2024-05-08T23:05:40.548 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T23:05:40.548 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/multimds_misc} --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161 --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/orig.config.yaml 2024-05-08T23:05:40.599 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:05:40.698 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/multimds_misc} email: pdonnell@redhat.com first_in_suite: false job_id: '7698161' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Scrub error on inode sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMjn2ExfIpGtilDuJsQnR8cPJ7BhJ8gVwL96HZ/GR5sNqD8dJzV2HNrDpiBjbq+tUdFX8WF+OXaFe/oLzSSlhXo= smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyW3AfJX+NG4r8sSr414NF4otg3ZZ29GpZaDX52NnoCn2tYrIDT1cAO9MhExM0HkxaCVe7WPSNkwzFe6UMPxUM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_multimds_misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T23:05:40.698 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T23:05:40.699 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T23:05:40.699 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T23:05:40.701 INFO:teuthology.task.internal:Checking packages... 2024-05-08T23:05:40.725 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T23:05:40.725 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T23:05:40.725 INFO:teuthology.packaging:ref: None 2024-05-08T23:05:40.725 INFO:teuthology.packaging:tag: None 2024-05-08T23:05:40.725 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:05:40.725 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:05:40.725 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:05:40.868 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T23:05:40.869 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T23:05:40.875 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T23:05:40.875 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T23:05:40.897 INFO:teuthology.task.internal:Saving configuration 2024-05-08T23:05:40.915 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T23:05:40.921 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T23:05:40.947 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:59:25.702620', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMjn2ExfIpGtilDuJsQnR8cPJ7BhJ8gVwL96HZ/GR5sNqD8dJzV2HNrDpiBjbq+tUdFX8WF+OXaFe/oLzSSlhXo='} 2024-05-08T23:05:40.972 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:59:25.701189', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyW3AfJX+NG4r8sSr414NF4otg3ZZ29GpZaDX52NnoCn2tYrIDT1cAO9MhExM0HkxaCVe7WPSNkwzFe6UMPxUM='} 2024-05-08T23:05:40.972 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T23:05:40.979 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-08T23:05:40.979 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-08T23:05:40.979 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T23:05:41.050 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8c13ddd160>, signals=[15]) 2024-05-08T23:05:41.050 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T23:05:41.058 INFO:teuthology.task.internal:Opening connections... 2024-05-08T23:05:41.058 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:05:41.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:05:41.141 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:05:41.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:05:41.216 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T23:05:41.223 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-05-08T23:05:41.245 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-05-08T23:05:41.246 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel" 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora" 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.6" 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8" 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-05-08T23:05:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T23:05:41.308 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T23:05:41.308 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T23:05:41.308 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:05:41.308 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:05:41.308 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T23:05:41.308 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T23:05:41.308 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T23:05:41.308 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T23:05:41.309 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-05-08T23:05:41.338 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-05-08T23:05:41.359 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-05-08T23:05:41.360 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:ID="rhel" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="fedora" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="8.6" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el8" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T23:05:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T23:05:41.421 INFO:teuthology.orchestra.run.smithi049.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T23:05:41.421 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:05:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:05:41.421 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T23:05:41.421 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T23:05:41.421 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T23:05:41.421 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T23:05:41.421 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-05-08T23:05:41.446 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T23:05:41.456 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T23:05:41.462 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T23:05:41.462 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:05:41.465 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:05:41.484 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T23:05:41.490 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T23:05:41.491 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:05:41.526 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:05:41.562 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T23:05:41.615 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T23:05:41.631 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-08T23:05:41.631 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-08T23:05:41.631 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-08T23:05:41.631 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:05:41.632 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:05:41.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:05:41.652 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-05-08T23:05:41.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:05:41.653 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-08T23:05:41.712 INFO:teuthology.orchestra.run.smithi049.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:05:41.712 INFO:teuthology.task.kernel:Running kernel on smithi049: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:05:41.712 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-05-08T23:05:41.714 INFO:teuthology.orchestra.run.smithi005.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:05:41.715 INFO:teuthology.task.kernel:Running kernel on smithi005: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:05:41.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-05-08T23:05:43.277 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-05-08T23:05:43.277 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2024-05-08T23:05:43.349 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2024-05-08T23:05:43.349 INFO:teuthology.orchestra.run.smithi005.stdout:Unable to read consumer identity 2024-05-08T23:05:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:05:43.553 INFO:teuthology.orchestra.run.smithi049.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:05:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:05:43.607 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:05:43.607 INFO:teuthology.orchestra.run.smithi005.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:05:43.607 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:05:45.312 INFO:teuthology.orchestra.run.smithi049.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-08T23:05:45.314 INFO:teuthology.orchestra.run.smithi005.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2024-05-08T23:05:56.011 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 16 MB 00:10 2024-05-08T23:05:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 318 kB/s | 24 kB 00:00 2024-05-08T23:06:00.697 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T23:06:00.698 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T23:06:00.778 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-08T23:06:00.780 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-08T23:06:00.780 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-08T23:06:00.926 DEBUG:teuthology.orchestra.run.smithi049:> echo no | sudo yum reinstall kernel || true 2024-05-08T23:06:00.932 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:15 2024-05-08T23:06:01.282 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-05-08T23:06:01.283 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2024-05-08T23:06:01.535 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:06:01.535 INFO:teuthology.orchestra.run.smithi049.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:06:01.535 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:06:01.858 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 11:05:59 PM UTC. 2024-05-08T23:06:01.933 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T23:06:01.936 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T23:06:01.989 INFO:teuthology.orchestra.run.smithi049.stderr:Error: No packages marked for reinstall. 2024-05-08T23:06:02.038 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum reinstall -y kernel || true 2024-05-08T23:06:02.391 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-05-08T23:06:02.391 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2024-05-08T23:06:02.626 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:06:02.627 INFO:teuthology.orchestra.run.smithi049.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:06:02.627 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:06:02.948 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 11:05:59 PM UTC. 2024-05-08T23:06:03.012 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T23:06:03.013 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T23:06:03.059 INFO:teuthology.orchestra.run.smithi049.stderr:Error: No packages marked for reinstall. 2024-05-08T23:06:03.113 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T23:06:03.273 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:06:03.274 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:06:03.274 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T23:06:03.274 DEBUG:teuthology.parallel:result is None 2024-05-08T23:06:04.987 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 525 kB/s | 24 kB 00:00 2024-05-08T23:06:05.766 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T23:06:05.766 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T23:06:05.896 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-08T23:06:05.898 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-08T23:06:05.898 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-08T23:06:06.034 DEBUG:teuthology.orchestra.run.smithi005:> echo no | sudo yum reinstall kernel || true 2024-05-08T23:06:06.466 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2024-05-08T23:06:06.466 INFO:teuthology.orchestra.run.smithi005.stdout:Unable to read consumer identity 2024-05-08T23:06:06.692 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:06:06.692 INFO:teuthology.orchestra.run.smithi005.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:06:06.692 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:06:07.056 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 11:06:04 PM UTC. 2024-05-08T23:06:07.155 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T23:06:07.156 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T23:06:07.191 INFO:teuthology.orchestra.run.smithi005.stderr:Error: No packages marked for reinstall. 2024-05-08T23:06:07.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum reinstall -y kernel || true 2024-05-08T23:06:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2024-05-08T23:06:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:Unable to read consumer identity 2024-05-08T23:06:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:06:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:06:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:06:08.242 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:04 ago on Wed 08 May 2024 11:06:04 PM UTC. 2024-05-08T23:06:08.315 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T23:06:08.315 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T23:06:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:Error: No packages marked for reinstall. 2024-05-08T23:06:08.397 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T23:06:08.578 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:06:08.578 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:06:08.578 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T23:06:08.578 DEBUG:teuthology.parallel:result is None 2024-05-08T23:06:08.578 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T23:06:08.587 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T23:06:08.587 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:06:08.589 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:06:08.610 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T23:06:08.617 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T23:06:08.628 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T23:06:08.628 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:06:08.649 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:06:08.684 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T23:06:08.691 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T23:06:08.691 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:06:08.721 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:06:08.754 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:06:08.775 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:06:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:06:08.918 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:06:08.920 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T23:06:08.929 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T23:06:08.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:06:08.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:06:08.966 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T23:06:08.975 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T23:06:08.975 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:06:09.009 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:06:09.029 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:06:09.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:06:09.193 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:06:09.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:06:09.291 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:06:09.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:06:09.366 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:06:09.415 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:06:09.470 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:06:09.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:06:09.603 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:06:09.603 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:06:09.675 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-05-08T23:06:09.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-05-08T23:06:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:06:09.795 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:06:10.164 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T23:06:10.171 INFO:teuthology.task.internal:Starting timer... 2024-05-08T23:06:10.171 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T23:06:10.233 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T23:06:10.243 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T23:06:10.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-05-08T23:06:10.337 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-05-08T23:06:10.340 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-05-08T23:06:10.406 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-05-08T23:06:10.408 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T23:06:10.408 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-05-08T23:06:10.440 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-05-08T23:06:10.440 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-05-08T23:06:10.463 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-05-08T23:06:10.463 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi049.front.sepia.ceph.com': 'permissive'} 2024-05-08T23:06:10.463 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:06:10.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:06:10.513 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:06:10.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:06:10.541 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T23:06:10.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:06:10.586 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:06:10.613 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T23:06:10.623 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T23:06:10.639 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T23:06:10.641 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi049.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T23:16:56.996 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi049.front.sepia.ceph.com')] 2024-05-08T23:16:56.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-08T23:16:56.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:16:57.082 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-08T23:16:57.159 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-08T23:16:57.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-08T23:16:57.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:16:57.243 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-05-08T23:16:57.313 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-08T23:16:57.313 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T23:16:57.323 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T23:16:57.323 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:16:57.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:16:57.326 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:16:57.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:16:57.362 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:16:57.380 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:16:57.389 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:16:57.406 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:16:57.461 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-05-08T23:16:57.476 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-05-08T23:16:57.482 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-05-08T23:16:57.493 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:16:57.495 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-05-08T23:16:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:16:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:16:57.526 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:16:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:bash: ntpq: command not found 2024-05-08T23:16:57.601 INFO:teuthology.orchestra.run.smithi049.stderr:bash: ntpq: command not found 2024-05-08T23:16:57.602 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:16:57.602 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-08T23:16:57.602 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:16:57.603 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:16:57.603 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:16:57.603 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:16:57.605 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:16:57.605 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-05-08T23:16:57.605 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:16:57.605 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:16:57.605 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:16:57.605 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:16:57.606 INFO:teuthology.run_tasks:Running task install... 2024-05-08T23:16:57.613 DEBUG:teuthology.task.install:project ceph 2024-05-08T23:16:57.614 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T23:16:57.614 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T23:16:57.614 INFO:teuthology.task.install:Using flavor: default 2024-05-08T23:16:57.619 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T23:16:57.620 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T23:16:57.620 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:16:57.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:16:57.622 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:16:57.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:16:57.846 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:16:57.846 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:16:57.850 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:16:57.850 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:16:57.956 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:16:57.956 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:16:57.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:16:57.991 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:16:57.992 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:16:58.017 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:16:58.017 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:16:58.017 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:16:58.046 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:16:58.046 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:16:58.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:16:58.115 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:16:58.155 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:16:58.187 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-05-08T23:16:58.189 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-05-08T23:16:58.194 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:16:58.261 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-05-08T23:16:58.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-05-08T23:16:58.622 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2024-05-08T23:16:58.661 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-05-08T23:17:00.095 INFO:teuthology.orchestra.run.smithi005.stdout:63 files removed 2024-05-08T23:17:00.096 INFO:teuthology.orchestra.run.smithi049.stdout:63 files removed 2024-05-08T23:17:00.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:17:00.151 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:17:00.509 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-05-08T23:17:00.536 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2024-05-08T23:17:02.419 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 202 kB/s | 80 kB 00:00 2024-05-08T23:17:02.462 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 220 kB/s | 80 kB 00:00 2024-05-08T23:17:02.744 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-05-08T23:17:02.837 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-05-08T23:17:03.045 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2024-05-08T23:17:03.204 INFO:teuthology.orchestra.run.smithi049.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-05-08T23:17:03.220 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi005.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-05-08T23:17:04.672 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-05-08T23:17:05.422 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 16 MB 00:02 2024-05-08T23:17:08.464 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 405 kB/s | 24 kB 00:00 2024-05-08T23:17:09.124 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-05-08T23:17:09.397 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 428 kB/s | 24 kB 00:00 2024-05-08T23:17:10.066 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-05-08T23:17:12.058 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 67 MB 00:01 2024-05-08T23:17:13.100 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 61 MB 00:01 2024-05-08T23:17:23.727 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 61 MB 00:01 2024-05-08T23:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 67 MB 00:01 2024-05-08T23:17:40.326 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:17:40.329 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:17:40.346 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:17:40.347 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:17:40.348 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:17:40.349 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:17:40.352 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:17:40.356 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:17:40.358 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:17:40.591 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-08T23:17:40.607 INFO:teuthology.orchestra.run.smithi049.stdout:=================================================================================================================================== 2024-05-08T23:17:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-08T23:17:40.607 INFO:teuthology.orchestra.run.smithi049.stdout:=================================================================================================================================== 2024-05-08T23:17:40.607 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-08T23:17:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:17:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:17:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:17:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:17:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:17:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:17:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-08T23:17:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:17:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:17:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:17:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:17:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:17:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:17:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling module streams: 2024-05-08T23:17:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql 10 2024-05-08T23:17:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:17:40.619 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-08T23:17:40.619 INFO:teuthology.orchestra.run.smithi049.stdout:=================================================================================================================================== 2024-05-08T23:17:40.619 INFO:teuthology.orchestra.run.smithi049.stdout:Install 130 Packages 2024-05-08T23:17:40.619 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 16 Packages 2024-05-08T23:17:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:17:40.619 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 215 M 2024-05-08T23:17:40.619 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-08T23:17:41.094 INFO:teuthology.orchestra.run.smithi049.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 50 kB/s | 6.4 kB 00:00 2024-05-08T23:17:41.286 INFO:teuthology.orchestra.run.smithi049.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 4.4 MB/s | 860 kB 00:00 2024-05-08T23:17:41.344 INFO:teuthology.orchestra.run.smithi049.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 2.8 MB/s | 166 kB 00:00 2024-05-08T23:17:41.603 INFO:teuthology.orchestra.run.smithi049.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 8.8 MB/s | 2.3 MB 00:00 2024-05-08T23:17:41.811 INFO:teuthology.orchestra.run.smithi049.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 7.5 MB/s | 1.6 MB 00:00 2024-05-08T23:17:41.886 INFO:teuthology.orchestra.run.smithi049.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 6.0 MB/s | 5.5 MB 00:00 2024-05-08T23:17:42.403 INFO:teuthology.orchestra.run.smithi049.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 8.2 MB/s | 4.8 MB 00:00 2024-05-08T23:17:42.703 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:17:42.704 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:17:42.726 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:17:42.727 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:17:42.729 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:17:42.730 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:17:42.733 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:17:42.738 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:17:42.740 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:17:42.987 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-08T23:17:43.003 INFO:teuthology.orchestra.run.smithi005.stdout:=================================================================================================================================== 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout:=================================================================================================================================== 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:17:43.004 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:17:43.006 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:17:43.007 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:17:43.008 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:17:43.009 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:17:43.010 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:17:43.011 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:17:43.012 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:17:43.013 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:17:43.014 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:17:43.015 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling module streams: 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql 10 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout:=================================================================================================================================== 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout:Install 130 Packages 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 16 Packages 2024-05-08T23:17:43.016 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:17:43.017 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 215 M 2024-05-08T23:17:43.017 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-08T23:17:43.384 INFO:teuthology.orchestra.run.smithi005.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2024-05-08T23:17:43.559 INFO:teuthology.orchestra.run.smithi005.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 4.8 MB/s | 860 kB 00:00 2024-05-08T23:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout:(8/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x86 11 MB/s | 18 MB 00:01 2024-05-08T23:17:43.618 INFO:teuthology.orchestra.run.smithi005.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 2.8 MB/s | 166 kB 00:00 2024-05-08T23:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout:(9/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 7.7 MB/s | 22 MB 00:02 2024-05-08T23:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout:(10/146): ceph-selinux-18.2.2-1322.gd21e4f96.el 100 kB/s | 24 kB 00:00 2024-05-08T23:17:43.926 INFO:teuthology.orchestra.run.smithi005.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 7.4 MB/s | 2.3 MB 00:00 2024-05-08T23:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 2.0 MB/s | 261 kB 00:00 2024-05-08T23:17:44.277 INFO:teuthology.orchestra.run.smithi005.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 4.5 MB/s | 1.6 MB 00:00 2024-05-08T23:17:44.377 INFO:teuthology.orchestra.run.smithi005.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 4.3 MB/s | 5.5 MB 00:01 2024-05-08T23:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 54 kB/s | 31 kB 00:00 2024-05-08T23:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.8 MB/s | 782 kB 00:00 2024-05-08T23:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.7 MB/s | 142 kB 00:00 2024-05-08T23:17:45.022 INFO:teuthology.orchestra.run.smithi049.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 878 kB/s | 139 kB 00:00 2024-05-08T23:17:45.181 INFO:teuthology.orchestra.run.smithi049.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.9 MB/s | 474 kB 00:00 2024-05-08T23:17:45.335 INFO:teuthology.orchestra.run.smithi005.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 4.6 MB/s | 4.8 MB 00:01 2024-05-08T23:17:45.631 INFO:teuthology.orchestra.run.smithi049.stdout:(17/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 4.1 MB/s | 13 MB 00:03 2024-05-08T23:17:45.714 INFO:teuthology.orchestra.run.smithi049.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 538 kB/s | 45 kB 00:00 2024-05-08T23:17:45.815 INFO:teuthology.orchestra.run.smithi049.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.3 MB/s | 129 kB 00:00 2024-05-08T23:17:45.973 INFO:teuthology.orchestra.run.smithi049.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.2 MB/s | 197 kB 00:00 2024-05-08T23:17:46.207 INFO:teuthology.orchestra.run.smithi049.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.6 MB/s | 370 kB 00:00 2024-05-08T23:17:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 1.5 MB/s | 365 kB 00:00 2024-05-08T23:17:46.549 INFO:teuthology.orchestra.run.smithi049.stdout:(23/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 4.8 MB/s | 6.5 MB 00:01 2024-05-08T23:17:46.583 INFO:teuthology.orchestra.run.smithi049.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 797 kB/s | 107 kB 00:00 2024-05-08T23:17:46.633 INFO:teuthology.orchestra.run.smithi049.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.1 MB/s | 95 kB 00:00 2024-05-08T23:17:46.733 INFO:teuthology.orchestra.run.smithi049.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.8 MB/s | 181 kB 00:00 2024-05-08T23:17:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 224 kB/s | 24 kB 00:00 2024-05-08T23:17:46.950 INFO:teuthology.orchestra.run.smithi049.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.2 MB/s | 138 kB 00:00 2024-05-08T23:17:47.767 INFO:teuthology.orchestra.run.smithi049.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 4.3 MB/s | 3.5 MB 00:00 2024-05-08T23:17:48.035 INFO:teuthology.orchestra.run.smithi005.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 4.4 MB/s | 22 MB 00:04 2024-05-08T23:17:48.144 INFO:teuthology.orchestra.run.smithi005.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 225 kB/s | 24 kB 00:00 2024-05-08T23:17:48.559 INFO:teuthology.orchestra.run.smithi049.stdout:(30/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 1.7 MB/s | 3.5 MB 00:01 2024-05-08T23:17:48.768 INFO:teuthology.orchestra.run.smithi049.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.2 MB/s | 245 kB 00:00 2024-05-08T23:17:48.804 INFO:teuthology.orchestra.run.smithi005.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 4.2 MB/s | 18 MB 00:04 2024-05-08T23:17:48.861 INFO:teuthology.orchestra.run.smithi049.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 535 kB/s | 49 kB 00:00 2024-05-08T23:17:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 922 kB/s | 261 kB 00:00 2024-05-08T23:17:49.227 INFO:teuthology.orchestra.run.smithi049.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 44 kB/s | 16 kB 00:00 2024-05-08T23:17:49.237 INFO:teuthology.orchestra.run.smithi005.stdout:(12/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 3.4 MB/s | 13 MB 00:03 2024-05-08T23:17:49.262 INFO:teuthology.orchestra.run.smithi005.stdout:(13/146): libcephfs-devel-18.2.2-1322.gd21e4f96 177 kB/s | 31 kB 00:00 2024-05-08T23:17:49.379 INFO:teuthology.orchestra.run.smithi005.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.2 MB/s | 142 kB 00:00 2024-05-08T23:17:49.504 INFO:teuthology.orchestra.run.smithi005.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 1.1 MB/s | 139 kB 00:00 2024-05-08T23:17:49.555 INFO:teuthology.orchestra.run.smithi005.stdout:(16/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.4 MB/s | 782 kB 00:00 2024-05-08T23:17:49.610 INFO:teuthology.orchestra.run.smithi049.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 686 kB/s | 263 kB 00:00 2024-05-08T23:17:49.789 INFO:teuthology.orchestra.run.smithi005.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 1.6 MB/s | 474 kB 00:00 2024-05-08T23:17:49.802 INFO:teuthology.orchestra.run.smithi049.stdout:(35/146): ceph-mgr-diskprediction-local-18.2.2- 3.6 MB/s | 7.4 MB 00:02 2024-05-08T23:17:49.888 INFO:teuthology.orchestra.run.smithi005.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 446 kB/s | 45 kB 00:00 2024-05-08T23:17:49.902 INFO:teuthology.orchestra.run.smithi049.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 181 kB/s | 18 kB 00:00 2024-05-08T23:17:49.953 INFO:teuthology.orchestra.run.smithi049.stdout:(37/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 497 kB/s | 170 kB 00:00 2024-05-08T23:17:49.997 INFO:teuthology.orchestra.run.smithi005.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.2 MB/s | 129 kB 00:00 2024-05-08T23:17:50.197 INFO:teuthology.orchestra.run.smithi005.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 984 kB/s | 197 kB 00:00 2024-05-08T23:17:50.395 INFO:teuthology.orchestra.run.smithi049.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 133 kB/s | 59 kB 00:00 2024-05-08T23:17:50.464 INFO:teuthology.orchestra.run.smithi005.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.4 MB/s | 370 kB 00:00 2024-05-08T23:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 471 kB/s | 306 kB 00:00 2024-05-08T23:17:50.579 INFO:teuthology.orchestra.run.smithi049.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 408 kB/s | 75 kB 00:00 2024-05-08T23:17:50.713 INFO:teuthology.orchestra.run.smithi049.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 327 kB/s | 44 kB 00:00 2024-05-08T23:17:50.731 INFO:teuthology.orchestra.run.smithi005.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 1.3 MB/s | 365 kB 00:00 2024-05-08T23:17:50.797 INFO:teuthology.orchestra.run.smithi049.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2024-05-08T23:17:50.831 INFO:teuthology.orchestra.run.smithi005.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.0 MB/s | 107 kB 00:00 2024-05-08T23:17:50.905 INFO:teuthology.orchestra.run.smithi049.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 308 kB/s | 33 kB 00:00 2024-05-08T23:17:50.940 INFO:teuthology.orchestra.run.smithi005.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 880 kB/s | 95 kB 00:00 2024-05-08T23:17:50.946 INFO:teuthology.orchestra.run.smithi049.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 792 kB/s | 185 kB 00:00 2024-05-08T23:17:51.076 INFO:teuthology.orchestra.run.smithi049.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 95 kB/s | 16 kB 00:00 2024-05-08T23:17:51.122 INFO:teuthology.orchestra.run.smithi049.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 984 kB/s | 173 kB 00:00 2024-05-08T23:17:51.215 INFO:teuthology.orchestra.run.smithi049.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.7 MB/s | 384 kB 00:00 2024-05-08T23:17:51.248 INFO:teuthology.orchestra.run.smithi049.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 648 kB/s | 81 kB 00:00 2024-05-08T23:17:51.355 INFO:teuthology.orchestra.run.smithi049.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 79 kB/s | 11 kB 00:00 2024-05-08T23:17:51.382 INFO:teuthology.orchestra.run.smithi049.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 131 kB/s | 18 kB 00:00 2024-05-08T23:17:51.499 INFO:teuthology.orchestra.run.smithi049.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 418 kB/s | 49 kB 00:00 2024-05-08T23:17:51.574 INFO:teuthology.orchestra.run.smithi049.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 5.3 MB/s | 1.2 MB 00:00 2024-05-08T23:17:51.640 INFO:teuthology.orchestra.run.smithi005.stdout:(25/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 3.1 MB/s | 6.5 MB 00:02 2024-05-08T23:17:51.649 INFO:teuthology.orchestra.run.smithi049.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 391 kB/s | 59 kB 00:00 2024-05-08T23:17:51.683 INFO:teuthology.orchestra.run.smithi049.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 554 kB/s | 60 kB 00:00 2024-05-08T23:17:51.749 INFO:teuthology.orchestra.run.smithi005.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.6 MB/s | 181 kB 00:00 2024-05-08T23:17:51.792 INFO:teuthology.orchestra.run.smithi049.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2024-05-08T23:17:51.817 INFO:teuthology.orchestra.run.smithi049.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2024-05-08T23:17:51.849 INFO:teuthology.orchestra.run.smithi005.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 242 kB/s | 24 kB 00:00 2024-05-08T23:17:51.909 INFO:teuthology.orchestra.run.smithi049.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 288 kB/s | 34 kB 00:00 2024-05-08T23:17:51.941 INFO:teuthology.orchestra.run.smithi005.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.5 MB/s | 138 kB 00:00 2024-05-08T23:17:51.959 INFO:teuthology.orchestra.run.smithi049.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2024-05-08T23:17:52.018 INFO:teuthology.orchestra.run.smithi049.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 592 kB/s | 64 kB 00:00 2024-05-08T23:17:52.068 INFO:teuthology.orchestra.run.smithi049.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2024-05-08T23:17:52.148 INFO:teuthology.orchestra.run.smithi049.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 183 kB/s | 24 kB 00:00 2024-05-08T23:17:52.185 INFO:teuthology.orchestra.run.smithi049.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2024-05-08T23:17:52.252 INFO:teuthology.orchestra.run.smithi049.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 266 kB/s | 27 kB 00:00 2024-05-08T23:17:52.336 INFO:teuthology.orchestra.run.smithi049.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 1.7 MB/s | 255 kB 00:00 2024-05-08T23:17:52.369 INFO:teuthology.orchestra.run.smithi049.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2024-05-08T23:17:52.461 INFO:teuthology.orchestra.run.smithi049.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 489 kB/s | 61 kB 00:00 2024-05-08T23:17:52.486 INFO:teuthology.orchestra.run.smithi049.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 731 kB/s | 85 kB 00:00 2024-05-08T23:17:52.570 INFO:teuthology.orchestra.run.smithi049.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2024-05-08T23:17:52.596 INFO:teuthology.orchestra.run.smithi049.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-05-08T23:17:52.633 INFO:teuthology.orchestra.run.smithi005.stdout:(29/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 2.0 MB/s | 3.5 MB 00:01 2024-05-08T23:17:52.681 INFO:teuthology.orchestra.run.smithi049.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 8.9 MB/s | 1.7 MB 00:00 2024-05-08T23:17:52.796 INFO:teuthology.orchestra.run.smithi049.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 156 kB/s | 18 kB 00:00 2024-05-08T23:17:52.863 INFO:teuthology.orchestra.run.smithi049.stdout:(72/146): python3-packaging-16.8-9.el8.noarch.r 180 kB/s | 48 kB 00:00 2024-05-08T23:17:52.943 INFO:teuthology.orchestra.run.smithi049.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 291 kB/s | 43 kB 00:00 2024-05-08T23:17:52.972 INFO:teuthology.orchestra.run.smithi049.stdout:(74/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2024-05-08T23:17:53.073 INFO:teuthology.orchestra.run.smithi049.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 843 kB/s | 109 kB 00:00 2024-05-08T23:17:53.108 INFO:teuthology.orchestra.run.smithi005.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 3.0 MB/s | 3.5 MB 00:01 2024-05-08T23:17:53.123 INFO:teuthology.orchestra.run.smithi049.stdout:(76/146): python3-requests-oauthlib-1.0.0-1.el8 284 kB/s | 43 kB 00:00 2024-05-08T23:17:53.231 INFO:teuthology.orchestra.run.smithi049.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 118 kB/s | 19 kB 00:00 2024-05-08T23:17:53.282 INFO:teuthology.orchestra.run.smithi049.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 276 kB/s | 44 kB 00:00 2024-05-08T23:17:53.283 INFO:teuthology.orchestra.run.smithi005.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.4 MB/s | 245 kB 00:00 2024-05-08T23:17:53.357 INFO:teuthology.orchestra.run.smithi049.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2024-05-08T23:17:53.391 INFO:teuthology.orchestra.run.smithi049.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2024-05-08T23:17:53.392 INFO:teuthology.orchestra.run.smithi005.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 454 kB/s | 49 kB 00:00 2024-05-08T23:17:53.476 INFO:teuthology.orchestra.run.smithi005.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 196 kB/s | 16 kB 00:00 2024-05-08T23:17:53.499 INFO:teuthology.orchestra.run.smithi049.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2024-05-08T23:17:53.524 INFO:teuthology.orchestra.run.smithi049.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.5 MB/s | 200 kB 00:00 2024-05-08T23:17:53.583 INFO:teuthology.orchestra.run.smithi049.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 577 kB/s | 48 kB 00:00 2024-05-08T23:17:53.643 INFO:teuthology.orchestra.run.smithi005.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.5 MB/s | 263 kB 00:00 2024-05-08T23:17:53.667 INFO:teuthology.orchestra.run.smithi049.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 613 kB/s | 87 kB 00:00 2024-05-08T23:17:53.743 INFO:teuthology.orchestra.run.smithi005.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.7 MB/s | 170 kB 00:00 2024-05-08T23:17:53.750 INFO:teuthology.orchestra.run.smithi049.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 345 kB/s | 58 kB 00:00 2024-05-08T23:17:53.767 INFO:teuthology.orchestra.run.smithi049.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 385 kB/s | 39 kB 00:00 2024-05-08T23:17:53.835 INFO:teuthology.orchestra.run.smithi005.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 197 kB/s | 18 kB 00:00 2024-05-08T23:17:53.876 INFO:teuthology.orchestra.run.smithi049.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 663 kB/s | 83 kB 00:00 2024-05-08T23:17:53.918 INFO:teuthology.orchestra.run.smithi049.stdout:(88/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.2 MB/s | 645 kB 00:00 2024-05-08T23:17:54.011 INFO:teuthology.orchestra.run.smithi049.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-05-08T23:17:54.093 INFO:teuthology.orchestra.run.smithi049.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-05-08T23:17:54.177 INFO:teuthology.orchestra.run.smithi049.stdout:(91/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 438 kB/s | 73 kB 00:00 2024-05-08T23:17:54.202 INFO:teuthology.orchestra.run.smithi049.stdout:(92/146): elfutils-libelf-devel-0.189-3.el8.x86 568 kB/s | 62 kB 00:00 2024-05-08T23:17:54.311 INFO:teuthology.orchestra.run.smithi049.stdout:(93/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.1 MB/s | 562 kB 00:00 2024-05-08T23:17:54.386 INFO:teuthology.orchestra.run.smithi049.stdout:(94/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-08T23:17:54.420 INFO:teuthology.orchestra.run.smithi049.stdout:(95/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 434 kB/s | 47 kB 00:00 2024-05-08T23:17:54.553 INFO:teuthology.orchestra.run.smithi049.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 3.3 MB/s | 559 kB 00:00 2024-05-08T23:17:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:(97/146): NetworkManager-initscripts-updown-1.4 864 kB/s | 144 kB 00:00 2024-05-08T23:17:54.662 INFO:teuthology.orchestra.run.smithi049.stdout:(98/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 919 kB/s | 100 kB 00:00 2024-05-08T23:17:54.729 INFO:teuthology.orchestra.run.smithi049.stdout:(99/146): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2024-05-08T23:17:54.754 INFO:teuthology.orchestra.run.smithi049.stdout:(100/146): python3-prettytable-0.7.2-14.el8.noa 478 kB/s | 44 kB 00:00 2024-05-08T23:17:54.793 INFO:teuthology.orchestra.run.smithi005.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 320 kB/s | 306 kB 00:00 2024-05-08T23:17:54.921 INFO:teuthology.orchestra.run.smithi049.stdout:(101/146): python3-pyasn1-modules-0.3.7-6.el8.n 573 kB/s | 110 kB 00:00 2024-05-08T23:17:54.935 INFO:teuthology.orchestra.run.smithi005.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 415 kB/s | 59 kB 00:00 2024-05-08T23:17:54.988 INFO:teuthology.orchestra.run.smithi049.stdout:(102/146): python3-jsonpatch-1.21-2.el8.noarch. 114 kB/s | 27 kB 00:00 2024-05-08T23:17:55.069 INFO:teuthology.orchestra.run.smithi005.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 563 kB/s | 75 kB 00:00 2024-05-08T23:17:55.694 INFO:teuthology.orchestra.run.smithi005.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 761 kB/s | 476 kB 00:00 2024-05-08T23:17:55.773 INFO:teuthology.orchestra.run.smithi049.stdout:(103/146): ceph-test-18.2.2-1322.gd21e4f96.el8. 3.9 MB/s | 47 MB 00:11 2024-05-08T23:17:55.798 INFO:teuthology.orchestra.run.smithi049.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 145 kB/s | 126 kB 00:00 2024-05-08T23:17:55.811 INFO:teuthology.orchestra.run.smithi005.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 374 kB/s | 44 kB 00:00 2024-05-08T23:17:55.839 INFO:teuthology.orchestra.run.smithi049.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 538 kB/s | 457 kB 00:00 2024-05-08T23:17:56.115 INFO:teuthology.orchestra.run.smithi049.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 1.0 MB/s | 320 kB 00:00 2024-05-08T23:17:56.149 INFO:teuthology.orchestra.run.smithi049.stdout:(107/146): libtool-2.4.6-25.el8.x86_64.rpm 2.2 MB/s | 709 kB 00:00 2024-05-08T23:17:56.182 INFO:teuthology.orchestra.run.smithi049.stdout:(108/146): bison-3.0.4-10.el8.x86_64.rpm 1.6 MB/s | 688 kB 00:00 2024-05-08T23:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout:(109/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.3 MB/s | 259 kB 00:00 2024-05-08T23:17:56.310 INFO:teuthology.orchestra.run.smithi005.stdout:(42/146): ceph-mgr-diskprediction-local-18.2.2- 2.0 MB/s | 7.4 MB 00:03 2024-05-08T23:17:56.437 INFO:teuthology.orchestra.run.smithi005.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 296 kB/s | 185 kB 00:00 2024-05-08T23:17:56.458 INFO:teuthology.orchestra.run.smithi049.stdout:(110/146): uuid-1.6.2-43.el8.x86_64.rpm 232 kB/s | 64 kB 00:00 2024-05-08T23:17:56.516 INFO:teuthology.orchestra.run.smithi049.stdout:(111/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 400 kB/s | 103 kB 00:00 2024-05-08T23:17:56.541 INFO:teuthology.orchestra.run.smithi049.stdout:(112/146): python3-markupsafe-0.23-19.el8.x86_6 91 kB/s | 39 kB 00:00 2024-05-08T23:17:56.659 INFO:teuthology.orchestra.run.smithi049.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 3.7 MB/s | 538 kB 00:00 2024-05-08T23:17:56.729 INFO:teuthology.orchestra.run.smithi005.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 79 kB/s | 33 kB 00:00 2024-05-08T23:17:56.746 INFO:teuthology.orchestra.run.smithi005.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 53 kB/s | 16 kB 00:00 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi049.stdout:(114/146): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2024-05-08T23:17:56.917 INFO:teuthology.orchestra.run.smithi049.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.2 MB/s | 323 kB 00:00 2024-05-08T23:17:56.985 INFO:teuthology.orchestra.run.smithi049.stdout:(116/146): python3-numpy-1.14.3-10.el8.x86_64.r 8.4 MB/s | 3.7 MB 00:00 2024-05-08T23:17:57.036 INFO:teuthology.orchestra.run.smithi049.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.2 MB/s | 225 kB 00:00 2024-05-08T23:17:57.218 INFO:teuthology.orchestra.run.smithi049.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.2 MB/s | 354 kB 00:00 2024-05-08T23:17:57.296 INFO:teuthology.orchestra.run.smithi005.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 304 kB/s | 173 kB 00:00 2024-05-08T23:17:57.306 INFO:teuthology.orchestra.run.smithi049.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 581 kB/s | 157 kB 00:00 2024-05-08T23:17:57.452 INFO:teuthology.orchestra.run.smithi049.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 11 MB/s | 5.2 MB 00:00 2024-05-08T23:17:57.497 INFO:teuthology.orchestra.run.smithi005.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 407 kB/s | 81 kB 00:00 2024-05-08T23:17:57.519 INFO:teuthology.orchestra.run.smithi049.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-05-08T23:17:57.522 INFO:teuthology.orchestra.run.smithi005.stdout:(48/146): python3-cherrypy-18.4.0-1.el8.noarch. 496 kB/s | 384 kB 00:00 2024-05-08T23:17:57.611 INFO:teuthology.orchestra.run.smithi049.stdout:(122/146): automake-1.16.1-8.el8.noarch.rpm 2.3 MB/s | 714 kB 00:00 2024-05-08T23:17:57.614 INFO:teuthology.orchestra.run.smithi005.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-05-08T23:17:57.631 INFO:teuthology.orchestra.run.smithi005.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-05-08T23:17:57.686 INFO:teuthology.orchestra.run.smithi049.stdout:(123/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-08T23:17:57.740 INFO:teuthology.orchestra.run.smithi005.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2024-05-08T23:17:57.873 INFO:teuthology.orchestra.run.smithi005.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 440 kB/s | 59 kB 00:00 2024-05-08T23:17:57.970 INFO:teuthology.orchestra.run.smithi049.stdout:(124/146): libpq-13.11-1.el8.x86_64.rpm 382 kB/s | 198 kB 00:00 2024-05-08T23:17:57.982 INFO:teuthology.orchestra.run.smithi005.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 557 kB/s | 60 kB 00:00 2024-05-08T23:17:58.020 INFO:teuthology.orchestra.run.smithi049.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 133 kB/s | 54 kB 00:00 2024-05-08T23:17:58.295 INFO:teuthology.orchestra.run.smithi049.stdout:(126/146): python3-scipy-1.0.0-21.module+el8.9. 23 MB/s | 14 MB 00:00 2024-05-08T23:17:58.366 INFO:teuthology.orchestra.run.smithi049.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 4.4 MB/s | 1.5 MB 00:00 2024-05-08T23:17:58.373 INFO:teuthology.orchestra.run.smithi005.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 722 kB/s | 283 kB 00:00 2024-05-08T23:17:58.496 INFO:teuthology.orchestra.run.smithi049.stdout:(128/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 358 kB/s | 188 kB 00:00 2024-05-08T23:17:58.507 INFO:teuthology.orchestra.run.smithi005.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 117 kB/s | 16 kB 00:00 2024-05-08T23:17:58.596 INFO:teuthology.orchestra.run.smithi049.stdout:(129/146): postgresql-contrib-10.23-4.module+el 2.6 MB/s | 812 kB 00:00 2024-05-08T23:17:58.616 INFO:teuthology.orchestra.run.smithi005.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2024-05-08T23:17:58.830 INFO:teuthology.orchestra.run.smithi049.stdout:(130/146): postgresql-server-10.23-4.module+el8 11 MB/s | 5.1 MB 00:00 2024-05-08T23:17:58.908 INFO:teuthology.orchestra.run.smithi005.stdout:(57/146): python3-routes-2.4.1-12.el8.noarch.rp 674 kB/s | 196 kB 00:00 2024-05-08T23:17:58.996 INFO:teuthology.orchestra.run.smithi049.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2024-05-08T23:17:59.025 INFO:teuthology.orchestra.run.smithi005.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 550 kB/s | 64 kB 00:00 2024-05-08T23:17:59.075 INFO:teuthology.orchestra.run.smithi005.stdout:(59/146): python3-kubernetes-11.0.0-6.el8.noarc 817 kB/s | 1.2 MB 00:01 2024-05-08T23:17:59.159 INFO:teuthology.orchestra.run.smithi005.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2024-05-08T23:17:59.192 INFO:teuthology.orchestra.run.smithi005.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 205 kB/s | 24 kB 00:00 2024-05-08T23:17:59.193 INFO:teuthology.orchestra.run.smithi049.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2024-05-08T23:17:59.276 INFO:teuthology.orchestra.run.smithi005.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2024-05-08T23:17:59.310 INFO:teuthology.orchestra.run.smithi005.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2024-05-08T23:17:59.380 INFO:teuthology.orchestra.run.smithi049.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.5 MB/s | 854 kB 00:00 2024-05-08T23:17:59.464 INFO:teuthology.orchestra.run.smithi049.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 8.3 MB/s | 711 kB 00:00 2024-05-08T23:17:59.610 INFO:teuthology.orchestra.run.smithi005.stdout:(64/146): python3-webob-1.8.5-1.el8.1.noarch.rp 838 kB/s | 251 kB 00:00 2024-05-08T23:17:59.660 INFO:teuthology.orchestra.run.smithi005.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 665 kB/s | 255 kB 00:00 2024-05-08T23:17:59.698 INFO:teuthology.orchestra.run.smithi049.stdout:(135/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 6.3 MB/s | 1.5 MB 00:00 2024-05-08T23:17:59.727 INFO:teuthology.orchestra.run.smithi005.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 524 kB/s | 61 kB 00:00 2024-05-08T23:17:59.823 INFO:teuthology.orchestra.run.smithi049.stdout:(136/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.6 MB/s | 333 kB 00:00 2024-05-08T23:17:59.844 INFO:teuthology.orchestra.run.smithi005.stdout:(67/146): python3-zc-lockfile-2.0-2.el8.noarch. 197 kB/s | 23 kB 00:00 2024-05-08T23:17:59.869 INFO:teuthology.orchestra.run.smithi005.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 409 kB/s | 85 kB 00:00 2024-05-08T23:17:59.895 INFO:teuthology.orchestra.run.smithi005.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-05-08T23:18:00.013 INFO:teuthology.orchestra.run.smithi049.stdout:(137/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.6 MB/s | 1.1 MB 00:00 2024-05-08T23:18:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:(70/146): python3-packaging-16.8-9.el8.noarch.r 289 kB/s | 48 kB 00:00 2024-05-08T23:18:00.107 INFO:teuthology.orchestra.run.smithi049.stdout:(138/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 2.4 MB/s | 3.6 MB 00:01 2024-05-08T23:18:00.140 INFO:teuthology.orchestra.run.smithi049.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 221 kB 00:00 2024-05-08T23:18:00.170 INFO:teuthology.orchestra.run.smithi005.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 185 kB/s | 20 kB 00:00 2024-05-08T23:18:00.279 INFO:teuthology.orchestra.run.smithi005.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 406 kB/s | 44 kB 00:00 2024-05-08T23:18:00.292 INFO:teuthology.orchestra.run.smithi049.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 254 kB 00:00 2024-05-08T23:18:00.333 INFO:teuthology.orchestra.run.smithi049.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.2 MB/s | 237 kB 00:00 2024-05-08T23:18:00.416 INFO:teuthology.orchestra.run.smithi049.stdout:(142/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 1.2 MB/s | 99 kB 00:00 2024-05-08T23:18:00.437 INFO:teuthology.orchestra.run.smithi005.stdout:(73/146): python3-pyasn1-modules-0.3.7-6.el8.no 696 kB/s | 110 kB 00:00 2024-05-08T23:18:00.441 INFO:teuthology.orchestra.run.smithi049.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.2 MB/s | 180 kB 00:00 2024-05-08T23:18:00.571 INFO:teuthology.orchestra.run.smithi005.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 198 kB/s | 27 kB 00:00 2024-05-08T23:18:00.600 INFO:teuthology.orchestra.run.smithi049.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 14 MB/s | 2.5 MB 00:00 2024-05-08T23:18:00.625 INFO:teuthology.orchestra.run.smithi049.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 560 kB/s | 103 kB 00:00 2024-05-08T23:18:00.697 INFO:teuthology.orchestra.run.smithi005.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2024-05-08T23:18:00.839 INFO:teuthology.orchestra.run.smithi005.stdout:(76/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.2 MB/s | 457 kB 00:00 2024-05-08T23:18:00.956 INFO:teuthology.orchestra.run.smithi005.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 5.8 MB/s | 688 kB 00:00 2024-05-08T23:18:01.106 INFO:teuthology.orchestra.run.smithi005.stdout:(78/146): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-05-08T23:18:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:(79/146): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2024-05-08T23:18:01.382 INFO:teuthology.orchestra.run.smithi005.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 332 kB/s | 39 kB 00:00 2024-05-08T23:18:01.557 INFO:teuthology.orchestra.run.smithi005.stdout:(81/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2024-05-08T23:18:01.715 INFO:teuthology.orchestra.run.smithi005.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 404 kB/s | 64 kB 00:00 2024-05-08T23:18:01.774 INFO:teuthology.orchestra.run.smithi005.stdout:(83/146): thrift-0.13.0-2.el8.x86_64.rpm 920 kB/s | 1.7 MB 00:01 2024-05-08T23:18:01.841 INFO:teuthology.orchestra.run.smithi005.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 827 kB/s | 103 kB 00:00 2024-05-08T23:18:02.008 INFO:teuthology.orchestra.run.smithi005.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 3.2 MB/s | 538 kB 00:00 2024-05-08T23:18:02.075 INFO:teuthology.orchestra.run.smithi005.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-05-08T23:18:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:(146/146): librados2-18.2.2-1322.gd21e4f96.el8. 1.0 MB/s | 3.7 MB 00:03 2024-05-08T23:18:02.087 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:18:02.087 INFO:teuthology.orchestra.run.smithi049.stdout:Total 10 MB/s | 215 MB 00:21 2024-05-08T23:18:02.267 INFO:teuthology.orchestra.run.smithi005.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2024-05-08T23:18:02.301 INFO:teuthology.orchestra.run.smithi005.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.4 MB/s | 323 kB 00:00 2024-05-08T23:18:02.368 INFO:teuthology.orchestra.run.smithi005.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2024-05-08T23:18:02.843 INFO:teuthology.orchestra.run.smithi005.stdout:(90/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 3.2 MB/s | 47 MB 00:14 2024-05-08T23:18:02.877 INFO:teuthology.orchestra.run.smithi005.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 615 kB/s | 354 kB 00:00 2024-05-08T23:18:03.035 INFO:teuthology.orchestra.run.smithi005.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 7.8 MB/s | 5.2 MB 00:00 2024-05-08T23:18:03.060 INFO:teuthology.orchestra.run.smithi005.stdout:(93/146): python3-mako-1.0.6-14.el8.noarch.rpm 724 kB/s | 157 kB 00:00 2024-05-08T23:18:03.177 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-08T23:18:03.178 INFO:teuthology.orchestra.run.smithi005.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-05-08T23:18:03.219 INFO:teuthology.orchestra.run.smithi005.stdout:(95/146): automake-1.16.1-8.el8.noarch.rpm 3.8 MB/s | 714 kB 00:00 2024-05-08T23:18:03.244 INFO:teuthology.orchestra.run.smithi005.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-05-08T23:18:03.336 INFO:teuthology.orchestra.run.smithi005.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-08T23:18:03.370 INFO:teuthology.orchestra.run.smithi005.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 360 kB/s | 54 kB 00:00 2024-05-08T23:18:03.479 INFO:teuthology.orchestra.run.smithi005.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-08T23:18:03.546 INFO:teuthology.orchestra.run.smithi005.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 8.6 MB/s | 1.5 MB 00:00 2024-05-08T23:18:03.588 INFO:teuthology.orchestra.run.smithi005.stdout:(101/146): postgresql-contrib-10.23-4.module+el 7.3 MB/s | 812 kB 00:00 2024-05-08T23:18:03.606 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-08T23:18:03.606 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-08T23:18:03.771 INFO:teuthology.orchestra.run.smithi005.stdout:(102/146): python3-scipy-1.0.0-21.module+el8.9. 26 MB/s | 14 MB 00:00 2024-05-08T23:18:03.788 INFO:teuthology.orchestra.run.smithi005.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 89 kB/s | 18 kB 00:00 2024-05-08T23:18:03.922 INFO:teuthology.orchestra.run.smithi005.stdout:(104/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-05-08T23:18:03.947 INFO:teuthology.orchestra.run.smithi005.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 243 kB/s | 43 kB 00:00 2024-05-08T23:18:03.998 INFO:teuthology.orchestra.run.smithi005.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 742 kB/s | 155 kB 00:00 2024-05-08T23:18:04.031 INFO:teuthology.orchestra.run.smithi005.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 511 kB/s | 43 kB 00:00 2024-05-08T23:18:04.106 INFO:teuthology.orchestra.run.smithi005.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 172 kB/s | 19 kB 00:00 2024-05-08T23:18:04.132 INFO:teuthology.orchestra.run.smithi005.stdout:(109/146): python3-pycparser-2.14-14.el8.noarch 522 kB/s | 109 kB 00:00 2024-05-08T23:18:04.157 INFO:teuthology.orchestra.run.smithi005.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 349 kB/s | 44 kB 00:00 2024-05-08T23:18:04.215 INFO:teuthology.orchestra.run.smithi005.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2024-05-08T23:18:04.274 INFO:teuthology.orchestra.run.smithi005.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-05-08T23:18:04.299 INFO:teuthology.orchestra.run.smithi005.stdout:(113/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2024-05-08T23:18:04.325 INFO:teuthology.orchestra.run.smithi005.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.8 MB/s | 200 kB 00:00 2024-05-08T23:18:04.416 INFO:teuthology.orchestra.run.smithi005.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 745 kB/s | 87 kB 00:00 2024-05-08T23:18:04.442 INFO:teuthology.orchestra.run.smithi005.stdout:(116/146): keyutils-libs-devel-1.5.10-9.el8.x86 288 kB/s | 48 kB 00:00 2024-05-08T23:18:04.484 INFO:teuthology.orchestra.run.smithi005.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 363 kB/s | 58 kB 00:00 2024-05-08T23:18:04.626 INFO:teuthology.orchestra.run.smithi005.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 185 kB/s | 39 kB 00:00 2024-05-08T23:18:04.651 INFO:teuthology.orchestra.run.smithi005.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 397 kB/s | 83 kB 00:00 2024-05-08T23:18:04.759 INFO:teuthology.orchestra.run.smithi005.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.3 MB/s | 645 kB 00:00 2024-05-08T23:18:04.785 INFO:teuthology.orchestra.run.smithi005.stdout:(121/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.7 MB/s | 238 kB 00:00 2024-05-08T23:18:04.881 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-08T23:18:04.893 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-08T23:18:04.919 INFO:teuthology.orchestra.run.smithi005.stdout:(122/146): libquadmath-8.5.0-20.el8.x86_64.rpm 589 kB/s | 172 kB 00:00 2024-05-08T23:18:04.944 INFO:teuthology.orchestra.run.smithi005.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 389 kB/s | 62 kB 00:00 2024-05-08T23:18:04.969 INFO:teuthology.orchestra.run.smithi005.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 347 kB/s | 73 kB 00:00 2024-05-08T23:18:05.095 INFO:teuthology.orchestra.run.smithi005.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.1 MB/s | 562 kB 00:00 2024-05-08T23:18:05.170 INFO:teuthology.orchestra.run.smithi005.stdout:(126/146): openssl-devel-1.1.1k-12.el8_9.x86_64 10 MB/s | 2.3 MB 00:00 2024-05-08T23:18:05.195 INFO:teuthology.orchestra.run.smithi005.stdout:(127/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 209 kB/s | 47 kB 00:00 2024-05-08T23:18:05.262 INFO:teuthology.orchestra.run.smithi005.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 3.3 MB/s | 559 kB 00:00 2024-05-08T23:18:05.312 INFO:teuthology.orchestra.run.smithi005.stdout:(129/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 853 kB/s | 100 kB 00:00 2024-05-08T23:18:05.413 INFO:teuthology.orchestra.run.smithi005.stdout:(130/146): NetworkManager-initscripts-updown-1. 594 kB/s | 144 kB 00:00 2024-05-08T23:18:05.555 INFO:teuthology.orchestra.run.smithi005.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2024-05-08T23:18:05.680 INFO:teuthology.orchestra.run.smithi005.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.4 MB/s | 303 kB 00:00 2024-05-08T23:18:05.847 INFO:teuthology.orchestra.run.smithi005.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.0 MB/s | 854 kB 00:00 2024-05-08T23:18:06.072 INFO:teuthology.orchestra.run.smithi005.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.1 MB/s | 711 kB 00:00 2024-05-08T23:18:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:(135/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.7 MB/s | 1.5 MB 00:00 2024-05-08T23:18:06.406 INFO:teuthology.orchestra.run.smithi005.stdout:(136/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.3 MB/s | 333 kB 00:00 2024-05-08T23:18:06.640 INFO:teuthology.orchestra.run.smithi005.stdout:(137/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 4.6 MB/s | 1.1 MB 00:00 2024-05-08T23:18:06.987 INFO:teuthology.orchestra.run.smithi005.stdout:(138/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 637 kB/s | 221 kB 00:00 2024-05-08T23:18:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:(139/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 254 kB 00:00 2024-05-08T23:18:07.349 INFO:teuthology.orchestra.run.smithi005.stdout:(140/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.1 MB/s | 237 kB 00:00 2024-05-08T23:18:07.491 INFO:teuthology.orchestra.run.smithi005.stdout:(141/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.2 MB/s | 180 kB 00:00 2024-05-08T23:18:07.641 INFO:teuthology.orchestra.run.smithi005.stdout:(142/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 661 kB/s | 99 kB 00:00 2024-05-08T23:18:07.725 INFO:teuthology.orchestra.run.smithi005.stdout:(143/146): librados2-18.2.2-1322.gd21e4f96.el8. 1.5 MB/s | 3.7 MB 00:02 2024-05-08T23:18:07.833 INFO:teuthology.orchestra.run.smithi005.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 13 MB/s | 2.5 MB 00:00 2024-05-08T23:18:07.875 INFO:teuthology.orchestra.run.smithi005.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 685 kB/s | 103 kB 00:00 2024-05-08T23:18:09.917 INFO:teuthology.orchestra.run.smithi005.stdout:(146/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 793 kB/s | 3.6 MB 00:04 2024-05-08T23:18:10.312 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:18:10.312 INFO:teuthology.orchestra.run.smithi005.stdout:Total 8.0 MB/s | 215 MB 00:26 2024-05-08T23:18:11.043 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-08T23:18:11.349 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-08T23:18:11.491 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:18:11.495 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-08T23:18:11.495 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-08T23:18:11.809 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:18:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:18:12.193 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:18:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:18:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:18:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:18:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:18:12.852 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-08T23:18:12.863 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-08T23:18:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:18:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T23:18:13.605 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T23:18:13.707 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T23:18:13.913 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:18:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:18:14.564 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:18:15.006 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:18:15.188 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:18:15.308 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:18:15.529 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:18:15.610 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:18:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:18:16.154 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:18:16.241 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:18:16.437 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:18:16.695 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:18:16.908 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:18:17.312 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:18:17.551 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:18:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:18:17.801 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:18:17.962 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:18:18.022 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:18:18.225 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T23:18:18.479 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-08T23:18:18.545 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T23:18:18.627 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:18:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T23:18:18.842 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:18:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T23:18:19.065 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:18:19.152 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T23:18:19.254 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:18:19.345 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T23:18:19.391 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:18:19.454 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:18:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:18:19.715 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:18:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:18:19.999 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T23:18:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:18:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:18:20.195 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T23:18:20.315 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T23:18:20.491 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:18:20.837 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T23:18:20.932 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:18:21.020 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T23:18:21.124 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:18:21.545 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:18:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:18:21.781 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T23:18:21.863 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:18:22.004 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:18:22.076 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T23:18:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:18:22.322 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T23:18:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:18:22.683 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:18:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:18:22.769 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:18:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:18:22.951 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:18:23.024 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:18:23.173 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:18:23.291 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:18:23.330 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:18:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T23:18:23.727 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:18:23.930 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:18:24.041 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:18:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:18:24.282 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:18:24.365 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:18:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:18:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:18:24.823 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:18:25.360 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T23:18:25.492 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:18:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:18:25.785 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T23:18:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:18:26.498 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:18:26.824 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T23:18:27.151 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T23:18:27.265 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:18:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T23:18:27.539 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T23:18:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:18:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T23:18:27.782 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:18:27.880 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T23:18:28.043 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:18:28.204 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T23:18:28.261 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:18:28.456 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T23:18:28.477 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:18:28.556 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:18:28.735 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:18:28.786 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:18:28.858 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:18:28.881 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:18:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:18:29.103 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:18:29.262 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:18:29.364 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:18:29.594 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:18:30.066 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:18:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T23:18:30.461 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:18:30.490 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:18:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T23:18:30.702 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T23:18:30.772 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T23:18:30.942 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T23:18:31.108 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T23:18:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:18:31.329 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T23:18:31.543 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T23:18:31.616 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T23:18:31.810 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T23:18:32.052 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T23:18:32.204 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T23:18:32.424 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-05-08T23:18:32.645 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-05-08T23:18:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-05-08T23:18:33.023 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-05-08T23:18:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:18:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-05-08T23:18:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T23:18:33.591 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-05-08T23:18:33.626 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:18:33.855 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:18:33.882 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-05-08T23:18:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:18:34.115 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-05-08T23:18:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:18:34.301 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-05-08T23:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:18:34.541 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-05-08T23:18:34.551 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:18:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:18:34.812 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:18:34.957 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-05-08T23:18:35.024 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:18:35.268 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:18:35.289 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-05-08T23:18:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:18:35.586 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-05-08T23:18:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-05-08T23:18:35.953 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:18:35.988 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-05-08T23:18:36.231 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:18:36.244 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-05-08T23:18:36.252 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:18:36.452 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T23:18:36.508 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-05-08T23:18:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-05-08T23:18:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T23:18:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-05-08T23:18:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T23:18:37.164 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-05-08T23:18:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T23:18:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T23:18:37.498 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-05-08T23:18:37.704 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T23:18:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T23:18:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:18:38.088 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:18:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T23:18:38.358 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:18:38.515 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T23:18:38.575 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:18:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T23:18:38.790 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:18:38.955 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:18:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T23:18:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:18:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:18:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T23:18:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:18:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:18:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T23:18:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:18:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T23:18:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:18:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T23:18:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:18:40.547 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T23:18:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:18:40.923 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T23:18:40.945 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:18:41.158 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T23:18:41.200 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:18:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T23:18:41.417 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:18:41.569 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T23:18:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:18:41.811 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T23:18:41.996 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T23:18:42.064 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T23:18:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-08T23:18:42.566 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-08T23:18:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T23:18:42.807 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:18:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T23:18:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T23:18:43.141 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:18:43.156 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-08T23:18:43.310 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T23:18:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:18:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T23:18:43.723 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:18:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:18:43.915 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:18:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:18:44.167 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:18:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:18:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:18:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:18:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:18:44.934 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:18:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:18:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:18:45.325 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:18:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:18:45.399 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:18:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:18:45.608 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:18:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:18:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:18:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:18:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:18:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:18:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:18:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:18:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:18:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:18:48.823 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:18:49.050 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:18:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:18:49.706 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:18:49.713 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:18:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:18:49.983 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:18:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:18:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:18:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:18:50.928 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:18:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:18:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:18:51.112 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:18:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:18:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:18:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:18:56.176 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:18:56.462 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:18:56.555 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:18:56.703 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:19:11.606 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /dev 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /run 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /dev 2024-05-08T23:19:11.607 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:19:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:19:13.750 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:19:13.987 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:19:14.213 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:19:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:19:14.901 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:19:15.087 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:19:15.106 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:19:17.058 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:19:17.235 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:19:17.351 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:19:17.580 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:19:17.871 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:19:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:19:19.100 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /dev 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /run 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /dev 2024-05-08T23:19:19.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:19:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:19:19.466 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:19:20.285 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:19:20.469 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:19:20.568 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:19:20.944 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:19:21.123 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:19:21.162 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:19:21.281 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:19:21.335 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:19:21.348 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:19:21.500 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:19:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:19:21.924 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:19:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:19:22.641 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:19:22.663 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:19:22.665 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:19:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:19:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:19:23.130 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:19:23.294 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:19:23.663 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:19:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:19:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:19:24.819 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:19:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:19:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:19:25.427 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:19:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:19:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:19:27.867 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:19:28.015 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:19:28.124 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:19:28.502 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:19:28.567 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:19:28.641 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:19:28.686 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:19:28.848 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:19:28.863 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:19:29.576 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:19:29.897 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T23:19:30.214 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T23:19:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:19:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:19:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:19:30.637 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T23:19:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:19:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:19:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:19:31.032 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T23:19:31.225 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T23:19:31.240 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T23:19:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:19:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:19:31.512 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:19:31.757 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:19:32.010 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:19:34.510 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:19:34.817 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:19:34.856 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:19:35.306 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:19:35.667 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:19:35.921 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:19:36.142 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:19:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:19:36.374 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:19:36.697 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:19:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:19:37.040 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:19:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T23:19:37.265 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T23:19:37.351 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T23:19:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:19:37.582 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T23:19:37.598 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T23:19:37.799 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T23:19:37.810 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T23:19:37.994 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T23:19:38.150 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:19:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:19:38.368 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:19:38.545 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:19:38.545 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:19:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T23:19:38.836 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:19:38.836 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:19:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T23:19:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:19:39.086 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:19:39.087 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:19:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:19:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:19:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:19:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:19:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:19:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:19:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:19:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:19:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:19:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:19:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:19:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:19:40.777 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:19:40.777 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:19:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:19:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:19:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:19:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:19:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:19:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:19:42.609 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:19:42.612 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:19:42.612 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:19:42.613 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:19:42.614 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:19:42.615 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:19:42.616 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:19:42.617 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-05-08T23:19:42.618 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 93/162 2024-05-08T23:19:42.619 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 94/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 95/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 96/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 97/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 98/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 99/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T23:19:42.620 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T23:19:42.621 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:19:42.622 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:19:42.623 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:19:42.624 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:19:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:19:43.066 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:19:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:19:43.601 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:19:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:19:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:19:44.276 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:19:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:19:44.712 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:19:44.724 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:19:44.930 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T23:19:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:19:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:19:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:19:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:19:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:19:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:19:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:19:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:19:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:19:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:19:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:19:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:19:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:19:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:19:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:19:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:19:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:19:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:19:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:19:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:19:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:19:47.931 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:19:47.932 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:19:48.148 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:19:48.148 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:19:48.356 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:19:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:19:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:19:49.564 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:19:49.564 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:19:49.564 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:19:49.564 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:19:49.564 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:19:49.564 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:19:49.564 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:19:49.564 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:19:49.565 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:19:49.566 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:19:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-08T23:19:49.582 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T23:19:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:19:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:19:49.587 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:19:49.588 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:19:56.235 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:19:56.235 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-05-08T23:19:56.886 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:19:56.887 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.888 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.889 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:19:56.890 INFO:teuthology.orchestra.run.smithi049.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.891 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:19:56.892 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.893 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:19:56.894 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:19:56.895 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:19:56.896 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.897 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.898 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:19:56.898 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:19:56.898 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:19:56.898 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:19:56.898 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:19:56.898 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:19:56.898 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:19:56.898 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:19:56.899 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-08T23:19:57.127 DEBUG:teuthology.parallel:result is None 2024-05-08T23:20:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:20:04.128 INFO:teuthology.orchestra.run.smithi005.stdout:Installed products updated. 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:20:04.801 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.802 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:20:04.803 INFO:teuthology.orchestra.run.smithi005.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:20:04.804 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:20:04.805 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:20:04.806 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:20:04.806 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:20:04.806 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:20:04.806 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:20:04.806 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:20:04.806 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:20:04.806 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:20:04.806 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:20:04.807 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:20:04.807 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:20:04.807 INFO:teuthology.orchestra.run.smithi005.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:20:04.807 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:20:04.807 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.807 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.807 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:20:04.808 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:20:04.809 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:20:04.810 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:20:04.811 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:20:04.812 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:20:04.812 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:20:04.812 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:20:04.812 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:04.812 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-08T23:20:05.038 DEBUG:teuthology.parallel:result is None 2024-05-08T23:20:05.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:20:05.271 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:20:05.301 INFO:teuthology.orchestra.run.smithi005.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:20:05.302 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:20:05.302 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:20:05.304 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:20:05.452 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:20:05.482 INFO:teuthology.orchestra.run.smithi049.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:20:05.482 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:20:05.482 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:20:05.484 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T23:20:05.485 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:05.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:20:05.539 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:05.539 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:20:05.591 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T23:20:05.592 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:05.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:20:05.627 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:20:05.704 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:05.705 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:20:05.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:20:05.807 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T23:20:05.807 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:05.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:20:05.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:20:05.917 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:05.917 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:20:05.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:20:06.016 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T23:20:06.016 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:06.016 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:20:06.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:20:06.125 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:06.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:20:06.154 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:20:06.223 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T23:20:06.312 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T23:20:06.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /var/log/ceph 2024-05-08T23:20:06.314 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /var/log/ceph 2024-05-08T23:20:06.347 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T23:20:06.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T23:20:06.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T23:20:06.421 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T23:20:06.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T23:20:06.464 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T23:20:06.504 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-08T23:20:06.504 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-08T23:20:06.504 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-08T23:20:06.504 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:06.504 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T23:20:06.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T23:20:06.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-08T23:20:06.722 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:06.722 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T23:20:06.740 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T23:20:06.857 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-08T23:20:06.890 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T23:20:06.890 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T23:20:06.891 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161', 'branch': 'wip-pdonnell-testing-20240508.150423-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/multimds_misc}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7698161', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240508.150423-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240508.150423-reef', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 65, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2738, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'sleep_before_teardown': 0, 'subset': '17/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240508.150423-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi005.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMjn2ExfIpGtilDuJsQnR8cPJ7BhJ8gVwL96HZ/GR5sNqD8dJzV2HNrDpiBjbq+tUdFX8WF+OXaFe/oLzSSlhXo=', 'smithi049.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyW3AfJX+NG4r8sSr414NF4otg3ZZ29GpZaDX52NnoCn2tYrIDT1cAO9MhExM0HkxaCVe7WPSNkwzFe6UMPxUM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_multimds_misc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_19:22:21', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T23:20:06.891 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T23:20:06.894 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T23:20:06.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T23:20:06.956 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T23:20:07.010 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:07.010 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:20:07.050 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:20:07.051 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-05-08T23:20:07.112 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:20:07.113 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:20:07.113 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 181703 Links: 1 2024-05-08T23:20:07.113 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:20:07.113 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:20:07.113 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-08 23:19:49.167669321 +0000 2024-05-08T23:20:07.113 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-08 23:16:08.483934099 +0000 2024-05-08T23:20:07.113 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-08 23:16:08.483934099 +0000 2024-05-08T23:20:07.113 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-05-08T23:20:07.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:20:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-08T23:20:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-08T23:20:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000118792 s, 4.3 MB/s 2024-05-08T23:20:07.187 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:20:07.250 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 180695 Links: 1 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-08 23:19:49.167669321 +0000 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-08 23:16:08.843926468 +0000 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-08 23:16:08.843926468 +0000 2024-05-08T23:20:07.313 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-05-08T23:20:07.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:20:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-08T23:20:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-08T23:20:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.00012693 s, 4.0 MB/s 2024-05-08T23:20:07.386 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:20:07.450 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 181783 Links: 1 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-08 23:19:49.167669321 +0000 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-08 23:16:09.204918811 +0000 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-08 23:16:09.204918811 +0000 2024-05-08T23:20:07.513 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-05-08T23:20:07.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:20:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-08T23:20:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-08T23:20:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000124954 s, 4.1 MB/s 2024-05-08T23:20:07.586 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:20:07.650 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-05-08T23:20:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:20:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:20:07.712 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 182722 Links: 1 2024-05-08T23:20:07.712 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:20:07.713 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:20:07.713 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-08 23:19:49.167669321 +0000 2024-05-08T23:20:07.713 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-08 23:16:09.574910960 +0000 2024-05-08T23:20:07.713 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-08 23:16:09.574910960 +0000 2024-05-08T23:20:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-05-08T23:20:07.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:20:07.786 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-08T23:20:07.787 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-08T23:20:07.787 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000113583 s, 4.5 MB/s 2024-05-08T23:20:07.788 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:20:07.851 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:07.851 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:07.851 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:20:07.872 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:20:07.873 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-05-08T23:20:07.934 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:20:07.935 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:20:07.935 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 181012 Links: 1 2024-05-08T23:20:07.935 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:20:07.935 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:20:07.935 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-08 23:19:42.225768174 +0000 2024-05-08T23:20:07.935 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-08 23:15:46.309463242 +0000 2024-05-08T23:20:07.935 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-08 23:15:46.309463242 +0000 2024-05-08T23:20:07.935 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-05-08T23:20:07.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:20:08.006 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-08T23:20:08.006 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-08T23:20:08.007 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000114471 s, 4.5 MB/s 2024-05-08T23:20:08.008 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:20:08.069 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-05-08T23:20:08.128 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:20:08.128 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:20:08.128 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 181470 Links: 1 2024-05-08T23:20:08.128 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:20:08.129 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:20:08.129 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-08 23:19:42.225768174 +0000 2024-05-08T23:20:08.129 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-08 23:15:46.681453152 +0000 2024-05-08T23:20:08.129 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-08 23:15:46.681453152 +0000 2024-05-08T23:20:08.129 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-05-08T23:20:08.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:20:08.197 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-08T23:20:08.197 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-08T23:20:08.197 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000112806 s, 4.5 MB/s 2024-05-08T23:20:08.198 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:20:08.258 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 182658 Links: 1 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-08 23:19:42.225768174 +0000 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-08 23:15:47.028443741 +0000 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-08 23:15:47.028443741 +0000 2024-05-08T23:20:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-05-08T23:20:08.319 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:20:08.385 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-08T23:20:08.385 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-08T23:20:08.385 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000105393 s, 4.9 MB/s 2024-05-08T23:20:08.387 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:20:08.445 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 184411 Links: 1 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-08 23:19:42.225768174 +0000 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-08 23:15:47.380434194 +0000 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-08 23:15:47.380434194 +0000 2024-05-08T23:20:08.504 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-05-08T23:20:08.505 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:20:08.574 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-08T23:20:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-08T23:20:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000104983 s, 4.9 MB/s 2024-05-08T23:20:08.575 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:20:08.633 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:08.633 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T23:20:08.633 INFO:tasks.ceph:Generating config... 2024-05-08T23:20:08.634 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-08T23:20:08.635 INFO:tasks.ceph:[client] debug client = 20 2024-05-08T23:20:08.635 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T23:20:08.635 INFO:tasks.ceph:[client] pid file = 2024-05-08T23:20:08.635 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-08T23:20:08.635 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-08T23:20:08.635 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-08T23:20:08.635 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-08T23:20:08.635 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-08T23:20:08.635 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-08T23:20:08.635 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-08T23:20:08.635 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-08T23:20:08.635 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-08T23:20:08.635 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-08T23:20:08.635 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-08T23:20:08.636 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-08T23:20:08.636 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-08T23:20:08.636 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T23:20:08.636 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T23:20:08.636 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T23:20:08.636 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T23:20:08.636 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T23:20:08.636 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-08T23:20:08.636 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-08T23:20:08.636 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-08T23:20:08.636 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-08T23:20:08.636 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T23:20:08.636 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T23:20:08.636 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T23:20:08.636 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T23:20:08.637 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T23:20:08.637 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T23:20:08.638 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T23:20:08.638 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T23:20:08.638 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T23:20:08.638 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T23:20:08.638 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T23:20:08.638 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T23:20:08.638 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-08T23:20:08.638 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T23:20:08.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T23:20:08.724 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T23:20:08.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T23:20:08.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:20:08.843 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.5'), ('mon.b', '172.21.15.49'), ('mon.c', '[v2:172.21.15.49:3301,v1:172.21.15.49:6790]')] 2024-05-08T23:20:08.843 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.5,172.21.15.49,[v2:172.21.15.49:3301,v1:172.21.15.49:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-08T23:20:08.844 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:08.844 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T23:20:08.904 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.5 --add b 172.21.15.49 --addv c '[v2:172.21.15.49:3301,v1:172.21.15.49:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:09.003 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:09.003 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: generated fsid 4fd8ca56-5ac1-431d-b985-b50a13a8130a 2024-05-08T23:20:09.003 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = pacific 2024-05-08T23:20:09.003 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 0 2024-05-08T23:20:09.003 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 4fd8ca56-5ac1-431d-b985-b50a13a8130a 2024-05-08T23:20:09.004 INFO:teuthology.orchestra.run.smithi005.stdout:last_changed 2024-05-08T23:20:09.004491+0000 2024-05-08T23:20:09.004 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-05-08T23:20:09.004491+0000 2024-05-08T23:20:09.004 INFO:teuthology.orchestra.run.smithi005.stdout:min_mon_release 16 (pacific) 2024-05-08T23:20:09.004 INFO:teuthology.orchestra.run.smithi005.stdout:election_strategy: 1 2024-05-08T23:20:09.004 INFO:teuthology.orchestra.run.smithi005.stdout:0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-08T23:20:09.004 INFO:teuthology.orchestra.run.smithi005.stdout:1: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-05-08T23:20:09.004 INFO:teuthology.orchestra.run.smithi005.stdout:2: [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] mon.c 2024-05-08T23:20:09.004 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T23:20:09.005 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T23:20:09.025 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4fd8ca56-5ac1-431d-b985-b50a13a8130a... 2024-05-08T23:20:09.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T23:20:09.068 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2024-05-08T23:20:09.119 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2024-05-08T23:20:09.120 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T23:20:09.121 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.5,172.21.15.49,[v2:172.21.15.49:3301,v1:172.21.15.49:6790]" 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on skipped message = False 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 4fd8ca56-5ac1-431d-b985-b50a13a8130a 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2024-05-08T23:20:09.122 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2024-05-08T23:20:09.123 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2024-05-08T23:20:09.124 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2024-05-08T23:20:09.125 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-05-08T23:20:09.126 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:20:09.127 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2024-05-08T23:20:09.128 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = "" 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2024-05-08T23:20:09.129 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = True 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2024-05-08T23:20:09.130 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = True 2024-05-08T23:20:09.131 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2024-05-08T23:20:09.131 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2024-05-08T23:20:09.131 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2024-05-08T23:20:09.131 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-05-08T23:20:09.132 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-05-08T23:20:09.133 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-05-08T23:20:09.134 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.49,[v2:172.21.15.49:3301,v1:172.21.15.49:6790]" 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 4fd8ca56-5ac1-431d-b985-b50a13a8130a 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.135 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.136 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: bdev async discard = True 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: bdev enable discard = True 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore allocator = bitmap 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 1/20 2024-05-08T23:20:09.137 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 1/20 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 4/10 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-08T23:20:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-05-08T23:20:09.139 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-05-08T23:20:09.140 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-05-08T23:20:09.141 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-05-08T23:20:09.142 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2024-05-08T23:20:09.143 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-05-08T23:20:09.143 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2024-05-08T23:20:09.143 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-08T23:20:09.143 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-08T23:20:09.143 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-08T23:20:09.144 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T23:20:09.144 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T23:20:09.295 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T23:20:09.296 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:09.296 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T23:20:09.316 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:09.316 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T23:20:09.378 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:20:09.378 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:09.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T23:20:09.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:20:09.607 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:09.607 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:09.647 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:20:09.647 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:09.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T23:20:09.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:20:09.691 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:09.691 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:09.749 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T23:20:09.749 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T23:20:09.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T23:20:09.817 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T23:20:09.820 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T23:20:09.894 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T23:20:09.897 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T23:20:09.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T23:20:09.964 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T23:20:09.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-08T23:20:10.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T23:20:10.105 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T23:20:10.109 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-08T23:20:10.140 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T23:20:10.201 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T23:20:10.203 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-08T23:20:10.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T23:20:10.328 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T23:20:10.331 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-08T23:20:10.358 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T23:20:10.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T23:20:10.414 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T23:20:10.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-08T23:20:10.481 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-08T23:20:10.497 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T23:20:10.542 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T23:20:10.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-08T23:20:10.600 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-08T23:20:10.615 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T23:20:10.615 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T23:20:10.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T23:20:10.650 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:10.650 INFO:tasks.ceph:role: osd.0 2024-05-08T23:20:10.650 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:20:10.651 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:20:10.726 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:20:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-08T23:20:10.751 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-08T23:20:10.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T23:20:10.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T23:20:10.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T23:20:10.954 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:10.955 INFO:tasks.ceph:role: osd.1 2024-05-08T23:20:10.955 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:20:10.955 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T23:20:11.029 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:20:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:20:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:20:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:20:11.029 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:20:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:20:11.029 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:20:11.030 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:20:11.030 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:20:11.030 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:20:11.052 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-08T23:20:11.054 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-08T23:20:11.054 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T23:20:11.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T23:20:11.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T23:20:11.260 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:11.260 INFO:tasks.ceph:role: osd.2 2024-05-08T23:20:11.260 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:20:11.260 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T23:20:11.335 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:20:11.336 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:20:11.359 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-08T23:20:11.361 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-08T23:20:11.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T23:20:11.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T23:20:11.486 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T23:20:11.561 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:11.561 INFO:tasks.ceph:role: osd.3 2024-05-08T23:20:11.561 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:20:11.561 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T23:20:11.635 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:20:11.636 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:20:11.658 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-08T23:20:11.660 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-08T23:20:11.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T23:20:11.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T23:20:11.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:11.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:11.885+0000 7f9cc158a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T23:20:11.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:11.885+0000 7f9cc158a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T23:20:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:11.885+0000 7f9cc158a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:20:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:11.885+0000 7f9cc158a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:20:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:11.885+0000 7f9cc158a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:20:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:11.886+0000 7f9cc158a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T23:20:14.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T23:20:14.854 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:14.913+0000 7f1170b1f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T23:20:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:14.913+0000 7f1170b1f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T23:20:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:14.913+0000 7f1170b1f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:20:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:14.913+0000 7f1170b1f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:20:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:14.913+0000 7f1170b1f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:20:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:14.914+0000 7f1170b1f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T23:20:17.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T23:20:17.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:17.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:17.904+0000 7f47ce8047c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T23:20:17.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:17.904+0000 7f47ce8047c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T23:20:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:17.904+0000 7f47ce8047c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:20:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:17.904+0000 7f47ce8047c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:20:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:17.904+0000 7f47ce8047c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:20:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:17.905+0000 7f47ce8047c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T23:20:20.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T23:20:20.794 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:20.893+0000 7faca2f3c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T23:20:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:20.893+0000 7faca2f3c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T23:20:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:20.893+0000 7faca2f3c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:20:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:20.893+0000 7faca2f3c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:20:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:20.893+0000 7faca2f3c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:20:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:20.893+0000 7faca2f3c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T23:20:23.745 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T23:20:23.782 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T23:20:23.817 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:23.817 INFO:tasks.ceph:role: osd.4 2024-05-08T23:20:23.817 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:20:23.817 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:20:23.891 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:20:23.913 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-05-08T23:20:23.916 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-05-08T23:20:23.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T23:20:24.000 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T23:20:24.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T23:20:24.142 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:24.143 INFO:tasks.ceph:role: osd.5 2024-05-08T23:20:24.143 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:20:24.143 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T23:20:24.210 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:20:24.211 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:20:24.234 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-05-08T23:20:24.236 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-05-08T23:20:24.236 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T23:20:24.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T23:20:24.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T23:20:24.456 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:24.456 INFO:tasks.ceph:role: osd.6 2024-05-08T23:20:24.456 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:20:24.456 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T23:20:24.523 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:20:24.523 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:20:24.524 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:20:24.524 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:20:24.524 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:20:24.524 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:20:24.524 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:20:24.524 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:20:24.524 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:20:24.524 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:20:24.547 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-05-08T23:20:24.549 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-05-08T23:20:24.549 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T23:20:24.630 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T23:20:24.702 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T23:20:24.771 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:20:24.771 INFO:tasks.ceph:role: osd.7 2024-05-08T23:20:24.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:20:24.771 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T23:20:24.837 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:20:24.838 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:20:24.863 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-05-08T23:20:24.865 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-05-08T23:20:24.865 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T23:20:24.946 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T23:20:25.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:25.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:25.107+0000 7f68a5e6a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T23:20:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:25.107+0000 7f68a5e6a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T23:20:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:25.107+0000 7f68a5e6a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:20:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:25.107+0000 7f68a5e6a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:20:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:25.107+0000 7f68a5e6a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:20:25.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:25.108+0000 7f68a5e6a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T23:20:27.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T23:20:28.006 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:28.101+0000 7f54023f87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T23:20:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:28.101+0000 7f54023f87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T23:20:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:28.101+0000 7f54023f87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:20:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:28.101+0000 7f54023f87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:20:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:28.101+0000 7f54023f87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:20:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:28.101+0000 7f54023f87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T23:20:30.970 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T23:20:31.003 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:31.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:31.097+0000 7f102f0437c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T23:20:31.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:31.097+0000 7f102f0437c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T23:20:31.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:31.097+0000 7f102f0437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:20:31.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:31.097+0000 7f102f0437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:20:31.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:31.098+0000 7f102f0437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:20:31.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:31.098+0000 7f102f0437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T23:20:33.954 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T23:20:33.987 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:34.083+0000 7f4525da77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T23:20:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:34.083+0000 7f4525da77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T23:20:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:34.083+0000 7f4525da77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:20:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:34.083+0000 7f4525da77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:20:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:34.083+0000 7f4525da77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:20:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:34.083+0000 7f4525da77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T23:20:36.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:20:36.896 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:20:36.945 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T23:20:36.976 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T23:20:36.977 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:36.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T23:20:37.011 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:37.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-08T23:20:37.086 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:37.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-08T23:20:37.162 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:37.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T23:20:37.236 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:37.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T23:20:37.311 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:37.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T23:20:37.387 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:37.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T23:20:37.461 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:37.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T23:20:37.495 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:37.495 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-08T23:20:37.568 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:37.569 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-08T23:20:37.640 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:37.640 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T23:20:37.713 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:37.714 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T23:20:37.785 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:37.785 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T23:20:37.856 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:37.856 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T23:20:37.926 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:37.926 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T23:20:37.948 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:37.948 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-08T23:20:38.011 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:38.011 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T23:20:38.030 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:38.030 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-08T23:20:38.088 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T23:20:38.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T23:20:38.091 DEBUG:teuthology.orchestra.run.smithi049:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T23:20:38.132 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.y] 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCpCDxm1dPQMBAAhu0CK6DcucQteuTo7qN2IQ== 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCpCDxm4+KLORAAJGma8zcoGLift7K5wyvWlA== 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCqCDxmRixkBhAAk9s24ERPJ/6nCJY3g7QppA== 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.0] 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCrCDxmI4/bNBAAzNXcuLQCS9gaAJ0r4iYKlA== 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.1] 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCuCDxmMS6HNhAAcTP6nStED06bdM4MtbEPjQ== 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.2] 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCxCDxmDGv5NRAAljFQZXLGWwt+UXZ+HCdx0Q== 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.3] 2024-05-08T23:20:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQC0CDxmL+5LNRAAVPySIUtICBwIokJL9fHFtw== 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.x] 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCpCDxmntlgNRAA7JOIS+0NL1KmSSZhZjXlVQ== 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCqCDxm5iMNDBAANT+UrtTUBpYyQ9dzJyS01Q== 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCqCDxmPmurExAAxIzDzbLZ3s35XVJrvYiVCA== 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.4] 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQC5CDxm/F15BhAAEx9mKA9izSmb+xyxjEi9FA== 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.5] 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQC8CDxm0zAYBhAAjHT+TZFqk5MWvd0P5LmwfA== 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.6] 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQC/CDxmPcbiBRAAOmWNgaQGi6YGKTp8SJekUg== 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.7] 2024-05-08T23:20:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDCCDxm9FsDBRAA/lfmrpmL4UO3s/tJJ4o6yQ== 2024-05-08T23:20:38.135 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-08T23:20:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCqCDxm9tLAGBAAK8Lq89gJ902XqzXYAbYfkg== 2024-05-08T23:20:38.135 INFO:teuthology.orchestra.run.smithi005.stdout:[client.2] 2024-05-08T23:20:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCqCDxm2NzHHBAAUbeD58MntIoiwT8LekjIlw== 2024-05-08T23:20:38.135 INFO:teuthology.orchestra.run.smithi005.stdout:[client.1] 2024-05-08T23:20:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCqCDxmCStoIBAAhraObJ5LS9LtF97+hIcTFQ== 2024-05-08T23:20:38.135 INFO:teuthology.orchestra.run.smithi005.stdout:[client.3] 2024-05-08T23:20:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCqCDxmyvveIxAATEn7nOPHqtqSbdfQid25HA== 2024-05-08T23:20:38.157 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.y] 2024-05-08T23:20:38.157 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCpCDxm1dPQMBAAhu0CK6DcucQteuTo7qN2IQ== 2024-05-08T23:20:38.157 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2024-05-08T23:20:38.157 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCpCDxm4+KLORAAJGma8zcoGLift7K5wyvWlA== 2024-05-08T23:20:38.157 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2024-05-08T23:20:38.157 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCqCDxmRixkBhAAk9s24ERPJ/6nCJY3g7QppA== 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.0] 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCrCDxmI4/bNBAAzNXcuLQCS9gaAJ0r4iYKlA== 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.1] 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCuCDxmMS6HNhAAcTP6nStED06bdM4MtbEPjQ== 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.2] 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCxCDxmDGv5NRAAljFQZXLGWwt+UXZ+HCdx0Q== 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.3] 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQC0CDxmL+5LNRAAVPySIUtICBwIokJL9fHFtw== 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.x] 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCpCDxmntlgNRAA7JOIS+0NL1KmSSZhZjXlVQ== 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2024-05-08T23:20:38.158 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCqCDxm5iMNDBAANT+UrtTUBpYyQ9dzJyS01Q== 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCqCDxmPmurExAAxIzDzbLZ3s35XVJrvYiVCA== 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.4] 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQC5CDxm/F15BhAAEx9mKA9izSmb+xyxjEi9FA== 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.5] 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQC8CDxm0zAYBhAAjHT+TZFqk5MWvd0P5LmwfA== 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.6] 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQC/CDxmPcbiBRAAOmWNgaQGi6YGKTp8SJekUg== 2024-05-08T23:20:38.159 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.7] 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQDCCDxm9FsDBRAA/lfmrpmL4UO3s/tJJ4o6yQ== 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCqCDxm9tLAGBAAK8Lq89gJ902XqzXYAbYfkg== 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout:[client.2] 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCqCDxm2NzHHBAAUbeD58MntIoiwT8LekjIlw== 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout:[client.1] 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCqCDxmCStoIBAAhraObJ5LS9LtF97+hIcTFQ== 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout:[client.3] 2024-05-08T23:20:38.160 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCqCDxmyvveIxAATEn7nOPHqtqSbdfQid25HA== 2024-05-08T23:20:38.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:20:38.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:20:38.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:20:38.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:20:38.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:20:38.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:20:38.390 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.452 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.510 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:20:38.631 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:20:38.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:20:38.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:20:38.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:20:38.753 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:20:38.814 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.876 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:38.996 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:20:39.055 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:20:39.057 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:20:39.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:20:39.121 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:20:39.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:20:39.185 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:20:39.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:20:39.248 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:20:39.310 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T23:20:39.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T23:20:39.340 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:20:39.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T23:20:39.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T23:20:39.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:20:39.608 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T23:20:39.636 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T23:20:39.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:20:39.803 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T23:20:39.831 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:39.833 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:20:39.887 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T23:20:39.888 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T23:20:39.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T23:20:39.895 INFO:tasks.ceph.mon.a:Started 2024-05-08T23:20:39.895 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T23:20:39.895 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T23:20:39.929 INFO:tasks.ceph.mon.b:Started 2024-05-08T23:20:39.930 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T23:20:39.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T23:20:39.931 INFO:tasks.ceph.mon.c:Started 2024-05-08T23:20:39.932 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T23:20:39.932 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T23:20:39.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T23:20:39.934 INFO:tasks.ceph.mgr.y:Started 2024-05-08T23:20:39.934 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T23:20:39.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T23:20:39.936 INFO:tasks.ceph.mgr.x:Started 2024-05-08T23:20:39.937 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T23:20:39.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T23:20:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.077+0000 7fd0dbfbc700 1 Processor -- start 2024-05-08T23:20:40.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.078+0000 7fd0dbfbc700 1 -- start start 2024-05-08T23:20:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.078+0000 7fd0dbfbc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0dbfbc700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d40cc980 con 0x7fd0d406edf0 2024-05-08T23:20:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0dbfbc700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d40d00f0 con 0x7fd0d415bb20 2024-05-08T23:20:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0dbfbc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d406e420 con 0x7fd0d40cfd10 2024-05-08T23:20:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0da559700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0d406edf0 0x7fd0d4159f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60048/0 (socket says 172.21.15.5:60048) 2024-05-08T23:20:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0da559700 1 -- 172.21.15.5:0/3167167418 learned_addr learned my addr 172.21.15.5:0/3167167418 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:20:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:20:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.079+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:20:40.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.280+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:40.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.280+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:20:40.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.280+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:40.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.280+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:40.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.280+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:20:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.681+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.681+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:20:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.681+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.681+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:40.681+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:20:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:41.482+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:41.482+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:20:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:41.482+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:41.482+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:41.482+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:20:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 msgr2=0x7fd0d406dee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:20:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0cb7fe700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0d40cfd10 0x7fd0d406dee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 >> v1:172.21.15.49:6789/0 conn(0x7fd0d415bb20 legacy=0x7fd0d406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 >> v1:172.21.15.5:6789/0 conn(0x7fd0d406edf0 legacy=0x7fd0d4159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0cb7fe700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0cb7fe700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d406e420 con 0x7fd0b8006250 2024-05-08T23:20:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b8008db0 con 0x7fd0b80037a0 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9557700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9557700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.079+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:20:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9557700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:20:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9557700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:20:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:20:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.280+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:20:43.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:43.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:43.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9557700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:20:43.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9557700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:43.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:20:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:20:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:43.681+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:20:44.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9557700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9557700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9557700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9d58700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:20:44.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:44.482+0000 7fd0d9d58700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:20:45.744 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:45.744+0000 7f208dbd8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:20:45.746 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:45.747+0000 7f8905ca3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:20:45.827 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:45.827+0000 7f208dbd8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:20:45.836 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:45.837+0000 7f8905ca3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:20:45.904 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:45.904+0000 7f208dbd8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:20:45.920 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:45.920+0000 7f8905ca3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:20:45.977 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:45.977+0000 7f208dbd8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:20:46.000 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:46.000+0000 7f8905ca3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:20:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 msgr2=0x7fd0b8005c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:20:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0cb7fe700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.49:3300/0 conn(0x7fd0b80037a0 0x7fd0b8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 msgr2=0x7fd0b80086f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:20:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0cb7fe700 1 --2- 172.21.15.5:0/3167167418 >> v2:172.21.15.5:3300/0 conn(0x7fd0b8006250 0x7fd0b80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0da559700 1 -- 172.21.15.5:0/3167167418 reap_dead start 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0cb7fe700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0b8008280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d40ccbd0 con 0x7fd0d406edf0 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b80088a0 con 0x7fd0b8003bf0 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0cb7fe700 1 -- 172.21.15.5:0/3167167418 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d406e420 con 0x7fd0b8007e00 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0da559700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0b8008280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.079+0000 7fd0d8d56700 1 -- 172.21.15.5:0/3167167418 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609788771 0 0) 0x7fd0d40ccbd0 con 0x7fd0d406edf0 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0d8d56700 1 -- 172.21.15.5:0/3167167418 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c00035b0 con 0x7fd0d406edf0 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0d8d56700 1 -- 172.21.15.5:0/3167167418 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641533125 0 0) 0x7fd0b80088a0 con 0x7fd0b8003bf0 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0d8d56700 1 -- 172.21.15.5:0/3167167418 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d40ccbd0 con 0x7fd0b8003bf0 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0da559700 1 -- 172.21.15.5:0/3167167418 >> v1:172.21.15.49:6789/0 conn(0x7fd0b8003bf0 legacy=0x7fd0b80066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0da559700 1 -- 172.21.15.5:0/3167167418 >> v1:172.21.15.5:6789/0 conn(0x7fd0d406edf0 legacy=0x7fd0b800b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0da559700 1 -- 172.21.15.5:0/3167167418 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b80108c0 con 0x7fd0b8007e00 2024-05-08T23:20:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0da559700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0b8008280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4072a30 tx=0x7fd0cc0017e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ddc8a9cfcb0d5da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0d8d56700 1 -- 172.21.15.5:0/3167167418 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc005d70 con 0x7fd0b8007e00 2024-05-08T23:20:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0d8d56700 1 -- 172.21.15.5:0/3167167418 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc007110 con 0x7fd0b8007e00 2024-05-08T23:20:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.080+0000 7fd0d8d56700 1 -- 172.21.15.5:0/3167167418 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0074a0 con 0x7fd0b8007e00 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.081+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 msgr2=0x7fd0b8008280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.081+0000 7fd0dbfbc700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0b8008280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4072a30 tx=0x7fd0cc0017e0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.081+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/3167167418 shutdown_connections 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.081+0000 7fd0dbfbc700 1 --2- 172.21.15.5:0/3167167418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0b8008280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.081+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/3167167418 >> 172.21.15.5:0/3167167418 conn(0x7fd0d4099a50 msgr2=0x7fd0d40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.081+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/3167167418 shutdown_connections 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.081+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/3167167418 wait complete. 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.082+0000 7fd0dbfbc700 1 Processor -- start 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.082+0000 7fd0dbfbc700 1 -- start start 2024-05-08T23:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0dbfbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b8003bf0 0x7fd0d40c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0dbfbc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0d415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0dbfbc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d406edf0 0x7fd0d415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0dbfbc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40b0a80 con 0x7fd0d406edf0 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0dbfbc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40b1320 con 0x7fd0b8003bf0 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0dbfbc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d415ac30 con 0x7fd0b8007e00 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0da559700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d406edf0 0x7fd0d415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0d9557700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0d415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0d9d58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b8003bf0 0x7fd0d40c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0da559700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d406edf0 0x7fd0d415a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48306/0 (socket says 172.21.15.5:48306) 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0d9557700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0d415a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:56174/0 (socket says 172.21.15.5:56174) 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.083+0000 7fd0da559700 1 -- 172.21.15.5:0/4272250463 learned_addr learned my addr 172.21.15.5:0/4272250463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0da559700 1 -- 172.21.15.5:0/4272250463 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 msgr2=0x7fd0d415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0da559700 1 --2- 172.21.15.5:0/4272250463 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0d415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0da559700 1 -- 172.21.15.5:0/4272250463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b8003bf0 msgr2=0x7fd0d40c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0da559700 1 --2- 172.21.15.5:0/4272250463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b8003bf0 0x7fd0d40c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0da559700 1 -- 172.21.15.5:0/4272250463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d40cc980 con 0x7fd0d406edf0 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0d9557700 1 --2- 172.21.15.5:0/4272250463 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0d415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0da559700 1 --2- 172.21.15.5:0/4272250463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d406edf0 0x7fd0d415a6f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80061c0 tx=0x7fd0cc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0ca7fc700 1 -- 172.21.15.5:0/4272250463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0078f0 con 0x7fd0d406edf0 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0ca7fc700 1 -- 172.21.15.5:0/4272250463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc004a70 con 0x7fd0d406edf0 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d415e790 con 0x7fd0d406edf0 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0ca7fc700 1 -- 172.21.15.5:0/4272250463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc004e00 con 0x7fd0d406edf0 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d40b0fb0 con 0x7fd0d406edf0 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0ca7fc700 1 -- 172.21.15.5:0/4272250463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd0cc005050 con 0x7fd0d406edf0 2024-05-08T23:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.084+0000 7fd0ca7fc700 1 -- 172.21.15.5:0/4272250463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd0cc0146c0 con 0x7fd0d406edf0 2024-05-08T23:20:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.086+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d40b1490 con 0x7fd0d406edf0 2024-05-08T23:20:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.088+0000 7fd0ca7fc700 1 -- 172.21.15.5:0/4272250463 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd0cc025070 con 0x7fd0d406edf0 2024-05-08T23:20:46.158 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:46.158+0000 7f208dbd8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:20:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.169+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd0d4074bc0 con 0x7fd0d406edf0 2024-05-08T23:20:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.172+0000 7fd0ca7fc700 1 -- 172.21.15.5:0/4272250463 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd0cc019040 con 0x7fd0d406edf0 2024-05-08T23:20:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d406edf0 msgr2=0x7fd0d415a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 --2- 172.21.15.5:0/4272250463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d406edf0 0x7fd0d415a6f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80061c0 tx=0x7fd0cc001120 comp rx=0 tx=0).stop 2024-05-08T23:20:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 shutdown_connections 2024-05-08T23:20:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 --2- 172.21.15.5:0/4272250463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d406edf0 0x7fd0d415a6f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 --2- 172.21.15.5:0/4272250463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b8003bf0 0x7fd0d40c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 --2- 172.21.15.5:0/4272250463 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0b8007e00 0x7fd0d415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 >> 172.21.15.5:0/4272250463 conn(0x7fd0d4099a50 msgr2=0x7fd0d40ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 shutdown_connections 2024-05-08T23:20:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:46.173+0000 7fd0dbfbc700 1 -- 172.21.15.5:0/4272250463 wait complete. 2024-05-08T23:20:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-05-08T23:20:46.187 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T23:20:46.187 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T23:20:46.187 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:46.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T23:20:46.198 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:46.199+0000 7f8905ca3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:20:46.217 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:46.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T23:20:46.288 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:46.288 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T23:20:46.332 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:46.332+0000 7f208dbd8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:20:46.358 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:20:46.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T23:20:46.393 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:46.393+0000 7f8905ca3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:20:46.423 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:46.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T23:20:46.446 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:46.446+0000 7f208dbd8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:20:46.451 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:46.451 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T23:20:46.519 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:46.519 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T23:20:46.519 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:46.520+0000 7f8905ca3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:20:46.546 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:46.547+0000 7f208dbd8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:20:46.573 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:20:46.573 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T23:20:46.627 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:46.627+0000 7f208dbd8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:20:46.628 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:46.628+0000 7f8905ca3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:20:46.642 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd new 5d7f7048-8067-4dbf-aee2-245243a044ee 0 2024-05-08T23:20:46.714 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:46.714+0000 7f8905ca3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:20:46.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.856+0000 7f7cd1427700 1 Processor -- start 2024-05-08T23:20:46.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.856+0000 7f7cd1427700 1 -- start start 2024-05-08T23:20:46.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.857+0000 7f7cd1427700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ccc15b3b0 0x7f7ccc0cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.857+0000 7f7cd1427700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.857+0000 7f7cd1427700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ccc141960 0x7f7ccc140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.857+0000 7f7cd1427700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0cbda0 con 0x7f7ccc141960 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.857+0000 7f7cd1427700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0cc080 con 0x7f7ccc15b3b0 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.857+0000 7f7cd1427700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0cc360 con 0x7f7ccc15b790 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7cca7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7cca7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc0cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:56098/0 (socket says 172.21.15.49:56098) 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7cca7fc700 1 -- 172.21.15.49:0/3951615210 learned_addr learned my addr 172.21.15.49:0/3951615210 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7ccb7fe700 1 --2- 172.21.15.49:0/3951615210 >> v2:172.21.15.5:3300/0 conn(0x7f7ccc141960 0x7f7ccc140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7ccb7fe700 1 -- 172.21.15.49:0/3951615210 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 msgr2=0x7f7ccc0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7ccb7fe700 1 --2- 172.21.15.49:0/3951615210 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc0cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7ccb7fe700 1 -- 172.21.15.49:0/3951615210 >> v2:172.21.15.49:3300/0 conn(0x7f7ccc15b3b0 msgr2=0x7f7ccc0cf370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7ccb7fe700 1 --2- 172.21.15.49:0/3951615210 >> v2:172.21.15.49:3300/0 conn(0x7f7ccc15b3b0 0x7f7ccc0cf370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7ccb7fe700 1 -- 172.21.15.49:0/3951615210 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0cc640 con 0x7f7ccc141960 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7cca7fc700 1 --2- 172.21.15.49:0/3951615210 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc0cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.858+0000 7f7ccb7fe700 1 --2- 172.21.15.49:0/3951615210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc141960 0x7f7ccc140dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009d90 tx=0x7f7cc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8427c85c91daaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cc9ffb700 1 -- 172.21.15.49:0/3951615210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400e070 con 0x7f7ccc141960 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cc9ffb700 1 -- 172.21.15.49:0/3951615210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc400bae0 con 0x7f7ccc141960 2024-05-08T23:20:46.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cc9ffb700 1 -- 172.21.15.49:0/3951615210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400be90 con 0x7f7ccc141960 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cd1427700 1 -- 172.21.15.49:0/3951615210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc141960 msgr2=0x7f7ccc140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cd1427700 1 --2- 172.21.15.49:0/3951615210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc141960 0x7f7ccc140dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009d90 tx=0x7f7cc400b330 comp rx=0 tx=0).stop 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cd1427700 1 -- 172.21.15.49:0/3951615210 shutdown_connections 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cd1427700 1 --2- 172.21.15.49:0/3951615210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc141960 0x7f7ccc140dc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cd1427700 1 --2- 172.21.15.49:0/3951615210 >> v2:172.21.15.49:3300/0 conn(0x7f7ccc15b3b0 0x7f7ccc0cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cd1427700 1 --2- 172.21.15.49:0/3951615210 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc0cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.859+0000 7f7cd1427700 1 -- 172.21.15.49:0/3951615210 >> 172.21.15.49:0/3951615210 conn(0x7f7ccc0cfce0 msgr2=0x7f7ccc0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.860+0000 7f7cd1427700 1 -- 172.21.15.49:0/3951615210 shutdown_connections 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.860+0000 7f7cd1427700 1 -- 172.21.15.49:0/3951615210 wait complete. 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.861+0000 7f7cd1427700 1 Processor -- start 2024-05-08T23:20:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.861+0000 7f7cd1427700 1 -- start start 2024-05-08T23:20:46.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cd1427700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ccc141960 0x7f7ccc157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cd1427700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc15b3b0 0x7f7ccc131f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cd1427700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc132460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:46.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cd1427700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc133d30 con 0x7f7ccc15b3b0 2024-05-08T23:20:46.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cd1427700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc132c90 con 0x7f7ccc141960 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cd1427700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc132f70 con 0x7f7ccc15b790 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ccc141960 0x7f7ccc157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ccc141960 0x7f7ccc157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54692/0 (socket says 172.21.15.49:54692) 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 -- 172.21.15.49:0/1307310865 learned_addr learned my addr 172.21.15.49:0/1307310865 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccb7fe700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc132460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cca7fc700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc15b3b0 0x7f7ccc131f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 -- 172.21.15.49:0/1307310865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 msgr2=0x7f7ccc132460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc132460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 -- 172.21.15.49:0/1307310865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc15b3b0 msgr2=0x7f7ccc131f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc15b3b0 0x7f7ccc131f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 -- 172.21.15.49:0/1307310865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc009550 con 0x7f7ccc141960 2024-05-08T23:20:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7ccaffd700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ccc141960 0x7f7ccc157970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000b550 tx=0x7f7cc000b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cb37fe700 1 -- 172.21.15.49:0/1307310865 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0011830 con 0x7f7ccc141960 2024-05-08T23:20:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cb37fe700 1 -- 172.21.15.49:0/1307310865 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc0011990 con 0x7f7ccc141960 2024-05-08T23:20:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.862+0000 7f7cb37fe700 1 -- 172.21.15.49:0/1307310865 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0011d20 con 0x7f7ccc141960 2024-05-08T23:20:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.863+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc4009a70 con 0x7f7ccc141960 2024-05-08T23:20:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.863+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc133530 con 0x7f7ccc141960 2024-05-08T23:20:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.863+0000 7f7cb37fe700 1 -- 172.21.15.49:0/1307310865 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f7cc001a700 con 0x7f7ccc141960 2024-05-08T23:20:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.863+0000 7f7cb37fe700 1 -- 172.21.15.49:0/1307310865 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7cc0019710 con 0x7f7ccc141960 2024-05-08T23:20:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.864+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc074bc0 con 0x7f7ccc141960 2024-05-08T23:20:46.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.866+0000 7f7cb37fe700 1 -- 172.21.15.49:0/1307310865 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f7cc0039dc0 con 0x7f7ccc141960 2024-05-08T23:20:46.872 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:46.868+0000 7f208dbd8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:20:46.942 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:46.942+0000 7f208dbd8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:20:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.953+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5d7f7048-8067-4dbf-aee2-245243a044ee", "id": 0} v 0) v1 -- 0x7f7ccc0b1580 con 0x7f7ccc141960 2024-05-08T23:20:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.958+0000 7f7cb37fe700 1 -- 172.21.15.49:0/1307310865 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5d7f7048-8067-4dbf-aee2-245243a044ee", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7cc0028030 con 0x7f7ccc141960 2024-05-08T23:20:46.958 INFO:teuthology.orchestra.run.smithi049.stdout:0 2024-05-08T23:20:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ccc141960 msgr2=0x7f7ccc157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ccc141960 0x7f7ccc157970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000b550 tx=0x7f7cc000b860 comp rx=0 tx=0).stop 2024-05-08T23:20:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 shutdown_connections 2024-05-08T23:20:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ccc15b790 0x7f7ccc132460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ccc141960 0x7f7ccc157970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 --2- 172.21.15.49:0/1307310865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ccc15b3b0 0x7f7ccc131f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 >> 172.21.15.49:0/1307310865 conn(0x7f7ccc0cfce0 msgr2=0x7f7ccc06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 shutdown_connections 2024-05-08T23:20:46.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:46.959+0000 7f7cd1427700 1 -- 172.21.15.49:0/1307310865 wait complete. 2024-05-08T23:20:46.971 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd new 1128f99e-4bac-4e93-a928-21f11858a313 1 2024-05-08T23:20:46.972 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:46.972+0000 7f8905ca3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:20:47.052 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:47.052+0000 7f8905ca3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:20:47.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.096+0000 7fee894d3700 1 Processor -- start 2024-05-08T23:20:47.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.096+0000 7fee894d3700 1 -- start start 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee894d3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fee8406b070 0x7fee84168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee894d3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee84145040 con 0x7fee840b4d50 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee894d3700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee840b16e0 con 0x7fee840b0d80 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee894d3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee840b19c0 con 0x7fee8406b070 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee82ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fee840b0d80 0x7fee840b3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:32908/0 (socket says 172.21.15.49:32908) 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee82ffd700 1 -- 172.21.15.49:0/35591600 learned_addr learned my addr 172.21.15.49:0/35591600 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee837fe700 1 --2- 172.21.15.49:0/35591600 >> v2:172.21.15.5:3300/0 conn(0x7fee8406b070 0x7fee84168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780918482 0 0) 0x7fee840b16e0 con 0x7fee840b0d80 2024-05-08T23:20:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee6c0035b0 con 0x7fee840b0d80 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945849586 0 0) 0x7fee84145040 con 0x7fee840b4d50 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee840b16e0 con 0x7fee840b4d50 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 <== mon.1 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381702804 0 0) 0x7fee6c0035b0 con 0x7fee840b0d80 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 >> v2:172.21.15.5:3300/0 conn(0x7fee8406b070 msgr2=0x7fee84168490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 --2- 172.21.15.49:0/35591600 >> v2:172.21.15.5:3300/0 conn(0x7fee8406b070 0x7fee84168490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 >> v1:172.21.15.5:6789/0 conn(0x7fee840b4d50 legacy=0x7fee8406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840b1ca0 con 0x7fee840b0d80 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 <== mon.1 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fee74001670 con 0x7fee840b0d80 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 >> v1:172.21.15.49:6789/0 conn(0x7fee840b0d80 legacy=0x7fee840b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee6c003b90 0x7fee6c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c005210 0x7fee6c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840b19c0 con 0x7fee6c003b90 2024-05-08T23:20:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c00c090 con 0x7fee6c004850 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c00c250 con 0x7fee6c005210 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee827fc700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee82ffd700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee6c003b90 0x7fee6c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.097+0000 7fee837fe700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c005210 0x7fee6c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee827fc700 1 -- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c005210 msgr2=0x7fee6c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee827fc700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c005210 0x7fee6c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee827fc700 1 -- 172.21.15.49:0/35591600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee6c003b90 msgr2=0x7fee6c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee827fc700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee6c003b90 0x7fee6c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee827fc700 1 -- 172.21.15.49:0/35591600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c0035b0 con 0x7fee6c004850 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee837fe700 1 -- 172.21.15.49:0/35591600 reap_dead start 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee827fc700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee78001940 tx=0x7fee6c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec64b971e0123fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78001430 con 0x7fee6c004850 2024-05-08T23:20:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee78001d70 con 0x7fee6c004850 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.098+0000 7fee81ffb700 1 -- 172.21.15.49:0/35591600 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78005510 con 0x7fee6c004850 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.099+0000 7fee894d3700 1 -- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6c004850 msgr2=0x7fee6c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.099+0000 7fee894d3700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee78001940 tx=0x7fee6c005880 comp rx=0 tx=0).stop 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.099+0000 7fee894d3700 1 -- 172.21.15.49:0/35591600 shutdown_connections 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.099+0000 7fee894d3700 1 --2- 172.21.15.49:0/35591600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.099+0000 7fee894d3700 1 -- 172.21.15.49:0/35591600 >> 172.21.15.49:0/35591600 conn(0x7fee84099a50 msgr2=0x7fee840b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.099+0000 7fee894d3700 1 -- 172.21.15.49:0/35591600 shutdown_connections 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.099+0000 7fee894d3700 1 -- 172.21.15.49:0/35591600 wait complete. 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.100+0000 7fee894d3700 1 Processor -- start 2024-05-08T23:20:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.100+0000 7fee894d3700 1 -- start start 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee894d3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c004850 0x7fee8413deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee894d3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee8415dab0 0x7fee84150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee894d3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee841453d0 0x7fee84145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee894d3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84145dc0 con 0x7fee841453d0 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee894d3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee841f34e0 con 0x7fee8415dab0 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee894d3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee84150940 con 0x7fee6c004850 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c004850 0x7fee8413deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee8415dab0 0x7fee84150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c004850 0x7fee8413deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:56108/0 (socket says 172.21.15.49:56108) 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 -- 172.21.15.49:0/1829236906 learned_addr learned my addr 172.21.15.49:0/1829236906 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee837fe700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee841453d0 0x7fee84145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 -- 172.21.15.49:0/1829236906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee8415dab0 msgr2=0x7fee84150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee8415dab0 0x7fee84150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 -- 172.21.15.49:0/1829236906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee841453d0 msgr2=0x7fee84145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee841453d0 0x7fee84145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 -- 172.21.15.49:0/1829236906 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8415dfb0 con 0x7fee6c004850 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee827fc700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee8415dab0 0x7fee84150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.101+0000 7fee82ffd700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c004850 0x7fee8413deb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee740072c0 tx=0x7fee740075d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.102+0000 7fee6b7fe700 1 -- 172.21.15.49:0/1829236906 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74011640 con 0x7fee6c004850 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.102+0000 7fee6b7fe700 1 -- 172.21.15.49:0/1829236906 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee740117e0 con 0x7fee6c004850 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.102+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8415e290 con 0x7fee6c004850 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.102+0000 7fee6b7fe700 1 -- 172.21.15.49:0/1829236906 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74011b90 con 0x7fee6c004850 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.102+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8415e850 con 0x7fee6c004850 2024-05-08T23:20:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.102+0000 7fee6b7fe700 1 -- 172.21.15.49:0/1829236906 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fee7401d460 con 0x7fee6c004850 2024-05-08T23:20:47.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.102+0000 7fee6b7fe700 1 -- 172.21.15.49:0/1829236906 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fee74013910 con 0x7fee6c004850 2024-05-08T23:20:47.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.103+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee840b3710 con 0x7fee6c004850 2024-05-08T23:20:47.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.105+0000 7fee6b7fe700 1 -- 172.21.15.49:0/1829236906 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fee74013ad0 con 0x7fee6c004850 2024-05-08T23:20:47.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.191+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1128f99e-4bac-4e93-a928-21f11858a313", "id": 1} v 0) v1 -- 0x7fee84074bc0 con 0x7fee6c004850 2024-05-08T23:20:47.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.194+0000 7fee6b7fe700 1 -- 172.21.15.49:0/1829236906 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1128f99e-4bac-4e93-a928-21f11858a313", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fee74013cd0 con 0x7fee6c004850 2024-05-08T23:20:47.194 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-05-08T23:20:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c004850 msgr2=0x7fee8413deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c004850 0x7fee8413deb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee740072c0 tx=0x7fee740075d0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 shutdown_connections 2024-05-08T23:20:47.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee841453d0 0x7fee84145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee6c004850 0x7fee8413deb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 --2- 172.21.15.49:0/1829236906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee8415dab0 0x7fee84150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 >> 172.21.15.49:0/1829236906 conn(0x7fee84099a50 msgr2=0x7fee84144830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:47.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 shutdown_connections 2024-05-08T23:20:47.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.195+0000 7fee894d3700 1 -- 172.21.15.49:0/1829236906 wait complete. 2024-05-08T23:20:47.207 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd new 99a97325-4923-4c97-aa94-52a2eaa66907 2 2024-05-08T23:20:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.332+0000 7f9424ebe700 1 Processor -- start 2024-05-08T23:20:47.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.332+0000 7f9424ebe700 1 -- start start 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f9424ebe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f942006a960 0x7f9420099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f9424ebe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f94200aea70 0x7f94200aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f9424ebe700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942015c920 con 0x7f94200b5500 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f9424ebe700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200ad6b0 con 0x7f94200aea70 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f9424ebe700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200b2700 con 0x7f942006a960 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f941e59c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f942006a960 0x7f9420099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f941e59c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f942006a960 0x7f9420099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54714/0 (socket says 172.21.15.49:54714) 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f941e59c700 1 -- 172.21.15.49:0/3409650960 learned_addr learned my addr 172.21.15.49:0/3409650960 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.333+0000 7f941ed9d700 1 --2- 172.21.15.49:0/3409650960 >> v2:172.21.15.5:3300/0 conn(0x7f94200aea70 0x7f94200aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.334+0000 7f941e59c700 1 -- 172.21.15.49:0/3409650960 >> v2:172.21.15.5:3300/0 conn(0x7f94200aea70 msgr2=0x7f94200aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:47.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.334+0000 7f941e59c700 1 --2- 172.21.15.49:0/3409650960 >> v2:172.21.15.5:3300/0 conn(0x7f94200aea70 0x7f94200aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.334+0000 7f941e59c700 1 -- 172.21.15.49:0/3409650960 >> v1:172.21.15.5:6789/0 conn(0x7f94200b5500 legacy=0x7f942006ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.334+0000 7f941e59c700 1 -- 172.21.15.49:0/3409650960 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94200b2910 con 0x7f942006a960 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.334+0000 7f941e59c700 1 --2- 172.21.15.49:0/3409650960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f942006a960 0x7f9420099a50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94100098f0 tx=0x7f9410009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7668ddd963d82f13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.334+0000 7f941d59a700 1 -- 172.21.15.49:0/3409650960 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410011830 con 0x7f942006a960 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.334+0000 7f941d59a700 1 -- 172.21.15.49:0/3409650960 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410011990 con 0x7f942006a960 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.334+0000 7f941d59a700 1 -- 172.21.15.49:0/3409650960 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410011d20 con 0x7f942006a960 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f9424ebe700 1 -- 172.21.15.49:0/3409650960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f942006a960 msgr2=0x7f9420099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f9424ebe700 1 --2- 172.21.15.49:0/3409650960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f942006a960 0x7f9420099a50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94100098f0 tx=0x7f9410009c00 comp rx=0 tx=0).stop 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f9424ebe700 1 -- 172.21.15.49:0/3409650960 shutdown_connections 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f9424ebe700 1 --2- 172.21.15.49:0/3409650960 >> v2:172.21.15.5:3300/0 conn(0x7f94200aea70 0x7f94200aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f9424ebe700 1 --2- 172.21.15.49:0/3409650960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f942006a960 0x7f9420099a50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f9424ebe700 1 -- 172.21.15.49:0/3409650960 >> 172.21.15.49:0/3409650960 conn(0x7f94200b5a30 msgr2=0x7f94200b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f9424ebe700 1 -- 172.21.15.49:0/3409650960 shutdown_connections 2024-05-08T23:20:47.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f9424ebe700 1 -- 172.21.15.49:0/3409650960 wait complete. 2024-05-08T23:20:47.338 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:47.335+0000 7f208dbd8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:20:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.336+0000 7f9424ebe700 1 Processor -- start 2024-05-08T23:20:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.336+0000 7f9424ebe700 1 -- start start 2024-05-08T23:20:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f9424ebe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f942006a960 0x7f942015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f9424ebe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94200aea70 0x7f94201e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f9424ebe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f94200b5500 0x7f9420149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f9424ebe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94201e3ed0 con 0x7f942006a960 2024-05-08T23:20:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f9424ebe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94201e41b0 con 0x7f94200aea70 2024-05-08T23:20:47.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f9424ebe700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94201e4490 con 0x7f94200b5500 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94200aea70 0x7f94201e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941ed9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f94200b5500 0x7f9420149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94200aea70 0x7f94201e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54722/0 (socket says 172.21.15.49:54722) 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f942006a960 0x7f942015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941dd9b700 1 -- 172.21.15.49:0/2536141826 learned_addr learned my addr 172.21.15.49:0/2536141826 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941ed9d700 1 -- 172.21.15.49:0/2536141826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94200aea70 msgr2=0x7f94201e74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941ed9d700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94200aea70 0x7f94201e74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941ed9d700 1 -- 172.21.15.49:0/2536141826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f942006a960 msgr2=0x7f942015cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941ed9d700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f942006a960 0x7f942015cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.337+0000 7f941ed9d700 1 -- 172.21.15.49:0/2536141826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410009550 con 0x7f94200b5500 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f941e59c700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f942006a960 0x7f942015cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f941dd9b700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94200aea70 0x7f94201e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f941ed9d700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f94200b5500 0x7f9420149860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f941800efc0 tx=0x7f941800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f940effd700 1 -- 172.21.15.49:0/2536141826 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941800cc90 con 0x7f94200b5500 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f940effd700 1 -- 172.21.15.49:0/2536141826 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941800f040 con 0x7f94200b5500 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94201e4770 con 0x7f94200b5500 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f940effd700 1 -- 172.21.15.49:0/2536141826 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94180146c0 con 0x7f94200b5500 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94201550a0 con 0x7f94200b5500 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f940effd700 1 -- 172.21.15.49:0/2536141826 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9418014b20 con 0x7f94200b5500 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.338+0000 7f940effd700 1 -- 172.21.15.49:0/2536141826 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f941801dd70 con 0x7f94200b5500 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.339+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420074bc0 con 0x7f94200b5500 2024-05-08T23:20:47.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.341+0000 7f940effd700 1 -- 172.21.15.49:0/2536141826 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f941803bdc0 con 0x7f94200b5500 2024-05-08T23:20:47.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.427+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd new", "uuid": "99a97325-4923-4c97-aa94-52a2eaa66907", "id": 2} v 0) v1 -- 0x7f942006b590 con 0x7f94200b5500 2024-05-08T23:20:47.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.431+0000 7f940effd700 1 -- 172.21.15.49:0/2536141826 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "99a97325-4923-4c97-aa94-52a2eaa66907", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f941802a030 con 0x7f94200b5500 2024-05-08T23:20:47.432 INFO:teuthology.orchestra.run.smithi049.stdout:2 2024-05-08T23:20:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.431+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f94200b5500 msgr2=0x7f9420149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.431+0000 7f9424ebe700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f94200b5500 0x7f9420149860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f941800efc0 tx=0x7f941800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.431+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 shutdown_connections 2024-05-08T23:20:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.431+0000 7f9424ebe700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f94200b5500 0x7f9420149860 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.431+0000 7f9424ebe700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f942006a960 0x7f942015cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.431+0000 7f9424ebe700 1 --2- 172.21.15.49:0/2536141826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94200aea70 0x7f94201e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.431+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 >> 172.21.15.49:0/2536141826 conn(0x7f94200b5a30 msgr2=0x7f9420159730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.432+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 shutdown_connections 2024-05-08T23:20:47.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.432+0000 7f9424ebe700 1 -- 172.21.15.49:0/2536141826 wait complete. 2024-05-08T23:20:47.444 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd new e2783735-4661-4f39-b55c-00c3fcdc8124 3 2024-05-08T23:20:47.473 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:47.473+0000 7f8905ca3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:20:47.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.574+0000 7f40470ba700 1 Processor -- start 2024-05-08T23:20:47.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.574+0000 7f40470ba700 1 -- start start 2024-05-08T23:20:47.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f40470ba700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f40400ce120 0x7f404006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f40470ba700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400cb2a0 0x7f40400cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f40470ba700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404006fbf0 con 0x7f404015db00 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f40470ba700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404006edb0 con 0x7f40400ce120 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f40470ba700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404006f090 con 0x7f40400cb2a0 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f403ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f40400ce120 0x7f404006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f4045657700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400cb2a0 0x7f40400cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f403ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f40400ce120 0x7f404006e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54728/0 (socket says 172.21.15.49:54728) 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f403ffff700 1 -- 172.21.15.49:0/405191144 learned_addr learned my addr 172.21.15.49:0/405191144 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f403ffff700 1 -- 172.21.15.49:0/405191144 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400cb2a0 msgr2=0x7f40400cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f403ffff700 1 --2- 172.21.15.49:0/405191144 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400cb2a0 0x7f40400cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f403ffff700 1 -- 172.21.15.49:0/405191144 >> v1:172.21.15.49:6789/0 conn(0x7f404015db00 legacy=0x7f404006dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f403ffff700 1 -- 172.21.15.49:0/405191144 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404006f370 con 0x7f40400ce120 2024-05-08T23:20:47.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.575+0000 7f4045657700 1 --2- 172.21.15.49:0/405191144 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400cb2a0 0x7f40400cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f403ffff700 1 --2- 172.21.15.49:0/405191144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400ce120 0x7f404006e390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f402c0098f0 tx=0x7f402c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d389633a776d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f403f7fe700 1 -- 172.21.15.49:0/405191144 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c011830 con 0x7f40400ce120 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f403f7fe700 1 -- 172.21.15.49:0/405191144 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c011990 con 0x7f40400ce120 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f403f7fe700 1 -- 172.21.15.49:0/405191144 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c011d20 con 0x7f40400ce120 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f40470ba700 1 -- 172.21.15.49:0/405191144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400ce120 msgr2=0x7f404006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f40470ba700 1 --2- 172.21.15.49:0/405191144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400ce120 0x7f404006e390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f402c0098f0 tx=0x7f402c009c00 comp rx=0 tx=0).stop 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f40470ba700 1 -- 172.21.15.49:0/405191144 shutdown_connections 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f40470ba700 1 --2- 172.21.15.49:0/405191144 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400cb2a0 0x7f40400cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f40470ba700 1 --2- 172.21.15.49:0/405191144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400ce120 0x7f404006e390 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f40470ba700 1 -- 172.21.15.49:0/405191144 >> 172.21.15.49:0/405191144 conn(0x7f4040099a50 msgr2=0x7f40400d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.576+0000 7f40470ba700 1 -- 172.21.15.49:0/405191144 shutdown_connections 2024-05-08T23:20:47.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.577+0000 7f40470ba700 1 -- 172.21.15.49:0/405191144 wait complete. 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.578+0000 7f40470ba700 1 Processor -- start 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.578+0000 7f40470ba700 1 -- start start 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f40470ba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400cb2a0 0x7f40400b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f40470ba700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400ce120 0x7f40400b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f40470ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404015db00 0x7f4040151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f40470ba700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40400b3600 con 0x7f404015db00 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f40470ba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40400b38e0 con 0x7f40400cb2a0 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f40470ba700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40400b3bc0 con 0x7f40400ce120 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400ce120 0x7f40400b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400ce120 0x7f40400b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:56140/0 (socket says 172.21.15.49:56140) 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 -- 172.21.15.49:0/2340381187 learned_addr learned my addr 172.21.15.49:0/2340381187 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f4044e56700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400cb2a0 0x7f40400b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f4045657700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404015db00 0x7f4040151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 -- 172.21.15.49:0/2340381187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400cb2a0 msgr2=0x7f40400b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400cb2a0 0x7f40400b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 -- 172.21.15.49:0/2340381187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404015db00 msgr2=0x7f4040151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404015db00 0x7f4040151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 -- 172.21.15.49:0/2340381187 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038009a70 con 0x7f40400ce120 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f4045657700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404015db00 0x7f4040151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403ffff700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400ce120 0x7f40400b46a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f402c000c00 tx=0x7f402c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403d7fa700 1 -- 172.21.15.49:0/2340381187 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c004480 con 0x7f40400ce120 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403d7fa700 1 -- 172.21.15.49:0/2340381187 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c01f070 con 0x7f40400ce120 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c009550 con 0x7f40400ce120 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.579+0000 7f403d7fa700 1 -- 172.21.15.49:0/2340381187 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0038c0 con 0x7f40400ce120 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.580+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40400b4180 con 0x7f40400ce120 2024-05-08T23:20:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.580+0000 7f403d7fa700 1 -- 172.21.15.49:0/2340381187 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f402c021030 con 0x7f40400ce120 2024-05-08T23:20:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.580+0000 7f403d7fa700 1 -- 172.21.15.49:0/2340381187 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f402c01ae30 con 0x7f40400ce120 2024-05-08T23:20:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.581+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40400b4370 con 0x7f40400ce120 2024-05-08T23:20:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.583+0000 7f403d7fa700 1 -- 172.21.15.49:0/2340381187 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f402c015020 con 0x7f40400ce120 2024-05-08T23:20:47.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.670+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e2783735-4661-4f39-b55c-00c3fcdc8124", "id": 3} v 0) v1 -- 0x7f4040074bc0 con 0x7f40400ce120 2024-05-08T23:20:47.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.674+0000 7f403d7fa700 1 -- 172.21.15.49:0/2340381187 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2783735-4661-4f39-b55c-00c3fcdc8124", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f402c01a450 con 0x7f40400ce120 2024-05-08T23:20:47.675 INFO:teuthology.orchestra.run.smithi049.stdout:3 2024-05-08T23:20:47.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.674+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400ce120 msgr2=0x7f40400b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.674+0000 7f40470ba700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400ce120 0x7f40400b46a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f402c000c00 tx=0x7f402c004750 comp rx=0 tx=0).stop 2024-05-08T23:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.675+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 shutdown_connections 2024-05-08T23:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.675+0000 7f40470ba700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404015db00 0x7f4040151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.675+0000 7f40470ba700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40400cb2a0 0x7f40400b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.675+0000 7f40470ba700 1 --2- 172.21.15.49:0/2340381187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f40400ce120 0x7f40400b46a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.675+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 >> 172.21.15.49:0/2340381187 conn(0x7f4040099a50 msgr2=0x7f40400cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.675+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 shutdown_connections 2024-05-08T23:20:47.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.675+0000 7f40470ba700 1 -- 172.21.15.49:0/2340381187 wait complete. 2024-05-08T23:20:47.680 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:47.679+0000 7f208dbd8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:20:47.687 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd new 90b7fb65-a3d7-4e06-ab90-c04c884689c3 4 2024-05-08T23:20:47.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.831+0000 7f386c9af700 1 Processor -- start 2024-05-08T23:20:47.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.831+0000 7f386c9af700 1 -- start start 2024-05-08T23:20:47.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.832+0000 7f386c9af700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f386815ba10 0x7f38680cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.832+0000 7f386c9af700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38680cfde0 0x7f386806a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.832+0000 7f386c9af700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386806aea0 0x7f38680ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.832+0000 7f386c9af700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386815f3e0 con 0x7f386815ba10 2024-05-08T23:20:47.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.832+0000 7f386c9af700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38681603b0 con 0x7f38680cfde0 2024-05-08T23:20:47.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.832+0000 7f386c9af700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386806b2b0 con 0x7f386806aea0 2024-05-08T23:20:47.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.832+0000 7f3865d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38680cfde0 0x7f386806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3866d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386806aea0 0x7f38680ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3866d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386806aea0 0x7f38680ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:56154/0 (socket says 172.21.15.49:56154) 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3865d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38680cfde0 0x7f386806a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54748/0 (socket says 172.21.15.49:54748) 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3866d9d700 1 -- 172.21.15.49:0/3481480746 learned_addr learned my addr 172.21.15.49:0/3481480746 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f386659c700 1 --2- 172.21.15.49:0/3481480746 >> v2:172.21.15.5:3300/0 conn(0x7f386815ba10 0x7f38680cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3865d9b700 1 -- 172.21.15.49:0/3481480746 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386806aea0 msgr2=0x7f38680ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3865d9b700 1 --2- 172.21.15.49:0/3481480746 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386806aea0 0x7f38680ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3865d9b700 1 -- 172.21.15.49:0/3481480746 >> v2:172.21.15.5:3300/0 conn(0x7f386815ba10 msgr2=0x7f38680cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3865d9b700 1 --2- 172.21.15.49:0/3481480746 >> v2:172.21.15.5:3300/0 conn(0x7f386815ba10 0x7f38680cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3865d9b700 1 -- 172.21.15.49:0/3481480746 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386815f5f0 con 0x7f38680cfde0 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3866d9d700 1 --2- 172.21.15.49:0/3481480746 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386806aea0 0x7f38680ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f3865d9b700 1 --2- 172.21.15.49:0/3481480746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38680cfde0 0x7f386806a960 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3868071ef0 tx=0x7f385c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d8f46f0dcb834a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:47.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f386559a700 1 -- 172.21.15.49:0/3481480746 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c011830 con 0x7f38680cfde0 2024-05-08T23:20:47.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f386559a700 1 -- 172.21.15.49:0/3481480746 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c011990 con 0x7f38680cfde0 2024-05-08T23:20:47.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.833+0000 7f386559a700 1 -- 172.21.15.49:0/3481480746 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c011d20 con 0x7f38680cfde0 2024-05-08T23:20:47.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 -- 172.21.15.49:0/3481480746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38680cfde0 msgr2=0x7f386806a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 --2- 172.21.15.49:0/3481480746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38680cfde0 0x7f386806a960 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3868071ef0 tx=0x7f385c009b20 comp rx=0 tx=0).stop 2024-05-08T23:20:47.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 -- 172.21.15.49:0/3481480746 shutdown_connections 2024-05-08T23:20:47.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 --2- 172.21.15.49:0/3481480746 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386806aea0 0x7f38680ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 --2- 172.21.15.49:0/3481480746 >> v2:172.21.15.5:3300/0 conn(0x7f386815ba10 0x7f38680cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 --2- 172.21.15.49:0/3481480746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38680cfde0 0x7f386806a960 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 -- 172.21.15.49:0/3481480746 >> 172.21.15.49:0/3481480746 conn(0x7f3868099a50 msgr2=0x7f38680d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:47.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 -- 172.21.15.49:0/3481480746 shutdown_connections 2024-05-08T23:20:47.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.834+0000 7f386c9af700 1 -- 172.21.15.49:0/3481480746 wait complete. 2024-05-08T23:20:47.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.836+0000 7f386c9af700 1 Processor -- start 2024-05-08T23:20:47.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.836+0000 7f386c9af700 1 -- start start 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386c9af700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806aea0 0x7f38680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386c9af700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38680cfde0 0x7f38680b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386c9af700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386815ba10 0x7f38680b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386c9af700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386815d860 con 0x7f38680cfde0 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386c9af700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38680b1500 con 0x7f386806aea0 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386c9af700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38680b42c0 con 0x7f386815ba10 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806aea0 0x7f38680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f3866d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386815ba10 0x7f38680b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806aea0 0x7f38680b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54760/0 (socket says 172.21.15.49:54760) 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f3865d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38680cfde0 0x7f38680b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 -- 172.21.15.49:0/3286334323 learned_addr learned my addr 172.21.15.49:0/3286334323 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:47.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 -- 172.21.15.49:0/3286334323 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386815ba10 msgr2=0x7f38680b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386815ba10 0x7f38680b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 -- 172.21.15.49:0/3286334323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38680cfde0 msgr2=0x7f38680b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38680cfde0 0x7f38680b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 -- 172.21.15.49:0/3286334323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860009b30 con 0x7f386806aea0 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f3865d9b700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38680cfde0 0x7f38680b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f3866d9d700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386815ba10 0x7f38680b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386659c700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806aea0 0x7f38680b4f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f385800e870 tx=0x7f385800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f3856ffd700 1 -- 172.21.15.49:0/3286334323 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858009590 con 0x7f386806aea0 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f3856ffd700 1 -- 172.21.15.49:0/3286334323 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3858013680 con 0x7f386806aea0 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f3856ffd700 1 -- 172.21.15.49:0/3286334323 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38580139f0 con 0x7f386806aea0 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.838+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c0094a0 con 0x7f386806aea0 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.839+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386815dd50 con 0x7f386806aea0 2024-05-08T23:20:47.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.839+0000 7f3856ffd700 1 -- 172.21.15.49:0/3286334323 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f385801c450 con 0x7f386806aea0 2024-05-08T23:20:47.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.839+0000 7f3856ffd700 1 -- 172.21.15.49:0/3286334323 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f385801cc10 con 0x7f386806aea0 2024-05-08T23:20:47.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.839+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386815e3a0 con 0x7f386806aea0 2024-05-08T23:20:47.843 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:47.841+0000 7f8905ca3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:20:47.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.842+0000 7f3856ffd700 1 -- 172.21.15.49:0/3286334323 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f385803bdc0 con 0x7f386806aea0 2024-05-08T23:20:47.873 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:47.873+0000 7f208dbd8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:20:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.942+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd new", "uuid": "90b7fb65-a3d7-4e06-ab90-c04c884689c3", "id": 4} v 0) v1 -- 0x7f3868074bc0 con 0x7f386806aea0 2024-05-08T23:20:47.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.945+0000 7f3856ffd700 1 -- 172.21.15.49:0/3286334323 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "90b7fb65-a3d7-4e06-ab90-c04c884689c3", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f385802a070 con 0x7f386806aea0 2024-05-08T23:20:47.945 INFO:teuthology.orchestra.run.smithi049.stdout:4 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.945+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806aea0 msgr2=0x7f38680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.945+0000 7f386c9af700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806aea0 0x7f38680b4f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f385800e870 tx=0x7f385800eb80 comp rx=0 tx=0).stop 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.945+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 shutdown_connections 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.945+0000 7f386c9af700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f386815ba10 0x7f38680b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.945+0000 7f386c9af700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806aea0 0x7f38680b4f00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.945+0000 7f386c9af700 1 --2- 172.21.15.49:0/3286334323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38680cfde0 0x7f38680b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.946+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 >> 172.21.15.49:0/3286334323 conn(0x7f3868099a50 msgr2=0x7f38680b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.946+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 shutdown_connections 2024-05-08T23:20:47.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:47.946+0000 7f386c9af700 1 -- 172.21.15.49:0/3286334323 wait complete. 2024-05-08T23:20:47.958 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:47.957+0000 7f208dbd8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:20:47.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd new f2df4605-e5ec-41b9-8ec6-0bfc843354c7 5 2024-05-08T23:20:48.021 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:48.021+0000 7f8905ca3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:20:48.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.088+0000 7fa505d9d700 1 Processor -- start 2024-05-08T23:20:48.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.088+0000 7fa505d9d700 1 -- start start 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa505d9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa50015a540 0x7fa50015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa505d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50006eea0 0x7fa5000ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa505d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5000719e0 0x7fa5000ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa505d9d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5000b3230 con 0x7fa5000719e0 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa505d9d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa500072b20 con 0x7fa50015a540 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa505d9d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50015efe0 con 0x7fa50006eea0 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa4fffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50006eea0 0x7fa5000ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa504d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa50015a540 0x7fa50015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa504d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa50015a540 0x7fa50015eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54766/0 (socket says 172.21.15.49:54766) 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa4fffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50006eea0 0x7fa5000ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:56168/0 (socket says 172.21.15.49:56168) 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa504d9b700 1 -- 172.21.15.49:0/1906685528 learned_addr learned my addr 172.21.15.49:0/1906685528 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa50559c700 1 --2- 172.21.15.49:0/1906685528 >> v2:172.21.15.5:3300/0 conn(0x7fa5000719e0 0x7fa5000ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa4fffff700 1 -- 172.21.15.49:0/1906685528 >> v2:172.21.15.49:3300/0 conn(0x7fa50015a540 msgr2=0x7fa50015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa4fffff700 1 --2- 172.21.15.49:0/1906685528 >> v2:172.21.15.49:3300/0 conn(0x7fa50015a540 0x7fa50015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa4fffff700 1 -- 172.21.15.49:0/1906685528 >> v2:172.21.15.5:3300/0 conn(0x7fa5000719e0 msgr2=0x7fa5000ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa4fffff700 1 --2- 172.21.15.49:0/1906685528 >> v2:172.21.15.5:3300/0 conn(0x7fa5000719e0 0x7fa5000ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.089+0000 7fa4fffff700 1 -- 172.21.15.49:0/1906685528 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa500071fa0 con 0x7fa50006eea0 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa504d9b700 1 --2- 172.21.15.49:0/1906685528 >> v2:172.21.15.49:3300/0 conn(0x7fa50015a540 0x7fa50015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa4fffff700 1 --2- 172.21.15.49:0/1906685528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50006eea0 0x7fa5000ce320 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa50004ede0 tx=0x7fa4ec00bdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2386ae9e30769a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa4ff7fe700 1 -- 172.21.15.49:0/1906685528 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00dc70 con 0x7fa50006eea0 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa4ff7fe700 1 -- 172.21.15.49:0/1906685528 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec00ddd0 con 0x7fa50006eea0 2024-05-08T23:20:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa4ff7fe700 1 -- 172.21.15.49:0/1906685528 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec010690 con 0x7fa50006eea0 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa505d9d700 1 -- 172.21.15.49:0/1906685528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50006eea0 msgr2=0x7fa5000ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa505d9d700 1 --2- 172.21.15.49:0/1906685528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50006eea0 0x7fa5000ce320 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa50004ede0 tx=0x7fa4ec00bdc0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa505d9d700 1 -- 172.21.15.49:0/1906685528 shutdown_connections 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa505d9d700 1 --2- 172.21.15.49:0/1906685528 >> v2:172.21.15.5:3300/0 conn(0x7fa5000719e0 0x7fa5000ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa505d9d700 1 --2- 172.21.15.49:0/1906685528 >> v2:172.21.15.49:3300/0 conn(0x7fa50015a540 0x7fa50015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa505d9d700 1 --2- 172.21.15.49:0/1906685528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50006eea0 0x7fa5000ce320 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.090+0000 7fa505d9d700 1 -- 172.21.15.49:0/1906685528 >> 172.21.15.49:0/1906685528 conn(0x7fa5000d0210 msgr2=0x7fa5000d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.091+0000 7fa505d9d700 1 -- 172.21.15.49:0/1906685528 shutdown_connections 2024-05-08T23:20:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.091+0000 7fa505d9d700 1 -- 172.21.15.49:0/1906685528 wait complete. 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.092+0000 7fa505d9d700 1 Processor -- start 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.092+0000 7fa505d9d700 1 -- start start 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa505d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50006eea0 0x7fa50014e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa505d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5000719e0 0x7fa50014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa505d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50015a540 0x7fa500157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa505d9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50015c8a0 con 0x7fa50006eea0 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa505d9d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50015bf90 con 0x7fa5000719e0 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa505d9d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa500158070 con 0x7fa50015a540 2024-05-08T23:20:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa4fffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5000719e0 0x7fa50014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa50559c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50015a540 0x7fa500157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa4fffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5000719e0 0x7fa50014d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54774/0 (socket says 172.21.15.49:54774) 2024-05-08T23:20:48.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa4fffff700 1 -- 172.21.15.49:0/532254285 learned_addr learned my addr 172.21.15.49:0/532254285 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:48.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.093+0000 7fa504d9b700 1 --2- 172.21.15.49:0/532254285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50006eea0 0x7fa50014e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa50559c700 1 -- 172.21.15.49:0/532254285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5000719e0 msgr2=0x7fa50014d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa50559c700 1 --2- 172.21.15.49:0/532254285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5000719e0 0x7fa50014d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa50559c700 1 -- 172.21.15.49:0/532254285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50006eea0 msgr2=0x7fa50014e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa50559c700 1 --2- 172.21.15.49:0/532254285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50006eea0 0x7fa50014e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa50559c700 1 -- 172.21.15.49:0/532254285 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f4009550 con 0x7fa50015a540 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa50559c700 1 --2- 172.21.15.49:0/532254285 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50015a540 0x7fa500157b30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800eab0 tx=0x7fa4f800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa4fd7fa700 1 -- 172.21.15.49:0/532254285 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8010040 con 0x7fa50015a540 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa4fd7fa700 1 -- 172.21.15.49:0/532254285 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f800b970 con 0x7fa50015a540 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ec00b740 con 0x7fa50015a540 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa4fd7fa700 1 -- 172.21.15.49:0/532254285 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8009a80 con 0x7fa50015a540 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50015c1e0 con 0x7fa50015a540 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa4fd7fa700 1 -- 172.21.15.49:0/532254285 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa4f801d700 con 0x7fa50015a540 2024-05-08T23:20:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.094+0000 7fa4fd7fa700 1 -- 172.21.15.49:0/532254285 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fa4f8014b60 con 0x7fa50015a540 2024-05-08T23:20:48.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.095+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50015c9e0 con 0x7fa50015a540 2024-05-08T23:20:48.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.097+0000 7fa4fd7fa700 1 -- 172.21.15.49:0/532254285 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa4f803bdc0 con 0x7fa50015a540 2024-05-08T23:20:48.110 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:48.110+0000 7f8905ca3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:20:48.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.184+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f2df4605-e5ec-41b9-8ec6-0bfc843354c7", "id": 5} v 0) v1 -- 0x7fa500074b50 con 0x7fa50015a540 2024-05-08T23:20:48.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.187+0000 7fa4fd7fa700 1 -- 172.21.15.49:0/532254285 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f2df4605-e5ec-41b9-8ec6-0bfc843354c7", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa4f802a030 con 0x7fa50015a540 2024-05-08T23:20:48.188 INFO:teuthology.orchestra.run.smithi049.stdout:5 2024-05-08T23:20:48.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50015a540 msgr2=0x7fa500157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 --2- 172.21.15.49:0/532254285 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50015a540 0x7fa500157b30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800eab0 tx=0x7fa4f800edc0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 shutdown_connections 2024-05-08T23:20:48.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 --2- 172.21.15.49:0/532254285 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa50015a540 0x7fa500157b30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 --2- 172.21.15.49:0/532254285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50006eea0 0x7fa50014e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 --2- 172.21.15.49:0/532254285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5000719e0 0x7fa50014d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 >> 172.21.15.49:0/532254285 conn(0x7fa5000d0210 msgr2=0x7fa50015e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:48.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 shutdown_connections 2024-05-08T23:20:48.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.188+0000 7fa505d9d700 1 -- 172.21.15.49:0/532254285 wait complete. 2024-05-08T23:20:48.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd new ddcb50c8-ad38-42a2-a994-16033633bee8 6 2024-05-08T23:20:48.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.331+0000 7f9601b59700 1 Processor -- start 2024-05-08T23:20:48.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.331+0000 7f9601b59700 1 -- start start 2024-05-08T23:20:48.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f9601b59700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95fc15b930 0x7f95fc09a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f9601b59700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f95fc09a160 0x7f95fc0b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f9601b59700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc0cd440 con 0x7f95fc15b550 2024-05-08T23:20:48.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f9601b59700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc0ccba0 con 0x7f95fc09a160 2024-05-08T23:20:48.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f9601b59700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc0cce80 con 0x7f95fc15b930 2024-05-08T23:20:48.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95faffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95fc15b930 0x7f95fc09a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95faffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95fc15b930 0x7f95fc09a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54784/0 (socket says 172.21.15.49:54784) 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95faffd700 1 -- 172.21.15.49:0/3978045753 learned_addr learned my addr 172.21.15.49:0/3978045753 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95fbfff700 1 --2- 172.21.15.49:0/3978045753 >> v2:172.21.15.5:3300/0 conn(0x7f95fc09a160 0x7f95fc0b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95faffd700 1 -- 172.21.15.49:0/3978045753 >> v2:172.21.15.5:3300/0 conn(0x7f95fc09a160 msgr2=0x7f95fc0b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95faffd700 1 --2- 172.21.15.49:0/3978045753 >> v2:172.21.15.5:3300/0 conn(0x7f95fc09a160 0x7f95fc0b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95faffd700 1 -- 172.21.15.49:0/3978045753 >> v1:172.21.15.49:6789/0 conn(0x7f95fc15b550 legacy=0x7f95fc099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95faffd700 1 -- 172.21.15.49:0/3978045753 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0cd160 con 0x7f95fc15b930 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95faffd700 1 --2- 172.21.15.49:0/3978045753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc15b930 0x7f95fc09a8b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95fc067630 tx=0x7f95f0009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d57478e44da1172 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.332+0000 7f95fa7fc700 1 -- 172.21.15.49:0/3978045753 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0011830 con 0x7f95fc15b930 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f95fa7fc700 1 -- 172.21.15.49:0/3978045753 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f0011990 con 0x7f95fc15b930 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f95fa7fc700 1 -- 172.21.15.49:0/3978045753 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0011d20 con 0x7f95fc15b930 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f9601b59700 1 -- 172.21.15.49:0/3978045753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc15b930 msgr2=0x7f95fc09a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f9601b59700 1 --2- 172.21.15.49:0/3978045753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc15b930 0x7f95fc09a8b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95fc067630 tx=0x7f95f0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f9601b59700 1 -- 172.21.15.49:0/3978045753 shutdown_connections 2024-05-08T23:20:48.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f9601b59700 1 --2- 172.21.15.49:0/3978045753 >> v2:172.21.15.5:3300/0 conn(0x7f95fc09a160 0x7f95fc0b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f9601b59700 1 --2- 172.21.15.49:0/3978045753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc15b930 0x7f95fc09a8b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f9601b59700 1 -- 172.21.15.49:0/3978045753 >> 172.21.15.49:0/3978045753 conn(0x7f95fc0b2ec0 msgr2=0x7f95fc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:48.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f9601b59700 1 -- 172.21.15.49:0/3978045753 shutdown_connections 2024-05-08T23:20:48.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.333+0000 7f9601b59700 1 -- 172.21.15.49:0/3978045753 wait complete. 2024-05-08T23:20:48.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.335+0000 7f9601b59700 1 Processor -- start 2024-05-08T23:20:48.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.335+0000 7f9601b59700 1 -- start start 2024-05-08T23:20:48.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.335+0000 7f9601b59700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc09a160 0x7f95fc157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.335+0000 7f9601b59700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95fc15b550 0x7f95fc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.335+0000 7f9601b59700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f95fc15b930 0x7f95fc0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f9601b59700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0b5a50 con 0x7f95fc15b550 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f9601b59700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc158140 con 0x7f95fc09a160 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f9601b59700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0b5230 con 0x7f95fc15b930 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc09a160 0x7f95fc157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fbfff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f95fc15b930 0x7f95fc0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc09a160 0x7f95fc157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54800/0 (socket says 172.21.15.49:54800) 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 -- 172.21.15.49:0/849655808 learned_addr learned my addr 172.21.15.49:0/849655808 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95faffd700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95fc15b550 0x7f95fc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 -- 172.21.15.49:0/849655808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f95fc15b930 msgr2=0x7f95fc0ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f95fc15b930 0x7f95fc0ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 -- 172.21.15.49:0/849655808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95fc15b550 msgr2=0x7f95fc0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95fc15b550 0x7f95fc0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 -- 172.21.15.49:0/849655808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0009550 con 0x7f95fc09a160 2024-05-08T23:20:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fbfff700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f95fc15b930 0x7f95fc0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95fb7fe700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc09a160 0x7f95fc157c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c4e0 tx=0x7f95ec00c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95e3fff700 1 -- 172.21.15.49:0/849655808 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec0052d0 con 0x7f95fc09a160 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95e3fff700 1 -- 172.21.15.49:0/849655808 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec005430 con 0x7f95fc09a160 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc0b5490 con 0x7f95fc09a160 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f95e3fff700 1 -- 172.21.15.49:0/849655808 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec0038c0 con 0x7f95fc09a160 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.336+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc06bba0 con 0x7f95fc09a160 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.337+0000 7f95e3fff700 1 -- 172.21.15.49:0/849655808 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f95ec003d20 con 0x7f95fc09a160 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.337+0000 7f95e3fff700 1 -- 172.21.15.49:0/849655808 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f95ec011d60 con 0x7f95fc09a160 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.337+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc074bc0 con 0x7f95fc09a160 2024-05-08T23:20:48.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.340+0000 7f95e3fff700 1 -- 172.21.15.49:0/849655808 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f95ec017070 con 0x7f95fc09a160 2024-05-08T23:20:48.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.428+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ddcb50c8-ad38-42a2-a994-16033633bee8", "id": 6} v 0) v1 -- 0x7f95fc0cc770 con 0x7f95fc09a160 2024-05-08T23:20:48.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.431+0000 7f95e3fff700 1 -- 172.21.15.49:0/849655808 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddcb50c8-ad38-42a2-a994-16033633bee8", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f95ec011590 con 0x7f95fc09a160 2024-05-08T23:20:48.432 INFO:teuthology.orchestra.run.smithi049.stdout:6 2024-05-08T23:20:48.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc09a160 msgr2=0x7f95fc157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc09a160 0x7f95fc157c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c4e0 tx=0x7f95ec00c7f0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 shutdown_connections 2024-05-08T23:20:48.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f95fc15b930 0x7f95fc0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc09a160 0x7f95fc157c00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 --2- 172.21.15.49:0/849655808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95fc15b550 0x7f95fc0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 >> 172.21.15.49:0/849655808 conn(0x7f95fc0b2ec0 msgr2=0x7f95fc0cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:48.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 shutdown_connections 2024-05-08T23:20:48.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.432+0000 7f9601b59700 1 -- 172.21.15.49:0/849655808 wait complete. 2024-05-08T23:20:48.449 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd new e8e302a4-2b11-42b9-bc89-265425c636f6 7 2024-05-08T23:20:48.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.573+0000 7fae7cc21700 1 Processor -- start 2024-05-08T23:20:48.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.573+0000 7fae7cc21700 1 -- start start 2024-05-08T23:20:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.574+0000 7fae7cc21700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fae780d13a0 0x7fae780d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.574+0000 7fae7cc21700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae780cdc90 0x7fae780b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.574+0000 7fae7cc21700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae780b1200 con 0x7fae7806bd60 2024-05-08T23:20:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.574+0000 7fae7cc21700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780d1cc0 con 0x7fae780cdc90 2024-05-08T23:20:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.574+0000 7fae7cc21700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780ce5c0 con 0x7fae780d13a0 2024-05-08T23:20:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fae780d13a0 0x7fae780d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fae780d13a0 0x7fae780d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54808/0 (socket says 172.21.15.49:54808) 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7659c700 1 -- 172.21.15.49:0/3479295266 learned_addr learned my addr 172.21.15.49:0/3479295266 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae76d9d700 1 --2- 172.21.15.49:0/3479295266 >> v2:172.21.15.5:3300/0 conn(0x7fae780cdc90 0x7fae780b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7659c700 1 -- 172.21.15.49:0/3479295266 >> v2:172.21.15.5:3300/0 conn(0x7fae780cdc90 msgr2=0x7fae780b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7659c700 1 --2- 172.21.15.49:0/3479295266 >> v2:172.21.15.5:3300/0 conn(0x7fae780cdc90 0x7fae780b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7659c700 1 -- 172.21.15.49:0/3479295266 >> v1:172.21.15.49:6789/0 conn(0x7fae7806bd60 legacy=0x7fae780cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7659c700 1 -- 172.21.15.49:0/3479295266 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780ce8a0 con 0x7fae780d13a0 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae76d9d700 1 --2- 172.21.15.49:0/3479295266 >> v2:172.21.15.5:3300/0 conn(0x7fae780cdc90 0x7fae780b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7659c700 1 --2- 172.21.15.49:0/3479295266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae780d13a0 0x7fae780d1780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae7806e520 tx=0x7fae68009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc8599f21c7ed67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7559a700 1 -- 172.21.15.49:0/3479295266 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800b870 con 0x7fae780d13a0 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7559a700 1 -- 172.21.15.49:0/3479295266 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6800b9d0 con 0x7fae780d13a0 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.575+0000 7fae7559a700 1 -- 172.21.15.49:0/3479295266 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800bd60 con 0x7fae780d13a0 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.576+0000 7fae7cc21700 1 -- 172.21.15.49:0/3479295266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae780d13a0 msgr2=0x7fae780d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.576+0000 7fae7cc21700 1 --2- 172.21.15.49:0/3479295266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae780d13a0 0x7fae780d1780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae7806e520 tx=0x7fae68009bd0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.576+0000 7fae7cc21700 1 -- 172.21.15.49:0/3479295266 shutdown_connections 2024-05-08T23:20:48.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.576+0000 7fae7cc21700 1 --2- 172.21.15.49:0/3479295266 >> v2:172.21.15.5:3300/0 conn(0x7fae780cdc90 0x7fae780b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.576+0000 7fae7cc21700 1 --2- 172.21.15.49:0/3479295266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae780d13a0 0x7fae780d1780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.576+0000 7fae7cc21700 1 -- 172.21.15.49:0/3479295266 >> 172.21.15.49:0/3479295266 conn(0x7fae7808afe0 msgr2=0x7fae7808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:48.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.576+0000 7fae7cc21700 1 -- 172.21.15.49:0/3479295266 shutdown_connections 2024-05-08T23:20:48.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.576+0000 7fae7cc21700 1 -- 172.21.15.49:0/3479295266 wait complete. 2024-05-08T23:20:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.577+0000 7fae7cc21700 1 Processor -- start 2024-05-08T23:20:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.577+0000 7fae7cc21700 1 -- start start 2024-05-08T23:20:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7cc21700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae7806bd60 0x7fae7815d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7cc21700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae780cdc90 0x7fae780cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7cc21700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae780d13a0 0x7fae7815c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7cc21700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae780cb4f0 con 0x7fae780d13a0 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7cc21700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815d910 con 0x7fae7806bd60 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7cc21700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815dbf0 con 0x7fae780cdc90 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae7806bd60 0x7fae7815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae7806bd60 0x7fae7815d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54810/0 (socket says 172.21.15.49:54810) 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae7659c700 1 -- 172.21.15.49:0/2584669927 learned_addr learned my addr 172.21.15.49:0/2584669927 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae75d9b700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae780cdc90 0x7fae780cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.578+0000 7fae76d9d700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae780d13a0 0x7fae7815c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae7659c700 1 -- 172.21.15.49:0/2584669927 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae780cdc90 msgr2=0x7fae780cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae7659c700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae780cdc90 0x7fae780cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae7659c700 1 -- 172.21.15.49:0/2584669927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae780d13a0 msgr2=0x7fae7815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae7659c700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae780d13a0 0x7fae7815c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae7659c700 1 -- 172.21.15.49:0/2584669927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae68009550 con 0x7fae7806bd60 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae7659c700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae7806bd60 0x7fae7815d210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fae78067100 tx=0x7fae68004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae66ffd700 1 -- 172.21.15.49:0/2584669927 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68004480 con 0x7fae7806bd60 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae66ffd700 1 -- 172.21.15.49:0/2584669927 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae680045e0 con 0x7fae7806bd60 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7815ded0 con 0x7fae7806bd60 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae66ffd700 1 -- 172.21.15.49:0/2584669927 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680038e0 con 0x7fae7806bd60 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7815e490 con 0x7fae7806bd60 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae66ffd700 1 -- 172.21.15.49:0/2584669927 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fae68021070 con 0x7fae7806bd60 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.579+0000 7fae66ffd700 1 -- 172.21.15.49:0/2584669927 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fae68024ba0 con 0x7fae7806bd60 2024-05-08T23:20:48.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.580+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae78074bc0 con 0x7fae7806bd60 2024-05-08T23:20:48.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.584+0000 7fae66ffd700 1 -- 172.21.15.49:0/2584669927 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fae68029020 con 0x7fae7806bd60 2024-05-08T23:20:48.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.671+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e8e302a4-2b11-42b9-bc89-265425c636f6", "id": 7} v 0) v1 -- 0x7fae7806c260 con 0x7fae7806bd60 2024-05-08T23:20:48.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.674+0000 7fae66ffd700 1 -- 172.21.15.49:0/2584669927 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8e302a4-2b11-42b9-bc89-265425c636f6", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fae68015020 con 0x7fae7806bd60 2024-05-08T23:20:48.675 INFO:teuthology.orchestra.run.smithi049.stdout:7 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.675+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae7806bd60 msgr2=0x7fae7815d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.675+0000 7fae7cc21700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae7806bd60 0x7fae7815d210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fae78067100 tx=0x7fae68004750 comp rx=0 tx=0).stop 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.675+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 shutdown_connections 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.675+0000 7fae7cc21700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae780d13a0 0x7fae7815c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.675+0000 7fae7cc21700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae7806bd60 0x7fae7815d210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.675+0000 7fae7cc21700 1 --2- 172.21.15.49:0/2584669927 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae780cdc90 0x7fae780cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.675+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 >> 172.21.15.49:0/2584669927 conn(0x7fae7808afe0 msgr2=0x7fae7808b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.676+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 shutdown_connections 2024-05-08T23:20:48.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-08T23:20:48.676+0000 7fae7cc21700 1 -- 172.21.15.49:0/2584669927 wait complete. 2024-05-08T23:20:48.688 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T23:20:48.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T23:20:48.690 INFO:tasks.ceph.osd.0:Started 2024-05-08T23:20:48.691 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T23:20:48.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T23:20:48.693 INFO:tasks.ceph.osd.1:Started 2024-05-08T23:20:48.693 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T23:20:48.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T23:20:48.695 INFO:tasks.ceph.osd.2:Started 2024-05-08T23:20:48.695 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T23:20:48.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T23:20:48.697 INFO:tasks.ceph.osd.3:Started 2024-05-08T23:20:48.698 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T23:20:48.698 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T23:20:48.700 INFO:tasks.ceph.osd.4:Started 2024-05-08T23:20:48.700 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T23:20:48.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T23:20:48.703 INFO:tasks.ceph.osd.5:Started 2024-05-08T23:20:48.703 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T23:20:48.703 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T23:20:48.705 INFO:tasks.ceph.osd.6:Started 2024-05-08T23:20:48.705 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T23:20:48.705 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T23:20:48.710 INFO:tasks.ceph.osd.7:Started 2024-05-08T23:20:48.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:20:48.757 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:48.757+0000 7f208dbd8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:20:48.854 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:48.855+0000 7f208dbd8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:20:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.871+0000 7f9cecaa3700 1 Processor -- start 2024-05-08T23:20:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.871+0000 7f9cecaa3700 1 -- start start 2024-05-08T23:20:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.872+0000 7f9cecaa3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.872+0000 7f9cecaa3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ce80cd5f0 0x7f9ce8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.872+0000 7f9cecaa3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce809ac00 con 0x7f9ce815b810 2024-05-08T23:20:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.872+0000 7f9cecaa3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce80cda60 con 0x7f9ce80cd5f0 2024-05-08T23:20:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.872+0000 7f9cecaa3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80cf6d0 con 0x7f9ce80cf270 2024-05-08T23:20:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce659c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9ce815b810 0x7f9ce806ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49094/0 (socket says 172.21.15.5:49094) 2024-05-08T23:20:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce659c700 1 -- 172.21.15.5:0/1121995293 learned_addr learned my addr 172.21.15.5:0/1121995293 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:20:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce5d9b700 1 --2- 172.21.15.5:0/1121995293 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce6d9d700 1 --2- 172.21.15.5:0/1121995293 >> v2:172.21.15.49:3300/0 conn(0x7f9ce80cd5f0 0x7f9ce8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce559a700 1 -- 172.21.15.5:0/1121995293 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427504587 0 0) 0x7f9ce809ac00 con 0x7f9ce815b810 2024-05-08T23:20:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce559a700 1 -- 172.21.15.5:0/1121995293 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd00035b0 con 0x7f9ce815b810 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce5d9b700 1 -- 172.21.15.5:0/1121995293 >> v2:172.21.15.49:3300/0 conn(0x7f9ce80cd5f0 msgr2=0x7f9ce8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce5d9b700 1 --2- 172.21.15.5:0/1121995293 >> v2:172.21.15.49:3300/0 conn(0x7f9ce80cd5f0 0x7f9ce8099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce5d9b700 1 -- 172.21.15.5:0/1121995293 >> v1:172.21.15.5:6789/0 conn(0x7f9ce815b810 legacy=0x7f9ce806ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce5d9b700 1 -- 172.21.15.5:0/1121995293 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80cc5d0 con 0x7f9ce80cf270 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce6d9d700 1 --2- 172.21.15.5:0/1121995293 >> v2:172.21.15.49:3300/0 conn(0x7f9ce80cd5f0 0x7f9ce8099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce5d9b700 1 --2- 172.21.15.5:0/1121995293 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80cd0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009840 tx=0x7f9cdc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d92f1a81e28018 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.873+0000 7f9ce559a700 1 -- 172.21.15.5:0/1121995293 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc011830 con 0x7f9ce80cf270 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9ce559a700 1 -- 172.21.15.5:0/1121995293 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc011990 con 0x7f9ce80cf270 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9ce559a700 1 -- 172.21.15.5:0/1121995293 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc011d20 con 0x7f9ce80cf270 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9cecaa3700 1 -- 172.21.15.5:0/1121995293 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 msgr2=0x7f9ce80cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9cecaa3700 1 --2- 172.21.15.5:0/1121995293 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80cd0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009840 tx=0x7f9cdc009b50 comp rx=0 tx=0).stop 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9cecaa3700 1 -- 172.21.15.5:0/1121995293 shutdown_connections 2024-05-08T23:20:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9cecaa3700 1 --2- 172.21.15.5:0/1121995293 >> v2:172.21.15.49:3300/0 conn(0x7f9ce80cd5f0 0x7f9ce8099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9cecaa3700 1 --2- 172.21.15.5:0/1121995293 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80cd0b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9cecaa3700 1 -- 172.21.15.5:0/1121995293 >> 172.21.15.5:0/1121995293 conn(0x7f9ce80b2f80 msgr2=0x7f9ce80b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9cecaa3700 1 -- 172.21.15.5:0/1121995293 shutdown_connections 2024-05-08T23:20:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.874+0000 7f9cecaa3700 1 -- 172.21.15.5:0/1121995293 wait complete. 2024-05-08T23:20:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.876+0000 7f9cecaa3700 1 Processor -- start 2024-05-08T23:20:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.876+0000 7f9cecaa3700 1 -- start start 2024-05-08T23:20:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9cecaa3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce80cd5f0 0x7f9ce815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9cecaa3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9cecaa3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce815b810 0x7f9ce80ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9cecaa3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80c48a0 con 0x7f9ce80cd5f0 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9cecaa3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce815af90 con 0x7f9ce815b810 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9cecaa3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80c42e0 con 0x7f9ce80cf270 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce80cd5f0 0x7f9ce815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce815b810 0x7f9ce80ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce5d9b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce815b810 0x7f9ce80ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39852/0 (socket says 172.21.15.5:39852) 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce80cd5f0 0x7f9ce815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33524/0 (socket says 172.21.15.5:33524) 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 -- 172.21.15.5:0/2770471241 learned_addr learned my addr 172.21.15.5:0/2770471241 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce5d9b700 1 --2- 172.21.15.5:0/2770471241 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:52504/0 (socket says 172.21.15.5:52504) 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 -- 172.21.15.5:0/2770471241 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 msgr2=0x7f9ce80c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 --2- 172.21.15.5:0/2770471241 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 -- 172.21.15.5:0/2770471241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce80cd5f0 msgr2=0x7f9ce815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 --2- 172.21.15.5:0/2770471241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce80cd5f0 0x7f9ce815aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 -- 172.21.15.5:0/2770471241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce00094a0 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.877+0000 7f9ce6d9d700 1 --2- 172.21.15.5:0/2770471241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce815b810 0x7f9ce80ae0e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000eb60 tx=0x7f9ce000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.878+0000 7f9cd6ffd700 1 -- 172.21.15.5:0/2770471241 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00097a0 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.878+0000 7f9cd6ffd700 1 -- 172.21.15.5:0/2770471241 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0009e60 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.878+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc0094a0 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.878+0000 7f9cd6ffd700 1 -- 172.21.15.5:0/2770471241 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0004240 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.878+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce806bc10 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.878+0000 7f9cd6ffd700 1 -- 172.21.15.5:0/2770471241 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9ce0005a20 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.878+0000 7f9cd6ffd700 1 -- 172.21.15.5:0/2770471241 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9ce0003e10 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.879+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce80d0f30 con 0x7f9ce815b810 2024-05-08T23:20:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.881+0000 7f9cd6ffd700 1 -- 172.21.15.5:0/2770471241 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f9ce0020060 con 0x7f9ce815b810 2024-05-08T23:20:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.957+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ce806ba70 con 0x7f9ce815b810 2024-05-08T23:20:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.957+0000 7f9cd6ffd700 1 -- 172.21.15.5:0/2770471241 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f9ce806ba70 con 0x7f9ce815b810 2024-05-08T23:20:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:20:48.674248+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.958+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce815b810 msgr2=0x7f9ce80ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.958+0000 7f9cecaa3700 1 --2- 172.21.15.5:0/2770471241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce815b810 0x7f9ce80ae0e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000eb60 tx=0x7f9ce000ee70 comp rx=0 tx=0).stop 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.958+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 shutdown_connections 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.958+0000 7f9cecaa3700 1 --2- 172.21.15.5:0/2770471241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce815b810 0x7f9ce80ae0e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.958+0000 7f9cecaa3700 1 --2- 172.21.15.5:0/2770471241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce80cd5f0 0x7f9ce815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.958+0000 7f9cecaa3700 1 --2- 172.21.15.5:0/2770471241 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ce80cf270 0x7f9ce80c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.958+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 >> 172.21.15.5:0/2770471241 conn(0x7f9ce80b2f80 msgr2=0x7f9ce815fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.959+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 shutdown_connections 2024-05-08T23:20:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:48.959+0000 7f9cecaa3700 1 -- 172.21.15.5:0/2770471241 wait complete. 2024-05-08T23:20:48.972 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T23:20:48.972 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T23:20:48.977 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:48.977+0000 7f8905ca3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:20:48.993 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:48.993+0000 7f208dbd8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:20:49.071 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:49.071+0000 7f8905ca3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:20:49.222 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:49.222+0000 7f8905ca3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:20:49.312 INFO:tasks.ceph.osd.5.smithi049.stderr:2024-05-08T23:20:49.311+0000 7f9f511c07c0 -1 Falling back to public interface 2024-05-08T23:20:49.317 INFO:tasks.ceph.osd.6.smithi049.stderr:2024-05-08T23:20:49.316+0000 7fcafb9e57c0 -1 Falling back to public interface 2024-05-08T23:20:49.322 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-08T23:20:49.322+0000 7f4580fda7c0 -1 Falling back to public interface 2024-05-08T23:20:49.329 INFO:tasks.ceph.osd.4.smithi049.stderr:2024-05-08T23:20:49.327+0000 7f21b19447c0 -1 Falling back to public interface 2024-05-08T23:20:49.330 INFO:tasks.ceph.osd.7.smithi049.stderr:2024-05-08T23:20:49.329+0000 7fddcc8c97c0 -1 Falling back to public interface 2024-05-08T23:20:49.341 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-08T23:20:49.341+0000 7efd477247c0 -1 Falling back to public interface 2024-05-08T23:20:49.343 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-08T23:20:49.343+0000 7f0b75dcd7c0 -1 Falling back to public interface 2024-05-08T23:20:49.352 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-08T23:20:49.352+0000 7fd18e3e47c0 -1 Falling back to public interface 2024-05-08T23:20:49.578 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:49.578+0000 7f208dbd8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:20:49.745 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:49.746+0000 7f208dbd8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:20:49.778 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T23:20:49.864 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:49.864+0000 7f8905ca3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:20:49.904 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.904+0000 7feff33f3700 1 Processor -- start 2024-05-08T23:20:49.904 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.904+0000 7feff33f3700 1 -- start start 2024-05-08T23:20:49.907 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff33f3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fefec15b790 0x7fefec0cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:49.907 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff33f3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec0cd0c0 0x7fefec06a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:49.907 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff33f3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec0b32c0 con 0x7fefec06af00 2024-05-08T23:20:49.907 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff33f3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefec06b3b0 con 0x7fefec15b790 2024-05-08T23:20:49.907 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff33f3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0b35a0 con 0x7fefec0cd0c0 2024-05-08T23:20:49.907 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff1990700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fefec06af00 0x7fefec15f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49108/0 (socket says 172.21.15.5:49108) 2024-05-08T23:20:49.907 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff1990700 1 -- 172.21.15.5:0/1978285605 learned_addr learned my addr 172.21.15.5:0/1978285605 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:20:49.907 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff118f700 1 --2- 172.21.15.5:0/1978285605 >> v2:172.21.15.49:3300/0 conn(0x7fefec15b790 0x7fefec0cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.905+0000 7feff098e700 1 --2- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec0cd0c0 0x7fefec06a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7fefe3fff700 1 -- 172.21.15.5:0/1978285605 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722955228 0 0) 0x7fefec0b32c0 con 0x7fefec06af00 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7feff118f700 1 -- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec0cd0c0 msgr2=0x7fefec06a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7feff118f700 1 --2- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec0cd0c0 0x7fefec06a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7feff118f700 1 -- 172.21.15.5:0/1978285605 >> v1:172.21.15.5:6789/0 conn(0x7fefec06af00 legacy=0x7fefec15f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7feff118f700 1 -- 172.21.15.5:0/1978285605 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec15bb70 con 0x7fefec15b790 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7feff098e700 1 --2- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec0cd0c0 0x7fefec06a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7feff118f700 1 --2- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 0x7fefec0cc0a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009840 tx=0x7fefe8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307a4c02183fb39a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7fefe3fff700 1 -- 172.21.15.5:0/1978285605 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8011830 con 0x7fefec15b790 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7fefe3fff700 1 -- 172.21.15.5:0/1978285605 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8011990 con 0x7fefec15b790 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.906+0000 7fefe3fff700 1 -- 172.21.15.5:0/1978285605 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8011d20 con 0x7fefec15b790 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.907+0000 7feff33f3700 1 -- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 msgr2=0x7fefec0cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:49.908 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.907+0000 7feff33f3700 1 --2- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 0x7fefec0cc0a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009840 tx=0x7fefe8009b50 comp rx=0 tx=0).stop 2024-05-08T23:20:49.909 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.907+0000 7feff33f3700 1 -- 172.21.15.5:0/1978285605 shutdown_connections 2024-05-08T23:20:49.909 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.907+0000 7feff33f3700 1 --2- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 0x7fefec0cc0a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:49.909 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.907+0000 7feff33f3700 1 --2- 172.21.15.5:0/1978285605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec0cd0c0 0x7fefec06a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:49.910 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.907+0000 7feff33f3700 1 -- 172.21.15.5:0/1978285605 >> 172.21.15.5:0/1978285605 conn(0x7fefec0b1f80 msgr2=0x7fefec0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:49.910 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.907+0000 7feff33f3700 1 -- 172.21.15.5:0/1978285605 shutdown_connections 2024-05-08T23:20:49.910 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.907+0000 7feff33f3700 1 -- 172.21.15.5:0/1978285605 wait complete. 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.908+0000 7feff33f3700 1 Processor -- start 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.908+0000 7feff33f3700 1 -- start start 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff33f3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec06af00 0x7fefec159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff33f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefec0cd0c0 0x7fefec09bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff33f3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 0x7fefec09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff33f3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec15a0c0 con 0x7fefec0cd0c0 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff33f3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0cf360 con 0x7fefec15b790 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff33f3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0cf5b0 con 0x7fefec06af00 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff098e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefec0cd0c0 0x7fefec09bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:49.913 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff1990700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 0x7fefec09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff098e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefec0cd0c0 0x7fefec09bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33526/0 (socket says 172.21.15.5:33526) 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff098e700 1 -- 172.21.15.5:0/867085212 learned_addr learned my addr 172.21.15.5:0/867085212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.909+0000 7feff118f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec06af00 0x7fefec159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff1990700 1 -- 172.21.15.5:0/867085212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec06af00 msgr2=0x7fefec159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff1990700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec06af00 0x7fefec159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff1990700 1 -- 172.21.15.5:0/867085212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefec0cd0c0 msgr2=0x7fefec09bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff1990700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefec0cd0c0 0x7fefec09bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff1990700 1 -- 172.21.15.5:0/867085212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0094a0 con 0x7fefec15b790 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff118f700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec06af00 0x7fefec159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:49.914 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff098e700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefec0cd0c0 0x7fefec09bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff1990700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 0x7fefec09c480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fefe400caf0 tx=0x7fefe400ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7fefe1ffb700 1 -- 172.21.15.5:0/867085212 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe400a9e0 con 0x7fefec15b790 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7fefe1ffb700 1 -- 172.21.15.5:0/867085212 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe400ab60 con 0x7fefec15b790 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe80094a0 con 0x7fefec15b790 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7fefe1ffb700 1 -- 172.21.15.5:0/867085212 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40070d0 con 0x7fefec15b790 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7fefe1ffb700 1 -- 172.21.15.5:0/867085212 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fefe4007530 con 0x7fefec15b790 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.910+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec0cfb70 con 0x7fefec15b790 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.911+0000 7fefe1ffb700 1 -- 172.21.15.5:0/867085212 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fefe401c3b0 con 0x7fefec15b790 2024-05-08T23:20:49.915 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.911+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec04ec60 con 0x7fefec15b790 2024-05-08T23:20:49.916 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.914+0000 7fefe1ffb700 1 -- 172.21.15.5:0/867085212 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fefe4018070 con 0x7fefec15b790 2024-05-08T23:20:49.918 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:49.918+0000 7f208dbd8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:20:49.990 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.990+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefec0b63f0 con 0x7fefec15b790 2024-05-08T23:20:49.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.991+0000 7fefe1ffb700 1 -- 172.21.15.5:0/867085212 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fefe40077e0 con 0x7fefec15b790 2024-05-08T23:20:49.991 INFO:teuthology.misc.health.smithi005.stdout: 2024-05-08T23:20:49.991 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":10,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:20:48.674248+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 msgr2=0x7fefec09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 0x7fefec09c480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fefe400caf0 tx=0x7fefe400ce00 comp rx=0 tx=0).stop 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 shutdown_connections 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefec15b790 0x7fefec09c480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fefec06af00 0x7fefec159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 --2- 172.21.15.5:0/867085212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefec0cd0c0 0x7fefec09bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 >> 172.21.15.5:0/867085212 conn(0x7fefec0b1f80 msgr2=0x7fefec0b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 shutdown_connections 2024-05-08T23:20:49.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:49.992+0000 7feff33f3700 1 -- 172.21.15.5:0/867085212 wait complete. 2024-05-08T23:20:50.004 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T23:20:50.052 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:50.052+0000 7f8905ca3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:20:50.239 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:50.239+0000 7f8905ca3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:20:50.775 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:50.775+0000 7f208dbd8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:20:51.023 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:51.023+0000 7f208dbd8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:20:51.100 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:51.100+0000 7f208dbd8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:20:51.183 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:51.183+0000 7f8905ca3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:20:51.269 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:51.270+0000 7f208dbd8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:20:51.454 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:51.454+0000 7f8905ca3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:20:51.538 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:51.538+0000 7f8905ca3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:20:51.720 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-08T23:20:51.720+0000 7f8905ca3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:20:52.303 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:52.302+0000 7f2048e74700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T23:20:52.303 INFO:tasks.ceph.mgr.x.smithi049.stderr:2024-05-08T23:20:52.303+0000 7f2048e74700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T23:20:52.798 INFO:tasks.ceph.osd.5.smithi049.stderr:2024-05-08T23:20:52.799+0000 7f9f511c07c0 -1 osd.5 0 log_to_monitors true 2024-05-08T23:20:52.819 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-08T23:20:52.819+0000 7f4580fda7c0 -1 osd.2 0 log_to_monitors true 2024-05-08T23:20:52.840 INFO:tasks.ceph.osd.7.smithi049.stderr:2024-05-08T23:20:52.840+0000 7fddcc8c97c0 -1 osd.7 0 log_to_monitors true 2024-05-08T23:20:52.847 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-08T23:20:52.847+0000 7efd477247c0 -1 osd.3 0 log_to_monitors true 2024-05-08T23:20:52.854 INFO:tasks.ceph.osd.4.smithi049.stderr:2024-05-08T23:20:52.854+0000 7f21b19447c0 -1 osd.4 0 log_to_monitors true 2024-05-08T23:20:52.857 INFO:tasks.ceph.osd.6.smithi049.stderr:2024-05-08T23:20:52.858+0000 7fcafb9e57c0 -1 osd.6 0 log_to_monitors true 2024-05-08T23:20:52.858 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-08T23:20:52.858+0000 7f0b75dcd7c0 -1 osd.1 0 log_to_monitors true 2024-05-08T23:20:52.880 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-08T23:20:52.880+0000 7fd18e3e47c0 -1 osd.0 0 log_to_monitors true 2024-05-08T23:20:55.579 INFO:tasks.ceph.osd.6.smithi049.stderr:2024-05-08T23:20:55.576+0000 7fcaef59a700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T23:20:55.586 INFO:tasks.ceph.osd.6.smithi049.stderr:2024-05-08T23:20:55.587+0000 7fcae7975700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:20:55.589 INFO:tasks.ceph.osd.5.smithi049.stderr:2024-05-08T23:20:55.590+0000 7f9f44d75700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T23:20:55.605 INFO:tasks.ceph.osd.5.smithi049.stderr:2024-05-08T23:20:55.605+0000 7f9f3d150700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:20:55.659 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-08T23:20:55.659+0000 7efd3b2d9700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T23:20:55.661 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-08T23:20:55.662+0000 7f4574b8f700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T23:20:55.669 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-08T23:20:55.669+0000 7efd336b4700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:20:55.672 INFO:tasks.ceph.osd.4.smithi049.stderr:2024-05-08T23:20:55.672+0000 7f21a54f9700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T23:20:55.673 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-08T23:20:55.674+0000 7f456cf6a700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:20:55.682 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-08T23:20:55.682+0000 7fd181f99700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T23:20:55.686 INFO:tasks.ceph.osd.4.smithi049.stderr:2024-05-08T23:20:55.687+0000 7f219d8d4700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:20:55.699 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-08T23:20:55.699+0000 7fd17a374700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:20:55.733 INFO:tasks.ceph.osd.7.smithi049.stderr:2024-05-08T23:20:55.733+0000 7fddc047e700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T23:20:55.742 INFO:tasks.ceph.osd.7.smithi049.stderr:2024-05-08T23:20:55.742+0000 7fddb8859700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:20:55.745 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-08T23:20:55.746+0000 7f0b69982700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T23:20:55.763 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-08T23:20:55.763+0000 7f0b61d5d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:20:56.811 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T23:20:56.945 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.945+0000 7f3bd4198700 1 Processor -- start 2024-05-08T23:20:56.945 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.945+0000 7f3bd4198700 1 -- start start 2024-05-08T23:20:56.949 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.946+0000 7f3bd4198700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3bcc15dae0 0x7f3bcc0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:56.949 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.946+0000 7f3bd4198700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bcc0cc570 0x7f3bcc0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:56.949 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.946+0000 7f3bd4198700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc0cda20 con 0x7f3bcc06b3d0 2024-05-08T23:20:56.949 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.946+0000 7f3bd4198700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0d0140 con 0x7f3bcc15dae0 2024-05-08T23:20:56.949 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.946+0000 7f3bd4198700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc15dec0 con 0x7f3bcc0cc570 2024-05-08T23:20:56.949 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.946+0000 7f3bd1f34700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3bcc15dae0 0x7f3bcc0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.946+0000 7f3bd1f34700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3bcc15dae0 0x7f3bcc0cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33548/0 (socket says 172.21.15.5:33548) 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.946+0000 7f3bd1f34700 1 -- 172.21.15.5:0/1746922787 learned_addr learned my addr 172.21.15.5:0/1746922787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd2735700 1 --2- 172.21.15.5:0/1746922787 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc0cc570 0x7f3bcc0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd1f34700 1 -- 172.21.15.5:0/1746922787 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc0cc570 msgr2=0x7f3bcc0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd1f34700 1 --2- 172.21.15.5:0/1746922787 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc0cc570 0x7f3bcc0cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd1f34700 1 -- 172.21.15.5:0/1746922787 >> v1:172.21.15.49:6789/0 conn(0x7f3bcc06b3d0 legacy=0x7f3bcc1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd1f34700 1 -- 172.21.15.5:0/1746922787 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0cce10 con 0x7f3bcc15dae0 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd2735700 1 --2- 172.21.15.5:0/1746922787 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc0cc570 0x7f3bcc0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:56.950 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd1f34700 1 --2- 172.21.15.5:0/1746922787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15dae0 0x7f3bcc0cf6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc072d00 tx=0x7f3bc8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b329feb44961b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd0f32700 1 -- 172.21.15.5:0/1746922787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8004230 con 0x7f3bcc15dae0 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd0f32700 1 -- 172.21.15.5:0/1746922787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8004390 con 0x7f3bcc15dae0 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.947+0000 7f3bd0f32700 1 -- 172.21.15.5:0/1746922787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80036a0 con 0x7f3bcc15dae0 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.948+0000 7f3bd4198700 1 -- 172.21.15.5:0/1746922787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15dae0 msgr2=0x7f3bcc0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.948+0000 7f3bd4198700 1 --2- 172.21.15.5:0/1746922787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15dae0 0x7f3bcc0cf6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc072d00 tx=0x7f3bc8009bd0 comp rx=0 tx=0).stop 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.948+0000 7f3bd4198700 1 -- 172.21.15.5:0/1746922787 shutdown_connections 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.948+0000 7f3bd4198700 1 --2- 172.21.15.5:0/1746922787 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc0cc570 0x7f3bcc0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.948+0000 7f3bd4198700 1 --2- 172.21.15.5:0/1746922787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15dae0 0x7f3bcc0cf6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.948+0000 7f3bd4198700 1 -- 172.21.15.5:0/1746922787 >> 172.21.15.5:0/1746922787 conn(0x7f3bcc0b1f80 msgr2=0x7f3bcc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.948+0000 7f3bd4198700 1 -- 172.21.15.5:0/1746922787 shutdown_connections 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.948+0000 7f3bd4198700 1 -- 172.21.15.5:0/1746922787 wait complete. 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.949+0000 7f3bd4198700 1 Processor -- start 2024-05-08T23:20:56.951 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.949+0000 7f3bd4198700 1 -- start start 2024-05-08T23:20:56.954 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd4198700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc06b3d0 0x7f3bcc1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:56.954 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd4198700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cc570 0x7f3bcc157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd4198700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc15dae0 0x7f3bcc15fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd4198700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc158d60 con 0x7f3bcc0cc570 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd4198700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc15ec80 con 0x7f3bcc15dae0 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd4198700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc15ef60 con 0x7f3bcc06b3d0 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1733700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cc570 0x7f3bcc157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1733700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cc570 0x7f3bcc157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33560/0 (socket says 172.21.15.5:33560) 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1733700 1 -- 172.21.15.5:0/452794277 learned_addr learned my addr 172.21.15.5:0/452794277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd2735700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc15dae0 0x7f3bcc15fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1f34700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc06b3d0 0x7f3bcc1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1733700 1 -- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc06b3d0 msgr2=0x7f3bcc1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1733700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc06b3d0 0x7f3bcc1584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1733700 1 -- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc15dae0 msgr2=0x7f3bcc15fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1733700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc15dae0 0x7f3bcc15fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.950+0000 7f3bd1733700 1 -- 172.21.15.5:0/452794277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009550 con 0x7f3bcc0cc570 2024-05-08T23:20:56.955 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.951+0000 7f3bd1733700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cc570 0x7f3bcc157ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00b450 tx=0x7f3bbc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.951+0000 7f3bc27fc700 1 -- 172.21.15.5:0/452794277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc011830 con 0x7f3bcc0cc570 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.951+0000 7f3bc27fc700 1 -- 172.21.15.5:0/452794277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc011990 con 0x7f3bcc0cc570 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.951+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc15f240 con 0x7f3bcc0cc570 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.951+0000 7f3bc27fc700 1 -- 172.21.15.5:0/452794277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc011d40 con 0x7f3bcc0cc570 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.951+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc15f800 con 0x7f3bcc0cc570 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.952+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc15fa20 con 0x7f3bcc0cc570 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.952+0000 7f3bc27fc700 1 -- 172.21.15.5:0/452794277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f3bbc01a400 con 0x7f3bcc0cc570 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.953+0000 7f3bc27fc700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb006ac00 0x7f3bb006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.953+0000 7f3bc27fc700 1 -- 172.21.15.5:0/452794277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f3bbc0889a0 con 0x7f3bcc0cc570 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.953+0000 7f3bd1f34700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb006ac00 0x7f3bb006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:56.956 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.954+0000 7f3bd1f34700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb006ac00 0x7f3bb006d0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009130 tx=0x7f3bc8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:56.958 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:56.958+0000 7f3bc27fc700 1 -- 172.21.15.5:0/452794277 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3bbc0541c0 con 0x7f3bcc0cc570 2024-05-08T23:20:57.179 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.179+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bcc0ce720 con 0x7f3bcc0cc570 2024-05-08T23:20:57.180 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.180+0000 7f3bc27fc700 1 -- 172.21.15.5:0/452794277 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10344 (secure 0 0 0) 0x7f3bbc0577e0 con 0x7f3bcc0cc570 2024-05-08T23:20:57.180 INFO:teuthology.misc.health.smithi005.stdout: 2024-05-08T23:20:57.180 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":13,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:20:56.597803+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:20:57.183 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb006ac00 msgr2=0x7f3bb006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.183 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb006ac00 0x7f3bb006d0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009130 tx=0x7f3bc8008030 comp rx=0 tx=0).stop 2024-05-08T23:20:57.183 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cc570 msgr2=0x7f3bcc157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.183 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cc570 0x7f3bcc157ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00b450 tx=0x7f3bbc00b810 comp rx=0 tx=0).stop 2024-05-08T23:20:57.183 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 shutdown_connections 2024-05-08T23:20:57.183 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb006ac00 0x7f3bb006d0c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.183 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc15dae0 0x7f3bcc15fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.183 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc06b3d0 0x7f3bcc1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.184 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 --2- 172.21.15.5:0/452794277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cc570 0x7f3bcc157ca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.184 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 >> 172.21.15.5:0/452794277 conn(0x7f3bcc0b1f80 msgr2=0x7f3bcc0b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:57.184 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 shutdown_connections 2024-05-08T23:20:57.184 INFO:teuthology.misc.health.smithi005.stderr:2024-05-08T23:20:57.183+0000 7f3bd4198700 1 -- 172.21.15.5:0/452794277 wait complete. 2024-05-08T23:20:57.195 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T23:20:57.195 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T23:20:57.196 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:20:57.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:20:57.198 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:20:57.198 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:20:57.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:20:57.201 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:20:57.201 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:20:57.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:20:57.204 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:20:57.204 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:20:57.204 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:20:57.207 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:20:57.207 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-08T23:20:57.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:20:57.288 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:20:57.289 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:20:57.300 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:20:57.300 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:20:57.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.353+0000 7f0bae730700 1 Processor -- start 2024-05-08T23:20:57.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.353+0000 7f0bae730700 1 -- start start 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bae730700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80c6ea0 0x7f0ba80b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bae730700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba80b0a10 con 0x7f0ba815aa10 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bae730700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba80b0cf0 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bae730700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba80b0fd0 con 0x7f0ba80c6ea0 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bacccd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80c6ea0 0x7f0ba80b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0ba77fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0ba815aa10 0x7f0ba80c6790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49138/0 (socket says 172.21.15.5:49138) 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0ba7fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0ba80ce4d0 0x7f0ba80ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:55186/0 (socket says 172.21.15.5:55186) 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0ba7fff700 1 -- 172.21.15.5:0/1917389415 learned_addr learned my addr 172.21.15.5:0/1917389415 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0ba6ffd700 1 -- 172.21.15.5:0/1917389415 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502531459 0 0) 0x7f0ba80b0a10 con 0x7f0ba815aa10 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0ba6ffd700 1 -- 172.21.15.5:0/1917389415 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b900035b0 con 0x7f0ba815aa10 2024-05-08T23:20:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0ba6ffd700 1 -- 172.21.15.5:0/1917389415 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294753588 0 0) 0x7f0ba80b0cf0 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bacccd700 1 -- 172.21.15.5:0/1917389415 >> v1:172.21.15.49:6789/0 conn(0x7f0ba80ce4d0 legacy=0x7f0ba80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bacccd700 1 -- 172.21.15.5:0/1917389415 >> v1:172.21.15.5:6789/0 conn(0x7f0ba815aa10 legacy=0x7f0ba80c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bacccd700 1 -- 172.21.15.5:0/1917389415 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba80b12b0 con 0x7f0ba80c6ea0 2024-05-08T23:20:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.354+0000 7f0bacccd700 1 --2- 172.21.15.5:0/1917389415 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80c6ea0 0x7f0ba80b1ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009dc0 tx=0x7f0ba000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb83fdcb9f0a430a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0ba6ffd700 1 -- 172.21.15.5:0/1917389415 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000e070 con 0x7f0ba80c6ea0 2024-05-08T23:20:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0ba6ffd700 1 -- 172.21.15.5:0/1917389415 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba000bb20 con 0x7f0ba80c6ea0 2024-05-08T23:20:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0ba6ffd700 1 -- 172.21.15.5:0/1917389415 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000bed0 con 0x7f0ba80c6ea0 2024-05-08T23:20:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0bae730700 1 -- 172.21.15.5:0/1917389415 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80c6ea0 msgr2=0x7f0ba80b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0bae730700 1 --2- 172.21.15.5:0/1917389415 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80c6ea0 0x7f0ba80b1ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009dc0 tx=0x7f0ba000b330 comp rx=0 tx=0).stop 2024-05-08T23:20:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0bae730700 1 -- 172.21.15.5:0/1917389415 shutdown_connections 2024-05-08T23:20:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0bae730700 1 --2- 172.21.15.5:0/1917389415 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80c6ea0 0x7f0ba80b1ab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0bae730700 1 -- 172.21.15.5:0/1917389415 >> 172.21.15.5:0/1917389415 conn(0x7f0ba8099a50 msgr2=0x7f0ba80b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0bae730700 1 -- 172.21.15.5:0/1917389415 shutdown_connections 2024-05-08T23:20:57.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.355+0000 7f0bae730700 1 -- 172.21.15.5:0/1917389415 wait complete. 2024-05-08T23:20:57.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.357+0000 7f0bae730700 1 Processor -- start 2024-05-08T23:20:57.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.357+0000 7f0bae730700 1 -- start start 2024-05-08T23:20:57.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bae730700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ba80c6ea0 0x7f0ba806e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bae730700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80ce4d0 0x7f0ba8154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bae730700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ba815aa10 0x7f0ba813acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bae730700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba81553e0 con 0x7f0ba815aa10 2024-05-08T23:20:57.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bae730700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba806e760 con 0x7f0ba80c6ea0 2024-05-08T23:20:57.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bae730700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba806ea40 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bacccd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ba815aa10 0x7f0ba813acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0ba77fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80ce4d0 0x7f0ba8154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bacccd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ba815aa10 0x7f0ba813acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33590/0 (socket says 172.21.15.5:33590) 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ba80c6ea0 0x7f0ba806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bacccd700 1 -- 172.21.15.5:0/870126063 learned_addr learned my addr 172.21.15.5:0/870126063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0ba77fe700 1 -- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ba80c6ea0 msgr2=0x7f0ba806e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0ba77fe700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ba80c6ea0 0x7f0ba806e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0ba77fe700 1 -- 172.21.15.5:0/870126063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ba815aa10 msgr2=0x7f0ba813acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0ba77fe700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ba815aa10 0x7f0ba813acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0ba77fe700 1 -- 172.21.15.5:0/870126063 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba0009a70 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0bacccd700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ba815aa10 0x7f0ba813acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.358+0000 7f0ba77fe700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80ce4d0 0x7f0ba8154ea0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8072f10 tx=0x7f0b9c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.359+0000 7f0ba4ff9700 1 -- 172.21.15.5:0/870126063 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c004a70 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.359+0000 7f0ba4ff9700 1 -- 172.21.15.5:0/870126063 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c004bf0 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.359+0000 7f0ba4ff9700 1 -- 172.21.15.5:0/870126063 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0070d0 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.359+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba806ed20 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.359+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba806f2e0 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.360+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba8068000 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.360+0000 7f0ba4ff9700 1 -- 172.21.15.5:0/870126063 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f0b9c007230 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.361+0000 7f0ba4ff9700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac00 0x7f0b9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.361+0000 7f0ba4ff9700 1 -- 172.21.15.5:0/870126063 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f0b9c088250 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.361+0000 7f0ba7fff700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac00 0x7f0b9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.361+0000 7f0ba7fff700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac00 0x7f0b9406d0c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80b66a0 tx=0x7f0b98004a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:57.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.366+0000 7f0ba4ff9700 1 -- 172.21.15.5:0/870126063 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0b9c0539e0 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.584+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ba8159fd0 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.585+0000 7f0ba4ff9700 1 -- 172.21.15.5:0/870126063 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10344 (secure 0 0 0) 0x7f0b9c057000 con 0x7f0ba80ce4d0 2024-05-08T23:20:57.586 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:57.586 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:20:56.597803+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:20:57.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac00 msgr2=0x7f0b9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac00 0x7f0b9406d0c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80b66a0 tx=0x7f0b98004a50 comp rx=0 tx=0).stop 2024-05-08T23:20:57.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80ce4d0 msgr2=0x7f0ba8154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80ce4d0 0x7f0ba8154ea0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8072f10 tx=0x7f0b9c00c6e0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 shutdown_connections 2024-05-08T23:20:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac00 0x7f0b9406d0c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ba815aa10 0x7f0ba813acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ba80c6ea0 0x7f0ba806e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 --2- 172.21.15.5:0/870126063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ba80ce4d0 0x7f0ba8154ea0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 >> 172.21.15.5:0/870126063 conn(0x7f0ba8099a50 msgr2=0x7f0ba80c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 shutdown_connections 2024-05-08T23:20:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.588+0000 7f0bae730700 1 -- 172.21.15.5:0/870126063 wait complete. 2024-05-08T23:20:57.602 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-05-08T23:20:57.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:20:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.738+0000 7f0d6bce5700 1 Processor -- start 2024-05-08T23:20:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.738+0000 7f0d6bce5700 1 -- start start 2024-05-08T23:20:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d6bce5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0d6415db00 0x7f0d6406dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d6bce5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0d640ce120 0x7f0d6406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d6bce5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d6415e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d6bce5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6406fbf0 con 0x7f0d6415db00 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d6bce5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6406edb0 con 0x7f0d640ce120 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d6bce5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6406f090 con 0x7f0d6406e700 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d69a81700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0d6415db00 0x7f0d6406dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d69a81700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0d6415db00 0x7f0d6406dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36518/0 (socket says 172.21.15.5:36518) 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.739+0000 7f0d69a81700 1 -- 172.21.15.5:0/1663222199 learned_addr learned my addr 172.21.15.5:0/1663222199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d6a282700 1 --2- 172.21.15.5:0/1663222199 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d6415e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d69280700 1 --2- 172.21.15.5:0/1663222199 >> v2:172.21.15.49:3300/0 conn(0x7f0d640ce120 0x7f0d6406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d69a81700 1 -- 172.21.15.5:0/1663222199 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 msgr2=0x7f0d6415e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d69a81700 1 --2- 172.21.15.5:0/1663222199 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d6415e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d69a81700 1 -- 172.21.15.5:0/1663222199 >> v2:172.21.15.49:3300/0 conn(0x7f0d640ce120 msgr2=0x7f0d6406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d69a81700 1 --2- 172.21.15.5:0/1663222199 >> v2:172.21.15.49:3300/0 conn(0x7f0d640ce120 0x7f0d6406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d69a81700 1 -- 172.21.15.5:0/1663222199 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6406f370 con 0x7f0d6415db00 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d69280700 1 --2- 172.21.15.5:0/1663222199 >> v2:172.21.15.49:3300/0 conn(0x7f0d640ce120 0x7f0d6406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d69a81700 1 --2- 172.21.15.5:0/1663222199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d6406dc80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0d64072f60 tx=0x7f0d6000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8bd24f8309d763 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d68a7f700 1 -- 172.21.15.5:0/1663222199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000dc70 con 0x7f0d6415db00 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d68a7f700 1 -- 172.21.15.5:0/1663222199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6000ddd0 con 0x7f0d6415db00 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.740+0000 7f0d68a7f700 1 -- 172.21.15.5:0/1663222199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60010690 con 0x7f0d6415db00 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 -- 172.21.15.5:0/1663222199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 msgr2=0x7f0d6406dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/1663222199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d6406dc80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0d64072f60 tx=0x7f0d6000be70 comp rx=0 tx=0).stop 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 -- 172.21.15.5:0/1663222199 shutdown_connections 2024-05-08T23:20:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/1663222199 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d6415e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/1663222199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d6406dc80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/1663222199 >> v2:172.21.15.49:3300/0 conn(0x7f0d640ce120 0x7f0d6406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 -- 172.21.15.5:0/1663222199 >> 172.21.15.5:0/1663222199 conn(0x7f0d64099a50 msgr2=0x7f0d640d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 -- 172.21.15.5:0/1663222199 shutdown_connections 2024-05-08T23:20:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.741+0000 7f0d6bce5700 1 -- 172.21.15.5:0/1663222199 wait complete. 2024-05-08T23:20:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.742+0000 7f0d6bce5700 1 Processor -- start 2024-05-08T23:20:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.742+0000 7f0d6bce5700 1 -- start start 2024-05-08T23:20:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6bce5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d640c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6bce5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d640ce120 0x7f0d640c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6bce5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d640c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6bce5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d64151ca0 con 0x7f0d6415db00 2024-05-08T23:20:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6bce5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640c5dc0 con 0x7f0d640ce120 2024-05-08T23:20:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6bce5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640c51b0 con 0x7f0d6406e700 2024-05-08T23:20:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6a282700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d640c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6a282700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d640c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36532/0 (socket says 172.21.15.5:36532) 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6a282700 1 -- 172.21.15.5:0/99033426 learned_addr learned my addr 172.21.15.5:0/99033426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d69280700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d640ce120 0x7f0d640c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d69a81700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d640c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6a282700 1 -- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 msgr2=0x7f0d640c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6a282700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d640c4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6a282700 1 -- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d640ce120 msgr2=0x7f0d640c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6a282700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d640ce120 0x7f0d640c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d6a282700 1 -- 172.21.15.5:0/99033426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6000b7f0 con 0x7f0d6415db00 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.743+0000 7f0d69a81700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d640c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.744+0000 7f0d6a282700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d640c9140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00eed0 tx=0x7f0d5c00b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.744+0000 7f0d5a7fc700 1 -- 172.21.15.5:0/99033426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c010040 con 0x7f0d6415db00 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.744+0000 7f0d5a7fc700 1 -- 172.21.15.5:0/99033426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c00bb70 con 0x7f0d6415db00 2024-05-08T23:20:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.744+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d640c5490 con 0x7f0d6415db00 2024-05-08T23:20:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.744+0000 7f0d5a7fc700 1 -- 172.21.15.5:0/99033426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c014400 con 0x7f0d6415db00 2024-05-08T23:20:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.744+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d640c5a50 con 0x7f0d6415db00 2024-05-08T23:20:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.745+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d640c5f50 con 0x7f0d6415db00 2024-05-08T23:20:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.745+0000 7f0d5a7fc700 1 -- 172.21.15.5:0/99033426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f0d5c014560 con 0x7f0d6415db00 2024-05-08T23:20:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.746+0000 7f0d5a7fc700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d4806ac00 0x7f0d4806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.746+0000 7f0d5a7fc700 1 -- 172.21.15.5:0/99033426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4846+0+0 (secure 0 0 0) 0x7f0d5c0893a0 con 0x7f0d6415db00 2024-05-08T23:20:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.746+0000 7f0d69a81700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d4806ac00 0x7f0d4806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.746+0000 7f0d69a81700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d4806ac00 0x7f0d4806d0c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000beb0 tx=0x7f0d6001b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:57.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:57.751+0000 7f0d5a7fc700 1 -- 172.21.15.5:0/99033426 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0d5c054920 con 0x7f0d6415db00 2024-05-08T23:20:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.023+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d64068000 con 0x7f0d6415db00 2024-05-08T23:20:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.023+0000 7f0d5a7fc700 1 -- 172.21.15.5:0/99033426 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7f0d5c018500 con 0x7f0d6415db00 2024-05-08T23:20:58.024 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:20:58.024 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.025+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d4806ac00 msgr2=0x7f0d4806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.025+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d4806ac00 0x7f0d4806d0c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000beb0 tx=0x7f0d6001b040 comp rx=0 tx=0).stop 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.025+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 msgr2=0x7f0d640c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.025+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d640c9140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00eed0 tx=0x7f0d5c00b0b0 comp rx=0 tx=0).stop 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.026+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 shutdown_connections 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.026+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d4806ac00 0x7f0d4806d0c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.026+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d6415db00 0x7f0d640c9140 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.026+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d6406e700 0x7f0d640c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.026+0000 7f0d6bce5700 1 --2- 172.21.15.5:0/99033426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d640ce120 0x7f0d640c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.026+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 >> 172.21.15.5:0/99033426 conn(0x7f0d64099a50 msgr2=0x7f0d640cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.026+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 shutdown_connections 2024-05-08T23:20:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:20:58.026+0000 7f0d6bce5700 1 -- 172.21.15.5:0/99033426 wait complete. 2024-05-08T23:20:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-05-08T23:21:00.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.176+0000 7fbc9367b700 1 Processor -- start 2024-05-08T23:21:00.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.176+0000 7fbc9367b700 1 -- start start 2024-05-08T23:21:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc9367b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbc8c15f9b0 0x7fbc8c0cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc9367b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc8c162ca0 0x7fbc8c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc9367b700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8c06d540 con 0x7fbc8c06aea0 2024-05-08T23:21:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc9367b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c0b3840 con 0x7fbc8c162ca0 2024-05-08T23:21:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc9367b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c06dd50 con 0x7fbc8c15f9b0 2024-05-08T23:21:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc90c16700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc8c162ca0 0x7fbc8c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc90c16700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc8c162ca0 0x7fbc8c06a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36542/0 (socket says 172.21.15.5:36542) 2024-05-08T23:21:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc90c16700 1 -- 172.21.15.5:0/3637119077 learned_addr learned my addr 172.21.15.5:0/3637119077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc91417700 1 --2- 172.21.15.5:0/3637119077 >> v2:172.21.15.49:3300/0 conn(0x7fbc8c15f9b0 0x7fbc8c0cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.177+0000 7fbc90c16700 1 -- 172.21.15.5:0/3637119077 >> v2:172.21.15.49:3300/0 conn(0x7fbc8c15f9b0 msgr2=0x7fbc8c0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc90c16700 1 --2- 172.21.15.5:0/3637119077 >> v2:172.21.15.49:3300/0 conn(0x7fbc8c15f9b0 0x7fbc8c0cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc90c16700 1 -- 172.21.15.5:0/3637119077 >> v1:172.21.15.49:6789/0 conn(0x7fbc8c06aea0 legacy=0x7fbc8c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc90c16700 1 -- 172.21.15.5:0/3637119077 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c163100 con 0x7fbc8c162ca0 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc90c16700 1 --2- 172.21.15.5:0/3637119077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 0x7fbc8c06a960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0b5010 tx=0x7fbc7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e30de111be8633 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc83fff700 1 -- 172.21.15.5:0/3637119077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c011830 con 0x7fbc8c162ca0 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc83fff700 1 -- 172.21.15.5:0/3637119077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c011990 con 0x7fbc8c162ca0 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc83fff700 1 -- 172.21.15.5:0/3637119077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c011d20 con 0x7fbc8c162ca0 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc9367b700 1 -- 172.21.15.5:0/3637119077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 msgr2=0x7fbc8c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc9367b700 1 --2- 172.21.15.5:0/3637119077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 0x7fbc8c06a960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0b5010 tx=0x7fbc7c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc9367b700 1 -- 172.21.15.5:0/3637119077 shutdown_connections 2024-05-08T23:21:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc9367b700 1 --2- 172.21.15.5:0/3637119077 >> v2:172.21.15.49:3300/0 conn(0x7fbc8c15f9b0 0x7fbc8c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc9367b700 1 --2- 172.21.15.5:0/3637119077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 0x7fbc8c06a960 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.178+0000 7fbc9367b700 1 -- 172.21.15.5:0/3637119077 >> 172.21.15.5:0/3637119077 conn(0x7fbc8c0cfce0 msgr2=0x7fbc8c0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.179+0000 7fbc9367b700 1 -- 172.21.15.5:0/3637119077 shutdown_connections 2024-05-08T23:21:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.179+0000 7fbc9367b700 1 -- 172.21.15.5:0/3637119077 wait complete. 2024-05-08T23:21:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.180+0000 7fbc9367b700 1 Processor -- start 2024-05-08T23:21:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.180+0000 7fbc9367b700 1 -- start start 2024-05-08T23:21:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc9367b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc8c06aea0 0x7fbc8c1f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc9367b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc8c15f9b0 0x7fbc8c1f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc9367b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 0x7fbc8c1f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc9367b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c1f87e0 con 0x7fbc8c162ca0 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc9367b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c1f5250 con 0x7fbc8c06aea0 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc9367b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0c4db0 con 0x7fbc8c15f9b0 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc8c06aea0 0x7fbc8c1f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91c18700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 0x7fbc8c1f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc8c06aea0 0x7fbc8c1f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:42658/0 (socket says 172.21.15.5:42658) 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 -- 172.21.15.5:0/2085256017 learned_addr learned my addr 172.21.15.5:0/2085256017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc90c16700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc8c15f9b0 0x7fbc8c1f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 -- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc8c15f9b0 msgr2=0x7fbc8c1f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc8c15f9b0 0x7fbc8c1f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 -- 172.21.15.5:0/2085256017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 msgr2=0x7fbc8c1f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 0x7fbc8c1f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 -- 172.21.15.5:0/2085256017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c009550 con 0x7fbc8c06aea0 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91c18700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 0x7fbc8c1f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc90c16700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc8c15f9b0 0x7fbc8c1f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.181+0000 7fbc91417700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc8c06aea0 0x7fbc8c1f8a10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800e870 tx=0x7fbc8800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.182+0000 7fbc81ffb700 1 -- 172.21.15.5:0/2085256017 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88009590 con 0x7fbc8c06aea0 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.182+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c0c5090 con 0x7fbc8c06aea0 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.182+0000 7fbc81ffb700 1 -- 172.21.15.5:0/2085256017 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc88013680 con 0x7fbc8c06aea0 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.182+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c0c5650 con 0x7fbc8c06aea0 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.182+0000 7fbc81ffb700 1 -- 172.21.15.5:0/2085256017 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc880139f0 con 0x7fbc8c06aea0 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.183+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8c068000 con 0x7fbc8c06aea0 2024-05-08T23:21:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.184+0000 7fbc81ffb700 1 -- 172.21.15.5:0/2085256017 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7fbc88013b50 con 0x7fbc8c06aea0 2024-05-08T23:21:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.184+0000 7fbc81ffb700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc7006ac00 0x7fbc7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.184+0000 7fbc81ffb700 1 -- 172.21.15.5:0/2085256017 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbc88089670 con 0x7fbc8c06aea0 2024-05-08T23:21:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.184+0000 7fbc90c16700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc7006ac00 0x7fbc7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.185+0000 7fbc90c16700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc7006ac00 0x7fbc7006d0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009c10 tx=0x7fbc7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:00.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.194+0000 7fbc81ffb700 1 -- 172.21.15.5:0/2085256017 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbc88054be0 con 0x7fbc8c06aea0 2024-05-08T23:21:00.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.269+0000 7fbc81ffb700 1 -- 172.21.15.5:0/2085256017 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbc88054fb0 con 0x7fbc8c06aea0 2024-05-08T23:21:00.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.465+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc8c161c20 con 0x7fbc8c06aea0 2024-05-08T23:21:00.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.466+0000 7fbc81ffb700 1 -- 172.21.15.5:0/2085256017 <== mon.1 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3515 (secure 0 0 0) 0x7fbc880841c0 con 0x7fbc8c06aea0 2024-05-08T23:21:00.466 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:00.466 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-08T23:21:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc7006ac00 msgr2=0x7fbc7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc7006ac00 0x7fbc7006d0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009c10 tx=0x7fbc7c008030 comp rx=0 tx=0).stop 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc8c06aea0 msgr2=0x7fbc8c1f8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc8c06aea0 0x7fbc8c1f8a10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800e870 tx=0x7fbc8800eb80 comp rx=0 tx=0).stop 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 shutdown_connections 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc7006ac00 0x7fbc7006d0c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c162ca0 0x7fbc8c1f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc8c06aea0 0x7fbc8c1f8a10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 --2- 172.21.15.5:0/2085256017 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc8c15f9b0 0x7fbc8c1f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 >> 172.21.15.5:0/2085256017 conn(0x7fbc8c0cfce0 msgr2=0x7fbc8c165780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 shutdown_connections 2024-05-08T23:21:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.468+0000 7fbc9367b700 1 -- 172.21.15.5:0/2085256017 wait complete. 2024-05-08T23:21:00.470 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-05-08T23:21:00.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.618+0000 7f4aced72700 1 Processor -- start 2024-05-08T23:21:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.618+0000 7f4aced72700 1 -- start start 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4aced72700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac809a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4aced72700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80d0840 con 0x7f4ac8161030 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4aced72700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac806c600 con 0x7f4ac80cbbc0 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4aced72700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac806ccc0 con 0x7f4ac80b2480 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4accb0e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4ac8161030 0x7f4ac8099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56190/0 (socket says 172.21.15.5:56190) 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4accb0e700 1 -- 172.21.15.5:0/2899329894 learned_addr learned my addr 172.21.15.5:0/2899329894 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4ac7fff700 1 --2- 172.21.15.5:0/2899329894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac809a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4ac77fe700 1 -- 172.21.15.5:0/2899329894 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471021035 0 0) 0x7f4ac80d0840 con 0x7f4ac8161030 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4ac77fe700 1 -- 172.21.15.5:0/2899329894 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab80035b0 con 0x7f4ac8161030 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4ac77fe700 1 -- 172.21.15.5:0/2899329894 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440202950 0 0) 0x7f4ac806c600 con 0x7f4ac80cbbc0 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4ac77fe700 1 -- 172.21.15.5:0/2899329894 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac80d0840 con 0x7f4ac80cbbc0 2024-05-08T23:21:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4ac7fff700 1 -- 172.21.15.5:0/2899329894 >> v1:172.21.15.49:6789/0 conn(0x7f4ac80cbbc0 legacy=0x7f4ac80cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4ac7fff700 1 -- 172.21.15.5:0/2899329894 >> v1:172.21.15.5:6789/0 conn(0x7f4ac8161030 legacy=0x7f4ac8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.619+0000 7f4ac7fff700 1 -- 172.21.15.5:0/2899329894 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac806c8e0 con 0x7f4ac80b2480 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4ac7fff700 1 --2- 172.21.15.5:0/2899329894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac809a160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009870 tx=0x7f4ab4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1a70097afb08d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4ac77fe700 1 -- 172.21.15.5:0/2899329894 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4011830 con 0x7f4ac80b2480 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4ac77fe700 1 -- 172.21.15.5:0/2899329894 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab4011990 con 0x7f4ac80b2480 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4ac77fe700 1 -- 172.21.15.5:0/2899329894 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4011d20 con 0x7f4ac80b2480 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4aced72700 1 -- 172.21.15.5:0/2899329894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 msgr2=0x7f4ac809a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4aced72700 1 --2- 172.21.15.5:0/2899329894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac809a160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009870 tx=0x7f4ab4009b80 comp rx=0 tx=0).stop 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4aced72700 1 -- 172.21.15.5:0/2899329894 shutdown_connections 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4aced72700 1 --2- 172.21.15.5:0/2899329894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac809a160 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4aced72700 1 -- 172.21.15.5:0/2899329894 >> 172.21.15.5:0/2899329894 conn(0x7f4ac80b2f20 msgr2=0x7f4ac80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:00.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4aced72700 1 -- 172.21.15.5:0/2899329894 shutdown_connections 2024-05-08T23:21:00.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.620+0000 7f4aced72700 1 -- 172.21.15.5:0/2899329894 wait complete. 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.622+0000 7f4aced72700 1 Processor -- start 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.622+0000 7f4aced72700 1 -- start start 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4aced72700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac80b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4aced72700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ac80cbbc0 0x7f4ac80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4aced72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8161030 0x7f4ac80c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4aced72700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80d0660 con 0x7f4ac8161030 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4aced72700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80c54e0 con 0x7f4ac80cbbc0 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4aced72700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80b5380 con 0x7f4ac80b2480 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8161030 0x7f4ac80c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8161030 0x7f4ac80c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36558/0 (socket says 172.21.15.5:36558) 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 -- 172.21.15.5:0/2564849011 learned_addr learned my addr 172.21.15.5:0/2564849011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4ac7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ac80cbbc0 0x7f4ac80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4accb0e700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac80b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 -- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 msgr2=0x7f4ac80b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac80b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 -- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ac80cbbc0 msgr2=0x7f4ac80b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ac80cbbc0 0x7f4ac80b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 -- 172.21.15.5:0/2564849011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab40094a0 con 0x7f4ac8161030 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4ac7fff700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ac80cbbc0 0x7f4ac80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.623+0000 7f4acd30f700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8161030 0x7f4ac80c9780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000cc20 tx=0x7f4ac000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.624+0000 7f4ac57fa700 1 -- 172.21.15.5:0/2564849011 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0003c00 con 0x7f4ac8161030 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.624+0000 7f4ac57fa700 1 -- 172.21.15.5:0/2564849011 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0003d60 con 0x7f4ac8161030 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.624+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac80b5660 con 0x7f4ac8161030 2024-05-08T23:21:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.624+0000 7f4ac57fa700 1 -- 172.21.15.5:0/2564849011 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000aae0 con 0x7f4ac8161030 2024-05-08T23:21:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.624+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac80c5170 con 0x7f4ac8161030 2024-05-08T23:21:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.625+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac8068000 con 0x7f4ac8161030 2024-05-08T23:21:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.625+0000 7f4ac57fa700 1 -- 172.21.15.5:0/2564849011 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4ac000ac40 con 0x7f4ac8161030 2024-05-08T23:21:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.625+0000 7f4ac57fa700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4aac06ac90 0x7f4aac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.626+0000 7f4ac57fa700 1 -- 172.21.15.5:0/2564849011 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4ac0088fe0 con 0x7f4ac8161030 2024-05-08T23:21:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.626+0000 7f4accb0e700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4aac06ac90 0x7f4aac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.626+0000 7f4accb0e700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4aac06ac90 0x7f4aac06d150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4abc007fd0 tx=0x7f4abc007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.631+0000 7f4ac57fa700 1 -- 172.21.15.5:0/2564849011 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4ac0054520 con 0x7f4ac8161030 2024-05-08T23:21:00.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.846+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ac80d0f50 con 0x7f4ac8161030 2024-05-08T23:21:00.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.847+0000 7f4ac57fa700 1 -- 172.21.15.5:0/2564849011 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12286 (secure 0 0 0) 0x7f4ac0057b40 con 0x7f4ac8161030 2024-05-08T23:21:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:00.847 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:20:59.602729+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.849+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4aac06ac90 msgr2=0x7f4aac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.849+0000 7f4aced72700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4aac06ac90 0x7f4aac06d150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4abc007fd0 tx=0x7f4abc007dc0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.849+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8161030 msgr2=0x7f4ac80c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.849+0000 7f4aced72700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8161030 0x7f4ac80c9780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000cc20 tx=0x7f4ac000cf30 comp rx=0 tx=0).stop 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.850+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 shutdown_connections 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.850+0000 7f4aced72700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4aac06ac90 0x7f4aac06d150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.850+0000 7f4aced72700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8161030 0x7f4ac80c9780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.850+0000 7f4aced72700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4ac80b2480 0x7f4ac80b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.850+0000 7f4aced72700 1 --2- 172.21.15.5:0/2564849011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ac80cbbc0 0x7f4ac80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.850+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 >> 172.21.15.5:0/2564849011 conn(0x7f4ac80b2f20 msgr2=0x7f4ac806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.850+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 shutdown_connections 2024-05-08T23:21:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:00.850+0000 7f4aced72700 1 -- 172.21.15.5:0/2564849011 wait complete. 2024-05-08T23:21:00.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:00.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.003+0000 7f463aab4700 1 Processor -- start 2024-05-08T23:21:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.003+0000 7f463aab4700 1 -- start start 2024-05-08T23:21:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f463aab4700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f463415b550 0x7f463409a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f463aab4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463409a160 0x7f46340b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f463aab4700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46340cd440 con 0x7f463415b930 2024-05-08T23:21:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f463aab4700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340ccba0 con 0x7f463415b550 2024-05-08T23:21:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f463aab4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340cce80 con 0x7f463409a160 2024-05-08T23:21:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4639051700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463409a160 0x7f46340b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4638850700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f463415b550 0x7f463409a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4633fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f463415b930 0x7f4634099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:46210/0 (socket says 172.21.15.5:46210) 2024-05-08T23:21:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4633fff700 1 -- 172.21.15.5:0/2328830157 learned_addr learned my addr 172.21.15.5:0/2328830157 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f46337fe700 1 -- 172.21.15.5:0/2328830157 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693025091 0 0) 0x7f46340cd440 con 0x7f463415b930 2024-05-08T23:21:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f46337fe700 1 -- 172.21.15.5:0/2328830157 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f461c0035b0 con 0x7f463415b930 2024-05-08T23:21:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4639051700 1 -- 172.21.15.5:0/2328830157 >> v2:172.21.15.49:3300/0 conn(0x7f463415b550 msgr2=0x7f463409a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4639051700 1 --2- 172.21.15.5:0/2328830157 >> v2:172.21.15.49:3300/0 conn(0x7f463415b550 0x7f463409a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4639051700 1 -- 172.21.15.5:0/2328830157 >> v1:172.21.15.49:6789/0 conn(0x7f463415b930 legacy=0x7f4634099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4639051700 1 -- 172.21.15.5:0/2328830157 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340cd160 con 0x7f463409a160 2024-05-08T23:21:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4638850700 1 --2- 172.21.15.5:0/2328830157 >> v2:172.21.15.49:3300/0 conn(0x7f463415b550 0x7f463409a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.004+0000 7f4639051700 1 --2- 172.21.15.5:0/2328830157 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463409a160 0x7f46340b0cc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f462c009d90 tx=0x7f462c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a43143f4ae79fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f46337fe700 1 -- 172.21.15.5:0/2328830157 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00e070 con 0x7f463409a160 2024-05-08T23:21:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f46337fe700 1 -- 172.21.15.5:0/2328830157 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c00bb20 con 0x7f463409a160 2024-05-08T23:21:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f46337fe700 1 -- 172.21.15.5:0/2328830157 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c012430 con 0x7f463409a160 2024-05-08T23:21:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f463aab4700 1 -- 172.21.15.5:0/2328830157 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463409a160 msgr2=0x7f46340b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f463aab4700 1 --2- 172.21.15.5:0/2328830157 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463409a160 0x7f46340b0cc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f462c009d90 tx=0x7f462c00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f463aab4700 1 -- 172.21.15.5:0/2328830157 shutdown_connections 2024-05-08T23:21:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f463aab4700 1 --2- 172.21.15.5:0/2328830157 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463409a160 0x7f46340b0cc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f463aab4700 1 --2- 172.21.15.5:0/2328830157 >> v2:172.21.15.49:3300/0 conn(0x7f463415b550 0x7f463409a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f463aab4700 1 -- 172.21.15.5:0/2328830157 >> 172.21.15.5:0/2328830157 conn(0x7f46340b2ec0 msgr2=0x7f46340b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f463aab4700 1 -- 172.21.15.5:0/2328830157 shutdown_connections 2024-05-08T23:21:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.005+0000 7f463aab4700 1 -- 172.21.15.5:0/2328830157 wait complete. 2024-05-08T23:21:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.007+0000 7f463aab4700 1 Processor -- start 2024-05-08T23:21:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.007+0000 7f463aab4700 1 -- start start 2024-05-08T23:21:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f463aab4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463409a160 0x7f46340c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f463aab4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f463415b550 0x7f46341579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f463aab4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463415b930 0x7f46340ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f463aab4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c8320 con 0x7f463409a160 2024-05-08T23:21:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f463aab4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c5190 con 0x7f463415b550 2024-05-08T23:21:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f463aab4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b5570 con 0x7f463415b930 2024-05-08T23:21:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463409a160 0x7f46340c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4633fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f463415b550 0x7f46341579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463409a160 0x7f46340c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36568/0 (socket says 172.21.15.5:36568) 2024-05-08T23:21:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 -- 172.21.15.5:0/1301985357 learned_addr learned my addr 172.21.15.5:0/1301985357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:01.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4639051700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463415b930 0x7f46340ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 -- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463415b930 msgr2=0x7f46340ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463415b930 0x7f46340ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 -- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f463415b550 msgr2=0x7f46341579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f463415b550 0x7f46341579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 -- 172.21.15.5:0/1301985357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240094a0 con 0x7f463409a160 2024-05-08T23:21:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f4638850700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463409a160 0x7f46340c4c50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f462400ec10 tx=0x7f462400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f46317fa700 1 -- 172.21.15.5:0/1301985357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400c7f0 con 0x7f463409a160 2024-05-08T23:21:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f46317fa700 1 -- 172.21.15.5:0/1301985357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462400c950 con 0x7f463409a160 2024-05-08T23:21:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c009a70 con 0x7f463409a160 2024-05-08T23:21:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.008+0000 7f46317fa700 1 -- 172.21.15.5:0/1301985357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400ccc0 con 0x7f463409a160 2024-05-08T23:21:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.009+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46340c7d60 con 0x7f463409a160 2024-05-08T23:21:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.009+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46340b5850 con 0x7f463409a160 2024-05-08T23:21:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.010+0000 7f46317fa700 1 -- 172.21.15.5:0/1301985357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4624013470 con 0x7f463409a160 2024-05-08T23:21:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.010+0000 7f46317fa700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f462006ac90 0x7f462006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.010+0000 7f46317fa700 1 -- 172.21.15.5:0/1301985357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4624089060 con 0x7f463409a160 2024-05-08T23:21:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.011+0000 7f4633fff700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f462006ac90 0x7f462006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.011+0000 7f4633fff700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f462006ac90 0x7f462006d150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f463409f330 tx=0x7f4628007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.015+0000 7f46317fa700 1 -- 172.21.15.5:0/1301985357 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f46240bc900 con 0x7f463409a160 2024-05-08T23:21:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.233+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46340679a0 con 0x7f463409a160 2024-05-08T23:21:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.233+0000 7f46317fa700 1 -- 172.21.15.5:0/1301985357 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46240544f0 con 0x7f463409a160 2024-05-08T23:21:01.234 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:01.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f462006ac90 msgr2=0x7f462006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f462006ac90 0x7f462006d150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f463409f330 tx=0x7f4628007dc0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463409a160 msgr2=0x7f46340c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463409a160 0x7f46340c4c50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f462400ec10 tx=0x7f462400ef20 comp rx=0 tx=0).stop 2024-05-08T23:21:01.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 shutdown_connections 2024-05-08T23:21:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f462006ac90 0x7f462006d150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f463415b930 0x7f46340ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463409a160 0x7f46340c4c50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 --2- 172.21.15.5:0/1301985357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f463415b550 0x7f46341579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 >> 172.21.15.5:0/1301985357 conn(0x7f46340b2ec0 msgr2=0x7f4634099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.235+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 shutdown_connections 2024-05-08T23:21:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.236+0000 7f463aab4700 1 -- 172.21.15.5:0/1301985357 wait complete. 2024-05-08T23:21:01.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:21:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.385+0000 7fa41b979700 1 Processor -- start 2024-05-08T23:21:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.385+0000 7fa41b979700 1 -- start start 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa41b979700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41415b7b0 0x7fa4140cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa41b979700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa4140ac1c0 0x7fa4140ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa41b979700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140cb910 con 0x7fa41406d0a0 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa41b979700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4141600f0 con 0x7fa4140ac1c0 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa41b979700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140acb80 con 0x7fa41415b7b0 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa418f14700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa4140ac1c0 0x7fa4140ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa419f16700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa41406d0a0 0x7fa41406d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56196/0 (socket says 172.21.15.5:56196) 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa418f14700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa4140ac1c0 0x7fa4140ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36578/0 (socket says 172.21.15.5:36578) 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.386+0000 7fa419f16700 1 -- 172.21.15.5:0/733791686 learned_addr learned my addr 172.21.15.5:0/733791686 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa419715700 1 --2- 172.21.15.5:0/733791686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41415b7b0 0x7fa4140cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa418f14700 1 -- 172.21.15.5:0/733791686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41415b7b0 msgr2=0x7fa4140cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa418f14700 1 --2- 172.21.15.5:0/733791686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41415b7b0 0x7fa4140cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa418f14700 1 -- 172.21.15.5:0/733791686 >> v1:172.21.15.5:6789/0 conn(0x7fa41406d0a0 legacy=0x7fa41406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa418f14700 1 -- 172.21.15.5:0/733791686 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41415f190 con 0x7fa4140ac1c0 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa40bfff700 1 -- 172.21.15.5:0/733791686 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101369485 0 0) 0x7fa4140cb910 con 0x7fa41406d0a0 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa418f14700 1 --2- 172.21.15.5:0/733791686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ac640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa404009840 tx=0x7fa404009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109a128ca40794cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa40bfff700 1 -- 172.21.15.5:0/733791686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404011830 con 0x7fa4140ac1c0 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa40bfff700 1 -- 172.21.15.5:0/733791686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa404011990 con 0x7fa4140ac1c0 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.387+0000 7fa40bfff700 1 -- 172.21.15.5:0/733791686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404011d20 con 0x7fa4140ac1c0 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.388+0000 7fa41b979700 1 -- 172.21.15.5:0/733791686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 msgr2=0x7fa4140ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.388+0000 7fa41b979700 1 --2- 172.21.15.5:0/733791686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ac640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa404009840 tx=0x7fa404009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.388+0000 7fa41b979700 1 -- 172.21.15.5:0/733791686 shutdown_connections 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.388+0000 7fa41b979700 1 --2- 172.21.15.5:0/733791686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41415b7b0 0x7fa4140cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.388+0000 7fa41b979700 1 --2- 172.21.15.5:0/733791686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ac640 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.388+0000 7fa41b979700 1 -- 172.21.15.5:0/733791686 >> 172.21.15.5:0/733791686 conn(0x7fa4140cf270 msgr2=0x7fa4140cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.388+0000 7fa41b979700 1 -- 172.21.15.5:0/733791686 shutdown_connections 2024-05-08T23:21:01.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.388+0000 7fa41b979700 1 -- 172.21.15.5:0/733791686 wait complete. 2024-05-08T23:21:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.389+0000 7fa41b979700 1 Processor -- start 2024-05-08T23:21:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.389+0000 7fa41b979700 1 -- start start 2024-05-08T23:21:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa41b979700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41406d0a0 0x7fa4141ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa41b979700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa41b979700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140ce8e0 0x7fa41414e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa41b979700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140cd3d0 con 0x7fa4140ac1c0 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa41b979700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4141eeaf0 con 0x7fa4140ce8e0 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa41b979700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140cd5e0 con 0x7fa41406d0a0 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36584/0 (socket says 172.21.15.5:36584) 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 -- 172.21.15.5:0/3223797012 learned_addr learned my addr 172.21.15.5:0/3223797012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa419715700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41406d0a0 0x7fa4141ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa419f16700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140ce8e0 0x7fa41414e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 -- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41406d0a0 msgr2=0x7fa4141ee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41406d0a0 0x7fa4141ee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 -- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140ce8e0 msgr2=0x7fa41414e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140ce8e0 0x7fa41414e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 -- 172.21.15.5:0/3223797012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040094a0 con 0x7fa4140ac1c0 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa419f16700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140ce8e0 0x7fa41414e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.390+0000 7fa418f14700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ce3a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4040090b0 tx=0x7fa404004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.391+0000 7fa409ffb700 1 -- 172.21.15.5:0/3223797012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4040038a0 con 0x7fa4140ac1c0 2024-05-08T23:21:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.391+0000 7fa409ffb700 1 -- 172.21.15.5:0/3223797012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40401f070 con 0x7fa4140ac1c0 2024-05-08T23:21:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.391+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4140cd8c0 con 0x7fa4140ac1c0 2024-05-08T23:21:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.391+0000 7fa409ffb700 1 -- 172.21.15.5:0/3223797012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404003c20 con 0x7fa4140ac1c0 2024-05-08T23:21:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.391+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4140cde80 con 0x7fa4140ac1c0 2024-05-08T23:21:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.392+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4140678f0 con 0x7fa4140ac1c0 2024-05-08T23:21:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.392+0000 7fa409ffb700 1 -- 172.21.15.5:0/3223797012 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa404026480 con 0x7fa4140ac1c0 2024-05-08T23:21:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.393+0000 7fa409ffb700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3f806ac90 0x7fa3f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.393+0000 7fa409ffb700 1 -- 172.21.15.5:0/3223797012 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa404021070 con 0x7fa4140ac1c0 2024-05-08T23:21:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.393+0000 7fa419715700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3f806ac90 0x7fa3f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.394+0000 7fa419715700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3f806ac90 0x7fa3f806d150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa41000b9b0 tx=0x7fa410008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.398+0000 7fa409ffb700 1 -- 172.21.15.5:0/3223797012 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa404055f50 con 0x7fa4140ac1c0 2024-05-08T23:21:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.617+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa4140cfed0 con 0x7fa4140ac1c0 2024-05-08T23:21:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.617+0000 7fa409ffb700 1 -- 172.21.15.5:0/3223797012 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa404015020 con 0x7fa4140ac1c0 2024-05-08T23:21:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:01.618 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:21:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:21:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-08T23:21:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-08T23:21:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:21:01.618 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:21:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3f806ac90 msgr2=0x7fa3f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3f806ac90 0x7fa3f806d150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa41000b9b0 tx=0x7fa410008040 comp rx=0 tx=0).stop 2024-05-08T23:21:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 msgr2=0x7fa4140ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ce3a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4040090b0 tx=0x7fa404004750 comp rx=0 tx=0).stop 2024-05-08T23:21:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 shutdown_connections 2024-05-08T23:21:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3f806ac90 0x7fa3f806d150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140ce8e0 0x7fa41414e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa41406d0a0 0x7fa4141ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 --2- 172.21.15.5:0/3223797012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4140ac1c0 0x7fa4140ce3a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 >> 172.21.15.5:0/3223797012 conn(0x7fa4140cf270 msgr2=0x7fa414159560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 shutdown_connections 2024-05-08T23:21:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.620+0000 7fa41b979700 1 -- 172.21.15.5:0/3223797012 wait complete. 2024-05-08T23:21:01.633 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:21:01.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:21:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.774+0000 7f6de9080700 1 Processor -- start 2024-05-08T23:21:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.774+0000 7f6de9080700 1 -- start start 2024-05-08T23:21:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de9080700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6de40cbf50 0x7f6de4161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de9080700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de415e760 0x7f6de406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de9080700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de40cf970 con 0x7f6de4162240 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de9080700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de40cfc50 con 0x7f6de40cbf50 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de9080700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40cff30 con 0x7f6de415e760 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de2d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6de40cbf50 0x7f6de4161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de259c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6de4162240 0x7f6de415e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56202/0 (socket says 172.21.15.5:56202) 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de2d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6de40cbf50 0x7f6de4161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36598/0 (socket says 172.21.15.5:36598) 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de259c700 1 -- 172.21.15.5:0/1690861100 learned_addr learned my addr 172.21.15.5:0/1690861100 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de359e700 1 --2- 172.21.15.5:0/1690861100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de415e760 0x7f6de406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de2d9d700 1 -- 172.21.15.5:0/1690861100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de415e760 msgr2=0x7f6de406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de2d9d700 1 --2- 172.21.15.5:0/1690861100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de415e760 0x7f6de406a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de2d9d700 1 -- 172.21.15.5:0/1690861100 >> v1:172.21.15.5:6789/0 conn(0x7f6de4162240 legacy=0x7f6de415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de2d9d700 1 -- 172.21.15.5:0/1690861100 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de41617e0 con 0x7f6de40cbf50 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de1d9b700 1 -- 172.21.15.5:0/1690861100 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722069050 0 0) 0x7f6de40cf970 con 0x7f6de4162240 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de2d9d700 1 --2- 172.21.15.5:0/1690861100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40cbf50 0x7f6de4161d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009840 tx=0x7f6dd4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763e641d9b45ca8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de1d9b700 1 -- 172.21.15.5:0/1690861100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4011830 con 0x7f6de40cbf50 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de1d9b700 1 -- 172.21.15.5:0/1690861100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd4011990 con 0x7f6de40cbf50 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.775+0000 7f6de1d9b700 1 -- 172.21.15.5:0/1690861100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4011d20 con 0x7f6de40cbf50 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.776+0000 7f6de9080700 1 -- 172.21.15.5:0/1690861100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40cbf50 msgr2=0x7f6de4161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.776+0000 7f6de9080700 1 --2- 172.21.15.5:0/1690861100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40cbf50 0x7f6de4161d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009840 tx=0x7f6dd4009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.776+0000 7f6de9080700 1 -- 172.21.15.5:0/1690861100 shutdown_connections 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.776+0000 7f6de9080700 1 --2- 172.21.15.5:0/1690861100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de415e760 0x7f6de406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.776+0000 7f6de9080700 1 --2- 172.21.15.5:0/1690861100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40cbf50 0x7f6de4161d00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.776+0000 7f6de9080700 1 -- 172.21.15.5:0/1690861100 >> 172.21.15.5:0/1690861100 conn(0x7f6de4099a50 msgr2=0x7f6de40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.776+0000 7f6de9080700 1 -- 172.21.15.5:0/1690861100 shutdown_connections 2024-05-08T23:21:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.776+0000 7f6de9080700 1 -- 172.21.15.5:0/1690861100 wait complete. 2024-05-08T23:21:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.778+0000 7f6de9080700 1 Processor -- start 2024-05-08T23:21:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.778+0000 7f6de9080700 1 -- start start 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de9080700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de40cbf50 0x7f6de415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de9080700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de4162240 0x7f6de4158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de9080700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40b2020 0x7f6de409ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de9080700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de415c340 con 0x7f6de40b2020 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de9080700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40b54e0 con 0x7f6de40cbf50 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de9080700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40b57c0 con 0x7f6de4162240 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40b2020 0x7f6de409ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40b2020 0x7f6de409ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36608/0 (socket says 172.21.15.5:36608) 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de359e700 1 -- 172.21.15.5:0/970878211 learned_addr learned my addr 172.21.15.5:0/970878211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de259c700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de4162240 0x7f6de4158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.779+0000 7f6de2d9d700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de40cbf50 0x7f6de415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6de359e700 1 -- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de4162240 msgr2=0x7f6de4158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6de359e700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de4162240 0x7f6de4158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6de359e700 1 -- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de40cbf50 msgr2=0x7f6de415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6de359e700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de40cbf50 0x7f6de415bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6de359e700 1 -- 172.21.15.5:0/970878211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd40094a0 con 0x7f6de40b2020 2024-05-08T23:21:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6de359e700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40b2020 0x7f6de409ae00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00ef90 tx=0x7f6ddc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6dcb7fe700 1 -- 172.21.15.5:0/970878211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00cc90 con 0x7f6de40b2020 2024-05-08T23:21:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6dcb7fe700 1 -- 172.21.15.5:0/970878211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00f040 con 0x7f6de40b2020 2024-05-08T23:21:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de40b1820 con 0x7f6de40b2020 2024-05-08T23:21:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6dcb7fe700 1 -- 172.21.15.5:0/970878211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc014480 con 0x7f6de40b2020 2024-05-08T23:21:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.780+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de4159370 con 0x7f6de40b2020 2024-05-08T23:21:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.781+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de40afb90 con 0x7f6de40b2020 2024-05-08T23:21:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.782+0000 7f6dcb7fe700 1 -- 172.21.15.5:0/970878211 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6ddc0145e0 con 0x7f6de40b2020 2024-05-08T23:21:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.782+0000 7f6dcb7fe700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dd006ac90 0x7f6dd006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.782+0000 7f6dcb7fe700 1 -- 172.21.15.5:0/970878211 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6ddc089b00 con 0x7f6de40b2020 2024-05-08T23:21:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.782+0000 7f6de2d9d700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dd006ac90 0x7f6dd006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.783+0000 7f6de2d9d700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dd006ac90 0x7f6dd006d150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40090b0 tx=0x7f6dd4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:01.787+0000 7f6dcb7fe700 1 -- 172.21.15.5:0/970878211 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6ddc055030 con 0x7f6de40b2020 2024-05-08T23:21:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.003+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6de4074bc0 con 0x7f6de40b2020 2024-05-08T23:21:02.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.619+0000 7f6dcb7fe700 1 -- 172.21.15.5:0/970878211 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6ddc018020 con 0x7f6de40b2020 2024-05-08T23:21:02.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dd006ac90 msgr2=0x7f6dd006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dd006ac90 0x7f6dd006d150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40090b0 tx=0x7f6dd4008030 comp rx=0 tx=0).stop 2024-05-08T23:21:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40b2020 msgr2=0x7f6de409ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40b2020 0x7f6de409ae00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00ef90 tx=0x7f6ddc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 shutdown_connections 2024-05-08T23:21:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dd006ac90 0x7f6dd006d150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de40b2020 0x7f6de409ae00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de40cbf50 0x7f6de415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 --2- 172.21.15.5:0/970878211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de4162240 0x7f6de4158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 >> 172.21.15.5:0/970878211 conn(0x7f6de4099a50 msgr2=0x7f6de40ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.626+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 shutdown_connections 2024-05-08T23:21:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.627+0000 7f6de9080700 1 -- 172.21.15.5:0/970878211 wait complete. 2024-05-08T23:21:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-08T23:21:02.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:21:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.791+0000 7f6185cb8700 1 Processor -- start 2024-05-08T23:21:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.791+0000 7f6185cb8700 1 -- start start 2024-05-08T23:21:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f6185cb8700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61800d0f10 0x7f618015e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f6185cb8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800af5a0 0x7f618015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f6185cb8700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61800cf970 con 0x7f61800cbf50 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f6185cb8700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61800cfc50 con 0x7f61800d0f10 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f6185cb8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61800cff30 con 0x7f61800af5a0 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617f7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f61800cbf50 0x7f6180161d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56216/0 (socket says 172.21.15.5:56216) 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617f7fe700 1 -- 172.21.15.5:0/1000709085 learned_addr learned my addr 172.21.15.5:0/1000709085 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617effd700 1 --2- 172.21.15.5:0/1000709085 >> v2:172.21.15.49:3300/0 conn(0x7f61800d0f10 0x7f618015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617ffff700 1 --2- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800af5a0 0x7f618015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617e7fc700 1 -- 172.21.15.5:0/1000709085 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399413527 0 0) 0x7f61800cf970 con 0x7f61800cbf50 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617e7fc700 1 -- 172.21.15.5:0/1000709085 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61680035b0 con 0x7f61800cbf50 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617effd700 1 -- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800af5a0 msgr2=0x7f618015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:02.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617effd700 1 --2- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800af5a0 0x7f618015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617effd700 1 -- 172.21.15.5:0/1000709085 >> v1:172.21.15.5:6789/0 conn(0x7f61800cbf50 legacy=0x7f6180161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617effd700 1 -- 172.21.15.5:0/1000709085 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61801617e0 con 0x7f61800d0f10 2024-05-08T23:21:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.792+0000 7f617ffff700 1 --2- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800af5a0 0x7f618015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f617effd700 1 --2- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800d0f10 0x7f618015e050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6174009840 tx=0x7f6174009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78b9811152ad07a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f617e7fc700 1 -- 172.21.15.5:0/1000709085 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174011830 con 0x7f61800d0f10 2024-05-08T23:21:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f617e7fc700 1 -- 172.21.15.5:0/1000709085 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174011990 con 0x7f61800d0f10 2024-05-08T23:21:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f617e7fc700 1 -- 172.21.15.5:0/1000709085 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174011d20 con 0x7f61800d0f10 2024-05-08T23:21:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f6185cb8700 1 -- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800d0f10 msgr2=0x7f618015e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f6185cb8700 1 --2- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800d0f10 0x7f618015e050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6174009840 tx=0x7f6174009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f6185cb8700 1 -- 172.21.15.5:0/1000709085 shutdown_connections 2024-05-08T23:21:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f6185cb8700 1 --2- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800af5a0 0x7f618015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f6185cb8700 1 --2- 172.21.15.5:0/1000709085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800d0f10 0x7f618015e050 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f6185cb8700 1 -- 172.21.15.5:0/1000709085 >> 172.21.15.5:0/1000709085 conn(0x7f6180099a50 msgr2=0x7f61800d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f6185cb8700 1 -- 172.21.15.5:0/1000709085 shutdown_connections 2024-05-08T23:21:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.793+0000 7f6185cb8700 1 -- 172.21.15.5:0/1000709085 wait complete. 2024-05-08T23:21:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.795+0000 7f6185cb8700 1 Processor -- start 2024-05-08T23:21:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.795+0000 7f6185cb8700 1 -- start start 2024-05-08T23:21:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f6185cb8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61800af5a0 0x7f618015b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f6185cb8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800cbf50 0x7f618009bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f6185cb8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800d0f10 0x7f6180160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f6185cb8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618014b400 con 0x7f61800af5a0 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f6185cb8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618014b6e0 con 0x7f61800cbf50 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f6185cb8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618014b9c0 con 0x7f61800d0f10 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61800af5a0 0x7f618015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61800af5a0 0x7f618015b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36618/0 (socket says 172.21.15.5:36618) 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617f7fe700 1 -- 172.21.15.5:0/2230953488 learned_addr learned my addr 172.21.15.5:0/2230953488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617effd700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800cbf50 0x7f618009bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617ffff700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800d0f10 0x7f6180160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617f7fe700 1 -- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800d0f10 msgr2=0x7f6180160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617f7fe700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800d0f10 0x7f6180160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617f7fe700 1 -- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800cbf50 msgr2=0x7f618009bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617f7fe700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800cbf50 0x7f618009bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.796+0000 7f617f7fe700 1 -- 172.21.15.5:0/2230953488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178009a70 con 0x7f61800af5a0 2024-05-08T23:21:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.797+0000 7f617f7fe700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61800af5a0 0x7f618015b150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f617000c700 tx=0x7f617000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.797+0000 7f6167fff700 1 -- 172.21.15.5:0/2230953488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170005d70 con 0x7f61800af5a0 2024-05-08T23:21:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.797+0000 7f6167fff700 1 -- 172.21.15.5:0/2230953488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61700070d0 con 0x7f61800af5a0 2024-05-08T23:21:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.797+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61740094a0 con 0x7f61800af5a0 2024-05-08T23:21:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.797+0000 7f6167fff700 1 -- 172.21.15.5:0/2230953488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170007460 con 0x7f61800af5a0 2024-05-08T23:21:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.797+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618014bf80 con 0x7f61800af5a0 2024-05-08T23:21:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.798+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6180074bc0 con 0x7f61800af5a0 2024-05-08T23:21:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.799+0000 7f6167fff700 1 -- 172.21.15.5:0/2230953488 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f61700075c0 con 0x7f61800af5a0 2024-05-08T23:21:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.799+0000 7f6167fff700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f616c06ac90 0x7f616c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.799+0000 7f6167fff700 1 -- 172.21.15.5:0/2230953488 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6170088560 con 0x7f61800af5a0 2024-05-08T23:21:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.799+0000 7f617effd700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f616c06ac90 0x7f616c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.800+0000 7f617effd700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f616c06ac90 0x7f616c06d150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6174000c00 tx=0x7f6174008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:02.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:02.804+0000 7f6167fff700 1 -- 172.21.15.5:0/2230953488 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f61700539b0 con 0x7f61800af5a0 2024-05-08T23:21:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.022+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f61800b62b0 con 0x7f61800af5a0 2024-05-08T23:21:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.625+0000 7f6167fff700 1 -- 172.21.15.5:0/2230953488 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6170056fd0 con 0x7f61800af5a0 2024-05-08T23:21:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.633+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f616c06ac90 msgr2=0x7f616c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.633+0000 7f6185cb8700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f616c06ac90 0x7f616c06d150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6174000c00 tx=0x7f6174008030 comp rx=0 tx=0).stop 2024-05-08T23:21:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.633+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61800af5a0 msgr2=0x7f618015b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.633+0000 7f6185cb8700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61800af5a0 0x7f618015b150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f617000c700 tx=0x7f617000ca10 comp rx=0 tx=0).stop 2024-05-08T23:21:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.634+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 shutdown_connections 2024-05-08T23:21:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.634+0000 7f6185cb8700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f616c06ac90 0x7f616c06d150 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6174000c00 tx=0x7f6174008030 comp rx=0 tx=0).stop 2024-05-08T23:21:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.634+0000 7f6185cb8700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f61800d0f10 0x7f6180160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.634+0000 7f6185cb8700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61800af5a0 0x7f618015b150 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.634+0000 7f6185cb8700 1 --2- 172.21.15.5:0/2230953488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61800cbf50 0x7f618009bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.634+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 >> 172.21.15.5:0/2230953488 conn(0x7f6180099a50 msgr2=0x7f618006acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.635+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 shutdown_connections 2024-05-08T23:21:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.636+0000 7f6185cb8700 1 -- 172.21.15.5:0/2230953488 wait complete. 2024-05-08T23:21:03.646 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-08T23:21:03.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:21:03.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.803+0000 7fcbd0c4a700 1 Processor -- start 2024-05-08T23:21:03.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.803+0000 7fcbd0c4a700 1 -- start start 2024-05-08T23:21:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbd0c4a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcbcc0cbf50 0x7fcbcc161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbd0c4a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcbcc162240 0x7fcbcc15e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbd0c4a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc0d0f10 0x7fcbcc15e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbd0c4a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0cf970 con 0x7fcbcc162240 2024-05-08T23:21:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbd0c4a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0cfc50 con 0x7fcbcc0cbf50 2024-05-08T23:21:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbd0c4a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0cff30 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbc9d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcbcc162240 0x7fcbcc15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbc9d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcbcc162240 0x7fcbcc15e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36628/0 (socket says 172.21.15.5:36628) 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbc9d9b700 1 -- 172.21.15.5:0/1890328621 learned_addr learned my addr 172.21.15.5:0/1890328621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbca59c700 1 --2- 172.21.15.5:0/1890328621 >> v2:172.21.15.49:3300/0 conn(0x7fcbcc0cbf50 0x7fcbcc161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbcad9d700 1 --2- 172.21.15.5:0/1890328621 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc0d0f10 0x7fcbcc15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbc9d9b700 1 -- 172.21.15.5:0/1890328621 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc0d0f10 msgr2=0x7fcbcc15e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbc9d9b700 1 --2- 172.21.15.5:0/1890328621 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc0d0f10 0x7fcbcc15e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbc9d9b700 1 -- 172.21.15.5:0/1890328621 >> v2:172.21.15.49:3300/0 conn(0x7fcbcc0cbf50 msgr2=0x7fcbcc161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbc9d9b700 1 --2- 172.21.15.5:0/1890328621 >> v2:172.21.15.49:3300/0 conn(0x7fcbcc0cbf50 0x7fcbcc161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.804+0000 7fcbc9d9b700 1 -- 172.21.15.5:0/1890328621 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc1617e0 con 0x7fcbcc162240 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbc9d9b700 1 --2- 172.21.15.5:0/1890328621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc162240 0x7fcbcc15e050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0718f0 tx=0x7fcbc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26202a877b104c13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbc959a700 1 -- 172.21.15.5:0/1890328621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0011830 con 0x7fcbcc162240 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbc959a700 1 -- 172.21.15.5:0/1890328621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc0011990 con 0x7fcbcc162240 2024-05-08T23:21:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbc959a700 1 -- 172.21.15.5:0/1890328621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0011d20 con 0x7fcbcc162240 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/1890328621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc162240 msgr2=0x7fcbcc15e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/1890328621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc162240 0x7fcbcc15e050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0718f0 tx=0x7fcbc0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/1890328621 shutdown_connections 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/1890328621 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc0d0f10 0x7fcbcc15e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/1890328621 >> v2:172.21.15.49:3300/0 conn(0x7fcbcc0cbf50 0x7fcbcc161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/1890328621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc162240 0x7fcbcc15e050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.805+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/1890328621 >> 172.21.15.5:0/1890328621 conn(0x7fcbcc099a50 msgr2=0x7fcbcc0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.806+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/1890328621 shutdown_connections 2024-05-08T23:21:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.806+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/1890328621 wait complete. 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.807+0000 7fcbd0c4a700 1 Processor -- start 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.807+0000 7fcbd0c4a700 1 -- start start 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbd0c4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbcc0cbf50 0x7fcbcc1e9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbd0c4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc0d0f10 0x7fcbcc158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbd0c4a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc162240 0x7fcbcc1e7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbd0c4a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0b3520 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbd0c4a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc1599a0 con 0x7fcbcc0cbf50 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbd0c4a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0b2ef0 con 0x7fcbcc162240 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbc9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc0d0f10 0x7fcbcc158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbc9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc0d0f10 0x7fcbcc158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36642/0 (socket says 172.21.15.5:36642) 2024-05-08T23:21:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbc9d9b700 1 -- 172.21.15.5:0/839919566 learned_addr learned my addr 172.21.15.5:0/839919566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbca59c700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbcc0cbf50 0x7fcbcc1e9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbcad9d700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc162240 0x7fcbcc1e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbc9d9b700 1 -- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc162240 msgr2=0x7fcbcc1e7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbc9d9b700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc162240 0x7fcbcc1e7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbc9d9b700 1 -- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbcc0cbf50 msgr2=0x7fcbcc1e9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbc9d9b700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbcc0cbf50 0x7fcbcc1e9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbc9d9b700 1 -- 172.21.15.5:0/839919566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc0009550 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.808+0000 7fcbca59c700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbcc0cbf50 0x7fcbcc1e9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.809+0000 7fcbc9d9b700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc0d0f10 0x7fcbcc158ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009c10 tx=0x7fcbc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.809+0000 7fcbbaffd700 1 -- 172.21.15.5:0/839919566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0004510 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.809+0000 7fcbbaffd700 1 -- 172.21.15.5:0/839919566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc00036a0 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.809+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc0b1d40 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.809+0000 7fcbbaffd700 1 -- 172.21.15.5:0/839919566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0003a30 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.809+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc0b3140 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.810+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc0b3660 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.811+0000 7fcbbaffd700 1 -- 172.21.15.5:0/839919566 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcbc0003b90 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.811+0000 7fcbbaffd700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbac06ac90 0x7fcbac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.811+0000 7fcbbaffd700 1 -- 172.21.15.5:0/839919566 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fcbc008a3b0 con 0x7fcbcc0d0f10 2024-05-08T23:21:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.811+0000 7fcbca59c700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbac06ac90 0x7fcbac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.812+0000 7fcbca59c700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbac06ac90 0x7fcbac06d150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc1e9d20 tx=0x7fcbbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:03.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:03.816+0000 7fcbbaffd700 1 -- 172.21.15.5:0/839919566 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcbcc0b3660 con 0x7fcbcc0d0f10 2024-05-08T23:21:04.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.085+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcbcc0b3660 con 0x7fcbcc0d0f10 2024-05-08T23:21:04.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.090+0000 7fcbbaffd700 1 -- 172.21.15.5:0/839919566 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcbcc0b3660 con 0x7fcbcc0d0f10 2024-05-08T23:21:04.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbac06ac90 msgr2=0x7fcbac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbac06ac90 0x7fcbac06d150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc1e9d20 tx=0x7fcbbc008040 comp rx=0 tx=0).stop 2024-05-08T23:21:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc0d0f10 msgr2=0x7fcbcc158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc0d0f10 0x7fcbcc158ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009c10 tx=0x7fcbc0004750 comp rx=0 tx=0).stop 2024-05-08T23:21:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 shutdown_connections 2024-05-08T23:21:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbac06ac90 0x7fcbac06d150 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcbcc162240 0x7fcbcc1e7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbcc0cbf50 0x7fcbcc1e9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 --2- 172.21.15.5:0/839919566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbcc0d0f10 0x7fcbcc158ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.107+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 >> 172.21.15.5:0/839919566 conn(0x7fcbcc099a50 msgr2=0x7fcbcc1609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.108+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 shutdown_connections 2024-05-08T23:21:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.108+0000 7fcbd0c4a700 1 -- 172.21.15.5:0/839919566 wait complete. 2024-05-08T23:21:04.111 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T23:21:04.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.263+0000 7f601bcc2700 1 Processor -- start 2024-05-08T23:21:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.263+0000 7f601bcc2700 1 -- start start 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.264+0000 7f601bcc2700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f60140ceb00 0x7f60140cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.264+0000 7f601bcc2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140b3d30 0x7f60140b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.264+0000 7f601bcc2700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60141622e0 con 0x7f601406b970 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.264+0000 7f601bcc2700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6014162f60 con 0x7f60140ceb00 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.264+0000 7f601bcc2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60141625c0 con 0x7f60140b3d30 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.264+0000 7f601925d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f60140ceb00 0x7f60140cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.264+0000 7f601925d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f60140ceb00 0x7f60140cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36650/0 (socket says 172.21.15.5:36650) 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.264+0000 7f601925d700 1 -- 172.21.15.5:0/1984988802 learned_addr learned my addr 172.21.15.5:0/1984988802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f601a25f700 1 --2- 172.21.15.5:0/1984988802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140b3d30 0x7f60140b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f601925d700 1 -- 172.21.15.5:0/1984988802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140b3d30 msgr2=0x7f60140b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:04.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f601925d700 1 --2- 172.21.15.5:0/1984988802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140b3d30 0x7f60140b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f601925d700 1 -- 172.21.15.5:0/1984988802 >> v1:172.21.15.49:6789/0 conn(0x7f601406b970 legacy=0x7f601406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f601925d700 1 -- 172.21.15.5:0/1984988802 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60141628a0 con 0x7f60140ceb00 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f601925d700 1 --2- 172.21.15.5:0/1984988802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60140ceb00 0x7f60140cf2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60040098f0 tx=0x7f6004009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77299650ba2c5bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f6018a5c700 1 -- 172.21.15.5:0/1984988802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400b8a0 con 0x7f60140ceb00 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f6018a5c700 1 -- 172.21.15.5:0/1984988802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600400ba00 con 0x7f60140ceb00 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.265+0000 7f6018a5c700 1 -- 172.21.15.5:0/1984988802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400bd90 con 0x7f60140ceb00 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.266+0000 7f601bcc2700 1 -- 172.21.15.5:0/1984988802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60140ceb00 msgr2=0x7f60140cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.266+0000 7f601bcc2700 1 --2- 172.21.15.5:0/1984988802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60140ceb00 0x7f60140cf2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60040098f0 tx=0x7f6004009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.266+0000 7f601bcc2700 1 -- 172.21.15.5:0/1984988802 shutdown_connections 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.266+0000 7f601bcc2700 1 --2- 172.21.15.5:0/1984988802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140b3d30 0x7f60140b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.266+0000 7f601bcc2700 1 --2- 172.21.15.5:0/1984988802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60140ceb00 0x7f60140cf2d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.266+0000 7f601bcc2700 1 -- 172.21.15.5:0/1984988802 >> 172.21.15.5:0/1984988802 conn(0x7f60140713c0 msgr2=0x7f60140717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.266+0000 7f601bcc2700 1 -- 172.21.15.5:0/1984988802 shutdown_connections 2024-05-08T23:21:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.266+0000 7f601bcc2700 1 -- 172.21.15.5:0/1984988802 wait complete. 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.267+0000 7f601bcc2700 1 Processor -- start 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.267+0000 7f601bcc2700 1 -- start start 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f601bcc2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601406b970 0x7f60141429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f601bcc2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60140b3d30 0x7f6014154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f601bcc2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140ceb00 0x7f60141fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f601bcc2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6014142ee0 con 0x7f601406b970 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f601bcc2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6014154a60 con 0x7f60140b3d30 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f601bcc2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6014155430 con 0x7f60140ceb00 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f6019a5e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601406b970 0x7f60141429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f6019a5e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601406b970 0x7f60141429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36652/0 (socket says 172.21.15.5:36652) 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f6019a5e700 1 -- 172.21.15.5:0/2050123580 learned_addr learned my addr 172.21.15.5:0/2050123580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f601a25f700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140ceb00 0x7f60141fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f601925d700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60140b3d30 0x7f6014154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f6019a5e700 1 -- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140ceb00 msgr2=0x7f60141fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f6019a5e700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140ceb00 0x7f60141fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f6019a5e700 1 -- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60140b3d30 msgr2=0x7f6014154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f6019a5e700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60140b3d30 0x7f6014154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.268+0000 7f6019a5e700 1 -- 172.21.15.5:0/2050123580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6004009550 con 0x7f601406b970 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.269+0000 7f6019a5e700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601406b970 0x7f60141429a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f601000c4e0 tx=0x7f601000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.269+0000 7f600a7fc700 1 -- 172.21.15.5:0/2050123580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60100052d0 con 0x7f601406b970 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.269+0000 7f600a7fc700 1 -- 172.21.15.5:0/2050123580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6010005430 con 0x7f601406b970 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.269+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6014153e50 con 0x7f601406b970 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.269+0000 7f600a7fc700 1 -- 172.21.15.5:0/2050123580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60100038c0 con 0x7f601406b970 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.269+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6014154410 con 0x7f601406b970 2024-05-08T23:21:04.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.270+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6014067a80 con 0x7f601406b970 2024-05-08T23:21:04.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.270+0000 7f600a7fc700 1 -- 172.21.15.5:0/2050123580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6010003a20 con 0x7f601406b970 2024-05-08T23:21:04.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.271+0000 7f600a7fc700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ff806ac90 0x7f5ff806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.271+0000 7f600a7fc700 1 -- 172.21.15.5:0/2050123580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60100887e0 con 0x7f601406b970 2024-05-08T23:21:04.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.271+0000 7f601925d700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ff806ac90 0x7f5ff806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.272+0000 7f601925d700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ff806ac90 0x7f5ff806d150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60140cef40 tx=0x7f6004008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:04.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.276+0000 7f600a7fc700 1 -- 172.21.15.5:0/2050123580 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6010053a50 con 0x7f601406b970 2024-05-08T23:21:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.495+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6014068000 con 0x7f601406b970 2024-05-08T23:21:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.496+0000 7f600a7fc700 1 -- 172.21.15.5:0/2050123580 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15954 (secure 0 0 0) 0x7f6010017090 con 0x7f601406b970 2024-05-08T23:21:04.496 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:04.496 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":19,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:04.087036+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ff806ac90 msgr2=0x7f5ff806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ff806ac90 0x7f5ff806d150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60140cef40 tx=0x7f6004008030 comp rx=0 tx=0).stop 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601406b970 msgr2=0x7f60141429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601406b970 0x7f60141429a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f601000c4e0 tx=0x7f601000c8a0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 shutdown_connections 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ff806ac90 0x7f5ff806d150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60140ceb00 0x7f60141fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601406b970 0x7f60141429a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 --2- 172.21.15.5:0/2050123580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60140b3d30 0x7f6014154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.498+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 >> 172.21.15.5:0/2050123580 conn(0x7f60140713c0 msgr2=0x7f60140b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.499+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 shutdown_connections 2024-05-08T23:21:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.499+0000 7f601bcc2700 1 -- 172.21.15.5:0/2050123580 wait complete. 2024-05-08T23:21:04.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.649+0000 7f853ae5e700 1 Processor -- start 2024-05-08T23:21:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.649+0000 7f853ae5e700 1 -- start start 2024-05-08T23:21:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.650+0000 7f853ae5e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85340d3f40 0x7f8534161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.650+0000 7f853ae5e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f853415db80 0x7f85340cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.650+0000 7f853ae5e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8534162450 con 0x7f853415d7a0 2024-05-08T23:21:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.650+0000 7f853ae5e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340cee00 con 0x7f85340d3f40 2024-05-08T23:21:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.650+0000 7f853ae5e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340b27e0 con 0x7f853415db80 2024-05-08T23:21:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.650+0000 7f8533fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85340d3f40 0x7f8534161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.650+0000 7f8533fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85340d3f40 0x7f8534161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36666/0 (socket says 172.21.15.5:36666) 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.650+0000 7f8533fff700 1 -- 172.21.15.5:0/3463869189 learned_addr learned my addr 172.21.15.5:0/3463869189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f85393fb700 1 --2- 172.21.15.5:0/3463869189 >> v2:172.21.15.49:3300/0 conn(0x7f853415db80 0x7f85340cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f8533fff700 1 -- 172.21.15.5:0/3463869189 >> v2:172.21.15.49:3300/0 conn(0x7f853415db80 msgr2=0x7f85340cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f8533fff700 1 --2- 172.21.15.5:0/3463869189 >> v2:172.21.15.49:3300/0 conn(0x7f853415db80 0x7f85340cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f8533fff700 1 -- 172.21.15.5:0/3463869189 >> v1:172.21.15.49:6789/0 conn(0x7f853415d7a0 legacy=0x7f85340b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f8533fff700 1 -- 172.21.15.5:0/3463869189 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340d4400 con 0x7f85340d3f40 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f8533fff700 1 --2- 172.21.15.5:0/3463869189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f8534161f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f853408b9e0 tx=0x7f8528009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bb5d0572c4970 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f85337fe700 1 -- 172.21.15.5:0/3463869189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528005e30 con 0x7f85340d3f40 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f85337fe700 1 -- 172.21.15.5:0/3463869189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528004030 con 0x7f85340d3f40 2024-05-08T23:21:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.651+0000 7f85337fe700 1 -- 172.21.15.5:0/3463869189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280043e0 con 0x7f85340d3f40 2024-05-08T23:21:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.652+0000 7f853ae5e700 1 -- 172.21.15.5:0/3463869189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 msgr2=0x7f8534161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.652+0000 7f853ae5e700 1 --2- 172.21.15.5:0/3463869189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f8534161f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f853408b9e0 tx=0x7f8528009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.652+0000 7f853ae5e700 1 -- 172.21.15.5:0/3463869189 shutdown_connections 2024-05-08T23:21:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.652+0000 7f853ae5e700 1 --2- 172.21.15.5:0/3463869189 >> v2:172.21.15.49:3300/0 conn(0x7f853415db80 0x7f85340cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.652+0000 7f853ae5e700 1 --2- 172.21.15.5:0/3463869189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f8534161f10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.652+0000 7f853ae5e700 1 -- 172.21.15.5:0/3463869189 >> 172.21.15.5:0/3463869189 conn(0x7f853408afe0 msgr2=0x7f85340d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.652+0000 7f853ae5e700 1 -- 172.21.15.5:0/3463869189 shutdown_connections 2024-05-08T23:21:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.652+0000 7f853ae5e700 1 -- 172.21.15.5:0/3463869189 wait complete. 2024-05-08T23:21:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.653+0000 7f853ae5e700 1 Processor -- start 2024-05-08T23:21:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.653+0000 7f853ae5e700 1 -- start start 2024-05-08T23:21:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f853ae5e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f853415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f853ae5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853415d7a0 0x7f85340c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f853ae5e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f853415db80 0x7f85340cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f853ae5e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340c77b0 con 0x7f85340d3f40 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f853ae5e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340c7a90 con 0x7f853415d7a0 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f853ae5e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85340c7d70 con 0x7f853415db80 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f8538bfa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f853415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f8538bfa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f853415a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36672/0 (socket says 172.21.15.5:36672) 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f8538bfa700 1 -- 172.21.15.5:0/2805808061 learned_addr learned my addr 172.21.15.5:0/2805808061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f8533fff700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853415d7a0 0x7f85340c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.654+0000 7f85393fb700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f853415db80 0x7f85340cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f8538bfa700 1 -- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f853415db80 msgr2=0x7f85340cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f8538bfa700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f853415db80 0x7f85340cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f8538bfa700 1 -- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853415d7a0 msgr2=0x7f85340c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f8538bfa700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853415d7a0 0x7f85340c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f8538bfa700 1 -- 172.21.15.5:0/2805808061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8528009500 con 0x7f85340d3f40 2024-05-08T23:21:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f8533fff700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853415d7a0 0x7f85340c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f8538bfa700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f853415a070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f852400c4e0 tx=0x7f852400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f85317fa700 1 -- 172.21.15.5:0/2805808061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240052d0 con 0x7f85340d3f40 2024-05-08T23:21:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f85317fa700 1 -- 172.21.15.5:0/2805808061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8524005430 con 0x7f85340d3f40 2024-05-08T23:21:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85340c8050 con 0x7f85340d3f40 2024-05-08T23:21:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f85317fa700 1 -- 172.21.15.5:0/2805808061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240038c0 con 0x7f85340d3f40 2024-05-08T23:21:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.655+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853415a5b0 con 0x7f85340d3f40 2024-05-08T23:21:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.656+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85340cb970 con 0x7f85340d3f40 2024-05-08T23:21:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.657+0000 7f85317fa700 1 -- 172.21.15.5:0/2805808061 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8524003a20 con 0x7f85340d3f40 2024-05-08T23:21:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.657+0000 7f85317fa700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f852006ac90 0x7f852006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.657+0000 7f85317fa700 1 -- 172.21.15.5:0/2805808061 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8524088670 con 0x7f85340d3f40 2024-05-08T23:21:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.657+0000 7f8533fff700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f852006ac90 0x7f852006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.658+0000 7f8533fff700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f852006ac90 0x7f852006d150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f853408b9e0 tx=0x7f8528008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.662+0000 7f85317fa700 1 -- 172.21.15.5:0/2805808061 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8524053850 con 0x7f85340d3f40 2024-05-08T23:21:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.877+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8534162450 con 0x7f85340d3f40 2024-05-08T23:21:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.878+0000 7f85317fa700 1 -- 172.21.15.5:0/2805808061 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15954 (secure 0 0 0) 0x7f8534162450 con 0x7f85340d3f40 2024-05-08T23:21:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:04.879 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":19,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:04.087036+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f852006ac90 msgr2=0x7f852006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f852006ac90 0x7f852006d150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f853408b9e0 tx=0x7f8528008070 comp rx=0 tx=0).stop 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 msgr2=0x7f853415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f853415a070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f852400c4e0 tx=0x7f852400c8a0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 shutdown_connections 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f852006ac90 0x7f852006d150 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f853415db80 0x7f85340cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85340d3f40 0x7f853415a070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 --2- 172.21.15.5:0/2805808061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853415d7a0 0x7f85340c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:04.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 >> 172.21.15.5:0/2805808061 conn(0x7f853408afe0 msgr2=0x7f853415cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:04.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 shutdown_connections 2024-05-08T23:21:04.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:04.881+0000 7f853ae5e700 1 -- 172.21.15.5:0/2805808061 wait complete. 2024-05-08T23:21:04.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:21:05.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.033+0000 7fbeb77ce700 1 Processor -- start 2024-05-08T23:21:05.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.033+0000 7fbeb77ce700 1 -- start start 2024-05-08T23:21:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb77ce700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0161a90 0x7fbeb0162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb77ce700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbeb00b39e0 0x7fbeb006d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb77ce700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb00b5810 con 0x7fbeb00b3e70 2024-05-08T23:21:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb77ce700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb0163150 con 0x7fbeb00b39e0 2024-05-08T23:21:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb77ce700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00b4c00 con 0x7fbeb0161a90 2024-05-08T23:21:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb5d6b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbeb00b39e0 0x7fbeb006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb4d69700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbeb00b3e70 0x7fbeb00b32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56230/0 (socket says 172.21.15.5:56230) 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb4d69700 1 -- 172.21.15.5:0/820706372 learned_addr learned my addr 172.21.15.5:0/820706372 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.034+0000 7fbeb556a700 1 --2- 172.21.15.5:0/820706372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0161a90 0x7fbeb0162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbeb5d6b700 1 -- 172.21.15.5:0/820706372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0161a90 msgr2=0x7fbeb0162c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbeb5d6b700 1 --2- 172.21.15.5:0/820706372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0161a90 0x7fbeb0162c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbeb5d6b700 1 -- 172.21.15.5:0/820706372 >> v1:172.21.15.5:6789/0 conn(0x7fbeb00b3e70 legacy=0x7fbeb00b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbeb5d6b700 1 -- 172.21.15.5:0/820706372 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb00b4ee0 con 0x7fbeb00b39e0 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbea7fff700 1 -- 172.21.15.5:0/820706372 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887323977 0 0) 0x7fbeb00b5810 con 0x7fbeb00b3e70 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbeb5d6b700 1 --2- 172.21.15.5:0/820706372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b39e0 0x7fbeb006d540 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009d90 tx=0x7fbea800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a72cb46e35bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbea7fff700 1 -- 172.21.15.5:0/820706372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800e070 con 0x7fbeb00b39e0 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbea7fff700 1 -- 172.21.15.5:0/820706372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea800bb20 con 0x7fbeb00b39e0 2024-05-08T23:21:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.035+0000 7fbea7fff700 1 -- 172.21.15.5:0/820706372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8012430 con 0x7fbeb00b39e0 2024-05-08T23:21:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.036+0000 7fbeb77ce700 1 -- 172.21.15.5:0/820706372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b39e0 msgr2=0x7fbeb006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.036+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/820706372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b39e0 0x7fbeb006d540 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009d90 tx=0x7fbea800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.036+0000 7fbeb77ce700 1 -- 172.21.15.5:0/820706372 shutdown_connections 2024-05-08T23:21:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.036+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/820706372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b39e0 0x7fbeb006d540 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.036+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/820706372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0161a90 0x7fbeb0162c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.036+0000 7fbeb77ce700 1 -- 172.21.15.5:0/820706372 >> 172.21.15.5:0/820706372 conn(0x7fbeb00722d0 msgr2=0x7fbeb006a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.036+0000 7fbeb77ce700 1 -- 172.21.15.5:0/820706372 shutdown_connections 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.036+0000 7fbeb77ce700 1 -- 172.21.15.5:0/820706372 wait complete. 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.037+0000 7fbeb77ce700 1 Processor -- start 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.037+0000 7fbeb77ce700 1 -- start start 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb77ce700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb00b39e0 0x7fbeb00a8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb77ce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b3e70 0x7fbeb01faaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb77ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb0161a90 0x7fbeb01fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb77ce700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb014c980 con 0x7fbeb00b3e70 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb77ce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb014cc60 con 0x7fbeb0161a90 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb77ce700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb014cf40 con 0x7fbeb00b39e0 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b3e70 0x7fbeb01faaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b3e70 0x7fbeb01faaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36690/0 (socket says 172.21.15.5:36690) 2024-05-08T23:21:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 -- 172.21.15.5:0/2285142271 learned_addr learned my addr 172.21.15.5:0/2285142271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb5d6b700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb0161a90 0x7fbeb01fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb556a700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb00b39e0 0x7fbeb00a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 -- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb00b39e0 msgr2=0x7fbeb00a8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb00b39e0 0x7fbeb00a8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 -- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb0161a90 msgr2=0x7fbeb01fafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb0161a90 0x7fbeb01fafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 -- 172.21.15.5:0/2285142271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8009a70 con 0x7fbeb00b3e70 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb556a700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb00b39e0 0x7fbeb00a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb4d69700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b3e70 0x7fbeb01faaa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbea00011e0 tx=0x7fbea000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbea5ffb700 1 -- 172.21.15.5:0/2285142271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0004c80 con 0x7fbeb00b3e70 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbea5ffb700 1 -- 172.21.15.5:0/2285142271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea0004de0 con 0x7fbeb00b3e70 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb014d220 con 0x7fbeb00b3e70 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbea5ffb700 1 -- 172.21.15.5:0/2285142271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0007310 con 0x7fbeb00b3e70 2024-05-08T23:21:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.039+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb014c5a0 con 0x7fbeb00b3e70 2024-05-08T23:21:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.040+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb014d740 con 0x7fbeb00b3e70 2024-05-08T23:21:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.040+0000 7fbea5ffb700 1 -- 172.21.15.5:0/2285142271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbea0007470 con 0x7fbeb00b3e70 2024-05-08T23:21:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.041+0000 7fbea5ffb700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9406ac90 0x7fbe9406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.041+0000 7fbea5ffb700 1 -- 172.21.15.5:0/2285142271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbea0089120 con 0x7fbeb00b3e70 2024-05-08T23:21:05.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.041+0000 7fbeb556a700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9406ac90 0x7fbe9406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.041+0000 7fbeb556a700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9406ac90 0x7fbe9406d150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009b90 tx=0x7fbeac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.045+0000 7fbea5ffb700 1 -- 172.21.15.5:0/2285142271 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbeb014d740 con 0x7fbeb00b3e70 2024-05-08T23:21:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:05.314+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbeb014d740 con 0x7fbeb00b3e70 2024-05-08T23:21:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.095+0000 7fbea5ffb700 1 -- 172.21.15.5:0/2285142271 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbeb014d740 con 0x7fbeb00b3e70 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.098+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9406ac90 msgr2=0x7fbe9406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.098+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9406ac90 0x7fbe9406d150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009b90 tx=0x7fbeac008040 comp rx=0 tx=0).stop 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.098+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b3e70 msgr2=0x7fbeb01faaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.098+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b3e70 0x7fbeb01faaa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbea00011e0 tx=0x7fbea000ca30 comp rx=0 tx=0).stop 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.099+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 shutdown_connections 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.099+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9406ac90 0x7fbe9406d150 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.099+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb0161a90 0x7fbeb01fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.099+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb00b39e0 0x7fbeb00a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.099+0000 7fbeb77ce700 1 --2- 172.21.15.5:0/2285142271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb00b3e70 0x7fbeb01faaa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.099+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 >> 172.21.15.5:0/2285142271 conn(0x7fbeb00722d0 msgr2=0x7fbeb0162860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.099+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 shutdown_connections 2024-05-08T23:21:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.099+0000 7fbeb77ce700 1 -- 172.21.15.5:0/2285142271 wait complete. 2024-05-08T23:21:06.113 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:21:06.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:21:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.256+0000 7fd2abc43700 1 Processor -- start 2024-05-08T23:21:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.257+0000 7fd2abc43700 1 -- start start 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.257+0000 7fd2abc43700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd2a415d900 0x7fd2a4161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.257+0000 7fd2abc43700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a415dce0 con 0x7fd2a406c080 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.257+0000 7fd2abc43700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a40ce5b0 con 0x7fd2a40cda40 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.257+0000 7fd2abc43700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a4161510 con 0x7fd2a415d900 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2aa1e0700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd2a406c080 0x7fd2a40ada00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56234/0 (socket says 172.21.15.5:56234) 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2aa1e0700 1 -- 172.21.15.5:0/4214894696 learned_addr learned my addr 172.21.15.5:0/4214894696 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a99df700 1 --2- 172.21.15.5:0/4214894696 >> v2:172.21.15.49:3300/0 conn(0x7fd2a415d900 0x7fd2a4161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695045413 0 0) 0x7fd2a415dce0 con 0x7fd2a406c080 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2900035b0 con 0x7fd2a406c080 2024-05-08T23:21:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3816088143 0 0) 0x7fd2900035b0 con 0x7fd2a406c080 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 >> v2:172.21.15.49:3300/0 conn(0x7fd2a415d900 msgr2=0x7fd2a4161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 --2- 172.21.15.5:0/4214894696 >> v2:172.21.15.49:3300/0 conn(0x7fd2a415d900 0x7fd2a4161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 >> v1:172.21.15.49:6789/0 conn(0x7fd2a40cda40 legacy=0x7fd2a406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a41617f0 con 0x7fd2a406c080 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fd29c0019f0 con 0x7fd2a406c080 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 >> v1:172.21.15.5:6789/0 conn(0x7fd2a406c080 legacy=0x7fd2a40ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd290003a00 0x7fd290003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd290004850 0x7fd290004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290005210 0x7fd29000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4161510 con 0x7fd290003a00 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29000c1b0 con 0x7fd290005210 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29000c370 con 0x7fd290004850 2024-05-08T23:21:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a99df700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd290003a00 0x7fd290003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2a91de700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd290004850 0x7fd290004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.258+0000 7fd2aa1e0700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290005210 0x7fd29000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a99df700 1 -- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd290004850 msgr2=0x7fd290004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a99df700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd290004850 0x7fd290004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a99df700 1 -- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290005210 msgr2=0x7fd29000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a99df700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290005210 0x7fd29000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a99df700 1 -- 172.21.15.5:0/4214894696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900035b0 con 0x7fd290003a00 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2aa1e0700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290005210 0x7fd29000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2aa1e0700 1 -- 172.21.15.5:0/4214894696 reap_dead start 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a99df700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd290003a00 0x7fd290003e20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b60 tx=0x7fd290005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e38d433b1050838 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000e6c0 con 0x7fd290003a00 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a000e820 con 0x7fd290003a00 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.259+0000 7fd2a89dd700 1 -- 172.21.15.5:0/4214894696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000eb90 con 0x7fd290003a00 2024-05-08T23:21:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.260+0000 7fd2abc43700 1 -- 172.21.15.5:0/4214894696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd290003a00 msgr2=0x7fd290003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.260+0000 7fd2abc43700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd290003a00 0x7fd290003e20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b60 tx=0x7fd290005880 comp rx=0 tx=0).stop 2024-05-08T23:21:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.260+0000 7fd2abc43700 1 -- 172.21.15.5:0/4214894696 shutdown_connections 2024-05-08T23:21:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.260+0000 7fd2abc43700 1 --2- 172.21.15.5:0/4214894696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd290003a00 0x7fd290003e20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.260+0000 7fd2abc43700 1 -- 172.21.15.5:0/4214894696 >> 172.21.15.5:0/4214894696 conn(0x7fd2a40713c0 msgr2=0x7fd2a40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.260+0000 7fd2abc43700 1 -- 172.21.15.5:0/4214894696 shutdown_connections 2024-05-08T23:21:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.260+0000 7fd2abc43700 1 -- 172.21.15.5:0/4214894696 wait complete. 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.261+0000 7fd2abc43700 1 Processor -- start 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.261+0000 7fd2abc43700 1 -- start start 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2abc43700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290003a00 0x7fd2a40b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2abc43700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4159a00 0x7fd2a41454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2abc43700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2a40b3f80 0x7fd2a4154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2abc43700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4159e60 con 0x7fd2a4159a00 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2abc43700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4145a30 con 0x7fd290003a00 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2abc43700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40b4760 con 0x7fd2a40b3f80 2024-05-08T23:21:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a91de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4159a00 0x7fd2a41454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a91de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4159a00 0x7fd2a41454f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36712/0 (socket says 172.21.15.5:36712) 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a91de700 1 -- 172.21.15.5:0/3746014753 learned_addr learned my addr 172.21.15.5:0/3746014753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a99df700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290003a00 0x7fd2a40b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2aa1e0700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2a40b3f80 0x7fd2a4154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a91de700 1 -- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2a40b3f80 msgr2=0x7fd2a4154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a91de700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2a40b3f80 0x7fd2a4154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a91de700 1 -- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290003a00 msgr2=0x7fd2a40b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a91de700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290003a00 0x7fd2a40b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a91de700 1 -- 172.21.15.5:0/3746014753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a40b4a40 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.262+0000 7fd2a99df700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290003a00 0x7fd2a40b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.263+0000 7fd2a91de700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4159a00 0x7fd2a41454f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd29800f820 tx=0x7fd29800fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.263+0000 7fd2967fc700 1 -- 172.21.15.5:0/3746014753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2980052e0 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.263+0000 7fd2967fc700 1 -- 172.21.15.5:0/3746014753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd298005440 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.263+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a40b4d20 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.263+0000 7fd2967fc700 1 -- 172.21.15.5:0/3746014753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2980038c0 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.263+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a40b52e0 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.264+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a4074bc0 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.264+0000 7fd2967fc700 1 -- 172.21.15.5:0/3746014753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd298003a20 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.265+0000 7fd2967fc700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd28806ac90 0x7fd28806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.265+0000 7fd2967fc700 1 -- 172.21.15.5:0/3746014753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd298089200 con 0x7fd2a4159a00 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.265+0000 7fd2a99df700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd28806ac90 0x7fd28806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.266+0000 7fd2a99df700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd28806ac90 0x7fd28806d150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00090b0 tx=0x7fd2a0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.270+0000 7fd2967fc700 1 -- 172.21.15.5:0/3746014753 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd298054350 con 0x7fd2a4159a00 2024-05-08T23:21:06.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:06.536+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd2a4155af0 con 0x7fd2a4159a00 2024-05-08T23:21:06.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:21:06.980 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:21:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.098+0000 7fd2967fc700 1 -- 172.21.15.5:0/3746014753 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd298057970 con 0x7fd2a4159a00 2024-05-08T23:21:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd28806ac90 msgr2=0x7fd28806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd28806ac90 0x7fd28806d150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00090b0 tx=0x7fd2a0008030 comp rx=0 tx=0).stop 2024-05-08T23:21:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4159a00 msgr2=0x7fd2a41454f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4159a00 0x7fd2a41454f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd29800f820 tx=0x7fd29800fbe0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 shutdown_connections 2024-05-08T23:21:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd28806ac90 0x7fd28806d150 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2a40b3f80 0x7fd2a4154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd290003a00 0x7fd2a40b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 --2- 172.21.15.5:0/3746014753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4159a00 0x7fd2a41454f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 >> 172.21.15.5:0/3746014753 conn(0x7fd2a40713c0 msgr2=0x7fd2a4158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 shutdown_connections 2024-05-08T23:21:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.101+0000 7fd2abc43700 1 -- 172.21.15.5:0/3746014753 wait complete. 2024-05-08T23:21:07.115 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:21:07.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:07.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.259+0000 7f4cdf29e700 1 Processor -- start 2024-05-08T23:21:07.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.259+0000 7f4cdf29e700 1 -- start start 2024-05-08T23:21:07.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdf29e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 0x7f4cd80d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdf29e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4cd806bd80 0x7f4cd809a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdf29e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd80cf990 con 0x7f4cd806c3d0 2024-05-08T23:21:07.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdf29e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd80d04b0 con 0x7f4cd806bd80 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdf29e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd809ad60 con 0x7f4cd815da10 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdc839700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4cd806bd80 0x7f4cd809a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdc839700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4cd806bd80 0x7f4cd809a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36720/0 (socket says 172.21.15.5:36720) 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdc839700 1 -- 172.21.15.5:0/1010283802 learned_addr learned my addr 172.21.15.5:0/1010283802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdd03a700 1 --2- 172.21.15.5:0/1010283802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 0x7f4cd80d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdc839700 1 -- 172.21.15.5:0/1010283802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 msgr2=0x7f4cd80d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdc839700 1 --2- 172.21.15.5:0/1010283802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 0x7f4cd80d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdc839700 1 -- 172.21.15.5:0/1010283802 >> v1:172.21.15.49:6789/0 conn(0x7f4cd806c3d0 legacy=0x7f4cd80cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdc839700 1 -- 172.21.15.5:0/1010283802 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd80cfbc0 con 0x7f4cd806bd80 2024-05-08T23:21:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.260+0000 7f4cdc839700 1 --2- 172.21.15.5:0/1010283802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd809a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80098f0 tx=0x7f4cc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394273838e526c8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4ccffff700 1 -- 172.21.15.5:0/1010283802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8011830 con 0x7f4cd806bd80 2024-05-08T23:21:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4ccffff700 1 -- 172.21.15.5:0/1010283802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8011990 con 0x7f4cd806bd80 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4ccffff700 1 -- 172.21.15.5:0/1010283802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8011d20 con 0x7f4cd806bd80 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4cdf29e700 1 -- 172.21.15.5:0/1010283802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 msgr2=0x7f4cd809a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/1010283802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd809a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80098f0 tx=0x7f4cc8009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4cdf29e700 1 -- 172.21.15.5:0/1010283802 shutdown_connections 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/1010283802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 0x7f4cd80d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/1010283802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd809a430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4cdf29e700 1 -- 172.21.15.5:0/1010283802 >> 172.21.15.5:0/1010283802 conn(0x7f4cd80b3010 msgr2=0x7f4cd80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4cdf29e700 1 -- 172.21.15.5:0/1010283802 shutdown_connections 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.261+0000 7f4cdf29e700 1 -- 172.21.15.5:0/1010283802 wait complete. 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.263+0000 7f4cdf29e700 1 Processor -- start 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.263+0000 7f4cdf29e700 1 -- start start 2024-05-08T23:21:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.263+0000 7f4cdf29e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdf29e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd806c3d0 0x7f4cd814be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdf29e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 0x7f4cd80b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdf29e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80c9fa0 con 0x7f4cd806bd80 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdf29e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd814c3c0 con 0x7f4cd806c3d0 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdf29e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80c9640 con 0x7f4cd815da10 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd814eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36736/0 (socket says 172.21.15.5:36736) 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 -- 172.21.15.5:0/445436453 learned_addr learned my addr 172.21.15.5:0/445436453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdc839700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd806c3d0 0x7f4cd814be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd83b700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 0x7f4cd80b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 -- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 msgr2=0x7f4cd80b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 0x7f4cd80b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 -- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd806c3d0 msgr2=0x7f4cd814be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd806c3d0 0x7f4cd814be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 -- 172.21.15.5:0/445436453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc8009550 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdd03a700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd814eab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9b0 tx=0x7f4cd400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4ccdffb700 1 -- 172.21.15.5:0/445436453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4009710 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4ccdffb700 1 -- 172.21.15.5:0/445436453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd400ee50 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd80b7000 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.264+0000 7f4ccdffb700 1 -- 172.21.15.5:0/445436453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400ca50 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.265+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd80c98e0 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.266+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd8068000 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.266+0000 7f4ccdffb700 1 -- 172.21.15.5:0/445436453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4cd400cbb0 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.267+0000 7f4ccdffb700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4cbc06ac90 0x7f4cbc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.267+0000 7f4ccdffb700 1 -- 172.21.15.5:0/445436453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cd40892e0 con 0x7f4cd806bd80 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.267+0000 7f4cdc839700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4cbc06ac90 0x7f4cbc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.267+0000 7f4cdc839700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4cbc06ac90 0x7f4cbc06d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4cd806c1c0 tx=0x7f4cc8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.271+0000 7f4ccdffb700 1 -- 172.21.15.5:0/445436453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4cd40544b0 con 0x7f4cd806bd80 2024-05-08T23:21:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.540+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cd80b3c70 con 0x7f4cd806bd80 2024-05-08T23:21:07.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.541+0000 7f4ccdffb700 1 -- 172.21.15.5:0/445436453 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4719 (secure 0 0 0) 0x7f4cd400ce60 con 0x7f4cd806bd80 2024-05-08T23:21:07.540 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:07.541 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:21:04.087368+0000","modified":"2024-05-08T23:21:07.094437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4cbc06ac90 msgr2=0x7f4cbc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4cbc06ac90 0x7f4cbc06d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4cd806c1c0 tx=0x7f4cc8008030 comp rx=0 tx=0).stop 2024-05-08T23:21:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 msgr2=0x7f4cd814eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd814eab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9b0 tx=0x7f4cd400ecc0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 shutdown_connections 2024-05-08T23:21:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4cbc06ac90 0x7f4cbc06d150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4cd815da10 0x7f4cd80b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cd806bd80 0x7f4cd814eab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 --2- 172.21.15.5:0/445436453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd806c3d0 0x7f4cd814be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 >> 172.21.15.5:0/445436453 conn(0x7f4cd80b3010 msgr2=0x7f4cd80b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 shutdown_connections 2024-05-08T23:21:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.544+0000 7f4cdf29e700 1 -- 172.21.15.5:0/445436453 wait complete. 2024-05-08T23:21:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-08T23:21:07.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:07.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.692+0000 7f5f5d8d3700 1 Processor -- start 2024-05-08T23:21:07.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.692+0000 7f5f5d8d3700 1 -- start start 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f5d8d3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5f58161b80 0x7f5f580cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f5d8d3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5806c9c0 con 0x7f5f5815db60 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f5d8d3700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5806cca0 con 0x7f5f5815d780 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f5d8d3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5806cf80 con 0x7f5f58161b80 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f567fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5f5815db60 0x7f5f580d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44544/0 (socket says 172.21.15.5:44544) 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f567fc700 1 -- 172.21.15.5:0/627249386 learned_addr learned my addr 172.21.15.5:0/627249386 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f577fe700 1 --2- 172.21.15.5:0/627249386 >> v2:172.21.15.49:3300/0 conn(0x7f5f58161b80 0x7f5f580cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f55ffb700 1 -- 172.21.15.5:0/627249386 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116648189 0 0) 0x7f5f5806c9c0 con 0x7f5f5815db60 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.693+0000 7f5f55ffb700 1 -- 172.21.15.5:0/627249386 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f400035b0 con 0x7f5f5815db60 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.694+0000 7f5f577fe700 1 -- 172.21.15.5:0/627249386 >> v1:172.21.15.49:6789/0 conn(0x7f5f5815d780 legacy=0x7f5f58161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.694+0000 7f5f55ffb700 1 -- 172.21.15.5:0/627249386 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689624522 0 0) 0x7f5f5806cca0 con 0x7f5f5815d780 2024-05-08T23:21:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.694+0000 7f5f577fe700 1 -- 172.21.15.5:0/627249386 >> v1:172.21.15.5:6789/0 conn(0x7f5f5815db60 legacy=0x7f5f580d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.694+0000 7f5f577fe700 1 -- 172.21.15.5:0/627249386 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5806d260 con 0x7f5f58161b80 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.694+0000 7f5f577fe700 1 --2- 172.21.15.5:0/627249386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f58161b80 0x7f5f580cf400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009d90 tx=0x7f5f5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57239ef46acc2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.694+0000 7f5f55ffb700 1 -- 172.21.15.5:0/627249386 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5000e070 con 0x7f5f58161b80 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.694+0000 7f5f55ffb700 1 -- 172.21.15.5:0/627249386 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f50003a20 con 0x7f5f58161b80 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.694+0000 7f5f55ffb700 1 -- 172.21.15.5:0/627249386 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c001670 con 0x7f5f58161b80 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.695+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/627249386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f58161b80 msgr2=0x7f5f580cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.695+0000 7f5f5d8d3700 1 --2- 172.21.15.5:0/627249386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f58161b80 0x7f5f580cf400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009d90 tx=0x7f5f5000b330 comp rx=0 tx=0).stop 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.695+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/627249386 shutdown_connections 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.695+0000 7f5f5d8d3700 1 --2- 172.21.15.5:0/627249386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f58161b80 0x7f5f580cf400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.695+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/627249386 >> 172.21.15.5:0/627249386 conn(0x7f5f5808afe0 msgr2=0x7f5f580d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.695+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/627249386 shutdown_connections 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.695+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/627249386 wait complete. 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.696+0000 7f5f5d8d3700 1 Processor -- start 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.696+0000 7f5f5d8d3700 1 -- start start 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f5d8d3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5815d780 0x7f5f581605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f5d8d3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f5815db60 0x7f5f580b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f5d8d3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58161b80 0x7f5f580b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f5d8d3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58160b00 con 0x7f5f58161b80 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f5d8d3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f580cc860 con 0x7f5f5815d780 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f5d8d3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f580ccb40 con 0x7f5f5815db60 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f577fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58161b80 0x7f5f580b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f577fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58161b80 0x7f5f580b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60430/0 (socket says 172.21.15.5:60430) 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f577fe700 1 -- 172.21.15.5:0/1871606806 learned_addr learned my addr 172.21.15.5:0/1871606806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f56ffd700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5815d780 0x7f5f581605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.697+0000 7f5f567fc700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f5815db60 0x7f5f580b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f577fe700 1 -- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f5815db60 msgr2=0x7f5f580b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f577fe700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f5815db60 0x7f5f580b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f577fe700 1 -- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5815d780 msgr2=0x7f5f581605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f577fe700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5815d780 0x7f5f581605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f577fe700 1 -- 172.21.15.5:0/1871606806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f50009a70 con 0x7f5f58161b80 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f577fe700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58161b80 0x7f5f580b4a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f50015040 tx=0x7f5f50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f3ffff700 1 -- 172.21.15.5:0/1871606806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50004540 con 0x7f5f58161b80 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f3ffff700 1 -- 172.21.15.5:0/1871606806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f500046e0 con 0x7f5f58161b80 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f580cce20 con 0x7f5f58161b80 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f3ffff700 1 -- 172.21.15.5:0/1871606806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50009410 con 0x7f5f58161b80 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.698+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f580b3420 con 0x7f5f58161b80 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.699+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f580b2a40 con 0x7f5f58161b80 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.700+0000 7f5f3ffff700 1 -- 172.21.15.5:0/1871606806 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f50012450 con 0x7f5f58161b80 2024-05-08T23:21:07.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.700+0000 7f5f3ffff700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4406ac90 0x7f5f4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.700+0000 7f5f3ffff700 1 -- 172.21.15.5:0/1871606806 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f5008aac0 con 0x7f5f58161b80 2024-05-08T23:21:07.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.700+0000 7f5f56ffd700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4406ac90 0x7f5f4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.701+0000 7f5f56ffd700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4406ac90 0x7f5f4406d150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000f60 tx=0x7f5f48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.705+0000 7f5f3ffff700 1 -- 172.21.15.5:0/1871606806 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f500597c0 con 0x7f5f58161b80 2024-05-08T23:21:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.922+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f5815e300 con 0x7f5f58161b80 2024-05-08T23:21:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.923+0000 7f5f3ffff700 1 -- 172.21.15.5:0/1871606806 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7f5f5815e300 con 0x7f5f58161b80 2024-05-08T23:21:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:07.924 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4406ac90 msgr2=0x7f5f4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4406ac90 0x7f5f4406d150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000f60 tx=0x7f5f48009040 comp rx=0 tx=0).stop 2024-05-08T23:21:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58161b80 msgr2=0x7f5f580b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58161b80 0x7f5f580b4a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f50015040 tx=0x7f5f50004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 shutdown_connections 2024-05-08T23:21:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4406ac90 0x7f5f4406d150 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58161b80 0x7f5f580b4a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5815d780 0x7f5f581605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 --2- 172.21.15.5:0/1871606806 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f5815db60 0x7f5f580b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 >> 172.21.15.5:0/1871606806 conn(0x7f5f5808afe0 msgr2=0x7f5f580d3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 shutdown_connections 2024-05-08T23:21:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:07.926+0000 7f5f5d8d3700 1 -- 172.21.15.5:0/1871606806 wait complete. 2024-05-08T23:21:07.940 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:21:07.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:21:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.075+0000 7f2f0930d700 1 Processor -- start 2024-05-08T23:21:08.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.076+0000 7f2f0930d700 1 -- start start 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.076+0000 7f2f0930d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f04161b80 0x7f2f040cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.076+0000 7f2f0930d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0406c9c0 con 0x7f2f0415db60 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.076+0000 7f2f0930d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0406cca0 con 0x7f2f0415d780 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.076+0000 7f2f0930d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0406cf80 con 0x7f2f04161b80 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f037fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f04161b80 0x7f2f040cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f027fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2f0415db60 0x7f2f040d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44552/0 (socket says 172.21.15.5:44552) 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f027fc700 1 -- 172.21.15.5:0/2431108296 learned_addr learned my addr 172.21.15.5:0/2431108296 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f037fe700 1 -- 172.21.15.5:0/2431108296 >> v1:172.21.15.49:6789/0 conn(0x7f2f0415d780 legacy=0x7f2f04161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f037fe700 1 -- 172.21.15.5:0/2431108296 >> v1:172.21.15.5:6789/0 conn(0x7f2f0415db60 legacy=0x7f2f040d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f037fe700 1 -- 172.21.15.5:0/2431108296 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0406d260 con 0x7f2f04161b80 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f01ffb700 1 -- 172.21.15.5:0/2431108296 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725273622 0 0) 0x7f2f0406c9c0 con 0x7f2f0415db60 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f037fe700 1 --2- 172.21.15.5:0/2431108296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f04161b80 0x7f2f040cf400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009d90 tx=0x7f2efc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584b325b8082c21d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f01ffb700 1 -- 172.21.15.5:0/2431108296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00e070 con 0x7f2f04161b80 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f01ffb700 1 -- 172.21.15.5:0/2431108296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc00bbd0 con 0x7f2f04161b80 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.077+0000 7f2f01ffb700 1 -- 172.21.15.5:0/2431108296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc012430 con 0x7f2f04161b80 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.078+0000 7f2f0930d700 1 -- 172.21.15.5:0/2431108296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f04161b80 msgr2=0x7f2f040cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.078+0000 7f2f0930d700 1 --2- 172.21.15.5:0/2431108296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f04161b80 0x7f2f040cf400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009d90 tx=0x7f2efc00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.078+0000 7f2f0930d700 1 -- 172.21.15.5:0/2431108296 shutdown_connections 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.078+0000 7f2f0930d700 1 --2- 172.21.15.5:0/2431108296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f04161b80 0x7f2f040cf400 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.078+0000 7f2f0930d700 1 -- 172.21.15.5:0/2431108296 >> 172.21.15.5:0/2431108296 conn(0x7f2f0408afe0 msgr2=0x7f2f040d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.078+0000 7f2f0930d700 1 -- 172.21.15.5:0/2431108296 shutdown_connections 2024-05-08T23:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.078+0000 7f2f0930d700 1 -- 172.21.15.5:0/2431108296 wait complete. 2024-05-08T23:21:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.079+0000 7f2f0930d700 1 Processor -- start 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f0930d700 1 -- start start 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f0930d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0415d780 0x7f2f040b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f0930d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0415db60 0x7f2f04160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f0930d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f04161b80 0x7f2f040b8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f0930d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040cb480 con 0x7f2f0415d780 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f0930d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f041611d0 con 0x7f2f0415db60 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f0930d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040cb130 con 0x7f2f04161b80 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0415d780 0x7f2f040b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0415d780 0x7f2f040b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60450/0 (socket says 172.21.15.5:60450) 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.080+0000 7f2f02ffd700 1 -- 172.21.15.5:0/49998239 learned_addr learned my addr 172.21.15.5:0/49998239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f027fc700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0415db60 0x7f2f04160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f037fe700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f04161b80 0x7f2f040b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f02ffd700 1 -- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f04161b80 msgr2=0x7f2f040b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f02ffd700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f04161b80 0x7f2f040b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f02ffd700 1 -- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0415db60 msgr2=0x7f2f04160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f02ffd700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0415db60 0x7f2f04160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f02ffd700 1 -- 172.21.15.5:0/49998239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc009a70 con 0x7f2f0415d780 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f037fe700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f04161b80 0x7f2f040b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f02ffd700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0415d780 0x7f2f040b6220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c4e0 tx=0x7f2ef400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2eeb7fe700 1 -- 172.21.15.5:0/49998239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40052d0 con 0x7f2f0415d780 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2eeb7fe700 1 -- 172.21.15.5:0/49998239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4005430 con 0x7f2f0415d780 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f040b68e0 con 0x7f2f0415d780 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2eeb7fe700 1 -- 172.21.15.5:0/49998239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40038c0 con 0x7f2f0415d780 2024-05-08T23:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.081+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f040b6ea0 con 0x7f2f0415d780 2024-05-08T23:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.082+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f040678f0 con 0x7f2f0415d780 2024-05-08T23:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.083+0000 7f2eeb7fe700 1 -- 172.21.15.5:0/49998239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2ef4003a20 con 0x7f2f0415d780 2024-05-08T23:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.083+0000 7f2eeb7fe700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef006ac90 0x7f2ef006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.083+0000 7f2eeb7fe700 1 -- 172.21.15.5:0/49998239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ef40886e0 con 0x7f2f0415d780 2024-05-08T23:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.083+0000 7f2f027fc700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef006ac90 0x7f2ef006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.084+0000 7f2f027fc700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef006ac90 0x7f2ef006d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8007b10 tx=0x7f2ef8007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.088+0000 7f2eeb7fe700 1 -- 172.21.15.5:0/49998239 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2ef4053950 con 0x7f2f0415d780 2024-05-08T23:21:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.299+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f040b9530 con 0x7f2ef006ac90 2024-05-08T23:21:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.308+0000 7f2eeb7fe700 1 -- 172.21.15.5:0/49998239 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249918 (secure 0 0 0) 0x7f2f040b9530 con 0x7f2ef006ac90 2024-05-08T23:21:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef006ac90 msgr2=0x7f2ef006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef006ac90 0x7f2ef006d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8007b10 tx=0x7f2ef8007a60 comp rx=0 tx=0).stop 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0415d780 msgr2=0x7f2f040b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0415d780 0x7f2f040b6220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c4e0 tx=0x7f2ef400c7f0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 shutdown_connections 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef006ac90 0x7f2ef006d150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f04161b80 0x7f2f040b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0415d780 0x7f2f040b6220 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 --2- 172.21.15.5:0/49998239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0415db60 0x7f2f04160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 >> 172.21.15.5:0/49998239 conn(0x7f2f0408afe0 msgr2=0x7f2f0415cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 shutdown_connections 2024-05-08T23:21:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.311+0000 7f2f0930d700 1 -- 172.21.15.5:0/49998239 wait complete. 2024-05-08T23:21:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:21:08.315 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:21:08.315+0000 7ff160674700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:21:08.325 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-05-08T23:21:07.293985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218816,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755904,"statfs":{"total":773094113280,"available":772870045696,"internally_reserved":0,"allocated":2433024,"data_stored":1546584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.675148"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T23:21:00.284248+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:21:00.284248+0000","last_peered":"2024-05-08T23:21:00.284248+0000","last_clean":"2024-05-08T23:21:00.284248+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:21:00.284248+0000","last_undegraded":"2024-05-08T23:21:00.284248+0000","last_fullsized":"2024-05-08T23:21:00.284248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8959999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:21:08.330 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:21:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.505+0000 7fbe3a327700 1 Processor -- start 2024-05-08T23:21:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.505+0000 7fbe3a327700 1 -- start start 2024-05-08T23:21:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe3a327700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe340d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe3a327700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbe3406c3d0 0x7fbe3406c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe3a327700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe340cf990 con 0x7fbe3406bd80 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe3a327700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3406cdc0 con 0x7fbe3406c3d0 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe3a327700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3409ab40 con 0x7fbe3415da10 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe388c4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbe3406c3d0 0x7fbe3406c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe388c4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbe3406c3d0 0x7fbe3406c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60464/0 (socket says 172.21.15.5:60464) 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe388c4700 1 -- 172.21.15.5:0/786043763 learned_addr learned my addr 172.21.15.5:0/786043763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe33fff700 1 --2- 172.21.15.5:0/786043763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe340d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe388c4700 1 -- 172.21.15.5:0/786043763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 msgr2=0x7fbe340d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe388c4700 1 --2- 172.21.15.5:0/786043763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe340d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe388c4700 1 -- 172.21.15.5:0/786043763 >> v1:172.21.15.49:6789/0 conn(0x7fbe3406bd80 legacy=0x7fbe3409a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe388c4700 1 -- 172.21.15.5:0/786043763 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340d04b0 con 0x7fbe3406c3d0 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe33fff700 1 --2- 172.21.15.5:0/786043763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe340d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe388c4700 1 --2- 172.21.15.5:0/786043763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3406c880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009e80 tx=0x7fbe2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb3cf18c4f7be4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.506+0000 7fbe32ffd700 1 -- 172.21.15.5:0/786043763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00e070 con 0x7fbe3406c3d0 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe32ffd700 1 -- 172.21.15.5:0/786043763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00b980 con 0x7fbe3406c3d0 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe32ffd700 1 -- 172.21.15.5:0/786043763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00bd30 con 0x7fbe3406c3d0 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe3a327700 1 -- 172.21.15.5:0/786043763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 msgr2=0x7fbe3406c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe3a327700 1 --2- 172.21.15.5:0/786043763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3406c880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009e80 tx=0x7fbe2c00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe3a327700 1 -- 172.21.15.5:0/786043763 shutdown_connections 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe3a327700 1 --2- 172.21.15.5:0/786043763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3406c880 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe3a327700 1 --2- 172.21.15.5:0/786043763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe340d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe3a327700 1 -- 172.21.15.5:0/786043763 >> 172.21.15.5:0/786043763 conn(0x7fbe340b3010 msgr2=0x7fbe340b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe3a327700 1 -- 172.21.15.5:0/786043763 shutdown_connections 2024-05-08T23:21:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.507+0000 7fbe3a327700 1 -- 172.21.15.5:0/786043763 wait complete. 2024-05-08T23:21:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.508+0000 7fbe3a327700 1 Processor -- start 2024-05-08T23:21:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.509+0000 7fbe3a327700 1 -- start start 2024-05-08T23:21:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.509+0000 7fbe3a327700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3406bd80 0x7fbe34149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.509+0000 7fbe3a327700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3414c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.509+0000 7fbe3a327700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe341f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.509+0000 7fbe3a327700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340b5aa0 con 0x7fbe3406c3d0 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.509+0000 7fbe3a327700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340b5190 con 0x7fbe3406bd80 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.509+0000 7fbe3a327700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3414c6a0 con 0x7fbe3415da10 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3414c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3414c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60466/0 (socket says 172.21.15.5:60466) 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 -- 172.21.15.5:0/4248027928 learned_addr learned my addr 172.21.15.5:0/4248027928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe33fff700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3406bd80 0x7fbe34149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe388c4700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe341f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 -- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 msgr2=0x7fbe341f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe341f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 -- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3406bd80 msgr2=0x7fbe34149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3406bd80 0x7fbe34149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 -- 172.21.15.5:0/4248027928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c009b30 con 0x7fbe3406c3d0 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe388c4700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe341f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe33fff700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3406bd80 0x7fbe34149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe337fe700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3414c160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c420 tx=0x7fbe2800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe30ff9700 1 -- 172.21.15.5:0/4248027928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280052d0 con 0x7fbe3406c3d0 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe30ff9700 1 -- 172.21.15.5:0/4248027928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28005430 con 0x7fbe3406c3d0 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe340c8200 con 0x7fbe3406c3d0 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe30ff9700 1 -- 172.21.15.5:0/4248027928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280038c0 con 0x7fbe3406c3d0 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.510+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe340b56c0 con 0x7fbe3406c3d0 2024-05-08T23:21:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.511+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe340b5be0 con 0x7fbe3406c3d0 2024-05-08T23:21:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.512+0000 7fbe30ff9700 1 -- 172.21.15.5:0/4248027928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbe28003a20 con 0x7fbe3406c3d0 2024-05-08T23:21:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.512+0000 7fbe30ff9700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe2006ac90 0x7fbe2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.512+0000 7fbe30ff9700 1 -- 172.21.15.5:0/4248027928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe28088750 con 0x7fbe3406c3d0 2024-05-08T23:21:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.513+0000 7fbe33fff700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe2006ac90 0x7fbe2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.513+0000 7fbe33fff700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe2006ac90 0x7fbe2006d150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe240099f0 tx=0x7fbe24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.517+0000 7fbe30ff9700 1 -- 172.21.15.5:0/4248027928 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbe340b5be0 con 0x7fbe3406c3d0 2024-05-08T23:21:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.728+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe340cfbc0 con 0x7fbe2006ac90 2024-05-08T23:21:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.736+0000 7fbe30ff9700 1 -- 172.21.15.5:0/4248027928 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249918 (secure 0 0 0) 0x7fbe24009680 con 0x7fbe2006ac90 2024-05-08T23:21:08.739 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.739+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe2006ac90 msgr2=0x7fbe2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe2006ac90 0x7fbe2006d150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe240099f0 tx=0x7fbe24008040 comp rx=0 tx=0).stop 2024-05-08T23:21:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 msgr2=0x7fbe3414c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3414c160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c420 tx=0x7fbe2800c7e0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 shutdown_connections 2024-05-08T23:21:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe2006ac90 0x7fbe2006d150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe3415da10 0x7fbe341f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3406bd80 0x7fbe34149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 --2- 172.21.15.5:0/4248027928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3406c3d0 0x7fbe3414c160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 >> 172.21.15.5:0/4248027928 conn(0x7fbe340b3010 msgr2=0x7fbe34099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 shutdown_connections 2024-05-08T23:21:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.740+0000 7fbe3a327700 1 -- 172.21.15.5:0/4248027928 wait complete. 2024-05-08T23:21:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:21:08.753 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-05-08T23:21:07.293985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218816,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755904,"statfs":{"total":773094113280,"available":772870045696,"internally_reserved":0,"allocated":2433024,"data_stored":1546584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.675148"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T23:21:00.284248+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:21:00.284248+0000","last_peered":"2024-05-08T23:21:00.284248+0000","last_clean":"2024-05-08T23:21:00.284248+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:21:00.284248+0000","last_undegraded":"2024-05-08T23:21:00.284248+0000","last_fullsized":"2024-05-08T23:21:00.284248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8959999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:21:08.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:21:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.932+0000 7fedbac24700 1 Processor -- start 2024-05-08T23:21:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.932+0000 7fedbac24700 1 -- start start 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.933+0000 7fedbac24700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fedb415da10 0x7fedb40d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.933+0000 7fedbac24700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406bd80 0x7fedb409a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.933+0000 7fedbac24700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb40cf990 con 0x7fedb406c3d0 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.933+0000 7fedbac24700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb40d04b0 con 0x7fedb415da10 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.933+0000 7fedbac24700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb409ad60 con 0x7fedb406bd80 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.933+0000 7fedb89c0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fedb415da10 0x7fedb40d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.933+0000 7fedb89c0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fedb415da10 0x7fedb40d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60474/0 (socket says 172.21.15.5:60474) 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.933+0000 7fedb89c0700 1 -- 172.21.15.5:0/4060920758 learned_addr learned my addr 172.21.15.5:0/4060920758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb3fff700 1 --2- 172.21.15.5:0/4060920758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406bd80 0x7fedb409a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb89c0700 1 -- 172.21.15.5:0/4060920758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406bd80 msgr2=0x7fedb409a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb89c0700 1 --2- 172.21.15.5:0/4060920758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406bd80 0x7fedb409a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb89c0700 1 -- 172.21.15.5:0/4060920758 >> v1:172.21.15.49:6789/0 conn(0x7fedb406c3d0 legacy=0x7fedb40cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb89c0700 1 -- 172.21.15.5:0/4060920758 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb40cfbc0 con 0x7fedb415da10 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb89c0700 1 --2- 172.21.15.5:0/4060920758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40d0940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feda40098f0 tx=0x7feda4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3112239d777899a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb37fe700 1 -- 172.21.15.5:0/4060920758 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400b8a0 con 0x7fedb415da10 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb37fe700 1 -- 172.21.15.5:0/4060920758 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda400ba00 con 0x7fedb415da10 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.934+0000 7fedb37fe700 1 -- 172.21.15.5:0/4060920758 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400bd90 con 0x7fedb415da10 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.935+0000 7fedbac24700 1 -- 172.21.15.5:0/4060920758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 msgr2=0x7fedb40d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.935+0000 7fedbac24700 1 --2- 172.21.15.5:0/4060920758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40d0940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feda40098f0 tx=0x7feda4009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.935+0000 7fedbac24700 1 -- 172.21.15.5:0/4060920758 shutdown_connections 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.935+0000 7fedbac24700 1 --2- 172.21.15.5:0/4060920758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40d0940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.935+0000 7fedbac24700 1 --2- 172.21.15.5:0/4060920758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406bd80 0x7fedb409a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.935+0000 7fedbac24700 1 -- 172.21.15.5:0/4060920758 >> 172.21.15.5:0/4060920758 conn(0x7fedb40b3010 msgr2=0x7fedb40b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.935+0000 7fedbac24700 1 -- 172.21.15.5:0/4060920758 shutdown_connections 2024-05-08T23:21:08.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.935+0000 7fedbac24700 1 -- 172.21.15.5:0/4060920758 wait complete. 2024-05-08T23:21:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.936+0000 7fedbac24700 1 Processor -- start 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.936+0000 7fedbac24700 1 -- start start 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedbac24700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb406bd80 0x7fedb40b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedbac24700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406c3d0 0x7fedb40af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedbac24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedbac24700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb414c7e0 con 0x7fedb415da10 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedbac24700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40c8690 con 0x7fedb406bd80 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedbac24700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40c8970 con 0x7fedb406c3d0 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb91c1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb91c1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60482/0 (socket says 172.21.15.5:60482) 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb91c1700 1 -- 172.21.15.5:0/1004703024 learned_addr learned my addr 172.21.15.5:0/1004703024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb89c0700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb406bd80 0x7fedb40b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb3fff700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406c3d0 0x7fedb40af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb91c1700 1 -- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406c3d0 msgr2=0x7fedb40af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb91c1700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406c3d0 0x7fedb40af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb91c1700 1 -- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb406bd80 msgr2=0x7fedb40b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb91c1700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb406bd80 0x7fedb40b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.937+0000 7fedb91c1700 1 -- 172.21.15.5:0/1004703024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda4009550 con 0x7fedb415da10 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.938+0000 7fedb91c1700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40afb20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fedac00cb50 tx=0x7fedac00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.938+0000 7fedb17fa700 1 -- 172.21.15.5:0/1004703024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac003c00 con 0x7fedb415da10 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.938+0000 7fedb17fa700 1 -- 172.21.15.5:0/1004703024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac003d60 con 0x7fedb415da10 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.938+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb40c8c50 con 0x7fedb415da10 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.938+0000 7fedb17fa700 1 -- 172.21.15.5:0/1004703024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac00ab70 con 0x7fedb415da10 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.938+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb40c9450 con 0x7fedb415da10 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.939+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb40c9170 con 0x7fedb415da10 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.939+0000 7fedb17fa700 1 -- 172.21.15.5:0/1004703024 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fedac00acd0 con 0x7fedb415da10 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.940+0000 7fedb17fa700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feda006acc0 0x7feda006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.940+0000 7fedb17fa700 1 -- 172.21.15.5:0/1004703024 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedac089400 con 0x7fedb415da10 2024-05-08T23:21:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.940+0000 7fedb89c0700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feda006acc0 0x7feda006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.940+0000 7fedb89c0700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feda006acc0 0x7feda006d180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feda4000c00 tx=0x7feda4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:08.944+0000 7fedb17fa700 1 -- 172.21.15.5:0/1004703024 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fedb40c9170 con 0x7fedb415da10 2024-05-08T23:21:09.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.156+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedb40d0140 con 0x7feda006acc0 2024-05-08T23:21:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.163+0000 7fedb17fa700 1 -- 172.21.15.5:0/1004703024 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249918 (secure 0 0 0) 0x7feda400bb70 con 0x7feda006acc0 2024-05-08T23:21:09.166 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feda006acc0 msgr2=0x7feda006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feda006acc0 0x7feda006d180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feda4000c00 tx=0x7feda4008030 comp rx=0 tx=0).stop 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 msgr2=0x7fedb40afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40afb20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fedac00cb50 tx=0x7fedac00cf10 comp rx=0 tx=0).stop 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 shutdown_connections 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feda006acc0 0x7feda006d180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedb415da10 0x7fedb40afb20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedb406bd80 0x7fedb40b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 --2- 172.21.15.5:0/1004703024 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fedb406c3d0 0x7fedb40af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 >> 172.21.15.5:0/1004703024 conn(0x7fedb40b3010 msgr2=0x7fedb40737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 shutdown_connections 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.167+0000 7fedbac24700 1 -- 172.21.15.5:0/1004703024 wait complete. 2024-05-08T23:21:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:21:09.180 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-05-08T23:21:07.293985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218816,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755904,"statfs":{"total":773094113280,"available":772870045696,"internally_reserved":0,"allocated":2433024,"data_stored":1546584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.675148"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T23:21:00.284248+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:21:00.284248+0000","last_peered":"2024-05-08T23:21:00.284248+0000","last_clean":"2024-05-08T23:21:00.284248+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:21:00.284248+0000","last_undegraded":"2024-05-08T23:21:00.284248+0000","last_fullsized":"2024-05-08T23:21:00.284248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:02.613341+0000","last_change":"2024-05-08T23:21:02.613341+0000","last_active":"2024-05-08T23:21:02.613341+0000","last_peered":"2024-05-08T23:21:02.613341+0000","last_clean":"2024-05-08T23:21:02.613341+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:02.613341+0000","last_undegraded":"2024-05-08T23:21:02.613341+0000","last_fullsized":"2024-05-08T23:21:02.613341+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:21:03.618536+0000","last_change":"2024-05-08T23:21:03.618536+0000","last_active":"2024-05-08T23:21:03.618536+0000","last_peered":"2024-05-08T23:21:03.618536+0000","last_clean":"2024-05-08T23:21:03.618536+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:21:03.618536+0000","last_undegraded":"2024-05-08T23:21:03.618536+0000","last_fullsized":"2024-05-08T23:21:03.618536+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8959999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:21:09.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-05-08T23:21:09.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-05-08T23:21:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-05-08T23:21:09.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-05-08T23:21:09.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-05-08T23:21:09.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-05-08T23:21:09.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-05-08T23:21:09.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-05-08T23:21:09.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-05-08T23:21:09.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-05-08T23:21:09.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-05-08T23:21:09.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-05-08T23:21:09.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-05-08T23:21:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:02.613341+0000', 'last_change': '2024-05-08T23:21:02.613341+0000', 'last_active': '2024-05-08T23:21:02.613341+0000', 'last_peered': '2024-05-08T23:21:02.613341+0000', 'last_clean': '2024-05-08T23:21:02.613341+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:02.613341+0000', 'last_undegraded': '2024-05-08T23:21:02.613341+0000', 'last_fullsized': '2024-05-08T23:21:02.613341+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:02.613341+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-05-08T23:21:09.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-05-08T23:21:09.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-05-08T23:21:09.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-05-08T23:21:09.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:21:03.618536+0000', 'last_change': '2024-05-08T23:21:03.618536+0000', 'last_active': '2024-05-08T23:21:03.618536+0000', 'last_peered': '2024-05-08T23:21:03.618536+0000', 'last_clean': '2024-05-08T23:21:03.618536+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:21:03.618536+0000', 'last_undegraded': '2024-05-08T23:21:03.618536+0000', 'last_fullsized': '2024-05-08T23:21:03.618536+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'last_clean_scrub_stamp': '2024-05-08T23:21:03.618536+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:21:09.203 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:21:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.359+0000 7f1695d8d700 1 Processor -- start 2024-05-08T23:21:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.359+0000 7f1695d8d700 1 -- start start 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f1695d8d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f16900aef20 0x7f169015c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f1695d8d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f169015f230 0x7f16900cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f1695d8d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16900cd3b0 con 0x7f16900cca70 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f1695d8d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1690141df0 con 0x7f169015f230 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f1695d8d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16900ccec0 con 0x7f16900aef20 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f168effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f169015f230 0x7f16900cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f168effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f169015f230 0x7f16900cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60484/0 (socket says 172.21.15.5:60484) 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f168effd700 1 -- 172.21.15.5:0/2201788304 learned_addr learned my addr 172.21.15.5:0/2201788304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f168f7fe700 1 --2- 172.21.15.5:0/2201788304 >> v2:172.21.15.49:3300/0 conn(0x7f16900aef20 0x7f169015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f168effd700 1 -- 172.21.15.5:0/2201788304 >> v2:172.21.15.49:3300/0 conn(0x7f16900aef20 msgr2=0x7f169015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f168effd700 1 --2- 172.21.15.5:0/2201788304 >> v2:172.21.15.49:3300/0 conn(0x7f16900aef20 0x7f169015c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f168effd700 1 -- 172.21.15.5:0/2201788304 >> v1:172.21.15.49:6789/0 conn(0x7f16900cca70 legacy=0x7f16900ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.360+0000 7f168effd700 1 -- 172.21.15.5:0/2201788304 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900af300 con 0x7f169015f230 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f168effd700 1 --2- 172.21.15.5:0/2201788304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900cc530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16900b5010 tx=0x7f1684009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65214ce94636d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f168e7fc700 1 -- 172.21.15.5:0/2201788304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684004230 con 0x7f169015f230 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f168e7fc700 1 -- 172.21.15.5:0/2201788304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1684004390 con 0x7f169015f230 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f168e7fc700 1 -- 172.21.15.5:0/2201788304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16840036a0 con 0x7f169015f230 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f1695d8d700 1 -- 172.21.15.5:0/2201788304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 msgr2=0x7f16900cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f1695d8d700 1 --2- 172.21.15.5:0/2201788304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900cc530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16900b5010 tx=0x7f1684009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f1695d8d700 1 -- 172.21.15.5:0/2201788304 shutdown_connections 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f1695d8d700 1 --2- 172.21.15.5:0/2201788304 >> v2:172.21.15.49:3300/0 conn(0x7f16900aef20 0x7f169015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f1695d8d700 1 --2- 172.21.15.5:0/2201788304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900cc530 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.361+0000 7f1695d8d700 1 -- 172.21.15.5:0/2201788304 >> 172.21.15.5:0/2201788304 conn(0x7f16900cfce0 msgr2=0x7f16900d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.362+0000 7f1695d8d700 1 -- 172.21.15.5:0/2201788304 shutdown_connections 2024-05-08T23:21:09.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.362+0000 7f1695d8d700 1 -- 172.21.15.5:0/2201788304 wait complete. 2024-05-08T23:21:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.363+0000 7f1695d8d700 1 Processor -- start 2024-05-08T23:21:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.363+0000 7f1695d8d700 1 -- start start 2024-05-08T23:21:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f1695d8d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16900aef20 0x7f16901558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f1695d8d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16900cca70 0x7f16900b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f1695d8d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f1695d8d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1690131bb0 con 0x7f169015f230 2024-05-08T23:21:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f1695d8d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1690159c00 con 0x7f16900cca70 2024-05-08T23:21:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f1695d8d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1690159260 con 0x7f16900aef20 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60496/0 (socket says 172.21.15.5:60496) 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 -- 172.21.15.5:0/1250608156 learned_addr learned my addr 172.21.15.5:0/1250608156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168f7fe700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16900aef20 0x7f16901558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168effd700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16900cca70 0x7f16900b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 -- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16900aef20 msgr2=0x7f16901558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16900aef20 0x7f16901558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 -- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16900cca70 msgr2=0x7f16900b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16900cca70 0x7f16900b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 -- 172.21.15.5:0/1250608156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009550 con 0x7f169015f230 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168f7fe700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16900aef20 0x7f16901558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.364+0000 7f168ffff700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900b0be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f168800cc10 tx=0x7f168800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.365+0000 7f1677fff700 1 -- 172.21.15.5:0/1250608156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168800abb0 con 0x7f169015f230 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.365+0000 7f1677fff700 1 -- 172.21.15.5:0/1250608156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168800ad10 con 0x7f169015f230 2024-05-08T23:21:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.365+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1690159540 con 0x7f169015f230 2024-05-08T23:21:09.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.365+0000 7f1677fff700 1 -- 172.21.15.5:0/1250608156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688012660 con 0x7f169015f230 2024-05-08T23:21:09.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.365+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1690155e20 con 0x7f169015f230 2024-05-08T23:21:09.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.366+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1690068000 con 0x7f169015f230 2024-05-08T23:21:09.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.366+0000 7f1677fff700 1 -- 172.21.15.5:0/1250608156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f16880127c0 con 0x7f169015f230 2024-05-08T23:21:09.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.367+0000 7f1677fff700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f167c06ac90 0x7f167c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.367+0000 7f1677fff700 1 -- 172.21.15.5:0/1250608156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16880899f0 con 0x7f169015f230 2024-05-08T23:21:09.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.367+0000 7f168f7fe700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f167c06ac90 0x7f167c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.368+0000 7f168f7fe700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f167c06ac90 0x7f167c06d150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1690159050 tx=0x7f1680008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:09.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.371+0000 7f1677fff700 1 -- 172.21.15.5:0/1250608156 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1688054c60 con 0x7f169015f230 2024-05-08T23:21:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.680+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f169015ff80 con 0x7f169015f230 2024-05-08T23:21:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.680+0000 7f1677fff700 1 -- 172.21.15.5:0/1250608156 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1020 (secure 0 0 0) 0x7f1688016080 con 0x7f169015f230 2024-05-08T23:21:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:09.681 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":24,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":224067584,"bytes_avail":772870045696,"bytes_total":773094113280,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4263}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-05-08T23:20:45.275295+0000","services":{}},"progress_events":{}} 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.682+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f167c06ac90 msgr2=0x7f167c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f167c06ac90 0x7f167c06d150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1690159050 tx=0x7f1680008040 comp rx=0 tx=0).stop 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 msgr2=0x7f16900b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900b0be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f168800cc10 tx=0x7f168800a060 comp rx=0 tx=0).stop 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 shutdown_connections 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f167c06ac90 0x7f167c06d150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f169015f230 0x7f16900b0be0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16900aef20 0x7f16901558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 --2- 172.21.15.5:0/1250608156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16900cca70 0x7f16900b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 >> 172.21.15.5:0/1250608156 conn(0x7f16900cfce0 msgr2=0x7f169006f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 shutdown_connections 2024-05-08T23:21:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.683+0000 7f1695d8d700 1 -- 172.21.15.5:0/1250608156 wait complete. 2024-05-08T23:21:09.696 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T23:21:09.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:21:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.833+0000 7f6a6d3fa700 1 Processor -- start 2024-05-08T23:21:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.833+0000 7f6a6d3fa700 1 -- start start 2024-05-08T23:21:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a6d3fa700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a68157e40 0x7f6a68070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a6d3fa700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a680cf420 con 0x7f6a6815ea00 2024-05-08T23:21:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a6d3fa700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a68161fc0 con 0x7f6a68161a30 2024-05-08T23:21:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a6d3fa700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a68158220 con 0x7f6a68157e40 2024-05-08T23:21:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a66ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a68157e40 0x7f6a68070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a667fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6a6815ea00 0x7f6a68161320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44556/0 (socket says 172.21.15.5:44556) 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a667fc700 1 -- 172.21.15.5:0/2950611172 learned_addr learned my addr 172.21.15.5:0/2950611172 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a65ffb700 1 -- 172.21.15.5:0/2950611172 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976072375 0 0) 0x7f6a680cf420 con 0x7f6a6815ea00 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a65ffb700 1 -- 172.21.15.5:0/2950611172 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a500035b0 con 0x7f6a6815ea00 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a66ffd700 1 -- 172.21.15.5:0/2950611172 >> v1:172.21.15.49:6789/0 conn(0x7f6a68161a30 legacy=0x7f6a680b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a66ffd700 1 -- 172.21.15.5:0/2950611172 >> v1:172.21.15.5:6789/0 conn(0x7f6a6815ea00 legacy=0x7f6a68161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.834+0000 7f6a66ffd700 1 -- 172.21.15.5:0/2950611172 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6815df20 con 0x7f6a68157e40 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a66ffd700 1 --2- 172.21.15.5:0/2950611172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68157e40 0x7f6a68070fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c58a7c06f79540e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a65ffb700 1 -- 172.21.15.5:0/2950611172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800e070 con 0x7f6a68157e40 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a65ffb700 1 -- 172.21.15.5:0/2950611172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5800bb20 con 0x7f6a68157e40 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a65ffb700 1 -- 172.21.15.5:0/2950611172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58012430 con 0x7f6a68157e40 2024-05-08T23:21:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2950611172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68157e40 msgr2=0x7f6a68070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a6d3fa700 1 --2- 172.21.15.5:0/2950611172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68157e40 0x7f6a68070fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2950611172 shutdown_connections 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a6d3fa700 1 --2- 172.21.15.5:0/2950611172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68157e40 0x7f6a68070fa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2950611172 >> 172.21.15.5:0/2950611172 conn(0x7f6a680b1470 msgr2=0x7f6a680b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2950611172 shutdown_connections 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.835+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2950611172 wait complete. 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.837+0000 7f6a6d3fa700 1 Processor -- start 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.837+0000 7f6a6d3fa700 1 -- start start 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a6d3fa700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a68157e40 0x7f6a681411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a6d3fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a6815ea00 0x7f6a68151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a6d3fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68161a30 0x7f6a68141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a6d3fa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681e9090 con 0x7f6a68161a30 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a6d3fa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681416f0 con 0x7f6a6815ea00 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a6d3fa700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681e8410 con 0x7f6a68157e40 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68161a30 0x7f6a68141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a667fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a6815ea00 0x7f6a68151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68161a30 0x7f6a68141940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60518/0 (socket says 172.21.15.5:60518) 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 -- 172.21.15.5:0/2103816852 learned_addr learned my addr 172.21.15.5:0/2103816852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a66ffd700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a68157e40 0x7f6a681411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 -- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a68157e40 msgr2=0x7f6a681411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a68157e40 0x7f6a681411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 -- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a6815ea00 msgr2=0x7f6a68151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a6815ea00 0x7f6a68151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 -- 172.21.15.5:0/2103816852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58009a70 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68161a30 0x7f6a68141940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c3c0 tx=0x7f6a6000c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a4f7fe700 1 -- 172.21.15.5:0/2103816852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a600052d0 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.838+0000 7f6a4f7fe700 1 -- 172.21.15.5:0/2103816852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a60005430 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.839+0000 7f6a4f7fe700 1 -- 172.21.15.5:0/2103816852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a600038a0 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.839+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a681e86f0 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.839+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a681e8cb0 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.839+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a681e91d0 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.840+0000 7f6a4f7fe700 1 -- 172.21.15.5:0/2103816852 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6a60003a00 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.840+0000 7f6a4f7fe700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a5406ac90 0x7f6a5406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.840+0000 7f6a4f7fe700 1 -- 172.21.15.5:0/2103816852 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a60088520 con 0x7f6a68161a30 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.841+0000 7f6a66ffd700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a5406ac90 0x7f6a5406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.841+0000 7f6a66ffd700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a5406ac90 0x7f6a5406d150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a58014040 tx=0x7f6a58011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:09.845+0000 7f6a4f7fe700 1 -- 172.21.15.5:0/2103816852 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6a681e91d0 con 0x7f6a68161a30 2024-05-08T23:21:10.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.058+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a68157a30 con 0x7f6a5406ac90 2024-05-08T23:21:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.066+0000 7f6a4f7fe700 1 -- 172.21.15.5:0/2103816852 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264704 (secure 0 0 0) 0x7f6a68157a30 con 0x7f6a5406ac90 2024-05-08T23:21:10.070 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a5406ac90 msgr2=0x7f6a5406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a5406ac90 0x7f6a5406d150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a58014040 tx=0x7f6a58011000 comp rx=0 tx=0).stop 2024-05-08T23:21:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68161a30 msgr2=0x7f6a68141940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68161a30 0x7f6a68141940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c3c0 tx=0x7f6a6000c6d0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 shutdown_connections 2024-05-08T23:21:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a5406ac90 0x7f6a5406d150 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68161a30 0x7f6a68141940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a68157e40 0x7f6a681411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 --2- 172.21.15.5:0/2103816852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a6815ea00 0x7f6a68151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 >> 172.21.15.5:0/2103816852 conn(0x7f6a680b1470 msgr2=0x7f6a680acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 shutdown_connections 2024-05-08T23:21:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:10.070+0000 7f6a6d3fa700 1 -- 172.21.15.5:0/2103816852 wait complete. 2024-05-08T23:21:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:21:10.084 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-05-08T23:21:09.294423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001223"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356525+0000","last_change":"2024-05-08T23:21:04.117587+0000","last_active":"2024-05-08T23:21:05.356525+0000","last_peered":"2024-05-08T23:21:05.356525+0000","last_clean":"2024-05-08T23:21:05.356525+0000","last_became_active":"2024-05-08T23:21:04.117296+0000","last_became_peered":"2024-05-08T23:21:04.117296+0000","last_unstale":"2024-05-08T23:21:05.356525+0000","last_undegraded":"2024-05-08T23:21:05.356525+0000","last_fullsized":"2024-05-08T23:21:05.356525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093945+0000","last_change":"2024-05-08T23:21:04.113872+0000","last_active":"2024-05-08T23:21:05.093945+0000","last_peered":"2024-05-08T23:21:05.093945+0000","last_clean":"2024-05-08T23:21:05.093945+0000","last_became_active":"2024-05-08T23:21:04.113525+0000","last_became_peered":"2024-05-08T23:21:04.113525+0000","last_unstale":"2024-05-08T23:21:05.093945+0000","last_undegraded":"2024-05-08T23:21:05.093945+0000","last_fullsized":"2024-05-08T23:21:05.093945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356124+0000","last_change":"2024-05-08T23:21:04.100510+0000","last_active":"2024-05-08T23:21:05.356124+0000","last_peered":"2024-05-08T23:21:05.356124+0000","last_clean":"2024-05-08T23:21:05.356124+0000","last_became_active":"2024-05-08T23:21:04.100227+0000","last_became_peered":"2024-05-08T23:21:04.100227+0000","last_unstale":"2024-05-08T23:21:05.356124+0000","last_undegraded":"2024-05-08T23:21:05.356124+0000","last_fullsized":"2024-05-08T23:21:05.356124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.368136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356793+0000","last_change":"2024-05-08T23:21:04.110918+0000","last_active":"2024-05-08T23:21:05.356793+0000","last_peered":"2024-05-08T23:21:05.356793+0000","last_clean":"2024-05-08T23:21:05.356793+0000","last_became_active":"2024-05-08T23:21:04.110450+0000","last_became_peered":"2024-05-08T23:21:04.110450+0000","last_unstale":"2024-05-08T23:21:05.356793+0000","last_undegraded":"2024-05-08T23:21:05.356793+0000","last_fullsized":"2024-05-08T23:21:05.356793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356803+0000","last_change":"2024-05-08T23:21:04.113276+0000","last_active":"2024-05-08T23:21:05.356803+0000","last_peered":"2024-05-08T23:21:05.356803+0000","last_clean":"2024-05-08T23:21:05.356803+0000","last_became_active":"2024-05-08T23:21:04.112995+0000","last_became_peered":"2024-05-08T23:21:04.112995+0000","last_unstale":"2024-05-08T23:21:05.356803+0000","last_undegraded":"2024-05-08T23:21:05.356803+0000","last_fullsized":"2024-05-08T23:21:05.356803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356039+0000","last_change":"2024-05-08T23:21:04.106788+0000","last_active":"2024-05-08T23:21:05.356039+0000","last_peered":"2024-05-08T23:21:05.356039+0000","last_clean":"2024-05-08T23:21:05.356039+0000","last_became_active":"2024-05-08T23:21:04.106459+0000","last_became_peered":"2024-05-08T23:21:04.106459+0000","last_unstale":"2024-05-08T23:21:05.356039+0000","last_undegraded":"2024-05-08T23:21:05.356039+0000","last_fullsized":"2024-05-08T23:21:05.356039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356576+0000","last_change":"2024-05-08T23:21:04.112766+0000","last_active":"2024-05-08T23:21:05.356576+0000","last_peered":"2024-05-08T23:21:05.356576+0000","last_clean":"2024-05-08T23:21:05.356576+0000","last_became_active":"2024-05-08T23:21:04.112455+0000","last_became_peered":"2024-05-08T23:21:04.112455+0000","last_unstale":"2024-05-08T23:21:05.356576+0000","last_undegraded":"2024-05-08T23:21:05.356576+0000","last_fullsized":"2024-05-08T23:21:05.356576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:16.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092125+0000","last_change":"2024-05-08T23:21:04.103185+0000","last_active":"2024-05-08T23:21:05.092125+0000","last_peered":"2024-05-08T23:21:05.092125+0000","last_clean":"2024-05-08T23:21:05.092125+0000","last_became_active":"2024-05-08T23:21:04.102828+0000","last_became_peered":"2024-05-08T23:21:04.102828+0000","last_unstale":"2024-05-08T23:21:05.092125+0000","last_undegraded":"2024-05-08T23:21:05.092125+0000","last_fullsized":"2024-05-08T23:21:05.092125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356537+0000","last_change":"2024-05-08T23:21:04.122716+0000","last_active":"2024-05-08T23:21:05.356537+0000","last_peered":"2024-05-08T23:21:05.356537+0000","last_clean":"2024-05-08T23:21:05.356537+0000","last_became_active":"2024-05-08T23:21:04.122410+0000","last_became_peered":"2024-05-08T23:21:04.122410+0000","last_unstale":"2024-05-08T23:21:05.356537+0000","last_undegraded":"2024-05-08T23:21:05.356537+0000","last_fullsized":"2024-05-08T23:21:05.356537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:42.560490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355980+0000","last_change":"2024-05-08T23:21:04.111916+0000","last_active":"2024-05-08T23:21:05.355980+0000","last_peered":"2024-05-08T23:21:05.355980+0000","last_clean":"2024-05-08T23:21:05.355980+0000","last_became_active":"2024-05-08T23:21:04.111520+0000","last_became_peered":"2024-05-08T23:21:04.111520+0000","last_unstale":"2024-05-08T23:21:05.355980+0000","last_undegraded":"2024-05-08T23:21:05.355980+0000","last_fullsized":"2024-05-08T23:21:05.355980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355636+0000","last_change":"2024-05-08T23:21:04.113128+0000","last_active":"2024-05-08T23:21:05.355636+0000","last_peered":"2024-05-08T23:21:05.355636+0000","last_clean":"2024-05-08T23:21:05.355636+0000","last_became_active":"2024-05-08T23:21:04.112746+0000","last_became_peered":"2024-05-08T23:21:04.112746+0000","last_unstale":"2024-05-08T23:21:05.355636+0000","last_undegraded":"2024-05-08T23:21:05.355636+0000","last_fullsized":"2024-05-08T23:21:05.355636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:07.596384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093836+0000","last_change":"2024-05-08T23:21:04.118631+0000","last_active":"2024-05-08T23:21:05.093836+0000","last_peered":"2024-05-08T23:21:05.093836+0000","last_clean":"2024-05-08T23:21:05.093836+0000","last_became_active":"2024-05-08T23:21:04.118285+0000","last_became_peered":"2024-05-08T23:21:04.118285+0000","last_unstale":"2024-05-08T23:21:05.093836+0000","last_undegraded":"2024-05-08T23:21:05.093836+0000","last_fullsized":"2024-05-08T23:21:05.093836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.260488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092563+0000","last_change":"2024-05-08T23:21:04.114717+0000","last_active":"2024-05-08T23:21:05.092563+0000","last_peered":"2024-05-08T23:21:05.092563+0000","last_clean":"2024-05-08T23:21:05.092563+0000","last_became_active":"2024-05-08T23:21:04.114456+0000","last_became_peered":"2024-05-08T23:21:04.114456+0000","last_unstale":"2024-05-08T23:21:05.092563+0000","last_undegraded":"2024-05-08T23:21:05.092563+0000","last_fullsized":"2024-05-08T23:21:05.092563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.119343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355161+0000","last_change":"2024-05-08T23:21:04.110105+0000","last_active":"2024-05-08T23:21:05.355161+0000","last_peered":"2024-05-08T23:21:05.355161+0000","last_clean":"2024-05-08T23:21:05.355161+0000","last_became_active":"2024-05-08T23:21:04.109818+0000","last_became_peered":"2024-05-08T23:21:04.109818+0000","last_unstale":"2024-05-08T23:21:05.355161+0000","last_undegraded":"2024-05-08T23:21:05.355161+0000","last_fullsized":"2024-05-08T23:21:05.355161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093437+0000","last_change":"2024-05-08T23:21:04.114173+0000","last_active":"2024-05-08T23:21:05.093437+0000","last_peered":"2024-05-08T23:21:05.093437+0000","last_clean":"2024-05-08T23:21:05.093437+0000","last_became_active":"2024-05-08T23:21:04.108376+0000","last_became_peered":"2024-05-08T23:21:04.108376+0000","last_unstale":"2024-05-08T23:21:05.093437+0000","last_undegraded":"2024-05-08T23:21:05.093437+0000","last_fullsized":"2024-05-08T23:21:05.093437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355476+0000","last_change":"2024-05-08T23:21:04.100458+0000","last_active":"2024-05-08T23:21:05.355476+0000","last_peered":"2024-05-08T23:21:05.355476+0000","last_clean":"2024-05-08T23:21:05.355476+0000","last_became_active":"2024-05-08T23:21:04.100166+0000","last_became_peered":"2024-05-08T23:21:04.100166+0000","last_unstale":"2024-05-08T23:21:05.355476+0000","last_undegraded":"2024-05-08T23:21:05.355476+0000","last_fullsized":"2024-05-08T23:21:05.355476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357377+0000","last_change":"2024-05-08T23:21:04.109204+0000","last_active":"2024-05-08T23:21:05.357377+0000","last_peered":"2024-05-08T23:21:05.357377+0000","last_clean":"2024-05-08T23:21:05.357377+0000","last_became_active":"2024-05-08T23:21:04.108507+0000","last_became_peered":"2024-05-08T23:21:04.108507+0000","last_unstale":"2024-05-08T23:21:05.357377+0000","last_undegraded":"2024-05-08T23:21:05.357377+0000","last_fullsized":"2024-05-08T23:21:05.357377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:16.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355050+0000","last_change":"2024-05-08T23:21:04.114847+0000","last_active":"2024-05-08T23:21:05.355050+0000","last_peered":"2024-05-08T23:21:05.355050+0000","last_clean":"2024-05-08T23:21:05.355050+0000","last_became_active":"2024-05-08T23:21:04.114090+0000","last_became_peered":"2024-05-08T23:21:04.114090+0000","last_unstale":"2024-05-08T23:21:05.355050+0000","last_undegraded":"2024-05-08T23:21:05.355050+0000","last_fullsized":"2024-05-08T23:21:05.355050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.260488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356040+0000","last_change":"2024-05-08T23:21:04.113465+0000","last_active":"2024-05-08T23:21:05.356040+0000","last_peered":"2024-05-08T23:21:05.356040+0000","last_clean":"2024-05-08T23:21:05.356040+0000","last_became_active":"2024-05-08T23:21:04.113179+0000","last_became_peered":"2024-05-08T23:21:04.113179+0000","last_unstale":"2024-05-08T23:21:05.356040+0000","last_undegraded":"2024-05-08T23:21:05.356040+0000","last_fullsized":"2024-05-08T23:21:05.356040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355379+0000","last_change":"2024-05-08T23:21:03.646371+0000","last_active":"2024-05-08T23:21:05.355379+0000","last_peered":"2024-05-08T23:21:05.355379+0000","last_clean":"2024-05-08T23:21:05.355379+0000","last_became_active":"2024-05-08T23:21:03.646004+0000","last_became_peered":"2024-05-08T23:21:03.646004+0000","last_unstale":"2024-05-08T23:21:05.355379+0000","last_undegraded":"2024-05-08T23:21:05.355379+0000","last_fullsized":"2024-05-08T23:21:05.355379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:32.482556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092991+0000","last_change":"2024-05-08T23:21:04.115651+0000","last_active":"2024-05-08T23:21:05.092991+0000","last_peered":"2024-05-08T23:21:05.092991+0000","last_clean":"2024-05-08T23:21:05.092991+0000","last_became_active":"2024-05-08T23:21:04.115384+0000","last_became_peered":"2024-05-08T23:21:04.115384+0000","last_unstale":"2024-05-08T23:21:05.092991+0000","last_undegraded":"2024-05-08T23:21:05.092991+0000","last_fullsized":"2024-05-08T23:21:05.092991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093060+0000","last_change":"2024-05-08T23:21:03.645350+0000","last_active":"2024-05-08T23:21:05.093060+0000","last_peered":"2024-05-08T23:21:05.093060+0000","last_clean":"2024-05-08T23:21:05.093060+0000","last_became_active":"2024-05-08T23:21:03.645081+0000","last_became_peered":"2024-05-08T23:21:03.645081+0000","last_unstale":"2024-05-08T23:21:05.093060+0000","last_undegraded":"2024-05-08T23:21:05.093060+0000","last_fullsized":"2024-05-08T23:21:05.093060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355362+0000","last_change":"2024-05-08T23:21:04.112500+0000","last_active":"2024-05-08T23:21:05.355362+0000","last_peered":"2024-05-08T23:21:05.355362+0000","last_clean":"2024-05-08T23:21:05.355362+0000","last_became_active":"2024-05-08T23:21:04.111988+0000","last_became_peered":"2024-05-08T23:21:04.111988+0000","last_unstale":"2024-05-08T23:21:05.355362+0000","last_undegraded":"2024-05-08T23:21:05.355362+0000","last_fullsized":"2024-05-08T23:21:05.355362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356422+0000","last_change":"2024-05-08T23:21:03.647784+0000","last_active":"2024-05-08T23:21:05.356422+0000","last_peered":"2024-05-08T23:21:05.356422+0000","last_clean":"2024-05-08T23:21:05.356422+0000","last_became_active":"2024-05-08T23:21:03.646667+0000","last_became_peered":"2024-05-08T23:21:03.646667+0000","last_unstale":"2024-05-08T23:21:05.356422+0000","last_undegraded":"2024-05-08T23:21:05.356422+0000","last_fullsized":"2024-05-08T23:21:05.356422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355900+0000","last_change":"2024-05-08T23:21:04.104612+0000","last_active":"2024-05-08T23:21:05.355900+0000","last_peered":"2024-05-08T23:21:05.355900+0000","last_clean":"2024-05-08T23:21:05.355900+0000","last_became_active":"2024-05-08T23:21:04.104304+0000","last_became_peered":"2024-05-08T23:21:04.104304+0000","last_unstale":"2024-05-08T23:21:05.355900+0000","last_undegraded":"2024-05-08T23:21:05.355900+0000","last_fullsized":"2024-05-08T23:21:05.355900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356075+0000","last_change":"2024-05-08T23:21:03.648270+0000","last_active":"2024-05-08T23:21:05.356075+0000","last_peered":"2024-05-08T23:21:05.356075+0000","last_clean":"2024-05-08T23:21:05.356075+0000","last_became_active":"2024-05-08T23:21:03.647784+0000","last_became_peered":"2024-05-08T23:21:03.647784+0000","last_unstale":"2024-05-08T23:21:05.356075+0000","last_undegraded":"2024-05-08T23:21:05.356075+0000","last_fullsized":"2024-05-08T23:21:05.356075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357179+0000","last_change":"2024-05-08T23:21:04.106021+0000","last_active":"2024-05-08T23:21:05.357179+0000","last_peered":"2024-05-08T23:21:05.357179+0000","last_clean":"2024-05-08T23:21:05.357179+0000","last_became_active":"2024-05-08T23:21:04.105614+0000","last_became_peered":"2024-05-08T23:21:04.105614+0000","last_unstale":"2024-05-08T23:21:05.357179+0000","last_undegraded":"2024-05-08T23:21:05.357179+0000","last_fullsized":"2024-05-08T23:21:05.357179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.119343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092611+0000","last_change":"2024-05-08T23:21:03.642600+0000","last_active":"2024-05-08T23:21:05.092611+0000","last_peered":"2024-05-08T23:21:05.092611+0000","last_clean":"2024-05-08T23:21:05.092611+0000","last_became_active":"2024-05-08T23:21:03.642234+0000","last_became_peered":"2024-05-08T23:21:03.642234+0000","last_unstale":"2024-05-08T23:21:05.092611+0000","last_undegraded":"2024-05-08T23:21:05.092611+0000","last_fullsized":"2024-05-08T23:21:05.092611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093418+0000","last_change":"2024-05-08T23:21:04.117783+0000","last_active":"2024-05-08T23:21:05.093418+0000","last_peered":"2024-05-08T23:21:05.093418+0000","last_clean":"2024-05-08T23:21:05.093418+0000","last_became_active":"2024-05-08T23:21:04.117324+0000","last_became_peered":"2024-05-08T23:21:04.117324+0000","last_unstale":"2024-05-08T23:21:05.093418+0000","last_undegraded":"2024-05-08T23:21:05.093418+0000","last_fullsized":"2024-05-08T23:21:05.093418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.433453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355523+0000","last_change":"2024-05-08T23:21:03.644641+0000","last_active":"2024-05-08T23:21:05.355523+0000","last_peered":"2024-05-08T23:21:05.355523+0000","last_clean":"2024-05-08T23:21:05.355523+0000","last_became_active":"2024-05-08T23:21:03.644361+0000","last_became_peered":"2024-05-08T23:21:03.644361+0000","last_unstale":"2024-05-08T23:21:05.355523+0000","last_undegraded":"2024-05-08T23:21:05.355523+0000","last_fullsized":"2024-05-08T23:21:05.355523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355562+0000","last_change":"2024-05-08T23:21:04.105553+0000","last_active":"2024-05-08T23:21:05.355562+0000","last_peered":"2024-05-08T23:21:05.355562+0000","last_clean":"2024-05-08T23:21:05.355562+0000","last_became_active":"2024-05-08T23:21:04.105041+0000","last_became_peered":"2024-05-08T23:21:04.105041+0000","last_unstale":"2024-05-08T23:21:05.355562+0000","last_undegraded":"2024-05-08T23:21:05.355562+0000","last_fullsized":"2024-05-08T23:21:05.355562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.311829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092401+0000","last_change":"2024-05-08T23:21:03.639910+0000","last_active":"2024-05-08T23:21:05.092401+0000","last_peered":"2024-05-08T23:21:05.092401+0000","last_clean":"2024-05-08T23:21:05.092401+0000","last_became_active":"2024-05-08T23:21:03.638934+0000","last_became_peered":"2024-05-08T23:21:03.638934+0000","last_unstale":"2024-05-08T23:21:05.092401+0000","last_undegraded":"2024-05-08T23:21:05.092401+0000","last_fullsized":"2024-05-08T23:21:05.092401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092248+0000","last_change":"2024-05-08T23:21:04.108315+0000","last_active":"2024-05-08T23:21:05.092248+0000","last_peered":"2024-05-08T23:21:05.092248+0000","last_clean":"2024-05-08T23:21:05.092248+0000","last_became_active":"2024-05-08T23:21:04.108040+0000","last_became_peered":"2024-05-08T23:21:04.108040+0000","last_unstale":"2024-05-08T23:21:05.092248+0000","last_undegraded":"2024-05-08T23:21:05.092248+0000","last_fullsized":"2024-05-08T23:21:05.092248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355919+0000","last_change":"2024-05-08T23:21:03.641788+0000","last_active":"2024-05-08T23:21:05.355919+0000","last_peered":"2024-05-08T23:21:05.355919+0000","last_clean":"2024-05-08T23:21:05.355919+0000","last_became_active":"2024-05-08T23:21:03.641004+0000","last_became_peered":"2024-05-08T23:21:03.641004+0000","last_unstale":"2024-05-08T23:21:05.355919+0000","last_undegraded":"2024-05-08T23:21:05.355919+0000","last_fullsized":"2024-05-08T23:21:05.355919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093537+0000","last_change":"2024-05-08T23:21:04.114572+0000","last_active":"2024-05-08T23:21:05.093537+0000","last_peered":"2024-05-08T23:21:05.093537+0000","last_clean":"2024-05-08T23:21:05.093537+0000","last_became_active":"2024-05-08T23:21:04.114402+0000","last_became_peered":"2024-05-08T23:21:04.114402+0000","last_unstale":"2024-05-08T23:21:05.093537+0000","last_undegraded":"2024-05-08T23:21:05.093537+0000","last_fullsized":"2024-05-08T23:21:05.093537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356964+0000","last_change":"2024-05-08T23:21:03.648330+0000","last_active":"2024-05-08T23:21:05.356964+0000","last_peered":"2024-05-08T23:21:05.356964+0000","last_clean":"2024-05-08T23:21:05.356964+0000","last_became_active":"2024-05-08T23:21:03.648033+0000","last_became_peered":"2024-05-08T23:21:03.648033+0000","last_unstale":"2024-05-08T23:21:05.356964+0000","last_undegraded":"2024-05-08T23:21:05.356964+0000","last_fullsized":"2024-05-08T23:21:05.356964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356917+0000","last_change":"2024-05-08T23:21:04.108163+0000","last_active":"2024-05-08T23:21:05.356917+0000","last_peered":"2024-05-08T23:21:05.356917+0000","last_clean":"2024-05-08T23:21:05.356917+0000","last_became_active":"2024-05-08T23:21:04.107854+0000","last_became_peered":"2024-05-08T23:21:04.107854+0000","last_unstale":"2024-05-08T23:21:05.356917+0000","last_undegraded":"2024-05-08T23:21:05.356917+0000","last_fullsized":"2024-05-08T23:21:05.356917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:09.028590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356841+0000","last_change":"2024-05-08T23:21:03.640432+0000","last_active":"2024-05-08T23:21:05.356841+0000","last_peered":"2024-05-08T23:21:05.356841+0000","last_clean":"2024-05-08T23:21:05.356841+0000","last_became_active":"2024-05-08T23:21:03.640121+0000","last_became_peered":"2024-05-08T23:21:03.640121+0000","last_unstale":"2024-05-08T23:21:05.356841+0000","last_undegraded":"2024-05-08T23:21:05.356841+0000","last_fullsized":"2024-05-08T23:21:05.356841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355475+0000","last_change":"2024-05-08T23:21:04.113136+0000","last_active":"2024-05-08T23:21:05.355475+0000","last_peered":"2024-05-08T23:21:05.355475+0000","last_clean":"2024-05-08T23:21:05.355475+0000","last_became_active":"2024-05-08T23:21:04.112831+0000","last_became_peered":"2024-05-08T23:21:04.112831+0000","last_unstale":"2024-05-08T23:21:05.355475+0000","last_undegraded":"2024-05-08T23:21:05.355475+0000","last_fullsized":"2024-05-08T23:21:05.355475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:03.946264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093793+0000","last_change":"2024-05-08T23:21:03.647271+0000","last_active":"2024-05-08T23:21:05.093793+0000","last_peered":"2024-05-08T23:21:05.093793+0000","last_clean":"2024-05-08T23:21:05.093793+0000","last_became_active":"2024-05-08T23:21:03.646992+0000","last_became_peered":"2024-05-08T23:21:03.646992+0000","last_unstale":"2024-05-08T23:21:05.093793+0000","last_undegraded":"2024-05-08T23:21:05.093793+0000","last_fullsized":"2024-05-08T23:21:05.093793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:43.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093761+0000","last_change":"2024-05-08T23:21:04.106622+0000","last_active":"2024-05-08T23:21:05.093761+0000","last_peered":"2024-05-08T23:21:05.093761+0000","last_clean":"2024-05-08T23:21:05.093761+0000","last_became_active":"2024-05-08T23:21:04.106248+0000","last_became_peered":"2024-05-08T23:21:04.106248+0000","last_unstale":"2024-05-08T23:21:05.093761+0000","last_undegraded":"2024-05-08T23:21:05.093761+0000","last_fullsized":"2024-05-08T23:21:05.093761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.311829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356428+0000","last_change":"2024-05-08T23:21:03.642565+0000","last_active":"2024-05-08T23:21:05.356428+0000","last_peered":"2024-05-08T23:21:05.356428+0000","last_clean":"2024-05-08T23:21:05.356428+0000","last_became_active":"2024-05-08T23:21:03.642253+0000","last_became_peered":"2024-05-08T23:21:03.642253+0000","last_unstale":"2024-05-08T23:21:05.356428+0000","last_undegraded":"2024-05-08T23:21:05.356428+0000","last_fullsized":"2024-05-08T23:21:05.356428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356171+0000","last_change":"2024-05-08T23:21:04.113888+0000","last_active":"2024-05-08T23:21:05.356171+0000","last_peered":"2024-05-08T23:21:05.356171+0000","last_clean":"2024-05-08T23:21:05.356171+0000","last_became_active":"2024-05-08T23:21:04.113607+0000","last_became_peered":"2024-05-08T23:21:04.113607+0000","last_unstale":"2024-05-08T23:21:05.356171+0000","last_undegraded":"2024-05-08T23:21:05.356171+0000","last_fullsized":"2024-05-08T23:21:05.356171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:19.795476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355509+0000","last_change":"2024-05-08T23:21:03.641751+0000","last_active":"2024-05-08T23:21:05.355509+0000","last_peered":"2024-05-08T23:21:05.355509+0000","last_clean":"2024-05-08T23:21:05.355509+0000","last_became_active":"2024-05-08T23:21:03.641441+0000","last_became_peered":"2024-05-08T23:21:03.641441+0000","last_unstale":"2024-05-08T23:21:05.355509+0000","last_undegraded":"2024-05-08T23:21:05.355509+0000","last_fullsized":"2024-05-08T23:21:05.355509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:58.822252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355038+0000","last_change":"2024-05-08T23:21:04.106177+0000","last_active":"2024-05-08T23:21:05.355038+0000","last_peered":"2024-05-08T23:21:05.355038+0000","last_clean":"2024-05-08T23:21:05.355038+0000","last_became_active":"2024-05-08T23:21:04.105844+0000","last_became_peered":"2024-05-08T23:21:04.105844+0000","last_unstale":"2024-05-08T23:21:05.355038+0000","last_undegraded":"2024-05-08T23:21:05.355038+0000","last_fullsized":"2024-05-08T23:21:05.355038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356436+0000","last_change":"2024-05-08T23:21:03.646376+0000","last_active":"2024-05-08T23:21:05.356436+0000","last_peered":"2024-05-08T23:21:05.356436+0000","last_clean":"2024-05-08T23:21:05.356436+0000","last_became_active":"2024-05-08T23:21:03.646071+0000","last_became_peered":"2024-05-08T23:21:03.646071+0000","last_unstale":"2024-05-08T23:21:05.356436+0000","last_undegraded":"2024-05-08T23:21:05.356436+0000","last_fullsized":"2024-05-08T23:21:05.356436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356915+0000","last_change":"2024-05-08T23:21:04.120630+0000","last_active":"2024-05-08T23:21:05.356915+0000","last_peered":"2024-05-08T23:21:05.356915+0000","last_clean":"2024-05-08T23:21:05.356915+0000","last_became_active":"2024-05-08T23:21:04.120315+0000","last_became_peered":"2024-05-08T23:21:04.120315+0000","last_unstale":"2024-05-08T23:21:05.356915+0000","last_undegraded":"2024-05-08T23:21:05.356915+0000","last_fullsized":"2024-05-08T23:21:05.356915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:57.953372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356590+0000","last_change":"2024-05-08T23:21:03.645556+0000","last_active":"2024-05-08T23:21:05.356590+0000","last_peered":"2024-05-08T23:21:05.356590+0000","last_clean":"2024-05-08T23:21:05.356590+0000","last_became_active":"2024-05-08T23:21:03.645083+0000","last_became_peered":"2024-05-08T23:21:03.645083+0000","last_unstale":"2024-05-08T23:21:05.356590+0000","last_undegraded":"2024-05-08T23:21:05.356590+0000","last_fullsized":"2024-05-08T23:21:05.356590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355274+0000","last_change":"2024-05-08T23:21:04.108235+0000","last_active":"2024-05-08T23:21:05.355274+0000","last_peered":"2024-05-08T23:21:05.355274+0000","last_clean":"2024-05-08T23:21:05.355274+0000","last_became_active":"2024-05-08T23:21:04.107933+0000","last_became_peered":"2024-05-08T23:21:04.107933+0000","last_unstale":"2024-05-08T23:21:05.355274+0000","last_undegraded":"2024-05-08T23:21:05.355274+0000","last_fullsized":"2024-05-08T23:21:05.355274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354849+0000","last_change":"2024-05-08T23:21:03.636549+0000","last_active":"2024-05-08T23:21:05.354849+0000","last_peered":"2024-05-08T23:21:05.354849+0000","last_clean":"2024-05-08T23:21:05.354849+0000","last_became_active":"2024-05-08T23:21:03.636259+0000","last_became_peered":"2024-05-08T23:21:03.636259+0000","last_unstale":"2024-05-08T23:21:05.354849+0000","last_undegraded":"2024-05-08T23:21:05.354849+0000","last_fullsized":"2024-05-08T23:21:05.354849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356690+0000","last_change":"2024-05-08T23:21:04.113963+0000","last_active":"2024-05-08T23:21:05.356690+0000","last_peered":"2024-05-08T23:21:05.356690+0000","last_clean":"2024-05-08T23:21:05.356690+0000","last_became_active":"2024-05-08T23:21:04.113645+0000","last_became_peered":"2024-05-08T23:21:04.113645+0000","last_unstale":"2024-05-08T23:21:05.356690+0000","last_undegraded":"2024-05-08T23:21:05.356690+0000","last_fullsized":"2024-05-08T23:21:05.356690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093744+0000","last_change":"2024-05-08T23:21:03.644725+0000","last_active":"2024-05-08T23:21:05.093744+0000","last_peered":"2024-05-08T23:21:05.093744+0000","last_clean":"2024-05-08T23:21:05.093744+0000","last_became_active":"2024-05-08T23:21:03.644432+0000","last_became_peered":"2024-05-08T23:21:03.644432+0000","last_unstale":"2024-05-08T23:21:05.093744+0000","last_undegraded":"2024-05-08T23:21:05.093744+0000","last_fullsized":"2024-05-08T23:21:05.093744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:58.822252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356426+0000","last_change":"2024-05-08T23:21:04.116753+0000","last_active":"2024-05-08T23:21:05.356426+0000","last_peered":"2024-05-08T23:21:05.356426+0000","last_clean":"2024-05-08T23:21:05.356426+0000","last_became_active":"2024-05-08T23:21:04.116366+0000","last_became_peered":"2024-05-08T23:21:04.116366+0000","last_unstale":"2024-05-08T23:21:05.356426+0000","last_undegraded":"2024-05-08T23:21:05.356426+0000","last_fullsized":"2024-05-08T23:21:05.356426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093071+0000","last_change":"2024-05-08T23:21:03.643882+0000","last_active":"2024-05-08T23:21:05.093071+0000","last_peered":"2024-05-08T23:21:05.093071+0000","last_clean":"2024-05-08T23:21:05.093071+0000","last_became_active":"2024-05-08T23:21:03.643533+0000","last_became_peered":"2024-05-08T23:21:03.643533+0000","last_unstale":"2024-05-08T23:21:05.093071+0000","last_undegraded":"2024-05-08T23:21:05.093071+0000","last_fullsized":"2024-05-08T23:21:05.093071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357279+0000","last_change":"2024-05-08T23:21:04.109134+0000","last_active":"2024-05-08T23:21:05.357279+0000","last_peered":"2024-05-08T23:21:05.357279+0000","last_clean":"2024-05-08T23:21:05.357279+0000","last_became_active":"2024-05-08T23:21:04.108799+0000","last_became_peered":"2024-05-08T23:21:04.108799+0000","last_unstale":"2024-05-08T23:21:05.357279+0000","last_undegraded":"2024-05-08T23:21:05.357279+0000","last_fullsized":"2024-05-08T23:21:05.357279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356346+0000","last_change":"2024-05-08T23:21:03.644161+0000","last_active":"2024-05-08T23:21:05.356346+0000","last_peered":"2024-05-08T23:21:05.356346+0000","last_clean":"2024-05-08T23:21:05.356346+0000","last_became_active":"2024-05-08T23:21:03.643341+0000","last_became_peered":"2024-05-08T23:21:03.643341+0000","last_unstale":"2024-05-08T23:21:05.356346+0000","last_undegraded":"2024-05-08T23:21:05.356346+0000","last_fullsized":"2024-05-08T23:21:05.356346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093723+0000","last_change":"2024-05-08T23:21:04.108965+0000","last_active":"2024-05-08T23:21:05.093723+0000","last_peered":"2024-05-08T23:21:05.093723+0000","last_clean":"2024-05-08T23:21:05.093723+0000","last_became_active":"2024-05-08T23:21:04.107917+0000","last_became_peered":"2024-05-08T23:21:04.107917+0000","last_unstale":"2024-05-08T23:21:05.093723+0000","last_undegraded":"2024-05-08T23:21:05.093723+0000","last_fullsized":"2024-05-08T23:21:05.093723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:00.670245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093185+0000","last_change":"2024-05-08T23:21:04.115959+0000","last_active":"2024-05-08T23:21:05.093185+0000","last_peered":"2024-05-08T23:21:05.093185+0000","last_clean":"2024-05-08T23:21:05.093185+0000","last_became_active":"2024-05-08T23:21:04.115696+0000","last_became_peered":"2024-05-08T23:21:04.115696+0000","last_unstale":"2024-05-08T23:21:05.093185+0000","last_undegraded":"2024-05-08T23:21:05.093185+0000","last_fullsized":"2024-05-08T23:21:05.093185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354856+0000","last_change":"2024-05-08T23:21:03.632217+0000","last_active":"2024-05-08T23:21:05.354856+0000","last_peered":"2024-05-08T23:21:05.354856+0000","last_clean":"2024-05-08T23:21:05.354856+0000","last_became_active":"2024-05-08T23:21:03.631883+0000","last_became_peered":"2024-05-08T23:21:03.631883+0000","last_unstale":"2024-05-08T23:21:05.354856+0000","last_undegraded":"2024-05-08T23:21:05.354856+0000","last_fullsized":"2024-05-08T23:21:05.354856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092402+0000","last_change":"2024-05-08T23:21:04.104594+0000","last_active":"2024-05-08T23:21:05.092402+0000","last_peered":"2024-05-08T23:21:05.092402+0000","last_clean":"2024-05-08T23:21:05.092402+0000","last_became_active":"2024-05-08T23:21:04.104335+0000","last_became_peered":"2024-05-08T23:21:04.104335+0000","last_unstale":"2024-05-08T23:21:05.092402+0000","last_undegraded":"2024-05-08T23:21:05.092402+0000","last_fullsized":"2024-05-08T23:21:05.092402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355796+0000","last_change":"2024-05-08T23:21:03.637311+0000","last_active":"2024-05-08T23:21:05.355796+0000","last_peered":"2024-05-08T23:21:05.355796+0000","last_clean":"2024-05-08T23:21:05.355796+0000","last_became_active":"2024-05-08T23:21:03.636529+0000","last_became_peered":"2024-05-08T23:21:03.636529+0000","last_unstale":"2024-05-08T23:21:05.355796+0000","last_undegraded":"2024-05-08T23:21:05.355796+0000","last_fullsized":"2024-05-08T23:21:05.355796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-08T23:21:04.101711+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:21:04.101711+0000","last_peered":"2024-05-08T23:21:04.101711+0000","last_clean":"2024-05-08T23:21:04.101711+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:21:04.101711+0000","last_undegraded":"2024-05-08T23:21:04.101711+0000","last_fullsized":"2024-05-08T23:21:04.101711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355927+0000","last_change":"2024-05-08T23:21:04.108232+0000","last_active":"2024-05-08T23:21:05.355927+0000","last_peered":"2024-05-08T23:21:05.355927+0000","last_clean":"2024-05-08T23:21:05.355927+0000","last_became_active":"2024-05-08T23:21:04.107908+0000","last_became_peered":"2024-05-08T23:21:04.107908+0000","last_unstale":"2024-05-08T23:21:05.355927+0000","last_undegraded":"2024-05-08T23:21:05.355927+0000","last_fullsized":"2024-05-08T23:21:05.355927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:17.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356470+0000","last_change":"2024-05-08T23:21:03.645567+0000","last_active":"2024-05-08T23:21:05.356470+0000","last_peered":"2024-05-08T23:21:05.356470+0000","last_clean":"2024-05-08T23:21:05.356470+0000","last_became_active":"2024-05-08T23:21:03.644844+0000","last_became_peered":"2024-05-08T23:21:03.644844+0000","last_unstale":"2024-05-08T23:21:05.356470+0000","last_undegraded":"2024-05-08T23:21:05.356470+0000","last_fullsized":"2024-05-08T23:21:05.356470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:43.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356283+0000","last_change":"2024-05-08T23:21:03.646987+0000","last_active":"2024-05-08T23:21:05.356283+0000","last_peered":"2024-05-08T23:21:05.356283+0000","last_clean":"2024-05-08T23:21:05.356283+0000","last_became_active":"2024-05-08T23:21:03.646283+0000","last_became_peered":"2024-05-08T23:21:03.646283+0000","last_unstale":"2024-05-08T23:21:05.356283+0000","last_undegraded":"2024-05-08T23:21:05.356283+0000","last_fullsized":"2024-05-08T23:21:05.356283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:59.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092960+0000","last_change":"2024-05-08T23:21:04.106165+0000","last_active":"2024-05-08T23:21:05.092960+0000","last_peered":"2024-05-08T23:21:05.092960+0000","last_clean":"2024-05-08T23:21:05.092960+0000","last_became_active":"2024-05-08T23:21:04.105879+0000","last_became_peered":"2024-05-08T23:21:04.105879+0000","last_unstale":"2024-05-08T23:21:05.092960+0000","last_undegraded":"2024-05-08T23:21:05.092960+0000","last_fullsized":"2024-05-08T23:21:05.092960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.368136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:07.296809+0000","last_change":"2024-05-08T23:21:03.638777+0000","last_active":"2024-05-08T23:21:07.296809+0000","last_peered":"2024-05-08T23:21:07.296809+0000","last_clean":"2024-05-08T23:21:07.296809+0000","last_became_active":"2024-05-08T23:21:03.638494+0000","last_became_peered":"2024-05-08T23:21:03.638494+0000","last_unstale":"2024-05-08T23:21:07.296809+0000","last_undegraded":"2024-05-08T23:21:07.296809+0000","last_fullsized":"2024-05-08T23:21:07.296809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093027+0000","last_change":"2024-05-08T23:21:04.113442+0000","last_active":"2024-05-08T23:21:05.093027+0000","last_peered":"2024-05-08T23:21:05.093027+0000","last_clean":"2024-05-08T23:21:05.093027+0000","last_became_active":"2024-05-08T23:21:04.113144+0000","last_became_peered":"2024-05-08T23:21:04.113144+0000","last_unstale":"2024-05-08T23:21:05.093027+0000","last_undegraded":"2024-05-08T23:21:05.093027+0000","last_fullsized":"2024-05-08T23:21:05.093027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355626+0000","last_change":"2024-05-08T23:21:03.644120+0000","last_active":"2024-05-08T23:21:05.355626+0000","last_peered":"2024-05-08T23:21:05.355626+0000","last_clean":"2024-05-08T23:21:05.355626+0000","last_became_active":"2024-05-08T23:21:03.643541+0000","last_became_peered":"2024-05-08T23:21:03.643541+0000","last_unstale":"2024-05-08T23:21:05.355626+0000","last_undegraded":"2024-05-08T23:21:05.355626+0000","last_fullsized":"2024-05-08T23:21:05.355626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355406+0000","last_change":"2024-05-08T23:21:04.115769+0000","last_active":"2024-05-08T23:21:05.355406+0000","last_peered":"2024-05-08T23:21:05.355406+0000","last_clean":"2024-05-08T23:21:05.355406+0000","last_became_active":"2024-05-08T23:21:04.115446+0000","last_became_peered":"2024-05-08T23:21:04.115446+0000","last_unstale":"2024-05-08T23:21:05.355406+0000","last_undegraded":"2024-05-08T23:21:05.355406+0000","last_fullsized":"2024-05-08T23:21:05.355406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356111+0000","last_change":"2024-05-08T23:21:03.647284+0000","last_active":"2024-05-08T23:21:05.356111+0000","last_peered":"2024-05-08T23:21:05.356111+0000","last_clean":"2024-05-08T23:21:05.356111+0000","last_became_active":"2024-05-08T23:21:03.645828+0000","last_became_peered":"2024-05-08T23:21:03.645828+0000","last_unstale":"2024-05-08T23:21:05.356111+0000","last_undegraded":"2024-05-08T23:21:05.356111+0000","last_fullsized":"2024-05-08T23:21:05.356111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:54.428258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093694+0000","last_change":"2024-05-08T23:21:04.105901+0000","last_active":"2024-05-08T23:21:05.093694+0000","last_peered":"2024-05-08T23:21:05.093694+0000","last_clean":"2024-05-08T23:21:05.093694+0000","last_became_active":"2024-05-08T23:21:04.105495+0000","last_became_peered":"2024-05-08T23:21:04.105495+0000","last_unstale":"2024-05-08T23:21:05.093694+0000","last_undegraded":"2024-05-08T23:21:05.093694+0000","last_fullsized":"2024-05-08T23:21:05.093694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:44.288365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093485+0000","last_change":"2024-05-08T23:21:03.643905+0000","last_active":"2024-05-08T23:21:05.093485+0000","last_peered":"2024-05-08T23:21:05.093485+0000","last_clean":"2024-05-08T23:21:05.093485+0000","last_became_active":"2024-05-08T23:21:03.643555+0000","last_became_peered":"2024-05-08T23:21:03.643555+0000","last_unstale":"2024-05-08T23:21:05.093485+0000","last_undegraded":"2024-05-08T23:21:05.093485+0000","last_fullsized":"2024-05-08T23:21:05.093485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092205+0000","last_change":"2024-05-08T23:21:04.117123+0000","last_active":"2024-05-08T23:21:05.092205+0000","last_peered":"2024-05-08T23:21:05.092205+0000","last_clean":"2024-05-08T23:21:05.092205+0000","last_became_active":"2024-05-08T23:21:04.116830+0000","last_became_peered":"2024-05-08T23:21:04.116830+0000","last_unstale":"2024-05-08T23:21:05.092205+0000","last_undegraded":"2024-05-08T23:21:05.092205+0000","last_fullsized":"2024-05-08T23:21:05.092205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093648+0000","last_change":"2024-05-08T23:21:03.642867+0000","last_active":"2024-05-08T23:21:05.093648+0000","last_peered":"2024-05-08T23:21:05.093648+0000","last_clean":"2024-05-08T23:21:05.093648+0000","last_became_active":"2024-05-08T23:21:03.642439+0000","last_became_peered":"2024-05-08T23:21:03.642439+0000","last_unstale":"2024-05-08T23:21:05.093648+0000","last_undegraded":"2024-05-08T23:21:05.093648+0000","last_fullsized":"2024-05-08T23:21:05.093648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092672+0000","last_change":"2024-05-08T23:21:04.115928+0000","last_active":"2024-05-08T23:21:05.092672+0000","last_peered":"2024-05-08T23:21:05.092672+0000","last_clean":"2024-05-08T23:21:05.092672+0000","last_became_active":"2024-05-08T23:21:04.113858+0000","last_became_peered":"2024-05-08T23:21:04.113858+0000","last_unstale":"2024-05-08T23:21:05.092672+0000","last_undegraded":"2024-05-08T23:21:05.092672+0000","last_fullsized":"2024-05-08T23:21:05.092672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357022+0000","last_change":"2024-05-08T23:21:03.649228+0000","last_active":"2024-05-08T23:21:05.357022+0000","last_peered":"2024-05-08T23:21:05.357022+0000","last_clean":"2024-05-08T23:21:05.357022+0000","last_became_active":"2024-05-08T23:21:03.648868+0000","last_became_peered":"2024-05-08T23:21:03.648868+0000","last_unstale":"2024-05-08T23:21:05.357022+0000","last_undegraded":"2024-05-08T23:21:05.357022+0000","last_fullsized":"2024-05-08T23:21:05.357022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:34.255292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355422+0000","last_change":"2024-05-08T23:21:04.110982+0000","last_active":"2024-05-08T23:21:05.355422+0000","last_peered":"2024-05-08T23:21:05.355422+0000","last_clean":"2024-05-08T23:21:05.355422+0000","last_became_active":"2024-05-08T23:21:04.110643+0000","last_became_peered":"2024-05-08T23:21:04.110643+0000","last_unstale":"2024-05-08T23:21:05.355422+0000","last_undegraded":"2024-05-08T23:21:05.355422+0000","last_fullsized":"2024-05-08T23:21:05.355422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355712+0000","last_change":"2024-05-08T23:21:03.637418+0000","last_active":"2024-05-08T23:21:05.355712+0000","last_peered":"2024-05-08T23:21:05.355712+0000","last_clean":"2024-05-08T23:21:05.355712+0000","last_became_active":"2024-05-08T23:21:03.637139+0000","last_became_peered":"2024-05-08T23:21:03.637139+0000","last_unstale":"2024-05-08T23:21:05.355712+0000","last_undegraded":"2024-05-08T23:21:05.355712+0000","last_fullsized":"2024-05-08T23:21:05.355712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356254+0000","last_change":"2024-05-08T23:21:04.112240+0000","last_active":"2024-05-08T23:21:05.356254+0000","last_peered":"2024-05-08T23:21:05.356254+0000","last_clean":"2024-05-08T23:21:05.356254+0000","last_became_active":"2024-05-08T23:21:04.111914+0000","last_became_peered":"2024-05-08T23:21:04.111914+0000","last_unstale":"2024-05-08T23:21:05.356254+0000","last_undegraded":"2024-05-08T23:21:05.356254+0000","last_fullsized":"2024-05-08T23:21:05.356254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:09.028590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093821+0000","last_change":"2024-05-08T23:21:03.643712+0000","last_active":"2024-05-08T23:21:05.093821+0000","last_peered":"2024-05-08T23:21:05.093821+0000","last_clean":"2024-05-08T23:21:05.093821+0000","last_became_active":"2024-05-08T23:21:03.643387+0000","last_became_peered":"2024-05-08T23:21:03.643387+0000","last_unstale":"2024-05-08T23:21:05.093821+0000","last_undegraded":"2024-05-08T23:21:05.093821+0000","last_fullsized":"2024-05-08T23:21:05.093821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:59.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356035+0000","last_change":"2024-05-08T23:21:04.110728+0000","last_active":"2024-05-08T23:21:05.356035+0000","last_peered":"2024-05-08T23:21:05.356035+0000","last_clean":"2024-05-08T23:21:05.356035+0000","last_became_active":"2024-05-08T23:21:04.110094+0000","last_became_peered":"2024-05-08T23:21:04.110094+0000","last_unstale":"2024-05-08T23:21:05.356035+0000","last_undegraded":"2024-05-08T23:21:05.356035+0000","last_fullsized":"2024-05-08T23:21:05.356035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:30:28.180773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355130+0000","last_change":"2024-05-08T23:21:03.638990+0000","last_active":"2024-05-08T23:21:05.355130+0000","last_peered":"2024-05-08T23:21:05.355130+0000","last_clean":"2024-05-08T23:21:05.355130+0000","last_became_active":"2024-05-08T23:21:03.638696+0000","last_became_peered":"2024-05-08T23:21:03.638696+0000","last_unstale":"2024-05-08T23:21:05.355130+0000","last_undegraded":"2024-05-08T23:21:05.355130+0000","last_fullsized":"2024-05-08T23:21:05.355130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:16.993057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356830+0000","last_change":"2024-05-08T23:21:04.117976+0000","last_active":"2024-05-08T23:21:05.356830+0000","last_peered":"2024-05-08T23:21:05.356830+0000","last_clean":"2024-05-08T23:21:05.356830+0000","last_became_active":"2024-05-08T23:21:04.117682+0000","last_became_peered":"2024-05-08T23:21:04.117682+0000","last_unstale":"2024-05-08T23:21:05.356830+0000","last_undegraded":"2024-05-08T23:21:05.356830+0000","last_fullsized":"2024-05-08T23:21:05.356830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:30:28.180773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355265+0000","last_change":"2024-05-08T23:21:04.107601+0000","last_active":"2024-05-08T23:21:05.355265+0000","last_peered":"2024-05-08T23:21:05.355265+0000","last_clean":"2024-05-08T23:21:05.355265+0000","last_became_active":"2024-05-08T23:21:04.107309+0000","last_became_peered":"2024-05-08T23:21:04.107309+0000","last_unstale":"2024-05-08T23:21:05.355265+0000","last_undegraded":"2024-05-08T23:21:05.355265+0000","last_fullsized":"2024-05-08T23:21:05.355265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354968+0000","last_change":"2024-05-08T23:21:04.106808+0000","last_active":"2024-05-08T23:21:05.354968+0000","last_peered":"2024-05-08T23:21:05.354968+0000","last_clean":"2024-05-08T23:21:05.354968+0000","last_became_active":"2024-05-08T23:21:04.106628+0000","last_became_peered":"2024-05-08T23:21:04.106628+0000","last_unstale":"2024-05-08T23:21:05.354968+0000","last_undegraded":"2024-05-08T23:21:05.354968+0000","last_fullsized":"2024-05-08T23:21:05.354968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.433453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356256+0000","last_change":"2024-05-08T23:21:04.105100+0000","last_active":"2024-05-08T23:21:05.356256+0000","last_peered":"2024-05-08T23:21:05.356256+0000","last_clean":"2024-05-08T23:21:05.356256+0000","last_became_active":"2024-05-08T23:21:04.104785+0000","last_became_peered":"2024-05-08T23:21:04.104785+0000","last_unstale":"2024-05-08T23:21:05.356256+0000","last_undegraded":"2024-05-08T23:21:05.356256+0000","last_fullsized":"2024-05-08T23:21:05.356256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:19.795476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355441+0000","last_change":"2024-05-08T23:21:04.105145+0000","last_active":"2024-05-08T23:21:05.355441+0000","last_peered":"2024-05-08T23:21:05.355441+0000","last_clean":"2024-05-08T23:21:05.355441+0000","last_became_active":"2024-05-08T23:21:04.104842+0000","last_became_peered":"2024-05-08T23:21:04.104842+0000","last_unstale":"2024-05-08T23:21:05.355441+0000","last_undegraded":"2024-05-08T23:21:05.355441+0000","last_fullsized":"2024-05-08T23:21:05.355441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092800+0000","last_change":"2024-05-08T23:21:04.106935+0000","last_active":"2024-05-08T23:21:05.092800+0000","last_peered":"2024-05-08T23:21:05.092800+0000","last_clean":"2024-05-08T23:21:05.092800+0000","last_became_active":"2024-05-08T23:21:04.106671+0000","last_became_peered":"2024-05-08T23:21:04.106671+0000","last_unstale":"2024-05-08T23:21:05.092800+0000","last_undegraded":"2024-05-08T23:21:05.092800+0000","last_fullsized":"2024-05-08T23:21:05.092800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:07.596384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093704+0000","last_change":"2024-05-08T23:21:04.112023+0000","last_active":"2024-05-08T23:21:05.093704+0000","last_peered":"2024-05-08T23:21:05.093704+0000","last_clean":"2024-05-08T23:21:05.093704+0000","last_became_active":"2024-05-08T23:21:04.111819+0000","last_became_peered":"2024-05-08T23:21:04.111819+0000","last_unstale":"2024-05-08T23:21:05.093704+0000","last_undegraded":"2024-05-08T23:21:05.093704+0000","last_fullsized":"2024-05-08T23:21:05.093704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355844+0000","last_change":"2024-05-08T23:21:04.111963+0000","last_active":"2024-05-08T23:21:05.355844+0000","last_peered":"2024-05-08T23:21:05.355844+0000","last_clean":"2024-05-08T23:21:05.355844+0000","last_became_active":"2024-05-08T23:21:04.111629+0000","last_became_peered":"2024-05-08T23:21:04.111629+0000","last_unstale":"2024-05-08T23:21:05.355844+0000","last_undegraded":"2024-05-08T23:21:05.355844+0000","last_fullsized":"2024-05-08T23:21:05.355844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092931+0000","last_change":"2024-05-08T23:21:04.113608+0000","last_active":"2024-05-08T23:21:05.092931+0000","last_peered":"2024-05-08T23:21:05.092931+0000","last_clean":"2024-05-08T23:21:05.092931+0000","last_became_active":"2024-05-08T23:21:04.113322+0000","last_became_peered":"2024-05-08T23:21:04.113322+0000","last_unstale":"2024-05-08T23:21:05.092931+0000","last_undegraded":"2024-05-08T23:21:05.092931+0000","last_fullsized":"2024-05-08T23:21:05.092931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356052+0000","last_change":"2024-05-08T23:21:04.109658+0000","last_active":"2024-05-08T23:21:05.356052+0000","last_peered":"2024-05-08T23:21:05.356052+0000","last_clean":"2024-05-08T23:21:05.356052+0000","last_became_active":"2024-05-08T23:21:04.109222+0000","last_became_peered":"2024-05-08T23:21:04.109222+0000","last_unstale":"2024-05-08T23:21:05.356052+0000","last_undegraded":"2024-05-08T23:21:05.356052+0000","last_fullsized":"2024-05-08T23:21:05.356052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092319+0000","last_change":"2024-05-08T23:21:04.107408+0000","last_active":"2024-05-08T23:21:05.092319+0000","last_peered":"2024-05-08T23:21:05.092319+0000","last_clean":"2024-05-08T23:21:05.092319+0000","last_became_active":"2024-05-08T23:21:04.107123+0000","last_became_peered":"2024-05-08T23:21:04.107123+0000","last_unstale":"2024-05-08T23:21:05.092319+0000","last_undegraded":"2024-05-08T23:21:05.092319+0000","last_fullsized":"2024-05-08T23:21:05.092319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356155+0000","last_change":"2024-05-08T23:21:04.108500+0000","last_active":"2024-05-08T23:21:05.356155+0000","last_peered":"2024-05-08T23:21:05.356155+0000","last_clean":"2024-05-08T23:21:05.356155+0000","last_became_active":"2024-05-08T23:21:04.107974+0000","last_became_peered":"2024-05-08T23:21:04.107974+0000","last_unstale":"2024-05-08T23:21:05.356155+0000","last_undegraded":"2024-05-08T23:21:05.356155+0000","last_fullsized":"2024-05-08T23:21:05.356155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355333+0000","last_change":"2024-05-08T23:21:04.112817+0000","last_active":"2024-05-08T23:21:05.355333+0000","last_peered":"2024-05-08T23:21:05.355333+0000","last_clean":"2024-05-08T23:21:05.355333+0000","last_became_active":"2024-05-08T23:21:04.112519+0000","last_became_peered":"2024-05-08T23:21:04.112519+0000","last_unstale":"2024-05-08T23:21:05.355333+0000","last_undegraded":"2024-05-08T23:21:05.355333+0000","last_fullsized":"2024-05-08T23:21:05.355333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356539+0000","last_change":"2024-05-08T23:21:04.113697+0000","last_active":"2024-05-08T23:21:05.356539+0000","last_peered":"2024-05-08T23:21:05.356539+0000","last_clean":"2024-05-08T23:21:05.356539+0000","last_became_active":"2024-05-08T23:21:04.113363+0000","last_became_peered":"2024-05-08T23:21:04.113363+0000","last_unstale":"2024-05-08T23:21:05.356539+0000","last_undegraded":"2024-05-08T23:21:05.356539+0000","last_fullsized":"2024-05-08T23:21:05.356539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:21:13.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:21:13.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.234+0000 7fd0d0a9d700 1 Processor -- start 2024-05-08T23:21:13.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.234+0000 7fd0d0a9d700 1 -- start start 2024-05-08T23:21:13.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.235+0000 7fd0d0a9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0cc152850 0x7fd0cc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.235+0000 7fd0d0a9d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0ce100 con 0x7fd0cc0cbf30 2024-05-08T23:21:13.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.235+0000 7fd0d0a9d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0cc360 con 0x7fd0cc152470 2024-05-08T23:21:13.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.235+0000 7fd0d0a9d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0cc15c220 con 0x7fd0cc152850 2024-05-08T23:21:13.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c9d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0cc152850 0x7fd0cc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c9d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0cc152850 0x7fd0cc15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60530/0 (socket says 172.21.15.5:60530) 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c9d9b700 1 -- 172.21.15.5:0/3685500463 learned_addr learned my addr 172.21.15.5:0/3685500463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0cad9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0cc0cbf30 0x7fd0cc06a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44564/0 (socket says 172.21.15.5:44564) 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c9d9b700 1 -- 172.21.15.5:0/3685500463 >> v1:172.21.15.49:6789/0 conn(0x7fd0cc152470 legacy=0x7fd0cc0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c9d9b700 1 -- 172.21.15.5:0/3685500463 >> v1:172.21.15.5:6789/0 conn(0x7fd0cc0cbf30 legacy=0x7fd0cc06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c9d9b700 1 -- 172.21.15.5:0/3685500463 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc15c500 con 0x7fd0cc152850 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c9d9b700 1 --2- 172.21.15.5:0/3685500463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc15bbf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40098f0 tx=0x7fd0b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36e9e52f90208d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c959a700 1 -- 172.21.15.5:0/3685500463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400b8a0 con 0x7fd0cc152850 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c959a700 1 -- 172.21.15.5:0/3685500463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b400ba00 con 0x7fd0cc152850 2024-05-08T23:21:13.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.236+0000 7fd0c959a700 1 -- 172.21.15.5:0/3685500463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400bd90 con 0x7fd0cc152850 2024-05-08T23:21:13.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.237+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/3685500463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 msgr2=0x7fd0cc15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.237+0000 7fd0d0a9d700 1 --2- 172.21.15.5:0/3685500463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc15bbf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40098f0 tx=0x7fd0b4009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:13.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.237+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/3685500463 shutdown_connections 2024-05-08T23:21:13.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.237+0000 7fd0d0a9d700 1 --2- 172.21.15.5:0/3685500463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc15bbf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.237+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/3685500463 >> 172.21.15.5:0/3685500463 conn(0x7fd0cc099a50 msgr2=0x7fd0cc0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:13.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.237+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/3685500463 shutdown_connections 2024-05-08T23:21:13.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.237+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/3685500463 wait complete. 2024-05-08T23:21:13.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.239+0000 7fd0d0a9d700 1 Processor -- start 2024-05-08T23:21:13.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.239+0000 7fd0d0a9d700 1 -- start start 2024-05-08T23:21:13.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0d0a9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0cc0cbf30 0x7fd0cc0c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0d0a9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0cc152470 0x7fd0cc155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0d0a9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc0b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0d0a9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c3970 con 0x7fd0cc152850 2024-05-08T23:21:13.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0d0a9d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b55b0 con 0x7fd0cc152470 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0d0a9d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b5890 con 0x7fd0cc0cbf30 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc0b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc0b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60540/0 (socket says 172.21.15.5:60540) 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0cad9d700 1 -- 172.21.15.5:0/1086244217 learned_addr learned my addr 172.21.15.5:0/1086244217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0c9d9b700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0cc152470 0x7fd0cc155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0ca59c700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0cc0cbf30 0x7fd0cc0c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0cad9d700 1 -- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0cc0cbf30 msgr2=0x7fd0cc0c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0cad9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0cc0cbf30 0x7fd0cc0c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0cad9d700 1 -- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0cc152470 msgr2=0x7fd0cc155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0cad9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0cc152470 0x7fd0cc155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.240+0000 7fd0cad9d700 1 -- 172.21.15.5:0/1086244217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4009550 con 0x7fd0cc152850 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.241+0000 7fd0ca59c700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0cc0cbf30 0x7fd0cc0c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.241+0000 7fd0cad9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc0b4800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400cbd0 tx=0x7fd0c400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.241+0000 7fd0c2ffd700 1 -- 172.21.15.5:0/1086244217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400ab00 con 0x7fd0cc152850 2024-05-08T23:21:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.241+0000 7fd0c2ffd700 1 -- 172.21.15.5:0/1086244217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c400ac60 con 0x7fd0cc152850 2024-05-08T23:21:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.241+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc0b5b70 con 0x7fd0cc152850 2024-05-08T23:21:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.241+0000 7fd0c2ffd700 1 -- 172.21.15.5:0/1086244217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4007070 con 0x7fd0cc152850 2024-05-08T23:21:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.241+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc155a70 con 0x7fd0cc152850 2024-05-08T23:21:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.242+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc0b6090 con 0x7fd0cc152850 2024-05-08T23:21:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.242+0000 7fd0c2ffd700 1 -- 172.21.15.5:0/1086244217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd0c40071d0 con 0x7fd0cc152850 2024-05-08T23:21:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.243+0000 7fd0c2ffd700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0ac06ac90 0x7fd0ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.243+0000 7fd0c2ffd700 1 -- 172.21.15.5:0/1086244217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0c4088fa0 con 0x7fd0cc152850 2024-05-08T23:21:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.243+0000 7fd0ca59c700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0ac06ac90 0x7fd0ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.244+0000 7fd0ca59c700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0ac06ac90 0x7fd0ac06d150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0b1dd0 tx=0x7fd0bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:13.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.248+0000 7fd0c2ffd700 1 -- 172.21.15.5:0/1086244217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd0c40bc900 con 0x7fd0cc152850 2024-05-08T23:21:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.459+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0cc155c60 con 0x7fd0ac06ac90 2024-05-08T23:21:13.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.468+0000 7fd0c2ffd700 1 -- 172.21.15.5:0/1086244217 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264577 (secure 0 0 0) 0x7fd0cc155c60 con 0x7fd0ac06ac90 2024-05-08T23:21:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:13.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0ac06ac90 msgr2=0x7fd0ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0ac06ac90 0x7fd0ac06d150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0b1dd0 tx=0x7fd0bc009040 comp rx=0 tx=0).stop 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 msgr2=0x7fd0cc0b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc0b4800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400cbd0 tx=0x7fd0c400cee0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 shutdown_connections 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0ac06ac90 0x7fd0ac06d150 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc152850 0x7fd0cc0b4800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0cc0cbf30 0x7fd0cc0c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 --2- 172.21.15.5:0/1086244217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0cc152470 0x7fd0cc155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 >> 172.21.15.5:0/1086244217 conn(0x7fd0cc099a50 msgr2=0x7fd0cc15e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 shutdown_connections 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.471+0000 7fd0d0a9d700 1 -- 172.21.15.5:0/1086244217 wait complete. 2024-05-08T23:21:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:21:13.485 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-08T23:21:13.295544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.205437"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356525+0000","last_change":"2024-05-08T23:21:04.117587+0000","last_active":"2024-05-08T23:21:05.356525+0000","last_peered":"2024-05-08T23:21:05.356525+0000","last_clean":"2024-05-08T23:21:05.356525+0000","last_became_active":"2024-05-08T23:21:04.117296+0000","last_became_peered":"2024-05-08T23:21:04.117296+0000","last_unstale":"2024-05-08T23:21:05.356525+0000","last_undegraded":"2024-05-08T23:21:05.356525+0000","last_fullsized":"2024-05-08T23:21:05.356525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093945+0000","last_change":"2024-05-08T23:21:04.113872+0000","last_active":"2024-05-08T23:21:05.093945+0000","last_peered":"2024-05-08T23:21:05.093945+0000","last_clean":"2024-05-08T23:21:05.093945+0000","last_became_active":"2024-05-08T23:21:04.113525+0000","last_became_peered":"2024-05-08T23:21:04.113525+0000","last_unstale":"2024-05-08T23:21:05.093945+0000","last_undegraded":"2024-05-08T23:21:05.093945+0000","last_fullsized":"2024-05-08T23:21:05.093945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356124+0000","last_change":"2024-05-08T23:21:04.100510+0000","last_active":"2024-05-08T23:21:05.356124+0000","last_peered":"2024-05-08T23:21:05.356124+0000","last_clean":"2024-05-08T23:21:05.356124+0000","last_became_active":"2024-05-08T23:21:04.100227+0000","last_became_peered":"2024-05-08T23:21:04.100227+0000","last_unstale":"2024-05-08T23:21:05.356124+0000","last_undegraded":"2024-05-08T23:21:05.356124+0000","last_fullsized":"2024-05-08T23:21:05.356124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.368136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356793+0000","last_change":"2024-05-08T23:21:04.110918+0000","last_active":"2024-05-08T23:21:05.356793+0000","last_peered":"2024-05-08T23:21:05.356793+0000","last_clean":"2024-05-08T23:21:05.356793+0000","last_became_active":"2024-05-08T23:21:04.110450+0000","last_became_peered":"2024-05-08T23:21:04.110450+0000","last_unstale":"2024-05-08T23:21:05.356793+0000","last_undegraded":"2024-05-08T23:21:05.356793+0000","last_fullsized":"2024-05-08T23:21:05.356793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356803+0000","last_change":"2024-05-08T23:21:04.113276+0000","last_active":"2024-05-08T23:21:05.356803+0000","last_peered":"2024-05-08T23:21:05.356803+0000","last_clean":"2024-05-08T23:21:05.356803+0000","last_became_active":"2024-05-08T23:21:04.112995+0000","last_became_peered":"2024-05-08T23:21:04.112995+0000","last_unstale":"2024-05-08T23:21:05.356803+0000","last_undegraded":"2024-05-08T23:21:05.356803+0000","last_fullsized":"2024-05-08T23:21:05.356803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356039+0000","last_change":"2024-05-08T23:21:04.106788+0000","last_active":"2024-05-08T23:21:05.356039+0000","last_peered":"2024-05-08T23:21:05.356039+0000","last_clean":"2024-05-08T23:21:05.356039+0000","last_became_active":"2024-05-08T23:21:04.106459+0000","last_became_peered":"2024-05-08T23:21:04.106459+0000","last_unstale":"2024-05-08T23:21:05.356039+0000","last_undegraded":"2024-05-08T23:21:05.356039+0000","last_fullsized":"2024-05-08T23:21:05.356039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356576+0000","last_change":"2024-05-08T23:21:04.112766+0000","last_active":"2024-05-08T23:21:05.356576+0000","last_peered":"2024-05-08T23:21:05.356576+0000","last_clean":"2024-05-08T23:21:05.356576+0000","last_became_active":"2024-05-08T23:21:04.112455+0000","last_became_peered":"2024-05-08T23:21:04.112455+0000","last_unstale":"2024-05-08T23:21:05.356576+0000","last_undegraded":"2024-05-08T23:21:05.356576+0000","last_fullsized":"2024-05-08T23:21:05.356576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:16.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092125+0000","last_change":"2024-05-08T23:21:04.103185+0000","last_active":"2024-05-08T23:21:05.092125+0000","last_peered":"2024-05-08T23:21:05.092125+0000","last_clean":"2024-05-08T23:21:05.092125+0000","last_became_active":"2024-05-08T23:21:04.102828+0000","last_became_peered":"2024-05-08T23:21:04.102828+0000","last_unstale":"2024-05-08T23:21:05.092125+0000","last_undegraded":"2024-05-08T23:21:05.092125+0000","last_fullsized":"2024-05-08T23:21:05.092125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356537+0000","last_change":"2024-05-08T23:21:04.122716+0000","last_active":"2024-05-08T23:21:05.356537+0000","last_peered":"2024-05-08T23:21:05.356537+0000","last_clean":"2024-05-08T23:21:05.356537+0000","last_became_active":"2024-05-08T23:21:04.122410+0000","last_became_peered":"2024-05-08T23:21:04.122410+0000","last_unstale":"2024-05-08T23:21:05.356537+0000","last_undegraded":"2024-05-08T23:21:05.356537+0000","last_fullsized":"2024-05-08T23:21:05.356537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:42.560490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355980+0000","last_change":"2024-05-08T23:21:04.111916+0000","last_active":"2024-05-08T23:21:05.355980+0000","last_peered":"2024-05-08T23:21:05.355980+0000","last_clean":"2024-05-08T23:21:05.355980+0000","last_became_active":"2024-05-08T23:21:04.111520+0000","last_became_peered":"2024-05-08T23:21:04.111520+0000","last_unstale":"2024-05-08T23:21:05.355980+0000","last_undegraded":"2024-05-08T23:21:05.355980+0000","last_fullsized":"2024-05-08T23:21:05.355980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355636+0000","last_change":"2024-05-08T23:21:04.113128+0000","last_active":"2024-05-08T23:21:05.355636+0000","last_peered":"2024-05-08T23:21:05.355636+0000","last_clean":"2024-05-08T23:21:05.355636+0000","last_became_active":"2024-05-08T23:21:04.112746+0000","last_became_peered":"2024-05-08T23:21:04.112746+0000","last_unstale":"2024-05-08T23:21:05.355636+0000","last_undegraded":"2024-05-08T23:21:05.355636+0000","last_fullsized":"2024-05-08T23:21:05.355636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:07.596384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093836+0000","last_change":"2024-05-08T23:21:04.118631+0000","last_active":"2024-05-08T23:21:05.093836+0000","last_peered":"2024-05-08T23:21:05.093836+0000","last_clean":"2024-05-08T23:21:05.093836+0000","last_became_active":"2024-05-08T23:21:04.118285+0000","last_became_peered":"2024-05-08T23:21:04.118285+0000","last_unstale":"2024-05-08T23:21:05.093836+0000","last_undegraded":"2024-05-08T23:21:05.093836+0000","last_fullsized":"2024-05-08T23:21:05.093836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.260488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092563+0000","last_change":"2024-05-08T23:21:04.114717+0000","last_active":"2024-05-08T23:21:05.092563+0000","last_peered":"2024-05-08T23:21:05.092563+0000","last_clean":"2024-05-08T23:21:05.092563+0000","last_became_active":"2024-05-08T23:21:04.114456+0000","last_became_peered":"2024-05-08T23:21:04.114456+0000","last_unstale":"2024-05-08T23:21:05.092563+0000","last_undegraded":"2024-05-08T23:21:05.092563+0000","last_fullsized":"2024-05-08T23:21:05.092563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.119343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355161+0000","last_change":"2024-05-08T23:21:04.110105+0000","last_active":"2024-05-08T23:21:05.355161+0000","last_peered":"2024-05-08T23:21:05.355161+0000","last_clean":"2024-05-08T23:21:05.355161+0000","last_became_active":"2024-05-08T23:21:04.109818+0000","last_became_peered":"2024-05-08T23:21:04.109818+0000","last_unstale":"2024-05-08T23:21:05.355161+0000","last_undegraded":"2024-05-08T23:21:05.355161+0000","last_fullsized":"2024-05-08T23:21:05.355161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093437+0000","last_change":"2024-05-08T23:21:04.114173+0000","last_active":"2024-05-08T23:21:05.093437+0000","last_peered":"2024-05-08T23:21:05.093437+0000","last_clean":"2024-05-08T23:21:05.093437+0000","last_became_active":"2024-05-08T23:21:04.108376+0000","last_became_peered":"2024-05-08T23:21:04.108376+0000","last_unstale":"2024-05-08T23:21:05.093437+0000","last_undegraded":"2024-05-08T23:21:05.093437+0000","last_fullsized":"2024-05-08T23:21:05.093437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355476+0000","last_change":"2024-05-08T23:21:04.100458+0000","last_active":"2024-05-08T23:21:05.355476+0000","last_peered":"2024-05-08T23:21:05.355476+0000","last_clean":"2024-05-08T23:21:05.355476+0000","last_became_active":"2024-05-08T23:21:04.100166+0000","last_became_peered":"2024-05-08T23:21:04.100166+0000","last_unstale":"2024-05-08T23:21:05.355476+0000","last_undegraded":"2024-05-08T23:21:05.355476+0000","last_fullsized":"2024-05-08T23:21:05.355476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357377+0000","last_change":"2024-05-08T23:21:04.109204+0000","last_active":"2024-05-08T23:21:05.357377+0000","last_peered":"2024-05-08T23:21:05.357377+0000","last_clean":"2024-05-08T23:21:05.357377+0000","last_became_active":"2024-05-08T23:21:04.108507+0000","last_became_peered":"2024-05-08T23:21:04.108507+0000","last_unstale":"2024-05-08T23:21:05.357377+0000","last_undegraded":"2024-05-08T23:21:05.357377+0000","last_fullsized":"2024-05-08T23:21:05.357377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:16.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355050+0000","last_change":"2024-05-08T23:21:04.114847+0000","last_active":"2024-05-08T23:21:05.355050+0000","last_peered":"2024-05-08T23:21:05.355050+0000","last_clean":"2024-05-08T23:21:05.355050+0000","last_became_active":"2024-05-08T23:21:04.114090+0000","last_became_peered":"2024-05-08T23:21:04.114090+0000","last_unstale":"2024-05-08T23:21:05.355050+0000","last_undegraded":"2024-05-08T23:21:05.355050+0000","last_fullsized":"2024-05-08T23:21:05.355050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.260488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356040+0000","last_change":"2024-05-08T23:21:04.113465+0000","last_active":"2024-05-08T23:21:05.356040+0000","last_peered":"2024-05-08T23:21:05.356040+0000","last_clean":"2024-05-08T23:21:05.356040+0000","last_became_active":"2024-05-08T23:21:04.113179+0000","last_became_peered":"2024-05-08T23:21:04.113179+0000","last_unstale":"2024-05-08T23:21:05.356040+0000","last_undegraded":"2024-05-08T23:21:05.356040+0000","last_fullsized":"2024-05-08T23:21:05.356040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355379+0000","last_change":"2024-05-08T23:21:03.646371+0000","last_active":"2024-05-08T23:21:05.355379+0000","last_peered":"2024-05-08T23:21:05.355379+0000","last_clean":"2024-05-08T23:21:05.355379+0000","last_became_active":"2024-05-08T23:21:03.646004+0000","last_became_peered":"2024-05-08T23:21:03.646004+0000","last_unstale":"2024-05-08T23:21:05.355379+0000","last_undegraded":"2024-05-08T23:21:05.355379+0000","last_fullsized":"2024-05-08T23:21:05.355379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:32.482556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092991+0000","last_change":"2024-05-08T23:21:04.115651+0000","last_active":"2024-05-08T23:21:05.092991+0000","last_peered":"2024-05-08T23:21:05.092991+0000","last_clean":"2024-05-08T23:21:05.092991+0000","last_became_active":"2024-05-08T23:21:04.115384+0000","last_became_peered":"2024-05-08T23:21:04.115384+0000","last_unstale":"2024-05-08T23:21:05.092991+0000","last_undegraded":"2024-05-08T23:21:05.092991+0000","last_fullsized":"2024-05-08T23:21:05.092991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093060+0000","last_change":"2024-05-08T23:21:03.645350+0000","last_active":"2024-05-08T23:21:05.093060+0000","last_peered":"2024-05-08T23:21:05.093060+0000","last_clean":"2024-05-08T23:21:05.093060+0000","last_became_active":"2024-05-08T23:21:03.645081+0000","last_became_peered":"2024-05-08T23:21:03.645081+0000","last_unstale":"2024-05-08T23:21:05.093060+0000","last_undegraded":"2024-05-08T23:21:05.093060+0000","last_fullsized":"2024-05-08T23:21:05.093060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355362+0000","last_change":"2024-05-08T23:21:04.112500+0000","last_active":"2024-05-08T23:21:05.355362+0000","last_peered":"2024-05-08T23:21:05.355362+0000","last_clean":"2024-05-08T23:21:05.355362+0000","last_became_active":"2024-05-08T23:21:04.111988+0000","last_became_peered":"2024-05-08T23:21:04.111988+0000","last_unstale":"2024-05-08T23:21:05.355362+0000","last_undegraded":"2024-05-08T23:21:05.355362+0000","last_fullsized":"2024-05-08T23:21:05.355362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356422+0000","last_change":"2024-05-08T23:21:03.647784+0000","last_active":"2024-05-08T23:21:05.356422+0000","last_peered":"2024-05-08T23:21:05.356422+0000","last_clean":"2024-05-08T23:21:05.356422+0000","last_became_active":"2024-05-08T23:21:03.646667+0000","last_became_peered":"2024-05-08T23:21:03.646667+0000","last_unstale":"2024-05-08T23:21:05.356422+0000","last_undegraded":"2024-05-08T23:21:05.356422+0000","last_fullsized":"2024-05-08T23:21:05.356422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355900+0000","last_change":"2024-05-08T23:21:04.104612+0000","last_active":"2024-05-08T23:21:05.355900+0000","last_peered":"2024-05-08T23:21:05.355900+0000","last_clean":"2024-05-08T23:21:05.355900+0000","last_became_active":"2024-05-08T23:21:04.104304+0000","last_became_peered":"2024-05-08T23:21:04.104304+0000","last_unstale":"2024-05-08T23:21:05.355900+0000","last_undegraded":"2024-05-08T23:21:05.355900+0000","last_fullsized":"2024-05-08T23:21:05.355900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356075+0000","last_change":"2024-05-08T23:21:03.648270+0000","last_active":"2024-05-08T23:21:05.356075+0000","last_peered":"2024-05-08T23:21:05.356075+0000","last_clean":"2024-05-08T23:21:05.356075+0000","last_became_active":"2024-05-08T23:21:03.647784+0000","last_became_peered":"2024-05-08T23:21:03.647784+0000","last_unstale":"2024-05-08T23:21:05.356075+0000","last_undegraded":"2024-05-08T23:21:05.356075+0000","last_fullsized":"2024-05-08T23:21:05.356075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357179+0000","last_change":"2024-05-08T23:21:04.106021+0000","last_active":"2024-05-08T23:21:05.357179+0000","last_peered":"2024-05-08T23:21:05.357179+0000","last_clean":"2024-05-08T23:21:05.357179+0000","last_became_active":"2024-05-08T23:21:04.105614+0000","last_became_peered":"2024-05-08T23:21:04.105614+0000","last_unstale":"2024-05-08T23:21:05.357179+0000","last_undegraded":"2024-05-08T23:21:05.357179+0000","last_fullsized":"2024-05-08T23:21:05.357179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.119343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092611+0000","last_change":"2024-05-08T23:21:03.642600+0000","last_active":"2024-05-08T23:21:05.092611+0000","last_peered":"2024-05-08T23:21:05.092611+0000","last_clean":"2024-05-08T23:21:05.092611+0000","last_became_active":"2024-05-08T23:21:03.642234+0000","last_became_peered":"2024-05-08T23:21:03.642234+0000","last_unstale":"2024-05-08T23:21:05.092611+0000","last_undegraded":"2024-05-08T23:21:05.092611+0000","last_fullsized":"2024-05-08T23:21:05.092611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093418+0000","last_change":"2024-05-08T23:21:04.117783+0000","last_active":"2024-05-08T23:21:05.093418+0000","last_peered":"2024-05-08T23:21:05.093418+0000","last_clean":"2024-05-08T23:21:05.093418+0000","last_became_active":"2024-05-08T23:21:04.117324+0000","last_became_peered":"2024-05-08T23:21:04.117324+0000","last_unstale":"2024-05-08T23:21:05.093418+0000","last_undegraded":"2024-05-08T23:21:05.093418+0000","last_fullsized":"2024-05-08T23:21:05.093418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.433453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355523+0000","last_change":"2024-05-08T23:21:03.644641+0000","last_active":"2024-05-08T23:21:05.355523+0000","last_peered":"2024-05-08T23:21:05.355523+0000","last_clean":"2024-05-08T23:21:05.355523+0000","last_became_active":"2024-05-08T23:21:03.644361+0000","last_became_peered":"2024-05-08T23:21:03.644361+0000","last_unstale":"2024-05-08T23:21:05.355523+0000","last_undegraded":"2024-05-08T23:21:05.355523+0000","last_fullsized":"2024-05-08T23:21:05.355523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355562+0000","last_change":"2024-05-08T23:21:04.105553+0000","last_active":"2024-05-08T23:21:05.355562+0000","last_peered":"2024-05-08T23:21:05.355562+0000","last_clean":"2024-05-08T23:21:05.355562+0000","last_became_active":"2024-05-08T23:21:04.105041+0000","last_became_peered":"2024-05-08T23:21:04.105041+0000","last_unstale":"2024-05-08T23:21:05.355562+0000","last_undegraded":"2024-05-08T23:21:05.355562+0000","last_fullsized":"2024-05-08T23:21:05.355562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.311829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092401+0000","last_change":"2024-05-08T23:21:03.639910+0000","last_active":"2024-05-08T23:21:05.092401+0000","last_peered":"2024-05-08T23:21:05.092401+0000","last_clean":"2024-05-08T23:21:05.092401+0000","last_became_active":"2024-05-08T23:21:03.638934+0000","last_became_peered":"2024-05-08T23:21:03.638934+0000","last_unstale":"2024-05-08T23:21:05.092401+0000","last_undegraded":"2024-05-08T23:21:05.092401+0000","last_fullsized":"2024-05-08T23:21:05.092401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092248+0000","last_change":"2024-05-08T23:21:04.108315+0000","last_active":"2024-05-08T23:21:05.092248+0000","last_peered":"2024-05-08T23:21:05.092248+0000","last_clean":"2024-05-08T23:21:05.092248+0000","last_became_active":"2024-05-08T23:21:04.108040+0000","last_became_peered":"2024-05-08T23:21:04.108040+0000","last_unstale":"2024-05-08T23:21:05.092248+0000","last_undegraded":"2024-05-08T23:21:05.092248+0000","last_fullsized":"2024-05-08T23:21:05.092248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355919+0000","last_change":"2024-05-08T23:21:03.641788+0000","last_active":"2024-05-08T23:21:05.355919+0000","last_peered":"2024-05-08T23:21:05.355919+0000","last_clean":"2024-05-08T23:21:05.355919+0000","last_became_active":"2024-05-08T23:21:03.641004+0000","last_became_peered":"2024-05-08T23:21:03.641004+0000","last_unstale":"2024-05-08T23:21:05.355919+0000","last_undegraded":"2024-05-08T23:21:05.355919+0000","last_fullsized":"2024-05-08T23:21:05.355919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093537+0000","last_change":"2024-05-08T23:21:04.114572+0000","last_active":"2024-05-08T23:21:05.093537+0000","last_peered":"2024-05-08T23:21:05.093537+0000","last_clean":"2024-05-08T23:21:05.093537+0000","last_became_active":"2024-05-08T23:21:04.114402+0000","last_became_peered":"2024-05-08T23:21:04.114402+0000","last_unstale":"2024-05-08T23:21:05.093537+0000","last_undegraded":"2024-05-08T23:21:05.093537+0000","last_fullsized":"2024-05-08T23:21:05.093537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356964+0000","last_change":"2024-05-08T23:21:03.648330+0000","last_active":"2024-05-08T23:21:05.356964+0000","last_peered":"2024-05-08T23:21:05.356964+0000","last_clean":"2024-05-08T23:21:05.356964+0000","last_became_active":"2024-05-08T23:21:03.648033+0000","last_became_peered":"2024-05-08T23:21:03.648033+0000","last_unstale":"2024-05-08T23:21:05.356964+0000","last_undegraded":"2024-05-08T23:21:05.356964+0000","last_fullsized":"2024-05-08T23:21:05.356964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356917+0000","last_change":"2024-05-08T23:21:04.108163+0000","last_active":"2024-05-08T23:21:05.356917+0000","last_peered":"2024-05-08T23:21:05.356917+0000","last_clean":"2024-05-08T23:21:05.356917+0000","last_became_active":"2024-05-08T23:21:04.107854+0000","last_became_peered":"2024-05-08T23:21:04.107854+0000","last_unstale":"2024-05-08T23:21:05.356917+0000","last_undegraded":"2024-05-08T23:21:05.356917+0000","last_fullsized":"2024-05-08T23:21:05.356917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:09.028590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356841+0000","last_change":"2024-05-08T23:21:03.640432+0000","last_active":"2024-05-08T23:21:05.356841+0000","last_peered":"2024-05-08T23:21:05.356841+0000","last_clean":"2024-05-08T23:21:05.356841+0000","last_became_active":"2024-05-08T23:21:03.640121+0000","last_became_peered":"2024-05-08T23:21:03.640121+0000","last_unstale":"2024-05-08T23:21:05.356841+0000","last_undegraded":"2024-05-08T23:21:05.356841+0000","last_fullsized":"2024-05-08T23:21:05.356841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355475+0000","last_change":"2024-05-08T23:21:04.113136+0000","last_active":"2024-05-08T23:21:05.355475+0000","last_peered":"2024-05-08T23:21:05.355475+0000","last_clean":"2024-05-08T23:21:05.355475+0000","last_became_active":"2024-05-08T23:21:04.112831+0000","last_became_peered":"2024-05-08T23:21:04.112831+0000","last_unstale":"2024-05-08T23:21:05.355475+0000","last_undegraded":"2024-05-08T23:21:05.355475+0000","last_fullsized":"2024-05-08T23:21:05.355475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:03.946264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093793+0000","last_change":"2024-05-08T23:21:03.647271+0000","last_active":"2024-05-08T23:21:05.093793+0000","last_peered":"2024-05-08T23:21:05.093793+0000","last_clean":"2024-05-08T23:21:05.093793+0000","last_became_active":"2024-05-08T23:21:03.646992+0000","last_became_peered":"2024-05-08T23:21:03.646992+0000","last_unstale":"2024-05-08T23:21:05.093793+0000","last_undegraded":"2024-05-08T23:21:05.093793+0000","last_fullsized":"2024-05-08T23:21:05.093793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:43.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093761+0000","last_change":"2024-05-08T23:21:04.106622+0000","last_active":"2024-05-08T23:21:05.093761+0000","last_peered":"2024-05-08T23:21:05.093761+0000","last_clean":"2024-05-08T23:21:05.093761+0000","last_became_active":"2024-05-08T23:21:04.106248+0000","last_became_peered":"2024-05-08T23:21:04.106248+0000","last_unstale":"2024-05-08T23:21:05.093761+0000","last_undegraded":"2024-05-08T23:21:05.093761+0000","last_fullsized":"2024-05-08T23:21:05.093761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.311829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356428+0000","last_change":"2024-05-08T23:21:03.642565+0000","last_active":"2024-05-08T23:21:05.356428+0000","last_peered":"2024-05-08T23:21:05.356428+0000","last_clean":"2024-05-08T23:21:05.356428+0000","last_became_active":"2024-05-08T23:21:03.642253+0000","last_became_peered":"2024-05-08T23:21:03.642253+0000","last_unstale":"2024-05-08T23:21:05.356428+0000","last_undegraded":"2024-05-08T23:21:05.356428+0000","last_fullsized":"2024-05-08T23:21:05.356428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356171+0000","last_change":"2024-05-08T23:21:04.113888+0000","last_active":"2024-05-08T23:21:05.356171+0000","last_peered":"2024-05-08T23:21:05.356171+0000","last_clean":"2024-05-08T23:21:05.356171+0000","last_became_active":"2024-05-08T23:21:04.113607+0000","last_became_peered":"2024-05-08T23:21:04.113607+0000","last_unstale":"2024-05-08T23:21:05.356171+0000","last_undegraded":"2024-05-08T23:21:05.356171+0000","last_fullsized":"2024-05-08T23:21:05.356171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:19.795476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355509+0000","last_change":"2024-05-08T23:21:03.641751+0000","last_active":"2024-05-08T23:21:05.355509+0000","last_peered":"2024-05-08T23:21:05.355509+0000","last_clean":"2024-05-08T23:21:05.355509+0000","last_became_active":"2024-05-08T23:21:03.641441+0000","last_became_peered":"2024-05-08T23:21:03.641441+0000","last_unstale":"2024-05-08T23:21:05.355509+0000","last_undegraded":"2024-05-08T23:21:05.355509+0000","last_fullsized":"2024-05-08T23:21:05.355509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:58.822252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355038+0000","last_change":"2024-05-08T23:21:04.106177+0000","last_active":"2024-05-08T23:21:05.355038+0000","last_peered":"2024-05-08T23:21:05.355038+0000","last_clean":"2024-05-08T23:21:05.355038+0000","last_became_active":"2024-05-08T23:21:04.105844+0000","last_became_peered":"2024-05-08T23:21:04.105844+0000","last_unstale":"2024-05-08T23:21:05.355038+0000","last_undegraded":"2024-05-08T23:21:05.355038+0000","last_fullsized":"2024-05-08T23:21:05.355038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356436+0000","last_change":"2024-05-08T23:21:03.646376+0000","last_active":"2024-05-08T23:21:05.356436+0000","last_peered":"2024-05-08T23:21:05.356436+0000","last_clean":"2024-05-08T23:21:05.356436+0000","last_became_active":"2024-05-08T23:21:03.646071+0000","last_became_peered":"2024-05-08T23:21:03.646071+0000","last_unstale":"2024-05-08T23:21:05.356436+0000","last_undegraded":"2024-05-08T23:21:05.356436+0000","last_fullsized":"2024-05-08T23:21:05.356436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356915+0000","last_change":"2024-05-08T23:21:04.120630+0000","last_active":"2024-05-08T23:21:05.356915+0000","last_peered":"2024-05-08T23:21:05.356915+0000","last_clean":"2024-05-08T23:21:05.356915+0000","last_became_active":"2024-05-08T23:21:04.120315+0000","last_became_peered":"2024-05-08T23:21:04.120315+0000","last_unstale":"2024-05-08T23:21:05.356915+0000","last_undegraded":"2024-05-08T23:21:05.356915+0000","last_fullsized":"2024-05-08T23:21:05.356915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:57.953372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356590+0000","last_change":"2024-05-08T23:21:03.645556+0000","last_active":"2024-05-08T23:21:05.356590+0000","last_peered":"2024-05-08T23:21:05.356590+0000","last_clean":"2024-05-08T23:21:05.356590+0000","last_became_active":"2024-05-08T23:21:03.645083+0000","last_became_peered":"2024-05-08T23:21:03.645083+0000","last_unstale":"2024-05-08T23:21:05.356590+0000","last_undegraded":"2024-05-08T23:21:05.356590+0000","last_fullsized":"2024-05-08T23:21:05.356590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355274+0000","last_change":"2024-05-08T23:21:04.108235+0000","last_active":"2024-05-08T23:21:05.355274+0000","last_peered":"2024-05-08T23:21:05.355274+0000","last_clean":"2024-05-08T23:21:05.355274+0000","last_became_active":"2024-05-08T23:21:04.107933+0000","last_became_peered":"2024-05-08T23:21:04.107933+0000","last_unstale":"2024-05-08T23:21:05.355274+0000","last_undegraded":"2024-05-08T23:21:05.355274+0000","last_fullsized":"2024-05-08T23:21:05.355274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354849+0000","last_change":"2024-05-08T23:21:03.636549+0000","last_active":"2024-05-08T23:21:05.354849+0000","last_peered":"2024-05-08T23:21:05.354849+0000","last_clean":"2024-05-08T23:21:05.354849+0000","last_became_active":"2024-05-08T23:21:03.636259+0000","last_became_peered":"2024-05-08T23:21:03.636259+0000","last_unstale":"2024-05-08T23:21:05.354849+0000","last_undegraded":"2024-05-08T23:21:05.354849+0000","last_fullsized":"2024-05-08T23:21:05.354849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356690+0000","last_change":"2024-05-08T23:21:04.113963+0000","last_active":"2024-05-08T23:21:05.356690+0000","last_peered":"2024-05-08T23:21:05.356690+0000","last_clean":"2024-05-08T23:21:05.356690+0000","last_became_active":"2024-05-08T23:21:04.113645+0000","last_became_peered":"2024-05-08T23:21:04.113645+0000","last_unstale":"2024-05-08T23:21:05.356690+0000","last_undegraded":"2024-05-08T23:21:05.356690+0000","last_fullsized":"2024-05-08T23:21:05.356690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093744+0000","last_change":"2024-05-08T23:21:03.644725+0000","last_active":"2024-05-08T23:21:05.093744+0000","last_peered":"2024-05-08T23:21:05.093744+0000","last_clean":"2024-05-08T23:21:05.093744+0000","last_became_active":"2024-05-08T23:21:03.644432+0000","last_became_peered":"2024-05-08T23:21:03.644432+0000","last_unstale":"2024-05-08T23:21:05.093744+0000","last_undegraded":"2024-05-08T23:21:05.093744+0000","last_fullsized":"2024-05-08T23:21:05.093744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:58.822252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356426+0000","last_change":"2024-05-08T23:21:04.116753+0000","last_active":"2024-05-08T23:21:05.356426+0000","last_peered":"2024-05-08T23:21:05.356426+0000","last_clean":"2024-05-08T23:21:05.356426+0000","last_became_active":"2024-05-08T23:21:04.116366+0000","last_became_peered":"2024-05-08T23:21:04.116366+0000","last_unstale":"2024-05-08T23:21:05.356426+0000","last_undegraded":"2024-05-08T23:21:05.356426+0000","last_fullsized":"2024-05-08T23:21:05.356426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093071+0000","last_change":"2024-05-08T23:21:03.643882+0000","last_active":"2024-05-08T23:21:05.093071+0000","last_peered":"2024-05-08T23:21:05.093071+0000","last_clean":"2024-05-08T23:21:05.093071+0000","last_became_active":"2024-05-08T23:21:03.643533+0000","last_became_peered":"2024-05-08T23:21:03.643533+0000","last_unstale":"2024-05-08T23:21:05.093071+0000","last_undegraded":"2024-05-08T23:21:05.093071+0000","last_fullsized":"2024-05-08T23:21:05.093071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357279+0000","last_change":"2024-05-08T23:21:04.109134+0000","last_active":"2024-05-08T23:21:05.357279+0000","last_peered":"2024-05-08T23:21:05.357279+0000","last_clean":"2024-05-08T23:21:05.357279+0000","last_became_active":"2024-05-08T23:21:04.108799+0000","last_became_peered":"2024-05-08T23:21:04.108799+0000","last_unstale":"2024-05-08T23:21:05.357279+0000","last_undegraded":"2024-05-08T23:21:05.357279+0000","last_fullsized":"2024-05-08T23:21:05.357279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356346+0000","last_change":"2024-05-08T23:21:03.644161+0000","last_active":"2024-05-08T23:21:05.356346+0000","last_peered":"2024-05-08T23:21:05.356346+0000","last_clean":"2024-05-08T23:21:05.356346+0000","last_became_active":"2024-05-08T23:21:03.643341+0000","last_became_peered":"2024-05-08T23:21:03.643341+0000","last_unstale":"2024-05-08T23:21:05.356346+0000","last_undegraded":"2024-05-08T23:21:05.356346+0000","last_fullsized":"2024-05-08T23:21:05.356346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093723+0000","last_change":"2024-05-08T23:21:04.108965+0000","last_active":"2024-05-08T23:21:05.093723+0000","last_peered":"2024-05-08T23:21:05.093723+0000","last_clean":"2024-05-08T23:21:05.093723+0000","last_became_active":"2024-05-08T23:21:04.107917+0000","last_became_peered":"2024-05-08T23:21:04.107917+0000","last_unstale":"2024-05-08T23:21:05.093723+0000","last_undegraded":"2024-05-08T23:21:05.093723+0000","last_fullsized":"2024-05-08T23:21:05.093723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:00.670245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093185+0000","last_change":"2024-05-08T23:21:04.115959+0000","last_active":"2024-05-08T23:21:05.093185+0000","last_peered":"2024-05-08T23:21:05.093185+0000","last_clean":"2024-05-08T23:21:05.093185+0000","last_became_active":"2024-05-08T23:21:04.115696+0000","last_became_peered":"2024-05-08T23:21:04.115696+0000","last_unstale":"2024-05-08T23:21:05.093185+0000","last_undegraded":"2024-05-08T23:21:05.093185+0000","last_fullsized":"2024-05-08T23:21:05.093185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354856+0000","last_change":"2024-05-08T23:21:03.632217+0000","last_active":"2024-05-08T23:21:05.354856+0000","last_peered":"2024-05-08T23:21:05.354856+0000","last_clean":"2024-05-08T23:21:05.354856+0000","last_became_active":"2024-05-08T23:21:03.631883+0000","last_became_peered":"2024-05-08T23:21:03.631883+0000","last_unstale":"2024-05-08T23:21:05.354856+0000","last_undegraded":"2024-05-08T23:21:05.354856+0000","last_fullsized":"2024-05-08T23:21:05.354856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092402+0000","last_change":"2024-05-08T23:21:04.104594+0000","last_active":"2024-05-08T23:21:05.092402+0000","last_peered":"2024-05-08T23:21:05.092402+0000","last_clean":"2024-05-08T23:21:05.092402+0000","last_became_active":"2024-05-08T23:21:04.104335+0000","last_became_peered":"2024-05-08T23:21:04.104335+0000","last_unstale":"2024-05-08T23:21:05.092402+0000","last_undegraded":"2024-05-08T23:21:05.092402+0000","last_fullsized":"2024-05-08T23:21:05.092402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355796+0000","last_change":"2024-05-08T23:21:03.637311+0000","last_active":"2024-05-08T23:21:05.355796+0000","last_peered":"2024-05-08T23:21:05.355796+0000","last_clean":"2024-05-08T23:21:05.355796+0000","last_became_active":"2024-05-08T23:21:03.636529+0000","last_became_peered":"2024-05-08T23:21:03.636529+0000","last_unstale":"2024-05-08T23:21:05.355796+0000","last_undegraded":"2024-05-08T23:21:05.355796+0000","last_fullsized":"2024-05-08T23:21:05.355796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-08T23:21:04.101711+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:21:04.101711+0000","last_peered":"2024-05-08T23:21:04.101711+0000","last_clean":"2024-05-08T23:21:04.101711+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:21:04.101711+0000","last_undegraded":"2024-05-08T23:21:04.101711+0000","last_fullsized":"2024-05-08T23:21:04.101711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355927+0000","last_change":"2024-05-08T23:21:04.108232+0000","last_active":"2024-05-08T23:21:05.355927+0000","last_peered":"2024-05-08T23:21:05.355927+0000","last_clean":"2024-05-08T23:21:05.355927+0000","last_became_active":"2024-05-08T23:21:04.107908+0000","last_became_peered":"2024-05-08T23:21:04.107908+0000","last_unstale":"2024-05-08T23:21:05.355927+0000","last_undegraded":"2024-05-08T23:21:05.355927+0000","last_fullsized":"2024-05-08T23:21:05.355927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:17.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356470+0000","last_change":"2024-05-08T23:21:03.645567+0000","last_active":"2024-05-08T23:21:05.356470+0000","last_peered":"2024-05-08T23:21:05.356470+0000","last_clean":"2024-05-08T23:21:05.356470+0000","last_became_active":"2024-05-08T23:21:03.644844+0000","last_became_peered":"2024-05-08T23:21:03.644844+0000","last_unstale":"2024-05-08T23:21:05.356470+0000","last_undegraded":"2024-05-08T23:21:05.356470+0000","last_fullsized":"2024-05-08T23:21:05.356470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:43.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356283+0000","last_change":"2024-05-08T23:21:03.646987+0000","last_active":"2024-05-08T23:21:05.356283+0000","last_peered":"2024-05-08T23:21:05.356283+0000","last_clean":"2024-05-08T23:21:05.356283+0000","last_became_active":"2024-05-08T23:21:03.646283+0000","last_became_peered":"2024-05-08T23:21:03.646283+0000","last_unstale":"2024-05-08T23:21:05.356283+0000","last_undegraded":"2024-05-08T23:21:05.356283+0000","last_fullsized":"2024-05-08T23:21:05.356283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:59.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092960+0000","last_change":"2024-05-08T23:21:04.106165+0000","last_active":"2024-05-08T23:21:05.092960+0000","last_peered":"2024-05-08T23:21:05.092960+0000","last_clean":"2024-05-08T23:21:05.092960+0000","last_became_active":"2024-05-08T23:21:04.105879+0000","last_became_peered":"2024-05-08T23:21:04.105879+0000","last_unstale":"2024-05-08T23:21:05.092960+0000","last_undegraded":"2024-05-08T23:21:05.092960+0000","last_fullsized":"2024-05-08T23:21:05.092960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.368136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:07.296809+0000","last_change":"2024-05-08T23:21:03.638777+0000","last_active":"2024-05-08T23:21:07.296809+0000","last_peered":"2024-05-08T23:21:07.296809+0000","last_clean":"2024-05-08T23:21:07.296809+0000","last_became_active":"2024-05-08T23:21:03.638494+0000","last_became_peered":"2024-05-08T23:21:03.638494+0000","last_unstale":"2024-05-08T23:21:07.296809+0000","last_undegraded":"2024-05-08T23:21:07.296809+0000","last_fullsized":"2024-05-08T23:21:07.296809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093027+0000","last_change":"2024-05-08T23:21:04.113442+0000","last_active":"2024-05-08T23:21:05.093027+0000","last_peered":"2024-05-08T23:21:05.093027+0000","last_clean":"2024-05-08T23:21:05.093027+0000","last_became_active":"2024-05-08T23:21:04.113144+0000","last_became_peered":"2024-05-08T23:21:04.113144+0000","last_unstale":"2024-05-08T23:21:05.093027+0000","last_undegraded":"2024-05-08T23:21:05.093027+0000","last_fullsized":"2024-05-08T23:21:05.093027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355626+0000","last_change":"2024-05-08T23:21:03.644120+0000","last_active":"2024-05-08T23:21:05.355626+0000","last_peered":"2024-05-08T23:21:05.355626+0000","last_clean":"2024-05-08T23:21:05.355626+0000","last_became_active":"2024-05-08T23:21:03.643541+0000","last_became_peered":"2024-05-08T23:21:03.643541+0000","last_unstale":"2024-05-08T23:21:05.355626+0000","last_undegraded":"2024-05-08T23:21:05.355626+0000","last_fullsized":"2024-05-08T23:21:05.355626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355406+0000","last_change":"2024-05-08T23:21:04.115769+0000","last_active":"2024-05-08T23:21:05.355406+0000","last_peered":"2024-05-08T23:21:05.355406+0000","last_clean":"2024-05-08T23:21:05.355406+0000","last_became_active":"2024-05-08T23:21:04.115446+0000","last_became_peered":"2024-05-08T23:21:04.115446+0000","last_unstale":"2024-05-08T23:21:05.355406+0000","last_undegraded":"2024-05-08T23:21:05.355406+0000","last_fullsized":"2024-05-08T23:21:05.355406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356111+0000","last_change":"2024-05-08T23:21:03.647284+0000","last_active":"2024-05-08T23:21:05.356111+0000","last_peered":"2024-05-08T23:21:05.356111+0000","last_clean":"2024-05-08T23:21:05.356111+0000","last_became_active":"2024-05-08T23:21:03.645828+0000","last_became_peered":"2024-05-08T23:21:03.645828+0000","last_unstale":"2024-05-08T23:21:05.356111+0000","last_undegraded":"2024-05-08T23:21:05.356111+0000","last_fullsized":"2024-05-08T23:21:05.356111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:54.428258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093694+0000","last_change":"2024-05-08T23:21:04.105901+0000","last_active":"2024-05-08T23:21:05.093694+0000","last_peered":"2024-05-08T23:21:05.093694+0000","last_clean":"2024-05-08T23:21:05.093694+0000","last_became_active":"2024-05-08T23:21:04.105495+0000","last_became_peered":"2024-05-08T23:21:04.105495+0000","last_unstale":"2024-05-08T23:21:05.093694+0000","last_undegraded":"2024-05-08T23:21:05.093694+0000","last_fullsized":"2024-05-08T23:21:05.093694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:44.288365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093485+0000","last_change":"2024-05-08T23:21:03.643905+0000","last_active":"2024-05-08T23:21:05.093485+0000","last_peered":"2024-05-08T23:21:05.093485+0000","last_clean":"2024-05-08T23:21:05.093485+0000","last_became_active":"2024-05-08T23:21:03.643555+0000","last_became_peered":"2024-05-08T23:21:03.643555+0000","last_unstale":"2024-05-08T23:21:05.093485+0000","last_undegraded":"2024-05-08T23:21:05.093485+0000","last_fullsized":"2024-05-08T23:21:05.093485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092205+0000","last_change":"2024-05-08T23:21:04.117123+0000","last_active":"2024-05-08T23:21:05.092205+0000","last_peered":"2024-05-08T23:21:05.092205+0000","last_clean":"2024-05-08T23:21:05.092205+0000","last_became_active":"2024-05-08T23:21:04.116830+0000","last_became_peered":"2024-05-08T23:21:04.116830+0000","last_unstale":"2024-05-08T23:21:05.092205+0000","last_undegraded":"2024-05-08T23:21:05.092205+0000","last_fullsized":"2024-05-08T23:21:05.092205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093648+0000","last_change":"2024-05-08T23:21:03.642867+0000","last_active":"2024-05-08T23:21:05.093648+0000","last_peered":"2024-05-08T23:21:05.093648+0000","last_clean":"2024-05-08T23:21:05.093648+0000","last_became_active":"2024-05-08T23:21:03.642439+0000","last_became_peered":"2024-05-08T23:21:03.642439+0000","last_unstale":"2024-05-08T23:21:05.093648+0000","last_undegraded":"2024-05-08T23:21:05.093648+0000","last_fullsized":"2024-05-08T23:21:05.093648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092672+0000","last_change":"2024-05-08T23:21:04.115928+0000","last_active":"2024-05-08T23:21:05.092672+0000","last_peered":"2024-05-08T23:21:05.092672+0000","last_clean":"2024-05-08T23:21:05.092672+0000","last_became_active":"2024-05-08T23:21:04.113858+0000","last_became_peered":"2024-05-08T23:21:04.113858+0000","last_unstale":"2024-05-08T23:21:05.092672+0000","last_undegraded":"2024-05-08T23:21:05.092672+0000","last_fullsized":"2024-05-08T23:21:05.092672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357022+0000","last_change":"2024-05-08T23:21:03.649228+0000","last_active":"2024-05-08T23:21:05.357022+0000","last_peered":"2024-05-08T23:21:05.357022+0000","last_clean":"2024-05-08T23:21:05.357022+0000","last_became_active":"2024-05-08T23:21:03.648868+0000","last_became_peered":"2024-05-08T23:21:03.648868+0000","last_unstale":"2024-05-08T23:21:05.357022+0000","last_undegraded":"2024-05-08T23:21:05.357022+0000","last_fullsized":"2024-05-08T23:21:05.357022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:34.255292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355422+0000","last_change":"2024-05-08T23:21:04.110982+0000","last_active":"2024-05-08T23:21:05.355422+0000","last_peered":"2024-05-08T23:21:05.355422+0000","last_clean":"2024-05-08T23:21:05.355422+0000","last_became_active":"2024-05-08T23:21:04.110643+0000","last_became_peered":"2024-05-08T23:21:04.110643+0000","last_unstale":"2024-05-08T23:21:05.355422+0000","last_undegraded":"2024-05-08T23:21:05.355422+0000","last_fullsized":"2024-05-08T23:21:05.355422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355712+0000","last_change":"2024-05-08T23:21:03.637418+0000","last_active":"2024-05-08T23:21:05.355712+0000","last_peered":"2024-05-08T23:21:05.355712+0000","last_clean":"2024-05-08T23:21:05.355712+0000","last_became_active":"2024-05-08T23:21:03.637139+0000","last_became_peered":"2024-05-08T23:21:03.637139+0000","last_unstale":"2024-05-08T23:21:05.355712+0000","last_undegraded":"2024-05-08T23:21:05.355712+0000","last_fullsized":"2024-05-08T23:21:05.355712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356254+0000","last_change":"2024-05-08T23:21:04.112240+0000","last_active":"2024-05-08T23:21:05.356254+0000","last_peered":"2024-05-08T23:21:05.356254+0000","last_clean":"2024-05-08T23:21:05.356254+0000","last_became_active":"2024-05-08T23:21:04.111914+0000","last_became_peered":"2024-05-08T23:21:04.111914+0000","last_unstale":"2024-05-08T23:21:05.356254+0000","last_undegraded":"2024-05-08T23:21:05.356254+0000","last_fullsized":"2024-05-08T23:21:05.356254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:09.028590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093821+0000","last_change":"2024-05-08T23:21:03.643712+0000","last_active":"2024-05-08T23:21:05.093821+0000","last_peered":"2024-05-08T23:21:05.093821+0000","last_clean":"2024-05-08T23:21:05.093821+0000","last_became_active":"2024-05-08T23:21:03.643387+0000","last_became_peered":"2024-05-08T23:21:03.643387+0000","last_unstale":"2024-05-08T23:21:05.093821+0000","last_undegraded":"2024-05-08T23:21:05.093821+0000","last_fullsized":"2024-05-08T23:21:05.093821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:59.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356035+0000","last_change":"2024-05-08T23:21:04.110728+0000","last_active":"2024-05-08T23:21:05.356035+0000","last_peered":"2024-05-08T23:21:05.356035+0000","last_clean":"2024-05-08T23:21:05.356035+0000","last_became_active":"2024-05-08T23:21:04.110094+0000","last_became_peered":"2024-05-08T23:21:04.110094+0000","last_unstale":"2024-05-08T23:21:05.356035+0000","last_undegraded":"2024-05-08T23:21:05.356035+0000","last_fullsized":"2024-05-08T23:21:05.356035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:30:28.180773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355130+0000","last_change":"2024-05-08T23:21:03.638990+0000","last_active":"2024-05-08T23:21:05.355130+0000","last_peered":"2024-05-08T23:21:05.355130+0000","last_clean":"2024-05-08T23:21:05.355130+0000","last_became_active":"2024-05-08T23:21:03.638696+0000","last_became_peered":"2024-05-08T23:21:03.638696+0000","last_unstale":"2024-05-08T23:21:05.355130+0000","last_undegraded":"2024-05-08T23:21:05.355130+0000","last_fullsized":"2024-05-08T23:21:05.355130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:16.993057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356830+0000","last_change":"2024-05-08T23:21:04.117976+0000","last_active":"2024-05-08T23:21:05.356830+0000","last_peered":"2024-05-08T23:21:05.356830+0000","last_clean":"2024-05-08T23:21:05.356830+0000","last_became_active":"2024-05-08T23:21:04.117682+0000","last_became_peered":"2024-05-08T23:21:04.117682+0000","last_unstale":"2024-05-08T23:21:05.356830+0000","last_undegraded":"2024-05-08T23:21:05.356830+0000","last_fullsized":"2024-05-08T23:21:05.356830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:30:28.180773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355265+0000","last_change":"2024-05-08T23:21:04.107601+0000","last_active":"2024-05-08T23:21:05.355265+0000","last_peered":"2024-05-08T23:21:05.355265+0000","last_clean":"2024-05-08T23:21:05.355265+0000","last_became_active":"2024-05-08T23:21:04.107309+0000","last_became_peered":"2024-05-08T23:21:04.107309+0000","last_unstale":"2024-05-08T23:21:05.355265+0000","last_undegraded":"2024-05-08T23:21:05.355265+0000","last_fullsized":"2024-05-08T23:21:05.355265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354968+0000","last_change":"2024-05-08T23:21:04.106808+0000","last_active":"2024-05-08T23:21:05.354968+0000","last_peered":"2024-05-08T23:21:05.354968+0000","last_clean":"2024-05-08T23:21:05.354968+0000","last_became_active":"2024-05-08T23:21:04.106628+0000","last_became_peered":"2024-05-08T23:21:04.106628+0000","last_unstale":"2024-05-08T23:21:05.354968+0000","last_undegraded":"2024-05-08T23:21:05.354968+0000","last_fullsized":"2024-05-08T23:21:05.354968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.433453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356256+0000","last_change":"2024-05-08T23:21:04.105100+0000","last_active":"2024-05-08T23:21:05.356256+0000","last_peered":"2024-05-08T23:21:05.356256+0000","last_clean":"2024-05-08T23:21:05.356256+0000","last_became_active":"2024-05-08T23:21:04.104785+0000","last_became_peered":"2024-05-08T23:21:04.104785+0000","last_unstale":"2024-05-08T23:21:05.356256+0000","last_undegraded":"2024-05-08T23:21:05.356256+0000","last_fullsized":"2024-05-08T23:21:05.356256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:19.795476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355441+0000","last_change":"2024-05-08T23:21:04.105145+0000","last_active":"2024-05-08T23:21:05.355441+0000","last_peered":"2024-05-08T23:21:05.355441+0000","last_clean":"2024-05-08T23:21:05.355441+0000","last_became_active":"2024-05-08T23:21:04.104842+0000","last_became_peered":"2024-05-08T23:21:04.104842+0000","last_unstale":"2024-05-08T23:21:05.355441+0000","last_undegraded":"2024-05-08T23:21:05.355441+0000","last_fullsized":"2024-05-08T23:21:05.355441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092800+0000","last_change":"2024-05-08T23:21:04.106935+0000","last_active":"2024-05-08T23:21:05.092800+0000","last_peered":"2024-05-08T23:21:05.092800+0000","last_clean":"2024-05-08T23:21:05.092800+0000","last_became_active":"2024-05-08T23:21:04.106671+0000","last_became_peered":"2024-05-08T23:21:04.106671+0000","last_unstale":"2024-05-08T23:21:05.092800+0000","last_undegraded":"2024-05-08T23:21:05.092800+0000","last_fullsized":"2024-05-08T23:21:05.092800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:07.596384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093704+0000","last_change":"2024-05-08T23:21:04.112023+0000","last_active":"2024-05-08T23:21:05.093704+0000","last_peered":"2024-05-08T23:21:05.093704+0000","last_clean":"2024-05-08T23:21:05.093704+0000","last_became_active":"2024-05-08T23:21:04.111819+0000","last_became_peered":"2024-05-08T23:21:04.111819+0000","last_unstale":"2024-05-08T23:21:05.093704+0000","last_undegraded":"2024-05-08T23:21:05.093704+0000","last_fullsized":"2024-05-08T23:21:05.093704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355844+0000","last_change":"2024-05-08T23:21:04.111963+0000","last_active":"2024-05-08T23:21:05.355844+0000","last_peered":"2024-05-08T23:21:05.355844+0000","last_clean":"2024-05-08T23:21:05.355844+0000","last_became_active":"2024-05-08T23:21:04.111629+0000","last_became_peered":"2024-05-08T23:21:04.111629+0000","last_unstale":"2024-05-08T23:21:05.355844+0000","last_undegraded":"2024-05-08T23:21:05.355844+0000","last_fullsized":"2024-05-08T23:21:05.355844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092931+0000","last_change":"2024-05-08T23:21:04.113608+0000","last_active":"2024-05-08T23:21:05.092931+0000","last_peered":"2024-05-08T23:21:05.092931+0000","last_clean":"2024-05-08T23:21:05.092931+0000","last_became_active":"2024-05-08T23:21:04.113322+0000","last_became_peered":"2024-05-08T23:21:04.113322+0000","last_unstale":"2024-05-08T23:21:05.092931+0000","last_undegraded":"2024-05-08T23:21:05.092931+0000","last_fullsized":"2024-05-08T23:21:05.092931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356052+0000","last_change":"2024-05-08T23:21:04.109658+0000","last_active":"2024-05-08T23:21:05.356052+0000","last_peered":"2024-05-08T23:21:05.356052+0000","last_clean":"2024-05-08T23:21:05.356052+0000","last_became_active":"2024-05-08T23:21:04.109222+0000","last_became_peered":"2024-05-08T23:21:04.109222+0000","last_unstale":"2024-05-08T23:21:05.356052+0000","last_undegraded":"2024-05-08T23:21:05.356052+0000","last_fullsized":"2024-05-08T23:21:05.356052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092319+0000","last_change":"2024-05-08T23:21:04.107408+0000","last_active":"2024-05-08T23:21:05.092319+0000","last_peered":"2024-05-08T23:21:05.092319+0000","last_clean":"2024-05-08T23:21:05.092319+0000","last_became_active":"2024-05-08T23:21:04.107123+0000","last_became_peered":"2024-05-08T23:21:04.107123+0000","last_unstale":"2024-05-08T23:21:05.092319+0000","last_undegraded":"2024-05-08T23:21:05.092319+0000","last_fullsized":"2024-05-08T23:21:05.092319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356155+0000","last_change":"2024-05-08T23:21:04.108500+0000","last_active":"2024-05-08T23:21:05.356155+0000","last_peered":"2024-05-08T23:21:05.356155+0000","last_clean":"2024-05-08T23:21:05.356155+0000","last_became_active":"2024-05-08T23:21:04.107974+0000","last_became_peered":"2024-05-08T23:21:04.107974+0000","last_unstale":"2024-05-08T23:21:05.356155+0000","last_undegraded":"2024-05-08T23:21:05.356155+0000","last_fullsized":"2024-05-08T23:21:05.356155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355333+0000","last_change":"2024-05-08T23:21:04.112817+0000","last_active":"2024-05-08T23:21:05.355333+0000","last_peered":"2024-05-08T23:21:05.355333+0000","last_clean":"2024-05-08T23:21:05.355333+0000","last_became_active":"2024-05-08T23:21:04.112519+0000","last_became_peered":"2024-05-08T23:21:04.112519+0000","last_unstale":"2024-05-08T23:21:05.355333+0000","last_undegraded":"2024-05-08T23:21:05.355333+0000","last_fullsized":"2024-05-08T23:21:05.355333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356539+0000","last_change":"2024-05-08T23:21:04.113697+0000","last_active":"2024-05-08T23:21:05.356539+0000","last_peered":"2024-05-08T23:21:05.356539+0000","last_clean":"2024-05-08T23:21:05.356539+0000","last_became_active":"2024-05-08T23:21:04.113363+0000","last_became_peered":"2024-05-08T23:21:04.113363+0000","last_unstale":"2024-05-08T23:21:05.356539+0000","last_undegraded":"2024-05-08T23:21:05.356539+0000","last_fullsized":"2024-05-08T23:21:05.356539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:21:13.489 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:21:13.489 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T23:21:13.489 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T23:21:13.490 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T23:21:13.490 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T23:21:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.664+0000 7f09dfee5700 1 Processor -- start 2024-05-08T23:21:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.665+0000 7f09dfee5700 1 -- start start 2024-05-08T23:21:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.665+0000 7f09dfee5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09d8143340 0x7f09d815d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dfee5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d80cf9a0 con 0x7f09d815dcb0 2024-05-08T23:21:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dfee5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d80ceb90 con 0x7f09d80ce700 2024-05-08T23:21:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dfee5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d815e160 con 0x7f09d8143340 2024-05-08T23:21:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dd480700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f09d815dcb0 0x7f09d80cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44576/0 (socket says 172.21.15.5:44576) 2024-05-08T23:21:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09ddc81700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09d8143340 0x7f09d815d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dd480700 1 -- 172.21.15.5:0/430536502 learned_addr learned my addr 172.21.15.5:0/430536502 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09ddc81700 1 -- 172.21.15.5:0/430536502 >> v1:172.21.15.49:6789/0 conn(0x7f09d80ce700 legacy=0x7f09d80b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09ddc81700 1 -- 172.21.15.5:0/430536502 >> v1:172.21.15.5:6789/0 conn(0x7f09d815dcb0 legacy=0x7f09d80cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09ddc81700 1 -- 172.21.15.5:0/430536502 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80cee20 con 0x7f09d8143340 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dcc7f700 1 -- 172.21.15.5:0/430536502 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716556177 0 0) 0x7f09d4001110 con 0x7f09d815dcb0 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09ddc81700 1 --2- 172.21.15.5:0/430536502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d8143340 0x7f09d815d770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009840 tx=0x7f09cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f935c6cb0e492971 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dcc7f700 1 -- 172.21.15.5:0/430536502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc011830 con 0x7f09d8143340 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dcc7f700 1 -- 172.21.15.5:0/430536502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc011990 con 0x7f09d8143340 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.666+0000 7f09dcc7f700 1 -- 172.21.15.5:0/430536502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc011d20 con 0x7f09d8143340 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.667+0000 7f09dfee5700 1 -- 172.21.15.5:0/430536502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d8143340 msgr2=0x7f09d815d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.667+0000 7f09dfee5700 1 --2- 172.21.15.5:0/430536502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d8143340 0x7f09d815d770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009840 tx=0x7f09cc009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.667+0000 7f09dfee5700 1 -- 172.21.15.5:0/430536502 shutdown_connections 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.667+0000 7f09dfee5700 1 --2- 172.21.15.5:0/430536502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d8143340 0x7f09d815d770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.667+0000 7f09dfee5700 1 -- 172.21.15.5:0/430536502 >> 172.21.15.5:0/430536502 conn(0x7f09d80722a0 msgr2=0x7f09d806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.667+0000 7f09dfee5700 1 -- 172.21.15.5:0/430536502 shutdown_connections 2024-05-08T23:21:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.667+0000 7f09dfee5700 1 -- 172.21.15.5:0/430536502 wait complete. 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.668+0000 7f09dfee5700 1 Processor -- start 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.668+0000 7f09dfee5700 1 -- start start 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09dfee5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d80ce700 0x7f09d8159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09dfee5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8143340 0x7f09d81439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09dfee5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09d815dcb0 0x7f09d814e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09dfee5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d8139400 con 0x7f09d80ce700 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09dfee5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80b4100 con 0x7f09d8143340 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09dfee5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80b43e0 con 0x7f09d815dcb0 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09ddc81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d80ce700 0x7f09d8159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09ddc81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d80ce700 0x7f09d8159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60560/0 (socket says 172.21.15.5:60560) 2024-05-08T23:21:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.669+0000 7f09ddc81700 1 -- 172.21.15.5:0/521562857 learned_addr learned my addr 172.21.15.5:0/521562857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09dd480700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8143340 0x7f09d81439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09de482700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09d815dcb0 0x7f09d814e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ddc81700 1 -- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09d815dcb0 msgr2=0x7f09d814e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ddc81700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09d815dcb0 0x7f09d814e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ddc81700 1 -- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8143340 msgr2=0x7f09d81439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ddc81700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8143340 0x7f09d81439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ddc81700 1 -- 172.21.15.5:0/521562857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0094a0 con 0x7f09d80ce700 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09de482700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09d815dcb0 0x7f09d814e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ddc81700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d80ce700 0x7f09d8159950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0090b0 tx=0x7f09cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ca7fc700 1 -- 172.21.15.5:0/521562857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc01a900 con 0x7f09d80ce700 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ca7fc700 1 -- 172.21.15.5:0/521562857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc01aa60 con 0x7f09d80ce700 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d80b46c0 con 0x7f09d80ce700 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09ca7fc700 1 -- 172.21.15.5:0/521562857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc01add0 con 0x7f09d80ce700 2024-05-08T23:21:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.670+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d815b320 con 0x7f09d80ce700 2024-05-08T23:21:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.671+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d815b900 con 0x7f09d80ce700 2024-05-08T23:21:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.672+0000 7f09ca7fc700 1 -- 172.21.15.5:0/521562857 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f09cc022070 con 0x7f09d80ce700 2024-05-08T23:21:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.672+0000 7f09ca7fc700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09bc06ac90 0x7f09bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.672+0000 7f09ca7fc700 1 -- 172.21.15.5:0/521562857 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09cc08a640 con 0x7f09d80ce700 2024-05-08T23:21:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.672+0000 7f09dd480700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09bc06ac90 0x7f09bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.673+0000 7f09dd480700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09bc06ac90 0x7f09bc06d150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09d8159f50 tx=0x7f09d4007d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:13.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.677+0000 7f09ca7fc700 1 -- 172.21.15.5:0/521562857 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f09d815b900 con 0x7f09d80ce700 2024-05-08T23:21:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.960+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f09d815b900 con 0x7f09d80ce700 2024-05-08T23:21:13.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.964+0000 7f09ca7fc700 1 -- 172.21.15.5:0/521562857 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170937 (secure 0 0 0) 0x7f09d815b900 con 0x7f09d80ce700 2024-05-08T23:21:13.966 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09bc06ac90 msgr2=0x7f09bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09bc06ac90 0x7f09bc06d150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09d8159f50 tx=0x7f09d4007d90 comp rx=0 tx=0).stop 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d80ce700 msgr2=0x7f09d8159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d80ce700 0x7f09d8159950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0090b0 tx=0x7f09cc004750 comp rx=0 tx=0).stop 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 shutdown_connections 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09bc06ac90 0x7f09bc06d150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09d815dcb0 0x7f09d814e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09d80ce700 0x7f09d8159950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 --2- 172.21.15.5:0/521562857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09d8143340 0x7f09d81439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:13.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 >> 172.21.15.5:0/521562857 conn(0x7f09d80722a0 msgr2=0x7f09d814f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:13.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 shutdown_connections 2024-05-08T23:21:13.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:13.967+0000 7f09dfee5700 1 -- 172.21.15.5:0/521562857 wait complete. 2024-05-08T23:21:13.981 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":46734},{"type":"v1","addr":"172.21.15.49:6833","nonce":46734}]},"active_addr":"172.21.15.49:6833/46734","active_change":"2024-05-08T23:20:51.284558+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":4183644670}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3392233889}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1572162438}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1699767123}]}]} 2024-05-08T23:21:13.984 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T23:21:13.984 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T23:21:13.984 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:14.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.122+0000 7fc6e3b81700 1 Processor -- start 2024-05-08T23:21:14.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.122+0000 7fc6e3b81700 1 -- start start 2024-05-08T23:21:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e3b81700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc6dc09a6d0 0x7fc6dc162f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e3b81700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc06c210 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e3b81700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc099bf0 con 0x7fc6dc162b40 2024-05-08T23:21:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e3b81700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc06c4f0 con 0x7fc6dc09a6d0 2024-05-08T23:21:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e211e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc6dc09a6d0 0x7fc6dc162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e111c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc6dc0c4c00 0x7fc6dc06ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44586/0 (socket says 172.21.15.5:44586) 2024-05-08T23:21:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e111c700 1 -- 172.21.15.5:0/3974257510 learned_addr learned my addr 172.21.15.5:0/3974257510 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e211e700 1 -- 172.21.15.5:0/3974257510 >> v1:172.21.15.49:6789/0 conn(0x7fc6dc162b40 legacy=0x7fc6dc0c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e211e700 1 -- 172.21.15.5:0/3974257510 >> v1:172.21.15.5:6789/0 conn(0x7fc6dc0c4c00 legacy=0x7fc6dc06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.123+0000 7fc6e211e700 1 -- 172.21.15.5:0/3974257510 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc099ed0 con 0x7fc6dc09a6d0 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e211e700 1 --2- 172.21.15.5:0/3974257510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc09a6d0 0x7fc6dc162f20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009d90 tx=0x7fc6d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ccca1e42a115e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e091b700 1 -- 172.21.15.5:0/3974257510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d400e070 con 0x7fc6dc09a6d0 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e091b700 1 -- 172.21.15.5:0/3974257510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d400bbd0 con 0x7fc6dc09a6d0 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e091b700 1 -- 172.21.15.5:0/3974257510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4012430 con 0x7fc6dc09a6d0 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e3b81700 1 -- 172.21.15.5:0/3974257510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc09a6d0 msgr2=0x7fc6dc162f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e3b81700 1 --2- 172.21.15.5:0/3974257510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc09a6d0 0x7fc6dc162f20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009d90 tx=0x7fc6d400b330 comp rx=0 tx=0).stop 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e3b81700 1 -- 172.21.15.5:0/3974257510 shutdown_connections 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e3b81700 1 --2- 172.21.15.5:0/3974257510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc09a6d0 0x7fc6dc162f20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e3b81700 1 -- 172.21.15.5:0/3974257510 >> 172.21.15.5:0/3974257510 conn(0x7fc6dc0b3010 msgr2=0x7fc6dc0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e3b81700 1 -- 172.21.15.5:0/3974257510 shutdown_connections 2024-05-08T23:21:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.124+0000 7fc6e3b81700 1 -- 172.21.15.5:0/3974257510 wait complete. 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.126+0000 7fc6e3b81700 1 Processor -- start 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.126+0000 7fc6e3b81700 1 -- start start 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e3b81700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6dc09a6d0 0x7fc6dc0c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e3b81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc0c4c00 0x7fc6dc1f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e3b81700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6dc162b40 0x7fc6dc0af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e3b81700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0cb880 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e3b81700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0ca870 con 0x7fc6dc162b40 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e3b81700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0c7d30 con 0x7fc6dc09a6d0 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc0c4c00 0x7fc6dc1f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc0c4c00 0x7fc6dc1f7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60592/0 (socket says 172.21.15.5:60592) 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 -- 172.21.15.5:0/2807103006 learned_addr learned my addr 172.21.15.5:0/2807103006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e211e700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6dc162b40 0x7fc6dc0af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e191d700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6dc09a6d0 0x7fc6dc0c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 -- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6dc09a6d0 msgr2=0x7fc6dc0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6dc09a6d0 0x7fc6dc0c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 -- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6dc162b40 msgr2=0x7fc6dc0af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6dc162b40 0x7fc6dc0af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 -- 172.21.15.5:0/2807103006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d4009a70 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e111c700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc0c4c00 0x7fc6dc1f7e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c520 tx=0x7fc6cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6d27fc700 1 -- 172.21.15.5:0/2807103006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc005d70 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6d27fc700 1 -- 172.21.15.5:0/2807103006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc0070d0 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.127+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc0caac0 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.128+0000 7fc6d27fc700 1 -- 172.21.15.5:0/2807103006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc007440 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.128+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc0cb080 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.128+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc070020 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.129+0000 7fc6d27fc700 1 -- 172.21.15.5:0/2807103006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc6cc0075a0 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.130+0000 7fc6d27fc700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6c006ac90 0x7fc6c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.130+0000 7fc6d27fc700 1 -- 172.21.15.5:0/2807103006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6cc088ec0 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.130+0000 7fc6e191d700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6c006ac90 0x7fc6c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.130+0000 7fc6e191d700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6c006ac90 0x7fc6c006d150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8000f60 tx=0x7fc6d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.134+0000 7fc6d27fc700 1 -- 172.21.15.5:0/2807103006 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc6cc054230 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.352+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6dc068000 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.353+0000 7fc6d27fc700 1 -- 172.21.15.5:0/2807103006 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7fc6cc057850 con 0x7fc6dc0c4c00 2024-05-08T23:21:14.353 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:14.353 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:14.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6c006ac90 msgr2=0x7fc6c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6c006ac90 0x7fc6c006d150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8000f60 tx=0x7fc6d8009040 comp rx=0 tx=0).stop 2024-05-08T23:21:14.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc0c4c00 msgr2=0x7fc6dc1f7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc0c4c00 0x7fc6dc1f7e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c520 tx=0x7fc6cc00c8e0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 shutdown_connections 2024-05-08T23:21:14.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6c006ac90 0x7fc6c006d150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6dc162b40 0x7fc6dc0af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6dc09a6d0 0x7fc6dc0c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 --2- 172.21.15.5:0/2807103006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc0c4c00 0x7fc6dc1f7e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 >> 172.21.15.5:0/2807103006 conn(0x7fc6dc0b3010 msgr2=0x7fc6dc1613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:14.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 shutdown_connections 2024-05-08T23:21:14.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.356+0000 7fc6e3b81700 1 -- 172.21.15.5:0/2807103006 wait complete. 2024-05-08T23:21:14.370 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T23:21:14.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:14.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.506+0000 7fb270c1d700 1 Processor -- start 2024-05-08T23:21:14.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.506+0000 7fb270c1d700 1 -- start start 2024-05-08T23:21:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.507+0000 7fb270c1d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c13e5f0 0x7fb26c15fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.507+0000 7fb270c1d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb26c15c200 0x7fb26c15c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.507+0000 7fb270c1d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c0cf900 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.507+0000 7fb270c1d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c15baf0 con 0x7fb26c15c200 2024-05-08T23:21:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb270c1d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c15cb60 con 0x7fb26c13e5f0 2024-05-08T23:21:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb269d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb26c0ce8d0 0x7fb26c06a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44600/0 (socket says 172.21.15.5:44600) 2024-05-08T23:21:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb269d9b700 1 -- 172.21.15.5:0/1389275764 learned_addr learned my addr 172.21.15.5:0/1389275764 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26a59c700 1 --2- 172.21.15.5:0/1389275764 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c13e5f0 0x7fb26c15fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26ad9d700 1 --2- 172.21.15.5:0/1389275764 >> v2:172.21.15.49:3300/0 conn(0x7fb26c15c200 0x7fb26c15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26959a700 1 -- 172.21.15.5:0/1389275764 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658754322 0 0) 0x7fb26c0cf900 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26959a700 1 -- 172.21.15.5:0/1389275764 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2540035b0 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26a59c700 1 -- 172.21.15.5:0/1389275764 >> v2:172.21.15.49:3300/0 conn(0x7fb26c15c200 msgr2=0x7fb26c15c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26a59c700 1 --2- 172.21.15.5:0/1389275764 >> v2:172.21.15.49:3300/0 conn(0x7fb26c15c200 0x7fb26c15c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26a59c700 1 -- 172.21.15.5:0/1389275764 >> v1:172.21.15.5:6789/0 conn(0x7fb26c0ce8d0 legacy=0x7fb26c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26a59c700 1 -- 172.21.15.5:0/1389275764 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c15bd70 con 0x7fb26c13e5f0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.508+0000 7fb26a59c700 1 --2- 172.21.15.5:0/1389275764 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c13e5f0 0x7fb26c15fbb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009840 tx=0x7fb25c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2940075db047a723 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb26959a700 1 -- 172.21.15.5:0/1389275764 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c011830 con 0x7fb26c13e5f0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb26959a700 1 -- 172.21.15.5:0/1389275764 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c011990 con 0x7fb26c13e5f0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb26959a700 1 -- 172.21.15.5:0/1389275764 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c011d20 con 0x7fb26c13e5f0 2024-05-08T23:21:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb270c1d700 1 -- 172.21.15.5:0/1389275764 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c13e5f0 msgr2=0x7fb26c15fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb270c1d700 1 --2- 172.21.15.5:0/1389275764 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c13e5f0 0x7fb26c15fbb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009840 tx=0x7fb25c009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb270c1d700 1 -- 172.21.15.5:0/1389275764 shutdown_connections 2024-05-08T23:21:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb270c1d700 1 --2- 172.21.15.5:0/1389275764 >> v2:172.21.15.49:3300/0 conn(0x7fb26c15c200 0x7fb26c15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb270c1d700 1 --2- 172.21.15.5:0/1389275764 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c13e5f0 0x7fb26c15fbb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb270c1d700 1 -- 172.21.15.5:0/1389275764 >> 172.21.15.5:0/1389275764 conn(0x7fb26c0cfce0 msgr2=0x7fb26c0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb270c1d700 1 -- 172.21.15.5:0/1389275764 shutdown_connections 2024-05-08T23:21:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.509+0000 7fb270c1d700 1 -- 172.21.15.5:0/1389275764 wait complete. 2024-05-08T23:21:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.511+0000 7fb270c1d700 1 Processor -- start 2024-05-08T23:21:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.511+0000 7fb270c1d700 1 -- start start 2024-05-08T23:21:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb270c1d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c0ce8d0 0x7fb26c13d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb270c1d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26c13e5f0 0x7fb26c13ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb270c1d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c15c200 0x7fb26c13f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb270c1d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0c8f30 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb270c1d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c13dd60 con 0x7fb26c13e5f0 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb270c1d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0c91a0 con 0x7fb26c15c200 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c0ce8d0 0x7fb26c13d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c0ce8d0 0x7fb26c13d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60596/0 (socket says 172.21.15.5:60596) 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 -- 172.21.15.5:0/2068697639 learned_addr learned my addr 172.21.15.5:0/2068697639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb269d9b700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26c13e5f0 0x7fb26c13ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26ad9d700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c15c200 0x7fb26c13f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 -- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c15c200 msgr2=0x7fb26c13f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c15c200 0x7fb26c13f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 -- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26c13e5f0 msgr2=0x7fb26c13ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26c13e5f0 0x7fb26c13ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 -- 172.21.15.5:0/2068697639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c0094a0 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb26a59c700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c0ce8d0 0x7fb26c13d820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0090b0 tx=0x7fb25c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.512+0000 7fb25affd700 1 -- 172.21.15.5:0/2068697639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0036a0 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.513+0000 7fb25affd700 1 -- 172.21.15.5:0/2068697639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c01f070 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.513+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c0c9480 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.513+0000 7fb25affd700 1 -- 172.21.15.5:0/2068697639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c003a40 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.513+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c0b44a0 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.514+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c068000 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.514+0000 7fb25affd700 1 -- 172.21.15.5:0/2068697639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb25c003ba0 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.515+0000 7fb25affd700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb24c06ac90 0x7fb24c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.515+0000 7fb25affd700 1 -- 172.21.15.5:0/2068697639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb25c021070 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.515+0000 7fb269d9b700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb24c06ac90 0x7fb24c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.515+0000 7fb269d9b700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb24c06ac90 0x7fb24c06d150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0b4860 tx=0x7fb260009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:14.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.520+0000 7fb25affd700 1 -- 172.21.15.5:0/2068697639 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb25c055da0 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.737+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb26c15d900 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.738+0000 7fb25affd700 1 -- 172.21.15.5:0/2068697639 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7fb25c02b090 con 0x7fb26c0ce8d0 2024-05-08T23:21:14.738 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:14.739 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb24c06ac90 msgr2=0x7fb24c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb24c06ac90 0x7fb24c06d150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0b4860 tx=0x7fb260009040 comp rx=0 tx=0).stop 2024-05-08T23:21:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c0ce8d0 msgr2=0x7fb26c13d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c0ce8d0 0x7fb26c13d820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0090b0 tx=0x7fb25c004750 comp rx=0 tx=0).stop 2024-05-08T23:21:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 shutdown_connections 2024-05-08T23:21:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb24c06ac90 0x7fb24c06d150 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb26c15c200 0x7fb26c13f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c0ce8d0 0x7fb26c13d820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 --2- 172.21.15.5:0/2068697639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26c13e5f0 0x7fb26c13ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:14.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 >> 172.21.15.5:0/2068697639 conn(0x7fb26c0cfce0 msgr2=0x7fb26c0d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:14.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 shutdown_connections 2024-05-08T23:21:14.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.741+0000 7fb270c1d700 1 -- 172.21.15.5:0/2068697639 wait complete. 2024-05-08T23:21:14.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T23:21:14.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T23:21:14.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T23:21:14.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T23:21:14.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T23:21:14.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T23:21:14.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T23:21:14.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T23:21:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7fde4fbde700 1 Processor -- start 2024-05-08T23:21:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7fde4fbde700 1 -- start start 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.997+0000 7f55155b3700 1 Processor -- start 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.997+0000 7f55155b3700 1 -- start start 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f55155b3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5510098a80 0x7f551006ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f55155b3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55100b1510 0x7f55100ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f55155b3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551006d4b0 con 0x7f551006e4f0 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f55155b3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551006d790 con 0x7f5510098a80 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f55155b3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551006da70 con 0x7f55100b1510 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f550effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5510098a80 0x7f551006ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f550effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5510098a80 0x7f551006ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60600/0 (socket says 172.21.15.5:60600) 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f550effd700 1 -- 172.21.15.5:0/4133759993 learned_addr learned my addr 172.21.15.5:0/4133759993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:14.999+0000 7f550effd700 1 -- 172.21.15.5:0/4133759993 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55100b1510 msgr2=0x7f55100ac3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f550effd700 1 --2- 172.21.15.5:0/4133759993 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55100b1510 0x7f55100ac3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f550effd700 1 -- 172.21.15.5:0/4133759993 >> v1:172.21.15.5:6789/0 conn(0x7f551006e4f0 legacy=0x7f55100b0e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:15.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f550effd700 1 -- 172.21.15.5:0/4133759993 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551006dd50 con 0x7f5510098a80 2024-05-08T23:21:15.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f44501f9700 1 Processor -- start 2024-05-08T23:21:15.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fa973879700 1 Processor -- start 2024-05-08T23:21:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f550effd700 1 --2- 172.21.15.5:0/4133759993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5510098a80 0x7f551006ce20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55000078e0 tx=0x7f5500007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3983128cb31ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fde4fbde700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde480b1510 0x7fde480b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fde4fbde700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fde48163440 0x7fde480b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fde4fbde700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4806ce20 con 0x7fde4815c380 2024-05-08T23:21:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fde4fbde700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4806da10 con 0x7fde48163440 2024-05-08T23:21:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fde4fbde700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde481656b0 con 0x7fde480b1510 2024-05-08T23:21:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f550dffb700 1 -- 172.21.15.5:0/4133759993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550000f070 con 0x7f5510098a80 2024-05-08T23:21:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f550dffb700 1 -- 172.21.15.5:0/4133759993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500004030 con 0x7f5510098a80 2024-05-08T23:21:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f550dffb700 1 -- 172.21.15.5:0/4133759993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000043c0 con 0x7f5510098a80 2024-05-08T23:21:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f1d00ff5700 1 Processor -- start 2024-05-08T23:21:15.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fde4e17b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fde48163440 0x7fde480b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fde4e17b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fde48163440 0x7fde480b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36206/0 (socket says 172.21.15.5:36206) 2024-05-08T23:21:15.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fde4e17b700 1 -- 172.21.15.5:0/1253289020 learned_addr learned my addr 172.21.15.5:0/1253289020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.001+0000 7f1d00ff5700 1 -- start start 2024-05-08T23:21:15.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.001+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210211297 0 0) 0x7fde4806ce20 con 0x7fde4815c380 2024-05-08T23:21:15.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.001+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde340035b0 con 0x7fde4815c380 2024-05-08T23:21:15.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7fa973879700 1 -- start start 2024-05-08T23:21:15.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.001+0000 7fa973879700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa96c153f20 0x7fa96c0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.001+0000 7fa973879700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 0x7fa96c1633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.001+0000 7fa973879700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c0ac1c0 con 0x7fa96c15aae0 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.001+0000 7fa973879700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96c154300 con 0x7fa96c153f20 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.001+0000 7fa973879700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c06ccb0 con 0x7fa96c0b19d0 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa970e14700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa96c15aae0 0x7fa96c0b12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44614/0 (socket says 172.21.15.5:44614) 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa970e14700 1 -- 172.21.15.5:0/697062940 learned_addr learned my addr 172.21.15.5:0/697062940 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa971615700 1 --2- 172.21.15.5:0/697062940 >> v2:172.21.15.5:3300/0 conn(0x7fa96c153f20 0x7fa96c0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa971e16700 1 --2- 172.21.15.5:0/697062940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 0x7fa96c1633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.000+0000 7f44501f9700 1 -- start start 2024-05-08T23:21:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f44501f9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44480cd4e0 0x7f444806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f44501f9700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f444806f130 0x7f44480ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f44501f9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f44480ceb20 0x7f444815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f44501f9700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44480cefb0 con 0x7f44480ceb20 2024-05-08T23:21:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f44501f9700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444815c280 con 0x7f444806f130 2024-05-08T23:21:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f44501f9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444806e780 con 0x7f44480cd4e0 2024-05-08T23:21:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f444df95700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44480cd4e0 0x7f444806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa963fff700 1 -- 172.21.15.5:0/697062940 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267321230 0 0) 0x7fa96c0ac1c0 con 0x7fa96c15aae0 2024-05-08T23:21:15.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa963fff700 1 -- 172.21.15.5:0/697062940 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9580035b0 con 0x7fa96c15aae0 2024-05-08T23:21:15.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f444df95700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44480cd4e0 0x7f444806f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:47928/0 (socket says 172.21.15.5:47928) 2024-05-08T23:21:15.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f444df95700 1 -- 172.21.15.5:0/2435586131 learned_addr learned my addr 172.21.15.5:0/2435586131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 466041264 0 0) 0x7fde340035b0 con 0x7fde4815c380 2024-05-08T23:21:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde480b1510 msgr2=0x7fde480b2250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde480b1510 0x7fde480b2250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 >> v2:172.21.15.49:3300/0 conn(0x7fde48163440 msgr2=0x7fde480b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 --2- 172.21.15.5:0/1253289020 >> v2:172.21.15.49:3300/0 conn(0x7fde48163440 0x7fde480b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4815c760 con 0x7fde4815c380 2024-05-08T23:21:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fde38001670 con 0x7fde4815c380 2024-05-08T23:21:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 >> v1:172.21.15.5:6789/0 conn(0x7fde4815c380 legacy=0x7fde480b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34003a00 0x7fde34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde34004850 0x7fde34004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde34005210 0x7fde3400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4806da10 con 0x7fde34003a00 2024-05-08T23:21:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3400c080 con 0x7fde34005210 2024-05-08T23:21:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3400c240 con 0x7fde34004850 2024-05-08T23:21:15.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f444d794700 1 --2- 172.21.15.5:0/2435586131 >> v2:172.21.15.49:3300/0 conn(0x7f444806f130 0x7f44480ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f1d00ff5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1cfc15c000 0x7f1cfc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f1d00ff5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1cfc15c3e0 0x7f1cfc06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f1d00ff5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc0b48b0 con 0x7f1cfc0b35e0 2024-05-08T23:21:15.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f1d00ff5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc06dee0 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7f1d00ff5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc0b42f0 con 0x7f1cfc15c000 2024-05-08T23:21:15.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa971e16700 1 -- 172.21.15.5:0/697062940 >> v2:172.21.15.5:3300/0 conn(0x7fa96c153f20 msgr2=0x7fa96c0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa971e16700 1 --2- 172.21.15.5:0/697062940 >> v2:172.21.15.5:3300/0 conn(0x7fa96c153f20 0x7fa96c0b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa971e16700 1 -- 172.21.15.5:0/697062940 >> v1:172.21.15.5:6789/0 conn(0x7fa96c15aae0 legacy=0x7fa96c0b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.002+0000 7fa971e16700 1 -- 172.21.15.5:0/697062940 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c06cf90 con 0x7fa96c0b19d0 2024-05-08T23:21:15.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f444e796700 1 --2- 172.21.15.5:0/2435586131 >> v2:172.21.15.5:3300/0 conn(0x7f44480ceb20 0x7f444815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f1cfa59c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1cfc15c000 0x7f1cfc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f1cfad9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1cfc0b35e0 0x7f1cfc0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44618/0 (socket says 172.21.15.5:44618) 2024-05-08T23:21:15.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f1cfad9d700 1 -- 172.21.15.5:0/912192610 learned_addr learned my addr 172.21.15.5:0/912192610 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f444d794700 1 -- 172.21.15.5:0/2435586131 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44480cd4e0 msgr2=0x7f444806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f444d794700 1 --2- 172.21.15.5:0/2435586131 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44480cd4e0 0x7f444806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f444d794700 1 -- 172.21.15.5:0/2435586131 >> v2:172.21.15.5:3300/0 conn(0x7f44480ceb20 msgr2=0x7f444815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f444d794700 1 --2- 172.21.15.5:0/2435586131 >> v2:172.21.15.5:3300/0 conn(0x7f44480ceb20 0x7f444815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f444d794700 1 -- 172.21.15.5:0/2435586131 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444806ea60 con 0x7f444806f130 2024-05-08T23:21:15.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f1cf9d9b700 1 --2- 172.21.15.5:0/912192610 >> v2:172.21.15.5:3300/0 conn(0x7f1cfc15c3e0 0x7f1cfc06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7fde4e17b700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde34005210 0x7fde3400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7fa971e16700 1 --2- 172.21.15.5:0/697062940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 0x7fa96c1633e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa96400bfa0 tx=0x7fa96400d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b40c72566a41bb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7fde4d97a700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34003a00 0x7fde34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f1cfa59c700 1 -- 172.21.15.5:0/912192610 >> v2:172.21.15.5:3300/0 conn(0x7f1cfc15c3e0 msgr2=0x7f1cfc06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f1cfa59c700 1 --2- 172.21.15.5:0/912192610 >> v2:172.21.15.5:3300/0 conn(0x7f1cfc15c3e0 0x7f1cfc06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f1cfa59c700 1 -- 172.21.15.5:0/912192610 >> v1:172.21.15.5:6789/0 conn(0x7f1cfc0b35e0 legacy=0x7f1cfc0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f1cfa59c700 1 -- 172.21.15.5:0/912192610 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc0b45d0 con 0x7f1cfc15c000 2024-05-08T23:21:15.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f1cfa59c700 1 --2- 172.21.15.5:0/912192610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cfc15c000 0x7f1cfc06b970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc070fe0 tx=0x7f1cec009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475bdc13222ca649 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fa963fff700 1 -- 172.21.15.5:0/697062940 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964010070 con 0x7fa96c0b19d0 2024-05-08T23:21:15.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fa963fff700 1 -- 172.21.15.5:0/697062940 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964003a20 con 0x7fa96c0b19d0 2024-05-08T23:21:15.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fa963fff700 1 -- 172.21.15.5:0/697062940 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964003dd0 con 0x7fa96c0b19d0 2024-05-08T23:21:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f55155b3700 1 -- 172.21.15.5:0/4133759993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5510098a80 msgr2=0x7f551006ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f55155b3700 1 --2- 172.21.15.5:0/4133759993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5510098a80 0x7f551006ce20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55000078e0 tx=0x7f5500007bf0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fde4d179700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde34004850 0x7fde34004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.003+0000 7f444d794700 1 --2- 172.21.15.5:0/2435586131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f444806f130 0x7f44480ce5e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44480706a0 tx=0x7f4438009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1391d3e547960691 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f55155b3700 1 -- 172.21.15.5:0/4133759993 shutdown_connections 2024-05-08T23:21:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f55155b3700 1 --2- 172.21.15.5:0/4133759993 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55100b1510 0x7f55100ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f55155b3700 1 --2- 172.21.15.5:0/4133759993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5510098a80 0x7f551006ce20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f55155b3700 1 -- 172.21.15.5:0/4133759993 >> 172.21.15.5:0/4133759993 conn(0x7f55100b4690 msgr2=0x7f55100b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7f55155b3700 1 -- 172.21.15.5:0/4133759993 shutdown_connections 2024-05-08T23:21:15.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fde4d97a700 1 -- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde34004850 msgr2=0x7fde34004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fde4d97a700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde34004850 0x7fde34004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fde4d97a700 1 -- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde34005210 msgr2=0x7fde3400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fde4d97a700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde34005210 0x7fde3400ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.004+0000 7fde4d97a700 1 -- 172.21.15.5:0/1253289020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4815c760 con 0x7fde34003a00 2024-05-08T23:21:15.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fda72b12700 1 Processor -- start 2024-05-08T23:21:15.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fde4d97a700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34003a00 0x7fde34003e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c290 tx=0x7fde34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d355c567fad2b154 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7f1cf959a700 1 -- 172.21.15.5:0/912192610 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec004230 con 0x7f1cfc15c000 2024-05-08T23:21:15.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7f1cf959a700 1 -- 172.21.15.5:0/912192610 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec004390 con 0x7f1cfc15c000 2024-05-08T23:21:15.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7f1cf959a700 1 -- 172.21.15.5:0/912192610 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0036a0 con 0x7f1cfc15c000 2024-05-08T23:21:15.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fa973879700 1 -- 172.21.15.5:0/697062940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 msgr2=0x7fa96c1633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fa973879700 1 --2- 172.21.15.5:0/697062940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 0x7fa96c1633e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa96400bfa0 tx=0x7fa96400d330 comp rx=0 tx=0).stop 2024-05-08T23:21:15.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7f55155b3700 1 -- 172.21.15.5:0/4133759993 wait complete. 2024-05-08T23:21:15.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fa973879700 1 -- 172.21.15.5:0/697062940 shutdown_connections 2024-05-08T23:21:15.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fa973879700 1 --2- 172.21.15.5:0/697062940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 0x7fa96c1633e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fa973879700 1 --2- 172.21.15.5:0/697062940 >> v2:172.21.15.5:3300/0 conn(0x7fa96c153f20 0x7fa96c0b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fa973879700 1 -- 172.21.15.5:0/697062940 >> 172.21.15.5:0/697062940 conn(0x7fa96c0b4690 msgr2=0x7fa96c0b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3800cd90 con 0x7fde34003a00 2024-05-08T23:21:15.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde38012420 con 0x7fde34003a00 2024-05-08T23:21:15.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fde4c978700 1 -- 172.21.15.5:0/1253289020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380127d0 con 0x7fde34003a00 2024-05-08T23:21:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7fa973879700 1 -- 172.21.15.5:0/697062940 shutdown_connections 2024-05-08T23:21:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7f1d00ff5700 1 -- 172.21.15.5:0/912192610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cfc15c000 msgr2=0x7f1cfc06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7f1d00ff5700 1 --2- 172.21.15.5:0/912192610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cfc15c000 0x7f1cfc06b970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc070fe0 tx=0x7f1cec009b20 comp rx=0 tx=0).stop 2024-05-08T23:21:15.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.005+0000 7fda72b12700 1 -- start start 2024-05-08T23:21:15.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7fda72b12700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fda6c15c000 0x7fda6c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7fda72b12700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0b3cf0 0x7fda6c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7fda72b12700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c0b48b0 con 0x7fda6c15c3e0 2024-05-08T23:21:15.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7fda72b12700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6c06dee0 con 0x7fda6c15c000 2024-05-08T23:21:15.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7fda72b12700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c0b42f0 con 0x7fda6c0b3cf0 2024-05-08T23:21:15.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fde4fbde700 1 -- 172.21.15.5:0/1253289020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34003a00 msgr2=0x7fde34003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fde4fbde700 1 --2- 172.21.15.5:0/1253289020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34003a00 0x7fde34003e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c290 tx=0x7fde34005880 comp rx=0 tx=0).stop 2024-05-08T23:21:15.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7f1d00ff5700 1 -- 172.21.15.5:0/912192610 shutdown_connections 2024-05-08T23:21:15.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7f1d00ff5700 1 --2- 172.21.15.5:0/912192610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cfc15c000 0x7f1cfc06b970 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7f1d00ff5700 1 --2- 172.21.15.5:0/912192610 >> v2:172.21.15.5:3300/0 conn(0x7f1cfc15c3e0 0x7f1cfc06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7f1d00ff5700 1 -- 172.21.15.5:0/912192610 >> 172.21.15.5:0/912192610 conn(0x7f1cfc0cd550 msgr2=0x7f1cfc0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7f55155b3700 1 Processor -- start 2024-05-08T23:21:15.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda6bfff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fda6c15c3e0 0x7fda6c0b35e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:59996/0 (socket says 172.21.15.5:59996) 2024-05-08T23:21:15.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda6bfff700 1 -- 172.21.15.5:0/1800309429 learned_addr learned my addr 172.21.15.5:0/1800309429 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7f55155b3700 1 -- start start 2024-05-08T23:21:15.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda708ae700 1 --2- 172.21.15.5:0/1800309429 >> v2:172.21.15.5:3300/0 conn(0x7fda6c15c000 0x7fda6c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda710af700 1 --2- 172.21.15.5:0/1800309429 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0b3cf0 0x7fda6c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda6b7fe700 1 -- 172.21.15.5:0/1800309429 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964842401 0 0) 0x7fda60001110 con 0x7fda6c15c3e0 2024-05-08T23:21:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda6b7fe700 1 -- 172.21.15.5:0/1800309429 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda540035b0 con 0x7fda6c15c3e0 2024-05-08T23:21:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda708ae700 1 -- 172.21.15.5:0/1800309429 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0b3cf0 msgr2=0x7fda6c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda708ae700 1 --2- 172.21.15.5:0/1800309429 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0b3cf0 0x7fda6c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda708ae700 1 -- 172.21.15.5:0/1800309429 >> v1:172.21.15.49:6789/0 conn(0x7fda6c15c3e0 legacy=0x7fda6c0b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.007+0000 7fda708ae700 1 -- 172.21.15.5:0/1800309429 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0b45d0 con 0x7fda6c15c000 2024-05-08T23:21:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fde4e17b700 1 -- 172.21.15.5:0/1253289020 reap_dead start 2024-05-08T23:21:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.006+0000 7fa973879700 1 -- 172.21.15.5:0/697062940 wait complete. 2024-05-08T23:21:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fde4fbde700 1 -- 172.21.15.5:0/1253289020 shutdown_connections 2024-05-08T23:21:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fde4fbde700 1 -- 172.21.15.5:0/1253289020 >> 172.21.15.5:0/1253289020 conn(0x7fde480b4690 msgr2=0x7fde480b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fa973879700 1 Processor -- start 2024-05-08T23:21:15.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fda708ae700 1 --2- 172.21.15.5:0/1800309429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15c000 0x7fda6c06b970 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda64009d90 tx=0x7fda6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6968f9a733b8d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fde4fbde700 1 -- 172.21.15.5:0/1253289020 shutdown_connections 2024-05-08T23:21:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fde4fbde700 1 -- 172.21.15.5:0/1253289020 wait complete. 2024-05-08T23:21:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f55155b3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f551006e4f0 0x7f551008df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f55155b3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5510098a80 0x7f55100c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f55155b3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55100b1510 0x7f55100aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f55155b3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551008e460 con 0x7f55100b1510 2024-05-08T23:21:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f55155b3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100aef00 con 0x7f551006e4f0 2024-05-08T23:21:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f55155b3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100af1a0 con 0x7f5510098a80 2024-05-08T23:21:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f550e7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5510098a80 0x7f55100c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fda6b7fe700 1 -- 172.21.15.5:0/1800309429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400e070 con 0x7fda6c15c000 2024-05-08T23:21:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fda6b7fe700 1 -- 172.21.15.5:0/1800309429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6400bb20 con 0x7fda6c15c000 2024-05-08T23:21:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fda6b7fe700 1 -- 172.21.15.5:0/1800309429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64012430 con 0x7fda6c15c000 2024-05-08T23:21:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f550effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f551006e4f0 0x7f551008df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f550e7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5510098a80 0x7f55100c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:47972/0 (socket says 172.21.15.5:47972) 2024-05-08T23:21:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7f550e7fc700 1 -- 172.21.15.5:0/2262639176 learned_addr learned my addr 172.21.15.5:0/2262639176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f444cf93700 1 -- 172.21.15.5:0/2435586131 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438012b80 con 0x7f444806f130 2024-05-08T23:21:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f444cf93700 1 -- 172.21.15.5:0/2435586131 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4438012ce0 con 0x7f444806f130 2024-05-08T23:21:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f444cf93700 1 -- 172.21.15.5:0/2435586131 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443801b620 con 0x7f444806f130 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f550f7fe700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55100b1510 0x7f55100aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f550effd700 1 -- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5510098a80 msgr2=0x7f55100c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f550effd700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5510098a80 0x7f55100c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f550effd700 1 -- 172.21.15.5:0/2262639176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55100b1510 msgr2=0x7f55100aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f550effd700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55100b1510 0x7f55100aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f550effd700 1 -- 172.21.15.5:0/2262639176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55000075c0 con 0x7f551006e4f0 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7f550effd700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f551006e4f0 0x7f551008df20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5500015000 tx=0x7f5500004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.008+0000 7fa973879700 1 -- start start 2024-05-08T23:21:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa973879700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 0x7fa96c0986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa973879700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa96c153f20 0x7fa96c14e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa973879700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa96c15aae0 0x7fa96c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa973879700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c2021e0 con 0x7fa96c15aae0 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa973879700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c201560 con 0x7fa96c153f20 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa973879700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c201840 con 0x7fa96c0b19d0 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa971615700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 0x7fa96c0986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa970e14700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa96c153f20 0x7fa96c14e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa970e14700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa96c153f20 0x7fa96c14e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36244/0 (socket says 172.21.15.5:36244) 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fa970e14700 1 -- 172.21.15.5:0/2257648563 learned_addr learned my addr 172.21.15.5:0/2257648563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fda72b12700 1 -- 172.21.15.5:0/1800309429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15c000 msgr2=0x7fda6c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fda72b12700 1 --2- 172.21.15.5:0/1800309429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15c000 0x7fda6c06b970 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda64009d90 tx=0x7fda6400b330 comp rx=0 tx=0).stop 2024-05-08T23:21:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fda72b12700 1 -- 172.21.15.5:0/1800309429 shutdown_connections 2024-05-08T23:21:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fda72b12700 1 --2- 172.21.15.5:0/1800309429 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0b3cf0 0x7fda6c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fda72b12700 1 --2- 172.21.15.5:0/1800309429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15c000 0x7fda6c06b970 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.009+0000 7fda72b12700 1 -- 172.21.15.5:0/1800309429 >> 172.21.15.5:0/1800309429 conn(0x7fda6c0cd550 msgr2=0x7fda6c0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f44501f9700 1 -- 172.21.15.5:0/2435586131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f444806f130 msgr2=0x7f44480ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f44501f9700 1 --2- 172.21.15.5:0/2435586131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f444806f130 0x7f44480ce5e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44480706a0 tx=0x7f4438009b20 comp rx=0 tx=0).stop 2024-05-08T23:21:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f54f77fe700 1 -- 172.21.15.5:0/2262639176 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500003ab0 con 0x7f551006e4f0 2024-05-08T23:21:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f54f77fe700 1 -- 172.21.15.5:0/2262639176 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500003c10 con 0x7f551006e4f0 2024-05-08T23:21:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f54f77fe700 1 -- 172.21.15.5:0/2262639176 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500013430 con 0x7f551006e4f0 2024-05-08T23:21:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f44501f9700 1 -- 172.21.15.5:0/2435586131 shutdown_connections 2024-05-08T23:21:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f44501f9700 1 --2- 172.21.15.5:0/2435586131 >> v2:172.21.15.5:3300/0 conn(0x7f44480ceb20 0x7f444815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f44501f9700 1 --2- 172.21.15.5:0/2435586131 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44480cd4e0 0x7f444806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f44501f9700 1 --2- 172.21.15.5:0/2435586131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f444806f130 0x7f44480ce5e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7f44501f9700 1 -- 172.21.15.5:0/2435586131 >> 172.21.15.5:0/2435586131 conn(0x7f44480d0320 msgr2=0x7f44480d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fde4fbde700 1 Processor -- start 2024-05-08T23:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55100af480 con 0x7f551006e4f0 2024-05-08T23:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55100afa40 con 0x7f551006e4f0 2024-05-08T23:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fa971e16700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa96c15aae0 0x7fa96c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fa971e16700 1 -- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 msgr2=0x7fa96c0986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fa971e16700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa96c0b19d0 0x7fa96c0986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fa971e16700 1 -- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa96c153f20 msgr2=0x7fa96c14e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fa971e16700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa96c153f20 0x7fa96c14e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fa971e16700 1 -- 172.21.15.5:0/2257648563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96400bc80 con 0x7fa96c15aae0 2024-05-08T23:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fa971e16700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa96c15aae0 0x7fa96c156650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa964018000 tx=0x7fa964004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fda72b12700 1 -- 172.21.15.5:0/1800309429 shutdown_connections 2024-05-08T23:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fde4fbde700 1 -- start start 2024-05-08T23:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fde4fbde700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde481465e0 0x7fde48147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fde4fbde700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde481469c0 0x7fde48160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fde4fbde700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde48160a50 0x7fde48152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fde4fbde700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48147860 con 0x7fde481465e0 2024-05-08T23:21:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fde4fbde700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48160ed0 con 0x7fde48160a50 2024-05-08T23:21:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fde4fbde700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde48166760 con 0x7fde481469c0 2024-05-08T23:21:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640040b0 con 0x7fa96c15aae0 2024-05-08T23:21:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964004230 con 0x7fa96c15aae0 2024-05-08T23:21:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640045e0 con 0x7fa96c15aae0 2024-05-08T23:21:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c201b20 con 0x7fa96c15aae0 2024-05-08T23:21:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c201180 con 0x7fa96c15aae0 2024-05-08T23:21:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d97a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde481465e0 0x7fde48147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d97a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde481465e0 0x7fde48147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60682/0 (socket says 172.21.15.5:60682) 2024-05-08T23:21:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d97a700 1 -- 172.21.15.5:0/391106030 learned_addr learned my addr 172.21.15.5:0/391106030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d179700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde481469c0 0x7fde48160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.011+0000 7f1d00ff5700 1 -- 172.21.15.5:0/912192610 shutdown_connections 2024-05-08T23:21:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d179700 1 -- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde48160a50 msgr2=0x7fde48152aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:21:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d179700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde48160a50 0x7fde48152aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d179700 1 -- 172.21.15.5:0/391106030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde481465e0 msgr2=0x7fde48147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d179700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde481465e0 0x7fde48147c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fde4d179700 1 -- 172.21.15.5:0/391106030 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48146f30 con 0x7fde481469c0 2024-05-08T23:21:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.013+0000 7fde4d179700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde481469c0 0x7fde48160510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde480b4640 tx=0x7fde4400e880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.013+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44013680 con 0x7fde481469c0 2024-05-08T23:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.013+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde44013800 con 0x7fde481469c0 2024-05-08T23:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.013+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44013bb0 con 0x7fde481469c0 2024-05-08T23:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.013+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde481481c0 con 0x7fde481469c0 2024-05-08T23:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.013+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde48147480 con 0x7fde481469c0 2024-05-08T23:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa96c06d830 con 0x7fa96c15aae0 2024-05-08T23:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.010+0000 7fda72b12700 1 -- 172.21.15.5:0/1800309429 wait complete. 2024-05-08T23:21:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7fda72b12700 1 Processor -- start 2024-05-08T23:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.012+0000 7f1d00ff5700 1 -- 172.21.15.5:0/912192610 wait complete. 2024-05-08T23:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.013+0000 7f54f77fe700 1 -- 172.21.15.5:0/2262639176 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f550001a050 con 0x7f551006e4f0 2024-05-08T23:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.013+0000 7f54f77fe700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f54f806aef0 0x7f54f806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7f54f77fe700 1 -- 172.21.15.5:0/2262639176 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f550008a660 con 0x7f551006e4f0 2024-05-08T23:21:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7fb1e5f6c700 1 Processor -- start 2024-05-08T23:21:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7f550e7fc700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f54f806aef0 0x7f54f806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7f44501f9700 1 -- 172.21.15.5:0/2435586131 shutdown_connections 2024-05-08T23:21:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7f44501f9700 1 -- 172.21.15.5:0/2435586131 wait complete. 2024-05-08T23:21:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7f55155b3700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f54f00015e0 0x7f54f0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 --> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f0006bd0 con 0x7f54f00015e0 2024-05-08T23:21:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa964004740 con 0x7fa96c15aae0 2024-05-08T23:21:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7fb1e5f6c700 1 -- start start 2024-05-08T23:21:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7f550f7fe700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f54f00015e0 0x7f54f0003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7f550e7fc700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f54f806aef0 0x7f54f806d3b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5504009ad0 tx=0x7f5504009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7fa961ffb700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa95006ad60 0x7fa95006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa96408a560 con 0x7fa96c15aae0 2024-05-08T23:21:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7fa961ffb700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7fa950072f00 0x7fa950075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 --> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9500759d0 con 0x7fa950072f00 2024-05-08T23:21:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde30000fc0 con 0x7fde481469c0 2024-05-08T23:21:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7f9603fff700 1 Processor -- start 2024-05-08T23:21:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7f9603fff700 1 -- start start 2024-05-08T23:21:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7f1d00ff5700 1 Processor -- start 2024-05-08T23:21:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7f550f7fe700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f54f00015e0 0x7f54f0003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fa970e14700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7fa950072f00 0x7fa950075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fa971615700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa95006ad60 0x7fa95006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa964055860 con 0x7fa96c15aae0 2024-05-08T23:21:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fb1e5f6c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb1e00b2be0 0x7fb1e00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fb1e5f6c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1e00b3040 0x7fb1e00b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fb1e5f6c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e00ac1c0 con 0x7fb1e015aae0 2024-05-08T23:21:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fb1e5f6c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e006cf90 con 0x7fb1e00b2be0 2024-05-08T23:21:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fb1e5f6c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e006d270 con 0x7fb1e00b3040 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.014+0000 7fda72b12700 1 -- start start 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7fda72b12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15c000 0x7fda6c149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7fda72b12700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c15c3e0 0x7fda6c13e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.015+0000 7fda72b12700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c13ccd0 0x7fda6c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fda72b12700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c13de60 con 0x7fda6c15c000 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fda72b12700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c13ec50 con 0x7fda6c15c3e0 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fda72b12700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c13d250 con 0x7fda6c13ccd0 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fda6bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c15c3e0 0x7fda6c13e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7f44501f9700 1 Processor -- start 2024-05-08T23:21:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fda710af700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c13ccd0 0x7fda6c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fda6bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c15c3e0 0x7fda6c13e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36258/0 (socket says 172.21.15.5:36258) 2024-05-08T23:21:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fda6bfff700 1 -- 172.21.15.5:0/3900305902 learned_addr learned my addr 172.21.15.5:0/3900305902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fda708ae700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15c000 0x7fda6c149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f54f77fe700 1 -- 172.21.15.5:0/2262639176 <== osd.4 v2:172.21.15.49:6816/47400 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f54f0006bd0 con 0x7f54f00015e0 2024-05-08T23:21:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fa971615700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa95006ad60 0x7fa95006d220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0aea30 tx=0x7fa95c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fb1df7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb1e015aae0 0x7fb1e0153cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60000/0 (socket says 172.21.15.5:60000) 2024-05-08T23:21:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fb1df7fe700 1 -- 172.21.15.5:0/1331864846 learned_addr learned my addr 172.21.15.5:0/1331864846 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f9603fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9604161b40 0x7f96040b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f9603fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96040b2720 0x7f96040cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f9603fff700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96040d0000 con 0x7f96040cf5a0 2024-05-08T23:21:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f9603fff700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040b2ba0 con 0x7f96040b2720 2024-05-08T23:21:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f9603fff700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96040ac1c0 con 0x7f9604161b40 2024-05-08T23:21:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f96027fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96040b2720 0x7f96040cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f96027fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96040b2720 0x7f96040cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60710/0 (socket says 172.21.15.5:60710) 2024-05-08T23:21:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f96027fc700 1 -- 172.21.15.5:0/863294281 learned_addr learned my addr 172.21.15.5:0/863294281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fa970e14700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7fa950072f00 0x7fa950075320 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7f44501f9700 1 -- start start 2024-05-08T23:21:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f44501f9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444806f130 0x7f44480c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f44501f9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480cd4e0 0x7f44480c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f44501f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44480ceb20 0x7f44481580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f44501f9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44480c5270 con 0x7f44480cd4e0 2024-05-08T23:21:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f44501f9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44480c5c40 con 0x7f44480ceb20 2024-05-08T23:21:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f44501f9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44480b4710 con 0x7f444806f130 2024-05-08T23:21:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f9602ffd700 1 --2- 172.21.15.5:0/863294281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9604161b40 0x7f96040b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fb1dffff700 1 --2- 172.21.15.5:0/1331864846 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1e00b3040 0x7fb1e00b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fda6bfff700 1 -- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c13ccd0 msgr2=0x7fda6c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fda6bfff700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c13ccd0 0x7fda6c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fda6bfff700 1 -- 172.21.15.5:0/3900305902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15c000 msgr2=0x7fda6c149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fda6bfff700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15c000 0x7fda6c149a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fda6bfff700 1 -- 172.21.15.5:0/3900305902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64009a70 con 0x7fda6c15c3e0 2024-05-08T23:21:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fda6bfff700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c15c3e0 0x7fda6c13e710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda6000c500 tx=0x7fda6000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.016+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fde44013d10 con 0x7fde481469c0 2024-05-08T23:21:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fde3e7fc700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde2c06ad60 0x7fde2c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde44088f10 con 0x7fde481469c0 2024-05-08T23:21:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fde4d97a700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde2c06ad60 0x7fde2c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fde3e7fc700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fde2c072f00 0x7fde2c075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 --> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde2c0759d0 con 0x7fde2c072f00 2024-05-08T23:21:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f96027fc700 1 -- 172.21.15.5:0/863294281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9604161b40 msgr2=0x7f96040b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f96027fc700 1 --2- 172.21.15.5:0/863294281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9604161b40 0x7f96040b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f96027fc700 1 -- 172.21.15.5:0/863294281 >> v1:172.21.15.5:6789/0 conn(0x7f96040cf5a0 legacy=0x7f96041657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.017+0000 7f96027fc700 1 -- 172.21.15.5:0/863294281 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040cfd20 con 0x7f96040b2720 2024-05-08T23:21:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444df95700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444806f130 0x7f44480c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444df95700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444806f130 0x7f44480c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:48030/0 (socket says 172.21.15.5:48030) 2024-05-08T23:21:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444df95700 1 -- 172.21.15.5:0/2058947519 learned_addr learned my addr 172.21.15.5:0/2058947519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde440540d0 con 0x7fde481469c0 2024-05-08T23:21:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f1d00ff5700 1 -- start start 2024-05-08T23:21:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444e796700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44480ceb20 0x7f44481580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fb1deffd700 1 --2- 172.21.15.5:0/1331864846 >> v2:172.21.15.5:3300/0 conn(0x7fb1e00b2be0 0x7fb1e00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fde4e17b700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fde2c072f00 0x7fde2c075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fb1de7fc700 1 -- 172.21.15.5:0/1331864846 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315449633 0 0) 0x7fb1e00ac1c0 con 0x7fb1e015aae0 2024-05-08T23:21:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fb1de7fc700 1 -- 172.21.15.5:0/1331864846 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c80035b0 con 0x7fb1e015aae0 2024-05-08T23:21:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fb1deffd700 1 -- 172.21.15.5:0/1331864846 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1e00b3040 msgr2=0x7fb1e00b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fb1deffd700 1 --2- 172.21.15.5:0/1331864846 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1e00b3040 0x7fb1e00b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fb1deffd700 1 -- 172.21.15.5:0/1331864846 >> v1:172.21.15.49:6789/0 conn(0x7fb1e015aae0 legacy=0x7fb1e0153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fb1deffd700 1 -- 172.21.15.5:0/1331864846 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e006d550 con 0x7fb1e00b2be0 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444df95700 1 -- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44480ceb20 msgr2=0x7f44481580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444df95700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44480ceb20 0x7f44481580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444df95700 1 -- 172.21.15.5:0/2058947519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480cd4e0 msgr2=0x7f44480c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444df95700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480cd4e0 0x7f44480c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7f444df95700 1 -- 172.21.15.5:0/2058947519 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380094a0 con 0x7f444806f130 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f444df95700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444806f130 0x7f44480c6950 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f444400e960 tx=0x7f444400ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400eeb0 con 0x7f444806f130 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444400c850 con 0x7f444806f130 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400cbc0 con 0x7f444806f130 2024-05-08T23:21:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fde4d97a700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde2c06ad60 0x7fde2c06d220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde38000ff0 tx=0x7fde38009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fde4e17b700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fde2c072f00 0x7fde2c075320 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1d00ff5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cfc15c000 0x7f1cfc2000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1d00ff5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cfc15c3e0 0x7f1cfc163370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1d00ff5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1cfc153ff0 0x7f1cfc1638b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1d00ff5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc200610 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1d00ff5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc164940 con 0x7f1cfc15c000 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1d00ff5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc153dd0 con 0x7f1cfc153ff0 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1cf9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cfc15c3e0 0x7f1cfc163370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1cf9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cfc15c3e0 0x7f1cfc163370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60714/0 (socket says 172.21.15.5:60714) 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7f1cf9d9b700 1 -- 172.21.15.5:0/391981034 learned_addr learned my addr 172.21.15.5:0/391981034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f1cf9d9b700 1 -- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1cfc153ff0 msgr2=0x7f1cfc1638b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f1cf9d9b700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1cfc153ff0 0x7f1cfc1638b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f1cfa59c700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cfc15c000 0x7f1cfc2000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f1cf9d9b700 1 -- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cfc15c000 msgr2=0x7f1cfc2000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f1cf9d9b700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cfc15c000 0x7f1cfc2000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f1cf9d9b700 1 -- 172.21.15.5:0/391981034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec0094a0 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 <== osd.5 v2:172.21.15.49:6800/47399 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa9500759d0 con 0x7fa950072f00 2024-05-08T23:21:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.018+0000 7fb1deffd700 1 --2- 172.21.15.5:0/1331864846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e00b2be0 0x7fb1e00b0e70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009840 tx=0x7fb1d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddc4f24de303692 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fb1de7fc700 1 -- 172.21.15.5:0/1331864846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4011830 con 0x7fb1e00b2be0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fb1de7fc700 1 -- 172.21.15.5:0/1331864846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d4011990 con 0x7fb1e00b2be0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fb1de7fc700 1 -- 172.21.15.5:0/1331864846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4011d20 con 0x7fb1e00b2be0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60005d70 con 0x7fda6c15c3e0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda600070d0 con 0x7fda6c15c3e0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60007440 con 0x7fda6c15c3e0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c13d530 con 0x7fda6c15c3e0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.019+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c13daf0 con 0x7fda6c15c3e0 2024-05-08T23:21:15.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f96027fc700 1 --2- 172.21.15.5:0/863294281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96040b2720 0x7f96040cf060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009e50 tx=0x7f95fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22886d374b535d9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 <== osd.1 v2:172.21.15.5:6814/47133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fde2c0759d0 con 0x7fde2c072f00 2024-05-08T23:21:15.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f9601ffb700 1 -- 172.21.15.5:0/863294281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00e070 con 0x7f96040b2720 2024-05-08T23:21:15.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f9601ffb700 1 -- 172.21.15.5:0/863294281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc003a20 con 0x7f96040b2720 2024-05-08T23:21:15.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f9601ffb700 1 -- 172.21.15.5:0/863294281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc003dd0 con 0x7f96040b2720 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1331864846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e00b2be0 msgr2=0x7fb1e00b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7fb1e5f6c700 1 --2- 172.21.15.5:0/1331864846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e00b2be0 0x7fb1e00b0e70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009840 tx=0x7fb1d4009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1331864846 shutdown_connections 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7fb1e5f6c700 1 --2- 172.21.15.5:0/1331864846 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1e00b3040 0x7fb1e00b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7fb1e5f6c700 1 --2- 172.21.15.5:0/1331864846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e00b2be0 0x7fb1e00b0e70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1331864846 >> 172.21.15.5:0/1331864846 conn(0x7fb1e00b4690 msgr2=0x7fb1e00b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44480c4940 con 0x7f444806f130 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.020+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44480c4f00 con 0x7f444806f130 2024-05-08T23:21:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4430000fc0 con 0x7f444806f130 2024-05-08T23:21:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1331864846 shutdown_connections 2024-05-08T23:21:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7f1cf9d9b700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cfc15c3e0 0x7f1cfc163370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000e840 tx=0x7f1cf000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda6c06e460 con 0x7fda6c15c3e0 2024-05-08T23:21:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7f9603fff700 1 -- 172.21.15.5:0/863294281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96040b2720 msgr2=0x7f96040cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.021+0000 7f9603fff700 1 --2- 172.21.15.5:0/863294281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96040b2720 0x7f96040cf060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009e50 tx=0x7f95fc00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0009590 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0013680 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0013a10 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f9603fff700 1 -- 172.21.15.5:0/863294281 shutdown_connections 2024-05-08T23:21:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f9603fff700 1 --2- 172.21.15.5:0/863294281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9604161b40 0x7f96040b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f9603fff700 1 --2- 172.21.15.5:0/863294281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96040b2720 0x7f96040cf060 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f9603fff700 1 -- 172.21.15.5:0/863294281 >> 172.21.15.5:0/863294281 conn(0x7f96040b4700 msgr2=0x7f96040b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7f9603fff700 1 -- 172.21.15.5:0/863294281 shutdown_connections 2024-05-08T23:21:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7f9603fff700 1 -- 172.21.15.5:0/863294281 wait complete. 2024-05-08T23:21:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc0cf8e0 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc0cf290 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.022+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fda600075a0 con 0x7fda6c15c3e0 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7fda697fa700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5806ad60 0x7fda5806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda60088420 con 0x7fda6c15c3e0 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7fda708ae700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5806ad60 0x7fda5806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7fda697fa700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7fda58072f00 0x7fda58075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 --> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda580759d0 con 0x7fda58072f00 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda60053690 con 0x7fda6c15c3e0 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.023+0000 7fda710af700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7fda58072f00 0x7fda58075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.024+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f444400cd20 con 0x7f444806f130 2024-05-08T23:21:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.024+0000 7fda708ae700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5806ad60 0x7fda5806d220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda64014040 tx=0x7fda64011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.024+0000 7f443e7fc700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f443406ae40 0x7f443406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.024+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4444089680 con 0x7f444806f130 2024-05-08T23:21:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.025+0000 7f443e7fc700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f4434072fe0 0x7f4434075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.025+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4434075ab0 con 0x7f4434072fe0 2024-05-08T23:21:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.025+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44440899f0 con 0x7f444806f130 2024-05-08T23:21:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.025+0000 7f9603fff700 1 Processor -- start 2024-05-08T23:21:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.026+0000 7f444e796700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f4434072fe0 0x7f4434075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.026+0000 7f9603fff700 1 -- start start 2024-05-08T23:21:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.026+0000 7fda710af700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7fda58072f00 0x7fda58075320 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.026+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1cf0013b70 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1cfc06e460 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f1ceaffd700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1cdc06ace0 0x7f1cdc06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f9603fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96040b2720 0x7f960415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f9603fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96040cf5a0 0x7f960409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f9603fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9604161b40 0x7f96041fad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f9603fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9604163a20 con 0x7f9604161b40 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f9603fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040ce610 con 0x7f96040cf5a0 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f9603fff700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96040ce8f0 con 0x7f96040b2720 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f9602ffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96040b2720 0x7f960415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f96037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9604161b40 0x7f96041fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f96037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9604161b40 0x7f96041fad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60716/0 (socket says 172.21.15.5:60716) 2024-05-08T23:21:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f96037fe700 1 -- 172.21.15.5:0/2073377222 learned_addr learned my addr 172.21.15.5:0/2073377222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cf0089200 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f1ceaffd700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f1cdc072e80 0x7f1cdc0752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 --> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cdc075950 con 0x7f1cdc072e80 2024-05-08T23:21:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7f1cfa59c700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1cdc06ace0 0x7f1cdc06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.027+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 <== osd.7 v2:172.21.15.49:6822/47408 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fda580759d0 con 0x7fda58072f00 2024-05-08T23:21:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f9602ffd700 1 -- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96040cf5a0 msgr2=0x7f960409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f9602ffd700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96040cf5a0 0x7f960409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f9602ffd700 1 -- 172.21.15.5:0/2073377222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9604161b40 msgr2=0x7f96041fad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f9602ffd700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9604161b40 0x7f96041fad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f9602ffd700 1 -- 172.21.15.5:0/2073377222 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc009b30 con 0x7f96040b2720 2024-05-08T23:21:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f1cfa59c700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1cdc06ace0 0x7f1cdc06d1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c00 tx=0x7f1cec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f9602ffd700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96040b2720 0x7f960415d6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95f800e7e0 tx=0x7f95f800eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8013610 con 0x7f96040b2720 2024-05-08T23:21:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f8013770 con 0x7f96040b2720 2024-05-08T23:21:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8013ae0 con 0x7f96040b2720 2024-05-08T23:21:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf0057ee0 con 0x7f1cfc15c3e0 2024-05-08T23:21:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.029+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9604163c90 con 0x7f96040b2720 2024-05-08T23:21:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.029+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960415dc20 con 0x7f96040b2720 2024-05-08T23:21:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.029+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f960406d630 con 0x7f96040b2720 2024-05-08T23:21:15.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.026+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1331864846 wait complete. 2024-05-08T23:21:15.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.028+0000 7fb1e5f6c700 1 Processor -- start 2024-05-08T23:21:15.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.030+0000 7f444d794700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f443406ae40 0x7f443406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.030+0000 7f444e796700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f4434072fe0 0x7f4434075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.031+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f95f8013c40 con 0x7f96040b2720 2024-05-08T23:21:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.031+0000 7f444d794700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f443406ae40 0x7f443406d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44380090b0 tx=0x7f4438014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.031+0000 7f95eb7fe700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f006ac90 0x7f95f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.031+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 <== osd.0 v2:172.21.15.5:6824/47134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4434075ab0 con 0x7f4434072fe0 2024-05-08T23:21:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.031+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95f80890c0 con 0x7f96040b2720 2024-05-08T23:21:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.032+0000 7f96027fc700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f006ac90 0x7f95f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.032+0000 7f95eb7fe700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7f95f0072e30 0x7f95f0075250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.032+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 --> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95f0075900 con 0x7f95f0072e30 2024-05-08T23:21:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.032+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95f8089430 con 0x7f96040b2720 2024-05-08T23:21:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.032+0000 7f96037fe700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7f95f0072e30 0x7f95f0075250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.032+0000 7f96027fc700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f006ac90 0x7f95f006d150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f96040ae0a0 tx=0x7f95fc010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.032+0000 7f96037fe700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7f95f0072e30 0x7f95f0075250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.033+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 <== osd.2 v2:172.21.15.5:6800/47131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f95f0075900 con 0x7f95f0072e30 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.037+0000 7fb1e5f6c700 1 -- start start 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.038+0000 7fb1e5f6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e00b2be0 0x7fb1e00ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.038+0000 7fb1e5f6c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1e00b3040 0x7fb1e0154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.038+0000 7fb1e5f6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e015aae0 0x7fb1e01505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.038+0000 7fb1e5f6c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e0153820 con 0x7fb1e015aae0 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.038+0000 7fb1e5f6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e01541c0 con 0x7fb1e00b2be0 2024-05-08T23:21:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.038+0000 7fb1e5f6c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e0153540 con 0x7fb1e00b3040 2024-05-08T23:21:15.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.039+0000 7f1cfad9d700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f1cdc072e80 0x7f1cdc0752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.040+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e015aae0 0x7fb1e01505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.040+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e015aae0 0x7fb1e01505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60724/0 (socket says 172.21.15.5:60724) 2024-05-08T23:21:15.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.040+0000 7fb1dffff700 1 -- 172.21.15.5:0/1129608804 learned_addr learned my addr 172.21.15.5:0/1129608804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.041+0000 7fb1dffff700 1 -- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1e00b3040 msgr2=0x7fb1e0154480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:21:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.041+0000 7fb1dffff700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1e00b3040 0x7fb1e0154480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.041+0000 7fb1dffff700 1 -- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e00b2be0 msgr2=0x7fb1e00ac3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:21:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.041+0000 7fb1dffff700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e00b2be0 0x7fb1e00ac3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.041+0000 7fb1dffff700 1 -- 172.21.15.5:0/1129608804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40094a0 con 0x7fb1e015aae0 2024-05-08T23:21:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.041+0000 7fb1dffff700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e015aae0 0x7fb1e01505e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8007e40 tx=0x7fb1d800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.041+0000 7f1cfad9d700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f1cdc072e80 0x7f1cdc0752a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.042+0000 7fb1c7fff700 1 -- 172.21.15.5:0/1129608804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800eea0 con 0x7fb1e015aae0 2024-05-08T23:21:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.042+0000 7fb1c7fff700 1 -- 172.21.15.5:0/1129608804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8014420 con 0x7fb1e015aae0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.042+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e0168e70 con 0x7fb1e015aae0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.042+0000 7fb1c7fff700 1 -- 172.21.15.5:0/1129608804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80147d0 con 0x7fb1e015aae0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.042+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e0169430 con 0x7fb1e015aae0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.045+0000 7fb1c7fff700 1 -- 172.21.15.5:0/1129608804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb1d8014930 con 0x7fb1e015aae0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.045+0000 7fb1c7fff700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1cc06ac90 0x7fb1cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.045+0000 7fb1c7fff700 1 -- 172.21.15.5:0/1129608804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1d8089bd0 con 0x7fb1e015aae0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.046+0000 7fb1e5f6c700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fb1e00ad4f0 0x7fb1e0169950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.046+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1e00b18f0 con 0x7fb1e00ad4f0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.048+0000 7fb1df7fe700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1cc06ac90 0x7fb1cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.048+0000 7fb1deffd700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fb1e00ad4f0 0x7fb1e0169950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.048+0000 7fb1df7fe700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1cc06ac90 0x7fb1cc06d150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0007fd0 tx=0x7fb1d0007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.049+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 <== osd.3 v2:172.21.15.5:6808/47142 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1cdc075950 con 0x7f1cdc072e80 2024-05-08T23:21:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.052+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 --> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa96c06d550 con 0x7fa950072f00 2024-05-08T23:21:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.052+0000 7fa961ffb700 1 -- 172.21.15.5:0/2257648563 <== osd.5 v2:172.21.15.49:6800/47399 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa968004e30 con 0x7fa950072f00 2024-05-08T23:21:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.053+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 --> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda6c0b48b0 con 0x7fda58072f00 2024-05-08T23:21:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.052+0000 7fb1deffd700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fb1e00ad4f0 0x7fb1e0169950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.053+0000 7fb1c7fff700 1 -- 172.21.15.5:0/1129608804 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb1d4011630 con 0x7fb1e00ad4f0 2024-05-08T23:21:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.054+0000 7fda697fa700 1 -- 172.21.15.5:0/3900305902 <== osd.7 v2:172.21.15.49:6822/47408 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda6c0b48b0 con 0x7fda58072f00 2024-05-08T23:21:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.054+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 --> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54f0005cb0 con 0x7f54f00015e0 2024-05-08T23:21:15.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.054+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 --> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f960406d350 con 0x7f95f0072e30 2024-05-08T23:21:15.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.055+0000 7f54f77fe700 1 -- 172.21.15.5:0/2262639176 <== osd.4 v2:172.21.15.49:6816/47400 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f54f0005cb0 con 0x7f54f00015e0 2024-05-08T23:21:15.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.055+0000 7f95eb7fe700 1 -- 172.21.15.5:0/2073377222 <== osd.2 v2:172.21.15.5:6800/47131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f95f4009600 con 0x7f95f0072e30 2024-05-08T23:21:15.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.055+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f54f00015e0 msgr2=0x7f54f0003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.055+0000 7f55155b3700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f54f00015e0 0x7f54f0003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.055+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f54f806aef0 msgr2=0x7f54f806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.055+0000 7f55155b3700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f54f806aef0 0x7f54f806d3b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5504009ad0 tx=0x7f5504009210 comp rx=0 tx=0).stop 2024-05-08T23:21:15.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.055+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f551006e4f0 msgr2=0x7f551008df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.055+0000 7f55155b3700 1 --2- 172.21.15.5:0/2262639176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f551006e4f0 0x7f551008df20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5500015000 tx=0x7f5500004750 comp rx=0 tx=0).stop 2024-05-08T23:21:15.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.056+0000 7f550f7fe700 1 -- 172.21.15.5:0/2262639176 reap_dead start 2024-05-08T23:21:15.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.056+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 shutdown_connections 2024-05-08T23:21:15.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.056+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 >> 172.21.15.5:0/2262639176 conn(0x7f55100b4690 msgr2=0x7f55100b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.056+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 shutdown_connections 2024-05-08T23:21:15.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.056+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 --> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fde30002d40 con 0x7fde2c072f00 2024-05-08T23:21:15.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.056+0000 7f55155b3700 1 -- 172.21.15.5:0/2262639176 wait complete. 2024-05-08T23:21:15.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.056+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7fa950072f00 msgr2=0x7fa950075320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.056+0000 7fa973879700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7fa950072f00 0x7fa950075320 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.057+0000 7fde3e7fc700 1 -- 172.21.15.5:0/391106030 <== osd.1 v2:172.21.15.5:6814/47133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fde40009e90 con 0x7fde2c072f00 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.057+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fde2c072f00 msgr2=0x7fde2c075320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.057+0000 7fde4fbde700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fde2c072f00 0x7fde2c075320 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.057+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde2c06ad60 msgr2=0x7fde2c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.057+0000 7fde4fbde700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde2c06ad60 0x7fde2c06d220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde38000ff0 tx=0x7fde38009030 comp rx=0 tx=0).stop 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.057+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde481469c0 msgr2=0x7fde48160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.057+0000 7fde4fbde700 1 --2- 172.21.15.5:0/391106030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde481469c0 0x7fde48160510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde480b4640 tx=0x7fde4400e880 comp rx=0 tx=0).stop 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.057+0000 7fde4e17b700 1 -- 172.21.15.5:0/391106030 reap_dead start 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.058+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 shutdown_connections 2024-05-08T23:21:15.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.058+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 >> 172.21.15.5:0/391106030 conn(0x7fde480b4690 msgr2=0x7fde48163c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.058+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7f95f0072e30 msgr2=0x7f95f0075250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.058+0000 7f9603fff700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7f95f0072e30 0x7f95f0075250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.058+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 shutdown_connections 2024-05-08T23:21:15.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.058+0000 7fde4fbde700 1 -- 172.21.15.5:0/391106030 wait complete. 2024-05-08T23:21:15.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.060+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa95006ad60 msgr2=0x7fa95006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.060+0000 7fa973879700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa95006ad60 0x7fa95006d220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0aea30 tx=0x7fa95c008040 comp rx=0 tx=0).stop 2024-05-08T23:21:15.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.060+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa96c15aae0 msgr2=0x7fa96c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.060+0000 7fa973879700 1 --2- 172.21.15.5:0/2257648563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa96c15aae0 0x7fa96c156650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa964018000 tx=0x7fa964004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.060+0000 7fa971e16700 1 -- 172.21.15.5:0/2257648563 reap_dead start 2024-05-08T23:21:15.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.061+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 shutdown_connections 2024-05-08T23:21:15.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.061+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 >> 172.21.15.5:0/2257648563 conn(0x7fa96c0b4690 msgr2=0x7fa96c158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.063+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4430002d40 con 0x7f4434072fe0 2024-05-08T23:21:15.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.063+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f006ac90 msgr2=0x7f95f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.063+0000 7f9603fff700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f006ac90 0x7f95f006d150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f96040ae0a0 tx=0x7f95fc010000 comp rx=0 tx=0).stop 2024-05-08T23:21:15.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.063+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96040b2720 msgr2=0x7f960415d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.063+0000 7f9603fff700 1 --2- 172.21.15.5:0/2073377222 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96040b2720 0x7f960415d6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95f800e7e0 tx=0x7f95f800eba0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.063+0000 7f96037fe700 1 -- 172.21.15.5:0/2073377222 reap_dead start 2024-05-08T23:21:15.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.063+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 shutdown_connections 2024-05-08T23:21:15.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.063+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 >> 172.21.15.5:0/2073377222 conn(0x7f96040b4700 msgr2=0x7f96040af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.064+0000 7f443e7fc700 1 -- 172.21.15.5:0/2058947519 <== osd.0 v2:172.21.15.5:6824/47134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4430002d40 con 0x7f4434072fe0 2024-05-08T23:21:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.064+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 shutdown_connections 2024-05-08T23:21:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.064+0000 7fa973879700 1 -- 172.21.15.5:0/2257648563 wait complete. 2024-05-08T23:21:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.064+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f4434072fe0 msgr2=0x7f4434075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.064+0000 7f44501f9700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f4434072fe0 0x7f4434075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.065+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f443406ae40 msgr2=0x7f443406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.065+0000 7f44501f9700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f443406ae40 0x7f443406d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44380090b0 tx=0x7f4438014000 comp rx=0 tx=0).stop 2024-05-08T23:21:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.065+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444806f130 msgr2=0x7f44480c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.065+0000 7f44501f9700 1 --2- 172.21.15.5:0/2058947519 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444806f130 0x7f44480c6950 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f444400e960 tx=0x7f444400ec70 comp rx=0 tx=0).stop 2024-05-08T23:21:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.065+0000 7f444e796700 1 -- 172.21.15.5:0/2058947519 reap_dead start 2024-05-08T23:21:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.065+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 shutdown_connections 2024-05-08T23:21:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.065+0000 7f9603fff700 1 -- 172.21.15.5:0/2073377222 wait complete. 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.067+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7fda58072f00 msgr2=0x7fda58075320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.067+0000 7fda72b12700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7fda58072f00 0x7fda58075320 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.075+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5806ad60 msgr2=0x7fda5806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.075+0000 7fda72b12700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5806ad60 0x7fda5806d220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda64014040 tx=0x7fda64011000 comp rx=0 tx=0).stop 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.075+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c15c3e0 msgr2=0x7fda6c13e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.075+0000 7fda72b12700 1 --2- 172.21.15.5:0/3900305902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c15c3e0 0x7fda6c13e710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda6000c500 tx=0x7fda6000c810 comp rx=0 tx=0).stop 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.075+0000 7fda710af700 1 -- 172.21.15.5:0/3900305902 reap_dead start 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.075+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 shutdown_connections 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.075+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 >> 172.21.15.5:0/3900305902 conn(0x7fda6c0b3cf0 msgr2=0x7fda6c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.076+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 shutdown_connections 2024-05-08T23:21:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.078+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 shutdown_connections 2024-05-08T23:21:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.078+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 >> 172.21.15.5:0/2058947519 conn(0x7f44480d0320 msgr2=0x7f44480cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.078+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 shutdown_connections 2024-05-08T23:21:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.078+0000 7f44501f9700 1 -- 172.21.15.5:0/2058947519 wait complete. 2024-05-08T23:21:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.082+0000 7fda72b12700 1 -- 172.21.15.5:0/3900305902 wait complete. 2024-05-08T23:21:15.109 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-08T23:21:15.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T23:21:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.086+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 --> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1cfc159920 con 0x7f1cdc072e80 2024-05-08T23:21:15.110 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:15.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T23:21:15.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.087+0000 7f1ceaffd700 1 -- 172.21.15.5:0/391981034 <== osd.3 v2:172.21.15.5:6808/47142 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1cfc159920 con 0x7f1cdc072e80 2024-05-08T23:21:15.112 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:15.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T23:21:15.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.088+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f1cdc072e80 msgr2=0x7f1cdc0752a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.088+0000 7f1d00ff5700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f1cdc072e80 0x7f1cdc0752a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.088+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb1e0154a20 con 0x7fb1e00ad4f0 2024-05-08T23:21:15.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.088+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1cdc06ace0 msgr2=0x7f1cdc06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.088+0000 7f1d00ff5700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1cdc06ace0 0x7f1cdc06d1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c00 tx=0x7f1cec008030 comp rx=0 tx=0).stop 2024-05-08T23:21:15.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.088+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cfc15c3e0 msgr2=0x7f1cfc163370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.088+0000 7f1d00ff5700 1 --2- 172.21.15.5:0/391981034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cfc15c3e0 0x7f1cfc163370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000e840 tx=0x7f1cf000ec00 comp rx=0 tx=0).stop 2024-05-08T23:21:15.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.089+0000 7f1cfad9d700 1 -- 172.21.15.5:0/391981034 reap_dead start 2024-05-08T23:21:15.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.089+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 shutdown_connections 2024-05-08T23:21:15.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.089+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 >> 172.21.15.5:0/391981034 conn(0x7f1cfc0b35e0 msgr2=0x7f1cfc06b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.089+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 shutdown_connections 2024-05-08T23:21:15.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.089+0000 7f1d00ff5700 1 -- 172.21.15.5:0/391981034 wait complete. 2024-05-08T23:21:15.115 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:15.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.089+0000 7fb1c7fff700 1 -- 172.21.15.5:0/1129608804 <== osd.6 v2:172.21.15.49:6808/47398 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb1e0154a20 con 0x7fb1e00ad4f0 2024-05-08T23:21:15.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T23:21:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.089+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fb1e00ad4f0 msgr2=0x7fb1e0169950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.089+0000 7fb1e5f6c700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fb1e00ad4f0 0x7fb1e0169950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1cc06ac90 msgr2=0x7fb1cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1e5f6c700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1cc06ac90 0x7fb1cc06d150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0007fd0 tx=0x7fb1d0007980 comp rx=0 tx=0).stop 2024-05-08T23:21:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e015aae0 msgr2=0x7fb1e01505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1e5f6c700 1 --2- 172.21.15.5:0/1129608804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1e015aae0 0x7fb1e01505e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8007e40 tx=0x7fb1d800e3c0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1dffff700 1 -- 172.21.15.5:0/1129608804 reap_dead start 2024-05-08T23:21:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 shutdown_connections 2024-05-08T23:21:15.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 >> 172.21.15.5:0/1129608804 conn(0x7fb1e00b4690 msgr2=0x7fb1e006dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 shutdown_connections 2024-05-08T23:21:15.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.090+0000 7fb1e5f6c700 1 -- 172.21.15.5:0/1129608804 wait complete. 2024-05-08T23:21:15.118 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:15.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T23:21:15.119 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:15.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T23:21:15.120 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-08T23:21:15.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T23:21:15.121 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-08T23:21:15.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T23:21:15.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.389+0000 7fbb2014e700 1 Processor -- start 2024-05-08T23:21:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.389+0000 7fbb2014e700 1 -- start start 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.391+0000 7fbb2014e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb18153e90 0x7fbb180b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.391+0000 7fbb2014e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb180b13b0 0x7fbb180b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.391+0000 7fbb2014e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb180ac1c0 con 0x7fbb1815aae0 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.391+0000 7fbb2014e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1806ccb0 con 0x7fbb18153e90 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.391+0000 7fbb2014e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1806cf90 con 0x7fbb180b13b0 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.391+0000 7fbb1deea700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbb1815aae0 0x7fbb180b2be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60014/0 (socket says 172.21.15.5:60014) 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.391+0000 7fbb1deea700 1 -- 172.21.15.5:0/866620694 learned_addr learned my addr 172.21.15.5:0/866620694 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.391+0000 7fbb1e6eb700 1 --2- 172.21.15.5:0/866620694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb180b13b0 0x7fbb180b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1e6eb700 1 -- 172.21.15.5:0/866620694 >> v2:172.21.15.5:3300/0 conn(0x7fbb18153e90 msgr2=0x7fbb180b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1e6eb700 1 --2- 172.21.15.5:0/866620694 >> v2:172.21.15.5:3300/0 conn(0x7fbb18153e90 0x7fbb180b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1cee8700 1 -- 172.21.15.5:0/866620694 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681456212 0 0) 0x7fbb180ac1c0 con 0x7fbb1815aae0 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1e6eb700 1 -- 172.21.15.5:0/866620694 >> v1:172.21.15.49:6789/0 conn(0x7fbb1815aae0 legacy=0x7fbb180b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1e6eb700 1 -- 172.21.15.5:0/866620694 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb181542f0 con 0x7fbb180b13b0 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1e6eb700 1 --2- 172.21.15.5:0/866620694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb180b13b0 0x7fbb180b1820 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009840 tx=0x7fbb14009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6eb2717b517444 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1cee8700 1 -- 172.21.15.5:0/866620694 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14011830 con 0x7fbb180b13b0 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1cee8700 1 -- 172.21.15.5:0/866620694 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb14011990 con 0x7fbb180b13b0 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.392+0000 7fbb1cee8700 1 -- 172.21.15.5:0/866620694 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14011d20 con 0x7fbb180b13b0 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.393+0000 7fbb2014e700 1 -- 172.21.15.5:0/866620694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb180b13b0 msgr2=0x7fbb180b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.393+0000 7fbb2014e700 1 --2- 172.21.15.5:0/866620694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb180b13b0 0x7fbb180b1820 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009840 tx=0x7fbb14009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.393+0000 7fc3c8189700 1 Processor -- start 2024-05-08T23:21:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.394+0000 7fc3c8189700 1 -- start start 2024-05-08T23:21:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fb13bced700 1 Processor -- start 2024-05-08T23:21:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fc3c8189700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3c0161a80 0x7fc3c00b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fc3c8189700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc3c00ce970 0x7fc3c0070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fc3c8189700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c0071910 con 0x7fc3c00714c0 2024-05-08T23:21:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fc3c8189700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c00b3cc0 con 0x7fc3c00ce970 2024-05-08T23:21:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fc3c8189700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c00b3fa0 con 0x7fc3c0161a80 2024-05-08T23:21:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fbb2014e700 1 -- 172.21.15.5:0/866620694 shutdown_connections 2024-05-08T23:21:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fbb2014e700 1 --2- 172.21.15.5:0/866620694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb180b13b0 0x7fbb180b1820 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fbb2014e700 1 --2- 172.21.15.5:0/866620694 >> v2:172.21.15.5:3300/0 conn(0x7fbb18153e90 0x7fbb180b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fbb2014e700 1 -- 172.21.15.5:0/866620694 >> 172.21.15.5:0/866620694 conn(0x7fbb180b4690 msgr2=0x7fbb180b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7f51077b7700 1 Processor -- start 2024-05-08T23:21:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fc3c6726700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc3c00714c0 0x7fc3c01656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60026/0 (socket says 172.21.15.5:60026) 2024-05-08T23:21:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fc3c6726700 1 -- 172.21.15.5:0/3438347883 learned_addr learned my addr 172.21.15.5:0/3438347883 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.395+0000 7fb13bced700 1 -- start start 2024-05-08T23:21:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7f51077b7700 1 -- start start 2024-05-08T23:21:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938548182 0 0) 0x7fc3b0001110 con 0x7fc3c00714c0 2024-05-08T23:21:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3ac0035b0 con 0x7fc3c00714c0 2024-05-08T23:21:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 <== mon.1 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3571470419 0 0) 0x7fc3ac0035b0 con 0x7fc3c00714c0 2024-05-08T23:21:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3c0161a80 msgr2=0x7fc3c00b30e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3c0161a80 0x7fc3c00b30e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 >> v2:172.21.15.49:3300/0 conn(0x7fc3c00ce970 msgr2=0x7fc3c0070f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 --2- 172.21.15.5:0/3438347883 >> v2:172.21.15.49:3300/0 conn(0x7fc3c00ce970 0x7fc3c0070f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c00b4280 con 0x7fc3c00714c0 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 <== mon.1 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fc3b0001800 con 0x7fc3c00714c0 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 >> v1:172.21.15.49:6789/0 conn(0x7fc3c00714c0 legacy=0x7fc3c01656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac003a00 0x7fc3ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3ac004850 0x7fc3ac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3ac005260 0x7fc3ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c00b3cc0 con 0x7fc3ac003a00 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c00b3fa0 con 0x7fc3ac005260 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac00c1f0 con 0x7fc3ac004850 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7ffa65d2e700 1 Processor -- start 2024-05-08T23:21:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.396+0000 7ffa65d2e700 1 -- start start 2024-05-08T23:21:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7f51077b7700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f51000c3980 0x7f51000ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7f51077b7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51000b2730 0x7f51000cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7f51077b7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51000ce980 con 0x7f51000cc460 2024-05-08T23:21:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7f51077b7700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51000cdc80 con 0x7f51000b2730 2024-05-08T23:21:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7f51077b7700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51000cd100 con 0x7f51000c3980 2024-05-08T23:21:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fc3c6726700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3ac005260 0x7fc3ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7f5105553700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f51000cc460 0x7f51000c3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60038/0 (socket says 172.21.15.5:60038) 2024-05-08T23:21:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7f5105553700 1 -- 172.21.15.5:0/1320583637 learned_addr learned my addr 172.21.15.5:0/1320583637 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fc3c6726700 1 -- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3ac004850 msgr2=0x7fc3ac004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fc3c6726700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3ac004850 0x7fc3ac004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fc3c6726700 1 -- 172.21.15.5:0/3438347883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac003a00 msgr2=0x7fc3ac003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fc3c6726700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac003a00 0x7fc3ac003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fc3c6726700 1 -- 172.21.15.5:0/3438347883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c00b4280 con 0x7fc3ac005260 2024-05-08T23:21:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fb13bced700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb13415f900 0x7fb13406ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fb13bced700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb13406b2c0 0x7fb1341621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fb13bced700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13406b720 con 0x7fb134162710 2024-05-08T23:21:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fb13bced700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb134070f30 con 0x7fb13406b2c0 2024-05-08T23:21:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fb13bced700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb134162ba0 con 0x7fb13415f900 2024-05-08T23:21:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fb13a28a700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb134162710 0x7fb1340b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44626/0 (socket says 172.21.15.5:44626) 2024-05-08T23:21:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.397+0000 7fb13a28a700 1 -- 172.21.15.5:0/2727737079 learned_addr learned my addr 172.21.15.5:0/2727737079 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7ffa65d2e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa60160e40 0x7ffa600d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7ffa65d2e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600cf470 con 0x7ffa600cfff0 2024-05-08T23:21:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7ffa65d2e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600d13b0 con 0x7ffa600d3fe0 2024-05-08T23:21:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7ffa65d2e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa600cf6b0 con 0x7ffa60160e40 2024-05-08T23:21:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7f50f7fff700 1 -- 172.21.15.5:0/1320583637 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84162470 0 0) 0x7f51000ce980 con 0x7f51000cc460 2024-05-08T23:21:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7f50f7fff700 1 -- 172.21.15.5:0/1320583637 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50ec0035b0 con 0x7f51000cc460 2024-05-08T23:21:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7fc3c6726700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3ac005260 0x7fc3ac00ba10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000c2f0 tx=0x7fc3ac005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381c2d43234f5cde server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0007e20 con 0x7fc3ac005260 2024-05-08T23:21:15.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b000ce10 con 0x7fc3ac005260 2024-05-08T23:21:15.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7fc3c4f23700 1 -- 172.21.15.5:0/3438347883 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0005b60 con 0x7fc3ac005260 2024-05-08T23:21:15.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7f5104d52700 1 --2- 172.21.15.5:0/1320583637 >> v2:172.21.15.49:3300/0 conn(0x7f51000c3980 0x7f51000ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7f5104d52700 1 -- 172.21.15.5:0/1320583637 >> v2:172.21.15.5:3300/0 conn(0x7f51000b2730 msgr2=0x7f51000cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7f5104d52700 1 --2- 172.21.15.5:0/1320583637 >> v2:172.21.15.5:3300/0 conn(0x7f51000b2730 0x7f51000cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7f5104d52700 1 -- 172.21.15.5:0/1320583637 >> v1:172.21.15.49:6789/0 conn(0x7f51000cc460 legacy=0x7f51000c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7fb139288700 1 --2- 172.21.15.5:0/2727737079 >> v2:172.21.15.5:3300/0 conn(0x7fb13406b2c0 0x7fb1341621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195615147 0 0) 0x7fb13406b720 con 0x7fb134162710 2024-05-08T23:21:15.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1200035b0 con 0x7fb134162710 2024-05-08T23:21:15.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7f5104d52700 1 -- 172.21.15.5:0/1320583637 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51000cd3e0 con 0x7f51000c3980 2024-05-08T23:21:15.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7ffa5ffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ffa600cfff0 0x7ffa600b6490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44642/0 (socket says 172.21.15.5:44642) 2024-05-08T23:21:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.398+0000 7ffa5ffff700 1 -- 172.21.15.5:0/3323785437 learned_addr learned my addr 172.21.15.5:0/3323785437 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777209483 0 0) 0x7ffa58001490 con 0x7ffa600cfff0 2024-05-08T23:21:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa480035b0 con 0x7ffa600cfff0 2024-05-08T23:21:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7ffa5f7fe700 1 --2- 172.21.15.5:0/3323785437 >> v2:172.21.15.5:3300/0 conn(0x7ffa60160e40 0x7ffa600d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7f5104d52700 1 --2- 172.21.15.5:0/1320583637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 0x7f51000ccb20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f51000d12f0 tx=0x7f50fc009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=600a9a9b302fac90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7f50f7fff700 1 -- 172.21.15.5:0/1320583637 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc004230 con 0x7f51000c3980 2024-05-08T23:21:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7f50f7fff700 1 -- 172.21.15.5:0/1320583637 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc004390 con 0x7f51000c3980 2024-05-08T23:21:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7f50f7fff700 1 -- 172.21.15.5:0/1320583637 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0036a0 con 0x7f51000c3980 2024-05-08T23:21:15.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fc3c8189700 1 -- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3ac005260 msgr2=0x7fc3ac00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fc3c8189700 1 --2- 172.21.15.5:0/3438347883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3ac005260 0x7fc3ac00ba10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000c2f0 tx=0x7fc3ac005880 comp rx=0 tx=0).stop 2024-05-08T23:21:15.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fc3c6726700 1 -- 172.21.15.5:0/3438347883 reap_dead start 2024-05-08T23:21:15.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1511651901 0 0) 0x7fb1200035b0 con 0x7fb134162710 2024-05-08T23:21:15.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb13415f900 msgr2=0x7fb13406ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb13415f900 0x7fb13406ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 >> v2:172.21.15.5:3300/0 conn(0x7fb13406b2c0 msgr2=0x7fb1341621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 --2- 172.21.15.5:0/2727737079 >> v2:172.21.15.5:3300/0 conn(0x7fb13406b2c0 0x7fb1341621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340711d0 con 0x7fb134162710 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fb130001670 con 0x7fb134162710 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb139288700 1 --2- 172.21.15.5:0/2727737079 >> v2:172.21.15.5:3300/0 conn(0x7fb13406b2c0 0x7fb1341621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 >> v1:172.21.15.5:6789/0 conn(0x7fb134162710 legacy=0x7fb1340b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb120003a00 0x7fb120003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb120004850 0x7fb120004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb12000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134070f30 con 0x7fb120005260 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134162ba0 con 0x7fb120004850 2024-05-08T23:21:15.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12000c210 con 0x7fb120003a00 2024-05-08T23:21:15.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fc3c8189700 1 -- 172.21.15.5:0/3438347883 shutdown_connections 2024-05-08T23:21:15.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fc3c8189700 1 -- 172.21.15.5:0/3438347883 >> 172.21.15.5:0/3438347883 conn(0x7fc3c006af90 msgr2=0x7fc3c00b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.399+0000 7fb139288700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb120004850 0x7fb120004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fc3c8189700 1 -- 172.21.15.5:0/3438347883 shutdown_connections 2024-05-08T23:21:15.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fb13a28a700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb12000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fb139a89700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb120003a00 0x7fb120003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349457431 0 0) 0x7ffa600d13b0 con 0x7ffa600d3fe0 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58001490 con 0x7ffa600d3fe0 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 406005190 0 0) 0x7ffa480035b0 con 0x7ffa600cfff0 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 >> v2:172.21.15.5:3300/0 conn(0x7ffa60160e40 msgr2=0x7ffa600d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 --2- 172.21.15.5:0/3323785437 >> v2:172.21.15.5:3300/0 conn(0x7ffa60160e40 0x7ffa600d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 >> v1:172.21.15.49:6789/0 conn(0x7ffa600d3fe0 legacy=0x7ffa600ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600d1690 con 0x7ffa600cfff0 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7ffa58001b00 con 0x7ffa600cfff0 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 >> v1:172.21.15.5:6789/0 conn(0x7ffa600cfff0 legacy=0x7ffa600b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa48003b90 0x7ffa48003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48004850 0x7ffa48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa48005210 0x7ffa48014220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600cf6b0 con 0x7ffa48005210 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa48014870 con 0x7ffa48003b90 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa48014a30 con 0x7ffa48004850 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7ffa5ffff700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa48005210 0x7ffa48014220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7ffa5f7fe700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa48003b90 0x7ffa48003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7ffa5ffff700 1 -- 172.21.15.5:0/3323785437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48004850 msgr2=0x7ffa48004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7ffa5ffff700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48004850 0x7ffa48004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7ffa5ffff700 1 -- 172.21.15.5:0/3323785437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa48003b90 msgr2=0x7ffa48003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7ffa5ffff700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa48003b90 0x7ffa48003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7ffa5ffff700 1 -- 172.21.15.5:0/3323785437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480035b0 con 0x7ffa48005210 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7ffa5ffff700 1 -- 172.21.15.5:0/3323785437 reap_dead start 2024-05-08T23:21:15.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7ffa5ffff700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa48005210 0x7ffa48014220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800a680 tx=0x7ffa48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8791f899e26b011f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580072d0 con 0x7ffa48005210 2024-05-08T23:21:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58007470 con 0x7ffa48005210 2024-05-08T23:21:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7ffa5e7fc700 1 -- 172.21.15.5:0/3323785437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58007820 con 0x7ffa48005210 2024-05-08T23:21:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fbb2014e700 1 -- 172.21.15.5:0/866620694 shutdown_connections 2024-05-08T23:21:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fbb2014e700 1 -- 172.21.15.5:0/866620694 wait complete. 2024-05-08T23:21:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7fbb2014e700 1 Processor -- start 2024-05-08T23:21:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7f51077b7700 1 -- 172.21.15.5:0/1320583637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 msgr2=0x7f51000ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7f51077b7700 1 --2- 172.21.15.5:0/1320583637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 0x7f51000ccb20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f51000d12f0 tx=0x7f50fc009b20 comp rx=0 tx=0).stop 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7f51077b7700 1 -- 172.21.15.5:0/1320583637 shutdown_connections 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7f51077b7700 1 --2- 172.21.15.5:0/1320583637 >> v2:172.21.15.5:3300/0 conn(0x7f51000b2730 0x7f51000cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7f51077b7700 1 --2- 172.21.15.5:0/1320583637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 0x7f51000ccb20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7f51077b7700 1 -- 172.21.15.5:0/1320583637 >> 172.21.15.5:0/1320583637 conn(0x7f5100099a50 msgr2=0x7f51000b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7f51077b7700 1 -- 172.21.15.5:0/1320583637 shutdown_connections 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fc3c8189700 1 -- 172.21.15.5:0/3438347883 wait complete. 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7fc3c8189700 1 Processor -- start 2024-05-08T23:21:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.401+0000 7fc3c8189700 1 -- start start 2024-05-08T23:21:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fb13a28a700 1 -- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb120003a00 msgr2=0x7fb120003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fb13a28a700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb120003a00 0x7fb120003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fb13a28a700 1 -- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb120004850 msgr2=0x7fb120004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fb13a28a700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb120004850 0x7fb120004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7fb13a28a700 1 -- 172.21.15.5:0/2727737079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340711d0 con 0x7fb120005260 2024-05-08T23:21:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7fb13a28a700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb12000ba10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb130001eb0 tx=0x7fb120005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68ecc1837b9fef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.400+0000 7f51077b7700 1 -- 172.21.15.5:0/1320583637 wait complete. 2024-05-08T23:21:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7f51077b7700 1 Processor -- start 2024-05-08T23:21:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7ffa65d2e700 1 -- 172.21.15.5:0/3323785437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa48005210 msgr2=0x7ffa48014220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7ffa65d2e700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa48005210 0x7ffa48014220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800a680 tx=0x7ffa48005880 comp rx=0 tx=0).stop 2024-05-08T23:21:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7ffa65d2e700 1 -- 172.21.15.5:0/3323785437 shutdown_connections 2024-05-08T23:21:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7ffa65d2e700 1 --2- 172.21.15.5:0/3323785437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa48005210 0x7ffa48014220 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7ffa65d2e700 1 -- 172.21.15.5:0/3323785437 >> 172.21.15.5:0/3323785437 conn(0x7ffa6008afe0 msgr2=0x7ffa600d3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7fbb2014e700 1 -- start start 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fbb2014e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb180b13b0 0x7fbb18158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fbb2014e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb18153e90 0x7fbb18156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fbb2014e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb1815aae0 0x7fbb1816a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fbb2014e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18150e20 con 0x7fbb180b13b0 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb2014e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb181508f0 con 0x7fbb18153e90 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb2014e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1816a830 con 0x7fbb1815aae0 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1d6e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb18153e90 0x7fbb18156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1d6e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb18153e90 0x7fbb18156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36364/0 (socket says 172.21.15.5:36364) 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7f51077b7700 1 -- start start 2024-05-08T23:21:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f51077b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51000b2730 0x7f51001ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f51077b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 0x7f51000c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f51077b7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51000cc460 0x7f51000b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f51077b7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51000c4e20 con 0x7f51000b2730 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f51077b7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5100200370 con 0x7f51000c3980 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f51077b7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5100200650 con 0x7f51000cc460 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c8189700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac005260 0x7fc3c0163d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c8189700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3c0160d10 0x7fc3c01488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c8189700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c0148de0 0x7fc3c014c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c8189700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0164900 con 0x7fc3ac005260 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c8189700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0164340 con 0x7fc3c0148de0 2024-05-08T23:21:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c8189700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0164620 con 0x7fc3c0160d10 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c5f25700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac005260 0x7fc3c0163d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c5f25700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac005260 0x7fc3c0163d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60804/0 (socket says 172.21.15.5:60804) 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fc3c5f25700 1 -- 172.21.15.5:0/3476229661 learned_addr learned my addr 172.21.15.5:0/3476229661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3c6726700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c0148de0 0x7fc3c014c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3c5f25700 1 -- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3c0160d10 msgr2=0x7fc3c01488a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3c5f25700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3c0160d10 0x7fc3c01488a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3c5f25700 1 -- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c0148de0 msgr2=0x7fc3c014c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3c5f25700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c0148de0 0x7fc3c014c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3c5f25700 1 -- 172.21.15.5:0/3476229661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c0162180 con 0x7fc3ac005260 2024-05-08T23:21:15.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13000ce10 con 0x7fb120005260 2024-05-08T23:21:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb130005960 con 0x7fb120005260 2024-05-08T23:21:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.402+0000 7fb138a87700 1 -- 172.21.15.5:0/2727737079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130005cf0 con 0x7fb120005260 2024-05-08T23:21:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fb13bced700 1 -- 172.21.15.5:0/2727737079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 msgr2=0x7fb12000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fb13bced700 1 --2- 172.21.15.5:0/2727737079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb12000ba10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb130001eb0 tx=0x7fb120005880 comp rx=0 tx=0).stop 2024-05-08T23:21:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.403+0000 7fb13a28a700 1 -- 172.21.15.5:0/2727737079 reap_dead start 2024-05-08T23:21:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1d6e9700 1 -- 172.21.15.5:0/332215898 learned_addr learned my addr 172.21.15.5:0/332215898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1e6eb700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb1815aae0 0x7fbb1816a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1deea700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb180b13b0 0x7fbb18158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f5105d54700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51000cc460 0x7f51000b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f5105d54700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51000cc460 0x7f51000b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:48110/0 (socket says 172.21.15.5:48110) 2024-05-08T23:21:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f5105d54700 1 -- 172.21.15.5:0/2849354838 learned_addr learned my addr 172.21.15.5:0/2849354838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f5104d52700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 0x7f51000c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3c5f25700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac005260 0x7fc3c0163d60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800b760 tx=0x7fc3b800bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1d6e9700 1 -- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb1815aae0 msgr2=0x7fbb1816a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1d6e9700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb1815aae0 0x7fbb1816a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1d6e9700 1 -- 172.21.15.5:0/332215898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb180b13b0 msgr2=0x7fbb18158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1d6e9700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb180b13b0 0x7fbb18158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1d6e9700 1 -- 172.21.15.5:0/332215898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb140094a0 con 0x7fbb18153e90 2024-05-08T23:21:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3b67fc700 1 -- 172.21.15.5:0/3476229661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8010860 con 0x7fc3ac005260 2024-05-08T23:21:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3b67fc700 1 -- 172.21.15.5:0/3476229661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b8010a00 con 0x7fc3ac005260 2024-05-08T23:21:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fc3b67fc700 1 -- 172.21.15.5:0/3476229661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8010db0 con 0x7fc3ac005260 2024-05-08T23:21:15.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fb13bced700 1 -- 172.21.15.5:0/2727737079 shutdown_connections 2024-05-08T23:21:15.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fb13bced700 1 -- 172.21.15.5:0/2727737079 >> 172.21.15.5:0/2727737079 conn(0x7fb1340b1290 msgr2=0x7fb1340b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7fbb1deea700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb180b13b0 0x7fbb18158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:15.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.404+0000 7f5105553700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51000b2730 0x7f51001ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7fbb1d6e9700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb18153e90 0x7fbb18156e60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800b590 tx=0x7fbb0800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7f5104d52700 1 -- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51000cc460 msgr2=0x7f51000b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7f5104d52700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51000cc460 0x7f51000b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7f5104d52700 1 -- 172.21.15.5:0/2849354838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51000b2730 msgr2=0x7f51001ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7f5104d52700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51000b2730 0x7f51001ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7f5104d52700 1 -- 172.21.15.5:0/2849354838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc0094a0 con 0x7f51000c3980 2024-05-08T23:21:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7fbb0e7fc700 1 -- 172.21.15.5:0/332215898 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08011830 con 0x7fbb18153e90 2024-05-08T23:21:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7fbb0e7fc700 1 -- 172.21.15.5:0/332215898 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08011990 con 0x7fbb18153e90 2024-05-08T23:21:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7fbb0e7fc700 1 -- 172.21.15.5:0/332215898 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08011d40 con 0x7fbb18153e90 2024-05-08T23:21:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7ffa65d2e700 1 -- 172.21.15.5:0/3323785437 shutdown_connections 2024-05-08T23:21:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7f5104d52700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 0x7f51000c5550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f50fc015000 tx=0x7f50fc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb181563c0 con 0x7fbb18153e90 2024-05-08T23:21:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb18156660 con 0x7fbb18153e90 2024-05-08T23:21:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7f50f5ffb700 1 -- 172.21.15.5:0/2849354838 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003de0 con 0x7f51000c3980 2024-05-08T23:21:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7f50f5ffb700 1 -- 172.21.15.5:0/2849354838 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc01e070 con 0x7f51000c3980 2024-05-08T23:21:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7f50f5ffb700 1 -- 172.21.15.5:0/2849354838 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc011870 con 0x7f51000c3980 2024-05-08T23:21:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.406+0000 7fb13bced700 1 -- 172.21.15.5:0/2727737079 shutdown_connections 2024-05-08T23:21:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5100200930 con 0x7f51000c3980 2024-05-08T23:21:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.405+0000 7ffa65d2e700 1 -- 172.21.15.5:0/3323785437 wait complete. 2024-05-08T23:21:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7ffa65d2e700 1 Processor -- start 2024-05-08T23:21:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7f43afa30700 1 Processor -- start 2024-05-08T23:21:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5100200060 con 0x7f51000c3980 2024-05-08T23:21:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7f43afa30700 1 -- start start 2024-05-08T23:21:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43afa30700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a80b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43afa30700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43a80cda40 0x7f43a80cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43afa30700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f43a806b970 0x7f43a806be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43afa30700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a806c340 con 0x7f43a80cda40 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43afa30700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a806ed70 con 0x7f43a806b970 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43afa30700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80ce570 con 0x7f43a8142bc0 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43ad7cc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a80b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43ad7cc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a80b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:48114/0 (socket says 172.21.15.5:48114) 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f43ad7cc700 1 -- 172.21.15.5:0/732963693 learned_addr learned my addr 172.21.15.5:0/732963693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7ffa65d2e700 1 -- start start 2024-05-08T23:21:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa65d2e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48005210 0x7ffa6015d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa65d2e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa600c8eb0 0x7ffa600c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa65d2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa600c9330 0x7ffa600c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa65d2e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600cce90 con 0x7ffa600c9330 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa65d2e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa60071ad0 con 0x7ffa600c8eb0 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa65d2e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600cd2b0 con 0x7ffa48005210 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa5f7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48005210 0x7ffa6015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa5f7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48005210 0x7ffa6015d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:48124/0 (socket says 172.21.15.5:48124) 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa5f7fe700 1 -- 172.21.15.5:0/2022627145 learned_addr learned my addr 172.21.15.5:0/2022627145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7ffa5ffff700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa600c9330 0x7ffa600c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7ffa5ffff700 1 -- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48005210 msgr2=0x7ffa6015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7ffa5ffff700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48005210 0x7ffa6015d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7ffa5ffff700 1 -- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa600c8eb0 msgr2=0x7ffa600c9870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7ffa5ffff700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa600c8eb0 0x7ffa600c9870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7ffa5ffff700 1 -- 172.21.15.5:0/2022627145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600cc210 con 0x7ffa600c9330 2024-05-08T23:21:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7ffa5ffff700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa600c9330 0x7ffa600c9db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa6008bc40 tx=0x7ffa600b4e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51000c5270 con 0x7f51000c3980 2024-05-08T23:21:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.406+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c0162460 con 0x7fc3ac005260 2024-05-08T23:21:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.406+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c014cce0 con 0x7fc3ac005260 2024-05-08T23:21:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fc3b67fc700 1 -- 172.21.15.5:0/3476229661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc3b800f3b0 con 0x7fc3ac005260 2024-05-08T23:21:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fc3b67fc700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc3a406ad60 0x7fc3a406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f7cad5d0700 1 Processor -- start 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7f7cad5d0700 1 -- start start 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7f7cad5d0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ca815a770 0x7f7ca80ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7f7cad5d0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ca80cea20 0x7f7ca80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7f7cad5d0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ca80ccbd0 0x7f7ca80cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7f7cad5d0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80caa30 con 0x7f7ca80ccbd0 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7f7cad5d0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80cf790 con 0x7f7ca80cea20 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7f7cad5d0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80cfa70 con 0x7f7ca815a770 2024-05-08T23:21:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ca80cea20 0x7f7ca80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ca80cea20 0x7f7ca80cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36384/0 (socket says 172.21.15.5:36384) 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 -- 172.21.15.5:0/651116779 learned_addr learned my addr 172.21.15.5:0/651116779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca77fe700 1 --2- 172.21.15.5:0/651116779 >> v2:172.21.15.5:3300/0 conn(0x7f7ca80ccbd0 0x7f7ca80cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 -- 172.21.15.5:0/651116779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ca815a770 msgr2=0x7f7ca80ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 --2- 172.21.15.5:0/651116779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ca815a770 0x7f7ca80ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 -- 172.21.15.5:0/651116779 >> v2:172.21.15.5:3300/0 conn(0x7f7ca80ccbd0 msgr2=0x7f7ca80cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 --2- 172.21.15.5:0/651116779 >> v2:172.21.15.5:3300/0 conn(0x7f7ca80ccbd0 0x7f7ca80cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 -- 172.21.15.5:0/651116779 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80cfd50 con 0x7f7ca80cea20 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f7ca67fc700 1 --2- 172.21.15.5:0/651116779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80cea20 0x7f7ca80cee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ca806ca50 tx=0x7f7c90009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe55a03bd5c2dc5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.407+0000 7fb13bced700 1 -- 172.21.15.5:0/2727737079 wait complete. 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7fb13bced700 1 Processor -- start 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.409+0000 7fb13bced700 1 -- start start 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7fb13bced700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb1341510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7fb13bced700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb13415e120 0x7fb13414ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7fb13bced700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb13409c330 0x7fb13409c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7fb13bced700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134151610 con 0x7fb120005260 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7fb13bced700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13414bf50 con 0x7fb13409c330 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7fb13bced700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13409ccf0 con 0x7fb13415e120 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb139a89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb1341510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb139a89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb1341510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60836/0 (socket says 172.21.15.5:60836) 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb139a89700 1 -- 172.21.15.5:0/2473679007 learned_addr learned my addr 172.21.15.5:0/2473679007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb13a28a700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb13409c330 0x7fb13409c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb139a89700 1 -- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb13415e120 msgr2=0x7fb13414ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb139a89700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb13415e120 0x7fb13414ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb139a89700 1 -- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb13409c330 msgr2=0x7fb13409c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb139a89700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb13409c330 0x7fb13409c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fb139a89700 1 -- 172.21.15.5:0/2473679007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13414c160 con 0x7fb120005260 2024-05-08T23:21:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.410+0000 7fbb0e7fc700 1 -- 172.21.15.5:0/332215898 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbb080193b0 con 0x7fbb18153e90 2024-05-08T23:21:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fbb0e7fc700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbafc06ad90 0x7fbafc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fbb0e7fc700 1 -- 172.21.15.5:0/332215898 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb08088ef0 con 0x7fbb18153e90 2024-05-08T23:21:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7fc3b67fc700 1 -- 172.21.15.5:0/3476229661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3b8089600 con 0x7fc3ac005260 2024-05-08T23:21:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fb139a89700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb1341510d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb12400e820 tx=0x7fb12400eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c0068000 con 0x7fc3ac005260 2024-05-08T23:21:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f43ad7cc700 1 -- 172.21.15.5:0/732963693 >> v2:172.21.15.49:3300/0 conn(0x7f43a806b970 msgr2=0x7f43a806be00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f43ad7cc700 1 --2- 172.21.15.5:0/732963693 >> v2:172.21.15.49:3300/0 conn(0x7f43a806b970 0x7f43a806be00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f43ad7cc700 1 -- 172.21.15.5:0/732963693 >> v2:172.21.15.5:3300/0 conn(0x7f43a80cda40 msgr2=0x7f43a80cdec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f43ad7cc700 1 --2- 172.21.15.5:0/732963693 >> v2:172.21.15.5:3300/0 conn(0x7f43a80cda40 0x7f43a80cdec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f43ad7cc700 1 -- 172.21.15.5:0/732963693 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a80ce850 con 0x7f43a8142bc0 2024-05-08T23:21:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f43ad7cc700 1 --2- 172.21.15.5:0/732963693 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a80b2bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f43a8072b60 tx=0x7f43a400be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0808cd94d3fcc34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7ffa47fff700 1 -- 172.21.15.5:0/2022627145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58001f50 con 0x7ffa600c9330 2024-05-08T23:21:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7ffa47fff700 1 -- 172.21.15.5:0/2022627145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58001ad0 con 0x7ffa600c9330 2024-05-08T23:21:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7ffa47fff700 1 -- 172.21.15.5:0/2022627145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5801f600 con 0x7ffa600c9330 2024-05-08T23:21:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb000052f0 con 0x7fbb18153e90 2024-05-08T23:21:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fbb1deea700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbafc06ad90 0x7fbafc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f50f5ffb700 1 -- 172.21.15.5:0/2849354838 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f50fc0119d0 con 0x7f51000c3980 2024-05-08T23:21:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.411+0000 7f50f5ffb700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f50e406ad60 0x7f50e406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f50f5ffb700 1 -- 172.21.15.5:0/2849354838 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50fc08a300 con 0x7f51000c3980 2024-05-08T23:21:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f5105553700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f50e406ad60 0x7f50e406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fb12a7fc700 1 -- 172.21.15.5:0/2473679007 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124013680 con 0x7fb120005260 2024-05-08T23:21:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fb12a7fc700 1 -- 172.21.15.5:0/2473679007 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240137e0 con 0x7fb120005260 2024-05-08T23:21:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13414c440 con 0x7fb120005260 2024-05-08T23:21:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fb12a7fc700 1 -- 172.21.15.5:0/2473679007 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124013b90 con 0x7fb120005260 2024-05-08T23:21:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fc3c5724700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc3a406ad60 0x7fc3a406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa600cc4f0 con 0x7ffa600c9330 2024-05-08T23:21:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa600ccab0 con 0x7ffa600c9330 2024-05-08T23:21:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7fc3c5724700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc3a406ad60 0x7fc3a406d220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00ced70 tx=0x7fc3bc006c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13414ca00 con 0x7fb120005260 2024-05-08T23:21:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7fbb1deea700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbafc06ad90 0x7fbafc06d250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbb100012e0 tx=0x7fbb10001a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7efdead11700 1 Processor -- start 2024-05-08T23:21:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1340678f0 con 0x7fb120005260 2024-05-08T23:21:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7efdead11700 1 -- start start 2024-05-08T23:21:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f439ffff700 1 -- 172.21.15.5:0/732963693 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400dd70 con 0x7f43a8142bc0 2024-05-08T23:21:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f439ffff700 1 -- 172.21.15.5:0/732963693 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a4012450 con 0x7f43a8142bc0 2024-05-08T23:21:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.412+0000 7f439ffff700 1 -- 172.21.15.5:0/732963693 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4012800 con 0x7f43a8142bc0 2024-05-08T23:21:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7f43afa30700 1 -- 172.21.15.5:0/732963693 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 msgr2=0x7f43a80b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.413+0000 7f43afa30700 1 --2- 172.21.15.5:0/732963693 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a80b2bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f43a8072b60 tx=0x7f43a400be70 comp rx=0 tx=0).stop 2024-05-08T23:21:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.414+0000 7f43afa30700 1 -- 172.21.15.5:0/732963693 shutdown_connections 2024-05-08T23:21:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.414+0000 7f43afa30700 1 --2- 172.21.15.5:0/732963693 >> v2:172.21.15.49:3300/0 conn(0x7f43a806b970 0x7f43a806be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.414+0000 7f43afa30700 1 --2- 172.21.15.5:0/732963693 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a80b2bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.414+0000 7f43afa30700 1 --2- 172.21.15.5:0/732963693 >> v2:172.21.15.5:3300/0 conn(0x7f43a80cda40 0x7f43a80cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.414+0000 7f43afa30700 1 -- 172.21.15.5:0/732963693 >> 172.21.15.5:0/732963693 conn(0x7f43a80cd550 msgr2=0x7f43a80713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.414+0000 7f43afa30700 1 -- 172.21.15.5:0/732963693 shutdown_connections 2024-05-08T23:21:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.414+0000 7f43afa30700 1 -- 172.21.15.5:0/732963693 wait complete. 2024-05-08T23:21:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.414+0000 7f5105553700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f50e406ad60 0x7f50e406d220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f50f8001d90 tx=0x7f50f8001d20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efdead11700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde40b0e00 0x7efde40b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efdead11700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efde40b2c60 0x7efde415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efdead11700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde415f5a0 con 0x7efde406e890 2024-05-08T23:21:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efdead11700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde406ce20 con 0x7efde40b2c60 2024-05-08T23:21:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efdead11700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde40b1720 con 0x7efde40b0e00 2024-05-08T23:21:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efde92ae700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efde406e890 0x7efde415ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44658/0 (socket says 172.21.15.5:44658) 2024-05-08T23:21:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efde92ae700 1 -- 172.21.15.5:0/2751647943 learned_addr learned my addr 172.21.15.5:0/2751647943 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efde3fff700 1 --2- 172.21.15.5:0/2751647943 >> v2:172.21.15.49:3300/0 conn(0x7efde40b2c60 0x7efde415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7efde8aad700 1 --2- 172.21.15.5:0/2751647943 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde40b0e00 0x7efde40b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7f43afa30700 1 Processor -- start 2024-05-08T23:21:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7f43afa30700 1 -- start start 2024-05-08T23:21:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.415+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa600ccfd0 con 0x7ffa600c9330 2024-05-08T23:21:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7efde3fff700 1 -- 172.21.15.5:0/2751647943 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde40b0e00 msgr2=0x7efde40b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7efde3fff700 1 --2- 172.21.15.5:0/2751647943 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde40b0e00 0x7efde40b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7efde3fff700 1 -- 172.21.15.5:0/2751647943 >> v1:172.21.15.5:6789/0 conn(0x7efde406e890 legacy=0x7efde415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7efde3fff700 1 -- 172.21.15.5:0/2751647943 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde415e500 con 0x7efde40b2c60 2024-05-08T23:21:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7efde3fff700 1 --2- 172.21.15.5:0/2751647943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde40b2c60 0x7efde415dcd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009840 tx=0x7efdd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53ba38d67f27d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43afa30700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43a80cda40 0x7f43a815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43afa30700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a815e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43afa30700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a815cac0 0x7f43a8158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43afa30700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8164360 con 0x7f43a80cda40 2024-05-08T23:21:15.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43afa30700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8164600 con 0x7f43a815cac0 2024-05-08T23:21:15.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43afa30700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a815f560 con 0x7f43a8142bc0 2024-05-08T23:21:15.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43ad7cc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43a80cda40 0x7f43a815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43ad7cc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43a80cda40 0x7f43a815de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60846/0 (socket says 172.21.15.5:60846) 2024-05-08T23:21:15.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.416+0000 7f43ad7cc700 1 -- 172.21.15.5:0/3925711593 learned_addr learned my addr 172.21.15.5:0/3925711593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7efde37fe700 1 -- 172.21.15.5:0/2751647943 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8011830 con 0x7efde40b2c60 2024-05-08T23:21:15.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f50f5ffb700 1 -- 172.21.15.5:0/2849354838 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f51000c5270 con 0x7f51000c3980 2024-05-08T23:21:15.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f7ca5ffb700 1 -- 172.21.15.5:0/651116779 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000b870 con 0x7f7ca80cea20 2024-05-08T23:21:15.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43acfcb700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43adfcd700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a815cac0 0x7f43a8158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43ad7cc700 1 -- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 msgr2=0x7f43a815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43ad7cc700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a815e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43ad7cc700 1 -- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a815cac0 msgr2=0x7f43a8158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43ad7cc700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a815cac0 0x7f43a8158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43ad7cc700 1 -- 172.21.15.5:0/3925711593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a400b7f0 con 0x7f43a80cda40 2024-05-08T23:21:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43acfcb700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.417+0000 7f43ad7cc700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43a80cda40 0x7f43a815de00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43a400d6d0 tx=0x7f43a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.419+0000 7fc3b67fc700 1 -- 172.21.15.5:0/3476229661 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc3b80547d0 con 0x7fc3ac005260 2024-05-08T23:21:15.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.421+0000 7ffa47fff700 1 -- 172.21.15.5:0/2022627145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffa5801a070 con 0x7ffa600c9330 2024-05-08T23:21:15.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.421+0000 7ffa47fff700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa4c06ace0 0x7ffa4c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.422+0000 7ffa47fff700 1 -- 172.21.15.5:0/2022627145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa5808b0b0 con 0x7ffa600c9330 2024-05-08T23:21:15.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.422+0000 7ffa5f7fe700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa4c06ace0 0x7ffa4c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.422+0000 7fbb0e7fc700 1 -- 172.21.15.5:0/332215898 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbb08054140 con 0x7fbb18153e90 2024-05-08T23:21:15.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.422+0000 7ffa5f7fe700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa4c06ace0 0x7ffa4c06d1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009d00 tx=0x7ffa50009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.421+0000 7efde37fe700 1 -- 172.21.15.5:0/2751647943 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd8011990 con 0x7efde40b2c60 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.421+0000 7efde37fe700 1 -- 172.21.15.5:0/2751647943 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8011d20 con 0x7efde40b2c60 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7efdead11700 1 -- 172.21.15.5:0/2751647943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde40b2c60 msgr2=0x7efde415dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7efdead11700 1 --2- 172.21.15.5:0/2751647943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde40b2c60 0x7efde415dcd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009840 tx=0x7efdd8009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7efdead11700 1 -- 172.21.15.5:0/2751647943 shutdown_connections 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7efdead11700 1 --2- 172.21.15.5:0/2751647943 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde40b0e00 0x7efde40b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7efdead11700 1 --2- 172.21.15.5:0/2751647943 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde40b2c60 0x7efde415dcd0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009840 tx=0x7efdd8009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7efdead11700 1 -- 172.21.15.5:0/2751647943 >> 172.21.15.5:0/2751647943 conn(0x7efde40b4690 msgr2=0x7efde40b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7efdead11700 1 -- 172.21.15.5:0/2751647943 shutdown_connections 2024-05-08T23:21:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7efdead11700 1 -- 172.21.15.5:0/2751647943 wait complete. 2024-05-08T23:21:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7f7ca5ffb700 1 -- 172.21.15.5:0/651116779 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9000b9d0 con 0x7f7ca80cea20 2024-05-08T23:21:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.423+0000 7f7ca5ffb700 1 -- 172.21.15.5:0/651116779 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000bd40 con 0x7f7ca80cea20 2024-05-08T23:21:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.424+0000 7f7cad5d0700 1 -- 172.21.15.5:0/651116779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80cea20 msgr2=0x7f7ca80cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.424+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/651116779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80cea20 0x7f7ca80cee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ca806ca50 tx=0x7f7c90009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.425+0000 7efdead11700 1 Processor -- start 2024-05-08T23:21:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.424+0000 7fb12a7fc700 1 -- 172.21.15.5:0/2473679007 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb124013cf0 con 0x7fb120005260 2024-05-08T23:21:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.425+0000 7fb12a7fc700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb11806ad60 0x7fb11806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.425+0000 7fb12a7fc700 1 -- 172.21.15.5:0/2473679007 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb124089520 con 0x7fb120005260 2024-05-08T23:21:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.424+0000 7f439dffb700 1 -- 172.21.15.5:0/3925711593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4012070 con 0x7f43a80cda40 2024-05-08T23:21:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.424+0000 7f439dffb700 1 -- 172.21.15.5:0/3925711593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a40038e0 con 0x7f43a80cda40 2024-05-08T23:21:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.424+0000 7f439dffb700 1 -- 172.21.15.5:0/3925711593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4003c90 con 0x7f43a80cda40 2024-05-08T23:21:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.424+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a8164360 con 0x7f43a80cda40 2024-05-08T23:21:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.425+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a8159c30 con 0x7f43a80cda40 2024-05-08T23:21:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7fb139288700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb11806ad60 0x7fb11806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.425+0000 7efdead11700 1 -- start start 2024-05-08T23:21:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efdead11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde406e890 0x7efde40ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efdead11700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde40b0e00 0x7efde4154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efdead11700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde4155da0 0x7efde41551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efdead11700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde40b0660 con 0x7efde40b0e00 2024-05-08T23:21:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efdead11700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde409a1a0 con 0x7efde406e890 2024-05-08T23:21:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efdead11700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde4099620 con 0x7efde4155da0 2024-05-08T23:21:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7fb139288700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb11806ad60 0x7fb11806d220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0070b0 tx=0x7fb12c007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efde92ae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde4155da0 0x7efde41551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efde92ae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde4155da0 0x7efde41551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:48164/0 (socket says 172.21.15.5:48164) 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.426+0000 7efde92ae700 1 -- 172.21.15.5:0/3206134789 learned_addr learned my addr 172.21.15.5:0/3206134789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde8aad700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde406e890 0x7efde40ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde92ae700 1 -- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde406e890 msgr2=0x7efde40ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde92ae700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde406e890 0x7efde40ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde92ae700 1 -- 172.21.15.5:0/3206134789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde40b0e00 msgr2=0x7efde4154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde92ae700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde40b0e00 0x7efde4154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde92ae700 1 -- 172.21.15.5:0/3206134789 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd80094a0 con 0x7efde4155da0 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde92ae700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde4155da0 0x7efde41551f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efddc00caa0 tx=0x7efddc00ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde17fa700 1 -- 172.21.15.5:0/3206134789 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc00aa40 con 0x7efde4155da0 2024-05-08T23:21:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efde17fa700 1 -- 172.21.15.5:0/3206134789 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc00aba0 con 0x7efde4155da0 2024-05-08T23:21:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.427+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde4099900 con 0x7efde4155da0 2024-05-08T23:21:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.428+0000 7f7cad5d0700 1 -- 172.21.15.5:0/651116779 shutdown_connections 2024-05-08T23:21:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.428+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/651116779 >> v2:172.21.15.5:3300/0 conn(0x7f7ca80ccbd0 0x7f7ca80cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.428+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/651116779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ca815a770 0x7f7ca80ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.428+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/651116779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80cea20 0x7f7ca80cee80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.428+0000 7f7cad5d0700 1 -- 172.21.15.5:0/651116779 >> 172.21.15.5:0/651116779 conn(0x7f7ca80b1f80 msgr2=0x7f7ca80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.428+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde4099ec0 con 0x7efde4155da0 2024-05-08T23:21:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.428+0000 7ffa47fff700 1 -- 172.21.15.5:0/2022627145 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffa58056300 con 0x7ffa600c9330 2024-05-08T23:21:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.429+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde41580f0 con 0x7efde4155da0 2024-05-08T23:21:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.431+0000 7efde17fa700 1 -- 172.21.15.5:0/3206134789 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc012400 con 0x7efde4155da0 2024-05-08T23:21:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.430+0000 7f7cad5d0700 1 -- 172.21.15.5:0/651116779 shutdown_connections 2024-05-08T23:21:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.430+0000 7f7cad5d0700 1 -- 172.21.15.5:0/651116779 wait complete. 2024-05-08T23:21:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.432+0000 7f7cad5d0700 1 Processor -- start 2024-05-08T23:21:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.433+0000 7efde17fa700 1 -- 172.21.15.5:0/3206134789 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efddc01b3b0 con 0x7efde4155da0 2024-05-08T23:21:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.433+0000 7efde17fa700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efdd006ace0 0x7efdd006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7efde17fa700 1 -- 172.21.15.5:0/3206134789 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efddc089400 con 0x7efde4155da0 2024-05-08T23:21:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7efde8aad700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efdd006ace0 0x7efdd006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.435+0000 7efde8aad700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efdd006ace0 0x7efdd006d1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efde40cfa80 tx=0x7efdd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.432+0000 7f7cad5d0700 1 -- start start 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.433+0000 7f7cad5d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80ccbd0 0x7f7ca8149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.433+0000 7f7cad5d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80cea20 0x7f7ca814b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.433+0000 7f7cad5d0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ca815a770 0x7f7ca80c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.433+0000 7f7cad5d0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8145920 con 0x7f7ca80cea20 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.433+0000 7f7cad5d0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca814bfb0 con 0x7f7ca80ccbd0 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.433+0000 7f7cad5d0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca814c290 con 0x7f7ca815a770 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80ccbd0 0x7f7ca8149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80ccbd0 0x7f7ca8149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36434/0 (socket says 172.21.15.5:36434) 2024-05-08T23:21:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca6ffd700 1 -- 172.21.15.5:0/57625630 learned_addr learned my addr 172.21.15.5:0/57625630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca67fc700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80cea20 0x7f7ca814b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca67fc700 1 -- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ca815a770 msgr2=0x7f7ca80c6240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:21:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca67fc700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ca815a770 0x7f7ca80c6240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca67fc700 1 -- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80ccbd0 msgr2=0x7f7ca8149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca67fc700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80ccbd0 0x7f7ca8149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.434+0000 7f7ca67fc700 1 -- 172.21.15.5:0/57625630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90009550 con 0x7f7ca80cea20 2024-05-08T23:21:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.435+0000 7f7ca67fc700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80cea20 0x7f7ca814b740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c90016000 tx=0x7f7c90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.435+0000 7efde17fa700 1 -- 172.21.15.5:0/3206134789 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efddc054640 con 0x7efde4155da0 2024-05-08T23:21:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.435+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43880052f0 con 0x7f43a80cda40 2024-05-08T23:21:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.438+0000 7f439dffb700 1 -- 172.21.15.5:0/3925711593 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f43a40125c0 con 0x7f43a80cda40 2024-05-08T23:21:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.439+0000 7f439dffb700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f438c06ad60 0x7f438c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.439+0000 7f439dffb700 1 -- 172.21.15.5:0/3925711593 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43a408a530 con 0x7f43a80cda40 2024-05-08T23:21:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.440+0000 7f43acfcb700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f438c06ad60 0x7f438c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.436+0000 7fb12a7fc700 1 -- 172.21.15.5:0/2473679007 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb1240546e0 con 0x7fb120005260 2024-05-08T23:21:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.440+0000 7f7c9f7fe700 1 -- 172.21.15.5:0/57625630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90004380 con 0x7f7ca80cea20 2024-05-08T23:21:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.440+0000 7f7c9f7fe700 1 -- 172.21.15.5:0/57625630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9001f070 con 0x7f7ca80cea20 2024-05-08T23:21:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.440+0000 7f7c9f7fe700 1 -- 172.21.15.5:0/57625630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c900036a0 con 0x7f7ca80cea20 2024-05-08T23:21:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.441+0000 7f43acfcb700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f438c06ad60 0x7f438c06d220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4398006fd0 tx=0x7f43980090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.441+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca814c980 con 0x7f7ca80cea20 2024-05-08T23:21:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.441+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca814cf40 con 0x7f7ca80cea20 2024-05-08T23:21:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.445+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c0052f0 con 0x7f7ca80cea20 2024-05-08T23:21:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.449+0000 7f439dffb700 1 -- 172.21.15.5:0/3925711593 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f43a40558a0 con 0x7f43a80cda40 2024-05-08T23:21:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.451+0000 7f7c9f7fe700 1 -- 172.21.15.5:0/57625630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7c90021030 con 0x7f7ca80cea20 2024-05-08T23:21:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.452+0000 7f7c9f7fe700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c9406aef0 0x7f7c9406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.452+0000 7f7c9f7fe700 1 -- 172.21.15.5:0/57625630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c9008ab60 con 0x7f7ca80cea20 2024-05-08T23:21:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.452+0000 7f7ca6ffd700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c9406aef0 0x7f7c9406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.452+0000 7f7c9f7fe700 1 -- 172.21.15.5:0/57625630 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7c900555f0 con 0x7f7ca80cea20 2024-05-08T23:21:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.479+0000 7f7ca6ffd700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c9406aef0 0x7f7c9406d3b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c98006fd0 tx=0x7f7c98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:15.872 INFO:teuthology.orchestra.run.smithi005.stdout:55834574853 2024-05-08T23:21:15.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.869+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f51000c5270 con 0x7f51000c3980 2024-05-08T23:21:15.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.870+0000 7f50f5ffb700 1 -- 172.21.15.5:0/2849354838 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50fc020020 con 0x7f51000c3980 2024-05-08T23:21:15.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.872+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc3c014cfc0 con 0x7fc3ac005260 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f50e406ad60 msgr2=0x7f50e406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f50e406ad60 0x7f50e406d220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f50f8001d90 tx=0x7f50f8001d20 comp rx=0 tx=0).stop 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 msgr2=0x7f51000c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 0x7f51000c5550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f50fc015000 tx=0x7f50fc004750 comp rx=0 tx=0).stop 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 shutdown_connections 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f50e406ad60 0x7f50e406d220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51000cc460 0x7f51000b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51000b2730 0x7f51001ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 --2- 172.21.15.5:0/2849354838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51000c3980 0x7f51000c5550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 >> 172.21.15.5:0/2849354838 conn(0x7f5100099a50 msgr2=0x7f51000ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 shutdown_connections 2024-05-08T23:21:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7f51077b7700 1 -- 172.21.15.5:0/2849354838 wait complete. 2024-05-08T23:21:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.874+0000 7fc3b67fc700 1 -- 172.21.15.5:0/3476229661 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3b801e030 con 0x7fc3ac005260 2024-05-08T23:21:15.875 INFO:teuthology.orchestra.run.smithi005.stdout:55834574853 2024-05-08T23:21:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.877+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc3a406ad60 msgr2=0x7fc3a406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.877+0000 7fc3c8189700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc3a406ad60 0x7fc3a406d220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00ced70 tx=0x7fc3bc006c60 comp rx=0 tx=0).stop 2024-05-08T23:21:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.877+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac005260 msgr2=0x7fc3c0163d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.877+0000 7fc3c8189700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac005260 0x7fc3c0163d60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800b760 tx=0x7fc3b800bb20 comp rx=0 tx=0).stop 2024-05-08T23:21:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.878+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 shutdown_connections 2024-05-08T23:21:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.878+0000 7fc3c8189700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc3a406ad60 0x7fc3a406d220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.878+0000 7fc3c8189700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3c0148de0 0x7fc3c014c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.878+0000 7fc3c8189700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3ac005260 0x7fc3c0163d60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.878+0000 7fc3c8189700 1 --2- 172.21.15.5:0/3476229661 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3c0160d10 0x7fc3c01488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.878+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 >> 172.21.15.5:0/3476229661 conn(0x7fc3c006af90 msgr2=0x7fc3c006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.878+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 shutdown_connections 2024-05-08T23:21:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.878+0000 7fc3c8189700 1 -- 172.21.15.5:0/3476229661 wait complete. 2024-05-08T23:21:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.881+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb000055a0 con 0x7fbb18153e90 2024-05-08T23:21:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.882+0000 7fbb0e7fc700 1 -- 172.21.15.5:0/332215898 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb08015030 con 0x7fbb18153e90 2024-05-08T23:21:15.890 INFO:teuthology.orchestra.run.smithi005.stdout:55834574853 2024-05-08T23:21:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.891+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ffa600cf470 con 0x7ffa600c9330 2024-05-08T23:21:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.892+0000 7ffa47fff700 1 -- 172.21.15.5:0/2022627145 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffa600cf470 con 0x7ffa600c9330 2024-05-08T23:21:15.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.892+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbafc06ad90 msgr2=0x7fbafc06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.892+0000 7fbb2014e700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbafc06ad90 0x7fbafc06d250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbb100012e0 tx=0x7fbb10001a30 comp rx=0 tx=0).stop 2024-05-08T23:21:15.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.892+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb18153e90 msgr2=0x7fbb18156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.892+0000 7fbb2014e700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb18153e90 0x7fbb18156e60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800b590 tx=0x7fbb0800b8a0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:55834574853 2024-05-08T23:21:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.893+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 shutdown_connections 2024-05-08T23:21:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.893+0000 7fbb2014e700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbafc06ad90 0x7fbafc06d250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.893+0000 7fbb2014e700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb1815aae0 0x7fbb1816a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.893+0000 7fbb2014e700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb180b13b0 0x7fbb18158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.893+0000 7fbb2014e700 1 --2- 172.21.15.5:0/332215898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb18153e90 0x7fbb18156e60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.893+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 >> 172.21.15.5:0/332215898 conn(0x7fbb180b4690 msgr2=0x7fbb18158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.893+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7efde40678f0 con 0x7efde4155da0 2024-05-08T23:21:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.893+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb1340cdf60 con 0x7fb120005260 2024-05-08T23:21:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.894+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 shutdown_connections 2024-05-08T23:21:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.894+0000 7fbb2014e700 1 -- 172.21.15.5:0/332215898 wait complete. 2024-05-08T23:21:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.894+0000 7efde17fa700 1 -- 172.21.15.5:0/3206134789 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efddc017090 con 0x7efde4155da0 2024-05-08T23:21:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.894+0000 7fb12a7fc700 1 -- 172.21.15.5:0/2473679007 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb124057d00 con 0x7fb120005260 2024-05-08T23:21:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:15.896 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.897+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa4c06ace0 msgr2=0x7ffa4c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.897+0000 7ffa65d2e700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa4c06ace0 0x7ffa4c06d1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009d00 tx=0x7ffa50009210 comp rx=0 tx=0).stop 2024-05-08T23:21:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.897+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa600c9330 msgr2=0x7ffa600c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.897+0000 7ffa65d2e700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa600c9330 0x7ffa600c9db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa6008bc40 tx=0x7ffa600b4e10 comp rx=0 tx=0).stop 2024-05-08T23:21:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.897+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4388005870 con 0x7f43a80cda40 2024-05-08T23:21:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 shutdown_connections 2024-05-08T23:21:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7ffa65d2e700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa4c06ace0 0x7ffa4c06d1a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7ffa65d2e700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa600c9330 0x7ffa600c9db0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7ffa65d2e700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa48005210 0x7ffa6015d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7ffa65d2e700 1 --2- 172.21.15.5:0/2022627145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa600c8eb0 0x7ffa600c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 >> 172.21.15.5:0/2022627145 conn(0x7ffa6008afe0 msgr2=0x7ffa600737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7f439dffb700 1 -- 172.21.15.5:0/3925711593 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f43a4058ec0 con 0x7f43a80cda40 2024-05-08T23:21:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 shutdown_connections 2024-05-08T23:21:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7ffa65d2e700 1 -- 172.21.15.5:0/2022627145 wait complete. 2024-05-08T23:21:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb11806ad60 msgr2=0x7fb11806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7fb13bced700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb11806ad60 0x7fb11806d220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0070b0 tx=0x7fb12c007040 comp rx=0 tx=0).stop 2024-05-08T23:21:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 msgr2=0x7fb1341510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.898+0000 7fb13bced700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb1341510d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb12400e820 tx=0x7fb12400eb30 comp rx=0 tx=0).stop 2024-05-08T23:21:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 shutdown_connections 2024-05-08T23:21:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7fb13bced700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb11806ad60 0x7fb11806d220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7fb13bced700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb13409c330 0x7fb13409c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7fb13bced700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb120005260 0x7fb1341510d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7fb13bced700 1 --2- 172.21.15.5:0/2473679007 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb13415e120 0x7fb13414ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 >> 172.21.15.5:0/2473679007 conn(0x7fb1340b1290 msgr2=0x7fb1340b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 shutdown_connections 2024-05-08T23:21:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7fb13bced700 1 -- 172.21.15.5:0/2473679007 wait complete. 2024-05-08T23:21:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efdd006ace0 msgr2=0x7efdd006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efdd006ace0 0x7efdd006d1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efde40cfa80 tx=0x7efdd4008040 comp rx=0 tx=0).stop 2024-05-08T23:21:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde4155da0 msgr2=0x7efde41551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde4155da0 0x7efde41551f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efddc00caa0 tx=0x7efddc00ce60 comp rx=0 tx=0).stop 2024-05-08T23:21:15.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-05-08T23:21:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 shutdown_connections 2024-05-08T23:21:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efdd006ace0 0x7efdd006d1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efde4155da0 0x7efde41551f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde406e890 0x7efde40ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 --2- 172.21.15.5:0/3206134789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde40b0e00 0x7efde4154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.899+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 >> 172.21.15.5:0/3206134789 conn(0x7efde40b4690 msgr2=0x7efde41609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-05-08T23:21:15.906 INFO:teuthology.orchestra.run.smithi005.stdout:55834574853 2024-05-08T23:21:15.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.900+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 shutdown_connections 2024-05-08T23:21:15.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.900+0000 7efdead11700 1 -- 172.21.15.5:0/3206134789 wait complete. 2024-05-08T23:21:15.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.901+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7c8c005870 con 0x7f7ca80cea20 2024-05-08T23:21:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f438c06ad60 msgr2=0x7f438c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f438c06ad60 0x7f438c06d220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4398006fd0 tx=0x7f43980090e0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43a80cda40 msgr2=0x7f43a815de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43a80cda40 0x7f43a815de00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43a400d6d0 tx=0x7f43a4004750 comp rx=0 tx=0).stop 2024-05-08T23:21:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 shutdown_connections 2024-05-08T23:21:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f438c06ad60 0x7f438c06d220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a815cac0 0x7f43a8158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43a80cda40 0x7f43a815de00 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43a400d6d0 tx=0x7f43a4004750 comp rx=0 tx=0).stop 2024-05-08T23:21:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 --2- 172.21.15.5:0/3925711593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f43a8142bc0 0x7f43a815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.904+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 >> 172.21.15.5:0/3925711593 conn(0x7f43a806b970 msgr2=0x7f43a80cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.906+0000 7f7c9f7fe700 1 -- 172.21.15.5:0/57625630 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c90015020 con 0x7f7ca80cea20 2024-05-08T23:21:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.907+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 shutdown_connections 2024-05-08T23:21:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.908+0000 7f43afa30700 1 -- 172.21.15.5:0/3925711593 wait complete. 2024-05-08T23:21:15.909 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:15.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c9406aef0 msgr2=0x7f7c9406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c9406aef0 0x7f7c9406d3b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c98006fd0 tx=0x7f7c98008040 comp rx=0 tx=0).stop 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80cea20 msgr2=0x7f7ca814b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80cea20 0x7f7ca814b740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c90016000 tx=0x7f7c90004750 comp rx=0 tx=0).stop 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 shutdown_connections 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c9406aef0 0x7f7c9406d3b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ca815a770 0x7f7ca80c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ca80ccbd0 0x7f7ca8149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 --2- 172.21.15.5:0/57625630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80cea20 0x7f7ca814b740 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.913+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 >> 172.21.15.5:0/57625630 conn(0x7f7ca80b1f80 msgr2=0x7f7ca80d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.914+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 shutdown_connections 2024-05-08T23:21:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:15.914+0000 7f7cad5d0700 1 -- 172.21.15.5:0/57625630 wait complete. 2024-05-08T23:21:15.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-05-08T23:21:15.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-05-08T23:21:15.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-05-08T23:21:15.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-05-08T23:21:15.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-05-08T23:21:15.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-05-08T23:21:16.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T23:21:16.906 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T23:21:16.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T23:21:16.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T23:21:16.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T23:21:16.925 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T23:21:16.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T23:21:16.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T23:21:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.124+0000 7f840e83a700 1 Processor -- start 2024-05-08T23:21:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.124+0000 7f840e83a700 1 -- start start 2024-05-08T23:21:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.126+0000 7f840e83a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84080b19d0 0x7f84081643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.126+0000 7f840e83a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84080ac1c0 con 0x7f840815aae0 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.126+0000 7f840e83a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840806ccb0 con 0x7f8408153f20 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.126+0000 7f840e83a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8408154300 con 0x7f84080b19d0 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.126+0000 7f84077fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f840815aae0 0x7f84080b12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44666/0 (socket says 172.21.15.5:44666) 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.126+0000 7f84077fe700 1 -- 172.21.15.5:0/3524456250 learned_addr learned my addr 172.21.15.5:0/3524456250 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049512620 0 0) 0x7f84080ac1c0 con 0x7f840815aae0 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f00035b0 con 0x7f840815aae0 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3253453637 0 0) 0x7f83f00035b0 con 0x7f840815aae0 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 >> v2:172.21.15.49:3300/0 conn(0x7f84080b19d0 msgr2=0x7f84081643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 --2- 172.21.15.5:0/3524456250 >> v2:172.21.15.49:3300/0 conn(0x7f84080b19d0 0x7f84081643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 >> v1:172.21.15.49:6789/0 conn(0x7f8408153f20 legacy=0x7f84080b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840806cf90 con 0x7f840815aae0 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f83f8001670 con 0x7f840815aae0 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 >> v1:172.21.15.5:6789/0 conn(0x7f840815aae0 legacy=0x7f84080b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83f0003a00 0x7f83f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f0004850 0x7f83f0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0005330 0x7f83f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8408154300 con 0x7f83f0005330 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f000c210 con 0x7f83f0004850 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.127+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f000c3d0 con 0x7f83f0003a00 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.128+0000 7f84077fe700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f0004850 0x7f83f0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.128+0000 7f8407fff700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83f0003a00 0x7f83f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f84077fe700 1 -- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83f0003a00 msgr2=0x7f83f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f84077fe700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83f0003a00 0x7f83f0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f84077fe700 1 -- 172.21.15.5:0/3524456250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0005330 msgr2=0x7f83f000ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f84077fe700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0005330 0x7f83f000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f84077fe700 1 -- 172.21.15.5:0/3524456250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840806cf90 con 0x7f83f0004850 2024-05-08T23:21:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f84077fe700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f0004850 0x7f83f0004cd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d3d0 tx=0x7f83f0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4015e7c9bbf26514 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f840cdd7700 1 -- 172.21.15.5:0/3524456250 reap_dead start 2024-05-08T23:21:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f800f400 con 0x7f83f0004850 2024-05-08T23:21:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f800f560 con 0x7f83f0004850 2024-05-08T23:21:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.129+0000 7f8406ffd700 1 -- 172.21.15.5:0/3524456250 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f800f910 con 0x7f83f0004850 2024-05-08T23:21:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.131+0000 7f840e83a700 1 -- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f0004850 msgr2=0x7f83f0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.131+0000 7f840e83a700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f0004850 0x7f83f0004cd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d3d0 tx=0x7f83f0005880 comp rx=0 tx=0).stop 2024-05-08T23:21:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.131+0000 7f51b2f0f700 1 Processor -- start 2024-05-08T23:21:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.131+0000 7f51b2f0f700 1 -- start start 2024-05-08T23:21:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f840e83a700 1 -- 172.21.15.5:0/3524456250 shutdown_connections 2024-05-08T23:21:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f840e83a700 1 --2- 172.21.15.5:0/3524456250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f0004850 0x7f83f0004cd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f840e83a700 1 -- 172.21.15.5:0/3524456250 >> 172.21.15.5:0/3524456250 conn(0x7f84080b4690 msgr2=0x7f84080b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f840e83a700 1 -- 172.21.15.5:0/3524456250 shutdown_connections 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f51b2f0f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51ac06b490 0x7f51ac1621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f51b2f0f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac162ba0 con 0x7f51ac15f900 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f51b2f0f700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0711d0 con 0x7f51ac162710 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f51b2f0f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac070f30 con 0x7f51ac06b490 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.133+0000 7f51b0cab700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f51ac15f900 0x7f51ac06ad80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44672/0 (socket says 172.21.15.5:44672) 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.133+0000 7f51b0cab700 1 -- 172.21.15.5:0/1283630092 learned_addr learned my addr 172.21.15.5:0/1283630092 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416215096 0 0) 0x7f51ac162ba0 con 0x7f51ac15f900 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51940035b0 con 0x7f51ac15f900 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2318376133 0 0) 0x7f51940035b0 con 0x7f51ac15f900 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51ac06b490 msgr2=0x7f51ac1621d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f51ac06b490 0x7f51ac1621d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 >> v1:172.21.15.49:6789/0 conn(0x7f51ac162710 legacy=0x7f51ac071c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac162e30 con 0x7f51ac15f900 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f519c001670 con 0x7f51ac15f900 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 >> v1:172.21.15.5:6789/0 conn(0x7f51ac15f900 legacy=0x7f51ac06ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5194003a30 0x7f5194003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194004850 0x7f5194004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5194005280 0x7f519400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac070f30 con 0x7f5194005280 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519400c190 con 0x7f5194003a30 2024-05-08T23:21:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.134+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519400c350 con 0x7f5194004850 2024-05-08T23:21:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.132+0000 7f840e83a700 1 -- 172.21.15.5:0/3524456250 wait complete. 2024-05-08T23:21:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.135+0000 7f840e83a700 1 Processor -- start 2024-05-08T23:21:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.135+0000 7f51b14ac700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5194005280 0x7f519400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.136+0000 7f51b14ac700 1 -- 172.21.15.5:0/1283630092 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194004850 msgr2=0x7f5194004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.136+0000 7f51b14ac700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194004850 0x7f5194004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.136+0000 7f51b14ac700 1 -- 172.21.15.5:0/1283630092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5194003a30 msgr2=0x7f5194003e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.136+0000 7f51b14ac700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5194003a30 0x7f5194003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.136+0000 7f51b14ac700 1 -- 172.21.15.5:0/1283630092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac162e30 con 0x7f5194005280 2024-05-08T23:21:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.136+0000 7f51b14ac700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5194005280 0x7f519400ba10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009880 tx=0x7f5194005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d672f335cfb25c12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0011620 con 0x7f5194005280 2024-05-08T23:21:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a0011780 con 0x7f5194005280 2024-05-08T23:21:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f51ab7fe700 1 -- 172.21.15.5:0/1283630092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0011b30 con 0x7f5194005280 2024-05-08T23:21:17.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f51b2f0f700 1 -- 172.21.15.5:0/1283630092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5194005280 msgr2=0x7f519400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f51b2f0f700 1 --2- 172.21.15.5:0/1283630092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5194005280 0x7f519400ba10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009880 tx=0x7f5194005880 comp rx=0 tx=0).stop 2024-05-08T23:21:17.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f51b14ac700 1 -- 172.21.15.5:0/1283630092 reap_dead start 2024-05-08T23:21:17.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.136+0000 7f840e83a700 1 -- start start 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f840e83a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0004850 0x7f84081533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f840e83a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84081455d0 0x7f8408151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f840e83a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8408145a50 0x7f8408151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f840e83a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8408154890 con 0x7f83f0004850 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f840e83a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84080adae0 con 0x7f84081455d0 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f840e83a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84080addc0 con 0x7f8408145a50 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f8407fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0004850 0x7f84081533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f8407fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0004850 0x7f84081533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60878/0 (socket says 172.21.15.5:60878) 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.137+0000 7f8407fff700 1 -- 172.21.15.5:0/1691133398 learned_addr learned my addr 172.21.15.5:0/1691133398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f8407fff700 1 -- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8408145a50 msgr2=0x7f8408151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f8407fff700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8408145a50 0x7f8408151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f8407fff700 1 -- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84081455d0 msgr2=0x7f8408151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f8407fff700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84081455d0 0x7f8408151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f8407fff700 1 -- 172.21.15.5:0/1691133398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080ae0a0 con 0x7f83f0004850 2024-05-08T23:21:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.138+0000 7f8407fff700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0004850 0x7f84081533f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00f840 tx=0x7f83fc00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.140+0000 7f51b2f0f700 1 -- 172.21.15.5:0/1283630092 shutdown_connections 2024-05-08T23:21:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.140+0000 7f51b2f0f700 1 -- 172.21.15.5:0/1283630092 >> 172.21.15.5:0/1283630092 conn(0x7f51ac0b1290 msgr2=0x7f51ac0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.141+0000 7f51b2f0f700 1 -- 172.21.15.5:0/1283630092 shutdown_connections 2024-05-08T23:21:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.141+0000 7f51b2f0f700 1 -- 172.21.15.5:0/1283630092 wait complete. 2024-05-08T23:21:17.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.143+0000 7f51b2f0f700 1 Processor -- start 2024-05-08T23:21:17.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.141+0000 7f8404ff9700 1 -- 172.21.15.5:0/1691133398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc004b70 con 0x7f83f0004850 2024-05-08T23:21:17.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.141+0000 7f8404ff9700 1 -- 172.21.15.5:0/1691133398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc014680 con 0x7f83f0004850 2024-05-08T23:21:17.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.141+0000 7f8404ff9700 1 -- 172.21.15.5:0/1691133398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc014a10 con 0x7f83f0004850 2024-05-08T23:21:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.143+0000 7f51b2f0f700 1 -- start start 2024-05-08T23:21:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.144+0000 7f51b2f0f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194005280 0x7f51ac1f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.144+0000 7f51b2f0f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac0cba60 0x7f51ac14e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.145+0000 7f51b2f0f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac1fb230 0x7f51ac163780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.145+0000 7f51b2f0f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1f8520 con 0x7f51ac0cba60 2024-05-08T23:21:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.145+0000 7f51b2f0f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1f7c80 con 0x7f51ac1fb230 2024-05-08T23:21:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.145+0000 7f51b2f0f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1f7f60 con 0x7f5194005280 2024-05-08T23:21:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.145+0000 7f51abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac0cba60 0x7f51ac14e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.141+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8408154520 con 0x7f83f0004850 2024-05-08T23:21:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.142+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8408151f90 con 0x7f83f0004850 2024-05-08T23:21:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.145+0000 7f51abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac0cba60 0x7f51ac14e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60884/0 (socket says 172.21.15.5:60884) 2024-05-08T23:21:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.145+0000 7f51abfff700 1 -- 172.21.15.5:0/2100419781 learned_addr learned my addr 172.21.15.5:0/2100419781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.146+0000 7f51b0cab700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194005280 0x7f51ac1f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.146+0000 7f51abfff700 1 -- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194005280 msgr2=0x7f51ac1f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.146+0000 7f51abfff700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194005280 0x7f51ac1f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.146+0000 7f51abfff700 1 -- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac1fb230 msgr2=0x7f51ac163780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:21:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.146+0000 7f51abfff700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac1fb230 0x7f51ac163780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.146+0000 7f51abfff700 1 -- 172.21.15.5:0/2100419781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac1f8240 con 0x7f51ac0cba60 2024-05-08T23:21:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.146+0000 7f51b0cab700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194005280 0x7f51ac1f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.147+0000 7f8404ff9700 1 -- 172.21.15.5:0/1691133398 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f83fc014b70 con 0x7f83f0004850 2024-05-08T23:21:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.147+0000 7f8404ff9700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83ec06ad60 0x7f83ec06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.147+0000 7f8404ff9700 1 -- 172.21.15.5:0/1691133398 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83fc089880 con 0x7f83f0004850 2024-05-08T23:21:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.148+0000 7f84077fe700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83ec06ad60 0x7f83ec06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.148+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e40052f0 con 0x7f83f0004850 2024-05-08T23:21:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.148+0000 7f84077fe700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83ec06ad60 0x7f83ec06d220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f84080aef50 tx=0x7f83f800b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.146+0000 7f51abfff700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac0cba60 0x7f51ac14e6d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f51a400bab0 tx=0x7f51a400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.167+0000 7f51a97fa700 1 -- 172.21.15.5:0/2100419781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a400ca70 con 0x7f51ac0cba60 2024-05-08T23:21:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.167+0000 7f51a97fa700 1 -- 172.21.15.5:0/2100419781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a400cbd0 con 0x7f51ac0cba60 2024-05-08T23:21:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.167+0000 7f51a97fa700 1 -- 172.21.15.5:0/2100419781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4014440 con 0x7f51ac0cba60 2024-05-08T23:21:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.167+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac1fad10 con 0x7f51ac0cba60 2024-05-08T23:21:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.167+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac14ec10 con 0x7f51ac0cba60 2024-05-08T23:21:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.169+0000 7f8404ff9700 1 -- 172.21.15.5:0/1691133398 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f83fc018080 con 0x7f83f0004850 2024-05-08T23:21:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.171+0000 7f51a97fa700 1 -- 172.21.15.5:0/2100419781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f51a40145a0 con 0x7f51ac0cba60 2024-05-08T23:21:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.172+0000 7f51a97fa700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f519006ad60 0x7f519006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.172+0000 7f51a97fa700 1 -- 172.21.15.5:0/2100419781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51a4088f90 con 0x7f51ac0cba60 2024-05-08T23:21:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.173+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac1fab70 con 0x7f51ac0cba60 2024-05-08T23:21:17.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.181+0000 7f51b0cab700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f519006ad60 0x7f519006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.182+0000 7f51a97fa700 1 -- 172.21.15.5:0/2100419781 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f51ac1fab70 con 0x7f51ac0cba60 2024-05-08T23:21:17.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.186+0000 7f51b0cab700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f519006ad60 0x7f519006d220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f519c001820 tx=0x7f519c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.194+0000 7fbec07f8700 1 Processor -- start 2024-05-08T23:21:17.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.195+0000 7fbec07f8700 1 -- start start 2024-05-08T23:21:17.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.193+0000 7fb6da673700 1 Processor -- start 2024-05-08T23:21:17.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.193+0000 7fb6da673700 1 -- start start 2024-05-08T23:21:17.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.195+0000 7fb6da673700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d4098a80 0x7fb6d406ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.195+0000 7fb6da673700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d406d4b0 con 0x7fb6d40b1510 2024-05-08T23:21:17.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.195+0000 7fb6da673700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d406d790 con 0x7fb6d406e4f0 2024-05-08T23:21:17.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.195+0000 7fb6da673700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d406da70 con 0x7fb6d4098a80 2024-05-08T23:21:17.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.195+0000 7fb6d3fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d4098a80 0x7fb6d406ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.195+0000 7fb6d3fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d4098a80 0x7fb6d406ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:48226/0 (socket says 172.21.15.5:48226) 2024-05-08T23:21:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.195+0000 7fb6d3fff700 1 -- 172.21.15.5:0/3883197133 learned_addr learned my addr 172.21.15.5:0/3883197133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.196+0000 7fb6d2ffd700 1 -- 172.21.15.5:0/3883197133 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840856012 0 0) 0x7fb6d406d4b0 con 0x7fb6d40b1510 2024-05-08T23:21:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.197+0000 7ff67002f700 1 Processor -- start 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.196+0000 7fbec07f8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbeb815aae0 0x7fbeb8153f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.196+0000 7fbec07f8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb80b1580 0x7fbeb81655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.196+0000 7fbec07f8700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb80ac1c0 con 0x7fbeb80b2be0 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.196+0000 7fbec07f8700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb806ccb0 con 0x7fbeb815aae0 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.196+0000 7fbec07f8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb806cf90 con 0x7fbeb80b1580 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.197+0000 7fbebdd93700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbeb80b2be0 0x7fbeb80b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44680/0 (socket says 172.21.15.5:44680) 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.197+0000 7fbebdd93700 1 -- 172.21.15.5:0/3705746065 learned_addr learned my addr 172.21.15.5:0/3705746065 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.197+0000 7fbebed95700 1 --2- 172.21.15.5:0/3705746065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb80b1580 0x7fbeb81655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7fbebe594700 1 --2- 172.21.15.5:0/3705746065 >> v2:172.21.15.5:3300/0 conn(0x7fbeb815aae0 0x7fbeb8153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7fbebed95700 1 -- 172.21.15.5:0/3705746065 >> v2:172.21.15.5:3300/0 conn(0x7fbeb815aae0 msgr2=0x7fbeb8153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7fbebed95700 1 --2- 172.21.15.5:0/3705746065 >> v2:172.21.15.5:3300/0 conn(0x7fbeb815aae0 0x7fbeb8153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7fbebed95700 1 -- 172.21.15.5:0/3705746065 >> v1:172.21.15.5:6789/0 conn(0x7fbeb80b2be0 legacy=0x7fbeb80b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7fbebed95700 1 -- 172.21.15.5:0/3705746065 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb806d270 con 0x7fbeb80b1580 2024-05-08T23:21:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7fb6d2ffd700 1 -- 172.21.15.5:0/3883197133 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6bc003590 con 0x7fb6d40b1510 2024-05-08T23:21:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7fb6d3fff700 1 -- 172.21.15.5:0/3883197133 >> v1:172.21.15.49:6789/0 conn(0x7fb6d406e4f0 legacy=0x7fb6d40b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7fb6d3fff700 1 -- 172.21.15.5:0/3883197133 >> v1:172.21.15.5:6789/0 conn(0x7fb6d40b1510 legacy=0x7fb6d40ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7fb6d3fff700 1 -- 172.21.15.5:0/3883197133 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d406dd50 con 0x7fb6d4098a80 2024-05-08T23:21:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.197+0000 7ff67002f700 1 -- start start 2024-05-08T23:21:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.198+0000 7ff67002f700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff66815f900 0x7ff66806ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7ff67002f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff66806b2c0 0x7ff6681621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7ff67002f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6680711d0 con 0x7ff668162710 2024-05-08T23:21:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7ff67002f700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66806b750 con 0x7ff66806b2c0 2024-05-08T23:21:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7ff67002f700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff668070f30 con 0x7ff66815f900 2024-05-08T23:21:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7ff66ddcb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff66815f900 0x7ff66806ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7ff66ddcb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff66815f900 0x7ff66806ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36492/0 (socket says 172.21.15.5:36492) 2024-05-08T23:21:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7ff66ddcb700 1 -- 172.21.15.5:0/283823765 learned_addr learned my addr 172.21.15.5:0/283823765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7fb6d3fff700 1 --2- 172.21.15.5:0/3883197133 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d4098a80 0x7fb6d406ce20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009920 tx=0x7fb6c4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9f11f8762ec1f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7fb6d2ffd700 1 -- 172.21.15.5:0/3883197133 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4004230 con 0x7fb6d4098a80 2024-05-08T23:21:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7fb6d2ffd700 1 -- 172.21.15.5:0/3883197133 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c4004390 con 0x7fb6d4098a80 2024-05-08T23:21:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7fb6d2ffd700 1 -- 172.21.15.5:0/3883197133 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40036a0 con 0x7fb6d4098a80 2024-05-08T23:21:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7fbebed95700 1 --2- 172.21.15.5:0/3705746065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb80b1580 0x7fbeb81655f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009840 tx=0x7fbeb0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1d1dbe9aa4bd59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7fbebd592700 1 -- 172.21.15.5:0/3705746065 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0011830 con 0x7fbeb80b1580 2024-05-08T23:21:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7fbebd592700 1 -- 172.21.15.5:0/3705746065 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb0011990 con 0x7fbeb80b1580 2024-05-08T23:21:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7fbebd592700 1 -- 172.21.15.5:0/3705746065 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0011d20 con 0x7fbeb80b1580 2024-05-08T23:21:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7fca7c228700 1 Processor -- start 2024-05-08T23:21:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fb6da673700 1 -- 172.21.15.5:0/3883197133 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d4098a80 msgr2=0x7fb6d406ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fb6da673700 1 --2- 172.21.15.5:0/3883197133 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d4098a80 0x7fb6d406ce20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009920 tx=0x7fb6c4009c30 comp rx=0 tx=0).stop 2024-05-08T23:21:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.199+0000 7ff66d5ca700 1 --2- 172.21.15.5:0/283823765 >> v2:172.21.15.5:3300/0 conn(0x7ff66806b2c0 0x7ff6681621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7ff66d5ca700 1 -- 172.21.15.5:0/283823765 >> v2:172.21.15.49:3300/0 conn(0x7ff66815f900 msgr2=0x7ff66806ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7ff66d5ca700 1 --2- 172.21.15.5:0/283823765 >> v2:172.21.15.49:3300/0 conn(0x7ff66815f900 0x7ff66806ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7ff66d5ca700 1 -- 172.21.15.5:0/283823765 >> v1:172.21.15.5:6789/0 conn(0x7ff668162710 legacy=0x7ff6680b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7ff66d5ca700 1 -- 172.21.15.5:0/283823765 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668162b80 con 0x7ff66806b2c0 2024-05-08T23:21:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.200+0000 7ff66d5ca700 1 --2- 172.21.15.5:0/283823765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff6681621d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff66806b8d0 tx=0x7ff664009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e3986732cad6e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fb6da673700 1 -- 172.21.15.5:0/3883197133 shutdown_connections 2024-05-08T23:21:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fb6da673700 1 --2- 172.21.15.5:0/3883197133 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d4098a80 0x7fb6d406ce20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fb6da673700 1 -- 172.21.15.5:0/3883197133 >> 172.21.15.5:0/3883197133 conn(0x7fb6d40b4690 msgr2=0x7fb6d40b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fbec07f8700 1 -- 172.21.15.5:0/3705746065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb80b1580 msgr2=0x7fbeb81655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fbec07f8700 1 --2- 172.21.15.5:0/3705746065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb80b1580 0x7fbeb81655f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009840 tx=0x7fbeb0009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fb6da673700 1 -- 172.21.15.5:0/3883197133 shutdown_connections 2024-05-08T23:21:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fbec07f8700 1 -- 172.21.15.5:0/3705746065 shutdown_connections 2024-05-08T23:21:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fbec07f8700 1 --2- 172.21.15.5:0/3705746065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb80b1580 0x7fbeb81655f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fbec07f8700 1 --2- 172.21.15.5:0/3705746065 >> v2:172.21.15.5:3300/0 conn(0x7fbeb815aae0 0x7fbeb8153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fbec07f8700 1 -- 172.21.15.5:0/3705746065 >> 172.21.15.5:0/3705746065 conn(0x7fbeb80b4690 msgr2=0x7fbeb80b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fca7c228700 1 -- start start 2024-05-08T23:21:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7fca7c228700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca740678f0 0x7fca74143110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7fca7c228700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fca7406e2f0 0x7fca74071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7fca7c228700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7406ebc0 con 0x7fca74070f80 2024-05-08T23:21:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7fca7c228700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca74071970 con 0x7fca740678f0 2024-05-08T23:21:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7fca7c228700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca74143650 con 0x7fca7406e2f0 2024-05-08T23:21:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7ff66cdc9700 1 -- 172.21.15.5:0/283823765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664011830 con 0x7ff66806b2c0 2024-05-08T23:21:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7ff66cdc9700 1 -- 172.21.15.5:0/283823765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff664011990 con 0x7ff66806b2c0 2024-05-08T23:21:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7ff66cdc9700 1 -- 172.21.15.5:0/283823765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664011d20 con 0x7ff66806b2c0 2024-05-08T23:21:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7ff67002f700 1 -- 172.21.15.5:0/283823765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 msgr2=0x7ff6681621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7ff67002f700 1 --2- 172.21.15.5:0/283823765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff6681621d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff66806b8d0 tx=0x7ff664009b20 comp rx=0 tx=0).stop 2024-05-08T23:21:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.202+0000 7fca79fc4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca740678f0 0x7fca74143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fca79fc4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca740678f0 0x7fca74143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60910/0 (socket says 172.21.15.5:60910) 2024-05-08T23:21:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fca79fc4700 1 -- 172.21.15.5:0/2681215114 learned_addr learned my addr 172.21.15.5:0/2681215114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fca7a7c5700 1 --2- 172.21.15.5:0/2681215114 >> v2:172.21.15.49:3300/0 conn(0x7fca7406e2f0 0x7fca74071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fca79fc4700 1 -- 172.21.15.5:0/2681215114 >> v2:172.21.15.49:3300/0 conn(0x7fca7406e2f0 msgr2=0x7fca74071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fca79fc4700 1 --2- 172.21.15.5:0/2681215114 >> v2:172.21.15.49:3300/0 conn(0x7fca7406e2f0 0x7fca74071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fca79fc4700 1 -- 172.21.15.5:0/2681215114 >> v1:172.21.15.49:6789/0 conn(0x7fca74070f80 legacy=0x7fca7406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fca79fc4700 1 -- 172.21.15.5:0/2681215114 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740b2880 con 0x7fca740678f0 2024-05-08T23:21:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.201+0000 7fb6da673700 1 -- 172.21.15.5:0/3883197133 wait complete. 2024-05-08T23:21:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fb6da673700 1 Processor -- start 2024-05-08T23:21:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.203+0000 7fbec07f8700 1 -- 172.21.15.5:0/3705746065 shutdown_connections 2024-05-08T23:21:17.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.204+0000 7fca79fc4700 1 --2- 172.21.15.5:0/2681215114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca740678f0 0x7fca74143110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fca740ae8b0 tx=0x7fca6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48061e8bf38fca2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.204+0000 7fbec07f8700 1 -- 172.21.15.5:0/3705746065 wait complete. 2024-05-08T23:21:17.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.204+0000 7fca78fc2700 1 -- 172.21.15.5:0/2681215114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00e070 con 0x7fca740678f0 2024-05-08T23:21:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.204+0000 7fca78fc2700 1 -- 172.21.15.5:0/2681215114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c00b9c0 con 0x7fca740678f0 2024-05-08T23:21:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.204+0000 7fca78fc2700 1 -- 172.21.15.5:0/2681215114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00bd70 con 0x7fca740678f0 2024-05-08T23:21:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fca7c228700 1 -- 172.21.15.5:0/2681215114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca740678f0 msgr2=0x7fca74143110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fca7c228700 1 --2- 172.21.15.5:0/2681215114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca740678f0 0x7fca74143110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fca740ae8b0 tx=0x7fca6c00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fbec07f8700 1 Processor -- start 2024-05-08T23:21:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fbec07f8700 1 -- start start 2024-05-08T23:21:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.204+0000 7fb6da673700 1 -- start start 2024-05-08T23:21:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fb6da673700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d406e4f0 0x7fb6d4207e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fb6da673700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d4098a80 0x7fb6d40ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fb6da673700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6d40b1510 0x7fb6d40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fb6da673700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40ad5f0 con 0x7fb6d40b1510 2024-05-08T23:21:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fb6da673700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40cf630 con 0x7fb6d4098a80 2024-05-08T23:21:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fb6da673700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40cf910 con 0x7fb6d406e4f0 2024-05-08T23:21:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d4098a80 0x7fb6d40ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d4098a80 0x7fb6d40ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36502/0 (socket says 172.21.15.5:36502) 2024-05-08T23:21:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 -- 172.21.15.5:0/534336418 learned_addr learned my addr 172.21.15.5:0/534336418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d8c10700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6d40b1510 0x7fb6d40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d3fff700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d406e4f0 0x7fb6d4207e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7ff67002f700 1 -- 172.21.15.5:0/283823765 shutdown_connections 2024-05-08T23:21:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7ff67002f700 1 --2- 172.21.15.5:0/283823765 >> v2:172.21.15.49:3300/0 conn(0x7ff66815f900 0x7ff66806ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7ff67002f700 1 --2- 172.21.15.5:0/283823765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff6681621d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7ff67002f700 1 -- 172.21.15.5:0/283823765 >> 172.21.15.5:0/283823765 conn(0x7ff6680b1290 msgr2=0x7ff6680b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 -- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d406e4f0 msgr2=0x7fb6d4207e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d406e4f0 0x7fb6d4207e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 -- 172.21.15.5:0/534336418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6d40b1510 msgr2=0x7fb6d40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6d40b1510 0x7fb6d40cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 -- 172.21.15.5:0/534336418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c4009550 con 0x7fb6d4098a80 2024-05-08T23:21:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.206+0000 7fb6d37fe700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d4098a80 0x7fb6d40ce8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800c4a0 tx=0x7fb6c800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fca7c228700 1 -- 172.21.15.5:0/2681215114 shutdown_connections 2024-05-08T23:21:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fca7c228700 1 --2- 172.21.15.5:0/2681215114 >> v2:172.21.15.49:3300/0 conn(0x7fca7406e2f0 0x7fca74071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fca7c228700 1 --2- 172.21.15.5:0/2681215114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca740678f0 0x7fca74143110 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fca740ae8b0 tx=0x7fca6c00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.205+0000 7fca7c228700 1 -- 172.21.15.5:0/2681215114 >> 172.21.15.5:0/2681215114 conn(0x7fca7406af90 msgr2=0x7fca740b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbec07f8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb80b1580 0x7fbeb814b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbec07f8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb80b2be0 0x7fbeb814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbec07f8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb815aae0 0x7fbeb80afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbec07f8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8155470 con 0x7fbeb80b2be0 2024-05-08T23:21:17.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbec07f8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb814bc50 con 0x7fbeb80b1580 2024-05-08T23:21:17.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbec07f8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8154860 con 0x7fbeb815aae0 2024-05-08T23:21:17.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fca7c228700 1 -- 172.21.15.5:0/2681215114 shutdown_connections 2024-05-08T23:21:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbebdd93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb80b2be0 0x7fbeb814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbebdd93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb80b2be0 0x7fbeb814e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60926/0 (socket says 172.21.15.5:60926) 2024-05-08T23:21:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbebdd93700 1 -- 172.21.15.5:0/323165319 learned_addr learned my addr 172.21.15.5:0/323165319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbebed95700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb815aae0 0x7fbeb80afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fbebe594700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb80b1580 0x7fbeb814b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.208+0000 7fbebe594700 1 -- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb815aae0 msgr2=0x7fbeb80afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.208+0000 7fbebe594700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb815aae0 0x7fbeb80afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.208+0000 7fbebe594700 1 -- 172.21.15.5:0/323165319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb80b2be0 msgr2=0x7fbeb814e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.208+0000 7fbebe594700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb80b2be0 0x7fbeb814e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.208+0000 7fbebe594700 1 -- 172.21.15.5:0/323165319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb00094a0 con 0x7fbeb80b1580 2024-05-08T23:21:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.208+0000 7fbebe594700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb80b1580 0x7fbeb814b710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbeac007e60 tx=0x7fbeac00e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7ff67002f700 1 -- 172.21.15.5:0/283823765 shutdown_connections 2024-05-08T23:21:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7ff67002f700 1 -- 172.21.15.5:0/283823765 wait complete. 2024-05-08T23:21:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7ff67002f700 1 Processor -- start 2024-05-08T23:21:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7fb6d0ff9700 1 -- 172.21.15.5:0/534336418 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80054e0 con 0x7fb6d4098a80 2024-05-08T23:21:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7fb6d0ff9700 1 -- 172.21.15.5:0/534336418 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8003680 con 0x7fb6d4098a80 2024-05-08T23:21:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7fb6d0ff9700 1 -- 172.21.15.5:0/534336418 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8003a30 con 0x7fb6d4098a80 2024-05-08T23:21:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.207+0000 7fca7c228700 1 -- 172.21.15.5:0/2681215114 wait complete. 2024-05-08T23:21:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7fca7c228700 1 Processor -- start 2024-05-08T23:21:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d40acfa0 con 0x7fb6d4098a80 2024-05-08T23:21:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d40cf340 con 0x7fb6d4098a80 2024-05-08T23:21:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fca7c228700 1 -- start start 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca7c228700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca7406e2f0 0x7fca741416c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca7c228700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fca74070f80 0x7fca7413fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca7c228700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca740a8970 0x7fca740ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca7c228700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca74141230 con 0x7fca7406e2f0 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca7c228700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca74140620 con 0x7fca740a8970 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca7c228700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca74140900 con 0x7fca74070f80 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca79fc4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca7406e2f0 0x7fca741416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca79fc4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca7406e2f0 0x7fca741416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60956/0 (socket says 172.21.15.5:60956) 2024-05-08T23:21:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.211+0000 7fca79fc4700 1 -- 172.21.15.5:0/1094547153 learned_addr learned my addr 172.21.15.5:0/1094547153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d40678f0 con 0x7fb6d4098a80 2024-05-08T23:21:17.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7ff67002f700 1 -- start start 2024-05-08T23:21:17.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7fca7a7c5700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca740a8970 0x7fca740ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff67002f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff668161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff67002f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff66815f900 0x7ff66814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff67002f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff668162710 0x7ff668097770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff67002f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff668160d40 con 0x7ff66806b2c0 2024-05-08T23:21:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff67002f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6681600c0 con 0x7ff668162710 2024-05-08T23:21:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff67002f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6681603a0 con 0x7ff66815f900 2024-05-08T23:21:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66ddcb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff668161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66ddcb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff668161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60958/0 (socket says 172.21.15.5:60958) 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66ddcb700 1 -- 172.21.15.5:0/3433793058 learned_addr learned my addr 172.21.15.5:0/3433793058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66d5ca700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff66815f900 0x7ff66814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66ddcb700 1 -- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff66815f900 msgr2=0x7ff66814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66e5cc700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff668162710 0x7ff668097770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66ddcb700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff66815f900 0x7ff66814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66ddcb700 1 -- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff668162710 msgr2=0x7ff668097770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66ddcb700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff668162710 0x7ff668097770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66ddcb700 1 -- 172.21.15.5:0/3433793058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640094a0 con 0x7ff66806b2c0 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7ff66d5ca700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff66815f900 0x7ff66814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7ff66e5cc700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff668162710 0x7ff668097770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7ff66ddcb700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff668161160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff66000efc0 tx=0x7ff66000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fbeb6ffd700 1 -- 172.21.15.5:0/323165319 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac00ede0 con 0x7fbeb80b1580 2024-05-08T23:21:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fbeb6ffd700 1 -- 172.21.15.5:0/323165319 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac014420 con 0x7fbeb80b1580 2024-05-08T23:21:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb8154b40 con 0x7fbeb80b1580 2024-05-08T23:21:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb8155100 con 0x7fbeb80b1580 2024-05-08T23:21:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7fb6d0ff9700 1 -- 172.21.15.5:0/534336418 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb6c8003b90 con 0x7fb6d4098a80 2024-05-08T23:21:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fb6d0ff9700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb6c006ac90 0x7fb6c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fb6d0ff9700 1 -- 172.21.15.5:0/534336418 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6c8088340 con 0x7fb6d4098a80 2024-05-08T23:21:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7ff65e7fc700 1 -- 172.21.15.5:0/3433793058 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000cbb0 con 0x7ff66806b2c0 2024-05-08T23:21:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7ff65e7fc700 1 -- 172.21.15.5:0/3433793058 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66000f040 con 0x7ff66806b2c0 2024-05-08T23:21:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7ff65e7fc700 1 -- 172.21.15.5:0/3433793058 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660014480 con 0x7ff66806b2c0 2024-05-08T23:21:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.214+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff668160680 con 0x7ff66806b2c0 2024-05-08T23:21:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.214+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66815fce0 con 0x7ff66806b2c0 2024-05-08T23:21:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fbeb6ffd700 1 -- 172.21.15.5:0/323165319 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac014790 con 0x7fbeb80b1580 2024-05-08T23:21:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.215+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6680678f0 con 0x7ff66806b2c0 2024-05-08T23:21:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7fca79fc4700 1 -- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fca74070f80 msgr2=0x7fca7413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7fca79fc4700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fca74070f80 0x7fca7413fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7fca79fc4700 1 -- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca740a8970 msgr2=0x7fca740ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7fca79fc4700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca740a8970 0x7fca740ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.212+0000 7fca79fc4700 1 -- 172.21.15.5:0/1094547153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c009b30 con 0x7fca7406e2f0 2024-05-08T23:21:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fca79fc4700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca7406e2f0 0x7fca741416c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fca6c012000 tx=0x7fca6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fca6a7fc700 1 -- 172.21.15.5:0/1094547153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0099b0 con 0x7fca7406e2f0 2024-05-08T23:21:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fca6a7fc700 1 -- 172.21.15.5:0/1094547153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c004250 con 0x7fca7406e2f0 2024-05-08T23:21:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.213+0000 7fca6a7fc700 1 -- 172.21.15.5:0/1094547153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c004600 con 0x7fca7406e2f0 2024-05-08T23:21:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.216+0000 7fb6d3fff700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb6c006ac90 0x7fb6c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.216+0000 7fb6d3fff700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb6c006ac90 0x7fb6c006d150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009fd0 tx=0x7fb6c4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.217+0000 7ff65e7fc700 1 -- 172.21.15.5:0/3433793058 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff6600145e0 con 0x7ff66806b2c0 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7fe473fff700 1 Processor -- start 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.209+0000 7fe473fff700 1 -- start start 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fe473fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe474072600 0x7fe4740b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fe473fff700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4740cea50 con 0x7fe4740c5b30 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fe473fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4740ae740 con 0x7fe47406e380 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fe473fff700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4740c6080 con 0x7fe474072600 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fe472ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe47406e380 0x7fe4740c40d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60078/0 (socket says 172.21.15.5:60078) 2024-05-08T23:21:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fe472ffd700 1 -- 172.21.15.5:0/1899134026 learned_addr learned my addr 172.21.15.5:0/1899134026 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414108849 0 0) 0x7fe4740ae740 con 0x7fe47406e380 2024-05-08T23:21:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.218+0000 7fb6d0ff9700 1 -- 172.21.15.5:0/534336418 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb6c8053510 con 0x7fb6d4098a80 2024-05-08T23:21:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.217+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea00052f0 con 0x7fbeb80b1580 2024-05-08T23:21:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.218+0000 7ff65e7fc700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff65406ae70 0x7ff65406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.218+0000 7ff65e7fc700 1 -- 172.21.15.5:0/3433793058 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff660089990 con 0x7ff66806b2c0 2024-05-08T23:21:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.219+0000 7ff66d5ca700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff65406ae70 0x7ff65406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.222+0000 7ff66d5ca700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff65406ae70 0x7ff65406d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff668072130 tx=0x7ff664008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca74140be0 con 0x7fca7406e2f0 2024-05-08T23:21:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca741401f0 con 0x7fca7406e2f0 2024-05-08T23:21:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.222+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca74068000 con 0x7fca7406e2f0 2024-05-08T23:21:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.223+0000 7fbfc9c1a700 1 Processor -- start 2024-05-08T23:21:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.222+0000 7fbeb6ffd700 1 -- 172.21.15.5:0/323165319 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbeac0148f0 con 0x7fbeb80b1580 2024-05-08T23:21:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.223+0000 7fbeb6ffd700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9c06ad60 0x7fbe9c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.223+0000 7fbeb6ffd700 1 -- 172.21.15.5:0/323165319 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeac08a440 con 0x7fbeb80b1580 2024-05-08T23:21:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.223+0000 7fbebdd93700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9c06ad60 0x7fbe9c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.217+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe45c003a00 con 0x7fe47406e380 2024-05-08T23:21:17.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.217+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174450196 0 0) 0x7fe4740cea50 con 0x7fe4740c5b30 2024-05-08T23:21:17.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.218+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4740ae740 con 0x7fe4740c5b30 2024-05-08T23:21:17.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.210+0000 7fe4737fe700 1 --2- 172.21.15.5:0/1899134026 >> v2:172.21.15.5:3300/0 conn(0x7fe474072600 0x7fe4740b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.219+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 <== mon.1 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1418524148 0 0) 0x7fe45c003a00 con 0x7fe47406e380 2024-05-08T23:21:17.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.219+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 >> v2:172.21.15.5:3300/0 conn(0x7fe474072600 msgr2=0x7fe4740b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.219+0000 7fe471ffb700 1 --2- 172.21.15.5:0/1899134026 >> v2:172.21.15.5:3300/0 conn(0x7fe474072600 0x7fe4740b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 >> v1:172.21.15.5:6789/0 conn(0x7fe4740c5b30 legacy=0x7fe474071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740cecc0 con 0x7fe47406e380 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 <== mon.1 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fe464001670 con 0x7fe47406e380 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 >> v1:172.21.15.49:6789/0 conn(0x7fe47406e380 legacy=0x7fe4740c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c004ca0 0x7fe45c0050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0056d0 0x7fe45c003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe45c0042a0 0x7fe45c009e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740c6080 con 0x7fe45c004ca0 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c004a50 con 0x7fe45c0056d0 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c00a390 con 0x7fe45c0042a0 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe4727fc700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0056d0 0x7fe45c003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.220+0000 7fe472ffd700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c004ca0 0x7fe45c0050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fe4737fe700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe45c0042a0 0x7fe45c009e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fe4727fc700 1 -- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe45c0042a0 msgr2=0x7fe45c009e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fe4727fc700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe45c0042a0 0x7fe45c009e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fe4727fc700 1 -- 172.21.15.5:0/1899134026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c004ca0 msgr2=0x7fe45c0050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fe4727fc700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c004ca0 0x7fe45c0050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fe4727fc700 1 -- 172.21.15.5:0/1899134026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740cecc0 con 0x7fe45c0056d0 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.221+0000 7fe4737fe700 1 -- 172.21.15.5:0/1899134026 reap_dead start 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.222+0000 7fe4727fc700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0056d0 0x7fe45c003c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe468001930 tx=0x7fe45c005cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c226d4f07e33f6a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.223+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4680052d0 con 0x7fe45c0056d0 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.224+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe468005430 con 0x7fe45c0056d0 2024-05-08T23:21:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.224+0000 7fe471ffb700 1 -- 172.21.15.5:0/1899134026 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4680038a0 con 0x7fe45c0056d0 2024-05-08T23:21:17.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.224+0000 7fbeb6ffd700 1 -- 172.21.15.5:0/323165319 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbeac055690 con 0x7fbeb80b1580 2024-05-08T23:21:17.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.223+0000 7fbfc9c1a700 1 -- start start 2024-05-08T23:21:17.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fe473fff700 1 -- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0056d0 msgr2=0x7fe45c003c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fe473fff700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0056d0 0x7fe45c003c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe468001930 tx=0x7fe45c005cd0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7ff65e7fc700 1 -- 172.21.15.5:0/3433793058 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff660054c10 con 0x7ff66806b2c0 2024-05-08T23:21:17.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.224+0000 7fbfc9c1a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbfc41597a0 0x7fbfc40cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.224+0000 7fbfc9c1a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc40b32a0 con 0x7fbfc40b0ee0 2024-05-08T23:21:17.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.224+0000 7fbfc9c1a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc40d0040 con 0x7fbfc415da50 2024-05-08T23:21:17.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.224+0000 7fbfc9c1a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc4159b80 con 0x7fbfc41597a0 2024-05-08T23:21:17.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fbfc3fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbfc40b0ee0 0x7fbfc415d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44718/0 (socket says 172.21.15.5:44718) 2024-05-08T23:21:17.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fbfc3fff700 1 -- 172.21.15.5:0/2407365700 learned_addr learned my addr 172.21.15.5:0/2407365700 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:17.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fbebdd93700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9c06ad60 0x7fbe9c06d220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbeb809d470 tx=0x7fbea40017b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fe473fff700 1 -- 172.21.15.5:0/1899134026 shutdown_connections 2024-05-08T23:21:17.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fe473fff700 1 --2- 172.21.15.5:0/1899134026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0056d0 0x7fe45c003c40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fe473fff700 1 -- 172.21.15.5:0/1899134026 >> 172.21.15.5:0/1899134026 conn(0x7fe4740b0d60 msgr2=0x7fe4740b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fe473fff700 1 -- 172.21.15.5:0/1899134026 shutdown_connections 2024-05-08T23:21:17.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.225+0000 7fe473fff700 1 -- 172.21.15.5:0/1899134026 wait complete. 2024-05-08T23:21:17.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.228+0000 7fe473fff700 1 Processor -- start 2024-05-08T23:21:17.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.227+0000 7fbfc8c18700 1 --2- 172.21.15.5:0/2407365700 >> v2:172.21.15.5:3300/0 conn(0x7fbfc41597a0 0x7fbfc40cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.227+0000 7fbfc8c18700 1 -- 172.21.15.5:0/2407365700 >> v1:172.21.15.49:6789/0 conn(0x7fbfc415da50 legacy=0x7fbfc40b1330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:17.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.227+0000 7fbfc8c18700 1 -- 172.21.15.5:0/2407365700 >> v1:172.21.15.5:6789/0 conn(0x7fbfc40b0ee0 legacy=0x7fbfc415d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.227+0000 7fbfc8c18700 1 -- 172.21.15.5:0/2407365700 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40b2fc0 con 0x7fbfc41597a0 2024-05-08T23:21:17.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.228+0000 7fbfc8c18700 1 --2- 172.21.15.5:0/2407365700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc41597a0 0x7fbfc40cfb00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009870 tx=0x7fbfb4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beebceb9db0730f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.228+0000 7fca6a7fc700 1 -- 172.21.15.5:0/1094547153 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fca6c004760 con 0x7fca7406e2f0 2024-05-08T23:21:17.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.229+0000 7fca6a7fc700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fca5806ad60 0x7fca5806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.229+0000 7fca6a7fc700 1 -- 172.21.15.5:0/1094547153 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca6c08ac60 con 0x7fca7406e2f0 2024-05-08T23:21:17.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.228+0000 7fe473fff700 1 -- start start 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.229+0000 7fe473fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c0056d0 0x7fe47408c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.229+0000 7fe473fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4740ad090 0x7fe4740ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.229+0000 7fe473fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe474203200 0x7fe4742036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.229+0000 7fe473fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740ada50 con 0x7fe45c0056d0 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.229+0000 7fe473fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b1e50 con 0x7fe4740ad090 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.229+0000 7fe473fff700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47408cf00 con 0x7fe474203200 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.230+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c0056d0 0x7fe47408c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.230+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c0056d0 0x7fe47408c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60986/0 (socket says 172.21.15.5:60986) 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.230+0000 7fe472ffd700 1 -- 172.21.15.5:0/2305642991 learned_addr learned my addr 172.21.15.5:0/2305642991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.231+0000 7fe472ffd700 1 -- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe474203200 msgr2=0x7fe4742036b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.231+0000 7fe472ffd700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe474203200 0x7fe4742036b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.231+0000 7fe472ffd700 1 -- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4740ad090 msgr2=0x7fe4740ad510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:21:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.231+0000 7fe472ffd700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4740ad090 0x7fe4740ad510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.231+0000 7fe472ffd700 1 -- 172.21.15.5:0/2305642991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740b20a0 con 0x7fe45c0056d0 2024-05-08T23:21:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.231+0000 7fe472ffd700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c0056d0 0x7fe47408c9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe464004fb0 tx=0x7fe4640073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.230+0000 7fca797c3700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fca5806ad60 0x7fca5806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.231+0000 7fca797c3700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fca5806ad60 0x7fca5806d220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca64000f60 tx=0x7fca64007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.235+0000 7fe45b7fe700 1 -- 172.21.15.5:0/2305642991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464033590 con 0x7fe45c0056d0 2024-05-08T23:21:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.235+0000 7fe45b7fe700 1 -- 172.21.15.5:0/2305642991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4640336f0 con 0x7fe45c0056d0 2024-05-08T23:21:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.235+0000 7fe45b7fe700 1 -- 172.21.15.5:0/2305642991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464033a80 con 0x7fe45c0056d0 2024-05-08T23:21:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.235+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4740b2380 con 0x7fe45c0056d0 2024-05-08T23:21:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.235+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4740b2940 con 0x7fe45c0056d0 2024-05-08T23:21:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.234+0000 7fca6a7fc700 1 -- 172.21.15.5:0/1094547153 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fca6c055e20 con 0x7fca7406e2f0 2024-05-08T23:21:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.234+0000 7fbfc37fe700 1 -- 172.21.15.5:0/2407365700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4004230 con 0x7fbfc41597a0 2024-05-08T23:21:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.234+0000 7fbfc37fe700 1 -- 172.21.15.5:0/2407365700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb4004390 con 0x7fbfc41597a0 2024-05-08T23:21:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.234+0000 7fbfc37fe700 1 -- 172.21.15.5:0/2407365700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40036a0 con 0x7fbfc41597a0 2024-05-08T23:21:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.235+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/2407365700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc41597a0 msgr2=0x7fbfc40cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.235+0000 7fbfc9c1a700 1 --2- 172.21.15.5:0/2407365700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc41597a0 0x7fbfc40cfb00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009870 tx=0x7fbfb4009b80 comp rx=0 tx=0).stop 2024-05-08T23:21:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.236+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/2407365700 shutdown_connections 2024-05-08T23:21:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.236+0000 7fbfc9c1a700 1 --2- 172.21.15.5:0/2407365700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc41597a0 0x7fbfc40cfb00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.236+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/2407365700 >> 172.21.15.5:0/2407365700 conn(0x7fbfc40b4700 msgr2=0x7fbfc40b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.236+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/2407365700 shutdown_connections 2024-05-08T23:21:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.236+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4740b2b60 con 0x7fe45c0056d0 2024-05-08T23:21:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.239+0000 7fe45b7fe700 1 -- 172.21.15.5:0/2305642991 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe464033be0 con 0x7fe45c0056d0 2024-05-08T23:21:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.239+0000 7fe45b7fe700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe46006ad60 0x7fe46006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.239+0000 7fe45b7fe700 1 -- 172.21.15.5:0/2305642991 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4640ab7d0 con 0x7fe45c0056d0 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.236+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/2407365700 wait complete. 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.238+0000 7fbfc9c1a700 1 Processor -- start 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.239+0000 7fbfc9c1a700 1 -- start start 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.240+0000 7fbfc9c1a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfc40b0ee0 0x7fbfc4098aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.240+0000 7fbfc9c1a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfc41597a0 0x7fbfc40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc9c1a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc415da50 0x7fbfc40cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc9c1a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40cd200 con 0x7fbfc415da50 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc9c1a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40ccd70 con 0x7fbfc40b0ee0 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc9c1a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40cc460 con 0x7fbfc41597a0 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfc41597a0 0x7fbfc40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc9419700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc415da50 0x7fbfc40cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc9419700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc415da50 0x7fbfc40cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60998/0 (socket says 172.21.15.5:60998) 2024-05-08T23:21:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc9419700 1 -- 172.21.15.5:0/3937880460 learned_addr learned my addr 172.21.15.5:0/3937880460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.241+0000 7fbfc8c18700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfc40b0ee0 0x7fbfc4098aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.242+0000 7fbfc9419700 1 -- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfc41597a0 msgr2=0x7fbfc40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.242+0000 7fbfc9419700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfc41597a0 0x7fbfc40cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.242+0000 7fbfc9419700 1 -- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfc40b0ee0 msgr2=0x7fbfc4098aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.242+0000 7fbfc9419700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfc40b0ee0 0x7fbfc4098aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.242+0000 7fbfc9419700 1 -- 172.21.15.5:0/3937880460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40094a0 con 0x7fbfc415da50 2024-05-08T23:21:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.242+0000 7fbfc9419700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc415da50 0x7fbfc40cbf20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b990 tx=0x7fbfbc00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.247+0000 7fe45b7fe700 1 -- 172.21.15.5:0/2305642991 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe464076a40 con 0x7fe45c0056d0 2024-05-08T23:21:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.247+0000 7fbfc17fa700 1 -- 172.21.15.5:0/3937880460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00d890 con 0x7fbfc415da50 2024-05-08T23:21:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.247+0000 7fbfc17fa700 1 -- 172.21.15.5:0/3937880460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc00d9f0 con 0x7fbfc415da50 2024-05-08T23:21:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.247+0000 7fbfc17fa700 1 -- 172.21.15.5:0/3937880460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00dda0 con 0x7fbfc415da50 2024-05-08T23:21:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.247+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc40cd3e0 con 0x7fbfc415da50 2024-05-08T23:21:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.247+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc40cca00 con 0x7fbfc415da50 2024-05-08T23:21:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.248+0000 7fe4727fc700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe46006ad60 0x7fe46006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.252+0000 7fbfc17fa700 1 -- 172.21.15.5:0/3937880460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbfbc006d50 con 0x7fbfc415da50 2024-05-08T23:21:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.253+0000 7fbfc17fa700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbfac06af50 0x7fbfac06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.253+0000 7fbfc17fa700 1 -- 172.21.15.5:0/3937880460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfbc089370 con 0x7fbfc415da50 2024-05-08T23:21:17.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.254+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa40052f0 con 0x7fbfc415da50 2024-05-08T23:21:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.265+0000 7fbfc17fa700 1 -- 172.21.15.5:0/3937880460 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbfbc054530 con 0x7fbfc415da50 2024-05-08T23:21:17.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.270+0000 7fbfc8c18700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbfac06af50 0x7fbfac06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.281+0000 7fe4727fc700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe46006ad60 0x7fe46006d220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe474202f90 tx=0x7fe468009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.281+0000 7fbfc8c18700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbfac06af50 0x7fbfac06d410 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009fd0 tx=0x7fbfb401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.628+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f51ac1fab70 con 0x7f51ac0cba60 2024-05-08T23:21:17.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.628+0000 7f51a97fa700 1 -- 172.21.15.5:0/2100419781 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51ac1fab70 con 0x7f51ac0cba60 2024-05-08T23:21:17.629 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:17.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.629+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f83e40055a0 con 0x7f83f0004850 2024-05-08T23:21:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.630+0000 7f8404ff9700 1 -- 172.21.15.5:0/1691133398 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f83fc058250 con 0x7f83f0004850 2024-05-08T23:21:17.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.632+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f519006ad60 msgr2=0x7f519006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.632+0000 7f51b2f0f700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f519006ad60 0x7f519006d220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f519c001820 tx=0x7f519c009040 comp rx=0 tx=0).stop 2024-05-08T23:21:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.632+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac0cba60 msgr2=0x7f51ac14e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.632+0000 7f51b2f0f700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac0cba60 0x7f51ac14e6d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f51a400bab0 tx=0x7f51a400be70 comp rx=0 tx=0).stop 2024-05-08T23:21:17.634 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 shutdown_connections 2024-05-08T23:21:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f51b2f0f700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f519006ad60 0x7f519006d220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f51b2f0f700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51ac1fb230 0x7f51ac163780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f51b2f0f700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5194005280 0x7f51ac1f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f51b2f0f700 1 --2- 172.21.15.5:0/2100419781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51ac0cba60 0x7f51ac14e6d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 >> 172.21.15.5:0/2100419781 conn(0x7f51ac0b1290 msgr2=0x7f51ac15fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 shutdown_connections 2024-05-08T23:21:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f51b2f0f700 1 -- 172.21.15.5:0/2100419781 wait complete. 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83ec06ad60 msgr2=0x7f83ec06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f840e83a700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83ec06ad60 0x7f83ec06d220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f84080aef50 tx=0x7f83f800b000 comp rx=0 tx=0).stop 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0004850 msgr2=0x7f84081533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.634+0000 7f840e83a700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0004850 0x7f84081533f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00f840 tx=0x7f83fc00fc00 comp rx=0 tx=0).stop 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.635+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 shutdown_connections 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.635+0000 7f840e83a700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83ec06ad60 0x7f83ec06d220 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.635+0000 7f840e83a700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8408145a50 0x7f8408151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.635+0000 7f840e83a700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83f0004850 0x7f84081533f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.635+0000 7f840e83a700 1 --2- 172.21.15.5:0/1691133398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84081455d0 0x7f8408151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.635+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 >> 172.21.15.5:0/1691133398 conn(0x7f84080b4690 msgr2=0x7f84080b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.635+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 shutdown_connections 2024-05-08T23:21:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.635+0000 7f840e83a700 1 -- 172.21.15.5:0/1691133398 wait complete. 2024-05-08T23:21:17.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-05-08T23:21:17.658 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:17.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-08T23:21:17.661 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.666+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb6d40b1a30 con 0x7fb6d4098a80 2024-05-08T23:21:17.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.666+0000 7fb6d0ff9700 1 -- 172.21.15.5:0/534336418 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6c8056b30 con 0x7fb6d4098a80 2024-05-08T23:21:17.666 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-08T23:21:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.670+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb6c006ac90 msgr2=0x7fb6c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.670+0000 7fb6da673700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb6c006ac90 0x7fb6c006d150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009fd0 tx=0x7fb6c4008030 comp rx=0 tx=0).stop 2024-05-08T23:21:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.670+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d4098a80 msgr2=0x7fb6d40ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.670+0000 7fb6da673700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d4098a80 0x7fb6d40ce8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800c4a0 tx=0x7fb6c800c7b0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.671+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 shutdown_connections 2024-05-08T23:21:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.671+0000 7fb6da673700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb6c006ac90 0x7fb6c006d150 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.671+0000 7fb6da673700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6d40b1510 0x7fb6d40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.671+0000 7fb6da673700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6d406e4f0 0x7fb6d4207e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.671+0000 7fb6da673700 1 --2- 172.21.15.5:0/534336418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d4098a80 0x7fb6d40ce8c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.671+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 >> 172.21.15.5:0/534336418 conn(0x7fb6d40b4690 msgr2=0x7fb6d416d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.673+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbea0005870 con 0x7fbeb80b1580 2024-05-08T23:21:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.674+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 shutdown_connections 2024-05-08T23:21:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.674+0000 7fb6da673700 1 -- 172.21.15.5:0/534336418 wait complete. 2024-05-08T23:21:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.674+0000 7fbeb6ffd700 1 -- 172.21.15.5:0/323165319 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbeac058cb0 con 0x7fbeb80b1580 2024-05-08T23:21:17.674 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.677+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff668071e80 con 0x7ff66806b2c0 2024-05-08T23:21:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.678+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9c06ad60 msgr2=0x7fbe9c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.678+0000 7fbec07f8700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9c06ad60 0x7fbe9c06d220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbeb809d470 tx=0x7fbea40017b0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.678+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb80b1580 msgr2=0x7fbeb814b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.678+0000 7fbec07f8700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb80b1580 0x7fbeb814b710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbeac007e60 tx=0x7fbeac00e3c0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.678+0000 7ff65e7fc700 1 -- 172.21.15.5:0/3433793058 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff660058230 con 0x7ff66806b2c0 2024-05-08T23:21:17.679 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:17.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.680+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fca740b4730 con 0x7fca7406e2f0 2024-05-08T23:21:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.681+0000 7fca6a7fc700 1 -- 172.21.15.5:0/1094547153 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca6c02a090 con 0x7fca7406e2f0 2024-05-08T23:21:17.681 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-08T23:21:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff65406ae70 msgr2=0x7ff65406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff65406ae70 0x7ff65406d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff668072130 tx=0x7ff664008030 comp rx=0 tx=0).stop 2024-05-08T23:21:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 msgr2=0x7ff668161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff668161160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff66000efc0 tx=0x7ff66000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 shutdown_connections 2024-05-08T23:21:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7fbec07f8700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe9c06ad60 0x7fbe9c06d220 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7fbec07f8700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb815aae0 0x7fbeb80afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7fbec07f8700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb80b1580 0x7fbeb814b710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7fbec07f8700 1 --2- 172.21.15.5:0/323165319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb80b2be0 0x7fbeb814e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 >> 172.21.15.5:0/323165319 conn(0x7fbeb80b4690 msgr2=0x7fbeb80ac9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 shutdown_connections 2024-05-08T23:21:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7fbec07f8700 1 -- 172.21.15.5:0/323165319 wait complete. 2024-05-08T23:21:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 shutdown_connections 2024-05-08T23:21:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff65406ae70 0x7ff65406d330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff668162710 0x7ff668097770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66806b2c0 0x7ff668161160 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 --2- 172.21.15.5:0/3433793058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff66815f900 0x7ff66814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 >> 172.21.15.5:0/3433793058 conn(0x7ff6680b1290 msgr2=0x7ff668072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 shutdown_connections 2024-05-08T23:21:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.682+0000 7ff67002f700 1 -- 172.21.15.5:0/3433793058 wait complete. 2024-05-08T23:21:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.684+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fca5806ad60 msgr2=0x7fca5806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.684+0000 7fca7c228700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fca5806ad60 0x7fca5806d220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca64000f60 tx=0x7fca64007a40 comp rx=0 tx=0).stop 2024-05-08T23:21:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.684+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca7406e2f0 msgr2=0x7fca741416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.684+0000 7fca7c228700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca7406e2f0 0x7fca741416c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fca6c012000 tx=0x7fca6c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.688+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 shutdown_connections 2024-05-08T23:21:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.688+0000 7fca7c228700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fca5806ad60 0x7fca5806d220 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.688+0000 7fca7c228700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca740a8970 0x7fca740ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.688+0000 7fca7c228700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca7406e2f0 0x7fca741416c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.688+0000 7fca7c228700 1 --2- 172.21.15.5:0/1094547153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fca74070f80 0x7fca7413fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.688+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 >> 172.21.15.5:0/1094547153 conn(0x7fca7406af90 msgr2=0x7fca740ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.688+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 shutdown_connections 2024-05-08T23:21:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.688+0000 7fca7c228700 1 -- 172.21.15.5:0/1094547153 wait complete. 2024-05-08T23:21:17.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-05-08T23:21:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.692+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbfa40055a0 con 0x7fbfc415da50 2024-05-08T23:21:17.692 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.692+0000 7fbfc17fa700 1 -- 172.21.15.5:0/3937880460 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbfbc057b50 con 0x7fbfc415da50 2024-05-08T23:21:17.692 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-08T23:21:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.693+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe474074b50 con 0x7fe45c0056d0 2024-05-08T23:21:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.693+0000 7fe45b7fe700 1 -- 172.21.15.5:0/2305642991 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe464045020 con 0x7fe45c0056d0 2024-05-08T23:21:17.693 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.695+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbfac06af50 msgr2=0x7fbfac06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.695+0000 7fbfc9c1a700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbfac06af50 0x7fbfac06d410 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009fd0 tx=0x7fbfb401c040 comp rx=0 tx=0).stop 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.695+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc415da50 msgr2=0x7fbfc40cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.695+0000 7fbfc9c1a700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc415da50 0x7fbfc40cbf20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b990 tx=0x7fbfbc00bca0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.696+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 shutdown_connections 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.696+0000 7fbfc9c1a700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbfac06af50 0x7fbfac06d410 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.696+0000 7fbfc9c1a700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc415da50 0x7fbfc40cbf20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.696+0000 7fbfc9c1a700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfc40b0ee0 0x7fbfc4098aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.696+0000 7fbfc9c1a700 1 --2- 172.21.15.5:0/3937880460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfc41597a0 0x7fbfc40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.696+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 >> 172.21.15.5:0/3937880460 conn(0x7fbfc40b4700 msgr2=0x7fbfc40b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.696+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 shutdown_connections 2024-05-08T23:21:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.696+0000 7fbfc9c1a700 1 -- 172.21.15.5:0/3937880460 wait complete. 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.697+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe46006ad60 msgr2=0x7fe46006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe46006ad60 0x7fe46006d220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe474202f90 tx=0x7fe468009030 comp rx=0 tx=0).stop 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c0056d0 msgr2=0x7fe47408c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c0056d0 0x7fe47408c9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe464004fb0 tx=0x7fe4640073f0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 shutdown_connections 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe46006ad60 0x7fe46006d220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe474203200 0x7fe4742036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe45c0056d0 0x7fe47408c9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 --2- 172.21.15.5:0/2305642991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4740ad090 0x7fe4740ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 >> 172.21.15.5:0/2305642991 conn(0x7fe4740b0d60 msgr2=0x7fe474071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 shutdown_connections 2024-05-08T23:21:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.698+0000 7fe473fff700 1 -- 172.21.15.5:0/2305642991 wait complete. 2024-05-08T23:21:17.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-05-08T23:21:17.706 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:17.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-05-08T23:21:17.709 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:17.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-05-08T23:21:17.718 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:17.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-05-08T23:21:17.719 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:17.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-05-08T23:21:17.720 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:17.720 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T23:21:17.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:21:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.899+0000 7fb41b59e700 1 Processor -- start 2024-05-08T23:21:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.899+0000 7fb41b59e700 1 -- start start 2024-05-08T23:21:17.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41b59e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c0b4190 0x7fb41c1652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41b59e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c0ae810 con 0x7fb41c157f60 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41b59e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c0adc00 con 0x7fb41c158340 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41b59e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0adee0 con 0x7fb41c0b4190 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41a59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb41c157f60 0x7fb41c15b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42506/0 (socket says 172.21.15.5:42506) 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41a59c700 1 -- 172.21.15.5:0/3654411517 learned_addr learned my addr 172.21.15.5:0/3654411517 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41ad9d700 1 --2- 172.21.15.5:0/3654411517 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c0b4190 0x7fb41c1652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41959a700 1 -- 172.21.15.5:0/3654411517 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070941657 0 0) 0x7fb41c0ae810 con 0x7fb41c157f60 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41959a700 1 -- 172.21.15.5:0/3654411517 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4040035b0 con 0x7fb41c157f60 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41959a700 1 -- 172.21.15.5:0/3654411517 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566499515 0 0) 0x7fb41c0adc00 con 0x7fb41c158340 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.900+0000 7fb41959a700 1 -- 172.21.15.5:0/3654411517 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb41c0ae810 con 0x7fb41c158340 2024-05-08T23:21:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.901+0000 7fb41959a700 1 -- 172.21.15.5:0/3654411517 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3682335463 0 0) 0x7fb4040035b0 con 0x7fb41c157f60 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.901+0000 7fb41ad9d700 1 -- 172.21.15.5:0/3654411517 >> v1:172.21.15.49:6789/0 conn(0x7fb41c158340 legacy=0x7fb41c0d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.901+0000 7fb41ad9d700 1 -- 172.21.15.5:0/3654411517 >> v1:172.21.15.5:6789/0 conn(0x7fb41c157f60 legacy=0x7fb41c15b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.901+0000 7fb41ad9d700 1 -- 172.21.15.5:0/3654411517 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0ae1c0 con 0x7fb41c0b4190 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.901+0000 7fb41ad9d700 1 --2- 172.21.15.5:0/3654411517 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c0b4190 0x7fb41c1652b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb414009dc0 tx=0x7fb41400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1375bf5ee153d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.901+0000 7fb41959a700 1 -- 172.21.15.5:0/3654411517 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41400e070 con 0x7fb41c0b4190 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.901+0000 7fb41959a700 1 -- 172.21.15.5:0/3654411517 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414003a20 con 0x7fb41c0b4190 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.901+0000 7fb41959a700 1 -- 172.21.15.5:0/3654411517 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c001670 con 0x7fb41c0b4190 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.902+0000 7fb41b59e700 1 -- 172.21.15.5:0/3654411517 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c0b4190 msgr2=0x7fb41c1652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.902+0000 7fb41b59e700 1 --2- 172.21.15.5:0/3654411517 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c0b4190 0x7fb41c1652b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb414009dc0 tx=0x7fb41400b330 comp rx=0 tx=0).stop 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.902+0000 7fb41b59e700 1 -- 172.21.15.5:0/3654411517 shutdown_connections 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.902+0000 7fb41b59e700 1 --2- 172.21.15.5:0/3654411517 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c0b4190 0x7fb41c1652b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.902+0000 7fb41b59e700 1 -- 172.21.15.5:0/3654411517 >> 172.21.15.5:0/3654411517 conn(0x7fb41c099a50 msgr2=0x7fb41c0b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.902+0000 7fb41b59e700 1 -- 172.21.15.5:0/3654411517 shutdown_connections 2024-05-08T23:21:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.902+0000 7fb41b59e700 1 -- 172.21.15.5:0/3654411517 wait complete. 2024-05-08T23:21:17.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.904+0000 7fb41b59e700 1 Processor -- start 2024-05-08T23:21:17.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.904+0000 7fb41b59e700 1 -- start start 2024-05-08T23:21:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41b59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb41c0b4190 0x7fb41c0ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41b59e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb41c157f60 0x7fb41c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41b59e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c158340 0x7fb41c06dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41b59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c158ab0 con 0x7fb41c0b4190 2024-05-08T23:21:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41b59e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0cf410 con 0x7fb41c157f60 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41b59e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0cf6b0 con 0x7fb41c158340 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb41c0b4190 0x7fb41c0ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb41c0b4190 0x7fb41c0ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52744/0 (socket says 172.21.15.5:52744) 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 -- 172.21.15.5:0/2485821066 learned_addr learned my addr 172.21.15.5:0/2485821066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb419d9b700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb41c157f60 0x7fb41c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41ad9d700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c158340 0x7fb41c06dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 -- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c158340 msgr2=0x7fb41c06dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c158340 0x7fb41c06dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 -- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb41c157f60 msgr2=0x7fb41c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb41c157f60 0x7fb41c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 -- 172.21.15.5:0/2485821066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414009a70 con 0x7fb41c0b4190 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb41a59c700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb41c0b4190 0x7fb41c0ceed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c750 tx=0x7fb40c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.905+0000 7fb40affd700 1 -- 172.21.15.5:0/2485821066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c004c80 con 0x7fb41c0b4190 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.906+0000 7fb40affd700 1 -- 172.21.15.5:0/2485821066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c004de0 con 0x7fb41c0b4190 2024-05-08T23:21:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.906+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c0cf990 con 0x7fb41c0b4190 2024-05-08T23:21:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.906+0000 7fb40affd700 1 -- 172.21.15.5:0/2485821066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0072d0 con 0x7fb41c0b4190 2024-05-08T23:21:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.906+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c158740 con 0x7fb41c0b4190 2024-05-08T23:21:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.906+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c074b50 con 0x7fb41c0b4190 2024-05-08T23:21:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.907+0000 7fb40affd700 1 -- 172.21.15.5:0/2485821066 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb40c007430 con 0x7fb41c0b4190 2024-05-08T23:21:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.907+0000 7fb40affd700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb3fc06ac90 0x7fb3fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.908+0000 7fb40affd700 1 -- 172.21.15.5:0/2485821066 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb40c0891b0 con 0x7fb41c0b4190 2024-05-08T23:21:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.908+0000 7fb419d9b700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb3fc06ac90 0x7fb3fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.908+0000 7fb419d9b700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb3fc06ac90 0x7fb3fc06d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb41c154410 tx=0x7fb410009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:17.912+0000 7fb40affd700 1 -- 172.21.15.5:0/2485821066 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb40c0542f0 con 0x7fb41c0b4190 2024-05-08T23:21:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.124+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb41c0ad7d0 con 0x7fb3fc06ac90 2024-05-08T23:21:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.132+0000 7fb40affd700 1 -- 172.21.15.5:0/2485821066 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264534 (secure 0 0 0) 0x7fb410001cd0 con 0x7fb3fc06ac90 2024-05-08T23:21:18.136 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb3fc06ac90 msgr2=0x7fb3fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb3fc06ac90 0x7fb3fc06d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb41c154410 tx=0x7fb410009040 comp rx=0 tx=0).stop 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb41c0b4190 msgr2=0x7fb41c0ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb41c0b4190 0x7fb41c0ceed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c750 tx=0x7fb40c00ca60 comp rx=0 tx=0).stop 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 shutdown_connections 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb3fc06ac90 0x7fb3fc06d150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb41c158340 0x7fb41c06dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb41c0b4190 0x7fb41c0ceed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 --2- 172.21.15.5:0/2485821066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb41c157f60 0x7fb41c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 >> 172.21.15.5:0/2485821066 conn(0x7fb41c099a50 msgr2=0x7fb41c06f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 shutdown_connections 2024-05-08T23:21:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.136+0000 7fb41b59e700 1 -- 172.21.15.5:0/2485821066 wait complete. 2024-05-08T23:21:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:21:18.150 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-08T23:21:17.296700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003154"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356525+0000","last_change":"2024-05-08T23:21:04.117587+0000","last_active":"2024-05-08T23:21:05.356525+0000","last_peered":"2024-05-08T23:21:05.356525+0000","last_clean":"2024-05-08T23:21:05.356525+0000","last_became_active":"2024-05-08T23:21:04.117296+0000","last_became_peered":"2024-05-08T23:21:04.117296+0000","last_unstale":"2024-05-08T23:21:05.356525+0000","last_undegraded":"2024-05-08T23:21:05.356525+0000","last_fullsized":"2024-05-08T23:21:05.356525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093945+0000","last_change":"2024-05-08T23:21:04.113872+0000","last_active":"2024-05-08T23:21:05.093945+0000","last_peered":"2024-05-08T23:21:05.093945+0000","last_clean":"2024-05-08T23:21:05.093945+0000","last_became_active":"2024-05-08T23:21:04.113525+0000","last_became_peered":"2024-05-08T23:21:04.113525+0000","last_unstale":"2024-05-08T23:21:05.093945+0000","last_undegraded":"2024-05-08T23:21:05.093945+0000","last_fullsized":"2024-05-08T23:21:05.093945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356124+0000","last_change":"2024-05-08T23:21:04.100510+0000","last_active":"2024-05-08T23:21:05.356124+0000","last_peered":"2024-05-08T23:21:05.356124+0000","last_clean":"2024-05-08T23:21:05.356124+0000","last_became_active":"2024-05-08T23:21:04.100227+0000","last_became_peered":"2024-05-08T23:21:04.100227+0000","last_unstale":"2024-05-08T23:21:05.356124+0000","last_undegraded":"2024-05-08T23:21:05.356124+0000","last_fullsized":"2024-05-08T23:21:05.356124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.368136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356793+0000","last_change":"2024-05-08T23:21:04.110918+0000","last_active":"2024-05-08T23:21:05.356793+0000","last_peered":"2024-05-08T23:21:05.356793+0000","last_clean":"2024-05-08T23:21:05.356793+0000","last_became_active":"2024-05-08T23:21:04.110450+0000","last_became_peered":"2024-05-08T23:21:04.110450+0000","last_unstale":"2024-05-08T23:21:05.356793+0000","last_undegraded":"2024-05-08T23:21:05.356793+0000","last_fullsized":"2024-05-08T23:21:05.356793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356803+0000","last_change":"2024-05-08T23:21:04.113276+0000","last_active":"2024-05-08T23:21:05.356803+0000","last_peered":"2024-05-08T23:21:05.356803+0000","last_clean":"2024-05-08T23:21:05.356803+0000","last_became_active":"2024-05-08T23:21:04.112995+0000","last_became_peered":"2024-05-08T23:21:04.112995+0000","last_unstale":"2024-05-08T23:21:05.356803+0000","last_undegraded":"2024-05-08T23:21:05.356803+0000","last_fullsized":"2024-05-08T23:21:05.356803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356039+0000","last_change":"2024-05-08T23:21:04.106788+0000","last_active":"2024-05-08T23:21:05.356039+0000","last_peered":"2024-05-08T23:21:05.356039+0000","last_clean":"2024-05-08T23:21:05.356039+0000","last_became_active":"2024-05-08T23:21:04.106459+0000","last_became_peered":"2024-05-08T23:21:04.106459+0000","last_unstale":"2024-05-08T23:21:05.356039+0000","last_undegraded":"2024-05-08T23:21:05.356039+0000","last_fullsized":"2024-05-08T23:21:05.356039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356576+0000","last_change":"2024-05-08T23:21:04.112766+0000","last_active":"2024-05-08T23:21:05.356576+0000","last_peered":"2024-05-08T23:21:05.356576+0000","last_clean":"2024-05-08T23:21:05.356576+0000","last_became_active":"2024-05-08T23:21:04.112455+0000","last_became_peered":"2024-05-08T23:21:04.112455+0000","last_unstale":"2024-05-08T23:21:05.356576+0000","last_undegraded":"2024-05-08T23:21:05.356576+0000","last_fullsized":"2024-05-08T23:21:05.356576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:16.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092125+0000","last_change":"2024-05-08T23:21:04.103185+0000","last_active":"2024-05-08T23:21:05.092125+0000","last_peered":"2024-05-08T23:21:05.092125+0000","last_clean":"2024-05-08T23:21:05.092125+0000","last_became_active":"2024-05-08T23:21:04.102828+0000","last_became_peered":"2024-05-08T23:21:04.102828+0000","last_unstale":"2024-05-08T23:21:05.092125+0000","last_undegraded":"2024-05-08T23:21:05.092125+0000","last_fullsized":"2024-05-08T23:21:05.092125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356537+0000","last_change":"2024-05-08T23:21:04.122716+0000","last_active":"2024-05-08T23:21:05.356537+0000","last_peered":"2024-05-08T23:21:05.356537+0000","last_clean":"2024-05-08T23:21:05.356537+0000","last_became_active":"2024-05-08T23:21:04.122410+0000","last_became_peered":"2024-05-08T23:21:04.122410+0000","last_unstale":"2024-05-08T23:21:05.356537+0000","last_undegraded":"2024-05-08T23:21:05.356537+0000","last_fullsized":"2024-05-08T23:21:05.356537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:42.560490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355980+0000","last_change":"2024-05-08T23:21:04.111916+0000","last_active":"2024-05-08T23:21:05.355980+0000","last_peered":"2024-05-08T23:21:05.355980+0000","last_clean":"2024-05-08T23:21:05.355980+0000","last_became_active":"2024-05-08T23:21:04.111520+0000","last_became_peered":"2024-05-08T23:21:04.111520+0000","last_unstale":"2024-05-08T23:21:05.355980+0000","last_undegraded":"2024-05-08T23:21:05.355980+0000","last_fullsized":"2024-05-08T23:21:05.355980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355636+0000","last_change":"2024-05-08T23:21:04.113128+0000","last_active":"2024-05-08T23:21:05.355636+0000","last_peered":"2024-05-08T23:21:05.355636+0000","last_clean":"2024-05-08T23:21:05.355636+0000","last_became_active":"2024-05-08T23:21:04.112746+0000","last_became_peered":"2024-05-08T23:21:04.112746+0000","last_unstale":"2024-05-08T23:21:05.355636+0000","last_undegraded":"2024-05-08T23:21:05.355636+0000","last_fullsized":"2024-05-08T23:21:05.355636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:07.596384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093836+0000","last_change":"2024-05-08T23:21:04.118631+0000","last_active":"2024-05-08T23:21:05.093836+0000","last_peered":"2024-05-08T23:21:05.093836+0000","last_clean":"2024-05-08T23:21:05.093836+0000","last_became_active":"2024-05-08T23:21:04.118285+0000","last_became_peered":"2024-05-08T23:21:04.118285+0000","last_unstale":"2024-05-08T23:21:05.093836+0000","last_undegraded":"2024-05-08T23:21:05.093836+0000","last_fullsized":"2024-05-08T23:21:05.093836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.260488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092563+0000","last_change":"2024-05-08T23:21:04.114717+0000","last_active":"2024-05-08T23:21:05.092563+0000","last_peered":"2024-05-08T23:21:05.092563+0000","last_clean":"2024-05-08T23:21:05.092563+0000","last_became_active":"2024-05-08T23:21:04.114456+0000","last_became_peered":"2024-05-08T23:21:04.114456+0000","last_unstale":"2024-05-08T23:21:05.092563+0000","last_undegraded":"2024-05-08T23:21:05.092563+0000","last_fullsized":"2024-05-08T23:21:05.092563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.119343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355161+0000","last_change":"2024-05-08T23:21:04.110105+0000","last_active":"2024-05-08T23:21:05.355161+0000","last_peered":"2024-05-08T23:21:05.355161+0000","last_clean":"2024-05-08T23:21:05.355161+0000","last_became_active":"2024-05-08T23:21:04.109818+0000","last_became_peered":"2024-05-08T23:21:04.109818+0000","last_unstale":"2024-05-08T23:21:05.355161+0000","last_undegraded":"2024-05-08T23:21:05.355161+0000","last_fullsized":"2024-05-08T23:21:05.355161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093437+0000","last_change":"2024-05-08T23:21:04.114173+0000","last_active":"2024-05-08T23:21:05.093437+0000","last_peered":"2024-05-08T23:21:05.093437+0000","last_clean":"2024-05-08T23:21:05.093437+0000","last_became_active":"2024-05-08T23:21:04.108376+0000","last_became_peered":"2024-05-08T23:21:04.108376+0000","last_unstale":"2024-05-08T23:21:05.093437+0000","last_undegraded":"2024-05-08T23:21:05.093437+0000","last_fullsized":"2024-05-08T23:21:05.093437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355476+0000","last_change":"2024-05-08T23:21:04.100458+0000","last_active":"2024-05-08T23:21:05.355476+0000","last_peered":"2024-05-08T23:21:05.355476+0000","last_clean":"2024-05-08T23:21:05.355476+0000","last_became_active":"2024-05-08T23:21:04.100166+0000","last_became_peered":"2024-05-08T23:21:04.100166+0000","last_unstale":"2024-05-08T23:21:05.355476+0000","last_undegraded":"2024-05-08T23:21:05.355476+0000","last_fullsized":"2024-05-08T23:21:05.355476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357377+0000","last_change":"2024-05-08T23:21:04.109204+0000","last_active":"2024-05-08T23:21:05.357377+0000","last_peered":"2024-05-08T23:21:05.357377+0000","last_clean":"2024-05-08T23:21:05.357377+0000","last_became_active":"2024-05-08T23:21:04.108507+0000","last_became_peered":"2024-05-08T23:21:04.108507+0000","last_unstale":"2024-05-08T23:21:05.357377+0000","last_undegraded":"2024-05-08T23:21:05.357377+0000","last_fullsized":"2024-05-08T23:21:05.357377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:16.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355050+0000","last_change":"2024-05-08T23:21:04.114847+0000","last_active":"2024-05-08T23:21:05.355050+0000","last_peered":"2024-05-08T23:21:05.355050+0000","last_clean":"2024-05-08T23:21:05.355050+0000","last_became_active":"2024-05-08T23:21:04.114090+0000","last_became_peered":"2024-05-08T23:21:04.114090+0000","last_unstale":"2024-05-08T23:21:05.355050+0000","last_undegraded":"2024-05-08T23:21:05.355050+0000","last_fullsized":"2024-05-08T23:21:05.355050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.260488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356040+0000","last_change":"2024-05-08T23:21:04.113465+0000","last_active":"2024-05-08T23:21:05.356040+0000","last_peered":"2024-05-08T23:21:05.356040+0000","last_clean":"2024-05-08T23:21:05.356040+0000","last_became_active":"2024-05-08T23:21:04.113179+0000","last_became_peered":"2024-05-08T23:21:04.113179+0000","last_unstale":"2024-05-08T23:21:05.356040+0000","last_undegraded":"2024-05-08T23:21:05.356040+0000","last_fullsized":"2024-05-08T23:21:05.356040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355379+0000","last_change":"2024-05-08T23:21:03.646371+0000","last_active":"2024-05-08T23:21:05.355379+0000","last_peered":"2024-05-08T23:21:05.355379+0000","last_clean":"2024-05-08T23:21:05.355379+0000","last_became_active":"2024-05-08T23:21:03.646004+0000","last_became_peered":"2024-05-08T23:21:03.646004+0000","last_unstale":"2024-05-08T23:21:05.355379+0000","last_undegraded":"2024-05-08T23:21:05.355379+0000","last_fullsized":"2024-05-08T23:21:05.355379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:32.482556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092991+0000","last_change":"2024-05-08T23:21:04.115651+0000","last_active":"2024-05-08T23:21:05.092991+0000","last_peered":"2024-05-08T23:21:05.092991+0000","last_clean":"2024-05-08T23:21:05.092991+0000","last_became_active":"2024-05-08T23:21:04.115384+0000","last_became_peered":"2024-05-08T23:21:04.115384+0000","last_unstale":"2024-05-08T23:21:05.092991+0000","last_undegraded":"2024-05-08T23:21:05.092991+0000","last_fullsized":"2024-05-08T23:21:05.092991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093060+0000","last_change":"2024-05-08T23:21:03.645350+0000","last_active":"2024-05-08T23:21:05.093060+0000","last_peered":"2024-05-08T23:21:05.093060+0000","last_clean":"2024-05-08T23:21:05.093060+0000","last_became_active":"2024-05-08T23:21:03.645081+0000","last_became_peered":"2024-05-08T23:21:03.645081+0000","last_unstale":"2024-05-08T23:21:05.093060+0000","last_undegraded":"2024-05-08T23:21:05.093060+0000","last_fullsized":"2024-05-08T23:21:05.093060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355362+0000","last_change":"2024-05-08T23:21:04.112500+0000","last_active":"2024-05-08T23:21:05.355362+0000","last_peered":"2024-05-08T23:21:05.355362+0000","last_clean":"2024-05-08T23:21:05.355362+0000","last_became_active":"2024-05-08T23:21:04.111988+0000","last_became_peered":"2024-05-08T23:21:04.111988+0000","last_unstale":"2024-05-08T23:21:05.355362+0000","last_undegraded":"2024-05-08T23:21:05.355362+0000","last_fullsized":"2024-05-08T23:21:05.355362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356422+0000","last_change":"2024-05-08T23:21:03.647784+0000","last_active":"2024-05-08T23:21:05.356422+0000","last_peered":"2024-05-08T23:21:05.356422+0000","last_clean":"2024-05-08T23:21:05.356422+0000","last_became_active":"2024-05-08T23:21:03.646667+0000","last_became_peered":"2024-05-08T23:21:03.646667+0000","last_unstale":"2024-05-08T23:21:05.356422+0000","last_undegraded":"2024-05-08T23:21:05.356422+0000","last_fullsized":"2024-05-08T23:21:05.356422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355900+0000","last_change":"2024-05-08T23:21:04.104612+0000","last_active":"2024-05-08T23:21:05.355900+0000","last_peered":"2024-05-08T23:21:05.355900+0000","last_clean":"2024-05-08T23:21:05.355900+0000","last_became_active":"2024-05-08T23:21:04.104304+0000","last_became_peered":"2024-05-08T23:21:04.104304+0000","last_unstale":"2024-05-08T23:21:05.355900+0000","last_undegraded":"2024-05-08T23:21:05.355900+0000","last_fullsized":"2024-05-08T23:21:05.355900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356075+0000","last_change":"2024-05-08T23:21:03.648270+0000","last_active":"2024-05-08T23:21:05.356075+0000","last_peered":"2024-05-08T23:21:05.356075+0000","last_clean":"2024-05-08T23:21:05.356075+0000","last_became_active":"2024-05-08T23:21:03.647784+0000","last_became_peered":"2024-05-08T23:21:03.647784+0000","last_unstale":"2024-05-08T23:21:05.356075+0000","last_undegraded":"2024-05-08T23:21:05.356075+0000","last_fullsized":"2024-05-08T23:21:05.356075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357179+0000","last_change":"2024-05-08T23:21:04.106021+0000","last_active":"2024-05-08T23:21:05.357179+0000","last_peered":"2024-05-08T23:21:05.357179+0000","last_clean":"2024-05-08T23:21:05.357179+0000","last_became_active":"2024-05-08T23:21:04.105614+0000","last_became_peered":"2024-05-08T23:21:04.105614+0000","last_unstale":"2024-05-08T23:21:05.357179+0000","last_undegraded":"2024-05-08T23:21:05.357179+0000","last_fullsized":"2024-05-08T23:21:05.357179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.119343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092611+0000","last_change":"2024-05-08T23:21:03.642600+0000","last_active":"2024-05-08T23:21:05.092611+0000","last_peered":"2024-05-08T23:21:05.092611+0000","last_clean":"2024-05-08T23:21:05.092611+0000","last_became_active":"2024-05-08T23:21:03.642234+0000","last_became_peered":"2024-05-08T23:21:03.642234+0000","last_unstale":"2024-05-08T23:21:05.092611+0000","last_undegraded":"2024-05-08T23:21:05.092611+0000","last_fullsized":"2024-05-08T23:21:05.092611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093418+0000","last_change":"2024-05-08T23:21:04.117783+0000","last_active":"2024-05-08T23:21:05.093418+0000","last_peered":"2024-05-08T23:21:05.093418+0000","last_clean":"2024-05-08T23:21:05.093418+0000","last_became_active":"2024-05-08T23:21:04.117324+0000","last_became_peered":"2024-05-08T23:21:04.117324+0000","last_unstale":"2024-05-08T23:21:05.093418+0000","last_undegraded":"2024-05-08T23:21:05.093418+0000","last_fullsized":"2024-05-08T23:21:05.093418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.433453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355523+0000","last_change":"2024-05-08T23:21:03.644641+0000","last_active":"2024-05-08T23:21:05.355523+0000","last_peered":"2024-05-08T23:21:05.355523+0000","last_clean":"2024-05-08T23:21:05.355523+0000","last_became_active":"2024-05-08T23:21:03.644361+0000","last_became_peered":"2024-05-08T23:21:03.644361+0000","last_unstale":"2024-05-08T23:21:05.355523+0000","last_undegraded":"2024-05-08T23:21:05.355523+0000","last_fullsized":"2024-05-08T23:21:05.355523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355562+0000","last_change":"2024-05-08T23:21:04.105553+0000","last_active":"2024-05-08T23:21:05.355562+0000","last_peered":"2024-05-08T23:21:05.355562+0000","last_clean":"2024-05-08T23:21:05.355562+0000","last_became_active":"2024-05-08T23:21:04.105041+0000","last_became_peered":"2024-05-08T23:21:04.105041+0000","last_unstale":"2024-05-08T23:21:05.355562+0000","last_undegraded":"2024-05-08T23:21:05.355562+0000","last_fullsized":"2024-05-08T23:21:05.355562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.311829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092401+0000","last_change":"2024-05-08T23:21:03.639910+0000","last_active":"2024-05-08T23:21:05.092401+0000","last_peered":"2024-05-08T23:21:05.092401+0000","last_clean":"2024-05-08T23:21:05.092401+0000","last_became_active":"2024-05-08T23:21:03.638934+0000","last_became_peered":"2024-05-08T23:21:03.638934+0000","last_unstale":"2024-05-08T23:21:05.092401+0000","last_undegraded":"2024-05-08T23:21:05.092401+0000","last_fullsized":"2024-05-08T23:21:05.092401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092248+0000","last_change":"2024-05-08T23:21:04.108315+0000","last_active":"2024-05-08T23:21:05.092248+0000","last_peered":"2024-05-08T23:21:05.092248+0000","last_clean":"2024-05-08T23:21:05.092248+0000","last_became_active":"2024-05-08T23:21:04.108040+0000","last_became_peered":"2024-05-08T23:21:04.108040+0000","last_unstale":"2024-05-08T23:21:05.092248+0000","last_undegraded":"2024-05-08T23:21:05.092248+0000","last_fullsized":"2024-05-08T23:21:05.092248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355919+0000","last_change":"2024-05-08T23:21:03.641788+0000","last_active":"2024-05-08T23:21:05.355919+0000","last_peered":"2024-05-08T23:21:05.355919+0000","last_clean":"2024-05-08T23:21:05.355919+0000","last_became_active":"2024-05-08T23:21:03.641004+0000","last_became_peered":"2024-05-08T23:21:03.641004+0000","last_unstale":"2024-05-08T23:21:05.355919+0000","last_undegraded":"2024-05-08T23:21:05.355919+0000","last_fullsized":"2024-05-08T23:21:05.355919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093537+0000","last_change":"2024-05-08T23:21:04.114572+0000","last_active":"2024-05-08T23:21:05.093537+0000","last_peered":"2024-05-08T23:21:05.093537+0000","last_clean":"2024-05-08T23:21:05.093537+0000","last_became_active":"2024-05-08T23:21:04.114402+0000","last_became_peered":"2024-05-08T23:21:04.114402+0000","last_unstale":"2024-05-08T23:21:05.093537+0000","last_undegraded":"2024-05-08T23:21:05.093537+0000","last_fullsized":"2024-05-08T23:21:05.093537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356964+0000","last_change":"2024-05-08T23:21:03.648330+0000","last_active":"2024-05-08T23:21:05.356964+0000","last_peered":"2024-05-08T23:21:05.356964+0000","last_clean":"2024-05-08T23:21:05.356964+0000","last_became_active":"2024-05-08T23:21:03.648033+0000","last_became_peered":"2024-05-08T23:21:03.648033+0000","last_unstale":"2024-05-08T23:21:05.356964+0000","last_undegraded":"2024-05-08T23:21:05.356964+0000","last_fullsized":"2024-05-08T23:21:05.356964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356917+0000","last_change":"2024-05-08T23:21:04.108163+0000","last_active":"2024-05-08T23:21:05.356917+0000","last_peered":"2024-05-08T23:21:05.356917+0000","last_clean":"2024-05-08T23:21:05.356917+0000","last_became_active":"2024-05-08T23:21:04.107854+0000","last_became_peered":"2024-05-08T23:21:04.107854+0000","last_unstale":"2024-05-08T23:21:05.356917+0000","last_undegraded":"2024-05-08T23:21:05.356917+0000","last_fullsized":"2024-05-08T23:21:05.356917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:09.028590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356841+0000","last_change":"2024-05-08T23:21:03.640432+0000","last_active":"2024-05-08T23:21:05.356841+0000","last_peered":"2024-05-08T23:21:05.356841+0000","last_clean":"2024-05-08T23:21:05.356841+0000","last_became_active":"2024-05-08T23:21:03.640121+0000","last_became_peered":"2024-05-08T23:21:03.640121+0000","last_unstale":"2024-05-08T23:21:05.356841+0000","last_undegraded":"2024-05-08T23:21:05.356841+0000","last_fullsized":"2024-05-08T23:21:05.356841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355475+0000","last_change":"2024-05-08T23:21:04.113136+0000","last_active":"2024-05-08T23:21:05.355475+0000","last_peered":"2024-05-08T23:21:05.355475+0000","last_clean":"2024-05-08T23:21:05.355475+0000","last_became_active":"2024-05-08T23:21:04.112831+0000","last_became_peered":"2024-05-08T23:21:04.112831+0000","last_unstale":"2024-05-08T23:21:05.355475+0000","last_undegraded":"2024-05-08T23:21:05.355475+0000","last_fullsized":"2024-05-08T23:21:05.355475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:03.946264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093793+0000","last_change":"2024-05-08T23:21:03.647271+0000","last_active":"2024-05-08T23:21:05.093793+0000","last_peered":"2024-05-08T23:21:05.093793+0000","last_clean":"2024-05-08T23:21:05.093793+0000","last_became_active":"2024-05-08T23:21:03.646992+0000","last_became_peered":"2024-05-08T23:21:03.646992+0000","last_unstale":"2024-05-08T23:21:05.093793+0000","last_undegraded":"2024-05-08T23:21:05.093793+0000","last_fullsized":"2024-05-08T23:21:05.093793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:43.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093761+0000","last_change":"2024-05-08T23:21:04.106622+0000","last_active":"2024-05-08T23:21:05.093761+0000","last_peered":"2024-05-08T23:21:05.093761+0000","last_clean":"2024-05-08T23:21:05.093761+0000","last_became_active":"2024-05-08T23:21:04.106248+0000","last_became_peered":"2024-05-08T23:21:04.106248+0000","last_unstale":"2024-05-08T23:21:05.093761+0000","last_undegraded":"2024-05-08T23:21:05.093761+0000","last_fullsized":"2024-05-08T23:21:05.093761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.311829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356428+0000","last_change":"2024-05-08T23:21:03.642565+0000","last_active":"2024-05-08T23:21:05.356428+0000","last_peered":"2024-05-08T23:21:05.356428+0000","last_clean":"2024-05-08T23:21:05.356428+0000","last_became_active":"2024-05-08T23:21:03.642253+0000","last_became_peered":"2024-05-08T23:21:03.642253+0000","last_unstale":"2024-05-08T23:21:05.356428+0000","last_undegraded":"2024-05-08T23:21:05.356428+0000","last_fullsized":"2024-05-08T23:21:05.356428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356171+0000","last_change":"2024-05-08T23:21:04.113888+0000","last_active":"2024-05-08T23:21:05.356171+0000","last_peered":"2024-05-08T23:21:05.356171+0000","last_clean":"2024-05-08T23:21:05.356171+0000","last_became_active":"2024-05-08T23:21:04.113607+0000","last_became_peered":"2024-05-08T23:21:04.113607+0000","last_unstale":"2024-05-08T23:21:05.356171+0000","last_undegraded":"2024-05-08T23:21:05.356171+0000","last_fullsized":"2024-05-08T23:21:05.356171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:19.795476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355509+0000","last_change":"2024-05-08T23:21:03.641751+0000","last_active":"2024-05-08T23:21:05.355509+0000","last_peered":"2024-05-08T23:21:05.355509+0000","last_clean":"2024-05-08T23:21:05.355509+0000","last_became_active":"2024-05-08T23:21:03.641441+0000","last_became_peered":"2024-05-08T23:21:03.641441+0000","last_unstale":"2024-05-08T23:21:05.355509+0000","last_undegraded":"2024-05-08T23:21:05.355509+0000","last_fullsized":"2024-05-08T23:21:05.355509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:58.822252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355038+0000","last_change":"2024-05-08T23:21:04.106177+0000","last_active":"2024-05-08T23:21:05.355038+0000","last_peered":"2024-05-08T23:21:05.355038+0000","last_clean":"2024-05-08T23:21:05.355038+0000","last_became_active":"2024-05-08T23:21:04.105844+0000","last_became_peered":"2024-05-08T23:21:04.105844+0000","last_unstale":"2024-05-08T23:21:05.355038+0000","last_undegraded":"2024-05-08T23:21:05.355038+0000","last_fullsized":"2024-05-08T23:21:05.355038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356436+0000","last_change":"2024-05-08T23:21:03.646376+0000","last_active":"2024-05-08T23:21:05.356436+0000","last_peered":"2024-05-08T23:21:05.356436+0000","last_clean":"2024-05-08T23:21:05.356436+0000","last_became_active":"2024-05-08T23:21:03.646071+0000","last_became_peered":"2024-05-08T23:21:03.646071+0000","last_unstale":"2024-05-08T23:21:05.356436+0000","last_undegraded":"2024-05-08T23:21:05.356436+0000","last_fullsized":"2024-05-08T23:21:05.356436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356915+0000","last_change":"2024-05-08T23:21:04.120630+0000","last_active":"2024-05-08T23:21:05.356915+0000","last_peered":"2024-05-08T23:21:05.356915+0000","last_clean":"2024-05-08T23:21:05.356915+0000","last_became_active":"2024-05-08T23:21:04.120315+0000","last_became_peered":"2024-05-08T23:21:04.120315+0000","last_unstale":"2024-05-08T23:21:05.356915+0000","last_undegraded":"2024-05-08T23:21:05.356915+0000","last_fullsized":"2024-05-08T23:21:05.356915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:57.953372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356590+0000","last_change":"2024-05-08T23:21:03.645556+0000","last_active":"2024-05-08T23:21:05.356590+0000","last_peered":"2024-05-08T23:21:05.356590+0000","last_clean":"2024-05-08T23:21:05.356590+0000","last_became_active":"2024-05-08T23:21:03.645083+0000","last_became_peered":"2024-05-08T23:21:03.645083+0000","last_unstale":"2024-05-08T23:21:05.356590+0000","last_undegraded":"2024-05-08T23:21:05.356590+0000","last_fullsized":"2024-05-08T23:21:05.356590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355274+0000","last_change":"2024-05-08T23:21:04.108235+0000","last_active":"2024-05-08T23:21:05.355274+0000","last_peered":"2024-05-08T23:21:05.355274+0000","last_clean":"2024-05-08T23:21:05.355274+0000","last_became_active":"2024-05-08T23:21:04.107933+0000","last_became_peered":"2024-05-08T23:21:04.107933+0000","last_unstale":"2024-05-08T23:21:05.355274+0000","last_undegraded":"2024-05-08T23:21:05.355274+0000","last_fullsized":"2024-05-08T23:21:05.355274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354849+0000","last_change":"2024-05-08T23:21:03.636549+0000","last_active":"2024-05-08T23:21:05.354849+0000","last_peered":"2024-05-08T23:21:05.354849+0000","last_clean":"2024-05-08T23:21:05.354849+0000","last_became_active":"2024-05-08T23:21:03.636259+0000","last_became_peered":"2024-05-08T23:21:03.636259+0000","last_unstale":"2024-05-08T23:21:05.354849+0000","last_undegraded":"2024-05-08T23:21:05.354849+0000","last_fullsized":"2024-05-08T23:21:05.354849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356690+0000","last_change":"2024-05-08T23:21:04.113963+0000","last_active":"2024-05-08T23:21:05.356690+0000","last_peered":"2024-05-08T23:21:05.356690+0000","last_clean":"2024-05-08T23:21:05.356690+0000","last_became_active":"2024-05-08T23:21:04.113645+0000","last_became_peered":"2024-05-08T23:21:04.113645+0000","last_unstale":"2024-05-08T23:21:05.356690+0000","last_undegraded":"2024-05-08T23:21:05.356690+0000","last_fullsized":"2024-05-08T23:21:05.356690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093744+0000","last_change":"2024-05-08T23:21:03.644725+0000","last_active":"2024-05-08T23:21:05.093744+0000","last_peered":"2024-05-08T23:21:05.093744+0000","last_clean":"2024-05-08T23:21:05.093744+0000","last_became_active":"2024-05-08T23:21:03.644432+0000","last_became_peered":"2024-05-08T23:21:03.644432+0000","last_unstale":"2024-05-08T23:21:05.093744+0000","last_undegraded":"2024-05-08T23:21:05.093744+0000","last_fullsized":"2024-05-08T23:21:05.093744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:58.822252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356426+0000","last_change":"2024-05-08T23:21:04.116753+0000","last_active":"2024-05-08T23:21:05.356426+0000","last_peered":"2024-05-08T23:21:05.356426+0000","last_clean":"2024-05-08T23:21:05.356426+0000","last_became_active":"2024-05-08T23:21:04.116366+0000","last_became_peered":"2024-05-08T23:21:04.116366+0000","last_unstale":"2024-05-08T23:21:05.356426+0000","last_undegraded":"2024-05-08T23:21:05.356426+0000","last_fullsized":"2024-05-08T23:21:05.356426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093071+0000","last_change":"2024-05-08T23:21:03.643882+0000","last_active":"2024-05-08T23:21:05.093071+0000","last_peered":"2024-05-08T23:21:05.093071+0000","last_clean":"2024-05-08T23:21:05.093071+0000","last_became_active":"2024-05-08T23:21:03.643533+0000","last_became_peered":"2024-05-08T23:21:03.643533+0000","last_unstale":"2024-05-08T23:21:05.093071+0000","last_undegraded":"2024-05-08T23:21:05.093071+0000","last_fullsized":"2024-05-08T23:21:05.093071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357279+0000","last_change":"2024-05-08T23:21:04.109134+0000","last_active":"2024-05-08T23:21:05.357279+0000","last_peered":"2024-05-08T23:21:05.357279+0000","last_clean":"2024-05-08T23:21:05.357279+0000","last_became_active":"2024-05-08T23:21:04.108799+0000","last_became_peered":"2024-05-08T23:21:04.108799+0000","last_unstale":"2024-05-08T23:21:05.357279+0000","last_undegraded":"2024-05-08T23:21:05.357279+0000","last_fullsized":"2024-05-08T23:21:05.357279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356346+0000","last_change":"2024-05-08T23:21:03.644161+0000","last_active":"2024-05-08T23:21:05.356346+0000","last_peered":"2024-05-08T23:21:05.356346+0000","last_clean":"2024-05-08T23:21:05.356346+0000","last_became_active":"2024-05-08T23:21:03.643341+0000","last_became_peered":"2024-05-08T23:21:03.643341+0000","last_unstale":"2024-05-08T23:21:05.356346+0000","last_undegraded":"2024-05-08T23:21:05.356346+0000","last_fullsized":"2024-05-08T23:21:05.356346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093723+0000","last_change":"2024-05-08T23:21:04.108965+0000","last_active":"2024-05-08T23:21:05.093723+0000","last_peered":"2024-05-08T23:21:05.093723+0000","last_clean":"2024-05-08T23:21:05.093723+0000","last_became_active":"2024-05-08T23:21:04.107917+0000","last_became_peered":"2024-05-08T23:21:04.107917+0000","last_unstale":"2024-05-08T23:21:05.093723+0000","last_undegraded":"2024-05-08T23:21:05.093723+0000","last_fullsized":"2024-05-08T23:21:05.093723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:00.670245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093185+0000","last_change":"2024-05-08T23:21:04.115959+0000","last_active":"2024-05-08T23:21:05.093185+0000","last_peered":"2024-05-08T23:21:05.093185+0000","last_clean":"2024-05-08T23:21:05.093185+0000","last_became_active":"2024-05-08T23:21:04.115696+0000","last_became_peered":"2024-05-08T23:21:04.115696+0000","last_unstale":"2024-05-08T23:21:05.093185+0000","last_undegraded":"2024-05-08T23:21:05.093185+0000","last_fullsized":"2024-05-08T23:21:05.093185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354856+0000","last_change":"2024-05-08T23:21:03.632217+0000","last_active":"2024-05-08T23:21:05.354856+0000","last_peered":"2024-05-08T23:21:05.354856+0000","last_clean":"2024-05-08T23:21:05.354856+0000","last_became_active":"2024-05-08T23:21:03.631883+0000","last_became_peered":"2024-05-08T23:21:03.631883+0000","last_unstale":"2024-05-08T23:21:05.354856+0000","last_undegraded":"2024-05-08T23:21:05.354856+0000","last_fullsized":"2024-05-08T23:21:05.354856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092402+0000","last_change":"2024-05-08T23:21:04.104594+0000","last_active":"2024-05-08T23:21:05.092402+0000","last_peered":"2024-05-08T23:21:05.092402+0000","last_clean":"2024-05-08T23:21:05.092402+0000","last_became_active":"2024-05-08T23:21:04.104335+0000","last_became_peered":"2024-05-08T23:21:04.104335+0000","last_unstale":"2024-05-08T23:21:05.092402+0000","last_undegraded":"2024-05-08T23:21:05.092402+0000","last_fullsized":"2024-05-08T23:21:05.092402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355796+0000","last_change":"2024-05-08T23:21:03.637311+0000","last_active":"2024-05-08T23:21:05.355796+0000","last_peered":"2024-05-08T23:21:05.355796+0000","last_clean":"2024-05-08T23:21:05.355796+0000","last_became_active":"2024-05-08T23:21:03.636529+0000","last_became_peered":"2024-05-08T23:21:03.636529+0000","last_unstale":"2024-05-08T23:21:05.355796+0000","last_undegraded":"2024-05-08T23:21:05.355796+0000","last_fullsized":"2024-05-08T23:21:05.355796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-08T23:21:04.101711+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:21:04.101711+0000","last_peered":"2024-05-08T23:21:04.101711+0000","last_clean":"2024-05-08T23:21:04.101711+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:21:04.101711+0000","last_undegraded":"2024-05-08T23:21:04.101711+0000","last_fullsized":"2024-05-08T23:21:04.101711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355927+0000","last_change":"2024-05-08T23:21:04.108232+0000","last_active":"2024-05-08T23:21:05.355927+0000","last_peered":"2024-05-08T23:21:05.355927+0000","last_clean":"2024-05-08T23:21:05.355927+0000","last_became_active":"2024-05-08T23:21:04.107908+0000","last_became_peered":"2024-05-08T23:21:04.107908+0000","last_unstale":"2024-05-08T23:21:05.355927+0000","last_undegraded":"2024-05-08T23:21:05.355927+0000","last_fullsized":"2024-05-08T23:21:05.355927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:17.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356470+0000","last_change":"2024-05-08T23:21:03.645567+0000","last_active":"2024-05-08T23:21:05.356470+0000","last_peered":"2024-05-08T23:21:05.356470+0000","last_clean":"2024-05-08T23:21:05.356470+0000","last_became_active":"2024-05-08T23:21:03.644844+0000","last_became_peered":"2024-05-08T23:21:03.644844+0000","last_unstale":"2024-05-08T23:21:05.356470+0000","last_undegraded":"2024-05-08T23:21:05.356470+0000","last_fullsized":"2024-05-08T23:21:05.356470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:43.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356283+0000","last_change":"2024-05-08T23:21:03.646987+0000","last_active":"2024-05-08T23:21:05.356283+0000","last_peered":"2024-05-08T23:21:05.356283+0000","last_clean":"2024-05-08T23:21:05.356283+0000","last_became_active":"2024-05-08T23:21:03.646283+0000","last_became_peered":"2024-05-08T23:21:03.646283+0000","last_unstale":"2024-05-08T23:21:05.356283+0000","last_undegraded":"2024-05-08T23:21:05.356283+0000","last_fullsized":"2024-05-08T23:21:05.356283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:59.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092960+0000","last_change":"2024-05-08T23:21:04.106165+0000","last_active":"2024-05-08T23:21:05.092960+0000","last_peered":"2024-05-08T23:21:05.092960+0000","last_clean":"2024-05-08T23:21:05.092960+0000","last_became_active":"2024-05-08T23:21:04.105879+0000","last_became_peered":"2024-05-08T23:21:04.105879+0000","last_unstale":"2024-05-08T23:21:05.092960+0000","last_undegraded":"2024-05-08T23:21:05.092960+0000","last_fullsized":"2024-05-08T23:21:05.092960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.368136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:07.296809+0000","last_change":"2024-05-08T23:21:03.638777+0000","last_active":"2024-05-08T23:21:07.296809+0000","last_peered":"2024-05-08T23:21:07.296809+0000","last_clean":"2024-05-08T23:21:07.296809+0000","last_became_active":"2024-05-08T23:21:03.638494+0000","last_became_peered":"2024-05-08T23:21:03.638494+0000","last_unstale":"2024-05-08T23:21:07.296809+0000","last_undegraded":"2024-05-08T23:21:07.296809+0000","last_fullsized":"2024-05-08T23:21:07.296809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093027+0000","last_change":"2024-05-08T23:21:04.113442+0000","last_active":"2024-05-08T23:21:05.093027+0000","last_peered":"2024-05-08T23:21:05.093027+0000","last_clean":"2024-05-08T23:21:05.093027+0000","last_became_active":"2024-05-08T23:21:04.113144+0000","last_became_peered":"2024-05-08T23:21:04.113144+0000","last_unstale":"2024-05-08T23:21:05.093027+0000","last_undegraded":"2024-05-08T23:21:05.093027+0000","last_fullsized":"2024-05-08T23:21:05.093027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355626+0000","last_change":"2024-05-08T23:21:03.644120+0000","last_active":"2024-05-08T23:21:05.355626+0000","last_peered":"2024-05-08T23:21:05.355626+0000","last_clean":"2024-05-08T23:21:05.355626+0000","last_became_active":"2024-05-08T23:21:03.643541+0000","last_became_peered":"2024-05-08T23:21:03.643541+0000","last_unstale":"2024-05-08T23:21:05.355626+0000","last_undegraded":"2024-05-08T23:21:05.355626+0000","last_fullsized":"2024-05-08T23:21:05.355626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355406+0000","last_change":"2024-05-08T23:21:04.115769+0000","last_active":"2024-05-08T23:21:05.355406+0000","last_peered":"2024-05-08T23:21:05.355406+0000","last_clean":"2024-05-08T23:21:05.355406+0000","last_became_active":"2024-05-08T23:21:04.115446+0000","last_became_peered":"2024-05-08T23:21:04.115446+0000","last_unstale":"2024-05-08T23:21:05.355406+0000","last_undegraded":"2024-05-08T23:21:05.355406+0000","last_fullsized":"2024-05-08T23:21:05.355406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356111+0000","last_change":"2024-05-08T23:21:03.647284+0000","last_active":"2024-05-08T23:21:05.356111+0000","last_peered":"2024-05-08T23:21:05.356111+0000","last_clean":"2024-05-08T23:21:05.356111+0000","last_became_active":"2024-05-08T23:21:03.645828+0000","last_became_peered":"2024-05-08T23:21:03.645828+0000","last_unstale":"2024-05-08T23:21:05.356111+0000","last_undegraded":"2024-05-08T23:21:05.356111+0000","last_fullsized":"2024-05-08T23:21:05.356111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:54.428258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093694+0000","last_change":"2024-05-08T23:21:04.105901+0000","last_active":"2024-05-08T23:21:05.093694+0000","last_peered":"2024-05-08T23:21:05.093694+0000","last_clean":"2024-05-08T23:21:05.093694+0000","last_became_active":"2024-05-08T23:21:04.105495+0000","last_became_peered":"2024-05-08T23:21:04.105495+0000","last_unstale":"2024-05-08T23:21:05.093694+0000","last_undegraded":"2024-05-08T23:21:05.093694+0000","last_fullsized":"2024-05-08T23:21:05.093694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:44.288365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093485+0000","last_change":"2024-05-08T23:21:03.643905+0000","last_active":"2024-05-08T23:21:05.093485+0000","last_peered":"2024-05-08T23:21:05.093485+0000","last_clean":"2024-05-08T23:21:05.093485+0000","last_became_active":"2024-05-08T23:21:03.643555+0000","last_became_peered":"2024-05-08T23:21:03.643555+0000","last_unstale":"2024-05-08T23:21:05.093485+0000","last_undegraded":"2024-05-08T23:21:05.093485+0000","last_fullsized":"2024-05-08T23:21:05.093485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092205+0000","last_change":"2024-05-08T23:21:04.117123+0000","last_active":"2024-05-08T23:21:05.092205+0000","last_peered":"2024-05-08T23:21:05.092205+0000","last_clean":"2024-05-08T23:21:05.092205+0000","last_became_active":"2024-05-08T23:21:04.116830+0000","last_became_peered":"2024-05-08T23:21:04.116830+0000","last_unstale":"2024-05-08T23:21:05.092205+0000","last_undegraded":"2024-05-08T23:21:05.092205+0000","last_fullsized":"2024-05-08T23:21:05.092205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093648+0000","last_change":"2024-05-08T23:21:03.642867+0000","last_active":"2024-05-08T23:21:05.093648+0000","last_peered":"2024-05-08T23:21:05.093648+0000","last_clean":"2024-05-08T23:21:05.093648+0000","last_became_active":"2024-05-08T23:21:03.642439+0000","last_became_peered":"2024-05-08T23:21:03.642439+0000","last_unstale":"2024-05-08T23:21:05.093648+0000","last_undegraded":"2024-05-08T23:21:05.093648+0000","last_fullsized":"2024-05-08T23:21:05.093648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092672+0000","last_change":"2024-05-08T23:21:04.115928+0000","last_active":"2024-05-08T23:21:05.092672+0000","last_peered":"2024-05-08T23:21:05.092672+0000","last_clean":"2024-05-08T23:21:05.092672+0000","last_became_active":"2024-05-08T23:21:04.113858+0000","last_became_peered":"2024-05-08T23:21:04.113858+0000","last_unstale":"2024-05-08T23:21:05.092672+0000","last_undegraded":"2024-05-08T23:21:05.092672+0000","last_fullsized":"2024-05-08T23:21:05.092672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357022+0000","last_change":"2024-05-08T23:21:03.649228+0000","last_active":"2024-05-08T23:21:05.357022+0000","last_peered":"2024-05-08T23:21:05.357022+0000","last_clean":"2024-05-08T23:21:05.357022+0000","last_became_active":"2024-05-08T23:21:03.648868+0000","last_became_peered":"2024-05-08T23:21:03.648868+0000","last_unstale":"2024-05-08T23:21:05.357022+0000","last_undegraded":"2024-05-08T23:21:05.357022+0000","last_fullsized":"2024-05-08T23:21:05.357022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:34.255292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355422+0000","last_change":"2024-05-08T23:21:04.110982+0000","last_active":"2024-05-08T23:21:05.355422+0000","last_peered":"2024-05-08T23:21:05.355422+0000","last_clean":"2024-05-08T23:21:05.355422+0000","last_became_active":"2024-05-08T23:21:04.110643+0000","last_became_peered":"2024-05-08T23:21:04.110643+0000","last_unstale":"2024-05-08T23:21:05.355422+0000","last_undegraded":"2024-05-08T23:21:05.355422+0000","last_fullsized":"2024-05-08T23:21:05.355422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355712+0000","last_change":"2024-05-08T23:21:03.637418+0000","last_active":"2024-05-08T23:21:05.355712+0000","last_peered":"2024-05-08T23:21:05.355712+0000","last_clean":"2024-05-08T23:21:05.355712+0000","last_became_active":"2024-05-08T23:21:03.637139+0000","last_became_peered":"2024-05-08T23:21:03.637139+0000","last_unstale":"2024-05-08T23:21:05.355712+0000","last_undegraded":"2024-05-08T23:21:05.355712+0000","last_fullsized":"2024-05-08T23:21:05.355712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356254+0000","last_change":"2024-05-08T23:21:04.112240+0000","last_active":"2024-05-08T23:21:05.356254+0000","last_peered":"2024-05-08T23:21:05.356254+0000","last_clean":"2024-05-08T23:21:05.356254+0000","last_became_active":"2024-05-08T23:21:04.111914+0000","last_became_peered":"2024-05-08T23:21:04.111914+0000","last_unstale":"2024-05-08T23:21:05.356254+0000","last_undegraded":"2024-05-08T23:21:05.356254+0000","last_fullsized":"2024-05-08T23:21:05.356254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:09.028590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093821+0000","last_change":"2024-05-08T23:21:03.643712+0000","last_active":"2024-05-08T23:21:05.093821+0000","last_peered":"2024-05-08T23:21:05.093821+0000","last_clean":"2024-05-08T23:21:05.093821+0000","last_became_active":"2024-05-08T23:21:03.643387+0000","last_became_peered":"2024-05-08T23:21:03.643387+0000","last_unstale":"2024-05-08T23:21:05.093821+0000","last_undegraded":"2024-05-08T23:21:05.093821+0000","last_fullsized":"2024-05-08T23:21:05.093821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:59.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356035+0000","last_change":"2024-05-08T23:21:04.110728+0000","last_active":"2024-05-08T23:21:05.356035+0000","last_peered":"2024-05-08T23:21:05.356035+0000","last_clean":"2024-05-08T23:21:05.356035+0000","last_became_active":"2024-05-08T23:21:04.110094+0000","last_became_peered":"2024-05-08T23:21:04.110094+0000","last_unstale":"2024-05-08T23:21:05.356035+0000","last_undegraded":"2024-05-08T23:21:05.356035+0000","last_fullsized":"2024-05-08T23:21:05.356035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:30:28.180773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355130+0000","last_change":"2024-05-08T23:21:03.638990+0000","last_active":"2024-05-08T23:21:05.355130+0000","last_peered":"2024-05-08T23:21:05.355130+0000","last_clean":"2024-05-08T23:21:05.355130+0000","last_became_active":"2024-05-08T23:21:03.638696+0000","last_became_peered":"2024-05-08T23:21:03.638696+0000","last_unstale":"2024-05-08T23:21:05.355130+0000","last_undegraded":"2024-05-08T23:21:05.355130+0000","last_fullsized":"2024-05-08T23:21:05.355130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:16.993057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356830+0000","last_change":"2024-05-08T23:21:04.117976+0000","last_active":"2024-05-08T23:21:05.356830+0000","last_peered":"2024-05-08T23:21:05.356830+0000","last_clean":"2024-05-08T23:21:05.356830+0000","last_became_active":"2024-05-08T23:21:04.117682+0000","last_became_peered":"2024-05-08T23:21:04.117682+0000","last_unstale":"2024-05-08T23:21:05.356830+0000","last_undegraded":"2024-05-08T23:21:05.356830+0000","last_fullsized":"2024-05-08T23:21:05.356830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:30:28.180773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355265+0000","last_change":"2024-05-08T23:21:04.107601+0000","last_active":"2024-05-08T23:21:05.355265+0000","last_peered":"2024-05-08T23:21:05.355265+0000","last_clean":"2024-05-08T23:21:05.355265+0000","last_became_active":"2024-05-08T23:21:04.107309+0000","last_became_peered":"2024-05-08T23:21:04.107309+0000","last_unstale":"2024-05-08T23:21:05.355265+0000","last_undegraded":"2024-05-08T23:21:05.355265+0000","last_fullsized":"2024-05-08T23:21:05.355265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354968+0000","last_change":"2024-05-08T23:21:04.106808+0000","last_active":"2024-05-08T23:21:05.354968+0000","last_peered":"2024-05-08T23:21:05.354968+0000","last_clean":"2024-05-08T23:21:05.354968+0000","last_became_active":"2024-05-08T23:21:04.106628+0000","last_became_peered":"2024-05-08T23:21:04.106628+0000","last_unstale":"2024-05-08T23:21:05.354968+0000","last_undegraded":"2024-05-08T23:21:05.354968+0000","last_fullsized":"2024-05-08T23:21:05.354968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.433453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356256+0000","last_change":"2024-05-08T23:21:04.105100+0000","last_active":"2024-05-08T23:21:05.356256+0000","last_peered":"2024-05-08T23:21:05.356256+0000","last_clean":"2024-05-08T23:21:05.356256+0000","last_became_active":"2024-05-08T23:21:04.104785+0000","last_became_peered":"2024-05-08T23:21:04.104785+0000","last_unstale":"2024-05-08T23:21:05.356256+0000","last_undegraded":"2024-05-08T23:21:05.356256+0000","last_fullsized":"2024-05-08T23:21:05.356256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:19.795476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355441+0000","last_change":"2024-05-08T23:21:04.105145+0000","last_active":"2024-05-08T23:21:05.355441+0000","last_peered":"2024-05-08T23:21:05.355441+0000","last_clean":"2024-05-08T23:21:05.355441+0000","last_became_active":"2024-05-08T23:21:04.104842+0000","last_became_peered":"2024-05-08T23:21:04.104842+0000","last_unstale":"2024-05-08T23:21:05.355441+0000","last_undegraded":"2024-05-08T23:21:05.355441+0000","last_fullsized":"2024-05-08T23:21:05.355441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092800+0000","last_change":"2024-05-08T23:21:04.106935+0000","last_active":"2024-05-08T23:21:05.092800+0000","last_peered":"2024-05-08T23:21:05.092800+0000","last_clean":"2024-05-08T23:21:05.092800+0000","last_became_active":"2024-05-08T23:21:04.106671+0000","last_became_peered":"2024-05-08T23:21:04.106671+0000","last_unstale":"2024-05-08T23:21:05.092800+0000","last_undegraded":"2024-05-08T23:21:05.092800+0000","last_fullsized":"2024-05-08T23:21:05.092800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:07.596384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093704+0000","last_change":"2024-05-08T23:21:04.112023+0000","last_active":"2024-05-08T23:21:05.093704+0000","last_peered":"2024-05-08T23:21:05.093704+0000","last_clean":"2024-05-08T23:21:05.093704+0000","last_became_active":"2024-05-08T23:21:04.111819+0000","last_became_peered":"2024-05-08T23:21:04.111819+0000","last_unstale":"2024-05-08T23:21:05.093704+0000","last_undegraded":"2024-05-08T23:21:05.093704+0000","last_fullsized":"2024-05-08T23:21:05.093704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355844+0000","last_change":"2024-05-08T23:21:04.111963+0000","last_active":"2024-05-08T23:21:05.355844+0000","last_peered":"2024-05-08T23:21:05.355844+0000","last_clean":"2024-05-08T23:21:05.355844+0000","last_became_active":"2024-05-08T23:21:04.111629+0000","last_became_peered":"2024-05-08T23:21:04.111629+0000","last_unstale":"2024-05-08T23:21:05.355844+0000","last_undegraded":"2024-05-08T23:21:05.355844+0000","last_fullsized":"2024-05-08T23:21:05.355844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092931+0000","last_change":"2024-05-08T23:21:04.113608+0000","last_active":"2024-05-08T23:21:05.092931+0000","last_peered":"2024-05-08T23:21:05.092931+0000","last_clean":"2024-05-08T23:21:05.092931+0000","last_became_active":"2024-05-08T23:21:04.113322+0000","last_became_peered":"2024-05-08T23:21:04.113322+0000","last_unstale":"2024-05-08T23:21:05.092931+0000","last_undegraded":"2024-05-08T23:21:05.092931+0000","last_fullsized":"2024-05-08T23:21:05.092931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356052+0000","last_change":"2024-05-08T23:21:04.109658+0000","last_active":"2024-05-08T23:21:05.356052+0000","last_peered":"2024-05-08T23:21:05.356052+0000","last_clean":"2024-05-08T23:21:05.356052+0000","last_became_active":"2024-05-08T23:21:04.109222+0000","last_became_peered":"2024-05-08T23:21:04.109222+0000","last_unstale":"2024-05-08T23:21:05.356052+0000","last_undegraded":"2024-05-08T23:21:05.356052+0000","last_fullsized":"2024-05-08T23:21:05.356052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092319+0000","last_change":"2024-05-08T23:21:04.107408+0000","last_active":"2024-05-08T23:21:05.092319+0000","last_peered":"2024-05-08T23:21:05.092319+0000","last_clean":"2024-05-08T23:21:05.092319+0000","last_became_active":"2024-05-08T23:21:04.107123+0000","last_became_peered":"2024-05-08T23:21:04.107123+0000","last_unstale":"2024-05-08T23:21:05.092319+0000","last_undegraded":"2024-05-08T23:21:05.092319+0000","last_fullsized":"2024-05-08T23:21:05.092319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356155+0000","last_change":"2024-05-08T23:21:04.108500+0000","last_active":"2024-05-08T23:21:05.356155+0000","last_peered":"2024-05-08T23:21:05.356155+0000","last_clean":"2024-05-08T23:21:05.356155+0000","last_became_active":"2024-05-08T23:21:04.107974+0000","last_became_peered":"2024-05-08T23:21:04.107974+0000","last_unstale":"2024-05-08T23:21:05.356155+0000","last_undegraded":"2024-05-08T23:21:05.356155+0000","last_fullsized":"2024-05-08T23:21:05.356155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355333+0000","last_change":"2024-05-08T23:21:04.112817+0000","last_active":"2024-05-08T23:21:05.355333+0000","last_peered":"2024-05-08T23:21:05.355333+0000","last_clean":"2024-05-08T23:21:05.355333+0000","last_became_active":"2024-05-08T23:21:04.112519+0000","last_became_peered":"2024-05-08T23:21:04.112519+0000","last_unstale":"2024-05-08T23:21:05.355333+0000","last_undegraded":"2024-05-08T23:21:05.355333+0000","last_fullsized":"2024-05-08T23:21:05.355333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356539+0000","last_change":"2024-05-08T23:21:04.113697+0000","last_active":"2024-05-08T23:21:05.356539+0000","last_peered":"2024-05-08T23:21:05.356539+0000","last_clean":"2024-05-08T23:21:05.356539+0000","last_became_active":"2024-05-08T23:21:04.113363+0000","last_became_peered":"2024-05-08T23:21:04.113363+0000","last_unstale":"2024-05-08T23:21:05.356539+0000","last_undegraded":"2024-05-08T23:21:05.356539+0000","last_fullsized":"2024-05-08T23:21:05.356539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:21:18.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:21:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.328+0000 7fb06814d700 1 Processor -- start 2024-05-08T23:21:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.328+0000 7fb06814d700 1 -- start start 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb06814d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb06015db00 0x7fb0600cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb06814d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06006f990 0x7fb06006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb06814d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06006ad20 con 0x7fb0600caa50 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb06814d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06015dee0 con 0x7fb06015db00 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb06814d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600ccad0 con 0x7fb06006f990 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb0666ea700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06006f990 0x7fb06006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb065ee9700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb06015db00 0x7fb0600cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb0656e8700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb0600caa50 0x7fb06006f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:46178/0 (socket says 172.21.15.5:46178) 2024-05-08T23:21:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb0656e8700 1 -- 172.21.15.5:0/1405853437 learned_addr learned my addr 172.21.15.5:0/1405853437 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb064ee7700 1 -- 172.21.15.5:0/1405853437 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022456365 0 0) 0x7fb06006ad20 con 0x7fb0600caa50 2024-05-08T23:21:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb064ee7700 1 -- 172.21.15.5:0/1405853437 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb04c0035b0 con 0x7fb0600caa50 2024-05-08T23:21:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb0666ea700 1 -- 172.21.15.5:0/1405853437 >> v2:172.21.15.49:3300/0 conn(0x7fb06015db00 msgr2=0x7fb0600cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.329+0000 7fb0666ea700 1 --2- 172.21.15.5:0/1405853437 >> v2:172.21.15.49:3300/0 conn(0x7fb06015db00 0x7fb0600cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb0666ea700 1 -- 172.21.15.5:0/1405853437 >> v1:172.21.15.49:6789/0 conn(0x7fb0600caa50 legacy=0x7fb06006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb0666ea700 1 -- 172.21.15.5:0/1405853437 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06006b000 con 0x7fb06006f990 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb065ee9700 1 --2- 172.21.15.5:0/1405853437 >> v2:172.21.15.49:3300/0 conn(0x7fb06015db00 0x7fb0600cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb0666ea700 1 --2- 172.21.15.5:0/1405853437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06006f990 0x7fb06006e320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb058009d90 tx=0x7fb05800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba08c1f0c1c92eae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb064ee7700 1 -- 172.21.15.5:0/1405853437 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800e070 con 0x7fb06006f990 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb064ee7700 1 -- 172.21.15.5:0/1405853437 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05800bb20 con 0x7fb06006f990 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb064ee7700 1 -- 172.21.15.5:0/1405853437 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058012430 con 0x7fb06006f990 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb06814d700 1 -- 172.21.15.5:0/1405853437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06006f990 msgr2=0x7fb06006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.330+0000 7fb06814d700 1 --2- 172.21.15.5:0/1405853437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06006f990 0x7fb06006e320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb058009d90 tx=0x7fb05800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.331+0000 7fb06814d700 1 -- 172.21.15.5:0/1405853437 shutdown_connections 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.331+0000 7fb06814d700 1 --2- 172.21.15.5:0/1405853437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06006f990 0x7fb06006e320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.331+0000 7fb06814d700 1 --2- 172.21.15.5:0/1405853437 >> v2:172.21.15.49:3300/0 conn(0x7fb06015db00 0x7fb0600cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.331+0000 7fb06814d700 1 -- 172.21.15.5:0/1405853437 >> 172.21.15.5:0/1405853437 conn(0x7fb060099a50 msgr2=0x7fb0600d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.331+0000 7fb06814d700 1 -- 172.21.15.5:0/1405853437 shutdown_connections 2024-05-08T23:21:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.331+0000 7fb06814d700 1 -- 172.21.15.5:0/1405853437 wait complete. 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.332+0000 7fb06814d700 1 Processor -- start 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.332+0000 7fb06814d700 1 -- start start 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb06814d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb06006f990 0x7fb0600c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb06814d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0600caa50 0x7fb060153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb06814d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06015db00 0x7fb060153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb06814d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600b47e0 con 0x7fb0600caa50 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb06814d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600b3bd0 con 0x7fb06006f990 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb06814d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600b3eb0 con 0x7fb06015db00 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0600caa50 0x7fb060153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0600caa50 0x7fb060153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52752/0 (socket says 172.21.15.5:52752) 2024-05-08T23:21:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 -- 172.21.15.5:0/4153539593 learned_addr learned my addr 172.21.15.5:0/4153539593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb065ee9700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb06006f990 0x7fb0600c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0666ea700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06015db00 0x7fb060153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 -- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06015db00 msgr2=0x7fb060153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06015db00 0x7fb060153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 -- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb06006f990 msgr2=0x7fb0600c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb06006f990 0x7fb0600c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 -- 172.21.15.5:0/4153539593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c0094a0 con 0x7fb0600caa50 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.333+0000 7fb0656e8700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0600caa50 0x7fb060153510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c700 tx=0x7fb05000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.334+0000 7fb0567fc700 1 -- 172.21.15.5:0/4153539593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050005d70 con 0x7fb0600caa50 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.334+0000 7fb0567fc700 1 -- 172.21.15.5:0/4153539593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0500070d0 con 0x7fb0600caa50 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.334+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb058009a70 con 0x7fb0600caa50 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.334+0000 7fb0567fc700 1 -- 172.21.15.5:0/4153539593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050007460 con 0x7fb0600caa50 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.334+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0600b4470 con 0x7fb0600caa50 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.335+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0600b4600 con 0x7fb0600caa50 2024-05-08T23:21:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.335+0000 7fb0567fc700 1 -- 172.21.15.5:0/4153539593 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb0500075c0 con 0x7fb0600caa50 2024-05-08T23:21:18.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.335+0000 7fb0567fc700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb04406ac90 0x7fb04406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.336+0000 7fb0567fc700 1 -- 172.21.15.5:0/4153539593 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb050088990 con 0x7fb0600caa50 2024-05-08T23:21:18.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.336+0000 7fb065ee9700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb04406ac90 0x7fb04406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.336+0000 7fb065ee9700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb04406ac90 0x7fb04406d150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009cd0 tx=0x7fb05c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:18.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.340+0000 7fb0567fc700 1 -- 172.21.15.5:0/4153539593 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb050053a50 con 0x7fb0600caa50 2024-05-08T23:21:18.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.553+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0600d1170 con 0x7fb04406ac90 2024-05-08T23:21:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.562+0000 7fb0567fc700 1 -- 172.21.15.5:0/4153539593 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264534 (secure 0 0 0) 0x7fb05c009e10 con 0x7fb04406ac90 2024-05-08T23:21:18.563 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb04406ac90 msgr2=0x7fb04406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb04406ac90 0x7fb04406d150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009cd0 tx=0x7fb05c009210 comp rx=0 tx=0).stop 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0600caa50 msgr2=0x7fb060153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0600caa50 0x7fb060153510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c700 tx=0x7fb05000ca10 comp rx=0 tx=0).stop 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 shutdown_connections 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb04406ac90 0x7fb04406d150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb06015db00 0x7fb060153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb06006f990 0x7fb0600c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 --2- 172.21.15.5:0/4153539593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0600caa50 0x7fb060153510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 >> 172.21.15.5:0/4153539593 conn(0x7fb060099a50 msgr2=0x7fb06015b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 shutdown_connections 2024-05-08T23:21:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.565+0000 7fb06814d700 1 -- 172.21.15.5:0/4153539593 wait complete. 2024-05-08T23:21:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:21:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-08T23:21:17.296700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003154"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356525+0000","last_change":"2024-05-08T23:21:04.117587+0000","last_active":"2024-05-08T23:21:05.356525+0000","last_peered":"2024-05-08T23:21:05.356525+0000","last_clean":"2024-05-08T23:21:05.356525+0000","last_became_active":"2024-05-08T23:21:04.117296+0000","last_became_peered":"2024-05-08T23:21:04.117296+0000","last_unstale":"2024-05-08T23:21:05.356525+0000","last_undegraded":"2024-05-08T23:21:05.356525+0000","last_fullsized":"2024-05-08T23:21:05.356525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093945+0000","last_change":"2024-05-08T23:21:04.113872+0000","last_active":"2024-05-08T23:21:05.093945+0000","last_peered":"2024-05-08T23:21:05.093945+0000","last_clean":"2024-05-08T23:21:05.093945+0000","last_became_active":"2024-05-08T23:21:04.113525+0000","last_became_peered":"2024-05-08T23:21:04.113525+0000","last_unstale":"2024-05-08T23:21:05.093945+0000","last_undegraded":"2024-05-08T23:21:05.093945+0000","last_fullsized":"2024-05-08T23:21:05.093945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356124+0000","last_change":"2024-05-08T23:21:04.100510+0000","last_active":"2024-05-08T23:21:05.356124+0000","last_peered":"2024-05-08T23:21:05.356124+0000","last_clean":"2024-05-08T23:21:05.356124+0000","last_became_active":"2024-05-08T23:21:04.100227+0000","last_became_peered":"2024-05-08T23:21:04.100227+0000","last_unstale":"2024-05-08T23:21:05.356124+0000","last_undegraded":"2024-05-08T23:21:05.356124+0000","last_fullsized":"2024-05-08T23:21:05.356124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.368136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356793+0000","last_change":"2024-05-08T23:21:04.110918+0000","last_active":"2024-05-08T23:21:05.356793+0000","last_peered":"2024-05-08T23:21:05.356793+0000","last_clean":"2024-05-08T23:21:05.356793+0000","last_became_active":"2024-05-08T23:21:04.110450+0000","last_became_peered":"2024-05-08T23:21:04.110450+0000","last_unstale":"2024-05-08T23:21:05.356793+0000","last_undegraded":"2024-05-08T23:21:05.356793+0000","last_fullsized":"2024-05-08T23:21:05.356793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356803+0000","last_change":"2024-05-08T23:21:04.113276+0000","last_active":"2024-05-08T23:21:05.356803+0000","last_peered":"2024-05-08T23:21:05.356803+0000","last_clean":"2024-05-08T23:21:05.356803+0000","last_became_active":"2024-05-08T23:21:04.112995+0000","last_became_peered":"2024-05-08T23:21:04.112995+0000","last_unstale":"2024-05-08T23:21:05.356803+0000","last_undegraded":"2024-05-08T23:21:05.356803+0000","last_fullsized":"2024-05-08T23:21:05.356803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356039+0000","last_change":"2024-05-08T23:21:04.106788+0000","last_active":"2024-05-08T23:21:05.356039+0000","last_peered":"2024-05-08T23:21:05.356039+0000","last_clean":"2024-05-08T23:21:05.356039+0000","last_became_active":"2024-05-08T23:21:04.106459+0000","last_became_peered":"2024-05-08T23:21:04.106459+0000","last_unstale":"2024-05-08T23:21:05.356039+0000","last_undegraded":"2024-05-08T23:21:05.356039+0000","last_fullsized":"2024-05-08T23:21:05.356039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356576+0000","last_change":"2024-05-08T23:21:04.112766+0000","last_active":"2024-05-08T23:21:05.356576+0000","last_peered":"2024-05-08T23:21:05.356576+0000","last_clean":"2024-05-08T23:21:05.356576+0000","last_became_active":"2024-05-08T23:21:04.112455+0000","last_became_peered":"2024-05-08T23:21:04.112455+0000","last_unstale":"2024-05-08T23:21:05.356576+0000","last_undegraded":"2024-05-08T23:21:05.356576+0000","last_fullsized":"2024-05-08T23:21:05.356576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:16.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092125+0000","last_change":"2024-05-08T23:21:04.103185+0000","last_active":"2024-05-08T23:21:05.092125+0000","last_peered":"2024-05-08T23:21:05.092125+0000","last_clean":"2024-05-08T23:21:05.092125+0000","last_became_active":"2024-05-08T23:21:04.102828+0000","last_became_peered":"2024-05-08T23:21:04.102828+0000","last_unstale":"2024-05-08T23:21:05.092125+0000","last_undegraded":"2024-05-08T23:21:05.092125+0000","last_fullsized":"2024-05-08T23:21:05.092125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356537+0000","last_change":"2024-05-08T23:21:04.122716+0000","last_active":"2024-05-08T23:21:05.356537+0000","last_peered":"2024-05-08T23:21:05.356537+0000","last_clean":"2024-05-08T23:21:05.356537+0000","last_became_active":"2024-05-08T23:21:04.122410+0000","last_became_peered":"2024-05-08T23:21:04.122410+0000","last_unstale":"2024-05-08T23:21:05.356537+0000","last_undegraded":"2024-05-08T23:21:05.356537+0000","last_fullsized":"2024-05-08T23:21:05.356537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:42.560490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355980+0000","last_change":"2024-05-08T23:21:04.111916+0000","last_active":"2024-05-08T23:21:05.355980+0000","last_peered":"2024-05-08T23:21:05.355980+0000","last_clean":"2024-05-08T23:21:05.355980+0000","last_became_active":"2024-05-08T23:21:04.111520+0000","last_became_peered":"2024-05-08T23:21:04.111520+0000","last_unstale":"2024-05-08T23:21:05.355980+0000","last_undegraded":"2024-05-08T23:21:05.355980+0000","last_fullsized":"2024-05-08T23:21:05.355980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355636+0000","last_change":"2024-05-08T23:21:04.113128+0000","last_active":"2024-05-08T23:21:05.355636+0000","last_peered":"2024-05-08T23:21:05.355636+0000","last_clean":"2024-05-08T23:21:05.355636+0000","last_became_active":"2024-05-08T23:21:04.112746+0000","last_became_peered":"2024-05-08T23:21:04.112746+0000","last_unstale":"2024-05-08T23:21:05.355636+0000","last_undegraded":"2024-05-08T23:21:05.355636+0000","last_fullsized":"2024-05-08T23:21:05.355636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:07.596384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093836+0000","last_change":"2024-05-08T23:21:04.118631+0000","last_active":"2024-05-08T23:21:05.093836+0000","last_peered":"2024-05-08T23:21:05.093836+0000","last_clean":"2024-05-08T23:21:05.093836+0000","last_became_active":"2024-05-08T23:21:04.118285+0000","last_became_peered":"2024-05-08T23:21:04.118285+0000","last_unstale":"2024-05-08T23:21:05.093836+0000","last_undegraded":"2024-05-08T23:21:05.093836+0000","last_fullsized":"2024-05-08T23:21:05.093836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.260488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092563+0000","last_change":"2024-05-08T23:21:04.114717+0000","last_active":"2024-05-08T23:21:05.092563+0000","last_peered":"2024-05-08T23:21:05.092563+0000","last_clean":"2024-05-08T23:21:05.092563+0000","last_became_active":"2024-05-08T23:21:04.114456+0000","last_became_peered":"2024-05-08T23:21:04.114456+0000","last_unstale":"2024-05-08T23:21:05.092563+0000","last_undegraded":"2024-05-08T23:21:05.092563+0000","last_fullsized":"2024-05-08T23:21:05.092563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.119343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355161+0000","last_change":"2024-05-08T23:21:04.110105+0000","last_active":"2024-05-08T23:21:05.355161+0000","last_peered":"2024-05-08T23:21:05.355161+0000","last_clean":"2024-05-08T23:21:05.355161+0000","last_became_active":"2024-05-08T23:21:04.109818+0000","last_became_peered":"2024-05-08T23:21:04.109818+0000","last_unstale":"2024-05-08T23:21:05.355161+0000","last_undegraded":"2024-05-08T23:21:05.355161+0000","last_fullsized":"2024-05-08T23:21:05.355161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093437+0000","last_change":"2024-05-08T23:21:04.114173+0000","last_active":"2024-05-08T23:21:05.093437+0000","last_peered":"2024-05-08T23:21:05.093437+0000","last_clean":"2024-05-08T23:21:05.093437+0000","last_became_active":"2024-05-08T23:21:04.108376+0000","last_became_peered":"2024-05-08T23:21:04.108376+0000","last_unstale":"2024-05-08T23:21:05.093437+0000","last_undegraded":"2024-05-08T23:21:05.093437+0000","last_fullsized":"2024-05-08T23:21:05.093437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355476+0000","last_change":"2024-05-08T23:21:04.100458+0000","last_active":"2024-05-08T23:21:05.355476+0000","last_peered":"2024-05-08T23:21:05.355476+0000","last_clean":"2024-05-08T23:21:05.355476+0000","last_became_active":"2024-05-08T23:21:04.100166+0000","last_became_peered":"2024-05-08T23:21:04.100166+0000","last_unstale":"2024-05-08T23:21:05.355476+0000","last_undegraded":"2024-05-08T23:21:05.355476+0000","last_fullsized":"2024-05-08T23:21:05.355476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357377+0000","last_change":"2024-05-08T23:21:04.109204+0000","last_active":"2024-05-08T23:21:05.357377+0000","last_peered":"2024-05-08T23:21:05.357377+0000","last_clean":"2024-05-08T23:21:05.357377+0000","last_became_active":"2024-05-08T23:21:04.108507+0000","last_became_peered":"2024-05-08T23:21:04.108507+0000","last_unstale":"2024-05-08T23:21:05.357377+0000","last_undegraded":"2024-05-08T23:21:05.357377+0000","last_fullsized":"2024-05-08T23:21:05.357377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:16.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355050+0000","last_change":"2024-05-08T23:21:04.114847+0000","last_active":"2024-05-08T23:21:05.355050+0000","last_peered":"2024-05-08T23:21:05.355050+0000","last_clean":"2024-05-08T23:21:05.355050+0000","last_became_active":"2024-05-08T23:21:04.114090+0000","last_became_peered":"2024-05-08T23:21:04.114090+0000","last_unstale":"2024-05-08T23:21:05.355050+0000","last_undegraded":"2024-05-08T23:21:05.355050+0000","last_fullsized":"2024-05-08T23:21:05.355050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.260488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356040+0000","last_change":"2024-05-08T23:21:04.113465+0000","last_active":"2024-05-08T23:21:05.356040+0000","last_peered":"2024-05-08T23:21:05.356040+0000","last_clean":"2024-05-08T23:21:05.356040+0000","last_became_active":"2024-05-08T23:21:04.113179+0000","last_became_peered":"2024-05-08T23:21:04.113179+0000","last_unstale":"2024-05-08T23:21:05.356040+0000","last_undegraded":"2024-05-08T23:21:05.356040+0000","last_fullsized":"2024-05-08T23:21:05.356040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355379+0000","last_change":"2024-05-08T23:21:03.646371+0000","last_active":"2024-05-08T23:21:05.355379+0000","last_peered":"2024-05-08T23:21:05.355379+0000","last_clean":"2024-05-08T23:21:05.355379+0000","last_became_active":"2024-05-08T23:21:03.646004+0000","last_became_peered":"2024-05-08T23:21:03.646004+0000","last_unstale":"2024-05-08T23:21:05.355379+0000","last_undegraded":"2024-05-08T23:21:05.355379+0000","last_fullsized":"2024-05-08T23:21:05.355379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:32.482556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092991+0000","last_change":"2024-05-08T23:21:04.115651+0000","last_active":"2024-05-08T23:21:05.092991+0000","last_peered":"2024-05-08T23:21:05.092991+0000","last_clean":"2024-05-08T23:21:05.092991+0000","last_became_active":"2024-05-08T23:21:04.115384+0000","last_became_peered":"2024-05-08T23:21:04.115384+0000","last_unstale":"2024-05-08T23:21:05.092991+0000","last_undegraded":"2024-05-08T23:21:05.092991+0000","last_fullsized":"2024-05-08T23:21:05.092991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093060+0000","last_change":"2024-05-08T23:21:03.645350+0000","last_active":"2024-05-08T23:21:05.093060+0000","last_peered":"2024-05-08T23:21:05.093060+0000","last_clean":"2024-05-08T23:21:05.093060+0000","last_became_active":"2024-05-08T23:21:03.645081+0000","last_became_peered":"2024-05-08T23:21:03.645081+0000","last_unstale":"2024-05-08T23:21:05.093060+0000","last_undegraded":"2024-05-08T23:21:05.093060+0000","last_fullsized":"2024-05-08T23:21:05.093060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355362+0000","last_change":"2024-05-08T23:21:04.112500+0000","last_active":"2024-05-08T23:21:05.355362+0000","last_peered":"2024-05-08T23:21:05.355362+0000","last_clean":"2024-05-08T23:21:05.355362+0000","last_became_active":"2024-05-08T23:21:04.111988+0000","last_became_peered":"2024-05-08T23:21:04.111988+0000","last_unstale":"2024-05-08T23:21:05.355362+0000","last_undegraded":"2024-05-08T23:21:05.355362+0000","last_fullsized":"2024-05-08T23:21:05.355362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356422+0000","last_change":"2024-05-08T23:21:03.647784+0000","last_active":"2024-05-08T23:21:05.356422+0000","last_peered":"2024-05-08T23:21:05.356422+0000","last_clean":"2024-05-08T23:21:05.356422+0000","last_became_active":"2024-05-08T23:21:03.646667+0000","last_became_peered":"2024-05-08T23:21:03.646667+0000","last_unstale":"2024-05-08T23:21:05.356422+0000","last_undegraded":"2024-05-08T23:21:05.356422+0000","last_fullsized":"2024-05-08T23:21:05.356422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355900+0000","last_change":"2024-05-08T23:21:04.104612+0000","last_active":"2024-05-08T23:21:05.355900+0000","last_peered":"2024-05-08T23:21:05.355900+0000","last_clean":"2024-05-08T23:21:05.355900+0000","last_became_active":"2024-05-08T23:21:04.104304+0000","last_became_peered":"2024-05-08T23:21:04.104304+0000","last_unstale":"2024-05-08T23:21:05.355900+0000","last_undegraded":"2024-05-08T23:21:05.355900+0000","last_fullsized":"2024-05-08T23:21:05.355900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356075+0000","last_change":"2024-05-08T23:21:03.648270+0000","last_active":"2024-05-08T23:21:05.356075+0000","last_peered":"2024-05-08T23:21:05.356075+0000","last_clean":"2024-05-08T23:21:05.356075+0000","last_became_active":"2024-05-08T23:21:03.647784+0000","last_became_peered":"2024-05-08T23:21:03.647784+0000","last_unstale":"2024-05-08T23:21:05.356075+0000","last_undegraded":"2024-05-08T23:21:05.356075+0000","last_fullsized":"2024-05-08T23:21:05.356075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357179+0000","last_change":"2024-05-08T23:21:04.106021+0000","last_active":"2024-05-08T23:21:05.357179+0000","last_peered":"2024-05-08T23:21:05.357179+0000","last_clean":"2024-05-08T23:21:05.357179+0000","last_became_active":"2024-05-08T23:21:04.105614+0000","last_became_peered":"2024-05-08T23:21:04.105614+0000","last_unstale":"2024-05-08T23:21:05.357179+0000","last_undegraded":"2024-05-08T23:21:05.357179+0000","last_fullsized":"2024-05-08T23:21:05.357179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.119343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092611+0000","last_change":"2024-05-08T23:21:03.642600+0000","last_active":"2024-05-08T23:21:05.092611+0000","last_peered":"2024-05-08T23:21:05.092611+0000","last_clean":"2024-05-08T23:21:05.092611+0000","last_became_active":"2024-05-08T23:21:03.642234+0000","last_became_peered":"2024-05-08T23:21:03.642234+0000","last_unstale":"2024-05-08T23:21:05.092611+0000","last_undegraded":"2024-05-08T23:21:05.092611+0000","last_fullsized":"2024-05-08T23:21:05.092611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093418+0000","last_change":"2024-05-08T23:21:04.117783+0000","last_active":"2024-05-08T23:21:05.093418+0000","last_peered":"2024-05-08T23:21:05.093418+0000","last_clean":"2024-05-08T23:21:05.093418+0000","last_became_active":"2024-05-08T23:21:04.117324+0000","last_became_peered":"2024-05-08T23:21:04.117324+0000","last_unstale":"2024-05-08T23:21:05.093418+0000","last_undegraded":"2024-05-08T23:21:05.093418+0000","last_fullsized":"2024-05-08T23:21:05.093418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.433453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355523+0000","last_change":"2024-05-08T23:21:03.644641+0000","last_active":"2024-05-08T23:21:05.355523+0000","last_peered":"2024-05-08T23:21:05.355523+0000","last_clean":"2024-05-08T23:21:05.355523+0000","last_became_active":"2024-05-08T23:21:03.644361+0000","last_became_peered":"2024-05-08T23:21:03.644361+0000","last_unstale":"2024-05-08T23:21:05.355523+0000","last_undegraded":"2024-05-08T23:21:05.355523+0000","last_fullsized":"2024-05-08T23:21:05.355523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355562+0000","last_change":"2024-05-08T23:21:04.105553+0000","last_active":"2024-05-08T23:21:05.355562+0000","last_peered":"2024-05-08T23:21:05.355562+0000","last_clean":"2024-05-08T23:21:05.355562+0000","last_became_active":"2024-05-08T23:21:04.105041+0000","last_became_peered":"2024-05-08T23:21:04.105041+0000","last_unstale":"2024-05-08T23:21:05.355562+0000","last_undegraded":"2024-05-08T23:21:05.355562+0000","last_fullsized":"2024-05-08T23:21:05.355562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.311829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092401+0000","last_change":"2024-05-08T23:21:03.639910+0000","last_active":"2024-05-08T23:21:05.092401+0000","last_peered":"2024-05-08T23:21:05.092401+0000","last_clean":"2024-05-08T23:21:05.092401+0000","last_became_active":"2024-05-08T23:21:03.638934+0000","last_became_peered":"2024-05-08T23:21:03.638934+0000","last_unstale":"2024-05-08T23:21:05.092401+0000","last_undegraded":"2024-05-08T23:21:05.092401+0000","last_fullsized":"2024-05-08T23:21:05.092401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092248+0000","last_change":"2024-05-08T23:21:04.108315+0000","last_active":"2024-05-08T23:21:05.092248+0000","last_peered":"2024-05-08T23:21:05.092248+0000","last_clean":"2024-05-08T23:21:05.092248+0000","last_became_active":"2024-05-08T23:21:04.108040+0000","last_became_peered":"2024-05-08T23:21:04.108040+0000","last_unstale":"2024-05-08T23:21:05.092248+0000","last_undegraded":"2024-05-08T23:21:05.092248+0000","last_fullsized":"2024-05-08T23:21:05.092248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:58.191543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355919+0000","last_change":"2024-05-08T23:21:03.641788+0000","last_active":"2024-05-08T23:21:05.355919+0000","last_peered":"2024-05-08T23:21:05.355919+0000","last_clean":"2024-05-08T23:21:05.355919+0000","last_became_active":"2024-05-08T23:21:03.641004+0000","last_became_peered":"2024-05-08T23:21:03.641004+0000","last_unstale":"2024-05-08T23:21:05.355919+0000","last_undegraded":"2024-05-08T23:21:05.355919+0000","last_fullsized":"2024-05-08T23:21:05.355919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093537+0000","last_change":"2024-05-08T23:21:04.114572+0000","last_active":"2024-05-08T23:21:05.093537+0000","last_peered":"2024-05-08T23:21:05.093537+0000","last_clean":"2024-05-08T23:21:05.093537+0000","last_became_active":"2024-05-08T23:21:04.114402+0000","last_became_peered":"2024-05-08T23:21:04.114402+0000","last_unstale":"2024-05-08T23:21:05.093537+0000","last_undegraded":"2024-05-08T23:21:05.093537+0000","last_fullsized":"2024-05-08T23:21:05.093537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356964+0000","last_change":"2024-05-08T23:21:03.648330+0000","last_active":"2024-05-08T23:21:05.356964+0000","last_peered":"2024-05-08T23:21:05.356964+0000","last_clean":"2024-05-08T23:21:05.356964+0000","last_became_active":"2024-05-08T23:21:03.648033+0000","last_became_peered":"2024-05-08T23:21:03.648033+0000","last_unstale":"2024-05-08T23:21:05.356964+0000","last_undegraded":"2024-05-08T23:21:05.356964+0000","last_fullsized":"2024-05-08T23:21:05.356964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356917+0000","last_change":"2024-05-08T23:21:04.108163+0000","last_active":"2024-05-08T23:21:05.356917+0000","last_peered":"2024-05-08T23:21:05.356917+0000","last_clean":"2024-05-08T23:21:05.356917+0000","last_became_active":"2024-05-08T23:21:04.107854+0000","last_became_peered":"2024-05-08T23:21:04.107854+0000","last_unstale":"2024-05-08T23:21:05.356917+0000","last_undegraded":"2024-05-08T23:21:05.356917+0000","last_fullsized":"2024-05-08T23:21:05.356917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:09.028590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356841+0000","last_change":"2024-05-08T23:21:03.640432+0000","last_active":"2024-05-08T23:21:05.356841+0000","last_peered":"2024-05-08T23:21:05.356841+0000","last_clean":"2024-05-08T23:21:05.356841+0000","last_became_active":"2024-05-08T23:21:03.640121+0000","last_became_peered":"2024-05-08T23:21:03.640121+0000","last_unstale":"2024-05-08T23:21:05.356841+0000","last_undegraded":"2024-05-08T23:21:05.356841+0000","last_fullsized":"2024-05-08T23:21:05.356841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355475+0000","last_change":"2024-05-08T23:21:04.113136+0000","last_active":"2024-05-08T23:21:05.355475+0000","last_peered":"2024-05-08T23:21:05.355475+0000","last_clean":"2024-05-08T23:21:05.355475+0000","last_became_active":"2024-05-08T23:21:04.112831+0000","last_became_peered":"2024-05-08T23:21:04.112831+0000","last_unstale":"2024-05-08T23:21:05.355475+0000","last_undegraded":"2024-05-08T23:21:05.355475+0000","last_fullsized":"2024-05-08T23:21:05.355475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:03.946264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093793+0000","last_change":"2024-05-08T23:21:03.647271+0000","last_active":"2024-05-08T23:21:05.093793+0000","last_peered":"2024-05-08T23:21:05.093793+0000","last_clean":"2024-05-08T23:21:05.093793+0000","last_became_active":"2024-05-08T23:21:03.646992+0000","last_became_peered":"2024-05-08T23:21:03.646992+0000","last_unstale":"2024-05-08T23:21:05.093793+0000","last_undegraded":"2024-05-08T23:21:05.093793+0000","last_fullsized":"2024-05-08T23:21:05.093793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:43.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093761+0000","last_change":"2024-05-08T23:21:04.106622+0000","last_active":"2024-05-08T23:21:05.093761+0000","last_peered":"2024-05-08T23:21:05.093761+0000","last_clean":"2024-05-08T23:21:05.093761+0000","last_became_active":"2024-05-08T23:21:04.106248+0000","last_became_peered":"2024-05-08T23:21:04.106248+0000","last_unstale":"2024-05-08T23:21:05.093761+0000","last_undegraded":"2024-05-08T23:21:05.093761+0000","last_fullsized":"2024-05-08T23:21:05.093761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.311829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356428+0000","last_change":"2024-05-08T23:21:03.642565+0000","last_active":"2024-05-08T23:21:05.356428+0000","last_peered":"2024-05-08T23:21:05.356428+0000","last_clean":"2024-05-08T23:21:05.356428+0000","last_became_active":"2024-05-08T23:21:03.642253+0000","last_became_peered":"2024-05-08T23:21:03.642253+0000","last_unstale":"2024-05-08T23:21:05.356428+0000","last_undegraded":"2024-05-08T23:21:05.356428+0000","last_fullsized":"2024-05-08T23:21:05.356428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356171+0000","last_change":"2024-05-08T23:21:04.113888+0000","last_active":"2024-05-08T23:21:05.356171+0000","last_peered":"2024-05-08T23:21:05.356171+0000","last_clean":"2024-05-08T23:21:05.356171+0000","last_became_active":"2024-05-08T23:21:04.113607+0000","last_became_peered":"2024-05-08T23:21:04.113607+0000","last_unstale":"2024-05-08T23:21:05.356171+0000","last_undegraded":"2024-05-08T23:21:05.356171+0000","last_fullsized":"2024-05-08T23:21:05.356171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:19.795476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355509+0000","last_change":"2024-05-08T23:21:03.641751+0000","last_active":"2024-05-08T23:21:05.355509+0000","last_peered":"2024-05-08T23:21:05.355509+0000","last_clean":"2024-05-08T23:21:05.355509+0000","last_became_active":"2024-05-08T23:21:03.641441+0000","last_became_peered":"2024-05-08T23:21:03.641441+0000","last_unstale":"2024-05-08T23:21:05.355509+0000","last_undegraded":"2024-05-08T23:21:05.355509+0000","last_fullsized":"2024-05-08T23:21:05.355509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:58.822252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355038+0000","last_change":"2024-05-08T23:21:04.106177+0000","last_active":"2024-05-08T23:21:05.355038+0000","last_peered":"2024-05-08T23:21:05.355038+0000","last_clean":"2024-05-08T23:21:05.355038+0000","last_became_active":"2024-05-08T23:21:04.105844+0000","last_became_peered":"2024-05-08T23:21:04.105844+0000","last_unstale":"2024-05-08T23:21:05.355038+0000","last_undegraded":"2024-05-08T23:21:05.355038+0000","last_fullsized":"2024-05-08T23:21:05.355038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356436+0000","last_change":"2024-05-08T23:21:03.646376+0000","last_active":"2024-05-08T23:21:05.356436+0000","last_peered":"2024-05-08T23:21:05.356436+0000","last_clean":"2024-05-08T23:21:05.356436+0000","last_became_active":"2024-05-08T23:21:03.646071+0000","last_became_peered":"2024-05-08T23:21:03.646071+0000","last_unstale":"2024-05-08T23:21:05.356436+0000","last_undegraded":"2024-05-08T23:21:05.356436+0000","last_fullsized":"2024-05-08T23:21:05.356436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356915+0000","last_change":"2024-05-08T23:21:04.120630+0000","last_active":"2024-05-08T23:21:05.356915+0000","last_peered":"2024-05-08T23:21:05.356915+0000","last_clean":"2024-05-08T23:21:05.356915+0000","last_became_active":"2024-05-08T23:21:04.120315+0000","last_became_peered":"2024-05-08T23:21:04.120315+0000","last_unstale":"2024-05-08T23:21:05.356915+0000","last_undegraded":"2024-05-08T23:21:05.356915+0000","last_fullsized":"2024-05-08T23:21:05.356915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:57.953372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356590+0000","last_change":"2024-05-08T23:21:03.645556+0000","last_active":"2024-05-08T23:21:05.356590+0000","last_peered":"2024-05-08T23:21:05.356590+0000","last_clean":"2024-05-08T23:21:05.356590+0000","last_became_active":"2024-05-08T23:21:03.645083+0000","last_became_peered":"2024-05-08T23:21:03.645083+0000","last_unstale":"2024-05-08T23:21:05.356590+0000","last_undegraded":"2024-05-08T23:21:05.356590+0000","last_fullsized":"2024-05-08T23:21:05.356590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355274+0000","last_change":"2024-05-08T23:21:04.108235+0000","last_active":"2024-05-08T23:21:05.355274+0000","last_peered":"2024-05-08T23:21:05.355274+0000","last_clean":"2024-05-08T23:21:05.355274+0000","last_became_active":"2024-05-08T23:21:04.107933+0000","last_became_peered":"2024-05-08T23:21:04.107933+0000","last_unstale":"2024-05-08T23:21:05.355274+0000","last_undegraded":"2024-05-08T23:21:05.355274+0000","last_fullsized":"2024-05-08T23:21:05.355274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354849+0000","last_change":"2024-05-08T23:21:03.636549+0000","last_active":"2024-05-08T23:21:05.354849+0000","last_peered":"2024-05-08T23:21:05.354849+0000","last_clean":"2024-05-08T23:21:05.354849+0000","last_became_active":"2024-05-08T23:21:03.636259+0000","last_became_peered":"2024-05-08T23:21:03.636259+0000","last_unstale":"2024-05-08T23:21:05.354849+0000","last_undegraded":"2024-05-08T23:21:05.354849+0000","last_fullsized":"2024-05-08T23:21:05.354849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:21.894155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356690+0000","last_change":"2024-05-08T23:21:04.113963+0000","last_active":"2024-05-08T23:21:05.356690+0000","last_peered":"2024-05-08T23:21:05.356690+0000","last_clean":"2024-05-08T23:21:05.356690+0000","last_became_active":"2024-05-08T23:21:04.113645+0000","last_became_peered":"2024-05-08T23:21:04.113645+0000","last_unstale":"2024-05-08T23:21:05.356690+0000","last_undegraded":"2024-05-08T23:21:05.356690+0000","last_fullsized":"2024-05-08T23:21:05.356690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093744+0000","last_change":"2024-05-08T23:21:03.644725+0000","last_active":"2024-05-08T23:21:05.093744+0000","last_peered":"2024-05-08T23:21:05.093744+0000","last_clean":"2024-05-08T23:21:05.093744+0000","last_became_active":"2024-05-08T23:21:03.644432+0000","last_became_peered":"2024-05-08T23:21:03.644432+0000","last_unstale":"2024-05-08T23:21:05.093744+0000","last_undegraded":"2024-05-08T23:21:05.093744+0000","last_fullsized":"2024-05-08T23:21:05.093744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:58.822252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356426+0000","last_change":"2024-05-08T23:21:04.116753+0000","last_active":"2024-05-08T23:21:05.356426+0000","last_peered":"2024-05-08T23:21:05.356426+0000","last_clean":"2024-05-08T23:21:05.356426+0000","last_became_active":"2024-05-08T23:21:04.116366+0000","last_became_peered":"2024-05-08T23:21:04.116366+0000","last_unstale":"2024-05-08T23:21:05.356426+0000","last_undegraded":"2024-05-08T23:21:05.356426+0000","last_fullsized":"2024-05-08T23:21:05.356426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:03.080761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093071+0000","last_change":"2024-05-08T23:21:03.643882+0000","last_active":"2024-05-08T23:21:05.093071+0000","last_peered":"2024-05-08T23:21:05.093071+0000","last_clean":"2024-05-08T23:21:05.093071+0000","last_became_active":"2024-05-08T23:21:03.643533+0000","last_became_peered":"2024-05-08T23:21:03.643533+0000","last_unstale":"2024-05-08T23:21:05.093071+0000","last_undegraded":"2024-05-08T23:21:05.093071+0000","last_fullsized":"2024-05-08T23:21:05.093071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357279+0000","last_change":"2024-05-08T23:21:04.109134+0000","last_active":"2024-05-08T23:21:05.357279+0000","last_peered":"2024-05-08T23:21:05.357279+0000","last_clean":"2024-05-08T23:21:05.357279+0000","last_became_active":"2024-05-08T23:21:04.108799+0000","last_became_peered":"2024-05-08T23:21:04.108799+0000","last_unstale":"2024-05-08T23:21:05.357279+0000","last_undegraded":"2024-05-08T23:21:05.357279+0000","last_fullsized":"2024-05-08T23:21:05.357279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356346+0000","last_change":"2024-05-08T23:21:03.644161+0000","last_active":"2024-05-08T23:21:05.356346+0000","last_peered":"2024-05-08T23:21:05.356346+0000","last_clean":"2024-05-08T23:21:05.356346+0000","last_became_active":"2024-05-08T23:21:03.643341+0000","last_became_peered":"2024-05-08T23:21:03.643341+0000","last_unstale":"2024-05-08T23:21:05.356346+0000","last_undegraded":"2024-05-08T23:21:05.356346+0000","last_fullsized":"2024-05-08T23:21:05.356346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093723+0000","last_change":"2024-05-08T23:21:04.108965+0000","last_active":"2024-05-08T23:21:05.093723+0000","last_peered":"2024-05-08T23:21:05.093723+0000","last_clean":"2024-05-08T23:21:05.093723+0000","last_became_active":"2024-05-08T23:21:04.107917+0000","last_became_peered":"2024-05-08T23:21:04.107917+0000","last_unstale":"2024-05-08T23:21:05.093723+0000","last_undegraded":"2024-05-08T23:21:05.093723+0000","last_fullsized":"2024-05-08T23:21:05.093723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:00.670245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093185+0000","last_change":"2024-05-08T23:21:04.115959+0000","last_active":"2024-05-08T23:21:05.093185+0000","last_peered":"2024-05-08T23:21:05.093185+0000","last_clean":"2024-05-08T23:21:05.093185+0000","last_became_active":"2024-05-08T23:21:04.115696+0000","last_became_peered":"2024-05-08T23:21:04.115696+0000","last_unstale":"2024-05-08T23:21:05.093185+0000","last_undegraded":"2024-05-08T23:21:05.093185+0000","last_fullsized":"2024-05-08T23:21:05.093185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354856+0000","last_change":"2024-05-08T23:21:03.632217+0000","last_active":"2024-05-08T23:21:05.354856+0000","last_peered":"2024-05-08T23:21:05.354856+0000","last_clean":"2024-05-08T23:21:05.354856+0000","last_became_active":"2024-05-08T23:21:03.631883+0000","last_became_peered":"2024-05-08T23:21:03.631883+0000","last_unstale":"2024-05-08T23:21:05.354856+0000","last_undegraded":"2024-05-08T23:21:05.354856+0000","last_fullsized":"2024-05-08T23:21:05.354856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092402+0000","last_change":"2024-05-08T23:21:04.104594+0000","last_active":"2024-05-08T23:21:05.092402+0000","last_peered":"2024-05-08T23:21:05.092402+0000","last_clean":"2024-05-08T23:21:05.092402+0000","last_became_active":"2024-05-08T23:21:04.104335+0000","last_became_peered":"2024-05-08T23:21:04.104335+0000","last_unstale":"2024-05-08T23:21:05.092402+0000","last_undegraded":"2024-05-08T23:21:05.092402+0000","last_fullsized":"2024-05-08T23:21:05.092402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.357822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355796+0000","last_change":"2024-05-08T23:21:03.637311+0000","last_active":"2024-05-08T23:21:05.355796+0000","last_peered":"2024-05-08T23:21:05.355796+0000","last_clean":"2024-05-08T23:21:05.355796+0000","last_became_active":"2024-05-08T23:21:03.636529+0000","last_became_peered":"2024-05-08T23:21:03.636529+0000","last_unstale":"2024-05-08T23:21:05.355796+0000","last_undegraded":"2024-05-08T23:21:05.355796+0000","last_fullsized":"2024-05-08T23:21:05.355796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:03.676884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-08T23:21:04.101711+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:21:04.101711+0000","last_peered":"2024-05-08T23:21:04.101711+0000","last_clean":"2024-05-08T23:21:04.101711+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:21:04.101711+0000","last_undegraded":"2024-05-08T23:21:04.101711+0000","last_fullsized":"2024-05-08T23:21:04.101711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355927+0000","last_change":"2024-05-08T23:21:04.108232+0000","last_active":"2024-05-08T23:21:05.355927+0000","last_peered":"2024-05-08T23:21:05.355927+0000","last_clean":"2024-05-08T23:21:05.355927+0000","last_became_active":"2024-05-08T23:21:04.107908+0000","last_became_peered":"2024-05-08T23:21:04.107908+0000","last_unstale":"2024-05-08T23:21:05.355927+0000","last_undegraded":"2024-05-08T23:21:05.355927+0000","last_fullsized":"2024-05-08T23:21:05.355927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:17.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356470+0000","last_change":"2024-05-08T23:21:03.645567+0000","last_active":"2024-05-08T23:21:05.356470+0000","last_peered":"2024-05-08T23:21:05.356470+0000","last_clean":"2024-05-08T23:21:05.356470+0000","last_became_active":"2024-05-08T23:21:03.644844+0000","last_became_peered":"2024-05-08T23:21:03.644844+0000","last_unstale":"2024-05-08T23:21:05.356470+0000","last_undegraded":"2024-05-08T23:21:05.356470+0000","last_fullsized":"2024-05-08T23:21:05.356470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:43.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356283+0000","last_change":"2024-05-08T23:21:03.646987+0000","last_active":"2024-05-08T23:21:05.356283+0000","last_peered":"2024-05-08T23:21:05.356283+0000","last_clean":"2024-05-08T23:21:05.356283+0000","last_became_active":"2024-05-08T23:21:03.646283+0000","last_became_peered":"2024-05-08T23:21:03.646283+0000","last_unstale":"2024-05-08T23:21:05.356283+0000","last_undegraded":"2024-05-08T23:21:05.356283+0000","last_fullsized":"2024-05-08T23:21:05.356283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:59.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092960+0000","last_change":"2024-05-08T23:21:04.106165+0000","last_active":"2024-05-08T23:21:05.092960+0000","last_peered":"2024-05-08T23:21:05.092960+0000","last_clean":"2024-05-08T23:21:05.092960+0000","last_became_active":"2024-05-08T23:21:04.105879+0000","last_became_peered":"2024-05-08T23:21:04.105879+0000","last_unstale":"2024-05-08T23:21:05.092960+0000","last_undegraded":"2024-05-08T23:21:05.092960+0000","last_fullsized":"2024-05-08T23:21:05.092960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.368136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:07.296809+0000","last_change":"2024-05-08T23:21:03.638777+0000","last_active":"2024-05-08T23:21:07.296809+0000","last_peered":"2024-05-08T23:21:07.296809+0000","last_clean":"2024-05-08T23:21:07.296809+0000","last_became_active":"2024-05-08T23:21:03.638494+0000","last_became_peered":"2024-05-08T23:21:03.638494+0000","last_unstale":"2024-05-08T23:21:07.296809+0000","last_undegraded":"2024-05-08T23:21:07.296809+0000","last_fullsized":"2024-05-08T23:21:07.296809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093027+0000","last_change":"2024-05-08T23:21:04.113442+0000","last_active":"2024-05-08T23:21:05.093027+0000","last_peered":"2024-05-08T23:21:05.093027+0000","last_clean":"2024-05-08T23:21:05.093027+0000","last_became_active":"2024-05-08T23:21:04.113144+0000","last_became_peered":"2024-05-08T23:21:04.113144+0000","last_unstale":"2024-05-08T23:21:05.093027+0000","last_undegraded":"2024-05-08T23:21:05.093027+0000","last_fullsized":"2024-05-08T23:21:05.093027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355626+0000","last_change":"2024-05-08T23:21:03.644120+0000","last_active":"2024-05-08T23:21:05.355626+0000","last_peered":"2024-05-08T23:21:05.355626+0000","last_clean":"2024-05-08T23:21:05.355626+0000","last_became_active":"2024-05-08T23:21:03.643541+0000","last_became_peered":"2024-05-08T23:21:03.643541+0000","last_unstale":"2024-05-08T23:21:05.355626+0000","last_undegraded":"2024-05-08T23:21:05.355626+0000","last_fullsized":"2024-05-08T23:21:05.355626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355406+0000","last_change":"2024-05-08T23:21:04.115769+0000","last_active":"2024-05-08T23:21:05.355406+0000","last_peered":"2024-05-08T23:21:05.355406+0000","last_clean":"2024-05-08T23:21:05.355406+0000","last_became_active":"2024-05-08T23:21:04.115446+0000","last_became_peered":"2024-05-08T23:21:04.115446+0000","last_unstale":"2024-05-08T23:21:05.355406+0000","last_undegraded":"2024-05-08T23:21:05.355406+0000","last_fullsized":"2024-05-08T23:21:05.355406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:50.696437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356111+0000","last_change":"2024-05-08T23:21:03.647284+0000","last_active":"2024-05-08T23:21:05.356111+0000","last_peered":"2024-05-08T23:21:05.356111+0000","last_clean":"2024-05-08T23:21:05.356111+0000","last_became_active":"2024-05-08T23:21:03.645828+0000","last_became_peered":"2024-05-08T23:21:03.645828+0000","last_unstale":"2024-05-08T23:21:05.356111+0000","last_undegraded":"2024-05-08T23:21:05.356111+0000","last_fullsized":"2024-05-08T23:21:05.356111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:54.428258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093694+0000","last_change":"2024-05-08T23:21:04.105901+0000","last_active":"2024-05-08T23:21:05.093694+0000","last_peered":"2024-05-08T23:21:05.093694+0000","last_clean":"2024-05-08T23:21:05.093694+0000","last_became_active":"2024-05-08T23:21:04.105495+0000","last_became_peered":"2024-05-08T23:21:04.105495+0000","last_unstale":"2024-05-08T23:21:05.093694+0000","last_undegraded":"2024-05-08T23:21:05.093694+0000","last_fullsized":"2024-05-08T23:21:05.093694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:44.288365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093485+0000","last_change":"2024-05-08T23:21:03.643905+0000","last_active":"2024-05-08T23:21:05.093485+0000","last_peered":"2024-05-08T23:21:05.093485+0000","last_clean":"2024-05-08T23:21:05.093485+0000","last_became_active":"2024-05-08T23:21:03.643555+0000","last_became_peered":"2024-05-08T23:21:03.643555+0000","last_unstale":"2024-05-08T23:21:05.093485+0000","last_undegraded":"2024-05-08T23:21:05.093485+0000","last_fullsized":"2024-05-08T23:21:05.093485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:32.018800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092205+0000","last_change":"2024-05-08T23:21:04.117123+0000","last_active":"2024-05-08T23:21:05.092205+0000","last_peered":"2024-05-08T23:21:05.092205+0000","last_clean":"2024-05-08T23:21:05.092205+0000","last_became_active":"2024-05-08T23:21:04.116830+0000","last_became_peered":"2024-05-08T23:21:04.116830+0000","last_unstale":"2024-05-08T23:21:05.092205+0000","last_undegraded":"2024-05-08T23:21:05.092205+0000","last_fullsized":"2024-05-08T23:21:05.092205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:02.247647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093648+0000","last_change":"2024-05-08T23:21:03.642867+0000","last_active":"2024-05-08T23:21:05.093648+0000","last_peered":"2024-05-08T23:21:05.093648+0000","last_clean":"2024-05-08T23:21:05.093648+0000","last_became_active":"2024-05-08T23:21:03.642439+0000","last_became_peered":"2024-05-08T23:21:03.642439+0000","last_unstale":"2024-05-08T23:21:05.093648+0000","last_undegraded":"2024-05-08T23:21:05.093648+0000","last_fullsized":"2024-05-08T23:21:05.093648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:51.538122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092672+0000","last_change":"2024-05-08T23:21:04.115928+0000","last_active":"2024-05-08T23:21:05.092672+0000","last_peered":"2024-05-08T23:21:05.092672+0000","last_clean":"2024-05-08T23:21:05.092672+0000","last_became_active":"2024-05-08T23:21:04.113858+0000","last_became_peered":"2024-05-08T23:21:04.113858+0000","last_unstale":"2024-05-08T23:21:05.092672+0000","last_undegraded":"2024-05-08T23:21:05.092672+0000","last_fullsized":"2024-05-08T23:21:05.092672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:35.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.357022+0000","last_change":"2024-05-08T23:21:03.649228+0000","last_active":"2024-05-08T23:21:05.357022+0000","last_peered":"2024-05-08T23:21:05.357022+0000","last_clean":"2024-05-08T23:21:05.357022+0000","last_became_active":"2024-05-08T23:21:03.648868+0000","last_became_peered":"2024-05-08T23:21:03.648868+0000","last_unstale":"2024-05-08T23:21:05.357022+0000","last_undegraded":"2024-05-08T23:21:05.357022+0000","last_fullsized":"2024-05-08T23:21:05.357022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:34.255292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355422+0000","last_change":"2024-05-08T23:21:04.110982+0000","last_active":"2024-05-08T23:21:05.355422+0000","last_peered":"2024-05-08T23:21:05.355422+0000","last_clean":"2024-05-08T23:21:05.355422+0000","last_became_active":"2024-05-08T23:21:04.110643+0000","last_became_peered":"2024-05-08T23:21:04.110643+0000","last_unstale":"2024-05-08T23:21:05.355422+0000","last_undegraded":"2024-05-08T23:21:05.355422+0000","last_fullsized":"2024-05-08T23:21:05.355422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:21.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355712+0000","last_change":"2024-05-08T23:21:03.637418+0000","last_active":"2024-05-08T23:21:05.355712+0000","last_peered":"2024-05-08T23:21:05.355712+0000","last_clean":"2024-05-08T23:21:05.355712+0000","last_became_active":"2024-05-08T23:21:03.637139+0000","last_became_peered":"2024-05-08T23:21:03.637139+0000","last_unstale":"2024-05-08T23:21:05.355712+0000","last_undegraded":"2024-05-08T23:21:05.355712+0000","last_fullsized":"2024-05-08T23:21:05.355712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:36.851412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356254+0000","last_change":"2024-05-08T23:21:04.112240+0000","last_active":"2024-05-08T23:21:05.356254+0000","last_peered":"2024-05-08T23:21:05.356254+0000","last_clean":"2024-05-08T23:21:05.356254+0000","last_became_active":"2024-05-08T23:21:04.111914+0000","last_became_peered":"2024-05-08T23:21:04.111914+0000","last_unstale":"2024-05-08T23:21:05.356254+0000","last_undegraded":"2024-05-08T23:21:05.356254+0000","last_fullsized":"2024-05-08T23:21:05.356254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:09.028590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093821+0000","last_change":"2024-05-08T23:21:03.643712+0000","last_active":"2024-05-08T23:21:05.093821+0000","last_peered":"2024-05-08T23:21:05.093821+0000","last_clean":"2024-05-08T23:21:05.093821+0000","last_became_active":"2024-05-08T23:21:03.643387+0000","last_became_peered":"2024-05-08T23:21:03.643387+0000","last_unstale":"2024-05-08T23:21:05.093821+0000","last_undegraded":"2024-05-08T23:21:05.093821+0000","last_fullsized":"2024-05-08T23:21:05.093821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:59.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356035+0000","last_change":"2024-05-08T23:21:04.110728+0000","last_active":"2024-05-08T23:21:05.356035+0000","last_peered":"2024-05-08T23:21:05.356035+0000","last_clean":"2024-05-08T23:21:05.356035+0000","last_became_active":"2024-05-08T23:21:04.110094+0000","last_became_peered":"2024-05-08T23:21:04.110094+0000","last_unstale":"2024-05-08T23:21:05.356035+0000","last_undegraded":"2024-05-08T23:21:05.356035+0000","last_fullsized":"2024-05-08T23:21:05.356035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:30:28.180773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355130+0000","last_change":"2024-05-08T23:21:03.638990+0000","last_active":"2024-05-08T23:21:05.355130+0000","last_peered":"2024-05-08T23:21:05.355130+0000","last_clean":"2024-05-08T23:21:05.355130+0000","last_became_active":"2024-05-08T23:21:03.638696+0000","last_became_peered":"2024-05-08T23:21:03.638696+0000","last_unstale":"2024-05-08T23:21:05.355130+0000","last_undegraded":"2024-05-08T23:21:05.355130+0000","last_fullsized":"2024-05-08T23:21:05.355130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:02.613341+0000","last_clean_scrub_stamp":"2024-05-08T23:21:02.613341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:16.993057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356830+0000","last_change":"2024-05-08T23:21:04.117976+0000","last_active":"2024-05-08T23:21:05.356830+0000","last_peered":"2024-05-08T23:21:05.356830+0000","last_clean":"2024-05-08T23:21:05.356830+0000","last_became_active":"2024-05-08T23:21:04.117682+0000","last_became_peered":"2024-05-08T23:21:04.117682+0000","last_unstale":"2024-05-08T23:21:05.356830+0000","last_undegraded":"2024-05-08T23:21:05.356830+0000","last_fullsized":"2024-05-08T23:21:05.356830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:30:28.180773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355265+0000","last_change":"2024-05-08T23:21:04.107601+0000","last_active":"2024-05-08T23:21:05.355265+0000","last_peered":"2024-05-08T23:21:05.355265+0000","last_clean":"2024-05-08T23:21:05.355265+0000","last_became_active":"2024-05-08T23:21:04.107309+0000","last_became_peered":"2024-05-08T23:21:04.107309+0000","last_unstale":"2024-05-08T23:21:05.355265+0000","last_undegraded":"2024-05-08T23:21:05.355265+0000","last_fullsized":"2024-05-08T23:21:05.355265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.354968+0000","last_change":"2024-05-08T23:21:04.106808+0000","last_active":"2024-05-08T23:21:05.354968+0000","last_peered":"2024-05-08T23:21:05.354968+0000","last_clean":"2024-05-08T23:21:05.354968+0000","last_became_active":"2024-05-08T23:21:04.106628+0000","last_became_peered":"2024-05-08T23:21:04.106628+0000","last_unstale":"2024-05-08T23:21:05.354968+0000","last_undegraded":"2024-05-08T23:21:05.354968+0000","last_fullsized":"2024-05-08T23:21:05.354968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.433453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356256+0000","last_change":"2024-05-08T23:21:04.105100+0000","last_active":"2024-05-08T23:21:05.356256+0000","last_peered":"2024-05-08T23:21:05.356256+0000","last_clean":"2024-05-08T23:21:05.356256+0000","last_became_active":"2024-05-08T23:21:04.104785+0000","last_became_peered":"2024-05-08T23:21:04.104785+0000","last_unstale":"2024-05-08T23:21:05.356256+0000","last_undegraded":"2024-05-08T23:21:05.356256+0000","last_fullsized":"2024-05-08T23:21:05.356256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:19.795476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355441+0000","last_change":"2024-05-08T23:21:04.105145+0000","last_active":"2024-05-08T23:21:05.355441+0000","last_peered":"2024-05-08T23:21:05.355441+0000","last_clean":"2024-05-08T23:21:05.355441+0000","last_became_active":"2024-05-08T23:21:04.104842+0000","last_became_peered":"2024-05-08T23:21:04.104842+0000","last_unstale":"2024-05-08T23:21:05.355441+0000","last_undegraded":"2024-05-08T23:21:05.355441+0000","last_fullsized":"2024-05-08T23:21:05.355441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092800+0000","last_change":"2024-05-08T23:21:04.106935+0000","last_active":"2024-05-08T23:21:05.092800+0000","last_peered":"2024-05-08T23:21:05.092800+0000","last_clean":"2024-05-08T23:21:05.092800+0000","last_became_active":"2024-05-08T23:21:04.106671+0000","last_became_peered":"2024-05-08T23:21:04.106671+0000","last_unstale":"2024-05-08T23:21:05.092800+0000","last_undegraded":"2024-05-08T23:21:05.092800+0000","last_fullsized":"2024-05-08T23:21:05.092800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:07.596384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.093704+0000","last_change":"2024-05-08T23:21:04.112023+0000","last_active":"2024-05-08T23:21:05.093704+0000","last_peered":"2024-05-08T23:21:05.093704+0000","last_clean":"2024-05-08T23:21:05.093704+0000","last_became_active":"2024-05-08T23:21:04.111819+0000","last_became_peered":"2024-05-08T23:21:04.111819+0000","last_unstale":"2024-05-08T23:21:05.093704+0000","last_undegraded":"2024-05-08T23:21:05.093704+0000","last_fullsized":"2024-05-08T23:21:05.093704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355844+0000","last_change":"2024-05-08T23:21:04.111963+0000","last_active":"2024-05-08T23:21:05.355844+0000","last_peered":"2024-05-08T23:21:05.355844+0000","last_clean":"2024-05-08T23:21:05.355844+0000","last_became_active":"2024-05-08T23:21:04.111629+0000","last_became_peered":"2024-05-08T23:21:04.111629+0000","last_unstale":"2024-05-08T23:21:05.355844+0000","last_undegraded":"2024-05-08T23:21:05.355844+0000","last_fullsized":"2024-05-08T23:21:05.355844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092931+0000","last_change":"2024-05-08T23:21:04.113608+0000","last_active":"2024-05-08T23:21:05.092931+0000","last_peered":"2024-05-08T23:21:05.092931+0000","last_clean":"2024-05-08T23:21:05.092931+0000","last_became_active":"2024-05-08T23:21:04.113322+0000","last_became_peered":"2024-05-08T23:21:04.113322+0000","last_unstale":"2024-05-08T23:21:05.092931+0000","last_undegraded":"2024-05-08T23:21:05.092931+0000","last_fullsized":"2024-05-08T23:21:05.092931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356052+0000","last_change":"2024-05-08T23:21:04.109658+0000","last_active":"2024-05-08T23:21:05.356052+0000","last_peered":"2024-05-08T23:21:05.356052+0000","last_clean":"2024-05-08T23:21:05.356052+0000","last_became_active":"2024-05-08T23:21:04.109222+0000","last_became_peered":"2024-05-08T23:21:04.109222+0000","last_unstale":"2024-05-08T23:21:05.356052+0000","last_undegraded":"2024-05-08T23:21:05.356052+0000","last_fullsized":"2024-05-08T23:21:05.356052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:39.159395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.092319+0000","last_change":"2024-05-08T23:21:04.107408+0000","last_active":"2024-05-08T23:21:05.092319+0000","last_peered":"2024-05-08T23:21:05.092319+0000","last_clean":"2024-05-08T23:21:05.092319+0000","last_became_active":"2024-05-08T23:21:04.107123+0000","last_became_peered":"2024-05-08T23:21:04.107123+0000","last_unstale":"2024-05-08T23:21:05.092319+0000","last_undegraded":"2024-05-08T23:21:05.092319+0000","last_fullsized":"2024-05-08T23:21:05.092319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356155+0000","last_change":"2024-05-08T23:21:04.108500+0000","last_active":"2024-05-08T23:21:05.356155+0000","last_peered":"2024-05-08T23:21:05.356155+0000","last_clean":"2024-05-08T23:21:05.356155+0000","last_became_active":"2024-05-08T23:21:04.107974+0000","last_became_peered":"2024-05-08T23:21:04.107974+0000","last_unstale":"2024-05-08T23:21:05.356155+0000","last_undegraded":"2024-05-08T23:21:05.356155+0000","last_fullsized":"2024-05-08T23:21:05.356155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:37.648885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.355333+0000","last_change":"2024-05-08T23:21:04.112817+0000","last_active":"2024-05-08T23:21:05.355333+0000","last_peered":"2024-05-08T23:21:05.355333+0000","last_clean":"2024-05-08T23:21:05.355333+0000","last_became_active":"2024-05-08T23:21:04.112519+0000","last_became_peered":"2024-05-08T23:21:04.112519+0000","last_unstale":"2024-05-08T23:21:05.355333+0000","last_undegraded":"2024-05-08T23:21:05.355333+0000","last_fullsized":"2024-05-08T23:21:05.355333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:15.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:21:05.356539+0000","last_change":"2024-05-08T23:21:04.113697+0000","last_active":"2024-05-08T23:21:05.356539+0000","last_peered":"2024-05-08T23:21:05.356539+0000","last_clean":"2024-05-08T23:21:05.356539+0000","last_became_active":"2024-05-08T23:21:04.113363+0000","last_became_peered":"2024-05-08T23:21:04.113363+0000","last_unstale":"2024-05-08T23:21:05.356539+0000","last_undegraded":"2024-05-08T23:21:05.356539+0000","last_fullsized":"2024-05-08T23:21:05.356539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:21:03.618536+0000","last_clean_scrub_stamp":"2024-05-08T23:21:03.618536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:22.318766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:21:18.584 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T23:21:18.584 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T23:21:18.584 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T23:21:18.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T23:21:18.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.762+0000 7f79454ec700 1 Processor -- start 2024-05-08T23:21:18.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.762+0000 7f79454ec700 1 -- start start 2024-05-08T23:21:18.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.762+0000 7f79454ec700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f794015d9c0 0x7f79400cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.762+0000 7f79454ec700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79400cf7a0 0x7f79400d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.762+0000 7f79454ec700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400d0880 con 0x7f794006b810 2024-05-08T23:21:18.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.762+0000 7f79454ec700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400d0b60 con 0x7f79400cf7a0 2024-05-08T23:21:18.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.762+0000 7f79454ec700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400ce5b0 con 0x7f794015d9c0 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79400cf7a0 0x7f79400d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79400cf7a0 0x7f79400d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52760/0 (socket says 172.21.15.5:52760) 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793f7fe700 1 -- 172.21.15.5:0/909134050 learned_addr learned my addr 172.21.15.5:0/909134050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793effd700 1 --2- 172.21.15.5:0/909134050 >> v2:172.21.15.49:3300/0 conn(0x7f794015d9c0 0x7f79400cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793f7fe700 1 -- 172.21.15.5:0/909134050 >> v2:172.21.15.49:3300/0 conn(0x7f794015d9c0 msgr2=0x7f79400cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793f7fe700 1 --2- 172.21.15.5:0/909134050 >> v2:172.21.15.49:3300/0 conn(0x7f794015d9c0 0x7f79400cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793f7fe700 1 -- 172.21.15.5:0/909134050 >> v1:172.21.15.49:6789/0 conn(0x7f794006b810 legacy=0x7f79400cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793f7fe700 1 -- 172.21.15.5:0/909134050 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400ce890 con 0x7f79400cf7a0 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793f7fe700 1 --2- 172.21.15.5:0/909134050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79400cf7a0 0x7f79400d16e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f794008b780 tx=0x7f793800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653dc60b60349045 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793dffb700 1 -- 172.21.15.5:0/909134050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800e070 con 0x7f79400cf7a0 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793dffb700 1 -- 172.21.15.5:0/909134050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793800b9c0 con 0x7f79400cf7a0 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.763+0000 7f793dffb700 1 -- 172.21.15.5:0/909134050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800bd70 con 0x7f79400cf7a0 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.764+0000 7f79454ec700 1 -- 172.21.15.5:0/909134050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79400cf7a0 msgr2=0x7f79400d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.764+0000 7f79454ec700 1 --2- 172.21.15.5:0/909134050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79400cf7a0 0x7f79400d16e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f794008b780 tx=0x7f793800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.764+0000 7f79454ec700 1 -- 172.21.15.5:0/909134050 shutdown_connections 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.764+0000 7f79454ec700 1 --2- 172.21.15.5:0/909134050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79400cf7a0 0x7f79400d16e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.764+0000 7f79454ec700 1 --2- 172.21.15.5:0/909134050 >> v2:172.21.15.49:3300/0 conn(0x7f794015d9c0 0x7f79400cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.764+0000 7f79454ec700 1 -- 172.21.15.5:0/909134050 >> 172.21.15.5:0/909134050 conn(0x7f794008afe0 msgr2=0x7f79400d3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.764+0000 7f79454ec700 1 -- 172.21.15.5:0/909134050 shutdown_connections 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.764+0000 7f79454ec700 1 -- 172.21.15.5:0/909134050 wait complete. 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.765+0000 7f79454ec700 1 Processor -- start 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f79454ec700 1 -- start start 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f79454ec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f794006b810 0x7f79401e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f79454ec700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79400cf7a0 0x7f794015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f79454ec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794015d9c0 0x7f79400b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f79454ec700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400cb3c0 con 0x7f794015d9c0 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f79454ec700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400ca7b0 con 0x7f794006b810 2024-05-08T23:21:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f79454ec700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400caa90 con 0x7f79400cf7a0 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f793f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794015d9c0 0x7f79400b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f793f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794015d9c0 0x7f79400b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52770/0 (socket says 172.21.15.5:52770) 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.766+0000 7f793f7fe700 1 -- 172.21.15.5:0/4226843859 learned_addr learned my addr 172.21.15.5:0/4226843859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f793effd700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f794006b810 0x7f79401e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f793e7fc700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79400cf7a0 0x7f794015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f793f7fe700 1 -- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79400cf7a0 msgr2=0x7f794015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f793f7fe700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79400cf7a0 0x7f794015bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f793f7fe700 1 -- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f794006b810 msgr2=0x7f79401e6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f793f7fe700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f794006b810 0x7f79401e6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f793f7fe700 1 -- 172.21.15.5:0/4226843859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938009b30 con 0x7f794015d9c0 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f793f7fe700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794015d9c0 0x7f79400b3ad0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f794008ba20 tx=0x7f7938004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f79277fe700 1 -- 172.21.15.5:0/4226843859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380099b0 con 0x7f794015d9c0 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f79277fe700 1 -- 172.21.15.5:0/4226843859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380092e0 con 0x7f794015d9c0 2024-05-08T23:21:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79400cad70 con 0x7f794015d9c0 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f79277fe700 1 -- 172.21.15.5:0/4226843859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793801c640 con 0x7f794015d9c0 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.767+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79401e7430 con 0x7f794015d9c0 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.768+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79400678f0 con 0x7f794015d9c0 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.769+0000 7f79277fe700 1 -- 172.21.15.5:0/4226843859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f793801c7a0 con 0x7f794015d9c0 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.769+0000 7f79277fe700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f792c06ac90 0x7f792c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.769+0000 7f79277fe700 1 -- 172.21.15.5:0/4226843859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f793808aca0 con 0x7f794015d9c0 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.769+0000 7f793effd700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f792c06ac90 0x7f792c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.770+0000 7f793effd700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f792c06ac90 0x7f792c06d150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f793000b9b0 tx=0x7f7930008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:18.774+0000 7f79277fe700 1 -- 172.21.15.5:0/4226843859 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7938055f10 con 0x7f794015d9c0 2024-05-08T23:21:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.042+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f79400d2740 con 0x7f794015d9c0 2024-05-08T23:21:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.042+0000 7f79277fe700 1 -- 172.21.15.5:0/4226843859 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7938059530 con 0x7f794015d9c0 2024-05-08T23:21:19.043 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:19.043 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T23:21:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f792c06ac90 msgr2=0x7f792c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f792c06ac90 0x7f792c06d150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f793000b9b0 tx=0x7f7930008040 comp rx=0 tx=0).stop 2024-05-08T23:21:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794015d9c0 msgr2=0x7f79400b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794015d9c0 0x7f79400b3ad0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f794008ba20 tx=0x7f7938004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 shutdown_connections 2024-05-08T23:21:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f792c06ac90 0x7f792c06d150 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794015d9c0 0x7f79400b3ad0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f794006b810 0x7f79401e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 --2- 172.21.15.5:0/4226843859 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79400cf7a0 0x7f794015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 >> 172.21.15.5:0/4226843859 conn(0x7f794008afe0 msgr2=0x7f794006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 shutdown_connections 2024-05-08T23:21:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.045+0000 7f79454ec700 1 -- 172.21.15.5:0/4226843859 wait complete. 2024-05-08T23:21:19.058 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T23:21:19.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.198+0000 7f36f7afb700 1 Processor -- start 2024-05-08T23:21:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.198+0000 7f36f7afb700 1 -- start start 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f7afb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f36f00cd4e0 0x7f36f006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f7afb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36f006f130 0x7f36f00ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f7afb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00ceb20 0x7f36f015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f7afb700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f00cefb0 con 0x7f36f006f130 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f7afb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f015c280 con 0x7f36f00cd4e0 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f7afb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f006e780 con 0x7f36f00ceb20 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f5096700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36f006f130 0x7f36f00ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f5096700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36f006f130 0x7f36f00ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52778/0 (socket says 172.21.15.5:52778) 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f5096700 1 -- 172.21.15.5:0/384149640 learned_addr learned my addr 172.21.15.5:0/384149640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f5897700 1 --2- 172.21.15.5:0/384149640 >> v2:172.21.15.49:3300/0 conn(0x7f36f00cd4e0 0x7f36f006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.199+0000 7f36f6098700 1 --2- 172.21.15.5:0/384149640 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00ceb20 0x7f36f015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f5096700 1 -- 172.21.15.5:0/384149640 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00ceb20 msgr2=0x7f36f015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f5096700 1 --2- 172.21.15.5:0/384149640 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00ceb20 0x7f36f015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f5096700 1 -- 172.21.15.5:0/384149640 >> v2:172.21.15.49:3300/0 conn(0x7f36f00cd4e0 msgr2=0x7f36f006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f5096700 1 --2- 172.21.15.5:0/384149640 >> v2:172.21.15.49:3300/0 conn(0x7f36f00cd4e0 0x7f36f006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f5096700 1 -- 172.21.15.5:0/384149640 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f006ea60 con 0x7f36f006f130 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f5096700 1 --2- 172.21.15.5:0/384149640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00ce5e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36f0072ef0 tx=0x7f36e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6666a30d9f1cbbaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f4895700 1 -- 172.21.15.5:0/384149640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0011830 con 0x7f36f006f130 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f4895700 1 -- 172.21.15.5:0/384149640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0011990 con 0x7f36f006f130 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.200+0000 7f36f4895700 1 -- 172.21.15.5:0/384149640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0011d20 con 0x7f36f006f130 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 -- 172.21.15.5:0/384149640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 msgr2=0x7f36f00ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 --2- 172.21.15.5:0/384149640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00ce5e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36f0072ef0 tx=0x7f36e0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 -- 172.21.15.5:0/384149640 shutdown_connections 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 --2- 172.21.15.5:0/384149640 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00ceb20 0x7f36f015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 --2- 172.21.15.5:0/384149640 >> v2:172.21.15.49:3300/0 conn(0x7f36f00cd4e0 0x7f36f006f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 --2- 172.21.15.5:0/384149640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00ce5e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 -- 172.21.15.5:0/384149640 >> 172.21.15.5:0/384149640 conn(0x7f36f00d0320 msgr2=0x7f36f00d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 -- 172.21.15.5:0/384149640 shutdown_connections 2024-05-08T23:21:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.201+0000 7f36f7afb700 1 -- 172.21.15.5:0/384149640 wait complete. 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.202+0000 7f36f7afb700 1 Processor -- start 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.203+0000 7f36f7afb700 1 -- start start 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.203+0000 7f36f7afb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.203+0000 7f36f7afb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00cd4e0 0x7f36f00b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.203+0000 7f36f7afb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f00ceb20 0x7f36f0152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f7afb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f0156ff0 con 0x7f36f006f130 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f7afb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00c4d00 con 0x7f36f00ceb20 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f7afb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00c6d70 con 0x7f36f00cd4e0 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52782/0 (socket says 172.21.15.5:52782) 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 -- 172.21.15.5:0/3705009337 learned_addr learned my addr 172.21.15.5:0/3705009337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f6098700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f00ceb20 0x7f36f0152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5096700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00cd4e0 0x7f36f00b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 -- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00cd4e0 msgr2=0x7f36f00b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00cd4e0 0x7f36f00b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 -- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f00ceb20 msgr2=0x7f36f0152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f00ceb20 0x7f36f0152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 -- 172.21.15.5:0/3705009337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e0009550 con 0x7f36f006f130 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f5897700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00c5190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00e930 tx=0x7f36ec00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36e67fc700 1 -- 172.21.15.5:0/3705009337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec009690 con 0x7f36f006f130 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36e67fc700 1 -- 172.21.15.5:0/3705009337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec00c850 con 0x7f36f006f130 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f00c67b0 con 0x7f36f006f130 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36e67fc700 1 -- 172.21.15.5:0/3705009337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec00cbe0 con 0x7f36f006f130 2024-05-08T23:21:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.204+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f00c4990 con 0x7f36f006f130 2024-05-08T23:21:19.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.205+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f00c4fe0 con 0x7f36f006f130 2024-05-08T23:21:19.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.206+0000 7f36e67fc700 1 -- 172.21.15.5:0/3705009337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f36ec00cd40 con 0x7f36f006f130 2024-05-08T23:21:19.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.206+0000 7f36e67fc700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36d406ac90 0x7f36d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.206+0000 7f36e67fc700 1 -- 172.21.15.5:0/3705009337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36ec089340 con 0x7f36f006f130 2024-05-08T23:21:19.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.207+0000 7f36f5096700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36d406ac90 0x7f36d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.207+0000 7f36f5096700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36d406ac90 0x7f36d406d150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009c10 tx=0x7f36e0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.211+0000 7f36e67fc700 1 -- 172.21.15.5:0/3705009337 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f36ec0545b0 con 0x7f36f006f130 2024-05-08T23:21:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.430+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36f0068000 con 0x7f36f006f130 2024-05-08T23:21:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.431+0000 7f36e67fc700 1 -- 172.21.15.5:0/3705009337 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7f36ec057bd0 con 0x7f36f006f130 2024-05-08T23:21:19.431 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:19.431 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36d406ac90 msgr2=0x7f36d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36d406ac90 0x7f36d406d150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009c10 tx=0x7f36e0008030 comp rx=0 tx=0).stop 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 msgr2=0x7f36f00c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00c5190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00e930 tx=0x7f36ec00ecf0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 shutdown_connections 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36d406ac90 0x7f36d406d150 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36f00ceb20 0x7f36f0152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f006f130 0x7f36f00c5190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 --2- 172.21.15.5:0/3705009337 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36f00cd4e0 0x7f36f00b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 >> 172.21.15.5:0/3705009337 conn(0x7f36f00d0320 msgr2=0x7f36f00ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 shutdown_connections 2024-05-08T23:21:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.433+0000 7f36f7afb700 1 -- 172.21.15.5:0/3705009337 wait complete. 2024-05-08T23:21:19.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:19.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.586+0000 7ff3db281700 1 Processor -- start 2024-05-08T23:21:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.586+0000 7ff3db281700 1 -- start start 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.587+0000 7ff3db281700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3d415b4e0 0x7ff3d40b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.587+0000 7ff3db281700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d40d1910 0x7ff3d40cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.587+0000 7ff3db281700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d40cf560 con 0x7ff3d415b8c0 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.587+0000 7ff3db281700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d40cefe0 con 0x7ff3d415b4e0 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.587+0000 7ff3db281700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40ccff0 con 0x7ff3d40d1910 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.587+0000 7ff3d901d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3d415b4e0 0x7ff3d40b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.587+0000 7ff3d901d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3d415b4e0 0x7ff3d40b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52790/0 (socket says 172.21.15.5:52790) 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.587+0000 7ff3d901d700 1 -- 172.21.15.5:0/1488207200 learned_addr learned my addr 172.21.15.5:0/1488207200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3d981e700 1 --2- 172.21.15.5:0/1488207200 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d40d1910 0x7ff3d40cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3d901d700 1 -- 172.21.15.5:0/1488207200 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d40d1910 msgr2=0x7ff3d40cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3d901d700 1 --2- 172.21.15.5:0/1488207200 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d40d1910 0x7ff3d40cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3d901d700 1 -- 172.21.15.5:0/1488207200 >> v1:172.21.15.49:6789/0 conn(0x7ff3d415b8c0 legacy=0x7ff3d40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3d901d700 1 -- 172.21.15.5:0/1488207200 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40cd2d0 con 0x7ff3d415b4e0 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3d901d700 1 --2- 172.21.15.5:0/1488207200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d415b4e0 0x7ff3d40b1200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00098f0 tx=0x7ff3d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbafcd6fd728e0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3cbfff700 1 -- 172.21.15.5:0/1488207200 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000b8a0 con 0x7ff3d415b4e0 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3cbfff700 1 -- 172.21.15.5:0/1488207200 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d000ba00 con 0x7ff3d415b4e0 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.588+0000 7ff3cbfff700 1 -- 172.21.15.5:0/1488207200 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000bd90 con 0x7ff3d415b4e0 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.589+0000 7ff3db281700 1 -- 172.21.15.5:0/1488207200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d415b4e0 msgr2=0x7ff3d40b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.589+0000 7ff3db281700 1 --2- 172.21.15.5:0/1488207200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d415b4e0 0x7ff3d40b1200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00098f0 tx=0x7ff3d0009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.589+0000 7ff3db281700 1 -- 172.21.15.5:0/1488207200 shutdown_connections 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.589+0000 7ff3db281700 1 --2- 172.21.15.5:0/1488207200 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d40d1910 0x7ff3d40cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.589+0000 7ff3db281700 1 --2- 172.21.15.5:0/1488207200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d415b4e0 0x7ff3d40b1200 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.589+0000 7ff3db281700 1 -- 172.21.15.5:0/1488207200 >> 172.21.15.5:0/1488207200 conn(0x7ff3d408afe0 msgr2=0x7ff3d408b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.589+0000 7ff3db281700 1 -- 172.21.15.5:0/1488207200 shutdown_connections 2024-05-08T23:21:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.589+0000 7ff3db281700 1 -- 172.21.15.5:0/1488207200 wait complete. 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.590+0000 7ff3db281700 1 Processor -- start 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.590+0000 7ff3db281700 1 -- start start 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3db281700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d40d1910 0x7ff3d41f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3db281700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d415b4e0 0x7ff3d4153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3db281700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d415b8c0 0x7ff3d4070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3db281700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4154030 con 0x7ff3d40d1910 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3db281700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d41f80b0 con 0x7ff3d415b8c0 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3db281700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d41f8390 con 0x7ff3d415b4e0 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d40d1910 0x7ff3d41f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d40d1910 0x7ff3d41f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52806/0 (socket says 172.21.15.5:52806) 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 -- 172.21.15.5:0/2776513888 learned_addr learned my addr 172.21.15.5:0/2776513888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 -- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d415b4e0 msgr2=0x7ff3d4153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d415b4e0 0x7ff3d4153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 -- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d415b8c0 msgr2=0x7ff3d4070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d415b8c0 0x7ff3d4070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 -- 172.21.15.5:0/2776513888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0009550 con 0x7ff3d40d1910 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.591+0000 7ff3d901d700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d40d1910 0x7ff3d41f7a40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0000c00 tx=0x7ff3d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.592+0000 7ff3c9ffb700 1 -- 172.21.15.5:0/2776513888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0004480 con 0x7ff3d40d1910 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.592+0000 7ff3c9ffb700 1 -- 172.21.15.5:0/2776513888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d001f070 con 0x7ff3d40d1910 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.592+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d41f8670 con 0x7ff3d40d1910 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.592+0000 7ff3c9ffb700 1 -- 172.21.15.5:0/2776513888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00038c0 con 0x7ff3d40d1910 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.592+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d406bb10 con 0x7ff3d40d1910 2024-05-08T23:21:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.593+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d4074bc0 con 0x7ff3d40d1910 2024-05-08T23:21:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.593+0000 7ff3c9ffb700 1 -- 172.21.15.5:0/2776513888 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff3d0021030 con 0x7ff3d40d1910 2024-05-08T23:21:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.594+0000 7ff3c9ffb700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff3b806ac90 0x7ff3b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.594+0000 7ff3c9ffb700 1 -- 172.21.15.5:0/2776513888 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3d008a980 con 0x7ff3d40d1910 2024-05-08T23:21:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.594+0000 7ff3d881c700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff3b806ac90 0x7ff3b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.594+0000 7ff3d881c700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff3b806ac90 0x7ff3b806d150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000f60 tx=0x7ff3c4007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.598+0000 7ff3c9ffb700 1 -- 172.21.15.5:0/2776513888 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff3d0055d10 con 0x7ff3d40d1910 2024-05-08T23:21:19.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.814+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3d40ce6f0 con 0x7ff3d40d1910 2024-05-08T23:21:19.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.814+0000 7ff3c9ffb700 1 -- 172.21.15.5:0/2776513888 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3d00150a0 con 0x7ff3d40d1910 2024-05-08T23:21:19.814 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:19.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.816+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff3b806ac90 msgr2=0x7ff3b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.816+0000 7ff3db281700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff3b806ac90 0x7ff3b806d150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000f60 tx=0x7ff3c4007a40 comp rx=0 tx=0).stop 2024-05-08T23:21:19.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d40d1910 msgr2=0x7ff3d41f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d40d1910 0x7ff3d41f7a40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0000c00 tx=0x7ff3d0004750 comp rx=0 tx=0).stop 2024-05-08T23:21:19.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 shutdown_connections 2024-05-08T23:21:19.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff3b806ac90 0x7ff3b806d150 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d415b8c0 0x7ff3d4070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d40d1910 0x7ff3d41f7a40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 --2- 172.21.15.5:0/2776513888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3d415b4e0 0x7ff3d4153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 >> 172.21.15.5:0/2776513888 conn(0x7ff3d408afe0 msgr2=0x7ff3d40cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:19.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 shutdown_connections 2024-05-08T23:21:19.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.817+0000 7ff3db281700 1 -- 172.21.15.5:0/2776513888 wait complete. 2024-05-08T23:21:19.830 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.969+0000 7f5e4b49d700 1 Processor -- start 2024-05-08T23:21:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.970+0000 7f5e4b49d700 1 -- start start 2024-05-08T23:21:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.970+0000 7f5e4b49d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5e440cc590 0x7f5e4406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e4b49d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4406f7c0 0x7f5e4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e4b49d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e440cb160 con 0x7f5e4415db00 2024-05-08T23:21:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e4b49d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4406fc50 con 0x7f5e440cc590 2024-05-08T23:21:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e4b49d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4415dee0 con 0x7f5e4406f7c0 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e48a38700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5e440cc590 0x7f5e4406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e49239700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5e4415db00 0x7f5e440caa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42516/0 (socket says 172.21.15.5:42516) 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e49239700 1 -- 172.21.15.5:0/4426958 learned_addr learned my addr 172.21.15.5:0/4426958 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e49a3a700 1 --2- 172.21.15.5:0/4426958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4406f7c0 0x7f5e4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e48a38700 1 -- 172.21.15.5:0/4426958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4406f7c0 msgr2=0x7f5e4406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e48a38700 1 --2- 172.21.15.5:0/4426958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4406f7c0 0x7f5e4406e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e48a38700 1 -- 172.21.15.5:0/4426958 >> v1:172.21.15.5:6789/0 conn(0x7f5e4415db00 legacy=0x7f5e440caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e48a38700 1 -- 172.21.15.5:0/4426958 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4406ad20 con 0x7f5e440cc590 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e37fff700 1 -- 172.21.15.5:0/4426958 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699217399 0 0) 0x7f5e440cb160 con 0x7f5e4415db00 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e48a38700 1 --2- 172.21.15.5:0/4426958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e4406f280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e38009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8756c86a8a6ea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e37fff700 1 -- 172.21.15.5:0/4426958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38011830 con 0x7f5e440cc590 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e37fff700 1 -- 172.21.15.5:0/4426958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38011990 con 0x7f5e440cc590 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.971+0000 7f5e37fff700 1 -- 172.21.15.5:0/4426958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38011d20 con 0x7f5e440cc590 2024-05-08T23:21:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.972+0000 7f5e4b49d700 1 -- 172.21.15.5:0/4426958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 msgr2=0x7f5e4406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.972+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/4426958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e4406f280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e38009b80 comp rx=0 tx=0).stop 2024-05-08T23:21:19.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.972+0000 7f5e4b49d700 1 -- 172.21.15.5:0/4426958 shutdown_connections 2024-05-08T23:21:19.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.972+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/4426958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4406f7c0 0x7f5e4406e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.972+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/4426958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e4406f280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.972+0000 7f5e4b49d700 1 -- 172.21.15.5:0/4426958 >> 172.21.15.5:0/4426958 conn(0x7f5e44099a50 msgr2=0x7f5e440d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:19.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.972+0000 7f5e4b49d700 1 -- 172.21.15.5:0/4426958 shutdown_connections 2024-05-08T23:21:19.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.972+0000 7f5e4b49d700 1 -- 172.21.15.5:0/4426958 wait complete. 2024-05-08T23:21:19.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.974+0000 7f5e4b49d700 1 Processor -- start 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.974+0000 7f5e4b49d700 1 -- start start 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.974+0000 7f5e4b49d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e4406f7c0 0x7f5e440b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.974+0000 7f5e4b49d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e440c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.974+0000 7f5e4b49d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4415db00 0x7f5e440c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.974+0000 7f5e4b49d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b4010 con 0x7f5e440cc590 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.974+0000 7f5e4b49d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b4ff0 con 0x7f5e4406f7c0 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.974+0000 7f5e4b49d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b52d0 con 0x7f5e4415db00 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e440c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e440c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52832/0 (socket says 172.21.15.5:52832) 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 -- 172.21.15.5:0/2631711117 learned_addr learned my addr 172.21.15.5:0/2631711117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e49239700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e4406f7c0 0x7f5e440b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e49a3a700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4415db00 0x7f5e440c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 -- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4415db00 msgr2=0x7f5e440c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4415db00 0x7f5e440c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 -- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e4406f7c0 msgr2=0x7f5e440b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e4406f7c0 0x7f5e440b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 -- 172.21.15.5:0/2631711117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380094a0 con 0x7f5e440cc590 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e48a38700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e440c75f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e38000c00 tx=0x7f5e38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e35ffb700 1 -- 172.21.15.5:0/2631711117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380036a0 con 0x7f5e440cc590 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e35ffb700 1 -- 172.21.15.5:0/2631711117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3801f070 con 0x7f5e440cc590 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e440c4f80 con 0x7f5e440cc590 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.975+0000 7f5e35ffb700 1 -- 172.21.15.5:0/2631711117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38003a20 con 0x7f5e440cc590 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.976+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e440c5540 con 0x7f5e440cc590 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.977+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e440d1170 con 0x7f5e440cc590 2024-05-08T23:21:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.977+0000 7f5e35ffb700 1 -- 172.21.15.5:0/2631711117 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5e38003b80 con 0x7f5e440cc590 2024-05-08T23:21:19.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.977+0000 7f5e35ffb700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5e2806acc0 0x7f5e2806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:19.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.977+0000 7f5e35ffb700 1 -- 172.21.15.5:0/2631711117 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e38021070 con 0x7f5e440cc590 2024-05-08T23:21:19.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.978+0000 7f5e49239700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5e2806acc0 0x7f5e2806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:19.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.978+0000 7f5e49239700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5e2806acc0 0x7f5e2806d180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e40000f60 tx=0x7f5e40007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:19.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:19.982+0000 7f5e35ffb700 1 -- 172.21.15.5:0/2631711117 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5e38055d40 con 0x7f5e440cc590 2024-05-08T23:21:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.200+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e440b4150 con 0x7f5e440cc590 2024-05-08T23:21:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.201+0000 7f5e35ffb700 1 -- 172.21.15.5:0/2631711117 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e38015020 con 0x7f5e440cc590 2024-05-08T23:21:20.201 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5e2806acc0 msgr2=0x7f5e2806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5e2806acc0 0x7f5e2806d180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e40000f60 tx=0x7f5e40007e70 comp rx=0 tx=0).stop 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 msgr2=0x7f5e440c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e440c75f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e38000c00 tx=0x7f5e38004750 comp rx=0 tx=0).stop 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 shutdown_connections 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5e2806acc0 0x7f5e2806d180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5e4415db00 0x7f5e440c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e4406f7c0 0x7f5e440b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 --2- 172.21.15.5:0/2631711117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e440cc590 0x7f5e440c75f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 >> 172.21.15.5:0/2631711117 conn(0x7f5e44099a50 msgr2=0x7f5e440b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 shutdown_connections 2024-05-08T23:21:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.203+0000 7f5e4b49d700 1 -- 172.21.15.5:0/2631711117 wait complete. 2024-05-08T23:21:20.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:20.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.356+0000 7ff290456700 1 Processor -- start 2024-05-08T23:21:20.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.356+0000 7ff290456700 1 -- start start 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff290456700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2880d3d20 0x7ff2880b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff290456700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff2880b3fc0 0x7ff2880b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff290456700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2880cf610 con 0x7ff2880cff30 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff290456700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2880d1fc0 con 0x7ff2880d3d20 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff290456700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2880d22a0 con 0x7ff2880b3fc0 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28d9f1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2880d3d20 0x7ff2880b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28d9f1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2880d3d20 0x7ff2880b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52834/0 (socket says 172.21.15.5:52834) 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28d9f1700 1 -- 172.21.15.5:0/767838173 learned_addr learned my addr 172.21.15.5:0/767838173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28e9f3700 1 --2- 172.21.15.5:0/767838173 >> v2:172.21.15.49:3300/0 conn(0x7ff2880b3fc0 0x7ff2880b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28d9f1700 1 -- 172.21.15.5:0/767838173 >> v2:172.21.15.49:3300/0 conn(0x7ff2880b3fc0 msgr2=0x7ff2880b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28d9f1700 1 --2- 172.21.15.5:0/767838173 >> v2:172.21.15.49:3300/0 conn(0x7ff2880b3fc0 0x7ff2880b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28d9f1700 1 -- 172.21.15.5:0/767838173 >> v1:172.21.15.49:6789/0 conn(0x7ff2880cff30 legacy=0x7ff2880d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28d9f1700 1 -- 172.21.15.5:0/767838173 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880d2580 con 0x7ff2880d3d20 2024-05-08T23:21:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.357+0000 7ff28d9f1700 1 --2- 172.21.15.5:0/767838173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880d3d20 0x7ff2880b3a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff2880b4fc0 tx=0x7ff278009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef14501400dd2927 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff28d1f0700 1 -- 172.21.15.5:0/767838173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800b870 con 0x7ff2880d3d20 2024-05-08T23:21:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff28d1f0700 1 -- 172.21.15.5:0/767838173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27800b9d0 con 0x7ff2880d3d20 2024-05-08T23:21:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff28d1f0700 1 -- 172.21.15.5:0/767838173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800bd60 con 0x7ff2880d3d20 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff290456700 1 -- 172.21.15.5:0/767838173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880d3d20 msgr2=0x7ff2880b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff290456700 1 --2- 172.21.15.5:0/767838173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880d3d20 0x7ff2880b3a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff2880b4fc0 tx=0x7ff278009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff290456700 1 -- 172.21.15.5:0/767838173 shutdown_connections 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff290456700 1 --2- 172.21.15.5:0/767838173 >> v2:172.21.15.49:3300/0 conn(0x7ff2880b3fc0 0x7ff2880b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff290456700 1 --2- 172.21.15.5:0/767838173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880d3d20 0x7ff2880b3a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff290456700 1 -- 172.21.15.5:0/767838173 >> 172.21.15.5:0/767838173 conn(0x7ff28808afe0 msgr2=0x7ff2880d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff290456700 1 -- 172.21.15.5:0/767838173 shutdown_connections 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.358+0000 7ff290456700 1 -- 172.21.15.5:0/767838173 wait complete. 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.360+0000 7ff290456700 1 Processor -- start 2024-05-08T23:21:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.360+0000 7ff290456700 1 -- start start 2024-05-08T23:21:20.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.360+0000 7ff290456700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2880b3fc0 0x7ff28814e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.360+0000 7ff290456700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880cff30 0x7ff28815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.360+0000 7ff290456700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2880d3d20 0x7ff2880b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.360+0000 7ff290456700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880b7540 con 0x7ff2880cff30 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.360+0000 7ff290456700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28814ef10 con 0x7ff2880b3fc0 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff290456700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28815ccf0 con 0x7ff2880d3d20 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880cff30 0x7ff28815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880cff30 0x7ff28815ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52846/0 (socket says 172.21.15.5:52846) 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 -- 172.21.15.5:0/3370859413 learned_addr learned my addr 172.21.15.5:0/3370859413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28e1f2700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2880b3fc0 0x7ff28814e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 -- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2880d3d20 msgr2=0x7ff2880b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2880d3d20 0x7ff2880b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 -- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2880b3fc0 msgr2=0x7ff28814e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2880b3fc0 0x7ff28814e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 -- 172.21.15.5:0/3370859413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009550 con 0x7ff2880cff30 2024-05-08T23:21:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff28d9f1700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880cff30 0x7ff28815ee10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff278004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff27effd700 1 -- 172.21.15.5:0/3370859413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278004540 con 0x7ff2880cff30 2024-05-08T23:21:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff27effd700 1 -- 172.21.15.5:0/3370859413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2780036a0 con 0x7ff2880cff30 2024-05-08T23:21:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff27effd700 1 -- 172.21.15.5:0/3370859413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278003a50 con 0x7ff2880cff30 2024-05-08T23:21:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28815cfd0 con 0x7ff2880cff30 2024-05-08T23:21:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.361+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28815d590 con 0x7ff2880cff30 2024-05-08T23:21:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.362+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2880c7b10 con 0x7ff2880cff30 2024-05-08T23:21:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.363+0000 7ff27effd700 1 -- 172.21.15.5:0/3370859413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff278021070 con 0x7ff2880cff30 2024-05-08T23:21:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.363+0000 7ff27effd700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff26c06ac90 0x7ff26c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.363+0000 7ff27effd700 1 -- 172.21.15.5:0/3370859413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff27808aa30 con 0x7ff2880cff30 2024-05-08T23:21:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.363+0000 7ff28e1f2700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff26c06ac90 0x7ff26c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.364+0000 7ff28e1f2700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff26c06ac90 0x7ff26c06d150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff284000f60 tx=0x7ff284009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:20.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.368+0000 7ff27effd700 1 -- 172.21.15.5:0/3370859413 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff278055ca0 con 0x7ff2880cff30 2024-05-08T23:21:20.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.583+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff288074bc0 con 0x7ff2880cff30 2024-05-08T23:21:20.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.584+0000 7ff27effd700 1 -- 172.21.15.5:0/3370859413 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2780290a0 con 0x7ff2880cff30 2024-05-08T23:21:20.584 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:20.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff26c06ac90 msgr2=0x7ff26c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff26c06ac90 0x7ff26c06d150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff284000f60 tx=0x7ff284009040 comp rx=0 tx=0).stop 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880cff30 msgr2=0x7ff28815ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880cff30 0x7ff28815ee10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff278004750 comp rx=0 tx=0).stop 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 shutdown_connections 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff26c06ac90 0x7ff26c06d150 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2880d3d20 0x7ff2880b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2880b3fc0 0x7ff28814e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 --2- 172.21.15.5:0/3370859413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2880cff30 0x7ff28815ee10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 >> 172.21.15.5:0/3370859413 conn(0x7ff28808afe0 msgr2=0x7ff2880b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 shutdown_connections 2024-05-08T23:21:20.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.586+0000 7ff290456700 1 -- 172.21.15.5:0/3370859413 wait complete. 2024-05-08T23:21:20.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.735+0000 7f0d82ab1700 1 Processor -- start 2024-05-08T23:21:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.735+0000 7f0d82ab1700 1 -- start start 2024-05-08T23:21:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d82ab1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0d7c0cdef0 0x7f0d7c0cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d82ab1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0d7c0cc040 0x7f0d7c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d82ab1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c0ce740 con 0x7f0d7c15a630 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d82ab1700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0cb670 con 0x7f0d7c0cdef0 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d82ab1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0cadc0 con 0x7f0d7c0cc040 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d7bfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0d7c0cdef0 0x7f0d7c0cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d8084d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0d7c15a630 0x7f0d7c0d0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42518/0 (socket says 172.21.15.5:42518) 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d8084d700 1 -- 172.21.15.5:0/2454814859 learned_addr learned my addr 172.21.15.5:0/2454814859 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d8104e700 1 --2- 172.21.15.5:0/2454814859 >> v2:172.21.15.49:3300/0 conn(0x7f0d7c0cc040 0x7f0d7c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d7bfff700 1 -- 172.21.15.5:0/2454814859 >> v2:172.21.15.49:3300/0 conn(0x7f0d7c0cc040 msgr2=0x7f0d7c15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d7bfff700 1 --2- 172.21.15.5:0/2454814859 >> v2:172.21.15.49:3300/0 conn(0x7f0d7c0cc040 0x7f0d7c15e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d7bfff700 1 -- 172.21.15.5:0/2454814859 >> v1:172.21.15.5:6789/0 conn(0x7f0d7c15a630 legacy=0x7f0d7c0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d7bfff700 1 -- 172.21.15.5:0/2454814859 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0cb060 con 0x7f0d7c0cdef0 2024-05-08T23:21:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.736+0000 7f0d7b7fe700 1 -- 172.21.15.5:0/2454814859 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683012775 0 0) 0x7f0d6c001110 con 0x7f0d7c15a630 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d7bfff700 1 --2- 172.21.15.5:0/2454814859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cdef0 0x7f0d7c0cbb00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0729b0 tx=0x7f0d64009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8679dc513b2d869 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d7b7fe700 1 -- 172.21.15.5:0/2454814859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64011830 con 0x7f0d7c0cdef0 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d7b7fe700 1 -- 172.21.15.5:0/2454814859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d64011990 con 0x7f0d7c0cdef0 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d7b7fe700 1 -- 172.21.15.5:0/2454814859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64011d20 con 0x7f0d7c0cdef0 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d82ab1700 1 -- 172.21.15.5:0/2454814859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cdef0 msgr2=0x7f0d7c0cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/2454814859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cdef0 0x7f0d7c0cbb00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0729b0 tx=0x7f0d64009b20 comp rx=0 tx=0).stop 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d82ab1700 1 -- 172.21.15.5:0/2454814859 shutdown_connections 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/2454814859 >> v2:172.21.15.49:3300/0 conn(0x7f0d7c0cc040 0x7f0d7c15e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/2454814859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cdef0 0x7f0d7c0cbb00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d82ab1700 1 -- 172.21.15.5:0/2454814859 >> 172.21.15.5:0/2454814859 conn(0x7f0d7c099a50 msgr2=0x7f0d7c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d82ab1700 1 -- 172.21.15.5:0/2454814859 shutdown_connections 2024-05-08T23:21:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.737+0000 7f0d82ab1700 1 -- 172.21.15.5:0/2454814859 wait complete. 2024-05-08T23:21:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.739+0000 7f0d82ab1700 1 Processor -- start 2024-05-08T23:21:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.739+0000 7f0d82ab1700 1 -- start start 2024-05-08T23:21:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d82ab1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cc040 0x7f0d7c0af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d82ab1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d7c0cdef0 0x7f0d7c157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d82ab1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d7c15a630 0x7f0d7c0c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d82ab1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0ca380 con 0x7f0d7c0cc040 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d82ab1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0afbd0 con 0x7f0d7c0cdef0 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d82ab1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0afe90 con 0x7f0d7c15a630 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cc040 0x7f0d7c0af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cc040 0x7f0d7c0af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52858/0 (socket says 172.21.15.5:52858) 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 -- 172.21.15.5:0/1997628234 learned_addr learned my addr 172.21.15.5:0/1997628234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8104e700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d7c15a630 0x7f0d7c0c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d7bfff700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d7c0cdef0 0x7f0d7c157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 -- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d7c15a630 msgr2=0x7f0d7c0c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d7c15a630 0x7f0d7c0c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 -- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d7c0cdef0 msgr2=0x7f0d7c157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d7c0cdef0 0x7f0d7c157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 -- 172.21.15.5:0/1997628234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640094a0 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8104e700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d7c15a630 0x7f0d7c0c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d7bfff700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d7c0cdef0 0x7f0d7c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.740+0000 7f0d8084d700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cc040 0x7f0d7c0af690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00c550 tx=0x7f0d6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.741+0000 7f0d797fa700 1 -- 172.21.15.5:0/1997628234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c004c80 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.741+0000 7f0d797fa700 1 -- 172.21.15.5:0/1997628234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c004e00 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.741+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c0b0170 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.741+0000 7f0d797fa700 1 -- 172.21.15.5:0/1997628234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c007310 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.741+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c0b0730 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.742+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c068000 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.742+0000 7f0d797fa700 1 -- 172.21.15.5:0/1997628234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0d6c007470 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.743+0000 7f0d797fa700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d5c06ac90 0x7f0d5c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.743+0000 7f0d797fa700 1 -- 172.21.15.5:0/1997628234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d6c088ea0 con 0x7f0d7c0cc040 2024-05-08T23:21:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.743+0000 7f0d7bfff700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d5c06ac90 0x7f0d5c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.743+0000 7f0d7bfff700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d5c06ac90 0x7f0d5c06d150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0d640090b0 tx=0x7f0d64008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:20.747+0000 7f0d797fa700 1 -- 172.21.15.5:0/1997628234 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0d6c054080 con 0x7f0d7c0cc040 2024-05-08T23:21:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.013+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d7c0ce740 con 0x7f0d7c0cc040 2024-05-08T23:21:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.014+0000 7f0d797fa700 1 -- 172.21.15.5:0/1997628234 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4719 (secure 0 0 0) 0x7f0d7c0ce740 con 0x7f0d7c0cc040 2024-05-08T23:21:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:21.014 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:21:04.087368+0000","modified":"2024-05-08T23:21:07.094437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d5c06ac90 msgr2=0x7f0d5c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d5c06ac90 0x7f0d5c06d150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0d640090b0 tx=0x7f0d64008030 comp rx=0 tx=0).stop 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cc040 msgr2=0x7f0d7c0af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cc040 0x7f0d7c0af690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00c550 tx=0x7f0d6c00c910 comp rx=0 tx=0).stop 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 shutdown_connections 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0d5c06ac90 0x7f0d5c06d150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0d7c15a630 0x7f0d7c0c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c0cc040 0x7f0d7c0af690 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 --2- 172.21.15.5:0/1997628234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d7c0cdef0 0x7f0d7c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 >> 172.21.15.5:0/1997628234 conn(0x7f0d7c099a50 msgr2=0x7f0d7c06e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 shutdown_connections 2024-05-08T23:21:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.017+0000 7f0d82ab1700 1 -- 172.21.15.5:0/1997628234 wait complete. 2024-05-08T23:21:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-08T23:21:21.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.168+0000 7f981487d700 1 Processor -- start 2024-05-08T23:21:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.168+0000 7f981487d700 1 -- start start 2024-05-08T23:21:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.169+0000 7f981487d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f981006ef30 0x7f981006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.169+0000 7f981487d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98100cef60 con 0x7f981015dcc0 2024-05-08T23:21:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f981487d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981015eee0 con 0x7f981015e310 2024-05-08T23:21:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f981487d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98100b2de0 con 0x7f981006ef30 2024-05-08T23:21:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980dd9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f981006ef30 0x7f981006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980e59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f981015dcc0 0x7f98100b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42520/0 (socket says 172.21.15.5:42520) 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980e59c700 1 -- 172.21.15.5:0/565791108 learned_addr learned my addr 172.21.15.5:0/565791108 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980dd9b700 1 -- 172.21.15.5:0/565791108 >> v1:172.21.15.49:6789/0 conn(0x7f981015e310 legacy=0x7f98100adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980dd9b700 1 -- 172.21.15.5:0/565791108 >> v1:172.21.15.5:6789/0 conn(0x7f981015dcc0 legacy=0x7f98100b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980dd9b700 1 -- 172.21.15.5:0/565791108 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98100ce980 con 0x7f981006ef30 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980d59a700 1 -- 172.21.15.5:0/565791108 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927217943 0 0) 0x7f981015eee0 con 0x7f981015e310 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980dd9b700 1 --2- 172.21.15.5:0/565791108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981006ef30 0x7f981006fa10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9804009840 tx=0x7f9804009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7e9c197c7dfffa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980d59a700 1 -- 172.21.15.5:0/565791108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980400f8a0 con 0x7f981006ef30 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980d59a700 1 -- 172.21.15.5:0/565791108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980400fa00 con 0x7f981006ef30 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.170+0000 7f980d59a700 1 -- 172.21.15.5:0/565791108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980400fd90 con 0x7f981006ef30 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.171+0000 7f981487d700 1 -- 172.21.15.5:0/565791108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981006ef30 msgr2=0x7f981006fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.171+0000 7f981487d700 1 --2- 172.21.15.5:0/565791108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981006ef30 0x7f981006fa10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9804009840 tx=0x7f9804009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.171+0000 7f981487d700 1 -- 172.21.15.5:0/565791108 shutdown_connections 2024-05-08T23:21:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.171+0000 7f981487d700 1 --2- 172.21.15.5:0/565791108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981006ef30 0x7f981006fa10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.171+0000 7f981487d700 1 -- 172.21.15.5:0/565791108 >> 172.21.15.5:0/565791108 conn(0x7f9810099a50 msgr2=0x7f98100b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.171+0000 7f981487d700 1 -- 172.21.15.5:0/565791108 shutdown_connections 2024-05-08T23:21:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.171+0000 7f981487d700 1 -- 172.21.15.5:0/565791108 wait complete. 2024-05-08T23:21:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.173+0000 7f981487d700 1 Processor -- start 2024-05-08T23:21:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.173+0000 7f981487d700 1 -- start start 2024-05-08T23:21:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.173+0000 7f981487d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f981006ef30 0x7f9810158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.173+0000 7f981487d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981015dcc0 0x7f98100d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.173+0000 7f981487d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f981015e310 0x7f98100d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.173+0000 7f981487d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100c37e0 con 0x7f981015dcc0 2024-05-08T23:21:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.173+0000 7f981487d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100c63a0 con 0x7f981015e310 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.173+0000 7f981487d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98101f7860 con 0x7f981006ef30 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981015dcc0 0x7f98100d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981015dcc0 0x7f98100d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52884/0 (socket says 172.21.15.5:52884) 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 -- 172.21.15.5:0/502206033 learned_addr learned my addr 172.21.15.5:0/502206033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980e59c700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f981006ef30 0x7f9810158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980ed9d700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f981015e310 0x7f98100d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 -- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f981006ef30 msgr2=0x7f9810158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f981006ef30 0x7f9810158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 -- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f981015e310 msgr2=0x7f98100d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f981015e310 0x7f98100d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 -- 172.21.15.5:0/502206033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98040094a0 con 0x7f981015dcc0 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f980dd9b700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981015dcc0 0x7f98100d06f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f98040090b0 tx=0x7f9804004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f97feffd700 1 -- 172.21.15.5:0/502206033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804010c80 con 0x7f981015dcc0 2024-05-08T23:21:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f97feffd700 1 -- 172.21.15.5:0/502206033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9804010de0 con 0x7f981015dcc0 2024-05-08T23:21:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98100c65f0 con 0x7f981015dcc0 2024-05-08T23:21:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f97feffd700 1 -- 172.21.15.5:0/502206033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804023640 con 0x7f981015dcc0 2024-05-08T23:21:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.174+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98100c6bb0 con 0x7f981015dcc0 2024-05-08T23:21:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.175+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9810068000 con 0x7f981015dcc0 2024-05-08T23:21:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.176+0000 7f97feffd700 1 -- 172.21.15.5:0/502206033 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9804021070 con 0x7f981015dcc0 2024-05-08T23:21:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.176+0000 7f97feffd700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f97f006ac90 0x7f97f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.176+0000 7f97feffd700 1 -- 172.21.15.5:0/502206033 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f980408a280 con 0x7f981015dcc0 2024-05-08T23:21:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.177+0000 7f980e59c700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f97f006ac90 0x7f97f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.177+0000 7f980e59c700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f97f006ac90 0x7f97f006d150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9800000f60 tx=0x7f9800009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.181+0000 7f97feffd700 1 -- 172.21.15.5:0/502206033 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f98040555a0 con 0x7f981015dcc0 2024-05-08T23:21:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.400+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98100cd980 con 0x7f981015dcc0 2024-05-08T23:21:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.401+0000 7f97feffd700 1 -- 172.21.15.5:0/502206033 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7f9804058bc0 con 0x7f981015dcc0 2024-05-08T23:21:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:21.402 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f97f006ac90 msgr2=0x7f97f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f97f006ac90 0x7f97f006d150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9800000f60 tx=0x7f9800009040 comp rx=0 tx=0).stop 2024-05-08T23:21:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981015dcc0 msgr2=0x7f98100d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981015dcc0 0x7f98100d06f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f98040090b0 tx=0x7f9804004750 comp rx=0 tx=0).stop 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 shutdown_connections 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f97f006ac90 0x7f97f006d150 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f981015e310 0x7f98100d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f981006ef30 0x7f9810158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 --2- 172.21.15.5:0/502206033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f981015dcc0 0x7f98100d06f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 >> 172.21.15.5:0/502206033 conn(0x7f9810099a50 msgr2=0x7f98100ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 shutdown_connections 2024-05-08T23:21:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.404+0000 7f981487d700 1 -- 172.21.15.5:0/502206033 wait complete. 2024-05-08T23:21:21.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:21.418 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.557+0000 7f115938c700 1 Processor -- start 2024-05-08T23:21:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.557+0000 7f115938c700 1 -- start start 2024-05-08T23:21:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f115938c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f115415d700 0x7f11540b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f115938c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11540b38b0 0x7f11541614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f115938c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11540b49b0 con 0x7f115415dae0 2024-05-08T23:21:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f115938c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11540b1150 con 0x7f115415d700 2024-05-08T23:21:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f115938c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11540b3da0 con 0x7f11540b38b0 2024-05-08T23:21:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1152ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f115415d700 0x7f11540b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1152ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f115415d700 0x7f11540b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52886/0 (socket says 172.21.15.5:52886) 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1152ffd700 1 -- 172.21.15.5:0/1263188164 learned_addr learned my addr 172.21.15.5:0/1263188164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f11527fc700 1 --2- 172.21.15.5:0/1263188164 >> v2:172.21.15.49:3300/0 conn(0x7f11540b38b0 0x7f11541614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1152ffd700 1 -- 172.21.15.5:0/1263188164 >> v2:172.21.15.49:3300/0 conn(0x7f11540b38b0 msgr2=0x7f11541614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1152ffd700 1 --2- 172.21.15.5:0/1263188164 >> v2:172.21.15.49:3300/0 conn(0x7f11540b38b0 0x7f11541614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1152ffd700 1 -- 172.21.15.5:0/1263188164 >> v1:172.21.15.49:6789/0 conn(0x7f115415dae0 legacy=0x7f11540d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1152ffd700 1 -- 172.21.15.5:0/1263188164 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540b4080 con 0x7f115415d700 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1152ffd700 1 --2- 172.21.15.5:0/1263188164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540b3370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11540ce190 tx=0x7f1144009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a105cec884ef99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1151ffb700 1 -- 172.21.15.5:0/1263188164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144005e30 con 0x7f115415d700 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1151ffb700 1 -- 172.21.15.5:0/1263188164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1144004030 con 0x7f115415d700 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.558+0000 7f1151ffb700 1 -- 172.21.15.5:0/1263188164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11440043e0 con 0x7f115415d700 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.559+0000 7f115938c700 1 -- 172.21.15.5:0/1263188164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 msgr2=0x7f11540b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.559+0000 7f115938c700 1 --2- 172.21.15.5:0/1263188164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540b3370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11540ce190 tx=0x7f1144009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.559+0000 7f115938c700 1 -- 172.21.15.5:0/1263188164 shutdown_connections 2024-05-08T23:21:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.559+0000 7f115938c700 1 --2- 172.21.15.5:0/1263188164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540b3370 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.559+0000 7f115938c700 1 --2- 172.21.15.5:0/1263188164 >> v2:172.21.15.49:3300/0 conn(0x7f11540b38b0 0x7f11541614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.559+0000 7f115938c700 1 -- 172.21.15.5:0/1263188164 >> 172.21.15.5:0/1263188164 conn(0x7f115408afe0 msgr2=0x7f11540d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.559+0000 7f115938c700 1 -- 172.21.15.5:0/1263188164 shutdown_connections 2024-05-08T23:21:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.559+0000 7f115938c700 1 -- 172.21.15.5:0/1263188164 wait complete. 2024-05-08T23:21:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.561+0000 7f115938c700 1 Processor -- start 2024-05-08T23:21:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.561+0000 7f115938c700 1 -- start start 2024-05-08T23:21:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f115938c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11540b38b0 0x7f11540c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f115938c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f115938c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f115415dae0 0x7f115415e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f115938c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11541f7590 con 0x7f115415d700 2024-05-08T23:21:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f115938c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11541f66c0 con 0x7f11540b38b0 2024-05-08T23:21:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f115938c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11541f6910 con 0x7f115415dae0 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f11527fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f11527fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52896/0 (socket says 172.21.15.5:52896) 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f11527fc700 1 -- 172.21.15.5:0/2926881100 learned_addr learned my addr 172.21.15.5:0/2926881100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f1152ffd700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11540b38b0 0x7f11540c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.562+0000 7f11537fe700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f115415dae0 0x7f115415e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f11527fc700 1 -- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f115415dae0 msgr2=0x7f115415e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f11527fc700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f115415dae0 0x7f115415e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f11527fc700 1 -- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11540b38b0 msgr2=0x7f11540c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f11527fc700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11540b38b0 0x7f11540c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f11527fc700 1 -- 172.21.15.5:0/2926881100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1144009500 con 0x7f115415d700 2024-05-08T23:21:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f11527fc700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540cf6b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f114800e9b0 tx=0x7f114800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f113b7fe700 1 -- 172.21.15.5:0/2926881100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148009710 con 0x7f115415d700 2024-05-08T23:21:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f113b7fe700 1 -- 172.21.15.5:0/2926881100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114800ee50 con 0x7f115415d700 2024-05-08T23:21:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11541f6bf0 con 0x7f115415d700 2024-05-08T23:21:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f113b7fe700 1 -- 172.21.15.5:0/2926881100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114800ca50 con 0x7f115415d700 2024-05-08T23:21:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.563+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11541f71b0 con 0x7f115415d700 2024-05-08T23:21:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.564+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11540b55f0 con 0x7f115415d700 2024-05-08T23:21:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.564+0000 7f113b7fe700 1 -- 172.21.15.5:0/2926881100 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f114800cbb0 con 0x7f115415d700 2024-05-08T23:21:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.565+0000 7f113b7fe700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f114006ac90 0x7f114006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.565+0000 7f113b7fe700 1 -- 172.21.15.5:0/2926881100 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1148089260 con 0x7f115415d700 2024-05-08T23:21:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.565+0000 7f1152ffd700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f114006ac90 0x7f114006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.566+0000 7f1152ffd700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f114006ac90 0x7f114006d150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1144009c10 tx=0x7f1144008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.570+0000 7f113b7fe700 1 -- 172.21.15.5:0/2926881100 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f11540b55f0 con 0x7f115415d700 2024-05-08T23:21:21.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.786+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11540b55f0 con 0x7f115415d700 2024-05-08T23:21:21.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.786+0000 7f113b7fe700 1 -- 172.21.15.5:0/2926881100 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11540b55f0 con 0x7f115415d700 2024-05-08T23:21:21.786 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f114006ac90 msgr2=0x7f114006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f114006ac90 0x7f114006d150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1144009c10 tx=0x7f1144008070 comp rx=0 tx=0).stop 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 msgr2=0x7f11540cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540cf6b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f114800e9b0 tx=0x7f114800ecc0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 shutdown_connections 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f114006ac90 0x7f114006d150 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f115415dae0 0x7f115415e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11540b38b0 0x7f11540c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 --2- 172.21.15.5:0/2926881100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115415d700 0x7f11540cf6b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 >> 172.21.15.5:0/2926881100 conn(0x7f115408afe0 msgr2=0x7f11540d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 shutdown_connections 2024-05-08T23:21:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.789+0000 7f115938c700 1 -- 172.21.15.5:0/2926881100 wait complete. 2024-05-08T23:21:21.804 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:21.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.943+0000 7f3ca71d1700 1 Processor -- start 2024-05-08T23:21:21.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.943+0000 7f3ca71d1700 1 -- start start 2024-05-08T23:21:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3ca71d1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca006a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3ca71d1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ca006aea0 0x7f3ca01454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3ca71d1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca0145040 con 0x7f3ca00b0d80 2024-05-08T23:21:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3ca71d1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca006b330 con 0x7f3ca006aea0 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3ca71d1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00b16e0 con 0x7f3ca00b4d50 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3ca4f6d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3ca00b0d80 0x7f3ca00b3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42534/0 (socket says 172.21.15.5:42534) 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3ca4f6d700 1 -- 172.21.15.5:0/498665994 learned_addr learned my addr 172.21.15.5:0/498665994 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3ca576e700 1 --2- 172.21.15.5:0/498665994 >> v2:172.21.15.49:3300/0 conn(0x7f3ca006aea0 0x7f3ca01454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.944+0000 7f3c9ffff700 1 --2- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3c9f7fe700 1 -- 172.21.15.5:0/498665994 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385846938 0 0) 0x7f3ca0145040 con 0x7f3ca00b0d80 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3c9f7fe700 1 -- 172.21.15.5:0/498665994 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c880035b0 con 0x7f3ca00b0d80 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3ca576e700 1 -- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 msgr2=0x7f3ca006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3c9f7fe700 1 -- 172.21.15.5:0/498665994 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2220103866 0 0) 0x7f3c880035b0 con 0x7f3ca00b0d80 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3ca576e700 1 --2- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3ca576e700 1 -- 172.21.15.5:0/498665994 >> v1:172.21.15.5:6789/0 conn(0x7f3ca00b0d80 legacy=0x7f3ca00b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3ca576e700 1 -- 172.21.15.5:0/498665994 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00b19c0 con 0x7f3ca006aea0 2024-05-08T23:21:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3c9ffff700 1 --2- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:21.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3ca576e700 1 --2- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca006aea0 0x7f3ca01454d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009d90 tx=0x7f3c9800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c36d795f82cd46c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3c9f7fe700 1 -- 172.21.15.5:0/498665994 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800e070 con 0x7f3ca006aea0 2024-05-08T23:21:21.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3c9f7fe700 1 -- 172.21.15.5:0/498665994 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c98003a20 con 0x7f3ca006aea0 2024-05-08T23:21:21.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.945+0000 7f3c9f7fe700 1 -- 172.21.15.5:0/498665994 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90001670 con 0x7f3ca006aea0 2024-05-08T23:21:21.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.946+0000 7f3ca71d1700 1 -- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca006aea0 msgr2=0x7f3ca01454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.946+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca006aea0 0x7f3ca01454d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009d90 tx=0x7f3c9800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.946+0000 7f3ca71d1700 1 -- 172.21.15.5:0/498665994 shutdown_connections 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.946+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca006aea0 0x7f3ca01454d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.946+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/498665994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.946+0000 7f3ca71d1700 1 -- 172.21.15.5:0/498665994 >> 172.21.15.5:0/498665994 conn(0x7f3ca0099a50 msgr2=0x7f3ca00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.946+0000 7f3ca71d1700 1 -- 172.21.15.5:0/498665994 shutdown_connections 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.946+0000 7f3ca71d1700 1 -- 172.21.15.5:0/498665994 wait complete. 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.947+0000 7f3ca71d1700 1 Processor -- start 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.947+0000 7f3ca71d1700 1 -- start start 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca71d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca006aea0 0x7f3ca01390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca71d1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca00b0d80 0x7f3ca01f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca71d1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca015d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca71d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00ca8a0 con 0x7f3ca006aea0 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca71d1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca01f4140 con 0x7f3ca00b0d80 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca71d1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca01f3e50 con 0x7f3ca00b4d50 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca4f6d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca006aea0 0x7f3ca01390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca4f6d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca006aea0 0x7f3ca01390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52908/0 (socket says 172.21.15.5:52908) 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.948+0000 7f3ca4f6d700 1 -- 172.21.15.5:0/3763288932 learned_addr learned my addr 172.21.15.5:0/3763288932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3c9ffff700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca00b0d80 0x7f3ca01f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca576e700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca4f6d700 1 -- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 msgr2=0x7f3ca015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca4f6d700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca015d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca4f6d700 1 -- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca00b0d80 msgr2=0x7f3ca01f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca4f6d700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca00b0d80 0x7f3ca01f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca4f6d700 1 -- 172.21.15.5:0/3763288932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c940094a0 con 0x7f3ca006aea0 2024-05-08T23:21:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca4f6d700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca006aea0 0x7f3ca01390d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3c900011e0 tx=0x7f3c9000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3c9d7fa700 1 -- 172.21.15.5:0/3763288932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90004c80 con 0x7f3ca006aea0 2024-05-08T23:21:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3c9d7fa700 1 -- 172.21.15.5:0/3763288932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c90004e00 con 0x7f3ca006aea0 2024-05-08T23:21:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c98009a70 con 0x7f3ca006aea0 2024-05-08T23:21:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3c9d7fa700 1 -- 172.21.15.5:0/3763288932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90007310 con 0x7f3ca006aea0 2024-05-08T23:21:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.949+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca009e070 con 0x7f3ca006aea0 2024-05-08T23:21:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.950+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca0074bc0 con 0x7f3ca006aea0 2024-05-08T23:21:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.951+0000 7f3c9d7fa700 1 -- 172.21.15.5:0/3763288932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3c90007470 con 0x7f3ca006aea0 2024-05-08T23:21:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.951+0000 7f3c9d7fa700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c8c06ac90 0x7f3c8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.951+0000 7f3c9d7fa700 1 -- 172.21.15.5:0/3763288932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c90089080 con 0x7f3ca006aea0 2024-05-08T23:21:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.951+0000 7f3c9ffff700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c8c06ac90 0x7f3c8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.952+0000 7f3c9ffff700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c8c06ac90 0x7f3c8c06d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009cd0 tx=0x7f3c94009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:21.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:21.956+0000 7f3c9d7fa700 1 -- 172.21.15.5:0/3763288932 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3c900541e0 con 0x7f3ca006aea0 2024-05-08T23:21:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.179+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ca006dd90 con 0x7f3ca006aea0 2024-05-08T23:21:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.179+0000 7f3c9d7fa700 1 -- 172.21.15.5:0/3763288932 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c90057800 con 0x7f3ca006aea0 2024-05-08T23:21:22.180 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.181+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c8c06ac90 msgr2=0x7f3c8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.181+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c8c06ac90 0x7f3c8c06d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009cd0 tx=0x7f3c94009210 comp rx=0 tx=0).stop 2024-05-08T23:21:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.181+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca006aea0 msgr2=0x7f3ca01390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.181+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca006aea0 0x7f3ca01390d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3c900011e0 tx=0x7f3c9000c9e0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.182+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 shutdown_connections 2024-05-08T23:21:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.182+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c8c06ac90 0x7f3c8c06d150 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.182+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ca00b4d50 0x7f3ca015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.182+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca006aea0 0x7f3ca01390d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.182+0000 7f3ca71d1700 1 --2- 172.21.15.5:0/3763288932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ca00b0d80 0x7f3ca01f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.182+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 >> 172.21.15.5:0/3763288932 conn(0x7f3ca0099a50 msgr2=0x7f3ca0146400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.182+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 shutdown_connections 2024-05-08T23:21:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.182+0000 7f3ca71d1700 1 -- 172.21.15.5:0/3763288932 wait complete. 2024-05-08T23:21:22.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.335+0000 7fba7564a700 1 Processor -- start 2024-05-08T23:21:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.335+0000 7fba7564a700 1 -- start start 2024-05-08T23:21:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba7564a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba7015dde0 0x7fba700cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba7564a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b4c80 0x7fba700d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba7564a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fba700b2e30 0x7fba700b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba7564a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba700ceef0 con 0x7fba7015dde0 2024-05-08T23:21:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba6e7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b4c80 0x7fba700d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba7564a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7015e1c0 con 0x7fba700b2e30 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba7564a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba700cf140 con 0x7fba700b4c80 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba6effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba7015dde0 0x7fba700cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba6e7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b4c80 0x7fba700d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:32782/0 (socket says 172.21.15.5:32782) 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.336+0000 7fba6e7fc700 1 -- 172.21.15.5:0/188240826 learned_addr learned my addr 172.21.15.5:0/188240826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6f7fe700 1 --2- 172.21.15.5:0/188240826 >> v2:172.21.15.49:3300/0 conn(0x7fba700b2e30 0x7fba700b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6effd700 1 -- 172.21.15.5:0/188240826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b4c80 msgr2=0x7fba700d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6effd700 1 --2- 172.21.15.5:0/188240826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b4c80 0x7fba700d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6effd700 1 -- 172.21.15.5:0/188240826 >> v2:172.21.15.49:3300/0 conn(0x7fba700b2e30 msgr2=0x7fba700b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6effd700 1 --2- 172.21.15.5:0/188240826 >> v2:172.21.15.49:3300/0 conn(0x7fba700b2e30 0x7fba700b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6effd700 1 -- 172.21.15.5:0/188240826 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700cf420 con 0x7fba7015dde0 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6effd700 1 --2- 172.21.15.5:0/188240826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700cffa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fba7008b480 tx=0x7fba60009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc8d50ea1160c9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6dffb700 1 -- 172.21.15.5:0/188240826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60011830 con 0x7fba7015dde0 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6dffb700 1 -- 172.21.15.5:0/188240826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba60011990 con 0x7fba7015dde0 2024-05-08T23:21:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.337+0000 7fba6dffb700 1 -- 172.21.15.5:0/188240826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60011d20 con 0x7fba7015dde0 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 -- 172.21.15.5:0/188240826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 msgr2=0x7fba700cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 --2- 172.21.15.5:0/188240826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700cffa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fba7008b480 tx=0x7fba60009b20 comp rx=0 tx=0).stop 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 -- 172.21.15.5:0/188240826 shutdown_connections 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 --2- 172.21.15.5:0/188240826 >> v2:172.21.15.49:3300/0 conn(0x7fba700b2e30 0x7fba700b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 --2- 172.21.15.5:0/188240826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700cffa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 --2- 172.21.15.5:0/188240826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b4c80 0x7fba700d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 -- 172.21.15.5:0/188240826 >> 172.21.15.5:0/188240826 conn(0x7fba7008afe0 msgr2=0x7fba700d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 -- 172.21.15.5:0/188240826 shutdown_connections 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.338+0000 7fba7564a700 1 -- 172.21.15.5:0/188240826 wait complete. 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.339+0000 7fba7564a700 1 Processor -- start 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.339+0000 7fba7564a700 1 -- start start 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba7564a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b2e30 0x7fba70151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba7564a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba700b4c80 0x7fba700b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba7564a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba7564a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba70151e80 con 0x7fba7015dde0 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba7564a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba700cb720 con 0x7fba700b4c80 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba7564a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7015e460 con 0x7fba700b2e30 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52930/0 (socket says 172.21.15.5:52930) 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6f7fe700 1 -- 172.21.15.5:0/2958080374 learned_addr learned my addr 172.21.15.5:0/2958080374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6effd700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b2e30 0x7fba70151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6e7fc700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba700b4c80 0x7fba700b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6f7fe700 1 -- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b2e30 msgr2=0x7fba70151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6f7fe700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b2e30 0x7fba70151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6f7fe700 1 -- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba700b4c80 msgr2=0x7fba700b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.340+0000 7fba6f7fe700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba700b4c80 0x7fba700b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.341+0000 7fba6f7fe700 1 -- 172.21.15.5:0/2958080374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600094a0 con 0x7fba7015dde0 2024-05-08T23:21:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.341+0000 7fba6f7fe700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700b10c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba6800eed0 tx=0x7fba6800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.341+0000 7fba577fe700 1 -- 172.21.15.5:0/2958080374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68010040 con 0x7fba7015dde0 2024-05-08T23:21:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.341+0000 7fba577fe700 1 -- 172.21.15.5:0/2958080374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6800bd30 con 0x7fba7015dde0 2024-05-08T23:21:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.341+0000 7fba577fe700 1 -- 172.21.15.5:0/2958080374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68009ea0 con 0x7fba7015dde0 2024-05-08T23:21:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.341+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7015e740 con 0x7fba7015dde0 2024-05-08T23:21:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.341+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7015ed00 con 0x7fba7015dde0 2024-05-08T23:21:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.342+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7015ef20 con 0x7fba7015dde0 2024-05-08T23:21:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.343+0000 7fba577fe700 1 -- 172.21.15.5:0/2958080374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fba68014640 con 0x7fba7015dde0 2024-05-08T23:21:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.343+0000 7fba577fe700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba5c06acc0 0x7fba5c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.343+0000 7fba577fe700 1 -- 172.21.15.5:0/2958080374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba68089bb0 con 0x7fba7015dde0 2024-05-08T23:21:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.343+0000 7fba6effd700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba5c06acc0 0x7fba5c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.344+0000 7fba6effd700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba5c06acc0 0x7fba5c06d180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba600090b0 tx=0x7fba60008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.348+0000 7fba577fe700 1 -- 172.21.15.5:0/2958080374 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fba68054e20 con 0x7fba7015dde0 2024-05-08T23:21:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.573+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba700b94e0 con 0x7fba7015dde0 2024-05-08T23:21:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.573+0000 7fba577fe700 1 -- 172.21.15.5:0/2958080374 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba680180a0 con 0x7fba7015dde0 2024-05-08T23:21:22.574 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.575+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba5c06acc0 msgr2=0x7fba5c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.575+0000 7fba7564a700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba5c06acc0 0x7fba5c06d180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba600090b0 tx=0x7fba60008030 comp rx=0 tx=0).stop 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.575+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 msgr2=0x7fba700b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.575+0000 7fba7564a700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700b10c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba6800eed0 tx=0x7fba6800b390 comp rx=0 tx=0).stop 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.576+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 shutdown_connections 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.576+0000 7fba7564a700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba5c06acc0 0x7fba5c06d180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.576+0000 7fba7564a700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba7015dde0 0x7fba700b10c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.576+0000 7fba7564a700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba700b2e30 0x7fba70151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.576+0000 7fba7564a700 1 --2- 172.21.15.5:0/2958080374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba700b4c80 0x7fba700b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.576+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 >> 172.21.15.5:0/2958080374 conn(0x7fba7008afe0 msgr2=0x7fba700b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:22.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.576+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 shutdown_connections 2024-05-08T23:21:22.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.576+0000 7fba7564a700 1 -- 172.21.15.5:0/2958080374 wait complete. 2024-05-08T23:21:22.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.727+0000 7f0213442700 1 Processor -- start 2024-05-08T23:21:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.727+0000 7f0213442700 1 -- start start 2024-05-08T23:21:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f0213442700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f020c0ce120 0x7f020c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f0213442700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f020c0cb2a0 0x7f020c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f0213442700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c06fbf0 con 0x7f020c15db00 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f0213442700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c06edb0 con 0x7f020c0ce120 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f0213442700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c06f090 con 0x7f020c0cb2a0 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f02109dd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f020c0ce120 0x7f020c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f02109dd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f020c0ce120 0x7f020c06e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52942/0 (socket says 172.21.15.5:52942) 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f02109dd700 1 -- 172.21.15.5:0/1222879681 learned_addr learned my addr 172.21.15.5:0/1222879681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f02119df700 1 --2- 172.21.15.5:0/1222879681 >> v2:172.21.15.49:3300/0 conn(0x7f020c0cb2a0 0x7f020c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f02109dd700 1 -- 172.21.15.5:0/1222879681 >> v2:172.21.15.49:3300/0 conn(0x7f020c0cb2a0 msgr2=0x7f020c0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.728+0000 7f02109dd700 1 --2- 172.21.15.5:0/1222879681 >> v2:172.21.15.49:3300/0 conn(0x7f020c0cb2a0 0x7f020c0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.729+0000 7f02109dd700 1 -- 172.21.15.5:0/1222879681 >> v1:172.21.15.49:6789/0 conn(0x7f020c15db00 legacy=0x7f020c06dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.729+0000 7f02109dd700 1 -- 172.21.15.5:0/1222879681 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c06f370 con 0x7f020c0ce120 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.729+0000 7f02109dd700 1 --2- 172.21.15.5:0/1222879681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0ce120 0x7f020c06e390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f020c0727d0 tx=0x7f01fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da070dbda215a30e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.729+0000 7f0203fff700 1 -- 172.21.15.5:0/1222879681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc004230 con 0x7f020c0ce120 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.729+0000 7f0203fff700 1 -- 172.21.15.5:0/1222879681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc004390 con 0x7f020c0ce120 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.729+0000 7f0203fff700 1 -- 172.21.15.5:0/1222879681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0036a0 con 0x7f020c0ce120 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.730+0000 7f0213442700 1 -- 172.21.15.5:0/1222879681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0ce120 msgr2=0x7f020c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.730+0000 7f0213442700 1 --2- 172.21.15.5:0/1222879681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0ce120 0x7f020c06e390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f020c0727d0 tx=0x7f01fc009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.730+0000 7f0213442700 1 -- 172.21.15.5:0/1222879681 shutdown_connections 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.730+0000 7f0213442700 1 --2- 172.21.15.5:0/1222879681 >> v2:172.21.15.49:3300/0 conn(0x7f020c0cb2a0 0x7f020c0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.730+0000 7f0213442700 1 --2- 172.21.15.5:0/1222879681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0ce120 0x7f020c06e390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.730+0000 7f0213442700 1 -- 172.21.15.5:0/1222879681 >> 172.21.15.5:0/1222879681 conn(0x7f020c099a50 msgr2=0x7f020c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.730+0000 7f0213442700 1 -- 172.21.15.5:0/1222879681 shutdown_connections 2024-05-08T23:21:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.730+0000 7f0213442700 1 -- 172.21.15.5:0/1222879681 wait complete. 2024-05-08T23:21:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.731+0000 7f0213442700 1 Processor -- start 2024-05-08T23:21:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.731+0000 7f0213442700 1 -- start start 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.732+0000 7f0213442700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0cb2a0 0x7f020c14c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.732+0000 7f0213442700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f020c0ce120 0x7f020c14ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.732+0000 7f0213442700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f020c15db00 0x7f020c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.732+0000 7f0213442700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c14cf30 con 0x7f020c0cb2a0 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.732+0000 7f0213442700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c14d0a0 con 0x7f020c0ce120 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.732+0000 7f0213442700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c14d2b0 con 0x7f020c15db00 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0cb2a0 0x7f020c14c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0cb2a0 0x7f020c14c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52954/0 (socket says 172.21.15.5:52954) 2024-05-08T23:21:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 -- 172.21.15.5:0/1266452377 learned_addr learned my addr 172.21.15.5:0/1266452377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02109dd700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f020c0ce120 0x7f020c14ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02119df700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f020c15db00 0x7f020c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 -- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f020c15db00 msgr2=0x7f020c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f020c15db00 0x7f020c06a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 -- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f020c0ce120 msgr2=0x7f020c14ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f020c0ce120 0x7f020c14ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 -- 172.21.15.5:0/1266452377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc009550 con 0x7f020c0cb2a0 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f02111de700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0cb2a0 0x7f020c14c9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f020800c4d0 tx=0x7f020800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f0201ffb700 1 -- 172.21.15.5:0/1266452377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208011790 con 0x7f020c0cb2a0 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f0201ffb700 1 -- 172.21.15.5:0/1266452377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02080118f0 con 0x7f020c0cb2a0 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c14d590 con 0x7f020c0cb2a0 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.733+0000 7f0201ffb700 1 -- 172.21.15.5:0/1266452377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208011ca0 con 0x7f020c0cb2a0 2024-05-08T23:21:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.734+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c14db50 con 0x7f020c0cb2a0 2024-05-08T23:21:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.735+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c068000 con 0x7f020c0cb2a0 2024-05-08T23:21:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.735+0000 7f0201ffb700 1 -- 172.21.15.5:0/1266452377 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f020801c460 con 0x7f020c0cb2a0 2024-05-08T23:21:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.735+0000 7f0201ffb700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01f006ac90 0x7f01f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.736+0000 7f02109dd700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01f006ac90 0x7f01f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.736+0000 7f0201ffb700 1 -- 172.21.15.5:0/1266452377 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0208089010 con 0x7f020c0cb2a0 2024-05-08T23:21:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.736+0000 7f02109dd700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01f006ac90 0x7f01f006d150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009130 tx=0x7f01fc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:22.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:22.740+0000 7f0201ffb700 1 -- 172.21.15.5:0/1266452377 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f02080541f0 con 0x7f020c0cb2a0 2024-05-08T23:21:23.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.007+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f020c15eee0 con 0x7f020c0cb2a0 2024-05-08T23:21:23.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.008+0000 7f0201ffb700 1 -- 172.21.15.5:0/1266452377 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4719 (secure 0 0 0) 0x7f0208057810 con 0x7f020c0cb2a0 2024-05-08T23:21:23.008 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:23.008 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:21:04.087368+0000","modified":"2024-05-08T23:21:07.094437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:23.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01f006ac90 msgr2=0x7f01f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01f006ac90 0x7f01f006d150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009130 tx=0x7f01fc008030 comp rx=0 tx=0).stop 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0cb2a0 msgr2=0x7f020c14c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0cb2a0 0x7f020c14c9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f020800c4d0 tx=0x7f020800c7e0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 shutdown_connections 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01f006ac90 0x7f01f006d150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f020c15db00 0x7f020c06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f020c0cb2a0 0x7f020c14c9f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 --2- 172.21.15.5:0/1266452377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f020c0ce120 0x7f020c14ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 >> 172.21.15.5:0/1266452377 conn(0x7f020c099a50 msgr2=0x7f020c15e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 shutdown_connections 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.010+0000 7f0213442700 1 -- 172.21.15.5:0/1266452377 wait complete. 2024-05-08T23:21:23.011 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-08T23:21:23.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.160+0000 7fa3fbeca700 1 Processor -- start 2024-05-08T23:21:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.160+0000 7fa3fbeca700 1 -- start start 2024-05-08T23:21:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.161+0000 7fa3fbeca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f4168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.161+0000 7fa3fbeca700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f409a870 con 0x7fa3f4161030 2024-05-08T23:21:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.161+0000 7fa3fbeca700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f40d0840 con 0x7fa3f40b2480 2024-05-08T23:21:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.161+0000 7fa3fbeca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f406ccc0 con 0x7fa3f40ccbd0 2024-05-08T23:21:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.161+0000 7fa3f9c66700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa3f4161030 0x7fa3f4099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42542/0 (socket says 172.21.15.5:42542) 2024-05-08T23:21:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.161+0000 7fa3f9c66700 1 -- 172.21.15.5:0/778807398 learned_addr learned my addr 172.21.15.5:0/778807398 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.161+0000 7fa3fa467700 1 --2- 172.21.15.5:0/778807398 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f4168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.162+0000 7fa3fa467700 1 -- 172.21.15.5:0/778807398 >> v1:172.21.15.49:6789/0 conn(0x7fa3f40b2480 legacy=0x7fa3f409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.162+0000 7fa3fa467700 1 -- 172.21.15.5:0/778807398 >> v1:172.21.15.5:6789/0 conn(0x7fa3f4161030 legacy=0x7fa3f4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.162+0000 7fa3fa467700 1 -- 172.21.15.5:0/778807398 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f406c600 con 0x7fa3f40ccbd0 2024-05-08T23:21:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.162+0000 7fa3fa467700 1 --2- 172.21.15.5:0/778807398 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f4168580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009dc0 tx=0x7fa3ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e56c1684a221bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:23.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3f8c64700 1 -- 172.21.15.5:0/778807398 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00e070 con 0x7fa3f40ccbd0 2024-05-08T23:21:23.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3f8c64700 1 -- 172.21.15.5:0/778807398 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec00bbd0 con 0x7fa3f40ccbd0 2024-05-08T23:21:23.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3f8c64700 1 -- 172.21.15.5:0/778807398 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec012430 con 0x7fa3f40ccbd0 2024-05-08T23:21:23.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3fbeca700 1 -- 172.21.15.5:0/778807398 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 msgr2=0x7fa3f4168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3fbeca700 1 --2- 172.21.15.5:0/778807398 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f4168580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009dc0 tx=0x7fa3ec00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3fbeca700 1 -- 172.21.15.5:0/778807398 shutdown_connections 2024-05-08T23:21:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3fbeca700 1 --2- 172.21.15.5:0/778807398 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f4168580 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3fbeca700 1 -- 172.21.15.5:0/778807398 >> 172.21.15.5:0/778807398 conn(0x7fa3f40b2f20 msgr2=0x7fa3f40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3fbeca700 1 -- 172.21.15.5:0/778807398 shutdown_connections 2024-05-08T23:21:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.165+0000 7fa3fbeca700 1 -- 172.21.15.5:0/778807398 wait complete. 2024-05-08T23:21:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.167+0000 7fa3fbeca700 1 Processor -- start 2024-05-08T23:21:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.167+0000 7fa3fbeca700 1 -- start start 2024-05-08T23:21:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fbeca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3f40b2480 0x7fa3f414b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fbeca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f40c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fbeca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f4161030 0x7fa3f40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fbeca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40d0660 con 0x7fa3f4161030 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fbeca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40b7130 con 0x7fa3f40b2480 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fbeca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40cafa0 con 0x7fa3f40ccbd0 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fa467700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f4161030 0x7fa3f40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fa467700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f4161030 0x7fa3f40ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52964/0 (socket says 172.21.15.5:52964) 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fa467700 1 -- 172.21.15.5:0/1201701028 learned_addr learned my addr 172.21.15.5:0/1201701028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3f9c66700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3f40b2480 0x7fa3f414b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3f9465700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f40c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fa467700 1 -- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 msgr2=0x7fa3f40c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fa467700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f40c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fa467700 1 -- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3f40b2480 msgr2=0x7fa3f414b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fa467700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3f40b2480 0x7fa3f414b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3fa467700 1 -- 172.21.15.5:0/1201701028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec009a70 con 0x7fa3f4161030 2024-05-08T23:21:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.168+0000 7fa3f9c66700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3f40b2480 0x7fa3f414b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.169+0000 7fa3fa467700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f4161030 0x7fa3f40ca100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec013000 tx=0x7fa3ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.169+0000 7fa3ea7fc700 1 -- 172.21.15.5:0/1201701028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec012da0 con 0x7fa3f4161030 2024-05-08T23:21:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.169+0000 7fa3ea7fc700 1 -- 172.21.15.5:0/1201701028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec020070 con 0x7fa3f4161030 2024-05-08T23:21:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.169+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f40ca990 con 0x7fa3f4161030 2024-05-08T23:21:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.169+0000 7fa3ea7fc700 1 -- 172.21.15.5:0/1201701028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec004280 con 0x7fa3f4161030 2024-05-08T23:21:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.169+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f40cac30 con 0x7fa3f4161030 2024-05-08T23:21:23.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.170+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f4068000 con 0x7fa3f4161030 2024-05-08T23:21:23.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.170+0000 7fa3ea7fc700 1 -- 172.21.15.5:0/1201701028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa3ec0043e0 con 0x7fa3f4161030 2024-05-08T23:21:23.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.171+0000 7fa3ea7fc700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3d806ac90 0x7fa3d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.171+0000 7fa3ea7fc700 1 -- 172.21.15.5:0/1201701028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3ec022070 con 0x7fa3f4161030 2024-05-08T23:21:23.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.171+0000 7fa3f9c66700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3d806ac90 0x7fa3d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.171+0000 7fa3f9c66700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3d806ac90 0x7fa3d806d150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0005910 tx=0x7fa3f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:23.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.175+0000 7fa3ea7fc700 1 -- 172.21.15.5:0/1201701028 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa3ec056470 con 0x7fa3f4161030 2024-05-08T23:21:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.394+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3f40d03a0 con 0x7fa3f4161030 2024-05-08T23:21:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.395+0000 7fa3ea7fc700 1 -- 172.21.15.5:0/1201701028 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7fa3ec059a90 con 0x7fa3f4161030 2024-05-08T23:21:23.395 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:23.396 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.397+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3d806ac90 msgr2=0x7fa3d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.397+0000 7fa3fbeca700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3d806ac90 0x7fa3d806d150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0005910 tx=0x7fa3f0009040 comp rx=0 tx=0).stop 2024-05-08T23:21:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f4161030 msgr2=0x7fa3f40ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f4161030 0x7fa3f40ca100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec013000 tx=0x7fa3ec004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 shutdown_connections 2024-05-08T23:21:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa3d806ac90 0x7fa3d806d150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f4161030 0x7fa3f40ca100 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3f40b2480 0x7fa3f414b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 --2- 172.21.15.5:0/1201701028 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3f40ccbd0 0x7fa3f40c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 >> 172.21.15.5:0/1201701028 conn(0x7fa3f40b2f20 msgr2=0x7fa3f406bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 shutdown_connections 2024-05-08T23:21:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.398+0000 7fa3fbeca700 1 -- 172.21.15.5:0/1201701028 wait complete. 2024-05-08T23:21:23.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.548+0000 7f5f1d1ae700 1 Processor -- start 2024-05-08T23:21:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.548+0000 7f5f1d1ae700 1 -- start start 2024-05-08T23:21:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f1d1ae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 0x7f5f18099f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f1d1ae700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1809aca0 con 0x7f5f1815b810 2024-05-08T23:21:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f1d1ae700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f180cd9e0 con 0x7f5f18099a50 2024-05-08T23:21:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f1d1ae700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1809a440 con 0x7f5f180cd4e0 2024-05-08T23:21:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f16d9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5f1815b810 0x7f5f180cffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42550/0 (socket says 172.21.15.5:42550) 2024-05-08T23:21:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f16d9d700 1 -- 172.21.15.5:0/772732655 learned_addr learned my addr 172.21.15.5:0/772732655 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f1759e700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 0x7f5f18099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450340603 0 0) 0x7f5f1809aca0 con 0x7f5f1815b810 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f000035b0 con 0x7f5f1815b810 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.549+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038967775 0 0) 0x7f5f180cd9e0 con 0x7f5f18099a50 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f1809aca0 con 0x7f5f18099a50 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3066954899 0 0) 0x7f5f000035b0 con 0x7f5f1815b810 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 msgr2=0x7f5f18099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 0x7f5f18099f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 >> v1:172.21.15.49:6789/0 conn(0x7f5f18099a50 legacy=0x7f5f180ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180cf220 con 0x7f5f1815b810 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f5f08001670 con 0x7f5f1815b810 2024-05-08T23:21:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1759e700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 0x7f5f18099f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 >> v1:172.21.15.5:6789/0 conn(0x7f5f1815b810 legacy=0x7f5f180cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f00003bc0 0x7f5f00003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00004850 0x7f5f00004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 0x7f5f00005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1809a440 con 0x7f5f00004850 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0000bb20 con 0x7f5f00003bc0 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0000bce0 con 0x7f5f180cd4e0 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1659c700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00004850 0x7f5f00004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f16d9d700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f00003bc0 0x7f5f00003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1759e700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 0x7f5f00005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1659c700 1 -- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 msgr2=0x7f5f00005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1659c700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f180cd4e0 0x7f5f00005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1659c700 1 -- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f00003bc0 msgr2=0x7f5f00003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1659c700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f00003bc0 0x7f5f00003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1659c700 1 -- 172.21.15.5:0/772732655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10009a70 con 0x7f5f00004850 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f1659c700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00004850 0x7f5f00004cd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c001980 tx=0x7f5f00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b436e5922f67276 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c001470 con 0x7f5f00004850 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.550+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c0052d0 con 0x7f5f00004850 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.551+0000 7f5f15d9b700 1 -- 172.21.15.5:0/772732655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c003680 con 0x7f5f00004850 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.551+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/772732655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00004850 msgr2=0x7f5f00004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.551+0000 7f5f1d1ae700 1 --2- 172.21.15.5:0/772732655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00004850 0x7f5f00004cd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c001980 tx=0x7f5f00005880 comp rx=0 tx=0).stop 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.551+0000 7f5f1759e700 1 -- 172.21.15.5:0/772732655 reap_dead start 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.551+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/772732655 shutdown_connections 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.551+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/772732655 >> 172.21.15.5:0/772732655 conn(0x7f5f180b3010 msgr2=0x7f5f180b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.551+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/772732655 shutdown_connections 2024-05-08T23:21:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.551+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/772732655 wait complete. 2024-05-08T23:21:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.553+0000 7f5f1d1ae700 1 Processor -- start 2024-05-08T23:21:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.553+0000 7f5f1d1ae700 1 -- start start 2024-05-08T23:21:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1d1ae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f00004850 0x7f5f180b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1d1ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f181f4270 0x7f5f181f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1d1ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f181594f0 0x7f5f18159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1d1ae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f18159eb0 con 0x7f5f181f4270 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1d1ae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1815d580 con 0x7f5f181594f0 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1d1ae700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f181f4de0 con 0x7f5f00004850 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f181f4270 0x7f5f181f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f181f4270 0x7f5f181f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52976/0 (socket says 172.21.15.5:52976) 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 -- 172.21.15.5:0/3092891660 learned_addr learned my addr 172.21.15.5:0/3092891660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f16d9d700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f00004850 0x7f5f180b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1759e700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f181594f0 0x7f5f18159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 -- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f00004850 msgr2=0x7f5f180b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f00004850 0x7f5f180b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 -- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f181594f0 msgr2=0x7f5f18159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f181594f0 0x7f5f18159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 -- 172.21.15.5:0/3092891660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f181f50c0 con 0x7f5f181f4270 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5f1659c700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f181f4270 0x7f5f181f46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c3c0 tx=0x7f5f0c001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5eff7fe700 1 -- 172.21.15.5:0/3092891660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c001410 con 0x7f5f181f4270 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.554+0000 7f5eff7fe700 1 -- 172.21.15.5:0/3092891660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c007040 con 0x7f5f181f4270 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.555+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1815d210 con 0x7f5f181f4270 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.555+0000 7f5eff7fe700 1 -- 172.21.15.5:0/3092891660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0073f0 con 0x7f5f181f4270 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.555+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f180b72f0 con 0x7f5f181f4270 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.555+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f180d0670 con 0x7f5f181f4270 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.556+0000 7f5eff7fe700 1 -- 172.21.15.5:0/3092891660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f0c007550 con 0x7f5f181f4270 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.556+0000 7f5eff7fe700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f0406ac90 0x7f5f0406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.557+0000 7f5eff7fe700 1 -- 172.21.15.5:0/3092891660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f0c08a2d0 con 0x7f5f181f4270 2024-05-08T23:21:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.557+0000 7f5f16d9d700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f0406ac90 0x7f5f0406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.557+0000 7f5f16d9d700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f0406ac90 0x7f5f0406d150 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f08001eb0 tx=0x7f5f08000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.561+0000 7f5eff7fe700 1 -- 172.21.15.5:0/3092891660 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f0c0555f0 con 0x7f5f181f4270 2024-05-08T23:21:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.829+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f18068000 con 0x7f5f181f4270 2024-05-08T23:21:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.830+0000 7f5eff7fe700 1 -- 172.21.15.5:0/3092891660 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4719 (secure 0 0 0) 0x7f5f0c058c10 con 0x7f5f181f4270 2024-05-08T23:21:23.830 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:23.830 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:21:04.087368+0000","modified":"2024-05-08T23:21:07.094437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f0406ac90 msgr2=0x7f5f0406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f0406ac90 0x7f5f0406d150 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f08001eb0 tx=0x7f5f08000f80 comp rx=0 tx=0).stop 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f181f4270 msgr2=0x7f5f181f46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f181f4270 0x7f5f181f46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c3c0 tx=0x7f5f0c001310 comp rx=0 tx=0).stop 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 shutdown_connections 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f0406ac90 0x7f5f0406d150 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f181594f0 0x7f5f18159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f00004850 0x7f5f180b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 --2- 172.21.15.5:0/3092891660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f181f4270 0x7f5f181f46f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 >> 172.21.15.5:0/3092891660 conn(0x7f5f180b3010 msgr2=0x7f5f180b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 shutdown_connections 2024-05-08T23:21:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:23.832+0000 7f5f1d1ae700 1 -- 172.21.15.5:0/3092891660 wait complete. 2024-05-08T23:21:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-08T23:21:24.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:21:04.087368+0000', 'modified': '2024-05-08T23:21:07.094437+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/1939093817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 1939093817}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1939093817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:21:24.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:21:24.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T23:21:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.396+0000 7f7342ad1700 1 Processor -- start 2024-05-08T23:21:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.396+0000 7f7342ad1700 1 -- start start 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f7342ad1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f733c15f830 0x7f733c1664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f7342ad1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c06be40 con 0x7f733c06e200 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f7342ad1700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c06c2d0 con 0x7f733c155da0 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f7342ad1700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733c06dcb0 con 0x7f733c15f830 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f734106e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f733c15f830 0x7f733c1664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f733bfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f733c06e200 0x7f733c15f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42562/0 (socket says 172.21.15.5:42562) 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f733bfff700 1 -- 172.21.15.5:0/1677477132 learned_addr learned my addr 172.21.15.5:0/1677477132 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f734106e700 1 -- 172.21.15.5:0/1677477132 >> v1:172.21.15.49:6789/0 conn(0x7f733c155da0 legacy=0x7f733c06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f734106e700 1 -- 172.21.15.5:0/1677477132 >> v1:172.21.15.5:6789/0 conn(0x7f733c06e200 legacy=0x7f733c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f734106e700 1 -- 172.21.15.5:0/1677477132 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c06e660 con 0x7f733c15f830 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f733b7fe700 1 -- 172.21.15.5:0/1677477132 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158882965 0 0) 0x7f733c06be40 con 0x7f733c06e200 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f734106e700 1 --2- 172.21.15.5:0/1677477132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c15f830 0x7f733c1664b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7334009d90 tx=0x7f733400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e834bda31c4cfa70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.397+0000 7f733b7fe700 1 -- 172.21.15.5:0/1677477132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733400e070 con 0x7f733c15f830 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f733b7fe700 1 -- 172.21.15.5:0/1677477132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733400bbd0 con 0x7f733c15f830 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f733b7fe700 1 -- 172.21.15.5:0/1677477132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334012430 con 0x7f733c15f830 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f7342ad1700 1 -- 172.21.15.5:0/1677477132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c15f830 msgr2=0x7f733c1664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f7342ad1700 1 --2- 172.21.15.5:0/1677477132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c15f830 0x7f733c1664b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7334009d90 tx=0x7f733400b330 comp rx=0 tx=0).stop 2024-05-08T23:21:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f7342ad1700 1 -- 172.21.15.5:0/1677477132 shutdown_connections 2024-05-08T23:21:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f7342ad1700 1 --2- 172.21.15.5:0/1677477132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c15f830 0x7f733c1664b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f7342ad1700 1 -- 172.21.15.5:0/1677477132 >> 172.21.15.5:0/1677477132 conn(0x7f733c0cc580 msgr2=0x7f733c0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f7342ad1700 1 -- 172.21.15.5:0/1677477132 shutdown_connections 2024-05-08T23:21:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.398+0000 7f7342ad1700 1 -- 172.21.15.5:0/1677477132 wait complete. 2024-05-08T23:21:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.399+0000 7f7342ad1700 1 Processor -- start 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.400+0000 7f7342ad1700 1 -- start start 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.400+0000 7f7342ad1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f733c06e200 0x7f733c1411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.400+0000 7f7342ad1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c155da0 0x7f733c0cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.400+0000 7f7342ad1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f733c15f830 0x7f733c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.400+0000 7f7342ad1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0ce440 con 0x7f733c155da0 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.400+0000 7f7342ad1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0cd870 con 0x7f733c06e200 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.400+0000 7f7342ad1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0cdb10 con 0x7f733c15f830 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c155da0 0x7f733c0cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c155da0 0x7f733c0cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53002/0 (socket says 172.21.15.5:53002) 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 -- 172.21.15.5:0/1521187370 learned_addr learned my addr 172.21.15.5:0/1521187370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f734086d700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f733c06e200 0x7f733c1411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f734106e700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f733c15f830 0x7f733c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 -- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f733c15f830 msgr2=0x7f733c0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f733c15f830 0x7f733c0c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 -- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f733c06e200 msgr2=0x7f733c1411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f733c06e200 0x7f733c1411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 -- 172.21.15.5:0/1521187370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7334009a70 con 0x7f733c155da0 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f733bfff700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c155da0 0x7f733c0cef40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7328001eb0 tx=0x7f732800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f73397fa700 1 -- 172.21.15.5:0/1521187370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328004c80 con 0x7f733c155da0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f73397fa700 1 -- 172.21.15.5:0/1521187370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7328004de0 con 0x7f733c155da0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f7342ad1700 1 -- 172.21.15.5:0/1521187370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c0cddf0 con 0x7f733c155da0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f73397fa700 1 -- 172.21.15.5:0/1521187370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328007310 con 0x7f733c155da0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.401+0000 7f7342ad1700 1 -- 172.21.15.5:0/1521187370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c0ceb30 con 0x7f733c155da0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.403+0000 7f73397fa700 1 -- 172.21.15.5:0/1521187370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7328007470 con 0x7f733c155da0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.403+0000 7f73397fa700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f732006ac90 0x7f732006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.404+0000 7f73397fa700 1 -- 172.21.15.5:0/1521187370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7328088f80 con 0x7f733c155da0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.404+0000 7f734086d700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f732006ac90 0x7f732006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.404+0000 7f734086d700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f732006ac90 0x7f732006d150 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7330000f60 tx=0x7f7330009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.405+0000 7f734ec8bb80 1 Processor -- start 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.405+0000 7f734ec8bb80 1 -- start start 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628203ff100 con 0x5628204d2d80 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628203fcac0 con 0x5628204a6320 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628203f83b0 con 0x5628204cb750 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734106e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734106e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204d3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53016/0 (socket says 172.21.15.5:53016) 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.406+0000 7f734106e700 1 -- 172.21.15.5:0/3839022537 learned_addr learned my addr 172.21.15.5:0/3839022537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f733bfff700 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f734086d700 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f734106e700 1 -- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 msgr2=0x5628204a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f734106e700 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f734106e700 1 -- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 msgr2=0x5628204a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f734106e700 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f734106e700 1 -- 172.21.15.5:0/3839022537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562820279360 con 0x5628204d2d80 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f733bfff700 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f734106e700 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204d3200 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7334013000 tx=0x7f7334011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5c89946b29316f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f73267fc700 1 -- 172.21.15.5:0/3839022537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340126e0 con 0x5628204d2d80 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f73267fc700 1 -- 172.21.15.5:0/3839022537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334012840 con 0x5628204d2d80 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.407+0000 7f73267fc700 1 -- 172.21.15.5:0/3839022537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334012bb0 con 0x5628204d2d80 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3839022537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 msgr2=0x5628204d3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204d3200 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7334013000 tx=0x7f7334011000 comp rx=0 tx=0).stop 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3839022537 shutdown_connections 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204d3200 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/3839022537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3839022537 >> 172.21.15.5:0/3839022537 conn(0x5628204061c0 msgr2=0x5628204edd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3839022537 shutdown_connections 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3839022537 wait complete. 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.408+0000 7f734ec8bb80 1 -- start start 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.409+0000 7f734ec8bb80 1 Processor -- start 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.409+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.409+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.409+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 0x5628204eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.409+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562820394510 con 0x5628204a6320 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.409+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628203941b0 con 0x5628204cb750 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.409+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56282027b650 con 0x5628204d2d80 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734ec8bb80 10 client.0 _ll_register_callbacks cb 0x56282039dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204fd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53028/0 (socket says 172.21.15.5:53028) 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 -- 172.21.15.5:0/1920266392 learned_addr learned my addr 172.21.15.5:0/1920266392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734106e700 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 0x5628204eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f733bfff700 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 -- 172.21.15.5:0/1920266392 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 msgr2=0x5628204eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 0x5628204eb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 -- 172.21.15.5:0/1920266392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 msgr2=0x5628204d6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204d6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 -- 172.21.15.5:0/1920266392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c06e660 con 0x5628204a6320 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734086d700 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204fd9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f73300039d0 tx=0x7f7330000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945387ea9eb4acf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f731b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f731b7fe700 1 -- 172.21.15.5:0/1920266392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330015570 con 0x5628204a6320 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f731b7fe700 1 -- 172.21.15.5:0/1920266392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7330021a80 con 0x5628204a6320 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734ec8bb80 1 -- 172.21.15.5:0/1920266392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562820351220 con 0x5628204a6320 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f731b7fe700 1 -- 172.21.15.5:0/1920266392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7330021c40 con 0x5628204a6320 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.410+0000 7f734ec8bb80 10 client.4701 fetch_fsmap learned FSMap version 7 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f734ec8bb80 1 -- 172.21.15.5:0/1920266392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x562820225b10 con 0x5628204a6320 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f731b7fe700 1 -- 172.21.15.5:0/1920266392 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562820351220 con 0x5628204a6320 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f731b7fe700 1 -- 172.21.15.5:0/1920266392 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7330021420 con 0x5628204a6320 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f734ec8bb80 10 client.4701 fetch_fsmap finished waiting for FSMap version 7 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f734ec8bb80 10 client.4701 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f734ec8bb80 20 client.4701 populate_metadata read hostname 'smithi005' 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] conn(0x5628204d43f0 0x5628204d47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f734ec8bb80 4 client.4701 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f734ec8bb80 1 -- 172.21.15.5:0/1920266392 --> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562820477790 con 0x5628204d43f0 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f733bfff700 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] conn(0x5628204d43f0 0x5628204d47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f733bfff700 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] conn(0x5628204d43f0 0x5628204d47d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6688f73dba4ede4 server_cookie=8d4bca93a094659d in_seq=0 out_seq=0 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.411+0000 7f731b7fe700 10 client.4701 ms_handle_connect on v2:172.21.15.5:6833/1939093817 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f731b7fe700 1 -- 172.21.15.5:0/1920266392 <== mds.0 v2:172.21.15.5:6833/1939093817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562820477790 con 0x5628204d43f0 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f731b7fe700 10 client.4701 handle_command_reply: tid=0 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 client.4701 shutdown 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 -- 172.21.15.5:0/1920266392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 msgr2=0x5628204fd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204fd9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f73300039d0 tx=0x7f7330000ed0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 -- 172.21.15.5:0/1920266392 shutdown_connections 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] conn(0x5628204d43f0 0x5628204d47d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 0x5628204eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204fd9f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/1920266392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204d6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f731b7fe700 0 client.4701 ms_handle_reset on v2:172.21.15.5:6833/1939093817 2024-05-08T23:21:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 -- 172.21.15.5:0/1920266392 >> 172.21.15.5:0/1920266392 conn(0x5628204061c0 msgr2=0x5628204d23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 -- 172.21.15.5:0/1920266392 shutdown_connections 2024-05-08T23:21:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.412+0000 7f734ec8bb80 1 -- 172.21.15.5:0/1920266392 wait complete. 2024-05-08T23:21:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.413+0000 7f734ec8bb80 20 client.4701 trim_cache size 0 max 16384 2024-05-08T23:21:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.432+0000 7f734ec8bb80 1 Processor -- start 2024-05-08T23:21:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.432+0000 7f734ec8bb80 1 -- start start 2024-05-08T23:21:24.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204ec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 0x5628204ec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628204d6730 con 0x5628204a6320 2024-05-08T23:21:24.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628205101b0 con 0x5628204cb750 2024-05-08T23:21:24.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628204aa3e0 con 0x5628204d2d80 2024-05-08T23:21:24.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53034/0 (socket says 172.21.15.5:53034) 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 -- 172.21.15.5:0/3800541500 learned_addr learned my addr 172.21.15.5:0/3800541500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f733bfff700 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734106e700 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 0x5628204ec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 -- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 msgr2=0x5628204ec9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 0x5628204ec9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 -- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 msgr2=0x5628204ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 -- 172.21.15.5:0/3800541500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628203be510 con 0x5628204a6320 2024-05-08T23:21:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f733bfff700 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204ec460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f734086d700 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204a1e80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73300051d0 tx=0x7f733001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b11151a10bdcb86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f7326ffd700 1 -- 172.21.15.5:0/3800541500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330021ba0 con 0x5628204a6320 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f7326ffd700 1 -- 172.21.15.5:0/3800541500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733001b050 con 0x5628204a6320 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.433+0000 7f7326ffd700 1 -- 172.21.15.5:0/3800541500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330036450 con 0x5628204a6320 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3800541500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 msgr2=0x5628204a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204a1e80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73300051d0 tx=0x7f733001e000 comp rx=0 tx=0).stop 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3800541500 shutdown_connections 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204d2d80 0x5628204ec9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204a6320 0x5628204a1e80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/3800541500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204cb750 0x5628204ec460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3800541500 >> 172.21.15.5:0/3800541500 conn(0x5628204061c0 msgr2=0x5628204a90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3800541500 shutdown_connections 2024-05-08T23:21:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.434+0000 7f734ec8bb80 1 -- 172.21.15.5:0/3800541500 wait complete. 2024-05-08T23:21:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.435+0000 7f734ec8bb80 1 -- start start 2024-05-08T23:21:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.435+0000 7f734ec8bb80 1 Processor -- start 2024-05-08T23:21:24.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734ec8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204e4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628204a8130 con 0x5628204d2d80 2024-05-08T23:21:24.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562820506ba0 con 0x5628204a6320 2024-05-08T23:21:24.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734ec8bb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628204e4540 con 0x5628204cb750 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734ec8bb80 10 client.0 _ll_register_callbacks cb 0x56282039dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204e4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53040/0 (socket says 172.21.15.5:53040) 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 -- 172.21.15.5:0/716953781 learned_addr learned my addr 172.21.15.5:0/716953781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734086d700 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f733bfff700 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 -- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 msgr2=0x5628204a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 -- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 msgr2=0x5628204ec4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204ec4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 -- 172.21.15.5:0/716953781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562820225b10 con 0x5628204d2d80 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f733bfff700 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f734106e700 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204e4000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7334009df0 tx=0x7f7334004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca03dbb8de908e2f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.436+0000 7f731b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f731b7fe700 1 -- 172.21.15.5:0/716953781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340126e0 con 0x5628204d2d80 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f731b7fe700 1 -- 172.21.15.5:0/716953781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7334012e40 con 0x5628204d2d80 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 1 -- 172.21.15.5:0/716953781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7334012840 con 0x5628204d2d80 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f731b7fe700 1 -- 172.21.15.5:0/716953781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7334012430 con 0x5628204d2d80 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 10 client.4707 fetch_fsmap learned FSMap version 7 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 1 -- 172.21.15.5:0/716953781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x56282027dc80 con 0x5628204d2d80 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f731b7fe700 1 -- 172.21.15.5:0/716953781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334012840 con 0x5628204d2d80 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f731b7fe700 1 -- 172.21.15.5:0/716953781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7334037070 con 0x5628204d2d80 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 10 client.4707 fetch_fsmap finished waiting for FSMap version 7 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 10 client.4707 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 20 client.4707 populate_metadata read hostname 'smithi005' 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] conn(0x5628205562c0 0x5628205566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 4 client.4707 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:21:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734ec8bb80 1 -- 172.21.15.5:0/716953781 --> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5628203f0e70 con 0x5628205562c0 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.437+0000 7f734086d700 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] conn(0x5628205562c0 0x5628205566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f731b7fe700 10 client.4707 ms_handle_connect on v2:172.21.15.5:6833/1939093817 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734086d700 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] conn(0x5628205562c0 0x5628205566a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d31f6bac1de16 server_cookie=4e10c717a6bbe9f7 in_seq=0 out_seq=0 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f731b7fe700 1 -- 172.21.15.5:0/716953781 <== mds.0 v2:172.21.15.5:6833/1939093817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5628203f0e70 con 0x5628205562c0 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f731b7fe700 10 client.4707 handle_command_reply: tid=0 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 client.4707 shutdown 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 -- 172.21.15.5:0/716953781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 msgr2=0x5628204e4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204e4000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7334009df0 tx=0x7f7334004b40 comp rx=0 tx=0).stop 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 -- 172.21.15.5:0/716953781 shutdown_connections 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628204d2d80 0x5628204e4000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628204a6320 0x5628204ec4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.5:6833/1939093817,v1:172.21.15.5:6835/1939093817] conn(0x5628205562c0 0x5628205566a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 --2- 172.21.15.5:0/716953781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628204cb750 0x5628204a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f731b7fe700 0 client.4707 ms_handle_reset on v2:172.21.15.5:6833/1939093817 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 -- 172.21.15.5:0/716953781 >> 172.21.15.5:0/716953781 conn(0x5628204061c0 msgr2=0x5628204a90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 -- 172.21.15.5:0/716953781 shutdown_connections 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 1 -- 172.21.15.5:0/716953781 wait complete. 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.438+0000 7f734ec8bb80 20 client.4707 trim_cache size 0 max 16384 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4263, 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:21:24.446 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:21:24.447 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:21:24.447 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 20.350075930999999, 2024-05-08T23:21:24.447 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 7, 2024-05-08T23:21:24.447 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 20, 2024-05-08T23:21:24.447 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 20, 2024-05-08T23:21:24.447 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 27.146508069999999 2024-05-08T23:21:24.447 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 -- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f732006ac90 msgr2=0x7f732006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f732006ac90 0x7f732006d150 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7330000f60 tx=0x7f7330009040 comp rx=0 tx=0).stop 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 -- 172.21.15.5:0/1521187370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c155da0 msgr2=0x7f733c0cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c155da0 0x7f733c0cef40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7328001eb0 tx=0x7f732800c8b0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 -- 172.21.15.5:0/1521187370 shutdown_connections 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f732006ac90 0x7f732006d150 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f733c15f830 0x7f733c0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f733c06e200 0x7f733c1411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 --2- 172.21.15.5:0/1521187370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f733c155da0 0x7f733c0cef40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 -- 172.21.15.5:0/1521187370 >> 172.21.15.5:0/1521187370 conn(0x7f733c0cc580 msgr2=0x7f733c06d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 -- 172.21.15.5:0/1521187370 shutdown_connections 2024-05-08T23:21:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:24.439+0000 7f7342ad1700 1 -- 172.21.15.5:0/1521187370 wait complete. 2024-05-08T23:21:24.453 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T23:21:24.464 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T23:21:24.464 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T23:21:24.464 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T23:21:24.464 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T23:21:24.489 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T23:21:24.489 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-08T23:21:24.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:24.490 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:21:24.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:24.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-08T23:21:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-05-08T23:21:24.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:21:24.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:24.590 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:21:24.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:24.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-08T23:21:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-05-08T23:21:24.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:21:24.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:24.731 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-08T23:21:24.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:24.753 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-05-08T23:21:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-05-08T23:21:24.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:21:24.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:24.831 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-08T23:21:24.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:24.891 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-05-08T23:21:24.964 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-05-08T23:21:24.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:21:24.966 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-05-08T23:21:24.966 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:21:24.966 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:21:24.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:21:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.111+0000 7f262e3a6700 1 Processor -- start 2024-05-08T23:21:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.111+0000 7f262e3a6700 1 -- start start 2024-05-08T23:21:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f262e3a6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f262815daf0 0x7f262806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f262e3a6700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26280cf6b0 con 0x7f262815d710 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f262e3a6700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26280cf8e0 con 0x7f262806bd60 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f262e3a6700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26280cfbc0 con 0x7f262815daf0 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f26277fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f262815daf0 0x7f262806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f26277fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f262815daf0 0x7f262806c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53046/0 (socket says 172.21.15.5:53046) 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f2627fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f262815d710 0x7f262809a6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42568/0 (socket says 172.21.15.5:42568) 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f26277fe700 1 -- 172.21.15.5:0/198479881 learned_addr learned my addr 172.21.15.5:0/198479881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f26277fe700 1 -- 172.21.15.5:0/198479881 >> v1:172.21.15.49:6789/0 conn(0x7f262806bd60 legacy=0x7f2628167560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f26277fe700 1 -- 172.21.15.5:0/198479881 >> v1:172.21.15.5:6789/0 conn(0x7f262815d710 legacy=0x7f262809a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f26277fe700 1 -- 172.21.15.5:0/198479881 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280cfea0 con 0x7f262815daf0 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.112+0000 7f2626ffd700 1 -- 172.21.15.5:0/198479881 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841451697 0 0) 0x7f26280cf6b0 con 0x7f262815d710 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f26277fe700 1 --2- 172.21.15.5:0/198479881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262815daf0 0x7f262806c480 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26180098f0 tx=0x7f2618009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48c1ce9434f52eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f2626ffd700 1 -- 172.21.15.5:0/198479881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800b8a0 con 0x7f262815daf0 2024-05-08T23:21:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f2626ffd700 1 -- 172.21.15.5:0/198479881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261800ba00 con 0x7f262815daf0 2024-05-08T23:21:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f2626ffd700 1 -- 172.21.15.5:0/198479881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800bd90 con 0x7f262815daf0 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f262e3a6700 1 -- 172.21.15.5:0/198479881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262815daf0 msgr2=0x7f262806c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f262e3a6700 1 --2- 172.21.15.5:0/198479881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262815daf0 0x7f262806c480 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26180098f0 tx=0x7f2618009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f262e3a6700 1 -- 172.21.15.5:0/198479881 shutdown_connections 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f262e3a6700 1 --2- 172.21.15.5:0/198479881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262815daf0 0x7f262806c480 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f262e3a6700 1 -- 172.21.15.5:0/198479881 >> 172.21.15.5:0/198479881 conn(0x7f26280b2f60 msgr2=0x7f26280b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f262e3a6700 1 -- 172.21.15.5:0/198479881 shutdown_connections 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.113+0000 7f262e3a6700 1 -- 172.21.15.5:0/198479881 wait complete. 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.115+0000 7f262e3a6700 1 Processor -- start 2024-05-08T23:21:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.115+0000 7f262e3a6700 1 -- start start 2024-05-08T23:21:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f262e3a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262806bd60 0x7f26280c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f262e3a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262815d710 0x7f26280c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f262e3a6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26280cb0f0 0x7f26280c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f262e3a6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262814e180 con 0x7f262806bd60 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f262e3a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280b7150 con 0x7f262815d710 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f262e3a6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280c75e0 con 0x7f26280cb0f0 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262806bd60 0x7f26280c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262806bd60 0x7f26280c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53060/0 (socket says 172.21.15.5:53060) 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 -- 172.21.15.5:0/3901401456 learned_addr learned my addr 172.21.15.5:0/3901401456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f26277fe700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262815d710 0x7f26280c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f262c943700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26280cb0f0 0x7f26280c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 -- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26280cb0f0 msgr2=0x7f26280c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26280cb0f0 0x7f26280c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 -- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262815d710 msgr2=0x7f26280c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262815d710 0x7f26280c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 -- 172.21.15.5:0/3901401456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2618009550 con 0x7f262806bd60 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f262c943700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26280cb0f0 0x7f26280c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2627fff700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262806bd60 0x7f26280c70a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f261000c520 tx=0x7f261000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.116+0000 7f2624ff9700 1 -- 172.21.15.5:0/3901401456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610004c80 con 0x7f262806bd60 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.117+0000 7f2624ff9700 1 -- 172.21.15.5:0/3901401456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2610004e00 con 0x7f262806bd60 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.117+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26280c78a0 con 0x7f262806bd60 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.117+0000 7f2624ff9700 1 -- 172.21.15.5:0/3901401456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610007310 con 0x7f262806bd60 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.117+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26280c7e60 con 0x7f262806bd60 2024-05-08T23:21:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.117+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262814e2c0 con 0x7f262806bd60 2024-05-08T23:21:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.118+0000 7f2624ff9700 1 -- 172.21.15.5:0/3901401456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2610007470 con 0x7f262806bd60 2024-05-08T23:21:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.118+0000 7f2624ff9700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f260806ac90 0x7f260806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.119+0000 7f2624ff9700 1 -- 172.21.15.5:0/3901401456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2610088fe0 con 0x7f262806bd60 2024-05-08T23:21:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.119+0000 7f26277fe700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f260806ac90 0x7f260806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.119+0000 7f26277fe700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f260806ac90 0x7f260806d150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2618009130 tx=0x7f261801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.123+0000 7f2624ff9700 1 -- 172.21.15.5:0/3901401456 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f262814e2c0 con 0x7f262806bd60 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.394+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f262814e2c0 con 0x7f262806bd60 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.394+0000 7f2624ff9700 1 -- 172.21.15.5:0/3901401456 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f262814e2c0 con 0x7f262806bd60 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:21:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f260806ac90 msgr2=0x7f260806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f260806ac90 0x7f260806d150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2618009130 tx=0x7f261801c040 comp rx=0 tx=0).stop 2024-05-08T23:21:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262806bd60 msgr2=0x7f26280c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262806bd60 0x7f26280c70a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f261000c520 tx=0x7f261000c8e0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 shutdown_connections 2024-05-08T23:21:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f260806ac90 0x7f260806d150 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26280cb0f0 0x7f26280c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262806bd60 0x7f26280c70a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 --2- 172.21.15.5:0/3901401456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f262815d710 0x7f26280c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 >> 172.21.15.5:0/3901401456 conn(0x7f26280b2f60 msgr2=0x7f262815c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 shutdown_connections 2024-05-08T23:21:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:25.397+0000 7f262e3a6700 1 -- 172.21.15.5:0/3901401456 wait complete. 2024-05-08T23:21:25.411 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:21:25.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:21:25.414 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-08T23:21:25.414 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:21:25.414 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:21:25.414 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:21:25.414 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:21:25.414 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:21:25.414 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:21:25.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:25.415 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-08T23:21:25.434 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:21:25.434 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:21:25.434 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:21:25.434 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:25.434 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:21:25.434 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:25.434 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:25.434 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42232sec preferred_lft 42232sec 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:25.435 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-08T23:21:25.436 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:21:25.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:25.436 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:25.436 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:25.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:21:25.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-05-08T23:21:25.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-05-08T23:21:25.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:21:25.437 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:25.625 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:21:25.659 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-08T23:21:25.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:25.661 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-08T23:21:25.735 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-08T23:21:25.735 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:21:25.735 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:21:25.735 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:21:25.735 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:21:25.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:25.736 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:25.736 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:25.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:21:25.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:21:25.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:21:25.736 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:25.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:25.991 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:21:26.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:26.012 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-05-08T23:21:26.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:26.074 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:26.074 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:26.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:21:26.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T23:21:26.074 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:26.186 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T23:21:26.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:26.186 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:26.187 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:26.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T23:21:26.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:21:26.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T23:21:26.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T23:21:26.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T23:21:26.187 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:26.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:26.423 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:26.423 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:26.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-08T23:21:26.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:21:26.423 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:26.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:21:26.535 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:21:26.555 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:21:26.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:21:26.555 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:21:26.617 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T23:21:26.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:26.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:26.716 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:26.716 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:26.808 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:26.808 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:26.808 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:21:26.808 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:21:26.809 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:21:26.809 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:21:26.809 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:21:26.809 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:21:26.828 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:26.829 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:26.829 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:21:26.829 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:21:26.829 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:21:26.829 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:21:26.829 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:21:26.830 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:21:26.830 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:21:26.830 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:21:26.830 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:21:26.831 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:21:26.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:26.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:21:26.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:26.865 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:26.958 INFO:teuthology.orchestra.run.smithi005.stdout:client4603 2024-05-08T23:21:26.959 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:21:26.959 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:26.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:26.981 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:27.077 INFO:teuthology.orchestra.run.smithi005.stdout:client4603 2024-05-08T23:21:27.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:27.077 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:27.133 INFO:teuthology.orchestra.run.smithi005.stdout:client4603 2024-05-08T23:21:27.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:27.134 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4fd8ca56-5ac1-431d-b985-b50a13a8130a.client4603/status') 2024-05-08T23:21:27.196 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.4603 (3)172.21.15.5:0/301318566 2024-05-08T23:21:27.196 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-05-08T23:21:27.196 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-05-08T23:21:27.196 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-05-08T23:21:27.196 INFO:teuthology.orchestra.run.smithi005.stderr:68 bytes copied, 2.8812e-05 s, 2.4 MB/s 2024-05-08T23:21:27.198 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-05-08T23:21:27.198 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:21:27.198 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T23:21:27.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:21:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.340+0000 7f661b382700 1 Processor -- start 2024-05-08T23:21:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.340+0000 7f661b382700 1 -- start start 2024-05-08T23:21:27.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661b382700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f66140c4400 0x7f66140b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661b382700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f66140b16f0 0x7f66140cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661b382700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661406c2d0 con 0x7f66140b2690 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661b382700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661406ba70 con 0x7f66140b16f0 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661b382700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66140b2b90 con 0x7f66140c4400 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661991f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f66140b16f0 0x7f66140cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661891d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f66140b2690 0x7f66140c47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42570/0 (socket says 172.21.15.5:42570) 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661991f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f66140b16f0 0x7f66140cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53066/0 (socket says 172.21.15.5:53066) 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661891d700 1 -- 172.21.15.5:0/4188525867 learned_addr learned my addr 172.21.15.5:0/4188525867 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661911e700 1 --2- 172.21.15.5:0/4188525867 >> v2:172.21.15.49:3300/0 conn(0x7f66140c4400 0x7f66140b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661991f700 1 -- 172.21.15.5:0/4188525867 >> v2:172.21.15.49:3300/0 conn(0x7f66140c4400 msgr2=0x7f66140b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661991f700 1 --2- 172.21.15.5:0/4188525867 >> v2:172.21.15.49:3300/0 conn(0x7f66140c4400 0x7f66140b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661991f700 1 -- 172.21.15.5:0/4188525867 >> v1:172.21.15.5:6789/0 conn(0x7f66140b2690 legacy=0x7f66140c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f661991f700 1 -- 172.21.15.5:0/4188525867 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661406bc80 con 0x7f66140b16f0 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.341+0000 7f660bfff700 1 -- 172.21.15.5:0/4188525867 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55732520 0 0) 0x7f661406c2d0 con 0x7f66140b2690 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f661991f700 1 --2- 172.21.15.5:0/4188525867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140cba80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6614072130 tx=0x7f660c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac015ad169e032af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f660bfff700 1 -- 172.21.15.5:0/4188525867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00e070 con 0x7f66140b16f0 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f660bfff700 1 -- 172.21.15.5:0/4188525867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c00bbd0 con 0x7f66140b16f0 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f660bfff700 1 -- 172.21.15.5:0/4188525867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c012430 con 0x7f66140b16f0 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f661b382700 1 -- 172.21.15.5:0/4188525867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 msgr2=0x7f66140cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f661b382700 1 --2- 172.21.15.5:0/4188525867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140cba80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6614072130 tx=0x7f660c00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f661b382700 1 -- 172.21.15.5:0/4188525867 shutdown_connections 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f661b382700 1 --2- 172.21.15.5:0/4188525867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140cba80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f661b382700 1 --2- 172.21.15.5:0/4188525867 >> v2:172.21.15.49:3300/0 conn(0x7f66140c4400 0x7f66140b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.342+0000 7f661b382700 1 -- 172.21.15.5:0/4188525867 >> 172.21.15.5:0/4188525867 conn(0x7f66140b2f20 msgr2=0x7f66140b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.343+0000 7f661b382700 1 -- 172.21.15.5:0/4188525867 shutdown_connections 2024-05-08T23:21:27.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.343+0000 7f661b382700 1 -- 172.21.15.5:0/4188525867 wait complete. 2024-05-08T23:21:27.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.344+0000 7f661b382700 1 Processor -- start 2024-05-08T23:21:27.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.344+0000 7f661b382700 1 -- start start 2024-05-08T23:21:27.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661b382700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661b382700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66140b2690 0x7f66140ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661b382700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66140c4400 0x7f6614160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661b382700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66140c6ab0 con 0x7f66140b16f0 2024-05-08T23:21:27.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661b382700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66140c95c0 con 0x7f66140b2690 2024-05-08T23:21:27.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661b382700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140ca140 con 0x7f66140c4400 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53070/0 (socket says 172.21.15.5:53070) 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 -- 172.21.15.5:0/3431236653 learned_addr learned my addr 172.21.15.5:0/3431236653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661891d700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66140b2690 0x7f66140ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661991f700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66140c4400 0x7f6614160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 -- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66140c4400 msgr2=0x7f6614160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66140c4400 0x7f6614160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 -- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66140b2690 msgr2=0x7f66140ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66140b2690 0x7f66140ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 -- 172.21.15.5:0/3431236653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c009a70 con 0x7f66140b16f0 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.345+0000 7f661911e700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140c5700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f661000e810 tx=0x7f661000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.346+0000 7f6609ffb700 1 -- 172.21.15.5:0/3431236653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661000c070 con 0x7f66140b16f0 2024-05-08T23:21:27.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.346+0000 7f6609ffb700 1 -- 172.21.15.5:0/3431236653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610013610 con 0x7f66140b16f0 2024-05-08T23:21:27.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.346+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66140c5c40 con 0x7f66140b16f0 2024-05-08T23:21:27.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.346+0000 7f6609ffb700 1 -- 172.21.15.5:0/3431236653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66100139c0 con 0x7f66140b16f0 2024-05-08T23:21:27.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.346+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66140c9af0 con 0x7f66140b16f0 2024-05-08T23:21:27.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.347+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66140ca420 con 0x7f66140b16f0 2024-05-08T23:21:27.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.347+0000 7f6609ffb700 1 -- 172.21.15.5:0/3431236653 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6610013b20 con 0x7f66140b16f0 2024-05-08T23:21:27.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.348+0000 7f6609ffb700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65f806ac90 0x7f65f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.348+0000 7f6609ffb700 1 -- 172.21.15.5:0/3431236653 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6610089100 con 0x7f66140b16f0 2024-05-08T23:21:27.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.348+0000 7f661891d700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65f806ac90 0x7f65f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.348+0000 7f661891d700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65f806ac90 0x7f65f806d150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f661409f430 tx=0x7f6604007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.352+0000 7f6609ffb700 1 -- 172.21.15.5:0/3431236653 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6610054250 con 0x7f66140b16f0 2024-05-08T23:21:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.619+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6614068000 con 0x7f66140b16f0 2024-05-08T23:21:27.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.620+0000 7f6609ffb700 1 -- 172.21.15.5:0/3431236653 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6610057870 con 0x7f66140b16f0 2024-05-08T23:21:27.620 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:21:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65f806ac90 msgr2=0x7f65f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65f806ac90 0x7f65f806d150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f661409f430 tx=0x7f6604007de0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 msgr2=0x7f66140c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140c5700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f661000e810 tx=0x7f661000eb20 comp rx=0 tx=0).stop 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 shutdown_connections 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65f806ac90 0x7f65f806d150 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66140c4400 0x7f6614160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66140b16f0 0x7f66140c5700 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 --2- 172.21.15.5:0/3431236653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66140b2690 0x7f66140ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 >> 172.21.15.5:0/3431236653 conn(0x7f66140b2f20 msgr2=0x7f661406ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 shutdown_connections 2024-05-08T23:21:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:27.622+0000 7f661b382700 1 -- 172.21.15.5:0/3431236653 wait complete. 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:21:27.637 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-08T23:21:27.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:27.638 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-08T23:21:27.657 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:21:27.657 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:21:27.657 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:21:27.657 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:27.657 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:21:27.657 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:27.657 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42229sec preferred_lft 42229sec 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:27.658 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::1c2e:b3ff:fe6b:56f/64 scope link 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::84a4:2aff:fe7c:ce1a/64 scope link tentative 2024-05-08T23:21:27.659 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:27.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:27.660 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:21:27.729 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-08T23:21:27.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:27.730 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-08T23:21:27.795 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-08T23:21:27.796 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:21:27.796 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:21:27.796 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:21:27.796 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:21:27.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:27.796 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:27.796 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:27.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:21:27.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:21:27.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:21:27.797 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:27.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:27.920 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:21:27.941 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:21:27.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:27.941 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-05-08T23:21:28.003 INFO:teuthology.orchestra.run.smithi005.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-08T23:21:28.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.004 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:28.004 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:28.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:21:28.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-08T23:21:28.004 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:28.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T23:21:28.165 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:21:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:21:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:21:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:21:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:28.166 INFO:teuthology.orchestra.run.smithi005.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:21:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 62:d2:d4:bb:18:d5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T23:21:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T23:21:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::60d2:d4ff:febb:18d5/64 scope link tentative 2024-05-08T23:21:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:28.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.184 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T23:21:28.233 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:21:28.233 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:21:28.233 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:21:28.233 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:28.233 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:21:28.233 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:28.233 INFO:teuthology.orchestra.run.smithi005.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:21:28.234 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 62:d2:d4:bb:18:d5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T23:21:28.234 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T23:21:28.234 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:28.234 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::60d2:d4ff:febb:18d5/64 scope link tentative 2024-05-08T23:21:28.234 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:28.246 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-08T23:21:28.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.246 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:28.246 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:28.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-08T23:21:28.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T23:21:28.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-08T23:21:28.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-08T23:21:28.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-08T23:21:28.247 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:28.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.467 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:21:28.467 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:21:28.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.1 up 2024-05-08T23:21:28.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T23:21:28.468 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:21:28.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:21:28.580 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-08T23:21:28.599 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-08T23:21:28.600 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:21:28.600 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-08T23:21:28.661 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-05-08T23:21:28.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.661 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:28.761 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:28.761 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:28.781 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:28.781 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:28.781 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:21:28.781 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 2@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:21:28.781 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:21:28.781 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:21:28.781 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:21:28.782 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:21:28.800 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:28.800 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:28.800 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:21:28.800 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 2@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:21:28.801 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:21:28.801 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:21:28.801 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:21:28.801 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:21:28.801 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:21:28.801 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:21:28.801 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:21:28.803 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:21:28.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.804 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-08T23:21:28.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.835 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:28.917 INFO:teuthology.orchestra.run.smithi005.stdout:client4487 2024-05-08T23:21:28.918 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:21:28.918 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:28.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:28.981 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:29.063 INFO:teuthology.orchestra.run.smithi005.stdout:client4487 2024-05-08T23:21:29.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:29.063 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:29.146 INFO:teuthology.orchestra.run.smithi005.stdout:client4487 2024-05-08T23:21:29.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:29.146 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4fd8ca56-5ac1-431d-b985-b50a13a8130a.client4487/status') 2024-05-08T23:21:29.240 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.4487 (3)172.21.15.5:0/3372784605 2024-05-08T23:21:29.241 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-05-08T23:21:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-05-08T23:21:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-05-08T23:21:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:69 bytes copied, 2.7706e-05 s, 2.5 MB/s 2024-05-08T23:21:29.242 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-05-08T23:21:29.242 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:21:29.242 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T23:21:29.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:21:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.384+0000 7fbe35913700 1 Processor -- start 2024-05-08T23:21:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.384+0000 7fbe35913700 1 -- start start 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe35913700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbe3015db10 0x7fbe300d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe35913700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3006c400 con 0x7fbe300cbb00 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe35913700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe300d0cd0 con 0x7fbe300b2700 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe35913700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe300d09f0 con 0x7fbe3015db10 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2f7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbe300cbb00 0x7fbe301617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32996/0 (socket says 172.21.15.5:32996) 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2f7fe700 1 -- 172.21.15.5:0/2388480756 learned_addr learned my addr 172.21.15.5:0/2388480756 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2effd700 1 --2- 172.21.15.5:0/2388480756 >> v2:172.21.15.49:3300/0 conn(0x7fbe3015db10 0x7fbe300d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2dffb700 1 -- 172.21.15.5:0/2388480756 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651119631 0 0) 0x7fbe3006c400 con 0x7fbe300cbb00 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2dffb700 1 -- 172.21.15.5:0/2388480756 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe180035b0 con 0x7fbe300cbb00 2024-05-08T23:21:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2dffb700 1 -- 172.21.15.5:0/2388480756 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110621385 0 0) 0x7fbe300d0cd0 con 0x7fbe300b2700 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2dffb700 1 -- 172.21.15.5:0/2388480756 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe3006c400 con 0x7fbe300b2700 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2effd700 1 -- 172.21.15.5:0/2388480756 >> v1:172.21.15.49:6789/0 conn(0x7fbe300b2700 legacy=0x7fbe3006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2effd700 1 -- 172.21.15.5:0/2388480756 >> v1:172.21.15.5:6789/0 conn(0x7fbe300cbb00 legacy=0x7fbe301617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.385+0000 7fbe2effd700 1 -- 172.21.15.5:0/2388480756 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3006bdb0 con 0x7fbe3015db10 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe2effd700 1 --2- 172.21.15.5:0/2388480756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3015db10 0x7fbe300d0410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009840 tx=0x7fbe20009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e225f9eacde0f764 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe2dffb700 1 -- 172.21.15.5:0/2388480756 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20011830 con 0x7fbe3015db10 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe2dffb700 1 -- 172.21.15.5:0/2388480756 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe20011990 con 0x7fbe3015db10 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe2dffb700 1 -- 172.21.15.5:0/2388480756 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20011d20 con 0x7fbe3015db10 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe35913700 1 -- 172.21.15.5:0/2388480756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3015db10 msgr2=0x7fbe300d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe35913700 1 --2- 172.21.15.5:0/2388480756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3015db10 0x7fbe300d0410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009840 tx=0x7fbe20009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe35913700 1 -- 172.21.15.5:0/2388480756 shutdown_connections 2024-05-08T23:21:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe35913700 1 --2- 172.21.15.5:0/2388480756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3015db10 0x7fbe300d0410 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe35913700 1 -- 172.21.15.5:0/2388480756 >> 172.21.15.5:0/2388480756 conn(0x7fbe300b2fa0 msgr2=0x7fbe300b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe35913700 1 -- 172.21.15.5:0/2388480756 shutdown_connections 2024-05-08T23:21:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.386+0000 7fbe35913700 1 -- 172.21.15.5:0/2388480756 wait complete. 2024-05-08T23:21:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.388+0000 7fbe35913700 1 Processor -- start 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.388+0000 7fbe35913700 1 -- start start 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe35913700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe300b2700 0x7fbe3015f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe35913700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe300cbb00 0x7fbe30157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe35913700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3015db10 0x7fbe300c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe35913700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301575c0 con 0x7fbe3015db10 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe35913700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30160530 con 0x7fbe300b2700 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe35913700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3015f0b0 con 0x7fbe300cbb00 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3015db10 0x7fbe300c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3015db10 0x7fbe300c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41872/0 (socket says 172.21.15.5:41872) 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2f7fe700 1 -- 172.21.15.5:0/1846700031 learned_addr learned my addr 172.21.15.5:0/1846700031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2effd700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe300b2700 0x7fbe3015f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2e7fc700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe300cbb00 0x7fbe30157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2f7fe700 1 -- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe300cbb00 msgr2=0x7fbe30157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2f7fe700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe300cbb00 0x7fbe30157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2f7fe700 1 -- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe300b2700 msgr2=0x7fbe3015f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2f7fe700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe300b2700 0x7fbe3015f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2f7fe700 1 -- 172.21.15.5:0/1846700031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe200094a0 con 0x7fbe3015db10 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.389+0000 7fbe2effd700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe300b2700 0x7fbe3015f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.390+0000 7fbe2f7fe700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3015db10 0x7fbe300c6690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800cc00 tx=0x7fbe2800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.390+0000 7fbe17fff700 1 -- 172.21.15.5:0/1846700031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800ab40 con 0x7fbe3015db10 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.390+0000 7fbe17fff700 1 -- 172.21.15.5:0/1846700031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2800aca0 con 0x7fbe3015db10 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.390+0000 7fbe17fff700 1 -- 172.21.15.5:0/1846700031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28012620 con 0x7fbe3015db10 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.390+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3015eaa0 con 0x7fbe3015db10 2024-05-08T23:21:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.390+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3015ed40 con 0x7fbe3015db10 2024-05-08T23:21:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.391+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3015e900 con 0x7fbe3015db10 2024-05-08T23:21:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.391+0000 7fbe17fff700 1 -- 172.21.15.5:0/1846700031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbe2801b3b0 con 0x7fbe3015db10 2024-05-08T23:21:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.392+0000 7fbe17fff700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe1c06ac90 0x7fbe1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.392+0000 7fbe17fff700 1 -- 172.21.15.5:0/1846700031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe280890e0 con 0x7fbe3015db10 2024-05-08T23:21:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.392+0000 7fbe2effd700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe1c06ac90 0x7fbe1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.392+0000 7fbe2effd700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe1c06ac90 0x7fbe1c06d150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009fd0 tx=0x7fbe2001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.396+0000 7fbe17fff700 1 -- 172.21.15.5:0/1846700031 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbe3015e900 con 0x7fbe3015db10 2024-05-08T23:21:29.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.663+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbe3015e900 con 0x7fbe3015db10 2024-05-08T23:21:29.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.664+0000 7fbe17fff700 1 -- 172.21.15.5:0/1846700031 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbe3015e900 con 0x7fbe3015db10 2024-05-08T23:21:29.664 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe1c06ac90 msgr2=0x7fbe1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe1c06ac90 0x7fbe1c06d150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009fd0 tx=0x7fbe2001c040 comp rx=0 tx=0).stop 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3015db10 msgr2=0x7fbe300c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3015db10 0x7fbe300c6690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800cc00 tx=0x7fbe2800a060 comp rx=0 tx=0).stop 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 shutdown_connections 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe1c06ac90 0x7fbe1c06d150 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe3015db10 0x7fbe300c6690 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe300b2700 0x7fbe3015f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 --2- 172.21.15.5:0/1846700031 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe300cbb00 0x7fbe30157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 >> 172.21.15.5:0/1846700031 conn(0x7fbe300b2fa0 msgr2=0x7fbe3015adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:29.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 shutdown_connections 2024-05-08T23:21:29.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:29.666+0000 7fbe35913700 1 -- 172.21.15.5:0/1846700031 wait complete. 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:21:29.680 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T23:21:29.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:29.681 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-05-08T23:21:29.703 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:21:29.703 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:21:29.703 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:21:29.703 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:29.703 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-05-08T23:21:29.703 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2024-05-08T23:21:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:29.705 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:29.705 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:29.705 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-05-08T23:21:29.705 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:21:29.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:29.705 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:29.706 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:29.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:21:29.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-05-08T23:21:29.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-05-08T23:21:29.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:21:29.706 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:29.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:29.892 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:21:29.922 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-05-08T23:21:29.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:29.924 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-05-08T23:21:30.002 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-05-08T23:21:30.002 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:21:30.002 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:21:30.002 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:21:30.002 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:21:30.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:30.003 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:30.003 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:30.003 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:21:30.003 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:21:30.003 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:21:30.003 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:30.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:30.279 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-08T23:21:30.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:30.297 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-05-08T23:21:30.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:30.355 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:30.355 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:30.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T23:21:30.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-08T23:21:30.356 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:30.453 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-08T23:21:30.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:30.454 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:30.454 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:30.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-08T23:21:30.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:21:30.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-08T23:21:30.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-08T23:21:30.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-08T23:21:30.454 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:30.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:30.654 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:30.655 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:30.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-05-08T23:21:30.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:21:30.655 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:30.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:21:30.754 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T23:21:30.771 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T23:21:30.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:21:30.772 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T23:21:30.828 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-08T23:21:30.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:30.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:30.916 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:30.917 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:21:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 1@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:21:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:21:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:21:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:21:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:21:31.038 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 1@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:21:31.039 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:21:31.041 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:21:31.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:31.042 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T23:21:31.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:31.069 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:31.145 INFO:teuthology.orchestra.run.smithi049.stdout:client4621 2024-05-08T23:21:31.145 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:21:31.146 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:31.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:31.204 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:31.277 INFO:teuthology.orchestra.run.smithi049.stdout:client4621 2024-05-08T23:21:31.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:31.277 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:31.352 INFO:teuthology.orchestra.run.smithi049.stdout:client4621 2024-05-08T23:21:31.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:31.353 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4fd8ca56-5ac1-431d-b985-b50a13a8130a.client4621/status') 2024-05-08T23:21:31.435 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.4621 (3)192.168.0.1:0/393871167 2024-05-08T23:21:31.435 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-05-08T23:21:31.436 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-05-08T23:21:31.437 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-05-08T23:21:31.437 INFO:teuthology.orchestra.run.smithi049.stderr:68 bytes copied, 2.3316e-05 s, 2.9 MB/s 2024-05-08T23:21:31.437 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-05-08T23:21:31.437 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:21:31.437 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T23:21:31.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:21:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.583+0000 7f3a1f002700 1 Processor -- start 2024-05-08T23:21:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.583+0000 7f3a1f002700 1 -- start start 2024-05-08T23:21:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a1f002700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3a180d22f0 0x7f3a180ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a1f002700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a180b4430 con 0x7f3a180d13e0 2024-05-08T23:21:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a1f002700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a180b3820 con 0x7f3a1815d780 2024-05-08T23:21:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a1f002700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a180d45c0 con 0x7f3a180d22f0 2024-05-08T23:21:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a17fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3a180d13e0 0x7f3a180d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33008/0 (socket says 172.21.15.5:33008) 2024-05-08T23:21:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a17fff700 1 -- 172.21.15.5:0/3245800956 learned_addr learned my addr 172.21.15.5:0/3245800956 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a1d59f700 1 --2- 172.21.15.5:0/3245800956 >> v2:172.21.15.49:3300/0 conn(0x7f3a180d22f0 0x7f3a180ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a177fe700 1 -- 172.21.15.5:0/3245800956 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454435716 0 0) 0x7f3a180b4430 con 0x7f3a180d13e0 2024-05-08T23:21:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a177fe700 1 -- 172.21.15.5:0/3245800956 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a000035b0 con 0x7f3a180d13e0 2024-05-08T23:21:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a177fe700 1 -- 172.21.15.5:0/3245800956 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592859914 0 0) 0x7f3a180b3820 con 0x7f3a1815d780 2024-05-08T23:21:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a177fe700 1 -- 172.21.15.5:0/3245800956 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a180b4430 con 0x7f3a1815d780 2024-05-08T23:21:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a1d59f700 1 -- 172.21.15.5:0/3245800956 >> v1:172.21.15.49:6789/0 conn(0x7f3a1815d780 legacy=0x7f3a180b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a1d59f700 1 -- 172.21.15.5:0/3245800956 >> v1:172.21.15.5:6789/0 conn(0x7f3a180d13e0 legacy=0x7f3a180d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.585+0000 7f3a1d59f700 1 -- 172.21.15.5:0/3245800956 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180b3b00 con 0x7f3a180d22f0 2024-05-08T23:21:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a1d59f700 1 --2- 172.21.15.5:0/3245800956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a180d22f0 0x7f3a180ce3f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009d90 tx=0x7f3a1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=504d97396ad17c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a177fe700 1 -- 172.21.15.5:0/3245800956 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000e070 con 0x7f3a180d22f0 2024-05-08T23:21:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a177fe700 1 -- 172.21.15.5:0/3245800956 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1000bb20 con 0x7f3a180d22f0 2024-05-08T23:21:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a177fe700 1 -- 172.21.15.5:0/3245800956 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10012430 con 0x7f3a180d22f0 2024-05-08T23:21:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a1f002700 1 -- 172.21.15.5:0/3245800956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a180d22f0 msgr2=0x7f3a180ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a1f002700 1 --2- 172.21.15.5:0/3245800956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a180d22f0 0x7f3a180ce3f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009d90 tx=0x7f3a1000b330 comp rx=0 tx=0).stop 2024-05-08T23:21:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a1f002700 1 -- 172.21.15.5:0/3245800956 shutdown_connections 2024-05-08T23:21:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a1f002700 1 --2- 172.21.15.5:0/3245800956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a180d22f0 0x7f3a180ce3f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a1f002700 1 -- 172.21.15.5:0/3245800956 >> 172.21.15.5:0/3245800956 conn(0x7f3a1808afe0 msgr2=0x7f3a180d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a1f002700 1 -- 172.21.15.5:0/3245800956 shutdown_connections 2024-05-08T23:21:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.586+0000 7f3a1f002700 1 -- 172.21.15.5:0/3245800956 wait complete. 2024-05-08T23:21:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.588+0000 7f3a1f002700 1 Processor -- start 2024-05-08T23:21:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.588+0000 7f3a1f002700 1 -- start start 2024-05-08T23:21:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.589+0000 7f3a1f002700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a180d13e0 0x7f3a180a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.589+0000 7f3a1f002700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a180d22f0 0x7f3a18153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.589+0000 7f3a1f002700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1815d780 0x7f3a1814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.589+0000 7f3a1f002700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1814d9c0 con 0x7f3a180d22f0 2024-05-08T23:21:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.589+0000 7f3a1f002700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180b7250 con 0x7f3a1815d780 2024-05-08T23:21:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.589+0000 7f3a1f002700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180b7ab0 con 0x7f3a180d13e0 2024-05-08T23:21:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a180d22f0 0x7f3a18153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a180d22f0 0x7f3a18153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41880/0 (socket says 172.21.15.5:41880) 2024-05-08T23:21:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 -- 172.21.15.5:0/408506048 learned_addr learned my addr 172.21.15.5:0/408506048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a1d59f700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1815d780 0x7f3a1814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a1cd9e700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a180d13e0 0x7f3a180a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 -- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a180d13e0 msgr2=0x7f3a180a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a180d13e0 0x7f3a180a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 -- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1815d780 msgr2=0x7f3a1814d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1815d780 0x7f3a1814d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 -- 172.21.15.5:0/408506048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009a70 con 0x7f3a180d22f0 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a1d59f700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1815d780 0x7f3a1814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a17fff700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a180d22f0 0x7f3a18153520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c001eb0 tx=0x7f3a0c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a157fa700 1 -- 172.21.15.5:0/408506048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c005d70 con 0x7f3a180d22f0 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a157fa700 1 -- 172.21.15.5:0/408506048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c0070d0 con 0x7f3a180d22f0 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1815dff0 con 0x7f3a180d22f0 2024-05-08T23:21:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a157fa700 1 -- 172.21.15.5:0/408506048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c007460 con 0x7f3a180d22f0 2024-05-08T23:21:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.590+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a180b7740 con 0x7f3a180d22f0 2024-05-08T23:21:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.591+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a180d1a50 con 0x7f3a180d22f0 2024-05-08T23:21:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.592+0000 7f3a157fa700 1 -- 172.21.15.5:0/408506048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a0c0075c0 con 0x7f3a180d22f0 2024-05-08T23:21:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.592+0000 7f3a157fa700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a0406ac90 0x7f3a0406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.592+0000 7f3a157fa700 1 -- 172.21.15.5:0/408506048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a0c088700 con 0x7f3a180d22f0 2024-05-08T23:21:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.593+0000 7f3a1cd9e700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a0406ac90 0x7f3a0406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.593+0000 7f3a1cd9e700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a0406ac90 0x7f3a0406d150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a08007fd0 tx=0x7f3a08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:31.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.597+0000 7f3a157fa700 1 -- 172.21.15.5:0/408506048 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3a0c0538e0 con 0x7f3a180d22f0 2024-05-08T23:21:31.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.865+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3a180b7900 con 0x7f3a180d22f0 2024-05-08T23:21:31.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.866+0000 7f3a157fa700 1 -- 172.21.15.5:0/408506048 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3a0c017060 con 0x7f3a180d22f0 2024-05-08T23:21:31.866 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:21:31.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a0406ac90 msgr2=0x7f3a0406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:31.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a0406ac90 0x7f3a0406d150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a08007fd0 tx=0x7f3a08009040 comp rx=0 tx=0).stop 2024-05-08T23:21:31.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a180d22f0 msgr2=0x7f3a18153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a180d22f0 0x7f3a18153520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c001eb0 tx=0x7f3a0c00c7b0 comp rx=0 tx=0).stop 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 shutdown_connections 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a0406ac90 0x7f3a0406d150 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1815d780 0x7f3a1814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a180d13e0 0x7f3a180a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 --2- 172.21.15.5:0/408506048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a180d22f0 0x7f3a18153520 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 >> 172.21.15.5:0/408506048 conn(0x7f3a1808afe0 msgr2=0x7f3a180b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 shutdown_connections 2024-05-08T23:21:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:31.868+0000 7f3a1f002700 1 -- 172.21.15.5:0/408506048 wait complete. 2024-05-08T23:21:31.882 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:21:31.882 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:21:31.882 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-05-08T23:21:31.883 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-08T23:21:31.883 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T23:21:31.883 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:21:31.883 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:21:31.883 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:21:31.883 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-08T23:21:31.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:31.884 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-05-08T23:21:31.904 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:21:31.904 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:21:31.904 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:21:31.904 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:31.904 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-05-08T23:21:31.905 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:31.905 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:31.905 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:31.905 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-05-08T23:21:31.905 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:31.905 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:31.906 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-05-08T23:21:31.906 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:21:31.906 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:31.906 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-05-08T23:21:31.906 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:21:31.907 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42195sec preferred_lft 42195sec 2024-05-08T23:21:31.907 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2024-05-08T23:21:31.907 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:31.907 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:21:31.907 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:31.907 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-05-08T23:21:31.907 INFO:teuthology.orchestra.run.smithi049.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:21:31.907 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 76:a1:bd:eb:32:d6 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::640c:e9ff:fea6:b18b/64 scope link 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run.smithi049.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 76:a1:bd:eb:32:d6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::74a1:bdff:feeb:32d6/64 scope link tentative 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:31.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:31.908 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:21:31.972 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-05-08T23:21:31.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:31.974 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-05-08T23:21:32.033 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-05-08T23:21:32.033 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:21:32.034 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:21:32.034 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:21:32.034 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:21:32.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.034 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:32.034 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:32.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:21:32.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:21:32.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:21:32.034 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:32.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.143 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-08T23:21:32.161 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T23:21:32.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.161 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-05-08T23:21:32.221 INFO:teuthology.orchestra.run.smithi049.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-08T23:21:32.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.222 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:32.222 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:32.222 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T23:21:32.222 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-08T23:21:32.222 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:32.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.319 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T23:21:32.362 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 26:9f:4e:a8:c1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::249f:4eff:fea8:c10f/64 scope link tentative 2024-05-08T23:21:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:32.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.374 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 26:9f:4e:a8:c1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:32.414 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::249f:4eff:fea8:c10f/64 scope link tentative 2024-05-08T23:21:32.415 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:21:32.433 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-08T23:21:32.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.433 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:32.433 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:32.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-08T23:21:32.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T23:21:32.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-08T23:21:32.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-08T23:21:32.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-08T23:21:32.434 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:32.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.679 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2024-05-08T23:21:32.679 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-08T23:21:32.680 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.1 up 2024-05-08T23:21:32.680 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T23:21:32.680 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-08T23:21:32.779 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:21:32.779 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-08T23:21:32.796 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-08T23:21:32.796 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:21:32.797 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-08T23:21:32.854 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-05-08T23:21:32.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.854 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:32.941 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:32.941 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:21:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 3@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:21:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:21:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:21:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:21:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:21:32.979 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:21:32.979 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:21:32.979 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:21:32.979 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 3@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:21:32.979 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:21:32.980 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:21:32.980 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:21:32.980 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:21:32.980 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:21:32.980 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:21:32.980 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:21:32.981 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:21:32.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:32.982 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-08T23:21:33.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:33.010 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:33.083 INFO:teuthology.orchestra.run.smithi049.stdout:client4633 2024-05-08T23:21:33.084 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:21:33.084 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:33.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:33.142 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:33.216 INFO:teuthology.orchestra.run.smithi049.stdout:client4633 2024-05-08T23:21:33.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:33.217 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:21:33.291 INFO:teuthology.orchestra.run.smithi049.stdout:client4633 2024-05-08T23:21:33.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:33.292 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4fd8ca56-5ac1-431d-b985-b50a13a8130a.client4633/status') 2024-05-08T23:21:33.374 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.4633 (3)192.168.0.2:0/1016774207 2024-05-08T23:21:33.374 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-05-08T23:21:33.375 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-05-08T23:21:33.375 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-05-08T23:21:33.375 INFO:teuthology.orchestra.run.smithi049.stderr:69 bytes copied, 2.2906e-05 s, 3.0 MB/s 2024-05-08T23:21:33.375 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-08T23:21:33.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.539+0000 7fee08724700 1 Processor -- start 2024-05-08T23:21:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.539+0000 7fee08724700 1 -- start start 2024-05-08T23:21:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee08724700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fee00099a50 0x7fee00099f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee08724700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee000ccdd0 0x7fee000cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee08724700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0009aca0 con 0x7fee0015b810 2024-05-08T23:21:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee08724700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0009a440 con 0x7fee00099a50 2024-05-08T23:21:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee08724700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee000cd7a0 con 0x7fee000ccdd0 2024-05-08T23:21:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee06cc1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee000ccdd0 0x7fee000cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee05cbf700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fee00099a50 0x7fee00099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee06cc1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee000ccdd0 0x7fee000cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:44976/0 (socket says 172.21.15.5:44976) 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee05cbf700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fee00099a50 0x7fee00099f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54680/0 (socket says 172.21.15.5:54680) 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee064c0700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fee0015b810 0x7fee000cffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:49810/0 (socket says 172.21.15.5:49810) 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.540+0000 7fee06cc1700 1 -- 172.21.15.5:0/1072693943 learned_addr learned my addr 172.21.15.5:0/1072693943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee06cc1700 1 -- 172.21.15.5:0/1072693943 >> v2:172.21.15.49:3300/0 conn(0x7fee00099a50 msgr2=0x7fee00099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee06cc1700 1 --2- 172.21.15.5:0/1072693943 >> v2:172.21.15.49:3300/0 conn(0x7fee00099a50 0x7fee00099f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee06cc1700 1 -- 172.21.15.5:0/1072693943 >> v1:172.21.15.49:6789/0 conn(0x7fee0015b810 legacy=0x7fee000cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee06cc1700 1 -- 172.21.15.5:0/1072693943 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee000cd9e0 con 0x7fee000ccdd0 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee05cbf700 1 --2- 172.21.15.5:0/1072693943 >> v2:172.21.15.49:3300/0 conn(0x7fee00099a50 0x7fee00099f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee06cc1700 1 --2- 172.21.15.5:0/1072693943 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee000ccdd0 0x7fee000cd260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009e50 tx=0x7fedf800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898383a18c6209c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee054be700 1 -- 172.21.15.5:0/1072693943 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800e070 con 0x7fee000ccdd0 2024-05-08T23:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee054be700 1 -- 172.21.15.5:0/1072693943 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf800b910 con 0x7fee000ccdd0 2024-05-08T23:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.541+0000 7fee054be700 1 -- 172.21.15.5:0/1072693943 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800bcc0 con 0x7fee000ccdd0 2024-05-08T23:21:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.542+0000 7fee08724700 1 -- 172.21.15.5:0/1072693943 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee000ccdd0 msgr2=0x7fee000cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.542+0000 7fee08724700 1 --2- 172.21.15.5:0/1072693943 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee000ccdd0 0x7fee000cd260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009e50 tx=0x7fedf800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.542+0000 7fee08724700 1 -- 172.21.15.5:0/1072693943 shutdown_connections 2024-05-08T23:21:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.542+0000 7fee08724700 1 --2- 172.21.15.5:0/1072693943 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee000ccdd0 0x7fee000cd260 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.542+0000 7fee08724700 1 --2- 172.21.15.5:0/1072693943 >> v2:172.21.15.49:3300/0 conn(0x7fee00099a50 0x7fee00099f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.542+0000 7fee08724700 1 -- 172.21.15.5:0/1072693943 >> 172.21.15.5:0/1072693943 conn(0x7fee000b3010 msgr2=0x7fee000b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.542+0000 7fee08724700 1 -- 172.21.15.5:0/1072693943 shutdown_connections 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.542+0000 7fee08724700 1 -- 172.21.15.5:0/1072693943 wait complete. 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.543+0000 7fee08724700 1 Processor -- start 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.543+0000 7fee08724700 1 -- start start 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee08724700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee00099a50 0x7fee0015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee08724700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee000ccdd0 0x7fee001f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee08724700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee0015b810 0x7fee000b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee08724700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee001f5bc0 con 0x7fee000ccdd0 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee08724700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0015e140 con 0x7fee0015b810 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee08724700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0015e420 con 0x7fee00099a50 2024-05-08T23:21:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee05cbf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee000ccdd0 0x7fee001f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee05cbf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee000ccdd0 0x7fee001f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41884/0 (socket says 172.21.15.5:41884) 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee05cbf700 1 -- 172.21.15.5:0/3568195020 learned_addr learned my addr 172.21.15.5:0/3568195020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee06cc1700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee0015b810 0x7fee000b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee064c0700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee00099a50 0x7fee0015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee05cbf700 1 -- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee00099a50 msgr2=0x7fee0015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee05cbf700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee00099a50 0x7fee0015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee05cbf700 1 -- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee0015b810 msgr2=0x7fee000b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.544+0000 7fee05cbf700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee0015b810 0x7fee000b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.545+0000 7fee05cbf700 1 -- 172.21.15.5:0/3568195020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00094a0 con 0x7fee000ccdd0 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.545+0000 7fee06cc1700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee0015b810 0x7fee000b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.545+0000 7fee05cbf700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee000ccdd0 0x7fee001f5680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedf000ebe0 tx=0x7fedf000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.545+0000 7fedf6ffd700 1 -- 172.21.15.5:0/3568195020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000c850 con 0x7fee000ccdd0 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.545+0000 7fedf6ffd700 1 -- 172.21.15.5:0/3568195020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf000c9b0 con 0x7fee000ccdd0 2024-05-08T23:21:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.545+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf8009b30 con 0x7fee000ccdd0 2024-05-08T23:21:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.545+0000 7fedf6ffd700 1 -- 172.21.15.5:0/3568195020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000cd40 con 0x7fee000ccdd0 2024-05-08T23:21:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.545+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0015e9e0 con 0x7fee000ccdd0 2024-05-08T23:21:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.546+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee00068000 con 0x7fee000ccdd0 2024-05-08T23:21:33.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.546+0000 7fedf6ffd700 1 -- 172.21.15.5:0/3568195020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fedf0013470 con 0x7fee000ccdd0 2024-05-08T23:21:33.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.547+0000 7fedf6ffd700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fede406ac90 0x7fede406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.547+0000 7fedf6ffd700 1 -- 172.21.15.5:0/3568195020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedf00894c0 con 0x7fee000ccdd0 2024-05-08T23:21:33.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.547+0000 7fee064c0700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fede406ac90 0x7fede406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.548+0000 7fee064c0700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fede406ac90 0x7fede406d150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fedfc000f60 tx=0x7fedfc007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:33.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.552+0000 7fedf6ffd700 1 -- 172.21.15.5:0/3568195020 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fedf0054600 con 0x7fee000ccdd0 2024-05-08T23:21:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.771+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee00099f00 con 0x7fee000ccdd0 2024-05-08T23:21:33.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.772+0000 7fedf6ffd700 1 -- 172.21.15.5:0/3568195020 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7fedf000cea0 con 0x7fee000ccdd0 2024-05-08T23:21:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:33.774 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:33.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fede406ac90 msgr2=0x7fede406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fede406ac90 0x7fede406d150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fedfc000f60 tx=0x7fedfc007ed0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee000ccdd0 msgr2=0x7fee001f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee000ccdd0 0x7fee001f5680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedf000ebe0 tx=0x7fedf000eef0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 shutdown_connections 2024-05-08T23:21:33.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fede406ac90 0x7fede406d150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee0015b810 0x7fee000b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee00099a50 0x7fee0015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 --2- 172.21.15.5:0/3568195020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee000ccdd0 0x7fee001f5680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.774+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 >> 172.21.15.5:0/3568195020 conn(0x7fee000b3010 msgr2=0x7fee000b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:33.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.775+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 shutdown_connections 2024-05-08T23:21:33.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.775+0000 7fee08724700 1 -- 172.21.15.5:0/3568195020 wait complete. 2024-05-08T23:21:33.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:33.789 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:33.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.929+0000 7f5c9c684700 1 Processor -- start 2024-05-08T23:21:33.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.929+0000 7f5c9c684700 1 -- start start 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9c684700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c9415d8c0 0x7f5c940ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9c684700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b5550 0x7f5c941618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9c684700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c94099ce0 con 0x7f5c940b50d0 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9c684700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9406fde0 con 0x7f5c9415d8c0 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9c684700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9406f260 con 0x7f5c940b5550 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9a420700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c9415d8c0 0x7f5c940ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9a420700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c9415d8c0 0x7f5c940ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41896/0 (socket says 172.21.15.5:41896) 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9a420700 1 -- 172.21.15.5:0/654357464 learned_addr learned my addr 172.21.15.5:0/654357464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.930+0000 7f5c9ac21700 1 --2- 172.21.15.5:0/654357464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b5550 0x7f5c941618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9a420700 1 -- 172.21.15.5:0/654357464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b5550 msgr2=0x7f5c941618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9a420700 1 --2- 172.21.15.5:0/654357464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b5550 0x7f5c941618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9a420700 1 -- 172.21.15.5:0/654357464 >> v1:172.21.15.49:6789/0 conn(0x7f5c940b50d0 legacy=0x7f5c9406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9a420700 1 -- 172.21.15.5:0/654357464 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9406f540 con 0x7f5c9415d8c0 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9ac21700 1 --2- 172.21.15.5:0/654357464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b5550 0x7f5c941618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9a420700 1 --2- 172.21.15.5:0/654357464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9415d8c0 0x7f5c940ad260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009e50 tx=0x7f5c8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fa9e710585f5cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9941e700 1 -- 172.21.15.5:0/654357464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400e070 con 0x7f5c9415d8c0 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9941e700 1 -- 172.21.15.5:0/654357464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8400b9c0 con 0x7f5c9415d8c0 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9941e700 1 -- 172.21.15.5:0/654357464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400bd70 con 0x7f5c9415d8c0 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9c684700 1 -- 172.21.15.5:0/654357464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9415d8c0 msgr2=0x7f5c940ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.931+0000 7f5c9c684700 1 --2- 172.21.15.5:0/654357464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9415d8c0 0x7f5c940ad260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009e50 tx=0x7f5c8400b330 comp rx=0 tx=0).stop 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.932+0000 7f5c9c684700 1 -- 172.21.15.5:0/654357464 shutdown_connections 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.932+0000 7f5c9c684700 1 --2- 172.21.15.5:0/654357464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b5550 0x7f5c941618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.932+0000 7f5c9c684700 1 --2- 172.21.15.5:0/654357464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9415d8c0 0x7f5c940ad260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.932+0000 7f5c9c684700 1 -- 172.21.15.5:0/654357464 >> 172.21.15.5:0/654357464 conn(0x7f5c940b5a30 msgr2=0x7f5c940b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.932+0000 7f5c9c684700 1 -- 172.21.15.5:0/654357464 shutdown_connections 2024-05-08T23:21:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.932+0000 7f5c9c684700 1 -- 172.21.15.5:0/654357464 wait complete. 2024-05-08T23:21:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.933+0000 7f5c9c684700 1 Processor -- start 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.933+0000 7f5c9c684700 1 -- start start 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9c684700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b50d0 0x7f5c9415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9c684700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c940b5550 0x7f5c941423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9c684700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9415d8c0 0x7f5c941f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9c684700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9415d070 con 0x7f5c940b5550 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9c684700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94142930 con 0x7f5c9415d8c0 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9c684700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9415ee90 con 0x7f5c940b50d0 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c940b5550 0x7f5c941423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c940b5550 0x7f5c941423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41908/0 (socket says 172.21.15.5:41908) 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 -- 172.21.15.5:0/3566625747 learned_addr learned my addr 172.21.15.5:0/3566625747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9ac21700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9415d8c0 0x7f5c941f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9a420700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b50d0 0x7f5c9415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 -- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b50d0 msgr2=0x7f5c9415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b50d0 0x7f5c9415ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 -- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9415d8c0 msgr2=0x7f5c941f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9415d8c0 0x7f5c941f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 -- 172.21.15.5:0/3566625747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84009b30 con 0x7f5c940b5550 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9ac21700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9415d8c0 0x7f5c941f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c9a420700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b50d0 0x7f5c9415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.934+0000 7f5c99c1f700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c940b5550 0x7f5c941423f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c480 tx=0x7f5c8800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.935+0000 7f5c92ffd700 1 -- 172.21.15.5:0/3566625747 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c880052d0 con 0x7f5c940b5550 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.935+0000 7f5c92ffd700 1 -- 172.21.15.5:0/3566625747 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c88005430 con 0x7f5c940b5550 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.935+0000 7f5c92ffd700 1 -- 172.21.15.5:0/3566625747 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c880038c0 con 0x7f5c940b5550 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.935+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9415f170 con 0x7f5c940b5550 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.935+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9415f730 con 0x7f5c940b5550 2024-05-08T23:21:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.936+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c94074bc0 con 0x7f5c940b5550 2024-05-08T23:21:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.936+0000 7f5c92ffd700 1 -- 172.21.15.5:0/3566625747 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5c88003a20 con 0x7f5c940b5550 2024-05-08T23:21:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.937+0000 7f5c92ffd700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c8006ac90 0x7f5c8006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.937+0000 7f5c92ffd700 1 -- 172.21.15.5:0/3566625747 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c88088740 con 0x7f5c940b5550 2024-05-08T23:21:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.937+0000 7f5c9a420700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c8006ac90 0x7f5c8006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.937+0000 7f5c9a420700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c8006ac90 0x7f5c8006d150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000f80 tx=0x7f5c84009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:33.941+0000 7f5c92ffd700 1 -- 172.21.15.5:0/3566625747 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5c88053920 con 0x7f5c940b5550 2024-05-08T23:21:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.159+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c9406b640 con 0x7f5c940b5550 2024-05-08T23:21:34.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.160+0000 7f5c92ffd700 1 -- 172.21.15.5:0/3566625747 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c88017020 con 0x7f5c940b5550 2024-05-08T23:21:34.160 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:34.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c8006ac90 msgr2=0x7f5c8006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c8006ac90 0x7f5c8006d150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000f80 tx=0x7f5c84009000 comp rx=0 tx=0).stop 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c940b5550 msgr2=0x7f5c941423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c940b5550 0x7f5c941423f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c480 tx=0x7f5c8800c840 comp rx=0 tx=0).stop 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 shutdown_connections 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c8006ac90 0x7f5c8006d150 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9415d8c0 0x7f5c941f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c940b50d0 0x7f5c9415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 --2- 172.21.15.5:0/3566625747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c940b5550 0x7f5c941423f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 >> 172.21.15.5:0/3566625747 conn(0x7f5c940b5a30 msgr2=0x7f5c940b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 shutdown_connections 2024-05-08T23:21:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.162+0000 7f5c9c684700 1 -- 172.21.15.5:0/3566625747 wait complete. 2024-05-08T23:21:34.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:34.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.316+0000 7faa822dd700 1 Processor -- start 2024-05-08T23:21:34.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.316+0000 7faa822dd700 1 -- start start 2024-05-08T23:21:34.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.317+0000 7faa822dd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa7c0cbf50 0x7faa7c161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.317+0000 7faa822dd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c162240 0x7faa7c15e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.317+0000 7faa822dd700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c0cf970 con 0x7faa7c0d0f10 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.317+0000 7faa822dd700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c0cfc50 con 0x7faa7c0cbf50 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.317+0000 7faa822dd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0cff30 con 0x7faa7c162240 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7b7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c162240 0x7faa7c15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa7c0cbf50 0x7faa7c161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7b7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c162240 0x7faa7c15e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:45014/0 (socket says 172.21.15.5:45014) 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7b7fe700 1 -- 172.21.15.5:0/696757982 learned_addr learned my addr 172.21.15.5:0/696757982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7b7fe700 1 -- 172.21.15.5:0/696757982 >> v2:172.21.15.49:3300/0 conn(0x7faa7c0cbf50 msgr2=0x7faa7c161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7b7fe700 1 --2- 172.21.15.5:0/696757982 >> v2:172.21.15.49:3300/0 conn(0x7faa7c0cbf50 0x7faa7c161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7b7fe700 1 -- 172.21.15.5:0/696757982 >> v1:172.21.15.49:6789/0 conn(0x7faa7c0d0f10 legacy=0x7faa7c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7b7fe700 1 -- 172.21.15.5:0/696757982 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c1617e0 con 0x7faa7c162240 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.318+0000 7faa7b7fe700 1 --2- 172.21.15.5:0/696757982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c162240 0x7faa7c15e050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faa700098f0 tx=0x7faa70009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d03c3feb06338a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:34.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa7affd700 1 -- 172.21.15.5:0/696757982 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70011830 con 0x7faa7c162240 2024-05-08T23:21:34.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa7affd700 1 -- 172.21.15.5:0/696757982 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa70011990 con 0x7faa7c162240 2024-05-08T23:21:34.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa7affd700 1 -- 172.21.15.5:0/696757982 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70011d20 con 0x7faa7c162240 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa822dd700 1 -- 172.21.15.5:0/696757982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c162240 msgr2=0x7faa7c15e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa822dd700 1 --2- 172.21.15.5:0/696757982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c162240 0x7faa7c15e050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faa700098f0 tx=0x7faa70009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa822dd700 1 -- 172.21.15.5:0/696757982 shutdown_connections 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa822dd700 1 --2- 172.21.15.5:0/696757982 >> v2:172.21.15.49:3300/0 conn(0x7faa7c0cbf50 0x7faa7c161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa822dd700 1 --2- 172.21.15.5:0/696757982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c162240 0x7faa7c15e050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa822dd700 1 -- 172.21.15.5:0/696757982 >> 172.21.15.5:0/696757982 conn(0x7faa7c099a50 msgr2=0x7faa7c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa822dd700 1 -- 172.21.15.5:0/696757982 shutdown_connections 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.319+0000 7faa822dd700 1 -- 172.21.15.5:0/696757982 wait complete. 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.321+0000 7faa822dd700 1 Processor -- start 2024-05-08T23:21:34.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.321+0000 7faa822dd700 1 -- start start 2024-05-08T23:21:34.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa822dd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c0cbf50 0x7faa7c1e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa822dd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa7c0d0f10 0x7faa7c151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa822dd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa7c162240 0x7faa7c152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa822dd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1e9850 con 0x7faa7c162240 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa822dd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c15e810 con 0x7faa7c0d0f10 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa822dd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1ea1b0 con 0x7faa7c0cbf50 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa8087a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa7c162240 0x7faa7c152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa8087a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa7c162240 0x7faa7c152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41922/0 (socket says 172.21.15.5:41922) 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa8087a700 1 -- 172.21.15.5:0/2559443826 learned_addr learned my addr 172.21.15.5:0/2559443826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa7b7fe700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa7c0d0f10 0x7faa7c151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa7bfff700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c0cbf50 0x7faa7c1e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa7bfff700 1 -- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa7c0d0f10 msgr2=0x7faa7c151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa7bfff700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa7c0d0f10 0x7faa7c151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa7bfff700 1 -- 172.21.15.5:0/2559443826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa7c162240 msgr2=0x7faa7c152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa7bfff700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa7c162240 0x7faa7c152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa7bfff700 1 -- 172.21.15.5:0/2559443826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa70009550 con 0x7faa7c0cbf50 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa8087a700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa7c162240 0x7faa7c152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.322+0000 7faa7bfff700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c0cbf50 0x7faa7c1e9c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e900 tx=0x7faa6c00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.323+0000 7faa78ff9700 1 -- 172.21.15.5:0/2559443826 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c013680 con 0x7faa7c0cbf50 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.323+0000 7faa78ff9700 1 -- 172.21.15.5:0/2559443826 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c0137e0 con 0x7faa7c0cbf50 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.323+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c1e8bd0 con 0x7faa7c0cbf50 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.323+0000 7faa78ff9700 1 -- 172.21.15.5:0/2559443826 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c013b50 con 0x7faa7c0cbf50 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.323+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c1e9190 con 0x7faa7c0cbf50 2024-05-08T23:21:34.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.324+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c074bc0 con 0x7faa7c0cbf50 2024-05-08T23:21:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.324+0000 7faa78ff9700 1 -- 172.21.15.5:0/2559443826 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7faa6c013cb0 con 0x7faa7c0cbf50 2024-05-08T23:21:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.325+0000 7faa78ff9700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa6806ac90 0x7faa6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.325+0000 7faa78ff9700 1 -- 172.21.15.5:0/2559443826 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa6c0892a0 con 0x7faa7c0cbf50 2024-05-08T23:21:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.325+0000 7faa7b7fe700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa6806ac90 0x7faa6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.325+0000 7faa7b7fe700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa6806ac90 0x7faa6806d150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa70014040 tx=0x7faa70008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:34.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.330+0000 7faa78ff9700 1 -- 172.21.15.5:0/2559443826 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7faa6c054470 con 0x7faa7c0cbf50 2024-05-08T23:21:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.547+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa7c1e9ac0 con 0x7faa7c0cbf50 2024-05-08T23:21:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.548+0000 7faa78ff9700 1 -- 172.21.15.5:0/2559443826 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa6c057a90 con 0x7faa7c0cbf50 2024-05-08T23:21:34.548 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa6806ac90 msgr2=0x7faa6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa6806ac90 0x7faa6806d150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa70014040 tx=0x7faa70008030 comp rx=0 tx=0).stop 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c0cbf50 msgr2=0x7faa7c1e9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c0cbf50 0x7faa7c1e9c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e900 tx=0x7faa6c00ec10 comp rx=0 tx=0).stop 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 shutdown_connections 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa6806ac90 0x7faa6806d150 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa7c162240 0x7faa7c152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa7c0cbf50 0x7faa7c1e9c70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 --2- 172.21.15.5:0/2559443826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa7c0d0f10 0x7faa7c151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 >> 172.21.15.5:0/2559443826 conn(0x7faa7c099a50 msgr2=0x7faa7c15c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 shutdown_connections 2024-05-08T23:21:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.550+0000 7faa822dd700 1 -- 172.21.15.5:0/2559443826 wait complete. 2024-05-08T23:21:34.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.703+0000 7fde95729700 1 Processor -- start 2024-05-08T23:21:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.703+0000 7fde95729700 1 -- start start 2024-05-08T23:21:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde95729700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fde900d1e50 0x7fde900ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde95729700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde90161320 0x7fde9006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde95729700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde900cf560 con 0x7fde900d2230 2024-05-08T23:21:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde95729700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde900ce950 con 0x7fde90161320 2024-05-08T23:21:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde95729700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde900cec30 con 0x7fde900d1e50 2024-05-08T23:21:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde8f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde90161320 0x7fde9006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde8e7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fde900d2230 0x7fde90160c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33024/0 (socket says 172.21.15.5:33024) 2024-05-08T23:21:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde8e7fc700 1 -- 172.21.15.5:0/3322927121 learned_addr learned my addr 172.21.15.5:0/3322927121 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde8effd700 1 --2- 172.21.15.5:0/3322927121 >> v2:172.21.15.49:3300/0 conn(0x7fde900d1e50 0x7fde900ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde8f7fe700 1 -- 172.21.15.5:0/3322927121 >> v2:172.21.15.49:3300/0 conn(0x7fde900d1e50 msgr2=0x7fde900ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.704+0000 7fde8f7fe700 1 --2- 172.21.15.5:0/3322927121 >> v2:172.21.15.49:3300/0 conn(0x7fde900d1e50 0x7fde900ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde8f7fe700 1 -- 172.21.15.5:0/3322927121 >> v1:172.21.15.5:6789/0 conn(0x7fde900d2230 legacy=0x7fde90160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde8f7fe700 1 -- 172.21.15.5:0/3322927121 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde900cef10 con 0x7fde90161320 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde8dffb700 1 -- 172.21.15.5:0/3322927121 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616557270 0 0) 0x7fde900cf560 con 0x7fde900d2230 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde8f7fe700 1 --2- 172.21.15.5:0/3322927121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde90161320 0x7fde9006bad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fde9008d7c0 tx=0x7fde8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0d85245d0ce6b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde8dffb700 1 -- 172.21.15.5:0/3322927121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8800e070 con 0x7fde90161320 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde8dffb700 1 -- 172.21.15.5:0/3322927121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8800bb20 con 0x7fde90161320 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde8dffb700 1 -- 172.21.15.5:0/3322927121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88012430 con 0x7fde90161320 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde95729700 1 -- 172.21.15.5:0/3322927121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde90161320 msgr2=0x7fde9006bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde95729700 1 --2- 172.21.15.5:0/3322927121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde90161320 0x7fde9006bad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fde9008d7c0 tx=0x7fde8800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde95729700 1 -- 172.21.15.5:0/3322927121 shutdown_connections 2024-05-08T23:21:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde95729700 1 --2- 172.21.15.5:0/3322927121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde90161320 0x7fde9006bad0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.705+0000 7fde95729700 1 --2- 172.21.15.5:0/3322927121 >> v2:172.21.15.49:3300/0 conn(0x7fde900d1e50 0x7fde900ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.706+0000 7fde95729700 1 -- 172.21.15.5:0/3322927121 >> 172.21.15.5:0/3322927121 conn(0x7fde9008afe0 msgr2=0x7fde900d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.706+0000 7fde95729700 1 -- 172.21.15.5:0/3322927121 shutdown_connections 2024-05-08T23:21:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.706+0000 7fde95729700 1 -- 172.21.15.5:0/3322927121 wait complete. 2024-05-08T23:21:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.707+0000 7fde95729700 1 Processor -- start 2024-05-08T23:21:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.707+0000 7fde95729700 1 -- start start 2024-05-08T23:21:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde95729700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde900d1e50 0x7fde9015cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde95729700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde900d2230 0x7fde900d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde95729700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde90161320 0x7fde900d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde95729700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c7120 con 0x7fde900d2230 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde95729700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900cc960 con 0x7fde900d1e50 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde95729700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900ccc40 con 0x7fde90161320 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde900d2230 0x7fde900d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde900d2230 0x7fde900d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41940/0 (socket says 172.21.15.5:41940) 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8e7fc700 1 -- 172.21.15.5:0/3445813409 learned_addr learned my addr 172.21.15.5:0/3445813409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8effd700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde900d1e50 0x7fde9015cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8f7fe700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde90161320 0x7fde900d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8e7fc700 1 -- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde90161320 msgr2=0x7fde900d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8e7fc700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde90161320 0x7fde900d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8e7fc700 1 -- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde900d1e50 msgr2=0x7fde9015cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8e7fc700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde900d1e50 0x7fde9015cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8e7fc700 1 -- 172.21.15.5:0/3445813409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88009a70 con 0x7fde900d2230 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.708+0000 7fde8effd700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde900d1e50 0x7fde9015cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.709+0000 7fde8e7fc700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde900d2230 0x7fde900d0000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fde840011e0 tx=0x7fde8400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.709+0000 7fde777fe700 1 -- 172.21.15.5:0/3445813409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84004c80 con 0x7fde900d2230 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.709+0000 7fde777fe700 1 -- 172.21.15.5:0/3445813409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde84004e00 con 0x7fde900d2230 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.709+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde900ccf20 con 0x7fde900d2230 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.709+0000 7fde777fe700 1 -- 172.21.15.5:0/3445813409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84007310 con 0x7fde900d2230 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.709+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde900b42d0 con 0x7fde900d2230 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.710+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde900c7290 con 0x7fde900d2230 2024-05-08T23:21:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.710+0000 7fde777fe700 1 -- 172.21.15.5:0/3445813409 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fde84007470 con 0x7fde900d2230 2024-05-08T23:21:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.711+0000 7fde777fe700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde7c06ac90 0x7fde7c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.711+0000 7fde777fe700 1 -- 172.21.15.5:0/3445813409 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde84088ed0 con 0x7fde900d2230 2024-05-08T23:21:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.711+0000 7fde8effd700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde7c06ac90 0x7fde7c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.712+0000 7fde8effd700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde7c06ac90 0x7fde7c06d150 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde80009fd0 tx=0x7fde80009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:34.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.716+0000 7fde777fe700 1 -- 172.21.15.5:0/3445813409 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fde84054140 con 0x7fde900d2230 2024-05-08T23:21:34.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.937+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde90074bc0 con 0x7fde900d2230 2024-05-08T23:21:34.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.937+0000 7fde777fe700 1 -- 172.21.15.5:0/3445813409 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde84057760 con 0x7fde900d2230 2024-05-08T23:21:34.938 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde7c06ac90 msgr2=0x7fde7c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde7c06ac90 0x7fde7c06d150 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde80009fd0 tx=0x7fde80009210 comp rx=0 tx=0).stop 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde900d2230 msgr2=0x7fde900d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde900d2230 0x7fde900d0000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fde840011e0 tx=0x7fde8400ca30 comp rx=0 tx=0).stop 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 shutdown_connections 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde7c06ac90 0x7fde7c06d150 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde90161320 0x7fde900d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde900d1e50 0x7fde9015cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 --2- 172.21.15.5:0/3445813409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde900d2230 0x7fde900d0000 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.939+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 >> 172.21.15.5:0/3445813409 conn(0x7fde9008afe0 msgr2=0x7fde900b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.940+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 shutdown_connections 2024-05-08T23:21:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:34.940+0000 7fde95729700 1 -- 172.21.15.5:0/3445813409 wait complete. 2024-05-08T23:21:34.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:35.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.089+0000 7f34f816a700 1 Processor -- start 2024-05-08T23:21:35.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.089+0000 7f34f816a700 1 -- start start 2024-05-08T23:21:35.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.090+0000 7f34f816a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f34f015dcc0 0x7f34f006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.090+0000 7f34f816a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00cef60 con 0x7f34f015e310 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.090+0000 7f34f816a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f015eee0 con 0x7f34f006fa10 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.090+0000 7f34f816a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f00b2de0 con 0x7f34f015dcc0 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.090+0000 7f34f6707700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f34f015e310 0x7f34f015e6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33036/0 (socket says 172.21.15.5:33036) 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.090+0000 7f34f6707700 1 -- 172.21.15.5:0/4286050692 learned_addr learned my addr 172.21.15.5:0/4286050692 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f5f06700 1 --2- 172.21.15.5:0/4286050692 >> v2:172.21.15.49:3300/0 conn(0x7f34f015dcc0 0x7f34f006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f4f04700 1 -- 172.21.15.5:0/4286050692 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200672715 0 0) 0x7f34f00cef60 con 0x7f34f015e310 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f4f04700 1 -- 172.21.15.5:0/4286050692 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34dc0035b0 con 0x7f34f015e310 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f4f04700 1 -- 172.21.15.5:0/4286050692 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875269999 0 0) 0x7f34f015eee0 con 0x7f34f006fa10 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f4f04700 1 -- 172.21.15.5:0/4286050692 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f00cef60 con 0x7f34f006fa10 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f5f06700 1 -- 172.21.15.5:0/4286050692 >> v1:172.21.15.49:6789/0 conn(0x7f34f006fa10 legacy=0x7f34f00b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f5f06700 1 -- 172.21.15.5:0/4286050692 >> v1:172.21.15.5:6789/0 conn(0x7f34f015e310 legacy=0x7f34f015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f5f06700 1 -- 172.21.15.5:0/4286050692 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00ce980 con 0x7f34f015dcc0 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f5f06700 1 --2- 172.21.15.5:0/4286050692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 0x7f34f006ef30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009840 tx=0x7f34ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4b88b2a0cc3b17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.091+0000 7f34f4f04700 1 -- 172.21.15.5:0/4286050692 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec011830 con 0x7f34f015dcc0 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f4f04700 1 -- 172.21.15.5:0/4286050692 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec011990 con 0x7f34f015dcc0 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f4f04700 1 -- 172.21.15.5:0/4286050692 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec011d20 con 0x7f34f015dcc0 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f816a700 1 -- 172.21.15.5:0/4286050692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 msgr2=0x7f34f006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f816a700 1 --2- 172.21.15.5:0/4286050692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 0x7f34f006ef30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009840 tx=0x7f34ec009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f816a700 1 -- 172.21.15.5:0/4286050692 shutdown_connections 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f816a700 1 --2- 172.21.15.5:0/4286050692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 0x7f34f006ef30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f816a700 1 -- 172.21.15.5:0/4286050692 >> 172.21.15.5:0/4286050692 conn(0x7f34f0099a50 msgr2=0x7f34f00b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f816a700 1 -- 172.21.15.5:0/4286050692 shutdown_connections 2024-05-08T23:21:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.092+0000 7f34f816a700 1 -- 172.21.15.5:0/4286050692 wait complete. 2024-05-08T23:21:35.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.093+0000 7f34f816a700 1 Processor -- start 2024-05-08T23:21:35.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.094+0000 7f34f816a700 1 -- start start 2024-05-08T23:21:35.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.094+0000 7f34f816a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34f006fa10 0x7f34f006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.094+0000 7f34f816a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 0x7f34f00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.094+0000 7f34f816a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f015e310 0x7f34f00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.094+0000 7f34f816a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f015c2f0 con 0x7f34f015e310 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.094+0000 7f34f816a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f01f6a60 con 0x7f34f015dcc0 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.094+0000 7f34f816a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00b5a80 con 0x7f34f006fa10 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.094+0000 7f34f6707700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f015e310 0x7f34f00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f6707700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f015e310 0x7f34f00b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41946/0 (socket says 172.21.15.5:41946) 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f6707700 1 -- 172.21.15.5:0/12114514 learned_addr learned my addr 172.21.15.5:0/12114514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f5f06700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34f006fa10 0x7f34f006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f5705700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 0x7f34f00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f6707700 1 -- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34f006fa10 msgr2=0x7f34f006abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f6707700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34f006fa10 0x7f34f006abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f6707700 1 -- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 msgr2=0x7f34f00b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f6707700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 0x7f34f00b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f6707700 1 -- 172.21.15.5:0/12114514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0094a0 con 0x7f34f015e310 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f5705700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 0x7f34f00b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f6707700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f015e310 0x7f34f00b53b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f34e800cb10 tx=0x7f34e800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34e67fc700 1 -- 172.21.15.5:0/12114514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800ab00 con 0x7f34f015e310 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34e67fc700 1 -- 172.21.15.5:0/12114514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e800ac60 con 0x7f34f015e310 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f01f6cb0 con 0x7f34f015e310 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34e67fc700 1 -- 172.21.15.5:0/12114514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8012400 con 0x7f34f015e310 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.095+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f01f7270 con 0x7f34f015e310 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.096+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f01f7460 con 0x7f34f015e310 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.097+0000 7f34e67fc700 1 -- 172.21.15.5:0/12114514 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f34e801b3b0 con 0x7f34f015e310 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.097+0000 7f34e67fc700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f34d406ac90 0x7f34d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.098+0000 7f34e67fc700 1 -- 172.21.15.5:0/12114514 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34e8088eb0 con 0x7f34f015e310 2024-05-08T23:21:35.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.098+0000 7f34f5f06700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f34d406ac90 0x7f34d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.098+0000 7f34f5f06700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f34d406ac90 0x7f34d406d150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009fd0 tx=0x7f34ec01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.102+0000 7f34e67fc700 1 -- 172.21.15.5:0/12114514 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f34e80541d0 con 0x7f34f015e310 2024-05-08T23:21:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.320+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34f0068000 con 0x7f34f015e310 2024-05-08T23:21:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.321+0000 7f34e67fc700 1 -- 172.21.15.5:0/12114514 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7f34e80577f0 con 0x7f34f015e310 2024-05-08T23:21:35.321 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:35.321 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.323+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f34d406ac90 msgr2=0x7f34d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.323+0000 7f34f816a700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f34d406ac90 0x7f34d406d150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009fd0 tx=0x7f34ec01c040 comp rx=0 tx=0).stop 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f015e310 msgr2=0x7f34f00b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f015e310 0x7f34f00b53b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f34e800cb10 tx=0x7f34e800ced0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 shutdown_connections 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f34d406ac90 0x7f34d406d150 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f015e310 0x7f34f00b53b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34f006fa10 0x7f34f006abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 --2- 172.21.15.5:0/12114514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34f015dcc0 0x7f34f00b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 >> 172.21.15.5:0/12114514 conn(0x7f34f0099a50 msgr2=0x7f34f00aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 shutdown_connections 2024-05-08T23:21:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.324+0000 7f34f816a700 1 -- 172.21.15.5:0/12114514 wait complete. 2024-05-08T23:21:35.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:35.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:35.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.477+0000 7f2f4dce7700 1 Processor -- start 2024-05-08T23:21:35.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.477+0000 7f2f4dce7700 1 -- start start 2024-05-08T23:21:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.478+0000 7f2f4dce7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f480b1940 0x7f2f480cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.478+0000 7f2f4dce7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480b14b0 con 0x7f2f480c4440 2024-05-08T23:21:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.478+0000 7f2f4dce7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480b2ce0 con 0x7f2f480b03b0 2024-05-08T23:21:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.478+0000 7f2f4dce7700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4806c2f0 con 0x7f2f480b1940 2024-05-08T23:21:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.478+0000 7f2f47fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f480b1940 0x7f2f480cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.478+0000 7f2f47fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f480b1940 0x7f2f480cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41948/0 (socket says 172.21.15.5:41948) 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.478+0000 7f2f47fff700 1 -- 172.21.15.5:0/1017586871 learned_addr learned my addr 172.21.15.5:0/1017586871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.478+0000 7f2f477fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2f480c4440 0x7f2f480c4820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33046/0 (socket says 172.21.15.5:33046) 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.479+0000 7f2f47fff700 1 -- 172.21.15.5:0/1017586871 >> v1:172.21.15.49:6789/0 conn(0x7f2f480b03b0 legacy=0x7f2f480b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.479+0000 7f2f47fff700 1 -- 172.21.15.5:0/1017586871 >> v1:172.21.15.5:6789/0 conn(0x7f2f480c4440 legacy=0x7f2f480c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.479+0000 7f2f47fff700 1 -- 172.21.15.5:0/1017586871 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4806ba70 con 0x7f2f480b1940 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.479+0000 7f2f467fc700 1 -- 172.21.15.5:0/1017586871 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789734246 0 0) 0x7f2f480b14b0 con 0x7f2f480c4440 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.479+0000 7f2f47fff700 1 --2- 172.21.15.5:0/1017586871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480b1940 0x7f2f480cf040 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009e80 tx=0x7f2f4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ccce24a8b3aab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.479+0000 7f2f467fc700 1 -- 172.21.15.5:0/1017586871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000e070 con 0x7f2f480b1940 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.479+0000 7f2f467fc700 1 -- 172.21.15.5:0/1017586871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4000b980 con 0x7f2f480b1940 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.479+0000 7f2f467fc700 1 -- 172.21.15.5:0/1017586871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000bd30 con 0x7f2f480b1940 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.480+0000 7f2f4dce7700 1 -- 172.21.15.5:0/1017586871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480b1940 msgr2=0x7f2f480cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.480+0000 7f2f4dce7700 1 --2- 172.21.15.5:0/1017586871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480b1940 0x7f2f480cf040 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009e80 tx=0x7f2f4000b330 comp rx=0 tx=0).stop 2024-05-08T23:21:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.480+0000 7f2f4dce7700 1 -- 172.21.15.5:0/1017586871 shutdown_connections 2024-05-08T23:21:35.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.480+0000 7f2f4dce7700 1 --2- 172.21.15.5:0/1017586871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480b1940 0x7f2f480cf040 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.480+0000 7f2f4dce7700 1 -- 172.21.15.5:0/1017586871 >> 172.21.15.5:0/1017586871 conn(0x7f2f480b2f60 msgr2=0x7f2f480b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:35.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.480+0000 7f2f4dce7700 1 -- 172.21.15.5:0/1017586871 shutdown_connections 2024-05-08T23:21:35.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.480+0000 7f2f4dce7700 1 -- 172.21.15.5:0/1017586871 wait complete. 2024-05-08T23:21:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.481+0000 7f2f4dce7700 1 Processor -- start 2024-05-08T23:21:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.481+0000 7f2f4dce7700 1 -- start start 2024-05-08T23:21:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f4dce7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f480b03b0 0x7f2f4815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f4dce7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f480b1940 0x7f2f48070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f4dce7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480c4440 0x7f2f480b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f4dce7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f481fb9b0 con 0x7f2f480c4440 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f4dce7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b85c0 con 0x7f2f480b1940 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f4dce7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4815e4c0 con 0x7f2f480b03b0 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480c4440 0x7f2f480b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480c4440 0x7f2f480b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41954/0 (socket says 172.21.15.5:41954) 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f47fff700 1 -- 172.21.15.5:0/2498789069 learned_addr learned my addr 172.21.15.5:0/2498789069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.482+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f480b1940 0x7f2f48070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f477fe700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f480b03b0 0x7f2f4815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f47fff700 1 -- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f480b03b0 msgr2=0x7f2f4815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f47fff700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f480b03b0 0x7f2f4815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f47fff700 1 -- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f480b1940 msgr2=0x7f2f48070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f47fff700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f480b1940 0x7f2f48070450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f47fff700 1 -- 172.21.15.5:0/2498789069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009b30 con 0x7f2f480c4440 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f47fff700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480c4440 0x7f2f480b6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f48072460 tx=0x7f2f40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f2ffff700 1 -- 172.21.15.5:0/2498789069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4001c640 con 0x7f2f480c4440 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f2ffff700 1 -- 172.21.15.5:0/2498789069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4001c7a0 con 0x7f2f480c4440 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f481fae30 con 0x7f2f480c4440 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f2ffff700 1 -- 172.21.15.5:0/2498789069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4001cb50 con 0x7f2f480c4440 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.483+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f481fb3f0 con 0x7f2f480c4440 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.484+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f48074bc0 con 0x7f2f480c4440 2024-05-08T23:21:35.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.485+0000 7f2f2ffff700 1 -- 172.21.15.5:0/2498789069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2f4001ccb0 con 0x7f2f480c4440 2024-05-08T23:21:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.485+0000 7f2f2ffff700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3406ac90 0x7f2f3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.485+0000 7f2f2ffff700 1 -- 172.21.15.5:0/2498789069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f4008ada0 con 0x7f2f480c4440 2024-05-08T23:21:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.485+0000 7f2f477fe700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3406ac90 0x7f2f3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.486+0000 7f2f477fe700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3406ac90 0x7f2f3406d150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2f38000f60 tx=0x7f2f38007d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.490+0000 7f2f2ffff700 1 -- 172.21.15.5:0/2498789069 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2f40055f90 con 0x7f2f480c4440 2024-05-08T23:21:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.706+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f480c8030 con 0x7f2f480c4440 2024-05-08T23:21:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.707+0000 7f2f2ffff700 1 -- 172.21.15.5:0/2498789069 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f400595b0 con 0x7f2f480c4440 2024-05-08T23:21:35.707 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3406ac90 msgr2=0x7f2f3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3406ac90 0x7f2f3406d150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2f38000f60 tx=0x7f2f38007d90 comp rx=0 tx=0).stop 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480c4440 msgr2=0x7f2f480b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480c4440 0x7f2f480b6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f48072460 tx=0x7f2f40004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 shutdown_connections 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3406ac90 0x7f2f3406d150 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f480c4440 0x7f2f480b6260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f480b03b0 0x7f2f4815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 --2- 172.21.15.5:0/2498789069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f480b1940 0x7f2f48070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 >> 172.21.15.5:0/2498789069 conn(0x7f2f480b2f60 msgr2=0x7f2f481607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 shutdown_connections 2024-05-08T23:21:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.709+0000 7f2f4dce7700 1 -- 172.21.15.5:0/2498789069 wait complete. 2024-05-08T23:21:35.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.864+0000 7fbc02cd8700 1 Processor -- start 2024-05-08T23:21:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.864+0000 7fbc02cd8700 1 -- start start 2024-05-08T23:21:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc02cd8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbbfc0cc320 0x7fbbfc15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc02cd8700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc0b2380 con 0x7fbbfc0ce950 2024-05-08T23:21:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc02cd8700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc0b2660 con 0x7fbbfc159be0 2024-05-08T23:21:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc02cd8700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbfc0b3140 con 0x7fbbfc0cc320 2024-05-08T23:21:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc01275700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbbfc0cc320 0x7fbbfc15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbbfbfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbbfc0ce950 0x7fbbfc15a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33052/0 (socket says 172.21.15.5:33052) 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbbfbfff700 1 -- 172.21.15.5:0/4283865849 learned_addr learned my addr 172.21.15.5:0/4283865849 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc01275700 1 -- 172.21.15.5:0/4283865849 >> v1:172.21.15.49:6789/0 conn(0x7fbbfc159be0 legacy=0x7fbbfc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc01275700 1 -- 172.21.15.5:0/4283865849 >> v1:172.21.15.5:6789/0 conn(0x7fbbfc0ce950 legacy=0x7fbbfc15a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc01275700 1 -- 172.21.15.5:0/4283865849 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0b2940 con 0x7fbbfc0cc320 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbbfb7fe700 1 -- 172.21.15.5:0/4283865849 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064537377 0 0) 0x7fbbfc0b2380 con 0x7fbbfc0ce950 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbc01275700 1 --2- 172.21.15.5:0/4283865849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0cc320 0x7fbbfc15ac20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400aea0 tx=0x7fbbf400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4805cec14a7bd743 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbbfb7fe700 1 -- 172.21.15.5:0/4283865849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400f070 con 0x7fbbfc0cc320 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbbfb7fe700 1 -- 172.21.15.5:0/4283865849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf400c9c0 con 0x7fbbfc0cc320 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.865+0000 7fbbfb7fe700 1 -- 172.21.15.5:0/4283865849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400cd70 con 0x7fbbfc0cc320 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.866+0000 7fbc02cd8700 1 -- 172.21.15.5:0/4283865849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0cc320 msgr2=0x7fbbfc15ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.866+0000 7fbc02cd8700 1 --2- 172.21.15.5:0/4283865849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0cc320 0x7fbbfc15ac20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400aea0 tx=0x7fbbf400c330 comp rx=0 tx=0).stop 2024-05-08T23:21:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.866+0000 7fbc02cd8700 1 -- 172.21.15.5:0/4283865849 shutdown_connections 2024-05-08T23:21:35.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.866+0000 7fbc02cd8700 1 --2- 172.21.15.5:0/4283865849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0cc320 0x7fbbfc15ac20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.866+0000 7fbc02cd8700 1 -- 172.21.15.5:0/4283865849 >> 172.21.15.5:0/4283865849 conn(0x7fbbfc06a960 msgr2=0x7fbbfc06ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:35.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.866+0000 7fbc02cd8700 1 -- 172.21.15.5:0/4283865849 shutdown_connections 2024-05-08T23:21:35.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.866+0000 7fbc02cd8700 1 -- 172.21.15.5:0/4283865849 wait complete. 2024-05-08T23:21:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.867+0000 7fbc02cd8700 1 Processor -- start 2024-05-08T23:21:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.868+0000 7fbc02cd8700 1 -- start start 2024-05-08T23:21:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.868+0000 7fbc02cd8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbfc0cc320 0x7fbbfc14e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.868+0000 7fbc02cd8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0ce950 0x7fbbfc1e4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbc02cd8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbbfc159be0 0x7fbbfc156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbc02cd8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc155ae0 con 0x7fbbfc0ce950 2024-05-08T23:21:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbc02cd8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc14e660 con 0x7fbbfc0cc320 2024-05-08T23:21:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbc02cd8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc155d80 con 0x7fbbfc159be0 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0ce950 0x7fbbfc1e4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0ce950 0x7fbbfc1e4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41982/0 (socket says 172.21.15.5:41982) 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 -- 172.21.15.5:0/3880779113 learned_addr learned my addr 172.21.15.5:0/3880779113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbc01275700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbbfc159be0 0x7fbbfc156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbc00a74700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbfc0cc320 0x7fbbfc14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 -- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbbfc159be0 msgr2=0x7fbbfc156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbbfc159be0 0x7fbbfc156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 -- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbfc0cc320 msgr2=0x7fbbfc14e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbfc0cc320 0x7fbbfc14e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 -- 172.21.15.5:0/3880779113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf400ab80 con 0x7fbbfc0ce950 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbfbfff700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0ce950 0x7fbbfc1e4570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400c550 tx=0x7fbbe400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbf97fa700 1 -- 172.21.15.5:0/3880779113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4004c80 con 0x7fbbfc0ce950 2024-05-08T23:21:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbf97fa700 1 -- 172.21.15.5:0/3880779113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe4004e20 con 0x7fbbfc0ce950 2024-05-08T23:21:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.869+0000 7fbbf97fa700 1 -- 172.21.15.5:0/3880779113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4007310 con 0x7fbbfc0ce950 2024-05-08T23:21:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.870+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc156060 con 0x7fbbfc0ce950 2024-05-08T23:21:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.870+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc156620 con 0x7fbbfc0ce950 2024-05-08T23:21:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.871+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc156810 con 0x7fbbfc0ce950 2024-05-08T23:21:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.871+0000 7fbbf97fa700 1 -- 172.21.15.5:0/3880779113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbbe4007470 con 0x7fbbfc0ce950 2024-05-08T23:21:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.871+0000 7fbbf97fa700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbbdc06ac90 0x7fbbdc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.872+0000 7fbbf97fa700 1 -- 172.21.15.5:0/3880779113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbe4089040 con 0x7fbbfc0ce950 2024-05-08T23:21:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.872+0000 7fbc00a74700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbbdc06ac90 0x7fbbdc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.872+0000 7fbc00a74700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbbdc06ac90 0x7fbbdc06d150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbbec007fd0 tx=0x7fbbec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:35.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:35.876+0000 7fbbf97fa700 1 -- 172.21.15.5:0/3880779113 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbbe4054190 con 0x7fbbfc0ce950 2024-05-08T23:21:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.096+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbfc074bc0 con 0x7fbbfc0ce950 2024-05-08T23:21:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.096+0000 7fbbf97fa700 1 -- 172.21.15.5:0/3880779113 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbbe40577b0 con 0x7fbbfc0ce950 2024-05-08T23:21:36.097 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbbdc06ac90 msgr2=0x7fbbdc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbbdc06ac90 0x7fbbdc06d150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbbec007fd0 tx=0x7fbbec009040 comp rx=0 tx=0).stop 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0ce950 msgr2=0x7fbbfc1e4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0ce950 0x7fbbfc1e4570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400c550 tx=0x7fbbe400c860 comp rx=0 tx=0).stop 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 shutdown_connections 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbbdc06ac90 0x7fbbdc06d150 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbbfc159be0 0x7fbbfc156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbfc0cc320 0x7fbbfc14e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 --2- 172.21.15.5:0/3880779113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbfc0ce950 0x7fbbfc1e4570 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 >> 172.21.15.5:0/3880779113 conn(0x7fbbfc06a960 msgr2=0x7fbbfc0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 shutdown_connections 2024-05-08T23:21:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.099+0000 7fbc02cd8700 1 -- 172.21.15.5:0/3880779113 wait complete. 2024-05-08T23:21:36.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.254+0000 7f5a74e7c700 1 Processor -- start 2024-05-08T23:21:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.254+0000 7f5a74e7c700 1 -- start start 2024-05-08T23:21:36.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a74e7c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5a70161ff0 0x7f5a7006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a74e7c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5a7006beb0 0x7f5a700ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a74e7c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7006c2e0 con 0x7f5a7015d900 2024-05-08T23:21:36.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a74e7c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7015dce0 con 0x7f5a7006beb0 2024-05-08T23:21:36.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a74e7c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a700ce5b0 con 0x7f5a70161ff0 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6ed9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5a7006beb0 0x7f5a700ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6e59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5a7015d900 0x7f5a700cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33060/0 (socket says 172.21.15.5:33060) 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6e59c700 1 -- 172.21.15.5:0/3117193317 learned_addr learned my addr 172.21.15.5:0/3117193317 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6dd9b700 1 --2- 172.21.15.5:0/3117193317 >> v2:172.21.15.49:3300/0 conn(0x7f5a70161ff0 0x7f5a7006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6d59a700 1 -- 172.21.15.5:0/3117193317 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381229700 0 0) 0x7f5a7006c2e0 con 0x7f5a7015d900 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6d59a700 1 -- 172.21.15.5:0/3117193317 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a580035b0 con 0x7f5a7015d900 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6ed9d700 1 -- 172.21.15.5:0/3117193317 >> v2:172.21.15.49:3300/0 conn(0x7f5a70161ff0 msgr2=0x7f5a7006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6ed9d700 1 --2- 172.21.15.5:0/3117193317 >> v2:172.21.15.49:3300/0 conn(0x7f5a70161ff0 0x7f5a7006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6ed9d700 1 -- 172.21.15.5:0/3117193317 >> v1:172.21.15.5:6789/0 conn(0x7f5a7015d900 legacy=0x7f5a700cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.255+0000 7f5a6ed9d700 1 -- 172.21.15.5:0/3117193317 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a70161510 con 0x7f5a7006beb0 2024-05-08T23:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a6ed9d700 1 --2- 172.21.15.5:0/3117193317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7006beb0 0x7f5a700ada00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a7006eaa0 tx=0x7f5a6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e811fd3949cb1da8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a6d59a700 1 -- 172.21.15.5:0/3117193317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800e070 con 0x7f5a7006beb0 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a6d59a700 1 -- 172.21.15.5:0/3117193317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6800bb20 con 0x7f5a7006beb0 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a6d59a700 1 -- 172.21.15.5:0/3117193317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68012430 con 0x7f5a7006beb0 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a74e7c700 1 -- 172.21.15.5:0/3117193317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7006beb0 msgr2=0x7f5a700ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/3117193317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7006beb0 0x7f5a700ada00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a7006eaa0 tx=0x7f5a6800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a74e7c700 1 -- 172.21.15.5:0/3117193317 shutdown_connections 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/3117193317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7006beb0 0x7f5a700ada00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/3117193317 >> v2:172.21.15.49:3300/0 conn(0x7f5a70161ff0 0x7f5a7006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.256+0000 7f5a74e7c700 1 -- 172.21.15.5:0/3117193317 >> 172.21.15.5:0/3117193317 conn(0x7f5a700713c0 msgr2=0x7f5a700717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.257+0000 7f5a74e7c700 1 -- 172.21.15.5:0/3117193317 shutdown_connections 2024-05-08T23:21:36.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.257+0000 7f5a74e7c700 1 -- 172.21.15.5:0/3117193317 wait complete. 2024-05-08T23:21:36.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.258+0000 7f5a74e7c700 1 Processor -- start 2024-05-08T23:21:36.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.258+0000 7f5a74e7c700 1 -- start start 2024-05-08T23:21:36.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a74e7c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a7006beb0 0x7f5a700afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a74e7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7015d900 0x7f5a700af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a74e7c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a70161ff0 0x7f5a7014feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a74e7c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700afa10 con 0x7f5a7015d900 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a74e7c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700ad9c0 con 0x7f5a70161ff0 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a74e7c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a701503f0 con 0x7f5a7006beb0 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a6dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7015d900 0x7f5a700af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a6dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7015d900 0x7f5a700af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42006/0 (socket says 172.21.15.5:42006) 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a6dd9b700 1 -- 172.21.15.5:0/648345749 learned_addr learned my addr 172.21.15.5:0/648345749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a6ed9d700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a70161ff0 0x7f5a7014feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a6e59c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a7006beb0 0x7f5a700afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a6dd9b700 1 -- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a7006beb0 msgr2=0x7f5a700afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.259+0000 7f5a6dd9b700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a7006beb0 0x7f5a700afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a6dd9b700 1 -- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a70161ff0 msgr2=0x7f5a7014feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a6dd9b700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a70161ff0 0x7f5a7014feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a6dd9b700 1 -- 172.21.15.5:0/648345749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68009a70 con 0x7f5a7015d900 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a6dd9b700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7015d900 0x7f5a700af4d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400d830 tx=0x7f5a6400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a5effd700 1 -- 172.21.15.5:0/648345749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a640125f0 con 0x7f5a7015d900 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a5effd700 1 -- 172.21.15.5:0/648345749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a64012750 con 0x7f5a7015d900 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a700b0460 con 0x7f5a7015d900 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a5effd700 1 -- 172.21.15.5:0/648345749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64012b00 con 0x7f5a7015d900 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.260+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a700adc10 con 0x7f5a7015d900 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.261+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a700b5a40 con 0x7f5a7015d900 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.261+0000 7f5a5effd700 1 -- 172.21.15.5:0/648345749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5a64012c60 con 0x7f5a7015d900 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.262+0000 7f5a5effd700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a5006ac90 0x7f5a5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.262+0000 7f5a5effd700 1 -- 172.21.15.5:0/648345749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a640892b0 con 0x7f5a7015d900 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.262+0000 7f5a6e59c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a5006ac90 0x7f5a5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.263+0000 7f5a6e59c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a5006ac90 0x7f5a5006d150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a60007fd0 tx=0x7f5a60007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.267+0000 7f5a5effd700 1 -- 172.21.15.5:0/648345749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5a64054520 con 0x7f5a7015d900 2024-05-08T23:21:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.490+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a70074bc0 con 0x7f5a7015d900 2024-05-08T23:21:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.490+0000 7f5a5effd700 1 -- 172.21.15.5:0/648345749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a64057b40 con 0x7f5a7015d900 2024-05-08T23:21:36.491 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:36.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a5006ac90 msgr2=0x7f5a5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a5006ac90 0x7f5a5006d150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a60007fd0 tx=0x7f5a60007dc0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7015d900 msgr2=0x7f5a700af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7015d900 0x7f5a700af4d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400d830 tx=0x7f5a6400db40 comp rx=0 tx=0).stop 2024-05-08T23:21:36.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 shutdown_connections 2024-05-08T23:21:36.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a5006ac90 0x7f5a5006d150 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a70161ff0 0x7f5a7014feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a7006beb0 0x7f5a700afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 --2- 172.21.15.5:0/648345749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a7015d900 0x7f5a700af4d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 >> 172.21.15.5:0/648345749 conn(0x7f5a700713c0 msgr2=0x7f5a700b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 shutdown_connections 2024-05-08T23:21:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.493+0000 7f5a74e7c700 1 -- 172.21.15.5:0/648345749 wait complete. 2024-05-08T23:21:36.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.642+0000 7efdb9dfb700 1 Processor -- start 2024-05-08T23:21:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.643+0000 7efdb9dfb700 1 -- start start 2024-05-08T23:21:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.643+0000 7efdb9dfb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efdb415d7a0 0x7efdb4161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.643+0000 7efdb9dfb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efdb40d3f40 0x7efdb40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.643+0000 7efdb9dfb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415db80 0x7efdb40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.643+0000 7efdb9dfb700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb4162450 con 0x7efdb40d3f40 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.643+0000 7efdb9dfb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb40cee00 con 0x7efdb415d7a0 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.643+0000 7efdb9dfb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40b27e0 con 0x7efdb415db80 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efdb40d3f40 0x7efdb40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efdb40d3f40 0x7efdb40b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42014/0 (socket says 172.21.15.5:42014) 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 -- 172.21.15.5:0/1318282437 learned_addr learned my addr 172.21.15.5:0/1318282437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb3fff700 1 --2- 172.21.15.5:0/1318282437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415db80 0x7efdb40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb37fe700 1 --2- 172.21.15.5:0/1318282437 >> v2:172.21.15.49:3300/0 conn(0x7efdb415d7a0 0x7efdb4161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 -- 172.21.15.5:0/1318282437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415db80 msgr2=0x7efdb40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 --2- 172.21.15.5:0/1318282437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415db80 0x7efdb40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 -- 172.21.15.5:0/1318282437 >> v2:172.21.15.49:3300/0 conn(0x7efdb415d7a0 msgr2=0x7efdb4161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 --2- 172.21.15.5:0/1318282437 >> v2:172.21.15.49:3300/0 conn(0x7efdb415d7a0 0x7efdb4161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 -- 172.21.15.5:0/1318282437 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40d4400 con 0x7efdb40d3f40 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb37fe700 1 --2- 172.21.15.5:0/1318282437 >> v2:172.21.15.49:3300/0 conn(0x7efdb415d7a0 0x7efdb4161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb3fff700 1 --2- 172.21.15.5:0/1318282437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415db80 0x7efdb40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb2ffd700 1 --2- 172.21.15.5:0/1318282437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40b0190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdb40b61f0 tx=0x7efda8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7542263cbee8a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb27fc700 1 -- 172.21.15.5:0/1318282437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8005e30 con 0x7efdb40d3f40 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb27fc700 1 -- 172.21.15.5:0/1318282437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8004030 con 0x7efdb40d3f40 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.644+0000 7efdb27fc700 1 -- 172.21.15.5:0/1318282437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda80043e0 con 0x7efdb40d3f40 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 -- 172.21.15.5:0/1318282437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 msgr2=0x7efdb40b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/1318282437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40b0190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdb40b61f0 tx=0x7efda8009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 -- 172.21.15.5:0/1318282437 shutdown_connections 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/1318282437 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415db80 0x7efdb40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/1318282437 >> v2:172.21.15.49:3300/0 conn(0x7efdb415d7a0 0x7efdb4161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/1318282437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40b0190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 -- 172.21.15.5:0/1318282437 >> 172.21.15.5:0/1318282437 conn(0x7efdb408afe0 msgr2=0x7efdb40d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 -- 172.21.15.5:0/1318282437 shutdown_connections 2024-05-08T23:21:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.645+0000 7efdb9dfb700 1 -- 172.21.15.5:0/1318282437 wait complete. 2024-05-08T23:21:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.646+0000 7efdb9dfb700 1 Processor -- start 2024-05-08T23:21:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.647+0000 7efdb9dfb700 1 -- start start 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.647+0000 7efdb9dfb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.647+0000 7efdb9dfb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415d7a0 0x7efdb406cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.647+0000 7efdb9dfb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdb415db80 0x7efdb406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.647+0000 7efdb9dfb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb414a660 con 0x7efdb40d3f40 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.647+0000 7efdb9dfb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4149d00 con 0x7efdb415db80 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.647+0000 7efdb9dfb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4149fa0 con 0x7efdb415d7a0 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42026/0 (socket says 172.21.15.5:42026) 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 -- 172.21.15.5:0/615517411 learned_addr learned my addr 172.21.15.5:0/615517411 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb3fff700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdb415db80 0x7efdb406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb2ffd700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415d7a0 0x7efdb406cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 -- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415d7a0 msgr2=0x7efdb406cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415d7a0 0x7efdb406cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 -- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdb415db80 msgr2=0x7efdb406d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdb415db80 0x7efdb406d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 -- 172.21.15.5:0/615517411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda40094a0 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb37fe700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40ca6f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efda400ebe0 tx=0x7efda4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efd9bfff700 1 -- 172.21.15.5:0/615517411 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda400c910 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efd9bfff700 1 -- 172.21.15.5:0/615517411 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda400ca70 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac009a70 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efd9bfff700 1 -- 172.21.15.5:0/615517411 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda400cde0 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.648+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda8009500 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.650+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb414a7a0 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.650+0000 7efd9bfff700 1 -- 172.21.15.5:0/615517411 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efda4013470 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.651+0000 7efd9bfff700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efda006ac90 0x7efda006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.651+0000 7efd9bfff700 1 -- 172.21.15.5:0/615517411 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efda4089470 con 0x7efdb40d3f40 2024-05-08T23:21:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.651+0000 7efdb2ffd700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efda006ac90 0x7efda006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.651+0000 7efdb2ffd700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efda006ac90 0x7efda006d150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efda8009c10 tx=0x7efda8008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.655+0000 7efd9bfff700 1 -- 172.21.15.5:0/615517411 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efdb414a7a0 con 0x7efdb40d3f40 2024-05-08T23:21:36.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.874+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdb414a7a0 con 0x7efdb40d3f40 2024-05-08T23:21:36.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.875+0000 7efd9bfff700 1 -- 172.21.15.5:0/615517411 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7efdb414a7a0 con 0x7efdb40d3f40 2024-05-08T23:21:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:36.875 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efda006ac90 msgr2=0x7efda006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efda006ac90 0x7efda006d150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efda8009c10 tx=0x7efda8008070 comp rx=0 tx=0).stop 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 msgr2=0x7efdb40ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40ca6f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efda400ebe0 tx=0x7efda4009ec0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 shutdown_connections 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efda006ac90 0x7efda006d150 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdb415db80 0x7efdb406d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40d3f40 0x7efdb40ca6f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 --2- 172.21.15.5:0/615517411 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efdb415d7a0 0x7efdb406cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.877+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 >> 172.21.15.5:0/615517411 conn(0x7efdb408afe0 msgr2=0x7efdb40b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:36.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.878+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 shutdown_connections 2024-05-08T23:21:36.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:36.878+0000 7efdb9dfb700 1 -- 172.21.15.5:0/615517411 wait complete. 2024-05-08T23:21:36.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:36.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:37.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.030+0000 7fe5c60d1700 1 Processor -- start 2024-05-08T23:21:37.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.031+0000 7fe5c60d1700 1 -- start start 2024-05-08T23:21:37.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.031+0000 7fe5c60d1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5c00d0a10 0x7fe5c00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.031+0000 7fe5c60d1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c00cc380 0x7fe5c00cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.031+0000 7fe5c60d1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c009ad50 con 0x7fe5c0099a50 2024-05-08T23:21:37.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.031+0000 7fe5c60d1700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c006c060 con 0x7fe5c00d0a10 2024-05-08T23:21:37.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.031+0000 7fe5c60d1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c009a790 con 0x7fe5c00cc380 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5beffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5c00d0a10 0x7fe5c00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5beffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5c00d0a10 0x7fe5c00d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42028/0 (socket says 172.21.15.5:42028) 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5beffd700 1 -- 172.21.15.5:0/3853462332 learned_addr learned my addr 172.21.15.5:0/3853462332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5bf7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe5c0099a50 0x7fe5c0099e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33064/0 (socket says 172.21.15.5:33064) 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5bffff700 1 --2- 172.21.15.5:0/3853462332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c00cc380 0x7fe5c00cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5beffd700 1 -- 172.21.15.5:0/3853462332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c00cc380 msgr2=0x7fe5c00cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5beffd700 1 --2- 172.21.15.5:0/3853462332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c00cc380 0x7fe5c00cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5beffd700 1 -- 172.21.15.5:0/3853462332 >> v1:172.21.15.5:6789/0 conn(0x7fe5c0099a50 legacy=0x7fe5c0099e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5beffd700 1 -- 172.21.15.5:0/3853462332 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c009aa70 con 0x7fe5c00d0a10 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5be7fc700 1 -- 172.21.15.5:0/3853462332 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1596047835 0 0) 0x7fe5c009ad50 con 0x7fe5c0099a50 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5beffd700 1 --2- 172.21.15.5:0/3853462332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c00d03c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80098f0 tx=0x7fe5a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcd70fe3e37527a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5be7fc700 1 -- 172.21.15.5:0/3853462332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800b8a0 con 0x7fe5c00d0a10 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5be7fc700 1 -- 172.21.15.5:0/3853462332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a800ba00 con 0x7fe5c00d0a10 2024-05-08T23:21:37.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.032+0000 7fe5be7fc700 1 -- 172.21.15.5:0/3853462332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800bd90 con 0x7fe5c00d0a10 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.033+0000 7fe5c60d1700 1 -- 172.21.15.5:0/3853462332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 msgr2=0x7fe5c00d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.033+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/3853462332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c00d03c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80098f0 tx=0x7fe5a8009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.033+0000 7fe5c60d1700 1 -- 172.21.15.5:0/3853462332 shutdown_connections 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.033+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/3853462332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c00cc380 0x7fe5c00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.033+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/3853462332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c00d03c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.033+0000 7fe5c60d1700 1 -- 172.21.15.5:0/3853462332 >> 172.21.15.5:0/3853462332 conn(0x7fe5c00b2f30 msgr2=0x7fe5c00b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.033+0000 7fe5c60d1700 1 -- 172.21.15.5:0/3853462332 shutdown_connections 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.033+0000 7fe5c60d1700 1 -- 172.21.15.5:0/3853462332 wait complete. 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.034+0000 7fe5c60d1700 1 Processor -- start 2024-05-08T23:21:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.035+0000 7fe5c60d1700 1 -- start start 2024-05-08T23:21:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.035+0000 7fe5c60d1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c0099a50 0x7fe5c00b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.035+0000 7fe5c60d1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5c00cc380 0x7fe5c00b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.035+0000 7fe5c60d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.035+0000 7fe5c60d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c01f37b0 con 0x7fe5c00d0a10 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.035+0000 7fe5c60d1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c015f570 con 0x7fe5c00cc380 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.035+0000 7fe5c60d1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c015f850 con 0x7fe5c0099a50 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c015c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42032/0 (socket says 172.21.15.5:42032) 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 -- 172.21.15.5:0/1964652930 learned_addr learned my addr 172.21.15.5:0/1964652930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bf7fe700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c0099a50 0x7fe5c00b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5beffd700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5c00cc380 0x7fe5c00b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 -- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c0099a50 msgr2=0x7fe5c00b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c0099a50 0x7fe5c00b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 -- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5c00cc380 msgr2=0x7fe5c00b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5c00cc380 0x7fe5c00b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 -- 172.21.15.5:0/1964652930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a8009550 con 0x7fe5c00d0a10 2024-05-08T23:21:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bf7fe700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c0099a50 0x7fe5c00b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5beffd700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5c00cc380 0x7fe5c00b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5bffff700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c015c110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8007e40 tx=0x7fe5b800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5b7fff700 1 -- 172.21.15.5:0/1964652930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800eea0 con 0x7fe5c00d0a10 2024-05-08T23:21:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5b7fff700 1 -- 172.21.15.5:0/1964652930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8014420 con 0x7fe5c00d0a10 2024-05-08T23:21:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5b7fff700 1 -- 172.21.15.5:0/1964652930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80147b0 con 0x7fe5c00d0a10 2024-05-08T23:21:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c015fb30 con 0x7fe5c00d0a10 2024-05-08T23:21:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.036+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c015f220 con 0x7fe5c00d0a10 2024-05-08T23:21:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.037+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c0160050 con 0x7fe5c00d0a10 2024-05-08T23:21:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.038+0000 7fe5b7fff700 1 -- 172.21.15.5:0/1964652930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe5b8014910 con 0x7fe5c00d0a10 2024-05-08T23:21:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.038+0000 7fe5b7fff700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe5a006ac90 0x7fe5a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.038+0000 7fe5b7fff700 1 -- 172.21.15.5:0/1964652930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5b8089ac0 con 0x7fe5c00d0a10 2024-05-08T23:21:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.039+0000 7fe5bf7fe700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe5a006ac90 0x7fe5a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.039+0000 7fe5bf7fe700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe5a006ac90 0x7fe5a006d150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0000f60 tx=0x7fe5b0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.043+0000 7fe5b7fff700 1 -- 172.21.15.5:0/1964652930 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe5b8054d30 con 0x7fe5c00d0a10 2024-05-08T23:21:37.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.260+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5c0074bc0 con 0x7fe5c00d0a10 2024-05-08T23:21:37.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.261+0000 7fe5b7fff700 1 -- 172.21.15.5:0/1964652930 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5b8089e60 con 0x7fe5c00d0a10 2024-05-08T23:21:37.261 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe5a006ac90 msgr2=0x7fe5a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe5a006ac90 0x7fe5a006d150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0000f60 tx=0x7fe5b0007d30 comp rx=0 tx=0).stop 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 msgr2=0x7fe5c015c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c015c110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8007e40 tx=0x7fe5b800e3c0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 shutdown_connections 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe5a006ac90 0x7fe5a006d150 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5c00d0a10 0x7fe5c015c110 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe5c0099a50 0x7fe5c00b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 --2- 172.21.15.5:0/1964652930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5c00cc380 0x7fe5c00b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 >> 172.21.15.5:0/1964652930 conn(0x7fe5c00b2f30 msgr2=0x7fe5c00b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 shutdown_connections 2024-05-08T23:21:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.263+0000 7fe5c60d1700 1 -- 172.21.15.5:0/1964652930 wait complete. 2024-05-08T23:21:37.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:21:37.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:37.277 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:21:37.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.458+0000 7f8fbb574700 1 Processor -- start 2024-05-08T23:21:37.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.458+0000 7f8fbb574700 1 -- start start 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.459+0000 7f8fbb574700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8fb4145040 0x7f8fb40b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.459+0000 7f8fbb574700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8fb415fbb0 0x7f8fb40d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.459+0000 7f8fbb574700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb40af6e0 con 0x7f8fb40d0890 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.459+0000 7f8fbb574700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb40d0d20 con 0x7f8fb4145040 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.459+0000 7f8fbb574700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb40b6460 con 0x7f8fb415fbb0 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.459+0000 7f8fb9310700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8fb4145040 0x7f8fb40b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb9b11700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8fb40d0890 0x7f8fb406a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33644/0 (socket says 172.21.15.5:33644) 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb9310700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8fb4145040 0x7f8fb40b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41404/0 (socket says 172.21.15.5:41404) 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb9b11700 1 -- 172.21.15.5:0/2978454075 learned_addr learned my addr 172.21.15.5:0/2978454075 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb8b0f700 1 --2- 172.21.15.5:0/2978454075 >> v2:172.21.15.49:3300/0 conn(0x7f8fb415fbb0 0x7f8fb40d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb9310700 1 -- 172.21.15.5:0/2978454075 >> v2:172.21.15.49:3300/0 conn(0x7f8fb415fbb0 msgr2=0x7f8fb40d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb9310700 1 --2- 172.21.15.5:0/2978454075 >> v2:172.21.15.49:3300/0 conn(0x7f8fb415fbb0 0x7f8fb40d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb9310700 1 -- 172.21.15.5:0/2978454075 >> v1:172.21.15.5:6789/0 conn(0x7f8fb40d0890 legacy=0x7f8fb406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb9310700 1 -- 172.21.15.5:0/2978454075 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40aeb50 con 0x7f8fb4145040 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fabfff700 1 -- 172.21.15.5:0/2978454075 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263373243 0 0) 0x7f8fb40af6e0 con 0x7f8fb40d0890 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fb9310700 1 --2- 172.21.15.5:0/2978454075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb4145040 0x7f8fb40b5f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40b3000 tx=0x7f8fb0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8028b805e22931e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fabfff700 1 -- 172.21.15.5:0/2978454075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0011830 con 0x7f8fb4145040 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fabfff700 1 -- 172.21.15.5:0/2978454075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0011990 con 0x7f8fb4145040 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.460+0000 7f8fabfff700 1 -- 172.21.15.5:0/2978454075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0011d20 con 0x7f8fb4145040 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.461+0000 7f8fbb574700 1 -- 172.21.15.5:0/2978454075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb4145040 msgr2=0x7f8fb40b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.461+0000 7f8fbb574700 1 --2- 172.21.15.5:0/2978454075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb4145040 0x7f8fb40b5f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40b3000 tx=0x7f8fb0009b20 comp rx=0 tx=0).stop 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.461+0000 7f8fbb574700 1 -- 172.21.15.5:0/2978454075 shutdown_connections 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.461+0000 7f8fbb574700 1 --2- 172.21.15.5:0/2978454075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb4145040 0x7f8fb40b5f20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.461+0000 7f8fbb574700 1 --2- 172.21.15.5:0/2978454075 >> v2:172.21.15.49:3300/0 conn(0x7f8fb415fbb0 0x7f8fb40d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.461+0000 7f8fbb574700 1 -- 172.21.15.5:0/2978454075 >> 172.21.15.5:0/2978454075 conn(0x7f8fb4099a50 msgr2=0x7f8fb40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:37.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.461+0000 7f8fbb574700 1 -- 172.21.15.5:0/2978454075 shutdown_connections 2024-05-08T23:21:37.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.461+0000 7f8fbb574700 1 -- 172.21.15.5:0/2978454075 wait complete. 2024-05-08T23:21:37.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.462+0000 7f8fbb574700 1 Processor -- start 2024-05-08T23:21:37.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.462+0000 7f8fbb574700 1 -- start start 2024-05-08T23:21:37.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fbb574700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40d0890 0x7f8fb415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fbb574700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8fb4145040 0x7f8fb414d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fbb574700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fb415fbb0 0x7f8fb415bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fbb574700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb415ead0 con 0x7f8fb40d0890 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fbb574700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb415c4c0 con 0x7f8fb415fbb0 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fbb574700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb415c7a0 con 0x7f8fb4145040 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fb9310700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40d0890 0x7f8fb415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fb9310700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40d0890 0x7f8fb415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41414/0 (socket says 172.21.15.5:41414) 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.463+0000 7f8fb9310700 1 -- 172.21.15.5:0/1394586254 learned_addr learned my addr 172.21.15.5:0/1394586254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fb9b11700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fb415fbb0 0x7f8fb415bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fb8b0f700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8fb4145040 0x7f8fb414d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fb9310700 1 -- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8fb4145040 msgr2=0x7f8fb414d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fb9310700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8fb4145040 0x7f8fb414d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fb9310700 1 -- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fb415fbb0 msgr2=0x7f8fb415bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fb9310700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fb415fbb0 0x7f8fb415bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fb9310700 1 -- 172.21.15.5:0/1394586254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb00094a0 con 0x7f8fb40d0890 2024-05-08T23:21:37.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fb9310700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40d0890 0x7f8fb415e590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40b5790 tx=0x7f8fb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fa9ffb700 1 -- 172.21.15.5:0/1394586254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00038a0 con 0x7f8fb40d0890 2024-05-08T23:21:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fa9ffb700 1 -- 172.21.15.5:0/1394586254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb001f070 con 0x7f8fb40d0890 2024-05-08T23:21:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb414ce20 con 0x7f8fb40d0890 2024-05-08T23:21:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fa9ffb700 1 -- 172.21.15.5:0/1394586254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0003c40 con 0x7f8fb40d0890 2024-05-08T23:21:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.464+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb414db60 con 0x7f8fb40d0890 2024-05-08T23:21:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.465+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb4067ae0 con 0x7f8fb40d0890 2024-05-08T23:21:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.466+0000 7f8fa9ffb700 1 -- 172.21.15.5:0/1394586254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8fb0026480 con 0x7f8fb40d0890 2024-05-08T23:21:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.466+0000 7f8fa9ffb700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f9806ac90 0x7f8f9806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.466+0000 7f8fa9ffb700 1 -- 172.21.15.5:0/1394586254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fb0021070 con 0x7f8fb40d0890 2024-05-08T23:21:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.467+0000 7f8fb8b0f700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f9806ac90 0x7f8f9806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.467+0000 7f8fb8b0f700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f9806ac90 0x7f8f9806d150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8fb414aae0 tx=0x7f8fa4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.471+0000 7f8fa9ffb700 1 -- 172.21.15.5:0/1394586254 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8fb0055f50 con 0x7f8fb40d0890 2024-05-08T23:21:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.686+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fb415c9b0 con 0x7f8fb40d0890 2024-05-08T23:21:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.687+0000 7f8fa9ffb700 1 -- 172.21.15.5:0/1394586254 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8fb0059570 con 0x7f8fb40d0890 2024-05-08T23:21:37.687 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f9806ac90 msgr2=0x7f8f9806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f9806ac90 0x7f8f9806d150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8fb414aae0 tx=0x7f8fa4008040 comp rx=0 tx=0).stop 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40d0890 msgr2=0x7f8fb415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40d0890 0x7f8fb415e590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40b5790 tx=0x7f8fb0004750 comp rx=0 tx=0).stop 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 shutdown_connections 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f9806ac90 0x7f8f9806d150 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fb415fbb0 0x7f8fb415bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40d0890 0x7f8fb415e590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 --2- 172.21.15.5:0/1394586254 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8fb4145040 0x7f8fb414d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 >> 172.21.15.5:0/1394586254 conn(0x7f8fb4099a50 msgr2=0x7f8fb406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:37.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 shutdown_connections 2024-05-08T23:21:37.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.689+0000 7f8fbb574700 1 -- 172.21.15.5:0/1394586254 wait complete. 2024-05-08T23:21:37.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.841+0000 7f74c4592700 1 Processor -- start 2024-05-08T23:21:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.841+0000 7f74c4592700 1 -- start start 2024-05-08T23:21:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.842+0000 7f74c4592700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f74bc0d13a0 0x7f74bc0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.842+0000 7f74c4592700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74bc06bd60 0x7f74bc0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.842+0000 7f74c4592700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc0b1200 con 0x7f74bc0cdc90 2024-05-08T23:21:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.842+0000 7f74c4592700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0d1cc0 con 0x7f74bc0d13a0 2024-05-08T23:21:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.842+0000 7f74c4592700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0cfcc0 con 0x7f74bc06bd60 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c232e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f74bc0d13a0 0x7f74bc0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c232e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f74bc0d13a0 0x7f74bc0d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41430/0 (socket says 172.21.15.5:41430) 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c232e700 1 -- 172.21.15.5:0/2398489591 learned_addr learned my addr 172.21.15.5:0/2398489591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c1b2d700 1 --2- 172.21.15.5:0/2398489591 >> v2:172.21.15.49:3300/0 conn(0x7f74bc06bd60 0x7f74bc0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c232e700 1 -- 172.21.15.5:0/2398489591 >> v2:172.21.15.49:3300/0 conn(0x7f74bc06bd60 msgr2=0x7f74bc0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c232e700 1 --2- 172.21.15.5:0/2398489591 >> v2:172.21.15.49:3300/0 conn(0x7f74bc06bd60 0x7f74bc0cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c232e700 1 -- 172.21.15.5:0/2398489591 >> v1:172.21.15.49:6789/0 conn(0x7f74bc0cdc90 legacy=0x7f74bc0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c232e700 1 -- 172.21.15.5:0/2398489591 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0ce5c0 con 0x7f74bc0d13a0 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c232e700 1 --2- 172.21.15.5:0/2398489591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0d13a0 0x7f74bc0d1780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0b0f90 tx=0x7f74b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a1d68b7994742 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c132c700 1 -- 172.21.15.5:0/2398489591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8011830 con 0x7f74bc0d13a0 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c132c700 1 -- 172.21.15.5:0/2398489591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8011990 con 0x7f74bc0d13a0 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.843+0000 7f74c132c700 1 -- 172.21.15.5:0/2398489591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8011d20 con 0x7f74bc0d13a0 2024-05-08T23:21:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.844+0000 7f74c4592700 1 -- 172.21.15.5:0/2398489591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0d13a0 msgr2=0x7f74bc0d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.844+0000 7f74c4592700 1 --2- 172.21.15.5:0/2398489591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0d13a0 0x7f74bc0d1780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0b0f90 tx=0x7f74b8009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.844+0000 7f74c4592700 1 -- 172.21.15.5:0/2398489591 shutdown_connections 2024-05-08T23:21:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.844+0000 7f74c4592700 1 --2- 172.21.15.5:0/2398489591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0d13a0 0x7f74bc0d1780 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.844+0000 7f74c4592700 1 --2- 172.21.15.5:0/2398489591 >> v2:172.21.15.49:3300/0 conn(0x7f74bc06bd60 0x7f74bc0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.844+0000 7f74c4592700 1 -- 172.21.15.5:0/2398489591 >> 172.21.15.5:0/2398489591 conn(0x7f74bc08afe0 msgr2=0x7f74bc08b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.844+0000 7f74c4592700 1 -- 172.21.15.5:0/2398489591 shutdown_connections 2024-05-08T23:21:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.844+0000 7f74c4592700 1 -- 172.21.15.5:0/2398489591 wait complete. 2024-05-08T23:21:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.846+0000 7f74c4592700 1 Processor -- start 2024-05-08T23:21:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.846+0000 7f74c4592700 1 -- start start 2024-05-08T23:21:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c4592700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74bc06bd60 0x7f74bc0af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c4592700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0cdc90 0x7f74bc0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c4592700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74bc0d13a0 0x7f74bc15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c4592700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc15b640 con 0x7f74bc0cdc90 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c4592700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0aeb90 con 0x7f74bc06bd60 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c4592700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc1f34c0 con 0x7f74bc0d13a0 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c1b2d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0cdc90 0x7f74bc0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c1b2d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0cdc90 0x7f74bc0ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41432/0 (socket says 172.21.15.5:41432) 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c1b2d700 1 -- 172.21.15.5:0/1835902318 learned_addr learned my addr 172.21.15.5:0/1835902318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c232e700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74bc06bd60 0x7f74bc0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c2b2f700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74bc0d13a0 0x7f74bc15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c1b2d700 1 -- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74bc0d13a0 msgr2=0x7f74bc15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c1b2d700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74bc0d13a0 0x7f74bc15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c1b2d700 1 -- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74bc06bd60 msgr2=0x7f74bc0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c1b2d700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74bc06bd60 0x7f74bc0af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.847+0000 7f74c1b2d700 1 -- 172.21.15.5:0/1835902318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b8009550 con 0x7f74bc0cdc90 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.848+0000 7f74c1b2d700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0cdc90 0x7f74bc0ae650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00e870 tx=0x7f74ac00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.848+0000 7f74b2ffd700 1 -- 172.21.15.5:0/1835902318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac009590 con 0x7f74bc0cdc90 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.848+0000 7f74b2ffd700 1 -- 172.21.15.5:0/1835902318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac013680 con 0x7f74bc0cdc90 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.848+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74bc15aac0 con 0x7f74bc0cdc90 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.848+0000 7f74b2ffd700 1 -- 172.21.15.5:0/1835902318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0139f0 con 0x7f74bc0cdc90 2024-05-08T23:21:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.848+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc15b080 con 0x7f74bc0cdc90 2024-05-08T23:21:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.849+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74bc144920 con 0x7f74bc0cdc90 2024-05-08T23:21:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.849+0000 7f74b2ffd700 1 -- 172.21.15.5:0/1835902318 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f74ac013b50 con 0x7f74bc0cdc90 2024-05-08T23:21:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.850+0000 7f74b2ffd700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f74a006ac90 0x7f74a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.850+0000 7f74b2ffd700 1 -- 172.21.15.5:0/1835902318 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74ac089230 con 0x7f74bc0cdc90 2024-05-08T23:21:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.850+0000 7f74c232e700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f74a006ac90 0x7f74a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.851+0000 7f74c232e700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f74a006ac90 0x7f74a006d150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009c10 tx=0x7f74b8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:37.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:37.854+0000 7f74b2ffd700 1 -- 172.21.15.5:0/1835902318 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f74ac054380 con 0x7f74bc0cdc90 2024-05-08T23:21:38.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.074+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74bc074bc0 con 0x7f74bc0cdc90 2024-05-08T23:21:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.075+0000 7f74b2ffd700 1 -- 172.21.15.5:0/1835902318 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74ac0579a0 con 0x7f74bc0cdc90 2024-05-08T23:21:38.075 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f74a006ac90 msgr2=0x7f74a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f74a006ac90 0x7f74a006d150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009c10 tx=0x7f74b8008030 comp rx=0 tx=0).stop 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0cdc90 msgr2=0x7f74bc0ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0cdc90 0x7f74bc0ae650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00e870 tx=0x7f74ac00eb80 comp rx=0 tx=0).stop 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 shutdown_connections 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f74a006ac90 0x7f74a006d150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74bc0d13a0 0x7f74bc15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74bc06bd60 0x7f74bc0af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 --2- 172.21.15.5:0/1835902318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74bc0cdc90 0x7f74bc0ae650 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 >> 172.21.15.5:0/1835902318 conn(0x7f74bc08afe0 msgr2=0x7f74bc08b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 shutdown_connections 2024-05-08T23:21:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.077+0000 7f74c4592700 1 -- 172.21.15.5:0/1835902318 wait complete. 2024-05-08T23:21:38.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.228+0000 7ff80e774700 1 Processor -- start 2024-05-08T23:21:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.228+0000 7ff80e774700 1 -- start start 2024-05-08T23:21:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80e774700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff808099a50 0x7ff808099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80e774700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8080ccdd0 0x7ff8080cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80e774700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80809aca0 con 0x7ff80815b810 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80e774700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80809a410 con 0x7ff8080ccdd0 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80e774700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080cd7a0 con 0x7ff808099a50 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80cd11700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8080ccdd0 0x7ff8080cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80cd11700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8080ccdd0 0x7ff8080cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41436/0 (socket says 172.21.15.5:41436) 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff807fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff80815b810 0x7ff8080cffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33648/0 (socket says 172.21.15.5:33648) 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80cd11700 1 -- 172.21.15.5:0/3810741413 learned_addr learned my addr 172.21.15.5:0/3810741413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff8077fe700 1 --2- 172.21.15.5:0/3810741413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff808099a50 0x7ff808099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80cd11700 1 -- 172.21.15.5:0/3810741413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff808099a50 msgr2=0x7ff808099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80cd11700 1 --2- 172.21.15.5:0/3810741413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff808099a50 0x7ff808099ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.229+0000 7ff80cd11700 1 -- 172.21.15.5:0/3810741413 >> v1:172.21.15.5:6789/0 conn(0x7ff80815b810 legacy=0x7ff8080cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.230+0000 7ff80cd11700 1 -- 172.21.15.5:0/3810741413 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080cd9e0 con 0x7ff8080ccdd0 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.230+0000 7ff806ffd700 1 -- 172.21.15.5:0/3810741413 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463958822 0 0) 0x7ff80809aca0 con 0x7ff80815b810 2024-05-08T23:21:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.230+0000 7ff80cd11700 1 --2- 172.21.15.5:0/3810741413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8080ccdd0 0x7ff8080cd260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff800009e50 tx=0x7ff80000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445e9259283b2574 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.230+0000 7ff806ffd700 1 -- 172.21.15.5:0/3810741413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000e070 con 0x7ff8080ccdd0 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.230+0000 7ff806ffd700 1 -- 172.21.15.5:0/3810741413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80000b9c0 con 0x7ff8080ccdd0 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.230+0000 7ff806ffd700 1 -- 172.21.15.5:0/3810741413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000bd70 con 0x7ff8080ccdd0 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.230+0000 7ff80e774700 1 -- 172.21.15.5:0/3810741413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8080ccdd0 msgr2=0x7ff8080cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.230+0000 7ff80e774700 1 --2- 172.21.15.5:0/3810741413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8080ccdd0 0x7ff8080cd260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff800009e50 tx=0x7ff80000b330 comp rx=0 tx=0).stop 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.231+0000 7ff80e774700 1 -- 172.21.15.5:0/3810741413 shutdown_connections 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.231+0000 7ff80e774700 1 --2- 172.21.15.5:0/3810741413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8080ccdd0 0x7ff8080cd260 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.231+0000 7ff80e774700 1 --2- 172.21.15.5:0/3810741413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff808099a50 0x7ff808099ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.231+0000 7ff80e774700 1 -- 172.21.15.5:0/3810741413 >> 172.21.15.5:0/3810741413 conn(0x7ff8080b3010 msgr2=0x7ff8080b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.231+0000 7ff80e774700 1 -- 172.21.15.5:0/3810741413 shutdown_connections 2024-05-08T23:21:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.231+0000 7ff80e774700 1 -- 172.21.15.5:0/3810741413 wait complete. 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.232+0000 7ff80e774700 1 Processor -- start 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.233+0000 7ff80e774700 1 -- start start 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.233+0000 7ff80e774700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff808099a50 0x7ff808156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.233+0000 7ff80e774700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8080ccdd0 0x7ff8080af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.233+0000 7ff80e774700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80815b810 0x7ff8080afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.233+0000 7ff80e774700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080cb140 con 0x7ff808099a50 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.233+0000 7ff80e774700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080ca4c0 con 0x7ff80815b810 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.233+0000 7ff80e774700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080ca7a0 con 0x7ff8080ccdd0 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff808099a50 0x7ff808156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff808099a50 0x7ff808156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41448/0 (socket says 172.21.15.5:41448) 2024-05-08T23:21:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 -- 172.21.15.5:0/3994668848 learned_addr learned my addr 172.21.15.5:0/3994668848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff8077fe700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8080ccdd0 0x7ff8080af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff80cd11700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80815b810 0x7ff8080afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 -- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8080ccdd0 msgr2=0x7ff8080af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8080ccdd0 0x7ff8080af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 -- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80815b810 msgr2=0x7ff8080afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80815b810 0x7ff8080afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 -- 172.21.15.5:0/3994668848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800009b30 con 0x7ff808099a50 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff807fff700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff808099a50 0x7ff808156c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80011e0 tx=0x7ff7f800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff804ff9700 1 -- 172.21.15.5:0/3994668848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004c80 con 0x7ff808099a50 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff804ff9700 1 -- 172.21.15.5:0/3994668848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f8004e00 con 0x7ff808099a50 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8080caa80 con 0x7ff808099a50 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff804ff9700 1 -- 172.21.15.5:0/3994668848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8007310 con 0x7ff808099a50 2024-05-08T23:21:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.234+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff808157150 con 0x7ff808099a50 2024-05-08T23:21:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.235+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8080cb280 con 0x7ff808099a50 2024-05-08T23:21:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.236+0000 7ff804ff9700 1 -- 172.21.15.5:0/3994668848 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff7f8007470 con 0x7ff808099a50 2024-05-08T23:21:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.237+0000 7ff804ff9700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff7f406ac90 0x7ff7f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.237+0000 7ff804ff9700 1 -- 172.21.15.5:0/3994668848 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7f8088ed0 con 0x7ff808099a50 2024-05-08T23:21:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.237+0000 7ff8077fe700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff7f406ac90 0x7ff7f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.238+0000 7ff8077fe700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff7f406ac90 0x7ff7f406d150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8080cd770 tx=0x7ff7fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:38.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.241+0000 7ff804ff9700 1 -- 172.21.15.5:0/3994668848 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff8080cb280 con 0x7ff808099a50 2024-05-08T23:21:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.464+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff808068000 con 0x7ff808099a50 2024-05-08T23:21:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.465+0000 7ff804ff9700 1 -- 172.21.15.5:0/3994668848 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7ff7f8054140 con 0x7ff808099a50 2024-05-08T23:21:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:38.466 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff7f406ac90 msgr2=0x7ff7f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff7f406ac90 0x7ff7f406d150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8080cd770 tx=0x7ff7fc008040 comp rx=0 tx=0).stop 2024-05-08T23:21:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff808099a50 msgr2=0x7ff808156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff808099a50 0x7ff808156c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80011e0 tx=0x7ff7f800ca30 comp rx=0 tx=0).stop 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 shutdown_connections 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff7f406ac90 0x7ff7f406d150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80815b810 0x7ff8080afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff808099a50 0x7ff808156c10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 --2- 172.21.15.5:0/3994668848 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8080ccdd0 0x7ff8080af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 >> 172.21.15.5:0/3994668848 conn(0x7ff8080b3010 msgr2=0x7ff8080cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 shutdown_connections 2024-05-08T23:21:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.468+0000 7ff80e774700 1 -- 172.21.15.5:0/3994668848 wait complete. 2024-05-08T23:21:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:38.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.622+0000 7f596e864700 1 Processor -- start 2024-05-08T23:21:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.622+0000 7f596e864700 1 -- start start 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f596e864700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f596815d880 0x7f59680af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f596e864700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f596806f8c0 0x7f59680ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f596e864700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59680b3870 con 0x7f59680ad7a0 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f596e864700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59680b32f0 con 0x7f596806f8c0 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f596e864700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59680b4ee0 con 0x7f596815d880 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f59677fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f596806f8c0 0x7f59680ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f59677fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f596806f8c0 0x7f59680ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41458/0 (socket says 172.21.15.5:41458) 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f59677fe700 1 -- 172.21.15.5:0/1094874836 learned_addr learned my addr 172.21.15.5:0/1094874836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f5967fff700 1 --2- 172.21.15.5:0/1094874836 >> v2:172.21.15.49:3300/0 conn(0x7f596815d880 0x7f59680af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f59677fe700 1 -- 172.21.15.5:0/1094874836 >> v2:172.21.15.49:3300/0 conn(0x7f596815d880 msgr2=0x7f59680af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f59677fe700 1 --2- 172.21.15.5:0/1094874836 >> v2:172.21.15.49:3300/0 conn(0x7f596815d880 0x7f59680af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f59677fe700 1 -- 172.21.15.5:0/1094874836 >> v1:172.21.15.49:6789/0 conn(0x7f59680ad7a0 legacy=0x7f596806e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.623+0000 7f59677fe700 1 -- 172.21.15.5:0/1094874836 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596806fd50 con 0x7f596806f8c0 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.624+0000 7f59677fe700 1 --2- 172.21.15.5:0/1094874836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596806f8c0 0x7f59680ad260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5968067620 tx=0x7f595c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6f6105aadb4054 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.624+0000 7f5966ffd700 1 -- 172.21.15.5:0/1094874836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c004230 con 0x7f596806f8c0 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.624+0000 7f5966ffd700 1 -- 172.21.15.5:0/1094874836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c004390 con 0x7f596806f8c0 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.624+0000 7f5966ffd700 1 -- 172.21.15.5:0/1094874836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0036a0 con 0x7f596806f8c0 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.624+0000 7f596e864700 1 -- 172.21.15.5:0/1094874836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596806f8c0 msgr2=0x7f59680ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.624+0000 7f596e864700 1 --2- 172.21.15.5:0/1094874836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596806f8c0 0x7f59680ad260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5968067620 tx=0x7f595c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.624+0000 7f596e864700 1 -- 172.21.15.5:0/1094874836 shutdown_connections 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.625+0000 7f596e864700 1 --2- 172.21.15.5:0/1094874836 >> v2:172.21.15.49:3300/0 conn(0x7f596815d880 0x7f59680af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.625+0000 7f596e864700 1 --2- 172.21.15.5:0/1094874836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596806f8c0 0x7f59680ad260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.625+0000 7f596e864700 1 -- 172.21.15.5:0/1094874836 >> 172.21.15.5:0/1094874836 conn(0x7f59680b5810 msgr2=0x7f59680b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.625+0000 7f596e864700 1 -- 172.21.15.5:0/1094874836 shutdown_connections 2024-05-08T23:21:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.625+0000 7f596e864700 1 -- 172.21.15.5:0/1094874836 wait complete. 2024-05-08T23:21:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.626+0000 7f596e864700 1 Processor -- start 2024-05-08T23:21:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.626+0000 7f596e864700 1 -- start start 2024-05-08T23:21:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f596e864700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f596806f8c0 0x7f5968156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f596e864700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59680ad7a0 0x7f59680b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f596e864700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f596815d880 0x7f596809c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f596e864700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596809a3e0 con 0x7f59680ad7a0 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f596e864700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596809a6c0 con 0x7f596806f8c0 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f596e864700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596809a9a0 con 0x7f596815d880 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f59677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59680ad7a0 0x7f59680b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f59677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59680ad7a0 0x7f59680b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41472/0 (socket says 172.21.15.5:41472) 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f59677fe700 1 -- 172.21.15.5:0/4047865656 learned_addr learned my addr 172.21.15.5:0/4047865656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f5967fff700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f596806f8c0 0x7f5968156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f596ce01700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f596815d880 0x7f596809c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f59677fe700 1 -- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f596815d880 msgr2=0x7f596809c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f59677fe700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f596815d880 0x7f596809c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f59677fe700 1 -- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f596806f8c0 msgr2=0x7f5968156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f59677fe700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f596806f8c0 0x7f5968156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.627+0000 7f59677fe700 1 -- 172.21.15.5:0/4047865656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c009550 con 0x7f59680ad7a0 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.628+0000 7f59677fe700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59680ad7a0 0x7f59680b2650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f595c009130 tx=0x7f595c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.628+0000 7f5964ff9700 1 -- 172.21.15.5:0/4047865656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c003d40 con 0x7f59680ad7a0 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.628+0000 7f5964ff9700 1 -- 172.21.15.5:0/4047865656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c01f070 con 0x7f59680ad7a0 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.628+0000 7f5964ff9700 1 -- 172.21.15.5:0/4047865656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c011870 con 0x7f59680ad7a0 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.628+0000 7f596e864700 1 -- 172.21.15.5:0/4047865656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596809ac80 con 0x7f59680ad7a0 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.628+0000 7f596e864700 1 -- 172.21.15.5:0/4047865656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596809b240 con 0x7f59680ad7a0 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.629+0000 7f594e7fc700 1 -- 172.21.15.5:0/4047865656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400052f0 con 0x7f59680ad7a0 2024-05-08T23:21:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.629+0000 7f5964ff9700 1 -- 172.21.15.5:0/4047865656 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f595c021030 con 0x7f59680ad7a0 2024-05-08T23:21:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.630+0000 7f5964ff9700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f595406ac90 0x7f595406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.630+0000 7f5964ff9700 1 -- 172.21.15.5:0/4047865656 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f595c08a810 con 0x7f59680ad7a0 2024-05-08T23:21:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.630+0000 7f5967fff700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f595406ac90 0x7f595406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.631+0000 7f5967fff700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f595406ac90 0x7f595406d150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f59680b3050 tx=0x7f5958008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:38.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.635+0000 7f5964ff9700 1 -- 172.21.15.5:0/4047865656 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f595c055a80 con 0x7f59680ad7a0 2024-05-08T23:21:38.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.852+0000 7f594e7fc700 1 -- 172.21.15.5:0/4047865656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5940005b20 con 0x7f59680ad7a0 2024-05-08T23:21:38.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.853+0000 7f5964ff9700 1 -- 172.21.15.5:0/4047865656 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f595c029020 con 0x7f59680ad7a0 2024-05-08T23:21:38.853 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 -- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f595406ac90 msgr2=0x7f595406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f595406ac90 0x7f595406d150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f59680b3050 tx=0x7f5958008040 comp rx=0 tx=0).stop 2024-05-08T23:21:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 -- 172.21.15.5:0/4047865656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59680ad7a0 msgr2=0x7f59680b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59680ad7a0 0x7f59680b2650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f595c009130 tx=0x7f595c004750 comp rx=0 tx=0).stop 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 -- 172.21.15.5:0/4047865656 shutdown_connections 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f595406ac90 0x7f595406d150 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f596815d880 0x7f596809c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f596806f8c0 0x7f5968156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 --2- 172.21.15.5:0/4047865656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59680ad7a0 0x7f59680b2650 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 -- 172.21.15.5:0/4047865656 >> 172.21.15.5:0/4047865656 conn(0x7f59680b5810 msgr2=0x7f59680b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 -- 172.21.15.5:0/4047865656 shutdown_connections 2024-05-08T23:21:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:38.855+0000 7f594e7fc700 1 -- 172.21.15.5:0/4047865656 wait complete. 2024-05-08T23:21:38.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:39.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.008+0000 7f72209b7700 1 Processor -- start 2024-05-08T23:21:39.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.008+0000 7f72209b7700 1 -- start start 2024-05-08T23:21:39.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f72209b7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f721c15d900 0x7f721c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f72209b7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c15dce0 con 0x7f721c0cda40 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f72209b7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c0ce5b0 con 0x7f721c06c080 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f72209b7700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c161510 con 0x7f721c15d900 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f721a59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f721c15d900 0x7f721c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f7219d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f721c0cda40 0x7f721c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33652/0 (socket says 172.21.15.5:33652) 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f7219d9b700 1 -- 172.21.15.5:0/3734701727 learned_addr learned my addr 172.21.15.5:0/3734701727 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f721a59c700 1 -- 172.21.15.5:0/3734701727 >> v1:172.21.15.49:6789/0 conn(0x7f721c06c080 legacy=0x7f721c0ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f721a59c700 1 -- 172.21.15.5:0/3734701727 >> v1:172.21.15.5:6789/0 conn(0x7f721c0cda40 legacy=0x7f721c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f721a59c700 1 -- 172.21.15.5:0/3734701727 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c1617f0 con 0x7f721c15d900 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.009+0000 7f721a59c700 1 --2- 172.21.15.5:0/3734701727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c15d900 0x7f721c161ff0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f720c009840 tx=0x7f720c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbfd569b7ab0bfd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f721959a700 1 -- 172.21.15.5:0/3734701727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c011830 con 0x7f721c15d900 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f721959a700 1 -- 172.21.15.5:0/3734701727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c011990 con 0x7f721c15d900 2024-05-08T23:21:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f721959a700 1 -- 172.21.15.5:0/3734701727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c011d20 con 0x7f721c15d900 2024-05-08T23:21:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f72209b7700 1 -- 172.21.15.5:0/3734701727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c15d900 msgr2=0x7f721c161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f72209b7700 1 --2- 172.21.15.5:0/3734701727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c15d900 0x7f721c161ff0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f720c009840 tx=0x7f720c009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f72209b7700 1 -- 172.21.15.5:0/3734701727 shutdown_connections 2024-05-08T23:21:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f72209b7700 1 --2- 172.21.15.5:0/3734701727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c15d900 0x7f721c161ff0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f72209b7700 1 -- 172.21.15.5:0/3734701727 >> 172.21.15.5:0/3734701727 conn(0x7f721c0713c0 msgr2=0x7f721c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f72209b7700 1 -- 172.21.15.5:0/3734701727 shutdown_connections 2024-05-08T23:21:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.010+0000 7f72209b7700 1 -- 172.21.15.5:0/3734701727 wait complete. 2024-05-08T23:21:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.012+0000 7f72209b7700 1 Processor -- start 2024-05-08T23:21:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.012+0000 7f72209b7700 1 -- start start 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.012+0000 7f72209b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c06c080 0x7f721c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f72209b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f721c0cda40 0x7f721c1f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f72209b7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f721c15d900 0x7f721c14e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f72209b7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1f0f30 con 0x7f721c06c080 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f72209b7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0b3c00 con 0x7f721c0cda40 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f72209b7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1f41f0 con 0x7f721c15d900 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c06c080 0x7f721c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c06c080 0x7f721c0ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41494/0 (socket says 172.21.15.5:41494) 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 -- 172.21.15.5:0/3623533877 learned_addr learned my addr 172.21.15.5:0/3623533877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f7219d9b700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f721c0cda40 0x7f721c1f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721ad9d700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f721c15d900 0x7f721c14e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 -- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f721c15d900 msgr2=0x7f721c14e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f721c15d900 0x7f721c14e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 -- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f721c0cda40 msgr2=0x7f721c1f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f721c0cda40 0x7f721c1f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 -- 172.21.15.5:0/3623533877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0094a0 con 0x7f721c06c080 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f721a59c700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c06c080 0x7f721c0ae880 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f720c000c00 tx=0x7f720c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.013+0000 7f720affd700 1 -- 172.21.15.5:0/3623533877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0036a0 con 0x7f721c06c080 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.014+0000 7f720affd700 1 -- 172.21.15.5:0/3623533877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c01e070 con 0x7f721c06c080 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.014+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c0b3ec0 con 0x7f721c06c080 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.014+0000 7f720affd700 1 -- 172.21.15.5:0/3623533877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c003a40 con 0x7f721c06c080 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.014+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c14ec10 con 0x7f721c06c080 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.015+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c074bc0 con 0x7f721c06c080 2024-05-08T23:21:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.015+0000 7f720affd700 1 -- 172.21.15.5:0/3623533877 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f720c003ba0 con 0x7f721c06c080 2024-05-08T23:21:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.016+0000 7f720affd700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71fc06ac90 0x7f71fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.016+0000 7f720affd700 1 -- 172.21.15.5:0/3623533877 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f720c08a2a0 con 0x7f721c06c080 2024-05-08T23:21:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.016+0000 7f7219d9b700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71fc06ac90 0x7f71fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.016+0000 7f7219d9b700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71fc06ac90 0x7f71fc06d150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7210005910 tx=0x7f7210009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.020+0000 7f720affd700 1 -- 172.21.15.5:0/3623533877 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f720c055510 con 0x7f721c06c080 2024-05-08T23:21:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.236+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f721c0cedd0 con 0x7f721c06c080 2024-05-08T23:21:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.236+0000 7f720affd700 1 -- 172.21.15.5:0/3623533877 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f720c058b30 con 0x7f721c06c080 2024-05-08T23:21:39.236 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71fc06ac90 msgr2=0x7f71fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71fc06ac90 0x7f71fc06d150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7210005910 tx=0x7f7210009040 comp rx=0 tx=0).stop 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c06c080 msgr2=0x7f721c0ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c06c080 0x7f721c0ae880 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f720c000c00 tx=0x7f720c004750 comp rx=0 tx=0).stop 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 shutdown_connections 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71fc06ac90 0x7f71fc06d150 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f721c15d900 0x7f721c14e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f721c06c080 0x7f721c0ae880 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 --2- 172.21.15.5:0/3623533877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f721c0cda40 0x7f721c1f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.238+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 >> 172.21.15.5:0/3623533877 conn(0x7f721c0713c0 msgr2=0x7f721c15fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.239+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 shutdown_connections 2024-05-08T23:21:39.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.239+0000 7f72209b7700 1 -- 172.21.15.5:0/3623533877 wait complete. 2024-05-08T23:21:39.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:39.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.392+0000 7fde38717700 1 Processor -- start 2024-05-08T23:21:39.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.392+0000 7fde38717700 1 -- start start 2024-05-08T23:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde38717700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde3015f320 0x7fde3015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde38717700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde30070880 con 0x7fde3006ea80 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde38717700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde300cc0f0 con 0x7fde300704a0 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde38717700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3015fd10 con 0x7fde3015f320 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde35cb2700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde3015f320 0x7fde3015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde36cb4700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fde3006ea80 0x7fde3006ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33662/0 (socket says 172.21.15.5:33662) 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde36cb4700 1 -- 172.21.15.5:0/3516818369 learned_addr learned my addr 172.21.15.5:0/3516818369 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde35cb2700 1 -- 172.21.15.5:0/3516818369 >> v1:172.21.15.49:6789/0 conn(0x7fde300704a0 legacy=0x7fde300cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde35cb2700 1 -- 172.21.15.5:0/3516818369 >> v1:172.21.15.5:6789/0 conn(0x7fde3006ea80 legacy=0x7fde3006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde35cb2700 1 -- 172.21.15.5:0/3516818369 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde300cc3d0 con 0x7fde3015f320 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.393+0000 7fde354b1700 1 -- 172.21.15.5:0/3516818369 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126780783 0 0) 0x7fde30070880 con 0x7fde3006ea80 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde35cb2700 1 --2- 172.21.15.5:0/3516818369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde3015f7d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde20009840 tx=0x7fde20009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab0aeaba076ebf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde354b1700 1 -- 172.21.15.5:0/3516818369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20011830 con 0x7fde3015f320 2024-05-08T23:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde354b1700 1 -- 172.21.15.5:0/3516818369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde20011990 con 0x7fde3015f320 2024-05-08T23:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde354b1700 1 -- 172.21.15.5:0/3516818369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20011d20 con 0x7fde3015f320 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde38717700 1 -- 172.21.15.5:0/3516818369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 msgr2=0x7fde3015f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde38717700 1 --2- 172.21.15.5:0/3516818369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde3015f7d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde20009840 tx=0x7fde20009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde38717700 1 -- 172.21.15.5:0/3516818369 shutdown_connections 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde38717700 1 --2- 172.21.15.5:0/3516818369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde3015f7d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde38717700 1 -- 172.21.15.5:0/3516818369 >> 172.21.15.5:0/3516818369 conn(0x7fde300b2fa0 msgr2=0x7fde300b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde38717700 1 -- 172.21.15.5:0/3516818369 shutdown_connections 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.394+0000 7fde38717700 1 -- 172.21.15.5:0/3516818369 wait complete. 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.396+0000 7fde38717700 1 Processor -- start 2024-05-08T23:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.396+0000 7fde38717700 1 -- start start 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.396+0000 7fde38717700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde3006ea80 0x7fde300c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.396+0000 7fde38717700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde300704a0 0x7fde300ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.396+0000 7fde38717700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde300cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.396+0000 7fde38717700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300c7880 con 0x7fde3015f320 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde38717700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300b4ec0 con 0x7fde300704a0 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde38717700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde300cb290 con 0x7fde3006ea80 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde300cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde300cad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41504/0 (socket says 172.21.15.5:41504) 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 -- 172.21.15.5:0/2302233752 learned_addr learned my addr 172.21.15.5:0/2302233752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde35cb2700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde300704a0 0x7fde300ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde364b3700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde3006ea80 0x7fde300c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 -- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde3006ea80 msgr2=0x7fde300c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde3006ea80 0x7fde300c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 -- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde300704a0 msgr2=0x7fde300ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde300704a0 0x7fde300ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 -- 172.21.15.5:0/2302233752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde200094a0 con 0x7fde3015f320 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde36cb4700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde300cad50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde28002230 tx=0x7fde2800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde26ffd700 1 -- 172.21.15.5:0/2302233752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2800aac0 con 0x7fde3015f320 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde300b48e0 con 0x7fde3015f320 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde26ffd700 1 -- 172.21.15.5:0/2302233752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2800ac40 con 0x7fde3015f320 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.397+0000 7fde26ffd700 1 -- 172.21.15.5:0/2302233752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde280070d0 con 0x7fde3015f320 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.398+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde300b4b50 con 0x7fde3015f320 2024-05-08T23:21:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.398+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde300b5030 con 0x7fde3015f320 2024-05-08T23:21:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.399+0000 7fde26ffd700 1 -- 172.21.15.5:0/2302233752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fde28007250 con 0x7fde3015f320 2024-05-08T23:21:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.399+0000 7fde26ffd700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde1406ac90 0x7fde1406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.400+0000 7fde26ffd700 1 -- 172.21.15.5:0/2302233752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde28088c90 con 0x7fde3015f320 2024-05-08T23:21:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.400+0000 7fde364b3700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde1406ac90 0x7fde1406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.400+0000 7fde364b3700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde1406ac90 0x7fde1406d150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde2c007fd0 tx=0x7fde2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.404+0000 7fde26ffd700 1 -- 172.21.15.5:0/2302233752 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fde28053f00 con 0x7fde3015f320 2024-05-08T23:21:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.620+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde30074bc0 con 0x7fde3015f320 2024-05-08T23:21:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.621+0000 7fde26ffd700 1 -- 172.21.15.5:0/2302233752 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde28018020 con 0x7fde3015f320 2024-05-08T23:21:39.621 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde1406ac90 msgr2=0x7fde1406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde1406ac90 0x7fde1406d150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde2c007fd0 tx=0x7fde2c009040 comp rx=0 tx=0).stop 2024-05-08T23:21:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 msgr2=0x7fde300cad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde300cad50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde28002230 tx=0x7fde2800cf50 comp rx=0 tx=0).stop 2024-05-08T23:21:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 shutdown_connections 2024-05-08T23:21:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde1406ac90 0x7fde1406d150 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde3015f320 0x7fde300cad50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde3006ea80 0x7fde300c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 --2- 172.21.15.5:0/2302233752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde300704a0 0x7fde300ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 >> 172.21.15.5:0/2302233752 conn(0x7fde300b2fa0 msgr2=0x7fde300b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:39.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 shutdown_connections 2024-05-08T23:21:39.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.624+0000 7fde38717700 1 -- 172.21.15.5:0/2302233752 wait complete. 2024-05-08T23:21:39.663 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-05-08T23:21:39.663 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:39.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.800+0000 7f49d3d9e700 1 Processor -- start 2024-05-08T23:21:39.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.801+0000 7f49d3d9e700 1 -- start start 2024-05-08T23:21:39.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.801+0000 7f49d3d9e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f49cc0cf3c0 0x7f49cc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.801+0000 7f49d3d9e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc06f4a0 0x7f49cc06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.801+0000 7f49d3d9e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc0b2b10 con 0x7f49cc15a380 2024-05-08T23:21:39.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.801+0000 7f49d3d9e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc0ceba0 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.801+0000 7f49d3d9e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc06f950 con 0x7f49cc06f4a0 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d1339700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f49cc0cf3c0 0x7f49cc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d1b3a700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f49cc15a380 0x7f49cc0cbca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33668/0 (socket says 172.21.15.5:33668) 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d1b3a700 1 -- 172.21.15.5:0/1458238221 learned_addr learned my addr 172.21.15.5:0/1458238221 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d233b700 1 --2- 172.21.15.5:0/1458238221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc06f4a0 0x7f49cc06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d1339700 1 -- 172.21.15.5:0/1458238221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc06f4a0 msgr2=0x7f49cc06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d1339700 1 --2- 172.21.15.5:0/1458238221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc06f4a0 0x7f49cc06d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d1339700 1 -- 172.21.15.5:0/1458238221 >> v1:172.21.15.5:6789/0 conn(0x7f49cc15a380 legacy=0x7f49cc0cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d1339700 1 -- 172.21.15.5:0/1458238221 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0ce340 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d0b38700 1 -- 172.21.15.5:0/1458238221 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971784965 0 0) 0x7f49cc0b2b10 con 0x7f49cc15a380 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d1339700 1 --2- 172.21.15.5:0/1458238221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc06ef60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00bae0 tx=0x7f49bc00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d5a74acc5f1f6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d0b38700 1 -- 172.21.15.5:0/1458238221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00dc70 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d0b38700 1 -- 172.21.15.5:0/1458238221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc00ddd0 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.802+0000 7f49d0b38700 1 -- 172.21.15.5:0/1458238221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc010690 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.803+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1458238221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 msgr2=0x7f49cc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.803+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1458238221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc06ef60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00bae0 tx=0x7f49bc00bdf0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.803+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1458238221 shutdown_connections 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.803+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1458238221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc06f4a0 0x7f49cc06d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.803+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1458238221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc06ef60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.803+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1458238221 >> 172.21.15.5:0/1458238221 conn(0x7f49cc0cfce0 msgr2=0x7f49cc0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.803+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1458238221 shutdown_connections 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.803+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1458238221 wait complete. 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.804+0000 7f49d3d9e700 1 Processor -- start 2024-05-08T23:21:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.804+0000 7f49d3d9e700 1 -- start start 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d3d9e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49cc06f4a0 0x7f49cc148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d3d9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc1f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d3d9e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc15a380 0x7f49cc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d3d9e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc1492d0 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d3d9e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc14d670 con 0x7f49cc06f4a0 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d3d9e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc1f32d0 con 0x7f49cc15a380 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d1339700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc1f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d1339700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc1f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41522/0 (socket says 172.21.15.5:41522) 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.805+0000 7f49d1339700 1 -- 172.21.15.5:0/1178970891 learned_addr learned my addr 172.21.15.5:0/1178970891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d233b700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc15a380 0x7f49cc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d1b3a700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49cc06f4a0 0x7f49cc148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d1339700 1 -- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc15a380 msgr2=0x7f49cc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d1339700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc15a380 0x7f49cc1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d1339700 1 -- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49cc06f4a0 msgr2=0x7f49cc148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d1339700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49cc06f4a0 0x7f49cc148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d1339700 1 -- 172.21.15.5:0/1178970891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc00b740 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d1b3a700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49cc06f4a0 0x7f49cc148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d1339700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc1f2d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00b2c0 tx=0x7f49bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49c27fc700 1 -- 172.21.15.5:0/1178970891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc01f070 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49c27fc700 1 -- 172.21.15.5:0/1178970891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc003970 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49c27fc700 1 -- 172.21.15.5:0/1178970891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc003d20 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc14ca60 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.806+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc14d020 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.807+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc068000 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.808+0000 7f49c27fc700 1 -- 172.21.15.5:0/1178970891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f49bc019400 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.809+0000 7f49c27fc700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49b006ac90 0x7f49b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:39.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.809+0000 7f49c27fc700 1 -- 172.21.15.5:0/1178970891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49bc08c9b0 con 0x7f49cc0cf3c0 2024-05-08T23:21:39.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.809+0000 7f49d1b3a700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49b006ac90 0x7f49b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:39.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.809+0000 7f49d1b3a700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49b006ac90 0x7f49b006d150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49c8000f60 tx=0x7f49c8007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:39.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:39.813+0000 7f49c27fc700 1 -- 172.21.15.5:0/1178970891 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f49bc057e20 con 0x7f49cc0cf3c0 2024-05-08T23:21:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.033+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49cc06c1e0 con 0x7f49cc0cf3c0 2024-05-08T23:21:40.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.034+0000 7f49c27fc700 1 -- 172.21.15.5:0/1178970891 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7f49bc029090 con 0x7f49cc0cf3c0 2024-05-08T23:21:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:40.034 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49b006ac90 msgr2=0x7f49b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49b006ac90 0x7f49b006d150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49c8000f60 tx=0x7f49c8007ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 msgr2=0x7f49cc1f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc1f2d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00b2c0 tx=0x7f49bc004750 comp rx=0 tx=0).stop 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 shutdown_connections 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49b006ac90 0x7f49b006d150 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49cc15a380 0x7f49cc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49cc06f4a0 0x7f49cc148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 --2- 172.21.15.5:0/1178970891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc0cf3c0 0x7f49cc1f2d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.036+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 >> 172.21.15.5:0/1178970891 conn(0x7f49cc0cfce0 msgr2=0x7f49cc0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.037+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 shutdown_connections 2024-05-08T23:21:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.037+0000 7f49d3d9e700 1 -- 172.21.15.5:0/1178970891 wait complete. 2024-05-08T23:21:40.050 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:40.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:40.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.190+0000 7f3e47994700 1 Processor -- start 2024-05-08T23:21:40.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.191+0000 7f3e47994700 1 -- start start 2024-05-08T23:21:40.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.191+0000 7f3e47994700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e4015dcc0 0x7f3e4006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.191+0000 7f3e47994700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4006fa10 0x7f3e400b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.191+0000 7f3e47994700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e400cef60 con 0x7f3e4015e310 2024-05-08T23:21:40.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.191+0000 7f3e47994700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4015eee0 con 0x7f3e4015dcc0 2024-05-08T23:21:40.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.191+0000 7f3e47994700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400ce980 con 0x7f3e4006fa10 2024-05-08T23:21:40.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e45730700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e4015dcc0 0x7f3e4006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e45730700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e4015dcc0 0x7f3e4006ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41528/0 (socket says 172.21.15.5:41528) 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e45730700 1 -- 172.21.15.5:0/354792109 learned_addr learned my addr 172.21.15.5:0/354792109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e44f2f700 1 --2- 172.21.15.5:0/354792109 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4006fa10 0x7f3e400b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e45730700 1 -- 172.21.15.5:0/354792109 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4006fa10 msgr2=0x7f3e400b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e45730700 1 --2- 172.21.15.5:0/354792109 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4006fa10 0x7f3e400b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e45730700 1 -- 172.21.15.5:0/354792109 >> v1:172.21.15.49:6789/0 conn(0x7f3e4015e310 legacy=0x7f3e400adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e45730700 1 -- 172.21.15.5:0/354792109 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4015e0a0 con 0x7f3e4015dcc0 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e45730700 1 --2- 172.21.15.5:0/354792109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4015dcc0 0x7f3e4006ef30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0098f0 tx=0x7f3e3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2111d44faeccb2b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e33fff700 1 -- 172.21.15.5:0/354792109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c011830 con 0x7f3e4015dcc0 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e33fff700 1 -- 172.21.15.5:0/354792109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c011990 con 0x7f3e4015dcc0 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.192+0000 7f3e33fff700 1 -- 172.21.15.5:0/354792109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c011d20 con 0x7f3e4015dcc0 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.193+0000 7f3e47994700 1 -- 172.21.15.5:0/354792109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4015dcc0 msgr2=0x7f3e4006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.193+0000 7f3e47994700 1 --2- 172.21.15.5:0/354792109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4015dcc0 0x7f3e4006ef30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0098f0 tx=0x7f3e3c009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:40.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.193+0000 7f3e47994700 1 -- 172.21.15.5:0/354792109 shutdown_connections 2024-05-08T23:21:40.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.193+0000 7f3e47994700 1 --2- 172.21.15.5:0/354792109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4015dcc0 0x7f3e4006ef30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.193+0000 7f3e47994700 1 --2- 172.21.15.5:0/354792109 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4006fa10 0x7f3e400b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.193+0000 7f3e47994700 1 -- 172.21.15.5:0/354792109 >> 172.21.15.5:0/354792109 conn(0x7f3e40099a50 msgr2=0x7f3e400b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:40.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.193+0000 7f3e47994700 1 -- 172.21.15.5:0/354792109 shutdown_connections 2024-05-08T23:21:40.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.193+0000 7f3e47994700 1 -- 172.21.15.5:0/354792109 wait complete. 2024-05-08T23:21:40.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.194+0000 7f3e47994700 1 Processor -- start 2024-05-08T23:21:40.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.194+0000 7f3e47994700 1 -- start start 2024-05-08T23:21:40.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e47994700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4006fa10 0x7f3e40154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e47994700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e4015dcc0 0x7f3e400c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e47994700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4015e310 0x7f3e400ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e47994700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400c5110 con 0x7f3e4006fa10 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e47994700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400c4e30 con 0x7f3e4015dcc0 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e47994700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e401f11d0 con 0x7f3e4015e310 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e45730700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4006fa10 0x7f3e40154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e45730700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4006fa10 0x7f3e40154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41536/0 (socket says 172.21.15.5:41536) 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e45730700 1 -- 172.21.15.5:0/643318282 learned_addr learned my addr 172.21.15.5:0/643318282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e44f2f700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e4015dcc0 0x7f3e400c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.195+0000 7f3e45f31700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4015e310 0x7f3e400ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e45730700 1 -- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4015e310 msgr2=0x7f3e400ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e45730700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4015e310 0x7f3e400ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e45730700 1 -- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e4015dcc0 msgr2=0x7f3e400c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e45730700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e4015dcc0 0x7f3e400c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e45730700 1 -- 172.21.15.5:0/643318282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c009550 con 0x7f3e4006fa10 2024-05-08T23:21:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e45730700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4006fa10 0x7f3e40154160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c000c00 tx=0x7f3e3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e31ffb700 1 -- 172.21.15.5:0/643318282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c004400 con 0x7f3e4006fa10 2024-05-08T23:21:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e31ffb700 1 -- 172.21.15.5:0/643318282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c01f070 con 0x7f3e4006fa10 2024-05-08T23:21:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e400b2d90 con 0x7f3e4006fa10 2024-05-08T23:21:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e31ffb700 1 -- 172.21.15.5:0/643318282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0038e0 con 0x7f3e4006fa10 2024-05-08T23:21:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.196+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e401f1790 con 0x7f3e4006fa10 2024-05-08T23:21:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.197+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e400c7220 con 0x7f3e4006fa10 2024-05-08T23:21:40.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.198+0000 7f3e31ffb700 1 -- 172.21.15.5:0/643318282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3e3c021030 con 0x7f3e4006fa10 2024-05-08T23:21:40.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.198+0000 7f3e31ffb700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3e2406ac90 0x7f3e2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.198+0000 7f3e31ffb700 1 -- 172.21.15.5:0/643318282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e3c08a700 con 0x7f3e4006fa10 2024-05-08T23:21:40.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.199+0000 7f3e44f2f700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3e2406ac90 0x7f3e2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.199+0000 7f3e44f2f700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3e2406ac90 0x7f3e2406d150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3e400b2140 tx=0x7f3e34009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.203+0000 7f3e31ffb700 1 -- 172.21.15.5:0/643318282 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3e3c055850 con 0x7f3e4006fa10 2024-05-08T23:21:40.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.421+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e40074bc0 con 0x7f3e4006fa10 2024-05-08T23:21:40.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.421+0000 7f3e31ffb700 1 -- 172.21.15.5:0/643318282 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e3c058e70 con 0x7f3e4006fa10 2024-05-08T23:21:40.421 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3e2406ac90 msgr2=0x7f3e2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3e2406ac90 0x7f3e2406d150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3e400b2140 tx=0x7f3e34009210 comp rx=0 tx=0).stop 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4006fa10 msgr2=0x7f3e40154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4006fa10 0x7f3e40154160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c000c00 tx=0x7f3e3c004750 comp rx=0 tx=0).stop 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 shutdown_connections 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3e2406ac90 0x7f3e2406d150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e4015e310 0x7f3e400ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e4006fa10 0x7f3e40154160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 --2- 172.21.15.5:0/643318282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e4015dcc0 0x7f3e400c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 >> 172.21.15.5:0/643318282 conn(0x7f3e40099a50 msgr2=0x7f3e4006ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 shutdown_connections 2024-05-08T23:21:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.423+0000 7f3e47994700 1 -- 172.21.15.5:0/643318282 wait complete. 2024-05-08T23:21:40.437 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.576+0000 7f323d508700 1 Processor -- start 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.576+0000 7f323d508700 1 -- start start 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f323d508700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3238159be0 0x7f32380ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f323d508700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 0x7f323815a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f323d508700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380b2380 con 0x7f323815aa50 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f323d508700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32380cec00 con 0x7f3238159be0 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f323d508700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380b3140 con 0x7f32380ce780 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f3236ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3238159be0 0x7f32380ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f3236ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3238159be0 0x7f32380ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41552/0 (socket says 172.21.15.5:41552) 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f3236ffd700 1 -- 172.21.15.5:0/4257569792 learned_addr learned my addr 172.21.15.5:0/4257569792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.577+0000 7f32367fc700 1 --2- 172.21.15.5:0/4257569792 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 0x7f323815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.578+0000 7f3236ffd700 1 -- 172.21.15.5:0/4257569792 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 msgr2=0x7f323815a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.578+0000 7f3236ffd700 1 --2- 172.21.15.5:0/4257569792 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 0x7f323815a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.578+0000 7f3236ffd700 1 -- 172.21.15.5:0/4257569792 >> v1:172.21.15.49:6789/0 conn(0x7f323815aa50 legacy=0x7f32380cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.578+0000 7f3236ffd700 1 -- 172.21.15.5:0/4257569792 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380b2660 con 0x7f3238159be0 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.578+0000 7f3236ffd700 1 --2- 172.21.15.5:0/4257569792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f32380ce240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f322800bb90 tx=0x7f322800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e668411ef3796e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.578+0000 7f3235ffb700 1 -- 172.21.15.5:0/4257569792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322800dc40 con 0x7f3238159be0 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.578+0000 7f3235ffb700 1 -- 172.21.15.5:0/4257569792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322800dda0 con 0x7f3238159be0 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.578+0000 7f3235ffb700 1 -- 172.21.15.5:0/4257569792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228017630 con 0x7f3238159be0 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.579+0000 7f323d508700 1 -- 172.21.15.5:0/4257569792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 msgr2=0x7f32380ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.579+0000 7f323d508700 1 --2- 172.21.15.5:0/4257569792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f32380ce240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f322800bb90 tx=0x7f322800bea0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.579+0000 7f323d508700 1 -- 172.21.15.5:0/4257569792 shutdown_connections 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.579+0000 7f323d508700 1 --2- 172.21.15.5:0/4257569792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f32380ce240 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.579+0000 7f323d508700 1 --2- 172.21.15.5:0/4257569792 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 0x7f323815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.579+0000 7f323d508700 1 -- 172.21.15.5:0/4257569792 >> 172.21.15.5:0/4257569792 conn(0x7f323806a960 msgr2=0x7f323806ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.579+0000 7f323d508700 1 -- 172.21.15.5:0/4257569792 shutdown_connections 2024-05-08T23:21:40.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.579+0000 7f323d508700 1 -- 172.21.15.5:0/4257569792 wait complete. 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.580+0000 7f323d508700 1 Processor -- start 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.580+0000 7f323d508700 1 -- start start 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f323d508700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 0x7f32381548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f323d508700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f323809af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f323d508700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323815aa50 0x7f32380c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f323d508700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238099c30 con 0x7f3238159be0 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f323d508700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238099370 con 0x7f323815aa50 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f323d508700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380995e0 con 0x7f32380ce780 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32367fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f323809af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32367fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f323809af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41562/0 (socket says 172.21.15.5:41562) 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32367fc700 1 -- 172.21.15.5:0/2925550148 learned_addr learned my addr 172.21.15.5:0/2925550148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f3236ffd700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 0x7f32381548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32377fe700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323815aa50 0x7f32380c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32367fc700 1 -- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 msgr2=0x7f32381548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32367fc700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 0x7f32381548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32367fc700 1 -- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323815aa50 msgr2=0x7f32380c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32367fc700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323815aa50 0x7f32380c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.581+0000 7f32367fc700 1 -- 172.21.15.5:0/2925550148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322800b7f0 con 0x7f3238159be0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.582+0000 7f32367fc700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f323809af10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f322000e840 tx=0x7f322000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.582+0000 7f322f7fe700 1 -- 172.21.15.5:0/2925550148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220013610 con 0x7f3238159be0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.582+0000 7f322f7fe700 1 -- 172.21.15.5:0/2925550148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3220013770 con 0x7f3238159be0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.582+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32380998c0 con 0x7f3238159be0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.582+0000 7f322f7fe700 1 -- 172.21.15.5:0/2925550148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220013b20 con 0x7f3238159be0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.582+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323809b9b0 con 0x7f3238159be0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.583+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3238159440 con 0x7f3238159be0 2024-05-08T23:21:40.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.584+0000 7f322f7fe700 1 -- 172.21.15.5:0/2925550148 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3220013c80 con 0x7f3238159be0 2024-05-08T23:21:40.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.584+0000 7f322f7fe700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321806ac90 0x7f321806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.584+0000 7f322f7fe700 1 -- 172.21.15.5:0/2925550148 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3220088f80 con 0x7f3238159be0 2024-05-08T23:21:40.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.584+0000 7f3236ffd700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321806ac90 0x7f321806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.585+0000 7f3236ffd700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321806ac90 0x7f321806d150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3228019040 tx=0x7f3228014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.589+0000 7f322f7fe700 1 -- 172.21.15.5:0/2925550148 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3220054160 con 0x7f3238159be0 2024-05-08T23:21:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.806+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3238074bc0 con 0x7f3238159be0 2024-05-08T23:21:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.807+0000 7f322f7fe700 1 -- 172.21.15.5:0/2925550148 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3220057780 con 0x7f3238159be0 2024-05-08T23:21:40.807 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321806ac90 msgr2=0x7f321806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321806ac90 0x7f321806d150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3228019040 tx=0x7f3228014000 comp rx=0 tx=0).stop 2024-05-08T23:21:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 msgr2=0x7f323809af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f323809af10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f322000e840 tx=0x7f322000ec00 comp rx=0 tx=0).stop 2024-05-08T23:21:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 shutdown_connections 2024-05-08T23:21:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321806ac90 0x7f321806d150 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323815aa50 0x7f32380c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32380ce780 0x7f32381548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 --2- 172.21.15.5:0/2925550148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3238159be0 0x7f323809af10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.809+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 >> 172.21.15.5:0/2925550148 conn(0x7f323806a960 msgr2=0x7f323806df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.810+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 shutdown_connections 2024-05-08T23:21:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.810+0000 7f323d508700 1 -- 172.21.15.5:0/2925550148 wait complete. 2024-05-08T23:21:40.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.963+0000 7faa716e5700 1 Processor -- start 2024-05-08T23:21:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.963+0000 7faa716e5700 1 -- start start 2024-05-08T23:21:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa716e5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa6c15d700 0x7faa6c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa716e5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c0b38b0 0x7faa6c1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa716e5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c0b49b0 con 0x7faa6c15dae0 2024-05-08T23:21:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa716e5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c0b1150 con 0x7faa6c15d700 2024-05-08T23:21:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa716e5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0b3da0 con 0x7faa6c0b38b0 2024-05-08T23:21:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6a7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c0b38b0 0x7faa6c1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6affd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa6c15d700 0x7faa6c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6a7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c0b38b0 0x7faa6c1614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:48310/0 (socket says 172.21.15.5:48310) 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6a7fc700 1 -- 172.21.15.5:0/7636187 learned_addr learned my addr 172.21.15.5:0/7636187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6a7fc700 1 -- 172.21.15.5:0/7636187 >> v2:172.21.15.49:3300/0 conn(0x7faa6c15d700 msgr2=0x7faa6c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6a7fc700 1 --2- 172.21.15.5:0/7636187 >> v2:172.21.15.49:3300/0 conn(0x7faa6c15d700 0x7faa6c0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6a7fc700 1 -- 172.21.15.5:0/7636187 >> v1:172.21.15.49:6789/0 conn(0x7faa6c15dae0 legacy=0x7faa6c0d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6a7fc700 1 -- 172.21.15.5:0/7636187 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c0b4080 con 0x7faa6c0b38b0 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.964+0000 7faa6affd700 1 --2- 172.21.15.5:0/7636187 >> v2:172.21.15.49:3300/0 conn(0x7faa6c15d700 0x7faa6c0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa6a7fc700 1 --2- 172.21.15.5:0/7636187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c0b38b0 0x7faa6c1614f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0098f0 tx=0x7faa5c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef992e9ea52f989 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa69ffb700 1 -- 172.21.15.5:0/7636187 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c011830 con 0x7faa6c0b38b0 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa69ffb700 1 -- 172.21.15.5:0/7636187 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c011990 con 0x7faa6c0b38b0 2024-05-08T23:21:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa69ffb700 1 -- 172.21.15.5:0/7636187 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c011d20 con 0x7faa6c0b38b0 2024-05-08T23:21:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa716e5700 1 -- 172.21.15.5:0/7636187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c0b38b0 msgr2=0x7faa6c1614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa716e5700 1 --2- 172.21.15.5:0/7636187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c0b38b0 0x7faa6c1614f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0098f0 tx=0x7faa5c009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa716e5700 1 -- 172.21.15.5:0/7636187 shutdown_connections 2024-05-08T23:21:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa716e5700 1 --2- 172.21.15.5:0/7636187 >> v2:172.21.15.49:3300/0 conn(0x7faa6c15d700 0x7faa6c0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa716e5700 1 --2- 172.21.15.5:0/7636187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c0b38b0 0x7faa6c1614f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa716e5700 1 -- 172.21.15.5:0/7636187 >> 172.21.15.5:0/7636187 conn(0x7faa6c08afe0 msgr2=0x7faa6c0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa716e5700 1 -- 172.21.15.5:0/7636187 shutdown_connections 2024-05-08T23:21:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.965+0000 7faa716e5700 1 -- 172.21.15.5:0/7636187 wait complete. 2024-05-08T23:21:40.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.967+0000 7faa716e5700 1 Processor -- start 2024-05-08T23:21:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.967+0000 7faa716e5700 1 -- start start 2024-05-08T23:21:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa716e5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa6c0b38b0 0x7faa6c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa716e5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa6c15d700 0x7faa6c0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa716e5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c15dae0 0x7faa6c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa716e5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c1590b0 con 0x7faa6c15d700 2024-05-08T23:21:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa716e5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0b7cc0 con 0x7faa6c0b38b0 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa716e5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0b7fa0 con 0x7faa6c15dae0 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa6c15d700 0x7faa6c0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa6c15d700 0x7faa6c0c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41568/0 (socket says 172.21.15.5:41568) 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 -- 172.21.15.5:0/2157887334 learned_addr learned my addr 172.21.15.5:0/2157887334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6affd700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa6c0b38b0 0x7faa6c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6b7fe700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c15dae0 0x7faa6c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 -- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c15dae0 msgr2=0x7faa6c0b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c15dae0 0x7faa6c0b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 -- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa6c0b38b0 msgr2=0x7faa6c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa6c0b38b0 0x7faa6c158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 -- 172.21.15.5:0/2157887334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa54009430 con 0x7faa6c15d700 2024-05-08T23:21:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6b7fe700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c15dae0 0x7faa6c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.968+0000 7faa6a7fc700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa6c15d700 0x7faa6c0c7a30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009c40 tx=0x7faa5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.969+0000 7faa637fe700 1 -- 172.21.15.5:0/2157887334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c004590 con 0x7faa6c15d700 2024-05-08T23:21:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.969+0000 7faa637fe700 1 -- 172.21.15.5:0/2157887334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c01e070 con 0x7faa6c15d700 2024-05-08T23:21:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.969+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c009550 con 0x7faa6c15d700 2024-05-08T23:21:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.969+0000 7faa637fe700 1 -- 172.21.15.5:0/2157887334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0038e0 con 0x7faa6c15d700 2024-05-08T23:21:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.969+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c0b8560 con 0x7faa6c15d700 2024-05-08T23:21:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.970+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c074bc0 con 0x7faa6c15d700 2024-05-08T23:21:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.970+0000 7faa637fe700 1 -- 172.21.15.5:0/2157887334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7faa5c003a40 con 0x7faa6c15d700 2024-05-08T23:21:40.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.971+0000 7faa637fe700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa4c06ac90 0x7faa4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:40.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.971+0000 7faa637fe700 1 -- 172.21.15.5:0/2157887334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa5c08a2e0 con 0x7faa6c15d700 2024-05-08T23:21:40.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.971+0000 7faa6affd700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa4c06ac90 0x7faa4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:40.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.971+0000 7faa6affd700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa4c06ac90 0x7faa4c06d150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faa54009d50 tx=0x7faa540091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:40.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:40.976+0000 7faa637fe700 1 -- 172.21.15.5:0/2157887334 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7faa5c0553a0 con 0x7faa6c15d700 2024-05-08T23:21:41.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.195+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa6c0d0e00 con 0x7faa6c15d700 2024-05-08T23:21:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.196+0000 7faa637fe700 1 -- 172.21.15.5:0/2157887334 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa5c0589c0 con 0x7faa6c15d700 2024-05-08T23:21:41.196 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa4c06ac90 msgr2=0x7faa4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa4c06ac90 0x7faa4c06d150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faa54009d50 tx=0x7faa540091c0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa6c15d700 msgr2=0x7faa6c0c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa6c15d700 0x7faa6c0c7a30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009c40 tx=0x7faa5c004750 comp rx=0 tx=0).stop 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 shutdown_connections 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faa4c06ac90 0x7faa4c06d150 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faa6c15dae0 0x7faa6c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa6c0b38b0 0x7faa6c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 --2- 172.21.15.5:0/2157887334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa6c15d700 0x7faa6c0c7a30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 >> 172.21.15.5:0/2157887334 conn(0x7faa6c08afe0 msgr2=0x7faa6c15b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 shutdown_connections 2024-05-08T23:21:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.198+0000 7faa716e5700 1 -- 172.21.15.5:0/2157887334 wait complete. 2024-05-08T23:21:41.212 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-05-08T23:21:41.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.348+0000 7fab2c467700 1 Processor -- start 2024-05-08T23:21:41.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.349+0000 7fab2c467700 1 -- start start 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2c467700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab2415ef60 0x7fab240b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2c467700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf000 0x7fab240cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2c467700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240cfe30 con 0x7fab240cf9a0 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2c467700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240cd400 con 0x7fab2415ef60 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2c467700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2409ad40 con 0x7fab240cf000 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2a203700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab2415ef60 0x7fab240b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2a203700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab2415ef60 0x7fab240b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41576/0 (socket says 172.21.15.5:41576) 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2a203700 1 -- 172.21.15.5:0/3623488858 learned_addr learned my addr 172.21.15.5:0/3623488858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab29a02700 1 --2- 172.21.15.5:0/3623488858 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf000 0x7fab240cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2a203700 1 -- 172.21.15.5:0/3623488858 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf000 msgr2=0x7fab240cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2a203700 1 --2- 172.21.15.5:0/3623488858 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf000 0x7fab240cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2a203700 1 -- 172.21.15.5:0/3623488858 >> v1:172.21.15.49:6789/0 conn(0x7fab240cf9a0 legacy=0x7fab24099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.350+0000 7fab2a203700 1 -- 172.21.15.5:0/3623488858 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab240ccb60 con 0x7fab2415ef60 2024-05-08T23:21:41.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.351+0000 7fab2a203700 1 --2- 172.21.15.5:0/3623488858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab240b2340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab200098f0 tx=0x7fab20009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07116745f31e3f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.351+0000 7fab29201700 1 -- 172.21.15.5:0/3623488858 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000b8a0 con 0x7fab2415ef60 2024-05-08T23:21:41.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.351+0000 7fab29201700 1 -- 172.21.15.5:0/3623488858 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2000ba00 con 0x7fab2415ef60 2024-05-08T23:21:41.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.351+0000 7fab29201700 1 -- 172.21.15.5:0/3623488858 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000bd90 con 0x7fab2415ef60 2024-05-08T23:21:41.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.352+0000 7fab2c467700 1 -- 172.21.15.5:0/3623488858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 msgr2=0x7fab240b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.352+0000 7fab2c467700 1 --2- 172.21.15.5:0/3623488858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab240b2340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab200098f0 tx=0x7fab20009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:41.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.352+0000 7fab2c467700 1 -- 172.21.15.5:0/3623488858 shutdown_connections 2024-05-08T23:21:41.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.352+0000 7fab2c467700 1 --2- 172.21.15.5:0/3623488858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab240b2340 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.352+0000 7fab2c467700 1 --2- 172.21.15.5:0/3623488858 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf000 0x7fab240cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.352+0000 7fab2c467700 1 -- 172.21.15.5:0/3623488858 >> 172.21.15.5:0/3623488858 conn(0x7fab240b2f20 msgr2=0x7fab240b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.352+0000 7fab2c467700 1 -- 172.21.15.5:0/3623488858 shutdown_connections 2024-05-08T23:21:41.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.352+0000 7fab2c467700 1 -- 172.21.15.5:0/3623488858 wait complete. 2024-05-08T23:21:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.353+0000 7fab2c467700 1 Processor -- start 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.354+0000 7fab2c467700 1 -- start start 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.354+0000 7fab2c467700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab240cf000 0x7fab24151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.354+0000 7fab2c467700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf9a0 0x7fab2406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.354+0000 7fab2c467700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab2406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.354+0000 7fab2c467700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab24151fa0 con 0x7fab2415ef60 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.354+0000 7fab2c467700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240b0d80 con 0x7fab240cf000 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.354+0000 7fab2c467700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab240b1060 con 0x7fab240cf9a0 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.354+0000 7fab2aa04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab2406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2aa04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab2406d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41590/0 (socket says 172.21.15.5:41590) 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2aa04700 1 -- 172.21.15.5:0/656528032 learned_addr learned my addr 172.21.15.5:0/656528032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:41.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2a203700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab240cf000 0x7fab24151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab29a02700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf9a0 0x7fab2406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2aa04700 1 -- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf9a0 msgr2=0x7fab2406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2aa04700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf9a0 0x7fab2406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2aa04700 1 -- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab240cf000 msgr2=0x7fab24151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2aa04700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab240cf000 0x7fab24151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2aa04700 1 -- 172.21.15.5:0/656528032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab20009550 con 0x7fab2415ef60 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2aa04700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab2406d030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00cb50 tx=0x7fab1c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab1affd700 1 -- 172.21.15.5:0/656528032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c003c00 con 0x7fab2415ef60 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab1affd700 1 -- 172.21.15.5:0/656528032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c003d60 con 0x7fab2415ef60 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab240b1340 con 0x7fab2415ef60 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab1affd700 1 -- 172.21.15.5:0/656528032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00ab70 con 0x7fab2415ef60 2024-05-08T23:21:41.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.355+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab240afe70 con 0x7fab2415ef60 2024-05-08T23:21:41.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.356+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab24068000 con 0x7fab2415ef60 2024-05-08T23:21:41.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.357+0000 7fab1affd700 1 -- 172.21.15.5:0/656528032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fab1c00acd0 con 0x7fab2415ef60 2024-05-08T23:21:41.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.357+0000 7fab1affd700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab0806ac90 0x7fab0806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.357+0000 7fab1affd700 1 -- 172.21.15.5:0/656528032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab1c089380 con 0x7fab2415ef60 2024-05-08T23:21:41.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.358+0000 7fab2a203700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab0806ac90 0x7fab0806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.358+0000 7fab2a203700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab0806ac90 0x7fab0806d150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab20000c00 tx=0x7fab20008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.362+0000 7fab1affd700 1 -- 172.21.15.5:0/656528032 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fab1c054550 con 0x7fab2415ef60 2024-05-08T23:21:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:41.673+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) v1 -- 0x7fab240cedc0 con 0x7fab2415ef60 2024-05-08T23:21:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.218+0000 7fab1affd700 1 -- 172.21.15.5:0/656528032 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v57) v1 ==== 139+0+0 (secure 0 0 0) 0x7fab1c057b70 con 0x7fab2415ef60 2024-05-08T23:21:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.221+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab0806ac90 msgr2=0x7fab0806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.221+0000 7fab2c467700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab0806ac90 0x7fab0806d150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab20000c00 tx=0x7fab20008030 comp rx=0 tx=0).stop 2024-05-08T23:21:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.221+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 msgr2=0x7fab2406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.221+0000 7fab2c467700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab2406d030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00cb50 tx=0x7fab1c00cf10 comp rx=0 tx=0).stop 2024-05-08T23:21:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.221+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 shutdown_connections 2024-05-08T23:21:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.222+0000 7fab2c467700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab0806ac90 0x7fab0806d150 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.222+0000 7fab2c467700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2415ef60 0x7fab2406d030 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.222+0000 7fab2c467700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab240cf000 0x7fab24151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.222+0000 7fab2c467700 1 --2- 172.21.15.5:0/656528032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab240cf9a0 0x7fab2406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.222+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 >> 172.21.15.5:0/656528032 conn(0x7fab240b2f20 msgr2=0x7fab2406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.222+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 shutdown_connections 2024-05-08T23:21:42.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.222+0000 7fab2c467700 1 -- 172.21.15.5:0/656528032 wait complete. 2024-05-08T23:21:42.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T23:21:42.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.381+0000 7f5340a57700 1 Processor -- start 2024-05-08T23:21:42.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.381+0000 7f5340a57700 1 -- start start 2024-05-08T23:21:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.382+0000 7f5340a57700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c0cbf50 0x7f533c161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.382+0000 7f5340a57700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c0cf970 con 0x7f533c15e760 2024-05-08T23:21:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.382+0000 7f5340a57700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c0cfc50 con 0x7f533c162240 2024-05-08T23:21:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.382+0000 7f5340a57700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c0cff30 con 0x7f533c0cbf50 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.382+0000 7f533ad9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f533c15e760 0x7f533c06a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33670/0 (socket says 172.21.15.5:33670) 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.382+0000 7f533ad9d700 1 -- 172.21.15.5:0/3110772847 learned_addr learned my addr 172.21.15.5:0/3110772847 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533a59c700 1 --2- 172.21.15.5:0/3110772847 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c0cbf50 0x7f533c161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533959a700 1 -- 172.21.15.5:0/3110772847 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750974994 0 0) 0x7f533c0cf970 con 0x7f533c15e760 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533959a700 1 -- 172.21.15.5:0/3110772847 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53280035b0 con 0x7f533c15e760 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533959a700 1 -- 172.21.15.5:0/3110772847 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516455035 0 0) 0x7f533c0cfc50 con 0x7f533c162240 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533959a700 1 -- 172.21.15.5:0/3110772847 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f533c0cf970 con 0x7f533c162240 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533959a700 1 -- 172.21.15.5:0/3110772847 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2521443172 0 0) 0x7f53280035b0 con 0x7f533c15e760 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533a59c700 1 -- 172.21.15.5:0/3110772847 >> v1:172.21.15.49:6789/0 conn(0x7f533c162240 legacy=0x7f533c15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533a59c700 1 -- 172.21.15.5:0/3110772847 >> v1:172.21.15.5:6789/0 conn(0x7f533c15e760 legacy=0x7f533c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533a59c700 1 -- 172.21.15.5:0/3110772847 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c1617e0 con 0x7f533c0cbf50 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533a59c700 1 --2- 172.21.15.5:0/3110772847 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c0cbf50 0x7f533c161d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5324009870 tx=0x7f5324009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499c6b3cd2184717 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533959a700 1 -- 172.21.15.5:0/3110772847 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340019f0 con 0x7f533c0cbf50 2024-05-08T23:21:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533959a700 1 -- 172.21.15.5:0/3110772847 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5324004030 con 0x7f533c0cbf50 2024-05-08T23:21:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.383+0000 7f533959a700 1 -- 172.21.15.5:0/3110772847 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53240043c0 con 0x7f533c0cbf50 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.384+0000 7f5340a57700 1 -- 172.21.15.5:0/3110772847 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c0cbf50 msgr2=0x7f533c161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.384+0000 7f5340a57700 1 --2- 172.21.15.5:0/3110772847 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c0cbf50 0x7f533c161d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5324009870 tx=0x7f5324009b80 comp rx=0 tx=0).stop 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.384+0000 7f5340a57700 1 -- 172.21.15.5:0/3110772847 shutdown_connections 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.384+0000 7f5340a57700 1 --2- 172.21.15.5:0/3110772847 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c0cbf50 0x7f533c161d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.384+0000 7f5340a57700 1 -- 172.21.15.5:0/3110772847 >> 172.21.15.5:0/3110772847 conn(0x7f533c099a50 msgr2=0x7f533c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.384+0000 7f5340a57700 1 -- 172.21.15.5:0/3110772847 shutdown_connections 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.384+0000 7f5340a57700 1 -- 172.21.15.5:0/3110772847 wait complete. 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.386+0000 7f5340a57700 1 Processor -- start 2024-05-08T23:21:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.386+0000 7f5340a57700 1 -- start start 2024-05-08T23:21:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.386+0000 7f5340a57700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f533c0cbf50 0x7f533c153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.386+0000 7f5340a57700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c162240 0x7f533c160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.386+0000 7f5340a57700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f533c0afd20 0x7f533c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.386+0000 7f5340a57700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c0b06f0 con 0x7f533c0afd20 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.386+0000 7f5340a57700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1613f0 con 0x7f533c0cbf50 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.386+0000 7f5340a57700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1605e0 con 0x7f533c162240 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f533c0afd20 0x7f533c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f533c0afd20 0x7f533c0b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41592/0 (socket says 172.21.15.5:41592) 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 -- 172.21.15.5:0/2718138642 learned_addr learned my addr 172.21.15.5:0/2718138642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533a59c700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f533c0cbf50 0x7f533c153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f5339d9b700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c162240 0x7f533c160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 -- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c162240 msgr2=0x7f533c160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c162240 0x7f533c160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 -- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f533c0cbf50 msgr2=0x7f533c153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f533c0cbf50 0x7f533c153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 -- 172.21.15.5:0/2718138642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240094a0 con 0x7f533c0afd20 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533a59c700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f533c0cbf50 0x7f533c153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f533ad9d700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f533c0afd20 0x7f533c0b01b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f533400cc50 tx=0x7f533400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f5332ffd700 1 -- 172.21.15.5:0/2718138642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400aa70 con 0x7f533c0afd20 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f5332ffd700 1 -- 172.21.15.5:0/2718138642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533400abd0 con 0x7f533c0afd20 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c15f9d0 con 0x7f533c0afd20 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f5332ffd700 1 -- 172.21.15.5:0/2718138642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340070d0 con 0x7f533c0afd20 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.387+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c15ff90 con 0x7f533c0afd20 2024-05-08T23:21:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.389+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c067ac0 con 0x7f533c0afd20 2024-05-08T23:21:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.389+0000 7f5332ffd700 1 -- 172.21.15.5:0/2718138642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5334007230 con 0x7f533c0afd20 2024-05-08T23:21:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.389+0000 7f5332ffd700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f531c06ac90 0x7f531c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.390+0000 7f5332ffd700 1 -- 172.21.15.5:0/2718138642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5334089110 con 0x7f533c0afd20 2024-05-08T23:21:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.390+0000 7f533a59c700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f531c06ac90 0x7f531c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.390+0000 7f533a59c700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f531c06ac90 0x7f531c06d150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f53240097c0 tx=0x7f5324008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.394+0000 7f5332ffd700 1 -- 172.21.15.5:0/2718138642 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5334057d80 con 0x7f533c0afd20 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.612+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f533c074bc0 con 0x7f533c0afd20 2024-05-08T23:21:42.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.615+0000 7f5332ffd700 1 -- 172.21.15.5:0/2718138642 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5334057910 con 0x7f533c0afd20 2024-05-08T23:21:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f531c06ac90 msgr2=0x7f531c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f531c06ac90 0x7f531c06d150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f53240097c0 tx=0x7f5324008030 comp rx=0 tx=0).stop 2024-05-08T23:21:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f533c0afd20 msgr2=0x7f533c0b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f533c0afd20 0x7f533c0b01b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f533400cc50 tx=0x7f533400cf60 comp rx=0 tx=0).stop 2024-05-08T23:21:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 shutdown_connections 2024-05-08T23:21:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f531c06ac90 0x7f531c06d150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f533c0afd20 0x7f533c0b01b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f533c0cbf50 0x7f533c153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 --2- 172.21.15.5:0/2718138642 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f533c162240 0x7f533c160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 >> 172.21.15.5:0/2718138642 conn(0x7f533c099a50 msgr2=0x7f533c0cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 shutdown_connections 2024-05-08T23:21:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:42.618+0000 7f5340a57700 1 -- 172.21.15.5:0/2718138642 wait complete. 2024-05-08T23:21:42.635 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:21:42.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:21:42.686 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:21:42.740 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:21:42.740 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:21:42.741 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:42.762 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:21:42.762 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:42.824 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:21:42.824 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:42.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:43.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.071+0000 7f4090ca3700 1 Processor -- start 2024-05-08T23:21:43.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.071+0000 7f4090ca3700 1 -- start start 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f4090ca3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f408c15ef60 0x7f408c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f4090ca3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f408c0cf000 0x7f408c0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f4090ca3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c0cfe10 con 0x7f408c0cf9a0 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f4090ca3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408c0cd400 con 0x7f408c0cf000 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f4090ca3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408c09ad40 con 0x7f408c15ef60 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f4089d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f408c0cf000 0x7f408c0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f408ad9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f408c0cf9a0 0x7f408c099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33686/0 (socket says 172.21.15.5:33686) 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f408ad9d700 1 -- 172.21.15.5:0/3286314348 learned_addr learned my addr 172.21.15.5:0/3286314348 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.072+0000 7f408a59c700 1 --2- 172.21.15.5:0/3286314348 >> v2:172.21.15.49:3300/0 conn(0x7f408c15ef60 0x7f408c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f408959a700 1 -- 172.21.15.5:0/3286314348 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975568788 0 0) 0x7f408c0cfe10 con 0x7f408c0cf9a0 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f408959a700 1 -- 172.21.15.5:0/3286314348 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40780035b0 con 0x7f408c0cf9a0 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f4089d9b700 1 -- 172.21.15.5:0/3286314348 >> v2:172.21.15.49:3300/0 conn(0x7f408c15ef60 msgr2=0x7f408c0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f4089d9b700 1 --2- 172.21.15.5:0/3286314348 >> v2:172.21.15.49:3300/0 conn(0x7f408c15ef60 0x7f408c0b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f4089d9b700 1 -- 172.21.15.5:0/3286314348 >> v1:172.21.15.5:6789/0 conn(0x7f408c0cf9a0 legacy=0x7f408c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f4089d9b700 1 -- 172.21.15.5:0/3286314348 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c0ccb60 con 0x7f408c0cf000 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f4089d9b700 1 --2- 172.21.15.5:0/3286314348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c0cf460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f408c06e820 tx=0x7f4074009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341c2a0179a3a1af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f408959a700 1 -- 172.21.15.5:0/3286314348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074011830 con 0x7f408c0cf000 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f408959a700 1 -- 172.21.15.5:0/3286314348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074011990 con 0x7f408c0cf000 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.073+0000 7f408959a700 1 -- 172.21.15.5:0/3286314348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074011d20 con 0x7f408c0cf000 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.074+0000 7f4090ca3700 1 -- 172.21.15.5:0/3286314348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 msgr2=0x7f408c0cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.074+0000 7f4090ca3700 1 --2- 172.21.15.5:0/3286314348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c0cf460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f408c06e820 tx=0x7f4074009b20 comp rx=0 tx=0).stop 2024-05-08T23:21:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.074+0000 7f4090ca3700 1 -- 172.21.15.5:0/3286314348 shutdown_connections 2024-05-08T23:21:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.074+0000 7f4090ca3700 1 --2- 172.21.15.5:0/3286314348 >> v2:172.21.15.49:3300/0 conn(0x7f408c15ef60 0x7f408c0b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.074+0000 7f4090ca3700 1 --2- 172.21.15.5:0/3286314348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c0cf460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.074+0000 7f4090ca3700 1 -- 172.21.15.5:0/3286314348 >> 172.21.15.5:0/3286314348 conn(0x7f408c0b2f20 msgr2=0x7f408c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.074+0000 7f4090ca3700 1 -- 172.21.15.5:0/3286314348 shutdown_connections 2024-05-08T23:21:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.074+0000 7f4090ca3700 1 -- 172.21.15.5:0/3286314348 wait complete. 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.075+0000 7f4090ca3700 1 Processor -- start 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.075+0000 7f4090ca3700 1 -- start start 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f4090ca3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f4090ca3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f408c0cf9a0 0x7f408c0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f4090ca3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f408c15ef60 0x7f408c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f4090ca3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c15b210 con 0x7f408c0cf000 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f4090ca3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c06d980 con 0x7f408c0cf9a0 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f4090ca3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0c48b0 con 0x7f408c15ef60 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f408a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f408a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c15acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41612/0 (socket says 172.21.15.5:41612) 2024-05-08T23:21:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f408a59c700 1 -- 172.21.15.5:0/2525926193 learned_addr learned my addr 172.21.15.5:0/2525926193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f4089d9b700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f408c0cf9a0 0x7f408c0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f408ad9d700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f408c15ef60 0x7f408c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f408a59c700 1 -- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f408c15ef60 msgr2=0x7f408c0b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f408a59c700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f408c15ef60 0x7f408c0b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f408a59c700 1 -- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f408c0cf9a0 msgr2=0x7f408c0b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.076+0000 7f408a59c700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f408c0cf9a0 0x7f408c0b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.077+0000 7f408a59c700 1 -- 172.21.15.5:0/2525926193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40740094a0 con 0x7f408c0cf000 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.077+0000 7f4089d9b700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f408c0cf9a0 0x7f408c0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.077+0000 7f408a59c700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c15acd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f407c00e9b0 tx=0x7f407c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.077+0000 7f4082ffd700 1 -- 172.21.15.5:0/2525926193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c009710 con 0x7f408c0cf000 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.077+0000 7f4082ffd700 1 -- 172.21.15.5:0/2525926193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c00c850 con 0x7f408c0cf000 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.077+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c0c4b90 con 0x7f408c0cf000 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.077+0000 7f4082ffd700 1 -- 172.21.15.5:0/2525926193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00cbc0 con 0x7f408c0cf000 2024-05-08T23:21:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.077+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c0c5150 con 0x7f408c0cf000 2024-05-08T23:21:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.078+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408c0c5370 con 0x7f408c0cf000 2024-05-08T23:21:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.078+0000 7f4082ffd700 1 -- 172.21.15.5:0/2525926193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f407c00cd20 con 0x7f408c0cf000 2024-05-08T23:21:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.079+0000 7f4082ffd700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f406c06ac90 0x7f406c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.079+0000 7f4082ffd700 1 -- 172.21.15.5:0/2525926193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f407c0893a0 con 0x7f408c0cf000 2024-05-08T23:21:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.079+0000 7f4089d9b700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f406c06ac90 0x7f406c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.080+0000 7f4089d9b700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f406c06ac90 0x7f406c06d150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40740090b0 tx=0x7f407401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.084+0000 7f4082ffd700 1 -- 172.21.15.5:0/2525926193 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f407c054610 con 0x7f408c0cf000 2024-05-08T23:21:43.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.305+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f408c068000 con 0x7f408c0cf000 2024-05-08T23:21:43.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.306+0000 7f4082ffd700 1 -- 172.21.15.5:0/2525926193 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7f407c057c30 con 0x7f408c0cf000 2024-05-08T23:21:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:43.306 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f406c06ac90 msgr2=0x7f406c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f406c06ac90 0x7f406c06d150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40740090b0 tx=0x7f407401c040 comp rx=0 tx=0).stop 2024-05-08T23:21:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 msgr2=0x7f408c15acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c15acd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f407c00e9b0 tx=0x7f407c00ed70 comp rx=0 tx=0).stop 2024-05-08T23:21:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 shutdown_connections 2024-05-08T23:21:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f406c06ac90 0x7f406c06d150 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f408c15ef60 0x7f408c0b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f408c0cf000 0x7f408c15acd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 --2- 172.21.15.5:0/2525926193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f408c0cf9a0 0x7f408c0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 >> 172.21.15.5:0/2525926193 conn(0x7f408c0b2f20 msgr2=0x7f408c15ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 shutdown_connections 2024-05-08T23:21:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.308+0000 7f4090ca3700 1 -- 172.21.15.5:0/2525926193 wait complete. 2024-05-08T23:21:43.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:43.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.464+0000 7f2a9197b700 1 Processor -- start 2024-05-08T23:21:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.464+0000 7f2a9197b700 1 -- start start 2024-05-08T23:21:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a9197b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2a8c0d1e50 0x7f2a8c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a9197b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c161320 0x7f2a8c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a9197b700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8c0cf560 con 0x7f2a8c0d2230 2024-05-08T23:21:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a9197b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8c0ce950 con 0x7f2a8c0d1e50 2024-05-08T23:21:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a9197b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cec30 con 0x7f2a8c161320 2024-05-08T23:21:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a8a7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a8c0d2230 0x7f2a8c160c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33690/0 (socket says 172.21.15.5:33690) 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a8a7fc700 1 -- 172.21.15.5:0/1259844549 learned_addr learned my addr 172.21.15.5:0/1259844549 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a8affd700 1 --2- 172.21.15.5:0/1259844549 >> v2:172.21.15.49:3300/0 conn(0x7f2a8c0d1e50 0x7f2a8c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a8b7fe700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c161320 0x7f2a8c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447053054 0 0) 0x7f2a8c0cf560 con 0x7f2a8c0d2230 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a740035b0 con 0x7f2a8c0d2230 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 41899108 0 0) 0x7f2a740035b0 con 0x7f2a8c0d2230 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c161320 msgr2=0x7f2a8c06bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a89ffb700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c161320 0x7f2a8c06bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 >> v2:172.21.15.49:3300/0 conn(0x7f2a8c0d1e50 msgr2=0x7f2a8c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a89ffb700 1 --2- 172.21.15.5:0/1259844549 >> v2:172.21.15.49:3300/0 conn(0x7f2a8c0d1e50 0x7f2a8c0ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.465+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0cef10 con 0x7f2a8c0d2230 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f2a80001670 con 0x7f2a8c0d2230 2024-05-08T23:21:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 >> v1:172.21.15.5:6789/0 conn(0x7f2a8c0d2230 legacy=0x7f2a8c160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a74003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a74004850 0x7f2a74004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c161320 0x7f2a74005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0ce950 con 0x7f2a74003a00 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cec30 con 0x7f2a74004850 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7400ba10 con 0x7f2a8c161320 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8affd700 1 --2- 172.21.15.5:0/1259844549 >> v2:172.21.15.49:3300/0 conn(0x7f2a8c0d1e50 0x7f2a8c0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8affd700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a74003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8a7fc700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a74004850 0x7f2a74004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8b7fe700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c161320 0x7f2a74005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8affd700 1 -- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c161320 msgr2=0x7f2a74005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8affd700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c161320 0x7f2a74005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8affd700 1 -- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a74004850 msgr2=0x7f2a74004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8affd700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a74004850 0x7f2a74004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8affd700 1 -- 172.21.15.5:0/1259844549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a740035b0 con 0x7f2a74003a00 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a8affd700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a74003e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009d10 tx=0x7f2a74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5cc07a154f748d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c009e60 con 0x7f2a74003a00 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c00e9a0 con 0x7f2a74003a00 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.466+0000 7f2a89ffb700 1 -- 172.21.15.5:0/1259844549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00ed10 con 0x7f2a74003a00 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.467+0000 7f2a9197b700 1 -- 172.21.15.5:0/1259844549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 msgr2=0x7f2a74003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.467+0000 7f2a9197b700 1 --2- 172.21.15.5:0/1259844549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a74003e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009d10 tx=0x7f2a74005880 comp rx=0 tx=0).stop 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.467+0000 7f2a8b7fe700 1 -- 172.21.15.5:0/1259844549 reap_dead start 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.467+0000 7f2a9197b700 1 -- 172.21.15.5:0/1259844549 shutdown_connections 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.467+0000 7f2a9197b700 1 -- 172.21.15.5:0/1259844549 >> 172.21.15.5:0/1259844549 conn(0x7f2a8c08afe0 msgr2=0x7f2a8c0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.467+0000 7f2a9197b700 1 -- 172.21.15.5:0/1259844549 shutdown_connections 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.467+0000 7f2a9197b700 1 -- 172.21.15.5:0/1259844549 wait complete. 2024-05-08T23:21:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.468+0000 7f2a9197b700 1 Processor -- start 2024-05-08T23:21:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.469+0000 7f2a9197b700 1 -- start start 2024-05-08T23:21:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.469+0000 7f2a9197b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a8c15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.469+0000 7f2a9197b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c0ccb40 0x7f2a8c0ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.469+0000 7f2a9197b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a8c1579f0 0x7f2a8c157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.469+0000 7f2a9197b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c1f36c0 con 0x7f2a74003a00 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.469+0000 7f2a9197b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cd500 con 0x7f2a8c1579f0 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.469+0000 7f2a9197b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cd720 con 0x7f2a8c0ccb40 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a8c15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a8c15f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41632/0 (socket says 172.21.15.5:41632) 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 -- 172.21.15.5:0/3355128442 learned_addr learned my addr 172.21.15.5:0/3355128442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8b7fe700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a8c1579f0 0x7f2a8c157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8a7fc700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c0ccb40 0x7f2a8c0ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 -- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c0ccb40 msgr2=0x7f2a8c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c0ccb40 0x7f2a8c0ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 -- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a8c1579f0 msgr2=0x7f2a8c157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a8c1579f0 0x7f2a8c157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 -- 172.21.15.5:0/3355128442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c1583b0 con 0x7f2a74003a00 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a8affd700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a8c15f6e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00e790 tx=0x7f2a7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a73fff700 1 -- 172.21.15.5:0/3355128442 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c020070 con 0x7f2a74003a00 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a73fff700 1 -- 172.21.15.5:0/3355128442 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c009e60 con 0x7f2a74003a00 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c1f38d0 con 0x7f2a74003a00 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a73fff700 1 -- 172.21.15.5:0/3355128442 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c003ad0 con 0x7f2a74003a00 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.470+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c1f3b70 con 0x7f2a74003a00 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.471+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c1f44a0 con 0x7f2a74003a00 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.472+0000 7f2a73fff700 1 -- 172.21.15.5:0/3355128442 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2a7c003c30 con 0x7f2a74003a00 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.472+0000 7f2a73fff700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a7806ac90 0x7f2a7806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.472+0000 7f2a73fff700 1 -- 172.21.15.5:0/3355128442 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a7c08a960 con 0x7f2a74003a00 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.473+0000 7f2a8a7fc700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a7806ac90 0x7f2a7806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.473+0000 7f2a8a7fc700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a7806ac90 0x7f2a7806d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c1f7d70 tx=0x7f2a80007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.477+0000 7f2a73fff700 1 -- 172.21.15.5:0/3355128442 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2a7c055cf0 con 0x7f2a74003a00 2024-05-08T23:21:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.693+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a8c074bc0 con 0x7f2a74003a00 2024-05-08T23:21:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.693+0000 7f2a73fff700 1 -- 172.21.15.5:0/3355128442 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a7c059310 con 0x7f2a74003a00 2024-05-08T23:21:43.693 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a7806ac90 msgr2=0x7f2a7806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a7806ac90 0x7f2a7806d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c1f7d70 tx=0x7f2a80007980 comp rx=0 tx=0).stop 2024-05-08T23:21:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 msgr2=0x7f2a8c15f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a8c15f6e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00e790 tx=0x7f2a7c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 shutdown_connections 2024-05-08T23:21:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a7806ac90 0x7f2a7806d150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a8c1579f0 0x7f2a8c157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a74003a00 0x7f2a8c15f6e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 --2- 172.21.15.5:0/3355128442 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a8c0ccb40 0x7f2a8c0ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 >> 172.21.15.5:0/3355128442 conn(0x7f2a8c08afe0 msgr2=0x7f2a8c0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 shutdown_connections 2024-05-08T23:21:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.696+0000 7f2a9197b700 1 -- 172.21.15.5:0/3355128442 wait complete. 2024-05-08T23:21:43.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.849+0000 7ff24455a700 1 Processor -- start 2024-05-08T23:21:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.849+0000 7ff24455a700 1 -- start start 2024-05-08T23:21:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.850+0000 7ff24455a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 0x7ff23c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.850+0000 7ff24455a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c0cb160 con 0x7ff23c0cc590 2024-05-08T23:21:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.850+0000 7ff24455a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c06ad20 con 0x7ff23c15db00 2024-05-08T23:21:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.850+0000 7ff24455a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c15dee0 con 0x7ff23c06f990 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.850+0000 7ff241af5700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff23c0cc590 0x7ff23c06f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33706/0 (socket says 172.21.15.5:33706) 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.850+0000 7ff241af5700 1 -- 172.21.15.5:0/2501959201 learned_addr learned my addr 172.21.15.5:0/2501959201 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff242af7700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 0x7ff23c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706437404 0 0) 0x7ff23c0cb160 con 0x7ff23c0cc590 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2280035b0 con 0x7ff23c0cc590 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021435330 0 0) 0x7ff23c06ad20 con 0x7ff23c15db00 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff23c0cb160 con 0x7ff23c15db00 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3087467484 0 0) 0x7ff2280035b0 con 0x7ff23c0cc590 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 msgr2=0x7ff23c06e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 0x7ff23c06e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 >> v1:172.21.15.49:6789/0 conn(0x7ff23c15db00 legacy=0x7ff23c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c06b000 con 0x7ff23c0cc590 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7ff230001670 con 0x7ff23c0cc590 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff242af7700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 0x7ff23c06e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 >> v1:172.21.15.5:6789/0 conn(0x7ff23c0cc590 legacy=0x7ff23c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228003bc0 0x7ff228003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff228004850 0x7ff228004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 0x7ff228005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c15dee0 con 0x7ff228004850 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22800ba70 con 0x7ff228003bc0 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22800bc30 con 0x7ff23c06f990 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff241af5700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff228004850 0x7ff228004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.851+0000 7ff2422f6700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228003bc0 0x7ff228003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff242af7700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 0x7ff228005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff241af5700 1 -- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 msgr2=0x7ff228005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff241af5700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c06f990 0x7ff228005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff241af5700 1 -- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228003bc0 msgr2=0x7ff228003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff241af5700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228003bc0 0x7ff228003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff241af5700 1 -- 172.21.15.5:0/2501959201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234009a70 con 0x7ff228004850 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff241af5700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff228004850 0x7ff228004cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff230001eb0 tx=0x7ff228005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d32082beaa73d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000cd20 con 0x7ff228004850 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23000ce80 con 0x7ff228004850 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.852+0000 7ff2412f4700 1 -- 172.21.15.5:0/2501959201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230005ba0 con 0x7ff228004850 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.853+0000 7ff24455a700 1 -- 172.21.15.5:0/2501959201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff228004850 msgr2=0x7ff228004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.853+0000 7ff24455a700 1 --2- 172.21.15.5:0/2501959201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff228004850 0x7ff228004cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff230001eb0 tx=0x7ff228005880 comp rx=0 tx=0).stop 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.853+0000 7ff242af7700 1 -- 172.21.15.5:0/2501959201 reap_dead start 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.853+0000 7ff24455a700 1 -- 172.21.15.5:0/2501959201 shutdown_connections 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.853+0000 7ff24455a700 1 -- 172.21.15.5:0/2501959201 >> 172.21.15.5:0/2501959201 conn(0x7ff23c099a50 msgr2=0x7ff23c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.853+0000 7ff24455a700 1 -- 172.21.15.5:0/2501959201 shutdown_connections 2024-05-08T23:21:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.853+0000 7ff24455a700 1 -- 172.21.15.5:0/2501959201 wait complete. 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.854+0000 7ff24455a700 1 Processor -- start 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.854+0000 7ff24455a700 1 -- start start 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff24455a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228004850 0x7ff23c0add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff24455a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff23c0c67d0 0x7ff23c14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff24455a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c0af410 0x7ff23c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff24455a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0ae370 con 0x7ff23c0c67d0 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff24455a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0ae650 con 0x7ff228004850 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff24455a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0ae930 con 0x7ff23c0af410 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff241af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff23c0c67d0 0x7ff23c14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff241af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff23c0c67d0 0x7ff23c14f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41652/0 (socket says 172.21.15.5:41652) 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff241af5700 1 -- 172.21.15.5:0/3528787333 learned_addr learned my addr 172.21.15.5:0/3528787333 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff2422f6700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228004850 0x7ff23c0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff242af7700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c0af410 0x7ff23c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff241af5700 1 -- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c0af410 msgr2=0x7ff23c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff241af5700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c0af410 0x7ff23c15bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff241af5700 1 -- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228004850 msgr2=0x7ff23c0add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff241af5700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228004850 0x7ff23c0add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff241af5700 1 -- 172.21.15.5:0/3528787333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0aec10 con 0x7ff23c0c67d0 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff2422f6700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228004850 0x7ff23c0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.855+0000 7ff242af7700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c0af410 0x7ff23c15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.856+0000 7ff241af5700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff23c0c67d0 0x7ff23c14f7e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff230015040 tx=0x7ff230001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.856+0000 7ff22effd700 1 -- 172.21.15.5:0/3528787333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300072d0 con 0x7ff23c0c67d0 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.856+0000 7ff22effd700 1 -- 172.21.15.5:0/3528787333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230007450 con 0x7ff23c0c67d0 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.856+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c0aeef0 con 0x7ff23c0c67d0 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.856+0000 7ff22effd700 1 -- 172.21.15.5:0/3528787333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230007800 con 0x7ff23c0c67d0 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.856+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c0c6c50 con 0x7ff23c0c67d0 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.857+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c074bc0 con 0x7ff23c0c67d0 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.857+0000 7ff22effd700 1 -- 172.21.15.5:0/3528787333 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff23001e430 con 0x7ff23c0c67d0 2024-05-08T23:21:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.858+0000 7ff22effd700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff22006ac90 0x7ff22006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.858+0000 7ff22effd700 1 -- 172.21.15.5:0/3528787333 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff23008a060 con 0x7ff23c0c67d0 2024-05-08T23:21:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.858+0000 7ff2422f6700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff22006ac90 0x7ff22006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.858+0000 7ff2422f6700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff22006ac90 0x7ff22006d150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff238007fd0 tx=0x7ff238009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:43.862+0000 7ff22effd700 1 -- 172.21.15.5:0/3528787333 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff2300552d0 con 0x7ff23c0c67d0 2024-05-08T23:21:44.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.081+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff23c1e6060 con 0x7ff23c0c67d0 2024-05-08T23:21:44.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.081+0000 7ff22effd700 1 -- 172.21.15.5:0/3528787333 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2300588f0 con 0x7ff23c0c67d0 2024-05-08T23:21:44.081 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff22006ac90 msgr2=0x7ff22006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff22006ac90 0x7ff22006d150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff238007fd0 tx=0x7ff238009040 comp rx=0 tx=0).stop 2024-05-08T23:21:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff23c0c67d0 msgr2=0x7ff23c14f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff23c0c67d0 0x7ff23c14f7e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff230015040 tx=0x7ff230001980 comp rx=0 tx=0).stop 2024-05-08T23:21:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 shutdown_connections 2024-05-08T23:21:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff22006ac90 0x7ff22006d150 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff23c0af410 0x7ff23c15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff228004850 0x7ff23c0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 --2- 172.21.15.5:0/3528787333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff23c0c67d0 0x7ff23c14f7e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 >> 172.21.15.5:0/3528787333 conn(0x7ff23c099a50 msgr2=0x7ff23c0cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 shutdown_connections 2024-05-08T23:21:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.084+0000 7ff24455a700 1 -- 172.21.15.5:0/3528787333 wait complete. 2024-05-08T23:21:44.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:44.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.240+0000 7f66be9f0700 1 Processor -- start 2024-05-08T23:21:44.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.241+0000 7f66be9f0700 1 -- start start 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.241+0000 7f66be9f0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b809a790 0x7f66b8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.241+0000 7f66be9f0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b806d010 con 0x7f66b8099f90 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.241+0000 7f66be9f0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b815de40 con 0x7f66b815da60 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.241+0000 7f66be9f0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b809a420 con 0x7f66b809a790 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66bcf8d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f66b8099f90 0x7f66b80cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33708/0 (socket says 172.21.15.5:33708) 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66bcf8d700 1 -- 172.21.15.5:0/1104524782 learned_addr learned my addr 172.21.15.5:0/1104524782 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b77fe700 1 --2- 172.21.15.5:0/1104524782 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b809a790 0x7f66b8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b6ffd700 1 -- 172.21.15.5:0/1104524782 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195363305 0 0) 0x7f66b806d010 con 0x7f66b8099f90 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b6ffd700 1 -- 172.21.15.5:0/1104524782 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a40035b0 con 0x7f66b8099f90 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b6ffd700 1 -- 172.21.15.5:0/1104524782 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454268017 0 0) 0x7f66b815de40 con 0x7f66b815da60 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b6ffd700 1 -- 172.21.15.5:0/1104524782 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b806d010 con 0x7f66b815da60 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b6ffd700 1 -- 172.21.15.5:0/1104524782 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3092574030 0 0) 0x7f66a40035b0 con 0x7f66b8099f90 2024-05-08T23:21:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b77fe700 1 -- 172.21.15.5:0/1104524782 >> v1:172.21.15.49:6789/0 conn(0x7f66b815da60 legacy=0x7f66b80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b77fe700 1 -- 172.21.15.5:0/1104524782 >> v1:172.21.15.5:6789/0 conn(0x7f66b8099f90 legacy=0x7f66b80cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b77fe700 1 -- 172.21.15.5:0/1104524782 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80d0b20 con 0x7f66b809a790 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.242+0000 7f66b77fe700 1 --2- 172.21.15.5:0/1104524782 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b809a790 0x7f66b8099a50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009870 tx=0x7f66a0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3d3103cdc5fa4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66b6ffd700 1 -- 172.21.15.5:0/1104524782 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00019f0 con 0x7f66b809a790 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66b6ffd700 1 -- 172.21.15.5:0/1104524782 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0004030 con 0x7f66b809a790 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66b6ffd700 1 -- 172.21.15.5:0/1104524782 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a00043c0 con 0x7f66b809a790 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66be9f0700 1 -- 172.21.15.5:0/1104524782 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b809a790 msgr2=0x7f66b8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66be9f0700 1 --2- 172.21.15.5:0/1104524782 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b809a790 0x7f66b8099a50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009870 tx=0x7f66a0009b80 comp rx=0 tx=0).stop 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66be9f0700 1 -- 172.21.15.5:0/1104524782 shutdown_connections 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66be9f0700 1 --2- 172.21.15.5:0/1104524782 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b809a790 0x7f66b8099a50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66be9f0700 1 -- 172.21.15.5:0/1104524782 >> 172.21.15.5:0/1104524782 conn(0x7f66b80b2f60 msgr2=0x7f66b80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66be9f0700 1 -- 172.21.15.5:0/1104524782 shutdown_connections 2024-05-08T23:21:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.243+0000 7f66be9f0700 1 -- 172.21.15.5:0/1104524782 wait complete. 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.245+0000 7f66be9f0700 1 Processor -- start 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.245+0000 7f66be9f0700 1 -- start start 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66be9f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66b8099f90 0x7f66b8159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66be9f0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66b809a790 0x7f66b8160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66be9f0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b815da60 0x7f66b815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66be9f0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b815ffe0 con 0x7f66b809a790 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66be9f0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b815f3d0 con 0x7f66b8099f90 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66be9f0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b815f6b0 con 0x7f66b815da60 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66b809a790 0x7f66b8160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66b809a790 0x7f66b8160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41668/0 (socket says 172.21.15.5:41668) 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 -- 172.21.15.5:0/2570611644 learned_addr learned my addr 172.21.15.5:0/2570611644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b7fff700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66b8099f90 0x7f66b8159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66bcf8d700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b815da60 0x7f66b815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 -- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b815da60 msgr2=0x7f66b815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b815da60 0x7f66b815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 -- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66b8099f90 msgr2=0x7f66b8159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66b8099f90 0x7f66b8159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 -- 172.21.15.5:0/2570611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a00094a0 con 0x7f66b809a790 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b7fff700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66b8099f90 0x7f66b8159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b77fe700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66b809a790 0x7f66b8160640 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097c0 tx=0x7f66a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b4ff9700 1 -- 172.21.15.5:0/2570611644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0004570 con 0x7f66b809a790 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.246+0000 7f66b4ff9700 1 -- 172.21.15.5:0/2570611644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a001f070 con 0x7f66b809a790 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.247+0000 7f66be9f0700 1 -- 172.21.15.5:0/2570611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b815f990 con 0x7f66b809a790 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.247+0000 7f66b4ff9700 1 -- 172.21.15.5:0/2570611644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0011870 con 0x7f66b809a790 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.247+0000 7f66be9f0700 1 -- 172.21.15.5:0/2570611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b8159f60 con 0x7f66b809a790 2024-05-08T23:21:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.248+0000 7f66ae7fc700 1 -- 172.21.15.5:0/2570611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66900052f0 con 0x7f66b809a790 2024-05-08T23:21:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.248+0000 7f66b4ff9700 1 -- 172.21.15.5:0/2570611644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f66a00119d0 con 0x7f66b809a790 2024-05-08T23:21:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.248+0000 7f66b4ff9700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f669806ac90 0x7f669806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.249+0000 7f66b4ff9700 1 -- 172.21.15.5:0/2570611644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66a0021070 con 0x7f66b809a790 2024-05-08T23:21:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.249+0000 7f66b7fff700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f669806ac90 0x7f669806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.249+0000 7f66b7fff700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f669806ac90 0x7f669806d150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66a80011e0 tx=0x7f66a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.253+0000 7f66b4ff9700 1 -- 172.21.15.5:0/2570611644 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f66a0055cc0 con 0x7f66b809a790 2024-05-08T23:21:44.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.469+0000 7f66ae7fc700 1 -- 172.21.15.5:0/2570611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6690005b20 con 0x7f66b809a790 2024-05-08T23:21:44.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.470+0000 7f66b4ff9700 1 -- 172.21.15.5:0/2570611644 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66a002b020 con 0x7f66b809a790 2024-05-08T23:21:44.470 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 -- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f669806ac90 msgr2=0x7f669806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f669806ac90 0x7f669806d150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66a80011e0 tx=0x7f66a8009040 comp rx=0 tx=0).stop 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 -- 172.21.15.5:0/2570611644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66b809a790 msgr2=0x7f66b8160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66b809a790 0x7f66b8160640 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097c0 tx=0x7f66a0004750 comp rx=0 tx=0).stop 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 -- 172.21.15.5:0/2570611644 shutdown_connections 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f669806ac90 0x7f669806d150 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66b815da60 0x7f66b815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66b8099f90 0x7f66b8159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 --2- 172.21.15.5:0/2570611644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66b809a790 0x7f66b8160640 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 -- 172.21.15.5:0/2570611644 >> 172.21.15.5:0/2570611644 conn(0x7f66b80b2f60 msgr2=0x7f66b80cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 -- 172.21.15.5:0/2570611644 shutdown_connections 2024-05-08T23:21:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.472+0000 7f66ae7fc700 1 -- 172.21.15.5:0/2570611644 wait complete. 2024-05-08T23:21:44.486 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:21:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.623+0000 7f16eb7ad700 1 Processor -- start 2024-05-08T23:21:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.623+0000 7f16eb7ad700 1 -- start start 2024-05-08T23:21:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.624+0000 7f16eb7ad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e40c90e0 0x7f16e41612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.624+0000 7f16eb7ad700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16e40c94c0 0x7f16e406b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.624+0000 7f16eb7ad700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e4161800 con 0x7f16e4146a10 2024-05-08T23:21:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.624+0000 7f16eb7ad700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e406a960 con 0x7f16e40c94c0 2024-05-08T23:21:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.624+0000 7f16eb7ad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e4161a80 con 0x7f16e40c90e0 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.624+0000 7f16e8d48700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16e40c94c0 0x7f16e406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.624+0000 7f16e8d48700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16e40c94c0 0x7f16e406b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41680/0 (socket says 172.21.15.5:41680) 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.624+0000 7f16e8d48700 1 -- 172.21.15.5:0/3669769514 learned_addr learned my addr 172.21.15.5:0/3669769514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16e9549700 1 --2- 172.21.15.5:0/3669769514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e40c90e0 0x7f16e41612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16e8d48700 1 -- 172.21.15.5:0/3669769514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e40c90e0 msgr2=0x7f16e41612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16e8d48700 1 --2- 172.21.15.5:0/3669769514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e40c90e0 0x7f16e41612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16e8d48700 1 -- 172.21.15.5:0/3669769514 >> v1:172.21.15.49:6789/0 conn(0x7f16e4146a10 legacy=0x7f16e406b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16e8d48700 1 -- 172.21.15.5:0/3669769514 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e4161d60 con 0x7f16e40c94c0 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16e8d48700 1 --2- 172.21.15.5:0/3669769514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e406b090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16d40098f0 tx=0x7f16d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a443fe3a4852f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16dbfff700 1 -- 172.21.15.5:0/3669769514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4011830 con 0x7f16e40c94c0 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16dbfff700 1 -- 172.21.15.5:0/3669769514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d4011990 con 0x7f16e40c94c0 2024-05-08T23:21:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.625+0000 7f16dbfff700 1 -- 172.21.15.5:0/3669769514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4011d20 con 0x7f16e40c94c0 2024-05-08T23:21:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.626+0000 7f16eb7ad700 1 -- 172.21.15.5:0/3669769514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 msgr2=0x7f16e406b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.626+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/3669769514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e406b090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16d40098f0 tx=0x7f16d4009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.626+0000 7f16eb7ad700 1 -- 172.21.15.5:0/3669769514 shutdown_connections 2024-05-08T23:21:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.626+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/3669769514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e40c90e0 0x7f16e41612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.626+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/3669769514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e406b090 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.626+0000 7f16eb7ad700 1 -- 172.21.15.5:0/3669769514 >> 172.21.15.5:0/3669769514 conn(0x7f16e40cc580 msgr2=0x7f16e40cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.626+0000 7f16eb7ad700 1 -- 172.21.15.5:0/3669769514 shutdown_connections 2024-05-08T23:21:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.626+0000 7f16eb7ad700 1 -- 172.21.15.5:0/3669769514 wait complete. 2024-05-08T23:21:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.627+0000 7f16eb7ad700 1 Processor -- start 2024-05-08T23:21:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.627+0000 7f16eb7ad700 1 -- start start 2024-05-08T23:21:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16eb7ad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c90e0 0x7f16e413c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16eb7ad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e40b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16eb7ad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e4146a10 0x7f16e413dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16eb7ad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e413cb30 con 0x7f16e40c94c0 2024-05-08T23:21:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16eb7ad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40b2e20 con 0x7f16e40c90e0 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16eb7ad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e406d690 con 0x7f16e4146a10 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e8d48700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e40b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e8d48700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e40b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41694/0 (socket says 172.21.15.5:41694) 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e8d48700 1 -- 172.21.15.5:0/112611816 learned_addr learned my addr 172.21.15.5:0/112611816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e9549700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c90e0 0x7f16e413c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e9d4a700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e4146a10 0x7f16e413dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e8d48700 1 -- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e4146a10 msgr2=0x7f16e413dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e8d48700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e4146a10 0x7f16e413dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e8d48700 1 -- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c90e0 msgr2=0x7f16e413c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.628+0000 7f16e8d48700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c90e0 0x7f16e413c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.629+0000 7f16e8d48700 1 -- 172.21.15.5:0/112611816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4009550 con 0x7f16e40c94c0 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.629+0000 7f16e8d48700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e40b28e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16e40cd2d0 tx=0x7f16d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.629+0000 7f16d9ffb700 1 -- 172.21.15.5:0/112611816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4004550 con 0x7f16e40c94c0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.629+0000 7f16d9ffb700 1 -- 172.21.15.5:0/112611816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d401f070 con 0x7f16e40c94c0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.629+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e406d970 con 0x7f16e40c94c0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.629+0000 7f16d9ffb700 1 -- 172.21.15.5:0/112611816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d40038e0 con 0x7f16e40c94c0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.629+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e406df30 con 0x7f16e40c94c0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.630+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e4068000 con 0x7f16e40c94c0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.631+0000 7f16d9ffb700 1 -- 172.21.15.5:0/112611816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f16d4003a40 con 0x7f16e40c94c0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.631+0000 7f16d9ffb700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c806ac90 0x7f16c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.631+0000 7f16d9ffb700 1 -- 172.21.15.5:0/112611816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16d4021070 con 0x7f16e40c94c0 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.631+0000 7f16e9549700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c806ac90 0x7f16c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.632+0000 7f16e9549700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c806ac90 0x7f16c806d150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16e41406c0 tx=0x7f16e0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.636+0000 7f16d9ffb700 1 -- 172.21.15.5:0/112611816 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f16d4055d40 con 0x7f16e40c94c0 2024-05-08T23:21:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.870+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f16e406fa20 con 0x7f16e40c94c0 2024-05-08T23:21:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.870+0000 7f16d9ffb700 1 -- 172.21.15.5:0/112611816 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f16d4015070 con 0x7f16e40c94c0 2024-05-08T23:21:44.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c806ac90 msgr2=0x7f16c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c806ac90 0x7f16c806d150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16e41406c0 tx=0x7f16e0009210 comp rx=0 tx=0).stop 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 msgr2=0x7f16e40b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e40b28e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16e40cd2d0 tx=0x7f16d4004750 comp rx=0 tx=0).stop 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 shutdown_connections 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c806ac90 0x7f16c806d150 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16e4146a10 0x7f16e413dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c90e0 0x7f16e413c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 --2- 172.21.15.5:0/112611816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e40c94c0 0x7f16e40b28e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 >> 172.21.15.5:0/112611816 conn(0x7f16e40cc580 msgr2=0x7f16e40ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 shutdown_connections 2024-05-08T23:21:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:44.873+0000 7f16eb7ad700 1 -- 172.21.15.5:0/112611816 wait complete. 2024-05-08T23:21:44.875 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:21:44.887 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:21:44.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:44.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-08T23:21:44.933 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:21:44.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:44.934 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:21:44.966 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:21:44.966 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:44.989 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:21:44.989 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:45.050 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:21:45.050 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:45.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.259+0000 7f69a4153700 1 Processor -- start 2024-05-08T23:21:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.259+0000 7f69a4153700 1 -- start start 2024-05-08T23:21:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a4153700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f699c15d7a0 0x7f699c161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a4153700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f699c0d3f40 0x7f699c0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a4153700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c162450 con 0x7f699c15db80 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a4153700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699c0cee00 con 0x7f699c0d3f40 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a4153700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699c0b27e0 con 0x7f699c15d7a0 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a16ee700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f699c0d3f40 0x7f699c0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a26f0700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f699c15db80 0x7f699c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33722/0 (socket says 172.21.15.5:33722) 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a26f0700 1 -- 172.21.15.5:0/4018014664 learned_addr learned my addr 172.21.15.5:0/4018014664 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a1eef700 1 --2- 172.21.15.5:0/4018014664 >> v2:172.21.15.49:3300/0 conn(0x7f699c15d7a0 0x7f699c161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a0eed700 1 -- 172.21.15.5:0/4018014664 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735470023 0 0) 0x7f6994001490 con 0x7f699c15db80 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a0eed700 1 -- 172.21.15.5:0/4018014664 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69880035b0 con 0x7f699c15db80 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a16ee700 1 -- 172.21.15.5:0/4018014664 >> v2:172.21.15.49:3300/0 conn(0x7f699c15d7a0 msgr2=0x7f699c161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a16ee700 1 --2- 172.21.15.5:0/4018014664 >> v2:172.21.15.49:3300/0 conn(0x7f699c15d7a0 0x7f699c161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a16ee700 1 -- 172.21.15.5:0/4018014664 >> v1:172.21.15.5:6789/0 conn(0x7f699c15db80 legacy=0x7f699c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a16ee700 1 -- 172.21.15.5:0/4018014664 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0d4400 con 0x7f699c0d3f40 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.260+0000 7f69a16ee700 1 --2- 172.21.15.5:0/4018014664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c0d3f40 0x7f699c0b0190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f699c0b61f0 tx=0x7f698c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de49a88c728e129d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a0eed700 1 -- 172.21.15.5:0/4018014664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c005e30 con 0x7f699c0d3f40 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a0eed700 1 -- 172.21.15.5:0/4018014664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c006030 con 0x7f699c0d3f40 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a0eed700 1 -- 172.21.15.5:0/4018014664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c0063c0 con 0x7f699c0d3f40 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a4153700 1 -- 172.21.15.5:0/4018014664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c0d3f40 msgr2=0x7f699c0b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a4153700 1 --2- 172.21.15.5:0/4018014664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c0d3f40 0x7f699c0b0190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f699c0b61f0 tx=0x7f698c009b00 comp rx=0 tx=0).stop 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a4153700 1 -- 172.21.15.5:0/4018014664 shutdown_connections 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a4153700 1 --2- 172.21.15.5:0/4018014664 >> v2:172.21.15.49:3300/0 conn(0x7f699c15d7a0 0x7f699c161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a4153700 1 --2- 172.21.15.5:0/4018014664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c0d3f40 0x7f699c0b0190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a4153700 1 -- 172.21.15.5:0/4018014664 >> 172.21.15.5:0/4018014664 conn(0x7f699c08afe0 msgr2=0x7f699c0d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a4153700 1 -- 172.21.15.5:0/4018014664 shutdown_connections 2024-05-08T23:21:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.261+0000 7f69a4153700 1 -- 172.21.15.5:0/4018014664 wait complete. 2024-05-08T23:21:45.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.263+0000 7f69a4153700 1 Processor -- start 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.263+0000 7f69a4153700 1 -- start start 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a4153700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699c0d3f40 0x7f699c0b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a4153700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c15d7a0 0x7f699c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a4153700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699c15db80 0x7f699c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a4153700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0c8f70 con 0x7f699c15d7a0 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a4153700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0c9250 con 0x7f699c0d3f40 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a4153700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0c9530 con 0x7f699c15db80 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c15d7a0 0x7f699c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c15d7a0 0x7f699c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41712/0 (socket says 172.21.15.5:41712) 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 -- 172.21.15.5:0/3765970963 learned_addr learned my addr 172.21.15.5:0/3765970963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a1eef700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699c0d3f40 0x7f699c0b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a26f0700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699c15db80 0x7f699c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 -- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699c15db80 msgr2=0x7f699c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699c15db80 0x7f699c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 -- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699c0d3f40 msgr2=0x7f699c0b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699c0d3f40 0x7f699c0b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 -- 172.21.15.5:0/3765970963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c009430 con 0x7f699c15d7a0 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a26f0700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699c15db80 0x7f699c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a16ee700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c15d7a0 0x7f699c0cc8c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69927fc700 1 -- 172.21.15.5:0/3765970963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c006970 con 0x7f699c15d7a0 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69927fc700 1 -- 172.21.15.5:0/3765970963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c020070 con 0x7f699c15d7a0 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69927fc700 1 -- 172.21.15.5:0/3765970963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c004260 con 0x7f699c15d7a0 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c0c9810 con 0x7f699c15d7a0 2024-05-08T23:21:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.264+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c15aed0 con 0x7f699c15d7a0 2024-05-08T23:21:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.265+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c0c9cd0 con 0x7f699c15d7a0 2024-05-08T23:21:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.266+0000 7f69927fc700 1 -- 172.21.15.5:0/3765970963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f698c022030 con 0x7f699c15d7a0 2024-05-08T23:21:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.266+0000 7f69927fc700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f698006ac90 0x7f698006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.266+0000 7f69927fc700 1 -- 172.21.15.5:0/3765970963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f698c08a850 con 0x7f699c15d7a0 2024-05-08T23:21:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.267+0000 7f69a1eef700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f698006ac90 0x7f698006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.267+0000 7f69a1eef700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f698006ac90 0x7f698006d150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f699c0cd5a0 tx=0x7f6998008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.272+0000 7f69927fc700 1 -- 172.21.15.5:0/3765970963 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f698c055a20 con 0x7f699c15d7a0 2024-05-08T23:21:45.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.488+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f699c0cf920 con 0x7f699c15d7a0 2024-05-08T23:21:45.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.489+0000 7f69927fc700 1 -- 172.21.15.5:0/3765970963 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7f698c059040 con 0x7f699c15d7a0 2024-05-08T23:21:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:45.489 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f698006ac90 msgr2=0x7f698006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f698006ac90 0x7f698006d150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f699c0cd5a0 tx=0x7f6998008040 comp rx=0 tx=0).stop 2024-05-08T23:21:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c15d7a0 msgr2=0x7f699c0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c15d7a0 0x7f699c0cc8c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c004750 comp rx=0 tx=0).stop 2024-05-08T23:21:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 shutdown_connections 2024-05-08T23:21:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f698006ac90 0x7f698006d150 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699c15db80 0x7f699c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699c0d3f40 0x7f699c0b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 --2- 172.21.15.5:0/3765970963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699c15d7a0 0x7f699c0cc8c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 >> 172.21.15.5:0/3765970963 conn(0x7f699c08afe0 msgr2=0x7f699c0d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 shutdown_connections 2024-05-08T23:21:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.491+0000 7f69a4153700 1 -- 172.21.15.5:0/3765970963 wait complete. 2024-05-08T23:21:45.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:45.505 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.645+0000 7f6a894b5700 1 Processor -- start 2024-05-08T23:21:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.645+0000 7f6a894b5700 1 -- start start 2024-05-08T23:21:45.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.645+0000 7f6a894b5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a8415a7c0 0x7f6a840cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.645+0000 7f6a894b5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a840ca130 0x7f6a840b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.645+0000 7f6a894b5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a840cd7c0 con 0x7f6a840727b0 2024-05-08T23:21:45.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.645+0000 7f6a894b5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a840cd1b0 con 0x7f6a840ca130 2024-05-08T23:21:45.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.645+0000 7f6a894b5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8415aba0 con 0x7f6a8415a7c0 2024-05-08T23:21:45.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a837fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a840ca130 0x7f6a840b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a837fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a840ca130 0x7f6a840b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41728/0 (socket says 172.21.15.5:41728) 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a837fe700 1 -- 172.21.15.5:0/3598319008 learned_addr learned my addr 172.21.15.5:0/3598319008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a82ffd700 1 --2- 172.21.15.5:0/3598319008 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a8415a7c0 0x7f6a840cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a837fe700 1 -- 172.21.15.5:0/3598319008 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a8415a7c0 msgr2=0x7f6a840cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a837fe700 1 --2- 172.21.15.5:0/3598319008 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a8415a7c0 0x7f6a840cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a837fe700 1 -- 172.21.15.5:0/3598319008 >> v1:172.21.15.49:6789/0 conn(0x7f6a840727b0 legacy=0x7f6a840c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a837fe700 1 -- 172.21.15.5:0/3598319008 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a840b3e70 con 0x7f6a840ca130 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a837fe700 1 --2- 172.21.15.5:0/3598319008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a840ca130 0x7f6a840b24f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00d150 tx=0x7f6a7c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6504a8a495a69cea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a81ffb700 1 -- 172.21.15.5:0/3598319008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c010070 con 0x7f6a840ca130 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a81ffb700 1 -- 172.21.15.5:0/3598319008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c00dd10 con 0x7f6a840ca130 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.646+0000 7f6a81ffb700 1 -- 172.21.15.5:0/3598319008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00b310 con 0x7f6a840ca130 2024-05-08T23:21:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.647+0000 7f6a894b5700 1 -- 172.21.15.5:0/3598319008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a840ca130 msgr2=0x7f6a840b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.647+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3598319008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a840ca130 0x7f6a840b24f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00d150 tx=0x7f6a7c00d460 comp rx=0 tx=0).stop 2024-05-08T23:21:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.647+0000 7f6a894b5700 1 -- 172.21.15.5:0/3598319008 shutdown_connections 2024-05-08T23:21:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.647+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3598319008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a840ca130 0x7f6a840b24f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.647+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3598319008 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a8415a7c0 0x7f6a840cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.647+0000 7f6a894b5700 1 -- 172.21.15.5:0/3598319008 >> 172.21.15.5:0/3598319008 conn(0x7f6a8406a960 msgr2=0x7f6a8406ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.647+0000 7f6a894b5700 1 -- 172.21.15.5:0/3598319008 shutdown_connections 2024-05-08T23:21:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.647+0000 7f6a894b5700 1 -- 172.21.15.5:0/3598319008 wait complete. 2024-05-08T23:21:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.649+0000 7f6a894b5700 1 Processor -- start 2024-05-08T23:21:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.649+0000 7f6a894b5700 1 -- start start 2024-05-08T23:21:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.649+0000 7f6a894b5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a840727b0 0x7f6a841496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.649+0000 7f6a894b5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a840ca130 0x7f6a841558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.649+0000 7f6a894b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a8415a7c0 0x7f6a84155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.649+0000 7f6a894b5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a841570a0 con 0x7f6a8415a7c0 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.649+0000 7f6a894b5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a84156490 con 0x7f6a840ca130 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.649+0000 7f6a894b5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a84156770 con 0x7f6a840727b0 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a8415a7c0 0x7f6a84155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a8415a7c0 0x7f6a84155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41736/0 (socket says 172.21.15.5:41736) 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 -- 172.21.15.5:0/3960602217 learned_addr learned my addr 172.21.15.5:0/3960602217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a82ffd700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a840727b0 0x7f6a841496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a827fc700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a840ca130 0x7f6a841558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 -- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a840727b0 msgr2=0x7f6a841496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a840727b0 0x7f6a841496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 -- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a840ca130 msgr2=0x7f6a841558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a840ca130 0x7f6a841558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 -- 172.21.15.5:0/3960602217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c00bd40 con 0x7f6a8415a7c0 2024-05-08T23:21:45.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a837fe700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a8415a7c0 0x7f6a84155e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c014000 tx=0x7f6a7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a6b7fe700 1 -- 172.21.15.5:0/3960602217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c009a10 con 0x7f6a8415a7c0 2024-05-08T23:21:45.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a6b7fe700 1 -- 172.21.15.5:0/3960602217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c009bb0 con 0x7f6a8415a7c0 2024-05-08T23:21:45.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a84156a50 con 0x7f6a8415a7c0 2024-05-08T23:21:45.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a6b7fe700 1 -- 172.21.15.5:0/3960602217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00b4c0 con 0x7f6a8415a7c0 2024-05-08T23:21:45.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.650+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a841509e0 con 0x7f6a8415a7c0 2024-05-08T23:21:45.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.651+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a84074bc0 con 0x7f6a8415a7c0 2024-05-08T23:21:45.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.652+0000 7f6a6b7fe700 1 -- 172.21.15.5:0/3960602217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6a7c004040 con 0x7f6a8415a7c0 2024-05-08T23:21:45.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.652+0000 7f6a6b7fe700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a7006acc0 0x7f6a7006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.652+0000 7f6a6b7fe700 1 -- 172.21.15.5:0/3960602217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a7c08a8a0 con 0x7f6a8415a7c0 2024-05-08T23:21:45.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.652+0000 7f6a82ffd700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a7006acc0 0x7f6a7006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.653+0000 7f6a82ffd700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a7006acc0 0x7f6a7006d180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009b40 tx=0x7f6a74009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.657+0000 7f6a6b7fe700 1 -- 172.21.15.5:0/3960602217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6a7c055a90 con 0x7f6a8415a7c0 2024-05-08T23:21:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.871+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a84067b90 con 0x7f6a8415a7c0 2024-05-08T23:21:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.871+0000 7f6a6b7fe700 1 -- 172.21.15.5:0/3960602217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a7c0590b0 con 0x7f6a8415a7c0 2024-05-08T23:21:45.871 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a7006acc0 msgr2=0x7f6a7006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a7006acc0 0x7f6a7006d180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009b40 tx=0x7f6a74009210 comp rx=0 tx=0).stop 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a8415a7c0 msgr2=0x7f6a84155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a8415a7c0 0x7f6a84155e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c014000 tx=0x7f6a7c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 shutdown_connections 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6a7006acc0 0x7f6a7006d180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a8415a7c0 0x7f6a84155e10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a840727b0 0x7f6a841496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 --2- 172.21.15.5:0/3960602217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a840ca130 0x7f6a841558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 >> 172.21.15.5:0/3960602217 conn(0x7f6a8406a960 msgr2=0x7f6a84157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.873+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 shutdown_connections 2024-05-08T23:21:45.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:45.874+0000 7f6a894b5700 1 -- 172.21.15.5:0/3960602217 wait complete. 2024-05-08T23:21:45.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:46.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.026+0000 7f6f8e66e700 1 Processor -- start 2024-05-08T23:21:46.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.027+0000 7f6f8e66e700 1 -- start start 2024-05-08T23:21:46.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.027+0000 7f6f8e66e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f8815db00 0x7f6f880cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.027+0000 7f6f8e66e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8806ad20 con 0x7f6f880caa50 2024-05-08T23:21:46.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.027+0000 7f6f8e66e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8806b000 con 0x7f6f8806f990 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.027+0000 7f6f8e66e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8815dee0 con 0x7f6f8815db00 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f877fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6f880caa50 0x7f6f8806f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33730/0 (socket says 172.21.15.5:33730) 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f877fe700 1 -- 172.21.15.5:0/460357328 learned_addr learned my addr 172.21.15.5:0/460357328 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728008497 0 0) 0x7f6f8806ad20 con 0x7f6f880caa50 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f87fff700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f8815db00 0x7f6f880cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f700035b0 con 0x7f6f880caa50 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758983403 0 0) 0x7f6f700035b0 con 0x7f6f880caa50 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f8815db00 msgr2=0x7f6f880cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f8815db00 0x7f6f880cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 >> v1:172.21.15.49:6789/0 conn(0x7f6f8806f990 legacy=0x7f6f8806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8806b2e0 con 0x7f6f880caa50 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f6f7c001670 con 0x7f6f880caa50 2024-05-08T23:21:46.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 >> v1:172.21.15.5:6789/0 conn(0x7f6f880caa50 legacy=0x7f6f8806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f70003a30 0x7f6f70003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f700048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f8815db00 0x7f6f70005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8815dee0 con 0x7f6f700043e0 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7000ba10 con 0x7f6f70003a30 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7000bbd0 con 0x7f6f8815db00 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f877fe700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f700048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f87fff700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f70003a30 0x7f6f70003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.028+0000 7f6f8cc0b700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f8815db00 0x7f6f70005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f877fe700 1 -- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f8815db00 msgr2=0x7f6f70005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f877fe700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f8815db00 0x7f6f70005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f877fe700 1 -- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f70003a30 msgr2=0x7f6f70003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f877fe700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f70003a30 0x7f6f70003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f877fe700 1 -- 172.21.15.5:0/460357328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f700035b0 con 0x7f6f700043e0 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f87fff700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f70003a30 0x7f6f70003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f877fe700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f700048b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c2f0 tx=0x7f6f70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede5d687613b1fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00ce30 con 0x7f6f700043e0 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c005960 con 0x7f6f700043e0 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.029+0000 7f6f86ffd700 1 -- 172.21.15.5:0/460357328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c005d10 con 0x7f6f700043e0 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.030+0000 7f6f8e66e700 1 -- 172.21.15.5:0/460357328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 msgr2=0x7f6f700048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.030+0000 7f6f8e66e700 1 --2- 172.21.15.5:0/460357328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f700048b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c2f0 tx=0x7f6f70005880 comp rx=0 tx=0).stop 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.030+0000 7f6f8cc0b700 1 -- 172.21.15.5:0/460357328 reap_dead start 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.030+0000 7f6f8e66e700 1 -- 172.21.15.5:0/460357328 shutdown_connections 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.030+0000 7f6f8e66e700 1 -- 172.21.15.5:0/460357328 >> 172.21.15.5:0/460357328 conn(0x7f6f88099a50 msgr2=0x7f6f880d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.030+0000 7f6f8e66e700 1 -- 172.21.15.5:0/460357328 shutdown_connections 2024-05-08T23:21:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.030+0000 7f6f8e66e700 1 -- 172.21.15.5:0/460357328 wait complete. 2024-05-08T23:21:46.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.031+0000 7f6f8e66e700 1 Processor -- start 2024-05-08T23:21:46.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.031+0000 7f6f8e66e700 1 -- start start 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f8e66e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f880b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f8e66e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f880b2fe0 0x7f6f880b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f8e66e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f881f64b0 0x7f6f880c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f8e66e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b3470 con 0x7f6f700043e0 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f8e66e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b3750 con 0x7f6f881f64b0 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f8e66e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b3a30 con 0x7f6f880b2fe0 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f87fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f880b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f87fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f880b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41746/0 (socket says 172.21.15.5:41746) 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f87fff700 1 -- 172.21.15.5:0/4040423107 learned_addr learned my addr 172.21.15.5:0/4040423107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f8cc0b700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f881f64b0 0x7f6f880c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f877fe700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f880b2fe0 0x7f6f880b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f87fff700 1 -- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f880b2fe0 msgr2=0x7f6f880b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f87fff700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f880b2fe0 0x7f6f880b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.032+0000 7f6f87fff700 1 -- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f881f64b0 msgr2=0x7f6f880c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f87fff700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f881f64b0 0x7f6f880c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f87fff700 1 -- 172.21.15.5:0/4040423107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880b3d10 con 0x7f6f700043e0 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f8cc0b700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f881f64b0 0x7f6f880c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f877fe700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f880b2fe0 0x7f6f880b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f87fff700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f880b4990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6f78011ca0 tx=0x7f6f7800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f84ff9700 1 -- 172.21.15.5:0/4040423107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78011e40 con 0x7f6f700043e0 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f84ff9700 1 -- 172.21.15.5:0/4040423107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7800fd40 con 0x7f6f700043e0 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f880b3ff0 con 0x7f6f700043e0 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f84ff9700 1 -- 172.21.15.5:0/4040423107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78004260 con 0x7f6f700043e0 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.033+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f880c6290 con 0x7f6f700043e0 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.034+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f88074bc0 con 0x7f6f700043e0 2024-05-08T23:21:46.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.035+0000 7f6f84ff9700 1 -- 172.21.15.5:0/4040423107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6f780043c0 con 0x7f6f700043e0 2024-05-08T23:21:46.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.035+0000 7f6f84ff9700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f7406ac90 0x7f6f7406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.035+0000 7f6f84ff9700 1 -- 172.21.15.5:0/4040423107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f780898f0 con 0x7f6f700043e0 2024-05-08T23:21:46.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.035+0000 7f6f877fe700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f7406ac90 0x7f6f7406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.036+0000 7f6f877fe700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f7406ac90 0x7f6f7406d150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6f880ca940 tx=0x7f6f7c007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.040+0000 7f6f84ff9700 1 -- 172.21.15.5:0/4040423107 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6f78054b60 con 0x7f6f700043e0 2024-05-08T23:21:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.255+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f880cbf20 con 0x7f6f700043e0 2024-05-08T23:21:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.256+0000 7f6f84ff9700 1 -- 172.21.15.5:0/4040423107 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f78022020 con 0x7f6f700043e0 2024-05-08T23:21:46.256 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f7406ac90 msgr2=0x7f6f7406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f7406ac90 0x7f6f7406d150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6f880ca940 tx=0x7f6f7c007000 comp rx=0 tx=0).stop 2024-05-08T23:21:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 msgr2=0x7f6f880b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f880b4990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6f78011ca0 tx=0x7f6f7800f520 comp rx=0 tx=0).stop 2024-05-08T23:21:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 shutdown_connections 2024-05-08T23:21:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f7406ac90 0x7f6f7406d150 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f881f64b0 0x7f6f880c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f700043e0 0x7f6f880b4990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 --2- 172.21.15.5:0/4040423107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f880b2fe0 0x7f6f880b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 >> 172.21.15.5:0/4040423107 conn(0x7f6f88099a50 msgr2=0x7f6f880ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 shutdown_connections 2024-05-08T23:21:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.258+0000 7f6f8e66e700 1 -- 172.21.15.5:0/4040423107 wait complete. 2024-05-08T23:21:46.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:46.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.410+0000 7fd7d1d92700 1 Processor -- start 2024-05-08T23:21:46.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.410+0000 7fd7d1d92700 1 -- start start 2024-05-08T23:21:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.411+0000 7fd7d1d92700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc152680 0x7fd7cc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.411+0000 7fd7d1d92700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd7cc15c900 0x7fd7cc0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.411+0000 7fd7d1d92700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc06f100 con 0x7fd7cc06b970 2024-05-08T23:21:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.411+0000 7fd7d1d92700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc15bc10 con 0x7fd7cc15c900 2024-05-08T23:21:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.411+0000 7fd7d1d92700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc152a60 con 0x7fd7cc152680 2024-05-08T23:21:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc152680 0x7fd7cc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7caffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd7cc15c900 0x7fd7cc0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc152680 0x7fd7cc0b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:48544/0 (socket says 172.21.15.5:48544) 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7cb7fe700 1 -- 172.21.15.5:0/3626403027 learned_addr learned my addr 172.21.15.5:0/3626403027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7cb7fe700 1 -- 172.21.15.5:0/3626403027 >> v2:172.21.15.49:3300/0 conn(0x7fd7cc15c900 msgr2=0x7fd7cc0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7cb7fe700 1 --2- 172.21.15.5:0/3626403027 >> v2:172.21.15.49:3300/0 conn(0x7fd7cc15c900 0x7fd7cc0b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7cb7fe700 1 -- 172.21.15.5:0/3626403027 >> v1:172.21.15.49:6789/0 conn(0x7fd7cc06b970 legacy=0x7fd7cc06d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7cb7fe700 1 -- 172.21.15.5:0/3626403027 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0b5aa0 con 0x7fd7cc152680 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.412+0000 7fd7cb7fe700 1 --2- 172.21.15.5:0/3626403027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc152680 0x7fd7cc0b4f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098f0 tx=0x7fd7bc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f9efcd11f54bb8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7ca7fc700 1 -- 172.21.15.5:0/3626403027 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc011830 con 0x7fd7cc152680 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7ca7fc700 1 -- 172.21.15.5:0/3626403027 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc011990 con 0x7fd7cc152680 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7ca7fc700 1 -- 172.21.15.5:0/3626403027 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc011d20 con 0x7fd7cc152680 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3626403027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc152680 msgr2=0x7fd7cc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3626403027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc152680 0x7fd7cc0b4f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098f0 tx=0x7fd7bc009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:46.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3626403027 shutdown_connections 2024-05-08T23:21:46.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3626403027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc152680 0x7fd7cc0b4f30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3626403027 >> v2:172.21.15.49:3300/0 conn(0x7fd7cc15c900 0x7fd7cc0b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3626403027 >> 172.21.15.5:0/3626403027 conn(0x7fd7cc071400 msgr2=0x7fd7cc071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:46.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3626403027 shutdown_connections 2024-05-08T23:21:46.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.413+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3626403027 wait complete. 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.414+0000 7fd7d1d92700 1 Processor -- start 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.415+0000 7fd7d1d92700 1 -- start start 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.415+0000 7fd7d1d92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7cc06b970 0x7fd7cc150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.415+0000 7fd7d1d92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7cc152680 0x7fd7cc0aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.415+0000 7fd7d1d92700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc15c900 0x7fd7cc0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.415+0000 7fd7d1d92700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc151380 con 0x7fd7cc06b970 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.415+0000 7fd7d1d92700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0a9dd0 con 0x7fd7cc152680 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.415+0000 7fd7d1d92700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc15d970 con 0x7fd7cc15c900 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7cc06b970 0x7fd7cc150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7cc06b970 0x7fd7cc150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41758/0 (socket says 172.21.15.5:41758) 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 -- 172.21.15.5:0/3394175654 learned_addr learned my addr 172.21.15.5:0/3394175654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7caffd700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7cc152680 0x7fd7cc0aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cbfff700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc15c900 0x7fd7cc0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 -- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc15c900 msgr2=0x7fd7cc0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc15c900 0x7fd7cc0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 -- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7cc152680 msgr2=0x7fd7cc0aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7cc152680 0x7fd7cc0aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 -- 172.21.15.5:0/3394175654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc009550 con 0x7fd7cc06b970 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cbfff700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc15c900 0x7fd7cc0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7cb7fe700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7cc06b970 0x7fd7cc150e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc000c00 tx=0x7fd7bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7c3fff700 1 -- 172.21.15.5:0/3394175654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc004480 con 0x7fd7cc06b970 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7c3fff700 1 -- 172.21.15.5:0/3394175654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc01f070 con 0x7fd7cc06b970 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc15dc50 con 0x7fd7cc06b970 2024-05-08T23:21:46.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7c3fff700 1 -- 172.21.15.5:0/3394175654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0038c0 con 0x7fd7cc06b970 2024-05-08T23:21:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.416+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc15e210 con 0x7fd7cc06b970 2024-05-08T23:21:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.417+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc074bc0 con 0x7fd7cc06b970 2024-05-08T23:21:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.418+0000 7fd7c3fff700 1 -- 172.21.15.5:0/3394175654 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd7bc021030 con 0x7fd7cc06b970 2024-05-08T23:21:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.418+0000 7fd7c3fff700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7ac06ac90 0x7fd7ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.419+0000 7fd7c3fff700 1 -- 172.21.15.5:0/3394175654 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7bc08a7b0 con 0x7fd7cc06b970 2024-05-08T23:21:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.419+0000 7fd7caffd700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7ac06ac90 0x7fd7ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.419+0000 7fd7caffd700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7ac06ac90 0x7fd7ac06d150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc06ebc0 tx=0x7fd7b4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.423+0000 7fd7c3fff700 1 -- 172.21.15.5:0/3394175654 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd7bc055aa0 con 0x7fd7cc06b970 2024-05-08T23:21:46.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.635+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7cc071ff0 con 0x7fd7cc06b970 2024-05-08T23:21:46.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.635+0000 7fd7c3fff700 1 -- 172.21.15.5:0/3394175654 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7bc0150a0 con 0x7fd7cc06b970 2024-05-08T23:21:46.636 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7ac06ac90 msgr2=0x7fd7ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7ac06ac90 0x7fd7ac06d150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc06ebc0 tx=0x7fd7b4009210 comp rx=0 tx=0).stop 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7cc06b970 msgr2=0x7fd7cc150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7cc06b970 0x7fd7cc150e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc000c00 tx=0x7fd7bc004750 comp rx=0 tx=0).stop 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 shutdown_connections 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7ac06ac90 0x7fd7ac06d150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7cc15c900 0x7fd7cc0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7cc06b970 0x7fd7cc150e40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 --2- 172.21.15.5:0/3394175654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7cc152680 0x7fd7cc0aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.637+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 >> 172.21.15.5:0/3394175654 conn(0x7fd7cc071400 msgr2=0x7fd7cc0b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.638+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 shutdown_connections 2024-05-08T23:21:46.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.638+0000 7fd7d1d92700 1 -- 172.21.15.5:0/3394175654 wait complete. 2024-05-08T23:21:46.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:21:46.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.788+0000 7fd92169b700 1 Processor -- start 2024-05-08T23:21:46.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.788+0000 7fd92169b700 1 -- start start 2024-05-08T23:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd92169b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd91c160f00 0x7fd91c0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd92169b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd91c0b2f60 0x7fd91c0d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd92169b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0d2100 0x7fd91c161610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd92169b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91c0cdc50 con 0x7fd91c0b2f60 2024-05-08T23:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd92169b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91c0d2590 con 0x7fd91c160f00 2024-05-08T23:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd92169b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0ce3f0 con 0x7fd91c0d2100 2024-05-08T23:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd91a7fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd91c0b2f60 0x7fd91c0d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd91a7fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd91c0b2f60 0x7fd91c0d1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41768/0 (socket says 172.21.15.5:41768) 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd91a7fc700 1 -- 172.21.15.5:0/3505352115 learned_addr learned my addr 172.21.15.5:0/3505352115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd91affd700 1 --2- 172.21.15.5:0/3505352115 >> v2:172.21.15.49:3300/0 conn(0x7fd91c160f00 0x7fd91c0b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.789+0000 7fd91b7fe700 1 --2- 172.21.15.5:0/3505352115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0d2100 0x7fd91c161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd91a7fc700 1 -- 172.21.15.5:0/3505352115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0d2100 msgr2=0x7fd91c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd91a7fc700 1 --2- 172.21.15.5:0/3505352115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0d2100 0x7fd91c161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd91a7fc700 1 -- 172.21.15.5:0/3505352115 >> v2:172.21.15.49:3300/0 conn(0x7fd91c160f00 msgr2=0x7fd91c0b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd91a7fc700 1 --2- 172.21.15.5:0/3505352115 >> v2:172.21.15.49:3300/0 conn(0x7fd91c160f00 0x7fd91c0b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd91a7fc700 1 -- 172.21.15.5:0/3505352115 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0b42c0 con 0x7fd91c0b2f60 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd91affd700 1 --2- 172.21.15.5:0/3505352115 >> v2:172.21.15.49:3300/0 conn(0x7fd91c160f00 0x7fd91c0b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd91a7fc700 1 --2- 172.21.15.5:0/3505352115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0b2f60 0x7fd91c0d1bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0d1180 tx=0x7fd904009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f31cb669fb0cd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd919ffb700 1 -- 172.21.15.5:0/3505352115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90400b870 con 0x7fd91c0b2f60 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd919ffb700 1 -- 172.21.15.5:0/3505352115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90400b9d0 con 0x7fd91c0b2f60 2024-05-08T23:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.790+0000 7fd919ffb700 1 -- 172.21.15.5:0/3505352115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90400bd60 con 0x7fd91c0b2f60 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 -- 172.21.15.5:0/3505352115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0b2f60 msgr2=0x7fd91c0d1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 --2- 172.21.15.5:0/3505352115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0b2f60 0x7fd91c0d1bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0d1180 tx=0x7fd904009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 -- 172.21.15.5:0/3505352115 shutdown_connections 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 --2- 172.21.15.5:0/3505352115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0d2100 0x7fd91c161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 --2- 172.21.15.5:0/3505352115 >> v2:172.21.15.49:3300/0 conn(0x7fd91c160f00 0x7fd91c0b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 --2- 172.21.15.5:0/3505352115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0b2f60 0x7fd91c0d1bc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 -- 172.21.15.5:0/3505352115 >> 172.21.15.5:0/3505352115 conn(0x7fd91c08afe0 msgr2=0x7fd91c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 -- 172.21.15.5:0/3505352115 shutdown_connections 2024-05-08T23:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.791+0000 7fd92169b700 1 -- 172.21.15.5:0/3505352115 wait complete. 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.792+0000 7fd92169b700 1 Processor -- start 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.792+0000 7fd92169b700 1 -- start start 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd92169b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0b2f60 0x7fd91c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd92169b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0d2100 0x7fd91c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd92169b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91c160f00 0x7fd91c0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd92169b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0b7d30 con 0x7fd91c0d2100 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd92169b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0cd220 con 0x7fd91c160f00 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd92169b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0c9930 con 0x7fd91c0b2f60 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd91a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0d2100 0x7fd91c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd91a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0d2100 0x7fd91c15aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41778/0 (socket says 172.21.15.5:41778) 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd91a7fc700 1 -- 172.21.15.5:0/4139352765 learned_addr learned my addr 172.21.15.5:0/4139352765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd91b7fe700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91c160f00 0x7fd91c0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.793+0000 7fd91affd700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0b2f60 0x7fd91c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd91a7fc700 1 -- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0b2f60 msgr2=0x7fd91c0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd91a7fc700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0b2f60 0x7fd91c0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd91a7fc700 1 -- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91c160f00 msgr2=0x7fd91c0ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd91a7fc700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91c160f00 0x7fd91c0ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd91a7fc700 1 -- 172.21.15.5:0/4139352765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904009550 con 0x7fd91c0d2100 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd91b7fe700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91c160f00 0x7fd91c0ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd91affd700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0b2f60 0x7fd91c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd91a7fc700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0d2100 0x7fd91c15aff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd904000c00 tx=0x7fd904004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd9137fe700 1 -- 172.21.15.5:0/4139352765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904004540 con 0x7fd91c0d2100 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd9137fe700 1 -- 172.21.15.5:0/4139352765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9040036a0 con 0x7fd91c0d2100 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c0c9c10 con 0x7fd91c0d2100 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd9137fe700 1 -- 172.21.15.5:0/4139352765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904003a50 con 0x7fd91c0d2100 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.794+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c15c0d0 con 0x7fd91c0d2100 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.795+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c0d11b0 con 0x7fd91c0d2100 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.795+0000 7fd9137fe700 1 -- 172.21.15.5:0/4139352765 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd904021070 con 0x7fd91c0d2100 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.796+0000 7fd9137fe700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8fc06ac90 0x7fd8fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.796+0000 7fd9137fe700 1 -- 172.21.15.5:0/4139352765 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd90408ac10 con 0x7fd91c0d2100 2024-05-08T23:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.796+0000 7fd91affd700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8fc06ac90 0x7fd8fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.797+0000 7fd91affd700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8fc06ac90 0x7fd8fc06d150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd914003fd0 tx=0x7fd914009530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:46.801+0000 7fd9137fe700 1 -- 172.21.15.5:0/4139352765 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd904055fa0 con 0x7fd91c0d2100 2024-05-08T23:21:47.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.033+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd91c0678f0 con 0x7fd91c0d2100 2024-05-08T23:21:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.033+0000 7fd9137fe700 1 -- 172.21.15.5:0/4139352765 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd904015020 con 0x7fd91c0d2100 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8fc06ac90 msgr2=0x7fd8fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8fc06ac90 0x7fd8fc06d150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd914003fd0 tx=0x7fd914009530 comp rx=0 tx=0).stop 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0d2100 msgr2=0x7fd91c15aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0d2100 0x7fd91c15aff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd904000c00 tx=0x7fd904004750 comp rx=0 tx=0).stop 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 shutdown_connections 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8fc06ac90 0x7fd8fc06d150 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd91c160f00 0x7fd91c0ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd91c0b2f60 0x7fd91c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 --2- 172.21.15.5:0/4139352765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd91c0d2100 0x7fd91c15aff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 >> 172.21.15.5:0/4139352765 conn(0x7fd91c08afe0 msgr2=0x7fd91c06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 shutdown_connections 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.035+0000 7fd92169b700 1 -- 172.21.15.5:0/4139352765 wait complete. 2024-05-08T23:21:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:21:47.049 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T23:21:47.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:47.049 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-08T23:21:47.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:21:47.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:47.094 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T23:21:47.173 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:21:47.173 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:47.196 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:21:47.196 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:47.260 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:21:47.260 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:47.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.508+0000 7ff16fcce700 1 Processor -- start 2024-05-08T23:21:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.508+0000 7ff16fcce700 1 -- start start 2024-05-08T23:21:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16fcce700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff16806bd80 0x7ff1680d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16fcce700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1680cf990 con 0x7ff16815da10 2024-05-08T23:21:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16fcce700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1680d04b0 con 0x7ff16806c3d0 2024-05-08T23:21:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16fcce700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16809ab40 con 0x7ff16806bd80 2024-05-08T23:21:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16da6a700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff16815da10 0x7ff16809a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59822/0 (socket says 172.21.15.5:59822) 2024-05-08T23:21:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16da6a700 1 -- 172.21.15.5:0/401273152 learned_addr learned my addr 172.21.15.5:0/401273152 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16d269700 1 --2- 172.21.15.5:0/401273152 >> v2:172.21.15.49:3300/0 conn(0x7ff16806bd80 0x7ff1680d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698088845 0 0) 0x7ff1680cf990 con 0x7ff16815da10 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.509+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1540035b0 con 0x7ff16815da10 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706216827 0 0) 0x7ff1680d04b0 con 0x7ff16806c3d0 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1680cf990 con 0x7ff16806c3d0 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 453399489 0 0) 0x7ff1540035b0 con 0x7ff16815da10 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 >> v2:172.21.15.49:3300/0 conn(0x7ff16806bd80 msgr2=0x7ff1680d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 --2- 172.21.15.5:0/401273152 >> v2:172.21.15.49:3300/0 conn(0x7ff16806bd80 0x7ff1680d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 >> v1:172.21.15.49:6789/0 conn(0x7ff16806c3d0 legacy=0x7ff1680cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16809ad60 con 0x7ff16815da10 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7ff164001670 con 0x7ff16815da10 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16d269700 1 --2- 172.21.15.5:0/401273152 >> v2:172.21.15.49:3300/0 conn(0x7ff16806bd80 0x7ff1680d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 >> v1:172.21.15.5:6789/0 conn(0x7ff16815da10 legacy=0x7ff16809a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff154003b90 0x7ff154003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154004850 0x7ff154004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff154005210 0x7ff15400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16809ab40 con 0x7ff154003b90 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15400c090 con 0x7ff154005210 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15400c250 con 0x7ff154004850 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16da6a700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff154003b90 0x7ff154003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16e26b700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff154005210 0x7ff15400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16d269700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154004850 0x7ff154004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16da6a700 1 -- 172.21.15.5:0/401273152 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154004850 msgr2=0x7ff154004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16da6a700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154004850 0x7ff154004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16da6a700 1 -- 172.21.15.5:0/401273152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff154005210 msgr2=0x7ff15400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16da6a700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff154005210 0x7ff15400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16da6a700 1 -- 172.21.15.5:0/401273152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1540035b0 con 0x7ff154003b90 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.510+0000 7ff16e26b700 1 -- 172.21.15.5:0/401273152 reap_dead start 2024-05-08T23:21:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16da6a700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff154003b90 0x7ff154003f90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff16400c210 tx=0x7ff154005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0c3f7c9e096dc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16400cdb0 con 0x7ff154003b90 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff164001af0 con 0x7ff154003b90 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16ca68700 1 -- 172.21.15.5:0/401273152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164005ba0 con 0x7ff154003b90 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16fcce700 1 -- 172.21.15.5:0/401273152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff154003b90 msgr2=0x7ff154003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16fcce700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff154003b90 0x7ff154003f90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff16400c210 tx=0x7ff154005880 comp rx=0 tx=0).stop 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16fcce700 1 -- 172.21.15.5:0/401273152 shutdown_connections 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16fcce700 1 --2- 172.21.15.5:0/401273152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff154003b90 0x7ff154003f90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16fcce700 1 -- 172.21.15.5:0/401273152 >> 172.21.15.5:0/401273152 conn(0x7ff1680b3010 msgr2=0x7ff1680b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16fcce700 1 -- 172.21.15.5:0/401273152 shutdown_connections 2024-05-08T23:21:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.511+0000 7ff16fcce700 1 -- 172.21.15.5:0/401273152 wait complete. 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.513+0000 7ff16fcce700 1 Processor -- start 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.513+0000 7ff16fcce700 1 -- start start 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16fcce700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154003b90 0x7ff1681f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16fcce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1681490a0 0x7ff1680b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16fcce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1680b20c0 0x7ff1680b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16fcce700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1680b2ab0 con 0x7ff1681490a0 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16fcce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1680b43d0 con 0x7ff1680b20c0 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16fcce700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1681f38d0 con 0x7ff154003b90 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1681490a0 0x7ff1680b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1681490a0 0x7ff1680b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44796/0 (socket says 172.21.15.5:44796) 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 -- 172.21.15.5:0/1180195217 learned_addr learned my addr 172.21.15.5:0/1180195217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16e26b700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1680b20c0 0x7ff1680b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16da6a700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154003b90 0x7ff1681f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 -- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154003b90 msgr2=0x7ff1681f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154003b90 0x7ff1681f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 -- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1680b20c0 msgr2=0x7ff1680b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1680b20c0 0x7ff1680b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 -- 172.21.15.5:0/1180195217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1681f3bb0 con 0x7ff1681490a0 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16d269700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1681490a0 0x7ff1680b3e90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff15c011ca0 tx=0x7ff15c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff15a7fc700 1 -- 172.21.15.5:0/1180195217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00f9a0 con 0x7ff1681490a0 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff15a7fc700 1 -- 172.21.15.5:0/1180195217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c00fb00 con 0x7ff1681490a0 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1681f3e90 con 0x7ff1681490a0 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.514+0000 7ff15a7fc700 1 -- 172.21.15.5:0/1180195217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00feb0 con 0x7ff1681490a0 2024-05-08T23:21:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.515+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff168149520 con 0x7ff1681490a0 2024-05-08T23:21:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.515+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1680cfba0 con 0x7ff1681490a0 2024-05-08T23:21:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.516+0000 7ff15a7fc700 1 -- 172.21.15.5:0/1180195217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff15c004030 con 0x7ff1681490a0 2024-05-08T23:21:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.516+0000 7ff15a7fc700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff14c06ac90 0x7ff14c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.516+0000 7ff15a7fc700 1 -- 172.21.15.5:0/1180195217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff15c089820 con 0x7ff1681490a0 2024-05-08T23:21:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.517+0000 7ff16da6a700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff14c06ac90 0x7ff14c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.517+0000 7ff16da6a700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff14c06ac90 0x7ff14c06d150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff164000c00 tx=0x7ff164006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.521+0000 7ff15a7fc700 1 -- 172.21.15.5:0/1180195217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff1680cfba0 con 0x7ff1681490a0 2024-05-08T23:21:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.743+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1680cfba0 con 0x7ff1681490a0 2024-05-08T23:21:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.744+0000 7ff15a7fc700 1 -- 172.21.15.5:0/1180195217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7ff1680cfba0 con 0x7ff1681490a0 2024-05-08T23:21:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:47.744 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff14c06ac90 msgr2=0x7ff14c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff14c06ac90 0x7ff14c06d150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff164000c00 tx=0x7ff164006040 comp rx=0 tx=0).stop 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1681490a0 msgr2=0x7ff1680b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1681490a0 0x7ff1680b3e90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff15c011ca0 tx=0x7ff15c009ea0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 shutdown_connections 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff14c06ac90 0x7ff14c06d150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1680b20c0 0x7ff1680b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff154003b90 0x7ff1681f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 --2- 172.21.15.5:0/1180195217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1681490a0 0x7ff1680b3e90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 >> 172.21.15.5:0/1180195217 conn(0x7ff1680b3010 msgr2=0x7ff16806ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 shutdown_connections 2024-05-08T23:21:47.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.746+0000 7ff16fcce700 1 -- 172.21.15.5:0/1180195217 wait complete. 2024-05-08T23:21:47.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:47.761 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.899+0000 7f7d9017f700 1 Processor -- start 2024-05-08T23:21:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.899+0000 7f7d9017f700 1 -- start start 2024-05-08T23:21:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.900+0000 7f7d9017f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d880ae9d0 0x7f7d8816cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.900+0000 7f7d9017f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880b00b0 con 0x7f7d880c4af0 2024-05-08T23:21:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.900+0000 7f7d9017f700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880af010 con 0x7f7d8814b790 2024-05-08T23:21:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.900+0000 7f7d9017f700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880aee20 con 0x7f7d880ae9d0 2024-05-08T23:21:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.900+0000 7f7d8e71c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d880ae9d0 0x7f7d8816cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8d71a700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7d880c4af0 0x7f7d880ae2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59826/0 (socket says 172.21.15.5:59826) 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8d71a700 1 -- 172.21.15.5:0/4119868178 learned_addr learned my addr 172.21.15.5:0/4119868178 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8e71c700 1 -- 172.21.15.5:0/4119868178 >> v1:172.21.15.49:6789/0 conn(0x7f7d8814b790 legacy=0x7f7d880c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8e71c700 1 -- 172.21.15.5:0/4119868178 >> v1:172.21.15.5:6789/0 conn(0x7f7d880c4af0 legacy=0x7f7d880ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8e71c700 1 -- 172.21.15.5:0/4119868178 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d880af2f0 con 0x7f7d880ae9d0 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8cf19700 1 -- 172.21.15.5:0/4119868178 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246484787 0 0) 0x7f7d880b00b0 con 0x7f7d880c4af0 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8e71c700 1 --2- 172.21.15.5:0/4119868178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d8816cc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009d90 tx=0x7f7d8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576c970cd6b0b108 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8cf19700 1 -- 172.21.15.5:0/4119868178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000e070 con 0x7f7d880ae9d0 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8cf19700 1 -- 172.21.15.5:0/4119868178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000bbd0 con 0x7f7d880ae9d0 2024-05-08T23:21:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.901+0000 7f7d8cf19700 1 -- 172.21.15.5:0/4119868178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80012430 con 0x7f7d880ae9d0 2024-05-08T23:21:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.902+0000 7f7d9017f700 1 -- 172.21.15.5:0/4119868178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 msgr2=0x7f7d8816cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.902+0000 7f7d9017f700 1 --2- 172.21.15.5:0/4119868178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d8816cc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009d90 tx=0x7f7d8000b330 comp rx=0 tx=0).stop 2024-05-08T23:21:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.902+0000 7f7d9017f700 1 -- 172.21.15.5:0/4119868178 shutdown_connections 2024-05-08T23:21:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.902+0000 7f7d9017f700 1 --2- 172.21.15.5:0/4119868178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d8816cc70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.902+0000 7f7d9017f700 1 -- 172.21.15.5:0/4119868178 >> 172.21.15.5:0/4119868178 conn(0x7f7d8808afe0 msgr2=0x7f7d8808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.902+0000 7f7d9017f700 1 -- 172.21.15.5:0/4119868178 shutdown_connections 2024-05-08T23:21:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.902+0000 7f7d9017f700 1 -- 172.21.15.5:0/4119868178 wait complete. 2024-05-08T23:21:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.903+0000 7f7d9017f700 1 Processor -- start 2024-05-08T23:21:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.903+0000 7f7d9017f700 1 -- start start 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d9017f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d88161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d9017f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d880c4af0 0x7f7d880b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d9017f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8814b790 0x7f7d880b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d9017f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b54a0 con 0x7f7d880ae9d0 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d9017f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b5780 con 0x7f7d8814b790 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d9017f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b5a60 con 0x7f7d880c4af0 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d8df1b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d88161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d8df1b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d88161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44814/0 (socket says 172.21.15.5:44814) 2024-05-08T23:21:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d8df1b700 1 -- 172.21.15.5:0/1561426690 learned_addr learned my addr 172.21.15.5:0/1561426690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d8e71c700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8814b790 0x7f7d880b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.904+0000 7f7d8d71a700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d880c4af0 0x7f7d880b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d8df1b700 1 -- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d880c4af0 msgr2=0x7f7d880b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d8df1b700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d880c4af0 0x7f7d880b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d8df1b700 1 -- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8814b790 msgr2=0x7f7d880b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d8df1b700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8814b790 0x7f7d880b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d8df1b700 1 -- 172.21.15.5:0/1561426690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80009a70 con 0x7f7d880ae9d0 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d8df1b700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d88161440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400c450 tx=0x7f7d8400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d7e7fc700 1 -- 172.21.15.5:0/1561426690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d840052d0 con 0x7f7d880ae9d0 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d7e7fc700 1 -- 172.21.15.5:0/1561426690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d84005430 con 0x7f7d880ae9d0 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d880b5d40 con 0x7f7d880ae9d0 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d7e7fc700 1 -- 172.21.15.5:0/1561426690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d840038c0 con 0x7f7d880ae9d0 2024-05-08T23:21:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.905+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d880c6f70 con 0x7f7d880ae9d0 2024-05-08T23:21:47.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.906+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8809b3e0 con 0x7f7d880ae9d0 2024-05-08T23:21:47.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.907+0000 7f7d7e7fc700 1 -- 172.21.15.5:0/1561426690 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7d84003a20 con 0x7f7d880ae9d0 2024-05-08T23:21:47.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.907+0000 7f7d7e7fc700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d6c06ac90 0x7f7d6c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.907+0000 7f7d7e7fc700 1 -- 172.21.15.5:0/1561426690 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d84088640 con 0x7f7d880ae9d0 2024-05-08T23:21:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.907+0000 7f7d8d71a700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d6c06ac90 0x7f7d6c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.908+0000 7f7d8d71a700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d6c06ac90 0x7f7d6c06d150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007fd0 tx=0x7f7d78007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:47.912+0000 7f7d7e7fc700 1 -- 172.21.15.5:0/1561426690 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7d8809b3e0 con 0x7f7d880ae9d0 2024-05-08T23:21:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.126+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d8809b3e0 con 0x7f7d880ae9d0 2024-05-08T23:21:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.126+0000 7f7d7e7fc700 1 -- 172.21.15.5:0/1561426690 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d840538b0 con 0x7f7d880ae9d0 2024-05-08T23:21:48.126 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d6c06ac90 msgr2=0x7f7d6c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d6c06ac90 0x7f7d6c06d150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007fd0 tx=0x7f7d78007c20 comp rx=0 tx=0).stop 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 msgr2=0x7f7d88161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d88161440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400c450 tx=0x7f7d8400c810 comp rx=0 tx=0).stop 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 shutdown_connections 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d6c06ac90 0x7f7d6c06d150 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8814b790 0x7f7d880b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880ae9d0 0x7f7d88161440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 --2- 172.21.15.5:0/1561426690 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d880c4af0 0x7f7d880b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 >> 172.21.15.5:0/1561426690 conn(0x7f7d8808afe0 msgr2=0x7f7d8806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 shutdown_connections 2024-05-08T23:21:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.128+0000 7f7d9017f700 1 -- 172.21.15.5:0/1561426690 wait complete. 2024-05-08T23:21:48.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.281+0000 7f3401a0e700 1 Processor -- start 2024-05-08T23:21:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.282+0000 7f3401a0e700 1 -- start start 2024-05-08T23:21:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.282+0000 7f3401a0e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f33fc161a00 0x7f33fc0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.282+0000 7f3401a0e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33fc0d02d0 con 0x7f33fc0b3c00 2024-05-08T23:21:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.282+0000 7f3401a0e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33fc09ab70 con 0x7f33fc161de0 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.282+0000 7f3401a0e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33fc0d0ca0 con 0x7f33fc161a00 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33fa7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f33fc0b3c00 0x7f33fc09a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59832/0 (socket says 172.21.15.5:59832) 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33fa7fc700 1 -- 172.21.15.5:0/1166525684 learned_addr learned my addr 172.21.15.5:0/1166525684 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33faffd700 1 --2- 172.21.15.5:0/1166525684 >> v2:172.21.15.49:3300/0 conn(0x7f33fc161a00 0x7f33fc0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33f9ffb700 1 -- 172.21.15.5:0/1166525684 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285758981 0 0) 0x7f33fc0d02d0 con 0x7f33fc0b3c00 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33f9ffb700 1 -- 172.21.15.5:0/1166525684 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e40035b0 con 0x7f33fc0b3c00 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33f9ffb700 1 -- 172.21.15.5:0/1166525684 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318733754 0 0) 0x7f33fc09ab70 con 0x7f33fc161de0 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33f9ffb700 1 -- 172.21.15.5:0/1166525684 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33fc0d02d0 con 0x7f33fc161de0 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33f9ffb700 1 -- 172.21.15.5:0/1166525684 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2128005438 0 0) 0x7f33e40035b0 con 0x7f33fc0b3c00 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33faffd700 1 -- 172.21.15.5:0/1166525684 >> v1:172.21.15.49:6789/0 conn(0x7f33fc161de0 legacy=0x7f33fc160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33faffd700 1 -- 172.21.15.5:0/1166525684 >> v1:172.21.15.5:6789/0 conn(0x7f33fc0b3c00 legacy=0x7f33fc09a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33faffd700 1 -- 172.21.15.5:0/1166525684 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0cf9a0 con 0x7f33fc161a00 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.283+0000 7f33faffd700 1 --2- 172.21.15.5:0/1166525684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc161a00 0x7f33fc0d0760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009840 tx=0x7f33ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dd1193cde5c4557 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f33f9ffb700 1 -- 172.21.15.5:0/1166525684 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0001670 con 0x7f33fc161a00 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f33f9ffb700 1 -- 172.21.15.5:0/1166525684 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec004030 con 0x7f33fc161a00 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f33f9ffb700 1 -- 172.21.15.5:0/1166525684 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec0043c0 con 0x7f33fc161a00 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f3401a0e700 1 -- 172.21.15.5:0/1166525684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc161a00 msgr2=0x7f33fc0d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f3401a0e700 1 --2- 172.21.15.5:0/1166525684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc161a00 0x7f33fc0d0760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009840 tx=0x7f33ec009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f3401a0e700 1 -- 172.21.15.5:0/1166525684 shutdown_connections 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f3401a0e700 1 --2- 172.21.15.5:0/1166525684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc161a00 0x7f33fc0d0760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f3401a0e700 1 -- 172.21.15.5:0/1166525684 >> 172.21.15.5:0/1166525684 conn(0x7f33fc0b2f20 msgr2=0x7f33fc0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f3401a0e700 1 -- 172.21.15.5:0/1166525684 shutdown_connections 2024-05-08T23:21:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.284+0000 7f3401a0e700 1 -- 172.21.15.5:0/1166525684 wait complete. 2024-05-08T23:21:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.285+0000 7f3401a0e700 1 Processor -- start 2024-05-08T23:21:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.286+0000 7f3401a0e700 1 -- start start 2024-05-08T23:21:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.286+0000 7f3401a0e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc0b3c00 0x7f33fc0c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.286+0000 7f3401a0e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33fc161a00 0x7f33fc0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.286+0000 7f3401a0e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc161de0 0x7f33fc0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f3401a0e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0b5700 con 0x7f33fc161de0 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f3401a0e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0b5b20 con 0x7f33fc0b3c00 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f3401a0e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0b4a80 con 0x7f33fc161a00 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc161de0 0x7f33fc0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc0b3c00 0x7f33fc0c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fa7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33fc161a00 0x7f33fc0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc161de0 0x7f33fc0c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44824/0 (socket says 172.21.15.5:44824) 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 -- 172.21.15.5:0/3188438401 learned_addr learned my addr 172.21.15.5:0/3188438401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:48.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc0b3c00 0x7f33fc0c8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36730/0 (socket says 172.21.15.5:36730) 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 -- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33fc161a00 msgr2=0x7f33fc0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33fc161a00 0x7f33fc0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 -- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc0b3c00 msgr2=0x7f33fc0c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc0b3c00 0x7f33fc0c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 -- 172.21.15.5:0/3188438401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0094a0 con 0x7f33fc161de0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33fb7fe700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc161de0 0x7f33fc0c7300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33f400cbc0 tx=0x7f33f400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33e3fff700 1 -- 172.21.15.5:0/3188438401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4003c00 con 0x7f33fc161de0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33e3fff700 1 -- 172.21.15.5:0/3188438401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4003d60 con 0x7f33fc161de0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc0b4d60 con 0x7f33fc161de0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.287+0000 7f33e3fff700 1 -- 172.21.15.5:0/3188438401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400aaa0 con 0x7f33fc161de0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.288+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc0b5320 con 0x7f33fc161de0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.289+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc06d480 con 0x7f33fc161de0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.289+0000 7f33e3fff700 1 -- 172.21.15.5:0/3188438401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f33f400ac00 con 0x7f33fc161de0 2024-05-08T23:21:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.289+0000 7f33e3fff700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e806ac90 0x7f33e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.290+0000 7f33e3fff700 1 -- 172.21.15.5:0/3188438401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33f40892c0 con 0x7f33fc161de0 2024-05-08T23:21:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.290+0000 7f33faffd700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e806ac90 0x7f33e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.290+0000 7f33faffd700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e806ac90 0x7f33e806d150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0090b0 tx=0x7f33ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.294+0000 7f33e3fff700 1 -- 172.21.15.5:0/3188438401 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f33f40544a0 con 0x7f33fc161de0 2024-05-08T23:21:48.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.514+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33fc0b5840 con 0x7f33fc161de0 2024-05-08T23:21:48.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.515+0000 7f33e3fff700 1 -- 172.21.15.5:0/3188438401 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33fc0b5840 con 0x7f33fc161de0 2024-05-08T23:21:48.515 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:48.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e806ac90 msgr2=0x7f33e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e806ac90 0x7f33e806d150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0090b0 tx=0x7f33ec008030 comp rx=0 tx=0).stop 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc161de0 msgr2=0x7f33fc0c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc161de0 0x7f33fc0c7300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33f400cbc0 tx=0x7f33f400ced0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 shutdown_connections 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e806ac90 0x7f33e806d150 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc161de0 0x7f33fc0c7300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33fc0b3c00 0x7f33fc0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 --2- 172.21.15.5:0/3188438401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33fc161a00 0x7f33fc0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 >> 172.21.15.5:0/3188438401 conn(0x7f33fc0b2f20 msgr2=0x7f33fc06f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 shutdown_connections 2024-05-08T23:21:48.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.517+0000 7f3401a0e700 1 -- 172.21.15.5:0/3188438401 wait complete. 2024-05-08T23:21:48.531 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.671+0000 7f458f9d6700 1 Processor -- start 2024-05-08T23:21:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.671+0000 7f458f9d6700 1 -- start start 2024-05-08T23:21:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458f9d6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b4e0 0x7f45880b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458f9d6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f458815b8c0 0x7f45880d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458f9d6700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45880cf560 con 0x7f45880d1740 2024-05-08T23:21:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458f9d6700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45880cefe0 con 0x7f458815b8c0 2024-05-08T23:21:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458f9d6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45880d1bb0 con 0x7f458815b4e0 2024-05-08T23:21:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458d772700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b4e0 0x7f45880b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458cf71700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f458815b8c0 0x7f45880d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458d772700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b4e0 0x7f45880b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:37762/0 (socket says 172.21.15.5:37762) 2024-05-08T23:21:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458d772700 1 -- 172.21.15.5:0/1041144054 learned_addr learned my addr 172.21.15.5:0/1041144054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.672+0000 7f458df73700 1 --1- 172.21.15.5:0/1041144054 >> v1:172.21.15.49:6789/0 conn(0x7f45880d1740 0x7f45880cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:46180/0 (socket says 172.21.15.5:46180) 2024-05-08T23:21:48.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.673+0000 7f458cf71700 1 -- 172.21.15.5:0/1041144054 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b4e0 msgr2=0x7f45880b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:48.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.673+0000 7f458cf71700 1 --2- 172.21.15.5:0/1041144054 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b4e0 0x7f45880b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.673+0000 7f458cf71700 1 -- 172.21.15.5:0/1041144054 >> v1:172.21.15.49:6789/0 conn(0x7f45880d1740 legacy=0x7f45880cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.673+0000 7f458cf71700 1 -- 172.21.15.5:0/1041144054 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45880ccff0 con 0x7f458815b8c0 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.673+0000 7f458cf71700 1 --2- 172.21.15.5:0/1041144054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458815b8c0 0x7f45880d1200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4578009840 tx=0x7f4578009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43542b7d88d3dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.673+0000 7f457ffff700 1 -- 172.21.15.5:0/1041144054 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4578011830 con 0x7f458815b8c0 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.673+0000 7f457ffff700 1 -- 172.21.15.5:0/1041144054 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4578011990 con 0x7f458815b8c0 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.673+0000 7f457ffff700 1 -- 172.21.15.5:0/1041144054 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4578011d20 con 0x7f458815b8c0 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.674+0000 7f458f9d6700 1 -- 172.21.15.5:0/1041144054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458815b8c0 msgr2=0x7f45880d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.674+0000 7f458f9d6700 1 --2- 172.21.15.5:0/1041144054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458815b8c0 0x7f45880d1200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4578009840 tx=0x7f4578009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.674+0000 7f458f9d6700 1 -- 172.21.15.5:0/1041144054 shutdown_connections 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.674+0000 7f458f9d6700 1 --2- 172.21.15.5:0/1041144054 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b4e0 0x7f45880b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.674+0000 7f458f9d6700 1 --2- 172.21.15.5:0/1041144054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458815b8c0 0x7f45880d1200 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.674+0000 7f458f9d6700 1 -- 172.21.15.5:0/1041144054 >> 172.21.15.5:0/1041144054 conn(0x7f458808afe0 msgr2=0x7f458808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.674+0000 7f458f9d6700 1 -- 172.21.15.5:0/1041144054 shutdown_connections 2024-05-08T23:21:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.674+0000 7f458f9d6700 1 -- 172.21.15.5:0/1041144054 wait complete. 2024-05-08T23:21:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.675+0000 7f458f9d6700 1 Processor -- start 2024-05-08T23:21:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.675+0000 7f458f9d6700 1 -- start start 2024-05-08T23:21:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458f9d6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45880d1740 0x7f4588156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458f9d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f458815b4e0 0x7f458815e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458f9d6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b8c0 0x7f45880b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458f9d6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881568d0 con 0x7f458815b4e0 2024-05-08T23:21:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458f9d6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881f3ad0 con 0x7f45880d1740 2024-05-08T23:21:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458f9d6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45881f3db0 con 0x7f458815b8c0 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458cf71700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f458815b4e0 0x7f458815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458cf71700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f458815b4e0 0x7f458815e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44836/0 (socket says 172.21.15.5:44836) 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458cf71700 1 -- 172.21.15.5:0/2329753079 learned_addr learned my addr 172.21.15.5:0/2329753079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458df73700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b8c0 0x7f45880b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458d772700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45880d1740 0x7f4588156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458cf71700 1 -- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b8c0 msgr2=0x7f45880b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458cf71700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b8c0 0x7f45880b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458cf71700 1 -- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45880d1740 msgr2=0x7f4588156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.676+0000 7f458cf71700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45880d1740 0x7f4588156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.677+0000 7f458cf71700 1 -- 172.21.15.5:0/2329753079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780094a0 con 0x7f458815b4e0 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.677+0000 7f458cf71700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f458815b4e0 0x7f458815e640 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4578009fd0 tx=0x7f4578004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.677+0000 7f457dffb700 1 -- 172.21.15.5:0/2329753079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45780036a0 con 0x7f458815b4e0 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.677+0000 7f457dffb700 1 -- 172.21.15.5:0/2329753079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457801e070 con 0x7f458815b4e0 2024-05-08T23:21:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.677+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45881f4090 con 0x7f458815b4e0 2024-05-08T23:21:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.677+0000 7f457dffb700 1 -- 172.21.15.5:0/2329753079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4578003a40 con 0x7f458815b4e0 2024-05-08T23:21:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.677+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45880c9340 con 0x7f458815b4e0 2024-05-08T23:21:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.678+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45880c9590 con 0x7f458815b4e0 2024-05-08T23:21:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.679+0000 7f457dffb700 1 -- 172.21.15.5:0/2329753079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4578003ba0 con 0x7f458815b4e0 2024-05-08T23:21:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.679+0000 7f457dffb700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f456c06ac90 0x7f456c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.680+0000 7f457dffb700 1 -- 172.21.15.5:0/2329753079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f457808a250 con 0x7f458815b4e0 2024-05-08T23:21:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.680+0000 7f458d772700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f456c06ac90 0x7f456c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.680+0000 7f458d772700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f456c06ac90 0x7f456c06d150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f45880b73c0 tx=0x7f45840092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:48.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.684+0000 7f457dffb700 1 -- 172.21.15.5:0/2329753079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4578055420 con 0x7f458815b4e0 2024-05-08T23:21:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.902+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4588074bc0 con 0x7f458815b4e0 2024-05-08T23:21:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.903+0000 7f457dffb700 1 -- 172.21.15.5:0/2329753079 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4578020020 con 0x7f458815b4e0 2024-05-08T23:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f456c06ac90 msgr2=0x7f456c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f456c06ac90 0x7f456c06d150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f45880b73c0 tx=0x7f45840092c0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f458815b4e0 msgr2=0x7f458815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f458815b4e0 0x7f458815e640 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4578009fd0 tx=0x7f4578004750 comp rx=0 tx=0).stop 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 shutdown_connections 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f456c06ac90 0x7f456c06d150 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f458815b8c0 0x7f45880b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45880d1740 0x7f4588156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 --2- 172.21.15.5:0/2329753079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f458815b4e0 0x7f458815e640 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 >> 172.21.15.5:0/2329753079 conn(0x7f458808afe0 msgr2=0x7f458806f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 shutdown_connections 2024-05-08T23:21:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:48.905+0000 7f458f9d6700 1 -- 172.21.15.5:0/2329753079 wait complete. 2024-05-08T23:21:48.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:21:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.057+0000 7f1793058700 1 Processor -- start 2024-05-08T23:21:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.058+0000 7f1793058700 1 -- start start 2024-05-08T23:21:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.058+0000 7f1793058700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f178c0d3f40 0x7f178c161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.058+0000 7f1793058700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f178c15db80 0x7f178c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.058+0000 7f1793058700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c162450 con 0x7f178c15d7a0 2024-05-08T23:21:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.058+0000 7f1793058700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0cee00 con 0x7f178c15db80 2024-05-08T23:21:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.058+0000 7f1793058700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0b27e0 con 0x7f178c0d3f40 2024-05-08T23:21:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f1790df4700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f178c15d7a0 0x7f178c0b0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59842/0 (socket says 172.21.15.5:59842) 2024-05-08T23:21:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f17915f5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f178c15db80 0x7f178c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f1790df4700 1 -- 172.21.15.5:0/2710950408 learned_addr learned my addr 172.21.15.5:0/2710950408 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f178bfff700 1 --2- 172.21.15.5:0/2710950408 >> v2:172.21.15.49:3300/0 conn(0x7f178c0d3f40 0x7f178c161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f17915f5700 1 -- 172.21.15.5:0/2710950408 >> v2:172.21.15.49:3300/0 conn(0x7f178c0d3f40 msgr2=0x7f178c161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f17915f5700 1 --2- 172.21.15.5:0/2710950408 >> v2:172.21.15.49:3300/0 conn(0x7f178c0d3f40 0x7f178c161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f17915f5700 1 -- 172.21.15.5:0/2710950408 >> v1:172.21.15.5:6789/0 conn(0x7f178c15d7a0 legacy=0x7f178c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f178b7fe700 1 -- 172.21.15.5:0/2710950408 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069339374 0 0) 0x7f177c001110 con 0x7f178c15d7a0 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f17915f5700 1 -- 172.21.15.5:0/2710950408 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0d4400 con 0x7f178c15db80 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f17915f5700 1 --2- 172.21.15.5:0/2710950408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c0cf290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f178c0d0de0 tx=0x7f178400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c2b1da394aac54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f178b7fe700 1 -- 172.21.15.5:0/2710950408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400e070 con 0x7f178c15db80 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f178b7fe700 1 -- 172.21.15.5:0/2710950408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178400bb20 con 0x7f178c15db80 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.059+0000 7f178b7fe700 1 -- 172.21.15.5:0/2710950408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784012430 con 0x7f178c15db80 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.060+0000 7f1793058700 1 -- 172.21.15.5:0/2710950408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 msgr2=0x7f178c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.060+0000 7f1793058700 1 --2- 172.21.15.5:0/2710950408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c0cf290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f178c0d0de0 tx=0x7f178400b330 comp rx=0 tx=0).stop 2024-05-08T23:21:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.060+0000 7f1793058700 1 -- 172.21.15.5:0/2710950408 shutdown_connections 2024-05-08T23:21:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.060+0000 7f1793058700 1 --2- 172.21.15.5:0/2710950408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c0cf290 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.060+0000 7f1793058700 1 --2- 172.21.15.5:0/2710950408 >> v2:172.21.15.49:3300/0 conn(0x7f178c0d3f40 0x7f178c161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.060+0000 7f1793058700 1 -- 172.21.15.5:0/2710950408 >> 172.21.15.5:0/2710950408 conn(0x7f178c08afe0 msgr2=0x7f178c0d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.060+0000 7f1793058700 1 -- 172.21.15.5:0/2710950408 shutdown_connections 2024-05-08T23:21:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.060+0000 7f1793058700 1 -- 172.21.15.5:0/2710950408 wait complete. 2024-05-08T23:21:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.061+0000 7f1793058700 1 Processor -- start 2024-05-08T23:21:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.061+0000 7f1793058700 1 -- start start 2024-05-08T23:21:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f1793058700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f178c0d3f40 0x7f178c160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f1793058700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f178c15d7a0 0x7f178c0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f1793058700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c1f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f1793058700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0ca290 con 0x7f178c15db80 2024-05-08T23:21:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f1793058700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0b2ec0 con 0x7f178c0d3f40 2024-05-08T23:21:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f1793058700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c9a50 con 0x7f178c15d7a0 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f17915f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c1f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f17915f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c1f2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44852/0 (socket says 172.21.15.5:44852) 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.062+0000 7f17915f5700 1 -- 172.21.15.5:0/2105363215 learned_addr learned my addr 172.21.15.5:0/2105363215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f1790df4700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f178c0d3f40 0x7f178c160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f178bfff700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f178c15d7a0 0x7f178c0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17915f5700 1 -- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f178c15d7a0 msgr2=0x7f178c0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17915f5700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f178c15d7a0 0x7f178c0c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17915f5700 1 -- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f178c0d3f40 msgr2=0x7f178c160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17915f5700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f178c0d3f40 0x7f178c160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17915f5700 1 -- 172.21.15.5:0/2105363215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784009a70 con 0x7f178c15db80 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f178bfff700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f178c15d7a0 0x7f178c0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17915f5700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c1f2ae0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f178c08ba90 tx=0x7f1784004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17897fa700 1 -- 172.21.15.5:0/2105363215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784012430 con 0x7f178c15db80 2024-05-08T23:21:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17897fa700 1 -- 172.21.15.5:0/2105363215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1784020070 con 0x7f178c15db80 2024-05-08T23:21:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f17897fa700 1 -- 172.21.15.5:0/2105363215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784004280 con 0x7f178c15db80 2024-05-08T23:21:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c0c9cd0 con 0x7f178c15db80 2024-05-08T23:21:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.063+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c0b2b50 con 0x7f178c15db80 2024-05-08T23:21:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.064+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c0b29b0 con 0x7f178c15db80 2024-05-08T23:21:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.065+0000 7f17897fa700 1 -- 172.21.15.5:0/2105363215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f17840043e0 con 0x7f178c15db80 2024-05-08T23:21:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.065+0000 7f17897fa700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f176c06ac90 0x7f176c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.065+0000 7f17897fa700 1 -- 172.21.15.5:0/2105363215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1784022070 con 0x7f178c15db80 2024-05-08T23:21:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.066+0000 7f1790df4700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f176c06ac90 0x7f176c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.066+0000 7f1790df4700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f176c06ac90 0x7f176c06d150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f177c000f60 tx=0x7f177c007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.070+0000 7f17897fa700 1 -- 172.21.15.5:0/2105363215 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f178c0b29b0 con 0x7f178c15db80 2024-05-08T23:21:49.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.307+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f178c0b2950 con 0x7f178c15db80 2024-05-08T23:21:49.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.307+0000 7f17897fa700 1 -- 172.21.15.5:0/2105363215 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f17840561e0 con 0x7f178c15db80 2024-05-08T23:21:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f176c06ac90 msgr2=0x7f176c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f176c06ac90 0x7f176c06d150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f177c000f60 tx=0x7f177c007c20 comp rx=0 tx=0).stop 2024-05-08T23:21:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 msgr2=0x7f178c1f2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c1f2ae0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f178c08ba90 tx=0x7f1784004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 shutdown_connections 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f176c06ac90 0x7f176c06d150 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178c15db80 0x7f178c1f2ae0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f178c0d3f40 0x7f178c160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 --2- 172.21.15.5:0/2105363215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f178c15d7a0 0x7f178c0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 >> 172.21.15.5:0/2105363215 conn(0x7f178c08afe0 msgr2=0x7f178c15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 shutdown_connections 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.310+0000 7f1793058700 1 -- 172.21.15.5:0/2105363215 wait complete. 2024-05-08T23:21:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:21:49.324 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-08T23:21:49.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:49.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-08T23:21:49.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:21:49.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:49.369 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T23:21:49.389 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:21:49.389 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:49.411 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:21:49.411 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:49.472 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:21:49.472 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:21:49.533 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.678+0000 7f32aa7c0700 1 Processor -- start 2024-05-08T23:21:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.678+0000 7f32aa7c0700 1 -- start start 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32aa7c0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f32a415db00 0x7f32a40cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32aa7c0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32a406f990 0x7f32a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32aa7c0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a406ad20 con 0x7f32a40caa50 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32aa7c0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a415dee0 con 0x7f32a415db00 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32aa7c0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a40ccad0 con 0x7f32a406f990 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a3fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f32a415db00 0x7f32a40cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f32a40caa50 0x7f32a406f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59844/0 (socket says 172.21.15.5:59844) 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a37fe700 1 -- 172.21.15.5:0/1987240608 learned_addr learned my addr 172.21.15.5:0/1987240608 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a8d5d700 1 --2- 172.21.15.5:0/1987240608 >> v2:172.21.15.49:3300/0 conn(0x7f32a406f990 0x7f32a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a3fff700 1 -- 172.21.15.5:0/1987240608 >> v2:172.21.15.49:3300/0 conn(0x7f32a406f990 msgr2=0x7f32a406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a3fff700 1 --2- 172.21.15.5:0/1987240608 >> v2:172.21.15.49:3300/0 conn(0x7f32a406f990 0x7f32a406e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a3fff700 1 -- 172.21.15.5:0/1987240608 >> v1:172.21.15.5:6789/0 conn(0x7f32a40caa50 legacy=0x7f32a406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a3fff700 1 -- 172.21.15.5:0/1987240608 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a406b000 con 0x7f32a415db00 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.679+0000 7f32a2ffd700 1 -- 172.21.15.5:0/1987240608 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137154630 0 0) 0x7f32a406ad20 con 0x7f32a40caa50 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32a3fff700 1 --2- 172.21.15.5:0/1987240608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a40cc590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f32a4099f70 tx=0x7f3294009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f385ffadf7c49a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32a2ffd700 1 -- 172.21.15.5:0/1987240608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294011830 con 0x7f32a415db00 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32a2ffd700 1 -- 172.21.15.5:0/1987240608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3294011990 con 0x7f32a415db00 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32a2ffd700 1 -- 172.21.15.5:0/1987240608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294011d20 con 0x7f32a415db00 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1987240608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 msgr2=0x7f32a40cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1987240608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a40cc590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f32a4099f70 tx=0x7f3294009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1987240608 shutdown_connections 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1987240608 >> v2:172.21.15.49:3300/0 conn(0x7f32a406f990 0x7f32a406e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1987240608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a40cc590 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1987240608 >> 172.21.15.5:0/1987240608 conn(0x7f32a4099a50 msgr2=0x7f32a40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1987240608 shutdown_connections 2024-05-08T23:21:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.680+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1987240608 wait complete. 2024-05-08T23:21:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.682+0000 7f32aa7c0700 1 Processor -- start 2024-05-08T23:21:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.682+0000 7f32aa7c0700 1 -- start start 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32aa7c0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32a406f990 0x7f32a40b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32aa7c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32a40caa50 0x7f32a415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32aa7c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a41e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32aa7c0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40c4d50 con 0x7f32a415db00 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32aa7c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a41e6100 con 0x7f32a40caa50 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32aa7c0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40c40d0 con 0x7f32a406f990 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a41e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a41e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44870/0 (socket says 172.21.15.5:44870) 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 -- 172.21.15.5:0/1235308150 learned_addr learned my addr 172.21.15.5:0/1235308150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a37fe700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32a40caa50 0x7f32a415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a3fff700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32a406f990 0x7f32a40b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 -- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32a406f990 msgr2=0x7f32a40b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32a406f990 0x7f32a40b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 -- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32a40caa50 msgr2=0x7f32a415bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32a40caa50 0x7f32a415bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 -- 172.21.15.5:0/1235308150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940094a0 con 0x7f32a415db00 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a37fe700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32a40caa50 0x7f32a415bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.683+0000 7f32a8d5d700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a41e6c90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f329c007de0 tx=0x7f329c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.684+0000 7f32a0ff9700 1 -- 172.21.15.5:0/1235308150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00ede0 con 0x7f32a415db00 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.684+0000 7f32a0ff9700 1 -- 172.21.15.5:0/1235308150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c014420 con 0x7f32a415db00 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.684+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a40c43b0 con 0x7f32a415db00 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.684+0000 7f32a0ff9700 1 -- 172.21.15.5:0/1235308150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c0147d0 con 0x7f32a415db00 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.684+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a40c4970 con 0x7f32a415db00 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.685+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a40c4e90 con 0x7f32a415db00 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.685+0000 7f32a0ff9700 1 -- 172.21.15.5:0/1235308150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f329c014930 con 0x7f32a415db00 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.686+0000 7f32a0ff9700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f329006ac90 0x7f329006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.686+0000 7f32a0ff9700 1 -- 172.21.15.5:0/1235308150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f329c089ad0 con 0x7f32a415db00 2024-05-08T23:21:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.686+0000 7f32a3fff700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f329006ac90 0x7f329006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.686+0000 7f32a3fff700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f329006ac90 0x7f329006d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f32940097c0 tx=0x7f329401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.690+0000 7f32a0ff9700 1 -- 172.21.15.5:0/1235308150 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f32a40c4e90 con 0x7f32a415db00 2024-05-08T23:21:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.909+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32a40c4e90 con 0x7f32a415db00 2024-05-08T23:21:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.910+0000 7f32a0ff9700 1 -- 172.21.15.5:0/1235308150 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15941 (secure 0 0 0) 0x7f32a40c4e90 con 0x7f32a415db00 2024-05-08T23:21:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:49.910 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:05.088769+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f329006ac90 msgr2=0x7f329006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f329006ac90 0x7f329006d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f32940097c0 tx=0x7f329401c040 comp rx=0 tx=0).stop 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 msgr2=0x7f32a41e6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a41e6c90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f329c007de0 tx=0x7f329c00e3c0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 shutdown_connections 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f329006ac90 0x7f329006d150 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a415db00 0x7f32a41e6c90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32a406f990 0x7f32a40b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 --2- 172.21.15.5:0/1235308150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32a40caa50 0x7f32a415bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:49.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.912+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 >> 172.21.15.5:0/1235308150 conn(0x7f32a4099a50 msgr2=0x7f32a415ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:49.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.913+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 shutdown_connections 2024-05-08T23:21:49.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:49.913+0000 7f32aa7c0700 1 -- 172.21.15.5:0/1235308150 wait complete. 2024-05-08T23:21:49.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:49.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.067+0000 7fa4a8ef9700 1 Processor -- start 2024-05-08T23:21:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.067+0000 7fa4a8ef9700 1 -- start start 2024-05-08T23:21:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.068+0000 7fa4a8ef9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa4a415d880 0x7fa4a40af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.068+0000 7fa4a8ef9700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40b3870 con 0x7fa4a406f8c0 2024-05-08T23:21:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.068+0000 7fa4a8ef9700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40b4ee0 con 0x7fa4a406e7e0 2024-05-08T23:21:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.068+0000 7fa4a8ef9700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a40b32f0 con 0x7fa4a415d880 2024-05-08T23:21:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.068+0000 7fa4a259c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa4a415d880 0x7fa4a40af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.068+0000 7fa4a1d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa4a406f8c0 0x7fa4a40ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59854/0 (socket says 172.21.15.5:59854) 2024-05-08T23:21:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.068+0000 7fa4a1d9b700 1 -- 172.21.15.5:0/763870977 learned_addr learned my addr 172.21.15.5:0/763870977 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.069+0000 7fa4a259c700 1 -- 172.21.15.5:0/763870977 >> v1:172.21.15.49:6789/0 conn(0x7fa4a406e7e0 legacy=0x7fa4a406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.069+0000 7fa4a259c700 1 -- 172.21.15.5:0/763870977 >> v1:172.21.15.5:6789/0 conn(0x7fa4a406f8c0 legacy=0x7fa4a40ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.069+0000 7fa4a259c700 1 -- 172.21.15.5:0/763870977 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a406fd50 con 0x7fa4a415d880 2024-05-08T23:21:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.069+0000 7fa4a159a700 1 -- 172.21.15.5:0/763870977 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240134509 0 0) 0x7fa4a40b3870 con 0x7fa4a406f8c0 2024-05-08T23:21:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.069+0000 7fa4a259c700 1 --2- 172.21.15.5:0/763870977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a415d880 0x7fa4a40af990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa494009840 tx=0x7fa494009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7b9934fb23daf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.069+0000 7fa4a159a700 1 -- 172.21.15.5:0/763870977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494011830 con 0x7fa4a415d880 2024-05-08T23:21:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.069+0000 7fa4a159a700 1 -- 172.21.15.5:0/763870977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494011990 con 0x7fa4a415d880 2024-05-08T23:21:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.069+0000 7fa4a159a700 1 -- 172.21.15.5:0/763870977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494011d20 con 0x7fa4a415d880 2024-05-08T23:21:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.070+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/763870977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a415d880 msgr2=0x7fa4a40af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.070+0000 7fa4a8ef9700 1 --2- 172.21.15.5:0/763870977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a415d880 0x7fa4a40af990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa494009840 tx=0x7fa494009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.070+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/763870977 shutdown_connections 2024-05-08T23:21:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.070+0000 7fa4a8ef9700 1 --2- 172.21.15.5:0/763870977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a415d880 0x7fa4a40af990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.070+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/763870977 >> 172.21.15.5:0/763870977 conn(0x7fa4a40b5810 msgr2=0x7fa4a40b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.070+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/763870977 shutdown_connections 2024-05-08T23:21:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.070+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/763870977 wait complete. 2024-05-08T23:21:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.071+0000 7fa4a8ef9700 1 Processor -- start 2024-05-08T23:21:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.071+0000 7fa4a8ef9700 1 -- start start 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a8ef9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a406e7e0 0x7fa4a409ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a8ef9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa4a406f8c0 0x7fa4a41f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a8ef9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a415d880 0x7fa4a40ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a8ef9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a409b5b0 con 0x7fa4a406e7e0 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a8ef9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41f3fc0 con 0x7fa4a415d880 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a8ef9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a409a9a0 con 0x7fa4a406f8c0 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a406e7e0 0x7fa4a409ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a406e7e0 0x7fa4a409ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44884/0 (socket says 172.21.15.5:44884) 2024-05-08T23:21:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a259c700 1 -- 172.21.15.5:0/863895233 learned_addr learned my addr 172.21.15.5:0/863895233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a1d9b700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa4a406f8c0 0x7fa4a41f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.072+0000 7fa4a2d9d700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a415d880 0x7fa4a40ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa4a259c700 1 -- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa4a406f8c0 msgr2=0x7fa4a41f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa4a259c700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa4a406f8c0 0x7fa4a41f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa4a259c700 1 -- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a415d880 msgr2=0x7fa4a40ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa4a259c700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a415d880 0x7fa4a40ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa4a259c700 1 -- 172.21.15.5:0/863895233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4940094a0 con 0x7fa4a406e7e0 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa4a259c700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a406e7e0 0x7fa4a409ba40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa494014040 tx=0x7fa494004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa492ffd700 1 -- 172.21.15.5:0/863895233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49401a7e0 con 0x7fa4a406e7e0 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa492ffd700 1 -- 172.21.15.5:0/863895233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49401a940 con 0x7fa4a406e7e0 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a409ac80 con 0x7fa4a406e7e0 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa492ffd700 1 -- 172.21.15.5:0/863895233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49401acb0 con 0x7fa4a406e7e0 2024-05-08T23:21:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.073+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a409b240 con 0x7fa4a406e7e0 2024-05-08T23:21:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.074+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a409b890 con 0x7fa4a406e7e0 2024-05-08T23:21:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.075+0000 7fa492ffd700 1 -- 172.21.15.5:0/863895233 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa494022070 con 0x7fa4a406e7e0 2024-05-08T23:21:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.075+0000 7fa492ffd700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa48406ac90 0x7fa48406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.075+0000 7fa492ffd700 1 -- 172.21.15.5:0/863895233 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa494089f80 con 0x7fa4a406e7e0 2024-05-08T23:21:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.076+0000 7fa4a1d9b700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa48406ac90 0x7fa48406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.076+0000 7fa4a1d9b700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa48406ac90 0x7fa48406d150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40d0f50 tx=0x7fa498007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.080+0000 7fa492ffd700 1 -- 172.21.15.5:0/863895233 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa4940bd900 con 0x7fa4a406e7e0 2024-05-08T23:21:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.296+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4a4074bc0 con 0x7fa4a406e7e0 2024-05-08T23:21:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.296+0000 7fa492ffd700 1 -- 172.21.15.5:0/863895233 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa494029020 con 0x7fa4a406e7e0 2024-05-08T23:21:50.296 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.298+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa48406ac90 msgr2=0x7fa48406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.298+0000 7fa4a8ef9700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa48406ac90 0x7fa48406d150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40d0f50 tx=0x7fa498007a60 comp rx=0 tx=0).stop 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a406e7e0 msgr2=0x7fa4a409ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a406e7e0 0x7fa4a409ba40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa494014040 tx=0x7fa494004750 comp rx=0 tx=0).stop 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 shutdown_connections 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa48406ac90 0x7fa48406d150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a415d880 0x7fa4a40ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4a406e7e0 0x7fa4a409ba40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 --2- 172.21.15.5:0/863895233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa4a406f8c0 0x7fa4a41f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 >> 172.21.15.5:0/863895233 conn(0x7fa4a40b5810 msgr2=0x7fa4a415fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 shutdown_connections 2024-05-08T23:21:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.299+0000 7fa4a8ef9700 1 -- 172.21.15.5:0/863895233 wait complete. 2024-05-08T23:21:50.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:50.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.450+0000 7fe69b4de700 1 Processor -- start 2024-05-08T23:21:50.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.451+0000 7fe69b4de700 1 -- start start 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe69b4de700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe69415d8c0 0x7fe6940ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe69b4de700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6940b5550 0x7fe6941618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe69b4de700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe694099ce0 con 0x7fe6940b50d0 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe69b4de700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69406fde0 con 0x7fe6940b5550 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe69b4de700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69406f260 con 0x7fe69415d8c0 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe698a79700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe6940b50d0 0x7fe69406a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59856/0 (socket says 172.21.15.5:59856) 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe699a7b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6940b5550 0x7fe6941618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe698a79700 1 -- 172.21.15.5:0/3525335108 learned_addr learned my addr 172.21.15.5:0/3525335108 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe69927a700 1 --2- 172.21.15.5:0/3525335108 >> v2:172.21.15.49:3300/0 conn(0x7fe69415d8c0 0x7fe6940ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe68bfff700 1 -- 172.21.15.5:0/3525335108 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442107604 0 0) 0x7fe694099ce0 con 0x7fe6940b50d0 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe68bfff700 1 -- 172.21.15.5:0/3525335108 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6800035b0 con 0x7fe6940b50d0 2024-05-08T23:21:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe699a7b700 1 -- 172.21.15.5:0/3525335108 >> v2:172.21.15.49:3300/0 conn(0x7fe69415d8c0 msgr2=0x7fe6940ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe699a7b700 1 --2- 172.21.15.5:0/3525335108 >> v2:172.21.15.49:3300/0 conn(0x7fe69415d8c0 0x7fe6940ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe699a7b700 1 -- 172.21.15.5:0/3525335108 >> v1:172.21.15.5:6789/0 conn(0x7fe6940b50d0 legacy=0x7fe69406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe699a7b700 1 -- 172.21.15.5:0/3525335108 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69406f540 con 0x7fe6940b5550 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.452+0000 7fe699a7b700 1 --2- 172.21.15.5:0/3525335108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b5550 0x7fe6941618f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe69406d730 tx=0x7fe68c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f631cc7ca610ba9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe68bfff700 1 -- 172.21.15.5:0/3525335108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c00e070 con 0x7fe6940b5550 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe68bfff700 1 -- 172.21.15.5:0/3525335108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c00bb20 con 0x7fe6940b5550 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe68bfff700 1 -- 172.21.15.5:0/3525335108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c012430 con 0x7fe6940b5550 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe69b4de700 1 -- 172.21.15.5:0/3525335108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b5550 msgr2=0x7fe6941618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3525335108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b5550 0x7fe6941618f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe69406d730 tx=0x7fe68c00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe69b4de700 1 -- 172.21.15.5:0/3525335108 shutdown_connections 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3525335108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b5550 0x7fe6941618f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3525335108 >> v2:172.21.15.49:3300/0 conn(0x7fe69415d8c0 0x7fe6940ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe69b4de700 1 -- 172.21.15.5:0/3525335108 >> 172.21.15.5:0/3525335108 conn(0x7fe6940b5a30 msgr2=0x7fe6940b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe69b4de700 1 -- 172.21.15.5:0/3525335108 shutdown_connections 2024-05-08T23:21:50.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.453+0000 7fe69b4de700 1 -- 172.21.15.5:0/3525335108 wait complete. 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.455+0000 7fe69b4de700 1 Processor -- start 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.455+0000 7fe69b4de700 1 -- start start 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.455+0000 7fe69b4de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b50d0 0x7fe69413dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.455+0000 7fe69b4de700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6940b5550 0x7fe69413a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.455+0000 7fe69b4de700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69415d8c0 0x7fe694156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69b4de700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69413ae90 con 0x7fe6940b50d0 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69b4de700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe694156b40 con 0x7fe6940b5550 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69b4de700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe694156e20 con 0x7fe69415d8c0 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b50d0 0x7fe69413dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b50d0 0x7fe69413dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44908/0 (socket says 172.21.15.5:44908) 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 -- 172.21.15.5:0/3992142673 learned_addr learned my addr 172.21.15.5:0/3992142673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe698a79700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6940b5550 0x7fe69413a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe699a7b700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69415d8c0 0x7fe694156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 -- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69415d8c0 msgr2=0x7fe694156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69415d8c0 0x7fe694156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 -- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6940b5550 msgr2=0x7fe69413a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6940b5550 0x7fe69413a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 -- 172.21.15.5:0/3992142673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c009a70 con 0x7fe6940b50d0 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe699a7b700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69415d8c0 0x7fe694156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69927a700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b50d0 0x7fe69413dd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe69000e870 tx=0x7fe69000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe689ffb700 1 -- 172.21.15.5:0/3992142673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690009590 con 0x7fe6940b50d0 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe689ffb700 1 -- 172.21.15.5:0/3992142673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe690013680 con 0x7fe6940b50d0 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe694157100 con 0x7fe6940b50d0 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.456+0000 7fe689ffb700 1 -- 172.21.15.5:0/3992142673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6900139f0 con 0x7fe6940b50d0 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.457+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69415ae90 con 0x7fe6940b50d0 2024-05-08T23:21:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.458+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69415b200 con 0x7fe6940b50d0 2024-05-08T23:21:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.458+0000 7fe689ffb700 1 -- 172.21.15.5:0/3992142673 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe690013b50 con 0x7fe6940b50d0 2024-05-08T23:21:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.459+0000 7fe689ffb700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe67806ac90 0x7fe67806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.459+0000 7fe689ffb700 1 -- 172.21.15.5:0/3992142673 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe690089190 con 0x7fe6940b50d0 2024-05-08T23:21:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.459+0000 7fe698a79700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe67806ac90 0x7fe67806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.459+0000 7fe698a79700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe67806ac90 0x7fe67806d150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe69413e290 tx=0x7fe684007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.463+0000 7fe689ffb700 1 -- 172.21.15.5:0/3992142673 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe690054400 con 0x7fe6940b50d0 2024-05-08T23:21:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.679+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe694156c80 con 0x7fe6940b50d0 2024-05-08T23:21:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.680+0000 7fe689ffb700 1 -- 172.21.15.5:0/3992142673 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe694156c80 con 0x7fe6940b50d0 2024-05-08T23:21:50.680 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe67806ac90 msgr2=0x7fe67806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe67806ac90 0x7fe67806d150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe69413e290 tx=0x7fe684007dc0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b50d0 msgr2=0x7fe69413dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b50d0 0x7fe69413dd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe69000e870 tx=0x7fe69000eb80 comp rx=0 tx=0).stop 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 shutdown_connections 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe67806ac90 0x7fe67806d150 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69415d8c0 0x7fe694156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6940b50d0 0x7fe69413dd50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 --2- 172.21.15.5:0/3992142673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6940b5550 0x7fe69413a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.682+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 >> 172.21.15.5:0/3992142673 conn(0x7fe6940b5a30 msgr2=0x7fe69415dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.683+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 shutdown_connections 2024-05-08T23:21:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.683+0000 7fe69b4de700 1 -- 172.21.15.5:0/3992142673 wait complete. 2024-05-08T23:21:50.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.835+0000 7fd6443a4700 1 Processor -- start 2024-05-08T23:21:50.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.835+0000 7fd6443a4700 1 -- start start 2024-05-08T23:21:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.836+0000 7fd6443a4700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd63c06bd60 0x7fd63c0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd6443a4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c0cdc90 0x7fd63c0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd6443a4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0b1200 con 0x7fd63c0d13a0 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd6443a4700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0cfcc0 con 0x7fd63c06bd60 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd6443a4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0ce5c0 con 0x7fd63c0cdc90 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd642140700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd63c0d13a0 0x7fd63c0d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59868/0 (socket says 172.21.15.5:59868) 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd642140700 1 -- 172.21.15.5:0/1074829603 learned_addr learned my addr 172.21.15.5:0/1074829603 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64193f700 1 --2- 172.21.15.5:0/1074829603 >> v2:172.21.15.49:3300/0 conn(0x7fd63c06bd60 0x7fd63c0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd642941700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c0cdc90 0x7fd63c0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068714140 0 0) 0x7fd63c0b1200 con 0x7fd63c0d13a0 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6280035b0 con 0x7fd63c0d13a0 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232280330 0 0) 0x7fd6280035b0 con 0x7fd63c0d13a0 2024-05-08T23:21:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c0cdc90 msgr2=0x7fd63c0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c0cdc90 0x7fd63c0b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 >> v2:172.21.15.49:3300/0 conn(0x7fd63c06bd60 msgr2=0x7fd63c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 --2- 172.21.15.5:0/1074829603 >> v2:172.21.15.49:3300/0 conn(0x7fd63c06bd60 0x7fd63c0cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0ce8a0 con 0x7fd63c0d13a0 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fd638001670 con 0x7fd63c0d13a0 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 >> v1:172.21.15.5:6789/0 conn(0x7fd63c0d13a0 legacy=0x7fd63c0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628003a00 0x7fd628003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.837+0000 7fd64113e700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd628004850 0x7fd628004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd64113e700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd628005210 0x7fd62800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0cfcc0 con 0x7fd628005210 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0ce5c0 con 0x7fd628003a00 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62800bf50 con 0x7fd628004850 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642941700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c0cdc90 0x7fd63c0b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642941700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd628005210 0x7fd62800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd64193f700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd628004850 0x7fd628004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642140700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628003a00 0x7fd628003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642941700 1 -- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd628004850 msgr2=0x7fd628004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642941700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd628004850 0x7fd628004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642941700 1 -- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628003a00 msgr2=0x7fd628003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642941700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628003a00 0x7fd628003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642941700 1 -- 172.21.15.5:0/1074829603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280035b0 con 0x7fd628005210 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd642941700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd628005210 0x7fd62800ba10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd63400e220 tx=0x7fd628005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788df762fea2ef85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63400ecc0 con 0x7fd628005210 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63400f040 con 0x7fd628005210 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.838+0000 7fd64113e700 1 -- 172.21.15.5:0/1074829603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634003a20 con 0x7fd628005210 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.839+0000 7fd6443a4700 1 -- 172.21.15.5:0/1074829603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd628005210 msgr2=0x7fd62800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.839+0000 7fd6443a4700 1 --2- 172.21.15.5:0/1074829603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd628005210 0x7fd62800ba10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd63400e220 tx=0x7fd628005880 comp rx=0 tx=0).stop 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.839+0000 7fd642941700 1 -- 172.21.15.5:0/1074829603 reap_dead start 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.839+0000 7fd6443a4700 1 -- 172.21.15.5:0/1074829603 shutdown_connections 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.839+0000 7fd6443a4700 1 -- 172.21.15.5:0/1074829603 >> 172.21.15.5:0/1074829603 conn(0x7fd63c08afe0 msgr2=0x7fd63c08b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.839+0000 7fd6443a4700 1 -- 172.21.15.5:0/1074829603 shutdown_connections 2024-05-08T23:21:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.839+0000 7fd6443a4700 1 -- 172.21.15.5:0/1074829603 wait complete. 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.840+0000 7fd6443a4700 1 Processor -- start 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.840+0000 7fd6443a4700 1 -- start start 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd6443a4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628005210 0x7fd63c1f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd6443a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd63c1526b0 0x7fd63c1f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd6443a4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c15faf0 0x7fd63c0c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd6443a4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c151210 con 0x7fd63c1526b0 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd6443a4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0c58e0 con 0x7fd628005210 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd6443a4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0c5b60 con 0x7fd63c15faf0 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd64193f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd63c1526b0 0x7fd63c1f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd64193f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd63c1526b0 0x7fd63c1f68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44934/0 (socket says 172.21.15.5:44934) 2024-05-08T23:21:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd64193f700 1 -- 172.21.15.5:0/3046611385 learned_addr learned my addr 172.21.15.5:0/3046611385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd642140700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628005210 0x7fd63c1f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.841+0000 7fd642941700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c15faf0 0x7fd63c0c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd64193f700 1 -- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c15faf0 msgr2=0x7fd63c0c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd64193f700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c15faf0 0x7fd63c0c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd64193f700 1 -- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628005210 msgr2=0x7fd63c1f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd64193f700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628005210 0x7fd63c1f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd64193f700 1 -- 172.21.15.5:0/3046611385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0c5e40 con 0x7fd63c1526b0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd64193f700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd63c1526b0 0x7fd63c1f68c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd62c011ad0 tx=0x7fd62c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd632ffd700 1 -- 172.21.15.5:0/3046611385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00fa60 con 0x7fd63c1526b0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd632ffd700 1 -- 172.21.15.5:0/3046611385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c00fbc0 con 0x7fd63c1526b0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c0c6120 con 0x7fd63c1526b0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd632ffd700 1 -- 172.21.15.5:0/3046611385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c014440 con 0x7fd63c1526b0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.842+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c160160 con 0x7fd63c1526b0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.843+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c0c6340 con 0x7fd63c1526b0 2024-05-08T23:21:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.843+0000 7fd632ffd700 1 -- 172.21.15.5:0/3046611385 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd62c0145a0 con 0x7fd63c1526b0 2024-05-08T23:21:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.844+0000 7fd632ffd700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd62006ac90 0x7fd62006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.844+0000 7fd632ffd700 1 -- 172.21.15.5:0/3046611385 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd62c089b60 con 0x7fd63c1526b0 2024-05-08T23:21:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.844+0000 7fd642140700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd62006ac90 0x7fd62006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.845+0000 7fd642140700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd62006ac90 0x7fd62006d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd638001050 tx=0x7fd638009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:50.849+0000 7fd632ffd700 1 -- 172.21.15.5:0/3046611385 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd62c054df0 con 0x7fd63c1526b0 2024-05-08T23:21:51.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.063+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd63c074bc0 con 0x7fd63c1526b0 2024-05-08T23:21:51.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.063+0000 7fd632ffd700 1 -- 172.21.15.5:0/3046611385 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd62c0220a0 con 0x7fd63c1526b0 2024-05-08T23:21:51.063 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd62006ac90 msgr2=0x7fd62006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd62006ac90 0x7fd62006d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd638001050 tx=0x7fd638009040 comp rx=0 tx=0).stop 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd63c1526b0 msgr2=0x7fd63c1f68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd63c1526b0 0x7fd63c1f68c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd62c011ad0 tx=0x7fd62c011e90 comp rx=0 tx=0).stop 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 shutdown_connections 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd62006ac90 0x7fd62006d150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd63c15faf0 0x7fd63c0c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd628005210 0x7fd63c1f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.065+0000 7fd6443a4700 1 --2- 172.21.15.5:0/3046611385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd63c1526b0 0x7fd63c1f68c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.066+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 >> 172.21.15.5:0/3046611385 conn(0x7fd63c08afe0 msgr2=0x7fd63c070130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:51.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.066+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 shutdown_connections 2024-05-08T23:21:51.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.066+0000 7fd6443a4700 1 -- 172.21.15.5:0/3046611385 wait complete. 2024-05-08T23:21:51.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:21:51.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.216+0000 7f6f83349700 1 Processor -- start 2024-05-08T23:21:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.217+0000 7f6f83349700 1 -- start start 2024-05-08T23:21:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.217+0000 7f6f83349700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6f7c15dc40 0x7f6f7c09a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.217+0000 7f6f83349700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6f7c0d0270 0x7f6f7c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.217+0000 7f6f83349700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c099f90 0x7f6f7c06be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.217+0000 7f6f83349700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c09a420 con 0x7f6f7c0d0270 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.217+0000 7f6f83349700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c0d0700 con 0x7f6f7c15dc40 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.217+0000 7f6f83349700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0d09e0 con 0x7f6f7c099f90 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6f7c0d0270 0x7f6f7c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6f7c0d0270 0x7f6f7c099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44942/0 (socket says 172.21.15.5:44942) 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 -- 172.21.15.5:0/1941446550 learned_addr learned my addr 172.21.15.5:0/1941446550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f818e6700 1 --2- 172.21.15.5:0/1941446550 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c099f90 0x7f6f7c06be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f810e5700 1 --2- 172.21.15.5:0/1941446550 >> v2:172.21.15.49:3300/0 conn(0x7f6f7c15dc40 0x7f6f7c09a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 -- 172.21.15.5:0/1941446550 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c099f90 msgr2=0x7f6f7c06be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 --2- 172.21.15.5:0/1941446550 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c099f90 0x7f6f7c06be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 -- 172.21.15.5:0/1941446550 >> v2:172.21.15.49:3300/0 conn(0x7f6f7c15dc40 msgr2=0x7f6f7c09a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 --2- 172.21.15.5:0/1941446550 >> v2:172.21.15.49:3300/0 conn(0x7f6f7c15dc40 0x7f6f7c09a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 -- 172.21.15.5:0/1941446550 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0d0cc0 con 0x7f6f7c0d0270 2024-05-08T23:21:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f808e4700 1 --2- 172.21.15.5:0/1941446550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c0d0270 0x7f6f7c099a50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c072250 tx=0x7f6f6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbced09fbeb0f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f73fff700 1 -- 172.21.15.5:0/1941446550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c011830 con 0x7f6f7c0d0270 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f73fff700 1 -- 172.21.15.5:0/1941446550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c011990 con 0x7f6f7c0d0270 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.218+0000 7f6f73fff700 1 -- 172.21.15.5:0/1941446550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c011d20 con 0x7f6f7c0d0270 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 -- 172.21.15.5:0/1941446550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c0d0270 msgr2=0x7f6f7c099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 --2- 172.21.15.5:0/1941446550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c0d0270 0x7f6f7c099a50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c072250 tx=0x7f6f6c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 -- 172.21.15.5:0/1941446550 shutdown_connections 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 --2- 172.21.15.5:0/1941446550 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c099f90 0x7f6f7c06be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 --2- 172.21.15.5:0/1941446550 >> v2:172.21.15.49:3300/0 conn(0x7f6f7c15dc40 0x7f6f7c09a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 --2- 172.21.15.5:0/1941446550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c0d0270 0x7f6f7c099a50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 -- 172.21.15.5:0/1941446550 >> 172.21.15.5:0/1941446550 conn(0x7f6f7c0b2f20 msgr2=0x7f6f7c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 -- 172.21.15.5:0/1941446550 shutdown_connections 2024-05-08T23:21:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.219+0000 7f6f83349700 1 -- 172.21.15.5:0/1941446550 wait complete. 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.221+0000 7f6f83349700 1 Processor -- start 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.221+0000 7f6f83349700 1 -- start start 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.221+0000 7f6f83349700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f7c099f90 0x7f6f7c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.221+0000 7f6f83349700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c15dc40 0x7f6f7c1f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.221+0000 7f6f83349700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c158b70 0x7f6f7c1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.221+0000 7f6f83349700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0c9630 con 0x7f6f7c15dc40 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.221+0000 7f6f83349700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1f6f30 con 0x7f6f7c099f90 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.221+0000 7f6f83349700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1f7190 con 0x7f6f7c158b70 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c15dc40 0x7f6f7c1f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c15dc40 0x7f6f7c1f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44944/0 (socket says 172.21.15.5:44944) 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 -- 172.21.15.5:0/2917810040 learned_addr learned my addr 172.21.15.5:0/2917810040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f810e5700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f7c099f90 0x7f6f7c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f818e6700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c158b70 0x7f6f7c1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 -- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c158b70 msgr2=0x7f6f7c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c158b70 0x7f6f7c1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 -- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f7c099f90 msgr2=0x7f6f7c0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f7c099f90 0x7f6f7c0c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 -- 172.21.15.5:0/2917810040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009550 con 0x7f6f7c15dc40 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f808e4700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c15dc40 0x7f6f7c1f69f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009fd0 tx=0x7f6f6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f71ffb700 1 -- 172.21.15.5:0/2917810040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c004450 con 0x7f6f7c15dc40 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f71ffb700 1 -- 172.21.15.5:0/2917810040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c01f070 con 0x7f6f7c15dc40 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c1f7470 con 0x7f6f7c15dc40 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.222+0000 7f6f71ffb700 1 -- 172.21.15.5:0/2917810040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0038c0 con 0x7f6f7c15dc40 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.223+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c0c9840 con 0x7f6f7c15dc40 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.223+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c068000 con 0x7f6f7c15dc40 2024-05-08T23:21:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.224+0000 7f6f71ffb700 1 -- 172.21.15.5:0/2917810040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6f6c003a20 con 0x7f6f7c15dc40 2024-05-08T23:21:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.224+0000 7f6f71ffb700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f6006ac90 0x7f6f6006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.225+0000 7f6f71ffb700 1 -- 172.21.15.5:0/2917810040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f6c021070 con 0x7f6f7c15dc40 2024-05-08T23:21:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.225+0000 7f6f810e5700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f6006ac90 0x7f6f6006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.225+0000 7f6f810e5700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f6006ac90 0x7f6f6006d150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009b40 tx=0x7f6f78009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.229+0000 7f6f71ffb700 1 -- 172.21.15.5:0/2917810040 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6f6c055c00 con 0x7f6f7c15dc40 2024-05-08T23:21:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.466+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6f7c15c5c0 con 0x7f6f7c15dc40 2024-05-08T23:21:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.466+0000 7f6f71ffb700 1 -- 172.21.15.5:0/2917810040 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6f6c02b020 con 0x7f6f7c15dc40 2024-05-08T23:21:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f6006ac90 msgr2=0x7f6f6006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f6006ac90 0x7f6f6006d150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009b40 tx=0x7f6f78009210 comp rx=0 tx=0).stop 2024-05-08T23:21:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c15dc40 msgr2=0x7f6f7c1f69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c15dc40 0x7f6f7c1f69f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009fd0 tx=0x7f6f6c004750 comp rx=0 tx=0).stop 2024-05-08T23:21:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 shutdown_connections 2024-05-08T23:21:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f6006ac90 0x7f6f6006d150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f7c158b70 0x7f6f7c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f7c099f90 0x7f6f7c0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 --2- 172.21.15.5:0/2917810040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7c15dc40 0x7f6f7c1f69f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 >> 172.21.15.5:0/2917810040 conn(0x7f6f7c0b2f20 msgr2=0x7f6f7c15b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 shutdown_connections 2024-05-08T23:21:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.469+0000 7f6f83349700 1 -- 172.21.15.5:0/2917810040 wait complete. 2024-05-08T23:21:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:21:51.482 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-08T23:21:51.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:51.483 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-08T23:21:51.525 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:21:51.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:21:51.526 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T23:21:51.586 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T23:21:51.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.728+0000 7ff44da0f700 1 Processor -- start 2024-05-08T23:21:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.729+0000 7ff44da0f700 1 -- start start 2024-05-08T23:21:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.729+0000 7ff44da0f700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff44815d9f0 0x7ff4480d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.729+0000 7ff44da0f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4480cfb50 con 0x7ff44806b070 2024-05-08T23:21:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.729+0000 7ff44da0f700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4480cf560 con 0x7ff4480af4a0 2024-05-08T23:21:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.729+0000 7ff44da0f700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44815ddd0 con 0x7ff44815d9f0 2024-05-08T23:21:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff4477fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff44806b070 0x7ff4481614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59880/0 (socket says 172.21.15.5:59880) 2024-05-08T23:21:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff4477fe700 1 -- 172.21.15.5:0/2996163392 learned_addr learned my addr 172.21.15.5:0/2996163392 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff446ffd700 1 --2- 172.21.15.5:0/2996163392 >> v2:172.21.15.49:3300/0 conn(0x7ff44815d9f0 0x7ff4480d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191646370 0 0) 0x7ff4480cfb50 con 0x7ff44806b070 2024-05-08T23:21:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4300035b0 con 0x7ff44806b070 2024-05-08T23:21:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 729053902 0 0) 0x7ff4300035b0 con 0x7ff44806b070 2024-05-08T23:21:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 >> v2:172.21.15.49:3300/0 conn(0x7ff44815d9f0 msgr2=0x7ff4480d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 --2- 172.21.15.5:0/2996163392 >> v2:172.21.15.49:3300/0 conn(0x7ff44815d9f0 0x7ff4480d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 >> v1:172.21.15.49:6789/0 conn(0x7ff4480af4a0 legacy=0x7ff44806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4480cc460 con 0x7ff44806b070 2024-05-08T23:21:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7ff4400019f0 con 0x7ff44806b070 2024-05-08T23:21:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 >> v1:172.21.15.5:6789/0 conn(0x7ff44806b070 legacy=0x7ff4481614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430003a00 0x7ff430003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff430004850 0x7ff430004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff430005210 0x7ff43000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44815ddd0 con 0x7ff430004850 2024-05-08T23:21:51.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43000c1b0 con 0x7ff430003a00 2024-05-08T23:21:51.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43000c370 con 0x7ff430005210 2024-05-08T23:21:51.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff4467fc700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff430004850 0x7ff430004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.730+0000 7ff446ffd700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430003a00 0x7ff430003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff4477fe700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff430005210 0x7ff43000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff4467fc700 1 -- 172.21.15.5:0/2996163392 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff430005210 msgr2=0x7ff43000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:51.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff4467fc700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff430005210 0x7ff43000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff4467fc700 1 -- 172.21.15.5:0/2996163392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430003a00 msgr2=0x7ff430003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:51.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff4467fc700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430003a00 0x7ff430003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff4467fc700 1 -- 172.21.15.5:0/2996163392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4300035b0 con 0x7ff430004850 2024-05-08T23:21:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff4477fe700 1 -- 172.21.15.5:0/2996163392 reap_dead start 2024-05-08T23:21:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff4467fc700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff430004850 0x7ff430004cd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0017c0 tx=0x7ff430005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a57a802585231ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0052d0 con 0x7ff430004850 2024-05-08T23:21:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c005450 con 0x7ff430004850 2024-05-08T23:21:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.731+0000 7ff445ffb700 1 -- 172.21.15.5:0/2996163392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0038c0 con 0x7ff430004850 2024-05-08T23:21:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.732+0000 7ff44da0f700 1 -- 172.21.15.5:0/2996163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff430004850 msgr2=0x7ff430004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.732+0000 7ff44da0f700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff430004850 0x7ff430004cd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0017c0 tx=0x7ff430005880 comp rx=0 tx=0).stop 2024-05-08T23:21:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.732+0000 7ff44da0f700 1 -- 172.21.15.5:0/2996163392 shutdown_connections 2024-05-08T23:21:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.732+0000 7ff44da0f700 1 --2- 172.21.15.5:0/2996163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff430004850 0x7ff430004cd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.732+0000 7ff44da0f700 1 -- 172.21.15.5:0/2996163392 >> 172.21.15.5:0/2996163392 conn(0x7ff4480d02f0 msgr2=0x7ff4480d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.732+0000 7ff44da0f700 1 -- 172.21.15.5:0/2996163392 shutdown_connections 2024-05-08T23:21:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.732+0000 7ff44da0f700 1 -- 172.21.15.5:0/2996163392 wait complete. 2024-05-08T23:21:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.733+0000 7ff44da0f700 1 Processor -- start 2024-05-08T23:21:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.733+0000 7ff44da0f700 1 -- start start 2024-05-08T23:21:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff44da0f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430004850 0x7ff448155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff44da0f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4480b5880 0x7ff44815fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff44da0f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44815f7c0 0x7ff448160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff44da0f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480c48b0 con 0x7ff44815f7c0 2024-05-08T23:21:51.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff44da0f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480b5df0 con 0x7ff430004850 2024-05-08T23:21:51.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff44da0f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44815eac0 con 0x7ff4480b5880 2024-05-08T23:21:51.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44815f7c0 0x7ff448160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44815f7c0 0x7ff448160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44968/0 (socket says 172.21.15.5:44968) 2024-05-08T23:21:51.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff4477fe700 1 -- 172.21.15.5:0/3495011832 learned_addr learned my addr 172.21.15.5:0/3495011832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:51.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff446ffd700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430004850 0x7ff448155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff4467fc700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4480b5880 0x7ff44815fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff4477fe700 1 -- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4480b5880 msgr2=0x7ff44815fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff4477fe700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4480b5880 0x7ff44815fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff4477fe700 1 -- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430004850 msgr2=0x7ff448155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.734+0000 7ff4477fe700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430004850 0x7ff448155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff4477fe700 1 -- 172.21.15.5:0/3495011832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44815eda0 con 0x7ff44815f7c0 2024-05-08T23:21:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff446ffd700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430004850 0x7ff448155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff4467fc700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4480b5880 0x7ff44815fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff4477fe700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44815f7c0 0x7ff448160250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4480b6240 tx=0x7ff440007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff42ffff700 1 -- 172.21.15.5:0/3495011832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440017070 con 0x7ff44815f7c0 2024-05-08T23:21:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff42ffff700 1 -- 172.21.15.5:0/3495011832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44000ecb0 con 0x7ff44815f7c0 2024-05-08T23:21:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44815f080 con 0x7ff44815f7c0 2024-05-08T23:21:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff42ffff700 1 -- 172.21.15.5:0/3495011832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000ee40 con 0x7ff44815f7c0 2024-05-08T23:21:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.735+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4480c62b0 con 0x7ff44815f7c0 2024-05-08T23:21:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.736+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44815f5a0 con 0x7ff44815f7c0 2024-05-08T23:21:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.736+0000 7ff42ffff700 1 -- 172.21.15.5:0/3495011832 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff440013420 con 0x7ff44815f7c0 2024-05-08T23:21:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.737+0000 7ff42ffff700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff43406ac90 0x7ff43406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.737+0000 7ff42ffff700 1 -- 172.21.15.5:0/3495011832 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff440089020 con 0x7ff44815f7c0 2024-05-08T23:21:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.737+0000 7ff446ffd700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff43406ac90 0x7ff43406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.738+0000 7ff446ffd700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff43406ac90 0x7ff43406d150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff448069190 tx=0x7ff438008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:51.742+0000 7ff42ffff700 1 -- 172.21.15.5:0/3495011832 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff44815f5a0 con 0x7ff44815f7c0 2024-05-08T23:21:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.010+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff44815f5a0 con 0x7ff44815f7c0 2024-05-08T23:21:52.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.011+0000 7ff42ffff700 1 -- 172.21.15.5:0/3495011832 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4719 (secure 0 0 0) 0x7ff440054290 con 0x7ff44815f7c0 2024-05-08T23:21:52.011 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:52.011 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:21:04.087368+0000","modified":"2024-05-08T23:21:07.094437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff43406ac90 msgr2=0x7ff43406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff43406ac90 0x7ff43406d150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff448069190 tx=0x7ff438008040 comp rx=0 tx=0).stop 2024-05-08T23:21:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44815f7c0 msgr2=0x7ff448160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44815f7c0 0x7ff448160250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4480b6240 tx=0x7ff440007c90 comp rx=0 tx=0).stop 2024-05-08T23:21:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 shutdown_connections 2024-05-08T23:21:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff43406ac90 0x7ff43406d150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44815f7c0 0x7ff448160250 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff430004850 0x7ff448155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 --2- 172.21.15.5:0/3495011832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4480b5880 0x7ff44815fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 >> 172.21.15.5:0/3495011832 conn(0x7ff4480d02f0 msgr2=0x7ff4480d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 shutdown_connections 2024-05-08T23:21:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.013+0000 7ff44da0f700 1 -- 172.21.15.5:0/3495011832 wait complete. 2024-05-08T23:21:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-08T23:21:52.027 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:52.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.166+0000 7fb7fe8ef700 1 Processor -- start 2024-05-08T23:21:52.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.166+0000 7fb7fe8ef700 1 -- start start 2024-05-08T23:21:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.166+0000 7fb7fe8ef700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7f815d7a0 0x7fb7f8161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.166+0000 7fb7fe8ef700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f8162450 con 0x7fb7f815db80 2024-05-08T23:21:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.166+0000 7fb7fe8ef700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80cee00 con 0x7fb7f80d3f40 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.166+0000 7fb7fe8ef700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f80b27e0 con 0x7fb7f815d7a0 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7fce8c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb7f815db80 0x7fb7f80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59894/0 (socket says 172.21.15.5:59894) 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7fce8c700 1 -- 172.21.15.5:0/2784185450 learned_addr learned my addr 172.21.15.5:0/2784185450 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7f7fff700 1 --2- 172.21.15.5:0/2784185450 >> v2:172.21.15.49:3300/0 conn(0x7fb7f815d7a0 0x7fb7f8161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7f6ffd700 1 -- 172.21.15.5:0/2784185450 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100347108 0 0) 0x7fb7f0001490 con 0x7fb7f815db80 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7f6ffd700 1 -- 172.21.15.5:0/2784185450 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e00035b0 con 0x7fb7f815db80 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7f7fff700 1 -- 172.21.15.5:0/2784185450 >> v1:172.21.15.49:6789/0 conn(0x7fb7f80d3f40 legacy=0x7fb7f80b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7f7fff700 1 -- 172.21.15.5:0/2784185450 >> v1:172.21.15.5:6789/0 conn(0x7fb7f815db80 legacy=0x7fb7f80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7f7fff700 1 -- 172.21.15.5:0/2784185450 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f80d4400 con 0x7fb7f815d7a0 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7f7fff700 1 --2- 172.21.15.5:0/2784185450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d7a0 0x7fb7f8161f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009840 tx=0x7fb7e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d7a73a28eab88d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.167+0000 7fb7f6ffd700 1 -- 172.21.15.5:0/2784185450 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8011830 con 0x7fb7f815d7a0 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7f6ffd700 1 -- 172.21.15.5:0/2784185450 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e8011990 con 0x7fb7f815d7a0 2024-05-08T23:21:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7f6ffd700 1 -- 172.21.15.5:0/2784185450 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8011d20 con 0x7fb7f815d7a0 2024-05-08T23:21:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/2784185450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d7a0 msgr2=0x7fb7f8161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7fe8ef700 1 --2- 172.21.15.5:0/2784185450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d7a0 0x7fb7f8161f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009840 tx=0x7fb7e8009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/2784185450 shutdown_connections 2024-05-08T23:21:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7fe8ef700 1 --2- 172.21.15.5:0/2784185450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d7a0 0x7fb7f8161f10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/2784185450 >> 172.21.15.5:0/2784185450 conn(0x7fb7f808afe0 msgr2=0x7fb7f80d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/2784185450 shutdown_connections 2024-05-08T23:21:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.168+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/2784185450 wait complete. 2024-05-08T23:21:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.169+0000 7fb7fe8ef700 1 Processor -- start 2024-05-08T23:21:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.170+0000 7fb7fe8ef700 1 -- start start 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.170+0000 7fb7fe8ef700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f80d3f40 0x7fb7f80c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.170+0000 7fb7fe8ef700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7f815d7a0 0x7fb7f814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.170+0000 7fb7fe8ef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f815db80 0x7fb7f80b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.170+0000 7fb7fe8ef700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f80c7a40 con 0x7fb7f815db80 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.170+0000 7fb7fe8ef700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f815f240 con 0x7fb7f80d3f40 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.170+0000 7fb7fe8ef700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f815f520 con 0x7fb7f815d7a0 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f815db80 0x7fb7f80b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f815db80 0x7fb7f80b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44984/0 (socket says 172.21.15.5:44984) 2024-05-08T23:21:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 -- 172.21.15.5:0/997166080 learned_addr learned my addr 172.21.15.5:0/997166080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7f7fff700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f80d3f40 0x7fb7f80c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7f77fe700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7f815d7a0 0x7fb7f814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 -- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7f815d7a0 msgr2=0x7fb7f814da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7f815d7a0 0x7fb7f814da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 -- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f80d3f40 msgr2=0x7fb7f80c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f80d3f40 0x7fb7f80c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 -- 172.21.15.5:0/997166080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80094a0 con 0x7fb7f815db80 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fce8c700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f815db80 0x7fb7f80b3fa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000cae0 tx=0x7fb7f000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7f4ff9700 1 -- 172.21.15.5:0/997166080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000aa80 con 0x7fb7f815db80 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7f4ff9700 1 -- 172.21.15.5:0/997166080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f000abe0 con 0x7fb7f815db80 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f815f800 con 0x7fb7f815db80 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7f4ff9700 1 -- 172.21.15.5:0/997166080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0012400 con 0x7fb7f815db80 2024-05-08T23:21:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.171+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f814dfa0 con 0x7fb7f815db80 2024-05-08T23:21:52.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.172+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f815fd20 con 0x7fb7f815db80 2024-05-08T23:21:52.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.173+0000 7fb7f4ff9700 1 -- 172.21.15.5:0/997166080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb7f001b3b0 con 0x7fb7f815db80 2024-05-08T23:21:52.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.173+0000 7fb7f4ff9700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb7e406ac90 0x7fb7e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.173+0000 7fb7f4ff9700 1 -- 172.21.15.5:0/997166080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7f0088ec0 con 0x7fb7f815db80 2024-05-08T23:21:52.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.174+0000 7fb7f7fff700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb7e406ac90 0x7fb7e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.174+0000 7fb7f7fff700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb7e406ac90 0x7fb7e406d150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80090b0 tx=0x7fb7e8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.178+0000 7fb7f4ff9700 1 -- 172.21.15.5:0/997166080 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb7f815fd20 con 0x7fb7f815db80 2024-05-08T23:21:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.401+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7f8162450 con 0x7fb7f815db80 2024-05-08T23:21:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.402+0000 7fb7f4ff9700 1 -- 172.21.15.5:0/997166080 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16176 (secure 0 0 0) 0x7fb7f8162450 con 0x7fb7f815db80 2024-05-08T23:21:52.402 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:52.402 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:52.243971+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb7e406ac90 msgr2=0x7fb7e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb7e406ac90 0x7fb7e406d150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80090b0 tx=0x7fb7e8008030 comp rx=0 tx=0).stop 2024-05-08T23:21:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f815db80 msgr2=0x7fb7f80b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f815db80 0x7fb7f80b3fa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000cae0 tx=0x7fb7f000cea0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 shutdown_connections 2024-05-08T23:21:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb7e406ac90 0x7fb7e406d150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f815db80 0x7fb7f80b3fa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f80d3f40 0x7fb7f80c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 --2- 172.21.15.5:0/997166080 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7f815d7a0 0x7fb7f814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 >> 172.21.15.5:0/997166080 conn(0x7fb7f808afe0 msgr2=0x7fb7f806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 shutdown_connections 2024-05-08T23:21:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.404+0000 7fb7fe8ef700 1 -- 172.21.15.5:0/997166080 wait complete. 2024-05-08T23:21:52.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:21:02.035577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:21:03.055037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:21:52.418 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:21:52.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.558+0000 7f10c158e700 1 Processor -- start 2024-05-08T23:21:52.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.558+0000 7f10c158e700 1 -- start start 2024-05-08T23:21:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.559+0000 7f10c158e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f10bc15d900 0x7f10bc161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.559+0000 7f10c158e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10bc06c080 0x7f10bc0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.559+0000 7f10c158e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc15dce0 con 0x7f10bc0cda40 2024-05-08T23:21:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.559+0000 7f10c158e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc0ce5b0 con 0x7f10bc06c080 2024-05-08T23:21:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.559+0000 7f10c158e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc161510 con 0x7f10bc15d900 2024-05-08T23:21:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.559+0000 7f10bb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10bc06c080 0x7f10bc0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.559+0000 7f10bb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10bc06c080 0x7f10bc0cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44994/0 (socket says 172.21.15.5:44994) 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.559+0000 7f10bb7fe700 1 -- 172.21.15.5:0/2799289023 learned_addr learned my addr 172.21.15.5:0/2799289023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10baffd700 1 --2- 172.21.15.5:0/2799289023 >> v2:172.21.15.49:3300/0 conn(0x7f10bc15d900 0x7f10bc161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10bb7fe700 1 -- 172.21.15.5:0/2799289023 >> v2:172.21.15.49:3300/0 conn(0x7f10bc15d900 msgr2=0x7f10bc161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10bb7fe700 1 --2- 172.21.15.5:0/2799289023 >> v2:172.21.15.49:3300/0 conn(0x7f10bc15d900 0x7f10bc161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10bb7fe700 1 -- 172.21.15.5:0/2799289023 >> v1:172.21.15.49:6789/0 conn(0x7f10bc0cda40 legacy=0x7f10bc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10bb7fe700 1 -- 172.21.15.5:0/2799289023 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc1617f0 con 0x7f10bc06c080 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10bb7fe700 1 --2- 172.21.15.5:0/2799289023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc06c080 0x7f10bc0cdef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f10bc066960 tx=0x7f10b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f702876db22f613 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10b9ffb700 1 -- 172.21.15.5:0/2799289023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b400e070 con 0x7f10bc06c080 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10b9ffb700 1 -- 172.21.15.5:0/2799289023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b400b910 con 0x7f10bc06c080 2024-05-08T23:21:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.560+0000 7f10b9ffb700 1 -- 172.21.15.5:0/2799289023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b400bcc0 con 0x7f10bc06c080 2024-05-08T23:21:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.561+0000 7f10c158e700 1 -- 172.21.15.5:0/2799289023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc06c080 msgr2=0x7f10bc0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.561+0000 7f10c158e700 1 --2- 172.21.15.5:0/2799289023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc06c080 0x7f10bc0cdef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f10bc066960 tx=0x7f10b400b330 comp rx=0 tx=0).stop 2024-05-08T23:21:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.561+0000 7f10c158e700 1 -- 172.21.15.5:0/2799289023 shutdown_connections 2024-05-08T23:21:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.561+0000 7f10c158e700 1 --2- 172.21.15.5:0/2799289023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc06c080 0x7f10bc0cdef0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.561+0000 7f10c158e700 1 --2- 172.21.15.5:0/2799289023 >> v2:172.21.15.49:3300/0 conn(0x7f10bc15d900 0x7f10bc161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.561+0000 7f10c158e700 1 -- 172.21.15.5:0/2799289023 >> 172.21.15.5:0/2799289023 conn(0x7f10bc0713c0 msgr2=0x7f10bc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:52.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.561+0000 7f10c158e700 1 -- 172.21.15.5:0/2799289023 shutdown_connections 2024-05-08T23:21:52.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.561+0000 7f10c158e700 1 -- 172.21.15.5:0/2799289023 wait complete. 2024-05-08T23:21:52.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.563+0000 7f10c158e700 1 Processor -- start 2024-05-08T23:21:52.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.563+0000 7f10c158e700 1 -- start start 2024-05-08T23:21:52.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10c158e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10bc06c080 0x7f10bc159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10c158e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc0cda40 0x7f10bc15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10c158e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10bc15d900 0x7f10bc157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10c158e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc1f3700 con 0x7f10bc0cda40 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10c158e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc1f39e0 con 0x7f10bc15d900 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10c158e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc1f3cc0 con 0x7f10bc06c080 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc0cda40 0x7f10bc15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc0cda40 0x7f10bc15c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45004/0 (socket says 172.21.15.5:45004) 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 -- 172.21.15.5:0/1207874229 learned_addr learned my addr 172.21.15.5:0/1207874229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10baffd700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10bc06c080 0x7f10bc159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10bb7fe700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10bc15d900 0x7f10bc157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 -- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10bc06c080 msgr2=0x7f10bc159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10bc06c080 0x7f10bc159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 -- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10bc15d900 msgr2=0x7f10bc157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10bc15d900 0x7f10bc157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 -- 172.21.15.5:0/1207874229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b4009b30 con 0x7f10bc0cda40 2024-05-08T23:21:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10bb7fe700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10bc15d900 0x7f10bc157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:52.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.564+0000 7f10ba7fc700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc0cda40 0x7f10bc15c9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c4e0 tx=0x7f10b000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.565+0000 7f10a37fe700 1 -- 172.21.15.5:0/1207874229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00052d0 con 0x7f10bc0cda40 2024-05-08T23:21:52.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.565+0000 7f10a37fe700 1 -- 172.21.15.5:0/1207874229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0005430 con 0x7f10bc0cda40 2024-05-08T23:21:52.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.565+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc1f3fa0 con 0x7f10bc0cda40 2024-05-08T23:21:52.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.565+0000 7f10a37fe700 1 -- 172.21.15.5:0/1207874229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00038c0 con 0x7f10bc0cda40 2024-05-08T23:21:52.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.565+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc15cf00 con 0x7f10bc0cda40 2024-05-08T23:21:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.566+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc1f4190 con 0x7f10bc0cda40 2024-05-08T23:21:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.566+0000 7f10a37fe700 1 -- 172.21.15.5:0/1207874229 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f10b0003a20 con 0x7f10bc0cda40 2024-05-08T23:21:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.567+0000 7f10a37fe700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10a806ac90 0x7f10a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.567+0000 7f10a37fe700 1 -- 172.21.15.5:0/1207874229 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f10b0088880 con 0x7f10bc0cda40 2024-05-08T23:21:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.567+0000 7f10baffd700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10a806ac90 0x7f10a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.568+0000 7f10baffd700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10a806ac90 0x7f10a806d150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0b0e70 tx=0x7f10ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.571+0000 7f10a37fe700 1 -- 172.21.15.5:0/1207874229 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f10b00539c0 con 0x7f10bc0cda40 2024-05-08T23:21:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.787+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10bc1f3840 con 0x7f10bc0cda40 2024-05-08T23:21:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.787+0000 7f10a37fe700 1 -- 172.21.15.5:0/1207874229 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10b0017020 con 0x7f10bc0cda40 2024-05-08T23:21:52.787 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:21:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10a806ac90 msgr2=0x7f10a806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10a806ac90 0x7f10a806d150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0b0e70 tx=0x7f10ac008040 comp rx=0 tx=0).stop 2024-05-08T23:21:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc0cda40 msgr2=0x7f10bc15c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc0cda40 0x7f10bc15c9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c4e0 tx=0x7f10b000c8a0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 shutdown_connections 2024-05-08T23:21:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10a806ac90 0x7f10a806d150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10bc15d900 0x7f10bc157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10bc06c080 0x7f10bc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 --2- 172.21.15.5:0/1207874229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10bc0cda40 0x7f10bc15c9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 >> 172.21.15.5:0/1207874229 conn(0x7f10bc0713c0 msgr2=0x7f10bc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 shutdown_connections 2024-05-08T23:21:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.790+0000 7f10c158e700 1 -- 172.21.15.5:0/1207874229 wait complete. 2024-05-08T23:21:52.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:21:52.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.944+0000 7f6e3dca1700 1 Processor -- start 2024-05-08T23:21:52.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.944+0000 7f6e3dca1700 1 -- start start 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.944+0000 7f6e3dca1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6e3815b4e0 0x7f6e380b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.944+0000 7f6e3dca1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e3815b8c0 0x7f6e380d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.944+0000 7f6e3dca1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 0x7f6e380cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.944+0000 7f6e3dca1700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e380cefe0 con 0x7f6e3815b8c0 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.944+0000 7f6e3dca1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e380cf560 con 0x7f6e3815b4e0 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.944+0000 7f6e3dca1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380ccff0 con 0x7f6e380d1740 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e3815b8c0 0x7f6e380d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e3815b8c0 0x7f6e380d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45016/0 (socket says 172.21.15.5:45016) 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 -- 172.21.15.5:0/1296339717 learned_addr learned my addr 172.21.15.5:0/1296339717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e37fff700 1 --2- 172.21.15.5:0/1296339717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 0x7f6e380cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e377fe700 1 --2- 172.21.15.5:0/1296339717 >> v2:172.21.15.49:3300/0 conn(0x7f6e3815b4e0 0x7f6e380b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 -- 172.21.15.5:0/1296339717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 msgr2=0x7f6e380cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 --2- 172.21.15.5:0/1296339717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 0x7f6e380cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 -- 172.21.15.5:0/1296339717 >> v2:172.21.15.49:3300/0 conn(0x7f6e3815b4e0 msgr2=0x7f6e380b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 --2- 172.21.15.5:0/1296339717 >> v2:172.21.15.49:3300/0 conn(0x7f6e3815b4e0 0x7f6e380b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 -- 172.21.15.5:0/1296339717 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380cd2d0 con 0x7f6e3815b8c0 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e377fe700 1 --2- 172.21.15.5:0/1296339717 >> v2:172.21.15.49:3300/0 conn(0x7f6e3815b4e0 0x7f6e380b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e36ffd700 1 --2- 172.21.15.5:0/1296339717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e380d1200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e380b0f90 tx=0x7f6e2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e0b677ba569d68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e367fc700 1 -- 172.21.15.5:0/1296339717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00b870 con 0x7f6e3815b8c0 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e367fc700 1 -- 172.21.15.5:0/1296339717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c00b9d0 con 0x7f6e3815b8c0 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.945+0000 7f6e367fc700 1 -- 172.21.15.5:0/1296339717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00bd60 con 0x7f6e3815b8c0 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 -- 172.21.15.5:0/1296339717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 msgr2=0x7f6e380d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/1296339717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e380d1200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e380b0f90 tx=0x7f6e2c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 -- 172.21.15.5:0/1296339717 shutdown_connections 2024-05-08T23:21:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/1296339717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 0x7f6e380cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/1296339717 >> v2:172.21.15.49:3300/0 conn(0x7f6e3815b4e0 0x7f6e380b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/1296339717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e380d1200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 -- 172.21.15.5:0/1296339717 >> 172.21.15.5:0/1296339717 conn(0x7f6e3808afe0 msgr2=0x7f6e3808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 -- 172.21.15.5:0/1296339717 shutdown_connections 2024-05-08T23:21:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.946+0000 7f6e3dca1700 1 -- 172.21.15.5:0/1296339717 wait complete. 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.947+0000 7f6e3dca1700 1 Processor -- start 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.948+0000 7f6e3dca1700 1 -- start start 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.948+0000 7f6e3dca1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 0x7f6e381f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.948+0000 7f6e3dca1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e3815b4e0 0x7f6e38154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.948+0000 7f6e3dca1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e38157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.948+0000 7f6e3dca1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3815da70 con 0x7f6e3815b8c0 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.948+0000 7f6e3dca1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3815ce60 con 0x7f6e3815b4e0 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.948+0000 7f6e3dca1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3815d140 con 0x7f6e380d1740 2024-05-08T23:21:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e38157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e38157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45022/0 (socket says 172.21.15.5:45022) 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 -- 172.21.15.5:0/2205020212 learned_addr learned my addr 172.21.15.5:0/2205020212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e377fe700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 0x7f6e381f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e36ffd700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e3815b4e0 0x7f6e38154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 -- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 msgr2=0x7f6e381f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 0x7f6e381f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 -- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e3815b4e0 msgr2=0x7f6e38154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e3815b4e0 0x7f6e38154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 -- 172.21.15.5:0/2205020212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c009550 con 0x7f6e3815b8c0 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e37fff700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e38157c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000cd70 tx=0x7f6e3000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e1ffff700 1 -- 172.21.15.5:0/2205020212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30003a20 con 0x7f6e3815b8c0 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e1ffff700 1 -- 172.21.15.5:0/2205020212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30003b80 con 0x7f6e3815b8c0 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3815d420 con 0x7f6e3815b8c0 2024-05-08T23:21:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e1ffff700 1 -- 172.21.15.5:0/2205020212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30004020 con 0x7f6e3815b8c0 2024-05-08T23:21:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.949+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e38154d90 con 0x7f6e3815b8c0 2024-05-08T23:21:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.950+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e380ccbe0 con 0x7f6e3815b8c0 2024-05-08T23:21:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.951+0000 7f6e1ffff700 1 -- 172.21.15.5:0/2205020212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6e30004180 con 0x7f6e3815b8c0 2024-05-08T23:21:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.951+0000 7f6e1ffff700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e2406ac90 0x7f6e2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.951+0000 7f6e1ffff700 1 -- 172.21.15.5:0/2205020212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6e30012070 con 0x7f6e3815b8c0 2024-05-08T23:21:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.952+0000 7f6e377fe700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e2406ac90 0x7f6e2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.952+0000 7f6e377fe700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e2406ac90 0x7f6e2406d150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006fd0 tx=0x7f6e28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:52.956+0000 7f6e1ffff700 1 -- 172.21.15.5:0/2205020212 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6e30054be0 con 0x7f6e3815b8c0 2024-05-08T23:21:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.177+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e380d08d0 con 0x7f6e3815b8c0 2024-05-08T23:21:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.178+0000 7f6e1ffff700 1 -- 172.21.15.5:0/2205020212 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e30058200 con 0x7f6e3815b8c0 2024-05-08T23:21:53.177 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.180+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e2406ac90 msgr2=0x7f6e2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.180+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e2406ac90 0x7f6e2406d150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006fd0 tx=0x7f6e28008040 comp rx=0 tx=0).stop 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.180+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 msgr2=0x7f6e38157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.180+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e38157c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000cd70 tx=0x7f6e3000a570 comp rx=0 tx=0).stop 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.181+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 shutdown_connections 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.181+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e2406ac90 0x7f6e2406d150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.181+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e3815b8c0 0x7f6e38157c10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.181+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e380d1740 0x7f6e381f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.181+0000 7f6e3dca1700 1 --2- 172.21.15.5:0/2205020212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e3815b4e0 0x7f6e38154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.181+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 >> 172.21.15.5:0/2205020212 conn(0x7f6e3808afe0 msgr2=0x7f6e380cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.181+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 shutdown_connections 2024-05-08T23:21:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.181+0000 7f6e3dca1700 1 -- 172.21.15.5:0/2205020212 wait complete. 2024-05-08T23:21:53.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:21:53.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.337+0000 7f7f3de94700 1 Processor -- start 2024-05-08T23:21:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.337+0000 7f7f3de94700 1 -- start start 2024-05-08T23:21:53.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.338+0000 7f7f3de94700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815a430 0x7f7f3806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.338+0000 7f7f3de94700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f3815ed40 0x7f7f380cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.338+0000 7f7f3de94700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f380cfa20 con 0x7f7f3815e0f0 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.338+0000 7f7f3de94700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3806e520 con 0x7f7f3815ed40 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.338+0000 7f7f3de94700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38066960 con 0x7f7f3815a430 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.338+0000 7f7f36ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f3815ed40 0x7f7f380cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.338+0000 7f7f36ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f3815ed40 0x7f7f380cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45034/0 (socket says 172.21.15.5:45034) 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f36ffd700 1 -- 172.21.15.5:0/1774895654 learned_addr learned my addr 172.21.15.5:0/1774895654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f377fe700 1 --2- 172.21.15.5:0/1774895654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815a430 0x7f7f3806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f36ffd700 1 -- 172.21.15.5:0/1774895654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815a430 msgr2=0x7f7f3806e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f36ffd700 1 --2- 172.21.15.5:0/1774895654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815a430 0x7f7f3806e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f36ffd700 1 -- 172.21.15.5:0/1774895654 >> v1:172.21.15.49:6789/0 conn(0x7f7f3815e0f0 legacy=0x7f7f380b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f36ffd700 1 -- 172.21.15.5:0/1774895654 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38066c40 con 0x7f7f3815ed40 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f377fe700 1 --2- 172.21.15.5:0/1774895654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815a430 0x7f7f3806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f36ffd700 1 --2- 172.21.15.5:0/1774895654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815ed40 0x7f7f380cf4e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f280098f0 tx=0x7f7f28009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ce1388ef9123e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f367fc700 1 -- 172.21.15.5:0/1774895654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800b8a0 con 0x7f7f3815ed40 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f367fc700 1 -- 172.21.15.5:0/1774895654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2800ba00 con 0x7f7f3815ed40 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.339+0000 7f7f367fc700 1 -- 172.21.15.5:0/1774895654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800bd90 con 0x7f7f3815ed40 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.340+0000 7f7f3de94700 1 -- 172.21.15.5:0/1774895654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815ed40 msgr2=0x7f7f380cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.340+0000 7f7f3de94700 1 --2- 172.21.15.5:0/1774895654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815ed40 0x7f7f380cf4e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f280098f0 tx=0x7f7f28009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.340+0000 7f7f3de94700 1 -- 172.21.15.5:0/1774895654 shutdown_connections 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.340+0000 7f7f3de94700 1 --2- 172.21.15.5:0/1774895654 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815a430 0x7f7f3806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.340+0000 7f7f3de94700 1 --2- 172.21.15.5:0/1774895654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815ed40 0x7f7f380cf4e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.340+0000 7f7f3de94700 1 -- 172.21.15.5:0/1774895654 >> 172.21.15.5:0/1774895654 conn(0x7f7f380cfbb0 msgr2=0x7f7f380cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.340+0000 7f7f3de94700 1 -- 172.21.15.5:0/1774895654 shutdown_connections 2024-05-08T23:21:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.340+0000 7f7f3de94700 1 -- 172.21.15.5:0/1774895654 wait complete. 2024-05-08T23:21:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.341+0000 7f7f3de94700 1 Processor -- start 2024-05-08T23:21:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f3de94700 1 -- start start 2024-05-08T23:21:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f3de94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3815a430 0x7f7f3814f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f3de94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815e0f0 0x7f7f380c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f3de94700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815ed40 0x7f7f380afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f3de94700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380c98f0 con 0x7f7f3815e0f0 2024-05-08T23:21:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f3de94700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380c9bd0 con 0x7f7f3815a430 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f3de94700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380c9eb0 con 0x7f7f3815ed40 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f36ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815e0f0 0x7f7f380c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f36ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815e0f0 0x7f7f380c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45046/0 (socket says 172.21.15.5:45046) 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.342+0000 7f7f36ffd700 1 -- 172.21.15.5:0/2082205051 learned_addr learned my addr 172.21.15.5:0/2082205051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f377fe700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3815a430 0x7f7f3814f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f37fff700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815ed40 0x7f7f380afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f36ffd700 1 -- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815ed40 msgr2=0x7f7f380afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f36ffd700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815ed40 0x7f7f380afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f36ffd700 1 -- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3815a430 msgr2=0x7f7f3814f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f36ffd700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3815a430 0x7f7f3814f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f36ffd700 1 -- 172.21.15.5:0/2082205051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f28009550 con 0x7f7f3815e0f0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f37fff700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815ed40 0x7f7f380afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f36ffd700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815e0f0 0x7f7f380c8fa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009c40 tx=0x7f7f28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f2ffff700 1 -- 172.21.15.5:0/2082205051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28004590 con 0x7f7f3815e0f0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f2ffff700 1 -- 172.21.15.5:0/2082205051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2801e070 con 0x7f7f3815e0f0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f380ca190 con 0x7f7f3815e0f0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f2ffff700 1 -- 172.21.15.5:0/2082205051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f280038e0 con 0x7f7f3815e0f0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.343+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3814fbf0 con 0x7f7f3815e0f0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.345+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f38157470 con 0x7f7f3815e0f0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.345+0000 7f7f2ffff700 1 -- 172.21.15.5:0/2082205051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7f28003a40 con 0x7f7f3815e0f0 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.345+0000 7f7f2ffff700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f1806ac90 0x7f7f1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.345+0000 7f7f2ffff700 1 -- 172.21.15.5:0/2082205051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7f28020070 con 0x7f7f3815e0f0 2024-05-08T23:21:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.345+0000 7f7f377fe700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f1806ac90 0x7f7f1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.346+0000 7f7f377fe700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f1806ac90 0x7f7f1806d150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f200099d0 tx=0x7f7f20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.350+0000 7f7f2ffff700 1 -- 172.21.15.5:0/2082205051 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7f38157470 con 0x7f7f3815e0f0 2024-05-08T23:21:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.567+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f38157850 con 0x7f7f3815e0f0 2024-05-08T23:21:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.568+0000 7f7f2ffff700 1 -- 172.21.15.5:0/2082205051 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f38157850 con 0x7f7f3815e0f0 2024-05-08T23:21:53.567 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:21:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f1806ac90 msgr2=0x7f7f1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f1806ac90 0x7f7f1806d150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f200099d0 tx=0x7f7f20008040 comp rx=0 tx=0).stop 2024-05-08T23:21:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815e0f0 msgr2=0x7f7f380c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815e0f0 0x7f7f380c8fa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009c40 tx=0x7f7f28004750 comp rx=0 tx=0).stop 2024-05-08T23:21:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 shutdown_connections 2024-05-08T23:21:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f1806ac90 0x7f7f1806d150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f3815ed40 0x7f7f380afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3815a430 0x7f7f3814f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 --2- 172.21.15.5:0/2082205051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f3815e0f0 0x7f7f380c8fa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 >> 172.21.15.5:0/2082205051 conn(0x7f7f380cfbb0 msgr2=0x7f7f380cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 shutdown_connections 2024-05-08T23:21:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.570+0000 7f7f3de94700 1 -- 172.21.15.5:0/2082205051 wait complete. 2024-05-08T23:21:53.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.720+0000 7f934a60a700 1 Processor -- start 2024-05-08T23:21:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.720+0000 7f934a60a700 1 -- start start 2024-05-08T23:21:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f934a60a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934406b140 0x7f93440ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f934a60a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f934406a960 0x7f934415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f934a60a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440cdbb0 con 0x7f934415c1c0 2024-05-08T23:21:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f934a60a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440cd310 con 0x7f934406a960 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f934a60a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93440cd5f0 con 0x7f934406b140 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f93437fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f934406a960 0x7f934415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f93437fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f934406a960 0x7f934415bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45048/0 (socket says 172.21.15.5:45048) 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f93437fe700 1 -- 172.21.15.5:0/3015099024 learned_addr learned my addr 172.21.15.5:0/3015099024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f9343fff700 1 --2- 172.21.15.5:0/3015099024 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934406b140 0x7f93440ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f93437fe700 1 -- 172.21.15.5:0/3015099024 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934406b140 msgr2=0x7f93440ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f93437fe700 1 --2- 172.21.15.5:0/3015099024 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934406b140 0x7f93440ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f93437fe700 1 -- 172.21.15.5:0/3015099024 >> v1:172.21.15.49:6789/0 conn(0x7f934415c1c0 legacy=0x7f93440cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.721+0000 7f93437fe700 1 -- 172.21.15.5:0/3015099024 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440cd8d0 con 0x7f934406a960 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f93437fe700 1 --2- 172.21.15.5:0/3015099024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406a960 0x7f934415bc80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f93380098f0 tx=0x7f9338009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b172ce4f0adefb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f9342ffd700 1 -- 172.21.15.5:0/3015099024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338011830 con 0x7f934406a960 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f9342ffd700 1 -- 172.21.15.5:0/3015099024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338011990 con 0x7f934406a960 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f9342ffd700 1 -- 172.21.15.5:0/3015099024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338011d20 con 0x7f934406a960 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f934a60a700 1 -- 172.21.15.5:0/3015099024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406a960 msgr2=0x7f934415bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f934a60a700 1 --2- 172.21.15.5:0/3015099024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406a960 0x7f934415bc80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f93380098f0 tx=0x7f9338009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f934a60a700 1 -- 172.21.15.5:0/3015099024 shutdown_connections 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f934a60a700 1 --2- 172.21.15.5:0/3015099024 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934406b140 0x7f93440ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f934a60a700 1 --2- 172.21.15.5:0/3015099024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406a960 0x7f934415bc80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f934a60a700 1 -- 172.21.15.5:0/3015099024 >> 172.21.15.5:0/3015099024 conn(0x7f93440d0320 msgr2=0x7f93440d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f934a60a700 1 -- 172.21.15.5:0/3015099024 shutdown_connections 2024-05-08T23:21:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.722+0000 7f934a60a700 1 -- 172.21.15.5:0/3015099024 wait complete. 2024-05-08T23:21:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.724+0000 7f934a60a700 1 Processor -- start 2024-05-08T23:21:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.724+0000 7f934a60a700 1 -- start start 2024-05-08T23:21:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f934a60a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f934406a960 0x7f934413f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f934a60a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406b140 0x7f93440b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f934a60a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934415c1c0 0x7f93440b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f934a60a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440b5ab0 con 0x7f934406b140 2024-05-08T23:21:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f934a60a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93441f0d70 con 0x7f934406a960 2024-05-08T23:21:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f934a60a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93440b3f90 con 0x7f934415c1c0 2024-05-08T23:21:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f93437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406b140 0x7f93440b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f93437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406b140 0x7f93440b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45062/0 (socket says 172.21.15.5:45062) 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f93437fe700 1 -- 172.21.15.5:0/933312813 learned_addr learned my addr 172.21.15.5:0/933312813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f9343fff700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f934406a960 0x7f934413f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f9348ba7700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934415c1c0 0x7f93440b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f93437fe700 1 -- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934415c1c0 msgr2=0x7f93440b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f93437fe700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934415c1c0 0x7f93440b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f93437fe700 1 -- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f934406a960 msgr2=0x7f934413f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f93437fe700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f934406a960 0x7f934413f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.725+0000 7f93437fe700 1 -- 172.21.15.5:0/933312813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338009550 con 0x7f934406b140 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.726+0000 7f93437fe700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406b140 0x7f93440b5030 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9338000c00 tx=0x7f9338004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.726+0000 7f9340ff9700 1 -- 172.21.15.5:0/933312813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338004570 con 0x7f934406b140 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.726+0000 7f9340ff9700 1 -- 172.21.15.5:0/933312813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933801f070 con 0x7f934406b140 2024-05-08T23:21:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.726+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93440c3cc0 con 0x7f934406b140 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.726+0000 7f9340ff9700 1 -- 172.21.15.5:0/933312813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380038c0 con 0x7f934406b140 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.726+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93440b4550 con 0x7f934406b140 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.727+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9344068000 con 0x7f934406b140 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.727+0000 7f9340ff9700 1 -- 172.21.15.5:0/933312813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9338021030 con 0x7f934406b140 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.728+0000 7f9340ff9700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f933006ac90 0x7f933006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.728+0000 7f9340ff9700 1 -- 172.21.15.5:0/933312813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f933808a8b0 con 0x7f934406b140 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.728+0000 7f9343fff700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f933006ac90 0x7f933006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.729+0000 7f9343fff700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f933006ac90 0x7f933006d150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9334009b70 tx=0x7f9334008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.733+0000 7f9340ff9700 1 -- 172.21.15.5:0/933312813 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9338055b10 con 0x7f934406b140 2024-05-08T23:21:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.996+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93440c47e0 con 0x7f934406b140 2024-05-08T23:21:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.996+0000 7f9340ff9700 1 -- 172.21.15.5:0/933312813 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4719 (secure 0 0 0) 0x7f9338015020 con 0x7f934406b140 2024-05-08T23:21:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:53.996 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:21:04.087368+0000","modified":"2024-05-08T23:21:07.094437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f933006ac90 msgr2=0x7f933006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f933006ac90 0x7f933006d150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9334009b70 tx=0x7f9334008040 comp rx=0 tx=0).stop 2024-05-08T23:21:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406b140 msgr2=0x7f93440b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406b140 0x7f93440b5030 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9338000c00 tx=0x7f9338004750 comp rx=0 tx=0).stop 2024-05-08T23:21:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 shutdown_connections 2024-05-08T23:21:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f933006ac90 0x7f933006d150 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f934415c1c0 0x7f93440b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f934406a960 0x7f934413f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 --2- 172.21.15.5:0/933312813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934406b140 0x7f93440b5030 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 >> 172.21.15.5:0/933312813 conn(0x7f93440d0320 msgr2=0x7f93440d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 shutdown_connections 2024-05-08T23:21:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:53.999+0000 7f934a60a700 1 -- 172.21.15.5:0/933312813 wait complete. 2024-05-08T23:21:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-08T23:21:54.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:54.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.149+0000 7f8ef9bbd700 1 Processor -- start 2024-05-08T23:21:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.149+0000 7f8ef9bbd700 1 -- start start 2024-05-08T23:21:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.150+0000 7f8ef9bbd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ef40d13e0 0x7f8ef40d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.150+0000 7f8ef9bbd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef415db60 0x7f8ef40d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.150+0000 7f8ef9bbd700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef40b4430 con 0x7f8ef415d780 2024-05-08T23:21:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.150+0000 7f8ef9bbd700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef40d45c0 con 0x7f8ef40d13e0 2024-05-08T23:21:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.150+0000 7f8ef9bbd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40b3820 con 0x7f8ef415db60 2024-05-08T23:21:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.150+0000 7f8ef37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8ef415d780 0x7f8ef40b48c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59898/0 (socket says 172.21.15.5:59898) 2024-05-08T23:21:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.150+0000 7f8ef37fe700 1 -- 172.21.15.5:0/3650442605 learned_addr learned my addr 172.21.15.5:0/3650442605 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef2ffd700 1 --2- 172.21.15.5:0/3650442605 >> v2:172.21.15.49:3300/0 conn(0x7f8ef40d13e0 0x7f8ef40d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef3fff700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef415db60 0x7f8ef40d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257011027 0 0) 0x7f8ef40b4430 con 0x7f8ef415d780 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8edc0035b0 con 0x7f8ef415d780 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3144241166 0 0) 0x7f8edc0035b0 con 0x7f8ef415d780 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef415db60 msgr2=0x7f8ef40d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef415db60 0x7f8ef40d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 >> v2:172.21.15.49:3300/0 conn(0x7f8ef40d13e0 msgr2=0x7f8ef40d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 --2- 172.21.15.5:0/3650442605 >> v2:172.21.15.49:3300/0 conn(0x7f8ef40d13e0 0x7f8ef40d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40b3b00 con 0x7f8ef415d780 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f8ee40017f0 con 0x7f8ef415d780 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 >> v1:172.21.15.5:6789/0 conn(0x7f8ef415d780 legacy=0x7f8ef40b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef415db60 0x7f8edc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8edc003f80 0x7f8edc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8edc004d90 0x7f8edc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40d45c0 con 0x7f8edc004d90 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40b3820 con 0x7f8edc003f80 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc00ba40 con 0x7f8ef415db60 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef2ffd700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8edc003f80 0x7f8edc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef37fe700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef415db60 0x7f8edc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.151+0000 7f8ef3fff700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8edc004d90 0x7f8edc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef3fff700 1 -- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef415db60 msgr2=0x7f8edc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef3fff700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef415db60 0x7f8edc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef3fff700 1 -- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8edc003f80 msgr2=0x7f8edc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef3fff700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8edc003f80 0x7f8edc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef3fff700 1 -- 172.21.15.5:0/3650442605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0035b0 con 0x7f8edc004d90 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef3fff700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8edc004d90 0x7f8edc005210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009fc0 tx=0x7f8edc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954c669c409a236d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec00ec30 con 0x7f8edc004d90 2024-05-08T23:21:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec00f040 con 0x7f8edc004d90 2024-05-08T23:21:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.152+0000 7f8ef27fc700 1 -- 172.21.15.5:0/3650442605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec014480 con 0x7f8edc004d90 2024-05-08T23:21:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.153+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/3650442605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8edc004d90 msgr2=0x7f8edc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.153+0000 7f8ef9bbd700 1 --2- 172.21.15.5:0/3650442605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8edc004d90 0x7f8edc005210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009fc0 tx=0x7f8edc005880 comp rx=0 tx=0).stop 2024-05-08T23:21:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.153+0000 7f8ef3fff700 1 -- 172.21.15.5:0/3650442605 reap_dead start 2024-05-08T23:21:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.153+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/3650442605 shutdown_connections 2024-05-08T23:21:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.153+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/3650442605 >> 172.21.15.5:0/3650442605 conn(0x7f8ef408afe0 msgr2=0x7f8ef40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.153+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/3650442605 shutdown_connections 2024-05-08T23:21:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.153+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/3650442605 wait complete. 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.154+0000 7f8ef9bbd700 1 Processor -- start 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.154+0000 7f8ef9bbd700 1 -- start start 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef9bbd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ef40c8d30 0x7f8ef41f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef9bbd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ef4160be0 0x7f8ef40b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef9bbd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef40c9110 0x7f8ef40b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef9bbd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40b8870 con 0x7f8ef4160be0 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef9bbd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40b77d0 con 0x7f8ef40c8d30 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef9bbd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40b7ab0 con 0x7f8ef40c9110 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ef4160be0 0x7f8ef40b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ef4160be0 0x7f8ef40b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45072/0 (socket says 172.21.15.5:45072) 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef2ffd700 1 -- 172.21.15.5:0/1466903409 learned_addr learned my addr 172.21.15.5:0/1466903409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef37fe700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ef40c8d30 0x7f8ef41f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef3fff700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef40c9110 0x7f8ef40b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef2ffd700 1 -- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef40c9110 msgr2=0x7f8ef40b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef2ffd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef40c9110 0x7f8ef40b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef2ffd700 1 -- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ef40c8d30 msgr2=0x7f8ef41f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef2ffd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ef40c8d30 0x7f8ef41f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef2ffd700 1 -- 172.21.15.5:0/1466903409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40b7d90 con 0x7f8ef4160be0 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.155+0000 7f8ef37fe700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ef40c8d30 0x7f8ef41f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.156+0000 7f8ef2ffd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ef4160be0 0x7f8ef40b12f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8011aa0 tx=0x7f8ee8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.156+0000 7f8edbfff700 1 -- 172.21.15.5:0/1466903409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800f980 con 0x7f8ef4160be0 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.156+0000 7f8edbfff700 1 -- 172.21.15.5:0/1466903409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee800fb00 con 0x7f8ef4160be0 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.156+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef40b8070 con 0x7f8ef4160be0 2024-05-08T23:21:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.156+0000 7f8edbfff700 1 -- 172.21.15.5:0/1466903409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800feb0 con 0x7f8ef4160be0 2024-05-08T23:21:54.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.156+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef40b7240 con 0x7f8ef4160be0 2024-05-08T23:21:54.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.157+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef40cdda0 con 0x7f8ef4160be0 2024-05-08T23:21:54.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.157+0000 7f8edbfff700 1 -- 172.21.15.5:0/1466903409 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8ee8014440 con 0x7f8ef4160be0 2024-05-08T23:21:54.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.158+0000 7f8edbfff700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ee006ac90 0x7f8ee006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.158+0000 7f8edbfff700 1 -- 172.21.15.5:0/1466903409 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8ee8089b40 con 0x7f8ef4160be0 2024-05-08T23:21:54.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.158+0000 7f8ef37fe700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ee006ac90 0x7f8ee006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.159+0000 7f8ef37fe700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ee006ac90 0x7f8ee006d150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4000ef0 tx=0x7f8ee4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.163+0000 7f8edbfff700 1 -- 172.21.15.5:0/1466903409 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8ee8054e00 con 0x7f8ef4160be0 2024-05-08T23:21:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.380+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ef40d1a50 con 0x7f8ef4160be0 2024-05-08T23:21:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.382+0000 7f8edbfff700 1 -- 172.21.15.5:0/1466903409 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16176 (secure 0 0 0) 0x7f8ee8058420 con 0x7f8ef4160be0 2024-05-08T23:21:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:54.381 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:53.247940+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ee006ac90 msgr2=0x7f8ee006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ee006ac90 0x7f8ee006d150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4000ef0 tx=0x7f8ee4009040 comp rx=0 tx=0).stop 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ef4160be0 msgr2=0x7f8ef40b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ef4160be0 0x7f8ef40b12f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8011aa0 tx=0x7f8ee8011e60 comp rx=0 tx=0).stop 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 shutdown_connections 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ee006ac90 0x7f8ee006d150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ef40c9110 0x7f8ef40b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ef40c8d30 0x7f8ef41f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 --2- 172.21.15.5:0/1466903409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ef4160be0 0x7f8ef40b12f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.384+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 >> 172.21.15.5:0/1466903409 conn(0x7f8ef408afe0 msgr2=0x7f8ef40d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:54.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.385+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 shutdown_connections 2024-05-08T23:21:54.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.385+0000 7f8ef9bbd700 1 -- 172.21.15.5:0/1466903409 wait complete. 2024-05-08T23:21:54.398 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:21:54.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.537+0000 7fbc856f7700 1 Processor -- start 2024-05-08T23:21:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.537+0000 7fbc856f7700 1 -- start start 2024-05-08T23:21:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.537+0000 7fbc856f7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800cf7a0 0x7fbc801686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.537+0000 7fbc856f7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc800d0880 con 0x7fbc8006b810 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.537+0000 7fbc856f7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc800d0b60 con 0x7fbc8015d9c0 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.537+0000 7fbc856f7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800ce5b0 con 0x7fbc800cf7a0 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7e7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbc8006b810 0x7fbc800cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59912/0 (socket says 172.21.15.5:59912) 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7e7fc700 1 -- 172.21.15.5:0/1169719432 learned_addr learned my addr 172.21.15.5:0/1169719432 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7f7fe700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800cf7a0 0x7fbc801686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505621184 0 0) 0x7fbc800d0880 con 0x7fbc8006b810 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc680035b0 con 0x7fbc8006b810 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221984278 0 0) 0x7fbc800d0b60 con 0x7fbc8015d9c0 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc800d0880 con 0x7fbc8015d9c0 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529300274 0 0) 0x7fbc680035b0 con 0x7fbc8006b810 2024-05-08T23:21:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800cf7a0 msgr2=0x7fbc801686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800cf7a0 0x7fbc801686e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 >> v1:172.21.15.49:6789/0 conn(0x7fbc8015d9c0 legacy=0x7fbc800cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800ce890 con 0x7fbc8006b810 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fbc740017f0 con 0x7fbc8006b810 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 >> v1:172.21.15.5:6789/0 conn(0x7fbc8006b810 legacy=0x7fbc800cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800cf7a0 0x7fbc68003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc68004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc68004d90 0x7fbc68005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800ce5b0 con 0x7fbc68004120 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6800baf0 con 0x7fbc68004d90 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6800bcb0 con 0x7fbc800cf7a0 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.538+0000 7fbc7e7fc700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc68004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7effd700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800cf7a0 0x7fbc68003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7f7fe700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc68004d90 0x7fbc68005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7e7fc700 1 -- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800cf7a0 msgr2=0x7fbc68003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7e7fc700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800cf7a0 0x7fbc68003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7e7fc700 1 -- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc68004d90 msgr2=0x7fbc68005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7e7fc700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc68004d90 0x7fbc68005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7e7fc700 1 -- 172.21.15.5:0/1169719432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680035b0 con 0x7fbc68004120 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7e7fc700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc68004850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400c240 tx=0x7fbc68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb937410cd075f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400cd80 con 0x7fbc68004120 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc74007b70 con 0x7fbc68004120 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.539+0000 7fbc7dffb700 1 -- 172.21.15.5:0/1169719432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74007310 con 0x7fbc68004120 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.540+0000 7fbc856f7700 1 -- 172.21.15.5:0/1169719432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 msgr2=0x7fbc68004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.540+0000 7fbc856f7700 1 --2- 172.21.15.5:0/1169719432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc68004850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400c240 tx=0x7fbc68005880 comp rx=0 tx=0).stop 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.540+0000 7fbc7f7fe700 1 -- 172.21.15.5:0/1169719432 reap_dead start 2024-05-08T23:21:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.540+0000 7fbc856f7700 1 -- 172.21.15.5:0/1169719432 shutdown_connections 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.540+0000 7fbc856f7700 1 -- 172.21.15.5:0/1169719432 >> 172.21.15.5:0/1169719432 conn(0x7fbc8008afe0 msgr2=0x7fbc800d3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.540+0000 7fbc856f7700 1 -- 172.21.15.5:0/1169719432 shutdown_connections 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.540+0000 7fbc856f7700 1 -- 172.21.15.5:0/1169719432 wait complete. 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.541+0000 7fbc856f7700 1 Processor -- start 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.541+0000 7fbc856f7700 1 -- start start 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc856f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc800b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc856f7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800ca710 0x7fbc800cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc856f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc800cce50 0x7fbc800cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc856f7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800cb0d0 con 0x7fbc68004120 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc856f7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800b46d0 con 0x7fbc800cce50 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc856f7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800b49b0 con 0x7fbc800ca710 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc7effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc800b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc7effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc800b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45090/0 (socket says 172.21.15.5:45090) 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.542+0000 7fbc7effd700 1 -- 172.21.15.5:0/4257370695 learned_addr learned my addr 172.21.15.5:0/4257370695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc7effd700 1 -- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800ca710 msgr2=0x7fbc800cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc7effd700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800ca710 0x7fbc800cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc7effd700 1 -- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc800cce50 msgr2=0x7fbc800cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc7f7fe700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc800cce50 0x7fbc800cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc7effd700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc800cce50 0x7fbc800cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc7effd700 1 -- 172.21.15.5:0/4257370695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800b4c90 con 0x7fbc68004120 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc7effd700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc800b3ff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000f7e0 tx=0x7fbc7000fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc677fe700 1 -- 172.21.15.5:0/4257370695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc700052e0 con 0x7fbc68004120 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc677fe700 1 -- 172.21.15.5:0/4257370695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc70005440 con 0x7fbc68004120 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc800b4f70 con 0x7fbc68004120 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc677fe700 1 -- 172.21.15.5:0/4257370695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc700038c0 con 0x7fbc68004120 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.543+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc800b5770 con 0x7fbc68004120 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.544+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc800b5490 con 0x7fbc68004120 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.545+0000 7fbc677fe700 1 -- 172.21.15.5:0/4257370695 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbc70003a20 con 0x7fbc68004120 2024-05-08T23:21:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.545+0000 7fbc677fe700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc6c06ac90 0x7fbc6c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.545+0000 7fbc677fe700 1 -- 172.21.15.5:0/4257370695 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbc70089300 con 0x7fbc68004120 2024-05-08T23:21:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.545+0000 7fbc7e7fc700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc6c06ac90 0x7fbc6c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.546+0000 7fbc7e7fc700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc6c06ac90 0x7fbc6c06d150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc74000c00 tx=0x7fbc7400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.550+0000 7fbc677fe700 1 -- 172.21.15.5:0/4257370695 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbc800b5490 con 0x7fbc68004120 2024-05-08T23:21:54.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.816+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc800b5490 con 0x7fbc68004120 2024-05-08T23:21:54.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.817+0000 7fbc677fe700 1 -- 172.21.15.5:0/4257370695 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4719 (secure 0 0 0) 0x7fbc800b5490 con 0x7fbc68004120 2024-05-08T23:21:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:54.817 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:21:04.087368+0000","modified":"2024-05-08T23:21:07.094437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:54.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc6c06ac90 msgr2=0x7fbc6c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc6c06ac90 0x7fbc6c06d150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc74000c00 tx=0x7fbc7400f000 comp rx=0 tx=0).stop 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 msgr2=0x7fbc800b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc800b3ff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000f7e0 tx=0x7fbc7000fba0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 shutdown_connections 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc6c06ac90 0x7fbc6c06d150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc800cce50 0x7fbc800cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc68004120 0x7fbc800b3ff0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 --2- 172.21.15.5:0/4257370695 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbc800ca710 0x7fbc800cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 >> 172.21.15.5:0/4257370695 conn(0x7fbc8008afe0 msgr2=0x7fbc8015dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 shutdown_connections 2024-05-08T23:21:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.819+0000 7fbc856f7700 1 -- 172.21.15.5:0/4257370695 wait complete. 2024-05-08T23:21:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-08T23:21:54.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:54.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.970+0000 7f2606230700 1 Processor -- start 2024-05-08T23:21:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.970+0000 7f2606230700 1 -- start start 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.971+0000 7f2606230700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f260015db00 0x7f260006dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.971+0000 7f2606230700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260006fbf0 con 0x7f26000cb2a0 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.971+0000 7f2606230700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260006edb0 con 0x7f26000ce120 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.971+0000 7f2606230700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260006f090 con 0x7f260015db00 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.971+0000 7f25ff7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f260015db00 0x7f260006dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.971+0000 7f25fffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f26000cb2a0 0x7f26000cb6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59928/0 (socket says 172.21.15.5:59928) 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.971+0000 7f25fffff700 1 -- 172.21.15.5:0/2303633653 learned_addr learned my addr 172.21.15.5:0/2303633653 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.972+0000 7f25ff7fe700 1 -- 172.21.15.5:0/2303633653 >> v1:172.21.15.49:6789/0 conn(0x7f26000ce120 legacy=0x7f260006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.972+0000 7f25ff7fe700 1 -- 172.21.15.5:0/2303633653 >> v1:172.21.15.5:6789/0 conn(0x7f26000cb2a0 legacy=0x7f26000cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.972+0000 7f25ff7fe700 1 -- 172.21.15.5:0/2303633653 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260006f370 con 0x7f260015db00 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.972+0000 7f25ff7fe700 1 --2- 172.21.15.5:0/2303633653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f260015db00 0x7f260006dc80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009840 tx=0x7f25f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2fd499bda113d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.972+0000 7f25fe7fc700 1 -- 172.21.15.5:0/2303633653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0011830 con 0x7f260015db00 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.972+0000 7f25fe7fc700 1 -- 172.21.15.5:0/2303633653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f0011990 con 0x7f260015db00 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.972+0000 7f25fe7fc700 1 -- 172.21.15.5:0/2303633653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0011d20 con 0x7f260015db00 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.973+0000 7f2606230700 1 -- 172.21.15.5:0/2303633653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f260015db00 msgr2=0x7f260006dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.973+0000 7f2606230700 1 --2- 172.21.15.5:0/2303633653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f260015db00 0x7f260006dc80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009840 tx=0x7f25f0009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.973+0000 7f2606230700 1 -- 172.21.15.5:0/2303633653 shutdown_connections 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.973+0000 7f2606230700 1 --2- 172.21.15.5:0/2303633653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f260015db00 0x7f260006dc80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.973+0000 7f2606230700 1 -- 172.21.15.5:0/2303633653 >> 172.21.15.5:0/2303633653 conn(0x7f2600099a50 msgr2=0x7f26000d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.973+0000 7f2606230700 1 -- 172.21.15.5:0/2303633653 shutdown_connections 2024-05-08T23:21:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.973+0000 7f2606230700 1 -- 172.21.15.5:0/2303633653 wait complete. 2024-05-08T23:21:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.974+0000 7f2606230700 1 Processor -- start 2024-05-08T23:21:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.974+0000 7f2606230700 1 -- start start 2024-05-08T23:21:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f2606230700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26000cb2a0 0x7f2600154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f2606230700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26000ce120 0x7f26000b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f2606230700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f260015db00 0x7f260015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f2606230700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260015c130 con 0x7f26000cb2a0 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f2606230700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260015b890 con 0x7f26000ce120 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f2606230700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260015bb70 con 0x7f260015db00 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26000cb2a0 0x7f2600154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26000cb2a0 0x7f2600154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45098/0 (socket says 172.21.15.5:45098) 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25ff7fe700 1 -- 172.21.15.5:0/1764999662 learned_addr learned my addr 172.21.15.5:0/1764999662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25feffd700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26000ce120 0x7f26000b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25fffff700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f260015db00 0x7f260015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25ff7fe700 1 -- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f260015db00 msgr2=0x7f260015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25ff7fe700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f260015db00 0x7f260015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25ff7fe700 1 -- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26000ce120 msgr2=0x7f26000b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.975+0000 7f25ff7fe700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26000ce120 0x7f26000b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.976+0000 7f25ff7fe700 1 -- 172.21.15.5:0/1764999662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f00094a0 con 0x7f26000cb2a0 2024-05-08T23:21:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.976+0000 7f25ff7fe700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26000cb2a0 0x7f2600154da0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f25f0000c00 tx=0x7f25f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.976+0000 7f25e7fff700 1 -- 172.21.15.5:0/1764999662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f00038a0 con 0x7f26000cb2a0 2024-05-08T23:21:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.976+0000 7f25e7fff700 1 -- 172.21.15.5:0/1764999662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f001e070 con 0x7f26000cb2a0 2024-05-08T23:21:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.976+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260015be50 con 0x7f26000cb2a0 2024-05-08T23:21:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.976+0000 7f25e7fff700 1 -- 172.21.15.5:0/1764999662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0003c60 con 0x7f26000cb2a0 2024-05-08T23:21:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.976+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2600159260 con 0x7f26000cb2a0 2024-05-08T23:21:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.977+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26000b4030 con 0x7f26000cb2a0 2024-05-08T23:21:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.978+0000 7f25e7fff700 1 -- 172.21.15.5:0/1764999662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f25f002d450 con 0x7f26000cb2a0 2024-05-08T23:21:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.978+0000 7f25e7fff700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f25ec06ac90 0x7f25ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.978+0000 7f25e7fff700 1 -- 172.21.15.5:0/1764999662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f25f008a4f0 con 0x7f26000cb2a0 2024-05-08T23:21:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.978+0000 7f25feffd700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f25ec06ac90 0x7f25ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.979+0000 7f25feffd700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f25ec06ac90 0x7f25ec06d150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26000b63a0 tx=0x7f25f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:54.983+0000 7f25e7fff700 1 -- 172.21.15.5:0/1764999662 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f26000b4030 con 0x7f26000cb2a0 2024-05-08T23:21:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.250+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2600068000 con 0x7f26000cb2a0 2024-05-08T23:21:55.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.250+0000 7f25e7fff700 1 -- 172.21.15.5:0/1764999662 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4719 (secure 0 0 0) 0x7f25f00556d0 con 0x7f26000cb2a0 2024-05-08T23:21:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:55.251 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3822534655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3822534655},{"type":"v1","addr":"172.21.15.49:6836","nonce":3822534655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1921904828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1921904828},{"type":"v1","addr":"172.21.15.49:6837","nonce":1921904828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/432469275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":432469275},{"type":"v1","addr":"172.21.15.5:6834","nonce":432469275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:21:04.087368+0000","modified":"2024-05-08T23:21:07.094437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1939093817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1939093817},{"type":"v1","addr":"172.21.15.5:6835","nonce":1939093817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:55.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f25ec06ac90 msgr2=0x7f25ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f25ec06ac90 0x7f25ec06d150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26000b63a0 tx=0x7f25f4009040 comp rx=0 tx=0).stop 2024-05-08T23:21:55.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26000cb2a0 msgr2=0x7f2600154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26000cb2a0 0x7f2600154da0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f25f0000c00 tx=0x7f25f0004750 comp rx=0 tx=0).stop 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 shutdown_connections 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f25ec06ac90 0x7f25ec06d150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f260015db00 0x7f260015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26000cb2a0 0x7f2600154da0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 --2- 172.21.15.5:0/1764999662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26000ce120 0x7f26000b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 >> 172.21.15.5:0/1764999662 conn(0x7f2600099a50 msgr2=0x7f26000ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 shutdown_connections 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.253+0000 7f2606230700 1 -- 172.21.15.5:0/1764999662 wait complete. 2024-05-08T23:21:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-08T23:21:55.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:21:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.404+0000 7fd621e94700 1 Processor -- start 2024-05-08T23:21:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.404+0000 7fd621e94700 1 -- start start 2024-05-08T23:21:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd621e94700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd61c0cf840 0x7fd61c0af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd621e94700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd61c0d0eb0 0x7fd61c161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd621e94700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c0cd980 con 0x7fd61c06a960 2024-05-08T23:21:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd621e94700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61c0ccdb0 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd621e94700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61c0cd050 con 0x7fd61c0cf840 2024-05-08T23:21:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd61bfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd61c0d0eb0 0x7fd61c161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd61bfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd61c0d0eb0 0x7fd61c161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45114/0 (socket says 172.21.15.5:45114) 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd61bfff700 1 -- 172.21.15.5:0/3400365146 learned_addr learned my addr 172.21.15.5:0/3400365146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.405+0000 7fd61b7fe700 1 --2- 172.21.15.5:0/3400365146 >> v2:172.21.15.49:3300/0 conn(0x7fd61c0cf840 0x7fd61c0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.406+0000 7fd61bfff700 1 -- 172.21.15.5:0/3400365146 >> v2:172.21.15.49:3300/0 conn(0x7fd61c0cf840 msgr2=0x7fd61c0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.406+0000 7fd61bfff700 1 --2- 172.21.15.5:0/3400365146 >> v2:172.21.15.49:3300/0 conn(0x7fd61c0cf840 0x7fd61c0af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.406+0000 7fd61bfff700 1 -- 172.21.15.5:0/3400365146 >> v1:172.21.15.49:6789/0 conn(0x7fd61c06a960 legacy=0x7fd61c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.406+0000 7fd61bfff700 1 -- 172.21.15.5:0/3400365146 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0cd330 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.406+0000 7fd61bfff700 1 --2- 172.21.15.5:0/3400365146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c161620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd61c071f10 tx=0x7fd61400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564f9a9daf22eace server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.406+0000 7fd61a7fc700 1 -- 172.21.15.5:0/3400365146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61400e070 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.406+0000 7fd61a7fc700 1 -- 172.21.15.5:0/3400365146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61400b910 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.406+0000 7fd61a7fc700 1 -- 172.21.15.5:0/3400365146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61400bcc0 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.407+0000 7fd621e94700 1 -- 172.21.15.5:0/3400365146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 msgr2=0x7fd61c161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.407+0000 7fd621e94700 1 --2- 172.21.15.5:0/3400365146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c161620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd61c071f10 tx=0x7fd61400b330 comp rx=0 tx=0).stop 2024-05-08T23:21:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.407+0000 7fd621e94700 1 -- 172.21.15.5:0/3400365146 shutdown_connections 2024-05-08T23:21:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.407+0000 7fd621e94700 1 --2- 172.21.15.5:0/3400365146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c161620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.407+0000 7fd621e94700 1 --2- 172.21.15.5:0/3400365146 >> v2:172.21.15.49:3300/0 conn(0x7fd61c0cf840 0x7fd61c0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.407+0000 7fd621e94700 1 -- 172.21.15.5:0/3400365146 >> 172.21.15.5:0/3400365146 conn(0x7fd61c0d0320 msgr2=0x7fd61c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.407+0000 7fd621e94700 1 -- 172.21.15.5:0/3400365146 shutdown_connections 2024-05-08T23:21:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.407+0000 7fd621e94700 1 -- 172.21.15.5:0/3400365146 wait complete. 2024-05-08T23:21:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.408+0000 7fd621e94700 1 Processor -- start 2024-05-08T23:21:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.408+0000 7fd621e94700 1 -- start start 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd621e94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c06a960 0x7fd61c14b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd621e94700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd61c0cf840 0x7fd61c0c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd621e94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd621e94700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0b42f0 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd621e94700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c14bd20 con 0x7fd61c06a960 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd621e94700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0c4ad0 con 0x7fd61c0cf840 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd61bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd61bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c0c6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45130/0 (socket says 172.21.15.5:45130) 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd61bfff700 1 -- 172.21.15.5:0/3509860269 learned_addr learned my addr 172.21.15.5:0/3509860269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd61b7fe700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c06a960 0x7fd61c14b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.409+0000 7fd61affd700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd61c0cf840 0x7fd61c0c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd61bfff700 1 -- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd61c0cf840 msgr2=0x7fd61c0c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd61bfff700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd61c0cf840 0x7fd61c0c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd61bfff700 1 -- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c06a960 msgr2=0x7fd61c14b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd61bfff700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c06a960 0x7fd61c14b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd61bfff700 1 -- 172.21.15.5:0/3509860269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614009b30 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd61bfff700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c0c6880 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0704d0 tx=0x7fd614004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd603fff700 1 -- 172.21.15.5:0/3509860269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614004880 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd603fff700 1 -- 172.21.15.5:0/3509860269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6140099b0 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c0c4db0 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd603fff700 1 -- 172.21.15.5:0/3509860269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61401c680 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.410+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c0c5370 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.411+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c072df0 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.412+0000 7fd603fff700 1 -- 172.21.15.5:0/3509860269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd61401c7e0 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.412+0000 7fd603fff700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd60806ac90 0x7fd60806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.412+0000 7fd603fff700 1 -- 172.21.15.5:0/3509860269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd61408af10 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.412+0000 7fd61b7fe700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd60806ac90 0x7fd60806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.413+0000 7fd61b7fe700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd60806ac90 0x7fd60806d150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009be0 tx=0x7fd60c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.417+0000 7fd603fff700 1 -- 172.21.15.5:0/3509860269 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd6140560f0 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.638+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd61c0af950 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.639+0000 7fd603fff700 1 -- 172.21.15.5:0/3509860269 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16176 (secure 0 0 0) 0x7fd614059710 con 0x7fd61c0d0eb0 2024-05-08T23:21:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:21:55.639 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:21:53.247940+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:21:02.035577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:21:03.055037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:21:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd60806ac90 msgr2=0x7fd60806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd60806ac90 0x7fd60806d150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009be0 tx=0x7fd60c009210 comp rx=0 tx=0).stop 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 msgr2=0x7fd61c0c6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c0c6880 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0704d0 tx=0x7fd614004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 shutdown_connections 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd60806ac90 0x7fd60806d150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd61c0d0eb0 0x7fd61c0c6880 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd61c06a960 0x7fd61c14b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 --2- 172.21.15.5:0/3509860269 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd61c0cf840 0x7fd61c0c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 >> 172.21.15.5:0/3509860269 conn(0x7fd61c0d0320 msgr2=0x7fd61c0ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 shutdown_connections 2024-05-08T23:21:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.642+0000 7fd621e94700 1 -- 172.21.15.5:0/3509860269 wait complete. 2024-05-08T23:21:55.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:21:55.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.793+0000 7f608923a700 1 Processor -- start 2024-05-08T23:21:55.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.794+0000 7f608923a700 1 -- start start 2024-05-08T23:21:55.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.794+0000 7f608923a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f608406a960 0x7f608406ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.794+0000 7f608923a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60840d01f0 con 0x7f608415d980 2024-05-08T23:21:55.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.794+0000 7f608923a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60840c71b0 con 0x7f60840c5260 2024-05-08T23:21:55.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.794+0000 7f608923a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60840c5bc0 con 0x7f608406a960 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f608359e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f608406a960 0x7f608406ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f6082d9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f608415d980 0x7f60840b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59938/0 (socket says 172.21.15.5:59938) 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f6082d9d700 1 -- 172.21.15.5:0/3702633074 learned_addr learned my addr 172.21.15.5:0/3702633074 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f608359e700 1 -- 172.21.15.5:0/3702633074 >> v1:172.21.15.49:6789/0 conn(0x7f60840c5260 legacy=0x7f60840cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f608359e700 1 -- 172.21.15.5:0/3702633074 >> v1:172.21.15.5:6789/0 conn(0x7f608415d980 legacy=0x7f60840b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f6081d9b700 1 -- 172.21.15.5:0/3702633074 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315656048 0 0) 0x7f60840d01f0 con 0x7f608415d980 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f608359e700 1 -- 172.21.15.5:0/3702633074 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60840c69d0 con 0x7f608406a960 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f608359e700 1 --2- 172.21.15.5:0/3702633074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608406a960 0x7f608406ad40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f607c009d90 tx=0x7f607c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27034e8df8280a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f6081d9b700 1 -- 172.21.15.5:0/3702633074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c00e070 con 0x7f608406a960 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f6081d9b700 1 -- 172.21.15.5:0/3702633074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c00bbd0 con 0x7f608406a960 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.795+0000 7f6081d9b700 1 -- 172.21.15.5:0/3702633074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c012430 con 0x7f608406a960 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.796+0000 7f608923a700 1 -- 172.21.15.5:0/3702633074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608406a960 msgr2=0x7f608406ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.796+0000 7f608923a700 1 --2- 172.21.15.5:0/3702633074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608406a960 0x7f608406ad40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f607c009d90 tx=0x7f607c00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.796+0000 7f608923a700 1 -- 172.21.15.5:0/3702633074 shutdown_connections 2024-05-08T23:21:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.796+0000 7f608923a700 1 --2- 172.21.15.5:0/3702633074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608406a960 0x7f608406ad40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.796+0000 7f608923a700 1 -- 172.21.15.5:0/3702633074 >> 172.21.15.5:0/3702633074 conn(0x7f60840d03a0 msgr2=0x7f60840d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:55.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.796+0000 7f608923a700 1 -- 172.21.15.5:0/3702633074 shutdown_connections 2024-05-08T23:21:55.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.796+0000 7f608923a700 1 -- 172.21.15.5:0/3702633074 wait complete. 2024-05-08T23:21:55.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.798+0000 7f608923a700 1 Processor -- start 2024-05-08T23:21:55.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.798+0000 7f608923a700 1 -- start start 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.798+0000 7f608923a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f608406a960 0x7f6084158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.798+0000 7f608923a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60840c5260 0x7f60840b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.798+0000 7f608923a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608415d980 0x7f60840ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.798+0000 7f608923a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608409a3e0 con 0x7f608415d980 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.798+0000 7f608923a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608409a6c0 con 0x7f60840c5260 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.798+0000 7f608923a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608409a9a0 con 0x7f608406a960 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608415d980 0x7f60840ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608415d980 0x7f60840ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45152/0 (socket says 172.21.15.5:45152) 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 -- 172.21.15.5:0/3303039798 learned_addr learned my addr 172.21.15.5:0/3303039798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608259c700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60840c5260 0x7f60840b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f6082d9d700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f608406a960 0x7f6084158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 -- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f608406a960 msgr2=0x7f6084158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f608406a960 0x7f6084158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 -- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60840c5260 msgr2=0x7f60840b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60840c5260 0x7f60840b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 -- 172.21.15.5:0/3303039798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c009a70 con 0x7f608415d980 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f6082d9d700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f608406a960 0x7f6084158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608359e700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608415d980 0x7f60840ae550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f607c014040 tx=0x7f607c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f606b7fe700 1 -- 172.21.15.5:0/3303039798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c01b820 con 0x7f608415d980 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f606b7fe700 1 -- 172.21.15.5:0/3303039798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c020070 con 0x7f608415d980 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608409ac80 con 0x7f608415d980 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.799+0000 7f606b7fe700 1 -- 172.21.15.5:0/3303039798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c01bc00 con 0x7f608415d980 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.800+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60840b3600 con 0x7f608415d980 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.800+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608409b1a0 con 0x7f608415d980 2024-05-08T23:21:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.801+0000 7f606b7fe700 1 -- 172.21.15.5:0/3303039798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f607c026400 con 0x7f608415d980 2024-05-08T23:21:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.801+0000 7f606b7fe700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f607006ac90 0x7f607006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.801+0000 7f606b7fe700 1 -- 172.21.15.5:0/3303039798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f607c08b270 con 0x7f608415d980 2024-05-08T23:21:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.802+0000 7f6082d9d700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f607006ac90 0x7f607006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.802+0000 7f6082d9d700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f607006ac90 0x7f607006d150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6074000f60 tx=0x7f6074007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:55.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:55.806+0000 7f606b7fe700 1 -- 172.21.15.5:0/3303039798 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f608409b1a0 con 0x7f608415d980 2024-05-08T23:21:56.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.079+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f608409b1a0 con 0x7f608415d980 2024-05-08T23:21:56.080 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:21:56.080+0000 7f8a49d06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:21:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.087+0000 7f606b7fe700 1 -- 172.21.15.5:0/3303039798 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f608409b1a0 con 0x7f608415d980 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f607006ac90 msgr2=0x7f607006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f607006ac90 0x7f607006d150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6074000f60 tx=0x7f6074007e70 comp rx=0 tx=0).stop 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608415d980 msgr2=0x7f60840ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608415d980 0x7f60840ae550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f607c014040 tx=0x7f607c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 shutdown_connections 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f607006ac90 0x7f607006d150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608415d980 0x7f60840ae550 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f608406a960 0x7f6084158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 --2- 172.21.15.5:0/3303039798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60840c5260 0x7f60840b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.089+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 >> 172.21.15.5:0/3303039798 conn(0x7f60840d03a0 msgr2=0x7f608415cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.090+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 shutdown_connections 2024-05-08T23:21:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.090+0000 7f608923a700 1 -- 172.21.15.5:0/3303039798 wait complete. 2024-05-08T23:21:56.091 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:21:56.104 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:21:56.129 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:21:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.243+0000 7f49bd9f1700 1 Processor -- start 2024-05-08T23:21:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.243+0000 7f49bd9f1700 1 -- start start 2024-05-08T23:21:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49bd9f1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815b8c0 0x7f49b80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49bd9f1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49b80d1740 0x7f49b80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49bd9f1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b80cf560 con 0x7f49b815b4e0 2024-05-08T23:21:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49bd9f1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b80cefe0 con 0x7f49b80d1740 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49bd9f1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80d1bb0 con 0x7f49b815b8c0 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b6ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f49b815b4e0 0x7f49b80b1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59966/0 (socket says 172.21.15.5:59966) 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b6ffd700 1 -- 172.21.15.5:0/2746255793 learned_addr learned my addr 172.21.15.5:0/2746255793 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b77fe700 1 --2- 172.21.15.5:0/2746255793 >> v2:172.21.15.49:3300/0 conn(0x7f49b80d1740 0x7f49b80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311069931 0 0) 0x7f49b80cf560 con 0x7f49b815b4e0 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b67fc700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815b8c0 0x7f49b80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a00035b0 con 0x7f49b815b4e0 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1745282736 0 0) 0x7f49a00035b0 con 0x7f49b815b4e0 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815b8c0 msgr2=0x7f49b80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815b8c0 0x7f49b80d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 >> v2:172.21.15.49:3300/0 conn(0x7f49b80d1740 msgr2=0x7f49b80cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 --2- 172.21.15.5:0/2746255793 >> v2:172.21.15.49:3300/0 conn(0x7f49b80d1740 0x7f49b80cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b80ccff0 con 0x7f49b815b4e0 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f49a8001670 con 0x7f49b815b4e0 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.245+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 >> v1:172.21.15.5:6789/0 conn(0x7f49b815b4e0 legacy=0x7f49b80b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a0003a00 0x7f49a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a0004850 0x7f49a0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815b8c0 0x7f49a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80cefe0 con 0x7f49a0003a00 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80d1bb0 con 0x7f49a0004850 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a000ba10 con 0x7f49b815b8c0 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b6ffd700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a0003a00 0x7f49a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b67fc700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a0004850 0x7f49a0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b77fe700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815b8c0 0x7f49a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b6ffd700 1 -- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815b8c0 msgr2=0x7f49a0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b6ffd700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815b8c0 0x7f49a0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b6ffd700 1 -- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a0004850 msgr2=0x7f49a0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b6ffd700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a0004850 0x7f49a0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b6ffd700 1 -- 172.21.15.5:0/2746255793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a00035b0 con 0x7f49a0003a00 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b6ffd700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a0003a00 0x7f49a0003e20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f49a800c320 tx=0x7f49a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267639819d9d60e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a800ce60 con 0x7f49a0003a00 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a8005960 con 0x7f49a0003a00 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.246+0000 7f49b5ffb700 1 -- 172.21.15.5:0/2746255793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8005cf0 con 0x7f49a0003a00 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.247+0000 7f49bd9f1700 1 -- 172.21.15.5:0/2746255793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a0003a00 msgr2=0x7f49a0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.247+0000 7f49bd9f1700 1 --2- 172.21.15.5:0/2746255793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a0003a00 0x7f49a0003e20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f49a800c320 tx=0x7f49a0005880 comp rx=0 tx=0).stop 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.247+0000 7f49b77fe700 1 -- 172.21.15.5:0/2746255793 reap_dead start 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.247+0000 7f49bd9f1700 1 -- 172.21.15.5:0/2746255793 shutdown_connections 2024-05-08T23:21:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.247+0000 7f49bd9f1700 1 -- 172.21.15.5:0/2746255793 >> 172.21.15.5:0/2746255793 conn(0x7f49b808afe0 msgr2=0x7f49b808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.247+0000 7f49bd9f1700 1 -- 172.21.15.5:0/2746255793 shutdown_connections 2024-05-08T23:21:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.247+0000 7f49bd9f1700 1 -- 172.21.15.5:0/2746255793 wait complete. 2024-05-08T23:21:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.248+0000 7f49bd9f1700 1 Processor -- start 2024-05-08T23:21:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49bd9f1700 1 -- start start 2024-05-08T23:21:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49bd9f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b814fc00 0x7f49b815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49bd9f1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815d870 0x7f49b80af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49bd9f1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b815c2c0 0x7f49b80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49bd9f1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b815d450 con 0x7f49b814fc00 2024-05-08T23:21:56.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49bd9f1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80b4090 con 0x7f49b815c2c0 2024-05-08T23:21:56.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49bd9f1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b815c7d0 con 0x7f49b815d870 2024-05-08T23:21:56.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b814fc00 0x7f49b815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b814fc00 0x7f49b815bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45172/0 (socket says 172.21.15.5:45172) 2024-05-08T23:21:56.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.249+0000 7f49b6ffd700 1 -- 172.21.15.5:0/22557655 learned_addr learned my addr 172.21.15.5:0/22557655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b67fc700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815d870 0x7f49b80af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b77fe700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b815c2c0 0x7f49b80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b6ffd700 1 -- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815d870 msgr2=0x7f49b80af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b6ffd700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815d870 0x7f49b80af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b6ffd700 1 -- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b815c2c0 msgr2=0x7f49b80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b6ffd700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b815c2c0 0x7f49b80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b6ffd700 1 -- 172.21.15.5:0/22557655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b815cab0 con 0x7f49b814fc00 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b67fc700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815d870 0x7f49b80af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49b6ffd700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b814fc00 0x7f49b815bd80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f49a80048d0 tx=0x7f49a8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f499ffff700 1 -- 172.21.15.5:0/22557655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8001680 con 0x7f49b814fc00 2024-05-08T23:21:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f499ffff700 1 -- 172.21.15.5:0/22557655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a8007ba0 con 0x7f49b814fc00 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b815cd90 con 0x7f49b814fc00 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f499ffff700 1 -- 172.21.15.5:0/22557655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8007520 con 0x7f49b814fc00 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.250+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b80c9f70 con 0x7f49b814fc00 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.251+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b80ca130 con 0x7f49b814fc00 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.252+0000 7f499ffff700 1 -- 172.21.15.5:0/22557655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f49a8007680 con 0x7f49b814fc00 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.252+0000 7f499ffff700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49a406acc0 0x7f49a406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.252+0000 7f499ffff700 1 -- 172.21.15.5:0/22557655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f49a808a3a0 con 0x7f49b814fc00 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.253+0000 7f49b67fc700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49a406acc0 0x7f49a406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.253+0000 7f49b67fc700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49a406acc0 0x7f49a406d180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f49b80c9d50 tx=0x7f49ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.257+0000 7f499ffff700 1 -- 172.21.15.5:0/22557655 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f49b80ca130 con 0x7f49b814fc00 2024-05-08T23:21:56.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:56.526+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f49b80ca130 con 0x7f49b814fc00 2024-05-08T23:21:57.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.088+0000 7f499ffff700 1 -- 172.21.15.5:0/22557655 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f49b80ca130 con 0x7f49b814fc00 2024-05-08T23:21:57.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.091+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49a406acc0 msgr2=0x7f49a406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.091+0000 7f49bd9f1700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49a406acc0 0x7f49a406d180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f49b80c9d50 tx=0x7f49ac008040 comp rx=0 tx=0).stop 2024-05-08T23:21:57.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.091+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b814fc00 msgr2=0x7f49b815bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.091+0000 7f49bd9f1700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b814fc00 0x7f49b815bd80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f49a80048d0 tx=0x7f49a8001a00 comp rx=0 tx=0).stop 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.092+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 shutdown_connections 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.092+0000 7f49bd9f1700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f49a406acc0 0x7f49a406d180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.092+0000 7f49bd9f1700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b815c2c0 0x7f49b80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.092+0000 7f49bd9f1700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b814fc00 0x7f49b815bd80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.092+0000 7f49bd9f1700 1 --2- 172.21.15.5:0/22557655 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49b815d870 0x7f49b80af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.092+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 >> 172.21.15.5:0/22557655 conn(0x7f49b808afe0 msgr2=0x7f49b80b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.092+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 shutdown_connections 2024-05-08T23:21:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.092+0000 7f49bd9f1700 1 -- 172.21.15.5:0/22557655 wait complete. 2024-05-08T23:21:57.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:21:57.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.297+0000 7f32362f4700 1 Processor -- start 2024-05-08T23:21:57.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.297+0000 7f32362f4700 1 -- start start 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.297+0000 7f32362f4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300cff30 0x7f32300d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.297+0000 7f32362f4700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32300b4a90 0x7f32300ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.297+0000 7f32362f4700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32300d0850 con 0x7f32300d3d20 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.297+0000 7f32362f4700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32300cf610 con 0x7f32300b4a90 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.297+0000 7f32362f4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32300d1fc0 con 0x7f32300cff30 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322ffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300cff30 0x7f32300d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f3234891700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32300b4a90 0x7f32300ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322f7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f32300d3d20 0x7f32300ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:46310/0 (socket says 172.21.15.5:46310) 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322f7fe700 1 -- 172.21.15.5:0/4091868615 learned_addr learned my addr 172.21.15.5:0/4091868615 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322effd700 1 -- 172.21.15.5:0/4091868615 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653371221 0 0) 0x7f32300d0850 con 0x7f32300d3d20 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322effd700 1 -- 172.21.15.5:0/4091868615 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32180035b0 con 0x7f32300d3d20 2024-05-08T23:21:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322ffff700 1 -- 172.21.15.5:0/4091868615 >> v2:172.21.15.49:3300/0 conn(0x7f32300b4a90 msgr2=0x7f32300ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322ffff700 1 --2- 172.21.15.5:0/4091868615 >> v2:172.21.15.49:3300/0 conn(0x7f32300b4a90 0x7f32300ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322ffff700 1 -- 172.21.15.5:0/4091868615 >> v1:172.21.15.49:6789/0 conn(0x7f32300d3d20 legacy=0x7f32300ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322ffff700 1 -- 172.21.15.5:0/4091868615 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300d22a0 con 0x7f32300cff30 2024-05-08T23:21:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f3234891700 1 --2- 172.21.15.5:0/4091868615 >> v2:172.21.15.49:3300/0 conn(0x7f32300b4a90 0x7f32300ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.298+0000 7f322ffff700 1 --2- 172.21.15.5:0/4091868615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300cff30 0x7f32300d0310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3220009840 tx=0x7f3220009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a884c47426eadf94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f322effd700 1 -- 172.21.15.5:0/4091868615 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220011830 con 0x7f32300cff30 2024-05-08T23:21:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f322effd700 1 -- 172.21.15.5:0/4091868615 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3220011990 con 0x7f32300cff30 2024-05-08T23:21:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f322effd700 1 -- 172.21.15.5:0/4091868615 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220011d20 con 0x7f32300cff30 2024-05-08T23:21:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f32362f4700 1 -- 172.21.15.5:0/4091868615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300cff30 msgr2=0x7f32300d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f32362f4700 1 --2- 172.21.15.5:0/4091868615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300cff30 0x7f32300d0310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3220009840 tx=0x7f3220009b50 comp rx=0 tx=0).stop 2024-05-08T23:21:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f32362f4700 1 -- 172.21.15.5:0/4091868615 shutdown_connections 2024-05-08T23:21:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f32362f4700 1 --2- 172.21.15.5:0/4091868615 >> v2:172.21.15.49:3300/0 conn(0x7f32300b4a90 0x7f32300ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f32362f4700 1 --2- 172.21.15.5:0/4091868615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300cff30 0x7f32300d0310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f32362f4700 1 -- 172.21.15.5:0/4091868615 >> 172.21.15.5:0/4091868615 conn(0x7f323008afe0 msgr2=0x7f32300d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f32362f4700 1 -- 172.21.15.5:0/4091868615 shutdown_connections 2024-05-08T23:21:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.299+0000 7f32362f4700 1 -- 172.21.15.5:0/4091868615 wait complete. 2024-05-08T23:21:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.301+0000 7f32362f4700 1 Processor -- start 2024-05-08T23:21:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.301+0000 7f32362f4700 1 -- start start 2024-05-08T23:21:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f32362f4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32300b4a90 0x7f3230157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f32362f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32300cff30 0x7f32300b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f32362f4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300d3d20 0x7f32300b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f32362f4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301f36f0 con 0x7f32300cff30 2024-05-08T23:21:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f32362f4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32300ccaf0 con 0x7f32300b4a90 2024-05-08T23:21:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f32362f4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32300ccdd0 con 0x7f32300d3d20 2024-05-08T23:21:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32300cff30 0x7f32300b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32300b4a90 0x7f3230157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32300cff30 0x7f32300b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45174/0 (socket says 172.21.15.5:45174) 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 -- 172.21.15.5:0/3728117740 learned_addr learned my addr 172.21.15.5:0/3728117740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f3234891700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300d3d20 0x7f32300b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 -- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300d3d20 msgr2=0x7f32300b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300d3d20 0x7f32300b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 -- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32300b4a90 msgr2=0x7f3230157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32300b4a90 0x7f3230157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 -- 172.21.15.5:0/3728117740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3228009a70 con 0x7f32300cff30 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322ffff700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32300b4a90 0x7f3230157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.302+0000 7f322f7fe700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32300cff30 0x7f32300b6f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f322400c700 tx=0x7f322400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.303+0000 7f322cff9700 1 -- 172.21.15.5:0/3728117740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224005d70 con 0x7f32300cff30 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.303+0000 7f322cff9700 1 -- 172.21.15.5:0/3728117740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32240070d0 con 0x7f32300cff30 2024-05-08T23:21:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.303+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32200094a0 con 0x7f32300cff30 2024-05-08T23:21:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.303+0000 7f322cff9700 1 -- 172.21.15.5:0/3728117740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224007460 con 0x7f32300cff30 2024-05-08T23:21:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.303+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32300cd390 con 0x7f32300cff30 2024-05-08T23:21:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.304+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32300cd580 con 0x7f32300cff30 2024-05-08T23:21:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.304+0000 7f322cff9700 1 -- 172.21.15.5:0/3728117740 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f32240075c0 con 0x7f32300cff30 2024-05-08T23:21:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.305+0000 7f322cff9700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321c06ac90 0x7f321c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.305+0000 7f322cff9700 1 -- 172.21.15.5:0/3728117740 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f3224088920 con 0x7f32300cff30 2024-05-08T23:21:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.305+0000 7f322ffff700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321c06ac90 0x7f321c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.306+0000 7f322ffff700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321c06ac90 0x7f321c06d150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c00 tx=0x7f3220008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.310+0000 7f322cff9700 1 -- 172.21.15.5:0/3728117740 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3224053980 con 0x7f32300cff30 2024-05-08T23:21:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:57.527+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32301f3940 con 0x7f32300cff30 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.272+0000 7f322cff9700 1 -- 172.21.15.5:0/3728117740 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f32301f3940 con 0x7f32300cff30 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.277+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321c06ac90 msgr2=0x7f321c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.277+0000 7f32362f4700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321c06ac90 0x7f321c06d150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c00 tx=0x7f3220008030 comp rx=0 tx=0).stop 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.278+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32300cff30 msgr2=0x7f32300b6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.278+0000 7f32362f4700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32300cff30 0x7f32300b6f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f322400c700 tx=0x7f322400ca10 comp rx=0 tx=0).stop 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.278+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 shutdown_connections 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.278+0000 7f32362f4700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f321c06ac90 0x7f321c06d150 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c00 tx=0x7f3220008030 comp rx=0 tx=0).stop 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.278+0000 7f32362f4700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32300d3d20 0x7f32300b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.278+0000 7f32362f4700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32300b4a90 0x7f3230157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.278+0000 7f32362f4700 1 --2- 172.21.15.5:0/3728117740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32300cff30 0x7f32300b6f70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.278+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 >> 172.21.15.5:0/3728117740 conn(0x7f323008afe0 msgr2=0x7f32300b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.279+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 shutdown_connections 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.279+0000 7f32362f4700 1 -- 172.21.15.5:0/3728117740 wait complete. 2024-05-08T23:21:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:21:58.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:21:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.439+0000 7f8c5d559700 1 Processor -- start 2024-05-08T23:21:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.440+0000 7f8c5d559700 1 -- start start 2024-05-08T23:21:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.440+0000 7f8c5d559700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c5815d700 0x7f8c580b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.440+0000 7f8c5d559700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8c58161c00 0x7f8c580b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.440+0000 7f8c5d559700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c580b49b0 con 0x7f8c580b38b0 2024-05-08T23:21:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.440+0000 7f8c5d559700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c580b3da0 con 0x7f8c5815d700 2024-05-08T23:21:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.440+0000 7f8c5d559700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c580b4080 con 0x7f8c58161c00 2024-05-08T23:21:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c56ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c5815d700 0x7f8c580b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c56ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c5815d700 0x7f8c580b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58866/0 (socket says 172.21.15.5:58866) 2024-05-08T23:21:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c56ffd700 1 -- 172.21.15.5:0/3633511780 learned_addr learned my addr 172.21.15.5:0/3633511780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c577fe700 1 --2- 172.21.15.5:0/3633511780 >> v2:172.21.15.49:3300/0 conn(0x7f8c58161c00 0x7f8c580b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c56ffd700 1 -- 172.21.15.5:0/3633511780 >> v2:172.21.15.49:3300/0 conn(0x7f8c58161c00 msgr2=0x7f8c580b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c56ffd700 1 --2- 172.21.15.5:0/3633511780 >> v2:172.21.15.49:3300/0 conn(0x7f8c58161c00 0x7f8c580b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c56ffd700 1 -- 172.21.15.5:0/3633511780 >> v1:172.21.15.49:6789/0 conn(0x7f8c580b38b0 legacy=0x7f8c581614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c56ffd700 1 -- 172.21.15.5:0/3633511780 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c580b4360 con 0x7f8c5815d700 2024-05-08T23:21:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c56ffd700 1 --2- 172.21.15.5:0/3633511780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c580b3370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8c580d1510 tx=0x7f8c48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf8b93190ac9c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c55ffb700 1 -- 172.21.15.5:0/3633511780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48005e30 con 0x7f8c5815d700 2024-05-08T23:21:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c55ffb700 1 -- 172.21.15.5:0/3633511780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c48006030 con 0x7f8c5815d700 2024-05-08T23:21:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.441+0000 7f8c55ffb700 1 -- 172.21.15.5:0/3633511780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c480063e0 con 0x7f8c5815d700 2024-05-08T23:21:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.442+0000 7f8c5d559700 1 -- 172.21.15.5:0/3633511780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 msgr2=0x7f8c580b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.442+0000 7f8c5d559700 1 --2- 172.21.15.5:0/3633511780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c580b3370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8c580d1510 tx=0x7f8c48009bd0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.442+0000 7f8c5d559700 1 -- 172.21.15.5:0/3633511780 shutdown_connections 2024-05-08T23:21:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.442+0000 7f8c5d559700 1 --2- 172.21.15.5:0/3633511780 >> v2:172.21.15.49:3300/0 conn(0x7f8c58161c00 0x7f8c580b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.442+0000 7f8c5d559700 1 --2- 172.21.15.5:0/3633511780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c580b3370 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.442+0000 7f8c5d559700 1 -- 172.21.15.5:0/3633511780 >> 172.21.15.5:0/3633511780 conn(0x7f8c5808afe0 msgr2=0x7f8c580d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.442+0000 7f8c5d559700 1 -- 172.21.15.5:0/3633511780 shutdown_connections 2024-05-08T23:21:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.442+0000 7f8c5d559700 1 -- 172.21.15.5:0/3633511780 wait complete. 2024-05-08T23:21:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.443+0000 7f8c5d559700 1 Processor -- start 2024-05-08T23:21:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c5d559700 1 -- start start 2024-05-08T23:21:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c5d559700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c580b38b0 0x7f8c580ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c5d559700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c58151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c5d559700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c58161c00 0x7f8c580b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c5d559700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580c8160 con 0x7f8c5815d700 2024-05-08T23:21:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c5d559700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580c8440 con 0x7f8c580b38b0 2024-05-08T23:21:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c5d559700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580c8720 con 0x7f8c58161c00 2024-05-08T23:21:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c567fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c58151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c567fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c58151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58872/0 (socket says 172.21.15.5:58872) 2024-05-08T23:21:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.444+0000 7f8c567fc700 1 -- 172.21.15.5:0/293482705 learned_addr learned my addr 172.21.15.5:0/293482705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c56ffd700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c580b38b0 0x7f8c580ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c577fe700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c58161c00 0x7f8c580b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c567fc700 1 -- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c58161c00 msgr2=0x7f8c580b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c567fc700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c58161c00 0x7f8c580b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c567fc700 1 -- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c580b38b0 msgr2=0x7f8c580ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c567fc700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c580b38b0 0x7f8c580ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c567fc700 1 -- 172.21.15.5:0/293482705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48009500 con 0x7f8c5815d700 2024-05-08T23:21:58.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c577fe700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c58161c00 0x7f8c580b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:58.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c567fc700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c58151410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000c4d0 tx=0x7f8c4000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c4f7fe700 1 -- 172.21.15.5:0/293482705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40011790 con 0x7f8c5815d700 2024-05-08T23:21:58.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c4f7fe700 1 -- 172.21.15.5:0/293482705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c400118f0 con 0x7f8c5815d700 2024-05-08T23:21:58.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c580c8a00 con 0x7f8c5815d700 2024-05-08T23:21:58.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c4f7fe700 1 -- 172.21.15.5:0/293482705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40011ca0 con 0x7f8c5815d700 2024-05-08T23:21:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.445+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c58151950 con 0x7f8c5815d700 2024-05-08T23:21:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.446+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c580c8c40 con 0x7f8c5815d700 2024-05-08T23:21:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.447+0000 7f8c4f7fe700 1 -- 172.21.15.5:0/293482705 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8c4001c460 con 0x7f8c5815d700 2024-05-08T23:21:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.447+0000 7f8c4f7fe700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c3806ac90 0x7f8c3806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.447+0000 7f8c4f7fe700 1 -- 172.21.15.5:0/293482705 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f8c40088fc0 con 0x7f8c5815d700 2024-05-08T23:21:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.448+0000 7f8c56ffd700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c3806ac90 0x7f8c3806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.448+0000 7f8c56ffd700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c3806ac90 0x7f8c3806d150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c580d1510 tx=0x7f8c48008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.452+0000 7f8c4f7fe700 1 -- 172.21.15.5:0/293482705 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8c400bc900 con 0x7f8c5815d700 2024-05-08T23:21:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:58.668+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c580c82a0 con 0x7f8c5815d700 2024-05-08T23:21:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.277+0000 7f8c4f7fe700 1 -- 172.21.15.5:0/293482705 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8c580c82a0 con 0x7f8c5815d700 2024-05-08T23:21:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c3806ac90 msgr2=0x7f8c3806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c3806ac90 0x7f8c3806d150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c580d1510 tx=0x7f8c48008070 comp rx=0 tx=0).stop 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 msgr2=0x7f8c58151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c58151410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000c4d0 tx=0x7f8c4000c7e0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 shutdown_connections 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c3806ac90 0x7f8c3806d150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c58161c00 0x7f8c580b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c580b38b0 0x7f8c580ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 --2- 172.21.15.5:0/293482705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5815d700 0x7f8c58151410 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.281+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 >> 172.21.15.5:0/293482705 conn(0x7f8c5808afe0 msgr2=0x7f8c580d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.282+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 shutdown_connections 2024-05-08T23:21:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.282+0000 7f8c5d559700 1 -- 172.21.15.5:0/293482705 wait complete. 2024-05-08T23:21:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-08T23:21:59.300 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:21:59.301 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T23:21:59.301 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:21:59.301 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:21:59.301 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:21:59.301 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T23:21:59.301 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:21:59.302 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:21:59.302 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:21:59.302 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T23:21:59.302 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:21:59.302 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:21:59.302 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:21:59.302 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T23:21:59.303 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:21:59.303 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:21:59.313 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:21:59.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:21:59.314 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:21:59.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:21:59.315 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:21:59.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:21:59.316 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:21:59.316 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:21:59.356 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:21:59.356 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:59.357 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:21:59.357 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:59.357 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:21:59.358 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:59.358 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:21:59.358 DEBUG:teuthology.parallel:result is None 2024-05-08T23:21:59.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T23:21:59.448 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:21:59.448 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:21:59.459 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:21:59.459 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:21:59.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.522+0000 7f7ab4ae1700 1 Processor -- start 2024-05-08T23:21:59.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.522+0000 7f7ab4ae1700 1 -- start start 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.523+0000 7f7ab4ae1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab0160f00 0x7f7ab00b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.523+0000 7f7ab4ae1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00ce3f0 con 0x7f7ab00d22d0 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.523+0000 7f7ab4ae1700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00b42c0 con 0x7f7ab00b2f60 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.523+0000 7f7ab4ae1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00cdc50 con 0x7f7ab0160f00 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aaed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7ab00d22d0 0x7f7ab00ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34052/0 (socket says 172.21.15.5:34052) 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aaed9d700 1 -- 172.21.15.5:0/1162752229 learned_addr learned my addr 172.21.15.5:0/1162752229 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aae59c700 1 --2- 172.21.15.5:0/1162752229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab0160f00 0x7f7ab00b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aad59a700 1 -- 172.21.15.5:0/1162752229 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180198205 0 0) 0x7f7ab00ce3f0 con 0x7f7ab00d22d0 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aad59a700 1 -- 172.21.15.5:0/1162752229 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a980035b0 con 0x7f7ab00d22d0 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aad59a700 1 -- 172.21.15.5:0/1162752229 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368382955 0 0) 0x7f7ab00b42c0 con 0x7f7ab00b2f60 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aad59a700 1 -- 172.21.15.5:0/1162752229 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab00ce3f0 con 0x7f7ab00b2f60 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aae59c700 1 -- 172.21.15.5:0/1162752229 >> v1:172.21.15.49:6789/0 conn(0x7f7ab00b2f60 legacy=0x7f7ab00d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aae59c700 1 -- 172.21.15.5:0/1162752229 >> v1:172.21.15.5:6789/0 conn(0x7f7ab00d22d0 legacy=0x7f7ab00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aae59c700 1 -- 172.21.15.5:0/1162752229 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab00b3740 con 0x7f7ab0160f00 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aae59c700 1 --2- 172.21.15.5:0/1162752229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab0160f00 0x7f7ab00b47f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009850 tx=0x7f7aa0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e974ffde5ac4936c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aad59a700 1 -- 172.21.15.5:0/1162752229 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0005e30 con 0x7f7ab0160f00 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aad59a700 1 -- 172.21.15.5:0/1162752229 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa0006030 con 0x7f7ab0160f00 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.524+0000 7f7aad59a700 1 -- 172.21.15.5:0/1162752229 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00063e0 con 0x7f7ab0160f00 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.525+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1162752229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab0160f00 msgr2=0x7f7ab00b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.525+0000 7f7ab4ae1700 1 --2- 172.21.15.5:0/1162752229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab0160f00 0x7f7ab00b47f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009850 tx=0x7f7aa0009b60 comp rx=0 tx=0).stop 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.525+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1162752229 shutdown_connections 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.525+0000 7f7ab4ae1700 1 --2- 172.21.15.5:0/1162752229 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab0160f00 0x7f7ab00b47f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.525+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1162752229 >> 172.21.15.5:0/1162752229 conn(0x7f7ab008afe0 msgr2=0x7f7ab00d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.525+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1162752229 shutdown_connections 2024-05-08T23:21:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.525+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1162752229 wait complete. 2024-05-08T23:21:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.527+0000 7f7ab4ae1700 1 Processor -- start 2024-05-08T23:21:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.527+0000 7f7ab4ae1700 1 -- start start 2024-05-08T23:21:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7ab4ae1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab00b2f60 0x7f7ab00b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7ab4ae1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ab00d22d0 0x7f7ab0149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7ab4ae1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ab0160f00 0x7f7ab00b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7ab4ae1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00c9fd0 con 0x7f7ab0160f00 2024-05-08T23:21:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7ab4ae1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00b8850 con 0x7f7ab00d22d0 2024-05-08T23:21:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7ab4ae1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00b8b30 con 0x7f7ab00b2f60 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ab0160f00 0x7f7ab00b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ab0160f00 0x7f7ab00b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58906/0 (socket says 172.21.15.5:58906) 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 -- 172.21.15.5:0/1822164459 learned_addr learned my addr 172.21.15.5:0/1822164459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aae59c700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab00b2f60 0x7f7ab00b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aadd9b700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ab00d22d0 0x7f7ab0149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 -- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab00b2f60 msgr2=0x7f7ab00b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab00b2f60 0x7f7ab00b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 -- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ab00d22d0 msgr2=0x7f7ab0149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ab00d22d0 0x7f7ab0149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 -- 172.21.15.5:0/1822164459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa0009430 con 0x7f7ab0160f00 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aae59c700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab00b2f60 0x7f7ab00b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aadd9b700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ab00d22d0 0x7f7ab0149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7aaed9d700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ab0160f00 0x7f7ab00b7af0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800cb10 tx=0x7f7aa800ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.528+0000 7f7a9effd700 1 -- 172.21.15.5:0/1822164459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800a9e0 con 0x7f7ab0160f00 2024-05-08T23:21:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.529+0000 7f7a9effd700 1 -- 172.21.15.5:0/1822164459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa800ab40 con 0x7f7ab0160f00 2024-05-08T23:21:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.529+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00b8e10 con 0x7f7ab0160f00 2024-05-08T23:21:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.529+0000 7f7a9effd700 1 -- 172.21.15.5:0/1822164459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8012400 con 0x7f7ab0160f00 2024-05-08T23:21:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.529+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab0152e40 con 0x7f7ab0160f00 2024-05-08T23:21:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.530+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab00678f0 con 0x7f7ab0160f00 2024-05-08T23:21:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.530+0000 7f7a9effd700 1 -- 172.21.15.5:0/1822164459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7aa801b3b0 con 0x7f7ab0160f00 2024-05-08T23:21:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.530+0000 7f7a9effd700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a9006ac90 0x7f7a9006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.531+0000 7f7a9effd700 1 -- 172.21.15.5:0/1822164459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7aa8088b00 con 0x7f7ab0160f00 2024-05-08T23:21:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.531+0000 7f7aae59c700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a9006ac90 0x7f7a9006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.531+0000 7f7aae59c700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a9006ac90 0x7f7a9006d150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000c00 tx=0x7f7aa0008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.535+0000 7f7a9effd700 1 -- 172.21.15.5:0/1822164459 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7aa80540a0 con 0x7f7ab0160f00 2024-05-08T23:21:59.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:21:59.767+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7ab00d0d90 con 0x7f7ab0160f00 2024-05-08T23:22:00.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.491+0000 7f7a9effd700 1 -- 172.21.15.5:0/1822164459 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7aa80576c0 con 0x7f7ab0160f00 2024-05-08T23:22:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a9006ac90 msgr2=0x7f7a9006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a9006ac90 0x7f7a9006d150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000c00 tx=0x7f7aa0008070 comp rx=0 tx=0).stop 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ab0160f00 msgr2=0x7f7ab00b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ab0160f00 0x7f7ab00b7af0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800cb10 tx=0x7f7aa800ce20 comp rx=0 tx=0).stop 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 shutdown_connections 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a9006ac90 0x7f7a9006d150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ab0160f00 0x7f7ab00b7af0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ab00b2f60 0x7f7ab00b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 --2- 172.21.15.5:0/1822164459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ab00d22d0 0x7f7ab0149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 >> 172.21.15.5:0/1822164459 conn(0x7f7ab008afe0 msgr2=0x7f7ab00b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 shutdown_connections 2024-05-08T23:22:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.494+0000 7f7ab4ae1700 1 -- 172.21.15.5:0/1822164459 wait complete. 2024-05-08T23:22:00.496 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-08T23:22:00.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T23:22:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.650+0000 7f880ab79700 1 Processor -- start 2024-05-08T23:22:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.650+0000 7f880ab79700 1 -- start start 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f880ab79700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f88040cf840 0x7f88040af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f880ab79700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f880406a960 0x7f880415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f880ab79700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f880415da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f880ab79700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88040cd980 con 0x7f880406a960 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f880ab79700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88040ccdb0 con 0x7f88040cf840 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f880ab79700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040cd050 con 0x7f88040d0eb0 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8803fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f880406a960 0x7f880415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8803fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f880406a960 0x7f880415d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58916/0 (socket says 172.21.15.5:58916) 2024-05-08T23:22:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8803fff700 1 -- 172.21.15.5:0/265909261 learned_addr learned my addr 172.21.15.5:0/265909261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8808915700 1 --2- 172.21.15.5:0/265909261 >> v2:172.21.15.49:3300/0 conn(0x7f88040cf840 0x7f88040af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8809116700 1 --2- 172.21.15.5:0/265909261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f880415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8803fff700 1 -- 172.21.15.5:0/265909261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 msgr2=0x7f880415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8803fff700 1 --2- 172.21.15.5:0/265909261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f880415da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8803fff700 1 -- 172.21.15.5:0/265909261 >> v2:172.21.15.49:3300/0 conn(0x7f88040cf840 msgr2=0x7f88040af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8803fff700 1 --2- 172.21.15.5:0/265909261 >> v2:172.21.15.49:3300/0 conn(0x7f88040cf840 0x7f88040af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8803fff700 1 -- 172.21.15.5:0/265909261 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040cd330 con 0x7f880406a960 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.651+0000 7f8809116700 1 --2- 172.21.15.5:0/265909261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f880415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f8803fff700 1 --2- 172.21.15.5:0/265909261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f880415d4f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8804071f10 tx=0x7f87f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6e76878a98f9cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f88037fe700 1 -- 172.21.15.5:0/265909261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000b870 con 0x7f880406a960 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f88037fe700 1 -- 172.21.15.5:0/265909261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f000b9d0 con 0x7f880406a960 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f88037fe700 1 -- 172.21.15.5:0/265909261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000bd60 con 0x7f880406a960 2024-05-08T23:22:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f880ab79700 1 -- 172.21.15.5:0/265909261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 msgr2=0x7f880415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f880ab79700 1 --2- 172.21.15.5:0/265909261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f880415d4f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8804071f10 tx=0x7f87f0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f880ab79700 1 -- 172.21.15.5:0/265909261 shutdown_connections 2024-05-08T23:22:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f880ab79700 1 --2- 172.21.15.5:0/265909261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f880415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f880ab79700 1 --2- 172.21.15.5:0/265909261 >> v2:172.21.15.49:3300/0 conn(0x7f88040cf840 0x7f88040af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.652+0000 7f880ab79700 1 --2- 172.21.15.5:0/265909261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f880415d4f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.653+0000 7f880ab79700 1 -- 172.21.15.5:0/265909261 >> 172.21.15.5:0/265909261 conn(0x7f88040d0320 msgr2=0x7f88040d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.653+0000 7f880ab79700 1 -- 172.21.15.5:0/265909261 shutdown_connections 2024-05-08T23:22:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.653+0000 7f880ab79700 1 -- 172.21.15.5:0/265909261 wait complete. 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.654+0000 7f880ab79700 1 Processor -- start 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.654+0000 7f880ab79700 1 -- start start 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.655+0000 7f880ab79700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f8804153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.655+0000 7f880ab79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88040cf840 0x7f88040b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f880ab79700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f88040ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f880ab79700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804153f50 con 0x7f880406a960 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f880ab79700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804157c20 con 0x7f88040cf840 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f880ab79700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8804157f00 con 0x7f88040d0eb0 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f8804153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f8804153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58932/0 (socket says 172.21.15.5:58932) 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 -- 172.21.15.5:0/1070452633 learned_addr learned my addr 172.21.15.5:0/1070452633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8803fff700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88040cf840 0x7f88040b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8809116700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f88040ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 -- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 msgr2=0x7f88040ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f88040ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 -- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88040cf840 msgr2=0x7f88040b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88040cf840 0x7f88040b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 -- 172.21.15.5:0/1070452633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0009550 con 0x7f880406a960 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8803fff700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88040cf840 0x7f88040b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f8808915700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f8804153a10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f87f800d800 tx=0x7f87f800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.656+0000 7f88017fa700 1 -- 172.21.15.5:0/1070452633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80125f0 con 0x7f880406a960 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.657+0000 7f88017fa700 1 -- 172.21.15.5:0/1070452633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8012750 con 0x7f880406a960 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.657+0000 7f88017fa700 1 -- 172.21.15.5:0/1070452633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8012ae0 con 0x7f880406a960 2024-05-08T23:22:00.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.657+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88041581e0 con 0x7f880406a960 2024-05-08T23:22:00.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.657+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88040b4e70 con 0x7f880406a960 2024-05-08T23:22:00.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.658+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8804158830 con 0x7f880406a960 2024-05-08T23:22:00.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.658+0000 7f88017fa700 1 -- 172.21.15.5:0/1070452633 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f87f8012c40 con 0x7f880406a960 2024-05-08T23:22:00.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.659+0000 7f88017fa700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87e806ac90 0x7f87e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:00.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.659+0000 7f88017fa700 1 -- 172.21.15.5:0/1070452633 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87f8088f60 con 0x7f880406a960 2024-05-08T23:22:00.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.659+0000 7f8803fff700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87e806ac90 0x7f87e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:00.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.659+0000 7f8803fff700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87e806ac90 0x7f87e806d150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87f0014040 tx=0x7f87f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:00.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.663+0000 7f88017fa700 1 -- 172.21.15.5:0/1070452633 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8804158830 con 0x7f880406a960 2024-05-08T23:22:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.969+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8804158830 con 0x7f880406a960 2024-05-08T23:22:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.969+0000 7f88017fa700 1 -- 172.21.15.5:0/1070452633 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f8804158830 con 0x7f880406a960 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm4+KLORAAJGma8zcoGLift7K5wyvWlA==", 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm5iMNDBAANT+UrtTUBpYyQ9dzJyS01Q==", 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-08T23:22:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmRixkBhAAk9s24ERPJ/6nCJY3g7QppA==", 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmPmurExAAxIzDzbLZ3s35XVJrvYiVCA==", 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:22:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCrCDxmI4/bNBAAzNXcuLQCS9gaAJ0r4iYKlA==", 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCuCDxmMS6HNhAAcTP6nStED06bdM4MtbEPjQ==", 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCxCDxmDGv5NRAAljFQZXLGWwt+UXZ+HCdx0Q==", 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:22:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC0CDxmL+5LNRAAVPySIUtICBwIokJL9fHFtw==", 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC5CDxm/F15BhAAEx9mKA9izSmb+xyxjEi9FA==", 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC8CDxm0zAYBhAAjHT+TZFqk5MWvd0P5LmwfA==", 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:22:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC/CDxmPcbiBRAAOmWNgaQGi6YGKTp8SJekUg==", 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDCCDxm9FsDBRAA/lfmrpmL4UO3s/tJJ4o6yQ==", 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm9tLAGBAAK8Lq89gJ902XqzXYAbYfkg==", 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:22:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmCStoIBAAhraObJ5LS9LtF97+hIcTFQ==", 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm2NzHHBAAUbeD58MntIoiwT8LekjIlw==", 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmyvveIxAATEn7nOPHqtqSbdfQid25HA==", 2024-05-08T23:22:00.980 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm3ZIxDxAAM8M5YUfK9G2PkTzU4wY/+w==", 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:22:00.981 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mds", 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNCDxm4HNaEBAAFT/5dmTd/zWZIFh6Y34wFA==", 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mds" 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mgr", 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNCDxmFX9aEBAAuu9MEhtBWmSDI8QfMiVi2Q==", 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.982 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-osd", 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNCDxmSIhaEBAAWjc0sn8IJYZQ4v0UAZ1QUg==", 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-osd" 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd", 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNCDxmspVaEBAA1vFZmphc6E28fpqCN9AjSQ==", 2024-05-08T23:22:00.983 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNCDxmop9aEBAAAsUj0CYDaT8T6kaREXEPLA==", 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rgw", 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNCDxmpKhaEBAAZOTBR68y01OCNH3bT0sYRw==", 2024-05-08T23:22:00.984 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxmntlgNRAA7JOIS+0NL1KmSSZhZjXlVQ==", 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-08T23:22:00.985 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm1dPQMBAAhu0CK6DcucQteuTo7qN2IQ==", 2024-05-08T23:22:00.986 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:22:00.986 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:22:00.986 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:22:00.986 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:22:00.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:00.986 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-08T23:22:00.986 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:00.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87e806ac90 msgr2=0x7f87e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:00.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87e806ac90 0x7f87e806d150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87f0014040 tx=0x7f87f0008030 comp rx=0 tx=0).stop 2024-05-08T23:22:00.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 msgr2=0x7f8804153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:00.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f8804153a10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f87f800d800 tx=0x7f87f800db10 comp rx=0 tx=0).stop 2024-05-08T23:22:00.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 shutdown_connections 2024-05-08T23:22:00.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87e806ac90 0x7f87e806d150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f88040d0eb0 0x7f88040ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f880406a960 0x7f8804153a10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 --2- 172.21.15.5:0/1070452633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88040cf840 0x7f88040b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:00.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 >> 172.21.15.5:0/1070452633 conn(0x7f88040d0320 msgr2=0x7f88040c5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:00.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 shutdown_connections 2024-05-08T23:22:00.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:00.972+0000 7f880ab79700 1 -- 172.21.15.5:0/1070452633 wait complete. 2024-05-08T23:22:00.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-08T23:22:01.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.127+0000 7f696aa91700 1 Processor -- start 2024-05-08T23:22:01.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.127+0000 7f696aa91700 1 -- start start 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696aa91700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69640cf840 0x7f69640af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696aa91700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69640d0eb0 0x7f6964161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696aa91700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69640cd980 con 0x7f696406a960 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696aa91700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69640ccdb0 con 0x7f69640cf840 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696aa91700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69640cd050 con 0x7f69640d0eb0 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696882d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69640cf840 0x7f69640af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696882d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69640cf840 0x7f69640af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58938/0 (socket says 172.21.15.5:58938) 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696882d700 1 -- 172.21.15.5:0/3115449446 learned_addr learned my addr 172.21.15.5:0/3115449446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696902e700 1 --2- 172.21.15.5:0/3115449446 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69640d0eb0 0x7f6964161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696882d700 1 -- 172.21.15.5:0/3115449446 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69640d0eb0 msgr2=0x7f6964161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696882d700 1 --2- 172.21.15.5:0/3115449446 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69640d0eb0 0x7f6964161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696882d700 1 -- 172.21.15.5:0/3115449446 >> v1:172.21.15.49:6789/0 conn(0x7f696406a960 legacy=0x7f696415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696882d700 1 -- 172.21.15.5:0/3115449446 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69640cd330 con 0x7f69640cf840 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696902e700 1 --2- 172.21.15.5:0/3115449446 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69640d0eb0 0x7f6964161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f696882d700 1 --2- 172.21.15.5:0/3115449446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f69640af440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f69540098f0 tx=0x7f6954009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07990f97856faf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.128+0000 7f69637fe700 1 -- 172.21.15.5:0/3115449446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954004230 con 0x7f69640cf840 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f69637fe700 1 -- 172.21.15.5:0/3115449446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954004390 con 0x7f69640cf840 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f69637fe700 1 -- 172.21.15.5:0/3115449446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540036a0 con 0x7f69640cf840 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f696aa91700 1 -- 172.21.15.5:0/3115449446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 msgr2=0x7f69640af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f696aa91700 1 --2- 172.21.15.5:0/3115449446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f69640af440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f69540098f0 tx=0x7f6954009c00 comp rx=0 tx=0).stop 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f696aa91700 1 -- 172.21.15.5:0/3115449446 shutdown_connections 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f696aa91700 1 --2- 172.21.15.5:0/3115449446 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69640d0eb0 0x7f6964161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f696aa91700 1 --2- 172.21.15.5:0/3115449446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f69640af440 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f696aa91700 1 -- 172.21.15.5:0/3115449446 >> 172.21.15.5:0/3115449446 conn(0x7f69640d0320 msgr2=0x7f69640d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:01.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f696aa91700 1 -- 172.21.15.5:0/3115449446 shutdown_connections 2024-05-08T23:22:01.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.129+0000 7f696aa91700 1 -- 172.21.15.5:0/3115449446 wait complete. 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.130+0000 7f696aa91700 1 Processor -- start 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.131+0000 7f696aa91700 1 -- start start 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.131+0000 7f696aa91700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f696406a960 0x7f696415f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.131+0000 7f696aa91700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f696415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f696aa91700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69640d0eb0 0x7f69640ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f696aa91700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696415e8d0 con 0x7f69640cf840 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f696aa91700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696415ebb0 con 0x7f69640d0eb0 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f696aa91700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696415ee90 con 0x7f696406a960 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f696415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f696415e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58952/0 (socket says 172.21.15.5:58952) 2024-05-08T23:22:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 -- 172.21.15.5:0/2491836528 learned_addr learned my addr 172.21.15.5:0/2491836528 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f696902e700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69640d0eb0 0x7f69640ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f696882d700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f696406a960 0x7f696415f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 -- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f696406a960 msgr2=0x7f696415f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f696406a960 0x7f696415f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 -- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69640d0eb0 msgr2=0x7f69640ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69640d0eb0 0x7f69640ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 -- 172.21.15.5:0/2491836528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954009550 con 0x7f69640cf840 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f696882d700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f696406a960 0x7f696415f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f6963fff700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f696415e010 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f695800c4d0 tx=0x7f695800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:01.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f69617fa700 1 -- 172.21.15.5:0/2491836528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958011790 con 0x7f69640cf840 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f69617fa700 1 -- 172.21.15.5:0/2491836528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580118f0 con 0x7f69640cf840 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f69617fa700 1 -- 172.21.15.5:0/2491836528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958011ca0 con 0x7f69640cf840 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.132+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696415f170 con 0x7f69640cf840 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.133+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696415feb0 con 0x7f69640cf840 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.133+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696415f690 con 0x7f69640cf840 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.134+0000 7f69617fa700 1 -- 172.21.15.5:0/2491836528 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f695801c460 con 0x7f69640cf840 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.134+0000 7f69617fa700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f695006ac90 0x7f695006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.134+0000 7f69617fa700 1 -- 172.21.15.5:0/2491836528 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6958088de0 con 0x7f69640cf840 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.135+0000 7f696882d700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f695006ac90 0x7f695006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.135+0000 7f696882d700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f695006ac90 0x7f695006d150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6954009130 tx=0x7f6954008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:01.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.139+0000 7f69617fa700 1 -- 172.21.15.5:0/2491836528 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f696415f690 con 0x7f69640cf840 2024-05-08T23:22:01.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.449+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f696415f690 con 0x7f69640cf840 2024-05-08T23:22:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.453+0000 7f69617fa700 1 -- 172.21.15.5:0/2491836528 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f696415f690 con 0x7f69640cf840 2024-05-08T23:22:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.455+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f695006ac90 msgr2=0x7f695006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f695006ac90 0x7f695006d150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6954009130 tx=0x7f6954008030 comp rx=0 tx=0).stop 2024-05-08T23:22:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 msgr2=0x7f696415e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f696415e010 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f695800c4d0 tx=0x7f695800c7e0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 shutdown_connections 2024-05-08T23:22:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f695006ac90 0x7f695006d150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69640d0eb0 0x7f69640ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f696406a960 0x7f696415f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 --2- 172.21.15.5:0/2491836528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69640cf840 0x7f696415e010 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 >> 172.21.15.5:0/2491836528 conn(0x7f69640d0320 msgr2=0x7f69640ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 shutdown_connections 2024-05-08T23:22:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.456+0000 7f696aa91700 1 -- 172.21.15.5:0/2491836528 wait complete. 2024-05-08T23:22:01.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.607+0000 7f930e899700 1 Processor -- start 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.607+0000 7f930e899700 1 -- start start 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.608+0000 7f930e899700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f93080cf840 0x7f93080af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.608+0000 7f930e899700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 0x7f930815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.608+0000 7f930e899700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93080cd980 con 0x7f93080d0eb0 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.608+0000 7f930e899700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93080ccdb0 con 0x7f93080cf840 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.608+0000 7f930e899700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93080cd050 con 0x7f930806a960 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.608+0000 7f930ce36700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f93080d0eb0 0x7f930806dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34066/0 (socket says 172.21.15.5:34066) 2024-05-08T23:22:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.608+0000 7f930ce36700 1 -- 172.21.15.5:0/1510940641 learned_addr learned my addr 172.21.15.5:0/1510940641 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9307fff700 1 --2- 172.21.15.5:0/1510940641 >> v2:172.21.15.49:3300/0 conn(0x7f93080cf840 0x7f93080af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f93077fe700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 0x7f930815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121774362 0 0) 0x7f93080cd980 con 0x7f93080d0eb0 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92f00035b0 con 0x7f93080d0eb0 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 588109500 0 0) 0x7f92f00035b0 con 0x7f93080d0eb0 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 msgr2=0x7f930815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 0x7f930815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 >> v2:172.21.15.49:3300/0 conn(0x7f93080cf840 msgr2=0x7f93080af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 --2- 172.21.15.5:0/1510940641 >> v2:172.21.15.49:3300/0 conn(0x7f93080cf840 0x7f93080af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93080cd330 con 0x7f93080d0eb0 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f93000019f0 con 0x7f93080d0eb0 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f93077fe700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 0x7f930815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9307fff700 1 --2- 172.21.15.5:0/1510940641 >> v2:172.21.15.49:3300/0 conn(0x7f93080cf840 0x7f93080af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 >> v1:172.21.15.5:6789/0 conn(0x7f93080d0eb0 legacy=0x7f930806dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f0003a00 0x7f92f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f92f0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 0x7f92f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080ccdb0 con 0x7f92f0004850 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080cd050 con 0x7f92f0003a00 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f000ba10 con 0x7f930806a960 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f93077fe700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f92f0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f9307fff700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f0003a00 0x7f92f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.609+0000 7f930ce36700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 0x7f92f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f93077fe700 1 -- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 msgr2=0x7f92f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f93077fe700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f930806a960 0x7f92f0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f93077fe700 1 -- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f0003a00 msgr2=0x7f92f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f93077fe700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f0003a00 0x7f92f0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f93077fe700 1 -- 172.21.15.5:0/1510940641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc0094a0 con 0x7f92f0004850 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f93077fe700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f92f0004cd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009d00 tx=0x7f92f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2101a3e5618a778 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc009df0 con 0x7f92f0004850 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc00e9a0 con 0x7f92f0004850 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f9306ffd700 1 -- 172.21.15.5:0/1510940641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc00ed50 con 0x7f92f0004850 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f930e899700 1 -- 172.21.15.5:0/1510940641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 msgr2=0x7f92f0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f930e899700 1 --2- 172.21.15.5:0/1510940641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f92f0004cd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009d00 tx=0x7f92f0005880 comp rx=0 tx=0).stop 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.610+0000 7f930ce36700 1 -- 172.21.15.5:0/1510940641 reap_dead start 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.611+0000 7f930e899700 1 -- 172.21.15.5:0/1510940641 shutdown_connections 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.611+0000 7f930e899700 1 -- 172.21.15.5:0/1510940641 >> 172.21.15.5:0/1510940641 conn(0x7f93080d0320 msgr2=0x7f93080d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.611+0000 7f930e899700 1 -- 172.21.15.5:0/1510940641 shutdown_connections 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.611+0000 7f930e899700 1 -- 172.21.15.5:0/1510940641 wait complete. 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.612+0000 7f930e899700 1 Processor -- start 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.612+0000 7f930e899700 1 -- start start 2024-05-08T23:22:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f930e899700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f930814a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f930e899700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93080b5380 0x7f93080b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f930e899700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93080b2f20 0x7f93080b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f930e899700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080b5d40 con 0x7f92f0004850 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f930e899700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080b3c10 con 0x7f93080b5380 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f930e899700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93080b3ef0 con 0x7f93080b2f20 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f9307fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f930814a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f9307fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f930814a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58972/0 (socket says 172.21.15.5:58972) 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f9307fff700 1 -- 172.21.15.5:0/2153365004 learned_addr learned my addr 172.21.15.5:0/2153365004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f930ce36700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93080b2f20 0x7f93080b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f93077fe700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93080b5380 0x7f93080b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f9307fff700 1 -- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93080b2f20 msgr2=0x7f93080b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f9307fff700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93080b2f20 0x7f93080b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f9307fff700 1 -- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93080b5380 msgr2=0x7f93080b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f9307fff700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93080b5380 0x7f93080b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f9307fff700 1 -- 172.21.15.5:0/2153365004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93080b41d0 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.613+0000 7f93077fe700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93080b5380 0x7f93080b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.614+0000 7f9307fff700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f930814a8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f92f8011cd0 tx=0x7f92f800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.614+0000 7f9304ff9700 1 -- 172.21.15.5:0/2153365004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f800fb30 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.614+0000 7f9304ff9700 1 -- 172.21.15.5:0/2153365004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f800fc90 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.614+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93080b44b0 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.614+0000 7f9304ff9700 1 -- 172.21.15.5:0/2153365004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8004030 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.614+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93080b3910 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.615+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9308068000 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.615+0000 7f9304ff9700 1 -- 172.21.15.5:0/2153365004 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f92f8004190 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.616+0000 7f9304ff9700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92f406ac90 0x7f92f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.616+0000 7f9304ff9700 1 -- 172.21.15.5:0/2153365004 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f92f80896e0 con 0x7f92f0004850 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.616+0000 7f93077fe700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92f406ac90 0x7f92f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.616+0000 7f93077fe700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92f406ac90 0x7f92f406d150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9308155080 tx=0x7f92fc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:01.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.620+0000 7f9304ff9700 1 -- 172.21.15.5:0/2153365004 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f92f8054c10 con 0x7f92f0004850 2024-05-08T23:22:01.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.925+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f93080b62a0 con 0x7f92f0004850 2024-05-08T23:22:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.929+0000 7f9304ff9700 1 -- 172.21.15.5:0/2153365004 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f92f8022020 con 0x7f92f0004850 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92f406ac90 msgr2=0x7f92f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92f406ac90 0x7f92f406d150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9308155080 tx=0x7f92fc008030 comp rx=0 tx=0).stop 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 msgr2=0x7f930814a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f930814a8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f92f8011cd0 tx=0x7f92f800f520 comp rx=0 tx=0).stop 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 shutdown_connections 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92f406ac90 0x7f92f406d150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93080b2f20 0x7f93080b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92f0004850 0x7f930814a8f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 --2- 172.21.15.5:0/2153365004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93080b5380 0x7f93080b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 >> 172.21.15.5:0/2153365004 conn(0x7f93080d0320 msgr2=0x7f930815c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 shutdown_connections 2024-05-08T23:22:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:01.931+0000 7f930e899700 1 -- 172.21.15.5:0/2153365004 wait complete. 2024-05-08T23:22:01.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-08T23:22:02.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.083+0000 7fad8afe9700 1 Processor -- start 2024-05-08T23:22:02.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.083+0000 7fad8afe9700 1 -- start start 2024-05-08T23:22:02.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad8afe9700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fad8415f830 0x7fad841664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad8afe9700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8406be40 con 0x7fad84155da0 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad8afe9700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8406c2d0 con 0x7fad8406e200 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad8afe9700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8406dcb0 con 0x7fad8415f830 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad88d85700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fad84155da0 0x7fad8406d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34068/0 (socket says 172.21.15.5:34068) 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad88d85700 1 -- 172.21.15.5:0/720473640 learned_addr learned my addr 172.21.15.5:0/720473640 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad89586700 1 --2- 172.21.15.5:0/720473640 >> v2:172.21.15.49:3300/0 conn(0x7fad8415f830 0x7fad841664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad837fe700 1 -- 172.21.15.5:0/720473640 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565452052 0 0) 0x7fad8406be40 con 0x7fad84155da0 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad837fe700 1 -- 172.21.15.5:0/720473640 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad740035b0 con 0x7fad84155da0 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad837fe700 1 -- 172.21.15.5:0/720473640 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638531447 0 0) 0x7fad8406c2d0 con 0x7fad8406e200 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.084+0000 7fad837fe700 1 -- 172.21.15.5:0/720473640 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad8406be40 con 0x7fad8406e200 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.085+0000 7fad89586700 1 -- 172.21.15.5:0/720473640 >> v1:172.21.15.49:6789/0 conn(0x7fad8406e200 legacy=0x7fad8415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.085+0000 7fad837fe700 1 -- 172.21.15.5:0/720473640 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525918647 0 0) 0x7fad740035b0 con 0x7fad84155da0 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.085+0000 7fad89586700 1 -- 172.21.15.5:0/720473640 >> v1:172.21.15.5:6789/0 conn(0x7fad84155da0 legacy=0x7fad8406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.085+0000 7fad89586700 1 -- 172.21.15.5:0/720473640 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8406e660 con 0x7fad8415f830 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.085+0000 7fad89586700 1 --2- 172.21.15.5:0/720473640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8415f830 0x7fad841664b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009d90 tx=0x7fad7c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ee749f3c9d67e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.085+0000 7fad837fe700 1 -- 172.21.15.5:0/720473640 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00e070 con 0x7fad8415f830 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.085+0000 7fad837fe700 1 -- 172.21.15.5:0/720473640 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c003a20 con 0x7fad8415f830 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.085+0000 7fad837fe700 1 -- 172.21.15.5:0/720473640 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78001670 con 0x7fad8415f830 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.086+0000 7fad8afe9700 1 -- 172.21.15.5:0/720473640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8415f830 msgr2=0x7fad841664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.086+0000 7fad8afe9700 1 --2- 172.21.15.5:0/720473640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8415f830 0x7fad841664b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009d90 tx=0x7fad7c00b330 comp rx=0 tx=0).stop 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.086+0000 7fad8afe9700 1 -- 172.21.15.5:0/720473640 shutdown_connections 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.086+0000 7fad8afe9700 1 --2- 172.21.15.5:0/720473640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8415f830 0x7fad841664b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.086+0000 7fad8afe9700 1 -- 172.21.15.5:0/720473640 >> 172.21.15.5:0/720473640 conn(0x7fad840cc580 msgr2=0x7fad840cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:02.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.086+0000 7fad8afe9700 1 -- 172.21.15.5:0/720473640 shutdown_connections 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.086+0000 7fad8afe9700 1 -- 172.21.15.5:0/720473640 wait complete. 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.087+0000 7fad8afe9700 1 Processor -- start 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.087+0000 7fad8afe9700 1 -- start start 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad8afe9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8406e200 0x7fad84140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad8afe9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fad84155da0 0x7fad841e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad8afe9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad8415f830 0x7fad841404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad8afe9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad84141760 con 0x7fad8415f830 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad8afe9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840b28e0 con 0x7fad8406e200 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad8afe9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad84157840 con 0x7fad84155da0 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad89586700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad8415f830 0x7fad841404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad89586700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad8415f830 0x7fad841404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58974/0 (socket says 172.21.15.5:58974) 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad89586700 1 -- 172.21.15.5:0/230214058 learned_addr learned my addr 172.21.15.5:0/230214058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad88d85700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8406e200 0x7fad84140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.088+0000 7fad83fff700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fad84155da0 0x7fad841e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad89586700 1 -- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fad84155da0 msgr2=0x7fad841e4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad89586700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fad84155da0 0x7fad841e4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad89586700 1 -- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8406e200 msgr2=0x7fad84140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad89586700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8406e200 0x7fad84140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad89586700 1 -- 172.21.15.5:0/230214058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c009a70 con 0x7fad8415f830 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad89586700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad8415f830 0x7fad841404c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fad7c015040 tx=0x7fad7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad817fa700 1 -- 172.21.15.5:0/230214058 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0044c0 con 0x7fad8415f830 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad817fa700 1 -- 172.21.15.5:0/230214058 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c004620 con 0x7fad8415f830 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad84157b20 con 0x7fad8415f830 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad817fa700 1 -- 172.21.15.5:0/230214058 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c012430 con 0x7fad8415f830 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.089+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad841e56f0 con 0x7fad8415f830 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.090+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad84157cb0 con 0x7fad8415f830 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.091+0000 7fad817fa700 1 -- 172.21.15.5:0/230214058 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fad7c012590 con 0x7fad8415f830 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.091+0000 7fad817fa700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fad6806ac90 0x7fad6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.091+0000 7fad817fa700 1 -- 172.21.15.5:0/230214058 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fad7c08a3b0 con 0x7fad8415f830 2024-05-08T23:22:02.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.091+0000 7fad88d85700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fad6806ac90 0x7fad6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.092+0000 7fad88d85700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fad6806ac90 0x7fad6806d150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad78000fd0 tx=0x7fad78000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:02.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.096+0000 7fad817fa700 1 -- 172.21.15.5:0/230214058 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fad7c055a20 con 0x7fad8415f830 2024-05-08T23:22:02.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.403+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fad841e5030 con 0x7fad8415f830 2024-05-08T23:22:02.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.407+0000 7fad817fa700 1 -- 172.21.15.5:0/230214058 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fad7c0160a0 con 0x7fad8415f830 2024-05-08T23:22:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fad6806ac90 msgr2=0x7fad6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fad6806ac90 0x7fad6806d150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad78000fd0 tx=0x7fad78000ed0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad8415f830 msgr2=0x7fad841404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad8415f830 0x7fad841404c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fad7c015040 tx=0x7fad7c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 shutdown_connections 2024-05-08T23:22:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fad6806ac90 0x7fad6806d150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad8415f830 0x7fad841404c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8406e200 0x7fad84140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 --2- 172.21.15.5:0/230214058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fad84155da0 0x7fad841e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 >> 172.21.15.5:0/230214058 conn(0x7fad840cc580 msgr2=0x7fad840af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 shutdown_connections 2024-05-08T23:22:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.410+0000 7fad8afe9700 1 -- 172.21.15.5:0/230214058 wait complete. 2024-05-08T23:22:02.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-08T23:22:02.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.562+0000 7f84999ec700 1 Processor -- start 2024-05-08T23:22:02.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.562+0000 7f84999ec700 1 -- start start 2024-05-08T23:22:02.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.562+0000 7f84999ec700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f84940cffa0 0x7f84940d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.562+0000 7f84999ec700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84940b2200 0x7f84940b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.562+0000 7f84999ec700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84940d1530 con 0x7f849415d920 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.562+0000 7f84999ec700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84940d0960 con 0x7f84940b2200 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.562+0000 7f84999ec700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84940d1810 con 0x7f84940cffa0 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f8492ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f849415d920 0x7f84940cee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34080/0 (socket says 172.21.15.5:34080) 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f8492ffd700 1 -- 172.21.15.5:0/3239885689 learned_addr learned my addr 172.21.15.5:0/3239885689 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f84927fc700 1 --2- 172.21.15.5:0/3239885689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f84940cffa0 0x7f84940d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f84937fe700 1 --2- 172.21.15.5:0/3239885689 >> v2:172.21.15.49:3300/0 conn(0x7f84940b2200 0x7f84940b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f8491ffb700 1 -- 172.21.15.5:0/3239885689 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019988784 0 0) 0x7f84940d1530 con 0x7f849415d920 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f8491ffb700 1 -- 172.21.15.5:0/3239885689 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84800035b0 con 0x7f849415d920 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f84927fc700 1 -- 172.21.15.5:0/3239885689 >> v2:172.21.15.49:3300/0 conn(0x7f84940b2200 msgr2=0x7f84940b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f84927fc700 1 --2- 172.21.15.5:0/3239885689 >> v2:172.21.15.49:3300/0 conn(0x7f84940b2200 0x7f84940b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f84927fc700 1 -- 172.21.15.5:0/3239885689 >> v1:172.21.15.5:6789/0 conn(0x7f849415d920 legacy=0x7f84940cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f84927fc700 1 -- 172.21.15.5:0/3239885689 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940d1af0 con 0x7f84940cffa0 2024-05-08T23:22:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f84937fe700 1 --2- 172.21.15.5:0/3239885689 >> v2:172.21.15.49:3300/0 conn(0x7f84940b2200 0x7f84940b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.563+0000 7f84927fc700 1 --2- 172.21.15.5:0/3239885689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f84940cffa0 0x7f84940d0420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f847c009840 tx=0x7f847c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4881ce2a6206a8c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f8491ffb700 1 -- 172.21.15.5:0/3239885689 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c011830 con 0x7f84940cffa0 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f8491ffb700 1 -- 172.21.15.5:0/3239885689 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c011990 con 0x7f84940cffa0 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f8491ffb700 1 -- 172.21.15.5:0/3239885689 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c011d20 con 0x7f84940cffa0 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f84999ec700 1 -- 172.21.15.5:0/3239885689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f84940cffa0 msgr2=0x7f84940d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f84999ec700 1 --2- 172.21.15.5:0/3239885689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f84940cffa0 0x7f84940d0420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f847c009840 tx=0x7f847c009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f84999ec700 1 -- 172.21.15.5:0/3239885689 shutdown_connections 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f84999ec700 1 --2- 172.21.15.5:0/3239885689 >> v2:172.21.15.49:3300/0 conn(0x7f84940b2200 0x7f84940b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f84999ec700 1 --2- 172.21.15.5:0/3239885689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f84940cffa0 0x7f84940d0420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f84999ec700 1 -- 172.21.15.5:0/3239885689 >> 172.21.15.5:0/3239885689 conn(0x7f849408afe0 msgr2=0x7f84940d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f84999ec700 1 -- 172.21.15.5:0/3239885689 shutdown_connections 2024-05-08T23:22:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.564+0000 7f84999ec700 1 -- 172.21.15.5:0/3239885689 wait complete. 2024-05-08T23:22:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.566+0000 7f84999ec700 1 Processor -- start 2024-05-08T23:22:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.566+0000 7f84999ec700 1 -- start start 2024-05-08T23:22:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f84999ec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84940b2200 0x7f8494158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f84999ec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84940cffa0 0x7f84940b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f84999ec700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f849415d920 0x7f84941573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f84999ec700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941586a0 con 0x7f84940b2200 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f84999ec700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84940cd4f0 con 0x7f84940cffa0 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f84999ec700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84940c8040 con 0x7f849415d920 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f8492ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84940b2200 0x7f8494158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f8492ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84940b2200 0x7f8494158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58988/0 (socket says 172.21.15.5:58988) 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f8492ffd700 1 -- 172.21.15.5:0/1289056430 learned_addr learned my addr 172.21.15.5:0/1289056430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f84937fe700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f849415d920 0x7f84941573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f84927fc700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84940cffa0 0x7f84940b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f8492ffd700 1 -- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f849415d920 msgr2=0x7f84941573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f8492ffd700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f849415d920 0x7f84941573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f8492ffd700 1 -- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84940cffa0 msgr2=0x7f84940b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f8492ffd700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84940cffa0 0x7f84940b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.567+0000 7f8492ffd700 1 -- 172.21.15.5:0/1289056430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009a70 con 0x7f84940b2200 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.568+0000 7f8492ffd700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84940b2200 0x7f8494158160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8484001eb0 tx=0x7f848400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.568+0000 7f848bfff700 1 -- 172.21.15.5:0/1289056430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400cea0 con 0x7f84940b2200 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.568+0000 7f848bfff700 1 -- 172.21.15.5:0/1289056430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484005d70 con 0x7f84940b2200 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.568+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c0094a0 con 0x7f84940b2200 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.568+0000 7f848bfff700 1 -- 172.21.15.5:0/1289056430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84840072f0 con 0x7f84940b2200 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.568+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84940c8600 con 0x7f84940b2200 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.569+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84940678f0 con 0x7f84940b2200 2024-05-08T23:22:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.569+0000 7f848bfff700 1 -- 172.21.15.5:0/1289056430 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8484007450 con 0x7f84940b2200 2024-05-08T23:22:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.570+0000 7f848bfff700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f847406ac90 0x7f847406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.570+0000 7f848bfff700 1 -- 172.21.15.5:0/1289056430 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f84840883d0 con 0x7f84940b2200 2024-05-08T23:22:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.570+0000 7f84927fc700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f847406ac90 0x7f847406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.570+0000 7f84927fc700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f847406ac90 0x7f847406d150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f847c009fd0 tx=0x7f847c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.574+0000 7f848bfff700 1 -- 172.21.15.5:0/1289056430 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8484053910 con 0x7f84940b2200 2024-05-08T23:22:02.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.878+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f84940d0da0 con 0x7f84940b2200 2024-05-08T23:22:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.882+0000 7f848bfff700 1 -- 172.21.15.5:0/1289056430 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8484056f30 con 0x7f84940b2200 2024-05-08T23:22:02.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f847406ac90 msgr2=0x7f847406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f847406ac90 0x7f847406d150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f847c009fd0 tx=0x7f847c008030 comp rx=0 tx=0).stop 2024-05-08T23:22:02.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84940b2200 msgr2=0x7f8494158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:02.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84940b2200 0x7f8494158160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8484001eb0 tx=0x7f848400c7b0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 shutdown_connections 2024-05-08T23:22:02.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f847406ac90 0x7f847406d150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f849415d920 0x7f84941573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84940b2200 0x7f8494158160 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 --2- 172.21.15.5:0/1289056430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84940cffa0 0x7f84940b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:02.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 >> 172.21.15.5:0/1289056430 conn(0x7f849408afe0 msgr2=0x7f84940b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:02.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 shutdown_connections 2024-05-08T23:22:02.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:02.885+0000 7f84999ec700 1 -- 172.21.15.5:0/1289056430 wait complete. 2024-05-08T23:22:02.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-08T23:22:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.037+0000 7fc9b7d4a700 1 Processor -- start 2024-05-08T23:22:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.038+0000 7fc9b7d4a700 1 -- start start 2024-05-08T23:22:03.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.038+0000 7fc9b7d4a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc9b00b2aa0 0x7fc9b00c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.038+0000 7fc9b7d4a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00c60d0 0x7fc9b00ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.038+0000 7fc9b7d4a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b00c71f0 con 0x7fc9b015db00 2024-05-08T23:22:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.038+0000 7fc9b7d4a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b00ce630 con 0x7fc9b00b2aa0 2024-05-08T23:22:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b7d4a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b00c6560 con 0x7fc9b00c60d0 2024-05-08T23:22:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b52e5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc9b00b2aa0 0x7fc9b00c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b52e5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc9b00b2aa0 0x7fc9b00c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58998/0 (socket says 172.21.15.5:58998) 2024-05-08T23:22:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b52e5700 1 -- 172.21.15.5:0/2782756078 learned_addr learned my addr 172.21.15.5:0/2782756078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b62e7700 1 --2- 172.21.15.5:0/2782756078 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00c60d0 0x7fc9b00ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b52e5700 1 -- 172.21.15.5:0/2782756078 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00c60d0 msgr2=0x7fc9b00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b52e5700 1 --2- 172.21.15.5:0/2782756078 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00c60d0 0x7fc9b00ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b52e5700 1 -- 172.21.15.5:0/2782756078 >> v1:172.21.15.49:6789/0 conn(0x7fc9b015db00 legacy=0x7fc9b00cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b52e5700 1 -- 172.21.15.5:0/2782756078 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b00c6950 con 0x7fc9b00b2aa0 2024-05-08T23:22:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b52e5700 1 --2- 172.21.15.5:0/2782756078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00b2aa0 0x7fc9b00c5b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00098f0 tx=0x7fc9a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb436ba32194950a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b4ae4700 1 -- 172.21.15.5:0/2782756078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0011830 con 0x7fc9b00b2aa0 2024-05-08T23:22:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b4ae4700 1 -- 172.21.15.5:0/2782756078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0011990 con 0x7fc9b00b2aa0 2024-05-08T23:22:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.039+0000 7fc9b4ae4700 1 -- 172.21.15.5:0/2782756078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0011d20 con 0x7fc9b00b2aa0 2024-05-08T23:22:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.040+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/2782756078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00b2aa0 msgr2=0x7fc9b00c5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.040+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/2782756078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00b2aa0 0x7fc9b00c5b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00098f0 tx=0x7fc9a0009c00 comp rx=0 tx=0).stop 2024-05-08T23:22:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.040+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/2782756078 shutdown_connections 2024-05-08T23:22:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.040+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/2782756078 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00c60d0 0x7fc9b00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.040+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/2782756078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00b2aa0 0x7fc9b00c5b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.040+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/2782756078 >> 172.21.15.5:0/2782756078 conn(0x7fc9b00b1f80 msgr2=0x7fc9b00b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.040+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/2782756078 shutdown_connections 2024-05-08T23:22:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.040+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/2782756078 wait complete. 2024-05-08T23:22:03.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.042+0000 7fc9b7d4a700 1 Processor -- start 2024-05-08T23:22:03.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.042+0000 7fc9b7d4a700 1 -- start start 2024-05-08T23:22:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b7d4a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00b2aa0 0x7fc9b01f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b7d4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b00c60d0 0x7fc9b01f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b7d4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b015db00 0x7fc9b00b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b7d4a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0157d80 con 0x7fc9b015db00 2024-05-08T23:22:03.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b7d4a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01f3f50 con 0x7fc9b00c60d0 2024-05-08T23:22:03.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b7d4a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01f6bd0 con 0x7fc9b00b2aa0 2024-05-08T23:22:03.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b62e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b015db00 0x7fc9b00b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b62e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b015db00 0x7fc9b00b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59008/0 (socket says 172.21.15.5:59008) 2024-05-08T23:22:03.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b62e7700 1 -- 172.21.15.5:0/3523078037 learned_addr learned my addr 172.21.15.5:0/3523078037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:03.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b52e5700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b00c60d0 0x7fc9b01f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b5ae6700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00b2aa0 0x7fc9b01f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b62e7700 1 -- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00b2aa0 msgr2=0x7fc9b01f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b62e7700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00b2aa0 0x7fc9b01f6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b62e7700 1 -- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b00c60d0 msgr2=0x7fc9b01f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b62e7700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b00c60d0 0x7fc9b01f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.043+0000 7fc9b62e7700 1 -- 172.21.15.5:0/3523078037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a0009550 con 0x7fc9b015db00 2024-05-08T23:22:03.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.044+0000 7fc9b62e7700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b015db00 0x7fc9b00b0e00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800e950 tx=0x7fc9a800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:03.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.044+0000 7fc9a67fc700 1 -- 172.21.15.5:0/3523078037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8009730 con 0x7fc9b015db00 2024-05-08T23:22:03.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.044+0000 7fc9a67fc700 1 -- 172.21.15.5:0/3523078037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8013680 con 0x7fc9b015db00 2024-05-08T23:22:03.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.044+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b01f6eb0 con 0x7fc9b015db00 2024-05-08T23:22:03.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.044+0000 7fc9a67fc700 1 -- 172.21.15.5:0/3523078037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8013a10 con 0x7fc9b015db00 2024-05-08T23:22:03.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.044+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b01f41f0 con 0x7fc9b015db00 2024-05-08T23:22:03.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.045+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b01f43e0 con 0x7fc9b015db00 2024-05-08T23:22:03.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.046+0000 7fc9a67fc700 1 -- 172.21.15.5:0/3523078037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc9a8013b70 con 0x7fc9b015db00 2024-05-08T23:22:03.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.046+0000 7fc9a67fc700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc99406ac90 0x7fc99406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.046+0000 7fc9a67fc700 1 -- 172.21.15.5:0/3523078037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc9a8088fb0 con 0x7fc9b015db00 2024-05-08T23:22:03.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.046+0000 7fc9b5ae6700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc99406ac90 0x7fc99406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.047+0000 7fc9b5ae6700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc99406ac90 0x7fc99406d150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0012e0 tx=0x7fc9ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:03.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.051+0000 7fc9a67fc700 1 -- 172.21.15.5:0/3523078037 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc9a80bc900 con 0x7fc9b015db00 2024-05-08T23:22:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.358+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc9b0068000 con 0x7fc9b015db00 2024-05-08T23:22:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.361+0000 7fc9a67fc700 1 -- 172.21.15.5:0/3523078037 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc9a80544e0 con 0x7fc9b015db00 2024-05-08T23:22:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc99406ac90 msgr2=0x7fc99406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc99406ac90 0x7fc99406d150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0012e0 tx=0x7fc9ac009040 comp rx=0 tx=0).stop 2024-05-08T23:22:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b015db00 msgr2=0x7fc9b00b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b015db00 0x7fc9b00b0e00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800e950 tx=0x7fc9a800ec60 comp rx=0 tx=0).stop 2024-05-08T23:22:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 shutdown_connections 2024-05-08T23:22:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc99406ac90 0x7fc99406d150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b015db00 0x7fc9b00b0e00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9b00b2aa0 0x7fc9b01f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 --2- 172.21.15.5:0/3523078037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9b00c60d0 0x7fc9b01f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 >> 172.21.15.5:0/3523078037 conn(0x7fc9b00b1f80 msgr2=0x7fc9b00cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 shutdown_connections 2024-05-08T23:22:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.364+0000 7fc9b7d4a700 1 -- 172.21.15.5:0/3523078037 wait complete. 2024-05-08T23:22:03.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-08T23:22:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.515+0000 7f7d0be93700 1 Processor -- start 2024-05-08T23:22:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.515+0000 7f7d0be93700 1 -- start start 2024-05-08T23:22:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.516+0000 7f7d0be93700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d040cf840 0x7f7d040af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.516+0000 7f7d0be93700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7d0406a960 0x7f7d0415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.516+0000 7f7d0be93700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d040cd980 con 0x7f7d040d0eb0 2024-05-08T23:22:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.516+0000 7f7d0be93700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d040ccdb0 con 0x7f7d040cf840 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.516+0000 7f7d0be93700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d040cd050 con 0x7f7d0406a960 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.516+0000 7f7d09c2f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d040cf840 0x7f7d040af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d09c2f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d040cf840 0x7f7d040af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59014/0 (socket says 172.21.15.5:59014) 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d09c2f700 1 -- 172.21.15.5:0/1833599764 learned_addr learned my addr 172.21.15.5:0/1833599764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d0942e700 1 --2- 172.21.15.5:0/1833599764 >> v2:172.21.15.49:3300/0 conn(0x7f7d0406a960 0x7f7d0415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d09c2f700 1 -- 172.21.15.5:0/1833599764 >> v2:172.21.15.49:3300/0 conn(0x7f7d0406a960 msgr2=0x7f7d0415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d09c2f700 1 --2- 172.21.15.5:0/1833599764 >> v2:172.21.15.49:3300/0 conn(0x7f7d0406a960 0x7f7d0415d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d09c2f700 1 -- 172.21.15.5:0/1833599764 >> v1:172.21.15.49:6789/0 conn(0x7f7d040d0eb0 legacy=0x7f7d0406dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d09c2f700 1 -- 172.21.15.5:0/1833599764 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040cd330 con 0x7f7d040cf840 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d09c2f700 1 --2- 172.21.15.5:0/1833599764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040af440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d040717d0 tx=0x7f7d00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9d05dd1f6676b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d08c2d700 1 -- 172.21.15.5:0/1833599764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00004230 con 0x7f7d040cf840 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d08c2d700 1 -- 172.21.15.5:0/1833599764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00004390 con 0x7f7d040cf840 2024-05-08T23:22:03.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.517+0000 7f7d08c2d700 1 -- 172.21.15.5:0/1833599764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000036a0 con 0x7f7d040cf840 2024-05-08T23:22:03.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.518+0000 7f7d0be93700 1 -- 172.21.15.5:0/1833599764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 msgr2=0x7f7d040af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.518+0000 7f7d0be93700 1 --2- 172.21.15.5:0/1833599764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040af440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d040717d0 tx=0x7f7d00009bd0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.518+0000 7f7d0be93700 1 -- 172.21.15.5:0/1833599764 shutdown_connections 2024-05-08T23:22:03.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.518+0000 7f7d0be93700 1 --2- 172.21.15.5:0/1833599764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040af440 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.518+0000 7f7d0be93700 1 --2- 172.21.15.5:0/1833599764 >> v2:172.21.15.49:3300/0 conn(0x7f7d0406a960 0x7f7d0415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.518+0000 7f7d0be93700 1 -- 172.21.15.5:0/1833599764 >> 172.21.15.5:0/1833599764 conn(0x7f7d040d0320 msgr2=0x7f7d040d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:03.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.518+0000 7f7d0be93700 1 -- 172.21.15.5:0/1833599764 shutdown_connections 2024-05-08T23:22:03.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.518+0000 7f7d0be93700 1 -- 172.21.15.5:0/1833599764 wait complete. 2024-05-08T23:22:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.519+0000 7f7d0be93700 1 Processor -- start 2024-05-08T23:22:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.519+0000 7f7d0be93700 1 -- start start 2024-05-08T23:22:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0be93700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d0406a960 0x7f7d0415fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0be93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0be93700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d040d0eb0 0x7f7d040b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0be93700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040ada00 con 0x7f7d040cf840 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0be93700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040b3f70 con 0x7f7d0406a960 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0be93700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040b4250 con 0x7f7d040d0eb0 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0942e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0942e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59030/0 (socket says 172.21.15.5:59030) 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0942e700 1 -- 172.21.15.5:0/4075717087 learned_addr learned my addr 172.21.15.5:0/4075717087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d09c2f700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d0406a960 0x7f7d0415fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.520+0000 7f7d0a430700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d040d0eb0 0x7f7d040b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7d0942e700 1 -- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d040d0eb0 msgr2=0x7f7d040b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7d0942e700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d040d0eb0 0x7f7d040b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7d0942e700 1 -- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d0406a960 msgr2=0x7f7d0415fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7d0942e700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d0406a960 0x7f7d0415fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7d0942e700 1 -- 172.21.15.5:0/4075717087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00009550 con 0x7f7d040cf840 2024-05-08T23:22:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7d0942e700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040b5010 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e810 tx=0x7f7cf400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7cfa7fc700 1 -- 172.21.15.5:0/4075717087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400c070 con 0x7f7d040cf840 2024-05-08T23:22:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7cfa7fc700 1 -- 172.21.15.5:0/4075717087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4013610 con 0x7f7d040cf840 2024-05-08T23:22:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d040b4530 con 0x7f7d040cf840 2024-05-08T23:22:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7cfa7fc700 1 -- 172.21.15.5:0/4075717087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40139c0 con 0x7f7d040cf840 2024-05-08T23:22:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.521+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d040b4af0 con 0x7f7d040cf840 2024-05-08T23:22:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.522+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d04068000 con 0x7f7d040cf840 2024-05-08T23:22:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.523+0000 7f7cfa7fc700 1 -- 172.21.15.5:0/4075717087 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7cf4013b20 con 0x7f7d040cf840 2024-05-08T23:22:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.523+0000 7f7cfa7fc700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ce806ac90 0x7f7ce806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.523+0000 7f7cfa7fc700 1 -- 172.21.15.5:0/4075717087 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7cf4088d60 con 0x7f7d040cf840 2024-05-08T23:22:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.524+0000 7f7d09c2f700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ce806ac90 0x7f7ce806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.524+0000 7f7d09c2f700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ce806ac90 0x7f7ce806d150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009130 tx=0x7f7d00008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.528+0000 7f7cfa7fc700 1 -- 172.21.15.5:0/4075717087 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7cf40542a0 con 0x7f7d040cf840 2024-05-08T23:22:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.839+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f7d040b62a0 con 0x7f7d040cf840 2024-05-08T23:22:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.843+0000 7f7cfa7fc700 1 -- 172.21.15.5:0/4075717087 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7cf40578c0 con 0x7f7d040cf840 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ce806ac90 msgr2=0x7f7ce806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ce806ac90 0x7f7ce806d150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009130 tx=0x7f7d00008030 comp rx=0 tx=0).stop 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 msgr2=0x7f7d040b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040b5010 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e810 tx=0x7f7cf400eb20 comp rx=0 tx=0).stop 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 shutdown_connections 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ce806ac90 0x7f7ce806d150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d040d0eb0 0x7f7d040b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d0406a960 0x7f7d0415fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 --2- 172.21.15.5:0/4075717087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d040cf840 0x7f7d040b5010 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 >> 172.21.15.5:0/4075717087 conn(0x7f7d040d0320 msgr2=0x7f7d0406f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.845+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 shutdown_connections 2024-05-08T23:22:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.846+0000 7f7d0be93700 1 -- 172.21.15.5:0/4075717087 wait complete. 2024-05-08T23:22:03.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.996+0000 7f48184bb700 1 Processor -- start 2024-05-08T23:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.996+0000 7f48184bb700 1 -- start start 2024-05-08T23:22:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.997+0000 7f48184bb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f481006cae0 0x7f48100d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.997+0000 7f48184bb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48100cf990 con 0x7f481006bd80 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.997+0000 7f48184bb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48100d04b0 con 0x7f481015da10 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.997+0000 7f48184bb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481009ab40 con 0x7f481006cae0 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.997+0000 7f4815a56700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f481006bd80 0x7f481006c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34096/0 (socket says 172.21.15.5:34096) 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.997+0000 7f4815a56700 1 -- 172.21.15.5:0/1152572628 learned_addr learned my addr 172.21.15.5:0/1152572628 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4816a58700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f481006cae0 0x7f48100d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634327384 0 0) 0x7f48100cf990 con 0x7f481006bd80 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47fc0035b0 con 0x7f481006bd80 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3560951719 0 0) 0x7f47fc0035b0 con 0x7f481006bd80 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f481006cae0 msgr2=0x7f48100d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f481006cae0 0x7f48100d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 >> v1:172.21.15.49:6789/0 conn(0x7f481015da10 legacy=0x7f481009a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481009ad60 con 0x7f481006bd80 2024-05-08T23:22:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f4800001670 con 0x7f481006bd80 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 >> v1:172.21.15.5:6789/0 conn(0x7f481006bd80 legacy=0x7f481006c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47fc003a30 0x7f47fc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f481006cae0 0x7f47fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47fc004d90 0x7f47fc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481009ab40 con 0x7f47fc004d90 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc00bac0 con 0x7f47fc003a30 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc00bc80 con 0x7f481006cae0 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4816a58700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47fc004d90 0x7f47fc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4816257700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47fc003a30 0x7f47fc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.998+0000 7f4815a56700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f481006cae0 0x7f47fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4816a58700 1 -- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f481006cae0 msgr2=0x7f47fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4816a58700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f481006cae0 0x7f47fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4816a58700 1 -- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47fc003a30 msgr2=0x7f47fc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4816a58700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47fc003a30 0x7f47fc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4816a58700 1 -- 172.21.15.5:0/1152572628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc0035b0 con 0x7f47fc004d90 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4816a58700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47fc004d90 0x7f47fc005210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4810162730 tx=0x7f47fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d540e46a59a7f185 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800eba0 con 0x7f47fc004d90 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480800f040 con 0x7f47fc004d90 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4815255700 1 -- 172.21.15.5:0/1152572628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808014480 con 0x7f47fc004d90 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f48184bb700 1 -- 172.21.15.5:0/1152572628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47fc004d90 msgr2=0x7f47fc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f48184bb700 1 --2- 172.21.15.5:0/1152572628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47fc004d90 0x7f47fc005210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4810162730 tx=0x7f47fc005880 comp rx=0 tx=0).stop 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:03.999+0000 7f4816a58700 1 -- 172.21.15.5:0/1152572628 reap_dead start 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.000+0000 7f48184bb700 1 -- 172.21.15.5:0/1152572628 shutdown_connections 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.000+0000 7f48184bb700 1 -- 172.21.15.5:0/1152572628 >> 172.21.15.5:0/1152572628 conn(0x7f48100b3010 msgr2=0x7f48100b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.000+0000 7f48184bb700 1 -- 172.21.15.5:0/1152572628 shutdown_connections 2024-05-08T23:22:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.000+0000 7f48184bb700 1 -- 172.21.15.5:0/1152572628 wait complete. 2024-05-08T23:22:04.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.001+0000 7f48184bb700 1 Processor -- start 2024-05-08T23:22:04.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.001+0000 7f48184bb700 1 -- start start 2024-05-08T23:22:04.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f48184bb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47fc004d90 0x7f481014d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f48184bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481006ff50 0x7f48100b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f48184bb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48100af1a0 0x7f48100af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f48184bb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100afb60 con 0x7f48100af1a0 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f48184bb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c5360 con 0x7f481006ff50 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f48184bb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c5030 con 0x7f47fc004d90 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816a58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48100af1a0 0x7f48100af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816a58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48100af1a0 0x7f48100af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59058/0 (socket says 172.21.15.5:59058) 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816a58700 1 -- 172.21.15.5:0/236753615 learned_addr learned my addr 172.21.15.5:0/236753615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816257700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47fc004d90 0x7f481014d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4815a56700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481006ff50 0x7f48100b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816a58700 1 -- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47fc004d90 msgr2=0x7f481014d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816a58700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47fc004d90 0x7f481014d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816a58700 1 -- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481006ff50 msgr2=0x7f48100b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816a58700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481006ff50 0x7f48100b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816a58700 1 -- 172.21.15.5:0/236753615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100b65c0 con 0x7f48100af1a0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4816257700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47fc004d90 0x7f481014d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.002+0000 7f4815a56700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481006ff50 0x7f48100b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.003+0000 7f4816a58700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48100af1a0 0x7f48100af620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4808002230 tx=0x7f4808009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.003+0000 7f4806ffd700 1 -- 172.21.15.5:0/236753615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808014d20 con 0x7f48100af1a0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.003+0000 7f4806ffd700 1 -- 172.21.15.5:0/236753615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808009410 con 0x7f48100af1a0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.003+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48100b68a0 con 0x7f48100af1a0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.003+0000 7f4806ffd700 1 -- 172.21.15.5:0/236753615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808014ea0 con 0x7f48100af1a0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.003+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481014dc20 con 0x7f48100af1a0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.004+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810068000 con 0x7f48100af1a0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.004+0000 7f4806ffd700 1 -- 172.21.15.5:0/236753615 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4808023070 con 0x7f48100af1a0 2024-05-08T23:22:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.005+0000 7f4806ffd700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47f406ac90 0x7f47f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.005+0000 7f4806ffd700 1 -- 172.21.15.5:0/236753615 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f480808a5a0 con 0x7f48100af1a0 2024-05-08T23:22:04.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.005+0000 7f4816257700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47f406ac90 0x7f47f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.005+0000 7f4816257700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47f406ac90 0x7f47f406d150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f480c007fd0 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.009+0000 7f4806ffd700 1 -- 172.21.15.5:0/236753615 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4808055c80 con 0x7f48100af1a0 2024-05-08T23:22:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.230+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4810099ee0 con 0x7f48100af1a0 2024-05-08T23:22:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.231+0000 7f4806ffd700 1 -- 172.21.15.5:0/236753615 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12293 (secure 0 0 0) 0x7f48080592a0 con 0x7f48100af1a0 2024-05-08T23:22:04.231 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:04.231 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:22:00.487893+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:22:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47f406ac90 msgr2=0x7f47f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47f406ac90 0x7f47f406d150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f480c007fd0 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-05-08T23:22:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48100af1a0 msgr2=0x7f48100af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48100af1a0 0x7f48100af620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4808002230 tx=0x7f4808009750 comp rx=0 tx=0).stop 2024-05-08T23:22:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 shutdown_connections 2024-05-08T23:22:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47f406ac90 0x7f47f406d150 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48100af1a0 0x7f48100af620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47fc004d90 0x7f481014d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 --2- 172.21.15.5:0/236753615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481006ff50 0x7f48100b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.233+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 >> 172.21.15.5:0/236753615 conn(0x7f48100b3010 msgr2=0x7f48100d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:04.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.234+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 shutdown_connections 2024-05-08T23:22:04.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.234+0000 7f48184bb700 1 -- 172.21.15.5:0/236753615 wait complete. 2024-05-08T23:22:04.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:22:04.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:22:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.389+0000 7f471fe62700 1 Processor -- start 2024-05-08T23:22:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.390+0000 7f471fe62700 1 -- start start 2024-05-08T23:22:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471fe62700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f47180d1e50 0x7f47180ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471fe62700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4718161320 0x7f471806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471fe62700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47180cf560 con 0x7f47180d2230 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471fe62700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47180ce950 con 0x7f47180d1e50 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471fe62700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47180cec30 con 0x7f4718161320 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471d3fd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f47180d2230 0x7f4718160c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34102/0 (socket says 172.21.15.5:34102) 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471d3fd700 1 -- 172.21.15.5:0/3481396756 learned_addr learned my addr 172.21.15.5:0/3481396756 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471e3ff700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4718161320 0x7f471806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471dbfe700 1 --2- 172.21.15.5:0/3481396756 >> v2:172.21.15.49:3300/0 conn(0x7f47180d1e50 0x7f47180ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251087426 0 0) 0x7f47180cf560 con 0x7f47180d2230 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47040035b0 con 0x7f47180d2230 2024-05-08T23:22:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517291868 0 0) 0x7f47040035b0 con 0x7f47180d2230 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4718161320 msgr2=0x7f471806bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4718161320 0x7f471806bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 >> v2:172.21.15.49:3300/0 conn(0x7f47180d1e50 msgr2=0x7f47180ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 --2- 172.21.15.5:0/3481396756 >> v2:172.21.15.49:3300/0 conn(0x7f47180d1e50 0x7f47180ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47180cef10 con 0x7f47180d2230 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f4708001670 con 0x7f47180d2230 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 >> v1:172.21.15.5:6789/0 conn(0x7f47180d2230 legacy=0x7f4718160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704003a00 0x7f4704003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4704004850 0x7f4704004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.391+0000 7f471cbfc700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4718161320 0x7f4704005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180ce950 con 0x7f4704004850 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180cec30 con 0x7f4704003a00 2024-05-08T23:22:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470400ba10 con 0x7f4718161320 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471d3fd700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4704004850 0x7f4704004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471dbfe700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704003a00 0x7f4704003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471e3ff700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4718161320 0x7f4704005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471d3fd700 1 -- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4718161320 msgr2=0x7f4704005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471d3fd700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4718161320 0x7f4704005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471d3fd700 1 -- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704003a00 msgr2=0x7f4704003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471d3fd700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704003a00 0x7f4704003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471d3fd700 1 -- 172.21.15.5:0/3481396756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040035b0 con 0x7f4704004850 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471d3fd700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4704004850 0x7f4704004cd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f470800c320 tx=0x7f4704005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4a01312c032914 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800ce60 con 0x7f4704004850 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4708005960 con 0x7f4704004850 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.392+0000 7f471cbfc700 1 -- 172.21.15.5:0/3481396756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708005cf0 con 0x7f4704004850 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.393+0000 7f471fe62700 1 -- 172.21.15.5:0/3481396756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4704004850 msgr2=0x7f4704004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.393+0000 7f471fe62700 1 --2- 172.21.15.5:0/3481396756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4704004850 0x7f4704004cd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f470800c320 tx=0x7f4704005880 comp rx=0 tx=0).stop 2024-05-08T23:22:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.393+0000 7f471e3ff700 1 -- 172.21.15.5:0/3481396756 reap_dead start 2024-05-08T23:22:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.393+0000 7f471fe62700 1 -- 172.21.15.5:0/3481396756 shutdown_connections 2024-05-08T23:22:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.393+0000 7f471fe62700 1 -- 172.21.15.5:0/3481396756 >> 172.21.15.5:0/3481396756 conn(0x7f471808afe0 msgr2=0x7f47180d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.393+0000 7f471fe62700 1 -- 172.21.15.5:0/3481396756 shutdown_connections 2024-05-08T23:22:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.393+0000 7f471fe62700 1 -- 172.21.15.5:0/3481396756 wait complete. 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.394+0000 7f471fe62700 1 Processor -- start 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471fe62700 1 -- start start 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471fe62700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4704004850 0x7f471815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471fe62700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718160320 0x7f47180cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471fe62700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47181f2ae0 0x7f47180ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471fe62700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471815f280 con 0x7f4718160320 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471fe62700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471815f560 con 0x7f47181f2ae0 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471fe62700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471815f840 con 0x7f4704004850 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471d3fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718160320 0x7f47180cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471d3fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718160320 0x7f47180cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59068/0 (socket says 172.21.15.5:59068) 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.395+0000 7f471d3fd700 1 -- 172.21.15.5:0/3089927499 learned_addr learned my addr 172.21.15.5:0/3089927499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471dbfe700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4704004850 0x7f471815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471e3ff700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47181f2ae0 0x7f47180ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471d3fd700 1 -- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4704004850 msgr2=0x7f471815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471d3fd700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4704004850 0x7f471815eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471d3fd700 1 -- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47181f2ae0 msgr2=0x7f47180ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471d3fd700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47181f2ae0 0x7f47180ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471d3fd700 1 -- 172.21.15.5:0/3089927499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471815fb20 con 0x7f4718160320 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471e3ff700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47181f2ae0 0x7f47180ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471d3fd700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718160320 0x7f47180cc590 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f47040046a0 tx=0x7f4708001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f470e7fc700 1 -- 172.21.15.5:0/3089927499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708001680 con 0x7f4718160320 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f470e7fc700 1 -- 172.21.15.5:0/3089927499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4708004a80 con 0x7f4718160320 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471815fe00 con 0x7f4718160320 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f470e7fc700 1 -- 172.21.15.5:0/3089927499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708004e30 con 0x7f4718160320 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.396+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47181f2f90 con 0x7f4718160320 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.397+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47181f3270 con 0x7f4718160320 2024-05-08T23:22:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.398+0000 7f470e7fc700 1 -- 172.21.15.5:0/3089927499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f47080070d0 con 0x7f4718160320 2024-05-08T23:22:04.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.398+0000 7f470e7fc700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f46fc06ac90 0x7f46fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.398+0000 7f470e7fc700 1 -- 172.21.15.5:0/3089927499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f470808a300 con 0x7f4718160320 2024-05-08T23:22:04.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.398+0000 7f471dbfe700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f46fc06ac90 0x7f46fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.399+0000 7f471dbfe700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f46fc06ac90 0x7f46fc06d150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4714009be0 tx=0x7f4714009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.403+0000 7f470e7fc700 1 -- 172.21.15.5:0/3089927499 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4708055980 con 0x7f4718160320 2024-05-08T23:22:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.624+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4718160170 con 0x7f4718160320 2024-05-08T23:22:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.624+0000 7f470e7fc700 1 -- 172.21.15.5:0/3089927499 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4708007da0 con 0x7f4718160320 2024-05-08T23:22:04.624 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:22:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f46fc06ac90 msgr2=0x7f46fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f46fc06ac90 0x7f46fc06d150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4714009be0 tx=0x7f4714009210 comp rx=0 tx=0).stop 2024-05-08T23:22:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718160320 msgr2=0x7f47180cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718160320 0x7f47180cc590 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f47040046a0 tx=0x7f4708001a00 comp rx=0 tx=0).stop 2024-05-08T23:22:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 shutdown_connections 2024-05-08T23:22:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f46fc06ac90 0x7f46fc06d150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47181f2ae0 0x7f47180ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4704004850 0x7f471815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 --2- 172.21.15.5:0/3089927499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718160320 0x7f47180cc590 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 >> 172.21.15.5:0/3089927499 conn(0x7f471808afe0 msgr2=0x7f471806cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 shutdown_connections 2024-05-08T23:22:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.627+0000 7f471fe62700 1 -- 172.21.15.5:0/3089927499 wait complete. 2024-05-08T23:22:04.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:22:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.777+0000 7fd801d3d700 1 Processor -- start 2024-05-08T23:22:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.777+0000 7fd801d3d700 1 -- start start 2024-05-08T23:22:04.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd801d3d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0aef20 0x7fd7fc15c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd801d3d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd7fc15f230 0x7fd7fc0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd801d3d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7fc0cd3b0 con 0x7fd7fc0cca70 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd801d3d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7fc141df0 con 0x7fd7fc15f230 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd801d3d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc0ccf00 con 0x7fd7fc0aef20 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd7faffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd7fc15f230 0x7fd7fc0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd7fb7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0aef20 0x7fd7fc15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd7fbfff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd7fc0cca70 0x7fd7fc0ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:34540/0 (socket says 172.21.15.5:34540) 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.778+0000 7fd7fbfff700 1 -- 172.21.15.5:0/3995339740 learned_addr learned my addr 172.21.15.5:0/3995339740 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7faffd700 1 -- 172.21.15.5:0/3995339740 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0aef20 msgr2=0x7fd7fc15c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7faffd700 1 --2- 172.21.15.5:0/3995339740 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0aef20 0x7fd7fc15c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7faffd700 1 -- 172.21.15.5:0/3995339740 >> v1:172.21.15.49:6789/0 conn(0x7fd7fc0cca70 legacy=0x7fd7fc0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7faffd700 1 -- 172.21.15.5:0/3995339740 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc0af300 con 0x7fd7fc15f230 2024-05-08T23:22:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7fb7fe700 1 --2- 172.21.15.5:0/3995339740 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0aef20 0x7fd7fc15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7faffd700 1 --2- 172.21.15.5:0/3995339740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 0x7fd7fc0cc530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009840 tx=0x7fd7f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74adead4b972a080 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7fa7fc700 1 -- 172.21.15.5:0/3995339740 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0011830 con 0x7fd7fc15f230 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7fa7fc700 1 -- 172.21.15.5:0/3995339740 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f0011990 con 0x7fd7fc15f230 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.779+0000 7fd7fa7fc700 1 -- 172.21.15.5:0/3995339740 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0011d20 con 0x7fd7fc15f230 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.780+0000 7fd801d3d700 1 -- 172.21.15.5:0/3995339740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 msgr2=0x7fd7fc0cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.780+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3995339740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 0x7fd7fc0cc530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009840 tx=0x7fd7f0009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.780+0000 7fd801d3d700 1 -- 172.21.15.5:0/3995339740 shutdown_connections 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.780+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3995339740 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0aef20 0x7fd7fc15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.780+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3995339740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 0x7fd7fc0cc530 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.780+0000 7fd801d3d700 1 -- 172.21.15.5:0/3995339740 >> 172.21.15.5:0/3995339740 conn(0x7fd7fc0cfce0 msgr2=0x7fd7fc0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.780+0000 7fd801d3d700 1 -- 172.21.15.5:0/3995339740 shutdown_connections 2024-05-08T23:22:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.780+0000 7fd801d3d700 1 -- 172.21.15.5:0/3995339740 wait complete. 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.781+0000 7fd801d3d700 1 Processor -- start 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.781+0000 7fd801d3d700 1 -- start start 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd801d3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc0aef20 0x7fd7fc157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd801d3d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0cca70 0x7fd7fc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd801d3d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 0x7fd7fc1f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd801d3d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc156300 con 0x7fd7fc0aef20 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd801d3d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc156550 con 0x7fd7fc15f230 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd801d3d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc156830 con 0x7fd7fc0cca70 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd7fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc0aef20 0x7fd7fc157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd7fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc0aef20 0x7fd7fc157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59084/0 (socket says 172.21.15.5:59084) 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd7fb7fe700 1 -- 172.21.15.5:0/3170711734 learned_addr learned my addr 172.21.15.5:0/3170711734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd7faffd700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0cca70 0x7fd7fc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.782+0000 7fd7fbfff700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 0x7fd7fc1f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7fb7fe700 1 -- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0cca70 msgr2=0x7fd7fc0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7fb7fe700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0cca70 0x7fd7fc0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7fb7fe700 1 -- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 msgr2=0x7fd7fc1f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7fb7fe700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 0x7fd7fc1f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7fb7fe700 1 -- 172.21.15.5:0/3170711734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec0094a0 con 0x7fd7fc0aef20 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7fb7fe700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc0aef20 0x7fd7fc157fd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00ebe0 tx=0x7fd7ec00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7e3fff700 1 -- 172.21.15.5:0/3170711734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00c850 con 0x7fd7fc0aef20 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7e3fff700 1 -- 172.21.15.5:0/3170711734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec00c9b0 con 0x7fd7fc0aef20 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd7e3fff700 1 -- 172.21.15.5:0/3170711734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00cd40 con 0x7fd7fc0aef20 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f00094a0 con 0x7fd7fc0aef20 2024-05-08T23:22:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.783+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7fc156df0 con 0x7fd7fc0aef20 2024-05-08T23:22:04.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.784+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7fc157160 con 0x7fd7fc0aef20 2024-05-08T23:22:04.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.785+0000 7fd7e3fff700 1 -- 172.21.15.5:0/3170711734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd7ec013470 con 0x7fd7fc0aef20 2024-05-08T23:22:04.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.785+0000 7fd7e3fff700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7e806ac90 0x7fd7e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:04.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.786+0000 7fd7e3fff700 1 -- 172.21.15.5:0/3170711734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd7ec0890a0 con 0x7fd7fc0aef20 2024-05-08T23:22:04.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.786+0000 7fd7faffd700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7e806ac90 0x7fd7e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:04.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.786+0000 7fd7faffd700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7e806ac90 0x7fd7e806d150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0000c00 tx=0x7fd7f001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:04.790+0000 7fd7e3fff700 1 -- 172.21.15.5:0/3170711734 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd7ec0545d0 con 0x7fd7fc0aef20 2024-05-08T23:22:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.013+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd7fc068000 con 0x7fd7fc0aef20 2024-05-08T23:22:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.013+0000 7fd7e3fff700 1 -- 172.21.15.5:0/3170711734 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd7ec00cea0 con 0x7fd7fc0aef20 2024-05-08T23:22:05.013 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:05.013 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:22:05.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:22:05.013 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-08T23:22:05.013 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-08T23:22:05.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:05.014 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:22:05.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7e806ac90 msgr2=0x7fd7e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7e806ac90 0x7fd7e806d150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0000c00 tx=0x7fd7f001c040 comp rx=0 tx=0).stop 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc0aef20 msgr2=0x7fd7fc157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc0aef20 0x7fd7fc157fd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00ebe0 tx=0x7fd7ec00eef0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 shutdown_connections 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7e806ac90 0x7fd7e806d150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7fc15f230 0x7fd7fc1f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc0aef20 0x7fd7fc157fd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 --2- 172.21.15.5:0/3170711734 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7fc0cca70 0x7fd7fc0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 >> 172.21.15.5:0/3170711734 conn(0x7fd7fc0cfce0 msgr2=0x7fd7fc06dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 shutdown_connections 2024-05-08T23:22:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.015+0000 7fd801d3d700 1 -- 172.21.15.5:0/3170711734 wait complete. 2024-05-08T23:22:05.029 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:22:05.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:22:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.170+0000 7fb2fc838700 1 Processor -- start 2024-05-08T23:22:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.170+0000 7fb2fc838700 1 -- start start 2024-05-08T23:22:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.171+0000 7fb2fc838700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b4e0 0x7fb2f80b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.171+0000 7fb2fc838700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2f80d1910 0x7fb2f80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.171+0000 7fb2fc838700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f80cf560 con 0x7fb2f815b8c0 2024-05-08T23:22:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.171+0000 7fb2fc838700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f80cefe0 con 0x7fb2f80d1910 2024-05-08T23:22:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.171+0000 7fb2fc838700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80ccff0 con 0x7fb2f815b4e0 2024-05-08T23:22:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f6d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2f80d1910 0x7fb2f80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f6d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2f80d1910 0x7fb2f80cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59088/0 (socket says 172.21.15.5:59088) 2024-05-08T23:22:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f6d9d700 1 -- 172.21.15.5:0/548195525 learned_addr learned my addr 172.21.15.5:0/548195525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f659c700 1 --2- 172.21.15.5:0/548195525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b4e0 0x7fb2f80b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f6d9d700 1 -- 172.21.15.5:0/548195525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b4e0 msgr2=0x7fb2f80b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f6d9d700 1 --2- 172.21.15.5:0/548195525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b4e0 0x7fb2f80b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f6d9d700 1 -- 172.21.15.5:0/548195525 >> v1:172.21.15.49:6789/0 conn(0x7fb2f815b8c0 legacy=0x7fb2f80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f6d9d700 1 -- 172.21.15.5:0/548195525 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80cd2d0 con 0x7fb2f80d1910 2024-05-08T23:22:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f659c700 1 --2- 172.21.15.5:0/548195525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b4e0 0x7fb2f80b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f6d9d700 1 --2- 172.21.15.5:0/548195525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f80cc2c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009e50 tx=0x7fb2f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f194c37ef50e49a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f559a700 1 -- 172.21.15.5:0/548195525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000e070 con 0x7fb2f80d1910 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f559a700 1 -- 172.21.15.5:0/548195525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f000b9c0 con 0x7fb2f80d1910 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.172+0000 7fb2f559a700 1 -- 172.21.15.5:0/548195525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000bd70 con 0x7fb2f80d1910 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.173+0000 7fb2fc838700 1 -- 172.21.15.5:0/548195525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 msgr2=0x7fb2f80cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.173+0000 7fb2fc838700 1 --2- 172.21.15.5:0/548195525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f80cc2c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009e50 tx=0x7fb2f000b330 comp rx=0 tx=0).stop 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.173+0000 7fb2fc838700 1 -- 172.21.15.5:0/548195525 shutdown_connections 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.173+0000 7fb2fc838700 1 --2- 172.21.15.5:0/548195525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f80cc2c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.173+0000 7fb2fc838700 1 --2- 172.21.15.5:0/548195525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b4e0 0x7fb2f80b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.173+0000 7fb2fc838700 1 -- 172.21.15.5:0/548195525 >> 172.21.15.5:0/548195525 conn(0x7fb2f808afe0 msgr2=0x7fb2f808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.173+0000 7fb2fc838700 1 -- 172.21.15.5:0/548195525 shutdown_connections 2024-05-08T23:22:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.173+0000 7fb2fc838700 1 -- 172.21.15.5:0/548195525 wait complete. 2024-05-08T23:22:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.175+0000 7fb2fc838700 1 Processor -- start 2024-05-08T23:22:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.175+0000 7fb2fc838700 1 -- start start 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2fc838700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2fc838700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f815b4e0 0x7fb2f815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2fc838700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b8c0 0x7fb2f80af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2fc838700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f815c2c0 con 0x7fb2f80d1910 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2fc838700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f815d0c0 con 0x7fb2f815b4e0 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2fc838700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f815c4b0 con 0x7fb2f815b8c0 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f815d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59092/0 (socket says 172.21.15.5:59092) 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f659c700 1 -- 172.21.15.5:0/1679954454 learned_addr learned my addr 172.21.15.5:0/1679954454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f6d9d700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b8c0 0x7fb2f80af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f659c700 1 -- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b8c0 msgr2=0x7fb2f80af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f659c700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b8c0 0x7fb2f80af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f659c700 1 -- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f815b4e0 msgr2=0x7fb2f815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f659c700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f815b4e0 0x7fb2f815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.176+0000 7fb2f659c700 1 -- 172.21.15.5:0/1679954454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0009b30 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.177+0000 7fb2f659c700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f815d550 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800e840 tx=0x7fb2e800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.177+0000 7fb2eeffd700 1 -- 172.21.15.5:0/1679954454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8013610 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.177+0000 7fb2eeffd700 1 -- 172.21.15.5:0/1679954454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8013770 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.177+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f815c790 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.177+0000 7fb2eeffd700 1 -- 172.21.15.5:0/1679954454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8013b20 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.177+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f815cd50 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.178+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f815d3a0 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.178+0000 7fb2eeffd700 1 -- 172.21.15.5:0/1679954454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb2e8013c80 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.179+0000 7fb2eeffd700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb2d806acc0 0x7fb2d806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.179+0000 7fb2eeffd700 1 -- 172.21.15.5:0/1679954454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb2e8088bb0 con 0x7fb2f80d1910 2024-05-08T23:22:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.179+0000 7fb2f5d9b700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb2d806acc0 0x7fb2d806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.180+0000 7fb2f5d9b700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb2d806acc0 0x7fb2d806d180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000f60 tx=0x7fb2e0007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.184+0000 7fb2eeffd700 1 -- 172.21.15.5:0/1679954454 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb2e8054170 con 0x7fb2f80d1910 2024-05-08T23:22:05.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:05.405+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb2f8074bc0 con 0x7fb2f80d1910 2024-05-08T23:22:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.301+0000 7fb2eeffd700 1 -- 172.21.15.5:0/1679954454 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb2e80210b0 con 0x7fb2f80d1910 2024-05-08T23:22:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.306+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb2d806acc0 msgr2=0x7fb2d806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.306+0000 7fb2fc838700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb2d806acc0 0x7fb2d806d180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000f60 tx=0x7fb2e0007a40 comp rx=0 tx=0).stop 2024-05-08T23:22:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.306+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 msgr2=0x7fb2f815d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.306+0000 7fb2fc838700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f815d550 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800e840 tx=0x7fb2e800ec00 comp rx=0 tx=0).stop 2024-05-08T23:22:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.307+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 shutdown_connections 2024-05-08T23:22:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.307+0000 7fb2fc838700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb2d806acc0 0x7fb2d806d180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.307+0000 7fb2fc838700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2f815b8c0 0x7fb2f80af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.307+0000 7fb2fc838700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f80d1910 0x7fb2f815d550 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.307+0000 7fb2fc838700 1 --2- 172.21.15.5:0/1679954454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f815b4e0 0x7fb2f815bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.307+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 >> 172.21.15.5:0/1679954454 conn(0x7fb2f808afe0 msgr2=0x7fb2f80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.308+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 shutdown_connections 2024-05-08T23:22:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.308+0000 7fb2fc838700 1 -- 172.21.15.5:0/1679954454 wait complete. 2024-05-08T23:22:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-08T23:22:06.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:22:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.471+0000 7f699e617700 1 Processor -- start 2024-05-08T23:22:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.471+0000 7f699e617700 1 -- start start 2024-05-08T23:22:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.472+0000 7f699e617700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699815b4e0 0x7f69980b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.472+0000 7f699e617700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f699815b8c0 0x7f69980d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.472+0000 7f699e617700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69980cf560 con 0x7f69980d1740 2024-05-08T23:22:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.472+0000 7f699e617700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69980cefe0 con 0x7f699815b8c0 2024-05-08T23:22:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.472+0000 7f699e617700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980d1bb0 con 0x7f699815b4e0 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f69977fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f699815b8c0 0x7f69980d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f6997fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699815b4e0 0x7f69980b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f699cbb4700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f69980d1740 0x7f69980cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:34552/0 (socket says 172.21.15.5:34552) 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f699cbb4700 1 -- 172.21.15.5:0/2920354068 learned_addr learned my addr 172.21.15.5:0/2920354068 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f6996ffd700 1 -- 172.21.15.5:0/2920354068 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596139361 0 0) 0x7f69980cf560 con 0x7f69980d1740 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f6997fff700 1 -- 172.21.15.5:0/2920354068 >> v2:172.21.15.49:3300/0 conn(0x7f699815b8c0 msgr2=0x7f69980d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f6997fff700 1 --2- 172.21.15.5:0/2920354068 >> v2:172.21.15.49:3300/0 conn(0x7f699815b8c0 0x7f69980d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f6997fff700 1 -- 172.21.15.5:0/2920354068 >> v1:172.21.15.49:6789/0 conn(0x7f69980d1740 legacy=0x7f69980cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.473+0000 7f6997fff700 1 -- 172.21.15.5:0/2920354068 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980ccff0 con 0x7f699815b4e0 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f6997fff700 1 --2- 172.21.15.5:0/2920354068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699815b4e0 0x7f69980b1200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6988009840 tx=0x7f6988009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f0a15408200517 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f6996ffd700 1 -- 172.21.15.5:0/2920354068 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988011830 con 0x7f699815b4e0 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f6996ffd700 1 -- 172.21.15.5:0/2920354068 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6988011990 con 0x7f699815b4e0 2024-05-08T23:22:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f6996ffd700 1 -- 172.21.15.5:0/2920354068 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988011d20 con 0x7f699815b4e0 2024-05-08T23:22:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f699e617700 1 -- 172.21.15.5:0/2920354068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699815b4e0 msgr2=0x7f69980b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f699e617700 1 --2- 172.21.15.5:0/2920354068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699815b4e0 0x7f69980b1200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6988009840 tx=0x7f6988009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f699e617700 1 -- 172.21.15.5:0/2920354068 shutdown_connections 2024-05-08T23:22:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f699e617700 1 --2- 172.21.15.5:0/2920354068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f699815b4e0 0x7f69980b1200 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f699e617700 1 --2- 172.21.15.5:0/2920354068 >> v2:172.21.15.49:3300/0 conn(0x7f699815b8c0 0x7f69980d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f699e617700 1 -- 172.21.15.5:0/2920354068 >> 172.21.15.5:0/2920354068 conn(0x7f699808afe0 msgr2=0x7f699808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f699e617700 1 -- 172.21.15.5:0/2920354068 shutdown_connections 2024-05-08T23:22:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.474+0000 7f699e617700 1 -- 172.21.15.5:0/2920354068 wait complete. 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.476+0000 7f699e617700 1 Processor -- start 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.476+0000 7f699e617700 1 -- start start 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699e617700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69980d1740 0x7f6998146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699e617700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699815b4e0 0x7f69980c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699e617700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699815b8c0 0x7f699806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699e617700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69980c69a0 con 0x7f699815b8c0 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699e617700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69980c7520 con 0x7f699815b4e0 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699e617700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980c6bf0 con 0x7f69980d1740 2024-05-08T23:22:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699815b8c0 0x7f699806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699815b8c0 0x7f699806d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59102/0 (socket says 172.21.15.5:59102) 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 -- 172.21.15.5:0/1924663901 learned_addr learned my addr 172.21.15.5:0/1924663901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f69977fe700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699815b4e0 0x7f69980c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f6997fff700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69980d1740 0x7f6998146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 -- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69980d1740 msgr2=0x7f6998146990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69980d1740 0x7f6998146990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 -- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699815b4e0 msgr2=0x7f69980c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699815b4e0 0x7f69980c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 -- 172.21.15.5:0/1924663901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880094a0 con 0x7f699815b8c0 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f699cbb4700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699815b8c0 0x7f699806d1e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6990002230 tx=0x7f699000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.477+0000 7f6994ff9700 1 -- 172.21.15.5:0/1924663901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699000aac0 con 0x7f699815b8c0 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.478+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69980c6ed0 con 0x7f699815b8c0 2024-05-08T23:22:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.478+0000 7f6994ff9700 1 -- 172.21.15.5:0/1924663901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699000ac40 con 0x7f699815b8c0 2024-05-08T23:22:06.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.478+0000 7f6994ff9700 1 -- 172.21.15.5:0/1924663901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69900070d0 con 0x7f699815b8c0 2024-05-08T23:22:06.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.478+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6998154af0 con 0x7f699815b8c0 2024-05-08T23:22:06.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.479+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6998154d70 con 0x7f699815b8c0 2024-05-08T23:22:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.479+0000 7f6994ff9700 1 -- 172.21.15.5:0/1924663901 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6990007250 con 0x7f699815b8c0 2024-05-08T23:22:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.480+0000 7f6994ff9700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f697806ac90 0x7f697806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.480+0000 7f6994ff9700 1 -- 172.21.15.5:0/1924663901 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6990088c20 con 0x7f699815b8c0 2024-05-08T23:22:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.480+0000 7f6997fff700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f697806ac90 0x7f697806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.480+0000 7f6997fff700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f697806ac90 0x7f697806d150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f69880090b0 tx=0x7f6988008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.485+0000 7f6994ff9700 1 -- 172.21.15.5:0/1924663901 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6990053f60 con 0x7f699815b8c0 2024-05-08T23:22:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:06.701+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f69980c7690 con 0x7f699815b8c0 2024-05-08T23:22:07.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.307+0000 7f6994ff9700 1 -- 172.21.15.5:0/1924663901 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f69900180d0 con 0x7f699815b8c0 2024-05-08T23:22:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.318+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f697806ac90 msgr2=0x7f697806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.318+0000 7f699e617700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f697806ac90 0x7f697806d150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f69880090b0 tx=0x7f6988008030 comp rx=0 tx=0).stop 2024-05-08T23:22:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.318+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699815b8c0 msgr2=0x7f699806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.318+0000 7f699e617700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699815b8c0 0x7f699806d1e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6990002230 tx=0x7f699000cf50 comp rx=0 tx=0).stop 2024-05-08T23:22:07.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.321+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 shutdown_connections 2024-05-08T23:22:07.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.321+0000 7f699e617700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f697806ac90 0x7f697806d150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.321+0000 7f699e617700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f699815b8c0 0x7f699806d1e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.321+0000 7f699e617700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f69980d1740 0x7f6998146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.321+0000 7f699e617700 1 --2- 172.21.15.5:0/1924663901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f699815b4e0 0x7f69980c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.321+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 >> 172.21.15.5:0/1924663901 conn(0x7f699808afe0 msgr2=0x7f6998155880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:07.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.323+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 shutdown_connections 2024-05-08T23:22:07.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.324+0000 7f699e617700 1 -- 172.21.15.5:0/1924663901 wait complete. 2024-05-08T23:22:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-08T23:22:07.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:22:07.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:22:07.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:22:07.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.486+0000 7f3b41e75700 1 Processor -- start 2024-05-08T23:22:07.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.486+0000 7f3b41e75700 1 -- start start 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.487+0000 7f3b41e75700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3b3c15e760 0x7f3b3c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.487+0000 7f3b41e75700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c0cf970 con 0x7f3b3c0d0f10 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.487+0000 7f3b41e75700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c0cfc50 con 0x7f3b3c0cbf50 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.487+0000 7f3b41e75700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3c0cff30 con 0x7f3b3c15e760 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.487+0000 7f3b3bfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3b3c15e760 0x7f3b3c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.487+0000 7f3b3bfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3b3c15e760 0x7f3b3c06a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47084/0 (socket says 172.21.15.5:47084) 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.487+0000 7f3b3affd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3b3c0d0f10 0x7f3b3c15e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43804/0 (socket says 172.21.15.5:43804) 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.487+0000 7f3b3bfff700 1 -- 172.21.15.5:0/3246823335 learned_addr learned my addr 172.21.15.5:0/3246823335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.488+0000 7f3b3bfff700 1 -- 172.21.15.5:0/3246823335 >> v1:172.21.15.49:6789/0 conn(0x7f3b3c0cbf50 legacy=0x7f3b3c161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.488+0000 7f3b3bfff700 1 -- 172.21.15.5:0/3246823335 >> v1:172.21.15.5:6789/0 conn(0x7f3b3c0d0f10 legacy=0x7f3b3c15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.488+0000 7f3b3a7fc700 1 -- 172.21.15.5:0/3246823335 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211808553 0 0) 0x7f3b3c0cfc50 con 0x7f3b3c0cbf50 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.488+0000 7f3b3bfff700 1 -- 172.21.15.5:0/3246823335 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c1617e0 con 0x7f3b3c15e760 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.488+0000 7f3b3bfff700 1 --2- 172.21.15.5:0/3246823335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c15e760 0x7f3b3c06a960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009e50 tx=0x7f3b3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7716198abed217 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.488+0000 7f3b3a7fc700 1 -- 172.21.15.5:0/3246823335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400e070 con 0x7f3b3c15e760 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.488+0000 7f3b3a7fc700 1 -- 172.21.15.5:0/3246823335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3400b9c0 con 0x7f3b3c15e760 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.488+0000 7f3b3a7fc700 1 -- 172.21.15.5:0/3246823335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400bd70 con 0x7f3b3c15e760 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.489+0000 7f3b41e75700 1 -- 172.21.15.5:0/3246823335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c15e760 msgr2=0x7f3b3c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.489+0000 7f3b41e75700 1 --2- 172.21.15.5:0/3246823335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c15e760 0x7f3b3c06a960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009e50 tx=0x7f3b3400b330 comp rx=0 tx=0).stop 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.489+0000 7f3b41e75700 1 -- 172.21.15.5:0/3246823335 shutdown_connections 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.489+0000 7f3b41e75700 1 --2- 172.21.15.5:0/3246823335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c15e760 0x7f3b3c06a960 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.489+0000 7f3b41e75700 1 -- 172.21.15.5:0/3246823335 >> 172.21.15.5:0/3246823335 conn(0x7f3b3c099a50 msgr2=0x7f3b3c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.489+0000 7f3b41e75700 1 -- 172.21.15.5:0/3246823335 shutdown_connections 2024-05-08T23:22:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.489+0000 7f3b41e75700 1 -- 172.21.15.5:0/3246823335 wait complete. 2024-05-08T23:22:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.490+0000 7f3b41e75700 1 Processor -- start 2024-05-08T23:22:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.490+0000 7f3b41e75700 1 -- start start 2024-05-08T23:22:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b41e75700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b3c0cbf50 0x7f3b3c1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b41e75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b3c0d0f10 0x7f3b3c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b41e75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c0b54d0 0x7f3b3c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b41e75700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0afd20 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b41e75700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0b5a30 con 0x7f3b3c0d0f10 2024-05-08T23:22:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b41e75700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0b5d10 con 0x7f3b3c0cbf50 2024-05-08T23:22:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c0b54d0 0x7f3b3c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c0b54d0 0x7f3b3c0cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47086/0 (socket says 172.21.15.5:47086) 2024-05-08T23:22:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b3bfff700 1 -- 172.21.15.5:0/232324942 learned_addr learned my addr 172.21.15.5:0/232324942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b3affd700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b3c0d0f10 0x7f3b3c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.491+0000 7f3b3b7fe700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b3c0cbf50 0x7f3b3c1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b3bfff700 1 -- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b3c0cbf50 msgr2=0x7f3b3c1f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b3bfff700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b3c0cbf50 0x7f3b3c1f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b3bfff700 1 -- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b3c0d0f10 msgr2=0x7f3b3c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b3bfff700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b3c0d0f10 0x7f3b3c0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b3bfff700 1 -- 172.21.15.5:0/232324942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b34009b30 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b3bfff700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c0b54d0 0x7f3b3c0cc7c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b34012000 tx=0x7f3b34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b33fff700 1 -- 172.21.15.5:0/232324942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34004900 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b33fff700 1 -- 172.21.15.5:0/232324942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b340099b0 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c0aff70 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b33fff700 1 -- 172.21.15.5:0/232324942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3401b670 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.492+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c1f3ed0 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.493+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c0b0430 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.494+0000 7f3b33fff700 1 -- 172.21.15.5:0/232324942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3b3401b7d0 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.494+0000 7f3b33fff700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b1c06ac90 0x7f3b1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.494+0000 7f3b33fff700 1 -- 172.21.15.5:0/232324942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3b3408ab70 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.494+0000 7f3b3b7fe700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b1c06ac90 0x7f3b1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.495+0000 7f3b3b7fe700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b1c06ac90 0x7f3b1c06d150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000f60 tx=0x7f3b2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.499+0000 7f3b33fff700 1 -- 172.21.15.5:0/232324942 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3b34055e40 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.764+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b3c074bc0 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.769+0000 7f3b33fff700 1 -- 172.21.15.5:0/232324942 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3b34059460 con 0x7f3b3c0b54d0 2024-05-08T23:22:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b1c06ac90 msgr2=0x7f3b1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b1c06ac90 0x7f3b1c06d150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000f60 tx=0x7f3b2c009040 comp rx=0 tx=0).stop 2024-05-08T23:22:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c0b54d0 msgr2=0x7f3b3c0cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c0b54d0 0x7f3b3c0cc7c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b34012000 tx=0x7f3b34004ad0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 shutdown_connections 2024-05-08T23:22:07.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b1c06ac90 0x7f3b1c06d150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b3c0b54d0 0x7f3b3c0cc7c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b3c0cbf50 0x7f3b3c1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 --2- 172.21.15.5:0/232324942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b3c0d0f10 0x7f3b3c0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.775+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 >> 172.21.15.5:0/232324942 conn(0x7f3b3c099a50 msgr2=0x7f3b3c0add80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:07.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.781+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 shutdown_connections 2024-05-08T23:22:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.782+0000 7f3b41e75700 1 -- 172.21.15.5:0/232324942 wait complete. 2024-05-08T23:22:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 4 and data pool 5 2024-05-08T23:22:07.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:22:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.983+0000 7f5026240700 1 Processor -- start 2024-05-08T23:22:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.983+0000 7f5026240700 1 -- start start 2024-05-08T23:22:07.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f5026240700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f502015a380 0x7f50200cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f5026240700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f5026240700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50200b2b10 con 0x7f50200cf3c0 2024-05-08T23:22:07.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f5026240700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50200ceba0 con 0x7f502015a380 2024-05-08T23:22:07.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f5026240700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200ce340 con 0x7f502006f670 2024-05-08T23:22:07.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f501f7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f502015a380 0x7f50200cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f501ffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f501ffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502006d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:58692/0 (socket says 172.21.15.5:58692) 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f501f7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f502015a380 0x7f50200cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36504/0 (socket says 172.21.15.5:36504) 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f501effd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f50200cf3c0 0x7f502006ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60990/0 (socket says 172.21.15.5:60990) 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.984+0000 7f501ffff700 1 -- 172.21.15.5:0/1463439251 learned_addr learned my addr 172.21.15.5:0/1463439251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501e7fc700 1 -- 172.21.15.5:0/1463439251 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228135913 0 0) 0x7f50200b2b10 con 0x7f50200cf3c0 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501e7fc700 1 -- 172.21.15.5:0/1463439251 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50080035b0 con 0x7f50200cf3c0 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501f7fe700 1 -- 172.21.15.5:0/1463439251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 msgr2=0x7f502006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501f7fe700 1 --2- 172.21.15.5:0/1463439251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502006d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501f7fe700 1 -- 172.21.15.5:0/1463439251 >> v1:172.21.15.49:6789/0 conn(0x7f50200cf3c0 legacy=0x7f502006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501f7fe700 1 -- 172.21.15.5:0/1463439251 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50200ce5e0 con 0x7f502015a380 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501ffff700 1 --2- 172.21.15.5:0/1463439251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502006d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501f7fe700 1 --2- 172.21.15.5:0/1463439251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f502015a380 0x7f50200cbca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5010009840 tx=0x7f5010009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3138bae6423c2e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501e7fc700 1 -- 172.21.15.5:0/1463439251 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010011830 con 0x7f502015a380 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501e7fc700 1 -- 172.21.15.5:0/1463439251 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5010011990 con 0x7f502015a380 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.985+0000 7f501e7fc700 1 -- 172.21.15.5:0/1463439251 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010011d00 con 0x7f502015a380 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.986+0000 7f5026240700 1 -- 172.21.15.5:0/1463439251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f502015a380 msgr2=0x7f50200cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.986+0000 7f5026240700 1 --2- 172.21.15.5:0/1463439251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f502015a380 0x7f50200cbca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5010009840 tx=0x7f5010009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.986+0000 7f5026240700 1 -- 172.21.15.5:0/1463439251 shutdown_connections 2024-05-08T23:22:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.986+0000 7f5026240700 1 --2- 172.21.15.5:0/1463439251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502006d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.986+0000 7f5026240700 1 --2- 172.21.15.5:0/1463439251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f502015a380 0x7f50200cbca0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.986+0000 7f5026240700 1 -- 172.21.15.5:0/1463439251 >> 172.21.15.5:0/1463439251 conn(0x7f50200cfce0 msgr2=0x7f50200d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:07.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.986+0000 7f5026240700 1 -- 172.21.15.5:0/1463439251 shutdown_connections 2024-05-08T23:22:07.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.986+0000 7f5026240700 1 -- 172.21.15.5:0/1463439251 wait complete. 2024-05-08T23:22:07.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.988+0000 7f5026240700 1 Processor -- start 2024-05-08T23:22:07.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.988+0000 7f5026240700 1 -- start start 2024-05-08T23:22:07.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f5026240700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f5026240700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50200cf3c0 0x7f502006a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f5026240700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502015a380 0x7f502006aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f5026240700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502013c050 con 0x7f502015a380 2024-05-08T23:22:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f5026240700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502013c2e0 con 0x7f50200cf3c0 2024-05-08T23:22:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f5026240700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502013c5c0 con 0x7f502006f670 2024-05-08T23:22:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502015a380 0x7f502006aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502015a380 0x7f502006aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47088/0 (socket says 172.21.15.5:47088) 2024-05-08T23:22:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501ffff700 1 -- 172.21.15.5:0/448093686 learned_addr learned my addr 172.21.15.5:0/448093686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501f7fe700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501effd700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50200cf3c0 0x7f502006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501ffff700 1 -- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 msgr2=0x7f502015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501ffff700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501ffff700 1 -- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50200cf3c0 msgr2=0x7f502006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501ffff700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50200cf3c0 0x7f502006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.989+0000 7f501ffff700 1 -- 172.21.15.5:0/448093686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501800bd40 con 0x7f502015a380 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.990+0000 7f501ffff700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502015a380 0x7f502006aea0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f50180072e0 tx=0x7f501800e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.990+0000 7f5007fff700 1 -- 172.21.15.5:0/448093686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018003a20 con 0x7f502015a380 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.990+0000 7f5007fff700 1 -- 172.21.15.5:0/448093686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5018011040 con 0x7f502015a380 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.990+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50100094a0 con 0x7f502015a380 2024-05-08T23:22:07.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.990+0000 7f5007fff700 1 -- 172.21.15.5:0/448093686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018003da0 con 0x7f502015a380 2024-05-08T23:22:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.990+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502013cb80 con 0x7f502015a380 2024-05-08T23:22:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.991+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5020068000 con 0x7f502015a380 2024-05-08T23:22:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.992+0000 7f5007fff700 1 -- 172.21.15.5:0/448093686 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f501800e970 con 0x7f502015a380 2024-05-08T23:22:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.992+0000 7f5007fff700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f500c06ac90 0x7f500c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.992+0000 7f5007fff700 1 -- 172.21.15.5:0/448093686 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f501808b840 con 0x7f502015a380 2024-05-08T23:22:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.992+0000 7f501f7fe700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f500c06ac90 0x7f500c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.993+0000 7f501f7fe700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f500c06ac90 0x7f500c06d150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5010009fd0 tx=0x7f5010008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:07.997+0000 7f5007fff700 1 -- 172.21.15.5:0/448093686 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f50180569a0 con 0x7f502015a380 2024-05-08T23:22:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.215+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f502006fbb0 con 0x7f502015a380 2024-05-08T23:22:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.216+0000 7f5007fff700 1 -- 172.21.15.5:0/448093686 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16018 (secure 0 0 0) 0x7f5018059fc0 con 0x7f502015a380 2024-05-08T23:22:08.216 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:08.217 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:22:07.766190+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:22:05.437680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:22:06.733995+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f500c06ac90 msgr2=0x7f500c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f500c06ac90 0x7f500c06d150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5010009fd0 tx=0x7f5010008030 comp rx=0 tx=0).stop 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502015a380 msgr2=0x7f502006aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502015a380 0x7f502006aea0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f50180072e0 tx=0x7f501800e3a0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 shutdown_connections 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f500c06ac90 0x7f500c06d150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502015a380 0x7f502006aea0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f502006f670 0x7f502015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 --2- 172.21.15.5:0/448093686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50200cf3c0 0x7f502006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 >> 172.21.15.5:0/448093686 conn(0x7f50200cfce0 msgr2=0x7f50200af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 shutdown_connections 2024-05-08T23:22:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.219+0000 7f5026240700 1 -- 172.21.15.5:0/448093686 wait complete. 2024-05-08T23:22:08.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:22:08.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.370+0000 7f39899c4700 1 Processor -- start 2024-05-08T23:22:08.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.370+0000 7f39899c4700 1 -- start start 2024-05-08T23:22:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f39899c4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398415d9c0 0x7f39840cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f39899c4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f398406b810 0x7f39840cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f39899c4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39840d0880 con 0x7f398406bc90 2024-05-08T23:22:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f39899c4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39840d0b60 con 0x7f398406b810 2024-05-08T23:22:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f39899c4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39840ce5b0 con 0x7f398415d9c0 2024-05-08T23:22:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f39827fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f398406b810 0x7f39840cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f39837fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f398406bc90 0x7f39841613b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43810/0 (socket says 172.21.15.5:43810) 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f39837fe700 1 -- 172.21.15.5:0/1251460991 learned_addr learned my addr 172.21.15.5:0/1251460991 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.371+0000 7f3982ffd700 1 --2- 172.21.15.5:0/1251460991 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398415d9c0 0x7f39840cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f3981ffb700 1 -- 172.21.15.5:0/1251460991 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613654131 0 0) 0x7f39840d0880 con 0x7f398406bc90 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f3981ffb700 1 -- 172.21.15.5:0/1251460991 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39700035b0 con 0x7f398406bc90 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f39827fc700 1 -- 172.21.15.5:0/1251460991 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398415d9c0 msgr2=0x7f39840cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f39827fc700 1 --2- 172.21.15.5:0/1251460991 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398415d9c0 0x7f39840cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f39827fc700 1 -- 172.21.15.5:0/1251460991 >> v1:172.21.15.5:6789/0 conn(0x7f398406bc90 legacy=0x7f39841613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f39827fc700 1 -- 172.21.15.5:0/1251460991 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39840ce890 con 0x7f398406b810 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f39827fc700 1 --2- 172.21.15.5:0/1251460991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398406b810 0x7f39840cf090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f396c009820 tx=0x7f396c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9292d1f9800f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f3981ffb700 1 -- 172.21.15.5:0/1251460991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c005e30 con 0x7f398406b810 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f3981ffb700 1 -- 172.21.15.5:0/1251460991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c006030 con 0x7f398406b810 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.372+0000 7f3981ffb700 1 -- 172.21.15.5:0/1251460991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0063a0 con 0x7f398406b810 2024-05-08T23:22:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.373+0000 7f39899c4700 1 -- 172.21.15.5:0/1251460991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398406b810 msgr2=0x7f39840cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.373+0000 7f39899c4700 1 --2- 172.21.15.5:0/1251460991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398406b810 0x7f39840cf090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f396c009820 tx=0x7f396c009b30 comp rx=0 tx=0).stop 2024-05-08T23:22:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.373+0000 7f39899c4700 1 -- 172.21.15.5:0/1251460991 shutdown_connections 2024-05-08T23:22:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.373+0000 7f39899c4700 1 --2- 172.21.15.5:0/1251460991 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398415d9c0 0x7f39840cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.373+0000 7f39899c4700 1 --2- 172.21.15.5:0/1251460991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398406b810 0x7f39840cf090 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.373+0000 7f39899c4700 1 -- 172.21.15.5:0/1251460991 >> 172.21.15.5:0/1251460991 conn(0x7f398408afe0 msgr2=0x7f39840d3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.373+0000 7f39899c4700 1 -- 172.21.15.5:0/1251460991 shutdown_connections 2024-05-08T23:22:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.373+0000 7f39899c4700 1 -- 172.21.15.5:0/1251460991 wait complete. 2024-05-08T23:22:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.374+0000 7f39899c4700 1 Processor -- start 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.374+0000 7f39899c4700 1 -- start start 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39899c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f398406b810 0x7f39840b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39899c4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398406bc90 0x7f39840b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39899c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398415d9c0 0x7f398409d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39899c4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398414dd20 con 0x7f398415d9c0 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39899c4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398409dea0 con 0x7f398406b810 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39899c4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398409e180 con 0x7f398406bc90 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398415d9c0 0x7f398409d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398415d9c0 0x7f398409d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47100/0 (socket says 172.21.15.5:47100) 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39837fe700 1 -- 172.21.15.5:0/1704613244 learned_addr learned my addr 172.21.15.5:0/1704613244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f39827fc700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398406bc90 0x7f39840b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.375+0000 7f3982ffd700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f398406b810 0x7f39840b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39837fe700 1 -- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398406bc90 msgr2=0x7f39840b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39837fe700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398406bc90 0x7f39840b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39837fe700 1 -- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f398406b810 msgr2=0x7f39840b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39837fe700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f398406b810 0x7f39840b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39837fe700 1 -- 172.21.15.5:0/1704613244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c009430 con 0x7f398415d9c0 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39827fc700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398406bc90 0x7f39840b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39837fe700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398415d9c0 0x7f398409d2c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f397c00cb40 tx=0x7f397c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f397bfff700 1 -- 172.21.15.5:0/1704613244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c00aaa0 con 0x7f398415d9c0 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f397bfff700 1 -- 172.21.15.5:0/1704613244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c00ac00 con 0x7f398415d9c0 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398414bb00 con 0x7f398415d9c0 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f397bfff700 1 -- 172.21.15.5:0/1704613244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c012400 con 0x7f398415d9c0 2024-05-08T23:22:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.376+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398414b3e0 con 0x7f398415d9c0 2024-05-08T23:22:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.377+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398414c020 con 0x7f398415d9c0 2024-05-08T23:22:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.378+0000 7f397bfff700 1 -- 172.21.15.5:0/1704613244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f397c01b3b0 con 0x7f398415d9c0 2024-05-08T23:22:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.378+0000 7f397bfff700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f396406ac90 0x7f396406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.378+0000 7f397bfff700 1 -- 172.21.15.5:0/1704613244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f397c089080 con 0x7f398415d9c0 2024-05-08T23:22:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.378+0000 7f3982ffd700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f396406ac90 0x7f396406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.379+0000 7f3982ffd700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f396406ac90 0x7f396406d150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3974009b40 tx=0x7f3974009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:08.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.383+0000 7f397bfff700 1 -- 172.21.15.5:0/1704613244 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f398414c020 con 0x7f398415d9c0 2024-05-08T23:22:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.605+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f398414c020 con 0x7f398415d9c0 2024-05-08T23:22:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.606+0000 7f397bfff700 1 -- 172.21.15.5:0/1704613244 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16018 (secure 0 0 0) 0x7f398414c020 con 0x7f398415d9c0 2024-05-08T23:22:08.606 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:08.607 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:22:07.766190+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:22:05.437680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:22:06.733995+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:22:08.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f396406ac90 msgr2=0x7f396406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f396406ac90 0x7f396406d150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3974009b40 tx=0x7f3974009210 comp rx=0 tx=0).stop 2024-05-08T23:22:08.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398415d9c0 msgr2=0x7f398409d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398415d9c0 0x7f398409d2c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f397c00cb40 tx=0x7f397c00ce50 comp rx=0 tx=0).stop 2024-05-08T23:22:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 shutdown_connections 2024-05-08T23:22:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f396406ac90 0x7f396406d150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f398415d9c0 0x7f398409d2c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f398406b810 0x7f39840b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 --2- 172.21.15.5:0/1704613244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f398406bc90 0x7f39840b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 >> 172.21.15.5:0/1704613244 conn(0x7f398408afe0 msgr2=0x7f39840b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 shutdown_connections 2024-05-08T23:22:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.608+0000 7f39899c4700 1 -- 172.21.15.5:0/1704613244 wait complete. 2024-05-08T23:22:08.624 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:22:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.763+0000 7f1b1968c700 1 Processor -- start 2024-05-08T23:22:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.763+0000 7f1b1968c700 1 -- start start 2024-05-08T23:22:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b1968c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1b140d1e50 0x7f1b140ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b1968c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140d2230 0x7f1b14160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b1968c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b140cf560 con 0x7f1b140b2610 2024-05-08T23:22:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b1968c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1406cae0 con 0x7f1b140d1e50 2024-05-08T23:22:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b1968c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140ce950 con 0x7f1b140d2230 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b137fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1b140b2610 0x7f1b140d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43814/0 (socket says 172.21.15.5:43814) 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b137fe700 1 -- 172.21.15.5:0/3160738741 learned_addr learned my addr 172.21.15.5:0/3160738741 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b12ffd700 1 --2- 172.21.15.5:0/3160738741 >> v2:172.21.15.49:3300/0 conn(0x7f1b140d1e50 0x7f1b140ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.764+0000 7f1b127fc700 1 --2- 172.21.15.5:0/3160738741 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140d2230 0x7f1b14160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b11ffb700 1 -- 172.21.15.5:0/3160738741 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584535052 0 0) 0x7f1b140cf560 con 0x7f1b140b2610 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b11ffb700 1 -- 172.21.15.5:0/3160738741 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1afc0035b0 con 0x7f1b140b2610 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b127fc700 1 -- 172.21.15.5:0/3160738741 >> v2:172.21.15.49:3300/0 conn(0x7f1b140d1e50 msgr2=0x7f1b140ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b127fc700 1 --2- 172.21.15.5:0/3160738741 >> v2:172.21.15.49:3300/0 conn(0x7f1b140d1e50 0x7f1b140ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b127fc700 1 -- 172.21.15.5:0/3160738741 >> v1:172.21.15.5:6789/0 conn(0x7f1b140b2610 legacy=0x7f1b140d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b127fc700 1 -- 172.21.15.5:0/3160738741 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140cec30 con 0x7f1b140d2230 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b127fc700 1 --2- 172.21.15.5:0/3160738741 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140d2230 0x7f1b14160c10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009840 tx=0x7f1b08009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71951f522423c074 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b11ffb700 1 -- 172.21.15.5:0/3160738741 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011830 con 0x7f1b140d2230 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b11ffb700 1 -- 172.21.15.5:0/3160738741 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08011990 con 0x7f1b140d2230 2024-05-08T23:22:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.765+0000 7f1b11ffb700 1 -- 172.21.15.5:0/3160738741 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011d20 con 0x7f1b140d2230 2024-05-08T23:22:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.766+0000 7f1b1968c700 1 -- 172.21.15.5:0/3160738741 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140d2230 msgr2=0x7f1b14160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.766+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3160738741 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140d2230 0x7f1b14160c10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009840 tx=0x7f1b08009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.766+0000 7f1b1968c700 1 -- 172.21.15.5:0/3160738741 shutdown_connections 2024-05-08T23:22:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.766+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3160738741 >> v2:172.21.15.49:3300/0 conn(0x7f1b140d1e50 0x7f1b140ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.766+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3160738741 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140d2230 0x7f1b14160c10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.766+0000 7f1b1968c700 1 -- 172.21.15.5:0/3160738741 >> 172.21.15.5:0/3160738741 conn(0x7f1b1408afe0 msgr2=0x7f1b140d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.766+0000 7f1b1968c700 1 -- 172.21.15.5:0/3160738741 shutdown_connections 2024-05-08T23:22:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.766+0000 7f1b1968c700 1 -- 172.21.15.5:0/3160738741 wait complete. 2024-05-08T23:22:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.767+0000 7f1b1968c700 1 Processor -- start 2024-05-08T23:22:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.768+0000 7f1b1968c700 1 -- start start 2024-05-08T23:22:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.768+0000 7f1b1968c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140b2610 0x7f1b14146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.768+0000 7f1b1968c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b140d1e50 0x7f1b140c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.768+0000 7f1b1968c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b140d2230 0x7f1b1415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.768+0000 7f1b1968c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140cd3e0 con 0x7f1b140d1e50 2024-05-08T23:22:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.768+0000 7f1b1968c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140c9c00 con 0x7f1b140d2230 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.768+0000 7f1b1968c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140c9ee0 con 0x7f1b140b2610 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b140d1e50 0x7f1b140c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b140d1e50 0x7f1b140c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47108/0 (socket says 172.21.15.5:47108) 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 -- 172.21.15.5:0/3174821421 learned_addr learned my addr 172.21.15.5:0/3174821421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b12ffd700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140b2610 0x7f1b14146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b137fe700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b140d2230 0x7f1b1415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 -- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140b2610 msgr2=0x7f1b14146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140b2610 0x7f1b14146650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 -- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b140d2230 msgr2=0x7f1b1415f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b140d2230 0x7f1b1415f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 -- 172.21.15.5:0/3174821421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080094a0 con 0x7f1b140d1e50 2024-05-08T23:22:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.769+0000 7f1b127fc700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b140d1e50 0x7f1b140c7120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1b08014040 tx=0x7f1b08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.770+0000 7f1afb7fe700 1 -- 172.21.15.5:0/3174821421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08004590 con 0x7f1b140d1e50 2024-05-08T23:22:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.770+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1415ed90 con 0x7f1b140d1e50 2024-05-08T23:22:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.770+0000 7f1afb7fe700 1 -- 172.21.15.5:0/3174821421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0801f070 con 0x7f1b140d1e50 2024-05-08T23:22:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.770+0000 7f1afb7fe700 1 -- 172.21.15.5:0/3174821421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b080038a0 con 0x7f1b140d1e50 2024-05-08T23:22:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.770+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b140c7660 con 0x7f1b140d1e50 2024-05-08T23:22:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.771+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1415efd0 con 0x7f1b140d1e50 2024-05-08T23:22:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.772+0000 7f1afb7fe700 1 -- 172.21.15.5:0/3174821421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1b08003a00 con 0x7f1b140d1e50 2024-05-08T23:22:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.773+0000 7f1afb7fe700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1b0006acc0 0x7f1b0006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.773+0000 7f1b12ffd700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1b0006acc0 0x7f1b0006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.773+0000 7f1b12ffd700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1b0006acc0 0x7f1b0006d180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b1415bd00 tx=0x7f1b04009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.774+0000 7f1afb7fe700 1 -- 172.21.15.5:0/3174821421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b08021070 con 0x7f1b140d1e50 2024-05-08T23:22:08.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:08.780+0000 7f1afb7fe700 1 -- 172.21.15.5:0/3174821421 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1b1415efd0 con 0x7f1b140d1e50 2024-05-08T23:22:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.054+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1b1415ef80 con 0x7f1b140d1e50 2024-05-08T23:22:09.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.775+0000 7f1afb7fe700 1 -- 172.21.15.5:0/3174821421 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1b0802b0b0 con 0x7f1b140d1e50 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1b0006acc0 msgr2=0x7f1b0006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1b0006acc0 0x7f1b0006d180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b1415bd00 tx=0x7f1b04009210 comp rx=0 tx=0).stop 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b140d1e50 msgr2=0x7f1b140c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b140d1e50 0x7f1b140c7120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1b08014040 tx=0x7f1b08004750 comp rx=0 tx=0).stop 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 shutdown_connections 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1b0006acc0 0x7f1b0006d180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b140d2230 0x7f1b1415f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1b140b2610 0x7f1b14146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 --2- 172.21.15.5:0/3174821421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b140d1e50 0x7f1b140c7120 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 >> 172.21.15.5:0/3174821421 conn(0x7f1b1408afe0 msgr2=0x7f1b141548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 shutdown_connections 2024-05-08T23:22:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.778+0000 7f1b1968c700 1 -- 172.21.15.5:0/3174821421 wait complete. 2024-05-08T23:22:09.794 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:22:09.794 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:22:09.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.937+0000 7f8554ed5700 1 Processor -- start 2024-05-08T23:22:09.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.937+0000 7f8554ed5700 1 -- start start 2024-05-08T23:22:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.938+0000 7f8554ed5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85500cff30 0x7f85500d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.938+0000 7f8554ed5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500d3d20 0x7f85500ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.938+0000 7f8554ed5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500d0850 con 0x7f85500ce7a0 2024-05-08T23:22:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.938+0000 7f8554ed5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85500cf340 con 0x7f85500cff30 2024-05-08T23:22:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.938+0000 7f8554ed5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500cf610 con 0x7f85500d3d20 2024-05-08T23:22:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854e59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85500cff30 0x7f85500d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854ed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f85500ce7a0 0x7f85500cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43830/0 (socket says 172.21.15.5:43830) 2024-05-08T23:22:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854ed9d700 1 -- 172.21.15.5:0/1834963676 learned_addr learned my addr 172.21.15.5:0/1834963676 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854dd9b700 1 --2- 172.21.15.5:0/1834963676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500d3d20 0x7f85500ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854e59c700 1 -- 172.21.15.5:0/1834963676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500d3d20 msgr2=0x7f85500ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854e59c700 1 --2- 172.21.15.5:0/1834963676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500d3d20 0x7f85500ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854e59c700 1 -- 172.21.15.5:0/1834963676 >> v1:172.21.15.5:6789/0 conn(0x7f85500ce7a0 legacy=0x7f85500cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854e59c700 1 -- 172.21.15.5:0/1834963676 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500d1fc0 con 0x7f85500cff30 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854d59a700 1 -- 172.21.15.5:0/1834963676 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551044330 0 0) 0x7f85500d0850 con 0x7f85500ce7a0 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854e59c700 1 --2- 172.21.15.5:0/1834963676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85500d0310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8540009840 tx=0x7f8540009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdf6ee2ae5495e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854d59a700 1 -- 172.21.15.5:0/1834963676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540011830 con 0x7f85500cff30 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854d59a700 1 -- 172.21.15.5:0/1834963676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540011990 con 0x7f85500cff30 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.939+0000 7f854d59a700 1 -- 172.21.15.5:0/1834963676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540011d20 con 0x7f85500cff30 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.940+0000 7f8554ed5700 1 -- 172.21.15.5:0/1834963676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 msgr2=0x7f85500d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.940+0000 7f8554ed5700 1 --2- 172.21.15.5:0/1834963676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85500d0310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8540009840 tx=0x7f8540009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.940+0000 7f8554ed5700 1 -- 172.21.15.5:0/1834963676 shutdown_connections 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.940+0000 7f8554ed5700 1 --2- 172.21.15.5:0/1834963676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85500d0310 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.940+0000 7f8554ed5700 1 --2- 172.21.15.5:0/1834963676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500d3d20 0x7f85500ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.940+0000 7f8554ed5700 1 -- 172.21.15.5:0/1834963676 >> 172.21.15.5:0/1834963676 conn(0x7f855008afe0 msgr2=0x7f85500d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.940+0000 7f8554ed5700 1 -- 172.21.15.5:0/1834963676 shutdown_connections 2024-05-08T23:22:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.940+0000 7f8554ed5700 1 -- 172.21.15.5:0/1834963676 wait complete. 2024-05-08T23:22:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.941+0000 7f8554ed5700 1 Processor -- start 2024-05-08T23:22:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f8554ed5700 1 -- start start 2024-05-08T23:22:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f8554ed5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500ce7a0 0x7f85500ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f8554ed5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85501f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f8554ed5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85500d3d20 0x7f8550147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f8554ed5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500cc100 con 0x7f85500cff30 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f8554ed5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85501595d0 con 0x7f85500d3d20 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f8554ed5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8550147250 con 0x7f85500ce7a0 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f854dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85501f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f854dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85501f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47124/0 (socket says 172.21.15.5:47124) 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.942+0000 7f854dd9b700 1 -- 172.21.15.5:0/233972470 learned_addr learned my addr 172.21.15.5:0/233972470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854e59c700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500ce7a0 0x7f85500ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854ed9d700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85500d3d20 0x7f8550147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854dd9b700 1 -- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500ce7a0 msgr2=0x7f85500ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854dd9b700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500ce7a0 0x7f85500ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854dd9b700 1 -- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85500d3d20 msgr2=0x7f8550147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854dd9b700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85500d3d20 0x7f8550147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854dd9b700 1 -- 172.21.15.5:0/233972470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400094a0 con 0x7f85500cff30 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854e59c700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500ce7a0 0x7f85500ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f854dd9b700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85501f6070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f853800e810 tx=0x7f853800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:09.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f8546ffd700 1 -- 172.21.15.5:0/233972470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538013610 con 0x7f85500cff30 2024-05-08T23:22:09.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f8546ffd700 1 -- 172.21.15.5:0/233972470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8538013770 con 0x7f85500cff30 2024-05-08T23:22:09.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85500cb580 con 0x7f85500cff30 2024-05-08T23:22:09.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f8546ffd700 1 -- 172.21.15.5:0/233972470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538013b20 con 0x7f85500cff30 2024-05-08T23:22:09.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.943+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85500cbb40 con 0x7f85500cff30 2024-05-08T23:22:09.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.944+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8550145df0 con 0x7f85500cff30 2024-05-08T23:22:09.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.945+0000 7f8546ffd700 1 -- 172.21.15.5:0/233972470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8538013c80 con 0x7f85500cff30 2024-05-08T23:22:09.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.945+0000 7f8546ffd700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f853006ac90 0x7f853006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:09.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.945+0000 7f8546ffd700 1 -- 172.21.15.5:0/233972470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8538088fa0 con 0x7f85500cff30 2024-05-08T23:22:09.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.945+0000 7f854e59c700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f853006ac90 0x7f853006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:09.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.946+0000 7f854e59c700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f853006ac90 0x7f853006d150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f85400090b0 tx=0x7f8540008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:09.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:09.950+0000 7f8546ffd700 1 -- 172.21.15.5:0/233972470 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8538054190 con 0x7f85500cff30 2024-05-08T23:22:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.220+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8550145c10 con 0x7f85500cff30 2024-05-08T23:22:10.287 INFO:tasks.ceph.mds.d.smithi049.stderr:2024-05-08T23:22:10.288+0000 7f9f0b65f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:22:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.779+0000 7f8546ffd700 1 -- 172.21.15.5:0/233972470 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f85380577b0 con 0x7f85500cff30 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f853006ac90 msgr2=0x7f853006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f853006ac90 0x7f853006d150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f85400090b0 tx=0x7f8540008030 comp rx=0 tx=0).stop 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 msgr2=0x7f85501f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85501f6070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f853800e810 tx=0x7f853800ebd0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 shutdown_connections 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f853006ac90 0x7f853006d150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85500d3d20 0x7f8550147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f85500ce7a0 0x7f85500ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 --2- 172.21.15.5:0/233972470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85500cff30 0x7f85501f6070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 >> 172.21.15.5:0/233972470 conn(0x7f855008afe0 msgr2=0x7f85500cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:10.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.782+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 shutdown_connections 2024-05-08T23:22:10.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.783+0000 7f8554ed5700 1 -- 172.21.15.5:0/233972470 wait complete. 2024-05-08T23:22:10.799 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:22:10.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.942+0000 7f2b0d71f700 1 Processor -- start 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.942+0000 7f2b0d71f700 1 -- start start 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.943+0000 7f2b0d71f700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b080d13e0 0x7f2b080d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.943+0000 7f2b0d71f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b080b4430 con 0x7f2b0815d780 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.943+0000 7f2b0d71f700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b080b3820 con 0x7f2b0815db60 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.943+0000 7f2b0d71f700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b080d45c0 con 0x7f2b080d13e0 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.943+0000 7f2b06ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2b0815d780 0x7f2b080b48c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43846/0 (socket says 172.21.15.5:43846) 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.943+0000 7f2b06ffd700 1 -- 172.21.15.5:0/1868534961 learned_addr learned my addr 172.21.15.5:0/1868534961 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.943+0000 7f2b067fc700 1 --2- 172.21.15.5:0/1868534961 >> v2:172.21.15.49:3300/0 conn(0x7f2b080d13e0 0x7f2b080d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818251987 0 0) 0x7f2b080b4430 con 0x7f2b0815d780 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af40035b0 con 0x7f2b0815d780 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909293161 0 0) 0x7f2b080b3820 con 0x7f2b0815db60 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b080b4430 con 0x7f2b0815db60 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1437790686 0 0) 0x7f2af40035b0 con 0x7f2b0815d780 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 >> v2:172.21.15.49:3300/0 conn(0x7f2b080d13e0 msgr2=0x7f2b080d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 --2- 172.21.15.5:0/1868534961 >> v2:172.21.15.49:3300/0 conn(0x7f2b080d13e0 0x7f2b080d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 >> v1:172.21.15.49:6789/0 conn(0x7f2b0815db60 legacy=0x7f2b080ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080b3b00 con 0x7f2b0815d780 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f2af80017f0 con 0x7f2b0815d780 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 >> v1:172.21.15.5:6789/0 conn(0x7f2b0815d780 legacy=0x7f2b080b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003b90 0x7f2af4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4004850 0x7f2af4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4005210 0x7f2af400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080d45c0 con 0x7f2af4003b90 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af400c000 con 0x7f2af4005210 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af400c1c0 con 0x7f2af4004850 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b067fc700 1 --2- 172.21.15.5:0/1868534961 >> v2:172.21.15.49:3300/0 conn(0x7f2b080d13e0 0x7f2b080d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b06ffd700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003b90 0x7f2af4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b077fe700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4005210 0x7f2af400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b067fc700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4004850 0x7f2af4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b06ffd700 1 -- 172.21.15.5:0/1868534961 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4004850 msgr2=0x7f2af4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b06ffd700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4004850 0x7f2af4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b06ffd700 1 -- 172.21.15.5:0/1868534961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4005210 msgr2=0x7f2af400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.944+0000 7f2b06ffd700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4005210 0x7f2af400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b06ffd700 1 -- 172.21.15.5:0/1868534961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40035b0 con 0x7f2af4003b90 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b077fe700 1 -- 172.21.15.5:0/1868534961 reap_dead start 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b06ffd700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003b90 0x7f2af4003f90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2af8001eb0 tx=0x7f2af4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38610414f122aa33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af800cd20 con 0x7f2af4003b90 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af8001c00 con 0x7f2af4003b90 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b05ffb700 1 -- 172.21.15.5:0/1868534961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8007310 con 0x7f2af4003b90 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b0d71f700 1 -- 172.21.15.5:0/1868534961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003b90 msgr2=0x7f2af4003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b0d71f700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003b90 0x7f2af4003f90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2af8001eb0 tx=0x7f2af4005880 comp rx=0 tx=0).stop 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b0d71f700 1 -- 172.21.15.5:0/1868534961 shutdown_connections 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b0d71f700 1 --2- 172.21.15.5:0/1868534961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003b90 0x7f2af4003f90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.945+0000 7f2b0d71f700 1 -- 172.21.15.5:0/1868534961 >> 172.21.15.5:0/1868534961 conn(0x7f2b0808afe0 msgr2=0x7f2b080d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.946+0000 7f2b0d71f700 1 -- 172.21.15.5:0/1868534961 shutdown_connections 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.946+0000 7f2b0d71f700 1 -- 172.21.15.5:0/1868534961 wait complete. 2024-05-08T23:22:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.947+0000 7f2b0d71f700 1 Processor -- start 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.948+0000 7f2b0d71f700 1 -- start start 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.948+0000 7f2b0d71f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4003b90 0x7f2b080b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b0d71f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151990 0x7f2b080b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b0d71f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080ca6d0 0x7f2b080b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b0d71f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0815fe00 con 0x7f2b080ca6d0 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b0d71f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080b8a10 con 0x7f2b08151990 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b0d71f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080b6ad0 con 0x7f2af4003b90 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080ca6d0 0x7f2b080b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080ca6d0 0x7f2b080b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47138/0 (socket says 172.21.15.5:47138) 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 -- 172.21.15.5:0/3662440845 learned_addr learned my addr 172.21.15.5:0/3662440845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b06ffd700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4003b90 0x7f2b080b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b067fc700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151990 0x7f2b080b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 -- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4003b90 msgr2=0x7f2b080b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4003b90 0x7f2b080b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 -- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151990 msgr2=0x7f2b080b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151990 0x7f2b080b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 -- 172.21.15.5:0/3662440845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080caf20 con 0x7f2b080ca6d0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b077fe700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080ca6d0 0x7f2b080b1920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2b0808b800 tx=0x7f2b0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2aff7fe700 1 -- 172.21.15.5:0/3662440845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00003c10 con 0x7f2b080ca6d0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2aff7fe700 1 -- 172.21.15.5:0/3662440845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b00003d90 con 0x7f2b080ca6d0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.949+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b080cb200 con 0x7f2b080ca6d0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.950+0000 7f2aff7fe700 1 -- 172.21.15.5:0/3662440845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00005310 con 0x7f2b080ca6d0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.950+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b080cb7c0 con 0x7f2b080ca6d0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.951+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b080d1a50 con 0x7f2b080ca6d0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.951+0000 7f2aff7fe700 1 -- 172.21.15.5:0/3662440845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2b00005470 con 0x7f2b080ca6d0 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.952+0000 7f2aff7fe700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ae806ac90 0x7f2ae806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.952+0000 7f2aff7fe700 1 -- 172.21.15.5:0/3662440845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b000892b0 con 0x7f2b080ca6d0 2024-05-08T23:22:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.952+0000 7f2b06ffd700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ae806ac90 0x7f2ae806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.952+0000 7f2b06ffd700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ae806ac90 0x7f2ae806d150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000ef0 tx=0x7f2af8007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:10.956+0000 7f2aff7fe700 1 -- 172.21.15.5:0/3662440845 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2b000543f0 con 0x7f2b080ca6d0 2024-05-08T23:22:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.226+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b080d1a50 con 0x7f2b080ca6d0 2024-05-08T23:22:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.226+0000 7f2aff7fe700 1 -- 172.21.15.5:0/3662440845 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f2b00057a10 con 0x7f2b080ca6d0 2024-05-08T23:22:11.227 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:11.227 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:10.777230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4975},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:22:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ae806ac90 msgr2=0x7f2ae806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ae806ac90 0x7f2ae806d150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000ef0 tx=0x7f2af8007f90 comp rx=0 tx=0).stop 2024-05-08T23:22:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080ca6d0 msgr2=0x7f2b080b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080ca6d0 0x7f2b080b1920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2b0808b800 tx=0x7f2b0000ee70 comp rx=0 tx=0).stop 2024-05-08T23:22:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 shutdown_connections 2024-05-08T23:22:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ae806ac90 0x7f2ae806d150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080ca6d0 0x7f2b080b1920 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2af4003b90 0x7f2b080b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 --2- 172.21.15.5:0/3662440845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151990 0x7f2b080b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 >> 172.21.15.5:0/3662440845 conn(0x7f2b0808afe0 msgr2=0x7f2b080d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 shutdown_connections 2024-05-08T23:22:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.229+0000 7f2b0d71f700 1 -- 172.21.15.5:0/3662440845 wait complete. 2024-05-08T23:22:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 15 2024-05-08T23:22:11.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:22:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.380+0000 7f12ffabb700 1 Processor -- start 2024-05-08T23:22:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.380+0000 7f12ffabb700 1 -- start start 2024-05-08T23:22:11.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.381+0000 7f12ffabb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f815ef60 0x7f12f80b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.381+0000 7f12ffabb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f12f80cf000 0x7f12f80cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.381+0000 7f12ffabb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f80cfe50 con 0x7f12f80cf9c0 2024-05-08T23:22:11.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.381+0000 7f12ffabb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f80cd400 con 0x7f12f80cf000 2024-05-08T23:22:11.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.381+0000 7f12ffabb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f809ad40 con 0x7f12f815ef60 2024-05-08T23:22:11.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.381+0000 7f12fd857700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f815ef60 0x7f12f80b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.381+0000 7f12fe058700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f12f80cf9c0 0x7f12f8099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:32772/0 (socket says 172.21.15.5:32772) 2024-05-08T23:22:11.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.381+0000 7f12fd056700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f12f80cf000 0x7f12f80cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fe058700 1 -- 172.21.15.5:0/2883386220 learned_addr learned my addr 172.21.15.5:0/2883386220 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fc855700 1 -- 172.21.15.5:0/2883386220 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340377664 0 0) 0x7f12f80cfe50 con 0x7f12f80cf9c0 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fc855700 1 -- 172.21.15.5:0/2883386220 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e40035b0 con 0x7f12f80cf9c0 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fd857700 1 -- 172.21.15.5:0/2883386220 >> v2:172.21.15.49:3300/0 conn(0x7f12f80cf000 msgr2=0x7f12f80cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fd857700 1 --2- 172.21.15.5:0/2883386220 >> v2:172.21.15.49:3300/0 conn(0x7f12f80cf000 0x7f12f80cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fd857700 1 -- 172.21.15.5:0/2883386220 >> v1:172.21.15.49:6789/0 conn(0x7f12f80cf9c0 legacy=0x7f12f8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fd857700 1 -- 172.21.15.5:0/2883386220 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80ccb60 con 0x7f12f815ef60 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fd857700 1 --2- 172.21.15.5:0/2883386220 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f815ef60 0x7f12f80b2340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009840 tx=0x7f12f4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3a27a9211b9114 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fc855700 1 -- 172.21.15.5:0/2883386220 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4011830 con 0x7f12f815ef60 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fc855700 1 -- 172.21.15.5:0/2883386220 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f4011990 con 0x7f12f815ef60 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.382+0000 7f12fc855700 1 -- 172.21.15.5:0/2883386220 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4011d20 con 0x7f12f815ef60 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.383+0000 7f12ffabb700 1 -- 172.21.15.5:0/2883386220 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f815ef60 msgr2=0x7f12f80b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.383+0000 7f12ffabb700 1 --2- 172.21.15.5:0/2883386220 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f815ef60 0x7f12f80b2340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009840 tx=0x7f12f4009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:11.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.383+0000 7f12ffabb700 1 -- 172.21.15.5:0/2883386220 shutdown_connections 2024-05-08T23:22:11.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.383+0000 7f12ffabb700 1 --2- 172.21.15.5:0/2883386220 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f815ef60 0x7f12f80b2340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.383+0000 7f12ffabb700 1 --2- 172.21.15.5:0/2883386220 >> v2:172.21.15.49:3300/0 conn(0x7f12f80cf000 0x7f12f80cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.383+0000 7f12ffabb700 1 -- 172.21.15.5:0/2883386220 >> 172.21.15.5:0/2883386220 conn(0x7f12f80b2f20 msgr2=0x7f12f80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:11.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.383+0000 7f12ffabb700 1 -- 172.21.15.5:0/2883386220 shutdown_connections 2024-05-08T23:22:11.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.383+0000 7f12ffabb700 1 -- 172.21.15.5:0/2883386220 wait complete. 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.384+0000 7f12ffabb700 1 Processor -- start 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.384+0000 7f12ffabb700 1 -- start start 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12ffabb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f80cf000 0x7f12f8150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12ffabb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f80cf9c0 0x7f12f809bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12ffabb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f815ef60 0x7f12f815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12ffabb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f809f1d0 con 0x7f12f815ef60 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12ffabb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f8151cc0 con 0x7f12f80cf000 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12ffabb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f809e650 con 0x7f12f80cf9c0 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12fe058700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f815ef60 0x7f12f815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12fe058700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f815ef60 0x7f12f815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47146/0 (socket says 172.21.15.5:47146) 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12fe058700 1 -- 172.21.15.5:0/1782859463 learned_addr learned my addr 172.21.15.5:0/1782859463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12fd857700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f80cf000 0x7f12f8150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.385+0000 7f12fd056700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f80cf9c0 0x7f12f809bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12fe058700 1 -- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f80cf9c0 msgr2=0x7f12f809bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12fe058700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f80cf9c0 0x7f12f809bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12fe058700 1 -- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f80cf000 msgr2=0x7f12f8150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12fe058700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f80cf000 0x7f12f8150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12fe058700 1 -- 172.21.15.5:0/1782859463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f40094a0 con 0x7f12f815ef60 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12fe058700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f815ef60 0x7f12f815c220 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f12f000caa0 tx=0x7f12f000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12ee7fc700 1 -- 172.21.15.5:0/1782859463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f000a9e0 con 0x7f12f815ef60 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12ee7fc700 1 -- 172.21.15.5:0/1782859463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f000ab40 con 0x7f12f815ef60 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12ee7fc700 1 -- 172.21.15.5:0/1782859463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0012400 con 0x7f12f815ef60 2024-05-08T23:22:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f809e930 con 0x7f12f815ef60 2024-05-08T23:22:11.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.386+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f809eef0 con 0x7f12f815ef60 2024-05-08T23:22:11.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.387+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f8068000 con 0x7f12f815ef60 2024-05-08T23:22:11.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.388+0000 7f12ee7fc700 1 -- 172.21.15.5:0/1782859463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f12f001b3b0 con 0x7f12f815ef60 2024-05-08T23:22:11.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.388+0000 7f12ee7fc700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12dc06ac90 0x7f12dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.388+0000 7f12ee7fc700 1 -- 172.21.15.5:0/1782859463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12f0088e80 con 0x7f12f815ef60 2024-05-08T23:22:11.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.388+0000 7f12fd857700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12dc06ac90 0x7f12dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.389+0000 7f12fd857700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12dc06ac90 0x7f12dc06d150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12f4000c00 tx=0x7f12f4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.393+0000 7f12ee7fc700 1 -- 172.21.15.5:0/1782859463 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f12f00541b0 con 0x7f12f815ef60 2024-05-08T23:22:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.613+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12f80d05a0 con 0x7f12f815ef60 2024-05-08T23:22:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.615+0000 7f12ee7fc700 1 -- 172.21.15.5:0/1782859463 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16005 (secure 0 0 0) 0x7f12f00577d0 con 0x7f12f815ef60 2024-05-08T23:22:11.615 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:11.615 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:22:08.768203+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:22:05.437680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:22:06.733995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:22:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12dc06ac90 msgr2=0x7f12dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12dc06ac90 0x7f12dc06d150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12f4000c00 tx=0x7f12f4008030 comp rx=0 tx=0).stop 2024-05-08T23:22:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f815ef60 msgr2=0x7f12f815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f815ef60 0x7f12f815c220 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f12f000caa0 tx=0x7f12f000cdb0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 shutdown_connections 2024-05-08T23:22:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12dc06ac90 0x7f12dc06d150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f815ef60 0x7f12f815c220 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12f80cf000 0x7f12f8150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 --2- 172.21.15.5:0/1782859463 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12f80cf9c0 0x7f12f809bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 >> 172.21.15.5:0/1782859463 conn(0x7f12f80b2f20 msgr2=0x7f12f80b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 shutdown_connections 2024-05-08T23:22:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.617+0000 7f12ffabb700 1 -- 172.21.15.5:0/1782859463 wait complete. 2024-05-08T23:22:11.631 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:22:11.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:22:11.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.768+0000 7fd5a338c700 1 Processor -- start 2024-05-08T23:22:11.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.768+0000 7fd5a338c700 1 -- start start 2024-05-08T23:22:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a338c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd59c15b810 0x7fd59c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a338c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd59c09a160 0x7fd59c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a338c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59c09aca0 con 0x7fd59c0d0520 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a338c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c0cd9e0 con 0x7fd59c15b810 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a338c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c0cf220 con 0x7fd59c09a160 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a1128700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd59c15b810 0x7fd59c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a0927700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd59c0d0520 0x7fd59c099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43854/0 (socket says 172.21.15.5:43854) 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a1128700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd59c15b810 0x7fd59c0cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47160/0 (socket says 172.21.15.5:47160) 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a0927700 1 -- 172.21.15.5:0/2221479599 learned_addr learned my addr 172.21.15.5:0/2221479599 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a1929700 1 --2- 172.21.15.5:0/2221479599 >> v2:172.21.15.49:3300/0 conn(0x7fd59c09a160 0x7fd59c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a1128700 1 -- 172.21.15.5:0/2221479599 >> v2:172.21.15.49:3300/0 conn(0x7fd59c09a160 msgr2=0x7fd59c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a1128700 1 --2- 172.21.15.5:0/2221479599 >> v2:172.21.15.49:3300/0 conn(0x7fd59c09a160 0x7fd59c0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd5a1128700 1 -- 172.21.15.5:0/2221479599 >> v1:172.21.15.5:6789/0 conn(0x7fd59c0d0520 legacy=0x7fd59c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.769+0000 7fd593fff700 1 -- 172.21.15.5:0/2221479599 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209804593 0 0) 0x7fd59c09aca0 con 0x7fd59c0d0520 2024-05-08T23:22:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.770+0000 7fd5a1128700 1 -- 172.21.15.5:0/2221479599 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c0cf500 con 0x7fd59c15b810 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.770+0000 7fd5a1128700 1 --2- 172.21.15.5:0/2221479599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c15b810 0x7fd59c0cffe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0716f0 tx=0x7fd598009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f8024c90de8524 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.770+0000 7fd593fff700 1 -- 172.21.15.5:0/2221479599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598011830 con 0x7fd59c15b810 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.770+0000 7fd593fff700 1 -- 172.21.15.5:0/2221479599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598011990 con 0x7fd59c15b810 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.770+0000 7fd593fff700 1 -- 172.21.15.5:0/2221479599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598011d20 con 0x7fd59c15b810 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.770+0000 7fd5a338c700 1 -- 172.21.15.5:0/2221479599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c15b810 msgr2=0x7fd59c0cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.770+0000 7fd5a338c700 1 --2- 172.21.15.5:0/2221479599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c15b810 0x7fd59c0cffe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0716f0 tx=0x7fd598009b20 comp rx=0 tx=0).stop 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.771+0000 7fd5a338c700 1 -- 172.21.15.5:0/2221479599 shutdown_connections 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.771+0000 7fd5a338c700 1 --2- 172.21.15.5:0/2221479599 >> v2:172.21.15.49:3300/0 conn(0x7fd59c09a160 0x7fd59c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.771+0000 7fd5a338c700 1 --2- 172.21.15.5:0/2221479599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c15b810 0x7fd59c0cffe0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.771+0000 7fd5a338c700 1 -- 172.21.15.5:0/2221479599 >> 172.21.15.5:0/2221479599 conn(0x7fd59c0b3010 msgr2=0x7fd59c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.771+0000 7fd5a338c700 1 -- 172.21.15.5:0/2221479599 shutdown_connections 2024-05-08T23:22:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.771+0000 7fd5a338c700 1 -- 172.21.15.5:0/2221479599 wait complete. 2024-05-08T23:22:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.772+0000 7fd5a338c700 1 Processor -- start 2024-05-08T23:22:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.772+0000 7fd5a338c700 1 -- start start 2024-05-08T23:22:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a338c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd59c09a160 0x7fd59c1f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a338c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c0d0520 0x7fd59c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a338c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd59c15b810 0x7fd59c0b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a338c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0b70e0 con 0x7fd59c0d0520 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a338c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c15c5d0 con 0x7fd59c15b810 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a338c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0b6e00 con 0x7fd59c09a160 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c0d0520 0x7fd59c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c0d0520 0x7fd59c15c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47172/0 (socket says 172.21.15.5:47172) 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 -- 172.21.15.5:0/3523757752 learned_addr learned my addr 172.21.15.5:0/3523757752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a1128700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd59c09a160 0x7fd59c1f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a1929700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd59c15b810 0x7fd59c0b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 -- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd59c09a160 msgr2=0x7fd59c1f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd59c09a160 0x7fd59c1f0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 -- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd59c15b810 msgr2=0x7fd59c0b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd59c15b810 0x7fd59c0b6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 -- 172.21.15.5:0/3523757752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980094a0 con 0x7fd59c0d0520 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a1128700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd59c09a160 0x7fd59c1f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.773+0000 7fd5a0927700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c0d0520 0x7fd59c15c9f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00c750 tx=0x7fd58c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.774+0000 7fd591ffb700 1 -- 172.21.15.5:0/3523757752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004c80 con 0x7fd59c0d0520 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.774+0000 7fd591ffb700 1 -- 172.21.15.5:0/3523757752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c004e00 con 0x7fd59c0d0520 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.774+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c15bf10 con 0x7fd59c0d0520 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.774+0000 7fd591ffb700 1 -- 172.21.15.5:0/3523757752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c007310 con 0x7fd59c0d0520 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.774+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c15cf30 con 0x7fd59c0d0520 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.775+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c15c710 con 0x7fd59c0d0520 2024-05-08T23:22:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.775+0000 7fd591ffb700 1 -- 172.21.15.5:0/3523757752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd58c007470 con 0x7fd59c0d0520 2024-05-08T23:22:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.776+0000 7fd591ffb700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd58006ac90 0x7fd58006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.776+0000 7fd591ffb700 1 -- 172.21.15.5:0/3523757752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd58c0890e0 con 0x7fd59c0d0520 2024-05-08T23:22:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.776+0000 7fd5a1128700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd58006ac90 0x7fd58006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.777+0000 7fd5a1128700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd58006ac90 0x7fd58006d150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd5980090b0 tx=0x7fd598008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.780+0000 7fd591ffb700 1 -- 172.21.15.5:0/3523757752 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd59c15c710 con 0x7fd59c0d0520 2024-05-08T23:22:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:11.992+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd59c0cfac0 con 0x7fd58006ac90 2024-05-08T23:22:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.001+0000 7fd591ffb700 1 -- 172.21.15.5:0/3523757752 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285964 (secure 0 0 0) 0x7fd598011b00 con 0x7fd58006ac90 2024-05-08T23:22:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd58006ac90 msgr2=0x7fd58006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd58006ac90 0x7fd58006d150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd5980090b0 tx=0x7fd598008030 comp rx=0 tx=0).stop 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c0d0520 msgr2=0x7fd59c15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c0d0520 0x7fd59c15c9f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00c750 tx=0x7fd58c00cb10 comp rx=0 tx=0).stop 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 shutdown_connections 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd58006ac90 0x7fd58006d150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd59c15b810 0x7fd59c0b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd59c09a160 0x7fd59c1f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 --2- 172.21.15.5:0/3523757752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59c0d0520 0x7fd59c15c9f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 >> 172.21.15.5:0/3523757752 conn(0x7fd59c0b3010 msgr2=0x7fd59c0737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 shutdown_connections 2024-05-08T23:22:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.005+0000 7fd5a338c700 1 -- 172.21.15.5:0/3523757752 wait complete. 2024-05-08T23:22:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:22:12.020 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-05-08T23:22:11.309863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3973120,"data_stored":2242520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.010134"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312535+0000","last_change":"2024-05-08T23:22:07.312535+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312535+0000","last_undegraded":"2024-05-08T23:22:07.312535+0000","last_fullsized":"2024-05-08T23:22:07.312535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311851+0000","last_change":"2024-05-08T23:22:07.311851+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311851+0000","last_undegraded":"2024-05-08T23:22:07.311851+0000","last_fullsized":"2024-05-08T23:22:07.311851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:58.867905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310091+0000","last_change":"2024-05-08T23:22:07.310091+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310091+0000","last_undegraded":"2024-05-08T23:22:07.310091+0000","last_fullsized":"2024-05-08T23:22:07.310091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319925+0000","last_change":"2024-05-08T23:22:07.319925+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319925+0000","last_undegraded":"2024-05-08T23:22:07.319925+0000","last_fullsized":"2024-05-08T23:22:07.319925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310832+0000","last_change":"2024-05-08T23:22:07.310832+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310832+0000","last_undegraded":"2024-05-08T23:22:07.310832+0000","last_fullsized":"2024-05-08T23:22:07.310832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:27.657029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312605+0000","last_change":"2024-05-08T23:22:07.312605+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312605+0000","last_undegraded":"2024-05-08T23:22:07.312605+0000","last_fullsized":"2024-05-08T23:22:07.312605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:10.804990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.320203+0000","last_change":"2024-05-08T23:22:07.320203+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.320203+0000","last_undegraded":"2024-05-08T23:22:07.320203+0000","last_fullsized":"2024-05-08T23:22:07.320203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.320412+0000","last_change":"2024-05-08T23:22:07.320412+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.320412+0000","last_undegraded":"2024-05-08T23:22:07.320412+0000","last_fullsized":"2024-05-08T23:22:07.320412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313870+0000","last_change":"2024-05-08T23:22:07.313870+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313870+0000","last_undegraded":"2024-05-08T23:22:07.313870+0000","last_fullsized":"2024-05-08T23:22:07.313870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318448+0000","last_change":"2024-05-08T23:22:07.318448+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318448+0000","last_undegraded":"2024-05-08T23:22:07.318448+0000","last_fullsized":"2024-05-08T23:22:07.318448+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316240+0000","last_change":"2024-05-08T23:22:07.316240+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316240+0000","last_undegraded":"2024-05-08T23:22:07.316240+0000","last_fullsized":"2024-05-08T23:22:07.316240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309934+0000","last_change":"2024-05-08T23:22:07.309934+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309934+0000","last_undegraded":"2024-05-08T23:22:07.309934+0000","last_fullsized":"2024-05-08T23:22:07.309934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:42.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309244+0000","last_change":"2024-05-08T23:22:07.309244+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309244+0000","last_undegraded":"2024-05-08T23:22:07.309244+0000","last_fullsized":"2024-05-08T23:22:07.309244+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:50.423661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311023+0000","last_change":"2024-05-08T23:22:07.311023+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311023+0000","last_undegraded":"2024-05-08T23:22:07.311023+0000","last_fullsized":"2024-05-08T23:22:07.311023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:37.867491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309447+0000","last_change":"2024-05-08T23:22:07.309447+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309447+0000","last_undegraded":"2024-05-08T23:22:07.309447+0000","last_fullsized":"2024-05-08T23:22:07.309447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318925+0000","last_change":"2024-05-08T23:22:07.318925+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318925+0000","last_undegraded":"2024-05-08T23:22:07.318925+0000","last_fullsized":"2024-05-08T23:22:07.318925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318344+0000","last_change":"2024-05-08T23:22:07.318344+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318344+0000","last_undegraded":"2024-05-08T23:22:07.318344+0000","last_fullsized":"2024-05-08T23:22:07.318344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310922+0000","last_change":"2024-05-08T23:22:07.310922+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310922+0000","last_undegraded":"2024-05-08T23:22:07.310922+0000","last_fullsized":"2024-05-08T23:22:07.310922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:44.317582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310862+0000","last_change":"2024-05-08T23:22:07.310862+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310862+0000","last_undegraded":"2024-05-08T23:22:07.310862+0000","last_fullsized":"2024-05-08T23:22:07.310862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:20.807790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311659+0000","last_change":"2024-05-08T23:22:07.311659+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311659+0000","last_undegraded":"2024-05-08T23:22:07.311659+0000","last_fullsized":"2024-05-08T23:22:07.311659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:30.059653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318563+0000","last_change":"2024-05-08T23:22:07.318563+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318563+0000","last_undegraded":"2024-05-08T23:22:07.318563+0000","last_fullsized":"2024-05-08T23:22:07.318563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318074+0000","last_change":"2024-05-08T23:22:07.318074+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318074+0000","last_undegraded":"2024-05-08T23:22:07.318074+0000","last_fullsized":"2024-05-08T23:22:07.318074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313767+0000","last_change":"2024-05-08T23:22:07.313767+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313767+0000","last_undegraded":"2024-05-08T23:22:07.313767+0000","last_fullsized":"2024-05-08T23:22:07.313767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309544+0000","last_change":"2024-05-08T23:22:07.309544+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309544+0000","last_undegraded":"2024-05-08T23:22:07.309544+0000","last_fullsized":"2024-05-08T23:22:07.309544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317099+0000","last_change":"2024-05-08T23:22:07.317099+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317099+0000","last_undegraded":"2024-05-08T23:22:07.317099+0000","last_fullsized":"2024-05-08T23:22:07.317099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310141+0000","last_change":"2024-05-08T23:22:07.310141+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310141+0000","last_undegraded":"2024-05-08T23:22:07.310141+0000","last_fullsized":"2024-05-08T23:22:07.310141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:49.810291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316305+0000","last_change":"2024-05-08T23:22:07.316305+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316305+0000","last_undegraded":"2024-05-08T23:22:07.316305+0000","last_fullsized":"2024-05-08T23:22:07.316305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.308872+0000","last_change":"2024-05-08T23:22:07.308872+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308872+0000","last_undegraded":"2024-05-08T23:22:07.308872+0000","last_fullsized":"2024-05-08T23:22:07.308872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:09.803894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314342+0000","last_change":"2024-05-08T23:22:07.314342+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314342+0000","last_undegraded":"2024-05-08T23:22:07.314342+0000","last_fullsized":"2024-05-08T23:22:07.314342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319476+0000","last_change":"2024-05-08T23:22:07.319476+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319476+0000","last_undegraded":"2024-05-08T23:22:07.319476+0000","last_fullsized":"2024-05-08T23:22:07.319476+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311923+0000","last_change":"2024-05-08T23:22:07.311923+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311923+0000","last_undegraded":"2024-05-08T23:22:07.311923+0000","last_fullsized":"2024-05-08T23:22:07.311923+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:05.495518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316026+0000","last_change":"2024-05-08T23:22:07.316026+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316026+0000","last_undegraded":"2024-05-08T23:22:07.316026+0000","last_fullsized":"2024-05-08T23:22:07.316026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315002+0000","last_change":"2024-05-08T23:22:07.315002+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315002+0000","last_undegraded":"2024-05-08T23:22:07.315002+0000","last_fullsized":"2024-05-08T23:22:07.315002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313496+0000","last_change":"2024-05-08T23:22:07.313496+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313496+0000","last_undegraded":"2024-05-08T23:22:07.313496+0000","last_fullsized":"2024-05-08T23:22:07.313496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311991+0000","last_change":"2024-05-08T23:22:07.311991+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311991+0000","last_undegraded":"2024-05-08T23:22:07.311991+0000","last_fullsized":"2024-05-08T23:22:07.311991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.314508+0000","last_change":"2024-05-08T23:22:07.314508+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314508+0000","last_undegraded":"2024-05-08T23:22:07.314508+0000","last_fullsized":"2024-05-08T23:22:07.314508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:38.809279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317989+0000","last_change":"2024-05-08T23:22:07.317989+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317989+0000","last_undegraded":"2024-05-08T23:22:07.317989+0000","last_fullsized":"2024-05-08T23:22:07.317989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328595+0000","last_change":"2024-05-08T23:22:07.323498+0000","last_active":"2024-05-08T23:22:07.328595+0000","last_peered":"2024-05-08T23:22:07.328595+0000","last_clean":"2024-05-08T23:22:07.328595+0000","last_became_active":"2024-05-08T23:22:07.323230+0000","last_became_peered":"2024-05-08T23:22:07.323230+0000","last_unstale":"2024-05-08T23:22:07.328595+0000","last_undegraded":"2024-05-08T23:22:07.328595+0000","last_fullsized":"2024-05-08T23:22:07.328595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:50.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315788+0000","last_change":"2024-05-08T23:22:07.315788+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315788+0000","last_undegraded":"2024-05-08T23:22:07.315788+0000","last_fullsized":"2024-05-08T23:22:07.315788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308127+0000","last_change":"2024-05-08T23:22:07.308127+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308127+0000","last_undegraded":"2024-05-08T23:22:07.308127+0000","last_fullsized":"2024-05-08T23:22:07.308127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:40.830327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315411+0000","last_change":"2024-05-08T23:22:07.315411+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315411+0000","last_undegraded":"2024-05-08T23:22:07.315411+0000","last_fullsized":"2024-05-08T23:22:07.315411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327698+0000","last_change":"2024-05-08T23:22:07.324042+0000","last_active":"2024-05-08T23:22:07.327698+0000","last_peered":"2024-05-08T23:22:07.327698+0000","last_clean":"2024-05-08T23:22:07.327698+0000","last_became_active":"2024-05-08T23:22:07.323538+0000","last_became_peered":"2024-05-08T23:22:07.323538+0000","last_unstale":"2024-05-08T23:22:07.327698+0000","last_undegraded":"2024-05-08T23:22:07.327698+0000","last_fullsized":"2024-05-08T23:22:07.327698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:21.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.308908+0000","last_change":"2024-05-08T23:22:07.308908+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308908+0000","last_undegraded":"2024-05-08T23:22:07.308908+0000","last_fullsized":"2024-05-08T23:22:07.308908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327756+0000","last_change":"2024-05-08T23:22:07.323560+0000","last_active":"2024-05-08T23:22:07.327756+0000","last_peered":"2024-05-08T23:22:07.327756+0000","last_clean":"2024-05-08T23:22:07.327756+0000","last_became_active":"2024-05-08T23:22:07.323249+0000","last_became_peered":"2024-05-08T23:22:07.323249+0000","last_unstale":"2024-05-08T23:22:07.327756+0000","last_undegraded":"2024-05-08T23:22:07.327756+0000","last_fullsized":"2024-05-08T23:22:07.327756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:04.715359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317833+0000","last_change":"2024-05-08T23:22:07.317833+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317833+0000","last_undegraded":"2024-05-08T23:22:07.317833+0000","last_fullsized":"2024-05-08T23:22:07.317833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327285+0000","last_change":"2024-05-08T23:22:07.320297+0000","last_active":"2024-05-08T23:22:07.327285+0000","last_peered":"2024-05-08T23:22:07.327285+0000","last_clean":"2024-05-08T23:22:07.327285+0000","last_became_active":"2024-05-08T23:22:07.320014+0000","last_became_peered":"2024-05-08T23:22:07.320014+0000","last_unstale":"2024-05-08T23:22:07.327285+0000","last_undegraded":"2024-05-08T23:22:07.327285+0000","last_fullsized":"2024-05-08T23:22:07.327285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:51.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318358+0000","last_change":"2024-05-08T23:22:07.318358+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318358+0000","last_undegraded":"2024-05-08T23:22:07.318358+0000","last_fullsized":"2024-05-08T23:22:07.318358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328104+0000","last_change":"2024-05-08T23:22:07.326385+0000","last_active":"2024-05-08T23:22:07.328104+0000","last_peered":"2024-05-08T23:22:07.328104+0000","last_clean":"2024-05-08T23:22:07.328104+0000","last_became_active":"2024-05-08T23:22:07.326047+0000","last_became_peered":"2024-05-08T23:22:07.326047+0000","last_unstale":"2024-05-08T23:22:07.328104+0000","last_undegraded":"2024-05-08T23:22:07.328104+0000","last_fullsized":"2024-05-08T23:22:07.328104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:46.848050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309020+0000","last_change":"2024-05-08T23:22:07.309020+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309020+0000","last_undegraded":"2024-05-08T23:22:07.309020+0000","last_fullsized":"2024-05-08T23:22:07.309020+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:41.329227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328884+0000","last_change":"2024-05-08T23:22:07.324876+0000","last_active":"2024-05-08T23:22:07.328884+0000","last_peered":"2024-05-08T23:22:07.328884+0000","last_clean":"2024-05-08T23:22:07.328884+0000","last_became_active":"2024-05-08T23:22:07.324583+0000","last_became_peered":"2024-05-08T23:22:07.324583+0000","last_unstale":"2024-05-08T23:22:07.328884+0000","last_undegraded":"2024-05-08T23:22:07.328884+0000","last_fullsized":"2024-05-08T23:22:07.328884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:00.240669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310539+0000","last_change":"2024-05-08T23:22:07.310539+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310539+0000","last_undegraded":"2024-05-08T23:22:07.310539+0000","last_fullsized":"2024-05-08T23:22:07.310539+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:08.578978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313769+0000","last_change":"2024-05-08T23:22:07.313769+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313769+0000","last_undegraded":"2024-05-08T23:22:07.313769+0000","last_fullsized":"2024-05-08T23:22:07.313769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:37.291699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312087+0000","last_change":"2024-05-08T23:22:07.312087+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312087+0000","last_undegraded":"2024-05-08T23:22:07.312087+0000","last_fullsized":"2024-05-08T23:22:07.312087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:33.565554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308718+0000","last_change":"2024-05-08T23:22:07.308718+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308718+0000","last_undegraded":"2024-05-08T23:22:07.308718+0000","last_fullsized":"2024-05-08T23:22:07.308718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:40.686112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309329+0000","last_change":"2024-05-08T23:22:07.309329+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309329+0000","last_undegraded":"2024-05-08T23:22:07.309329+0000","last_fullsized":"2024-05-08T23:22:07.309329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328840+0000","last_change":"2024-05-08T23:22:07.324702+0000","last_active":"2024-05-08T23:22:07.328840+0000","last_peered":"2024-05-08T23:22:07.328840+0000","last_clean":"2024-05-08T23:22:07.328840+0000","last_became_active":"2024-05-08T23:22:07.323911+0000","last_became_peered":"2024-05-08T23:22:07.323911+0000","last_unstale":"2024-05-08T23:22:07.328840+0000","last_undegraded":"2024-05-08T23:22:07.328840+0000","last_fullsized":"2024-05-08T23:22:07.328840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:28.458137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317680+0000","last_change":"2024-05-08T23:22:07.317680+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317680+0000","last_undegraded":"2024-05-08T23:22:07.317680+0000","last_fullsized":"2024-05-08T23:22:07.317680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327463+0000","last_change":"2024-05-08T23:22:07.319729+0000","last_active":"2024-05-08T23:22:07.327463+0000","last_peered":"2024-05-08T23:22:07.327463+0000","last_clean":"2024-05-08T23:22:07.327463+0000","last_became_active":"2024-05-08T23:22:07.319420+0000","last_became_peered":"2024-05-08T23:22:07.319420+0000","last_unstale":"2024-05-08T23:22:07.327463+0000","last_undegraded":"2024-05-08T23:22:07.327463+0000","last_fullsized":"2024-05-08T23:22:07.327463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:13.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309969+0000","last_change":"2024-05-08T23:22:07.309969+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309969+0000","last_undegraded":"2024-05-08T23:22:07.309969+0000","last_fullsized":"2024-05-08T23:22:07.309969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:10.382589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327526+0000","last_change":"2024-05-08T23:22:07.327526+0000","last_active":"2024-05-08T23:22:07.327526+0000","last_peered":"2024-05-08T23:22:07.327526+0000","last_clean":"2024-05-08T23:22:07.327526+0000","last_became_active":"2024-05-08T23:22:07.327240+0000","last_became_peered":"2024-05-08T23:22:07.327240+0000","last_unstale":"2024-05-08T23:22:07.327526+0000","last_undegraded":"2024-05-08T23:22:07.327526+0000","last_fullsized":"2024-05-08T23:22:07.327526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317928+0000","last_change":"2024-05-08T23:22:07.317928+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317928+0000","last_undegraded":"2024-05-08T23:22:07.317928+0000","last_fullsized":"2024-05-08T23:22:07.317928+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328772+0000","last_change":"2024-05-08T23:22:07.324290+0000","last_active":"2024-05-08T23:22:07.328772+0000","last_peered":"2024-05-08T23:22:07.328772+0000","last_clean":"2024-05-08T23:22:07.328772+0000","last_became_active":"2024-05-08T23:22:07.323929+0000","last_became_peered":"2024-05-08T23:22:07.323929+0000","last_unstale":"2024-05-08T23:22:07.328772+0000","last_undegraded":"2024-05-08T23:22:07.328772+0000","last_fullsized":"2024-05-08T23:22:07.328772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:17.483564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312557+0000","last_change":"2024-05-08T23:22:07.312557+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312557+0000","last_undegraded":"2024-05-08T23:22:07.312557+0000","last_fullsized":"2024-05-08T23:22:07.312557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:38.296628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328617+0000","last_change":"2024-05-08T23:22:07.323609+0000","last_active":"2024-05-08T23:22:07.328617+0000","last_peered":"2024-05-08T23:22:07.328617+0000","last_clean":"2024-05-08T23:22:07.328617+0000","last_became_active":"2024-05-08T23:22:07.323344+0000","last_became_peered":"2024-05-08T23:22:07.323344+0000","last_unstale":"2024-05-08T23:22:07.328617+0000","last_undegraded":"2024-05-08T23:22:07.328617+0000","last_fullsized":"2024-05-08T23:22:07.328617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:41.903099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313857+0000","last_change":"2024-05-08T23:22:07.313857+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313857+0000","last_undegraded":"2024-05-08T23:22:07.313857+0000","last_fullsized":"2024-05-08T23:22:07.313857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.326254+0000","last_change":"2024-05-08T23:22:07.324272+0000","last_active":"2024-05-08T23:22:07.326254+0000","last_peered":"2024-05-08T23:22:07.326254+0000","last_clean":"2024-05-08T23:22:07.326254+0000","last_became_active":"2024-05-08T23:22:07.324005+0000","last_became_peered":"2024-05-08T23:22:07.324005+0000","last_unstale":"2024-05-08T23:22:07.326254+0000","last_undegraded":"2024-05-08T23:22:07.326254+0000","last_fullsized":"2024-05-08T23:22:07.326254+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310323+0000","last_change":"2024-05-08T23:22:07.310323+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310323+0000","last_undegraded":"2024-05-08T23:22:07.310323+0000","last_fullsized":"2024-05-08T23:22:07.310323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:31.969535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327612+0000","last_change":"2024-05-08T23:22:07.320014+0000","last_active":"2024-05-08T23:22:07.327612+0000","last_peered":"2024-05-08T23:22:07.327612+0000","last_clean":"2024-05-08T23:22:07.327612+0000","last_became_active":"2024-05-08T23:22:07.319073+0000","last_became_peered":"2024-05-08T23:22:07.319073+0000","last_unstale":"2024-05-08T23:22:07.327612+0000","last_undegraded":"2024-05-08T23:22:07.327612+0000","last_fullsized":"2024-05-08T23:22:07.327612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.661583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319446+0000","last_change":"2024-05-08T23:22:07.319446+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319446+0000","last_undegraded":"2024-05-08T23:22:07.319446+0000","last_fullsized":"2024-05-08T23:22:07.319446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328064+0000","last_change":"2024-05-08T23:22:07.325295+0000","last_active":"2024-05-08T23:22:07.328064+0000","last_peered":"2024-05-08T23:22:07.328064+0000","last_clean":"2024-05-08T23:22:07.328064+0000","last_became_active":"2024-05-08T23:22:07.324997+0000","last_became_peered":"2024-05-08T23:22:07.324997+0000","last_unstale":"2024-05-08T23:22:07.328064+0000","last_undegraded":"2024-05-08T23:22:07.328064+0000","last_fullsized":"2024-05-08T23:22:07.328064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:56.035940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309539+0000","last_change":"2024-05-08T23:22:07.309539+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309539+0000","last_undegraded":"2024-05-08T23:22:07.309539+0000","last_fullsized":"2024-05-08T23:22:07.309539+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327723+0000","last_change":"2024-05-08T23:22:07.323547+0000","last_active":"2024-05-08T23:22:07.327723+0000","last_peered":"2024-05-08T23:22:07.327723+0000","last_clean":"2024-05-08T23:22:07.327723+0000","last_became_active":"2024-05-08T23:22:07.323266+0000","last_became_peered":"2024-05-08T23:22:07.323266+0000","last_unstale":"2024-05-08T23:22:07.327723+0000","last_undegraded":"2024-05-08T23:22:07.327723+0000","last_fullsized":"2024-05-08T23:22:07.327723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309807+0000","last_change":"2024-05-08T23:22:07.309807+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309807+0000","last_undegraded":"2024-05-08T23:22:07.309807+0000","last_fullsized":"2024-05-08T23:22:07.309807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:47.852979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313480+0000","last_change":"2024-05-08T23:22:07.313480+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313480+0000","last_undegraded":"2024-05-08T23:22:07.313480+0000","last_fullsized":"2024-05-08T23:22:07.313480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:29.054724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309930+0000","last_change":"2024-05-08T23:22:07.309930+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309930+0000","last_undegraded":"2024-05-08T23:22:07.309930+0000","last_fullsized":"2024-05-08T23:22:07.309930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:15.611951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313374+0000","last_change":"2024-05-08T23:22:07.313374+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313374+0000","last_undegraded":"2024-05-08T23:22:07.313374+0000","last_fullsized":"2024-05-08T23:22:07.313374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309225+0000","last_change":"2024-05-08T23:22:07.309225+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309225+0000","last_undegraded":"2024-05-08T23:22:07.309225+0000","last_fullsized":"2024-05-08T23:22:07.309225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:45.661099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327791+0000","last_change":"2024-05-08T23:22:07.323427+0000","last_active":"2024-05-08T23:22:07.327791+0000","last_peered":"2024-05-08T23:22:07.327791+0000","last_clean":"2024-05-08T23:22:07.327791+0000","last_became_active":"2024-05-08T23:22:07.323163+0000","last_became_peered":"2024-05-08T23:22:07.323163+0000","last_unstale":"2024-05-08T23:22:07.327791+0000","last_undegraded":"2024-05-08T23:22:07.327791+0000","last_fullsized":"2024-05-08T23:22:07.327791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319698+0000","last_change":"2024-05-08T23:22:07.319698+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319698+0000","last_undegraded":"2024-05-08T23:22:07.319698+0000","last_fullsized":"2024-05-08T23:22:07.319698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308777+0000","last_change":"2024-05-08T23:22:07.308777+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308777+0000","last_undegraded":"2024-05-08T23:22:07.308777+0000","last_fullsized":"2024-05-08T23:22:07.308777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:04.490589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309538+0000","last_change":"2024-05-08T23:22:07.309538+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309538+0000","last_undegraded":"2024-05-08T23:22:07.309538+0000","last_fullsized":"2024-05-08T23:22:07.309538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:02.041531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.316598+0000","last_change":"2024-05-08T23:22:07.316598+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316598+0000","last_undegraded":"2024-05-08T23:22:07.316598+0000","last_fullsized":"2024-05-08T23:22:07.316598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310978+0000","last_change":"2024-05-08T23:22:07.310978+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310978+0000","last_undegraded":"2024-05-08T23:22:07.310978+0000","last_fullsized":"2024-05-08T23:22:07.310978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313873+0000","last_change":"2024-05-08T23:22:07.313873+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313873+0000","last_undegraded":"2024-05-08T23:22:07.313873+0000","last_fullsized":"2024-05-08T23:22:07.313873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313198+0000","last_change":"2024-05-08T23:22:07.313198+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313198+0000","last_undegraded":"2024-05-08T23:22:07.313198+0000","last_fullsized":"2024-05-08T23:22:07.313198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327107+0000","last_change":"2024-05-08T23:22:07.327107+0000","last_active":"2024-05-08T23:22:07.327107+0000","last_peered":"2024-05-08T23:22:07.327107+0000","last_clean":"2024-05-08T23:22:07.327107+0000","last_became_active":"2024-05-08T23:22:07.326542+0000","last_became_peered":"2024-05-08T23:22:07.326542+0000","last_unstale":"2024-05-08T23:22:07.327107+0000","last_undegraded":"2024-05-08T23:22:07.327107+0000","last_fullsized":"2024-05-08T23:22:07.327107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311985+0000","last_change":"2024-05-08T23:22:07.311985+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311985+0000","last_undegraded":"2024-05-08T23:22:07.311985+0000","last_fullsized":"2024-05-08T23:22:07.311985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.320882+0000","last_change":"2024-05-08T23:22:07.319178+0000","last_active":"2024-05-08T23:22:07.320882+0000","last_peered":"2024-05-08T23:22:07.320882+0000","last_clean":"2024-05-08T23:22:07.320882+0000","last_became_active":"2024-05-08T23:22:07.318864+0000","last_became_peered":"2024-05-08T23:22:07.318864+0000","last_unstale":"2024-05-08T23:22:07.320882+0000","last_undegraded":"2024-05-08T23:22:07.320882+0000","last_fullsized":"2024-05-08T23:22:07.320882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314790+0000","last_change":"2024-05-08T23:22:07.314790+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314790+0000","last_undegraded":"2024-05-08T23:22:07.314790+0000","last_fullsized":"2024-05-08T23:22:07.314790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.316706+0000","last_change":"2024-05-08T23:22:07.316706+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316706+0000","last_undegraded":"2024-05-08T23:22:07.316706+0000","last_fullsized":"2024-05-08T23:22:07.316706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309830+0000","last_change":"2024-05-08T23:22:07.309830+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309830+0000","last_undegraded":"2024-05-08T23:22:07.309830+0000","last_fullsized":"2024-05-08T23:22:07.309830+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:55.199701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328702+0000","last_change":"2024-05-08T23:22:07.326530+0000","last_active":"2024-05-08T23:22:07.328702+0000","last_peered":"2024-05-08T23:22:07.328702+0000","last_clean":"2024-05-08T23:22:07.328702+0000","last_became_active":"2024-05-08T23:22:07.326233+0000","last_became_peered":"2024-05-08T23:22:07.326233+0000","last_unstale":"2024-05-08T23:22:07.328702+0000","last_undegraded":"2024-05-08T23:22:07.328702+0000","last_fullsized":"2024-05-08T23:22:07.328702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:49.418732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314416+0000","last_change":"2024-05-08T23:22:07.314416+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314416+0000","last_undegraded":"2024-05-08T23:22:07.314416+0000","last_fullsized":"2024-05-08T23:22:07.314416+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328841+0000","last_change":"2024-05-08T23:22:07.325642+0000","last_active":"2024-05-08T23:22:07.328841+0000","last_peered":"2024-05-08T23:22:07.328841+0000","last_clean":"2024-05-08T23:22:07.328841+0000","last_became_active":"2024-05-08T23:22:07.325384+0000","last_became_peered":"2024-05-08T23:22:07.325384+0000","last_unstale":"2024-05-08T23:22:07.328841+0000","last_undegraded":"2024-05-08T23:22:07.328841+0000","last_fullsized":"2024-05-08T23:22:07.328841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:26.489244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315394+0000","last_change":"2024-05-08T23:22:07.315394+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315394+0000","last_undegraded":"2024-05-08T23:22:07.315394+0000","last_fullsized":"2024-05-08T23:22:07.315394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313997+0000","last_change":"2024-05-08T23:22:07.313997+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313997+0000","last_undegraded":"2024-05-08T23:22:07.313997+0000","last_fullsized":"2024-05-08T23:22:07.313997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T23:22:06.306594+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:22:06.306594+0000","last_peered":"2024-05-08T23:22:06.306594+0000","last_clean":"2024-05-08T23:22:06.306594+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:22:06.306594+0000","last_undegraded":"2024-05-08T23:22:06.306594+0000","last_fullsized":"2024-05-08T23:22:06.306594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.625},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47199999999999998}]},{"osd":1,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.501}]},{"osd":4,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.70099999999999996}]},{"osd":5,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.751},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.37}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.439},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50800000000000001}]},{"osd":4,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.626}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.9889999999999999,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.35399999999999998}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.7869999999999999,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.0519999999999996,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.379},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.6440000000000001,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.442}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.9139999999999997,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.63}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":7.1120000000000001,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.501},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.6970000000000001,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.62}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.734,"5min":7.734,"15min":7.734},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0220000000000002,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.54000000000000004}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.443}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74299999999999999}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.73299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.503}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":11.964,"5min":11.964,"15min":11.964},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.627}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.621},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.55000000000000004}]},{"osd":1,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.51900000000000002}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.0389999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.127000000000001,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.73899999999999999}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.79800000000000004}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.85199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.6710000000000003,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":4.71,"5min":4.71,"15min":4.71},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":8.6880000000000006,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.8940000000000001,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8559999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.627}]},{"osd":3,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.8770000000000007,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.5},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.427,"5min":10.427,"15min":10.427},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:22:12.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:22:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.199+0000 7f90118f0700 1 Processor -- start 2024-05-08T23:22:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.199+0000 7f90118f0700 1 -- start start 2024-05-08T23:22:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f90118f0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f900c15d780 0x7f900c0b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f90118f0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f900c0d22f0 0x7f900c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f90118f0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c0b4430 con 0x7f900c0d13e0 2024-05-08T23:22:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f90118f0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0d45c0 con 0x7f900c0d22f0 2024-05-08T23:22:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f90118f0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0b3820 con 0x7f900c15d780 2024-05-08T23:22:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f900b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f900c0d22f0 0x7f900c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f900b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f900c0d22f0 0x7f900c0ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47186/0 (socket says 172.21.15.5:47186) 2024-05-08T23:22:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f900a7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f900c0d13e0 0x7f900c0d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43864/0 (socket says 172.21.15.5:43864) 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.200+0000 7f900b7fe700 1 -- 172.21.15.5:0/687019836 learned_addr learned my addr 172.21.15.5:0/687019836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f900affd700 1 --2- 172.21.15.5:0/687019836 >> v2:172.21.15.49:3300/0 conn(0x7f900c15d780 0x7f900c0b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f900b7fe700 1 -- 172.21.15.5:0/687019836 >> v2:172.21.15.49:3300/0 conn(0x7f900c15d780 msgr2=0x7f900c0b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f900b7fe700 1 --2- 172.21.15.5:0/687019836 >> v2:172.21.15.49:3300/0 conn(0x7f900c15d780 0x7f900c0b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f900b7fe700 1 -- 172.21.15.5:0/687019836 >> v1:172.21.15.5:6789/0 conn(0x7f900c0d13e0 legacy=0x7f900c0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f900b7fe700 1 -- 172.21.15.5:0/687019836 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0b3b00 con 0x7f900c0d22f0 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f9009ffb700 1 -- 172.21.15.5:0/687019836 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801910700 0 0) 0x7f900c0b4430 con 0x7f900c0d13e0 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f900b7fe700 1 --2- 172.21.15.5:0/687019836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c0d22f0 0x7f900c0ce3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f900c06b640 tx=0x7f900400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc379112d776f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f9009ffb700 1 -- 172.21.15.5:0/687019836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400e070 con 0x7f900c0d22f0 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f9009ffb700 1 -- 172.21.15.5:0/687019836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400b9c0 con 0x7f900c0d22f0 2024-05-08T23:22:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.201+0000 7f9009ffb700 1 -- 172.21.15.5:0/687019836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400bd70 con 0x7f900c0d22f0 2024-05-08T23:22:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.202+0000 7f90118f0700 1 -- 172.21.15.5:0/687019836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c0d22f0 msgr2=0x7f900c0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.202+0000 7f90118f0700 1 --2- 172.21.15.5:0/687019836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c0d22f0 0x7f900c0ce3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f900c06b640 tx=0x7f900400b330 comp rx=0 tx=0).stop 2024-05-08T23:22:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.202+0000 7f90118f0700 1 -- 172.21.15.5:0/687019836 shutdown_connections 2024-05-08T23:22:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.202+0000 7f90118f0700 1 --2- 172.21.15.5:0/687019836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c0d22f0 0x7f900c0ce3f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.202+0000 7f90118f0700 1 --2- 172.21.15.5:0/687019836 >> v2:172.21.15.49:3300/0 conn(0x7f900c15d780 0x7f900c0b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.202+0000 7f90118f0700 1 -- 172.21.15.5:0/687019836 >> 172.21.15.5:0/687019836 conn(0x7f900c08afe0 msgr2=0x7f900c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.202+0000 7f90118f0700 1 -- 172.21.15.5:0/687019836 shutdown_connections 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.202+0000 7f90118f0700 1 -- 172.21.15.5:0/687019836 wait complete. 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.203+0000 7f90118f0700 1 Processor -- start 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.203+0000 7f90118f0700 1 -- start start 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f90118f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0d13e0 0x7f900c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f90118f0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f900c0d22f0 0x7f900c0b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f90118f0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c15d780 0x7f900c151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f90118f0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0b7850 con 0x7f900c15d780 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f90118f0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c6ee0 con 0x7f900c0d13e0 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f90118f0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0b72a0 con 0x7f900c0d22f0 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c15d780 0x7f900c151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c15d780 0x7f900c151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47192/0 (socket says 172.21.15.5:47192) 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900b7fe700 1 -- 172.21.15.5:0/3466769271 learned_addr learned my addr 172.21.15.5:0/3466769271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900affd700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0d13e0 0x7f900c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900a7fc700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f900c0d22f0 0x7f900c0b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900b7fe700 1 -- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f900c0d22f0 msgr2=0x7f900c0b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900b7fe700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f900c0d22f0 0x7f900c0b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900b7fe700 1 -- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0d13e0 msgr2=0x7f900c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900b7fe700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0d13e0 0x7f900c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900b7fe700 1 -- 172.21.15.5:0/3466769271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004009b30 con 0x7f900c15d780 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900a7fc700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f900c0d22f0 0x7f900c0b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.204+0000 7f900affd700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0d13e0 0x7f900c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.205+0000 7f900b7fe700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c15d780 0x7f900c151190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f900c08cf30 tx=0x7f9004004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.205+0000 7f8ff3fff700 1 -- 172.21.15.5:0/3466769271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040092e0 con 0x7f900c15d780 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.205+0000 7f8ff3fff700 1 -- 172.21.15.5:0/3466769271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900401c440 con 0x7f900c15d780 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.205+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c0b7570 con 0x7f900c15d780 2024-05-08T23:22:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.205+0000 7f8ff3fff700 1 -- 172.21.15.5:0/3466769271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900401c810 con 0x7f900c15d780 2024-05-08T23:22:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.205+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c0cbc40 con 0x7f900c15d780 2024-05-08T23:22:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.206+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c0d1a50 con 0x7f900c15d780 2024-05-08T23:22:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.206+0000 7f8ff3fff700 1 -- 172.21.15.5:0/3466769271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f900401c970 con 0x7f900c15d780 2024-05-08T23:22:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.207+0000 7f8ff3fff700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ff806ac90 0x7f8ff806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.207+0000 7f8ff3fff700 1 -- 172.21.15.5:0/3466769271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f900408aa20 con 0x7f900c15d780 2024-05-08T23:22:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.207+0000 7f900affd700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ff806ac90 0x7f8ff806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.208+0000 7f900affd700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ff806ac90 0x7f8ff806d150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc006fd0 tx=0x7f8ffc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.211+0000 7f8ff3fff700 1 -- 172.21.15.5:0/3466769271 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9004055b60 con 0x7f900c15d780 2024-05-08T23:22:12.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.423+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f900c0b34d0 con 0x7f8ff806ac90 2024-05-08T23:22:12.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.432+0000 7f8ff3fff700 1 -- 172.21.15.5:0/3466769271 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285964 (secure 0 0 0) 0x7f8ffc0096a0 con 0x7f8ff806ac90 2024-05-08T23:22:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ff806ac90 msgr2=0x7f8ff806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ff806ac90 0x7f8ff806d150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc006fd0 tx=0x7f8ffc008040 comp rx=0 tx=0).stop 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c15d780 msgr2=0x7f900c151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c15d780 0x7f900c151190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f900c08cf30 tx=0x7f9004004ad0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 shutdown_connections 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8ff806ac90 0x7f8ff806d150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f900c15d780 0x7f900c151190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0d13e0 0x7f900c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 --2- 172.21.15.5:0/3466769271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f900c0d22f0 0x7f900c0b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 >> 172.21.15.5:0/3466769271 conn(0x7f900c08afe0 msgr2=0x7f900c0d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:12.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 shutdown_connections 2024-05-08T23:22:12.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.435+0000 7f90118f0700 1 -- 172.21.15.5:0/3466769271 wait complete. 2024-05-08T23:22:12.439 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:22:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-05-08T23:22:11.309863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3973120,"data_stored":2242520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.010134"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312535+0000","last_change":"2024-05-08T23:22:07.312535+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312535+0000","last_undegraded":"2024-05-08T23:22:07.312535+0000","last_fullsized":"2024-05-08T23:22:07.312535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311851+0000","last_change":"2024-05-08T23:22:07.311851+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311851+0000","last_undegraded":"2024-05-08T23:22:07.311851+0000","last_fullsized":"2024-05-08T23:22:07.311851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:58.867905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310091+0000","last_change":"2024-05-08T23:22:07.310091+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310091+0000","last_undegraded":"2024-05-08T23:22:07.310091+0000","last_fullsized":"2024-05-08T23:22:07.310091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319925+0000","last_change":"2024-05-08T23:22:07.319925+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319925+0000","last_undegraded":"2024-05-08T23:22:07.319925+0000","last_fullsized":"2024-05-08T23:22:07.319925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310832+0000","last_change":"2024-05-08T23:22:07.310832+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310832+0000","last_undegraded":"2024-05-08T23:22:07.310832+0000","last_fullsized":"2024-05-08T23:22:07.310832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:27.657029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312605+0000","last_change":"2024-05-08T23:22:07.312605+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312605+0000","last_undegraded":"2024-05-08T23:22:07.312605+0000","last_fullsized":"2024-05-08T23:22:07.312605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:10.804990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.320203+0000","last_change":"2024-05-08T23:22:07.320203+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.320203+0000","last_undegraded":"2024-05-08T23:22:07.320203+0000","last_fullsized":"2024-05-08T23:22:07.320203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.320412+0000","last_change":"2024-05-08T23:22:07.320412+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.320412+0000","last_undegraded":"2024-05-08T23:22:07.320412+0000","last_fullsized":"2024-05-08T23:22:07.320412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313870+0000","last_change":"2024-05-08T23:22:07.313870+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313870+0000","last_undegraded":"2024-05-08T23:22:07.313870+0000","last_fullsized":"2024-05-08T23:22:07.313870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318448+0000","last_change":"2024-05-08T23:22:07.318448+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318448+0000","last_undegraded":"2024-05-08T23:22:07.318448+0000","last_fullsized":"2024-05-08T23:22:07.318448+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316240+0000","last_change":"2024-05-08T23:22:07.316240+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316240+0000","last_undegraded":"2024-05-08T23:22:07.316240+0000","last_fullsized":"2024-05-08T23:22:07.316240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309934+0000","last_change":"2024-05-08T23:22:07.309934+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309934+0000","last_undegraded":"2024-05-08T23:22:07.309934+0000","last_fullsized":"2024-05-08T23:22:07.309934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:42.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309244+0000","last_change":"2024-05-08T23:22:07.309244+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309244+0000","last_undegraded":"2024-05-08T23:22:07.309244+0000","last_fullsized":"2024-05-08T23:22:07.309244+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:50.423661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311023+0000","last_change":"2024-05-08T23:22:07.311023+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311023+0000","last_undegraded":"2024-05-08T23:22:07.311023+0000","last_fullsized":"2024-05-08T23:22:07.311023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:37.867491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309447+0000","last_change":"2024-05-08T23:22:07.309447+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309447+0000","last_undegraded":"2024-05-08T23:22:07.309447+0000","last_fullsized":"2024-05-08T23:22:07.309447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318925+0000","last_change":"2024-05-08T23:22:07.318925+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318925+0000","last_undegraded":"2024-05-08T23:22:07.318925+0000","last_fullsized":"2024-05-08T23:22:07.318925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318344+0000","last_change":"2024-05-08T23:22:07.318344+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318344+0000","last_undegraded":"2024-05-08T23:22:07.318344+0000","last_fullsized":"2024-05-08T23:22:07.318344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310922+0000","last_change":"2024-05-08T23:22:07.310922+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310922+0000","last_undegraded":"2024-05-08T23:22:07.310922+0000","last_fullsized":"2024-05-08T23:22:07.310922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:44.317582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310862+0000","last_change":"2024-05-08T23:22:07.310862+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310862+0000","last_undegraded":"2024-05-08T23:22:07.310862+0000","last_fullsized":"2024-05-08T23:22:07.310862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:20.807790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311659+0000","last_change":"2024-05-08T23:22:07.311659+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311659+0000","last_undegraded":"2024-05-08T23:22:07.311659+0000","last_fullsized":"2024-05-08T23:22:07.311659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:30.059653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318563+0000","last_change":"2024-05-08T23:22:07.318563+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318563+0000","last_undegraded":"2024-05-08T23:22:07.318563+0000","last_fullsized":"2024-05-08T23:22:07.318563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318074+0000","last_change":"2024-05-08T23:22:07.318074+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318074+0000","last_undegraded":"2024-05-08T23:22:07.318074+0000","last_fullsized":"2024-05-08T23:22:07.318074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313767+0000","last_change":"2024-05-08T23:22:07.313767+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313767+0000","last_undegraded":"2024-05-08T23:22:07.313767+0000","last_fullsized":"2024-05-08T23:22:07.313767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309544+0000","last_change":"2024-05-08T23:22:07.309544+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309544+0000","last_undegraded":"2024-05-08T23:22:07.309544+0000","last_fullsized":"2024-05-08T23:22:07.309544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317099+0000","last_change":"2024-05-08T23:22:07.317099+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317099+0000","last_undegraded":"2024-05-08T23:22:07.317099+0000","last_fullsized":"2024-05-08T23:22:07.317099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310141+0000","last_change":"2024-05-08T23:22:07.310141+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310141+0000","last_undegraded":"2024-05-08T23:22:07.310141+0000","last_fullsized":"2024-05-08T23:22:07.310141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:49.810291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316305+0000","last_change":"2024-05-08T23:22:07.316305+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316305+0000","last_undegraded":"2024-05-08T23:22:07.316305+0000","last_fullsized":"2024-05-08T23:22:07.316305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.308872+0000","last_change":"2024-05-08T23:22:07.308872+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308872+0000","last_undegraded":"2024-05-08T23:22:07.308872+0000","last_fullsized":"2024-05-08T23:22:07.308872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:09.803894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314342+0000","last_change":"2024-05-08T23:22:07.314342+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314342+0000","last_undegraded":"2024-05-08T23:22:07.314342+0000","last_fullsized":"2024-05-08T23:22:07.314342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319476+0000","last_change":"2024-05-08T23:22:07.319476+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319476+0000","last_undegraded":"2024-05-08T23:22:07.319476+0000","last_fullsized":"2024-05-08T23:22:07.319476+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311923+0000","last_change":"2024-05-08T23:22:07.311923+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311923+0000","last_undegraded":"2024-05-08T23:22:07.311923+0000","last_fullsized":"2024-05-08T23:22:07.311923+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:05.495518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316026+0000","last_change":"2024-05-08T23:22:07.316026+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316026+0000","last_undegraded":"2024-05-08T23:22:07.316026+0000","last_fullsized":"2024-05-08T23:22:07.316026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315002+0000","last_change":"2024-05-08T23:22:07.315002+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315002+0000","last_undegraded":"2024-05-08T23:22:07.315002+0000","last_fullsized":"2024-05-08T23:22:07.315002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313496+0000","last_change":"2024-05-08T23:22:07.313496+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313496+0000","last_undegraded":"2024-05-08T23:22:07.313496+0000","last_fullsized":"2024-05-08T23:22:07.313496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311991+0000","last_change":"2024-05-08T23:22:07.311991+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311991+0000","last_undegraded":"2024-05-08T23:22:07.311991+0000","last_fullsized":"2024-05-08T23:22:07.311991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.314508+0000","last_change":"2024-05-08T23:22:07.314508+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314508+0000","last_undegraded":"2024-05-08T23:22:07.314508+0000","last_fullsized":"2024-05-08T23:22:07.314508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:38.809279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317989+0000","last_change":"2024-05-08T23:22:07.317989+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317989+0000","last_undegraded":"2024-05-08T23:22:07.317989+0000","last_fullsized":"2024-05-08T23:22:07.317989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328595+0000","last_change":"2024-05-08T23:22:07.323498+0000","last_active":"2024-05-08T23:22:07.328595+0000","last_peered":"2024-05-08T23:22:07.328595+0000","last_clean":"2024-05-08T23:22:07.328595+0000","last_became_active":"2024-05-08T23:22:07.323230+0000","last_became_peered":"2024-05-08T23:22:07.323230+0000","last_unstale":"2024-05-08T23:22:07.328595+0000","last_undegraded":"2024-05-08T23:22:07.328595+0000","last_fullsized":"2024-05-08T23:22:07.328595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:50.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315788+0000","last_change":"2024-05-08T23:22:07.315788+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315788+0000","last_undegraded":"2024-05-08T23:22:07.315788+0000","last_fullsized":"2024-05-08T23:22:07.315788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308127+0000","last_change":"2024-05-08T23:22:07.308127+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308127+0000","last_undegraded":"2024-05-08T23:22:07.308127+0000","last_fullsized":"2024-05-08T23:22:07.308127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:40.830327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315411+0000","last_change":"2024-05-08T23:22:07.315411+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315411+0000","last_undegraded":"2024-05-08T23:22:07.315411+0000","last_fullsized":"2024-05-08T23:22:07.315411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327698+0000","last_change":"2024-05-08T23:22:07.324042+0000","last_active":"2024-05-08T23:22:07.327698+0000","last_peered":"2024-05-08T23:22:07.327698+0000","last_clean":"2024-05-08T23:22:07.327698+0000","last_became_active":"2024-05-08T23:22:07.323538+0000","last_became_peered":"2024-05-08T23:22:07.323538+0000","last_unstale":"2024-05-08T23:22:07.327698+0000","last_undegraded":"2024-05-08T23:22:07.327698+0000","last_fullsized":"2024-05-08T23:22:07.327698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:21.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.308908+0000","last_change":"2024-05-08T23:22:07.308908+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308908+0000","last_undegraded":"2024-05-08T23:22:07.308908+0000","last_fullsized":"2024-05-08T23:22:07.308908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327756+0000","last_change":"2024-05-08T23:22:07.323560+0000","last_active":"2024-05-08T23:22:07.327756+0000","last_peered":"2024-05-08T23:22:07.327756+0000","last_clean":"2024-05-08T23:22:07.327756+0000","last_became_active":"2024-05-08T23:22:07.323249+0000","last_became_peered":"2024-05-08T23:22:07.323249+0000","last_unstale":"2024-05-08T23:22:07.327756+0000","last_undegraded":"2024-05-08T23:22:07.327756+0000","last_fullsized":"2024-05-08T23:22:07.327756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:04.715359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317833+0000","last_change":"2024-05-08T23:22:07.317833+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317833+0000","last_undegraded":"2024-05-08T23:22:07.317833+0000","last_fullsized":"2024-05-08T23:22:07.317833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327285+0000","last_change":"2024-05-08T23:22:07.320297+0000","last_active":"2024-05-08T23:22:07.327285+0000","last_peered":"2024-05-08T23:22:07.327285+0000","last_clean":"2024-05-08T23:22:07.327285+0000","last_became_active":"2024-05-08T23:22:07.320014+0000","last_became_peered":"2024-05-08T23:22:07.320014+0000","last_unstale":"2024-05-08T23:22:07.327285+0000","last_undegraded":"2024-05-08T23:22:07.327285+0000","last_fullsized":"2024-05-08T23:22:07.327285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:51.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318358+0000","last_change":"2024-05-08T23:22:07.318358+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318358+0000","last_undegraded":"2024-05-08T23:22:07.318358+0000","last_fullsized":"2024-05-08T23:22:07.318358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328104+0000","last_change":"2024-05-08T23:22:07.326385+0000","last_active":"2024-05-08T23:22:07.328104+0000","last_peered":"2024-05-08T23:22:07.328104+0000","last_clean":"2024-05-08T23:22:07.328104+0000","last_became_active":"2024-05-08T23:22:07.326047+0000","last_became_peered":"2024-05-08T23:22:07.326047+0000","last_unstale":"2024-05-08T23:22:07.328104+0000","last_undegraded":"2024-05-08T23:22:07.328104+0000","last_fullsized":"2024-05-08T23:22:07.328104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:46.848050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309020+0000","last_change":"2024-05-08T23:22:07.309020+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309020+0000","last_undegraded":"2024-05-08T23:22:07.309020+0000","last_fullsized":"2024-05-08T23:22:07.309020+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:41.329227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328884+0000","last_change":"2024-05-08T23:22:07.324876+0000","last_active":"2024-05-08T23:22:07.328884+0000","last_peered":"2024-05-08T23:22:07.328884+0000","last_clean":"2024-05-08T23:22:07.328884+0000","last_became_active":"2024-05-08T23:22:07.324583+0000","last_became_peered":"2024-05-08T23:22:07.324583+0000","last_unstale":"2024-05-08T23:22:07.328884+0000","last_undegraded":"2024-05-08T23:22:07.328884+0000","last_fullsized":"2024-05-08T23:22:07.328884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:00.240669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310539+0000","last_change":"2024-05-08T23:22:07.310539+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310539+0000","last_undegraded":"2024-05-08T23:22:07.310539+0000","last_fullsized":"2024-05-08T23:22:07.310539+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:08.578978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313769+0000","last_change":"2024-05-08T23:22:07.313769+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313769+0000","last_undegraded":"2024-05-08T23:22:07.313769+0000","last_fullsized":"2024-05-08T23:22:07.313769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:37.291699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312087+0000","last_change":"2024-05-08T23:22:07.312087+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312087+0000","last_undegraded":"2024-05-08T23:22:07.312087+0000","last_fullsized":"2024-05-08T23:22:07.312087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:33.565554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308718+0000","last_change":"2024-05-08T23:22:07.308718+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308718+0000","last_undegraded":"2024-05-08T23:22:07.308718+0000","last_fullsized":"2024-05-08T23:22:07.308718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:40.686112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309329+0000","last_change":"2024-05-08T23:22:07.309329+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309329+0000","last_undegraded":"2024-05-08T23:22:07.309329+0000","last_fullsized":"2024-05-08T23:22:07.309329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328840+0000","last_change":"2024-05-08T23:22:07.324702+0000","last_active":"2024-05-08T23:22:07.328840+0000","last_peered":"2024-05-08T23:22:07.328840+0000","last_clean":"2024-05-08T23:22:07.328840+0000","last_became_active":"2024-05-08T23:22:07.323911+0000","last_became_peered":"2024-05-08T23:22:07.323911+0000","last_unstale":"2024-05-08T23:22:07.328840+0000","last_undegraded":"2024-05-08T23:22:07.328840+0000","last_fullsized":"2024-05-08T23:22:07.328840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:28.458137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317680+0000","last_change":"2024-05-08T23:22:07.317680+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317680+0000","last_undegraded":"2024-05-08T23:22:07.317680+0000","last_fullsized":"2024-05-08T23:22:07.317680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327463+0000","last_change":"2024-05-08T23:22:07.319729+0000","last_active":"2024-05-08T23:22:07.327463+0000","last_peered":"2024-05-08T23:22:07.327463+0000","last_clean":"2024-05-08T23:22:07.327463+0000","last_became_active":"2024-05-08T23:22:07.319420+0000","last_became_peered":"2024-05-08T23:22:07.319420+0000","last_unstale":"2024-05-08T23:22:07.327463+0000","last_undegraded":"2024-05-08T23:22:07.327463+0000","last_fullsized":"2024-05-08T23:22:07.327463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:13.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309969+0000","last_change":"2024-05-08T23:22:07.309969+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309969+0000","last_undegraded":"2024-05-08T23:22:07.309969+0000","last_fullsized":"2024-05-08T23:22:07.309969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:10.382589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327526+0000","last_change":"2024-05-08T23:22:07.327526+0000","last_active":"2024-05-08T23:22:07.327526+0000","last_peered":"2024-05-08T23:22:07.327526+0000","last_clean":"2024-05-08T23:22:07.327526+0000","last_became_active":"2024-05-08T23:22:07.327240+0000","last_became_peered":"2024-05-08T23:22:07.327240+0000","last_unstale":"2024-05-08T23:22:07.327526+0000","last_undegraded":"2024-05-08T23:22:07.327526+0000","last_fullsized":"2024-05-08T23:22:07.327526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317928+0000","last_change":"2024-05-08T23:22:07.317928+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317928+0000","last_undegraded":"2024-05-08T23:22:07.317928+0000","last_fullsized":"2024-05-08T23:22:07.317928+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328772+0000","last_change":"2024-05-08T23:22:07.324290+0000","last_active":"2024-05-08T23:22:07.328772+0000","last_peered":"2024-05-08T23:22:07.328772+0000","last_clean":"2024-05-08T23:22:07.328772+0000","last_became_active":"2024-05-08T23:22:07.323929+0000","last_became_peered":"2024-05-08T23:22:07.323929+0000","last_unstale":"2024-05-08T23:22:07.328772+0000","last_undegraded":"2024-05-08T23:22:07.328772+0000","last_fullsized":"2024-05-08T23:22:07.328772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:17.483564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312557+0000","last_change":"2024-05-08T23:22:07.312557+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312557+0000","last_undegraded":"2024-05-08T23:22:07.312557+0000","last_fullsized":"2024-05-08T23:22:07.312557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:38.296628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328617+0000","last_change":"2024-05-08T23:22:07.323609+0000","last_active":"2024-05-08T23:22:07.328617+0000","last_peered":"2024-05-08T23:22:07.328617+0000","last_clean":"2024-05-08T23:22:07.328617+0000","last_became_active":"2024-05-08T23:22:07.323344+0000","last_became_peered":"2024-05-08T23:22:07.323344+0000","last_unstale":"2024-05-08T23:22:07.328617+0000","last_undegraded":"2024-05-08T23:22:07.328617+0000","last_fullsized":"2024-05-08T23:22:07.328617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:41.903099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313857+0000","last_change":"2024-05-08T23:22:07.313857+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313857+0000","last_undegraded":"2024-05-08T23:22:07.313857+0000","last_fullsized":"2024-05-08T23:22:07.313857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.326254+0000","last_change":"2024-05-08T23:22:07.324272+0000","last_active":"2024-05-08T23:22:07.326254+0000","last_peered":"2024-05-08T23:22:07.326254+0000","last_clean":"2024-05-08T23:22:07.326254+0000","last_became_active":"2024-05-08T23:22:07.324005+0000","last_became_peered":"2024-05-08T23:22:07.324005+0000","last_unstale":"2024-05-08T23:22:07.326254+0000","last_undegraded":"2024-05-08T23:22:07.326254+0000","last_fullsized":"2024-05-08T23:22:07.326254+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310323+0000","last_change":"2024-05-08T23:22:07.310323+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310323+0000","last_undegraded":"2024-05-08T23:22:07.310323+0000","last_fullsized":"2024-05-08T23:22:07.310323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:31.969535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327612+0000","last_change":"2024-05-08T23:22:07.320014+0000","last_active":"2024-05-08T23:22:07.327612+0000","last_peered":"2024-05-08T23:22:07.327612+0000","last_clean":"2024-05-08T23:22:07.327612+0000","last_became_active":"2024-05-08T23:22:07.319073+0000","last_became_peered":"2024-05-08T23:22:07.319073+0000","last_unstale":"2024-05-08T23:22:07.327612+0000","last_undegraded":"2024-05-08T23:22:07.327612+0000","last_fullsized":"2024-05-08T23:22:07.327612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.661583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319446+0000","last_change":"2024-05-08T23:22:07.319446+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319446+0000","last_undegraded":"2024-05-08T23:22:07.319446+0000","last_fullsized":"2024-05-08T23:22:07.319446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328064+0000","last_change":"2024-05-08T23:22:07.325295+0000","last_active":"2024-05-08T23:22:07.328064+0000","last_peered":"2024-05-08T23:22:07.328064+0000","last_clean":"2024-05-08T23:22:07.328064+0000","last_became_active":"2024-05-08T23:22:07.324997+0000","last_became_peered":"2024-05-08T23:22:07.324997+0000","last_unstale":"2024-05-08T23:22:07.328064+0000","last_undegraded":"2024-05-08T23:22:07.328064+0000","last_fullsized":"2024-05-08T23:22:07.328064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:56.035940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309539+0000","last_change":"2024-05-08T23:22:07.309539+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309539+0000","last_undegraded":"2024-05-08T23:22:07.309539+0000","last_fullsized":"2024-05-08T23:22:07.309539+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327723+0000","last_change":"2024-05-08T23:22:07.323547+0000","last_active":"2024-05-08T23:22:07.327723+0000","last_peered":"2024-05-08T23:22:07.327723+0000","last_clean":"2024-05-08T23:22:07.327723+0000","last_became_active":"2024-05-08T23:22:07.323266+0000","last_became_peered":"2024-05-08T23:22:07.323266+0000","last_unstale":"2024-05-08T23:22:07.327723+0000","last_undegraded":"2024-05-08T23:22:07.327723+0000","last_fullsized":"2024-05-08T23:22:07.327723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309807+0000","last_change":"2024-05-08T23:22:07.309807+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309807+0000","last_undegraded":"2024-05-08T23:22:07.309807+0000","last_fullsized":"2024-05-08T23:22:07.309807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:47.852979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313480+0000","last_change":"2024-05-08T23:22:07.313480+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313480+0000","last_undegraded":"2024-05-08T23:22:07.313480+0000","last_fullsized":"2024-05-08T23:22:07.313480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:29.054724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309930+0000","last_change":"2024-05-08T23:22:07.309930+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309930+0000","last_undegraded":"2024-05-08T23:22:07.309930+0000","last_fullsized":"2024-05-08T23:22:07.309930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:15.611951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313374+0000","last_change":"2024-05-08T23:22:07.313374+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313374+0000","last_undegraded":"2024-05-08T23:22:07.313374+0000","last_fullsized":"2024-05-08T23:22:07.313374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309225+0000","last_change":"2024-05-08T23:22:07.309225+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309225+0000","last_undegraded":"2024-05-08T23:22:07.309225+0000","last_fullsized":"2024-05-08T23:22:07.309225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:45.661099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327791+0000","last_change":"2024-05-08T23:22:07.323427+0000","last_active":"2024-05-08T23:22:07.327791+0000","last_peered":"2024-05-08T23:22:07.327791+0000","last_clean":"2024-05-08T23:22:07.327791+0000","last_became_active":"2024-05-08T23:22:07.323163+0000","last_became_peered":"2024-05-08T23:22:07.323163+0000","last_unstale":"2024-05-08T23:22:07.327791+0000","last_undegraded":"2024-05-08T23:22:07.327791+0000","last_fullsized":"2024-05-08T23:22:07.327791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319698+0000","last_change":"2024-05-08T23:22:07.319698+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319698+0000","last_undegraded":"2024-05-08T23:22:07.319698+0000","last_fullsized":"2024-05-08T23:22:07.319698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308777+0000","last_change":"2024-05-08T23:22:07.308777+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308777+0000","last_undegraded":"2024-05-08T23:22:07.308777+0000","last_fullsized":"2024-05-08T23:22:07.308777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:04.490589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309538+0000","last_change":"2024-05-08T23:22:07.309538+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309538+0000","last_undegraded":"2024-05-08T23:22:07.309538+0000","last_fullsized":"2024-05-08T23:22:07.309538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:02.041531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.316598+0000","last_change":"2024-05-08T23:22:07.316598+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316598+0000","last_undegraded":"2024-05-08T23:22:07.316598+0000","last_fullsized":"2024-05-08T23:22:07.316598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310978+0000","last_change":"2024-05-08T23:22:07.310978+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310978+0000","last_undegraded":"2024-05-08T23:22:07.310978+0000","last_fullsized":"2024-05-08T23:22:07.310978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313873+0000","last_change":"2024-05-08T23:22:07.313873+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313873+0000","last_undegraded":"2024-05-08T23:22:07.313873+0000","last_fullsized":"2024-05-08T23:22:07.313873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313198+0000","last_change":"2024-05-08T23:22:07.313198+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313198+0000","last_undegraded":"2024-05-08T23:22:07.313198+0000","last_fullsized":"2024-05-08T23:22:07.313198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327107+0000","last_change":"2024-05-08T23:22:07.327107+0000","last_active":"2024-05-08T23:22:07.327107+0000","last_peered":"2024-05-08T23:22:07.327107+0000","last_clean":"2024-05-08T23:22:07.327107+0000","last_became_active":"2024-05-08T23:22:07.326542+0000","last_became_peered":"2024-05-08T23:22:07.326542+0000","last_unstale":"2024-05-08T23:22:07.327107+0000","last_undegraded":"2024-05-08T23:22:07.327107+0000","last_fullsized":"2024-05-08T23:22:07.327107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311985+0000","last_change":"2024-05-08T23:22:07.311985+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311985+0000","last_undegraded":"2024-05-08T23:22:07.311985+0000","last_fullsized":"2024-05-08T23:22:07.311985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.320882+0000","last_change":"2024-05-08T23:22:07.319178+0000","last_active":"2024-05-08T23:22:07.320882+0000","last_peered":"2024-05-08T23:22:07.320882+0000","last_clean":"2024-05-08T23:22:07.320882+0000","last_became_active":"2024-05-08T23:22:07.318864+0000","last_became_peered":"2024-05-08T23:22:07.318864+0000","last_unstale":"2024-05-08T23:22:07.320882+0000","last_undegraded":"2024-05-08T23:22:07.320882+0000","last_fullsized":"2024-05-08T23:22:07.320882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314790+0000","last_change":"2024-05-08T23:22:07.314790+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314790+0000","last_undegraded":"2024-05-08T23:22:07.314790+0000","last_fullsized":"2024-05-08T23:22:07.314790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.316706+0000","last_change":"2024-05-08T23:22:07.316706+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316706+0000","last_undegraded":"2024-05-08T23:22:07.316706+0000","last_fullsized":"2024-05-08T23:22:07.316706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309830+0000","last_change":"2024-05-08T23:22:07.309830+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309830+0000","last_undegraded":"2024-05-08T23:22:07.309830+0000","last_fullsized":"2024-05-08T23:22:07.309830+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:55.199701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328702+0000","last_change":"2024-05-08T23:22:07.326530+0000","last_active":"2024-05-08T23:22:07.328702+0000","last_peered":"2024-05-08T23:22:07.328702+0000","last_clean":"2024-05-08T23:22:07.328702+0000","last_became_active":"2024-05-08T23:22:07.326233+0000","last_became_peered":"2024-05-08T23:22:07.326233+0000","last_unstale":"2024-05-08T23:22:07.328702+0000","last_undegraded":"2024-05-08T23:22:07.328702+0000","last_fullsized":"2024-05-08T23:22:07.328702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:49.418732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314416+0000","last_change":"2024-05-08T23:22:07.314416+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314416+0000","last_undegraded":"2024-05-08T23:22:07.314416+0000","last_fullsized":"2024-05-08T23:22:07.314416+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328841+0000","last_change":"2024-05-08T23:22:07.325642+0000","last_active":"2024-05-08T23:22:07.328841+0000","last_peered":"2024-05-08T23:22:07.328841+0000","last_clean":"2024-05-08T23:22:07.328841+0000","last_became_active":"2024-05-08T23:22:07.325384+0000","last_became_peered":"2024-05-08T23:22:07.325384+0000","last_unstale":"2024-05-08T23:22:07.328841+0000","last_undegraded":"2024-05-08T23:22:07.328841+0000","last_fullsized":"2024-05-08T23:22:07.328841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:26.489244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315394+0000","last_change":"2024-05-08T23:22:07.315394+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315394+0000","last_undegraded":"2024-05-08T23:22:07.315394+0000","last_fullsized":"2024-05-08T23:22:07.315394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313997+0000","last_change":"2024-05-08T23:22:07.313997+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313997+0000","last_undegraded":"2024-05-08T23:22:07.313997+0000","last_fullsized":"2024-05-08T23:22:07.313997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T23:22:06.306594+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:22:06.306594+0000","last_peered":"2024-05-08T23:22:06.306594+0000","last_clean":"2024-05-08T23:22:06.306594+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:22:06.306594+0000","last_undegraded":"2024-05-08T23:22:06.306594+0000","last_fullsized":"2024-05-08T23:22:06.306594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.625},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47199999999999998}]},{"osd":1,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.501}]},{"osd":4,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.70099999999999996}]},{"osd":5,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.751},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.37}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.439},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50800000000000001}]},{"osd":4,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.626}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.9889999999999999,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.35399999999999998}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.7869999999999999,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.0519999999999996,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.379},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.6440000000000001,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.442}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.9139999999999997,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.63}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":7.1120000000000001,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.501},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.6970000000000001,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.62}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.734,"5min":7.734,"15min":7.734},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0220000000000002,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.54000000000000004}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.443}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74299999999999999}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.73299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.503}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":11.964,"5min":11.964,"15min":11.964},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.627}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.621},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.55000000000000004}]},{"osd":1,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.51900000000000002}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.0389999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.127000000000001,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.73899999999999999}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.79800000000000004}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.85199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.6710000000000003,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":4.71,"5min":4.71,"15min":4.71},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":8.6880000000000006,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.8940000000000001,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8559999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.627}]},{"osd":3,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.8770000000000007,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.5},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.427,"5min":10.427,"15min":10.427},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:22:12.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:22:12.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.628+0000 7f785cc76700 1 Processor -- start 2024-05-08T23:22:12.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.628+0000 7f785cc76700 1 -- start start 2024-05-08T23:22:12.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785cc76700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f785806b070 0x7f78580af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785cc76700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f78580b2960 0x7f78580ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785cc76700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78580cf410 con 0x7f785815db20 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785cc76700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785815df00 con 0x7f785806b070 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785cc76700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78580cf640 con 0x7f78580b2960 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785659c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f785815db20 0x7f785806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43878/0 (socket says 172.21.15.5:43878) 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785659c700 1 -- 172.21.15.5:0/4236764132 learned_addr learned my addr 172.21.15.5:0/4236764132 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f7856d9d700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f78580b2960 0x7f78580ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f7855d9b700 1 --2- 172.21.15.5:0/4236764132 >> v2:172.21.15.49:3300/0 conn(0x7f785806b070 0x7f78580af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390348256 0 0) 0x7f78580cf410 con 0x7f785815db20 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78400035b0 con 0x7f785815db20 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 760488207 0 0) 0x7f78400035b0 con 0x7f785815db20 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f78580b2960 msgr2=0x7f78580ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f78580b2960 0x7f78580ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 >> v2:172.21.15.49:3300/0 conn(0x7f785806b070 msgr2=0x7f78580af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 --2- 172.21.15.5:0/4236764132 >> v2:172.21.15.49:3300/0 conn(0x7f785806b070 0x7f78580af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580ad0f0 con 0x7f785815db20 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f7848001670 con 0x7f785815db20 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 >> v1:172.21.15.5:6789/0 conn(0x7f785815db20 legacy=0x7f785806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.629+0000 7f785559a700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7840003a00 0x7f7840003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785559a700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f78580b2960 0x7f7840004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785559a700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7840004d90 0x7f7840005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785815df00 con 0x7f7840004d90 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580cf640 con 0x7f7840003a00 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784000ba90 con 0x7f78580b2960 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f7856d9d700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7840004d90 0x7f7840005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785659c700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7840003a00 0x7f7840003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f7855d9b700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f78580b2960 0x7f7840004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f7856d9d700 1 -- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f78580b2960 msgr2=0x7f7840004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f7856d9d700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f78580b2960 0x7f7840004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f7856d9d700 1 -- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7840003a00 msgr2=0x7f7840003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f7856d9d700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7840003a00 0x7f7840003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f7856d9d700 1 -- 172.21.15.5:0/4236764132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400035b0 con 0x7f7840004d90 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f7856d9d700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7840004d90 0x7f7840005210 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7850009fc0 tx=0x7f7840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c195ba2c938fed6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785000ec30 con 0x7f7840004d90 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785000f040 con 0x7f7840004d90 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.630+0000 7f785559a700 1 -- 172.21.15.5:0/4236764132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850014480 con 0x7f7840004d90 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.631+0000 7f785cc76700 1 -- 172.21.15.5:0/4236764132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7840004d90 msgr2=0x7f7840005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.631+0000 7f785cc76700 1 --2- 172.21.15.5:0/4236764132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7840004d90 0x7f7840005210 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7850009fc0 tx=0x7f7840005880 comp rx=0 tx=0).stop 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.631+0000 7f7856d9d700 1 -- 172.21.15.5:0/4236764132 reap_dead start 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.631+0000 7f785cc76700 1 -- 172.21.15.5:0/4236764132 shutdown_connections 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.631+0000 7f785cc76700 1 -- 172.21.15.5:0/4236764132 >> 172.21.15.5:0/4236764132 conn(0x7f7858099a50 msgr2=0x7f78580b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:12.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.631+0000 7f785cc76700 1 -- 172.21.15.5:0/4236764132 shutdown_connections 2024-05-08T23:22:12.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.631+0000 7f785cc76700 1 -- 172.21.15.5:0/4236764132 wait complete. 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.633+0000 7f785cc76700 1 Processor -- start 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.633+0000 7f785cc76700 1 -- start start 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.633+0000 7f785cc76700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7840004d90 0x7f7858157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.633+0000 7f785cc76700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7858159db0 0x7f78580c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.633+0000 7f785cc76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78580c78a0 0x7f78580c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.633+0000 7f785cc76700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580c8270 con 0x7f7858159db0 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.633+0000 7f785cc76700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580cba30 con 0x7f78580c78a0 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.633+0000 7f785cc76700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7858159a80 con 0x7f7840004d90 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7858159db0 0x7f78580c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7858159db0 0x7f78580c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47208/0 (socket says 172.21.15.5:47208) 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 -- 172.21.15.5:0/2006898921 learned_addr learned my addr 172.21.15.5:0/2006898921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7856d9d700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78580c78a0 0x7f78580c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f785659c700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7840004d90 0x7f7858157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 -- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7840004d90 msgr2=0x7f7858157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7840004d90 0x7f7858157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 -- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78580c78a0 msgr2=0x7f78580c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78580c78a0 0x7f78580c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 -- 172.21.15.5:0/2006898921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78581591e0 con 0x7f7858159db0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7855d9b700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7858159db0 0x7f78580c64c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f784c011ad0 tx=0x7f784c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7846ffd700 1 -- 172.21.15.5:0/2006898921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c00fa60 con 0x7f7858159db0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7846ffd700 1 -- 172.21.15.5:0/2006898921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c00fbc0 con 0x7f7858159db0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78581594c0 con 0x7f7858159db0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f7846ffd700 1 -- 172.21.15.5:0/2006898921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c014440 con 0x7f7858159db0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.634+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78581584c0 con 0x7f7858159db0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.635+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7858068000 con 0x7f7858159db0 2024-05-08T23:22:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.636+0000 7f7846ffd700 1 -- 172.21.15.5:0/2006898921 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f784c0145a0 con 0x7f7858159db0 2024-05-08T23:22:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.636+0000 7f7846ffd700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f783806ac90 0x7f783806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.636+0000 7f7846ffd700 1 -- 172.21.15.5:0/2006898921 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f784c089a90 con 0x7f7858159db0 2024-05-08T23:22:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.637+0000 7f785659c700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f783806ac90 0x7f783806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.637+0000 7f785659c700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f783806ac90 0x7f783806d150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f7848009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.641+0000 7f7846ffd700 1 -- 172.21.15.5:0/2006898921 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f784c054d00 con 0x7f7858159db0 2024-05-08T23:22:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.852+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78580cfee0 con 0x7f783806ac90 2024-05-08T23:22:12.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.861+0000 7f7846ffd700 1 -- 172.21.15.5:0/2006898921 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285964 (secure 0 0 0) 0x7f7848001630 con 0x7f783806ac90 2024-05-08T23:22:12.865 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:12.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f783806ac90 msgr2=0x7f783806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f783806ac90 0x7f783806d150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f7848009040 comp rx=0 tx=0).stop 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7858159db0 msgr2=0x7f78580c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7858159db0 0x7f78580c64c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f784c011ad0 tx=0x7f784c011e90 comp rx=0 tx=0).stop 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 shutdown_connections 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f783806ac90 0x7f783806d150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78580c78a0 0x7f78580c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7840004d90 0x7f7858157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 --2- 172.21.15.5:0/2006898921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7858159db0 0x7f78580c64c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.864+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 >> 172.21.15.5:0/2006898921 conn(0x7f7858099a50 msgr2=0x7f78580ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.865+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 shutdown_connections 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:12.865+0000 7f785cc76700 1 -- 172.21.15.5:0/2006898921 wait complete. 2024-05-08T23:22:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:22:12.879 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-05-08T23:22:11.309863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3973120,"data_stored":2242520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.010134"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312535+0000","last_change":"2024-05-08T23:22:07.312535+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312535+0000","last_undegraded":"2024-05-08T23:22:07.312535+0000","last_fullsized":"2024-05-08T23:22:07.312535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311851+0000","last_change":"2024-05-08T23:22:07.311851+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311851+0000","last_undegraded":"2024-05-08T23:22:07.311851+0000","last_fullsized":"2024-05-08T23:22:07.311851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:58.867905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310091+0000","last_change":"2024-05-08T23:22:07.310091+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310091+0000","last_undegraded":"2024-05-08T23:22:07.310091+0000","last_fullsized":"2024-05-08T23:22:07.310091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319925+0000","last_change":"2024-05-08T23:22:07.319925+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319925+0000","last_undegraded":"2024-05-08T23:22:07.319925+0000","last_fullsized":"2024-05-08T23:22:07.319925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310832+0000","last_change":"2024-05-08T23:22:07.310832+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310832+0000","last_undegraded":"2024-05-08T23:22:07.310832+0000","last_fullsized":"2024-05-08T23:22:07.310832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:27.657029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312605+0000","last_change":"2024-05-08T23:22:07.312605+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312605+0000","last_undegraded":"2024-05-08T23:22:07.312605+0000","last_fullsized":"2024-05-08T23:22:07.312605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:10.804990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.320203+0000","last_change":"2024-05-08T23:22:07.320203+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.320203+0000","last_undegraded":"2024-05-08T23:22:07.320203+0000","last_fullsized":"2024-05-08T23:22:07.320203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.320412+0000","last_change":"2024-05-08T23:22:07.320412+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.320412+0000","last_undegraded":"2024-05-08T23:22:07.320412+0000","last_fullsized":"2024-05-08T23:22:07.320412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313870+0000","last_change":"2024-05-08T23:22:07.313870+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313870+0000","last_undegraded":"2024-05-08T23:22:07.313870+0000","last_fullsized":"2024-05-08T23:22:07.313870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318448+0000","last_change":"2024-05-08T23:22:07.318448+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318448+0000","last_undegraded":"2024-05-08T23:22:07.318448+0000","last_fullsized":"2024-05-08T23:22:07.318448+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316240+0000","last_change":"2024-05-08T23:22:07.316240+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316240+0000","last_undegraded":"2024-05-08T23:22:07.316240+0000","last_fullsized":"2024-05-08T23:22:07.316240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309934+0000","last_change":"2024-05-08T23:22:07.309934+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309934+0000","last_undegraded":"2024-05-08T23:22:07.309934+0000","last_fullsized":"2024-05-08T23:22:07.309934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:42.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309244+0000","last_change":"2024-05-08T23:22:07.309244+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309244+0000","last_undegraded":"2024-05-08T23:22:07.309244+0000","last_fullsized":"2024-05-08T23:22:07.309244+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:50.423661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311023+0000","last_change":"2024-05-08T23:22:07.311023+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311023+0000","last_undegraded":"2024-05-08T23:22:07.311023+0000","last_fullsized":"2024-05-08T23:22:07.311023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:37.867491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309447+0000","last_change":"2024-05-08T23:22:07.309447+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309447+0000","last_undegraded":"2024-05-08T23:22:07.309447+0000","last_fullsized":"2024-05-08T23:22:07.309447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318925+0000","last_change":"2024-05-08T23:22:07.318925+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318925+0000","last_undegraded":"2024-05-08T23:22:07.318925+0000","last_fullsized":"2024-05-08T23:22:07.318925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318344+0000","last_change":"2024-05-08T23:22:07.318344+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318344+0000","last_undegraded":"2024-05-08T23:22:07.318344+0000","last_fullsized":"2024-05-08T23:22:07.318344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310922+0000","last_change":"2024-05-08T23:22:07.310922+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310922+0000","last_undegraded":"2024-05-08T23:22:07.310922+0000","last_fullsized":"2024-05-08T23:22:07.310922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:44.317582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310862+0000","last_change":"2024-05-08T23:22:07.310862+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310862+0000","last_undegraded":"2024-05-08T23:22:07.310862+0000","last_fullsized":"2024-05-08T23:22:07.310862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:20.807790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311659+0000","last_change":"2024-05-08T23:22:07.311659+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311659+0000","last_undegraded":"2024-05-08T23:22:07.311659+0000","last_fullsized":"2024-05-08T23:22:07.311659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:30.059653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318563+0000","last_change":"2024-05-08T23:22:07.318563+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318563+0000","last_undegraded":"2024-05-08T23:22:07.318563+0000","last_fullsized":"2024-05-08T23:22:07.318563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318074+0000","last_change":"2024-05-08T23:22:07.318074+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318074+0000","last_undegraded":"2024-05-08T23:22:07.318074+0000","last_fullsized":"2024-05-08T23:22:07.318074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313767+0000","last_change":"2024-05-08T23:22:07.313767+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313767+0000","last_undegraded":"2024-05-08T23:22:07.313767+0000","last_fullsized":"2024-05-08T23:22:07.313767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309544+0000","last_change":"2024-05-08T23:22:07.309544+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309544+0000","last_undegraded":"2024-05-08T23:22:07.309544+0000","last_fullsized":"2024-05-08T23:22:07.309544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317099+0000","last_change":"2024-05-08T23:22:07.317099+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317099+0000","last_undegraded":"2024-05-08T23:22:07.317099+0000","last_fullsized":"2024-05-08T23:22:07.317099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310141+0000","last_change":"2024-05-08T23:22:07.310141+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310141+0000","last_undegraded":"2024-05-08T23:22:07.310141+0000","last_fullsized":"2024-05-08T23:22:07.310141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:49.810291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316305+0000","last_change":"2024-05-08T23:22:07.316305+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316305+0000","last_undegraded":"2024-05-08T23:22:07.316305+0000","last_fullsized":"2024-05-08T23:22:07.316305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.308872+0000","last_change":"2024-05-08T23:22:07.308872+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308872+0000","last_undegraded":"2024-05-08T23:22:07.308872+0000","last_fullsized":"2024-05-08T23:22:07.308872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:09.803894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314342+0000","last_change":"2024-05-08T23:22:07.314342+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314342+0000","last_undegraded":"2024-05-08T23:22:07.314342+0000","last_fullsized":"2024-05-08T23:22:07.314342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319476+0000","last_change":"2024-05-08T23:22:07.319476+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319476+0000","last_undegraded":"2024-05-08T23:22:07.319476+0000","last_fullsized":"2024-05-08T23:22:07.319476+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311923+0000","last_change":"2024-05-08T23:22:07.311923+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311923+0000","last_undegraded":"2024-05-08T23:22:07.311923+0000","last_fullsized":"2024-05-08T23:22:07.311923+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:05.495518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.316026+0000","last_change":"2024-05-08T23:22:07.316026+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316026+0000","last_undegraded":"2024-05-08T23:22:07.316026+0000","last_fullsized":"2024-05-08T23:22:07.316026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315002+0000","last_change":"2024-05-08T23:22:07.315002+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315002+0000","last_undegraded":"2024-05-08T23:22:07.315002+0000","last_fullsized":"2024-05-08T23:22:07.315002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313496+0000","last_change":"2024-05-08T23:22:07.313496+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313496+0000","last_undegraded":"2024-05-08T23:22:07.313496+0000","last_fullsized":"2024-05-08T23:22:07.313496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311991+0000","last_change":"2024-05-08T23:22:07.311991+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311991+0000","last_undegraded":"2024-05-08T23:22:07.311991+0000","last_fullsized":"2024-05-08T23:22:07.311991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.314508+0000","last_change":"2024-05-08T23:22:07.314508+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314508+0000","last_undegraded":"2024-05-08T23:22:07.314508+0000","last_fullsized":"2024-05-08T23:22:07.314508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:38.809279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317989+0000","last_change":"2024-05-08T23:22:07.317989+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317989+0000","last_undegraded":"2024-05-08T23:22:07.317989+0000","last_fullsized":"2024-05-08T23:22:07.317989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328595+0000","last_change":"2024-05-08T23:22:07.323498+0000","last_active":"2024-05-08T23:22:07.328595+0000","last_peered":"2024-05-08T23:22:07.328595+0000","last_clean":"2024-05-08T23:22:07.328595+0000","last_became_active":"2024-05-08T23:22:07.323230+0000","last_became_peered":"2024-05-08T23:22:07.323230+0000","last_unstale":"2024-05-08T23:22:07.328595+0000","last_undegraded":"2024-05-08T23:22:07.328595+0000","last_fullsized":"2024-05-08T23:22:07.328595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:50.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315788+0000","last_change":"2024-05-08T23:22:07.315788+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315788+0000","last_undegraded":"2024-05-08T23:22:07.315788+0000","last_fullsized":"2024-05-08T23:22:07.315788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308127+0000","last_change":"2024-05-08T23:22:07.308127+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308127+0000","last_undegraded":"2024-05-08T23:22:07.308127+0000","last_fullsized":"2024-05-08T23:22:07.308127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:40.830327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315411+0000","last_change":"2024-05-08T23:22:07.315411+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315411+0000","last_undegraded":"2024-05-08T23:22:07.315411+0000","last_fullsized":"2024-05-08T23:22:07.315411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327698+0000","last_change":"2024-05-08T23:22:07.324042+0000","last_active":"2024-05-08T23:22:07.327698+0000","last_peered":"2024-05-08T23:22:07.327698+0000","last_clean":"2024-05-08T23:22:07.327698+0000","last_became_active":"2024-05-08T23:22:07.323538+0000","last_became_peered":"2024-05-08T23:22:07.323538+0000","last_unstale":"2024-05-08T23:22:07.327698+0000","last_undegraded":"2024-05-08T23:22:07.327698+0000","last_fullsized":"2024-05-08T23:22:07.327698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:21.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.308908+0000","last_change":"2024-05-08T23:22:07.308908+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308908+0000","last_undegraded":"2024-05-08T23:22:07.308908+0000","last_fullsized":"2024-05-08T23:22:07.308908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327756+0000","last_change":"2024-05-08T23:22:07.323560+0000","last_active":"2024-05-08T23:22:07.327756+0000","last_peered":"2024-05-08T23:22:07.327756+0000","last_clean":"2024-05-08T23:22:07.327756+0000","last_became_active":"2024-05-08T23:22:07.323249+0000","last_became_peered":"2024-05-08T23:22:07.323249+0000","last_unstale":"2024-05-08T23:22:07.327756+0000","last_undegraded":"2024-05-08T23:22:07.327756+0000","last_fullsized":"2024-05-08T23:22:07.327756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:04.715359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317833+0000","last_change":"2024-05-08T23:22:07.317833+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317833+0000","last_undegraded":"2024-05-08T23:22:07.317833+0000","last_fullsized":"2024-05-08T23:22:07.317833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327285+0000","last_change":"2024-05-08T23:22:07.320297+0000","last_active":"2024-05-08T23:22:07.327285+0000","last_peered":"2024-05-08T23:22:07.327285+0000","last_clean":"2024-05-08T23:22:07.327285+0000","last_became_active":"2024-05-08T23:22:07.320014+0000","last_became_peered":"2024-05-08T23:22:07.320014+0000","last_unstale":"2024-05-08T23:22:07.327285+0000","last_undegraded":"2024-05-08T23:22:07.327285+0000","last_fullsized":"2024-05-08T23:22:07.327285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:51.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.318358+0000","last_change":"2024-05-08T23:22:07.318358+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.318358+0000","last_undegraded":"2024-05-08T23:22:07.318358+0000","last_fullsized":"2024-05-08T23:22:07.318358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328104+0000","last_change":"2024-05-08T23:22:07.326385+0000","last_active":"2024-05-08T23:22:07.328104+0000","last_peered":"2024-05-08T23:22:07.328104+0000","last_clean":"2024-05-08T23:22:07.328104+0000","last_became_active":"2024-05-08T23:22:07.326047+0000","last_became_peered":"2024-05-08T23:22:07.326047+0000","last_unstale":"2024-05-08T23:22:07.328104+0000","last_undegraded":"2024-05-08T23:22:07.328104+0000","last_fullsized":"2024-05-08T23:22:07.328104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:46.848050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309020+0000","last_change":"2024-05-08T23:22:07.309020+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309020+0000","last_undegraded":"2024-05-08T23:22:07.309020+0000","last_fullsized":"2024-05-08T23:22:07.309020+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:41.329227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328884+0000","last_change":"2024-05-08T23:22:07.324876+0000","last_active":"2024-05-08T23:22:07.328884+0000","last_peered":"2024-05-08T23:22:07.328884+0000","last_clean":"2024-05-08T23:22:07.328884+0000","last_became_active":"2024-05-08T23:22:07.324583+0000","last_became_peered":"2024-05-08T23:22:07.324583+0000","last_unstale":"2024-05-08T23:22:07.328884+0000","last_undegraded":"2024-05-08T23:22:07.328884+0000","last_fullsized":"2024-05-08T23:22:07.328884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:00.240669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310539+0000","last_change":"2024-05-08T23:22:07.310539+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310539+0000","last_undegraded":"2024-05-08T23:22:07.310539+0000","last_fullsized":"2024-05-08T23:22:07.310539+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:08.578978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313769+0000","last_change":"2024-05-08T23:22:07.313769+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313769+0000","last_undegraded":"2024-05-08T23:22:07.313769+0000","last_fullsized":"2024-05-08T23:22:07.313769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:37.291699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312087+0000","last_change":"2024-05-08T23:22:07.312087+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312087+0000","last_undegraded":"2024-05-08T23:22:07.312087+0000","last_fullsized":"2024-05-08T23:22:07.312087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:33.565554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308718+0000","last_change":"2024-05-08T23:22:07.308718+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308718+0000","last_undegraded":"2024-05-08T23:22:07.308718+0000","last_fullsized":"2024-05-08T23:22:07.308718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:40.686112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309329+0000","last_change":"2024-05-08T23:22:07.309329+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309329+0000","last_undegraded":"2024-05-08T23:22:07.309329+0000","last_fullsized":"2024-05-08T23:22:07.309329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328840+0000","last_change":"2024-05-08T23:22:07.324702+0000","last_active":"2024-05-08T23:22:07.328840+0000","last_peered":"2024-05-08T23:22:07.328840+0000","last_clean":"2024-05-08T23:22:07.328840+0000","last_became_active":"2024-05-08T23:22:07.323911+0000","last_became_peered":"2024-05-08T23:22:07.323911+0000","last_unstale":"2024-05-08T23:22:07.328840+0000","last_undegraded":"2024-05-08T23:22:07.328840+0000","last_fullsized":"2024-05-08T23:22:07.328840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:28.458137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317680+0000","last_change":"2024-05-08T23:22:07.317680+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317680+0000","last_undegraded":"2024-05-08T23:22:07.317680+0000","last_fullsized":"2024-05-08T23:22:07.317680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327463+0000","last_change":"2024-05-08T23:22:07.319729+0000","last_active":"2024-05-08T23:22:07.327463+0000","last_peered":"2024-05-08T23:22:07.327463+0000","last_clean":"2024-05-08T23:22:07.327463+0000","last_became_active":"2024-05-08T23:22:07.319420+0000","last_became_peered":"2024-05-08T23:22:07.319420+0000","last_unstale":"2024-05-08T23:22:07.327463+0000","last_undegraded":"2024-05-08T23:22:07.327463+0000","last_fullsized":"2024-05-08T23:22:07.327463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:13.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309969+0000","last_change":"2024-05-08T23:22:07.309969+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309969+0000","last_undegraded":"2024-05-08T23:22:07.309969+0000","last_fullsized":"2024-05-08T23:22:07.309969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:10.382589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327526+0000","last_change":"2024-05-08T23:22:07.327526+0000","last_active":"2024-05-08T23:22:07.327526+0000","last_peered":"2024-05-08T23:22:07.327526+0000","last_clean":"2024-05-08T23:22:07.327526+0000","last_became_active":"2024-05-08T23:22:07.327240+0000","last_became_peered":"2024-05-08T23:22:07.327240+0000","last_unstale":"2024-05-08T23:22:07.327526+0000","last_undegraded":"2024-05-08T23:22:07.327526+0000","last_fullsized":"2024-05-08T23:22:07.327526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.317928+0000","last_change":"2024-05-08T23:22:07.317928+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.317928+0000","last_undegraded":"2024-05-08T23:22:07.317928+0000","last_fullsized":"2024-05-08T23:22:07.317928+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328772+0000","last_change":"2024-05-08T23:22:07.324290+0000","last_active":"2024-05-08T23:22:07.328772+0000","last_peered":"2024-05-08T23:22:07.328772+0000","last_clean":"2024-05-08T23:22:07.328772+0000","last_became_active":"2024-05-08T23:22:07.323929+0000","last_became_peered":"2024-05-08T23:22:07.323929+0000","last_unstale":"2024-05-08T23:22:07.328772+0000","last_undegraded":"2024-05-08T23:22:07.328772+0000","last_fullsized":"2024-05-08T23:22:07.328772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:17.483564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.312557+0000","last_change":"2024-05-08T23:22:07.312557+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.312557+0000","last_undegraded":"2024-05-08T23:22:07.312557+0000","last_fullsized":"2024-05-08T23:22:07.312557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:38.296628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328617+0000","last_change":"2024-05-08T23:22:07.323609+0000","last_active":"2024-05-08T23:22:07.328617+0000","last_peered":"2024-05-08T23:22:07.328617+0000","last_clean":"2024-05-08T23:22:07.328617+0000","last_became_active":"2024-05-08T23:22:07.323344+0000","last_became_peered":"2024-05-08T23:22:07.323344+0000","last_unstale":"2024-05-08T23:22:07.328617+0000","last_undegraded":"2024-05-08T23:22:07.328617+0000","last_fullsized":"2024-05-08T23:22:07.328617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:41.903099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313857+0000","last_change":"2024-05-08T23:22:07.313857+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313857+0000","last_undegraded":"2024-05-08T23:22:07.313857+0000","last_fullsized":"2024-05-08T23:22:07.313857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.326254+0000","last_change":"2024-05-08T23:22:07.324272+0000","last_active":"2024-05-08T23:22:07.326254+0000","last_peered":"2024-05-08T23:22:07.326254+0000","last_clean":"2024-05-08T23:22:07.326254+0000","last_became_active":"2024-05-08T23:22:07.324005+0000","last_became_peered":"2024-05-08T23:22:07.324005+0000","last_unstale":"2024-05-08T23:22:07.326254+0000","last_undegraded":"2024-05-08T23:22:07.326254+0000","last_fullsized":"2024-05-08T23:22:07.326254+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310323+0000","last_change":"2024-05-08T23:22:07.310323+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310323+0000","last_undegraded":"2024-05-08T23:22:07.310323+0000","last_fullsized":"2024-05-08T23:22:07.310323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:31.969535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327612+0000","last_change":"2024-05-08T23:22:07.320014+0000","last_active":"2024-05-08T23:22:07.327612+0000","last_peered":"2024-05-08T23:22:07.327612+0000","last_clean":"2024-05-08T23:22:07.327612+0000","last_became_active":"2024-05-08T23:22:07.319073+0000","last_became_peered":"2024-05-08T23:22:07.319073+0000","last_unstale":"2024-05-08T23:22:07.327612+0000","last_undegraded":"2024-05-08T23:22:07.327612+0000","last_fullsized":"2024-05-08T23:22:07.327612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.661583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319446+0000","last_change":"2024-05-08T23:22:07.319446+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319446+0000","last_undegraded":"2024-05-08T23:22:07.319446+0000","last_fullsized":"2024-05-08T23:22:07.319446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328064+0000","last_change":"2024-05-08T23:22:07.325295+0000","last_active":"2024-05-08T23:22:07.328064+0000","last_peered":"2024-05-08T23:22:07.328064+0000","last_clean":"2024-05-08T23:22:07.328064+0000","last_became_active":"2024-05-08T23:22:07.324997+0000","last_became_peered":"2024-05-08T23:22:07.324997+0000","last_unstale":"2024-05-08T23:22:07.328064+0000","last_undegraded":"2024-05-08T23:22:07.328064+0000","last_fullsized":"2024-05-08T23:22:07.328064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:56.035940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309539+0000","last_change":"2024-05-08T23:22:07.309539+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309539+0000","last_undegraded":"2024-05-08T23:22:07.309539+0000","last_fullsized":"2024-05-08T23:22:07.309539+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327723+0000","last_change":"2024-05-08T23:22:07.323547+0000","last_active":"2024-05-08T23:22:07.327723+0000","last_peered":"2024-05-08T23:22:07.327723+0000","last_clean":"2024-05-08T23:22:07.327723+0000","last_became_active":"2024-05-08T23:22:07.323266+0000","last_became_peered":"2024-05-08T23:22:07.323266+0000","last_unstale":"2024-05-08T23:22:07.327723+0000","last_undegraded":"2024-05-08T23:22:07.327723+0000","last_fullsized":"2024-05-08T23:22:07.327723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309807+0000","last_change":"2024-05-08T23:22:07.309807+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309807+0000","last_undegraded":"2024-05-08T23:22:07.309807+0000","last_fullsized":"2024-05-08T23:22:07.309807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:47.852979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313480+0000","last_change":"2024-05-08T23:22:07.313480+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313480+0000","last_undegraded":"2024-05-08T23:22:07.313480+0000","last_fullsized":"2024-05-08T23:22:07.313480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:29.054724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309930+0000","last_change":"2024-05-08T23:22:07.309930+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309930+0000","last_undegraded":"2024-05-08T23:22:07.309930+0000","last_fullsized":"2024-05-08T23:22:07.309930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:15.611951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313374+0000","last_change":"2024-05-08T23:22:07.313374+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313374+0000","last_undegraded":"2024-05-08T23:22:07.313374+0000","last_fullsized":"2024-05-08T23:22:07.313374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309225+0000","last_change":"2024-05-08T23:22:07.309225+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309225+0000","last_undegraded":"2024-05-08T23:22:07.309225+0000","last_fullsized":"2024-05-08T23:22:07.309225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:45.661099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327791+0000","last_change":"2024-05-08T23:22:07.323427+0000","last_active":"2024-05-08T23:22:07.327791+0000","last_peered":"2024-05-08T23:22:07.327791+0000","last_clean":"2024-05-08T23:22:07.327791+0000","last_became_active":"2024-05-08T23:22:07.323163+0000","last_became_peered":"2024-05-08T23:22:07.323163+0000","last_unstale":"2024-05-08T23:22:07.327791+0000","last_undegraded":"2024-05-08T23:22:07.327791+0000","last_fullsized":"2024-05-08T23:22:07.327791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.319698+0000","last_change":"2024-05-08T23:22:07.319698+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.319698+0000","last_undegraded":"2024-05-08T23:22:07.319698+0000","last_fullsized":"2024-05-08T23:22:07.319698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.308777+0000","last_change":"2024-05-08T23:22:07.308777+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.308777+0000","last_undegraded":"2024-05-08T23:22:07.308777+0000","last_fullsized":"2024-05-08T23:22:07.308777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:04.490589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309538+0000","last_change":"2024-05-08T23:22:07.309538+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309538+0000","last_undegraded":"2024-05-08T23:22:07.309538+0000","last_fullsized":"2024-05-08T23:22:07.309538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:02.041531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.316598+0000","last_change":"2024-05-08T23:22:07.316598+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316598+0000","last_undegraded":"2024-05-08T23:22:07.316598+0000","last_fullsized":"2024-05-08T23:22:07.316598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.310978+0000","last_change":"2024-05-08T23:22:07.310978+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.310978+0000","last_undegraded":"2024-05-08T23:22:07.310978+0000","last_fullsized":"2024-05-08T23:22:07.310978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313873+0000","last_change":"2024-05-08T23:22:07.313873+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313873+0000","last_undegraded":"2024-05-08T23:22:07.313873+0000","last_fullsized":"2024-05-08T23:22:07.313873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.313198+0000","last_change":"2024-05-08T23:22:07.313198+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313198+0000","last_undegraded":"2024-05-08T23:22:07.313198+0000","last_fullsized":"2024-05-08T23:22:07.313198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.327107+0000","last_change":"2024-05-08T23:22:07.327107+0000","last_active":"2024-05-08T23:22:07.327107+0000","last_peered":"2024-05-08T23:22:07.327107+0000","last_clean":"2024-05-08T23:22:07.327107+0000","last_became_active":"2024-05-08T23:22:07.326542+0000","last_became_peered":"2024-05-08T23:22:07.326542+0000","last_unstale":"2024-05-08T23:22:07.327107+0000","last_undegraded":"2024-05-08T23:22:07.327107+0000","last_fullsized":"2024-05-08T23:22:07.327107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.311985+0000","last_change":"2024-05-08T23:22:07.311985+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.311985+0000","last_undegraded":"2024-05-08T23:22:07.311985+0000","last_fullsized":"2024-05-08T23:22:07.311985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.320882+0000","last_change":"2024-05-08T23:22:07.319178+0000","last_active":"2024-05-08T23:22:07.320882+0000","last_peered":"2024-05-08T23:22:07.320882+0000","last_clean":"2024-05-08T23:22:07.320882+0000","last_became_active":"2024-05-08T23:22:07.318864+0000","last_became_peered":"2024-05-08T23:22:07.318864+0000","last_unstale":"2024-05-08T23:22:07.320882+0000","last_undegraded":"2024-05-08T23:22:07.320882+0000","last_fullsized":"2024-05-08T23:22:07.320882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314790+0000","last_change":"2024-05-08T23:22:07.314790+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314790+0000","last_undegraded":"2024-05-08T23:22:07.314790+0000","last_fullsized":"2024-05-08T23:22:07.314790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.316706+0000","last_change":"2024-05-08T23:22:07.316706+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.316706+0000","last_undegraded":"2024-05-08T23:22:07.316706+0000","last_fullsized":"2024-05-08T23:22:07.316706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.309830+0000","last_change":"2024-05-08T23:22:07.309830+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.309830+0000","last_undegraded":"2024-05-08T23:22:07.309830+0000","last_fullsized":"2024-05-08T23:22:07.309830+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:55.199701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328702+0000","last_change":"2024-05-08T23:22:07.326530+0000","last_active":"2024-05-08T23:22:07.328702+0000","last_peered":"2024-05-08T23:22:07.328702+0000","last_clean":"2024-05-08T23:22:07.328702+0000","last_became_active":"2024-05-08T23:22:07.326233+0000","last_became_peered":"2024-05-08T23:22:07.326233+0000","last_unstale":"2024-05-08T23:22:07.328702+0000","last_undegraded":"2024-05-08T23:22:07.328702+0000","last_fullsized":"2024-05-08T23:22:07.328702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:49.418732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.314416+0000","last_change":"2024-05-08T23:22:07.314416+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.314416+0000","last_undegraded":"2024-05-08T23:22:07.314416+0000","last_fullsized":"2024-05-08T23:22:07.314416+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.328841+0000","last_change":"2024-05-08T23:22:07.325642+0000","last_active":"2024-05-08T23:22:07.328841+0000","last_peered":"2024-05-08T23:22:07.328841+0000","last_clean":"2024-05-08T23:22:07.328841+0000","last_became_active":"2024-05-08T23:22:07.325384+0000","last_became_peered":"2024-05-08T23:22:07.325384+0000","last_unstale":"2024-05-08T23:22:07.328841+0000","last_undegraded":"2024-05-08T23:22:07.328841+0000","last_fullsized":"2024-05-08T23:22:07.328841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:26.489244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-05-08T23:22:07.315394+0000","last_change":"2024-05-08T23:22:07.315394+0000","last_active":"2024-05-08T23:22:07.299109+0000","last_peered":"2024-05-08T23:22:07.299109+0000","last_clean":"2024-05-08T23:22:07.299109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.315394+0000","last_undegraded":"2024-05-08T23:22:07.315394+0000","last_fullsized":"2024-05-08T23:22:07.315394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-05-08T23:22:07.313997+0000","last_change":"2024-05-08T23:22:07.313997+0000","last_active":"2024-05-08T23:22:06.294180+0000","last_peered":"2024-05-08T23:22:06.294180+0000","last_clean":"2024-05-08T23:22:06.294180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:22:07.313997+0000","last_undegraded":"2024-05-08T23:22:07.313997+0000","last_fullsized":"2024-05-08T23:22:07.313997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T23:22:06.306594+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:22:06.306594+0000","last_peered":"2024-05-08T23:22:06.306594+0000","last_clean":"2024-05-08T23:22:06.306594+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:22:06.306594+0000","last_undegraded":"2024-05-08T23:22:06.306594+0000","last_fullsized":"2024-05-08T23:22:06.306594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.625},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47199999999999998}]},{"osd":1,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.501}]},{"osd":4,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.70099999999999996}]},{"osd":5,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.751},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.37}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.439},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.50800000000000001}]},{"osd":4,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.626}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.9889999999999999,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.35399999999999998}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.7869999999999999,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.0519999999999996,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.379},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.6440000000000001,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.442}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.9139999999999997,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.63}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":7.1120000000000001,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.501},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.6970000000000001,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.62}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.734,"5min":7.734,"15min":7.734},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0220000000000002,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.54000000000000004}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.443}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74299999999999999}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.73299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.503}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":11.964,"5min":11.964,"15min":11.964},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.627}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.621},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.55000000000000004}]},{"osd":1,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.51900000000000002}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.0389999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.127000000000001,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.73899999999999999}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.79800000000000004}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.85199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.6710000000000003,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":4.71,"5min":4.71,"15min":4.71},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":8.6880000000000006,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.8940000000000001,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8559999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.627}]},{"osd":3,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.8770000000000007,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.5},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.427,"5min":10.427,"15min":10.427},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:22:12.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-05-08T23:22:12.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.312535+0000', 'last_change': '2024-05-08T23:22:07.312535+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.312535+0000', 'last_undegraded': '2024-05-08T23:22:07.312535+0000', 'last_fullsized': '2024-05-08T23:22:07.312535+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.311851+0000', 'last_change': '2024-05-08T23:22:07.311851+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.311851+0000', 'last_undegraded': '2024-05-08T23:22:07.311851+0000', 'last_fullsized': '2024-05-08T23:22:07.311851+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:03:58.867905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.310091+0000', 'last_change': '2024-05-08T23:22:07.310091+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.310091+0000', 'last_undegraded': '2024-05-08T23:22:07.310091+0000', 'last_fullsized': '2024-05-08T23:22:07.310091+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.319925+0000', 'last_change': '2024-05-08T23:22:07.319925+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.319925+0000', 'last_undegraded': '2024-05-08T23:22:07.319925+0000', 'last_fullsized': '2024-05-08T23:22:07.319925+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.310832+0000', 'last_change': '2024-05-08T23:22:07.310832+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.310832+0000', 'last_undegraded': '2024-05-08T23:22:07.310832+0000', 'last_fullsized': '2024-05-08T23:22:07.310832+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T09:36:27.657029+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-05-08T23:22:12.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.312605+0000', 'last_change': '2024-05-08T23:22:07.312605+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.312605+0000', 'last_undegraded': '2024-05-08T23:22:07.312605+0000', 'last_fullsized': '2024-05-08T23:22:07.312605+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:17:10.804990+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.320203+0000', 'last_change': '2024-05-08T23:22:07.320203+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.320203+0000', 'last_undegraded': '2024-05-08T23:22:07.320203+0000', 'last_fullsized': '2024-05-08T23:22:07.320203+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.320412+0000', 'last_change': '2024-05-08T23:22:07.320412+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.320412+0000', 'last_undegraded': '2024-05-08T23:22:07.320412+0000', 'last_fullsized': '2024-05-08T23:22:07.320412+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.313870+0000', 'last_change': '2024-05-08T23:22:07.313870+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313870+0000', 'last_undegraded': '2024-05-08T23:22:07.313870+0000', 'last_fullsized': '2024-05-08T23:22:07.313870+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.318448+0000', 'last_change': '2024-05-08T23:22:07.318448+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.318448+0000', 'last_undegraded': '2024-05-08T23:22:07.318448+0000', 'last_fullsized': '2024-05-08T23:22:07.318448+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-05-08T23:22:12.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.316240+0000', 'last_change': '2024-05-08T23:22:07.316240+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.316240+0000', 'last_undegraded': '2024-05-08T23:22:07.316240+0000', 'last_fullsized': '2024-05-08T23:22:07.316240+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-05-08T23:22:12.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309934+0000', 'last_change': '2024-05-08T23:22:07.309934+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309934+0000', 'last_undegraded': '2024-05-08T23:22:07.309934+0000', 'last_fullsized': '2024-05-08T23:22:07.309934+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-09T23:48:42.227775+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-05-08T23:22:12.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309244+0000', 'last_change': '2024-05-08T23:22:07.309244+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309244+0000', 'last_undegraded': '2024-05-08T23:22:07.309244+0000', 'last_fullsized': '2024-05-08T23:22:07.309244+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:46:50.423661+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-05-08T23:22:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.311023+0000', 'last_change': '2024-05-08T23:22:07.311023+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.311023+0000', 'last_undegraded': '2024-05-08T23:22:07.311023+0000', 'last_fullsized': '2024-05-08T23:22:07.311023+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:58:37.867491+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-05-08T23:22:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309447+0000', 'last_change': '2024-05-08T23:22:07.309447+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309447+0000', 'last_undegraded': '2024-05-08T23:22:07.309447+0000', 'last_fullsized': '2024-05-08T23:22:07.309447+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-05-08T23:22:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.318925+0000', 'last_change': '2024-05-08T23:22:07.318925+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.318925+0000', 'last_undegraded': '2024-05-08T23:22:07.318925+0000', 'last_fullsized': '2024-05-08T23:22:07.318925+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-05-08T23:22:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.318344+0000', 'last_change': '2024-05-08T23:22:07.318344+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.318344+0000', 'last_undegraded': '2024-05-08T23:22:07.318344+0000', 'last_fullsized': '2024-05-08T23:22:07.318344+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.310922+0000', 'last_change': '2024-05-08T23:22:07.310922+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.310922+0000', 'last_undegraded': '2024-05-08T23:22:07.310922+0000', 'last_fullsized': '2024-05-08T23:22:07.310922+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:31:44.317582+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.310862+0000', 'last_change': '2024-05-08T23:22:07.310862+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.310862+0000', 'last_undegraded': '2024-05-08T23:22:07.310862+0000', 'last_fullsized': '2024-05-08T23:22:07.310862+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T06:49:20.807790+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.311659+0000', 'last_change': '2024-05-08T23:22:07.311659+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.311659+0000', 'last_undegraded': '2024-05-08T23:22:07.311659+0000', 'last_fullsized': '2024-05-08T23:22:07.311659+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:36:30.059653+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.318563+0000', 'last_change': '2024-05-08T23:22:07.318563+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.318563+0000', 'last_undegraded': '2024-05-08T23:22:07.318563+0000', 'last_fullsized': '2024-05-08T23:22:07.318563+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-05-08T23:22:12.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.318074+0000', 'last_change': '2024-05-08T23:22:07.318074+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.318074+0000', 'last_undegraded': '2024-05-08T23:22:07.318074+0000', 'last_fullsized': '2024-05-08T23:22:07.318074+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.313767+0000', 'last_change': '2024-05-08T23:22:07.313767+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313767+0000', 'last_undegraded': '2024-05-08T23:22:07.313767+0000', 'last_fullsized': '2024-05-08T23:22:07.313767+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309544+0000', 'last_change': '2024-05-08T23:22:07.309544+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309544+0000', 'last_undegraded': '2024-05-08T23:22:07.309544+0000', 'last_fullsized': '2024-05-08T23:22:07.309544+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.317099+0000', 'last_change': '2024-05-08T23:22:07.317099+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.317099+0000', 'last_undegraded': '2024-05-08T23:22:07.317099+0000', 'last_fullsized': '2024-05-08T23:22:07.317099+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.310141+0000', 'last_change': '2024-05-08T23:22:07.310141+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.310141+0000', 'last_undegraded': '2024-05-08T23:22:07.310141+0000', 'last_fullsized': '2024-05-08T23:22:07.310141+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T10:52:49.810291+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-05-08T23:22:12.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.316305+0000', 'last_change': '2024-05-08T23:22:07.316305+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.316305+0000', 'last_undegraded': '2024-05-08T23:22:07.316305+0000', 'last_fullsized': '2024-05-08T23:22:07.316305+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-05-08T23:22:12.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.308872+0000', 'last_change': '2024-05-08T23:22:07.308872+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.308872+0000', 'last_undegraded': '2024-05-08T23:22:07.308872+0000', 'last_fullsized': '2024-05-08T23:22:07.308872+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:21:09.803894+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-05-08T23:22:12.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.314342+0000', 'last_change': '2024-05-08T23:22:07.314342+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.314342+0000', 'last_undegraded': '2024-05-08T23:22:07.314342+0000', 'last_fullsized': '2024-05-08T23:22:07.314342+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:22:12.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-05-08T23:22:12.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.319476+0000', 'last_change': '2024-05-08T23:22:07.319476+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.319476+0000', 'last_undegraded': '2024-05-08T23:22:07.319476+0000', 'last_fullsized': '2024-05-08T23:22:07.319476+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.311923+0000', 'last_change': '2024-05-08T23:22:07.311923+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.311923+0000', 'last_undegraded': '2024-05-08T23:22:07.311923+0000', 'last_fullsized': '2024-05-08T23:22:07.311923+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:05:05.495518+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.316026+0000', 'last_change': '2024-05-08T23:22:07.316026+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.316026+0000', 'last_undegraded': '2024-05-08T23:22:07.316026+0000', 'last_fullsized': '2024-05-08T23:22:07.316026+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.315002+0000', 'last_change': '2024-05-08T23:22:07.315002+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.315002+0000', 'last_undegraded': '2024-05-08T23:22:07.315002+0000', 'last_fullsized': '2024-05-08T23:22:07.315002+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.313496+0000', 'last_change': '2024-05-08T23:22:07.313496+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313496+0000', 'last_undegraded': '2024-05-08T23:22:07.313496+0000', 'last_fullsized': '2024-05-08T23:22:07.313496+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': ['6', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-05-08T23:22:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.311991+0000', 'last_change': '2024-05-08T23:22:07.311991+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.311991+0000', 'last_undegraded': '2024-05-08T23:22:07.311991+0000', 'last_fullsized': '2024-05-08T23:22:07.311991+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.314508+0000', 'last_change': '2024-05-08T23:22:07.314508+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.314508+0000', 'last_undegraded': '2024-05-08T23:22:07.314508+0000', 'last_fullsized': '2024-05-08T23:22:07.314508+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:59:38.809279+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': ['0', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.317989+0000', 'last_change': '2024-05-08T23:22:07.317989+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.317989+0000', 'last_undegraded': '2024-05-08T23:22:07.317989+0000', 'last_fullsized': '2024-05-08T23:22:07.317989+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.315788+0000', 'last_change': '2024-05-08T23:22:07.315788+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.315788+0000', 'last_undegraded': '2024-05-08T23:22:07.315788+0000', 'last_fullsized': '2024-05-08T23:22:07.315788+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.308127+0000', 'last_change': '2024-05-08T23:22:07.308127+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.308127+0000', 'last_undegraded': '2024-05-08T23:22:07.308127+0000', 'last_fullsized': '2024-05-08T23:22:07.308127+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T08:14:40.830327+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-05-08T23:22:12.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.315411+0000', 'last_change': '2024-05-08T23:22:07.315411+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.315411+0000', 'last_undegraded': '2024-05-08T23:22:07.315411+0000', 'last_fullsized': '2024-05-08T23:22:07.315411+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-05-08T23:22:12.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.308908+0000', 'last_change': '2024-05-08T23:22:07.308908+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.308908+0000', 'last_undegraded': '2024-05-08T23:22:07.308908+0000', 'last_fullsized': '2024-05-08T23:22:07.308908+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-05-08T23:22:12.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.317833+0000', 'last_change': '2024-05-08T23:22:07.317833+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.317833+0000', 'last_undegraded': '2024-05-08T23:22:07.317833+0000', 'last_fullsized': '2024-05-08T23:22:07.317833+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:22:12.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-05-08T23:22:12.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.318358+0000', 'last_change': '2024-05-08T23:22:07.318358+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.318358+0000', 'last_undegraded': '2024-05-08T23:22:07.318358+0000', 'last_fullsized': '2024-05-08T23:22:07.318358+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-05-08T23:22:12.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309020+0000', 'last_change': '2024-05-08T23:22:07.309020+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309020+0000', 'last_undegraded': '2024-05-08T23:22:07.309020+0000', 'last_fullsized': '2024-05-08T23:22:07.309020+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:50:41.329227+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.310539+0000', 'last_change': '2024-05-08T23:22:07.310539+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.310539+0000', 'last_undegraded': '2024-05-08T23:22:07.310539+0000', 'last_fullsized': '2024-05-08T23:22:07.310539+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T04:19:08.578978+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.313769+0000', 'last_change': '2024-05-08T23:22:07.313769+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313769+0000', 'last_undegraded': '2024-05-08T23:22:07.313769+0000', 'last_fullsized': '2024-05-08T23:22:07.313769+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:18:37.291699+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': ['0', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.312087+0000', 'last_change': '2024-05-08T23:22:07.312087+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.312087+0000', 'last_undegraded': '2024-05-08T23:22:07.312087+0000', 'last_fullsized': '2024-05-08T23:22:07.312087+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T03:31:33.565554+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.308718+0000', 'last_change': '2024-05-08T23:22:07.308718+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.308718+0000', 'last_undegraded': '2024-05-08T23:22:07.308718+0000', 'last_fullsized': '2024-05-08T23:22:07.308718+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:21:40.686112+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-05-08T23:22:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309329+0000', 'last_change': '2024-05-08T23:22:07.309329+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309329+0000', 'last_undegraded': '2024-05-08T23:22:07.309329+0000', 'last_fullsized': '2024-05-08T23:22:07.309329+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.317680+0000', 'last_change': '2024-05-08T23:22:07.317680+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.317680+0000', 'last_undegraded': '2024-05-08T23:22:07.317680+0000', 'last_fullsized': '2024-05-08T23:22:07.317680+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309969+0000', 'last_change': '2024-05-08T23:22:07.309969+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309969+0000', 'last_undegraded': '2024-05-08T23:22:07.309969+0000', 'last_fullsized': '2024-05-08T23:22:07.309969+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T04:36:10.382589+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.317928+0000', 'last_change': '2024-05-08T23:22:07.317928+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.317928+0000', 'last_undegraded': '2024-05-08T23:22:07.317928+0000', 'last_fullsized': '2024-05-08T23:22:07.317928+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.312557+0000', 'last_change': '2024-05-08T23:22:07.312557+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.312557+0000', 'last_undegraded': '2024-05-08T23:22:07.312557+0000', 'last_fullsized': '2024-05-08T23:22:07.312557+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:18:38.296628+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.313857+0000', 'last_change': '2024-05-08T23:22:07.313857+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313857+0000', 'last_undegraded': '2024-05-08T23:22:07.313857+0000', 'last_fullsized': '2024-05-08T23:22:07.313857+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:22:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.310323+0000', 'last_change': '2024-05-08T23:22:07.310323+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.310323+0000', 'last_undegraded': '2024-05-08T23:22:07.310323+0000', 'last_fullsized': '2024-05-08T23:22:07.310323+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:09:31.969535+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.319446+0000', 'last_change': '2024-05-08T23:22:07.319446+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.319446+0000', 'last_undegraded': '2024-05-08T23:22:07.319446+0000', 'last_fullsized': '2024-05-08T23:22:07.319446+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309539+0000', 'last_change': '2024-05-08T23:22:07.309539+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309539+0000', 'last_undegraded': '2024-05-08T23:22:07.309539+0000', 'last_fullsized': '2024-05-08T23:22:07.309539+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309807+0000', 'last_change': '2024-05-08T23:22:07.309807+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309807+0000', 'last_undegraded': '2024-05-08T23:22:07.309807+0000', 'last_fullsized': '2024-05-08T23:22:07.309807+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:09:47.852979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.313480+0000', 'last_change': '2024-05-08T23:22:07.313480+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313480+0000', 'last_undegraded': '2024-05-08T23:22:07.313480+0000', 'last_fullsized': '2024-05-08T23:22:07.313480+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:36:29.054724+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': ['0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T23:22:12.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-05-08T23:22:12.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309930+0000', 'last_change': '2024-05-08T23:22:07.309930+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309930+0000', 'last_undegraded': '2024-05-08T23:22:07.309930+0000', 'last_fullsized': '2024-05-08T23:22:07.309930+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T06:01:15.611951+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2024-05-08T23:22:12.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.313374+0000', 'last_change': '2024-05-08T23:22:07.313374+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313374+0000', 'last_undegraded': '2024-05-08T23:22:07.313374+0000', 'last_fullsized': '2024-05-08T23:22:07.313374+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': ['6', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:22:12.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-05-08T23:22:12.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309225+0000', 'last_change': '2024-05-08T23:22:07.309225+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309225+0000', 'last_undegraded': '2024-05-08T23:22:07.309225+0000', 'last_fullsized': '2024-05-08T23:22:07.309225+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:54:45.661099+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-05-08T23:22:12.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.319698+0000', 'last_change': '2024-05-08T23:22:07.319698+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.319698+0000', 'last_undegraded': '2024-05-08T23:22:07.319698+0000', 'last_fullsized': '2024-05-08T23:22:07.319698+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.308777+0000', 'last_change': '2024-05-08T23:22:07.308777+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.308777+0000', 'last_undegraded': '2024-05-08T23:22:07.308777+0000', 'last_fullsized': '2024-05-08T23:22:07.308777+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:05:04.490589+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309538+0000', 'last_change': '2024-05-08T23:22:07.309538+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309538+0000', 'last_undegraded': '2024-05-08T23:22:07.309538+0000', 'last_fullsized': '2024-05-08T23:22:07.309538+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:44:02.041531+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.316598+0000', 'last_change': '2024-05-08T23:22:07.316598+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.316598+0000', 'last_undegraded': '2024-05-08T23:22:07.316598+0000', 'last_fullsized': '2024-05-08T23:22:07.316598+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.310978+0000', 'last_change': '2024-05-08T23:22:07.310978+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.310978+0000', 'last_undegraded': '2024-05-08T23:22:07.310978+0000', 'last_fullsized': '2024-05-08T23:22:07.310978+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2024-05-08T23:22:12.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.313873+0000', 'last_change': '2024-05-08T23:22:07.313873+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313873+0000', 'last_undegraded': '2024-05-08T23:22:07.313873+0000', 'last_fullsized': '2024-05-08T23:22:07.313873+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': ['0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.313198+0000', 'last_change': '2024-05-08T23:22:07.313198+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313198+0000', 'last_undegraded': '2024-05-08T23:22:07.313198+0000', 'last_fullsized': '2024-05-08T23:22:07.313198+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.311985+0000', 'last_change': '2024-05-08T23:22:07.311985+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.311985+0000', 'last_undegraded': '2024-05-08T23:22:07.311985+0000', 'last_fullsized': '2024-05-08T23:22:07.311985+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.314790+0000', 'last_change': '2024-05-08T23:22:07.314790+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.314790+0000', 'last_undegraded': '2024-05-08T23:22:07.314790+0000', 'last_fullsized': '2024-05-08T23:22:07.314790+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.316706+0000', 'last_change': '2024-05-08T23:22:07.316706+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.316706+0000', 'last_undegraded': '2024-05-08T23:22:07.316706+0000', 'last_fullsized': '2024-05-08T23:22:07.316706+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-05-08T23:22:12.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.309830+0000', 'last_change': '2024-05-08T23:22:07.309830+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.309830+0000', 'last_undegraded': '2024-05-08T23:22:07.309830+0000', 'last_fullsized': '2024-05-08T23:22:07.309830+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T06:48:55.199701+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:22:12.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-05-08T23:22:12.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.314416+0000', 'last_change': '2024-05-08T23:22:07.314416+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.314416+0000', 'last_undegraded': '2024-05-08T23:22:07.314416+0000', 'last_fullsized': '2024-05-08T23:22:07.314416+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:22:12.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-05-08T23:22:12.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:22:07.315394+0000', 'last_change': '2024-05-08T23:22:07.315394+0000', 'last_active': '2024-05-08T23:22:07.299109+0000', 'last_peered': '2024-05-08T23:22:07.299109+0000', 'last_clean': '2024-05-08T23:22:07.299109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.315394+0000', 'last_undegraded': '2024-05-08T23:22:07.315394+0000', 'last_fullsized': '2024-05-08T23:22:07.315394+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:07.299109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T23:22:12.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2024-05-08T23:22:12.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-05-08T23:22:07.313997+0000', 'last_change': '2024-05-08T23:22:07.313997+0000', 'last_active': '2024-05-08T23:22:06.294180+0000', 'last_peered': '2024-05-08T23:22:06.294180+0000', 'last_clean': '2024-05-08T23:22:06.294180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:22:07.313997+0000', 'last_undegraded': '2024-05-08T23:22:07.313997+0000', 'last_fullsized': '2024-05-08T23:22:07.313997+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'last_clean_scrub_stamp': '2024-05-08T23:22:06.294180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:22:12.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:22:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.057+0000 7fb26ea82700 1 Processor -- start 2024-05-08T23:22:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.057+0000 7fb26ea82700 1 -- start start 2024-05-08T23:22:13.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.057+0000 7fb26ea82700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2680b2340 0x7fb2680cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.057+0000 7fb26ea82700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb2680b4620 0x7fb2680b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.057+0000 7fb26ea82700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26809ad40 con 0x7fb26815ef60 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.057+0000 7fb26ea82700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2680cd400 con 0x7fb2680b2340 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.057+0000 7fb26ea82700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2680ccb60 con 0x7fb2680b4620 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb267fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2680b2340 0x7fb2680cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb26c81e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb26815ef60 0x7fb2680cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43890/0 (socket says 172.21.15.5:43890) 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb26c81e700 1 -- 172.21.15.5:0/1629037589 learned_addr learned my addr 172.21.15.5:0/1629037589 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb26d01f700 1 --2- 172.21.15.5:0/1629037589 >> v2:172.21.15.49:3300/0 conn(0x7fb2680b4620 0x7fb2680b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb267fff700 1 -- 172.21.15.5:0/1629037589 >> v2:172.21.15.49:3300/0 conn(0x7fb2680b4620 msgr2=0x7fb2680b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb267fff700 1 --2- 172.21.15.5:0/1629037589 >> v2:172.21.15.49:3300/0 conn(0x7fb2680b4620 0x7fb2680b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb267fff700 1 -- 172.21.15.5:0/1629037589 >> v1:172.21.15.5:6789/0 conn(0x7fb26815ef60 legacy=0x7fb2680cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb267fff700 1 -- 172.21.15.5:0/1629037589 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2680cce40 con 0x7fb2680b2340 2024-05-08T23:22:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb2677fe700 1 -- 172.21.15.5:0/1629037589 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430114110 0 0) 0x7fb26809ad40 con 0x7fb26815ef60 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb267fff700 1 --2- 172.21.15.5:0/1629037589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b2340 0x7fb2680cf710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb268072340 tx=0x7fb25c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68732ada2039a466 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb2677fe700 1 -- 172.21.15.5:0/1629037589 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c011830 con 0x7fb2680b2340 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb2677fe700 1 -- 172.21.15.5:0/1629037589 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c011990 con 0x7fb2680b2340 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.058+0000 7fb2677fe700 1 -- 172.21.15.5:0/1629037589 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c011d20 con 0x7fb2680b2340 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.059+0000 7fb26ea82700 1 -- 172.21.15.5:0/1629037589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b2340 msgr2=0x7fb2680cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.059+0000 7fb26ea82700 1 --2- 172.21.15.5:0/1629037589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b2340 0x7fb2680cf710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb268072340 tx=0x7fb25c009b20 comp rx=0 tx=0).stop 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.059+0000 7fb26ea82700 1 -- 172.21.15.5:0/1629037589 shutdown_connections 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.059+0000 7fb26ea82700 1 --2- 172.21.15.5:0/1629037589 >> v2:172.21.15.49:3300/0 conn(0x7fb2680b4620 0x7fb2680b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.059+0000 7fb26ea82700 1 --2- 172.21.15.5:0/1629037589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b2340 0x7fb2680cf710 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.059+0000 7fb26ea82700 1 -- 172.21.15.5:0/1629037589 >> 172.21.15.5:0/1629037589 conn(0x7fb2680b2f20 msgr2=0x7fb2680b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.059+0000 7fb26ea82700 1 -- 172.21.15.5:0/1629037589 shutdown_connections 2024-05-08T23:22:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.059+0000 7fb26ea82700 1 -- 172.21.15.5:0/1629037589 wait complete. 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.060+0000 7fb26ea82700 1 Processor -- start 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.060+0000 7fb26ea82700 1 -- start start 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb26ea82700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2680b2340 0x7fb268157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb26ea82700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b4620 0x7fb2680c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb26ea82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26815ef60 0x7fb2680ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb26ea82700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2680ae0e0 con 0x7fb2680b4620 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb26ea82700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2680aec60 con 0x7fb26815ef60 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb26ea82700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2680ae330 con 0x7fb2680b2340 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b4620 0x7fb2680c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b4620 0x7fb2680c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47230/0 (socket says 172.21.15.5:47230) 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 -- 172.21.15.5:0/3668288119 learned_addr learned my addr 172.21.15.5:0/3668288119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb26c81e700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2680b2340 0x7fb268157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb26d01f700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26815ef60 0x7fb2680ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 -- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2680b2340 msgr2=0x7fb268157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2680b2340 0x7fb268157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 -- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26815ef60 msgr2=0x7fb2680ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26815ef60 0x7fb2680ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 -- 172.21.15.5:0/3668288119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c0094a0 con 0x7fb2680b4620 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb267fff700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b4620 0x7fb2680c9f50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0090b0 tx=0x7fb25c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.062+0000 7fb2657fa700 1 -- 172.21.15.5:0/3668288119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0036a0 con 0x7fb2680b4620 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.063+0000 7fb2657fa700 1 -- 172.21.15.5:0/3668288119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c01e070 con 0x7fb2680b4620 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.063+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2680ae610 con 0x7fb2680b4620 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.063+0000 7fb2657fa700 1 -- 172.21.15.5:0/3668288119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c003a60 con 0x7fb2680b4620 2024-05-08T23:22:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.063+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb268158f20 con 0x7fb2680b4620 2024-05-08T23:22:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.064+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2680aedf0 con 0x7fb2680b4620 2024-05-08T23:22:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.064+0000 7fb2657fa700 1 -- 172.21.15.5:0/3668288119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb25c003bc0 con 0x7fb2680b4620 2024-05-08T23:22:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.064+0000 7fb2657fa700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb25406ac90 0x7fb25406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.065+0000 7fb2657fa700 1 -- 172.21.15.5:0/3668288119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb25c08a1f0 con 0x7fb2680b4620 2024-05-08T23:22:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.065+0000 7fb26c81e700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb25406ac90 0x7fb25406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.065+0000 7fb26c81e700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb25406ac90 0x7fb25406d150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb258000f60 tx=0x7fb258007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:13.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.069+0000 7fb2657fa700 1 -- 172.21.15.5:0/3668288119 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb25c0553e0 con 0x7fb2680b4620 2024-05-08T23:22:13.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.378+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb268072f10 con 0x7fb2680b4620 2024-05-08T23:22:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.378+0000 7fb2657fa700 1 -- 172.21.15.5:0/3668288119 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1240 (secure 0 0 0) 0x7fb25c058a00 con 0x7fb2680b4620 2024-05-08T23:22:13.379 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:13.379 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":88,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":21},{"state_name":"creating+activating","count":12}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225320960,"bytes_avail":772868792320,"bytes_total":773094113280,"inactive_pgs_ratio":0.7835051417350769},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":4975}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-05-08T23:21:59.307051+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:22:13.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb25406ac90 msgr2=0x7fb25406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb25406ac90 0x7fb25406d150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb258000f60 tx=0x7fb258007e70 comp rx=0 tx=0).stop 2024-05-08T23:22:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b4620 msgr2=0x7fb2680c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b4620 0x7fb2680c9f50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0090b0 tx=0x7fb25c004750 comp rx=0 tx=0).stop 2024-05-08T23:22:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 shutdown_connections 2024-05-08T23:22:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb25406ac90 0x7fb25406d150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb26815ef60 0x7fb2680ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb2680b2340 0x7fb268157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 --2- 172.21.15.5:0/3668288119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2680b4620 0x7fb2680c9f50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 >> 172.21.15.5:0/3668288119 conn(0x7fb2680b2f20 msgr2=0x7fb2680b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 shutdown_connections 2024-05-08T23:22:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.381+0000 7fb26ea82700 1 -- 172.21.15.5:0/3668288119 wait complete. 2024-05-08T23:22:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T23:22:13.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:22:13.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.532+0000 7f9de9524700 1 Processor -- start 2024-05-08T23:22:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.532+0000 7f9de9524700 1 -- start start 2024-05-08T23:22:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de9524700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de415db10 0x7f9de40d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de9524700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de406c400 con 0x7f9de40cbb00 2024-05-08T23:22:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de9524700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de40d0cd0 con 0x7f9de40b2700 2024-05-08T23:22:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de9524700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40d09f0 con 0x7f9de415db10 2024-05-08T23:22:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9de40cbb00 0x7f9de41617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43894/0 (socket says 172.21.15.5:43894) 2024-05-08T23:22:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de37fe700 1 -- 172.21.15.5:0/3634388391 learned_addr learned my addr 172.21.15.5:0/3634388391 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de2ffd700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de415db10 0x7f9de40d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616778155 0 0) 0x7f9de406c400 con 0x7f9de40cbb00 2024-05-08T23:22:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.533+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd00035b0 con 0x7f9de40cbb00 2024-05-08T23:22:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736037666 0 0) 0x7f9de40d0cd0 con 0x7f9de40b2700 2024-05-08T23:22:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de406c400 con 0x7f9de40b2700 2024-05-08T23:22:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3026431315 0 0) 0x7f9dd00035b0 con 0x7f9de40cbb00 2024-05-08T23:22:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de415db10 msgr2=0x7f9de40d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de415db10 0x7f9de40d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 >> v1:172.21.15.49:6789/0 conn(0x7f9de40b2700 legacy=0x7f9de406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de406bdb0 con 0x7f9de40cbb00 2024-05-08T23:22:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f9ddc0019f0 con 0x7f9de40cbb00 2024-05-08T23:22:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 >> v1:172.21.15.5:6789/0 conn(0x7f9de40cbb00 legacy=0x7f9de41617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de415db10 0x7f9dd0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd0004170 0x7f9dd0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd0004d90 0x7f9dd0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40d09f0 con 0x7f9dd0004d90 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd000bbd0 con 0x7f9dd0004170 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd000bd90 con 0x7f9de415db10 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de37fe700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd0004d90 0x7f9dd0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de27fc700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd0004170 0x7f9dd0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de2ffd700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de415db10 0x7f9dd0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de37fe700 1 -- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de415db10 msgr2=0x7f9dd0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de37fe700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de415db10 0x7f9dd0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de37fe700 1 -- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd0004170 msgr2=0x7f9dd0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de37fe700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd0004170 0x7f9dd0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.534+0000 7f9de37fe700 1 -- 172.21.15.5:0/3634388391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd00035b0 con 0x7f9dd0004d90 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de37fe700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd0004d90 0x7f9dd0005210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00a710 tx=0x7f9dd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f92f26792860e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0072d0 con 0x7f9dd0004d90 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc007430 con 0x7f9dd0004d90 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de1ffb700 1 -- 172.21.15.5:0/3634388391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0077e0 con 0x7f9dd0004d90 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de9524700 1 -- 172.21.15.5:0/3634388391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd0004d90 msgr2=0x7f9dd0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de9524700 1 --2- 172.21.15.5:0/3634388391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd0004d90 0x7f9dd0005210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00a710 tx=0x7f9dd0005880 comp rx=0 tx=0).stop 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de37fe700 1 -- 172.21.15.5:0/3634388391 reap_dead start 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de9524700 1 -- 172.21.15.5:0/3634388391 shutdown_connections 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de9524700 1 -- 172.21.15.5:0/3634388391 >> 172.21.15.5:0/3634388391 conn(0x7f9de40b2fa0 msgr2=0x7f9de40b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de9524700 1 -- 172.21.15.5:0/3634388391 shutdown_connections 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.535+0000 7f9de9524700 1 -- 172.21.15.5:0/3634388391 wait complete. 2024-05-08T23:22:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.537+0000 7f9de9524700 1 Processor -- start 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.537+0000 7f9de9524700 1 -- start start 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de9524700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de41547c0 0x7f9de40c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de9524700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de40c6030 0x7f9de40cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de9524700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de40ce270 0x7f9de41f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de9524700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40c98e0 con 0x7f9de40c6030 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de9524700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40ce6f0 con 0x7f9de41547c0 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de9524700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de41551a0 con 0x7f9de40ce270 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de40c6030 0x7f9de40cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de40c6030 0x7f9de40cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47244/0 (socket says 172.21.15.5:47244) 2024-05-08T23:22:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 -- 172.21.15.5:0/2485985023 learned_addr learned my addr 172.21.15.5:0/2485985023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de2ffd700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de41547c0 0x7f9de40c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de37fe700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de40ce270 0x7f9de41f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 -- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de40ce270 msgr2=0x7f9de41f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de40ce270 0x7f9de41f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 -- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de41547c0 msgr2=0x7f9de40c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de41547c0 0x7f9de40c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 -- 172.21.15.5:0/2485985023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40c8c60 con 0x7f9de40c6030 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9de27fc700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de40c6030 0x7f9de40cdd30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00f870 tx=0x7f9dcc00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.538+0000 7f9ddb7fe700 1 -- 172.21.15.5:0/2485985023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00fe70 con 0x7f9de40c6030 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.539+0000 7f9ddb7fe700 1 -- 172.21.15.5:0/2485985023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dcc0052e0 con 0x7f9de40c6030 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.539+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de40c8f40 con 0x7f9de40c6030 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.539+0000 7f9ddb7fe700 1 -- 172.21.15.5:0/2485985023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc003680 con 0x7f9de40c6030 2024-05-08T23:22:13.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.539+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de40c9500 con 0x7f9de40c6030 2024-05-08T23:22:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.540+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de40c96c0 con 0x7f9de40c6030 2024-05-08T23:22:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.540+0000 7f9ddb7fe700 1 -- 172.21.15.5:0/2485985023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9dcc0037e0 con 0x7f9de40c6030 2024-05-08T23:22:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.541+0000 7f9ddb7fe700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9dc406ac90 0x7f9dc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.541+0000 7f9ddb7fe700 1 -- 172.21.15.5:0/2485985023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dcc089420 con 0x7f9de40c6030 2024-05-08T23:22:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.541+0000 7f9de2ffd700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9dc406ac90 0x7f9dc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.542+0000 7f9de2ffd700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9dc406ac90 0x7f9dc406d150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009ba0 tx=0x7f9dd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.546+0000 7f9ddb7fe700 1 -- 172.21.15.5:0/2485985023 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9dcc054560 con 0x7f9de40c6030 2024-05-08T23:22:13.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.755+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9de40d1170 con 0x7f9dc406ac90 2024-05-08T23:22:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.764+0000 7f9ddb7fe700 1 -- 172.21.15.5:0/2485985023 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289282 (secure 0 0 0) 0x7f9de40d1170 con 0x7f9dc406ac90 2024-05-08T23:22:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9dc406ac90 msgr2=0x7f9dc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9dc406ac90 0x7f9dc406d150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009ba0 tx=0x7f9dd4008040 comp rx=0 tx=0).stop 2024-05-08T23:22:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de40c6030 msgr2=0x7f9de40cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de40c6030 0x7f9de40cdd30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00f870 tx=0x7f9dcc00fc30 comp rx=0 tx=0).stop 2024-05-08T23:22:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 shutdown_connections 2024-05-08T23:22:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9dc406ac90 0x7f9dc406d150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de40ce270 0x7f9de41f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de41547c0 0x7f9de40c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 --2- 172.21.15.5:0/2485985023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de40c6030 0x7f9de40cdd30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 >> 172.21.15.5:0/2485985023 conn(0x7f9de40b2fa0 msgr2=0x7f9de4099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 shutdown_connections 2024-05-08T23:22:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:13.768+0000 7f9de9524700 1 -- 172.21.15.5:0/2485985023 wait complete. 2024-05-08T23:22:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:22:13.782 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-05-08T23:22:13.310306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220456,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754264,"statfs":{"total":773094113280,"available":772868366336,"internally_reserved":0,"allocated":4333568,"data_stored":2401980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004224"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.795731+0000","last_change":"2024-05-08T23:22:07.792801+0000","last_active":"2024-05-08T23:22:07.795731+0000","last_peered":"2024-05-08T23:22:07.795731+0000","last_clean":"2024-05-08T23:22:07.795731+0000","last_became_active":"2024-05-08T23:22:07.792474+0000","last_became_peered":"2024-05-08T23:22:07.792474+0000","last_unstale":"2024-05-08T23:22:07.795731+0000","last_undegraded":"2024-05-08T23:22:07.795731+0000","last_fullsized":"2024-05-08T23:22:07.795731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799412+0000","last_change":"2024-05-08T23:22:07.794320+0000","last_active":"2024-05-08T23:22:07.799412+0000","last_peered":"2024-05-08T23:22:07.799412+0000","last_clean":"2024-05-08T23:22:07.799412+0000","last_became_active":"2024-05-08T23:22:07.789420+0000","last_became_peered":"2024-05-08T23:22:07.789420+0000","last_unstale":"2024-05-08T23:22:07.799412+0000","last_undegraded":"2024-05-08T23:22:07.799412+0000","last_fullsized":"2024-05-08T23:22:07.799412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:58.867905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799267+0000","last_change":"2024-05-08T23:22:07.798266+0000","last_active":"2024-05-08T23:22:07.799267+0000","last_peered":"2024-05-08T23:22:07.799267+0000","last_clean":"2024-05-08T23:22:07.799267+0000","last_became_active":"2024-05-08T23:22:07.797980+0000","last_became_peered":"2024-05-08T23:22:07.797980+0000","last_unstale":"2024-05-08T23:22:07.799267+0000","last_undegraded":"2024-05-08T23:22:07.799267+0000","last_fullsized":"2024-05-08T23:22:07.799267+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:46.755007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.784150+0000","last_change":"2024-05-08T23:22:07.781843+0000","last_active":"2024-05-08T23:22:07.784150+0000","last_peered":"2024-05-08T23:22:07.784150+0000","last_clean":"2024-05-08T23:22:07.784150+0000","last_became_active":"2024-05-08T23:22:07.781605+0000","last_became_peered":"2024-05-08T23:22:07.781605+0000","last_unstale":"2024-05-08T23:22:07.784150+0000","last_undegraded":"2024-05-08T23:22:07.784150+0000","last_fullsized":"2024-05-08T23:22:07.784150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797237+0000","last_change":"2024-05-08T23:22:07.795484+0000","last_active":"2024-05-08T23:22:07.797237+0000","last_peered":"2024-05-08T23:22:07.797237+0000","last_clean":"2024-05-08T23:22:07.797237+0000","last_became_active":"2024-05-08T23:22:07.795009+0000","last_became_peered":"2024-05-08T23:22:07.795009+0000","last_unstale":"2024-05-08T23:22:07.797237+0000","last_undegraded":"2024-05-08T23:22:07.797237+0000","last_fullsized":"2024-05-08T23:22:07.797237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:27.657029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.784296+0000","last_change":"2024-05-08T23:22:07.782730+0000","last_active":"2024-05-08T23:22:07.784296+0000","last_peered":"2024-05-08T23:22:07.784296+0000","last_clean":"2024-05-08T23:22:07.784296+0000","last_became_active":"2024-05-08T23:22:07.782475+0000","last_became_peered":"2024-05-08T23:22:07.782475+0000","last_unstale":"2024-05-08T23:22:07.784296+0000","last_undegraded":"2024-05-08T23:22:07.784296+0000","last_fullsized":"2024-05-08T23:22:07.784296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:10.804990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799764+0000","last_change":"2024-05-08T23:22:07.797086+0000","last_active":"2024-05-08T23:22:07.799764+0000","last_peered":"2024-05-08T23:22:07.799764+0000","last_clean":"2024-05-08T23:22:07.799764+0000","last_became_active":"2024-05-08T23:22:07.796747+0000","last_became_peered":"2024-05-08T23:22:07.796747+0000","last_unstale":"2024-05-08T23:22:07.799764+0000","last_undegraded":"2024-05-08T23:22:07.799764+0000","last_fullsized":"2024-05-08T23:22:07.799764+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:31.969535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796858+0000","last_change":"2024-05-08T23:22:07.791736+0000","last_active":"2024-05-08T23:22:07.796858+0000","last_peered":"2024-05-08T23:22:07.796858+0000","last_clean":"2024-05-08T23:22:07.796858+0000","last_became_active":"2024-05-08T23:22:07.791442+0000","last_became_peered":"2024-05-08T23:22:07.791442+0000","last_unstale":"2024-05-08T23:22:07.796858+0000","last_undegraded":"2024-05-08T23:22:07.796858+0000","last_fullsized":"2024-05-08T23:22:07.796858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:41.329227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792892+0000","last_change":"2024-05-08T23:22:07.784891+0000","last_active":"2024-05-08T23:22:07.792892+0000","last_peered":"2024-05-08T23:22:07.792892+0000","last_clean":"2024-05-08T23:22:07.792892+0000","last_became_active":"2024-05-08T23:22:07.784603+0000","last_became_peered":"2024-05-08T23:22:07.784603+0000","last_unstale":"2024-05-08T23:22:07.792892+0000","last_undegraded":"2024-05-08T23:22:07.792892+0000","last_fullsized":"2024-05-08T23:22:07.792892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797161+0000","last_change":"2024-05-08T23:22:07.786491+0000","last_active":"2024-05-08T23:22:07.797161+0000","last_peered":"2024-05-08T23:22:07.797161+0000","last_clean":"2024-05-08T23:22:07.797161+0000","last_became_active":"2024-05-08T23:22:07.786199+0000","last_became_peered":"2024-05-08T23:22:07.786199+0000","last_unstale":"2024-05-08T23:22:07.797161+0000","last_undegraded":"2024-05-08T23:22:07.797161+0000","last_fullsized":"2024-05-08T23:22:07.797161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797173+0000","last_change":"2024-05-08T23:22:07.794506+0000","last_active":"2024-05-08T23:22:07.797173+0000","last_peered":"2024-05-08T23:22:07.797173+0000","last_clean":"2024-05-08T23:22:07.797173+0000","last_became_active":"2024-05-08T23:22:07.794173+0000","last_became_peered":"2024-05-08T23:22:07.794173+0000","last_unstale":"2024-05-08T23:22:07.797173+0000","last_undegraded":"2024-05-08T23:22:07.797173+0000","last_fullsized":"2024-05-08T23:22:07.797173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.788628+0000","last_change":"2024-05-08T23:22:07.787087+0000","last_active":"2024-05-08T23:22:07.788628+0000","last_peered":"2024-05-08T23:22:07.788628+0000","last_clean":"2024-05-08T23:22:07.788628+0000","last_became_active":"2024-05-08T23:22:07.786743+0000","last_became_peered":"2024-05-08T23:22:07.786743+0000","last_unstale":"2024-05-08T23:22:07.788628+0000","last_undegraded":"2024-05-08T23:22:07.788628+0000","last_fullsized":"2024-05-08T23:22:07.788628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:42.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.786115+0000","last_change":"2024-05-08T23:22:07.784216+0000","last_active":"2024-05-08T23:22:07.786115+0000","last_peered":"2024-05-08T23:22:07.786115+0000","last_clean":"2024-05-08T23:22:07.786115+0000","last_became_active":"2024-05-08T23:22:07.783893+0000","last_became_peered":"2024-05-08T23:22:07.783893+0000","last_unstale":"2024-05-08T23:22:07.786115+0000","last_undegraded":"2024-05-08T23:22:07.786115+0000","last_fullsized":"2024-05-08T23:22:07.786115+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:38.296628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.794776+0000","last_change":"2024-05-08T23:22:07.787696+0000","last_active":"2024-05-08T23:22:07.794776+0000","last_peered":"2024-05-08T23:22:07.794776+0000","last_clean":"2024-05-08T23:22:07.794776+0000","last_became_active":"2024-05-08T23:22:07.784113+0000","last_became_peered":"2024-05-08T23:22:07.784113+0000","last_unstale":"2024-05-08T23:22:07.794776+0000","last_undegraded":"2024-05-08T23:22:07.794776+0000","last_fullsized":"2024-05-08T23:22:07.794776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:37.867491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792227+0000","last_change":"2024-05-08T23:22:07.790230+0000","last_active":"2024-05-08T23:22:07.792227+0000","last_peered":"2024-05-08T23:22:07.792227+0000","last_clean":"2024-05-08T23:22:07.792227+0000","last_became_active":"2024-05-08T23:22:07.789926+0000","last_became_peered":"2024-05-08T23:22:07.789926+0000","last_unstale":"2024-05-08T23:22:07.792227+0000","last_undegraded":"2024-05-08T23:22:07.792227+0000","last_fullsized":"2024-05-08T23:22:07.792227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:48.075638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799863+0000","last_change":"2024-05-08T23:22:07.798602+0000","last_active":"2024-05-08T23:22:07.799863+0000","last_peered":"2024-05-08T23:22:07.799863+0000","last_clean":"2024-05-08T23:22:07.799863+0000","last_became_active":"2024-05-08T23:22:07.797408+0000","last_became_peered":"2024-05-08T23:22:07.797408+0000","last_unstale":"2024-05-08T23:22:07.799863+0000","last_undegraded":"2024-05-08T23:22:07.799863+0000","last_fullsized":"2024-05-08T23:22:07.799863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:50.357763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797023+0000","last_change":"2024-05-08T23:22:07.793949+0000","last_active":"2024-05-08T23:22:07.797023+0000","last_peered":"2024-05-08T23:22:07.797023+0000","last_clean":"2024-05-08T23:22:07.797023+0000","last_became_active":"2024-05-08T23:22:07.793549+0000","last_became_peered":"2024-05-08T23:22:07.793549+0000","last_unstale":"2024-05-08T23:22:07.797023+0000","last_undegraded":"2024-05-08T23:22:07.797023+0000","last_fullsized":"2024-05-08T23:22:07.797023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799694+0000","last_change":"2024-05-08T23:22:07.796230+0000","last_active":"2024-05-08T23:22:07.799694+0000","last_peered":"2024-05-08T23:22:07.799694+0000","last_clean":"2024-05-08T23:22:07.799694+0000","last_became_active":"2024-05-08T23:22:07.796019+0000","last_became_peered":"2024-05-08T23:22:07.796019+0000","last_unstale":"2024-05-08T23:22:07.799694+0000","last_undegraded":"2024-05-08T23:22:07.799694+0000","last_fullsized":"2024-05-08T23:22:07.799694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:44.317582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799206+0000","last_change":"2024-05-08T23:22:07.794870+0000","last_active":"2024-05-08T23:22:07.799206+0000","last_peered":"2024-05-08T23:22:07.799206+0000","last_clean":"2024-05-08T23:22:07.799206+0000","last_became_active":"2024-05-08T23:22:07.794071+0000","last_became_peered":"2024-05-08T23:22:07.794071+0000","last_unstale":"2024-05-08T23:22:07.799206+0000","last_undegraded":"2024-05-08T23:22:07.799206+0000","last_fullsized":"2024-05-08T23:22:07.799206+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:20.807790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.790329+0000","last_change":"2024-05-08T23:22:07.789391+0000","last_active":"2024-05-08T23:22:07.790329+0000","last_peered":"2024-05-08T23:22:07.790329+0000","last_clean":"2024-05-08T23:22:07.790329+0000","last_became_active":"2024-05-08T23:22:07.789123+0000","last_became_peered":"2024-05-08T23:22:07.789123+0000","last_unstale":"2024-05-08T23:22:07.790329+0000","last_undegraded":"2024-05-08T23:22:07.790329+0000","last_fullsized":"2024-05-08T23:22:07.790329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:30.059653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796448+0000","last_change":"2024-05-08T23:22:07.791305+0000","last_active":"2024-05-08T23:22:07.796448+0000","last_peered":"2024-05-08T23:22:07.796448+0000","last_clean":"2024-05-08T23:22:07.796448+0000","last_became_active":"2024-05-08T23:22:07.789926+0000","last_became_peered":"2024-05-08T23:22:07.789926+0000","last_unstale":"2024-05-08T23:22:07.796448+0000","last_undegraded":"2024-05-08T23:22:07.796448+0000","last_fullsized":"2024-05-08T23:22:07.796448+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.793227+0000","last_change":"2024-05-08T23:22:07.785342+0000","last_active":"2024-05-08T23:22:07.793227+0000","last_peered":"2024-05-08T23:22:07.793227+0000","last_clean":"2024-05-08T23:22:07.793227+0000","last_became_active":"2024-05-08T23:22:07.785016+0000","last_became_peered":"2024-05-08T23:22:07.785016+0000","last_unstale":"2024-05-08T23:22:07.793227+0000","last_undegraded":"2024-05-08T23:22:07.793227+0000","last_fullsized":"2024-05-08T23:22:07.793227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.798868+0000","last_change":"2024-05-08T23:22:07.795399+0000","last_active":"2024-05-08T23:22:07.798868+0000","last_peered":"2024-05-08T23:22:07.798868+0000","last_clean":"2024-05-08T23:22:07.798868+0000","last_became_active":"2024-05-08T23:22:07.795120+0000","last_became_peered":"2024-05-08T23:22:07.795120+0000","last_unstale":"2024-05-08T23:22:07.798868+0000","last_undegraded":"2024-05-08T23:22:07.798868+0000","last_fullsized":"2024-05-08T23:22:07.798868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.788154+0000","last_change":"2024-05-08T23:22:07.785217+0000","last_active":"2024-05-08T23:22:07.788154+0000","last_peered":"2024-05-08T23:22:07.788154+0000","last_clean":"2024-05-08T23:22:07.788154+0000","last_became_active":"2024-05-08T23:22:07.784936+0000","last_became_peered":"2024-05-08T23:22:07.784936+0000","last_unstale":"2024-05-08T23:22:07.788154+0000","last_undegraded":"2024-05-08T23:22:07.788154+0000","last_fullsized":"2024-05-08T23:22:07.788154+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799679+0000","last_change":"2024-05-08T23:22:07.792957+0000","last_active":"2024-05-08T23:22:07.799679+0000","last_peered":"2024-05-08T23:22:07.799679+0000","last_clean":"2024-05-08T23:22:07.799679+0000","last_became_active":"2024-05-08T23:22:07.792691+0000","last_became_peered":"2024-05-08T23:22:07.792691+0000","last_unstale":"2024-05-08T23:22:07.799679+0000","last_undegraded":"2024-05-08T23:22:07.799679+0000","last_fullsized":"2024-05-08T23:22:07.799679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799713+0000","last_change":"2024-05-08T23:22:07.796720+0000","last_active":"2024-05-08T23:22:07.799713+0000","last_peered":"2024-05-08T23:22:07.799713+0000","last_clean":"2024-05-08T23:22:07.799713+0000","last_became_active":"2024-05-08T23:22:07.796359+0000","last_became_peered":"2024-05-08T23:22:07.796359+0000","last_unstale":"2024-05-08T23:22:07.799713+0000","last_undegraded":"2024-05-08T23:22:07.799713+0000","last_fullsized":"2024-05-08T23:22:07.799713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:49.810291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799658+0000","last_change":"2024-05-08T23:22:07.791923+0000","last_active":"2024-05-08T23:22:07.799658+0000","last_peered":"2024-05-08T23:22:07.799658+0000","last_clean":"2024-05-08T23:22:07.799658+0000","last_became_active":"2024-05-08T23:22:07.791650+0000","last_became_peered":"2024-05-08T23:22:07.791650+0000","last_unstale":"2024-05-08T23:22:07.799658+0000","last_undegraded":"2024-05-08T23:22:07.799658+0000","last_fullsized":"2024-05-08T23:22:07.799658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.787966+0000","last_change":"2024-05-08T23:22:07.786214+0000","last_active":"2024-05-08T23:22:07.787966+0000","last_peered":"2024-05-08T23:22:07.787966+0000","last_clean":"2024-05-08T23:22:07.787966+0000","last_became_active":"2024-05-08T23:22:07.783664+0000","last_became_peered":"2024-05-08T23:22:07.783664+0000","last_unstale":"2024-05-08T23:22:07.787966+0000","last_undegraded":"2024-05-08T23:22:07.787966+0000","last_fullsized":"2024-05-08T23:22:07.787966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:09.803894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796424+0000","last_change":"2024-05-08T23:22:07.791114+0000","last_active":"2024-05-08T23:22:07.796424+0000","last_peered":"2024-05-08T23:22:07.796424+0000","last_clean":"2024-05-08T23:22:07.796424+0000","last_became_active":"2024-05-08T23:22:07.789579+0000","last_became_peered":"2024-05-08T23:22:07.789579+0000","last_unstale":"2024-05-08T23:22:07.796424+0000","last_undegraded":"2024-05-08T23:22:07.796424+0000","last_fullsized":"2024-05-08T23:22:07.796424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796498+0000","last_change":"2024-05-08T23:22:07.793957+0000","last_active":"2024-05-08T23:22:07.796498+0000","last_peered":"2024-05-08T23:22:07.796498+0000","last_clean":"2024-05-08T23:22:07.796498+0000","last_became_active":"2024-05-08T23:22:07.793638+0000","last_became_peered":"2024-05-08T23:22:07.793638+0000","last_unstale":"2024-05-08T23:22:07.796498+0000","last_undegraded":"2024-05-08T23:22:07.796498+0000","last_fullsized":"2024-05-08T23:22:07.796498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:42.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.793350+0000","last_change":"2024-05-08T23:22:07.788892+0000","last_active":"2024-05-08T23:22:07.793350+0000","last_peered":"2024-05-08T23:22:07.793350+0000","last_clean":"2024-05-08T23:22:07.793350+0000","last_became_active":"2024-05-08T23:22:07.784706+0000","last_became_peered":"2024-05-08T23:22:07.784706+0000","last_unstale":"2024-05-08T23:22:07.793350+0000","last_undegraded":"2024-05-08T23:22:07.793350+0000","last_fullsized":"2024-05-08T23:22:07.793350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:05.495518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799845+0000","last_change":"2024-05-08T23:22:07.797424+0000","last_active":"2024-05-08T23:22:07.799845+0000","last_peered":"2024-05-08T23:22:07.799845+0000","last_clean":"2024-05-08T23:22:07.799845+0000","last_became_active":"2024-05-08T23:22:07.797107+0000","last_became_peered":"2024-05-08T23:22:07.797107+0000","last_unstale":"2024-05-08T23:22:07.799845+0000","last_undegraded":"2024-05-08T23:22:07.799845+0000","last_fullsized":"2024-05-08T23:22:07.799845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:46.755007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797793+0000","last_change":"2024-05-08T23:22:07.795811+0000","last_active":"2024-05-08T23:22:07.797793+0000","last_peered":"2024-05-08T23:22:07.797793+0000","last_clean":"2024-05-08T23:22:07.797793+0000","last_became_active":"2024-05-08T23:22:07.795559+0000","last_became_peered":"2024-05-08T23:22:07.795559+0000","last_unstale":"2024-05-08T23:22:07.797793+0000","last_undegraded":"2024-05-08T23:22:07.797793+0000","last_fullsized":"2024-05-08T23:22:07.797793+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.794244+0000","last_change":"2024-05-08T23:22:07.321400+0000","last_active":"2024-05-08T23:22:07.794244+0000","last_peered":"2024-05-08T23:22:07.794244+0000","last_clean":"2024-05-08T23:22:07.794244+0000","last_became_active":"2024-05-08T23:22:07.321115+0000","last_became_peered":"2024-05-08T23:22:07.321115+0000","last_unstale":"2024-05-08T23:22:07.794244+0000","last_undegraded":"2024-05-08T23:22:07.794244+0000","last_fullsized":"2024-05-08T23:22:07.794244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.795752+0000","last_change":"2024-05-08T23:22:07.794410+0000","last_active":"2024-05-08T23:22:07.795752+0000","last_peered":"2024-05-08T23:22:07.795752+0000","last_clean":"2024-05-08T23:22:07.795752+0000","last_became_active":"2024-05-08T23:22:07.793894+0000","last_became_peered":"2024-05-08T23:22:07.793894+0000","last_unstale":"2024-05-08T23:22:07.795752+0000","last_undegraded":"2024-05-08T23:22:07.795752+0000","last_fullsized":"2024-05-08T23:22:07.795752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.795678+0000","last_change":"2024-05-08T23:22:07.325196+0000","last_active":"2024-05-08T23:22:07.795678+0000","last_peered":"2024-05-08T23:22:07.795678+0000","last_clean":"2024-05-08T23:22:07.795678+0000","last_became_active":"2024-05-08T23:22:07.324913+0000","last_became_peered":"2024-05-08T23:22:07.324913+0000","last_unstale":"2024-05-08T23:22:07.795678+0000","last_undegraded":"2024-05-08T23:22:07.795678+0000","last_fullsized":"2024-05-08T23:22:07.795678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:38.809279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792981+0000","last_change":"2024-05-08T23:22:07.785830+0000","last_active":"2024-05-08T23:22:07.792981+0000","last_peered":"2024-05-08T23:22:07.792981+0000","last_clean":"2024-05-08T23:22:07.792981+0000","last_became_active":"2024-05-08T23:22:07.785570+0000","last_became_peered":"2024-05-08T23:22:07.785570+0000","last_unstale":"2024-05-08T23:22:07.792981+0000","last_undegraded":"2024-05-08T23:22:07.792981+0000","last_fullsized":"2024-05-08T23:22:07.792981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799074+0000","last_change":"2024-05-08T23:22:07.323498+0000","last_active":"2024-05-08T23:22:07.799074+0000","last_peered":"2024-05-08T23:22:07.799074+0000","last_clean":"2024-05-08T23:22:07.799074+0000","last_became_active":"2024-05-08T23:22:07.323230+0000","last_became_peered":"2024-05-08T23:22:07.323230+0000","last_unstale":"2024-05-08T23:22:07.799074+0000","last_undegraded":"2024-05-08T23:22:07.799074+0000","last_fullsized":"2024-05-08T23:22:07.799074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:50.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799634+0000","last_change":"2024-05-08T23:22:07.791888+0000","last_active":"2024-05-08T23:22:07.799634+0000","last_peered":"2024-05-08T23:22:07.799634+0000","last_clean":"2024-05-08T23:22:07.799634+0000","last_became_active":"2024-05-08T23:22:07.791592+0000","last_became_peered":"2024-05-08T23:22:07.791592+0000","last_unstale":"2024-05-08T23:22:07.799634+0000","last_undegraded":"2024-05-08T23:22:07.799634+0000","last_fullsized":"2024-05-08T23:22:07.799634+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797507+0000","last_change":"2024-05-08T23:22:07.324367+0000","last_active":"2024-05-08T23:22:07.797507+0000","last_peered":"2024-05-08T23:22:07.797507+0000","last_clean":"2024-05-08T23:22:07.797507+0000","last_became_active":"2024-05-08T23:22:07.324085+0000","last_became_peered":"2024-05-08T23:22:07.324085+0000","last_unstale":"2024-05-08T23:22:07.797507+0000","last_undegraded":"2024-05-08T23:22:07.797507+0000","last_fullsized":"2024-05-08T23:22:07.797507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:40.830327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797185+0000","last_change":"2024-05-08T23:22:07.794556+0000","last_active":"2024-05-08T23:22:07.797185+0000","last_peered":"2024-05-08T23:22:07.797185+0000","last_clean":"2024-05-08T23:22:07.797185+0000","last_became_active":"2024-05-08T23:22:07.794233+0000","last_became_peered":"2024-05-08T23:22:07.794233+0000","last_unstale":"2024-05-08T23:22:07.797185+0000","last_undegraded":"2024-05-08T23:22:07.797185+0000","last_fullsized":"2024-05-08T23:22:07.797185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:48.075638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.793741+0000","last_change":"2024-05-08T23:22:07.324042+0000","last_active":"2024-05-08T23:22:07.793741+0000","last_peered":"2024-05-08T23:22:07.793741+0000","last_clean":"2024-05-08T23:22:07.793741+0000","last_became_active":"2024-05-08T23:22:07.323538+0000","last_became_peered":"2024-05-08T23:22:07.323538+0000","last_unstale":"2024-05-08T23:22:07.793741+0000","last_undegraded":"2024-05-08T23:22:07.793741+0000","last_fullsized":"2024-05-08T23:22:07.793741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:21.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.788559+0000","last_change":"2024-05-08T23:22:07.786473+0000","last_active":"2024-05-08T23:22:07.788559+0000","last_peered":"2024-05-08T23:22:07.788559+0000","last_clean":"2024-05-08T23:22:07.788559+0000","last_became_active":"2024-05-08T23:22:07.786188+0000","last_became_peered":"2024-05-08T23:22:07.786188+0000","last_unstale":"2024-05-08T23:22:07.788559+0000","last_undegraded":"2024-05-08T23:22:07.788559+0000","last_fullsized":"2024-05-08T23:22:07.788559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:50.357763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.790246+0000","last_change":"2024-05-08T23:22:07.323560+0000","last_active":"2024-05-08T23:22:07.790246+0000","last_peered":"2024-05-08T23:22:07.790246+0000","last_clean":"2024-05-08T23:22:07.790246+0000","last_became_active":"2024-05-08T23:22:07.323249+0000","last_became_peered":"2024-05-08T23:22:07.323249+0000","last_unstale":"2024-05-08T23:22:07.790246+0000","last_undegraded":"2024-05-08T23:22:07.790246+0000","last_fullsized":"2024-05-08T23:22:07.790246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:04.715359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.793070+0000","last_change":"2024-05-08T23:22:07.785147+0000","last_active":"2024-05-08T23:22:07.793070+0000","last_peered":"2024-05-08T23:22:07.793070+0000","last_clean":"2024-05-08T23:22:07.793070+0000","last_became_active":"2024-05-08T23:22:07.784830+0000","last_became_peered":"2024-05-08T23:22:07.784830+0000","last_unstale":"2024-05-08T23:22:07.793070+0000","last_undegraded":"2024-05-08T23:22:07.793070+0000","last_fullsized":"2024-05-08T23:22:07.793070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.776278+0000","last_change":"2024-05-08T23:22:07.320297+0000","last_active":"2024-05-08T23:22:07.776278+0000","last_peered":"2024-05-08T23:22:07.776278+0000","last_clean":"2024-05-08T23:22:07.776278+0000","last_became_active":"2024-05-08T23:22:07.320014+0000","last_became_peered":"2024-05-08T23:22:07.320014+0000","last_unstale":"2024-05-08T23:22:07.776278+0000","last_undegraded":"2024-05-08T23:22:07.776278+0000","last_fullsized":"2024-05-08T23:22:07.776278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:51.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797023+0000","last_change":"2024-05-08T23:22:07.788761+0000","last_active":"2024-05-08T23:22:07.797023+0000","last_peered":"2024-05-08T23:22:07.797023+0000","last_clean":"2024-05-08T23:22:07.797023+0000","last_became_active":"2024-05-08T23:22:07.786643+0000","last_became_peered":"2024-05-08T23:22:07.786643+0000","last_unstale":"2024-05-08T23:22:07.797023+0000","last_undegraded":"2024-05-08T23:22:07.797023+0000","last_fullsized":"2024-05-08T23:22:07.797023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.772842+0000","last_change":"2024-05-08T23:22:07.326385+0000","last_active":"2024-05-08T23:22:07.772842+0000","last_peered":"2024-05-08T23:22:07.772842+0000","last_clean":"2024-05-08T23:22:07.772842+0000","last_became_active":"2024-05-08T23:22:07.326047+0000","last_became_peered":"2024-05-08T23:22:07.326047+0000","last_unstale":"2024-05-08T23:22:07.772842+0000","last_undegraded":"2024-05-08T23:22:07.772842+0000","last_fullsized":"2024-05-08T23:22:07.772842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:46.848050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.785753+0000","last_change":"2024-05-08T23:22:07.782563+0000","last_active":"2024-05-08T23:22:07.785753+0000","last_peered":"2024-05-08T23:22:07.785753+0000","last_clean":"2024-05-08T23:22:07.785753+0000","last_became_active":"2024-05-08T23:22:07.782290+0000","last_became_peered":"2024-05-08T23:22:07.782290+0000","last_unstale":"2024-05-08T23:22:07.785753+0000","last_undegraded":"2024-05-08T23:22:07.785753+0000","last_fullsized":"2024-05-08T23:22:07.785753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:41.329227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.776767+0000","last_change":"2024-05-08T23:22:07.324876+0000","last_active":"2024-05-08T23:22:07.776767+0000","last_peered":"2024-05-08T23:22:07.776767+0000","last_clean":"2024-05-08T23:22:07.776767+0000","last_became_active":"2024-05-08T23:22:07.324583+0000","last_became_peered":"2024-05-08T23:22:07.324583+0000","last_unstale":"2024-05-08T23:22:07.776767+0000","last_undegraded":"2024-05-08T23:22:07.776767+0000","last_fullsized":"2024-05-08T23:22:07.776767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:00.240669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799311+0000","last_change":"2024-05-08T23:22:07.795071+0000","last_active":"2024-05-08T23:22:07.799311+0000","last_peered":"2024-05-08T23:22:07.799311+0000","last_clean":"2024-05-08T23:22:07.799311+0000","last_became_active":"2024-05-08T23:22:07.793871+0000","last_became_peered":"2024-05-08T23:22:07.793871+0000","last_unstale":"2024-05-08T23:22:07.799311+0000","last_undegraded":"2024-05-08T23:22:07.799311+0000","last_fullsized":"2024-05-08T23:22:07.799311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:08.578978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.770219+0000","last_change":"2024-05-08T23:22:07.324516+0000","last_active":"2024-05-08T23:22:07.770219+0000","last_peered":"2024-05-08T23:22:07.770219+0000","last_clean":"2024-05-08T23:22:07.770219+0000","last_became_active":"2024-05-08T23:22:07.323212+0000","last_became_peered":"2024-05-08T23:22:07.323212+0000","last_unstale":"2024-05-08T23:22:07.770219+0000","last_undegraded":"2024-05-08T23:22:07.770219+0000","last_fullsized":"2024-05-08T23:22:07.770219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:37.291699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799829+0000","last_change":"2024-05-08T23:22:07.795115+0000","last_active":"2024-05-08T23:22:07.799829+0000","last_peered":"2024-05-08T23:22:07.799829+0000","last_clean":"2024-05-08T23:22:07.799829+0000","last_became_active":"2024-05-08T23:22:07.794815+0000","last_became_peered":"2024-05-08T23:22:07.794815+0000","last_unstale":"2024-05-08T23:22:07.799829+0000","last_undegraded":"2024-05-08T23:22:07.799829+0000","last_fullsized":"2024-05-08T23:22:07.799829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:33.565554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.794828+0000","last_change":"2024-05-08T23:22:07.324920+0000","last_active":"2024-05-08T23:22:07.794828+0000","last_peered":"2024-05-08T23:22:07.794828+0000","last_clean":"2024-05-08T23:22:07.794828+0000","last_became_active":"2024-05-08T23:22:07.324655+0000","last_became_peered":"2024-05-08T23:22:07.324655+0000","last_unstale":"2024-05-08T23:22:07.794828+0000","last_undegraded":"2024-05-08T23:22:07.794828+0000","last_fullsized":"2024-05-08T23:22:07.794828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:40.686112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796166+0000","last_change":"2024-05-08T23:22:07.790463+0000","last_active":"2024-05-08T23:22:07.796166+0000","last_peered":"2024-05-08T23:22:07.796166+0000","last_clean":"2024-05-08T23:22:07.796166+0000","last_became_active":"2024-05-08T23:22:07.790181+0000","last_became_peered":"2024-05-08T23:22:07.790181+0000","last_unstale":"2024-05-08T23:22:07.796166+0000","last_undegraded":"2024-05-08T23:22:07.796166+0000","last_fullsized":"2024-05-08T23:22:07.796166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:32.443021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.801567+0000","last_change":"2024-05-08T23:22:07.324702+0000","last_active":"2024-05-08T23:22:07.801567+0000","last_peered":"2024-05-08T23:22:07.801567+0000","last_clean":"2024-05-08T23:22:07.801567+0000","last_became_active":"2024-05-08T23:22:07.323911+0000","last_became_peered":"2024-05-08T23:22:07.323911+0000","last_unstale":"2024-05-08T23:22:07.801567+0000","last_undegraded":"2024-05-08T23:22:07.801567+0000","last_fullsized":"2024-05-08T23:22:07.801567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:28.458137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799010+0000","last_change":"2024-05-08T23:22:07.786765+0000","last_active":"2024-05-08T23:22:07.799010+0000","last_peered":"2024-05-08T23:22:07.799010+0000","last_clean":"2024-05-08T23:22:07.799010+0000","last_became_active":"2024-05-08T23:22:07.786447+0000","last_became_peered":"2024-05-08T23:22:07.786447+0000","last_unstale":"2024-05-08T23:22:07.799010+0000","last_undegraded":"2024-05-08T23:22:07.799010+0000","last_fullsized":"2024-05-08T23:22:07.799010+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796364+0000","last_change":"2024-05-08T23:22:07.319729+0000","last_active":"2024-05-08T23:22:07.796364+0000","last_peered":"2024-05-08T23:22:07.796364+0000","last_clean":"2024-05-08T23:22:07.796364+0000","last_became_active":"2024-05-08T23:22:07.319420+0000","last_became_peered":"2024-05-08T23:22:07.319420+0000","last_unstale":"2024-05-08T23:22:07.796364+0000","last_undegraded":"2024-05-08T23:22:07.796364+0000","last_fullsized":"2024-05-08T23:22:07.796364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:13.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.787794+0000","last_change":"2024-05-08T23:22:07.785658+0000","last_active":"2024-05-08T23:22:07.787794+0000","last_peered":"2024-05-08T23:22:07.787794+0000","last_clean":"2024-05-08T23:22:07.787794+0000","last_became_active":"2024-05-08T23:22:07.784171+0000","last_became_peered":"2024-05-08T23:22:07.784171+0000","last_unstale":"2024-05-08T23:22:07.787794+0000","last_undegraded":"2024-05-08T23:22:07.787794+0000","last_fullsized":"2024-05-08T23:22:07.787794+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:10.382589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.772252+0000","last_change":"2024-05-08T23:22:07.327526+0000","last_active":"2024-05-08T23:22:07.772252+0000","last_peered":"2024-05-08T23:22:07.772252+0000","last_clean":"2024-05-08T23:22:07.772252+0000","last_became_active":"2024-05-08T23:22:07.327240+0000","last_became_peered":"2024-05-08T23:22:07.327240+0000","last_unstale":"2024-05-08T23:22:07.772252+0000","last_undegraded":"2024-05-08T23:22:07.772252+0000","last_fullsized":"2024-05-08T23:22:07.772252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799655+0000","last_change":"2024-05-08T23:22:07.797083+0000","last_active":"2024-05-08T23:22:07.799655+0000","last_peered":"2024-05-08T23:22:07.799655+0000","last_clean":"2024-05-08T23:22:07.799655+0000","last_became_active":"2024-05-08T23:22:07.796233+0000","last_became_peered":"2024-05-08T23:22:07.796233+0000","last_unstale":"2024-05-08T23:22:07.799655+0000","last_undegraded":"2024-05-08T23:22:07.799655+0000","last_fullsized":"2024-05-08T23:22:07.799655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:32.443021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.776071+0000","last_change":"2024-05-08T23:22:07.324290+0000","last_active":"2024-05-08T23:22:07.776071+0000","last_peered":"2024-05-08T23:22:07.776071+0000","last_clean":"2024-05-08T23:22:07.776071+0000","last_became_active":"2024-05-08T23:22:07.323929+0000","last_became_peered":"2024-05-08T23:22:07.323929+0000","last_unstale":"2024-05-08T23:22:07.776071+0000","last_undegraded":"2024-05-08T23:22:07.776071+0000","last_fullsized":"2024-05-08T23:22:07.776071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:17.483564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796592+0000","last_change":"2024-05-08T23:22:07.794917+0000","last_active":"2024-05-08T23:22:07.796592+0000","last_peered":"2024-05-08T23:22:07.796592+0000","last_clean":"2024-05-08T23:22:07.796592+0000","last_became_active":"2024-05-08T23:22:07.793482+0000","last_became_peered":"2024-05-08T23:22:07.793482+0000","last_unstale":"2024-05-08T23:22:07.796592+0000","last_undegraded":"2024-05-08T23:22:07.796592+0000","last_fullsized":"2024-05-08T23:22:07.796592+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799707+0000","last_change":"2024-05-08T23:22:07.323609+0000","last_active":"2024-05-08T23:22:07.799707+0000","last_peered":"2024-05-08T23:22:07.799707+0000","last_clean":"2024-05-08T23:22:07.799707+0000","last_became_active":"2024-05-08T23:22:07.323344+0000","last_became_peered":"2024-05-08T23:22:07.323344+0000","last_unstale":"2024-05-08T23:22:07.799707+0000","last_undegraded":"2024-05-08T23:22:07.799707+0000","last_fullsized":"2024-05-08T23:22:07.799707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:41.903099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792697+0000","last_change":"2024-05-08T23:22:07.783864+0000","last_active":"2024-05-08T23:22:07.792697+0000","last_peered":"2024-05-08T23:22:07.792697+0000","last_clean":"2024-05-08T23:22:07.792697+0000","last_became_active":"2024-05-08T23:22:07.783242+0000","last_became_peered":"2024-05-08T23:22:07.783242+0000","last_unstale":"2024-05-08T23:22:07.792697+0000","last_undegraded":"2024-05-08T23:22:07.792697+0000","last_fullsized":"2024-05-08T23:22:07.792697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.800054+0000","last_change":"2024-05-08T23:22:07.324272+0000","last_active":"2024-05-08T23:22:07.800054+0000","last_peered":"2024-05-08T23:22:07.800054+0000","last_clean":"2024-05-08T23:22:07.800054+0000","last_became_active":"2024-05-08T23:22:07.324005+0000","last_became_peered":"2024-05-08T23:22:07.324005+0000","last_unstale":"2024-05-08T23:22:07.800054+0000","last_undegraded":"2024-05-08T23:22:07.800054+0000","last_fullsized":"2024-05-08T23:22:07.800054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796807+0000","last_change":"2024-05-08T23:22:07.792097+0000","last_active":"2024-05-08T23:22:07.796807+0000","last_peered":"2024-05-08T23:22:07.796807+0000","last_clean":"2024-05-08T23:22:07.796807+0000","last_became_active":"2024-05-08T23:22:07.791645+0000","last_became_peered":"2024-05-08T23:22:07.791645+0000","last_unstale":"2024-05-08T23:22:07.796807+0000","last_undegraded":"2024-05-08T23:22:07.796807+0000","last_fullsized":"2024-05-08T23:22:07.796807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:31.969535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792384+0000","last_change":"2024-05-08T23:22:07.320014+0000","last_active":"2024-05-08T23:22:07.792384+0000","last_peered":"2024-05-08T23:22:07.792384+0000","last_clean":"2024-05-08T23:22:07.792384+0000","last_became_active":"2024-05-08T23:22:07.319073+0000","last_became_peered":"2024-05-08T23:22:07.319073+0000","last_unstale":"2024-05-08T23:22:07.792384+0000","last_undegraded":"2024-05-08T23:22:07.792384+0000","last_fullsized":"2024-05-08T23:22:07.792384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.661583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.784131+0000","last_change":"2024-05-08T23:22:07.779107+0000","last_active":"2024-05-08T23:22:07.784131+0000","last_peered":"2024-05-08T23:22:07.784131+0000","last_clean":"2024-05-08T23:22:07.784131+0000","last_became_active":"2024-05-08T23:22:07.778839+0000","last_became_peered":"2024-05-08T23:22:07.778839+0000","last_unstale":"2024-05-08T23:22:07.784131+0000","last_undegraded":"2024-05-08T23:22:07.784131+0000","last_fullsized":"2024-05-08T23:22:07.784131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792867+0000","last_change":"2024-05-08T23:22:07.325295+0000","last_active":"2024-05-08T23:22:07.792867+0000","last_peered":"2024-05-08T23:22:07.792867+0000","last_clean":"2024-05-08T23:22:07.792867+0000","last_became_active":"2024-05-08T23:22:07.324997+0000","last_became_peered":"2024-05-08T23:22:07.324997+0000","last_unstale":"2024-05-08T23:22:07.792867+0000","last_undegraded":"2024-05-08T23:22:07.792867+0000","last_fullsized":"2024-05-08T23:22:07.792867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:56.035940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796024+0000","last_change":"2024-05-08T23:22:07.794794+0000","last_active":"2024-05-08T23:22:07.796024+0000","last_peered":"2024-05-08T23:22:07.796024+0000","last_clean":"2024-05-08T23:22:07.796024+0000","last_became_active":"2024-05-08T23:22:07.794503+0000","last_became_peered":"2024-05-08T23:22:07.794503+0000","last_unstale":"2024-05-08T23:22:07.796024+0000","last_undegraded":"2024-05-08T23:22:07.796024+0000","last_fullsized":"2024-05-08T23:22:07.796024+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.774719+0000","last_change":"2024-05-08T23:22:07.323547+0000","last_active":"2024-05-08T23:22:07.774719+0000","last_peered":"2024-05-08T23:22:07.774719+0000","last_clean":"2024-05-08T23:22:07.774719+0000","last_became_active":"2024-05-08T23:22:07.323266+0000","last_became_peered":"2024-05-08T23:22:07.323266+0000","last_unstale":"2024-05-08T23:22:07.774719+0000","last_undegraded":"2024-05-08T23:22:07.774719+0000","last_fullsized":"2024-05-08T23:22:07.774719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797211+0000","last_change":"2024-05-08T23:22:07.795639+0000","last_active":"2024-05-08T23:22:07.797211+0000","last_peered":"2024-05-08T23:22:07.797211+0000","last_clean":"2024-05-08T23:22:07.797211+0000","last_became_active":"2024-05-08T23:22:07.795353+0000","last_became_peered":"2024-05-08T23:22:07.795353+0000","last_unstale":"2024-05-08T23:22:07.797211+0000","last_undegraded":"2024-05-08T23:22:07.797211+0000","last_fullsized":"2024-05-08T23:22:07.797211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:54:32.601480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.787493+0000","last_change":"2024-05-08T23:22:07.322755+0000","last_active":"2024-05-08T23:22:07.787493+0000","last_peered":"2024-05-08T23:22:07.787493+0000","last_clean":"2024-05-08T23:22:07.787493+0000","last_became_active":"2024-05-08T23:22:07.321921+0000","last_became_peered":"2024-05-08T23:22:07.321921+0000","last_unstale":"2024-05-08T23:22:07.787493+0000","last_undegraded":"2024-05-08T23:22:07.787493+0000","last_fullsized":"2024-05-08T23:22:07.787493+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:29.054724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.787775+0000","last_change":"2024-05-08T23:22:07.785995+0000","last_active":"2024-05-08T23:22:07.787775+0000","last_peered":"2024-05-08T23:22:07.787775+0000","last_clean":"2024-05-08T23:22:07.787775+0000","last_became_active":"2024-05-08T23:22:07.785679+0000","last_became_peered":"2024-05-08T23:22:07.785679+0000","last_unstale":"2024-05-08T23:22:07.787775+0000","last_undegraded":"2024-05-08T23:22:07.787775+0000","last_fullsized":"2024-05-08T23:22:07.787775+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:15.611951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.779401+0000","last_change":"2024-05-08T23:22:07.327553+0000","last_active":"2024-05-08T23:22:07.779401+0000","last_peered":"2024-05-08T23:22:07.779401+0000","last_clean":"2024-05-08T23:22:07.779401+0000","last_became_active":"2024-05-08T23:22:07.327214+0000","last_became_peered":"2024-05-08T23:22:07.327214+0000","last_unstale":"2024-05-08T23:22:07.779401+0000","last_undegraded":"2024-05-08T23:22:07.779401+0000","last_fullsized":"2024-05-08T23:22:07.779401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.789962+0000","last_change":"2024-05-08T23:22:07.788317+0000","last_active":"2024-05-08T23:22:07.789962+0000","last_peered":"2024-05-08T23:22:07.789962+0000","last_clean":"2024-05-08T23:22:07.789962+0000","last_became_active":"2024-05-08T23:22:07.786118+0000","last_became_peered":"2024-05-08T23:22:07.786118+0000","last_unstale":"2024-05-08T23:22:07.789962+0000","last_undegraded":"2024-05-08T23:22:07.789962+0000","last_fullsized":"2024-05-08T23:22:07.789962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:45.661099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.774186+0000","last_change":"2024-05-08T23:22:07.323427+0000","last_active":"2024-05-08T23:22:07.774186+0000","last_peered":"2024-05-08T23:22:07.774186+0000","last_clean":"2024-05-08T23:22:07.774186+0000","last_became_active":"2024-05-08T23:22:07.323163+0000","last_became_peered":"2024-05-08T23:22:07.323163+0000","last_unstale":"2024-05-08T23:22:07.774186+0000","last_undegraded":"2024-05-08T23:22:07.774186+0000","last_fullsized":"2024-05-08T23:22:07.774186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799292+0000","last_change":"2024-05-08T23:22:07.796653+0000","last_active":"2024-05-08T23:22:07.799292+0000","last_peered":"2024-05-08T23:22:07.799292+0000","last_clean":"2024-05-08T23:22:07.799292+0000","last_became_active":"2024-05-08T23:22:07.796401+0000","last_became_peered":"2024-05-08T23:22:07.796401+0000","last_unstale":"2024-05-08T23:22:07.799292+0000","last_undegraded":"2024-05-08T23:22:07.799292+0000","last_fullsized":"2024-05-08T23:22:07.799292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:09.803894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.778060+0000","last_change":"2024-05-08T23:22:07.325578+0000","last_active":"2024-05-08T23:22:07.778060+0000","last_peered":"2024-05-08T23:22:07.778060+0000","last_clean":"2024-05-08T23:22:07.778060+0000","last_became_active":"2024-05-08T23:22:07.325303+0000","last_became_peered":"2024-05-08T23:22:07.325303+0000","last_unstale":"2024-05-08T23:22:07.778060+0000","last_undegraded":"2024-05-08T23:22:07.778060+0000","last_fullsized":"2024-05-08T23:22:07.778060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:04.490589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.789680+0000","last_change":"2024-05-08T23:22:07.788948+0000","last_active":"2024-05-08T23:22:07.789680+0000","last_peered":"2024-05-08T23:22:07.789680+0000","last_clean":"2024-05-08T23:22:07.789680+0000","last_became_active":"2024-05-08T23:22:07.788633+0000","last_became_peered":"2024-05-08T23:22:07.788633+0000","last_unstale":"2024-05-08T23:22:07.789680+0000","last_undegraded":"2024-05-08T23:22:07.789680+0000","last_fullsized":"2024-05-08T23:22:07.789680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:02.041531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799616+0000","last_change":"2024-05-08T23:22:07.330079+0000","last_active":"2024-05-08T23:22:07.799616+0000","last_peered":"2024-05-08T23:22:07.799616+0000","last_clean":"2024-05-08T23:22:07.799616+0000","last_became_active":"2024-05-08T23:22:07.329577+0000","last_became_peered":"2024-05-08T23:22:07.329577+0000","last_unstale":"2024-05-08T23:22:07.799616+0000","last_undegraded":"2024-05-08T23:22:07.799616+0000","last_fullsized":"2024-05-08T23:22:07.799616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.790578+0000","last_change":"2024-05-08T23:22:07.788317+0000","last_active":"2024-05-08T23:22:07.790578+0000","last_peered":"2024-05-08T23:22:07.790578+0000","last_clean":"2024-05-08T23:22:07.790578+0000","last_became_active":"2024-05-08T23:22:07.787987+0000","last_became_peered":"2024-05-08T23:22:07.787987+0000","last_unstale":"2024-05-08T23:22:07.790578+0000","last_undegraded":"2024-05-08T23:22:07.790578+0000","last_fullsized":"2024-05-08T23:22:07.790578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:17.509591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T23:22:08.868641+0000","last_change":"2024-05-08T23:22:07.322623+0000","last_active":"2024-05-08T23:22:08.868641+0000","last_peered":"2024-05-08T23:22:08.868641+0000","last_clean":"2024-05-08T23:22:08.868641+0000","last_became_active":"2024-05-08T23:22:07.322321+0000","last_became_peered":"2024-05-08T23:22:07.322321+0000","last_unstale":"2024-05-08T23:22:08.868641+0000","last_undegraded":"2024-05-08T23:22:08.868641+0000","last_fullsized":"2024-05-08T23:22:08.868641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796559+0000","last_change":"2024-05-08T23:22:07.788010+0000","last_active":"2024-05-08T23:22:07.796559+0000","last_peered":"2024-05-08T23:22:07.796559+0000","last_clean":"2024-05-08T23:22:07.796559+0000","last_became_active":"2024-05-08T23:22:07.787715+0000","last_became_peered":"2024-05-08T23:22:07.787715+0000","last_unstale":"2024-05-08T23:22:07.796559+0000","last_undegraded":"2024-05-08T23:22:07.796559+0000","last_fullsized":"2024-05-08T23:22:07.796559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.775208+0000","last_change":"2024-05-08T23:22:07.327107+0000","last_active":"2024-05-08T23:22:07.775208+0000","last_peered":"2024-05-08T23:22:07.775208+0000","last_clean":"2024-05-08T23:22:07.775208+0000","last_became_active":"2024-05-08T23:22:07.326542+0000","last_became_peered":"2024-05-08T23:22:07.326542+0000","last_unstale":"2024-05-08T23:22:07.775208+0000","last_undegraded":"2024-05-08T23:22:07.775208+0000","last_fullsized":"2024-05-08T23:22:07.775208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.791445+0000","last_change":"2024-05-08T23:22:07.784128+0000","last_active":"2024-05-08T23:22:07.791445+0000","last_peered":"2024-05-08T23:22:07.791445+0000","last_clean":"2024-05-08T23:22:07.791445+0000","last_became_active":"2024-05-08T23:22:07.783822+0000","last_became_peered":"2024-05-08T23:22:07.783822+0000","last_unstale":"2024-05-08T23:22:07.791445+0000","last_undegraded":"2024-05-08T23:22:07.791445+0000","last_fullsized":"2024-05-08T23:22:07.791445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.769881+0000","last_change":"2024-05-08T23:22:07.319178+0000","last_active":"2024-05-08T23:22:07.769881+0000","last_peered":"2024-05-08T23:22:07.769881+0000","last_clean":"2024-05-08T23:22:07.769881+0000","last_became_active":"2024-05-08T23:22:07.318864+0000","last_became_peered":"2024-05-08T23:22:07.318864+0000","last_unstale":"2024-05-08T23:22:07.769881+0000","last_undegraded":"2024-05-08T23:22:07.769881+0000","last_fullsized":"2024-05-08T23:22:07.769881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796633+0000","last_change":"2024-05-08T23:22:07.787863+0000","last_active":"2024-05-08T23:22:07.796633+0000","last_peered":"2024-05-08T23:22:07.796633+0000","last_clean":"2024-05-08T23:22:07.796633+0000","last_became_active":"2024-05-08T23:22:07.787166+0000","last_became_peered":"2024-05-08T23:22:07.787166+0000","last_unstale":"2024-05-08T23:22:07.796633+0000","last_undegraded":"2024-05-08T23:22:07.796633+0000","last_fullsized":"2024-05-08T23:22:07.796633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.772373+0000","last_change":"2024-05-08T23:22:07.329759+0000","last_active":"2024-05-08T23:22:07.772373+0000","last_peered":"2024-05-08T23:22:07.772373+0000","last_clean":"2024-05-08T23:22:07.772373+0000","last_became_active":"2024-05-08T23:22:07.329423+0000","last_became_peered":"2024-05-08T23:22:07.329423+0000","last_unstale":"2024-05-08T23:22:07.772373+0000","last_undegraded":"2024-05-08T23:22:07.772373+0000","last_fullsized":"2024-05-08T23:22:07.772373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799765+0000","last_change":"2024-05-08T23:22:07.795295+0000","last_active":"2024-05-08T23:22:07.799765+0000","last_peered":"2024-05-08T23:22:07.799765+0000","last_clean":"2024-05-08T23:22:07.799765+0000","last_became_active":"2024-05-08T23:22:07.793689+0000","last_became_peered":"2024-05-08T23:22:07.793689+0000","last_unstale":"2024-05-08T23:22:07.799765+0000","last_undegraded":"2024-05-08T23:22:07.799765+0000","last_fullsized":"2024-05-08T23:22:07.799765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:55.199701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.774286+0000","last_change":"2024-05-08T23:22:07.326530+0000","last_active":"2024-05-08T23:22:07.774286+0000","last_peered":"2024-05-08T23:22:07.774286+0000","last_clean":"2024-05-08T23:22:07.774286+0000","last_became_active":"2024-05-08T23:22:07.326233+0000","last_became_peered":"2024-05-08T23:22:07.326233+0000","last_unstale":"2024-05-08T23:22:07.774286+0000","last_undegraded":"2024-05-08T23:22:07.774286+0000","last_fullsized":"2024-05-08T23:22:07.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:49.418732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792793+0000","last_change":"2024-05-08T23:22:07.782326+0000","last_active":"2024-05-08T23:22:07.792793+0000","last_peered":"2024-05-08T23:22:07.792793+0000","last_clean":"2024-05-08T23:22:07.792793+0000","last_became_active":"2024-05-08T23:22:07.782054+0000","last_became_peered":"2024-05-08T23:22:07.782054+0000","last_unstale":"2024-05-08T23:22:07.792793+0000","last_undegraded":"2024-05-08T23:22:07.792793+0000","last_fullsized":"2024-05-08T23:22:07.792793+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.776058+0000","last_change":"2024-05-08T23:22:07.325642+0000","last_active":"2024-05-08T23:22:07.776058+0000","last_peered":"2024-05-08T23:22:07.776058+0000","last_clean":"2024-05-08T23:22:07.776058+0000","last_became_active":"2024-05-08T23:22:07.325384+0000","last_became_peered":"2024-05-08T23:22:07.325384+0000","last_unstale":"2024-05-08T23:22:07.776058+0000","last_undegraded":"2024-05-08T23:22:07.776058+0000","last_fullsized":"2024-05-08T23:22:07.776058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:26.489244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796437+0000","last_change":"2024-05-08T23:22:07.792644+0000","last_active":"2024-05-08T23:22:07.796437+0000","last_peered":"2024-05-08T23:22:07.796437+0000","last_clean":"2024-05-08T23:22:07.796437+0000","last_became_active":"2024-05-08T23:22:07.792347+0000","last_became_peered":"2024-05-08T23:22:07.792347+0000","last_unstale":"2024-05-08T23:22:07.796437+0000","last_undegraded":"2024-05-08T23:22:07.796437+0000","last_fullsized":"2024-05-08T23:22:07.796437+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.778375+0000","last_change":"2024-05-08T23:22:07.328621+0000","last_active":"2024-05-08T23:22:07.778375+0000","last_peered":"2024-05-08T23:22:07.778375+0000","last_clean":"2024-05-08T23:22:07.778375+0000","last_became_active":"2024-05-08T23:22:07.328297+0000","last_became_peered":"2024-05-08T23:22:07.328297+0000","last_unstale":"2024-05-08T23:22:07.778375+0000","last_undegraded":"2024-05-08T23:22:07.778375+0000","last_fullsized":"2024-05-08T23:22:07.778375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.779334+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:22:07.779334+0000","last_peered":"2024-05-08T23:22:07.779334+0000","last_clean":"2024-05-08T23:22:07.779334+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:22:07.779334+0000","last_undegraded":"2024-05-08T23:22:07.779334+0000","last_fullsized":"2024-05-08T23:22:07.779334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":742523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.496}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.432},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64200000000000002}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":152681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.495},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.55600000000000005}]},{"osd":3,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.438},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":153651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.29099999999999998}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.372},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.38500000000000001}]},{"osd":4,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.499},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.73099999999999998}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.75}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":978944,"data_stored":742433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.9889999999999999,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.7869999999999999,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.0519999999999996,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.30499999999999999}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.6440000000000001,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.9139999999999997,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.438},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.42299999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":7.1120000000000001,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.6970000000000001,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.734,"5min":7.734,"15min":7.734},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0220000000000002,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.46999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":152135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32200000000000001}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.36399999999999999}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.70899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":153651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.373},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.316}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":11.964,"5min":11.964,"15min":11.964},"last":0.438},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.501},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.57499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":152169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.0389999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.127000000000001,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.62}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.496}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.47599999999999998}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.441}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":152737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.6710000000000003,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":0.67300000000000004}]},{"osd":1,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":4.71,"5min":4.71,"15min":4.71},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":8.6880000000000006,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.625}]},{"osd":2,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.8940000000000001,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8559999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.66100000000000003}]},{"osd":3,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.8770000000000007,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.427,"5min":10.427,"15min":10.427},"last":0.64400000000000002}]},{"osd":4,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.441}]},{"osd":6,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.499},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:22:16.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:22:16.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.938+0000 7f45ef4ed700 1 Processor -- start 2024-05-08T23:22:16.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.938+0000 7f45ef4ed700 1 -- start start 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ef4ed700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45e80af9b0 0x7f45e8162c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ef4ed700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f45e80d0ec0 0x7f45e806a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ef4ed700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e806aea0 0x7f45e8161230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ef4ed700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e806b350 con 0x7f45e80af9b0 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ef4ed700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e8163160 con 0x7f45e80d0ec0 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ef4ed700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e80cfd60 con 0x7f45e806aea0 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ed289700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45e80af9b0 0x7f45e8162c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ed289700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45e80af9b0 0x7f45e8162c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47250/0 (socket says 172.21.15.5:47250) 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45ed289700 1 -- 172.21.15.5:0/1018863117 learned_addr learned my addr 172.21.15.5:0/1018863117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45eda8a700 1 --2- 172.21.15.5:0/1018863117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e806aea0 0x7f45e8161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.939+0000 7f45eca88700 1 --2- 172.21.15.5:0/1018863117 >> v2:172.21.15.49:3300/0 conn(0x7f45e80d0ec0 0x7f45e806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45ed289700 1 -- 172.21.15.5:0/1018863117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e806aea0 msgr2=0x7f45e8161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45ed289700 1 --2- 172.21.15.5:0/1018863117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e806aea0 0x7f45e8161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45ed289700 1 -- 172.21.15.5:0/1018863117 >> v2:172.21.15.49:3300/0 conn(0x7f45e80d0ec0 msgr2=0x7f45e806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45ed289700 1 --2- 172.21.15.5:0/1018863117 >> v2:172.21.15.49:3300/0 conn(0x7f45e80d0ec0 0x7f45e806a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45ed289700 1 -- 172.21.15.5:0/1018863117 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e80d0040 con 0x7f45e80af9b0 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45ed289700 1 --2- 172.21.15.5:0/1018863117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e8162c20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f45e40098f0 tx=0x7f45e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45f2f48b8b980d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45dbfff700 1 -- 172.21.15.5:0/1018863117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e400b860 con 0x7f45e80af9b0 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45dbfff700 1 -- 172.21.15.5:0/1018863117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e400b9c0 con 0x7f45e80af9b0 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.940+0000 7f45dbfff700 1 -- 172.21.15.5:0/1018863117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e400bd50 con 0x7f45e80af9b0 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 -- 172.21.15.5:0/1018863117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 msgr2=0x7f45e8162c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/1018863117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e8162c20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f45e40098f0 tx=0x7f45e4009c00 comp rx=0 tx=0).stop 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 -- 172.21.15.5:0/1018863117 shutdown_connections 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/1018863117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e806aea0 0x7f45e8161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/1018863117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e8162c20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/1018863117 >> v2:172.21.15.49:3300/0 conn(0x7f45e80d0ec0 0x7f45e806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 -- 172.21.15.5:0/1018863117 >> 172.21.15.5:0/1018863117 conn(0x7f45e80d03a0 msgr2=0x7f45e80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 -- 172.21.15.5:0/1018863117 shutdown_connections 2024-05-08T23:22:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.941+0000 7f45ef4ed700 1 -- 172.21.15.5:0/1018863117 wait complete. 2024-05-08T23:22:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.942+0000 7f45ef4ed700 1 Processor -- start 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.942+0000 7f45ef4ed700 1 -- start start 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45ef4ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e806aea0 0x7f45e80b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45ef4ed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45ef4ed700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e80d0ec0 0x7f45e812bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45ef4ed700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e812ab20 con 0x7f45e80af9b0 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45ef4ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e80b5050 con 0x7f45e806aea0 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45ef4ed700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e812c070 con 0x7f45e80d0ec0 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eca88700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eca88700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e80b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47264/0 (socket says 172.21.15.5:47264) 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eca88700 1 -- 172.21.15.5:0/4231727441 learned_addr learned my addr 172.21.15.5:0/4231727441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45ed289700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e806aea0 0x7f45e80b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eda8a700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e80d0ec0 0x7f45e812bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eca88700 1 -- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e80d0ec0 msgr2=0x7f45e812bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eca88700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e80d0ec0 0x7f45e812bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eca88700 1 -- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e806aea0 msgr2=0x7f45e80b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eca88700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e806aea0 0x7f45e80b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eca88700 1 -- 172.21.15.5:0/4231727441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e4009550 con 0x7f45e80af9b0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.943+0000 7f45eda8a700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e80d0ec0 0x7f45e812bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.944+0000 7f45eca88700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e80b4b10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00d7d0 tx=0x7f45dc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.944+0000 7f45d9ffb700 1 -- 172.21.15.5:0/4231727441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0125f0 con 0x7f45e80af9b0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.944+0000 7f45d9ffb700 1 -- 172.21.15.5:0/4231727441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc012750 con 0x7f45e80af9b0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.944+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e812ad70 con 0x7f45e80af9b0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.944+0000 7f45d9ffb700 1 -- 172.21.15.5:0/4231727441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc012ac0 con 0x7f45e80af9b0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.944+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e812b330 con 0x7f45e80af9b0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.945+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e806b490 con 0x7f45e80af9b0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.946+0000 7f45d9ffb700 1 -- 172.21.15.5:0/4231727441 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f45dc012c20 con 0x7f45e80af9b0 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.946+0000 7f45d9ffb700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f45cc06ac90 0x7f45cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:16.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.946+0000 7f45d9ffb700 1 -- 172.21.15.5:0/4231727441 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45dc089610 con 0x7f45e80af9b0 2024-05-08T23:22:16.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.946+0000 7f45ed289700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f45cc06ac90 0x7f45cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:16.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.947+0000 7f45ed289700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f45cc06ac90 0x7f45cc06d150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f45e4014040 tx=0x7f45e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:16.951+0000 7f45d9ffb700 1 -- 172.21.15.5:0/4231727441 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f45dc054880 con 0x7f45e80af9b0 2024-05-08T23:22:17.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.163+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45e80c6920 con 0x7f45cc06ac90 2024-05-08T23:22:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.172+0000 7f45d9ffb700 1 -- 172.21.15.5:0/4231727441 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289282 (secure 0 0 0) 0x7f45e400bb30 con 0x7f45cc06ac90 2024-05-08T23:22:17.176 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:17.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f45cc06ac90 msgr2=0x7f45cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f45cc06ac90 0x7f45cc06d150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f45e4014040 tx=0x7f45e4008030 comp rx=0 tx=0).stop 2024-05-08T23:22:17.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 msgr2=0x7f45e80b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e80b4b10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00d7d0 tx=0x7f45dc00db90 comp rx=0 tx=0).stop 2024-05-08T23:22:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 shutdown_connections 2024-05-08T23:22:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f45cc06ac90 0x7f45cc06d150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45e80d0ec0 0x7f45e812bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e806aea0 0x7f45e80b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 --2- 172.21.15.5:0/4231727441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e80af9b0 0x7f45e80b4b10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 >> 172.21.15.5:0/4231727441 conn(0x7f45e80d03a0 msgr2=0x7f45e80d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 shutdown_connections 2024-05-08T23:22:17.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.176+0000 7f45ef4ed700 1 -- 172.21.15.5:0/4231727441 wait complete. 2024-05-08T23:22:17.180 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:22:17.190 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-08T23:22:15.310969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220456,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754264,"statfs":{"total":773094113280,"available":772868366336,"internally_reserved":0,"allocated":4333568,"data_stored":2401980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002251"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.795731+0000","last_change":"2024-05-08T23:22:07.792801+0000","last_active":"2024-05-08T23:22:07.795731+0000","last_peered":"2024-05-08T23:22:07.795731+0000","last_clean":"2024-05-08T23:22:07.795731+0000","last_became_active":"2024-05-08T23:22:07.792474+0000","last_became_peered":"2024-05-08T23:22:07.792474+0000","last_unstale":"2024-05-08T23:22:07.795731+0000","last_undegraded":"2024-05-08T23:22:07.795731+0000","last_fullsized":"2024-05-08T23:22:07.795731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799412+0000","last_change":"2024-05-08T23:22:07.794320+0000","last_active":"2024-05-08T23:22:07.799412+0000","last_peered":"2024-05-08T23:22:07.799412+0000","last_clean":"2024-05-08T23:22:07.799412+0000","last_became_active":"2024-05-08T23:22:07.789420+0000","last_became_peered":"2024-05-08T23:22:07.789420+0000","last_unstale":"2024-05-08T23:22:07.799412+0000","last_undegraded":"2024-05-08T23:22:07.799412+0000","last_fullsized":"2024-05-08T23:22:07.799412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:58.867905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799267+0000","last_change":"2024-05-08T23:22:07.798266+0000","last_active":"2024-05-08T23:22:07.799267+0000","last_peered":"2024-05-08T23:22:07.799267+0000","last_clean":"2024-05-08T23:22:07.799267+0000","last_became_active":"2024-05-08T23:22:07.797980+0000","last_became_peered":"2024-05-08T23:22:07.797980+0000","last_unstale":"2024-05-08T23:22:07.799267+0000","last_undegraded":"2024-05-08T23:22:07.799267+0000","last_fullsized":"2024-05-08T23:22:07.799267+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:46.755007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.784150+0000","last_change":"2024-05-08T23:22:07.781843+0000","last_active":"2024-05-08T23:22:07.784150+0000","last_peered":"2024-05-08T23:22:07.784150+0000","last_clean":"2024-05-08T23:22:07.784150+0000","last_became_active":"2024-05-08T23:22:07.781605+0000","last_became_peered":"2024-05-08T23:22:07.781605+0000","last_unstale":"2024-05-08T23:22:07.784150+0000","last_undegraded":"2024-05-08T23:22:07.784150+0000","last_fullsized":"2024-05-08T23:22:07.784150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797237+0000","last_change":"2024-05-08T23:22:07.795484+0000","last_active":"2024-05-08T23:22:07.797237+0000","last_peered":"2024-05-08T23:22:07.797237+0000","last_clean":"2024-05-08T23:22:07.797237+0000","last_became_active":"2024-05-08T23:22:07.795009+0000","last_became_peered":"2024-05-08T23:22:07.795009+0000","last_unstale":"2024-05-08T23:22:07.797237+0000","last_undegraded":"2024-05-08T23:22:07.797237+0000","last_fullsized":"2024-05-08T23:22:07.797237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:27.657029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.784296+0000","last_change":"2024-05-08T23:22:07.782730+0000","last_active":"2024-05-08T23:22:07.784296+0000","last_peered":"2024-05-08T23:22:07.784296+0000","last_clean":"2024-05-08T23:22:07.784296+0000","last_became_active":"2024-05-08T23:22:07.782475+0000","last_became_peered":"2024-05-08T23:22:07.782475+0000","last_unstale":"2024-05-08T23:22:07.784296+0000","last_undegraded":"2024-05-08T23:22:07.784296+0000","last_fullsized":"2024-05-08T23:22:07.784296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:10.804990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799764+0000","last_change":"2024-05-08T23:22:07.797086+0000","last_active":"2024-05-08T23:22:07.799764+0000","last_peered":"2024-05-08T23:22:07.799764+0000","last_clean":"2024-05-08T23:22:07.799764+0000","last_became_active":"2024-05-08T23:22:07.796747+0000","last_became_peered":"2024-05-08T23:22:07.796747+0000","last_unstale":"2024-05-08T23:22:07.799764+0000","last_undegraded":"2024-05-08T23:22:07.799764+0000","last_fullsized":"2024-05-08T23:22:07.799764+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:31.969535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796858+0000","last_change":"2024-05-08T23:22:07.791736+0000","last_active":"2024-05-08T23:22:07.796858+0000","last_peered":"2024-05-08T23:22:07.796858+0000","last_clean":"2024-05-08T23:22:07.796858+0000","last_became_active":"2024-05-08T23:22:07.791442+0000","last_became_peered":"2024-05-08T23:22:07.791442+0000","last_unstale":"2024-05-08T23:22:07.796858+0000","last_undegraded":"2024-05-08T23:22:07.796858+0000","last_fullsized":"2024-05-08T23:22:07.796858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:41.329227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792892+0000","last_change":"2024-05-08T23:22:07.784891+0000","last_active":"2024-05-08T23:22:07.792892+0000","last_peered":"2024-05-08T23:22:07.792892+0000","last_clean":"2024-05-08T23:22:07.792892+0000","last_became_active":"2024-05-08T23:22:07.784603+0000","last_became_peered":"2024-05-08T23:22:07.784603+0000","last_unstale":"2024-05-08T23:22:07.792892+0000","last_undegraded":"2024-05-08T23:22:07.792892+0000","last_fullsized":"2024-05-08T23:22:07.792892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797161+0000","last_change":"2024-05-08T23:22:07.786491+0000","last_active":"2024-05-08T23:22:07.797161+0000","last_peered":"2024-05-08T23:22:07.797161+0000","last_clean":"2024-05-08T23:22:07.797161+0000","last_became_active":"2024-05-08T23:22:07.786199+0000","last_became_peered":"2024-05-08T23:22:07.786199+0000","last_unstale":"2024-05-08T23:22:07.797161+0000","last_undegraded":"2024-05-08T23:22:07.797161+0000","last_fullsized":"2024-05-08T23:22:07.797161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797173+0000","last_change":"2024-05-08T23:22:07.794506+0000","last_active":"2024-05-08T23:22:07.797173+0000","last_peered":"2024-05-08T23:22:07.797173+0000","last_clean":"2024-05-08T23:22:07.797173+0000","last_became_active":"2024-05-08T23:22:07.794173+0000","last_became_peered":"2024-05-08T23:22:07.794173+0000","last_unstale":"2024-05-08T23:22:07.797173+0000","last_undegraded":"2024-05-08T23:22:07.797173+0000","last_fullsized":"2024-05-08T23:22:07.797173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.788628+0000","last_change":"2024-05-08T23:22:07.787087+0000","last_active":"2024-05-08T23:22:07.788628+0000","last_peered":"2024-05-08T23:22:07.788628+0000","last_clean":"2024-05-08T23:22:07.788628+0000","last_became_active":"2024-05-08T23:22:07.786743+0000","last_became_peered":"2024-05-08T23:22:07.786743+0000","last_unstale":"2024-05-08T23:22:07.788628+0000","last_undegraded":"2024-05-08T23:22:07.788628+0000","last_fullsized":"2024-05-08T23:22:07.788628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:42.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.786115+0000","last_change":"2024-05-08T23:22:07.784216+0000","last_active":"2024-05-08T23:22:07.786115+0000","last_peered":"2024-05-08T23:22:07.786115+0000","last_clean":"2024-05-08T23:22:07.786115+0000","last_became_active":"2024-05-08T23:22:07.783893+0000","last_became_peered":"2024-05-08T23:22:07.783893+0000","last_unstale":"2024-05-08T23:22:07.786115+0000","last_undegraded":"2024-05-08T23:22:07.786115+0000","last_fullsized":"2024-05-08T23:22:07.786115+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:38.296628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.794776+0000","last_change":"2024-05-08T23:22:07.787696+0000","last_active":"2024-05-08T23:22:07.794776+0000","last_peered":"2024-05-08T23:22:07.794776+0000","last_clean":"2024-05-08T23:22:07.794776+0000","last_became_active":"2024-05-08T23:22:07.784113+0000","last_became_peered":"2024-05-08T23:22:07.784113+0000","last_unstale":"2024-05-08T23:22:07.794776+0000","last_undegraded":"2024-05-08T23:22:07.794776+0000","last_fullsized":"2024-05-08T23:22:07.794776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:37.867491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792227+0000","last_change":"2024-05-08T23:22:07.790230+0000","last_active":"2024-05-08T23:22:07.792227+0000","last_peered":"2024-05-08T23:22:07.792227+0000","last_clean":"2024-05-08T23:22:07.792227+0000","last_became_active":"2024-05-08T23:22:07.789926+0000","last_became_peered":"2024-05-08T23:22:07.789926+0000","last_unstale":"2024-05-08T23:22:07.792227+0000","last_undegraded":"2024-05-08T23:22:07.792227+0000","last_fullsized":"2024-05-08T23:22:07.792227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:48.075638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799863+0000","last_change":"2024-05-08T23:22:07.798602+0000","last_active":"2024-05-08T23:22:07.799863+0000","last_peered":"2024-05-08T23:22:07.799863+0000","last_clean":"2024-05-08T23:22:07.799863+0000","last_became_active":"2024-05-08T23:22:07.797408+0000","last_became_peered":"2024-05-08T23:22:07.797408+0000","last_unstale":"2024-05-08T23:22:07.799863+0000","last_undegraded":"2024-05-08T23:22:07.799863+0000","last_fullsized":"2024-05-08T23:22:07.799863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:50.357763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797023+0000","last_change":"2024-05-08T23:22:07.793949+0000","last_active":"2024-05-08T23:22:07.797023+0000","last_peered":"2024-05-08T23:22:07.797023+0000","last_clean":"2024-05-08T23:22:07.797023+0000","last_became_active":"2024-05-08T23:22:07.793549+0000","last_became_peered":"2024-05-08T23:22:07.793549+0000","last_unstale":"2024-05-08T23:22:07.797023+0000","last_undegraded":"2024-05-08T23:22:07.797023+0000","last_fullsized":"2024-05-08T23:22:07.797023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799694+0000","last_change":"2024-05-08T23:22:07.796230+0000","last_active":"2024-05-08T23:22:07.799694+0000","last_peered":"2024-05-08T23:22:07.799694+0000","last_clean":"2024-05-08T23:22:07.799694+0000","last_became_active":"2024-05-08T23:22:07.796019+0000","last_became_peered":"2024-05-08T23:22:07.796019+0000","last_unstale":"2024-05-08T23:22:07.799694+0000","last_undegraded":"2024-05-08T23:22:07.799694+0000","last_fullsized":"2024-05-08T23:22:07.799694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:44.317582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799206+0000","last_change":"2024-05-08T23:22:07.794870+0000","last_active":"2024-05-08T23:22:07.799206+0000","last_peered":"2024-05-08T23:22:07.799206+0000","last_clean":"2024-05-08T23:22:07.799206+0000","last_became_active":"2024-05-08T23:22:07.794071+0000","last_became_peered":"2024-05-08T23:22:07.794071+0000","last_unstale":"2024-05-08T23:22:07.799206+0000","last_undegraded":"2024-05-08T23:22:07.799206+0000","last_fullsized":"2024-05-08T23:22:07.799206+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:20.807790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.790329+0000","last_change":"2024-05-08T23:22:07.789391+0000","last_active":"2024-05-08T23:22:07.790329+0000","last_peered":"2024-05-08T23:22:07.790329+0000","last_clean":"2024-05-08T23:22:07.790329+0000","last_became_active":"2024-05-08T23:22:07.789123+0000","last_became_peered":"2024-05-08T23:22:07.789123+0000","last_unstale":"2024-05-08T23:22:07.790329+0000","last_undegraded":"2024-05-08T23:22:07.790329+0000","last_fullsized":"2024-05-08T23:22:07.790329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:30.059653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796448+0000","last_change":"2024-05-08T23:22:07.791305+0000","last_active":"2024-05-08T23:22:07.796448+0000","last_peered":"2024-05-08T23:22:07.796448+0000","last_clean":"2024-05-08T23:22:07.796448+0000","last_became_active":"2024-05-08T23:22:07.789926+0000","last_became_peered":"2024-05-08T23:22:07.789926+0000","last_unstale":"2024-05-08T23:22:07.796448+0000","last_undegraded":"2024-05-08T23:22:07.796448+0000","last_fullsized":"2024-05-08T23:22:07.796448+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.793227+0000","last_change":"2024-05-08T23:22:07.785342+0000","last_active":"2024-05-08T23:22:07.793227+0000","last_peered":"2024-05-08T23:22:07.793227+0000","last_clean":"2024-05-08T23:22:07.793227+0000","last_became_active":"2024-05-08T23:22:07.785016+0000","last_became_peered":"2024-05-08T23:22:07.785016+0000","last_unstale":"2024-05-08T23:22:07.793227+0000","last_undegraded":"2024-05-08T23:22:07.793227+0000","last_fullsized":"2024-05-08T23:22:07.793227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.798868+0000","last_change":"2024-05-08T23:22:07.795399+0000","last_active":"2024-05-08T23:22:07.798868+0000","last_peered":"2024-05-08T23:22:07.798868+0000","last_clean":"2024-05-08T23:22:07.798868+0000","last_became_active":"2024-05-08T23:22:07.795120+0000","last_became_peered":"2024-05-08T23:22:07.795120+0000","last_unstale":"2024-05-08T23:22:07.798868+0000","last_undegraded":"2024-05-08T23:22:07.798868+0000","last_fullsized":"2024-05-08T23:22:07.798868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.788154+0000","last_change":"2024-05-08T23:22:07.785217+0000","last_active":"2024-05-08T23:22:07.788154+0000","last_peered":"2024-05-08T23:22:07.788154+0000","last_clean":"2024-05-08T23:22:07.788154+0000","last_became_active":"2024-05-08T23:22:07.784936+0000","last_became_peered":"2024-05-08T23:22:07.784936+0000","last_unstale":"2024-05-08T23:22:07.788154+0000","last_undegraded":"2024-05-08T23:22:07.788154+0000","last_fullsized":"2024-05-08T23:22:07.788154+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:26.679362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799679+0000","last_change":"2024-05-08T23:22:07.792957+0000","last_active":"2024-05-08T23:22:07.799679+0000","last_peered":"2024-05-08T23:22:07.799679+0000","last_clean":"2024-05-08T23:22:07.799679+0000","last_became_active":"2024-05-08T23:22:07.792691+0000","last_became_peered":"2024-05-08T23:22:07.792691+0000","last_unstale":"2024-05-08T23:22:07.799679+0000","last_undegraded":"2024-05-08T23:22:07.799679+0000","last_fullsized":"2024-05-08T23:22:07.799679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:29.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799713+0000","last_change":"2024-05-08T23:22:07.796720+0000","last_active":"2024-05-08T23:22:07.799713+0000","last_peered":"2024-05-08T23:22:07.799713+0000","last_clean":"2024-05-08T23:22:07.799713+0000","last_became_active":"2024-05-08T23:22:07.796359+0000","last_became_peered":"2024-05-08T23:22:07.796359+0000","last_unstale":"2024-05-08T23:22:07.799713+0000","last_undegraded":"2024-05-08T23:22:07.799713+0000","last_fullsized":"2024-05-08T23:22:07.799713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:49.810291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799658+0000","last_change":"2024-05-08T23:22:07.791923+0000","last_active":"2024-05-08T23:22:07.799658+0000","last_peered":"2024-05-08T23:22:07.799658+0000","last_clean":"2024-05-08T23:22:07.799658+0000","last_became_active":"2024-05-08T23:22:07.791650+0000","last_became_peered":"2024-05-08T23:22:07.791650+0000","last_unstale":"2024-05-08T23:22:07.799658+0000","last_undegraded":"2024-05-08T23:22:07.799658+0000","last_fullsized":"2024-05-08T23:22:07.799658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:27.494173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.787966+0000","last_change":"2024-05-08T23:22:07.786214+0000","last_active":"2024-05-08T23:22:07.787966+0000","last_peered":"2024-05-08T23:22:07.787966+0000","last_clean":"2024-05-08T23:22:07.787966+0000","last_became_active":"2024-05-08T23:22:07.783664+0000","last_became_peered":"2024-05-08T23:22:07.783664+0000","last_unstale":"2024-05-08T23:22:07.787966+0000","last_undegraded":"2024-05-08T23:22:07.787966+0000","last_fullsized":"2024-05-08T23:22:07.787966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:09.803894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796424+0000","last_change":"2024-05-08T23:22:07.791114+0000","last_active":"2024-05-08T23:22:07.796424+0000","last_peered":"2024-05-08T23:22:07.796424+0000","last_clean":"2024-05-08T23:22:07.796424+0000","last_became_active":"2024-05-08T23:22:07.789579+0000","last_became_peered":"2024-05-08T23:22:07.789579+0000","last_unstale":"2024-05-08T23:22:07.796424+0000","last_undegraded":"2024-05-08T23:22:07.796424+0000","last_fullsized":"2024-05-08T23:22:07.796424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796498+0000","last_change":"2024-05-08T23:22:07.793957+0000","last_active":"2024-05-08T23:22:07.796498+0000","last_peered":"2024-05-08T23:22:07.796498+0000","last_clean":"2024-05-08T23:22:07.796498+0000","last_became_active":"2024-05-08T23:22:07.793638+0000","last_became_peered":"2024-05-08T23:22:07.793638+0000","last_unstale":"2024-05-08T23:22:07.796498+0000","last_undegraded":"2024-05-08T23:22:07.796498+0000","last_fullsized":"2024-05-08T23:22:07.796498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:42.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.793350+0000","last_change":"2024-05-08T23:22:07.788892+0000","last_active":"2024-05-08T23:22:07.793350+0000","last_peered":"2024-05-08T23:22:07.793350+0000","last_clean":"2024-05-08T23:22:07.793350+0000","last_became_active":"2024-05-08T23:22:07.784706+0000","last_became_peered":"2024-05-08T23:22:07.784706+0000","last_unstale":"2024-05-08T23:22:07.793350+0000","last_undegraded":"2024-05-08T23:22:07.793350+0000","last_fullsized":"2024-05-08T23:22:07.793350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:05.495518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799845+0000","last_change":"2024-05-08T23:22:07.797424+0000","last_active":"2024-05-08T23:22:07.799845+0000","last_peered":"2024-05-08T23:22:07.799845+0000","last_clean":"2024-05-08T23:22:07.799845+0000","last_became_active":"2024-05-08T23:22:07.797107+0000","last_became_peered":"2024-05-08T23:22:07.797107+0000","last_unstale":"2024-05-08T23:22:07.799845+0000","last_undegraded":"2024-05-08T23:22:07.799845+0000","last_fullsized":"2024-05-08T23:22:07.799845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:46.755007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797793+0000","last_change":"2024-05-08T23:22:07.795811+0000","last_active":"2024-05-08T23:22:07.797793+0000","last_peered":"2024-05-08T23:22:07.797793+0000","last_clean":"2024-05-08T23:22:07.797793+0000","last_became_active":"2024-05-08T23:22:07.795559+0000","last_became_peered":"2024-05-08T23:22:07.795559+0000","last_unstale":"2024-05-08T23:22:07.797793+0000","last_undegraded":"2024-05-08T23:22:07.797793+0000","last_fullsized":"2024-05-08T23:22:07.797793+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.794244+0000","last_change":"2024-05-08T23:22:07.321400+0000","last_active":"2024-05-08T23:22:07.794244+0000","last_peered":"2024-05-08T23:22:07.794244+0000","last_clean":"2024-05-08T23:22:07.794244+0000","last_became_active":"2024-05-08T23:22:07.321115+0000","last_became_peered":"2024-05-08T23:22:07.321115+0000","last_unstale":"2024-05-08T23:22:07.794244+0000","last_undegraded":"2024-05-08T23:22:07.794244+0000","last_fullsized":"2024-05-08T23:22:07.794244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.795752+0000","last_change":"2024-05-08T23:22:07.794410+0000","last_active":"2024-05-08T23:22:07.795752+0000","last_peered":"2024-05-08T23:22:07.795752+0000","last_clean":"2024-05-08T23:22:07.795752+0000","last_became_active":"2024-05-08T23:22:07.793894+0000","last_became_peered":"2024-05-08T23:22:07.793894+0000","last_unstale":"2024-05-08T23:22:07.795752+0000","last_undegraded":"2024-05-08T23:22:07.795752+0000","last_fullsized":"2024-05-08T23:22:07.795752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.795678+0000","last_change":"2024-05-08T23:22:07.325196+0000","last_active":"2024-05-08T23:22:07.795678+0000","last_peered":"2024-05-08T23:22:07.795678+0000","last_clean":"2024-05-08T23:22:07.795678+0000","last_became_active":"2024-05-08T23:22:07.324913+0000","last_became_peered":"2024-05-08T23:22:07.324913+0000","last_unstale":"2024-05-08T23:22:07.795678+0000","last_undegraded":"2024-05-08T23:22:07.795678+0000","last_fullsized":"2024-05-08T23:22:07.795678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:38.809279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792981+0000","last_change":"2024-05-08T23:22:07.785830+0000","last_active":"2024-05-08T23:22:07.792981+0000","last_peered":"2024-05-08T23:22:07.792981+0000","last_clean":"2024-05-08T23:22:07.792981+0000","last_became_active":"2024-05-08T23:22:07.785570+0000","last_became_peered":"2024-05-08T23:22:07.785570+0000","last_unstale":"2024-05-08T23:22:07.792981+0000","last_undegraded":"2024-05-08T23:22:07.792981+0000","last_fullsized":"2024-05-08T23:22:07.792981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799074+0000","last_change":"2024-05-08T23:22:07.323498+0000","last_active":"2024-05-08T23:22:07.799074+0000","last_peered":"2024-05-08T23:22:07.799074+0000","last_clean":"2024-05-08T23:22:07.799074+0000","last_became_active":"2024-05-08T23:22:07.323230+0000","last_became_peered":"2024-05-08T23:22:07.323230+0000","last_unstale":"2024-05-08T23:22:07.799074+0000","last_undegraded":"2024-05-08T23:22:07.799074+0000","last_fullsized":"2024-05-08T23:22:07.799074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:50.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799634+0000","last_change":"2024-05-08T23:22:07.791888+0000","last_active":"2024-05-08T23:22:07.799634+0000","last_peered":"2024-05-08T23:22:07.799634+0000","last_clean":"2024-05-08T23:22:07.799634+0000","last_became_active":"2024-05-08T23:22:07.791592+0000","last_became_peered":"2024-05-08T23:22:07.791592+0000","last_unstale":"2024-05-08T23:22:07.799634+0000","last_undegraded":"2024-05-08T23:22:07.799634+0000","last_fullsized":"2024-05-08T23:22:07.799634+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:51.525742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797507+0000","last_change":"2024-05-08T23:22:07.324367+0000","last_active":"2024-05-08T23:22:07.797507+0000","last_peered":"2024-05-08T23:22:07.797507+0000","last_clean":"2024-05-08T23:22:07.797507+0000","last_became_active":"2024-05-08T23:22:07.324085+0000","last_became_peered":"2024-05-08T23:22:07.324085+0000","last_unstale":"2024-05-08T23:22:07.797507+0000","last_undegraded":"2024-05-08T23:22:07.797507+0000","last_fullsized":"2024-05-08T23:22:07.797507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:40.830327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797185+0000","last_change":"2024-05-08T23:22:07.794556+0000","last_active":"2024-05-08T23:22:07.797185+0000","last_peered":"2024-05-08T23:22:07.797185+0000","last_clean":"2024-05-08T23:22:07.797185+0000","last_became_active":"2024-05-08T23:22:07.794233+0000","last_became_peered":"2024-05-08T23:22:07.794233+0000","last_unstale":"2024-05-08T23:22:07.797185+0000","last_undegraded":"2024-05-08T23:22:07.797185+0000","last_fullsized":"2024-05-08T23:22:07.797185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:48.075638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.793741+0000","last_change":"2024-05-08T23:22:07.324042+0000","last_active":"2024-05-08T23:22:07.793741+0000","last_peered":"2024-05-08T23:22:07.793741+0000","last_clean":"2024-05-08T23:22:07.793741+0000","last_became_active":"2024-05-08T23:22:07.323538+0000","last_became_peered":"2024-05-08T23:22:07.323538+0000","last_unstale":"2024-05-08T23:22:07.793741+0000","last_undegraded":"2024-05-08T23:22:07.793741+0000","last_fullsized":"2024-05-08T23:22:07.793741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:21.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.788559+0000","last_change":"2024-05-08T23:22:07.786473+0000","last_active":"2024-05-08T23:22:07.788559+0000","last_peered":"2024-05-08T23:22:07.788559+0000","last_clean":"2024-05-08T23:22:07.788559+0000","last_became_active":"2024-05-08T23:22:07.786188+0000","last_became_peered":"2024-05-08T23:22:07.786188+0000","last_unstale":"2024-05-08T23:22:07.788559+0000","last_undegraded":"2024-05-08T23:22:07.788559+0000","last_fullsized":"2024-05-08T23:22:07.788559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:50.357763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.790246+0000","last_change":"2024-05-08T23:22:07.323560+0000","last_active":"2024-05-08T23:22:07.790246+0000","last_peered":"2024-05-08T23:22:07.790246+0000","last_clean":"2024-05-08T23:22:07.790246+0000","last_became_active":"2024-05-08T23:22:07.323249+0000","last_became_peered":"2024-05-08T23:22:07.323249+0000","last_unstale":"2024-05-08T23:22:07.790246+0000","last_undegraded":"2024-05-08T23:22:07.790246+0000","last_fullsized":"2024-05-08T23:22:07.790246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:04.715359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.793070+0000","last_change":"2024-05-08T23:22:07.785147+0000","last_active":"2024-05-08T23:22:07.793070+0000","last_peered":"2024-05-08T23:22:07.793070+0000","last_clean":"2024-05-08T23:22:07.793070+0000","last_became_active":"2024-05-08T23:22:07.784830+0000","last_became_peered":"2024-05-08T23:22:07.784830+0000","last_unstale":"2024-05-08T23:22:07.793070+0000","last_undegraded":"2024-05-08T23:22:07.793070+0000","last_fullsized":"2024-05-08T23:22:07.793070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:52.748765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.776278+0000","last_change":"2024-05-08T23:22:07.320297+0000","last_active":"2024-05-08T23:22:07.776278+0000","last_peered":"2024-05-08T23:22:07.776278+0000","last_clean":"2024-05-08T23:22:07.776278+0000","last_became_active":"2024-05-08T23:22:07.320014+0000","last_became_peered":"2024-05-08T23:22:07.320014+0000","last_unstale":"2024-05-08T23:22:07.776278+0000","last_undegraded":"2024-05-08T23:22:07.776278+0000","last_fullsized":"2024-05-08T23:22:07.776278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:51.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797023+0000","last_change":"2024-05-08T23:22:07.788761+0000","last_active":"2024-05-08T23:22:07.797023+0000","last_peered":"2024-05-08T23:22:07.797023+0000","last_clean":"2024-05-08T23:22:07.797023+0000","last_became_active":"2024-05-08T23:22:07.786643+0000","last_became_peered":"2024-05-08T23:22:07.786643+0000","last_unstale":"2024-05-08T23:22:07.797023+0000","last_undegraded":"2024-05-08T23:22:07.797023+0000","last_fullsized":"2024-05-08T23:22:07.797023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.908028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.772842+0000","last_change":"2024-05-08T23:22:07.326385+0000","last_active":"2024-05-08T23:22:07.772842+0000","last_peered":"2024-05-08T23:22:07.772842+0000","last_clean":"2024-05-08T23:22:07.772842+0000","last_became_active":"2024-05-08T23:22:07.326047+0000","last_became_peered":"2024-05-08T23:22:07.326047+0000","last_unstale":"2024-05-08T23:22:07.772842+0000","last_undegraded":"2024-05-08T23:22:07.772842+0000","last_fullsized":"2024-05-08T23:22:07.772842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:46.848050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.785753+0000","last_change":"2024-05-08T23:22:07.782563+0000","last_active":"2024-05-08T23:22:07.785753+0000","last_peered":"2024-05-08T23:22:07.785753+0000","last_clean":"2024-05-08T23:22:07.785753+0000","last_became_active":"2024-05-08T23:22:07.782290+0000","last_became_peered":"2024-05-08T23:22:07.782290+0000","last_unstale":"2024-05-08T23:22:07.785753+0000","last_undegraded":"2024-05-08T23:22:07.785753+0000","last_fullsized":"2024-05-08T23:22:07.785753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:41.329227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.776767+0000","last_change":"2024-05-08T23:22:07.324876+0000","last_active":"2024-05-08T23:22:07.776767+0000","last_peered":"2024-05-08T23:22:07.776767+0000","last_clean":"2024-05-08T23:22:07.776767+0000","last_became_active":"2024-05-08T23:22:07.324583+0000","last_became_peered":"2024-05-08T23:22:07.324583+0000","last_unstale":"2024-05-08T23:22:07.776767+0000","last_undegraded":"2024-05-08T23:22:07.776767+0000","last_fullsized":"2024-05-08T23:22:07.776767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:00.240669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799311+0000","last_change":"2024-05-08T23:22:07.795071+0000","last_active":"2024-05-08T23:22:07.799311+0000","last_peered":"2024-05-08T23:22:07.799311+0000","last_clean":"2024-05-08T23:22:07.799311+0000","last_became_active":"2024-05-08T23:22:07.793871+0000","last_became_peered":"2024-05-08T23:22:07.793871+0000","last_unstale":"2024-05-08T23:22:07.799311+0000","last_undegraded":"2024-05-08T23:22:07.799311+0000","last_fullsized":"2024-05-08T23:22:07.799311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:19:08.578978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.770219+0000","last_change":"2024-05-08T23:22:07.324516+0000","last_active":"2024-05-08T23:22:07.770219+0000","last_peered":"2024-05-08T23:22:07.770219+0000","last_clean":"2024-05-08T23:22:07.770219+0000","last_became_active":"2024-05-08T23:22:07.323212+0000","last_became_peered":"2024-05-08T23:22:07.323212+0000","last_unstale":"2024-05-08T23:22:07.770219+0000","last_undegraded":"2024-05-08T23:22:07.770219+0000","last_fullsized":"2024-05-08T23:22:07.770219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:37.291699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799829+0000","last_change":"2024-05-08T23:22:07.795115+0000","last_active":"2024-05-08T23:22:07.799829+0000","last_peered":"2024-05-08T23:22:07.799829+0000","last_clean":"2024-05-08T23:22:07.799829+0000","last_became_active":"2024-05-08T23:22:07.794815+0000","last_became_peered":"2024-05-08T23:22:07.794815+0000","last_unstale":"2024-05-08T23:22:07.799829+0000","last_undegraded":"2024-05-08T23:22:07.799829+0000","last_fullsized":"2024-05-08T23:22:07.799829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:33.565554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.794828+0000","last_change":"2024-05-08T23:22:07.324920+0000","last_active":"2024-05-08T23:22:07.794828+0000","last_peered":"2024-05-08T23:22:07.794828+0000","last_clean":"2024-05-08T23:22:07.794828+0000","last_became_active":"2024-05-08T23:22:07.324655+0000","last_became_peered":"2024-05-08T23:22:07.324655+0000","last_unstale":"2024-05-08T23:22:07.794828+0000","last_undegraded":"2024-05-08T23:22:07.794828+0000","last_fullsized":"2024-05-08T23:22:07.794828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:40.686112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796166+0000","last_change":"2024-05-08T23:22:07.790463+0000","last_active":"2024-05-08T23:22:07.796166+0000","last_peered":"2024-05-08T23:22:07.796166+0000","last_clean":"2024-05-08T23:22:07.796166+0000","last_became_active":"2024-05-08T23:22:07.790181+0000","last_became_peered":"2024-05-08T23:22:07.790181+0000","last_unstale":"2024-05-08T23:22:07.796166+0000","last_undegraded":"2024-05-08T23:22:07.796166+0000","last_fullsized":"2024-05-08T23:22:07.796166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:32.443021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.801567+0000","last_change":"2024-05-08T23:22:07.324702+0000","last_active":"2024-05-08T23:22:07.801567+0000","last_peered":"2024-05-08T23:22:07.801567+0000","last_clean":"2024-05-08T23:22:07.801567+0000","last_became_active":"2024-05-08T23:22:07.323911+0000","last_became_peered":"2024-05-08T23:22:07.323911+0000","last_unstale":"2024-05-08T23:22:07.801567+0000","last_undegraded":"2024-05-08T23:22:07.801567+0000","last_fullsized":"2024-05-08T23:22:07.801567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:28.458137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799010+0000","last_change":"2024-05-08T23:22:07.786765+0000","last_active":"2024-05-08T23:22:07.799010+0000","last_peered":"2024-05-08T23:22:07.799010+0000","last_clean":"2024-05-08T23:22:07.799010+0000","last_became_active":"2024-05-08T23:22:07.786447+0000","last_became_peered":"2024-05-08T23:22:07.786447+0000","last_unstale":"2024-05-08T23:22:07.799010+0000","last_undegraded":"2024-05-08T23:22:07.799010+0000","last_fullsized":"2024-05-08T23:22:07.799010+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:01.245597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796364+0000","last_change":"2024-05-08T23:22:07.319729+0000","last_active":"2024-05-08T23:22:07.796364+0000","last_peered":"2024-05-08T23:22:07.796364+0000","last_clean":"2024-05-08T23:22:07.796364+0000","last_became_active":"2024-05-08T23:22:07.319420+0000","last_became_peered":"2024-05-08T23:22:07.319420+0000","last_unstale":"2024-05-08T23:22:07.796364+0000","last_undegraded":"2024-05-08T23:22:07.796364+0000","last_fullsized":"2024-05-08T23:22:07.796364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:13.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.787794+0000","last_change":"2024-05-08T23:22:07.785658+0000","last_active":"2024-05-08T23:22:07.787794+0000","last_peered":"2024-05-08T23:22:07.787794+0000","last_clean":"2024-05-08T23:22:07.787794+0000","last_became_active":"2024-05-08T23:22:07.784171+0000","last_became_peered":"2024-05-08T23:22:07.784171+0000","last_unstale":"2024-05-08T23:22:07.787794+0000","last_undegraded":"2024-05-08T23:22:07.787794+0000","last_fullsized":"2024-05-08T23:22:07.787794+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:10.382589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.772252+0000","last_change":"2024-05-08T23:22:07.327526+0000","last_active":"2024-05-08T23:22:07.772252+0000","last_peered":"2024-05-08T23:22:07.772252+0000","last_clean":"2024-05-08T23:22:07.772252+0000","last_became_active":"2024-05-08T23:22:07.327240+0000","last_became_peered":"2024-05-08T23:22:07.327240+0000","last_unstale":"2024-05-08T23:22:07.772252+0000","last_undegraded":"2024-05-08T23:22:07.772252+0000","last_fullsized":"2024-05-08T23:22:07.772252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799655+0000","last_change":"2024-05-08T23:22:07.797083+0000","last_active":"2024-05-08T23:22:07.799655+0000","last_peered":"2024-05-08T23:22:07.799655+0000","last_clean":"2024-05-08T23:22:07.799655+0000","last_became_active":"2024-05-08T23:22:07.796233+0000","last_became_peered":"2024-05-08T23:22:07.796233+0000","last_unstale":"2024-05-08T23:22:07.799655+0000","last_undegraded":"2024-05-08T23:22:07.799655+0000","last_fullsized":"2024-05-08T23:22:07.799655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:32.443021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.776071+0000","last_change":"2024-05-08T23:22:07.324290+0000","last_active":"2024-05-08T23:22:07.776071+0000","last_peered":"2024-05-08T23:22:07.776071+0000","last_clean":"2024-05-08T23:22:07.776071+0000","last_became_active":"2024-05-08T23:22:07.323929+0000","last_became_peered":"2024-05-08T23:22:07.323929+0000","last_unstale":"2024-05-08T23:22:07.776071+0000","last_undegraded":"2024-05-08T23:22:07.776071+0000","last_fullsized":"2024-05-08T23:22:07.776071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:17.483564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796592+0000","last_change":"2024-05-08T23:22:07.794917+0000","last_active":"2024-05-08T23:22:07.796592+0000","last_peered":"2024-05-08T23:22:07.796592+0000","last_clean":"2024-05-08T23:22:07.796592+0000","last_became_active":"2024-05-08T23:22:07.793482+0000","last_became_peered":"2024-05-08T23:22:07.793482+0000","last_unstale":"2024-05-08T23:22:07.796592+0000","last_undegraded":"2024-05-08T23:22:07.796592+0000","last_fullsized":"2024-05-08T23:22:07.796592+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799707+0000","last_change":"2024-05-08T23:22:07.323609+0000","last_active":"2024-05-08T23:22:07.799707+0000","last_peered":"2024-05-08T23:22:07.799707+0000","last_clean":"2024-05-08T23:22:07.799707+0000","last_became_active":"2024-05-08T23:22:07.323344+0000","last_became_peered":"2024-05-08T23:22:07.323344+0000","last_unstale":"2024-05-08T23:22:07.799707+0000","last_undegraded":"2024-05-08T23:22:07.799707+0000","last_fullsized":"2024-05-08T23:22:07.799707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:41.903099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792697+0000","last_change":"2024-05-08T23:22:07.783864+0000","last_active":"2024-05-08T23:22:07.792697+0000","last_peered":"2024-05-08T23:22:07.792697+0000","last_clean":"2024-05-08T23:22:07.792697+0000","last_became_active":"2024-05-08T23:22:07.783242+0000","last_became_peered":"2024-05-08T23:22:07.783242+0000","last_unstale":"2024-05-08T23:22:07.792697+0000","last_undegraded":"2024-05-08T23:22:07.792697+0000","last_fullsized":"2024-05-08T23:22:07.792697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:20.241779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.800054+0000","last_change":"2024-05-08T23:22:07.324272+0000","last_active":"2024-05-08T23:22:07.800054+0000","last_peered":"2024-05-08T23:22:07.800054+0000","last_clean":"2024-05-08T23:22:07.800054+0000","last_became_active":"2024-05-08T23:22:07.324005+0000","last_became_peered":"2024-05-08T23:22:07.324005+0000","last_unstale":"2024-05-08T23:22:07.800054+0000","last_undegraded":"2024-05-08T23:22:07.800054+0000","last_fullsized":"2024-05-08T23:22:07.800054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796807+0000","last_change":"2024-05-08T23:22:07.792097+0000","last_active":"2024-05-08T23:22:07.796807+0000","last_peered":"2024-05-08T23:22:07.796807+0000","last_clean":"2024-05-08T23:22:07.796807+0000","last_became_active":"2024-05-08T23:22:07.791645+0000","last_became_peered":"2024-05-08T23:22:07.791645+0000","last_unstale":"2024-05-08T23:22:07.796807+0000","last_undegraded":"2024-05-08T23:22:07.796807+0000","last_fullsized":"2024-05-08T23:22:07.796807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:31.969535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792384+0000","last_change":"2024-05-08T23:22:07.320014+0000","last_active":"2024-05-08T23:22:07.792384+0000","last_peered":"2024-05-08T23:22:07.792384+0000","last_clean":"2024-05-08T23:22:07.792384+0000","last_became_active":"2024-05-08T23:22:07.319073+0000","last_became_peered":"2024-05-08T23:22:07.319073+0000","last_unstale":"2024-05-08T23:22:07.792384+0000","last_undegraded":"2024-05-08T23:22:07.792384+0000","last_fullsized":"2024-05-08T23:22:07.792384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.661583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.784131+0000","last_change":"2024-05-08T23:22:07.779107+0000","last_active":"2024-05-08T23:22:07.784131+0000","last_peered":"2024-05-08T23:22:07.784131+0000","last_clean":"2024-05-08T23:22:07.784131+0000","last_became_active":"2024-05-08T23:22:07.778839+0000","last_became_peered":"2024-05-08T23:22:07.778839+0000","last_unstale":"2024-05-08T23:22:07.784131+0000","last_undegraded":"2024-05-08T23:22:07.784131+0000","last_fullsized":"2024-05-08T23:22:07.784131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:18.488493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792867+0000","last_change":"2024-05-08T23:22:07.325295+0000","last_active":"2024-05-08T23:22:07.792867+0000","last_peered":"2024-05-08T23:22:07.792867+0000","last_clean":"2024-05-08T23:22:07.792867+0000","last_became_active":"2024-05-08T23:22:07.324997+0000","last_became_peered":"2024-05-08T23:22:07.324997+0000","last_unstale":"2024-05-08T23:22:07.792867+0000","last_undegraded":"2024-05-08T23:22:07.792867+0000","last_fullsized":"2024-05-08T23:22:07.792867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:56.035940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796024+0000","last_change":"2024-05-08T23:22:07.794794+0000","last_active":"2024-05-08T23:22:07.796024+0000","last_peered":"2024-05-08T23:22:07.796024+0000","last_clean":"2024-05-08T23:22:07.796024+0000","last_became_active":"2024-05-08T23:22:07.794503+0000","last_became_peered":"2024-05-08T23:22:07.794503+0000","last_unstale":"2024-05-08T23:22:07.796024+0000","last_undegraded":"2024-05-08T23:22:07.796024+0000","last_fullsized":"2024-05-08T23:22:07.796024+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:39.814208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.774719+0000","last_change":"2024-05-08T23:22:07.323547+0000","last_active":"2024-05-08T23:22:07.774719+0000","last_peered":"2024-05-08T23:22:07.774719+0000","last_clean":"2024-05-08T23:22:07.774719+0000","last_became_active":"2024-05-08T23:22:07.323266+0000","last_became_peered":"2024-05-08T23:22:07.323266+0000","last_unstale":"2024-05-08T23:22:07.774719+0000","last_undegraded":"2024-05-08T23:22:07.774719+0000","last_fullsized":"2024-05-08T23:22:07.774719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.797211+0000","last_change":"2024-05-08T23:22:07.795639+0000","last_active":"2024-05-08T23:22:07.797211+0000","last_peered":"2024-05-08T23:22:07.797211+0000","last_clean":"2024-05-08T23:22:07.797211+0000","last_became_active":"2024-05-08T23:22:07.795353+0000","last_became_peered":"2024-05-08T23:22:07.795353+0000","last_unstale":"2024-05-08T23:22:07.797211+0000","last_undegraded":"2024-05-08T23:22:07.797211+0000","last_fullsized":"2024-05-08T23:22:07.797211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:54:32.601480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.787493+0000","last_change":"2024-05-08T23:22:07.322755+0000","last_active":"2024-05-08T23:22:07.787493+0000","last_peered":"2024-05-08T23:22:07.787493+0000","last_clean":"2024-05-08T23:22:07.787493+0000","last_became_active":"2024-05-08T23:22:07.321921+0000","last_became_peered":"2024-05-08T23:22:07.321921+0000","last_unstale":"2024-05-08T23:22:07.787493+0000","last_undegraded":"2024-05-08T23:22:07.787493+0000","last_fullsized":"2024-05-08T23:22:07.787493+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:29.054724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.787775+0000","last_change":"2024-05-08T23:22:07.785995+0000","last_active":"2024-05-08T23:22:07.787775+0000","last_peered":"2024-05-08T23:22:07.787775+0000","last_clean":"2024-05-08T23:22:07.787775+0000","last_became_active":"2024-05-08T23:22:07.785679+0000","last_became_peered":"2024-05-08T23:22:07.785679+0000","last_unstale":"2024-05-08T23:22:07.787775+0000","last_undegraded":"2024-05-08T23:22:07.787775+0000","last_fullsized":"2024-05-08T23:22:07.787775+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:15.611951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.779401+0000","last_change":"2024-05-08T23:22:07.327553+0000","last_active":"2024-05-08T23:22:07.779401+0000","last_peered":"2024-05-08T23:22:07.779401+0000","last_clean":"2024-05-08T23:22:07.779401+0000","last_became_active":"2024-05-08T23:22:07.327214+0000","last_became_peered":"2024-05-08T23:22:07.327214+0000","last_unstale":"2024-05-08T23:22:07.779401+0000","last_undegraded":"2024-05-08T23:22:07.779401+0000","last_fullsized":"2024-05-08T23:22:07.779401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.789962+0000","last_change":"2024-05-08T23:22:07.788317+0000","last_active":"2024-05-08T23:22:07.789962+0000","last_peered":"2024-05-08T23:22:07.789962+0000","last_clean":"2024-05-08T23:22:07.789962+0000","last_became_active":"2024-05-08T23:22:07.786118+0000","last_became_peered":"2024-05-08T23:22:07.786118+0000","last_unstale":"2024-05-08T23:22:07.789962+0000","last_undegraded":"2024-05-08T23:22:07.789962+0000","last_fullsized":"2024-05-08T23:22:07.789962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:45.661099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.774186+0000","last_change":"2024-05-08T23:22:07.323427+0000","last_active":"2024-05-08T23:22:07.774186+0000","last_peered":"2024-05-08T23:22:07.774186+0000","last_clean":"2024-05-08T23:22:07.774186+0000","last_became_active":"2024-05-08T23:22:07.323163+0000","last_became_peered":"2024-05-08T23:22:07.323163+0000","last_unstale":"2024-05-08T23:22:07.774186+0000","last_undegraded":"2024-05-08T23:22:07.774186+0000","last_fullsized":"2024-05-08T23:22:07.774186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799292+0000","last_change":"2024-05-08T23:22:07.796653+0000","last_active":"2024-05-08T23:22:07.799292+0000","last_peered":"2024-05-08T23:22:07.799292+0000","last_clean":"2024-05-08T23:22:07.799292+0000","last_became_active":"2024-05-08T23:22:07.796401+0000","last_became_peered":"2024-05-08T23:22:07.796401+0000","last_unstale":"2024-05-08T23:22:07.799292+0000","last_undegraded":"2024-05-08T23:22:07.799292+0000","last_fullsized":"2024-05-08T23:22:07.799292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:09.803894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.778060+0000","last_change":"2024-05-08T23:22:07.325578+0000","last_active":"2024-05-08T23:22:07.778060+0000","last_peered":"2024-05-08T23:22:07.778060+0000","last_clean":"2024-05-08T23:22:07.778060+0000","last_became_active":"2024-05-08T23:22:07.325303+0000","last_became_peered":"2024-05-08T23:22:07.325303+0000","last_unstale":"2024-05-08T23:22:07.778060+0000","last_undegraded":"2024-05-08T23:22:07.778060+0000","last_fullsized":"2024-05-08T23:22:07.778060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:04.490589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.789680+0000","last_change":"2024-05-08T23:22:07.788948+0000","last_active":"2024-05-08T23:22:07.789680+0000","last_peered":"2024-05-08T23:22:07.789680+0000","last_clean":"2024-05-08T23:22:07.789680+0000","last_became_active":"2024-05-08T23:22:07.788633+0000","last_became_peered":"2024-05-08T23:22:07.788633+0000","last_unstale":"2024-05-08T23:22:07.789680+0000","last_undegraded":"2024-05-08T23:22:07.789680+0000","last_fullsized":"2024-05-08T23:22:07.789680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:02.041531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799616+0000","last_change":"2024-05-08T23:22:07.330079+0000","last_active":"2024-05-08T23:22:07.799616+0000","last_peered":"2024-05-08T23:22:07.799616+0000","last_clean":"2024-05-08T23:22:07.799616+0000","last_became_active":"2024-05-08T23:22:07.329577+0000","last_became_peered":"2024-05-08T23:22:07.329577+0000","last_unstale":"2024-05-08T23:22:07.799616+0000","last_undegraded":"2024-05-08T23:22:07.799616+0000","last_fullsized":"2024-05-08T23:22:07.799616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:01.036602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.790578+0000","last_change":"2024-05-08T23:22:07.788317+0000","last_active":"2024-05-08T23:22:07.790578+0000","last_peered":"2024-05-08T23:22:07.790578+0000","last_clean":"2024-05-08T23:22:07.790578+0000","last_became_active":"2024-05-08T23:22:07.787987+0000","last_became_peered":"2024-05-08T23:22:07.787987+0000","last_unstale":"2024-05-08T23:22:07.790578+0000","last_undegraded":"2024-05-08T23:22:07.790578+0000","last_fullsized":"2024-05-08T23:22:07.790578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:17.509591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T23:22:08.868641+0000","last_change":"2024-05-08T23:22:07.322623+0000","last_active":"2024-05-08T23:22:08.868641+0000","last_peered":"2024-05-08T23:22:08.868641+0000","last_clean":"2024-05-08T23:22:08.868641+0000","last_became_active":"2024-05-08T23:22:07.322321+0000","last_became_peered":"2024-05-08T23:22:07.322321+0000","last_unstale":"2024-05-08T23:22:08.868641+0000","last_undegraded":"2024-05-08T23:22:08.868641+0000","last_fullsized":"2024-05-08T23:22:08.868641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:14.607022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796559+0000","last_change":"2024-05-08T23:22:07.788010+0000","last_active":"2024-05-08T23:22:07.796559+0000","last_peered":"2024-05-08T23:22:07.796559+0000","last_clean":"2024-05-08T23:22:07.796559+0000","last_became_active":"2024-05-08T23:22:07.787715+0000","last_became_peered":"2024-05-08T23:22:07.787715+0000","last_unstale":"2024-05-08T23:22:07.796559+0000","last_undegraded":"2024-05-08T23:22:07.796559+0000","last_fullsized":"2024-05-08T23:22:07.796559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.775208+0000","last_change":"2024-05-08T23:22:07.327107+0000","last_active":"2024-05-08T23:22:07.775208+0000","last_peered":"2024-05-08T23:22:07.775208+0000","last_clean":"2024-05-08T23:22:07.775208+0000","last_became_active":"2024-05-08T23:22:07.326542+0000","last_became_peered":"2024-05-08T23:22:07.326542+0000","last_unstale":"2024-05-08T23:22:07.775208+0000","last_undegraded":"2024-05-08T23:22:07.775208+0000","last_fullsized":"2024-05-08T23:22:07.775208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:33.446008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.791445+0000","last_change":"2024-05-08T23:22:07.784128+0000","last_active":"2024-05-08T23:22:07.791445+0000","last_peered":"2024-05-08T23:22:07.791445+0000","last_clean":"2024-05-08T23:22:07.791445+0000","last_became_active":"2024-05-08T23:22:07.783822+0000","last_became_peered":"2024-05-08T23:22:07.783822+0000","last_unstale":"2024-05-08T23:22:07.791445+0000","last_undegraded":"2024-05-08T23:22:07.791445+0000","last_fullsized":"2024-05-08T23:22:07.791445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:41.691040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.769881+0000","last_change":"2024-05-08T23:22:07.319178+0000","last_active":"2024-05-08T23:22:07.769881+0000","last_peered":"2024-05-08T23:22:07.769881+0000","last_clean":"2024-05-08T23:22:07.769881+0000","last_became_active":"2024-05-08T23:22:07.318864+0000","last_became_peered":"2024-05-08T23:22:07.318864+0000","last_unstale":"2024-05-08T23:22:07.769881+0000","last_undegraded":"2024-05-08T23:22:07.769881+0000","last_fullsized":"2024-05-08T23:22:07.769881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:26.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796633+0000","last_change":"2024-05-08T23:22:07.787863+0000","last_active":"2024-05-08T23:22:07.796633+0000","last_peered":"2024-05-08T23:22:07.796633+0000","last_clean":"2024-05-08T23:22:07.796633+0000","last_became_active":"2024-05-08T23:22:07.787166+0000","last_became_peered":"2024-05-08T23:22:07.787166+0000","last_unstale":"2024-05-08T23:22:07.796633+0000","last_undegraded":"2024-05-08T23:22:07.796633+0000","last_fullsized":"2024-05-08T23:22:07.796633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:41.972128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.772373+0000","last_change":"2024-05-08T23:22:07.329759+0000","last_active":"2024-05-08T23:22:07.772373+0000","last_peered":"2024-05-08T23:22:07.772373+0000","last_clean":"2024-05-08T23:22:07.772373+0000","last_became_active":"2024-05-08T23:22:07.329423+0000","last_became_peered":"2024-05-08T23:22:07.329423+0000","last_unstale":"2024-05-08T23:22:07.772373+0000","last_undegraded":"2024-05-08T23:22:07.772373+0000","last_fullsized":"2024-05-08T23:22:07.772373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.799765+0000","last_change":"2024-05-08T23:22:07.795295+0000","last_active":"2024-05-08T23:22:07.799765+0000","last_peered":"2024-05-08T23:22:07.799765+0000","last_clean":"2024-05-08T23:22:07.799765+0000","last_became_active":"2024-05-08T23:22:07.793689+0000","last_became_peered":"2024-05-08T23:22:07.793689+0000","last_unstale":"2024-05-08T23:22:07.799765+0000","last_undegraded":"2024-05-08T23:22:07.799765+0000","last_fullsized":"2024-05-08T23:22:07.799765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:55.199701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.774286+0000","last_change":"2024-05-08T23:22:07.326530+0000","last_active":"2024-05-08T23:22:07.774286+0000","last_peered":"2024-05-08T23:22:07.774286+0000","last_clean":"2024-05-08T23:22:07.774286+0000","last_became_active":"2024-05-08T23:22:07.326233+0000","last_became_peered":"2024-05-08T23:22:07.326233+0000","last_unstale":"2024-05-08T23:22:07.774286+0000","last_undegraded":"2024-05-08T23:22:07.774286+0000","last_fullsized":"2024-05-08T23:22:07.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:49.418732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.792793+0000","last_change":"2024-05-08T23:22:07.782326+0000","last_active":"2024-05-08T23:22:07.792793+0000","last_peered":"2024-05-08T23:22:07.792793+0000","last_clean":"2024-05-08T23:22:07.792793+0000","last_became_active":"2024-05-08T23:22:07.782054+0000","last_became_peered":"2024-05-08T23:22:07.782054+0000","last_unstale":"2024-05-08T23:22:07.792793+0000","last_undegraded":"2024-05-08T23:22:07.792793+0000","last_fullsized":"2024-05-08T23:22:07.792793+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:55.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.776058+0000","last_change":"2024-05-08T23:22:07.325642+0000","last_active":"2024-05-08T23:22:07.776058+0000","last_peered":"2024-05-08T23:22:07.776058+0000","last_clean":"2024-05-08T23:22:07.776058+0000","last_became_active":"2024-05-08T23:22:07.325384+0000","last_became_peered":"2024-05-08T23:22:07.325384+0000","last_unstale":"2024-05-08T23:22:07.776058+0000","last_undegraded":"2024-05-08T23:22:07.776058+0000","last_fullsized":"2024-05-08T23:22:07.776058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:26.489244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.796437+0000","last_change":"2024-05-08T23:22:07.792644+0000","last_active":"2024-05-08T23:22:07.796437+0000","last_peered":"2024-05-08T23:22:07.796437+0000","last_clean":"2024-05-08T23:22:07.796437+0000","last_became_active":"2024-05-08T23:22:07.792347+0000","last_became_peered":"2024-05-08T23:22:07.792347+0000","last_unstale":"2024-05-08T23:22:07.796437+0000","last_undegraded":"2024-05-08T23:22:07.796437+0000","last_fullsized":"2024-05-08T23:22:07.796437+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:07.299109+0000","last_clean_scrub_stamp":"2024-05-08T23:22:07.299109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:52.143215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.778375+0000","last_change":"2024-05-08T23:22:07.328621+0000","last_active":"2024-05-08T23:22:07.778375+0000","last_peered":"2024-05-08T23:22:07.778375+0000","last_clean":"2024-05-08T23:22:07.778375+0000","last_became_active":"2024-05-08T23:22:07.328297+0000","last_became_peered":"2024-05-08T23:22:07.328297+0000","last_unstale":"2024-05-08T23:22:07.778375+0000","last_undegraded":"2024-05-08T23:22:07.778375+0000","last_fullsized":"2024-05-08T23:22:07.778375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:22:06.294180+0000","last_clean_scrub_stamp":"2024-05-08T23:22:06.294180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.656170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-08T23:22:07.779334+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:22:07.779334+0000","last_peered":"2024-05-08T23:22:07.779334+0000","last_clean":"2024-05-08T23:22:07.779334+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:22:07.779334+0000","last_undegraded":"2024-05-08T23:22:07.779334+0000","last_fullsized":"2024-05-08T23:22:07.779334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":742523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.496}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.432},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64200000000000002}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":152681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.495},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.55600000000000005}]},{"osd":3,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.438},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed May 8 23:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":153651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.29099999999999998}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.372},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.38500000000000001}]},{"osd":4,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.499},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.73099999999999998}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.75}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":978944,"data_stored":742433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":7.9889999999999999,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.7869999999999999,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.0519999999999996,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.30499999999999999}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":7.6440000000000001,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.9139999999999997,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.438},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.42299999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":7.1120000000000001,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":7.6970000000000001,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.734,"5min":7.734,"15min":7.734},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0220000000000002,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.46999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":152135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32200000000000001}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.36399999999999999}]},{"osd":3,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.70899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":153651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.373},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.316}]},{"osd":2,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":11.964,"5min":11.964,"15min":11.964},"last":0.438},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.501},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed May 8 23:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.57499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":152169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.0389999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.127000000000001,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.62}]},{"osd":5,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.496}]},{"osd":6,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.47599999999999998}]},{"osd":7,"last update":"Wed May 8 23:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.441}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":152737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.6710000000000003,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":0.67300000000000004}]},{"osd":1,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":4.71,"5min":4.71,"15min":4.71},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":8.6880000000000006,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.625}]},{"osd":2,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.8940000000000001,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8559999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.66100000000000003}]},{"osd":3,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.8770000000000007,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.427,"5min":10.427,"15min":10.427},"last":0.64400000000000002}]},{"osd":4,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.441}]},{"osd":6,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed May 8 23:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.499},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:22:17.196 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:22:17.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T23:22:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.378+0000 7ff32c176700 1 Processor -- start 2024-05-08T23:22:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.379+0000 7ff32c176700 1 -- start start 2024-05-08T23:22:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.379+0000 7ff32c176700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff32415db00 0x7ff3240cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.379+0000 7ff32c176700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32406ad20 con 0x7ff32406f990 2024-05-08T23:22:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.379+0000 7ff32c176700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32406b000 con 0x7ff3240caa50 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.379+0000 7ff32c176700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32415dee0 con 0x7ff32415db00 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff329f12700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff32415db00 0x7ff3240cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff32a713700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff32406f990 0x7ff32406e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43902/0 (socket says 172.21.15.5:43902) 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff32a713700 1 -- 172.21.15.5:0/1901331184 learned_addr learned my addr 172.21.15.5:0/1901331184 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff329f12700 1 -- 172.21.15.5:0/1901331184 >> v1:172.21.15.49:6789/0 conn(0x7ff3240caa50 legacy=0x7ff32406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff329f12700 1 -- 172.21.15.5:0/1901331184 >> v1:172.21.15.5:6789/0 conn(0x7ff32406f990 legacy=0x7ff32406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff329f12700 1 -- 172.21.15.5:0/1901331184 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32406b2e0 con 0x7ff32415db00 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff328f10700 1 -- 172.21.15.5:0/1901331184 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162755647 0 0) 0x7ff32406ad20 con 0x7ff32406f990 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff329f12700 1 --2- 172.21.15.5:0/1901331184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32415db00 0x7ff3240cc590 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff320009840 tx=0x7ff320009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8e81ccb99fe6c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff328f10700 1 -- 172.21.15.5:0/1901331184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff320011830 con 0x7ff32415db00 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff328f10700 1 -- 172.21.15.5:0/1901331184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff320011990 con 0x7ff32415db00 2024-05-08T23:22:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.380+0000 7ff328f10700 1 -- 172.21.15.5:0/1901331184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff320011d20 con 0x7ff32415db00 2024-05-08T23:22:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.381+0000 7ff32c176700 1 -- 172.21.15.5:0/1901331184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32415db00 msgr2=0x7ff3240cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.381+0000 7ff32c176700 1 --2- 172.21.15.5:0/1901331184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32415db00 0x7ff3240cc590 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff320009840 tx=0x7ff320009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.381+0000 7ff32c176700 1 -- 172.21.15.5:0/1901331184 shutdown_connections 2024-05-08T23:22:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.381+0000 7ff32c176700 1 --2- 172.21.15.5:0/1901331184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32415db00 0x7ff3240cc590 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.381+0000 7ff32c176700 1 -- 172.21.15.5:0/1901331184 >> 172.21.15.5:0/1901331184 conn(0x7ff324099a50 msgr2=0x7ff3240d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.381+0000 7ff32c176700 1 -- 172.21.15.5:0/1901331184 shutdown_connections 2024-05-08T23:22:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.381+0000 7ff32c176700 1 -- 172.21.15.5:0/1901331184 wait complete. 2024-05-08T23:22:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.382+0000 7ff32c176700 1 Processor -- start 2024-05-08T23:22:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.382+0000 7ff32c176700 1 -- start start 2024-05-08T23:22:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff32c176700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32406f990 0x7ff32414b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff32c176700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3240caa50 0x7ff3240b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff32c176700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32415db00 0x7ff3240c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff32c176700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32414be20 con 0x7ff32406f990 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff32c176700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240c7600 con 0x7ff32415db00 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff32c176700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240c78e0 con 0x7ff3240caa50 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff329f12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32406f990 0x7ff32414b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff329f12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32406f990 0x7ff32414b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47286/0 (socket says 172.21.15.5:47286) 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.383+0000 7ff329f12700 1 -- 172.21.15.5:0/708091465 learned_addr learned my addr 172.21.15.5:0/708091465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff32a713700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32415db00 0x7ff3240c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff329711700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3240caa50 0x7ff3240b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff329f12700 1 -- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3240caa50 msgr2=0x7ff3240b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff329f12700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3240caa50 0x7ff3240b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff329f12700 1 -- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32415db00 msgr2=0x7ff3240c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff329f12700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32415db00 0x7ff3240c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff329f12700 1 -- 172.21.15.5:0/708091465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200094a0 con 0x7ff32406f990 2024-05-08T23:22:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff329f12700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32406f990 0x7ff32414b8e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff320014040 tx=0x7ff320004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff3167fc700 1 -- 172.21.15.5:0/708091465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32001a7e0 con 0x7ff32406f990 2024-05-08T23:22:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff3167fc700 1 -- 172.21.15.5:0/708091465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32001a940 con 0x7ff32406f990 2024-05-08T23:22:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3240c7bc0 con 0x7ff32406f990 2024-05-08T23:22:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff3167fc700 1 -- 172.21.15.5:0/708091465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32001acb0 con 0x7ff32406f990 2024-05-08T23:22:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.384+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3240c8180 con 0x7ff32406f990 2024-05-08T23:22:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.385+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32415d640 con 0x7ff32406f990 2024-05-08T23:22:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.386+0000 7ff3167fc700 1 -- 172.21.15.5:0/708091465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff320022070 con 0x7ff32406f990 2024-05-08T23:22:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.386+0000 7ff3167fc700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff30806ac90 0x7ff30806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.386+0000 7ff3167fc700 1 -- 172.21.15.5:0/708091465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff32008a500 con 0x7ff32406f990 2024-05-08T23:22:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.387+0000 7ff329711700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff30806ac90 0x7ff30806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.387+0000 7ff329711700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff30806ac90 0x7ff30806d150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff32415bc30 tx=0x7ff318009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.391+0000 7ff3167fc700 1 -- 172.21.15.5:0/708091465 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff3200556d0 con 0x7ff32406f990 2024-05-08T23:22:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.697+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff324074bc0 con 0x7ff32406f990 2024-05-08T23:22:17.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.701+0000 7ff3167fc700 1 -- 172.21.15.5:0/708091465 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff320058cf0 con 0x7ff32406f990 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff30806ac90 msgr2=0x7ff30806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff30806ac90 0x7ff30806d150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff32415bc30 tx=0x7ff318009040 comp rx=0 tx=0).stop 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32406f990 msgr2=0x7ff32414b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32406f990 0x7ff32414b8e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff320014040 tx=0x7ff320004750 comp rx=0 tx=0).stop 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 shutdown_connections 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff30806ac90 0x7ff30806d150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32415db00 0x7ff3240c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff32406f990 0x7ff32414b8e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 --2- 172.21.15.5:0/708091465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff3240caa50 0x7ff3240b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 >> 172.21.15.5:0/708091465 conn(0x7ff324099a50 msgr2=0x7ff3240cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 shutdown_connections 2024-05-08T23:22:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.703+0000 7ff32c176700 1 -- 172.21.15.5:0/708091465 wait complete. 2024-05-08T23:22:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-08T23:22:17.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:22:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.856+0000 7febdb3fb700 1 Processor -- start 2024-05-08T23:22:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.857+0000 7febdb3fb700 1 -- start start 2024-05-08T23:22:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.857+0000 7febdb3fb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd40ccbd0 0x7febd4168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.857+0000 7febdb3fb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd409a870 con 0x7febd4161030 2024-05-08T23:22:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.857+0000 7febdb3fb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40d0840 con 0x7febd40b2480 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.857+0000 7febdb3fb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd406ccc0 con 0x7febd40ccbd0 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febd9197700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7febd4161030 0x7febd4099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37534/0 (socket says 172.21.15.5:37534) 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febd9197700 1 -- 172.21.15.5:0/1026379267 learned_addr learned my addr 172.21.15.5:0/1026379267 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febd9998700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd40ccbd0 0x7febd4168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400092852 0 0) 0x7febd409a870 con 0x7febd4161030 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc00035b0 con 0x7febd4161030 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677930179 0 0) 0x7febc00035b0 con 0x7febd4161030 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd40ccbd0 msgr2=0x7febd4168580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd40ccbd0 0x7febd4168580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 >> v1:172.21.15.49:6789/0 conn(0x7febd40b2480 legacy=0x7febd409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd406c600 con 0x7febd4161030 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7febd0001670 con 0x7febd4161030 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 >> v1:172.21.15.5:6789/0 conn(0x7febd4161030 legacy=0x7febd4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc0003a30 0x7febc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc00043e0 0x7febc00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd40ccbd0 0x7febc0005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd406ccc0 con 0x7febc0003a30 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc000ba10 con 0x7febc00043e0 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.858+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc000bbd0 con 0x7febd40ccbd0 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd9197700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc0003a30 0x7febc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd8996700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc00043e0 0x7febc00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd9998700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd40ccbd0 0x7febc0005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd9197700 1 -- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd40ccbd0 msgr2=0x7febc0005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd9197700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd40ccbd0 0x7febc0005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd9197700 1 -- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc00043e0 msgr2=0x7febc00048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd9197700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc00043e0 0x7febc00048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd9197700 1 -- 172.21.15.5:0/1026379267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc00035b0 con 0x7febc0003a30 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febd9197700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc0003a30 0x7febc0003e10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7febd000c2f0 tx=0x7febc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62726299fd86fbbe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000ce30 con 0x7febc0003a30 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd0005960 con 0x7febc0003a30 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.859+0000 7febcbfff700 1 -- 172.21.15.5:0/1026379267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0005cf0 con 0x7febc0003a30 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.860+0000 7febdb3fb700 1 -- 172.21.15.5:0/1026379267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc0003a30 msgr2=0x7febc0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.860+0000 7febdb3fb700 1 --2- 172.21.15.5:0/1026379267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc0003a30 0x7febc0003e10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7febd000c2f0 tx=0x7febc0005880 comp rx=0 tx=0).stop 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.860+0000 7febd9998700 1 -- 172.21.15.5:0/1026379267 reap_dead start 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.860+0000 7febdb3fb700 1 -- 172.21.15.5:0/1026379267 shutdown_connections 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.860+0000 7febdb3fb700 1 -- 172.21.15.5:0/1026379267 >> 172.21.15.5:0/1026379267 conn(0x7febd40b2f20 msgr2=0x7febd40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.860+0000 7febdb3fb700 1 -- 172.21.15.5:0/1026379267 shutdown_connections 2024-05-08T23:22:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.860+0000 7febdb3fb700 1 -- 172.21.15.5:0/1026379267 wait complete. 2024-05-08T23:22:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.861+0000 7febdb3fb700 1 Processor -- start 2024-05-08T23:22:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.861+0000 7febdb3fb700 1 -- start start 2024-05-08T23:22:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febdb3fb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc0003a30 0x7febd40c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febdb3fb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd41553b0 0x7febd4154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febdb3fb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd40c7370 0x7febd40c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febdb3fb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40d0660 con 0x7febd40c7370 2024-05-08T23:22:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febdb3fb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40c53d0 con 0x7febc0003a30 2024-05-08T23:22:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febdb3fb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40c5e00 con 0x7febd41553b0 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd9998700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd40c7370 0x7febd40c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd9998700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd40c7370 0x7febd40c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57144/0 (socket says 172.21.15.5:57144) 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd9998700 1 -- 172.21.15.5:0/959065736 learned_addr learned my addr 172.21.15.5:0/959065736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd8996700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd41553b0 0x7febd4154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd9197700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc0003a30 0x7febd40c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd9998700 1 -- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd41553b0 msgr2=0x7febd4154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd9998700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd41553b0 0x7febd4154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd9998700 1 -- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc0003a30 msgr2=0x7febd40c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.862+0000 7febd9998700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc0003a30 0x7febd40c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.863+0000 7febd9998700 1 -- 172.21.15.5:0/959065736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40c5840 con 0x7febd40c7370 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.863+0000 7febd9998700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd40c7370 0x7febd40c7820 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7febd40cb7c0 tx=0x7febcc00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.863+0000 7febc9ffb700 1 -- 172.21.15.5:0/959065736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00e040 con 0x7febd40c7370 2024-05-08T23:22:17.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.863+0000 7febc9ffb700 1 -- 172.21.15.5:0/959065736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00f040 con 0x7febd40c7370 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.863+0000 7febc9ffb700 1 -- 172.21.15.5:0/959065736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00fc30 con 0x7febd40c7370 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.863+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40c5b20 con 0x7febd40c7370 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.863+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd415dc60 con 0x7febd40c7370 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.864+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd415de80 con 0x7febd40c7370 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.865+0000 7febc9ffb700 1 -- 172.21.15.5:0/959065736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7febcc00fd90 con 0x7febd40c7370 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.865+0000 7febc9ffb700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7febb806ac90 0x7febb806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.865+0000 7febc9ffb700 1 -- 172.21.15.5:0/959065736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febcc089920 con 0x7febd40c7370 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.865+0000 7febd9197700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7febb806ac90 0x7febb806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.866+0000 7febd9197700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7febb806ac90 0x7febb806d150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7febd0008fd0 tx=0x7febd0007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:17.870+0000 7febc9ffb700 1 -- 172.21.15.5:0/959065736 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7febcc054b90 con 0x7febd40c7370 2024-05-08T23:22:18.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.133+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febd406d2c0 con 0x7febd40c7370 2024-05-08T23:22:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.134+0000 7febc9ffb700 1 -- 172.21.15.5:0/959065736 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4719 (secure 0 0 0) 0x7febcc0581b0 con 0x7febd40c7370 2024-05-08T23:22:18.135 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:18.135 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:10.777230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4975},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:22:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7febb806ac90 msgr2=0x7febb806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7febb806ac90 0x7febb806d150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7febd0008fd0 tx=0x7febd0007000 comp rx=0 tx=0).stop 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd40c7370 msgr2=0x7febd40c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd40c7370 0x7febd40c7820 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7febd40cb7c0 tx=0x7febcc00f490 comp rx=0 tx=0).stop 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 shutdown_connections 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7febb806ac90 0x7febb806d150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd40c7370 0x7febd40c7820 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febc0003a30 0x7febd40c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 --2- 172.21.15.5:0/959065736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febd41553b0 0x7febd4154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 >> 172.21.15.5:0/959065736 conn(0x7febd40b2f20 msgr2=0x7febd406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 shutdown_connections 2024-05-08T23:22:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.136+0000 7febdb3fb700 1 -- 172.21.15.5:0/959065736 wait complete. 2024-05-08T23:22:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 15 2024-05-08T23:22:18.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:22:07.766519+0000', 'modified': '2024-05-08T23:22:10.777230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4975}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4975': {'gid': 4975, 'name': 'd', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6837/886789668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6835', 'nonce': 886789668}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 886789668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:22:18.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:22:18.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T23:22:18.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.699+0000 7f9cd452d700 1 Processor -- start 2024-05-08T23:22:18.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.699+0000 7f9cd452d700 1 -- start start 2024-05-08T23:22:18.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.700+0000 7f9cd452d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ccc15a770 0x7f9ccc0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.700+0000 7f9cd452d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9ccc0cea20 0x7f9ccc0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.700+0000 7f9cd452d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0ccbd0 0x7f9ccc0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.700+0000 7f9cd452d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc0caa30 con 0x7f9ccc0cea20 2024-05-08T23:22:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.700+0000 7f9cd452d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc0cf790 con 0x7f9ccc15a770 2024-05-08T23:22:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.700+0000 7f9cd452d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0cfa70 con 0x7f9ccc0ccbd0 2024-05-08T23:22:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9ccc0cea20 0x7f9ccc0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9ccc0cea20 0x7f9ccc0cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57158/0 (socket says 172.21.15.5:57158) 2024-05-08T23:22:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 -- 172.21.15.5:0/2487868564 learned_addr learned my addr 172.21.15.5:0/2487868564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/2487868564 >> v2:172.21.15.49:3300/0 conn(0x7f9ccc15a770 0x7f9ccc0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/2487868564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0ccbd0 0x7f9ccc0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 -- 172.21.15.5:0/2487868564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0ccbd0 msgr2=0x7f9ccc0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/2487868564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0ccbd0 0x7f9ccc0cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 -- 172.21.15.5:0/2487868564 >> v2:172.21.15.49:3300/0 conn(0x7f9ccc15a770 msgr2=0x7f9ccc0ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/2487868564 >> v2:172.21.15.49:3300/0 conn(0x7f9ccc15a770 0x7f9ccc0ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 -- 172.21.15.5:0/2487868564 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0cfd50 con 0x7f9ccc0cea20 2024-05-08T23:22:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/2487868564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc0cea20 0x7f9ccc0cee80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc071780 tx=0x7f9cc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc50214c70379a8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd12c7700 1 -- 172.21.15.5:0/2487868564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0011830 con 0x7f9ccc0cea20 2024-05-08T23:22:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd12c7700 1 -- 172.21.15.5:0/2487868564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc0011990 con 0x7f9ccc0cea20 2024-05-08T23:22:18.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.701+0000 7f9cd12c7700 1 -- 172.21.15.5:0/2487868564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0011d20 con 0x7f9ccc0cea20 2024-05-08T23:22:18.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 -- 172.21.15.5:0/2487868564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc0cea20 msgr2=0x7f9ccc0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2487868564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc0cea20 0x7f9ccc0cee80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc071780 tx=0x7f9cc0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 -- 172.21.15.5:0/2487868564 shutdown_connections 2024-05-08T23:22:18.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2487868564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0ccbd0 0x7f9ccc0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2487868564 >> v2:172.21.15.49:3300/0 conn(0x7f9ccc15a770 0x7f9ccc0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2487868564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc0cea20 0x7f9ccc0cee80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 -- 172.21.15.5:0/2487868564 >> 172.21.15.5:0/2487868564 conn(0x7f9ccc0b1f80 msgr2=0x7f9ccc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 -- 172.21.15.5:0/2487868564 shutdown_connections 2024-05-08T23:22:18.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.702+0000 7f9cd452d700 1 -- 172.21.15.5:0/2487868564 wait complete. 2024-05-08T23:22:18.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.703+0000 7f9cd452d700 1 Processor -- start 2024-05-08T23:22:18.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.703+0000 7f9cd452d700 1 -- start start 2024-05-08T23:22:18.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd452d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc0ccbd0 0x7f9ccc0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd452d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0cea20 0x7f9ccc1581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd452d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc15a770 0x7f9ccc0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd452d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0ca440 con 0x7f9ccc15a770 2024-05-08T23:22:18.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd452d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc158c80 con 0x7f9ccc0ccbd0 2024-05-08T23:22:18.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd452d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc158f60 con 0x7f9ccc0cea20 2024-05-08T23:22:18.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd2aca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc15a770 0x7f9ccc0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd2aca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc15a770 0x7f9ccc0c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57164/0 (socket says 172.21.15.5:57164) 2024-05-08T23:22:18.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd2aca700 1 -- 172.21.15.5:0/2795250909 learned_addr learned my addr 172.21.15.5:0/2795250909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:18.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0cea20 0x7f9ccc1581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.704+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc0ccbd0 0x7f9ccc0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cd2aca700 1 -- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0cea20 msgr2=0x7f9ccc1581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0cea20 0x7f9ccc1581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cd2aca700 1 -- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc0ccbd0 msgr2=0x7f9ccc0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc0ccbd0 0x7f9ccc0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cd2aca700 1 -- 172.21.15.5:0/2795250909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc0009550 con 0x7f9ccc15a770 2024-05-08T23:22:18.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc15a770 0x7f9ccc0c8270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400eed0 tx=0x7f9cc400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cbeffd700 1 -- 172.21.15.5:0/2795250909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4010040 con 0x7f9ccc15a770 2024-05-08T23:22:18.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cbeffd700 1 -- 172.21.15.5:0/2795250909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400bc80 con 0x7f9ccc15a770 2024-05-08T23:22:18.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cd452d700 1 -- 172.21.15.5:0/2795250909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc0c7a70 con 0x7f9ccc15a770 2024-05-08T23:22:18.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cbeffd700 1 -- 172.21.15.5:0/2795250909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4009e40 con 0x7f9ccc15a770 2024-05-08T23:22:18.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.705+0000 7f9cd452d700 1 -- 172.21.15.5:0/2795250909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc14f520 con 0x7f9ccc15a770 2024-05-08T23:22:18.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.707+0000 7f9cbeffd700 1 -- 172.21.15.5:0/2795250909 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9cc4014400 con 0x7f9ccc15a770 2024-05-08T23:22:18.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.707+0000 7f9cbeffd700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cb006ac90 0x7f9cb006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.707+0000 7f9cbeffd700 1 -- 172.21.15.5:0/2795250909 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cc4089660 con 0x7f9ccc15a770 2024-05-08T23:22:18.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.707+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cb006ac90 0x7f9cb006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.708+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cb006ac90 0x7f9cb006d150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009b40 tx=0x7f9cc8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.710+0000 7f9ce06e7b80 1 Processor -- start 2024-05-08T23:22:18.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.710+0000 7f9ce06e7b80 1 -- start start 2024-05-08T23:22:18.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 0x559acd2cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd285150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559acd1dc100 con 0x559acd2aff70 2024-05-08T23:22:18.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559acd1d9ac0 con 0x559acd283320 2024-05-08T23:22:18.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559acd1d53b0 con 0x559acd2ad380 2024-05-08T23:22:18.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57176/0 (socket says 172.21.15.5:57176) 2024-05-08T23:22:18.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 -- 172.21.15.5:0/464186616 learned_addr learned my addr 172.21.15.5:0/464186616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:18.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 0x559acd2cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd285150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 -- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 msgr2=0x559acd285150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd285150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 -- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 msgr2=0x559acd2cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 0x559acd2cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 -- 172.21.15.5:0/464186616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559acd056360 con 0x559acd2aff70 2024-05-08T23:22:18.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd285150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:18.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.711+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2b0350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8004550 tx=0x7f9cc8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36187a19298f12a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9caffff700 1 -- 172.21.15.5:0/464186616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc801c040 con 0x559acd2aff70 2024-05-08T23:22:18.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9caffff700 1 -- 172.21.15.5:0/464186616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc8004580 con 0x559acd2aff70 2024-05-08T23:22:18.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9caffff700 1 -- 172.21.15.5:0/464186616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8003ab0 con 0x559acd2aff70 2024-05-08T23:22:18.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/464186616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 msgr2=0x559acd2b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2b0350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8004550 tx=0x7f9cc8014000 comp rx=0 tx=0).stop 2024-05-08T23:22:18.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/464186616 shutdown_connections 2024-05-08T23:22:18.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd285150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2b0350 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/464186616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 0x559acd2cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/464186616 >> 172.21.15.5:0/464186616 conn(0x559acd1e31c0 msgr2=0x559acd2cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.712+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/464186616 shutdown_connections 2024-05-08T23:22:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.713+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/464186616 wait complete. 2024-05-08T23:22:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.713+0000 7f9ce06e7b80 1 -- start start 2024-05-08T23:22:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.713+0000 7f9ce06e7b80 1 Processor -- start 2024-05-08T23:22:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd283320 0x559acd2a6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd2ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2aff70 0x559acd2b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559acd171510 con 0x559acd283320 2024-05-08T23:22:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559acd1711b0 con 0x559acd2aff70 2024-05-08T23:22:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559acd058650 con 0x559acd2ad380 2024-05-08T23:22:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9ce06e7b80 10 client.0 _ll_register_callbacks cb 0x559acd17aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9cd22c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd283320 0x559acd2a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9cd22c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd283320 0x559acd2a6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57188/0 (socket says 172.21.15.5:57188) 2024-05-08T23:22:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.714+0000 7f9cd22c9700 1 -- 172.21.15.5:0/4215140294 learned_addr learned my addr 172.21.15.5:0/4215140294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd2ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2aff70 0x559acd2b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd22c9700 1 -- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 msgr2=0x559acd2ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd2ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd22c9700 1 -- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2aff70 msgr2=0x559acd2b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2aff70 0x559acd2b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd22c9700 1 -- 172.21.15.5:0/4215140294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc80041d0 con 0x559acd283320 2024-05-08T23:22:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2aff70 0x559acd2b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd283320 0x559acd2a6250 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80044f0 tx=0x7f9cc80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44f36a607df73b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cad7fa700 1 -- 172.21.15.5:0/4215140294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80227f0 con 0x559acd283320 2024-05-08T23:22:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cad7fa700 1 -- 172.21.15.5:0/4215140294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cc8036ae0 con 0x559acd283320 2024-05-08T23:22:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/4215140294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559acd12e220 con 0x559acd283320 2024-05-08T23:22:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cad7fa700 1 -- 172.21.15.5:0/4215140294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cc8036ce0 con 0x559acd283320 2024-05-08T23:22:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9ce06e7b80 10 client.5328 fetch_fsmap learned FSMap version 15 2024-05-08T23:22:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/4215140294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x559acd002b10 con 0x559acd283320 2024-05-08T23:22:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.715+0000 7f9cad7fa700 1 -- 172.21.15.5:0/4215140294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559acd12e220 con 0x559acd283320 2024-05-08T23:22:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9cad7fa700 1 -- 172.21.15.5:0/4215140294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9cc8022950 con 0x559acd283320 2024-05-08T23:22:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9ce06e7b80 10 client.5328 fetch_fsmap finished waiting for FSMap version 15 2024-05-08T23:22:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9ce06e7b80 10 client.5328 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:22:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9ce06e7b80 20 client.5328 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x559acd2a1f80 0x559acd2a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9ce06e7b80 4 client.5328 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/4215140294 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559acd254790 con 0x559acd2a1f80 2024-05-08T23:22:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x559acd2a1f80 0x559acd2a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x559acd2a1f80 0x559acd2a2360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5224e65c208d0573 server_cookie=364e6c3bd40eb859 in_seq=0 out_seq=0 2024-05-08T23:22:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.716+0000 7f9cad7fa700 10 client.5328 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.717+0000 7f9cad7fa700 1 -- 172.21.15.5:0/4215140294 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559acd254790 con 0x559acd2a1f80 2024-05-08T23:22:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.717+0000 7f9cad7fa700 10 client.5328 handle_command_reply: tid=0 2024-05-08T23:22:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.717+0000 7f9ce06e7b80 1 client.5328 shutdown 2024-05-08T23:22:18.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.717+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/4215140294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd283320 msgr2=0x559acd2a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.717+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd283320 0x559acd2a6250 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80044f0 tx=0x7f9cc80047c0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/4215140294 shutdown_connections 2024-05-08T23:22:18.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x559acd2a1f80 0x559acd2a2360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2aff70 0x559acd2b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd283320 0x559acd2a6250 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/4215140294 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd2ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9cad7fa700 0 client.5328 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:18.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/4215140294 >> 172.21.15.5:0/4215140294 conn(0x559acd1e31c0 msgr2=0x559acd2b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/4215140294 shutdown_connections 2024-05-08T23:22:18.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/4215140294 wait complete. 2024-05-08T23:22:18.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.718+0000 7f9ce06e7b80 20 client.5328 trim_cache size 0 max 16384 2024-05-08T23:22:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.737+0000 7f9ce06e7b80 1 Processor -- start 2024-05-08T23:22:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.737+0000 7f9ce06e7b80 1 -- start start 2024-05-08T23:22:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd283320 0x559acd2b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2ad380 0x559acd2b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559acd2a6250 con 0x559acd2aff70 2024-05-08T23:22:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559acd2a8f30 con 0x559acd2ad380 2024-05-08T23:22:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559acd2858b0 con 0x559acd283320 2024-05-08T23:22:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57190/0 (socket says 172.21.15.5:57190) 2024-05-08T23:22:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 -- 172.21.15.5:0/2449843032 learned_addr learned my addr 172.21.15.5:0/2449843032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd283320 0x559acd2b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2ad380 0x559acd2b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 -- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd283320 msgr2=0x559acd2b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd283320 0x559acd2b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 -- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2ad380 msgr2=0x559acd2b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2ad380 0x559acd2b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 -- 172.21.15.5:0/2449843032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559acd19b510 con 0x559acd2aff70 2024-05-08T23:22:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd283320 0x559acd2b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2ad380 0x559acd2b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2ab6b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009d50 tx=0x7f9cc4058d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc9f8e382bf0c7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.738+0000 7f9cbcff9700 1 -- 172.21.15.5:0/2449843032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40608f0 con 0x559acd2aff70 2024-05-08T23:22:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9cbcff9700 1 -- 172.21.15.5:0/2449843032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc4087c10 con 0x559acd2aff70 2024-05-08T23:22:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9cbcff9700 1 -- 172.21.15.5:0/2449843032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4058980 con 0x559acd2aff70 2024-05-08T23:22:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/2449843032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 msgr2=0x559acd2ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2ab6b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009d50 tx=0x7f9cc4058d30 comp rx=0 tx=0).stop 2024-05-08T23:22:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/2449843032 shutdown_connections 2024-05-08T23:22:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2ab6b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd283320 0x559acd2b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/2449843032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd2ad380 0x559acd2b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/2449843032 >> 172.21.15.5:0/2449843032 conn(0x559acd1e31c0 msgr2=0x559acd285d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/2449843032 shutdown_connections 2024-05-08T23:22:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.739+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/2449843032 wait complete. 2024-05-08T23:22:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.740+0000 7f9ce06e7b80 1 -- start start 2024-05-08T23:22:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.740+0000 7f9ce06e7b80 1 Processor -- start 2024-05-08T23:22:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 0x559acd2b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd2ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9ce06e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559acd2a3180 con 0x559acd2aff70 2024-05-08T23:22:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559acd2d7520 con 0x559acd283320 2024-05-08T23:22:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9ce06e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559acd2a2500 con 0x559acd2ad380 2024-05-08T23:22:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9ce06e7b80 10 client.0 _ll_register_callbacks cb 0x559acd17aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57194/0 (socket says 172.21.15.5:57194) 2024-05-08T23:22:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 -- 172.21.15.5:0/450882572 learned_addr learned my addr 172.21.15.5:0/450882572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 0x559acd2b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd1ac8700 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd2ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 -- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 msgr2=0x559acd2ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd2ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 -- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 msgr2=0x559acd2b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 0x559acd2b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 -- 172.21.15.5:0/450882572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc14f520 con 0x559acd2aff70 2024-05-08T23:22:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cd2aca700 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2abbf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4056850 tx=0x7f9cc4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ae8e45a543035a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.741+0000 7f9cad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9cad7fa700 1 -- 172.21.15.5:0/450882572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40608f0 con 0x559acd2aff70 2024-05-08T23:22:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9cad7fa700 1 -- 172.21.15.5:0/450882572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cc4060460 con 0x559acd2aff70 2024-05-08T23:22:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/450882572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cc8004580 con 0x559acd2aff70 2024-05-08T23:22:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9cad7fa700 1 -- 172.21.15.5:0/450882572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cc4060620 con 0x559acd2aff70 2024-05-08T23:22:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 10 client.5334 fetch_fsmap learned FSMap version 15 2024-05-08T23:22:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/450882572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x559acd05ac80 con 0x559acd2aff70 2024-05-08T23:22:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9cad7fa700 1 -- 172.21.15.5:0/450882572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc8004580 con 0x559acd2aff70 2024-05-08T23:22:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9cad7fa700 1 -- 172.21.15.5:0/450882572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9cc4060b40 con 0x559acd2aff70 2024-05-08T23:22:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 10 client.5334 fetch_fsmap finished waiting for FSMap version 15 2024-05-08T23:22:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 10 client.5334 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:22:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 20 client.5334 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x559acd2d63a0 0x559acd2d6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 4 client.5334 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:22:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/450882572 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559acd1cde70 con 0x559acd2d63a0 2024-05-08T23:22:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.742+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x559acd2d63a0 0x559acd2d6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9cad7fa700 10 client.5334 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9cd22c9700 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x559acd2d63a0 0x559acd2d6780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16659952f6d4239e server_cookie=2bce535a3e4bd63f in_seq=0 out_seq=0 2024-05-08T23:22:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9cad7fa700 1 -- 172.21.15.5:0/450882572 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x559acd1cde70 con 0x559acd2d63a0 2024-05-08T23:22:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9cad7fa700 10 client.5334 handle_command_reply: tid=0 2024-05-08T23:22:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 client.5334 shutdown 2024-05-08T23:22:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/450882572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 msgr2=0x559acd2abbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2abbf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4056850 tx=0x7f9cc4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/450882572 shutdown_connections 2024-05-08T23:22:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x559acd2d63a0 0x559acd2d6780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559acd2aff70 0x559acd2abbf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559acd283320 0x559acd2b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9cad7fa700 0 client.5334 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 --2- 172.21.15.5:0/450882572 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559acd2ad380 0x559acd2ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/450882572 >> 172.21.15.5:0/450882572 conn(0x559acd1e31c0 msgr2=0x559acd2ac610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/450882572 shutdown_connections 2024-05-08T23:22:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.743+0000 7f9ce06e7b80 1 -- 172.21.15.5:0/450882572 wait complete. 2024-05-08T23:22:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9ce06e7b80 20 client.5334 trim_cache size 0 max 16384 2024-05-08T23:22:18.763 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:18.763 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4975, 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 10.975545819000001, 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 15, 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 34, 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 34, 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 19.292988568999998 2024-05-08T23:22:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 -- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cb006ac90 msgr2=0x7f9cb006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cb006ac90 0x7f9cb006d150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009b40 tx=0x7f9cc8009210 comp rx=0 tx=0).stop 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 -- 172.21.15.5:0/2795250909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc15a770 msgr2=0x7f9ccc0c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc15a770 0x7f9ccc0c8270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400eed0 tx=0x7f9cc400b390 comp rx=0 tx=0).stop 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 -- 172.21.15.5:0/2795250909 shutdown_connections 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cb006ac90 0x7f9cb006d150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ccc15a770 0x7f9ccc0c8270 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ccc0ccbd0 0x7f9ccc0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 --2- 172.21.15.5:0/2795250909 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ccc0cea20 0x7f9ccc1581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 -- 172.21.15.5:0/2795250909 >> 172.21.15.5:0/2795250909 conn(0x7f9ccc0b1f80 msgr2=0x7f9ccc06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 -- 172.21.15.5:0/2795250909 shutdown_connections 2024-05-08T23:22:18.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.744+0000 7f9cd452d700 1 -- 172.21.15.5:0/2795250909 wait complete. 2024-05-08T23:22:18.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:22:18.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.905+0000 7feb5a177700 1 Processor -- start 2024-05-08T23:22:18.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.905+0000 7feb5a177700 1 -- start start 2024-05-08T23:22:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb5a177700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb5406f130 0x7feb5406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb5a177700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5415c280 con 0x7feb540cd4e0 2024-05-08T23:22:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb5a177700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5406e780 con 0x7feb540cecf0 2024-05-08T23:22:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb5a177700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5406ea60 con 0x7feb5406f130 2024-05-08T23:22:18.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb537fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7feb540cd4e0 0x7feb540ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37550/0 (socket says 172.21.15.5:37550) 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb537fe700 1 -- 172.21.15.5:0/3571434121 learned_addr learned my addr 172.21.15.5:0/3571434121 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb53fff700 1 --2- 172.21.15.5:0/3571434121 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb5406f130 0x7feb5406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb527fc700 1 -- 172.21.15.5:0/3571434121 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894790627 0 0) 0x7feb5415c280 con 0x7feb540cd4e0 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.906+0000 7feb527fc700 1 -- 172.21.15.5:0/3571434121 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb3c0035b0 con 0x7feb540cd4e0 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb527fc700 1 -- 172.21.15.5:0/3571434121 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300050027 0 0) 0x7feb5406e780 con 0x7feb540cecf0 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb527fc700 1 -- 172.21.15.5:0/3571434121 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb5415c280 con 0x7feb540cecf0 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb527fc700 1 -- 172.21.15.5:0/3571434121 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3297189247 0 0) 0x7feb3c0035b0 con 0x7feb540cd4e0 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb53fff700 1 -- 172.21.15.5:0/3571434121 >> v1:172.21.15.49:6789/0 conn(0x7feb540cecf0 legacy=0x7feb5415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb53fff700 1 -- 172.21.15.5:0/3571434121 >> v1:172.21.15.5:6789/0 conn(0x7feb540cd4e0 legacy=0x7feb540ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb53fff700 1 -- 172.21.15.5:0/3571434121 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5406ed40 con 0x7feb5406f130 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb53fff700 1 --2- 172.21.15.5:0/3571434121 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb5406f130 0x7feb5406f910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009dc0 tx=0x7feb4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18798a33c4dae9ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb527fc700 1 -- 172.21.15.5:0/3571434121 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00e070 con 0x7feb5406f130 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb527fc700 1 -- 172.21.15.5:0/3571434121 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c003a20 con 0x7feb5406f130 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.907+0000 7feb527fc700 1 -- 172.21.15.5:0/3571434121 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44001670 con 0x7feb5406f130 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.908+0000 7feb5a177700 1 -- 172.21.15.5:0/3571434121 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb5406f130 msgr2=0x7feb5406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.908+0000 7feb5a177700 1 --2- 172.21.15.5:0/3571434121 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb5406f130 0x7feb5406f910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009dc0 tx=0x7feb4c00b330 comp rx=0 tx=0).stop 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.908+0000 7feb5a177700 1 -- 172.21.15.5:0/3571434121 shutdown_connections 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.908+0000 7feb5a177700 1 --2- 172.21.15.5:0/3571434121 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb5406f130 0x7feb5406f910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.908+0000 7feb5a177700 1 -- 172.21.15.5:0/3571434121 >> 172.21.15.5:0/3571434121 conn(0x7feb540d0320 msgr2=0x7feb540d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.908+0000 7feb5a177700 1 -- 172.21.15.5:0/3571434121 shutdown_connections 2024-05-08T23:22:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.908+0000 7feb5a177700 1 -- 172.21.15.5:0/3571434121 wait complete. 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.909+0000 7feb5a177700 1 Processor -- start 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.909+0000 7feb5a177700 1 -- start start 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb5a177700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5406f130 0x7feb54157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb5a177700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb540cd4e0 0x7feb5414b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb5a177700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb540cecf0 0x7feb540b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb5a177700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540b49a0 con 0x7feb540cd4e0 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb5a177700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540b28f0 con 0x7feb5406f130 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb5a177700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb540b2bd0 con 0x7feb540cecf0 2024-05-08T23:22:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb52ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb540cd4e0 0x7feb5414b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb52ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb540cd4e0 0x7feb5414b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57196/0 (socket says 172.21.15.5:57196) 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.910+0000 7feb52ffd700 1 -- 172.21.15.5:0/2925861543 learned_addr learned my addr 172.21.15.5:0/2925861543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb537fe700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5406f130 0x7feb54157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb53fff700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb540cecf0 0x7feb540b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb52ffd700 1 -- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb540cecf0 msgr2=0x7feb540b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb52ffd700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb540cecf0 0x7feb540b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb52ffd700 1 -- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5406f130 msgr2=0x7feb54157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb52ffd700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5406f130 0x7feb54157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb52ffd700 1 -- 172.21.15.5:0/2925861543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c009a70 con 0x7feb540cd4e0 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb53fff700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb540cecf0 0x7feb540b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb52ffd700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb540cd4e0 0x7feb5414b6e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7feb4800c500 tx=0x7feb4800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb3bfff700 1 -- 172.21.15.5:0/2925861543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480052d0 con 0x7feb540cd4e0 2024-05-08T23:22:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb3bfff700 1 -- 172.21.15.5:0/2925861543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48005450 con 0x7feb540cd4e0 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb540b4630 con 0x7feb540cd4e0 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb3bfff700 1 -- 172.21.15.5:0/2925861543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480038c0 con 0x7feb540cd4e0 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.911+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb54157030 con 0x7feb540cd4e0 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.912+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb54068000 con 0x7feb540cd4e0 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.913+0000 7feb3bfff700 1 -- 172.21.15.5:0/2925861543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feb48003a20 con 0x7feb540cd4e0 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.913+0000 7feb3bfff700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb4006ac90 0x7feb4006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.913+0000 7feb3bfff700 1 -- 172.21.15.5:0/2925861543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb48088800 con 0x7feb540cd4e0 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.913+0000 7feb537fe700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb4006ac90 0x7feb4006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.914+0000 7feb537fe700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb4006ac90 0x7feb4006d150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feb44000f60 tx=0x7feb44001c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:18.918+0000 7feb3bfff700 1 -- 172.21.15.5:0/2925861543 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feb480539e0 con 0x7feb540cd4e0 2024-05-08T23:22:19.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.191+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb540b4ae0 con 0x7feb540cd4e0 2024-05-08T23:22:19.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.191+0000 7feb3bfff700 1 -- 172.21.15.5:0/2925861543 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7feb48017070 con 0x7feb540cd4e0 2024-05-08T23:22:19.191 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:22:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb4006ac90 msgr2=0x7feb4006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb4006ac90 0x7feb4006d150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feb44000f60 tx=0x7feb44001c00 comp rx=0 tx=0).stop 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb540cd4e0 msgr2=0x7feb5414b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb540cd4e0 0x7feb5414b6e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7feb4800c500 tx=0x7feb4800c8c0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 shutdown_connections 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb4006ac90 0x7feb4006d150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb540cecf0 0x7feb540b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5406f130 0x7feb54157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 --2- 172.21.15.5:0/2925861543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb540cd4e0 0x7feb5414b6e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 >> 172.21.15.5:0/2925861543 conn(0x7feb540d0320 msgr2=0x7feb5415b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 shutdown_connections 2024-05-08T23:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:19.193+0000 7feb5a177700 1 -- 172.21.15.5:0/2925861543 wait complete. 2024-05-08T23:22:19.207 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T23:22:19.207 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:22:19.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-08T23:22:19.208 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:22:19.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:22:19.208 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T23:22:19.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T23:22:19.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:22:19.208 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:22:19.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.208 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-08T23:22:19.228 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:22:19.228 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:22:19.228 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:22:19.228 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:22:19.228 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:22:19.228 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:22:19.228 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42178sec preferred_lft 42178sec 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:22:19.229 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::1c2e:b3ff:fe6b:56f/64 scope link 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:22:19.230 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:22:19.231 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:22:19.231 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::84a4:2aff:fe7c:ce1a/64 scope link 2024-05-08T23:22:19.231 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:22:19.231 INFO:teuthology.orchestra.run.smithi005.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:22:19.231 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether e2:3e:2c:7a:5d:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:22:19.231 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::e03e:2cff:fe7a:5db1/64 scope link 2024-05-08T23:22:19.231 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:22:19.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.231 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:22:19.299 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-08T23:22:19.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.302 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-08T23:22:19.366 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-08T23:22:19.367 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:22:19.367 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:22:19.367 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:22:19.367 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:22:19.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.367 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:22:19.367 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:22:19.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:22:19.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:22:19.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:22:19.368 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:22:19.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.492 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:22:19.512 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:22:19.512 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:22:19.512 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:22:19.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-08T23:22:19.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:22:19.589 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:22:19.651 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:22:19.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:22:19.652 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:22:19.714 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T23:22:19.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:22:19.812 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:22:19.812 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:22:19.853 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:22:19.853 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:22:19.855 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:22:19.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:22:19.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:19.888 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:22:19.987 INFO:teuthology.orchestra.run.smithi005.stdout:client5173 2024-05-08T23:22:19.987 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:22:19.987 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:22:20.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T23:22:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.201+0000 7f2074dda700 1 Processor -- start 2024-05-08T23:22:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.201+0000 7f2074dda700 1 -- start start 2024-05-08T23:22:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.202+0000 7f2074dda700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f207015dc60 0x7f20700ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.202+0000 7f2074dda700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2070072170 0x7f20700adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.202+0000 7f2074dda700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20700ae8e0 con 0x7f20700ae450 2024-05-08T23:22:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.202+0000 7f2074dda700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20700aed90 con 0x7f207015dc60 2024-05-08T23:22:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.202+0000 7f2074dda700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207006dcd0 con 0x7f2070072170 2024-05-08T23:22:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206ed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f20700ae450 0x7f20700b3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37556/0 (socket says 172.21.15.5:37556) 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206ed9d700 1 -- 172.21.15.5:0/168091675 learned_addr learned my addr 172.21.15.5:0/168091675 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206dd9b700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2070072170 0x7f20700adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206e59c700 1 --2- 172.21.15.5:0/168091675 >> v2:172.21.15.49:3300/0 conn(0x7f207015dc60 0x7f20700ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993436881 0 0) 0x7f20700ae8e0 con 0x7f20700ae450 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20580035b0 con 0x7f20700ae450 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884345207 0 0) 0x7f20580035b0 con 0x7f20700ae450 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2070072170 msgr2=0x7f20700adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2070072170 0x7f20700adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 >> v2:172.21.15.49:3300/0 conn(0x7f207015dc60 msgr2=0x7f20700ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 --2- 172.21.15.5:0/168091675 >> v2:172.21.15.49:3300/0 conn(0x7f207015dc60 0x7f20700ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207006e290 con 0x7f20700ae450 2024-05-08T23:22:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f20680019f0 con 0x7f20700ae450 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 >> v1:172.21.15.5:6789/0 conn(0x7f20700ae450 legacy=0x7f20700b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2058003a00 0x7f2058003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2058004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2070072170 0x7f2058005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20700aed90 con 0x7f2058004850 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207006dcd0 con 0x7f2058003a00 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205800ba90 con 0x7f2070072170 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.203+0000 7f206dd9b700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2058004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206e59c700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2058003a00 0x7f2058003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206ed9d700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2070072170 0x7f2058005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206dd9b700 1 -- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2070072170 msgr2=0x7f2058005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206dd9b700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2070072170 0x7f2058005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206dd9b700 1 -- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2058003a00 msgr2=0x7f2058003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206dd9b700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2058003a00 0x7f2058003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206dd9b700 1 -- 172.21.15.5:0/168091675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580035b0 con 0x7f2058004850 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206dd9b700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2058004cd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2064009b60 tx=0x7f2058005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7080fb4a865a06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206400e6c0 con 0x7f2058004850 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206400e820 con 0x7f2058004850 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.204+0000 7f206d59a700 1 -- 172.21.15.5:0/168091675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206400eb90 con 0x7f2058004850 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.205+0000 7f2074dda700 1 -- 172.21.15.5:0/168091675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 msgr2=0x7f2058004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.205+0000 7f2074dda700 1 --2- 172.21.15.5:0/168091675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2058004cd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2064009b60 tx=0x7f2058005880 comp rx=0 tx=0).stop 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.205+0000 7f206ed9d700 1 -- 172.21.15.5:0/168091675 reap_dead start 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.205+0000 7f2074dda700 1 -- 172.21.15.5:0/168091675 shutdown_connections 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.205+0000 7f2074dda700 1 -- 172.21.15.5:0/168091675 >> 172.21.15.5:0/168091675 conn(0x7f20700b5a30 msgr2=0x7f20700b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.205+0000 7f2074dda700 1 -- 172.21.15.5:0/168091675 shutdown_connections 2024-05-08T23:22:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.205+0000 7f2074dda700 1 -- 172.21.15.5:0/168091675 wait complete. 2024-05-08T23:22:20.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.206+0000 7f2074dda700 1 Processor -- start 2024-05-08T23:22:20.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.206+0000 7f2074dda700 1 -- start start 2024-05-08T23:22:20.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f2074dda700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2070147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f2074dda700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2070148420 0x7f207014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f2074dda700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f207014ed20 0x7f207014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f2074dda700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207014f1c0 con 0x7f2058004850 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f2074dda700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207014bf40 con 0x7f2070148420 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f2074dda700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207014c220 con 0x7f207014ed20 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f206e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2070147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f206e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2070147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57222/0 (socket says 172.21.15.5:57222) 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f206e59c700 1 -- 172.21.15.5:0/268716274 learned_addr learned my addr 172.21.15.5:0/268716274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f206dd9b700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2070148420 0x7f207014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.207+0000 7f206ed9d700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f207014ed20 0x7f207014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f206e59c700 1 -- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f207014ed20 msgr2=0x7f207014cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f206e59c700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f207014ed20 0x7f207014cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f206e59c700 1 -- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2070148420 msgr2=0x7f207014e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f206e59c700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2070148420 0x7f207014e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f206e59c700 1 -- 172.21.15.5:0/268716274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207014c500 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f206e59c700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2070147ab0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2060011aa0 tx=0x7f2060011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f205effd700 1 -- 172.21.15.5:0/268716274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206000f940 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f205effd700 1 -- 172.21.15.5:0/268716274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206000faa0 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207014c7e0 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f205effd700 1 -- 172.21.15.5:0/268716274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206000fe50 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.208+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207014d240 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.209+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2070074bc0 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.210+0000 7f205effd700 1 -- 172.21.15.5:0/268716274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2060014440 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.210+0000 7f205effd700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f205006ac90 0x7f205006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.210+0000 7f205effd700 1 -- 172.21.15.5:0/268716274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2060089a10 con 0x7f2058004850 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.211+0000 7f206dd9b700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f205006ac90 0x7f205006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.211+0000 7f206dd9b700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f205006ac90 0x7f205006d150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20640090b0 tx=0x7f206401f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.215+0000 7f205effd700 1 -- 172.21.15.5:0/268716274 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2060054bf0 con 0x7f2058004850 2024-05-08T23:22:20.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.487+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f207015a830 con 0x7f2058004850 2024-05-08T23:22:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.809+0000 7f205effd700 1 -- 172.21.15.5:0/268716274 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7f2060058210 con 0x7f2058004850 2024-05-08T23:22:20.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.812+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f205006ac90 msgr2=0x7f205006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.812+0000 7f2074dda700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f205006ac90 0x7f205006d150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20640090b0 tx=0x7f206401f040 comp rx=0 tx=0).stop 2024-05-08T23:22:20.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.812+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 msgr2=0x7f2070147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.812+0000 7f2074dda700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2070147ab0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2060011aa0 tx=0x7f2060011db0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.814+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 shutdown_connections 2024-05-08T23:22:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.814+0000 7f2074dda700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f205006ac90 0x7f205006d150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.814+0000 7f2074dda700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f207014ed20 0x7f207014cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.814+0000 7f2074dda700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2058004850 0x7f2070147ab0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.814+0000 7f2074dda700 1 --2- 172.21.15.5:0/268716274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2070148420 0x7f207014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.814+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 >> 172.21.15.5:0/268716274 conn(0x7f20700b5a30 msgr2=0x7f20700b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.814+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 shutdown_connections 2024-05-08T23:22:20.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.814+0000 7f2074dda700 1 -- 172.21.15.5:0/268716274 wait complete. 2024-05-08T23:22:20.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:22:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.972+0000 7fd0649e0700 1 Processor -- start 2024-05-08T23:22:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.972+0000 7fd0649e0700 1 -- start start 2024-05-08T23:22:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd0649e0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0600cf3c0 0x7fd06006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd0649e0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd06006f4a0 0x7fd06006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd0649e0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600b2b10 con 0x7fd06015a380 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd0649e0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0600ceba0 con 0x7fd06006f4a0 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd0649e0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06006f950 con 0x7fd0600cf3c0 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd05e59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd06015a380 0x7fd0600cbca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37558/0 (socket says 172.21.15.5:37558) 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd05e59c700 1 -- 172.21.15.5:0/1284006409 learned_addr learned my addr 172.21.15.5:0/1284006409 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd05dd9b700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0600cf3c0 0x7fd06006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd05ed9d700 1 --2- 172.21.15.5:0/1284006409 >> v2:172.21.15.49:3300/0 conn(0x7fd06006f4a0 0x7fd06006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495735903 0 0) 0x7fd0600b2b10 con 0x7fd06015a380 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.973+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0480035b0 con 0x7fd06015a380 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3366596221 0 0) 0x7fd0480035b0 con 0x7fd06015a380 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0600cf3c0 msgr2=0x7fd06006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0600cf3c0 0x7fd06006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 >> v2:172.21.15.49:3300/0 conn(0x7fd06006f4a0 msgr2=0x7fd06006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 --2- 172.21.15.5:0/1284006409 >> v2:172.21.15.49:3300/0 conn(0x7fd06006f4a0 0x7fd06006d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600ce340 con 0x7fd06015a380 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fd050001670 con 0x7fd06015a380 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 >> v1:172.21.15.5:6789/0 conn(0x7fd06015a380 legacy=0x7fd0600cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0600cf3c0 0x7fd048003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd048003f80 0x7fd048004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048004d90 0x7fd048005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600ceba0 con 0x7fd048003f80 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06006f950 con 0x7fd048004d90 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04800baf0 con 0x7fd0600cf3c0 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05dd9b700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd048003f80 0x7fd048004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05e59c700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0600cf3c0 0x7fd048003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05ed9d700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048004d90 0x7fd048005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05dd9b700 1 -- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0600cf3c0 msgr2=0x7fd048003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05dd9b700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0600cf3c0 0x7fd048003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05dd9b700 1 -- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048004d90 msgr2=0x7fd048005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05dd9b700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048004d90 0x7fd048005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05dd9b700 1 -- 172.21.15.5:0/1284006409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0480035b0 con 0x7fd048003f80 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.974+0000 7fd05dd9b700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd048003f80 0x7fd048004850 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd054009b60 tx=0x7fd048005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e076feb8c6c1c336 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05400e6c0 con 0x7fd048003f80 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05400e820 con 0x7fd048003f80 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd05d59a700 1 -- 172.21.15.5:0/1284006409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05400eb90 con 0x7fd048003f80 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd0649e0700 1 -- 172.21.15.5:0/1284006409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd048003f80 msgr2=0x7fd048004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd0649e0700 1 --2- 172.21.15.5:0/1284006409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd048003f80 0x7fd048004850 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd054009b60 tx=0x7fd048005880 comp rx=0 tx=0).stop 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd05ed9d700 1 -- 172.21.15.5:0/1284006409 reap_dead start 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd0649e0700 1 -- 172.21.15.5:0/1284006409 shutdown_connections 2024-05-08T23:22:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd0649e0700 1 -- 172.21.15.5:0/1284006409 >> 172.21.15.5:0/1284006409 conn(0x7fd0600cfce0 msgr2=0x7fd0600d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd0649e0700 1 -- 172.21.15.5:0/1284006409 shutdown_connections 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.975+0000 7fd0649e0700 1 -- 172.21.15.5:0/1284006409 wait complete. 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.977+0000 7fd0649e0700 1 Processor -- start 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.977+0000 7fd0649e0700 1 -- start start 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd0649e0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048003f80 0x7fd0601f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd0649e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd060152e80 0x7fd0601e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd0649e0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd06014e2e0 0x7fd0601e1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd0649e0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601e60b0 con 0x7fd060152e80 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd0649e0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601f2410 con 0x7fd048003f80 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd0649e0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd060153300 con 0x7fd06014e2e0 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd060152e80 0x7fd0601e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd060152e80 0x7fd0601e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57250/0 (socket says 172.21.15.5:57250) 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 -- 172.21.15.5:0/3998017492 learned_addr learned my addr 172.21.15.5:0/3998017492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:20.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05e59c700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048003f80 0x7fd0601f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05ed9d700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd06014e2e0 0x7fd0601e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 -- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd06014e2e0 msgr2=0x7fd0601e1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd06014e2e0 0x7fd0601e1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 -- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048003f80 msgr2=0x7fd0601f14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048003f80 0x7fd0601f14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 -- 172.21.15.5:0/3998017492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0601f1b70 con 0x7fd060152e80 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd05dd9b700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd060152e80 0x7fd0601e5b70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd0540090b0 tx=0x7fd0540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd04effd700 1 -- 172.21.15.5:0/3998017492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05400e6c0 con 0x7fd060152e80 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd04effd700 1 -- 172.21.15.5:0/3998017492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd054003ad0 con 0x7fd060152e80 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0601f1e50 con 0x7fd060152e80 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.978+0000 7fd04effd700 1 -- 172.21.15.5:0/3998017492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054003e80 con 0x7fd060152e80 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.979+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0601df820 con 0x7fd060152e80 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.979+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd060068000 con 0x7fd060152e80 2024-05-08T23:22:20.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.980+0000 7fd04effd700 1 -- 172.21.15.5:0/3998017492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd054013470 con 0x7fd060152e80 2024-05-08T23:22:20.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.981+0000 7fd04effd700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd04006ac90 0x7fd04006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.981+0000 7fd04effd700 1 -- 172.21.15.5:0/3998017492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd05408ab50 con 0x7fd060152e80 2024-05-08T23:22:20.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.981+0000 7fd05e59c700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd04006ac90 0x7fd04006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.982+0000 7fd05e59c700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd04006ac90 0x7fd04006d150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd050000c00 tx=0x7fd050009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:20.985+0000 7fd04effd700 1 -- 172.21.15.5:0/3998017492 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd054055ee0 con 0x7fd060152e80 2024-05-08T23:22:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.256+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd06006fbb0 con 0x7fd060152e80 2024-05-08T23:22:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.256+0000 7fd04effd700 1 -- 172.21.15.5:0/3998017492 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fd054017090 con 0x7fd060152e80 2024-05-08T23:22:21.257 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:21.257 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:20.810495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4975,"mds_1":4745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4745":{"gid":4745,"name":"c","rank":1,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd04006ac90 msgr2=0x7fd04006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd04006ac90 0x7fd04006d150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd050000c00 tx=0x7fd050009040 comp rx=0 tx=0).stop 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd060152e80 msgr2=0x7fd0601e5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd060152e80 0x7fd0601e5b70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd0540090b0 tx=0x7fd0540047c0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 shutdown_connections 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd04006ac90 0x7fd04006d150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd06014e2e0 0x7fd0601e1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd048003f80 0x7fd0601f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 --2- 172.21.15.5:0/3998017492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd060152e80 0x7fd0601e5b70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 >> 172.21.15.5:0/3998017492 conn(0x7fd0600cfce0 msgr2=0x7fd060150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 shutdown_connections 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:21.259+0000 7fd0649e0700 1 -- 172.21.15.5:0/3998017492 wait complete. 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-08T23:22:21.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:22:07.766519+0000', 'modified': '2024-05-08T23:22:20.810495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 4975, 'mds_1': 4745}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4745': {'gid': 4745, 'name': 'c', 'rank': 1, 'incarnation': 17, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.5:6835/146081129', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 146081129}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 146081129}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4975': {'gid': 4975, 'name': 'd', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6837/886789668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6835', 'nonce': 886789668}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 886789668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:22:21.676 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4745:up:creating 2024-05-08T23:22:22.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:22:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.820+0000 7fd20a144700 1 Processor -- start 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.820+0000 7fd20a144700 1 -- start start 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd20a144700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2040cf840 0x7fd2040af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd20a144700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd20406a960 0x7fd20415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd20a144700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2040cd980 con 0x7fd2040d0eb0 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd20a144700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2040ccdb0 con 0x7fd20406a960 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd20a144700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040cd050 con 0x7fd2040cf840 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd202ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd20406a960 0x7fd20415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd2037fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2040cf840 0x7fd2040af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd202ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd20406a960 0x7fd20415d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57256/0 (socket says 172.21.15.5:57256) 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.821+0000 7fd202ffd700 1 -- 172.21.15.5:0/519184312 learned_addr learned my addr 172.21.15.5:0/519184312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.822+0000 7fd202ffd700 1 -- 172.21.15.5:0/519184312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2040cf840 msgr2=0x7fd2040af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.822+0000 7fd202ffd700 1 --2- 172.21.15.5:0/519184312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2040cf840 0x7fd2040af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.822+0000 7fd202ffd700 1 -- 172.21.15.5:0/519184312 >> v1:172.21.15.49:6789/0 conn(0x7fd2040d0eb0 legacy=0x7fd20406dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.822+0000 7fd202ffd700 1 -- 172.21.15.5:0/519184312 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040cd330 con 0x7fd20406a960 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.822+0000 7fd202ffd700 1 --2- 172.21.15.5:0/519184312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20406a960 0x7fd20415d4f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009920 tx=0x7fd1f8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a486623a54b2ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.822+0000 7fd2027fc700 1 -- 172.21.15.5:0/519184312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8011830 con 0x7fd20406a960 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.822+0000 7fd2027fc700 1 -- 172.21.15.5:0/519184312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f8011990 con 0x7fd20406a960 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.822+0000 7fd2027fc700 1 -- 172.21.15.5:0/519184312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8011d20 con 0x7fd20406a960 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.823+0000 7fd20a144700 1 -- 172.21.15.5:0/519184312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20406a960 msgr2=0x7fd20415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.823+0000 7fd20a144700 1 --2- 172.21.15.5:0/519184312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20406a960 0x7fd20415d4f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009920 tx=0x7fd1f8009c30 comp rx=0 tx=0).stop 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.823+0000 7fd20a144700 1 -- 172.21.15.5:0/519184312 shutdown_connections 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.823+0000 7fd20a144700 1 --2- 172.21.15.5:0/519184312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd2040cf840 0x7fd2040af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.823+0000 7fd20a144700 1 --2- 172.21.15.5:0/519184312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20406a960 0x7fd20415d4f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.823+0000 7fd20a144700 1 -- 172.21.15.5:0/519184312 >> 172.21.15.5:0/519184312 conn(0x7fd2040d0320 msgr2=0x7fd2040d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.823+0000 7fd20a144700 1 -- 172.21.15.5:0/519184312 shutdown_connections 2024-05-08T23:22:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.823+0000 7fd20a144700 1 -- 172.21.15.5:0/519184312 wait complete. 2024-05-08T23:22:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.824+0000 7fd20a144700 1 Processor -- start 2024-05-08T23:22:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.824+0000 7fd20a144700 1 -- start start 2024-05-08T23:22:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd20a144700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd20406a960 0x7fd2040c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd20a144700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2040cf840 0x7fd20414a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd20a144700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2040d0eb0 0x7fd204155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd20a144700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040c6390 con 0x7fd2040cf840 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd20a144700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20414acf0 con 0x7fd2040d0eb0 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd20a144700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040c5af0 con 0x7fd20406a960 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd202ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2040cf840 0x7fd20414a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd202ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2040cf840 0x7fd20414a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57260/0 (socket says 172.21.15.5:57260) 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.825+0000 7fd202ffd700 1 -- 172.21.15.5:0/3943115020 learned_addr learned my addr 172.21.15.5:0/3943115020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd2037fe700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd20406a960 0x7fd2040c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd203fff700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2040d0eb0 0x7fd204155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd202ffd700 1 -- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd20406a960 msgr2=0x7fd2040c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd202ffd700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd20406a960 0x7fd2040c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd202ffd700 1 -- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2040d0eb0 msgr2=0x7fd204155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd202ffd700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2040d0eb0 0x7fd204155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd202ffd700 1 -- 172.21.15.5:0/3943115020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8009550 con 0x7fd2040cf840 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd203fff700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2040d0eb0 0x7fd204155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd2037fe700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd20406a960 0x7fd2040c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd202ffd700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2040cf840 0x7fd20414a7b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8000c00 tx=0x7fd1f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd1ebfff700 1 -- 172.21.15.5:0/3943115020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8004480 con 0x7fd2040cf840 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd1ebfff700 1 -- 172.21.15.5:0/3943115020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f801f070 con 0x7fd2040cf840 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2040c5dd0 con 0x7fd2040cf840 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd1ebfff700 1 -- 172.21.15.5:0/3943115020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80038e0 con 0x7fd2040cf840 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.826+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2040c3f20 con 0x7fd2040cf840 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.827+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2040c3d80 con 0x7fd2040cf840 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.828+0000 7fd1ebfff700 1 -- 172.21.15.5:0/3943115020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd1f8021030 con 0x7fd2040cf840 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.828+0000 7fd1ebfff700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd1f006ac90 0x7fd1f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.828+0000 7fd1ebfff700 1 -- 172.21.15.5:0/3943115020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1f808a640 con 0x7fd2040cf840 2024-05-08T23:22:22.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.828+0000 7fd2037fe700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd1f006ac90 0x7fd1f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:22.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.829+0000 7fd2037fe700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd1f006ac90 0x7fd1f006d150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009b60 tx=0x7fd1f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:22.833+0000 7fd1ebfff700 1 -- 172.21.15.5:0/3943115020 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd2040c3d80 con 0x7fd2040cf840 2024-05-08T23:22:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.100+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2040c3d80 con 0x7fd2040cf840 2024-05-08T23:22:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.101+0000 7fd1ebfff700 1 -- 172.21.15.5:0/3943115020 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4747 (secure 0 0 0) 0x7fd2040c3d80 con 0x7fd2040cf840 2024-05-08T23:22:23.102 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:23.102 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:22.820284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4975,"mds_1":4745,"mds_2":4742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4742":{"gid":4742,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4745":{"gid":4745,"name":"c","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:22:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd1f006ac90 msgr2=0x7fd1f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd1f006ac90 0x7fd1f006d150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009b60 tx=0x7fd1f4008040 comp rx=0 tx=0).stop 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2040cf840 msgr2=0x7fd20414a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2040cf840 0x7fd20414a7b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8000c00 tx=0x7fd1f8004750 comp rx=0 tx=0).stop 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 shutdown_connections 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd1f006ac90 0x7fd1f006d150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2040d0eb0 0x7fd204155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd20406a960 0x7fd2040c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 --2- 172.21.15.5:0/3943115020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2040cf840 0x7fd20414a7b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 >> 172.21.15.5:0/3943115020 conn(0x7fd2040d0320 msgr2=0x7fd2040b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 shutdown_connections 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.103+0000 7fd20a144700 1 -- 172.21.15.5:0/3943115020 wait complete. 2024-05-08T23:22:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 20 2024-05-08T23:22:23.520 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:22:07.766519+0000', 'modified': '2024-05-08T23:22:22.820284+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4975, 'mds_1': 4745, 'mds_2': 4742}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4742': {'gid': 4742, 'name': 'a', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.5:6834/295306227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 295306227}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 295306227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4745': {'gid': 4745, 'name': 'c', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.5:6835/146081129', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 146081129}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 146081129}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4975': {'gid': 4975, 'name': 'd', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6837/886789668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6835', 'nonce': 886789668}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 886789668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:22:23.520 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:22:23.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T23:22:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.668+0000 7f932084c700 1 Processor -- start 2024-05-08T23:22:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.668+0000 7f932084c700 1 -- start start 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f932084c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c09a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f932084c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c0b1410 con 0x7f931c15b900 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f932084c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c0b0e30 con 0x7f931c0b18a0 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f932084c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0cffd0 con 0x7f931c0cf9e0 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f931a59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f931c15b900 0x7f931c099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37574/0 (socket says 172.21.15.5:37574) 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f931a59c700 1 -- 172.21.15.5:0/788171593 learned_addr learned my addr 172.21.15.5:0/788171593 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f9319d9b700 1 --2- 172.21.15.5:0/788171593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c09a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f931959a700 1 -- 172.21.15.5:0/788171593 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104029573 0 0) 0x7f931c0b1410 con 0x7f931c15b900 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f931959a700 1 -- 172.21.15.5:0/788171593 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93040035b0 con 0x7f931c15b900 2024-05-08T23:22:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f931959a700 1 -- 172.21.15.5:0/788171593 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351361921 0 0) 0x7f931c0b0e30 con 0x7f931c0b18a0 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f931959a700 1 -- 172.21.15.5:0/788171593 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f931c0b1410 con 0x7f931c0b18a0 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f9319d9b700 1 -- 172.21.15.5:0/788171593 >> v1:172.21.15.49:6789/0 conn(0x7f931c0b18a0 legacy=0x7f931c0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f931959a700 1 -- 172.21.15.5:0/788171593 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3863615029 0 0) 0x7f93040035b0 con 0x7f931c15b900 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f9319d9b700 1 -- 172.21.15.5:0/788171593 >> v1:172.21.15.5:6789/0 conn(0x7f931c15b900 legacy=0x7f931c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.669+0000 7f9319d9b700 1 -- 172.21.15.5:0/788171593 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c0b10a0 con 0x7f931c0cf9e0 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.670+0000 7f9319d9b700 1 --2- 172.21.15.5:0/788171593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c09a160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9310009870 tx=0x7f9310009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552ae6f912998cef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.670+0000 7f931959a700 1 -- 172.21.15.5:0/788171593 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c001670 con 0x7f931c0cf9e0 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.670+0000 7f931959a700 1 -- 172.21.15.5:0/788171593 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310004030 con 0x7f931c0cf9e0 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.670+0000 7f931959a700 1 -- 172.21.15.5:0/788171593 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100043a0 con 0x7f931c0cf9e0 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.670+0000 7f932084c700 1 -- 172.21.15.5:0/788171593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 msgr2=0x7f931c09a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.670+0000 7f932084c700 1 --2- 172.21.15.5:0/788171593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c09a160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9310009870 tx=0x7f9310009b80 comp rx=0 tx=0).stop 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.671+0000 7f932084c700 1 -- 172.21.15.5:0/788171593 shutdown_connections 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.671+0000 7f932084c700 1 --2- 172.21.15.5:0/788171593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c09a160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.671+0000 7f932084c700 1 -- 172.21.15.5:0/788171593 >> 172.21.15.5:0/788171593 conn(0x7f931c0b2f20 msgr2=0x7f931c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.671+0000 7f932084c700 1 -- 172.21.15.5:0/788171593 shutdown_connections 2024-05-08T23:22:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.671+0000 7f932084c700 1 -- 172.21.15.5:0/788171593 wait complete. 2024-05-08T23:22:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.672+0000 7f932084c700 1 Processor -- start 2024-05-08T23:22:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.672+0000 7f932084c700 1 -- start start 2024-05-08T23:22:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f932084c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f931c0b18a0 0x7f931c06cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f932084c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c0c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f932084c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c15b900 0x7f931c1f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f932084c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0c7630 con 0x7f931c15b900 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f932084c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c1f7750 con 0x7f931c0b18a0 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f932084c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c1f67c0 con 0x7f931c0cf9e0 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f931ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c15b900 0x7f931c1f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f931ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c15b900 0x7f931c1f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57274/0 (socket says 172.21.15.5:57274) 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f931ad9d700 1 -- 172.21.15.5:0/3545212915 learned_addr learned my addr 172.21.15.5:0/3545212915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f9319d9b700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c0c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f931a59c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f931c0b18a0 0x7f931c06cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f931ad9d700 1 -- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 msgr2=0x7f931c0c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f931ad9d700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c0c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f931ad9d700 1 -- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f931c0b18a0 msgr2=0x7f931c06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.673+0000 7f931ad9d700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f931c0b18a0 0x7f931c06cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f931ad9d700 1 -- 172.21.15.5:0/3545212915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100094a0 con 0x7f931c15b900 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f931a59c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f931c0b18a0 0x7f931c06cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f9319d9b700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c0c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f931ad9d700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c15b900 0x7f931c1f7210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f931400cc20 tx=0x7f931400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f930affd700 1 -- 172.21.15.5:0/3545212915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314003c00 con 0x7f931c15b900 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f930affd700 1 -- 172.21.15.5:0/3545212915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9314003d60 con 0x7f931c15b900 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f932084c700 1 -- 172.21.15.5:0/3545212915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c06d640 con 0x7f931c15b900 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f930affd700 1 -- 172.21.15.5:0/3545212915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400abe0 con 0x7f931c15b900 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.674+0000 7f932084c700 1 -- 172.21.15.5:0/3545212915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c1f6a10 con 0x7f931c15b900 2024-05-08T23:22:23.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.675+0000 7f930affd700 1 -- 172.21.15.5:0/3545212915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f931400ad40 con 0x7f931c15b900 2024-05-08T23:22:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.676+0000 7f930affd700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92fc06ac90 0x7f92fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.676+0000 7f930affd700 1 -- 172.21.15.5:0/3545212915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93140894b0 con 0x7f931c15b900 2024-05-08T23:22:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.676+0000 7f931a59c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92fc06ac90 0x7f92fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.677+0000 7f931a59c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92fc06ac90 0x7f92fc06d150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f930c0011e0 tx=0x7f930c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.678+0000 7f932ca06b80 1 Processor -- start 2024-05-08T23:22:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.678+0000 7f932ca06b80 1 -- start start 2024-05-08T23:22:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a71050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 0x559c60a9b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f932ca06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c609c7100 con 0x559c60a6e320 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f932ca06b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c609c4ac0 con 0x559c60a93750 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f932ca06b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c609c03b0 con 0x559c60a9ad80 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f9319d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a71050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f931a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f9319d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a71050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57284/0 (socket says 172.21.15.5:57284) 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f9319d9b700 1 -- 172.21.15.5:0/659029618 learned_addr learned my addr 172.21.15.5:0/659029618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.679+0000 7f931ad9d700 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 0x559c60a9b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f9319d9b700 1 -- 172.21.15.5:0/659029618 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 msgr2=0x559c60a9b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f9319d9b700 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 0x559c60a9b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f9319d9b700 1 -- 172.21.15.5:0/659029618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 msgr2=0x559c60a6c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f9319d9b700 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a6c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f9319d9b700 1 -- 172.21.15.5:0/659029618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c60841360 con 0x559c60a6e320 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f931a59c700 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f9319d9b700 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a71050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9310004000 tx=0x7f9310010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d08389f395b730b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f92fbfff700 1 -- 172.21.15.5:0/659029618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100041a0 con 0x559c60a6e320 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f92fbfff700 1 -- 172.21.15.5:0/659029618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310004300 con 0x559c60a6e320 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.680+0000 7f92fbfff700 1 -- 172.21.15.5:0/659029618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100038a0 con 0x559c60a6e320 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 -- 172.21.15.5:0/659029618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 msgr2=0x559c60a71050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a71050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9310004000 tx=0x7f9310010000 comp rx=0 tx=0).stop 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 -- 172.21.15.5:0/659029618 shutdown_connections 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 0x559c60a9b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 --2- 172.21.15.5:0/659029618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a71050 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 -- 172.21.15.5:0/659029618 >> 172.21.15.5:0/659029618 conn(0x559c609ce1c0 msgr2=0x559c60ab5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 -- 172.21.15.5:0/659029618 shutdown_connections 2024-05-08T23:22:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.681+0000 7f932ca06b80 1 -- 172.21.15.5:0/659029618 wait complete. 2024-05-08T23:22:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.682+0000 7f932ca06b80 1 -- start start 2024-05-08T23:22:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.682+0000 7f932ca06b80 1 Processor -- start 2024-05-08T23:22:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a91e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60a69c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 0x559c60a6a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f932ca06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6095c510 con 0x559c60a6e320 2024-05-08T23:22:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f932ca06b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6095c1b0 con 0x559c60a9ad80 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f932ca06b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c60843650 con 0x559c60a93750 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f932ca06b80 10 client.0 _ll_register_callbacks cb 0x559c60965ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f931a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a91e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.683+0000 7f931a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a91e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57288/0 (socket says 172.21.15.5:57288) 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f931a59c700 1 -- 172.21.15.5:0/2291221306 learned_addr learned my addr 172.21.15.5:0/2291221306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f9319d9b700 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60a69c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f931ad9d700 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 0x559c60a6a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f931a59c700 1 -- 172.21.15.5:0/2291221306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 msgr2=0x559c60a69c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f931a59c700 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60a69c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f931a59c700 1 -- 172.21.15.5:0/2291221306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 msgr2=0x559c60a6a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f931a59c700 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 0x559c60a6a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f931a59c700 1 -- 172.21.15.5:0/2291221306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c005880 con 0x559c60a6e320 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f931a59c700 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a91e00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f930c007460 tx=0x7f930c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b86bfe6e932129 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f92f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f92f97fa700 1 -- 172.21.15.5:0/2291221306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00dd10 con 0x559c60a6e320 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f92f97fa700 1 -- 172.21.15.5:0/2291221306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f930c00de70 con 0x559c60a6e320 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f932ca06b80 1 -- 172.21.15.5:0/2291221306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c60919220 con 0x559c60a6e320 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f92f97fa700 1 -- 172.21.15.5:0/2291221306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f930c012410 con 0x559c60a6e320 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f932ca06b80 10 client.5370 fetch_fsmap learned FSMap version 20 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.684+0000 7f932ca06b80 1 -- 172.21.15.5:0/2291221306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x7f931c0b10a0 con 0x559c60a6e320 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f92f97fa700 1 -- 172.21.15.5:0/2291221306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c60919220 con 0x559c60a6e320 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f92f97fa700 1 -- 172.21.15.5:0/2291221306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 20) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f930c01f070 con 0x559c60a6e320 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f932ca06b80 10 client.5370 fetch_fsmap finished waiting for FSMap version 20 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f932ca06b80 10 client.5370 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f932ca06b80 20 client.5370 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f932ca06b80 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x559c60abcf70 0x559c60a9d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f932ca06b80 4 client.5370 mds_command: new command op to 4742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f932ca06b80 1 -- 172.21.15.5:0/2291221306 --> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c60a3f790 con 0x559c60abcf70 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f9319d9b700 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x559c60abcf70 0x559c60a9d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f9319d9b700 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x559c60abcf70 0x559c60a9d210 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75a86c0bb1fb2067 server_cookie=99822223f718b369 in_seq=0 out_seq=0 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.685+0000 7f92f97fa700 10 client.5370 ms_handle_connect on v2:172.21.15.5:6832/295306227 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f92f97fa700 1 -- 172.21.15.5:0/2291221306 <== mds.2 v2:172.21.15.5:6832/295306227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c60a3f790 con 0x559c60abcf70 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f92f97fa700 10 client.5370 handle_command_reply: tid=0 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 client.5370 shutdown 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 -- 172.21.15.5:0/2291221306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 msgr2=0x559c60a91e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a91e00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f930c007460 tx=0x7f930c000ed0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 -- 172.21.15.5:0/2291221306 shutdown_connections 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x559c60abcf70 0x559c60a9d210 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 0x559c60a6a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60a91e00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 --2- 172.21.15.5:0/2291221306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60a69c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f92f97fa700 0 client.5370 ms_handle_reset on v2:172.21.15.5:6832/295306227 2024-05-08T23:22:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 -- 172.21.15.5:0/2291221306 >> 172.21.15.5:0/2291221306 conn(0x559c609ce1c0 msgr2=0x559c60a70360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 -- 172.21.15.5:0/2291221306 shutdown_connections 2024-05-08T23:22:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.686+0000 7f932ca06b80 1 -- 172.21.15.5:0/2291221306 wait complete. 2024-05-08T23:22:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.687+0000 7f932ca06b80 20 client.5370 trim_cache size 0 max 16384 2024-05-08T23:22:23.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.706+0000 7f932ca06b80 1 Processor -- start 2024-05-08T23:22:23.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.706+0000 7f932ca06b80 1 -- start start 2024-05-08T23:22:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60aba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60ab9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 0x559c60a90ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f932ca06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60a70890 con 0x559c60a6e320 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f932ca06b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60abde80 con 0x559c60a9ad80 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f932ca06b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c60a933c0 con 0x559c60a93750 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f931a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60aba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f931a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60aba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57292/0 (socket says 172.21.15.5:57292) 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f931a59c700 1 -- 172.21.15.5:0/3926880602 learned_addr learned my addr 172.21.15.5:0/3926880602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f9319d9b700 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60ab9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f931ad9d700 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 0x559c60a90ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f931a59c700 1 -- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 msgr2=0x559c60ab9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f931a59c700 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60ab9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f931a59c700 1 -- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 msgr2=0x559c60a90ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.707+0000 7f931a59c700 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 0x559c60a90ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f931a59c700 1 -- 172.21.15.5:0/3926880602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c607edb10 con 0x559c60a6e320 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f9319d9b700 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60ab9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f931a59c700 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60aba3a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f930c0017a0 tx=0x7f930c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72b862b183f8218 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f9308ff9700 1 -- 172.21.15.5:0/3926880602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c012410 con 0x559c60a6e320 2024-05-08T23:22:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f9308ff9700 1 -- 172.21.15.5:0/3926880602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c018070 con 0x559c60a6e320 2024-05-08T23:22:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f9308ff9700 1 -- 172.21.15.5:0/3926880602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c012700 con 0x559c60a6e320 2024-05-08T23:22:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f932ca06b80 1 -- 172.21.15.5:0/3926880602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 msgr2=0x559c60aba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60aba3a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f930c0017a0 tx=0x7f930c019000 comp rx=0 tx=0).stop 2024-05-08T23:22:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f932ca06b80 1 -- 172.21.15.5:0/3926880602 shutdown_connections 2024-05-08T23:22:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a9ad80 0x559c60a90ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60aba3a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.708+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3926880602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a93750 0x559c60ab9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.709+0000 7f932ca06b80 1 -- 172.21.15.5:0/3926880602 >> 172.21.15.5:0/3926880602 conn(0x559c609ce1c0 msgr2=0x559c60a67190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.709+0000 7f932ca06b80 1 -- 172.21.15.5:0/3926880602 shutdown_connections 2024-05-08T23:22:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.709+0000 7f932ca06b80 1 -- 172.21.15.5:0/3926880602 wait complete. 2024-05-08T23:22:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.709+0000 7f932ca06b80 1 -- start start 2024-05-08T23:22:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.709+0000 7f932ca06b80 1 Processor -- start 2024-05-08T23:22:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60abd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a98c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f932ca06b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 0x559c60a991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f932ca06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60b1fb30 con 0x559c60a6e320 2024-05-08T23:22:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f932ca06b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60aba3a0 con 0x559c60a93750 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f932ca06b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c60abdc90 con 0x559c60a9ad80 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f932ca06b80 10 client.0 _ll_register_callbacks cb 0x559c60965ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60abd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60abd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57304/0 (socket says 172.21.15.5:57304) 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 -- 172.21.15.5:0/3115555243 learned_addr learned my addr 172.21.15.5:0/3115555243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f9319d9b700 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a98c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931ad9d700 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 0x559c60a991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 -- 172.21.15.5:0/3115555243 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 msgr2=0x559c60a991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 0x559c60a991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 -- 172.21.15.5:0/3115555243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 msgr2=0x559c60a98c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a98c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 -- 172.21.15.5:0/3115555243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c0b10a0 con 0x559c60a6e320 2024-05-08T23:22:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f931a59c700 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60abd750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f930c00d430 tx=0x7f930c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f953d783f9dd09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.711+0000 7f92f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f92f97fa700 1 -- 172.21.15.5:0/3115555243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c012750 con 0x559c60a6e320 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f92f97fa700 1 -- 172.21.15.5:0/3115555243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f930c012b20 con 0x559c60a6e320 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 1 -- 172.21.15.5:0/3115555243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9310004300 con 0x559c60a6e320 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f92f97fa700 1 -- 172.21.15.5:0/3115555243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f930c012d40 con 0x559c60a6e320 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 10 client.5376 fetch_fsmap learned FSMap version 20 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 1 -- 172.21.15.5:0/3115555243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x559c60986510 con 0x559c60a6e320 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f92f97fa700 1 -- 172.21.15.5:0/3115555243 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310004300 con 0x559c60a6e320 2024-05-08T23:22:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f92f97fa700 1 -- 172.21.15.5:0/3115555243 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 20) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f930c033040 con 0x559c60a6e320 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 10 client.5376 fetch_fsmap finished waiting for FSMap version 20 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 10 client.5376 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 20 client.5376 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x559c60b1dc10 0x559c60b1dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 4 client.5376 mds_command: new command op to 4742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f932ca06b80 1 -- 172.21.15.5:0/3115555243 --> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559c609b8e70 con 0x559c60b1dc10 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.712+0000 7f9319d9b700 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x559c60b1dc10 0x559c60b1dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f92f97fa700 10 client.5376 ms_handle_connect on v2:172.21.15.5:6832/295306227 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f9319d9b700 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x559c60b1dc10 0x559c60b1dff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c3ae9696e7a2c42 server_cookie=7c7bde6e5ea29f60 in_seq=0 out_seq=0 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f92f97fa700 1 -- 172.21.15.5:0/3115555243 <== mds.2 v2:172.21.15.5:6832/295306227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x559c609b8e70 con 0x559c60b1dc10 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f92f97fa700 10 client.5376 handle_command_reply: tid=0 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 client.5376 shutdown 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 -- 172.21.15.5:0/3115555243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 msgr2=0x559c60abd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60abd750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f930c00d430 tx=0x7f930c012000 comp rx=0 tx=0).stop 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 -- 172.21.15.5:0/3115555243 shutdown_connections 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x559c60b1dc10 0x559c60b1dff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559c60a9ad80 0x559c60a991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c60a6e320 0x559c60abd750 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 --2- 172.21.15.5:0/3115555243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c60a93750 0x559c60a98c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f92f97fa700 0 client.5376 ms_handle_reset on v2:172.21.15.5:6832/295306227 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 -- 172.21.15.5:0/3115555243 >> 172.21.15.5:0/3115555243 conn(0x559c609ce1c0 msgr2=0x559c60a9a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 -- 172.21.15.5:0/3115555243 shutdown_connections 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 1 -- 172.21.15.5:0/3115555243 wait complete. 2024-05-08T23:22:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.713+0000 7f932ca06b80 20 client.5376 trim_cache size 0 max 16384 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4742, 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 1.8931325349999999, 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 20, 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 34, 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 34, 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 24.251673754999999 2024-05-08T23:22:23.723 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 -- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92fc06ac90 msgr2=0x7f92fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92fc06ac90 0x7f92fc06d150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f930c0011e0 tx=0x7f930c009040 comp rx=0 tx=0).stop 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 -- 172.21.15.5:0/3545212915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c15b900 msgr2=0x7f931c1f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c15b900 0x7f931c1f7210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f931400cc20 tx=0x7f931400a060 comp rx=0 tx=0).stop 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 -- 172.21.15.5:0/3545212915 shutdown_connections 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f92fc06ac90 0x7f92fc06d150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c15b900 0x7f931c1f7210 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f931c0b18a0 0x7f931c06cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 --2- 172.21.15.5:0/3545212915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f931c0cf9e0 0x7f931c0c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 -- 172.21.15.5:0/3545212915 >> 172.21.15.5:0/3545212915 conn(0x7f931c0b2f20 msgr2=0x7f931c0cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 -- 172.21.15.5:0/3545212915 shutdown_connections 2024-05-08T23:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.714+0000 7f932084c700 1 -- 172.21.15.5:0/3545212915 wait complete. 2024-05-08T23:22:23.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.868+0000 7fcb0ab5a700 1 Processor -- start 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.868+0000 7fcb0ab5a700 1 -- start start 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb0ab5a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0409a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb0ab5a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb040b18a0 0x7fcb040b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb0ab5a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb040b1410 con 0x7fcb0415b900 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb0ab5a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb040b0e30 con 0x7fcb040b18a0 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb0ab5a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040cffd0 con 0x7fcb040cf9e0 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb090f7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb040b18a0 0x7fcb040b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb090f7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb040b18a0 0x7fcb040b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57310/0 (socket says 172.21.15.5:57310) 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb090f7700 1 -- 172.21.15.5:0/3298397407 learned_addr learned my addr 172.21.15.5:0/3298397407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.869+0000 7fcb03fff700 1 --2- 172.21.15.5:0/3298397407 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0409a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.870+0000 7fcb090f7700 1 -- 172.21.15.5:0/3298397407 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 msgr2=0x7fcb0409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.870+0000 7fcb090f7700 1 --2- 172.21.15.5:0/3298397407 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0409a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.870+0000 7fcb090f7700 1 -- 172.21.15.5:0/3298397407 >> v1:172.21.15.49:6789/0 conn(0x7fcb0415b900 legacy=0x7fcb04099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.870+0000 7fcb090f7700 1 -- 172.21.15.5:0/3298397407 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040b10a0 con 0x7fcb040b18a0 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.870+0000 7fcb090f7700 1 --2- 172.21.15.5:0/3298397407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb040b18a0 0x7fcb040b1d30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009e50 tx=0x7fcafc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a313764a194c9d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.870+0000 7fcb037fe700 1 -- 172.21.15.5:0/3298397407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00e070 con 0x7fcb040b18a0 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.870+0000 7fcb037fe700 1 -- 172.21.15.5:0/3298397407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc003a20 con 0x7fcb040b18a0 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.870+0000 7fcb037fe700 1 -- 172.21.15.5:0/3298397407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc003dd0 con 0x7fcb040b18a0 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.871+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/3298397407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb040b18a0 msgr2=0x7fcb040b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.871+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/3298397407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb040b18a0 0x7fcb040b1d30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009e50 tx=0x7fcafc00b330 comp rx=0 tx=0).stop 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.871+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/3298397407 shutdown_connections 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.871+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/3298397407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb040b18a0 0x7fcb040b1d30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.871+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/3298397407 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0409a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.871+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/3298397407 >> 172.21.15.5:0/3298397407 conn(0x7fcb040b2f20 msgr2=0x7fcb040b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.871+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/3298397407 shutdown_connections 2024-05-08T23:22:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.871+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/3298397407 wait complete. 2024-05-08T23:22:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.872+0000 7fcb0ab5a700 1 Processor -- start 2024-05-08T23:22:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.872+0000 7fcb0ab5a700 1 -- start start 2024-05-08T23:22:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb0ab5a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb040b18a0 0x7fcb040b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb0ab5a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb0ab5a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb0415b900 0x7fcb0415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb0ab5a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0415dd00 con 0x7fcb0415b900 2024-05-08T23:22:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb0ab5a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0415e6d0 con 0x7fcb040b18a0 2024-05-08T23:22:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb0ab5a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0415d0f0 con 0x7fcb040cf9e0 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb090f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb0415b900 0x7fcb0415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb090f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb0415b900 0x7fcb0415e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57320/0 (socket says 172.21.15.5:57320) 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb090f7700 1 -- 172.21.15.5:0/379419942 learned_addr learned my addr 172.21.15.5:0/379419942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb088f6700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb040b18a0 0x7fcb040b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb03fff700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb090f7700 1 -- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 msgr2=0x7fcb0414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb090f7700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0414ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb090f7700 1 -- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb040b18a0 msgr2=0x7fcb040b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb090f7700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb040b18a0 0x7fcb040b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb090f7700 1 -- 172.21.15.5:0/379419942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc009b30 con 0x7fcb0415b900 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.873+0000 7fcb03fff700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.874+0000 7fcb090f7700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb0415b900 0x7fcb0415e190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcb04072530 tx=0x7fcafc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.874+0000 7fcb017fa700 1 -- 172.21.15.5:0/379419942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0094d0 con 0x7fcb0415b900 2024-05-08T23:22:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.874+0000 7fcb017fa700 1 -- 172.21.15.5:0/379419942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc004240 con 0x7fcb0415b900 2024-05-08T23:22:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.874+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/379419942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0415d3d0 con 0x7fcb0415b900 2024-05-08T23:22:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.874+0000 7fcb017fa700 1 -- 172.21.15.5:0/379419942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0045f0 con 0x7fcb0415b900 2024-05-08T23:22:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.874+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/379419942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0415d990 con 0x7fcb0415b900 2024-05-08T23:22:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.876+0000 7fcb017fa700 1 -- 172.21.15.5:0/379419942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcafc019070 con 0x7fcb0415b900 2024-05-08T23:22:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.876+0000 7fcb017fa700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcaf006ac90 0x7fcaf006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.876+0000 7fcb017fa700 1 -- 172.21.15.5:0/379419942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcafc08a290 con 0x7fcb0415b900 2024-05-08T23:22:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.876+0000 7fcb088f6700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcaf006ac90 0x7fcaf006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.877+0000 7fcb088f6700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcaf006ac90 0x7fcaf006d150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009870 tx=0x7fcaf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.878+0000 7fcb16d14b80 1 Processor -- start 2024-05-08T23:22:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.878+0000 7fcb16d14b80 1 -- start start 2024-05-08T23:22:23.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac43f750 0x5555ac4180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 0x5555ac41d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac446d80 0x5555ac447200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac373100 con 0x5555ac446d80 2024-05-08T23:22:23.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac370ac0 con 0x5555ac41a320 2024-05-08T23:22:23.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555ac36c3b0 con 0x5555ac43f750 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb090f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac446d80 0x5555ac447200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb090f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac446d80 0x5555ac447200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57324/0 (socket says 172.21.15.5:57324) 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb090f7700 1 -- 172.21.15.5:0/2840268102 learned_addr learned my addr 172.21.15.5:0/2840268102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb03fff700 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 0x5555ac41d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.879+0000 7fcb088f6700 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac43f750 0x5555ac4180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcb090f7700 1 -- 172.21.15.5:0/2840268102 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac43f750 msgr2=0x5555ac4180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcb090f7700 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac43f750 0x5555ac4180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcb090f7700 1 -- 172.21.15.5:0/2840268102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 msgr2=0x5555ac41d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcb090f7700 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 0x5555ac41d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcb090f7700 1 -- 172.21.15.5:0/2840268102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555ac1ed360 con 0x5555ac446d80 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcb090f7700 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac446d80 0x5555ac447200 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0590b0 tx=0x7fcafc0598f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60de2231abf053b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcaea7fc700 1 -- 172.21.15.5:0/2840268102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc05b070 con 0x5555ac446d80 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcaea7fc700 1 -- 172.21.15.5:0/2840268102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc08a0c0 con 0x5555ac446d80 2024-05-08T23:22:23.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.880+0000 7fcaea7fc700 1 -- 172.21.15.5:0/2840268102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc05cf50 con 0x5555ac446d80 2024-05-08T23:22:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 -- 172.21.15.5:0/2840268102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac446d80 msgr2=0x5555ac447200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac446d80 0x5555ac447200 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0590b0 tx=0x7fcafc0598f0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 -- 172.21.15.5:0/2840268102 shutdown_connections 2024-05-08T23:22:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac446d80 0x5555ac447200 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac43f750 0x5555ac4180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/2840268102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 0x5555ac41d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 -- 172.21.15.5:0/2840268102 >> 172.21.15.5:0/2840268102 conn(0x5555ac37a1c0 msgr2=0x5555ac461d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 -- 172.21.15.5:0/2840268102 shutdown_connections 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 -- 172.21.15.5:0/2840268102 wait complete. 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.881+0000 7fcb16d14b80 1 -- start start 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.882+0000 7fcb16d14b80 1 Processor -- start 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 0x5555ac465ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac43f750 0x5555ac447160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac308510 con 0x5555ac43f750 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac3081b0 con 0x5555ac41a320 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555ac1ef650 con 0x5555ac446d80 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb16d14b80 10 client.0 _ll_register_callbacks cb 0x5555ac311ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac43f750 0x5555ac447160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac43f750 0x5555ac447160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57326/0 (socket says 172.21.15.5:57326) 2024-05-08T23:22:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 -- 172.21.15.5:0/3766050549 learned_addr learned my addr 172.21.15.5:0/3766050549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb088f6700 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 0x5555ac465ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb090f7700 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 -- 172.21.15.5:0/3766050549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 msgr2=0x5555ac4476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 -- 172.21.15.5:0/3766050549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 msgr2=0x5555ac465ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 0x5555ac465ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 -- 172.21.15.5:0/3766050549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0415d990 con 0x5555ac43f750 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcb03fff700 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac43f750 0x5555ac447160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcb040c8f90 tx=0x7fcaf8012660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd418f67c182cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcae37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.883+0000 7fcae37fe700 1 -- 172.21.15.5:0/3766050549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8004020 con 0x5555ac43f750 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcae37fe700 1 -- 172.21.15.5:0/3766050549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaf8004180 con 0x5555ac43f750 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3766050549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555ac2c5220 con 0x5555ac43f750 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcae37fe700 1 -- 172.21.15.5:0/3766050549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf8004340 con 0x5555ac43f750 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 10 client.5388 fetch_fsmap learned FSMap version 20 2024-05-08T23:22:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3766050549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x5555ac199b10 con 0x5555ac43f750 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcae37fe700 1 -- 172.21.15.5:0/3766050549 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5555ac2c5220 con 0x5555ac43f750 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcae37fe700 1 -- 172.21.15.5:0/3766050549 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 20) v1 ==== 3138+0+0 (secure 0 0 0) 0x7fcaf80036a0 con 0x5555ac43f750 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 10 client.5388 fetch_fsmap finished waiting for FSMap version 20 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 10 client.5388 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 20 client.5388 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5555ac4488c0 0x5555ac471070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 4 client.5388 mds_command: new command op to 4745 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3766050549 --> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555ac3eb790 con 0x5555ac4488c0 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.884+0000 7fcb088f6700 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5555ac4488c0 0x5555ac471070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.885+0000 7fcae37fe700 10 client.5388 ms_handle_connect on v2:172.21.15.5:6833/146081129 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.885+0000 7fcb088f6700 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5555ac4488c0 0x5555ac471070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1d0ff5539160756 server_cookie=6cb9c0377b0fae50 in_seq=0 out_seq=0 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.885+0000 7fcae37fe700 1 -- 172.21.15.5:0/3766050549 <== mds.1 v2:172.21.15.5:6833/146081129 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555ac3eb790 con 0x5555ac4488c0 2024-05-08T23:22:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.885+0000 7fcae37fe700 10 client.5388 handle_command_reply: tid=0 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.885+0000 7fcb16d14b80 1 client.5388 shutdown 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.885+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3766050549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac43f750 msgr2=0x5555ac447160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.885+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac43f750 0x5555ac447160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcb040c8f90 tx=0x7fcaf8012660 comp rx=0 tx=0).stop 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3766050549 shutdown_connections 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5555ac4488c0 0x5555ac471070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac41a320 0x5555ac465ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcae37fe700 0 client.5388 ms_handle_reset on v2:172.21.15.5:6833/146081129 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3766050549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac43f750 0x5555ac447160 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3766050549 >> 172.21.15.5:0/3766050549 conn(0x5555ac37a1c0 msgr2=0x5555ac41c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3766050549 shutdown_connections 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3766050549 wait complete. 2024-05-08T23:22:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.886+0000 7fcb16d14b80 20 client.5388 trim_cache size 0 max 16384 2024-05-08T23:22:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.905+0000 7fcb16d14b80 1 Processor -- start 2024-05-08T23:22:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.905+0000 7fcb16d14b80 1 -- start start 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac471eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac41d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac473be0 con 0x5555ac41a320 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac44a500 con 0x5555ac43f750 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555ac464860 con 0x5555ac446d80 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb088f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac471eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb088f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac471eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57342/0 (socket says 172.21.15.5:57342) 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb088f6700 1 -- 172.21.15.5:0/3051654653 learned_addr learned my addr 172.21.15.5:0/3051654653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb090f7700 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb03fff700 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac41d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb088f6700 1 -- 172.21.15.5:0/3051654653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 msgr2=0x5555ac4613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb088f6700 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb088f6700 1 -- 172.21.15.5:0/3051654653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 msgr2=0x5555ac41d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb088f6700 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac41d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.906+0000 7fcb088f6700 1 -- 172.21.15.5:0/3051654653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555ac332510 con 0x5555ac41a320 2024-05-08T23:22:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcb088f6700 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac471eb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5555ac443b90 tx=0x7fcaf4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92988761ed60aa3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcaeaffd700 1 -- 172.21.15.5:0/3051654653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4014e10 con 0x5555ac41a320 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcaeaffd700 1 -- 172.21.15.5:0/3051654653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf402b070 con 0x5555ac41a320 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcaeaffd700 1 -- 172.21.15.5:0/3051654653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf40237b0 con 0x5555ac41a320 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3051654653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 msgr2=0x5555ac471eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac471eb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5555ac443b90 tx=0x7fcaf4013000 comp rx=0 tx=0).stop 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3051654653 shutdown_connections 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac471eb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/3051654653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac41d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.907+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3051654653 >> 172.21.15.5:0/3051654653 conn(0x5555ac37a1c0 msgr2=0x5555ac413190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.908+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3051654653 shutdown_connections 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.908+0000 7fcb16d14b80 1 -- 172.21.15.5:0/3051654653 wait complete. 2024-05-08T23:22:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.908+0000 7fcb16d14b80 1 -- start start 2024-05-08T23:22:23.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.908+0000 7fcb16d14b80 1 Processor -- start 2024-05-08T23:22:23.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac4ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac4ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb16d14b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac41aa80 con 0x5555ac41a320 2024-05-08T23:22:23.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac4cb880 con 0x5555ac43f750 2024-05-08T23:22:23.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb16d14b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555ac4cbad0 con 0x5555ac446d80 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb16d14b80 10 client.0 _ll_register_callbacks cb 0x5555ac311ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb088f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac4ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb088f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac4ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57354/0 (socket says 172.21.15.5:57354) 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.909+0000 7fcb088f6700 1 -- 172.21.15.5:0/1711172264 learned_addr learned my addr 172.21.15.5:0/1711172264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb090f7700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb03fff700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac4ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb088f6700 1 -- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 msgr2=0x5555ac4cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb088f6700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb088f6700 1 -- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 msgr2=0x5555ac4ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb088f6700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac4ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb088f6700 1 -- 172.21.15.5:0/1711172264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf400e040 con 0x5555ac41a320 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb03fff700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac4ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb090f7700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb088f6700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac4ce240 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4014520 tx=0x7fcaf4004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f06d9aed99669d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcae37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcae37fe700 1 -- 172.21.15.5:0/1711172264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4028070 con 0x5555ac41a320 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcae37fe700 1 -- 172.21.15.5:0/1711172264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaf4023a80 con 0x5555ac41a320 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb16d14b80 1 -- 172.21.15.5:0/1711172264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcafc08a0c0 con 0x5555ac41a320 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcae37fe700 1 -- 172.21.15.5:0/1711172264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf4018040 con 0x5555ac41a320 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb16d14b80 10 client.5394 fetch_fsmap learned FSMap version 20 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcb16d14b80 1 -- 172.21.15.5:0/1711172264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x5555ac1f1c80 con 0x5555ac41a320 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.910+0000 7fcae37fe700 1 -- 172.21.15.5:0/1711172264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc08a0c0 con 0x5555ac41a320 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcae37fe700 1 -- 172.21.15.5:0/1711172264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 20) v1 ==== 3138+0+0 (secure 0 0 0) 0x7fcaf4023c40 con 0x5555ac41a320 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb16d14b80 10 client.5394 fetch_fsmap finished waiting for FSMap version 20 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb16d14b80 10 client.5394 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:22:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb16d14b80 20 client.5394 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5555ac4496c0 0x5555ac449aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb16d14b80 4 client.5394 mds_command: new command op to 4745 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb16d14b80 1 -- 172.21.15.5:0/1711172264 --> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5555ac364e70 con 0x5555ac4496c0 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb03fff700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5555ac4496c0 0x5555ac449aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcae37fe700 10 client.5394 ms_handle_connect on v2:172.21.15.5:6833/146081129 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb03fff700 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5555ac4496c0 0x5555ac449aa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37955157f9ca47c0 server_cookie=72383db54bd7745f in_seq=0 out_seq=0 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcae37fe700 1 -- 172.21.15.5:0/1711172264 <== mds.1 v2:172.21.15.5:6833/146081129 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5555ac364e70 con 0x5555ac4496c0 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcae37fe700 10 client.5394 handle_command_reply: tid=0 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.911+0000 7fcb16d14b80 1 client.5394 shutdown 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 -- 172.21.15.5:0/1711172264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 msgr2=0x5555ac4ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac4ce240 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4014520 tx=0x7fcaf4004710 comp rx=0 tx=0).stop 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 -- 172.21.15.5:0/1711172264 shutdown_connections 2024-05-08T23:22:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5555ac4496c0 0x5555ac449aa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5555ac446d80 0x5555ac4cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ac41a320 0x5555ac4ce240 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcae37fe700 0 client.5394 ms_handle_reset on v2:172.21.15.5:6833/146081129 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 --2- 172.21.15.5:0/1711172264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555ac43f750 0x5555ac4ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 -- 172.21.15.5:0/1711172264 >> 172.21.15.5:0/1711172264 conn(0x5555ac37a1c0 msgr2=0x5555ac483870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 -- 172.21.15.5:0/1711172264 shutdown_connections 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 1 -- 172.21.15.5:0/1711172264 wait complete. 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb16d14b80 20 client.5394 trim_cache size 0 max 16384 2024-05-08T23:22:23.919 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4745, 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 3.1003026619999998, 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 20, 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 34, 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 34, 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 24.449577714 2024-05-08T23:22:23.920 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcaf006ac90 msgr2=0x7fcaf006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcaf006ac90 0x7fcaf006d150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009870 tx=0x7fcaf4008040 comp rx=0 tx=0).stop 2024-05-08T23:22:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/379419942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb0415b900 msgr2=0x7fcb0415e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb0415b900 0x7fcb0415e190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcb04072530 tx=0x7fcafc004ad0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/379419942 shutdown_connections 2024-05-08T23:22:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcaf006ac90 0x7fcaf006d150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb0415b900 0x7fcb0415e190 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb040b18a0 0x7fcb040b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 --2- 172.21.15.5:0/379419942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcb040cf9e0 0x7fcb0414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.912+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/379419942 >> 172.21.15.5:0/379419942 conn(0x7fcb040b2f20 msgr2=0x7fcb040b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.913+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/379419942 shutdown_connections 2024-05-08T23:22:23.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:23.913+0000 7fcb0ab5a700 1 -- 172.21.15.5:0/379419942 wait complete. 2024-05-08T23:22:23.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T23:22:24.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.064+0000 7f998ad52700 1 Processor -- start 2024-05-08T23:22:24.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.064+0000 7f998ad52700 1 -- start start 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f998ad52700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f998406adc0 0x7f99840ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f998ad52700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840cd7c0 con 0x7f99840cc060 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f998ad52700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840cffb0 con 0x7f998406a960 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f998ad52700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99840ccc80 con 0x7f998406adc0 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f9988aee700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f99840cc060 0x7f99840cf8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37588/0 (socket says 172.21.15.5:37588) 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f9988aee700 1 -- 172.21.15.5:0/943626713 learned_addr learned my addr 172.21.15.5:0/943626713 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99892ef700 1 --2- 172.21.15.5:0/943626713 >> v2:172.21.15.49:3300/0 conn(0x7f998406adc0 0x7f99840ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99837fe700 1 -- 172.21.15.5:0/943626713 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505325476 0 0) 0x7f99840cd7c0 con 0x7f99840cc060 2024-05-08T23:22:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99837fe700 1 -- 172.21.15.5:0/943626713 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99700035b0 con 0x7f99840cc060 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99837fe700 1 -- 172.21.15.5:0/943626713 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559646126 0 0) 0x7f99840cffb0 con 0x7f998406a960 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99837fe700 1 -- 172.21.15.5:0/943626713 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99840cd7c0 con 0x7f998406a960 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99892ef700 1 -- 172.21.15.5:0/943626713 >> v1:172.21.15.49:6789/0 conn(0x7f998406a960 legacy=0x7f99840ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99892ef700 1 -- 172.21.15.5:0/943626713 >> v1:172.21.15.5:6789/0 conn(0x7f99840cc060 legacy=0x7f99840cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99892ef700 1 -- 172.21.15.5:0/943626713 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840cce90 con 0x7f998406adc0 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.066+0000 7f99892ef700 1 --2- 172.21.15.5:0/943626713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f998406adc0 0x7f99840ce940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f997c009d90 tx=0x7f997c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a245712270f9cd77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f99837fe700 1 -- 172.21.15.5:0/943626713 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c00e070 con 0x7f998406adc0 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f99837fe700 1 -- 172.21.15.5:0/943626713 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c00bb20 con 0x7f998406adc0 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f99837fe700 1 -- 172.21.15.5:0/943626713 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c00be90 con 0x7f998406adc0 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f998ad52700 1 -- 172.21.15.5:0/943626713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f998406adc0 msgr2=0x7f99840ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f998ad52700 1 --2- 172.21.15.5:0/943626713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f998406adc0 0x7f99840ce940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f997c009d90 tx=0x7f997c00b330 comp rx=0 tx=0).stop 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f998ad52700 1 -- 172.21.15.5:0/943626713 shutdown_connections 2024-05-08T23:22:24.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f998ad52700 1 --2- 172.21.15.5:0/943626713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f998406adc0 0x7f99840ce940 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f998ad52700 1 -- 172.21.15.5:0/943626713 >> 172.21.15.5:0/943626713 conn(0x7f9984099a50 msgr2=0x7f99840d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:24.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f998ad52700 1 -- 172.21.15.5:0/943626713 shutdown_connections 2024-05-08T23:22:24.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.067+0000 7f998ad52700 1 -- 172.21.15.5:0/943626713 wait complete. 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.069+0000 7f998ad52700 1 Processor -- start 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.069+0000 7f998ad52700 1 -- start start 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f998ad52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f998406a960 0x7f99840c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f998ad52700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f998406adc0 0x7f998414f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f998ad52700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99840cc060 0x7f99840c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f998ad52700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99840c83f0 con 0x7f998406a960 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f998ad52700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998414fbd0 con 0x7f99840cc060 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f998ad52700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99840c77e0 con 0x7f998406adc0 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f9988aee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f998406a960 0x7f99840c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f9988aee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f998406a960 0x7f99840c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57358/0 (socket says 172.21.15.5:57358) 2024-05-08T23:22:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.070+0000 7f9988aee700 1 -- 172.21.15.5:0/3381416036 learned_addr learned my addr 172.21.15.5:0/3381416036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f9983fff700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f998406adc0 0x7f998414f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f99892ef700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99840cc060 0x7f99840c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f9988aee700 1 -- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f998406adc0 msgr2=0x7f998414f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f9988aee700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f998406adc0 0x7f998414f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f9988aee700 1 -- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99840cc060 msgr2=0x7f99840c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f9988aee700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99840cc060 0x7f99840c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f9988aee700 1 -- 172.21.15.5:0/3381416036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c009a70 con 0x7f998406a960 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f99892ef700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99840cc060 0x7f99840c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f9988aee700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f998406a960 0x7f99840c6b10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f997400c700 tx=0x7f997400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f99817fa700 1 -- 172.21.15.5:0/3381416036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974005d70 con 0x7f998406a960 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f99817fa700 1 -- 172.21.15.5:0/3381416036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99740070d0 con 0x7f998406a960 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f998ad52700 1 -- 172.21.15.5:0/3381416036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99840c7ac0 con 0x7f998406a960 2024-05-08T23:22:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f99817fa700 1 -- 172.21.15.5:0/3381416036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974007480 con 0x7f998406a960 2024-05-08T23:22:24.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.071+0000 7f998ad52700 1 -- 172.21.15.5:0/3381416036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99840c8080 con 0x7f998406a960 2024-05-08T23:22:24.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.073+0000 7f99817fa700 1 -- 172.21.15.5:0/3381416036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f99740075e0 con 0x7f998406a960 2024-05-08T23:22:24.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.073+0000 7f99817fa700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f996406ac90 0x7f996406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.073+0000 7f99817fa700 1 -- 172.21.15.5:0/3381416036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99740889b0 con 0x7f998406a960 2024-05-08T23:22:24.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.074+0000 7f9983fff700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f996406ac90 0x7f996406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.074+0000 7f9983fff700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f996406ac90 0x7f996406d150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f996c0010e0 tx=0x7f996c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.075+0000 7f9996f0cb80 1 Processor -- start 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.076+0000 7f9996f0cb80 1 -- start start 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.076+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb5182c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.076+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5182e320 0x55cb51831050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.076+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5185ad80 0x55cb5185b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.076+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb51787100 con 0x55cb5185ad80 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.076+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb51784ac0 con 0x55cb51853750 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.076+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb517803b0 con 0x55cb5182e320 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.076+0000 7f99892ef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5185ad80 0x55cb5185b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f99892ef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5185ad80 0x55cb5185b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57360/0 (socket says 172.21.15.5:57360) 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f9988aee700 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb5182c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f9983fff700 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5182e320 0x55cb51831050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f99892ef700 1 -- 172.21.15.5:0/2053941612 learned_addr learned my addr 172.21.15.5:0/2053941612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f9988aee700 1 -- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5182e320 msgr2=0x55cb51831050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f9988aee700 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5182e320 0x55cb51831050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f9988aee700 1 -- 172.21.15.5:0/2053941612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5185ad80 msgr2=0x55cb5185b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f9988aee700 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5185ad80 0x55cb5185b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f9988aee700 1 -- 172.21.15.5:0/2053941612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb51601360 con 0x55cb51853750 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f99892ef700 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5185ad80 0x55cb5185b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f9988aee700 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb5182c0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9974086f40 tx=0x7f9974057290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c540aad91efd26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f997a7fc700 1 -- 172.21.15.5:0/2053941612 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974057860 con 0x55cb51853750 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f997a7fc700 1 -- 172.21.15.5:0/2053941612 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997405b690 con 0x55cb51853750 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.077+0000 7f997a7fc700 1 -- 172.21.15.5:0/2053941612 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997405fc50 con 0x55cb51853750 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 msgr2=0x55cb5182c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb5182c0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9974086f40 tx=0x7f9974057290 comp rx=0 tx=0).stop 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2053941612 shutdown_connections 2024-05-08T23:22:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5185ad80 0x55cb5185b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb5182c0c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2053941612 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5182e320 0x55cb51831050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2053941612 >> 172.21.15.5:0/2053941612 conn(0x55cb5178e1c0 msgr2=0x55cb51875d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2053941612 shutdown_connections 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.078+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2053941612 wait complete. 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.079+0000 7f9996f0cb80 1 -- start start 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.079+0000 7f9996f0cb80 1 Processor -- start 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51842150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb51853750 0x55cb51842690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5185ad80 0x55cb518430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5171c510 con 0x55cb5182e320 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5171c1b0 con 0x55cb5185ad80 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb51603650 con 0x55cb51853750 2024-05-08T23:22:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9996f0cb80 10 client.0 _ll_register_callbacks cb 0x55cb51725ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9988aee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51842150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9988aee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51842150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57370/0 (socket says 172.21.15.5:57370) 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9988aee700 1 -- 172.21.15.5:0/2037946318 learned_addr learned my addr 172.21.15.5:0/2037946318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f99892ef700 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5185ad80 0x55cb518430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9983fff700 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb51853750 0x55cb51842690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9988aee700 1 -- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb51853750 msgr2=0x55cb51842690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9988aee700 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb51853750 0x55cb51842690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9988aee700 1 -- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5185ad80 msgr2=0x55cb518430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9988aee700 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5185ad80 0x55cb518430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f9988aee700 1 -- 172.21.15.5:0/2037946318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99840cce90 con 0x55cb5182e320 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.080+0000 7f99892ef700 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5185ad80 0x55cb518430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9988aee700 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51842150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9974057560 tx=0x7f9974000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2755b6583b8ad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f99637fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f99637fe700 1 -- 172.21.15.5:0/2037946318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974057860 con 0x55cb5182e320 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f99637fe700 1 -- 172.21.15.5:0/2037946318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9974056d50 con 0x55cb5182e320 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2037946318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb516d9220 con 0x55cb5182e320 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f99637fe700 1 -- 172.21.15.5:0/2037946318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9974056f10 con 0x55cb5182e320 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 10 client.5406 fetch_fsmap learned FSMap version 20 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2037946318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x7f99840c8080 con 0x55cb5182e320 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f99637fe700 1 -- 172.21.15.5:0/2037946318 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb516d9220 con 0x55cb5182e320 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f99637fe700 1 -- 172.21.15.5:0/2037946318 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 20) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f997405f7c0 con 0x55cb5182e320 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 10 client.5406 fetch_fsmap finished waiting for FSMap version 20 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 10 client.5406 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 20 client.5406 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55cb5185d0a0 0x55cb51892890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 4 client.5406 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.081+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2037946318 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb517ff790 con 0x55cb5185d0a0 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.082+0000 7f9983fff700 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55cb5185d0a0 0x55cb51892890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.082+0000 7f99637fe700 10 client.5406 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.082+0000 7f9983fff700 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55cb5185d0a0 0x55cb51892890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155bb5539a4c7c32 server_cookie=990c66c5962f2e1f in_seq=0 out_seq=0 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f99637fe700 1 -- 172.21.15.5:0/2037946318 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb517ff790 con 0x55cb5185d0a0 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f99637fe700 10 client.5406 handle_command_reply: tid=0 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 client.5406 shutdown 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2037946318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 msgr2=0x55cb51842150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51842150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9974057560 tx=0x7f9974000ed0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2037946318 shutdown_connections 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55cb5185d0a0 0x55cb51892890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5185ad80 0x55cb518430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51842150 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2037946318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb51853750 0x55cb51842690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f99637fe700 0 client.5406 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2037946318 >> 172.21.15.5:0/2037946318 conn(0x55cb5178e1c0 msgr2=0x55cb5185b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2037946318 shutdown_connections 2024-05-08T23:22:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2037946318 wait complete. 2024-05-08T23:22:24.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.083+0000 7f9996f0cb80 20 client.5406 trim_cache size 0 max 16384 2024-05-08T23:22:24.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.102+0000 7f9996f0cb80 1 Processor -- start 2024-05-08T23:22:24.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.102+0000 7f9996f0cb80 1 -- start start 2024-05-08T23:22:24.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.103+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5182e320 0x55cb51853c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.103+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb51853750 0x55cb5185b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.103+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 0x55cb518d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.103+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb518549f0 con 0x55cb51853750 2024-05-08T23:22:24.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.103+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb51854190 con 0x55cb5182e320 2024-05-08T23:22:24.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.103+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb518257f0 con 0x55cb5185ad80 2024-05-08T23:22:24.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb51853750 0x55cb5185b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb51853750 0x55cb5185b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57378/0 (socket says 172.21.15.5:57378) 2024-05-08T23:22:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 -- 172.21.15.5:0/2017205247 learned_addr learned my addr 172.21.15.5:0/2017205247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9988aee700 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5182e320 0x55cb51853c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f99892ef700 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 0x55cb518d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 -- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 msgr2=0x55cb518d86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 0x55cb518d86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 -- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5182e320 msgr2=0x55cb51853c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5182e320 0x55cb51853c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 -- 172.21.15.5:0/2017205247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb515adb10 con 0x55cb51853750 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9988aee700 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5182e320 0x55cb51853c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f9983fff700 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb51853750 0x55cb5185b160 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f996c0011e0 tx=0x7f996c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e501b2e2910b5a03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f997affd700 1 -- 172.21.15.5:0/2017205247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0147c0 con 0x55cb51853750 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f997affd700 1 -- 172.21.15.5:0/2017205247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c0050d0 con 0x55cb51853750 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.104+0000 7f997affd700 1 -- 172.21.15.5:0/2017205247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c005440 con 0x55cb51853750 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2017205247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb51853750 msgr2=0x55cb5185b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb51853750 0x55cb5185b160 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f996c0011e0 tx=0x7f996c013000 comp rx=0 tx=0).stop 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2017205247 shutdown_connections 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 0x55cb518d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb5182e320 0x55cb51853c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/2017205247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb51853750 0x55cb5185b160 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2017205247 >> 172.21.15.5:0/2017205247 conn(0x55cb5178e1c0 msgr2=0x55cb5182d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:24.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2017205247 shutdown_connections 2024-05-08T23:22:24.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 -- 172.21.15.5:0/2017205247 wait complete. 2024-05-08T23:22:24.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.105+0000 7f9996f0cb80 1 -- start start 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.106+0000 7f9996f0cb80 1 Processor -- start 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.106+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51856f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.106+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb51857480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.106+0000 7f9996f0cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 0x55cb5185dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.106+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb5182a330 con 0x55cb5182e320 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.106+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb518704c0 con 0x55cb51853750 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.106+0000 7f9996f0cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb518973c0 con 0x55cb5185ad80 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9996f0cb80 10 client.0 _ll_register_callbacks cb 0x55cb51725ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51856f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51856f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57388/0 (socket says 172.21.15.5:57388) 2024-05-08T23:22:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 -- 172.21.15.5:0/3027629992 learned_addr learned my addr 172.21.15.5:0/3027629992 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f99892ef700 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 0x55cb5185dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9983fff700 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb51857480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 -- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 msgr2=0x55cb5185dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 0x55cb5185dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 -- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 msgr2=0x55cb51857480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb51857480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 -- 172.21.15.5:0/3027629992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99840cd7c0 con 0x55cb5182e320 2024-05-08T23:22:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9988aee700 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51856f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f99740571e0 tx=0x7f9974001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9d05198affe2d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f99637fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f99637fe700 1 -- 172.21.15.5:0/3027629992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400e040 con 0x55cb5182e320 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f99637fe700 1 -- 172.21.15.5:0/3027629992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9974057860 con 0x55cb5182e320 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9996f0cb80 1 -- 172.21.15.5:0/3027629992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f997405b690 con 0x55cb5182e320 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f99637fe700 1 -- 172.21.15.5:0/3027629992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99740012f0 con 0x55cb5182e320 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9996f0cb80 10 client.5412 fetch_fsmap learned FSMap version 20 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.107+0000 7f9996f0cb80 1 -- 172.21.15.5:0/3027629992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55cb51746510 con 0x55cb5182e320 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f99637fe700 1 -- 172.21.15.5:0/3027629992 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997405b690 con 0x55cb5182e320 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f99637fe700 1 -- 172.21.15.5:0/3027629992 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 20) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f99740760c0 con 0x55cb5182e320 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f9996f0cb80 10 client.5412 fetch_fsmap finished waiting for FSMap version 20 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f9996f0cb80 10 client.5412 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f9996f0cb80 20 client.5412 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55cb518e1830 0x55cb5185e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f9996f0cb80 4 client.5412 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f9996f0cb80 1 -- 172.21.15.5:0/3027629992 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cb51778e70 con 0x55cb518e1830 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f9983fff700 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55cb518e1830 0x55cb5185e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f99637fe700 10 client.5412 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.108+0000 7f9983fff700 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55cb518e1830 0x55cb5185e180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f41f7df4aeb7402 server_cookie=f7c8e9d06fbd1812 in_seq=0 out_seq=0 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f99637fe700 1 -- 172.21.15.5:0/3027629992 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55cb51778e70 con 0x55cb518e1830 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f99637fe700 10 client.5412 handle_command_reply: tid=0 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 client.5412 shutdown 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 -- 172.21.15.5:0/3027629992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 msgr2=0x55cb51856f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51856f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f99740571e0 tx=0x7f9974001150 comp rx=0 tx=0).stop 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 -- 172.21.15.5:0/3027629992 shutdown_connections 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55cb518e1830 0x55cb5185e180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb5185ad80 0x55cb5185dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb5182e320 0x55cb51856f40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f99637fe700 0 client.5412 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 --2- 172.21.15.5:0/3027629992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb51853750 0x55cb51857480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 -- 172.21.15.5:0/3027629992 >> 172.21.15.5:0/3027629992 conn(0x55cb5178e1c0 msgr2=0x55cb5182b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 -- 172.21.15.5:0/3027629992 shutdown_connections 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 1 -- 172.21.15.5:0/3027629992 wait complete. 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.109+0000 7f9996f0cb80 20 client.5412 trim_cache size 0 max 16384 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4975, 2024-05-08T23:22:24.117 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:22:24.118 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:22:24.118 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:22:24.118 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 16.341143911, 2024-05-08T23:22:24.118 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 20, 2024-05-08T23:22:24.118 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 34, 2024-05-08T23:22:24.118 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 34, 2024-05-08T23:22:24.118 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 24.658591863000002 2024-05-08T23:22:24.118 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 -- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f996406ac90 msgr2=0x7f996406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f996406ac90 0x7f996406d150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f996c0010e0 tx=0x7f996c009040 comp rx=0 tx=0).stop 2024-05-08T23:22:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 -- 172.21.15.5:0/3381416036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f998406a960 msgr2=0x7f99840c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f998406a960 0x7f99840c6b10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f997400c700 tx=0x7f997400cac0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 -- 172.21.15.5:0/3381416036 shutdown_connections 2024-05-08T23:22:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f996406ac90 0x7f996406d150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99840cc060 0x7f99840c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f998406a960 0x7f99840c6b10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 --2- 172.21.15.5:0/3381416036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f998406adc0 0x7f998414f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 -- 172.21.15.5:0/3381416036 >> 172.21.15.5:0/3381416036 conn(0x7f9984099a50 msgr2=0x7f998406f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 -- 172.21.15.5:0/3381416036 shutdown_connections 2024-05-08T23:22:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.110+0000 7f998ad52700 1 -- 172.21.15.5:0/3381416036 wait complete. 2024-05-08T23:22:24.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:22:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.259+0000 7f4dc72c0700 1 Processor -- start 2024-05-08T23:22:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.259+0000 7f4dc72c0700 1 -- start start 2024-05-08T23:22:24.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.260+0000 7f4dc72c0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4dc006cae0 0x7f4dc00d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.260+0000 7f4dc72c0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc00cf990 con 0x7f4dc015da10 2024-05-08T23:22:24.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.260+0000 7f4dc72c0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc00d04b0 con 0x7f4dc006bd80 2024-05-08T23:22:24.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.260+0000 7f4dc72c0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc009ab40 con 0x7f4dc006cae0 2024-05-08T23:22:24.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.260+0000 7f4dc505c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4dc015da10 0x7f4dc009a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37602/0 (socket says 172.21.15.5:37602) 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.260+0000 7f4dc505c700 1 -- 172.21.15.5:0/3768621363 learned_addr learned my addr 172.21.15.5:0/3768621363 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4dc585d700 1 --2- 172.21.15.5:0/3768621363 >> v2:172.21.15.49:3300/0 conn(0x7f4dc006cae0 0x7f4dc00d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4db7fff700 1 -- 172.21.15.5:0/3768621363 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943513759 0 0) 0x7f4dc00cf990 con 0x7f4dc015da10 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4db7fff700 1 -- 172.21.15.5:0/3768621363 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dac0035b0 con 0x7f4dc015da10 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4db7fff700 1 -- 172.21.15.5:0/3768621363 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2919060396 0 0) 0x7f4dac0035b0 con 0x7f4dc015da10 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4dc585d700 1 -- 172.21.15.5:0/3768621363 >> v1:172.21.15.49:6789/0 conn(0x7f4dc006bd80 legacy=0x7f4dc006c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4dc585d700 1 -- 172.21.15.5:0/3768621363 >> v1:172.21.15.5:6789/0 conn(0x7f4dc015da10 legacy=0x7f4dc009a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4dc585d700 1 -- 172.21.15.5:0/3768621363 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc009ad60 con 0x7f4dc006cae0 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4dc585d700 1 --2- 172.21.15.5:0/3768621363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006cae0 0x7f4dc00d0940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009d90 tx=0x7f4db800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a080e8025ca4f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4db7fff700 1 -- 172.21.15.5:0/3768621363 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db800e070 con 0x7f4dc006cae0 2024-05-08T23:22:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4db7fff700 1 -- 172.21.15.5:0/3768621363 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8003a20 con 0x7f4dc006cae0 2024-05-08T23:22:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.261+0000 7f4db7fff700 1 -- 172.21.15.5:0/3768621363 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc001670 con 0x7f4dc006cae0 2024-05-08T23:22:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.262+0000 7f4dc72c0700 1 -- 172.21.15.5:0/3768621363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006cae0 msgr2=0x7f4dc00d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.262+0000 7f4dc72c0700 1 --2- 172.21.15.5:0/3768621363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006cae0 0x7f4dc00d0940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009d90 tx=0x7f4db800b330 comp rx=0 tx=0).stop 2024-05-08T23:22:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.262+0000 7f4dc72c0700 1 -- 172.21.15.5:0/3768621363 shutdown_connections 2024-05-08T23:22:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.262+0000 7f4dc72c0700 1 --2- 172.21.15.5:0/3768621363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006cae0 0x7f4dc00d0940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.262+0000 7f4dc72c0700 1 -- 172.21.15.5:0/3768621363 >> 172.21.15.5:0/3768621363 conn(0x7f4dc00b3010 msgr2=0x7f4dc00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.262+0000 7f4dc72c0700 1 -- 172.21.15.5:0/3768621363 shutdown_connections 2024-05-08T23:22:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.262+0000 7f4dc72c0700 1 -- 172.21.15.5:0/3768621363 wait complete. 2024-05-08T23:22:24.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.263+0000 7f4dc72c0700 1 Processor -- start 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.263+0000 7f4dc72c0700 1 -- start start 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc72c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006bd80 0x7f4dc014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc72c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc006cae0 0x7f4dc01f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc72c0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4dc015da10 0x7f4dc01f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc72c0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc00b6700 con 0x7f4dc006cae0 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc72c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc00b68e0 con 0x7f4dc006bd80 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc72c0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc00b7380 con 0x7f4dc015da10 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc485b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc006cae0 0x7f4dc01f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc485b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc006cae0 0x7f4dc01f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57402/0 (socket says 172.21.15.5:57402) 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc485b700 1 -- 172.21.15.5:0/2428836475 learned_addr learned my addr 172.21.15.5:0/2428836475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc505c700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006bd80 0x7f4dc014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.264+0000 7f4dc585d700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4dc015da10 0x7f4dc01f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4dc485b700 1 -- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4dc015da10 msgr2=0x7f4dc01f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4dc485b700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4dc015da10 0x7f4dc01f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4dc485b700 1 -- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006bd80 msgr2=0x7f4dc014c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4dc485b700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006bd80 0x7f4dc014c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4dc485b700 1 -- 172.21.15.5:0/2428836475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8009a70 con 0x7f4dc006cae0 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4dc485b700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc006cae0 0x7f4dc01f23d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4db000c4b0 tx=0x7f4db000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4db5ffb700 1 -- 172.21.15.5:0/2428836475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db00052d0 con 0x7f4dc006cae0 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4db5ffb700 1 -- 172.21.15.5:0/2428836475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db0005430 con 0x7f4dc006cae0 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc00b44b0 con 0x7f4dc006cae0 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4db5ffb700 1 -- 172.21.15.5:0/2428836475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db00038c0 con 0x7f4dc006cae0 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.265+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc00b6e60 con 0x7f4dc006cae0 2024-05-08T23:22:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.266+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc0067960 con 0x7f4dc006cae0 2024-05-08T23:22:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.267+0000 7f4db5ffb700 1 -- 172.21.15.5:0/2428836475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4db0003a20 con 0x7f4dc006cae0 2024-05-08T23:22:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.268+0000 7f4db5ffb700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4da406ac90 0x7f4da406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.268+0000 7f4db5ffb700 1 -- 172.21.15.5:0/2428836475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4db0088b30 con 0x7f4dc006cae0 2024-05-08T23:22:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.268+0000 7f4dc505c700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4da406ac90 0x7f4da406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.268+0000 7f4dc505c700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4da406ac90 0x7f4da406d150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4dc01f67a0 tx=0x7f4dbc001da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.272+0000 7f4db5ffb700 1 -- 172.21.15.5:0/2428836475 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4db0053d00 con 0x7f4dc006cae0 2024-05-08T23:22:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.544+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dc0068000 con 0x7f4dc006cae0 2024-05-08T23:22:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.545+0000 7f4db5ffb700 1 -- 172.21.15.5:0/2428836475 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f4db0017090 con 0x7f4dc006cae0 2024-05-08T23:22:24.545 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:22:24.545 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:22.820284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4975,"mds_1":4745,"mds_2":4742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4742":{"gid":4742,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4745":{"gid":4745,"name":"c","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4da406ac90 msgr2=0x7f4da406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4da406ac90 0x7f4da406d150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4dc01f67a0 tx=0x7f4dbc001da0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc006cae0 msgr2=0x7f4dc01f23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc006cae0 0x7f4dc01f23d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4db000c4b0 tx=0x7f4db000c7c0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 shutdown_connections 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4da406ac90 0x7f4da406d150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4dc015da10 0x7f4dc01f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dc006bd80 0x7f4dc014c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 --2- 172.21.15.5:0/2428836475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc006cae0 0x7f4dc01f23d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.547+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 >> 172.21.15.5:0/2428836475 conn(0x7f4dc00b3010 msgr2=0x7f4dc00d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:24.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.548+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 shutdown_connections 2024-05-08T23:22:24.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:24.548+0000 7f4dc72c0700 1 -- 172.21.15.5:0/2428836475 wait complete. 2024-05-08T23:22:24.549 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 20 2024-05-08T23:22:24.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:24.562 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:22:24.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:24.592 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:22:24.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:24.661 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-05-08T23:22:24.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:24.738 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-05-08T23:22:24.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:24.801 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-05-08T23:22:24.864 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:22:25.021 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:22:25.022 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:25.022 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:22:27.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:22:27.181 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:22:27.182 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:27.182 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:22:29.184 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:22:29.343 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:22:29.343 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:29.344 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:22:31.345 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:22:31.501 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:22:31.502 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:31.502 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:22:33.503 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:22:33.661 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:22:33.661 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:33.662 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:22:35.662 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:22:35.822 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2024-05-08T23:22:35.823 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:35.823 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:22:35.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:22:35.988 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:22:35.988 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:35.988 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:22:37.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:22:37.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:22:37.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:22:38.154 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:22:38.154 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:38.154 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:22:40.156 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:22:40.319 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-08T23:22:40.365 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:22:40.365 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:22:40.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:22:40.535 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:22:40.536 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:22:40.536 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:22:40.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 flush journal 2024-05-08T23:22:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.676+0000 7f8d7320e700 1 Processor -- start 2024-05-08T23:22:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.677+0000 7f8d7320e700 1 -- start start 2024-05-08T23:22:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.677+0000 7f8d7320e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8d6c15dc60 0x7f8d6c0ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.677+0000 7f8d7320e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8d6c0ae620 0x7f8d6c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d7320e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6c0aed90 con 0x7f8d6c072170 2024-05-08T23:22:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d7320e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6c06dcd0 con 0x7f8d6c0ae620 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d7320e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6c06e290 con 0x7f8d6c15dc60 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d717ab700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8d6c0ae620 0x7f8d6c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d717ab700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8d6c0ae620 0x7f8d6c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57422/0 (socket says 172.21.15.5:57422) 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d717ab700 1 -- 172.21.15.5:0/1992500638 learned_addr learned my addr 172.21.15.5:0/1992500638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d70faa700 1 --2- 172.21.15.5:0/1992500638 >> v2:172.21.15.49:3300/0 conn(0x7f8d6c15dc60 0x7f8d6c0ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d717ab700 1 -- 172.21.15.5:0/1992500638 >> v2:172.21.15.49:3300/0 conn(0x7f8d6c15dc60 msgr2=0x7f8d6c0ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d717ab700 1 --2- 172.21.15.5:0/1992500638 >> v2:172.21.15.49:3300/0 conn(0x7f8d6c15dc60 0x7f8d6c0ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d717ab700 1 -- 172.21.15.5:0/1992500638 >> v1:172.21.15.49:6789/0 conn(0x7f8d6c072170 legacy=0x7f8d6c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d717ab700 1 -- 172.21.15.5:0/1992500638 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c0b2aa0 con 0x7f8d6c0ae620 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d717ab700 1 --2- 172.21.15.5:0/1992500638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c0ae620 0x7f8d6c06e720 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c06db20 tx=0x7f8d6400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1622cb5e369aec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d6b7fe700 1 -- 172.21.15.5:0/1992500638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64010070 con 0x7f8d6c0ae620 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d6b7fe700 1 -- 172.21.15.5:0/1992500638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400dce0 con 0x7f8d6c0ae620 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.678+0000 7f8d6b7fe700 1 -- 172.21.15.5:0/1992500638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400b310 con 0x7f8d6c0ae620 2024-05-08T23:22:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.679+0000 7f8d7320e700 1 -- 172.21.15.5:0/1992500638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c0ae620 msgr2=0x7f8d6c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.679+0000 7f8d7320e700 1 --2- 172.21.15.5:0/1992500638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c0ae620 0x7f8d6c06e720 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c06db20 tx=0x7f8d6400d430 comp rx=0 tx=0).stop 2024-05-08T23:22:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.679+0000 7f8d7320e700 1 -- 172.21.15.5:0/1992500638 shutdown_connections 2024-05-08T23:22:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.679+0000 7f8d7320e700 1 --2- 172.21.15.5:0/1992500638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c0ae620 0x7f8d6c06e720 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.679+0000 7f8d7320e700 1 --2- 172.21.15.5:0/1992500638 >> v2:172.21.15.49:3300/0 conn(0x7f8d6c15dc60 0x7f8d6c0ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.679+0000 7f8d7320e700 1 -- 172.21.15.5:0/1992500638 >> 172.21.15.5:0/1992500638 conn(0x7f8d6c0b5a30 msgr2=0x7f8d6c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.679+0000 7f8d7320e700 1 -- 172.21.15.5:0/1992500638 shutdown_connections 2024-05-08T23:22:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.679+0000 7f8d7320e700 1 -- 172.21.15.5:0/1992500638 wait complete. 2024-05-08T23:22:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.681+0000 7f8d7320e700 1 Processor -- start 2024-05-08T23:22:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.681+0000 7f8d7320e700 1 -- start start 2024-05-08T23:22:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d7320e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8d6c072170 0x7f8d6c0b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d7320e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d6c0ae620 0x7f8d6c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d7320e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c15dc60 0x7f8d6c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d7320e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c154e00 con 0x7f8d6c15dc60 2024-05-08T23:22:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d7320e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c150080 con 0x7f8d6c0ae620 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d7320e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c1f32d0 con 0x7f8d6c072170 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d717ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c15dc60 0x7f8d6c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d717ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c15dc60 0x7f8d6c156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57426/0 (socket says 172.21.15.5:57426) 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d717ab700 1 -- 172.21.15.5:0/2230352091 learned_addr learned my addr 172.21.15.5:0/2230352091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d6bfff700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d6c0ae620 0x7f8d6c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.682+0000 7f8d70faa700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8d6c072170 0x7f8d6c0b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d717ab700 1 -- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8d6c072170 msgr2=0x7f8d6c0b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d717ab700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8d6c072170 0x7f8d6c0b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d717ab700 1 -- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d6c0ae620 msgr2=0x7f8d6c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d717ab700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d6c0ae620 0x7f8d6c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d717ab700 1 -- 172.21.15.5:0/2230352091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6400bd40 con 0x7f8d6c15dc60 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d717ab700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c15dc60 0x7f8d6c156150 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8d64014000 tx=0x7f8d64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d697fa700 1 -- 172.21.15.5:0/2230352091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400b4c0 con 0x7f8d6c15dc60 2024-05-08T23:22:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d697fa700 1 -- 172.21.15.5:0/2230352091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400adb0 con 0x7f8d6c15dc60 2024-05-08T23:22:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d7320e700 1 -- 172.21.15.5:0/2230352091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6c156800 con 0x7f8d6c15dc60 2024-05-08T23:22:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d697fa700 1 -- 172.21.15.5:0/2230352091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004280 con 0x7f8d6c15dc60 2024-05-08T23:22:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.683+0000 7f8d7320e700 1 -- 172.21.15.5:0/2230352091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c150570 con 0x7f8d6c15dc60 2024-05-08T23:22:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.685+0000 7f8d697fa700 1 -- 172.21.15.5:0/2230352091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8d640043e0 con 0x7f8d6c15dc60 2024-05-08T23:22:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.685+0000 7f8d697fa700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8d4c06ac90 0x7f8d4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.685+0000 7f8d697fa700 1 -- 172.21.15.5:0/2230352091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d6408cc00 con 0x7f8d6c15dc60 2024-05-08T23:22:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.685+0000 7f8d70faa700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8d4c06ac90 0x7f8d4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.686+0000 7f8d70faa700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8d4c06ac90 0x7f8d4c06d150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009a10 tx=0x7f8d5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.687+0000 7f8d7f3c8b80 1 Processor -- start 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d7f3c8b80 1 -- start start 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c8d7a0 0x561214c66110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214c6b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c94dd0 0x561214c95250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561214bc1150 con 0x561214c94dd0 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561214bbeb10 con 0x561214c68370 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561214bba400 con 0x561214c8d7a0 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d717ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c94dd0 0x561214c95250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d717ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c94dd0 0x561214c95250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57438/0 (socket says 172.21.15.5:57438) 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.688+0000 7f8d717ab700 1 -- 172.21.15.5:0/3053893467 learned_addr learned my addr 172.21.15.5:0/3053893467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d6bfff700 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214c6b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d70faa700 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c8d7a0 0x561214c66110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d717ab700 1 -- 172.21.15.5:0/3053893467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c8d7a0 msgr2=0x561214c66110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d717ab700 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c8d7a0 0x561214c66110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d717ab700 1 -- 172.21.15.5:0/3053893467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 msgr2=0x561214c6b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d717ab700 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214c6b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d717ab700 1 -- 172.21.15.5:0/3053893467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561214a3b3b0 con 0x561214c94dd0 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d717ab700 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c94dd0 0x561214c95250 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d800 tx=0x7f8d64057f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b13fb960d91699 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d627fc700 1 -- 172.21.15.5:0/3053893467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6401be30 con 0x561214c94dd0 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d627fc700 1 -- 172.21.15.5:0/3053893467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6405c230 con 0x561214c94dd0 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.689+0000 7f8d627fc700 1 -- 172.21.15.5:0/3053893467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6405bb30 con 0x561214c94dd0 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3053893467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c94dd0 msgr2=0x561214c95250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c94dd0 0x561214c95250 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d800 tx=0x7f8d64057f70 comp rx=0 tx=0).stop 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3053893467 shutdown_connections 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c94dd0 0x561214c95250 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c8d7a0 0x561214c66110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3053893467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214c6b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3053893467 >> 172.21.15.5:0/3053893467 conn(0x561214bc8210 msgr2=0x561214cafdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3053893467 shutdown_connections 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3053893467 wait complete. 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.690+0000 7f8d7f3c8b80 1 -- start start 2024-05-08T23:22:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.691+0000 7f8d7f3c8b80 1 Processor -- start 2024-05-08T23:22:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214cbf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c98310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 0x561214c98850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561214b56560 con 0x561214c8d7a0 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561214b56200 con 0x561214c68370 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561214a3d6a0 con 0x561214c94dd0 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d7f3c8b80 10 client.0 _ll_register_callbacks cb 0x561214b5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d6bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c98310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d6bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c98310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57448/0 (socket says 172.21.15.5:57448) 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d6bfff700 1 -- 172.21.15.5:0/1225421662 learned_addr learned my addr 172.21.15.5:0/1225421662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d70faa700 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214cbf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.692+0000 7f8d717ab700 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 0x561214c98850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d6bfff700 1 -- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 msgr2=0x561214c98850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d6bfff700 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 0x561214c98850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d6bfff700 1 -- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 msgr2=0x561214cbf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d6bfff700 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214cbf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d6bfff700 1 -- 172.21.15.5:0/1225421662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c150570 con 0x561214c8d7a0 2024-05-08T23:22:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d70faa700 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214cbf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d6bfff700 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c98310 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8d54003b90 tx=0x7f8d54003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01cc28a7fe0d38e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/1225421662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54001980 con 0x561214c8d7a0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/1225421662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d54001ae0 con 0x561214c8d7a0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/1225421662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561214b13270 con 0x561214c8d7a0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/1225421662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d54001cc0 con 0x561214c8d7a0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d7f3c8b80 10 client.5430 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/1225421662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5612149e7b60 con 0x561214c8d7a0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.693+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/1225421662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561214b13270 con 0x561214c8d7a0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/1225421662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f8d540143c0 con 0x561214c8d7a0 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d7f3c8b80 10 client.5430 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d7f3c8b80 10 client.5430 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-05-08T23:22:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d7f3c8b80 20 client.5430 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x561214cb3670 0x561214c96eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d7f3c8b80 4 client.5430 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/1225421662 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561214c397e0 con 0x561214cb3670 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d70faa700 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x561214cb3670 0x561214c96eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d70faa700 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x561214cb3670 0x561214c96eb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f53dec5d94015cc server_cookie=299e3d9f0146daf3 in_seq=0 out_seq=0 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.694+0000 7f8d4b7fe700 10 client.5430 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/1225421662 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561214c397e0 con 0x561214cb3670 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d4b7fe700 10 client.5430 handle_command_reply: tid=0 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 client.5430 shutdown 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/1225421662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 msgr2=0x561214c98310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c98310 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8d54003b90 tx=0x7f8d54003ea0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/1225421662 shutdown_connections 2024-05-08T23:22:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x561214cb3670 0x561214c96eb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 0x561214c98850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c68370 0x561214cbf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d4b7fe700 0 client.5430 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/1225421662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c98310 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/1225421662 >> 172.21.15.5:0/1225421662 conn(0x561214bc8210 msgr2=0x561214c92110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/1225421662 shutdown_connections 2024-05-08T23:22:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/1225421662 wait complete. 2024-05-08T23:22:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.695+0000 7f8d7f3c8b80 20 client.5430 trim_cache size 0 max 16384 2024-05-08T23:22:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.711+0000 7f8d7f3c8b80 1 Processor -- start 2024-05-08T23:22:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.711+0000 7f8d7f3c8b80 1 -- start start 2024-05-08T23:22:40.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c68370 0x561214cb0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c8d7a0 0x561214cb08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 0x561214caf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561214c91f50 con 0x561214c68370 2024-05-08T23:22:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561214cc1bf0 con 0x561214c8d7a0 2024-05-08T23:22:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561214cd19d0 con 0x561214c94dd0 2024-05-08T23:22:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d70faa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c68370 0x561214cb0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d70faa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c68370 0x561214cb0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57450/0 (socket says 172.21.15.5:57450) 2024-05-08T23:22:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d70faa700 1 -- 172.21.15.5:0/491929369 learned_addr learned my addr 172.21.15.5:0/491929369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d70faa700 1 -- 172.21.15.5:0/491929369 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 msgr2=0x561214caf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.712+0000 7f8d70faa700 1 --2- 172.21.15.5:0/491929369 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 0x561214caf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d70faa700 1 -- 172.21.15.5:0/491929369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c8d7a0 msgr2=0x561214cb08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d70faa700 1 --2- 172.21.15.5:0/491929369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c8d7a0 0x561214cb08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d70faa700 1 -- 172.21.15.5:0/491929369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561214b80560 con 0x561214c68370 2024-05-08T23:22:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d70faa700 1 --2- 172.21.15.5:0/491929369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c68370 0x561214cb0390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0104d0 tx=0x7f8d5c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44987257d69520c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d62ffd700 1 -- 172.21.15.5:0/491929369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c015a00 con 0x561214c68370 2024-05-08T23:22:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d62ffd700 1 -- 172.21.15.5:0/491929369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c015b60 con 0x561214c68370 2024-05-08T23:22:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d62ffd700 1 -- 172.21.15.5:0/491929369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0260e0 con 0x561214c68370 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/491929369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c68370 msgr2=0x561214cb0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/491929369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c68370 0x561214cb0390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0104d0 tx=0x7f8d5c013000 comp rx=0 tx=0).stop 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/491929369 shutdown_connections 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/491929369 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c94dd0 0x561214caf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/491929369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c68370 0x561214cb0390 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/491929369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c8d7a0 0x561214cb08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.713+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/491929369 >> 172.21.15.5:0/491929369 conn(0x561214bc8210 msgr2=0x561214cb2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.714+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/491929369 shutdown_connections 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.714+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/491929369 wait complete. 2024-05-08T23:22:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.714+0000 7f8d7f3c8b80 1 -- start start 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.714+0000 7f8d7f3c8b80 1 Processor -- start 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c68370 0x561214c68750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c68c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d7f3c8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c94dd0 0x561214d18e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561214cc0620 con 0x561214c8d7a0 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561214cc0a80 con 0x561214c94dd0 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d7f3c8b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561214c6bb30 con 0x561214c68370 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d7f3c8b80 10 client.0 _ll_register_callbacks cb 0x561214b5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d6bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c68c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d6bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c68c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57456/0 (socket says 172.21.15.5:57456) 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.715+0000 7f8d6bfff700 1 -- 172.21.15.5:0/3398216308 learned_addr learned my addr 172.21.15.5:0/3398216308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d70faa700 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c68370 0x561214c68750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d717ab700 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c94dd0 0x561214d18e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d6bfff700 1 -- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c68370 msgr2=0x561214c68750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d6bfff700 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c68370 0x561214c68750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d6bfff700 1 -- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c94dd0 msgr2=0x561214d18e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d6bfff700 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c94dd0 0x561214d18e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d6bfff700 1 -- 172.21.15.5:0/3398216308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c00e050 con 0x561214c8d7a0 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d70faa700 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c68370 0x561214c68750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d6bfff700 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c68c90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8d54008fd0 tx=0x7f8d54000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5498eb7534b28dd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/3398216308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54014920 con 0x561214c8d7a0 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/3398216308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d54003870 con 0x561214c8d7a0 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3398216308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d6405c230 con 0x561214c8d7a0 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/3398216308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d54003a30 con 0x561214c8d7a0 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d7f3c8b80 10 client.5436 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3398216308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x561214a3fcd0 con 0x561214c8d7a0 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.716+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/3398216308 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6405c230 con 0x561214c8d7a0 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/3398216308 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f8d54014b60 con 0x561214c8d7a0 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d7f3c8b80 10 client.5436 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d7f3c8b80 10 client.5436 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d7f3c8b80 20 client.5436 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x561214c951f0 0x561214d19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d7f3c8b80 4 client.5436 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3398216308 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561214bb2ec0 con 0x561214c951f0 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d70faa700 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x561214c951f0 0x561214d19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d4b7fe700 10 client.5436 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.717+0000 7f8d70faa700 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x561214c951f0 0x561214d19340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53dc5432fcc7cc5c server_cookie=455c234532afceb0 in_seq=0 out_seq=0 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d4b7fe700 1 -- 172.21.15.5:0/3398216308 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561214bb2ec0 con 0x561214c951f0 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d4b7fe700 10 client.5436 handle_command_reply: tid=0 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 client.5436 shutdown 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3398216308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 msgr2=0x561214c68c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c68c90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8d54008fd0 tx=0x7f8d54000ed0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3398216308 shutdown_connections 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x561214c951f0 0x561214d19340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561214c94dd0 0x561214d18e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561214c68370 0x561214c68750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d4b7fe700 0 client.5436 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 --2- 172.21.15.5:0/3398216308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561214c8d7a0 0x561214c68c90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3398216308 >> 172.21.15.5:0/3398216308 conn(0x561214bc8210 msgr2=0x561214caf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3398216308 shutdown_connections 2024-05-08T23:22:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.730+0000 7f8d7f3c8b80 1 -- 172.21.15.5:0/3398216308 wait complete. 2024-05-08T23:22:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7f3c8b80 20 client.5436 trim_cache size 0 max 16384 2024-05-08T23:22:40.731 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:40.731 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:22:40.731 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:22:40.731 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 -- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8d4c06ac90 msgr2=0x7f8d4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8d4c06ac90 0x7f8d4c06d150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009a10 tx=0x7f8d5c008040 comp rx=0 tx=0).stop 2024-05-08T23:22:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 -- 172.21.15.5:0/2230352091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c15dc60 msgr2=0x7f8d6c156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c15dc60 0x7f8d6c156150 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8d64014000 tx=0x7f8d64004ad0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 -- 172.21.15.5:0/2230352091 shutdown_connections 2024-05-08T23:22:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8d4c06ac90 0x7f8d4c06d150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d6c15dc60 0x7f8d6c156150 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8d6c072170 0x7f8d6c0b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 --2- 172.21.15.5:0/2230352091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d6c0ae620 0x7f8d6c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 -- 172.21.15.5:0/2230352091 >> 172.21.15.5:0/2230352091 conn(0x7f8d6c0b5a30 msgr2=0x7f8d6c0ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 -- 172.21.15.5:0/2230352091 shutdown_connections 2024-05-08T23:22:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.731+0000 7f8d7320e700 1 -- 172.21.15.5:0/2230352091 wait complete. 2024-05-08T23:22:40.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:1 flush journal 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.885+0000 7f15d3017700 1 Processor -- start 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.885+0000 7f15d3017700 1 -- start start 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d3017700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f15cc154df0 0x7f15cc0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d3017700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f15cc0d0890 0x7f15cc0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d3017700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc15add0 con 0x7f15cc0b43b0 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d3017700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc15abe0 con 0x7f15cc0d0890 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d3017700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc0d0d20 con 0x7f15cc154df0 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15cbfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f15cc0d0890 0x7f15cc0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d15b4700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f15cc0b43b0 0x7f15cc0ada60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43506/0 (socket says 172.21.15.5:43506) 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d15b4700 1 -- 172.21.15.5:0/1477080957 learned_addr learned my addr 172.21.15.5:0/1477080957 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1477080957 >> v2:172.21.15.49:3300/0 conn(0x7f15cc154df0 0x7f15cc0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15cbfff700 1 -- 172.21.15.5:0/1477080957 >> v2:172.21.15.49:3300/0 conn(0x7f15cc154df0 msgr2=0x7f15cc0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1477080957 >> v2:172.21.15.49:3300/0 conn(0x7f15cc154df0 0x7f15cc0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15cbfff700 1 -- 172.21.15.5:0/1477080957 >> v1:172.21.15.5:6789/0 conn(0x7f15cc0b43b0 legacy=0x7f15cc0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15cbfff700 1 -- 172.21.15.5:0/1477080957 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc15b070 con 0x7f15cc0d0890 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15cb7fe700 1 -- 172.21.15.5:0/1477080957 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250107972 0 0) 0x7f15cc15add0 con 0x7f15cc0b43b0 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1477080957 >> v2:172.21.15.49:3300/0 conn(0x7f15cc154df0 0x7f15cc0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1477080957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc0d0890 0x7f15cc0b3e70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0675f0 tx=0x7f15b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a19206046d6d4f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.886+0000 7f15cb7fe700 1 -- 172.21.15.5:0/1477080957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4011830 con 0x7f15cc0d0890 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15cb7fe700 1 -- 172.21.15.5:0/1477080957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4011990 con 0x7f15cc0d0890 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15cb7fe700 1 -- 172.21.15.5:0/1477080957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4011d20 con 0x7f15cc0d0890 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15d3017700 1 -- 172.21.15.5:0/1477080957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc0d0890 msgr2=0x7f15cc0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15d3017700 1 --2- 172.21.15.5:0/1477080957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc0d0890 0x7f15cc0b3e70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0675f0 tx=0x7f15b4009b20 comp rx=0 tx=0).stop 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15d3017700 1 -- 172.21.15.5:0/1477080957 shutdown_connections 2024-05-08T23:22:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15d3017700 1 --2- 172.21.15.5:0/1477080957 >> v2:172.21.15.49:3300/0 conn(0x7f15cc154df0 0x7f15cc0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15d3017700 1 --2- 172.21.15.5:0/1477080957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc0d0890 0x7f15cc0b3e70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15d3017700 1 -- 172.21.15.5:0/1477080957 >> 172.21.15.5:0/1477080957 conn(0x7f15cc099a50 msgr2=0x7f15cc0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15d3017700 1 -- 172.21.15.5:0/1477080957 shutdown_connections 2024-05-08T23:22:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.887+0000 7f15d3017700 1 -- 172.21.15.5:0/1477080957 wait complete. 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.888+0000 7f15d3017700 1 Processor -- start 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.889+0000 7f15d3017700 1 -- start start 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.889+0000 7f15d3017700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0b43b0 0x7f15cc0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.889+0000 7f15d3017700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f15cc0d0890 0x7f15cc14b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d3017700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc154df0 0x7f15cc0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d3017700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc14b550 con 0x7f15cc154df0 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d3017700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0cf590 con 0x7f15cc0b43b0 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d3017700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0cd7a0 con 0x7f15cc0d0890 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc154df0 0x7f15cc0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc154df0 0x7f15cc0cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57468/0 (socket says 172.21.15.5:57468) 2024-05-08T23:22:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d0db3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0b43b0 0x7f15cc0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 -- 172.21.15.5:0/1335058630 learned_addr learned my addr 172.21.15.5:0/1335058630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f15cc0d0890 0x7f15cc14b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 -- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f15cc0d0890 msgr2=0x7f15cc14b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f15cc0d0890 0x7f15cc14b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 -- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0b43b0 msgr2=0x7f15cc0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0b43b0 0x7f15cc0cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 -- 172.21.15.5:0/1335058630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40094a0 con 0x7f15cc154df0 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d15b4700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc154df0 0x7f15cc0cf050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f15c400cba0 tx=0x7f15c400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15c97fa700 1 -- 172.21.15.5:0/1335058630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c400aaa0 con 0x7f15cc154df0 2024-05-08T23:22:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15c97fa700 1 -- 172.21.15.5:0/1335058630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c400ac20 con 0x7f15cc154df0 2024-05-08T23:22:40.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15d3017700 1 -- 172.21.15.5:0/1335058630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc0cda80 con 0x7f15cc154df0 2024-05-08T23:22:40.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.890+0000 7f15c97fa700 1 -- 172.21.15.5:0/1335058630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c40070d0 con 0x7f15cc154df0 2024-05-08T23:22:40.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.891+0000 7f15d3017700 1 -- 172.21.15.5:0/1335058630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc0ce040 con 0x7f15cc154df0 2024-05-08T23:22:40.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.893+0000 7f15c97fa700 1 -- 172.21.15.5:0/1335058630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f15c4007230 con 0x7f15cc154df0 2024-05-08T23:22:40.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.893+0000 7f15c97fa700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f15ac06ac90 0x7f15ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.893+0000 7f15c97fa700 1 -- 172.21.15.5:0/1335058630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15c4088fb0 con 0x7f15cc154df0 2024-05-08T23:22:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.893+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f15ac06ac90 0x7f15ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.894+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f15ac06ac90 0x7f15ac06d150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006fd0 tx=0x7f15bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.895+0000 7f15df1d1b80 1 Processor -- start 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.895+0000 7f15df1d1b80 1 -- start start 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f81b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 0x5560f81e3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f810f150 con 0x5560f81b6370 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f810cb10 con 0x5560f81e2dd0 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560f8108400 con 0x5560f81db7a0 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f81b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15d0db3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f81b90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57480/0 (socket says 172.21.15.5:57480) 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15cbfff700 1 -- 172.21.15.5:0/387873361 learned_addr learned my addr 172.21.15.5:0/387873361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15d15b4700 1 --2- 172.21.15.5:0/387873361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 0x5560f81e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15cbfff700 1 -- 172.21.15.5:0/387873361 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 msgr2=0x5560f81b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15cbfff700 1 --2- 172.21.15.5:0/387873361 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15cbfff700 1 -- 172.21.15.5:0/387873361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 msgr2=0x5560f81e3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15cbfff700 1 --2- 172.21.15.5:0/387873361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 0x5560f81e3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.896+0000 7f15cbfff700 1 -- 172.21.15.5:0/387873361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560f7f893b0 con 0x5560f81b6370 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15cbfff700 1 --2- 172.21.15.5:0/387873361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f81b90a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f15b4013000 tx=0x7f15b4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac77ef8b3a73dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15c27fc700 1 -- 172.21.15.5:0/387873361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4020070 con 0x5560f81b6370 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15c27fc700 1 -- 172.21.15.5:0/387873361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4011b00 con 0x5560f81b6370 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15c27fc700 1 -- 172.21.15.5:0/387873361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4004040 con 0x5560f81b6370 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 -- 172.21.15.5:0/387873361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 msgr2=0x5560f81b90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/387873361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f81b90a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f15b4013000 tx=0x7f15b4010000 comp rx=0 tx=0).stop 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 -- 172.21.15.5:0/387873361 shutdown_connections 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/387873361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 0x5560f81e3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/387873361 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/387873361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f81b90a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 -- 172.21.15.5:0/387873361 >> 172.21.15.5:0/387873361 conn(0x5560f8116210 msgr2=0x5560f81fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 -- 172.21.15.5:0/387873361 shutdown_connections 2024-05-08T23:22:40.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.897+0000 7f15df1d1b80 1 -- 172.21.15.5:0/387873361 wait complete. 2024-05-08T23:22:40.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.898+0000 7f15df1d1b80 1 -- start start 2024-05-08T23:22:40.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.898+0000 7f15df1d1b80 1 Processor -- start 2024-05-08T23:22:40.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 0x5560f81ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81e2dd0 0x5560f81da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f80a4560 con 0x5560f81e2dd0 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f80a4200 con 0x5560f81b6370 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560f7f8b6a0 con 0x5560f81db7a0 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15df1d1b80 10 client.0 _ll_register_callbacks cb 0x5560f80adb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15d15b4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81e2dd0 0x5560f81da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15d15b4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81e2dd0 0x5560f81da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57488/0 (socket says 172.21.15.5:57488) 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15d15b4700 1 -- 172.21.15.5:0/1379671216 learned_addr learned my addr 172.21.15.5:0/1379671216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 0x5560f81ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.899+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15d15b4700 1 -- 172.21.15.5:0/1379671216 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 msgr2=0x5560f81fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15d15b4700 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15d15b4700 1 -- 172.21.15.5:0/1379671216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 msgr2=0x5560f81ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15d15b4700 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 0x5560f81ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15d15b4700 1 -- 172.21.15.5:0/1379671216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc15b070 con 0x5560f81e2dd0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15d15b4700 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81e2dd0 0x5560f81da680 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f15c400a340 tx=0x7f15c4058420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad9f2cbea3f4239 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1379671216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4060040 con 0x5560f81e2dd0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1379671216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15c405bd10 con 0x5560f81e2dd0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1379671216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560f8061270 con 0x5560f81e2dd0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1379671216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15c40602d0 con 0x5560f81e2dd0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15df1d1b80 10 client.5451 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1379671216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5560f7f35b60 con 0x5560f81e2dd0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1379671216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5560f8061270 con 0x5560f81e2dd0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1379671216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f15c406bd20 con 0x5560f81e2dd0 2024-05-08T23:22:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15df1d1b80 10 client.5451 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15df1d1b80 10 client.5451 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.c 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15df1d1b80 20 client.5451 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.900+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5560f81e67a0 0x5560f820d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.901+0000 7f15df1d1b80 4 client.5451 mds_command: new command op to 4745 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.901+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1379671216 --> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560f81877e0 con 0x5560f81e67a0 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.901+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5560f81e67a0 0x5560f820d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.901+0000 7f15ab7fe700 10 client.5451 ms_handle_connect on v2:172.21.15.5:6833/146081129 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.901+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5560f81e67a0 0x5560f820d1f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf28e4392e9e3665 server_cookie=cffa05e18f4b7d81 in_seq=0 out_seq=0 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.901+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1379671216 <== mds.1 v2:172.21.15.5:6833/146081129 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5560f81877e0 con 0x5560f81e67a0 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.901+0000 7f15ab7fe700 10 client.5451 handle_command_reply: tid=0 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 client.5451 shutdown 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1379671216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81e2dd0 msgr2=0x5560f81da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81e2dd0 0x5560f81da680 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f15c400a340 tx=0x7f15c4058420 comp rx=0 tx=0).stop 2024-05-08T23:22:40.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1379671216 shutdown_connections 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5560f81e67a0 0x5560f820d1f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81e2dd0 0x5560f81da680 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 0x5560f81ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15ab7fe700 0 client.5451 ms_handle_reset on v2:172.21.15.5:6833/146081129 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1379671216 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1379671216 >> 172.21.15.5:0/1379671216 conn(0x5560f8116210 msgr2=0x5560f81b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1379671216 shutdown_connections 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1379671216 wait complete. 2024-05-08T23:22:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.902+0000 7f15df1d1b80 20 client.5451 trim_cache size 0 max 16384 2024-05-08T23:22:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.918+0000 7f15df1d1b80 1 Processor -- start 2024-05-08T23:22:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.918+0000 7f15df1d1b80 1 -- start start 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 0x5560f81b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81db7a0 0x5560f81b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81e2dd0 0x5560f81b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f820d000 con 0x5560f81db7a0 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f82202b0 con 0x5560f81b6370 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560f81b3c40 con 0x5560f81e2dd0 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81db7a0 0x5560f81b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81db7a0 0x5560f81b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57498/0 (socket says 172.21.15.5:57498) 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 -- 172.21.15.5:0/1859772588 learned_addr learned my addr 172.21.15.5:0/1859772588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 0x5560f81b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15d15b4700 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81e2dd0 0x5560f81b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 -- 172.21.15.5:0/1859772588 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81e2dd0 msgr2=0x5560f81b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81e2dd0 0x5560f81b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 -- 172.21.15.5:0/1859772588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 msgr2=0x5560f81b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 0x5560f81b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 -- 172.21.15.5:0/1859772588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560f80ce560 con 0x5560f81db7a0 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.919+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81db7a0 0x5560f81b1f10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009b60 tx=0x7f15b4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5694c1332882ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15c2ffd700 1 -- 172.21.15.5:0/1859772588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4020040 con 0x5560f81db7a0 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15c2ffd700 1 -- 172.21.15.5:0/1859772588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4006bb0 con 0x5560f81db7a0 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15c2ffd700 1 -- 172.21.15.5:0/1859772588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4009300 con 0x5560f81db7a0 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1859772588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81db7a0 msgr2=0x5560f81b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81db7a0 0x5560f81b1f10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009b60 tx=0x7f15b4011000 comp rx=0 tx=0).stop 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1859772588 shutdown_connections 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81e2dd0 0x5560f81b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81b6370 0x5560f81b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1859772588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81db7a0 0x5560f81b1f10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1859772588 >> 172.21.15.5:0/1859772588 conn(0x5560f8116210 msgr2=0x5560f81e02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1859772588 shutdown_connections 2024-05-08T23:22:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.920+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1859772588 wait complete. 2024-05-08T23:22:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.921+0000 7f15df1d1b80 1 -- start start 2024-05-08T23:22:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.921+0000 7f15df1d1b80 1 Processor -- start 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f8260ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81e3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15df1d1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 0x5560f820d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f826b3f0 con 0x5560f81b6370 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f81b2440 con 0x5560f81e2dd0 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15df1d1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560f81d9180 con 0x5560f81db7a0 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15df1d1b80 10 client.0 _ll_register_callbacks cb 0x5560f80adb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d0db3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f8260ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d0db3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f8260ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57502/0 (socket says 172.21.15.5:57502) 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d0db3700 1 -- 172.21.15.5:0/1279831543 learned_addr learned my addr 172.21.15.5:0/1279831543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d15b4700 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 0x5560f820d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d0db3700 1 -- 172.21.15.5:0/1279831543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 msgr2=0x5560f81e3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81e3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d0db3700 1 -- 172.21.15.5:0/1279831543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 msgr2=0x5560f820d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 0x5560f820d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.922+0000 7f15d0db3700 1 -- 172.21.15.5:0/1279831543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560f7f893b0 con 0x5560f81b6370 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15d0db3700 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f8260ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00a5e0 tx=0x7f15bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e46ec964bbe876 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1279831543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc026040 con 0x5560f81b6370 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1279831543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15bc01a040 con 0x5560f81b6370 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1279831543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15b4011b00 con 0x5560f81b6370 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1279831543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15bc029e60 con 0x5560f81b6370 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 10 client.5457 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1279831543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5560f7f8dcd0 con 0x5560f81b6370 2024-05-08T23:22:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1279831543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4011b00 con 0x5560f81b6370 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1279831543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f15bc015450 con 0x5560f81b6370 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 10 client.5457 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 10 client.5457 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.c 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 20 client.5457 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5560f81d9610 0x5560f820d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 4 client.5457 mds_command: new command op to 4745 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1279831543 --> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5560f8100ec0 con 0x5560f81d9610 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.923+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5560f81d9610 0x5560f820d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.924+0000 7f15ab7fe700 10 client.5457 ms_handle_connect on v2:172.21.15.5:6833/146081129 2024-05-08T23:22:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.924+0000 7f15cbfff700 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5560f81d9610 0x5560f820d730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fe691c4d53ea661 server_cookie=8bd15319080a2b30 in_seq=0 out_seq=0 2024-05-08T23:22:40.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15ab7fe700 1 -- 172.21.15.5:0/1279831543 <== mds.1 v2:172.21.15.5:6833/146081129 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5560f8100ec0 con 0x5560f81d9610 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15ab7fe700 10 client.5457 handle_command_reply: tid=0 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 client.5457 shutdown 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1279831543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 msgr2=0x5560f8260ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f8260ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00a5e0 tx=0x7f15bc0047c0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1279831543 shutdown_connections 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.5:6833/146081129,v1:172.21.15.5:6835/146081129] conn(0x5560f81d9610 0x5560f820d730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5560f81e2dd0 0x5560f820d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560f81b6370 0x5560f8260ec0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15ab7fe700 0 client.5457 ms_handle_reset on v2:172.21.15.5:6833/146081129 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 --2- 172.21.15.5:0/1279831543 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5560f81db7a0 0x5560f81e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1279831543 >> 172.21.15.5:0/1279831543 conn(0x5560f8116210 msgr2=0x5560f81af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1279831543 shutdown_connections 2024-05-08T23:22:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 1 -- 172.21.15.5:0/1279831543 wait complete. 2024-05-08T23:22:40.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.933+0000 7f15df1d1b80 20 client.5457 trim_cache size 0 max 16384 2024-05-08T23:22:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:40.934 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:22:40.934 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:22:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 -- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f15ac06ac90 msgr2=0x7f15ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f15ac06ac90 0x7f15ac06d150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006fd0 tx=0x7f15bc008040 comp rx=0 tx=0).stop 2024-05-08T23:22:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 -- 172.21.15.5:0/1335058630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc154df0 msgr2=0x7f15cc0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc154df0 0x7f15cc0cf050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f15c400cba0 tx=0x7f15c400ceb0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 -- 172.21.15.5:0/1335058630 shutdown_connections 2024-05-08T23:22:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f15ac06ac90 0x7f15ac06d150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15cc154df0 0x7f15cc0cf050 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0b43b0 0x7f15cc0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 --2- 172.21.15.5:0/1335058630 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f15cc0d0890 0x7f15cc14b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 -- 172.21.15.5:0/1335058630 >> 172.21.15.5:0/1335058630 conn(0x7f15cc099a50 msgr2=0x7f15cc14b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 -- 172.21.15.5:0/1335058630 shutdown_connections 2024-05-08T23:22:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:40.934+0000 7f15d3017700 1 -- 172.21.15.5:0/1335058630 wait complete. 2024-05-08T23:22:40.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:2 flush journal 2024-05-08T23:22:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.086+0000 7f027e47c700 1 Processor -- start 2024-05-08T23:22:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.086+0000 7f027e47c700 1 -- start start 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f027e47c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099a50 0x7f02780d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f027e47c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0278099e30 0x7f027809a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f027e47c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027806f930 con 0x7f02780cba90 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f027e47c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02780cfdf0 con 0x7f0278099e30 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f027e47c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780d00d0 con 0x7f0278099a50 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f0277fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099a50 0x7f02780d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f02777fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0278099e30 0x7f027809a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f027ca19700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f02780cba90 0x7f027806fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:52794/0 (socket says 172.21.15.5:52794) 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f027ca19700 1 -- 172.21.15.5:0/2630043383 learned_addr learned my addr 172.21.15.5:0/2630043383 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f02777fe700 1 -- 172.21.15.5:0/2630043383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099a50 msgr2=0x7f02780d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f02777fe700 1 --2- 172.21.15.5:0/2630043383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099a50 0x7f02780d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f02777fe700 1 -- 172.21.15.5:0/2630043383 >> v1:172.21.15.49:6789/0 conn(0x7f02780cba90 legacy=0x7f027806fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.087+0000 7f02777fe700 1 -- 172.21.15.5:0/2630043383 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780d03b0 con 0x7f0278099e30 2024-05-08T23:22:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f02777fe700 1 --2- 172.21.15.5:0/2630043383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099e30 0x7f027809a790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f026c009840 tx=0x7f026c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a3dcb9795428d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f0276ffd700 1 -- 172.21.15.5:0/2630043383 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c011830 con 0x7f0278099e30 2024-05-08T23:22:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f0276ffd700 1 -- 172.21.15.5:0/2630043383 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c011990 con 0x7f0278099e30 2024-05-08T23:22:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f0276ffd700 1 -- 172.21.15.5:0/2630043383 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c011d20 con 0x7f0278099e30 2024-05-08T23:22:41.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f027e47c700 1 -- 172.21.15.5:0/2630043383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099e30 msgr2=0x7f027809a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f027e47c700 1 --2- 172.21.15.5:0/2630043383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099e30 0x7f027809a790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f026c009840 tx=0x7f026c009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f027e47c700 1 -- 172.21.15.5:0/2630043383 shutdown_connections 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f027e47c700 1 --2- 172.21.15.5:0/2630043383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099a50 0x7f02780d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.088+0000 7f027e47c700 1 --2- 172.21.15.5:0/2630043383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099e30 0x7f027809a790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.089+0000 7f027e47c700 1 -- 172.21.15.5:0/2630043383 >> 172.21.15.5:0/2630043383 conn(0x7f02780b2f30 msgr2=0x7f02780b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.089+0000 7f027e47c700 1 -- 172.21.15.5:0/2630043383 shutdown_connections 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.089+0000 7f027e47c700 1 -- 172.21.15.5:0/2630043383 wait complete. 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.090+0000 7f027e47c700 1 Processor -- start 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.090+0000 7f027e47c700 1 -- start start 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027e47c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099a50 0x7f02780ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027e47c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099e30 0x7f0278154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027e47c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02780cba90 0x7f02780c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027e47c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780c9e10 con 0x7f02780cba90 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027e47c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780c8f20 con 0x7f0278099a50 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027e47c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780c9200 con 0x7f0278099e30 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027ca19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02780cba90 0x7f02780c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027ca19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02780cba90 0x7f02780c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57510/0 (socket says 172.21.15.5:57510) 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027ca19700 1 -- 172.21.15.5:0/1142062316 learned_addr learned my addr 172.21.15.5:0/1142062316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f02777fe700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099e30 0x7f0278154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f0277fff700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099a50 0x7f02780ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027ca19700 1 -- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099e30 msgr2=0x7f0278154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027ca19700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099e30 0x7f0278154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027ca19700 1 -- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099a50 msgr2=0x7f02780ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027ca19700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099a50 0x7f02780ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.091+0000 7f027ca19700 1 -- 172.21.15.5:0/1142062316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c0094a0 con 0x7f02780cba90 2024-05-08T23:22:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.092+0000 7f027ca19700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02780cba90 0x7f02780c6b30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f027000ca80 tx=0x7f027000cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.092+0000 7f0274ff9700 1 -- 172.21.15.5:0/1142062316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000a9e0 con 0x7f02780cba90 2024-05-08T23:22:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.092+0000 7f0274ff9700 1 -- 172.21.15.5:0/1142062316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027000ab40 con 0x7f02780cba90 2024-05-08T23:22:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.092+0000 7f027e47c700 1 -- 172.21.15.5:0/1142062316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02780c94e0 con 0x7f02780cba90 2024-05-08T23:22:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.092+0000 7f0274ff9700 1 -- 172.21.15.5:0/1142062316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270012400 con 0x7f02780cba90 2024-05-08T23:22:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.092+0000 7f027e47c700 1 -- 172.21.15.5:0/1142062316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02780c9aa0 con 0x7f02780cba90 2024-05-08T23:22:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.094+0000 7f0274ff9700 1 -- 172.21.15.5:0/1142062316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f027001b3b0 con 0x7f02780cba90 2024-05-08T23:22:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.094+0000 7f0274ff9700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f026406acc0 0x7f026406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.094+0000 7f0274ff9700 1 -- 172.21.15.5:0/1142062316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0270088dc0 con 0x7f02780cba90 2024-05-08T23:22:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.094+0000 7f0277fff700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f026406acc0 0x7f026406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.095+0000 7f0277fff700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f026406acc0 0x7f026406d180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0268009be0 tx=0x7f0268009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.097+0000 7f028a636b80 1 Processor -- start 2024-05-08T23:22:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.097+0000 7f028a636b80 1 -- start start 2024-05-08T23:22:41.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f028a636b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac37110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f028a636b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeac3c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f028a636b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac66250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f028a636b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aeab92150 con 0x557aeac65dd0 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f028a636b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aeab8fb10 con 0x557aeac39370 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f028a636b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aeab8b400 con 0x557aeac5e7a0 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac66250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac66250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57518/0 (socket says 172.21.15.5:57518) 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 -- 172.21.15.5:0/1469950099 learned_addr learned my addr 172.21.15.5:0/1469950099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f0277fff700 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac37110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f02777fe700 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeac3c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 -- 172.21.15.5:0/1469950099 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 msgr2=0x557aeac37110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac37110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 -- 172.21.15.5:0/1469950099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 msgr2=0x557aeac3c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeac3c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 -- 172.21.15.5:0/1469950099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aeaa0c3b0 con 0x557aeac65dd0 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f027ca19700 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac66250 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0270087310 tx=0x7f0270057980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc53e1be7cc71ea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f025dffb700 1 -- 172.21.15.5:0/1469950099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027005ba90 con 0x557aeac65dd0 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f025dffb700 1 -- 172.21.15.5:0/1469950099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0270060050 con 0x557aeac65dd0 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.098+0000 7f025dffb700 1 -- 172.21.15.5:0/1469950099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270057200 con 0x557aeac65dd0 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 -- 172.21.15.5:0/1469950099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 msgr2=0x557aeac66250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac66250 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0270087310 tx=0x7f0270057980 comp rx=0 tx=0).stop 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 -- 172.21.15.5:0/1469950099 shutdown_connections 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac66250 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac37110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 --2- 172.21.15.5:0/1469950099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeac3c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 -- 172.21.15.5:0/1469950099 >> 172.21.15.5:0/1469950099 conn(0x557aeab99210 msgr2=0x557aeac80dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 -- 172.21.15.5:0/1469950099 shutdown_connections 2024-05-08T23:22:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.099+0000 7f028a636b80 1 -- 172.21.15.5:0/1469950099 wait complete. 2024-05-08T23:22:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.100+0000 7f028a636b80 1 -- start start 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.100+0000 7f028a636b80 1 Processor -- start 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.101+0000 7f028a636b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeac82e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.101+0000 7f028a636b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f028a636b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f028a636b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aeab27560 con 0x557aeac65dd0 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f028a636b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aeab27200 con 0x557aeac39370 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f028a636b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aeaa0e6a0 con 0x557aeac5e7a0 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f028a636b80 10 client.0 _ll_register_callbacks cb 0x557aeab30b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f02777fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f0277fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeac82e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac810d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57526/0 (socket says 172.21.15.5:57526) 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 -- 172.21.15.5:0/1405110039 learned_addr learned my addr 172.21.15.5:0/1405110039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 -- 172.21.15.5:0/1405110039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 msgr2=0x557aeac691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 -- 172.21.15.5:0/1405110039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 msgr2=0x557aeac82e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeac82e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 -- 172.21.15.5:0/1405110039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02780c9aa0 con 0x557aeac65dd0 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f027ca19700 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac810d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f027000ca50 tx=0x7f0270001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291b264ac671de7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f0256ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f0256ffd700 1 -- 172.21.15.5:0/1405110039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270075070 con 0x557aeac65dd0 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f0256ffd700 1 -- 172.21.15.5:0/1405110039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f027005ba90 con 0x557aeac65dd0 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.102+0000 7f028a636b80 1 -- 172.21.15.5:0/1405110039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557aeaae4270 con 0x557aeac65dd0 2024-05-08T23:22:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f0256ffd700 1 -- 172.21.15.5:0/1405110039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f027005fbc0 con 0x557aeac65dd0 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f028a636b80 10 client.5466 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f028a636b80 1 -- 172.21.15.5:0/1405110039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x557aea9b8b60 con 0x557aeac65dd0 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f0256ffd700 1 -- 172.21.15.5:0/1405110039 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557aeaae4270 con 0x557aeac65dd0 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f0256ffd700 1 -- 172.21.15.5:0/1405110039 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f0270053b00 con 0x557aeac65dd0 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f028a636b80 10 client.5466 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f028a636b80 10 client.5466 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.a 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f028a636b80 20 client.5466 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f028a636b80 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x557aeac81610 0x557aeac93830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f028a636b80 4 client.5466 mds_command: new command op to 4742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f028a636b80 1 -- 172.21.15.5:0/1405110039 --> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557aeac0a7e0 con 0x557aeac81610 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f0277fff700 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x557aeac81610 0x557aeac93830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f0256ffd700 10 client.5466 ms_handle_connect on v2:172.21.15.5:6832/295306227 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.103+0000 7f0277fff700 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x557aeac81610 0x557aeac93830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b92fa2fa682e170e server_cookie=aa19c57f5b861107 in_seq=0 out_seq=0 2024-05-08T23:22:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f0256ffd700 1 -- 172.21.15.5:0/1405110039 <== mds.2 v2:172.21.15.5:6832/295306227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557aeac0a7e0 con 0x557aeac81610 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f0256ffd700 10 client.5466 handle_command_reply: tid=0 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 client.5466 shutdown 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 -- 172.21.15.5:0/1405110039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 msgr2=0x557aeac810d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac810d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f027000ca50 tx=0x7f0270001250 comp rx=0 tx=0).stop 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 -- 172.21.15.5:0/1405110039 shutdown_connections 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x557aeac81610 0x557aeac93830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac810d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeac82e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f0256ffd700 0 client.5466 ms_handle_reset on v2:172.21.15.5:6832/295306227 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 --2- 172.21.15.5:0/1405110039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 -- 172.21.15.5:0/1405110039 >> 172.21.15.5:0/1405110039 conn(0x557aeab99210 msgr2=0x557aeac3b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 -- 172.21.15.5:0/1405110039 shutdown_connections 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.104+0000 7f028a636b80 1 -- 172.21.15.5:0/1405110039 wait complete. 2024-05-08T23:22:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.105+0000 7f028a636b80 20 client.5466 trim_cache size 0 max 16384 2024-05-08T23:22:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.120+0000 7f028a636b80 1 Processor -- start 2024-05-08T23:22:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.120+0000 7f028a636b80 1 -- start start 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f028a636b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac39370 0x557aeac671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f028a636b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac5e7a0 0x557aeac67700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f028a636b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac65dd0 0x557aeac76950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f028a636b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aeac3cbd0 con 0x557aeac5e7a0 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f028a636b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aeac91090 con 0x557aeac65dd0 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f028a636b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aeac8f860 con 0x557aeac39370 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f02777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac5e7a0 0x557aeac67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f02777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac5e7a0 0x557aeac67700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57542/0 (socket says 172.21.15.5:57542) 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f02777fe700 1 -- 172.21.15.5:0/2538363227 learned_addr learned my addr 172.21.15.5:0/2538363227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f0277fff700 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac39370 0x557aeac671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f027ca19700 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac65dd0 0x557aeac76950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f02777fe700 1 -- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac39370 msgr2=0x557aeac671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.121+0000 7f02777fe700 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac39370 0x557aeac671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f02777fe700 1 -- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac65dd0 msgr2=0x557aeac76950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f02777fe700 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac65dd0 0x557aeac76950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f02777fe700 1 -- 172.21.15.5:0/2538363227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aeab51560 con 0x557aeac5e7a0 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f0277fff700 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac39370 0x557aeac671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f027ca19700 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac65dd0 0x557aeac76950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f02777fe700 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac5e7a0 0x557aeac67700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f026c009b90 tx=0x7f026c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cacbfdd301f3430 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f025e7fc700 1 -- 172.21.15.5:0/2538363227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c020040 con 0x557aeac5e7a0 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f025e7fc700 1 -- 172.21.15.5:0/2538363227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c0117e0 con 0x557aeac5e7a0 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f025e7fc700 1 -- 172.21.15.5:0/2538363227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c004280 con 0x557aeac5e7a0 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f028a636b80 1 -- 172.21.15.5:0/2538363227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac5e7a0 msgr2=0x557aeac67700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f028a636b80 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac5e7a0 0x557aeac67700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f026c009b90 tx=0x7f026c010000 comp rx=0 tx=0).stop 2024-05-08T23:22:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f028a636b80 1 -- 172.21.15.5:0/2538363227 shutdown_connections 2024-05-08T23:22:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f028a636b80 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac65dd0 0x557aeac76950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f028a636b80 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac39370 0x557aeac671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f028a636b80 1 --2- 172.21.15.5:0/2538363227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac5e7a0 0x557aeac67700 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.122+0000 7f028a636b80 1 -- 172.21.15.5:0/2538363227 >> 172.21.15.5:0/2538363227 conn(0x557aeab99210 msgr2=0x557aeac321e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.123+0000 7f028a636b80 1 -- 172.21.15.5:0/2538363227 shutdown_connections 2024-05-08T23:22:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.123+0000 7f028a636b80 1 -- 172.21.15.5:0/2538363227 wait complete. 2024-05-08T23:22:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.123+0000 7f028a636b80 1 -- start start 2024-05-08T23:22:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.123+0000 7f028a636b80 1 Processor -- start 2024-05-08T23:22:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f028a636b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeacd24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f028a636b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac387c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f028a636b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac3b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f028a636b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aeac80e50 con 0x557aeac65dd0 2024-05-08T23:22:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f028a636b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aeac93ef0 con 0x557aeac39370 2024-05-08T23:22:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f028a636b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aeac94140 con 0x557aeac5e7a0 2024-05-08T23:22:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f028a636b80 10 client.0 _ll_register_callbacks cb 0x557aeab30b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f027ca19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac3b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f027ca19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac3b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57548/0 (socket says 172.21.15.5:57548) 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f027ca19700 1 -- 172.21.15.5:0/3477803444 learned_addr learned my addr 172.21.15.5:0/3477803444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f0277fff700 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeacd24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.124+0000 7f02777fe700 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f027ca19700 1 -- 172.21.15.5:0/3477803444 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 msgr2=0x557aeac387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f027ca19700 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac387c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f027ca19700 1 -- 172.21.15.5:0/3477803444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 msgr2=0x557aeacd24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f027ca19700 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeacd24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f027ca19700 1 -- 172.21.15.5:0/3477803444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02780d03b0 con 0x557aeac65dd0 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f027ca19700 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac3b2e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0270001cb0 tx=0x7f0270088000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9759e356c0174d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f0256ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f0256ffd700 1 -- 172.21.15.5:0/3477803444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270075070 con 0x557aeac65dd0 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f0256ffd700 1 -- 172.21.15.5:0/3477803444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f027005ba90 con 0x557aeac65dd0 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 1 -- 172.21.15.5:0/3477803444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0270060050 con 0x557aeac65dd0 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f0256ffd700 1 -- 172.21.15.5:0/3477803444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0270056e70 con 0x557aeac65dd0 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 10 client.5472 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 1 -- 172.21.15.5:0/3477803444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x557aeaa10cd0 con 0x557aeac65dd0 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f0256ffd700 1 -- 172.21.15.5:0/3477803444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0270060050 con 0x557aeac65dd0 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f0256ffd700 1 -- 172.21.15.5:0/3477803444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f027006baa0 con 0x557aeac65dd0 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 10 client.5472 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 10 client.5472 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.a 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 20 client.5472 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x557aeac379f0 0x557aeac3b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 4 client.5472 mds_command: new command op to 4742 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.125+0000 7f028a636b80 1 -- 172.21.15.5:0/3477803444 --> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557aeab83ec0 con 0x557aeac379f0 2024-05-08T23:22:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.126+0000 7f0277fff700 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x557aeac379f0 0x557aeac3b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:41.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.126+0000 7f0256ffd700 10 client.5472 ms_handle_connect on v2:172.21.15.5:6832/295306227 2024-05-08T23:22:41.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.126+0000 7f0277fff700 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x557aeac379f0 0x557aeac3b820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e87663d7c7556d5 server_cookie=8214b12230a1eacd in_seq=0 out_seq=0 2024-05-08T23:22:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f0256ffd700 1 -- 172.21.15.5:0/3477803444 <== mds.2 v2:172.21.15.5:6832/295306227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557aeab83ec0 con 0x557aeac379f0 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f0256ffd700 10 client.5472 handle_command_reply: tid=0 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 client.5472 shutdown 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 -- 172.21.15.5:0/3477803444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 msgr2=0x557aeac3b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac3b2e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0270001cb0 tx=0x7f0270088000 comp rx=0 tx=0).stop 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 -- 172.21.15.5:0/3477803444 shutdown_connections 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.5:6832/295306227,v1:172.21.15.5:6834/295306227] conn(0x557aeac379f0 0x557aeac3b820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557aeac65dd0 0x557aeac3b2e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f0256ffd700 0 client.5472 ms_handle_reset on v2:172.21.15.5:6832/295306227 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557aeac39370 0x557aeacd24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 --2- 172.21.15.5:0/3477803444 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557aeac5e7a0 0x557aeac387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 -- 172.21.15.5:0/3477803444 >> 172.21.15.5:0/3477803444 conn(0x557aeab99210 msgr2=0x557aeac613f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 -- 172.21.15.5:0/3477803444 shutdown_connections 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.133+0000 7f028a636b80 1 -- 172.21.15.5:0/3477803444 wait complete. 2024-05-08T23:22:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f028a636b80 20 client.5472 trim_cache size 0 max 16384 2024-05-08T23:22:41.134 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:22:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:22:41.134 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 -- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f026406acc0 msgr2=0x7f026406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f026406acc0 0x7f026406d180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0268009be0 tx=0x7f0268009210 comp rx=0 tx=0).stop 2024-05-08T23:22:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 -- 172.21.15.5:0/1142062316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02780cba90 msgr2=0x7f02780c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02780cba90 0x7f02780c6b30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f027000ca80 tx=0x7f027000cd90 comp rx=0 tx=0).stop 2024-05-08T23:22:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 -- 172.21.15.5:0/1142062316 shutdown_connections 2024-05-08T23:22:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f026406acc0 0x7f026406d180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02780cba90 0x7f02780c6b30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0278099a50 0x7f02780ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 --2- 172.21.15.5:0/1142062316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0278099e30 0x7f0278154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:41.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 -- 172.21.15.5:0/1142062316 >> 172.21.15.5:0/1142062316 conn(0x7f02780b2f30 msgr2=0x7f02780ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:41.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 -- 172.21.15.5:0/1142062316 shutdown_connections 2024-05-08T23:22:41.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:41.134+0000 7f027e47c700 1 -- 172.21.15.5:0/1142062316 wait complete. 2024-05-08T23:22:41.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:22:41.147 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-05-08T23:22:41.173 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/ 2024-05-08T23:22:41.173 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4 2024-05-08T23:22:41.173 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5 2024-05-08T23:22:41.173 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6 2024-05-08T23:22:41.173 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7 2024-05-08T23:22:41.173 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-05-08T23:22:41.175 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:22:41.175 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:22:41.175 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:22:41.175 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:22:41.175 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-05-08T23:22:41.175 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:22:41.296 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-08T23:22:41.487 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:22:41.487 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:22:41.487 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:22:41.487 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:22:41.487 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-05-08T23:22:41.487 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:22:41.608 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-05-08T23:22:41.796 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:22:41.796 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:22:41.797 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:22:41.797 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:22:41.797 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-05-08T23:22:41.797 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:22:41.918 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-05-08T23:22:42.106 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:22:42.106 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:22:42.107 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:22:42.107 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:22:42.107 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-05-08T23:22:42.107 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:22:42.228 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627781 2024-05-08T23:22:42.418 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:22:42.418 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:22:42.418 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:22:42.418 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:22:42.419 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-05-08T23:22:42.419 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:22:42.539 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627782 2024-05-08T23:22:42.728 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:22:42.728 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:22:42.728 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:22:42.728 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:22:42.729 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-05-08T23:22:42.729 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:22:42.850 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627783 2024-05-08T23:22:43.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 tag path /d1/d2/d3 tag123 2024-05-08T23:22:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.228+0000 7fde53855700 1 Processor -- start 2024-05-08T23:22:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.228+0000 7fde53855700 1 -- start start 2024-05-08T23:22:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.229+0000 7fde53855700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fde4c14b930 0x7fde4c0b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.229+0000 7fde53855700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde4c06b070 0x7fde4c1633f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.229+0000 7fde53855700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4c0b16e0 con 0x7fde4c162d40 2024-05-08T23:22:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.229+0000 7fde53855700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c1631d0 con 0x7fde4c06b070 2024-05-08T23:22:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.229+0000 7fde53855700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c0b1980 con 0x7fde4c14b930 2024-05-08T23:22:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde51df2700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde4c06b070 0x7fde4c1633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde51df2700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde4c06b070 0x7fde4c1633f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57550/0 (socket says 172.21.15.5:57550) 2024-05-08T23:22:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde50df0700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fde4c162d40 0x7fde4c06a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43516/0 (socket says 172.21.15.5:43516) 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde51df2700 1 -- 172.21.15.5:0/4838714 learned_addr learned my addr 172.21.15.5:0/4838714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde515f1700 1 --2- 172.21.15.5:0/4838714 >> v2:172.21.15.49:3300/0 conn(0x7fde4c14b930 0x7fde4c0b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde51df2700 1 -- 172.21.15.5:0/4838714 >> v2:172.21.15.49:3300/0 conn(0x7fde4c14b930 msgr2=0x7fde4c0b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde51df2700 1 --2- 172.21.15.5:0/4838714 >> v2:172.21.15.49:3300/0 conn(0x7fde4c14b930 0x7fde4c0b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde51df2700 1 -- 172.21.15.5:0/4838714 >> v1:172.21.15.5:6789/0 conn(0x7fde4c162d40 legacy=0x7fde4c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde51df2700 1 -- 172.21.15.5:0/4838714 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0b1c60 con 0x7fde4c06b070 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde515f1700 1 --2- 172.21.15.5:0/4838714 >> v2:172.21.15.49:3300/0 conn(0x7fde4c14b930 0x7fde4c0b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde43fff700 1 -- 172.21.15.5:0/4838714 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889435949 0 0) 0x7fde4c0b16e0 con 0x7fde4c162d40 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde51df2700 1 --2- 172.21.15.5:0/4838714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c06b070 0x7fde4c1633f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0d1460 tx=0x7fde4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877c3c213cb09b0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde43fff700 1 -- 172.21.15.5:0/4838714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4400e070 con 0x7fde4c06b070 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde43fff700 1 -- 172.21.15.5:0/4838714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4400b9c0 con 0x7fde4c06b070 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.230+0000 7fde43fff700 1 -- 172.21.15.5:0/4838714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4400bd70 con 0x7fde4c06b070 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.231+0000 7fde53855700 1 -- 172.21.15.5:0/4838714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c06b070 msgr2=0x7fde4c1633f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.231+0000 7fde53855700 1 --2- 172.21.15.5:0/4838714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c06b070 0x7fde4c1633f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0d1460 tx=0x7fde4400b330 comp rx=0 tx=0).stop 2024-05-08T23:22:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.231+0000 7fde53855700 1 -- 172.21.15.5:0/4838714 shutdown_connections 2024-05-08T23:22:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.231+0000 7fde53855700 1 --2- 172.21.15.5:0/4838714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c06b070 0x7fde4c1633f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.231+0000 7fde53855700 1 --2- 172.21.15.5:0/4838714 >> v2:172.21.15.49:3300/0 conn(0x7fde4c14b930 0x7fde4c0b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.231+0000 7fde53855700 1 -- 172.21.15.5:0/4838714 >> 172.21.15.5:0/4838714 conn(0x7fde4c099a50 msgr2=0x7fde4c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.231+0000 7fde53855700 1 -- 172.21.15.5:0/4838714 shutdown_connections 2024-05-08T23:22:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.231+0000 7fde53855700 1 -- 172.21.15.5:0/4838714 wait complete. 2024-05-08T23:22:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.232+0000 7fde53855700 1 Processor -- start 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.232+0000 7fde53855700 1 -- start start 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde53855700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c06b070 0x7fde4c1fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde53855700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde4c14b930 0x7fde4c0b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde53855700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c162d40 0x7fde4c1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde53855700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c144bd0 con 0x7fde4c162d40 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde53855700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0b0c60 con 0x7fde4c06b070 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde53855700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1f76b0 con 0x7fde4c14b930 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde51df2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c162d40 0x7fde4c1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde51df2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c162d40 0x7fde4c1f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57562/0 (socket says 172.21.15.5:57562) 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde51df2700 1 -- 172.21.15.5:0/2067458141 learned_addr learned my addr 172.21.15.5:0/2067458141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.233+0000 7fde515f1700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c06b070 0x7fde4c1fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde50df0700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde4c14b930 0x7fde4c0b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde51df2700 1 -- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde4c14b930 msgr2=0x7fde4c0b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde51df2700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde4c14b930 0x7fde4c0b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde51df2700 1 -- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c06b070 msgr2=0x7fde4c1fb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde51df2700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c06b070 0x7fde4c1fb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde51df2700 1 -- 172.21.15.5:0/2067458141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44009b30 con 0x7fde4c162d40 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde51df2700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c162d40 0x7fde4c1f7170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fde4400b730 tx=0x7fde44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde41ffb700 1 -- 172.21.15.5:0/2067458141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440099b0 con 0x7fde4c162d40 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde41ffb700 1 -- 172.21.15.5:0/2067458141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde440092e0 con 0x7fde4c162d40 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde53855700 1 -- 172.21.15.5:0/2067458141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c144de0 con 0x7fde4c162d40 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde41ffb700 1 -- 172.21.15.5:0/2067458141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4401c660 con 0x7fde4c162d40 2024-05-08T23:22:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.234+0000 7fde53855700 1 -- 172.21.15.5:0/2067458141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c143200 con 0x7fde4c162d40 2024-05-08T23:22:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.236+0000 7fde41ffb700 1 -- 172.21.15.5:0/2067458141 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fde4401c7c0 con 0x7fde4c162d40 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.236+0000 7fde41ffb700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde3006ac90 0x7fde3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.236+0000 7fde41ffb700 1 -- 172.21.15.5:0/2067458141 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde4408ac80 con 0x7fde4c162d40 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.236+0000 7fde515f1700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde3006ac90 0x7fde3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.237+0000 7fde515f1700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde3006ac90 0x7fde3006d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde48009930 tx=0x7fde48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.238+0000 7fde5fa0fb80 1 Processor -- start 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.238+0000 7fde5fa0fb80 1 -- start start 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08ba810 0x5601a0893180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a0898110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08c1e40 0x5601a08c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a07ee1c0 con 0x5601a08953e0 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a07ebb80 con 0x5601a08c1e40 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601a07e7470 con 0x5601a08ba810 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde50df0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a0898110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde50df0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a0898110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57570/0 (socket says 172.21.15.5:57570) 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.239+0000 7fde50df0700 1 -- 172.21.15.5:0/3041698645 learned_addr learned my addr 172.21.15.5:0/3041698645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde51df2700 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08c1e40 0x5601a08c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde515f1700 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08ba810 0x5601a0893180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde50df0700 1 -- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08ba810 msgr2=0x5601a0893180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde50df0700 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08ba810 0x5601a0893180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde50df0700 1 -- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08c1e40 msgr2=0x5601a08c22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde50df0700 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08c1e40 0x5601a08c22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde50df0700 1 -- 172.21.15.5:0/3041698645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601a0668420 con 0x5601a08953e0 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde515f1700 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08ba810 0x5601a0893180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde50df0700 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a0898110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fde3c007be0 tx=0x7fde3c007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ed3a30a75b0034 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde36ffd700 1 -- 172.21.15.5:0/3041698645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c014480 con 0x5601a08953e0 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde36ffd700 1 -- 172.21.15.5:0/3041698645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c0145e0 con 0x5601a08953e0 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.240+0000 7fde36ffd700 1 -- 172.21.15.5:0/3041698645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c014950 con 0x5601a08953e0 2024-05-08T23:22:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3041698645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 msgr2=0x5601a0898110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a0898110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fde3c007be0 tx=0x7fde3c007330 comp rx=0 tx=0).stop 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3041698645 shutdown_connections 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08c1e40 0x5601a08c22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08ba810 0x5601a0893180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3041698645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a0898110 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3041698645 >> 172.21.15.5:0/3041698645 conn(0x5601a07f5280 msgr2=0x5601a08dce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3041698645 shutdown_connections 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3041698645 wait complete. 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.241+0000 7fde5fa0fb80 1 -- start start 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.242+0000 7fde5fa0fb80 1 Processor -- start 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a08de2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 0x5601a08bead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a07835d0 con 0x5601a08953e0 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a0783270 con 0x5601a08ba810 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601a066a710 con 0x5601a08c1e40 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde5fa0fb80 10 client.0 _ll_register_callbacks cb 0x5601a078cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde515f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a08de2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde515f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a08de2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57584/0 (socket says 172.21.15.5:57584) 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.243+0000 7fde515f1700 1 -- 172.21.15.5:0/2690826286 learned_addr learned my addr 172.21.15.5:0/2690826286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde50df0700 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde51df2700 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 0x5601a08bead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde515f1700 1 -- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 msgr2=0x5601a08bead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde515f1700 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 0x5601a08bead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde515f1700 1 -- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 msgr2=0x5601a08b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde515f1700 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde515f1700 1 -- 172.21.15.5:0/2690826286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c00f040 con 0x5601a08953e0 2024-05-08T23:22:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde515f1700 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a08de2c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fde48009fd0 tx=0x7fde480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374b6a85934fd71b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde2bfff700 1 -- 172.21.15.5:0/2690826286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48022620 con 0x5601a08953e0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde2bfff700 1 -- 172.21.15.5:0/2690826286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde48022780 con 0x5601a08953e0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2690826286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601a07402e0 con 0x5601a08953e0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde2bfff700 1 -- 172.21.15.5:0/2690826286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde48015e70 con 0x5601a08953e0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde5fa0fb80 10 client.5487 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2690826286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5601a0614bd0 con 0x5601a08953e0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.244+0000 7fde2bfff700 1 -- 172.21.15.5:0/2690826286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601a07402e0 con 0x5601a08953e0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde2bfff700 1 -- 172.21.15.5:0/2690826286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fde48022940 con 0x5601a08953e0 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde5fa0fb80 10 client.5487 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde5fa0fb80 10 client.5487 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde5fa0fb80 20 client.5487 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5601a08be3d0 0x5601a08bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde5fa0fb80 4 client.5487 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2690826286 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601a0866850 con 0x5601a08be3d0 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde50df0700 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5601a08be3d0 0x5601a08bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde50df0700 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5601a08be3d0 0x5601a08bfe20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca73dbbacef15390 server_cookie=bef73d02b7b2fcf1 in_seq=0 out_seq=0 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.245+0000 7fde2bfff700 10 client.5487 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde2bfff700 1 -- 172.21.15.5:0/2690826286 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601a0866850 con 0x5601a08be3d0 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde2bfff700 10 client.5487 handle_command_reply: tid=0 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 client.5487 shutdown 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2690826286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 msgr2=0x5601a08de2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a08de2c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fde48009fd0 tx=0x7fde480047c0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2690826286 shutdown_connections 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5601a08be3d0 0x5601a08bfe20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 0x5601a08bead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08953e0 0x5601a08de2c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2690826286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde2bfff700 0 client.5487 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2690826286 >> 172.21.15.5:0/2690826286 conn(0x5601a07f5280 msgr2=0x5601a08974c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2690826286 shutdown_connections 2024-05-08T23:22:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.246+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2690826286 wait complete. 2024-05-08T23:22:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.247+0000 7fde5fa0fb80 20 client.5487 trim_cache size 0 max 16384 2024-05-08T23:22:43.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.266+0000 7fde5fa0fb80 1 Processor -- start 2024-05-08T23:22:43.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.266+0000 7fde5fa0fb80 1 -- start start 2024-05-08T23:22:43.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08953e0 0x5601a08babf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08ba810 0x5601a08bb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 0x5601a08957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a08fdd40 con 0x5601a08ba810 2024-05-08T23:22:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a08920a0 con 0x5601a08953e0 2024-05-08T23:22:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601a08c2de0 con 0x5601a08c1e40 2024-05-08T23:22:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde50df0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08ba810 0x5601a08bb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde50df0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08ba810 0x5601a08bb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57590/0 (socket says 172.21.15.5:57590) 2024-05-08T23:22:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde50df0700 1 -- 172.21.15.5:0/2727868371 learned_addr learned my addr 172.21.15.5:0/2727868371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde50df0700 1 -- 172.21.15.5:0/2727868371 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 msgr2=0x5601a08957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde515f1700 1 --2- 172.21.15.5:0/2727868371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08953e0 0x5601a08babf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde50df0700 1 --2- 172.21.15.5:0/2727868371 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 0x5601a08957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde50df0700 1 -- 172.21.15.5:0/2727868371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08953e0 msgr2=0x5601a08babf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.267+0000 7fde50df0700 1 --2- 172.21.15.5:0/2727868371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08953e0 0x5601a08babf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde50df0700 1 -- 172.21.15.5:0/2727868371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601a07ad5d0 con 0x5601a08ba810 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde50df0700 1 --2- 172.21.15.5:0/2727868371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08ba810 0x5601a08bb130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fde3c013000 tx=0x7fde3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26b5e34e441d30f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde377fe700 1 -- 172.21.15.5:0/2727868371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c014480 con 0x5601a08ba810 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde377fe700 1 -- 172.21.15.5:0/2727868371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c025070 con 0x5601a08ba810 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde377fe700 1 -- 172.21.15.5:0/2727868371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c026d40 con 0x5601a08ba810 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2727868371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08ba810 msgr2=0x5601a08bb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2727868371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08ba810 0x5601a08bb130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fde3c013000 tx=0x7fde3c011000 comp rx=0 tx=0).stop 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2727868371 shutdown_connections 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2727868371 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08c1e40 0x5601a08957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2727868371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08953e0 0x5601a08babf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/2727868371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08ba810 0x5601a08bb130 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.268+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2727868371 >> 172.21.15.5:0/2727868371 conn(0x5601a07f5280 msgr2=0x5601a0894370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.269+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2727868371 shutdown_connections 2024-05-08T23:22:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.269+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/2727868371 wait complete. 2024-05-08T23:22:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.269+0000 7fde5fa0fb80 1 -- start start 2024-05-08T23:22:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.269+0000 7fde5fa0fb80 1 Processor -- start 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08953e0 0x5601a0943b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde5fa0fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08c1e40 0x5601a08c5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a09474d0 con 0x5601a08c1e40 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a0891e30 con 0x5601a08ba810 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde5fa0fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601a08959d0 con 0x5601a08953e0 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde5fa0fb80 10 client.0 _ll_register_callbacks cb 0x5601a078cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde51df2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08c1e40 0x5601a08c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde51df2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08c1e40 0x5601a08c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57596/0 (socket says 172.21.15.5:57596) 2024-05-08T23:22:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.270+0000 7fde51df2700 1 -- 172.21.15.5:0/3680855559 learned_addr learned my addr 172.21.15.5:0/3680855559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde50df0700 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde515f1700 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08953e0 0x5601a0943b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde51df2700 1 -- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08953e0 msgr2=0x5601a0943b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde51df2700 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08953e0 0x5601a0943b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde51df2700 1 -- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 msgr2=0x5601a08c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde51df2700 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde51df2700 1 -- 172.21.15.5:0/3680855559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601a0668420 con 0x5601a08c1e40 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde50df0700 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde51df2700 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08c1e40 0x5601a08c5530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fde440043d0 tx=0x7fde440595c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a5c82b9f5d7e67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde2bfff700 1 -- 172.21.15.5:0/3680855559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4405a010 con 0x5601a08c1e40 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde2bfff700 1 -- 172.21.15.5:0/3680855559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde44059bb0 con 0x5601a08c1e40 2024-05-08T23:22:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3680855559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde3c0145e0 con 0x5601a08c1e40 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde2bfff700 1 -- 172.21.15.5:0/3680855559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde4405d9e0 con 0x5601a08c1e40 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 10 client.5493 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3680855559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5601a066cd40 con 0x5601a08c1e40 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde2bfff700 1 -- 172.21.15.5:0/3680855559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c0145e0 con 0x5601a08c1e40 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde2bfff700 1 -- 172.21.15.5:0/3680855559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fde440590a0 con 0x5601a08c1e40 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 10 client.5493 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 10 client.5493 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 20 client.5493 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5601a08c5a70 0x5601a0949bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 4 client.5493 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}] 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.271+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3680855559 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}) v1 -- 0x5601a07dff30 con 0x5601a08c5a70 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.272+0000 7fde515f1700 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5601a08c5a70 0x5601a0949bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.272+0000 7fde515f1700 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5601a08c5a70 0x5601a0949bc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a6d1260d3f10aed server_cookie=4cdfe40c4a6b64eb in_seq=0 out_seq=0 2024-05-08T23:22:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.272+0000 7fde2bfff700 10 client.5493 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde2bfff700 1 -- 172.21.15.5:0/3680855559 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+80 (crc 0 0 0) 0x5601a07dff30 con 0x5601a08c5a70 2024-05-08T23:22:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde2bfff700 10 client.5493 handle_command_reply: tid=0 2024-05-08T23:22:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 client.5493 shutdown 2024-05-08T23:22:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3680855559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08c1e40 msgr2=0x5601a08c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08c1e40 0x5601a08c5530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fde440043d0 tx=0x7fde440595c0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3680855559 shutdown_connections 2024-05-08T23:22:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5601a08c5a70 0x5601a0949bc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a08c1e40 0x5601a08c5530 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5601a08953e0 0x5601a0943b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 --2- 172.21.15.5:0/3680855559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601a08ba810 0x5601a08c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3680855559 >> 172.21.15.5:0/3680855559 conn(0x5601a07f5280 msgr2=0x5601a08babf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde2bfff700 0 client.5493 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3680855559 shutdown_connections 2024-05-08T23:22:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.273+0000 7fde5fa0fb80 1 -- 172.21.15.5:0/3680855559 wait complete. 2024-05-08T23:22:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.274+0000 7fde5fa0fb80 20 client.5493 trim_cache size 0 max 16384 2024-05-08T23:22:43.282 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-08T23:22:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "tag123", 2024-05-08T23:22:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-08T23:22:43.282 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 -- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde3006ac90 msgr2=0x7fde3006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde3006ac90 0x7fde3006d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde48009930 tx=0x7fde48008040 comp rx=0 tx=0).stop 2024-05-08T23:22:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 -- 172.21.15.5:0/2067458141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c162d40 msgr2=0x7fde4c1f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c162d40 0x7fde4c1f7170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fde4400b730 tx=0x7fde44004ad0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 -- 172.21.15.5:0/2067458141 shutdown_connections 2024-05-08T23:22:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fde3006ac90 0x7fde3006d150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde4c162d40 0x7fde4c1f7170 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde4c06b070 0x7fde4c1fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 --2- 172.21.15.5:0/2067458141 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fde4c14b930 0x7fde4c0b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 -- 172.21.15.5:0/2067458141 >> 172.21.15.5:0/2067458141 conn(0x7fde4c099a50 msgr2=0x7fde4c06de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 -- 172.21.15.5:0/2067458141 shutdown_connections 2024-05-08T23:22:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.275+0000 7fde53855700 1 -- 172.21.15.5:0/2067458141 wait complete. 2024-05-08T23:22:43.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2024-05-08T23:22:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.427+0000 7f3a6a66d700 1 Processor -- start 2024-05-08T23:22:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.428+0000 7f3a6a66d700 1 -- start start 2024-05-08T23:22:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.428+0000 7f3a6a66d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3a640b6dd0 0x7f3a6409a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.428+0000 7f3a6a66d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a640d0930 0x7f3a64099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.428+0000 7f3a6a66d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a640cba20 con 0x7f3a64099f90 2024-05-08T23:22:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.428+0000 7f3a6a66d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6409a420 con 0x7f3a640b6dd0 2024-05-08T23:22:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.428+0000 7f3a6a66d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6409acd0 con 0x7f3a640d0930 2024-05-08T23:22:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a68c0a700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3a64099f90 0x7f3a6415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43524/0 (socket says 172.21.15.5:43524) 2024-05-08T23:22:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a68c0a700 1 -- 172.21.15.5:0/77853628 learned_addr learned my addr 172.21.15.5:0/77853628 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:22:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a63fff700 1 --2- 172.21.15.5:0/77853628 >> v2:172.21.15.49:3300/0 conn(0x7f3a640b6dd0 0x7f3a6409a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a637fe700 1 --2- 172.21.15.5:0/77853628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a640d0930 0x7f3a64099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a62ffd700 1 -- 172.21.15.5:0/77853628 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191973502 0 0) 0x7f3a640cba20 con 0x7f3a64099f90 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a62ffd700 1 -- 172.21.15.5:0/77853628 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a4c0035b0 con 0x7f3a64099f90 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a62ffd700 1 -- 172.21.15.5:0/77853628 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3641173068 0 0) 0x7f3a4c0035b0 con 0x7f3a64099f90 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a63fff700 1 -- 172.21.15.5:0/77853628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a640d0930 msgr2=0x7f3a64099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a63fff700 1 --2- 172.21.15.5:0/77853628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a640d0930 0x7f3a64099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a63fff700 1 -- 172.21.15.5:0/77853628 >> v1:172.21.15.5:6789/0 conn(0x7f3a64099f90 legacy=0x7f3a6415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a63fff700 1 -- 172.21.15.5:0/77853628 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a640cbc70 con 0x7f3a640b6dd0 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.429+0000 7f3a63fff700 1 --2- 172.21.15.5:0/77853628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 0x7f3a6409a790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009840 tx=0x7f3a54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad47b8afca74822 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a62ffd700 1 -- 172.21.15.5:0/77853628 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0019f0 con 0x7f3a640b6dd0 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a62ffd700 1 -- 172.21.15.5:0/77853628 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a54004030 con 0x7f3a640b6dd0 2024-05-08T23:22:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a62ffd700 1 -- 172.21.15.5:0/77853628 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a540043a0 con 0x7f3a640b6dd0 2024-05-08T23:22:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a6a66d700 1 -- 172.21.15.5:0/77853628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 msgr2=0x7f3a6409a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/77853628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 0x7f3a6409a790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009840 tx=0x7f3a54009b50 comp rx=0 tx=0).stop 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a6a66d700 1 -- 172.21.15.5:0/77853628 shutdown_connections 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/77853628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 0x7f3a6409a790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/77853628 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a640d0930 0x7f3a64099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a6a66d700 1 -- 172.21.15.5:0/77853628 >> 172.21.15.5:0/77853628 conn(0x7f3a640b2ec0 msgr2=0x7f3a640b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a6a66d700 1 -- 172.21.15.5:0/77853628 shutdown_connections 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.430+0000 7f3a6a66d700 1 -- 172.21.15.5:0/77853628 wait complete. 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.432+0000 7f3a6a66d700 1 Processor -- start 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.432+0000 7f3a6a66d700 1 -- start start 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a6a66d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a64099f90 0x7f3a640c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a6a66d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 0x7f3a640c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a6a66d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a640d0930 0x7f3a641f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a6a66d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640c7de0 con 0x7f3a640d0930 2024-05-08T23:22:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a6a66d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640c7820 con 0x7f3a640b6dd0 2024-05-08T23:22:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a6a66d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640c7b00 con 0x7f3a64099f90 2024-05-08T23:22:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a68c0a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a640d0930 0x7f3a641f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a68c0a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a640d0930 0x7f3a641f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57606/0 (socket says 172.21.15.5:57606) 2024-05-08T23:22:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.433+0000 7f3a68c0a700 1 -- 172.21.15.5:0/2113987910 learned_addr learned my addr 172.21.15.5:0/2113987910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a637fe700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 0x7f3a640c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a63fff700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a64099f90 0x7f3a640c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a68c0a700 1 -- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a64099f90 msgr2=0x7f3a640c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a64099f90 0x7f3a640c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a68c0a700 1 -- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 msgr2=0x7f3a640c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 0x7f3a640c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a68c0a700 1 -- 172.21.15.5:0/2113987910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a540094a0 con 0x7f3a640d0930 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a637fe700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 0x7f3a640c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a63fff700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a64099f90 0x7f3a640c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a640d0930 0x7f3a641f6fe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00cc00 tx=0x7f3a5c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a60ff9700 1 -- 172.21.15.5:0/2113987910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00ab80 con 0x7f3a640d0930 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a60ff9700 1 -- 172.21.15.5:0/2113987910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c00ad00 con 0x7f3a640d0930 2024-05-08T23:22:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a6a66d700 1 -- 172.21.15.5:0/2113987910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a640c8fc0 con 0x7f3a640d0930 2024-05-08T23:22:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a60ff9700 1 -- 172.21.15.5:0/2113987910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c007310 con 0x7f3a640d0930 2024-05-08T23:22:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.434+0000 7f3a6a66d700 1 -- 172.21.15.5:0/2113987910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a640c7470 con 0x7f3a640d0930 2024-05-08T23:22:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.436+0000 7f3a60ff9700 1 -- 172.21.15.5:0/2113987910 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a5c007470 con 0x7f3a640d0930 2024-05-08T23:22:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.437+0000 7f3a60ff9700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a5006ac90 0x7f3a5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.437+0000 7f3a60ff9700 1 -- 172.21.15.5:0/2113987910 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a5c0892d0 con 0x7f3a640d0930 2024-05-08T23:22:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.437+0000 7f3a63fff700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a5006ac90 0x7f3a5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.437+0000 7f3a63fff700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a5006ac90 0x7f3a5006d150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009b90 tx=0x7f3a54008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.439+0000 7f3a76827b80 1 Processor -- start 2024-05-08T23:22:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.439+0000 7f3a76827b80 1 -- start start 2024-05-08T23:22:43.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.439+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c1982f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.439+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1982f9370 0x55c1982fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.439+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c198325dd0 0x55c198326250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.439+0000 7f3a76827b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c198252150 con 0x55c1982f9370 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.439+0000 7f3a76827b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19824fb10 con 0x55c198325dd0 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.439+0000 7f3a76827b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c19824b400 con 0x55c19831e7a0 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1982f9370 0x55c1982fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1982f9370 0x55c1982fc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57622/0 (socket says 172.21.15.5:57622) 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 -- 172.21.15.5:0/54804502 learned_addr learned my addr 172.21.15.5:0/54804502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a63fff700 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c1982f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c198325dd0 0x55c198326250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 -- 172.21.15.5:0/54804502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 msgr2=0x55c1982f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c1982f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 -- 172.21.15.5:0/54804502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c198325dd0 msgr2=0x55c198326250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c198325dd0 0x55c198326250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 -- 172.21.15.5:0/54804502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1980cc3b0 con 0x55c1982f9370 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a637fe700 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1982f9370 0x55c1982fc0a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009c20 tx=0x7f3a58009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efcafa362ed29da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a49ffb700 1 -- 172.21.15.5:0/54804502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58010bb0 con 0x55c1982f9370 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a49ffb700 1 -- 172.21.15.5:0/54804502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58010d10 con 0x55c1982f9370 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.440+0000 7f3a49ffb700 1 -- 172.21.15.5:0/54804502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58004240 con 0x55c1982f9370 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 -- 172.21.15.5:0/54804502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1982f9370 msgr2=0x55c1982fc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1982f9370 0x55c1982fc0a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009c20 tx=0x7f3a58009210 comp rx=0 tx=0).stop 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 -- 172.21.15.5:0/54804502 shutdown_connections 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c198325dd0 0x55c198326250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c1982f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 --2- 172.21.15.5:0/54804502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1982f9370 0x55c1982fc0a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 -- 172.21.15.5:0/54804502 >> 172.21.15.5:0/54804502 conn(0x55c198259210 msgr2=0x55c198340dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 -- 172.21.15.5:0/54804502 shutdown_connections 2024-05-08T23:22:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.441+0000 7f3a76827b80 1 -- 172.21.15.5:0/54804502 wait complete. 2024-05-08T23:22:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.442+0000 7f3a76827b80 1 -- start start 2024-05-08T23:22:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.442+0000 7f3a76827b80 1 Processor -- start 2024-05-08T23:22:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.443+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c1983426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.443+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19831e7a0 0x55c198342c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.443+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c198325dd0 0x55c1983500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.443+0000 7f3a76827b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1981e7560 con 0x55c19831e7a0 2024-05-08T23:22:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.443+0000 7f3a76827b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1981e7200 con 0x55c1982f9370 2024-05-08T23:22:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.443+0000 7f3a76827b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1980ce6a0 con 0x55c198325dd0 2024-05-08T23:22:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a76827b80 10 client.0 _ll_register_callbacks cb 0x55c1981f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19831e7a0 0x55c198342c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19831e7a0 0x55c198342c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57636/0 (socket says 172.21.15.5:57636) 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 -- 172.21.15.5:0/1719312634 learned_addr learned my addr 172.21.15.5:0/1719312634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a63fff700 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c1983426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c198325dd0 0x55c1983500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 -- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c198325dd0 msgr2=0x55c1983500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c198325dd0 0x55c1983500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 -- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 msgr2=0x55c1983426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c1983426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 -- 172.21.15.5:0/1719312634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a58009880 con 0x55c19831e7a0 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a637fe700 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19831e7a0 0x55c198342c20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3a58008010 tx=0x7f3a580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb97b6f35ca3c86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1719312634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58021070 con 0x55c19831e7a0 2024-05-08T23:22:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1719312634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a58010e80 con 0x55c19831e7a0 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a76827b80 1 -- 172.21.15.5:0/1719312634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1981a4270 con 0x55c19831e7a0 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.444+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1719312634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a580096b0 con 0x55c19831e7a0 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a76827b80 10 client.5505 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a76827b80 1 -- 172.21.15.5:0/1719312634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55c198078b60 con 0x55c19831e7a0 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1719312634 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1981a4270 con 0x55c19831e7a0 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1719312634 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f3a58004370 con 0x55c19831e7a0 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a76827b80 10 client.5505 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a76827b80 10 client.5505 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a76827b80 20 client.5505 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55c198344d90 0x55c19835d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a76827b80 4 client.5505 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a76827b80 1 -- 172.21.15.5:0/1719312634 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1982ca7e0 con 0x55c198344d90 2024-05-08T23:22:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.445+0000 7f3a63fff700 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55c198344d90 0x55c19835d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a42ffd700 10 client.5505 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a63fff700 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55c198344d90 0x55c19835d830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a249e276b99206 server_cookie=9719f2cb74c0b7dc in_seq=0 out_seq=0 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1719312634 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1982ca7e0 con 0x55c198344d90 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a42ffd700 10 client.5505 handle_command_reply: tid=0 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 client.5505 shutdown 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 -- 172.21.15.5:0/1719312634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19831e7a0 msgr2=0x55c198342c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19831e7a0 0x55c198342c20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3a58008010 tx=0x7f3a580047c0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 -- 172.21.15.5:0/1719312634 shutdown_connections 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55c198344d90 0x55c19835d830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c198325dd0 0x55c1983500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c1983426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1719312634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19831e7a0 0x55c198342c20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a42ffd700 0 client.5505 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.446+0000 7f3a76827b80 1 -- 172.21.15.5:0/1719312634 >> 172.21.15.5:0/1719312634 conn(0x55c198259210 msgr2=0x55c1982fb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.447+0000 7f3a76827b80 1 -- 172.21.15.5:0/1719312634 shutdown_connections 2024-05-08T23:22:43.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.447+0000 7f3a76827b80 1 -- 172.21.15.5:0/1719312634 wait complete. 2024-05-08T23:22:43.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.447+0000 7f3a76827b80 20 client.5505 trim_cache size 0 max 16384 2024-05-08T23:22:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.465+0000 7f3a76827b80 1 Processor -- start 2024-05-08T23:22:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.465+0000 7f3a76827b80 1 -- start start 2024-05-08T23:22:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c198328d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c19831d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a76827b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1982f7cc0 con 0x55c198325dd0 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a76827b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19834f7b0 con 0x55c1982f9370 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a76827b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1982f6ec0 con 0x55c19831e7a0 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57644/0 (socket says 172.21.15.5:57644) 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 -- 172.21.15.5:0/725948087 learned_addr learned my addr 172.21.15.5:0/725948087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a63fff700 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c198328d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 -- 172.21.15.5:0/725948087 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 msgr2=0x55c19831d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a637fe700 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c19831d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c19831d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 -- 172.21.15.5:0/725948087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 msgr2=0x55c198328d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c198328d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 -- 172.21.15.5:0/725948087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c198211560 con 0x55c198325dd0 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.466+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831d9d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00a460 tx=0x7f3a5c05e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a70292d89b738c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a4a7fc700 1 -- 172.21.15.5:0/725948087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0890c0 con 0x55c198325dd0 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a4a7fc700 1 -- 172.21.15.5:0/725948087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c056470 con 0x55c198325dd0 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a4a7fc700 1 -- 172.21.15.5:0/725948087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c058570 con 0x55c198325dd0 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 -- 172.21.15.5:0/725948087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 msgr2=0x55c19831d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831d9d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00a460 tx=0x7f3a5c05e040 comp rx=0 tx=0).stop 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 -- 172.21.15.5:0/725948087 shutdown_connections 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831d9d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c198328d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 --2- 172.21.15.5:0/725948087 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c19831d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 -- 172.21.15.5:0/725948087 >> 172.21.15.5:0/725948087 conn(0x55c198259210 msgr2=0x55c198326ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 -- 172.21.15.5:0/725948087 shutdown_connections 2024-05-08T23:22:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.467+0000 7f3a76827b80 1 -- 172.21.15.5:0/725948087 wait complete. 2024-05-08T23:22:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.468+0000 7f3a76827b80 1 -- start start 2024-05-08T23:22:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.468+0000 7f3a76827b80 1 Processor -- start 2024-05-08T23:22:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c198392e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c19831f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a76827b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a76827b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19831f890 con 0x55c198325dd0 2024-05-08T23:22:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a76827b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c198341510 con 0x55c1982f9370 2024-05-08T23:22:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a76827b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c198322650 con 0x55c19831e7a0 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a76827b80 10 client.0 _ll_register_callbacks cb 0x55c1981f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a68c0a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a68c0a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57646/0 (socket says 172.21.15.5:57646) 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a68c0a700 1 -- 172.21.15.5:0/1610837372 learned_addr learned my addr 172.21.15.5:0/1610837372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a63fff700 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c198392e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a637fe700 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c19831f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a68c0a700 1 -- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 msgr2=0x55c19831f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c19831f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a68c0a700 1 -- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 msgr2=0x55c198392e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.469+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c198392e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a68c0a700 1 -- 172.21.15.5:0/1610837372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a640c7470 con 0x55c198325dd0 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a68c0a700 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831c4a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c057dd0 tx=0x7f3a5c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84884ba88e1f6bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:22:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1610837372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c087460 con 0x55c198325dd0 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1610837372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a5c058330 con 0x55c198325dd0 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 1 -- 172.21.15.5:0/1610837372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a58010d10 con 0x55c198325dd0 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1610837372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a5c058530 con 0x55c198325dd0 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 10 client.5511 fetch_fsmap learned FSMap version 22 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 1 -- 172.21.15.5:0/1610837372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55c1980d0cd0 con 0x55c198325dd0 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1610837372 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58010d10 con 0x55c198325dd0 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1610837372 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f3a5c060030 con 0x55c198325dd0 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 10 client.5511 fetch_fsmap finished waiting for FSMap version 22 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 10 client.5511 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 20 client.5511 populate_metadata read hostname 'smithi005' 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55c198340860 0x55c198340c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 4 client.5511 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:22:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.470+0000 7f3a76827b80 1 -- 172.21.15.5:0/1610837372 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c198243ec0 con 0x55c198340860 2024-05-08T23:22:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a63fff700 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55c198340860 0x55c198340c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a63fff700 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55c198340860 0x55c198340c70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ecbb7ea862a053 server_cookie=a8472eb2117b4f41 in_seq=0 out_seq=0 2024-05-08T23:22:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a42ffd700 10 client.5511 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a42ffd700 1 -- 172.21.15.5:0/1610837372 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+207 (crc 0 0 0) 0x55c198243ec0 con 0x55c198340860 2024-05-08T23:22:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a42ffd700 10 client.5511 handle_command_reply: tid=0 2024-05-08T23:22:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 client.5511 shutdown 2024-05-08T23:22:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 -- 172.21.15.5:0/1610837372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 msgr2=0x55c19831c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831c4a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c057dd0 tx=0x7f3a5c009040 comp rx=0 tx=0).stop 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 -- 172.21.15.5:0/1610837372 shutdown_connections 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x55c198340860 0x55c198340c70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c198325dd0 0x55c19831c4a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c1982f9370 0x55c198392e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a42ffd700 0 client.5511 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 --2- 172.21.15.5:0/1610837372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c19831e7a0 0x55c19831f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 -- 172.21.15.5:0/1610837372 >> 172.21.15.5:0/1610837372 conn(0x55c198259210 msgr2=0x55c198328d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 -- 172.21.15.5:0/1610837372 shutdown_connections 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.471+0000 7f3a76827b80 1 -- 172.21.15.5:0/1610837372 wait complete. 2024-05-08T23:22:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a76827b80 20 client.5511 trim_cache size 0 max 16384 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tag123": { 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "tag123", 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:22:43.480 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:22:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 -- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a5006ac90 msgr2=0x7f3a5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a5006ac90 0x7f3a5006d150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009b90 tx=0x7f3a54008030 comp rx=0 tx=0).stop 2024-05-08T23:22:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 -- 172.21.15.5:0/2113987910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a640d0930 msgr2=0x7f3a641f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a640d0930 0x7f3a641f6fe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00cc00 tx=0x7f3a5c00a060 comp rx=0 tx=0).stop 2024-05-08T23:22:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 -- 172.21.15.5:0/2113987910 shutdown_connections 2024-05-08T23:22:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a5006ac90 0x7f3a5006d150 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a640d0930 0x7f3a641f6fe0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a64099f90 0x7f3a640c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 --2- 172.21.15.5:0/2113987910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a640b6dd0 0x7f3a640c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 -- 172.21.15.5:0/2113987910 >> 172.21.15.5:0/2113987910 conn(0x7f3a640b2ec0 msgr2=0x7f3a64160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 -- 172.21.15.5:0/2113987910 shutdown_connections 2024-05-08T23:22:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:22:43.472+0000 7f3a6a66d700 1 -- 172.21.15.5:0/2113987910 wait complete. 2024-05-08T23:22:43.485 INFO:tasks.cephfs.filesystem:scrub status for tag:tag123 - {'path': '/d1/d2/d3', 'tag': 'tag123', 'options': 'recursive,force'} 2024-05-08T23:23:07.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:23:07.520 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:23:13.486 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2024-05-08T23:23:13.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.634+0000 7f0eaf31c700 1 Processor -- start 2024-05-08T23:23:13.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.634+0000 7f0eaf31c700 1 -- start start 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.635+0000 7f0eaf31c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ea815b8c0 0x7f0ea80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.635+0000 7f0eaf31c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea80d1740 0x7f0ea80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.635+0000 7f0eaf31c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea80cf560 con 0x7f0ea815b4e0 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.635+0000 7f0eaf31c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea80cefe0 con 0x7f0ea815b8c0 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.635+0000 7f0eaf31c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80ccff0 con 0x7f0ea80d1740 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.635+0000 7f0eac8b7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ea815b8c0 0x7f0ea80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.635+0000 7f0eac8b7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ea815b8c0 0x7f0ea80d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38264/0 (socket says 172.21.15.5:38264) 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.635+0000 7f0eac8b7700 1 -- 172.21.15.5:0/103407074 learned_addr learned my addr 172.21.15.5:0/103407074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/103407074 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea80d1740 0x7f0ea80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0eac8b7700 1 -- 172.21.15.5:0/103407074 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea80d1740 msgr2=0x7f0ea80cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/103407074 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea80d1740 0x7f0ea80cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0eac8b7700 1 -- 172.21.15.5:0/103407074 >> v1:172.21.15.49:6789/0 conn(0x7f0ea815b4e0 legacy=0x7f0ea80b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0eac8b7700 1 -- 172.21.15.5:0/103407074 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80cd2d0 con 0x7f0ea815b8c0 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/103407074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea815b8c0 0x7f0ea80d1200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0098f0 tx=0x7f0e9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bf573b3af5c77c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0e9bfff700 1 -- 172.21.15.5:0/103407074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00b8a0 con 0x7f0ea815b8c0 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0e9bfff700 1 -- 172.21.15.5:0/103407074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c00ba00 con 0x7f0ea815b8c0 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.636+0000 7f0e9bfff700 1 -- 172.21.15.5:0/103407074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00bd90 con 0x7f0ea815b8c0 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.637+0000 7f0eaf31c700 1 -- 172.21.15.5:0/103407074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea815b8c0 msgr2=0x7f0ea80d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.637+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/103407074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea815b8c0 0x7f0ea80d1200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0098f0 tx=0x7f0e9c009c00 comp rx=0 tx=0).stop 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.637+0000 7f0eaf31c700 1 -- 172.21.15.5:0/103407074 shutdown_connections 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.637+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/103407074 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea80d1740 0x7f0ea80cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.637+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/103407074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea815b8c0 0x7f0ea80d1200 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.637+0000 7f0eaf31c700 1 -- 172.21.15.5:0/103407074 >> 172.21.15.5:0/103407074 conn(0x7f0ea808afe0 msgr2=0x7f0ea808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.637+0000 7f0eaf31c700 1 -- 172.21.15.5:0/103407074 shutdown_connections 2024-05-08T23:23:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.637+0000 7f0eaf31c700 1 -- 172.21.15.5:0/103407074 wait complete. 2024-05-08T23:23:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.638+0000 7f0eaf31c700 1 Processor -- start 2024-05-08T23:23:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.639+0000 7f0eaf31c700 1 -- start start 2024-05-08T23:23:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0eaf31c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80d1740 0x7f0ea81f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0eaf31c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea815b4e0 0x7f0ea81f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0eaf31c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea815b8c0 0x7f0ea81f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0eaf31c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81f4360 con 0x7f0ea80d1740 2024-05-08T23:23:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0eaf31c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81f5370 con 0x7f0ea815b4e0 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0eaf31c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80c5d10 con 0x7f0ea815b8c0 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80d1740 0x7f0ea81f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80d1740 0x7f0ea81f4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38272/0 (socket says 172.21.15.5:38272) 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3675225682 learned_addr learned my addr 172.21.15.5:0/3675225682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea815b4e0 0x7f0ea81f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea815b8c0 0x7f0ea81f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea815b8c0 msgr2=0x7f0ea81f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea815b8c0 0x7f0ea81f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea815b4e0 msgr2=0x7f0ea81f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea815b4e0 0x7f0ea81f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3675225682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c009550 con 0x7f0ea80d1740 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea815b8c0 0x7f0ea81f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80d1740 0x7f0ea81f4780 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c510 tx=0x7f0ea400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0e99ffb700 1 -- 172.21.15.5:0/3675225682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40052d0 con 0x7f0ea80d1740 2024-05-08T23:23:13.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0e99ffb700 1 -- 172.21.15.5:0/3675225682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea4005430 con 0x7f0ea80d1740 2024-05-08T23:23:13.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.640+0000 7f0eaf31c700 1 -- 172.21.15.5:0/3675225682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea80c5ff0 con 0x7f0ea80d1740 2024-05-08T23:23:13.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.641+0000 7f0e99ffb700 1 -- 172.21.15.5:0/3675225682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40038c0 con 0x7f0ea80d1740 2024-05-08T23:23:13.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.641+0000 7f0eaf31c700 1 -- 172.21.15.5:0/3675225682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea80c65b0 con 0x7f0ea80d1740 2024-05-08T23:23:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.642+0000 7f0e99ffb700 1 -- 172.21.15.5:0/3675225682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0ea4003a20 con 0x7f0ea80d1740 2024-05-08T23:23:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.643+0000 7f0e99ffb700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e8c06acc0 0x7f0e8c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.643+0000 7f0e99ffb700 1 -- 172.21.15.5:0/3675225682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ea4088760 con 0x7f0ea80d1740 2024-05-08T23:23:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.643+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e8c06acc0 0x7f0e8c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.644+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e8c06acc0 0x7f0e8c06d180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c014040 tx=0x7f0e9c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.645+0000 7f0ebb4d6b80 1 Processor -- start 2024-05-08T23:23:13.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.645+0000 7f0ebb4d6b80 1 -- start start 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a79603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 0x5571a797c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a79351a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a788c150 con 0x5571a795ffc0 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a7889b10 con 0x5571a795d3d0 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571a7885400 con 0x5571a7933370 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a79603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a79603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38284/0 (socket says 172.21.15.5:38284) 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3815992177 learned_addr learned my addr 172.21.15.5:0/3815992177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 0x5571a797c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a79351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 msgr2=0x5571a797c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 0x5571a797c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 msgr2=0x5571a79351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a79351a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3815992177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a77063b0 con 0x5571a795ffc0 2024-05-08T23:23:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a79351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a79603a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4086bf0 tx=0x7f0ea4057230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405b9caf7baa2032 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0e92ffd700 1 -- 172.21.15.5:0/3815992177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40575f0 con 0x5571a795ffc0 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0e92ffd700 1 -- 172.21.15.5:0/3815992177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea405b420 con 0x5571a795ffc0 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.646+0000 7f0e92ffd700 1 -- 172.21.15.5:0/3815992177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea405f9e0 con 0x5571a795ffc0 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3815992177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 msgr2=0x5571a79603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a79603a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4086bf0 tx=0x7f0ea4057230 comp rx=0 tx=0).stop 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3815992177 shutdown_connections 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a79351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a79603a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3815992177 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 0x5571a797c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3815992177 >> 172.21.15.5:0/3815992177 conn(0x5571a7893210 msgr2=0x5571a797add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3815992177 shutdown_connections 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.647+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3815992177 wait complete. 2024-05-08T23:23:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.648+0000 7f0ebb4d6b80 1 -- start start 2024-05-08T23:23:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.648+0000 7f0ebb4d6b80 1 Processor -- start 2024-05-08T23:23:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 0x5571a7952610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a795dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a7953790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a7821560 con 0x5571a795ffc0 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a7821200 con 0x5571a795d3d0 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571a77086a0 con 0x5571a7933370 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ebb4d6b80 10 client.0 _ll_register_callbacks cb 0x5571a782ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead8b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a7953790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead8b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a7953790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38294/0 (socket says 172.21.15.5:38294) 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead8b9700 1 -- 172.21.15.5:0/801848865 learned_addr learned my addr 172.21.15.5:0/801848865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 0x5571a7952610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a795dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead8b9700 1 -- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 msgr2=0x5571a7952610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 0x5571a7952610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead8b9700 1 -- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 msgr2=0x5571a795dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a795dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.649+0000 7f0ead8b9700 1 -- 172.21.15.5:0/801848865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea80c65b0 con 0x5571a795ffc0 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a7953790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00079e0 tx=0x7f0ea0007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d7ef2717ced1ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0e87fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0e87fff700 1 -- 172.21.15.5:0/801848865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00118d0 con 0x5571a795ffc0 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0e87fff700 1 -- 172.21.15.5:0/801848865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ea0011a30 con 0x5571a795ffc0 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/801848865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a77de270 con 0x5571a795ffc0 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0e87fff700 1 -- 172.21.15.5:0/801848865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ea0011bf0 con 0x5571a795ffc0 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 10 client.5523 fetch_fsmap learned FSMap version 23 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/801848865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5571a76b2b60 con 0x5571a795ffc0 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0e87fff700 1 -- 172.21.15.5:0/801848865 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5571a77de270 con 0x5571a795ffc0 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0e87fff700 1 -- 172.21.15.5:0/801848865 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f0ea0014070 con 0x5571a795ffc0 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 10 client.5523 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 10 client.5523 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 20 client.5523 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5571a7953cd0 0x5571a79551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 4 client.5523 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.650+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/801848865 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571a79047e0 con 0x5571a7953cd0 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.651+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5571a7953cd0 0x5571a79551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.651+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5571a7953cd0 0x5571a79551b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee16cd68fdb7e1b5 server_cookie=5af9bc4e178dfe1c in_seq=0 out_seq=0 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.651+0000 7f0e87fff700 10 client.5523 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0e87fff700 1 -- 172.21.15.5:0/801848865 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571a79047e0 con 0x5571a7953cd0 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0e87fff700 10 client.5523 handle_command_reply: tid=0 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 client.5523 shutdown 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/801848865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 msgr2=0x5571a7953790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a7953790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00079e0 tx=0x7f0ea0007da0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/801848865 shutdown_connections 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5571a7953cd0 0x5571a79551b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795ffc0 0x5571a7953790 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a7933370 0x5571a7952610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0e87fff700 0 client.5523 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:23:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/801848865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a795dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/801848865 >> 172.21.15.5:0/801848865 conn(0x5571a7893210 msgr2=0x5571a79603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/801848865 shutdown_connections 2024-05-08T23:23:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/801848865 wait complete. 2024-05-08T23:23:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.652+0000 7f0ebb4d6b80 20 client.5523 trim_cache size 0 max 16384 2024-05-08T23:23:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.670+0000 7f0ebb4d6b80 1 Processor -- start 2024-05-08T23:23:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.670+0000 7f0ebb4d6b80 1 -- start start 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a7933370 0x5571a7932300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795d3d0 0x5571a7935d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 0x5571a7936260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a795e540 con 0x5571a795d3d0 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a795a870 con 0x5571a7933370 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571a7953b10 con 0x5571a795ffc0 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0eac8b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795d3d0 0x5571a7935d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0eac8b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795d3d0 0x5571a7935d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38300/0 (socket says 172.21.15.5:38300) 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.671+0000 7f0eac8b7700 1 -- 172.21.15.5:0/2650942678 learned_addr learned my addr 172.21.15.5:0/2650942678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a7933370 0x5571a7932300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 0x5571a7936260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0eac8b7700 1 -- 172.21.15.5:0/2650942678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 msgr2=0x5571a7936260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 0x5571a7936260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0eac8b7700 1 -- 172.21.15.5:0/2650942678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a7933370 msgr2=0x5571a7932300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a7933370 0x5571a7932300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0eac8b7700 1 -- 172.21.15.5:0/2650942678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a784b560 con 0x5571a795d3d0 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795d3d0 0x5571a7935d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c010 tx=0x7f0e9c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08855382b06fc0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0e937fe700 1 -- 172.21.15.5:0/2650942678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c02c040 con 0x5571a795d3d0 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0e937fe700 1 -- 172.21.15.5:0/2650942678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c0044d0 con 0x5571a795d3d0 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.672+0000 7f0e937fe700 1 -- 172.21.15.5:0/2650942678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c01f070 con 0x5571a795d3d0 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/2650942678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795d3d0 msgr2=0x5571a7935d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795d3d0 0x5571a7935d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c010 tx=0x7f0e9c011000 comp rx=0 tx=0).stop 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/2650942678 shutdown_connections 2024-05-08T23:23:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 0x5571a7936260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a7933370 0x5571a7932300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/2650942678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a795d3d0 0x5571a7935d20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/2650942678 >> 172.21.15.5:0/2650942678 conn(0x5571a7893210 msgr2=0x5571a7952420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/2650942678 shutdown_connections 2024-05-08T23:23:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/2650942678 wait complete. 2024-05-08T23:23:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 -- start start 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.673+0000 7f0ebb4d6b80 1 Processor -- start 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.674+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a7933370 0x5571a797ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.674+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a795f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.674+0000 7f0ebb4d6b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 0x5571a79eb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.674+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a79ebf50 con 0x5571a7933370 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.674+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a797b7a0 con 0x5571a795d3d0 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.674+0000 7f0ebb4d6b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571a79729b0 con 0x5571a795ffc0 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.674+0000 7f0ebb4d6b80 10 client.0 _ll_register_callbacks cb 0x5571a782ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.674+0000 7f0ead0b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a7933370 0x5571a797ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead0b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a7933370 0x5571a797ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38308/0 (socket says 172.21.15.5:38308) 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3907440085 learned_addr learned my addr 172.21.15.5:0/3907440085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead8b9700 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 0x5571a79eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a795f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 msgr2=0x5571a79eb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 0x5571a79eb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 msgr2=0x5571a795f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a795f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead0b8700 1 -- 172.21.15.5:0/3907440085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea80cd2d0 con 0x5571a7933370 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ead0b8700 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a7933370 0x5571a797ee00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ea406f170 tx=0x7f0ea400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87795444bf33bfce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0e87fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0e87fff700 1 -- 172.21.15.5:0/3907440085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4086040 con 0x5571a7933370 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0e87fff700 1 -- 172.21.15.5:0/3907440085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ea40575f0 con 0x5571a7933370 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3907440085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ea405b420 con 0x5571a7933370 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0e87fff700 1 -- 172.21.15.5:0/3907440085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ea40760b0 con 0x5571a7933370 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ebb4d6b80 10 client.5529 fetch_fsmap learned FSMap version 23 2024-05-08T23:23:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.675+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3907440085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5571a770acd0 con 0x5571a7933370 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0e87fff700 1 -- 172.21.15.5:0/3907440085 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea405b420 con 0x5571a7933370 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0e87fff700 1 -- 172.21.15.5:0/3907440085 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f0ea4086bf0 con 0x5571a7933370 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0ebb4d6b80 10 client.5529 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0ebb4d6b80 10 client.5529 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0ebb4d6b80 20 client.5529 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5571a79e7730 0x5571a79eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0ebb4d6b80 4 client.5529 mds_command: new command op to 4975 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3907440085 --> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5571a787dec0 con 0x5571a79e7730 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.676+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5571a79e7730 0x5571a79eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0eac8b7700 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5571a79e7730 0x5571a79eb6d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a61a490178b506 server_cookie=d64fae6cb94d9ed4 in_seq=0 out_seq=0 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0e87fff700 10 client.5529 ms_handle_connect on v2:172.21.15.49:6835/886789668 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0e87fff700 1 -- 172.21.15.5:0/3907440085 <== mds.0 v2:172.21.15.49:6835/886789668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5571a787dec0 con 0x5571a79e7730 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0e87fff700 10 client.5529 handle_command_reply: tid=0 2024-05-08T23:23:13.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 client.5529 shutdown 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3907440085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a7933370 msgr2=0x5571a797ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a7933370 0x5571a797ee00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ea406f170 tx=0x7f0ea400cf90 comp rx=0 tx=0).stop 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3907440085 shutdown_connections 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:6835/886789668,v1:172.21.15.49:6837/886789668] conn(0x5571a79e7730 0x5571a79eb6d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5571a795ffc0 0x5571a79eb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a7933370 0x5571a797ee00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0e87fff700 0 client.5529 ms_handle_reset on v2:172.21.15.49:6835/886789668 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 --2- 172.21.15.5:0/3907440085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571a795d3d0 0x5571a795f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3907440085 >> 172.21.15.5:0/3907440085 conn(0x5571a7893210 msgr2=0x5571a792fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3907440085 shutdown_connections 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 1 -- 172.21.15.5:0/3907440085 wait complete. 2024-05-08T23:23:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.677+0000 7f0ebb4d6b80 20 client.5529 trim_cache size 0 max 16384 2024-05-08T23:23:13.683 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:23:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-08T23:23:13.683 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-08T23:23:13.683 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 -- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e8c06acc0 msgr2=0x7f0e8c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e8c06acc0 0x7f0e8c06d180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c014040 tx=0x7f0e9c008030 comp rx=0 tx=0).stop 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 -- 172.21.15.5:0/3675225682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80d1740 msgr2=0x7f0ea81f4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80d1740 0x7f0ea81f4780 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c510 tx=0x7f0ea400c820 comp rx=0 tx=0).stop 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 -- 172.21.15.5:0/3675225682 shutdown_connections 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e8c06acc0 0x7f0e8c06d180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ea815b8c0 0x7f0ea81f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80d1740 0x7f0ea81f4780 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 --2- 172.21.15.5:0/3675225682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea815b4e0 0x7f0ea81f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 -- 172.21.15.5:0/3675225682 >> 172.21.15.5:0/3675225682 conn(0x7f0ea808afe0 msgr2=0x7f0ea8155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 -- 172.21.15.5:0/3675225682 shutdown_connections 2024-05-08T23:23:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.678+0000 7f0eaf31c700 1 -- 172.21.15.5:0/3675225682 wait complete. 2024-05-08T23:23:13.691 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:23:13.692 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000002.00000000 scrub_tag 2024-05-08T23:23:13.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.740+0000 7ff123dfe080 1 Processor -- start 2024-05-08T23:23:13.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.740+0000 7ff123dfe080 1 -- start start 2024-05-08T23:23:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff123dfe080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 0x55e32b1b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff123dfe080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b12b0c0 0x55e32b133980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff123dfe080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b1bc740 0x55e32b133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff123dfe080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32b0911a0 con 0x55e32b12b0c0 2024-05-08T23:23:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff123dfe080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32b02c6d0 con 0x55e32b1bc740 2024-05-08T23:23:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff123dfe080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e32b02c370 con 0x55e32b0ed820 2024-05-08T23:23:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff1196cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b12b0c0 0x55e32b133980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff1196cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b12b0c0 0x55e32b133980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38316/0 (socket says 172.21.15.5:38316) 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff1196cf700 1 -- 172.21.15.5:0/1845165580 learned_addr learned my addr 172.21.15.5:0/1845165580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff119ed0700 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 0x55e32b1b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff11a6d1700 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b1bc740 0x55e32b133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff1196cf700 1 -- 172.21.15.5:0/1845165580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 msgr2=0x55e32b1b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff1196cf700 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 0x55e32b1b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff1196cf700 1 -- 172.21.15.5:0/1845165580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b1bc740 msgr2=0x55e32b133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff1196cf700 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b1bc740 0x55e32b133ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.741+0000 7ff1196cf700 1 -- 172.21.15.5:0/1845165580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c005de0 con 0x55e32b12b0c0 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff1196cf700 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b12b0c0 0x55e32b133980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff110001d80 tx=0x7ff11000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789c02e3b29545de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff10bfff700 1 -- 172.21.15.5:0/1845165580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110014070 con 0x55e32b12b0c0 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff10bfff700 1 -- 172.21.15.5:0/1845165580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110008030 con 0x55e32b12b0c0 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff10bfff700 1 -- 172.21.15.5:0/1845165580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11000bc80 con 0x55e32b12b0c0 2024-05-08T23:23:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 -- 172.21.15.5:0/1845165580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b12b0c0 msgr2=0x55e32b133980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b12b0c0 0x55e32b133980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff110001d80 tx=0x7ff11000b4c0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 -- 172.21.15.5:0/1845165580 shutdown_connections 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b1bc740 0x55e32b133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 0x55e32b1b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 --2- 172.21.15.5:0/1845165580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b12b0c0 0x55e32b133980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 -- 172.21.15.5:0/1845165580 >> 172.21.15.5:0/1845165580 conn(0x55e32b116d60 msgr2=0x55e32b1190a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 -- 172.21.15.5:0/1845165580 shutdown_connections 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.742+0000 7ff123dfe080 1 -- 172.21.15.5:0/1845165580 wait complete. 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.743+0000 7ff123dfe080 1 Processor -- start 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.743+0000 7ff123dfe080 1 -- start start 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff123dfe080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 0x55e32b119e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff123dfe080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b12b0c0 0x55e32b11a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff123dfe080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b1bc740 0x55e32b258330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff123dfe080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32b023680 con 0x55e32b1bc740 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff123dfe080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32b1b0160 con 0x55e32b12b0c0 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff123dfe080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e32b259b60 con 0x55e32b0ed820 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b1bc740 0x55e32b258330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b1bc740 0x55e32b258330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38322/0 (socket says 172.21.15.5:38322) 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 -- 172.21.15.5:0/2148795434 learned_addr learned my addr 172.21.15.5:0/2148795434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff119ed0700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 0x55e32b119e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff1196cf700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b12b0c0 0x55e32b11a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 -- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 msgr2=0x55e32b119e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e32b0ed820 0x55e32b119e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 -- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b12b0c0 msgr2=0x55e32b11a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e32b12b0c0 0x55e32b11a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 -- 172.21.15.5:0/2148795434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11000a040 con 0x55e32b1bc740 2024-05-08T23:23:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.744+0000 7ff11a6d1700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b1bc740 0x55e32b258330 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff1140108e0 tx=0x7ff114010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.745+0000 7ff10affd700 1 -- 172.21.15.5:0/2148795434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114012670 con 0x55e32b1bc740 2024-05-08T23:23:13.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.745+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e32b03b8a0 con 0x55e32b1bc740 2024-05-08T23:23:13.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.745+0000 7ff10affd700 1 -- 172.21.15.5:0/2148795434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1140127f0 con 0x55e32b1bc740 2024-05-08T23:23:13.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.745+0000 7ff10affd700 1 -- 172.21.15.5:0/2148795434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114012ba0 con 0x55e32b1bc740 2024-05-08T23:23:13.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.745+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e32b02eed0 con 0x55e32b1bc740 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.746+0000 7ff10affd700 1 -- 172.21.15.5:0/2148795434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff114003040 con 0x55e32b1bc740 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.746+0000 7ff10affd700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0fc0695e0 0x7ff0fc06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.747+0000 7ff10affd700 1 -- 172.21.15.5:0/2148795434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff114017030 con 0x55e32b1bc740 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.747+0000 7ff119ed0700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0fc0695e0 0x7ff0fc06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.747+0000 7ff123dfe080 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x55e32b25b000 0x55e32b258d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.747+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 --> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] -- osd_op(unknown.0.0:1 4.12 4:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x55e32b0d8820 con 0x55e32b25b000 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.747+0000 7ff1196cf700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x55e32b25b000 0x55e32b258d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.747+0000 7ff119ed0700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0fc0695e0 0x7ff0fc06baa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00a4f0 tx=0x7ff10c00f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.747+0000 7ff1196cf700 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x55e32b25b000 0x55e32b258d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff1196cf700 1 -- 172.21.15.5:0/2148795434 <== osd.1 v2:172.21.15.5:6814/47133 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (10) out=10b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7ff110016070 con 0x55e32b25b000 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x55e32b25b000 msgr2=0x55e32b258d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x55e32b25b000 0x55e32b258d80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0fc0695e0 msgr2=0x7ff0fc06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0fc0695e0 0x7ff0fc06baa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00a4f0 tx=0x7ff10c00f000 comp rx=0 tx=0).stop 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b1bc740 msgr2=0x55e32b258330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 --2- 172.21.15.5:0/2148795434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e32b1bc740 0x55e32b258330 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff1140108e0 tx=0x7ff114010bf0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff11a6d1700 1 -- 172.21.15.5:0/2148795434 reap_dead start 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 shutdown_connections 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 >> 172.21.15.5:0/2148795434 conn(0x55e32b116d60 msgr2=0x55e32b130570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 shutdown_connections 2024-05-08T23:23:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.748+0000 7ff123dfe080 1 -- 172.21.15.5:0/2148795434 wait complete. 2024-05-08T23:23:13.753 INFO:teuthology.orchestra.run.smithi005.stdout:tag123 2024-05-08T23:23:13.753 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000003.00000000 scrub_tag 2024-05-08T23:23:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.802+0000 7f9e25465080 1 Processor -- start 2024-05-08T23:23:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e25465080 1 -- start start 2024-05-08T23:23:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e25465080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1e9f6fa0 0x563a1eb56560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e25465080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1ea25610 0x563a1ea3b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e25465080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea3d310 0x563a1eb539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e25465080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a1e99c1a0 con 0x563a1e9f6fa0 2024-05-08T23:23:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e25465080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a1e9376d0 con 0x563a1ea3d310 2024-05-08T23:23:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e25465080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a1e937370 con 0x563a1ea25610 2024-05-08T23:23:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e1b537700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1e9f6fa0 0x563a1eb56560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e1b537700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1e9f6fa0 0x563a1eb56560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38352/0 (socket says 172.21.15.5:38352) 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.803+0000 7f9e1b537700 1 -- 172.21.15.5:0/4267731940 learned_addr learned my addr 172.21.15.5:0/4267731940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1bd38700 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea3d310 0x563a1eb539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1ad36700 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1ea25610 0x563a1ea3b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1b537700 1 -- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1ea25610 msgr2=0x563a1ea3b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1b537700 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1ea25610 0x563a1ea3b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1b537700 1 -- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea3d310 msgr2=0x563a1eb539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1b537700 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea3d310 0x563a1eb539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1b537700 1 -- 172.21.15.5:0/4267731940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1400a1c0 con 0x563a1e9f6fa0 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1ad36700 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1ea25610 0x563a1ea3b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e1b537700 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1e9f6fa0 0x563a1eb56560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00b240 tx=0x7f9e0c00b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f08f8e42ff6580b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e19d34700 1 -- 172.21.15.5:0/4267731940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c014070 con 0x563a1e9f6fa0 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e19d34700 1 -- 172.21.15.5:0/4267731940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c00be60 con 0x563a1e9f6fa0 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.804+0000 7f9e19d34700 1 -- 172.21.15.5:0/4267731940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00f5f0 con 0x563a1e9f6fa0 2024-05-08T23:23:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 -- 172.21.15.5:0/4267731940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1e9f6fa0 msgr2=0x563a1eb56560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1e9f6fa0 0x563a1eb56560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00b240 tx=0x7f9e0c00b550 comp rx=0 tx=0).stop 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 -- 172.21.15.5:0/4267731940 shutdown_connections 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea3d310 0x563a1eb539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1e9f6fa0 0x563a1eb56560 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 --2- 172.21.15.5:0/4267731940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1ea25610 0x563a1ea3b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 -- 172.21.15.5:0/4267731940 >> 172.21.15.5:0/4267731940 conn(0x563a1e9f5ba0 msgr2=0x563a1e9f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 -- 172.21.15.5:0/4267731940 shutdown_connections 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.805+0000 7f9e25465080 1 -- 172.21.15.5:0/4267731940 wait complete. 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.806+0000 7f9e25465080 1 Processor -- start 2024-05-08T23:23:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.806+0000 7f9e25465080 1 -- start start 2024-05-08T23:23:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e25465080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1e9f6fa0 0x563a1ea18db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e25465080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea25610 0x563a1eac3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e25465080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1ea3d310 0x563a1ea176b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e25465080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a1e92e680 con 0x563a1ea3d310 2024-05-08T23:23:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e25465080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a1ea3bb10 con 0x563a1ea25610 2024-05-08T23:23:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e25465080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a1ea1eda0 con 0x563a1e9f6fa0 2024-05-08T23:23:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1ea3d310 0x563a1ea176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1ea3d310 0x563a1ea176b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38360/0 (socket says 172.21.15.5:38360) 2024-05-08T23:23:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 -- 172.21.15.5:0/2789405708 learned_addr learned my addr 172.21.15.5:0/2789405708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1b537700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1e9f6fa0 0x563a1ea18db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1ad36700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea25610 0x563a1eac3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 -- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1e9f6fa0 msgr2=0x563a1ea18db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a1e9f6fa0 0x563a1ea18db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 -- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea25610 msgr2=0x563a1eac3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a1ea25610 0x563a1eac3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 -- 172.21.15.5:0/2789405708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c001b40 con 0x563a1ea3d310 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e1bd38700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1ea3d310 0x563a1ea176b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9e14013040 tx=0x7f9e14003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.807+0000 7f9e18d32700 1 -- 172.21.15.5:0/2789405708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14010b80 con 0x563a1ea3d310 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.808+0000 7f9e18d32700 1 -- 172.21.15.5:0/2789405708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e14009030 con 0x563a1ea3d310 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.808+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a1e9468a0 con 0x563a1ea3d310 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.808+0000 7f9e18d32700 1 -- 172.21.15.5:0/2789405708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400ee80 con 0x563a1ea3d310 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.808+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a1e939ed0 con 0x563a1ea3d310 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.809+0000 7f9e18d32700 1 -- 172.21.15.5:0/2789405708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9e14024410 con 0x563a1ea3d310 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.809+0000 7f9e18d32700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e080695e0 0x7f9e0806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.810+0000 7f9e18d32700 1 -- 172.21.15.5:0/2789405708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e14089c00 con 0x563a1ea3d310 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.810+0000 7f9e1b537700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e080695e0 0x7f9e0806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.810+0000 7f9e25465080 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x563a1eac38a0 0x563a1ea3f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.810+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 --> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] -- osd_op(unknown.0.0:1 4.12 4:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x563a1ea1a590 con 0x563a1eac38a0 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.810+0000 7f9e1ad36700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x563a1eac38a0 0x563a1ea3f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.810+0000 7f9e1b537700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e080695e0 0x7f9e0806baa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00b240 tx=0x7f9e0c011040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.810+0000 7f9e1ad36700 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x563a1eac38a0 0x563a1ea3f180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e1ad36700 1 -- 172.21.15.5:0/2789405708 <== osd.1 v2:172.21.15.5:6814/47133 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr (10) out=10b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f9e10009990 con 0x563a1eac38a0 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x563a1eac38a0 msgr2=0x563a1ea3f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x563a1eac38a0 0x563a1ea3f180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e080695e0 msgr2=0x7f9e0806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e080695e0 0x7f9e0806baa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00b240 tx=0x7f9e0c011040 comp rx=0 tx=0).stop 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1ea3d310 msgr2=0x563a1ea176b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 --2- 172.21.15.5:0/2789405708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a1ea3d310 0x563a1ea176b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9e14013040 tx=0x7f9e14003330 comp rx=0 tx=0).stop 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e1bd38700 1 -- 172.21.15.5:0/2789405708 reap_dead start 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 shutdown_connections 2024-05-08T23:23:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 >> 172.21.15.5:0/2789405708 conn(0x563a1e9f5ba0 msgr2=0x563a1e9df6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 shutdown_connections 2024-05-08T23:23:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.811+0000 7f9e25465080 1 -- 172.21.15.5:0/2789405708 wait complete. 2024-05-08T23:23:13.817 INFO:teuthology.orchestra.run.smithi005.stdout:tag123 2024-05-08T23:23:13.818 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000004.00000000 scrub_tag 2024-05-08T23:23:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.867+0000 7f8c5737e080 1 Processor -- start 2024-05-08T23:23:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.867+0000 7f8c5737e080 1 -- start start 2024-05-08T23:23:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.867+0000 7f8c5737e080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48be68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c5737e080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 0x55ac48b12cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c5737e080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 0x55ac48be5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c5737e080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac48abb1a0 con 0x55ac48b15780 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c5737e080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac48a566d0 con 0x55ac48be9f30 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c5737e080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac48a56370 con 0x55ac48b5e2c0 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48be68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48be68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38384/0 (socket says 172.21.15.5:38384) 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 -- 172.21.15.5:0/1173137281 learned_addr learned my addr 172.21.15.5:0/1173137281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4cc4f700 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 0x55ac48b12cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 -- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 msgr2=0x55ac48b12cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 0x55ac48b12cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4dc51700 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 0x55ac48be5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 -- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 msgr2=0x55ac48be5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 0x55ac48be5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 -- 172.21.15.5:0/1173137281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44005de0 con 0x55ac48b15780 2024-05-08T23:23:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4dc51700 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 0x55ac48be5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c4d450700 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48be68d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000a7a0 tx=0x7f8c4000aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15087e2bed0553f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c3f7fe700 1 -- 172.21.15.5:0/1173137281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000f070 con 0x55ac48b15780 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c3f7fe700 1 -- 172.21.15.5:0/1173137281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c400084c0 con 0x55ac48b15780 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.868+0000 7f8c3f7fe700 1 -- 172.21.15.5:0/1173137281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40008830 con 0x55ac48b15780 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 -- 172.21.15.5:0/1173137281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 msgr2=0x55ac48be68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48be68d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000a7a0 tx=0x7f8c4000aab0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 -- 172.21.15.5:0/1173137281 shutdown_connections 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 0x55ac48be5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48be68d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 --2- 172.21.15.5:0/1173137281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 0x55ac48b12cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 -- 172.21.15.5:0/1173137281 >> 172.21.15.5:0/1173137281 conn(0x55ac48b3f2d0 msgr2=0x55ac48be9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 -- 172.21.15.5:0/1173137281 shutdown_connections 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.869+0000 7f8c5737e080 1 -- 172.21.15.5:0/1173137281 wait complete. 2024-05-08T23:23:13.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.870+0000 7f8c5737e080 1 Processor -- start 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.870+0000 7f8c5737e080 1 -- start start 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.870+0000 7f8c5737e080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48b5e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.870+0000 7f8c5737e080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 0x55ac48be16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.870+0000 7f8c5737e080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 0x55ac48beb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.870+0000 7f8c5737e080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac48a4d680 con 0x55ac48b15780 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.870+0000 7f8c5737e080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac48b380f0 con 0x55ac48be9f30 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.870+0000 7f8c5737e080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac48b38390 con 0x55ac48b5e2c0 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48b5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48b5e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38386/0 (socket says 172.21.15.5:38386) 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 -- 172.21.15.5:0/3983432701 learned_addr learned my addr 172.21.15.5:0/3983432701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4cc4f700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 0x55ac48be16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4dc51700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 0x55ac48beb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 -- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 msgr2=0x55ac48be16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac48b5e2c0 0x55ac48be16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 -- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 msgr2=0x55ac48beb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac48be9f30 0x55ac48beb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 -- 172.21.15.5:0/3983432701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4000a420 con 0x55ac48b15780 2024-05-08T23:23:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c4d450700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48b5e750 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c40018040 tx=0x7f8c40003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c3e7fc700 1 -- 172.21.15.5:0/3983432701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40008ec0 con 0x55ac48b15780 2024-05-08T23:23:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c3e7fc700 1 -- 172.21.15.5:0/3983432701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c40005370 con 0x55ac48b15780 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac48a658a0 con 0x55ac48b15780 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c3e7fc700 1 -- 172.21.15.5:0/3983432701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40020860 con 0x55ac48b15780 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.871+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac48a58ed0 con 0x55ac48b15780 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.873+0000 7f8c3e7fc700 1 -- 172.21.15.5:0/3983432701 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8c4001c070 con 0x55ac48b15780 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.873+0000 7f8c3e7fc700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c30069610 0x7f8c3006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.873+0000 7f8c3e7fc700 1 -- 172.21.15.5:0/3983432701 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c400890d0 con 0x55ac48b15780 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.873+0000 7f8c4cc4f700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c30069610 0x7f8c3006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.873+0000 7f8c5737e080 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55ac48b5d760 0x55ac48be5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.874+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 4.5 4:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x55ac48be60e0 con 0x55ac48b5d760 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.874+0000 7f8c4dc51700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55ac48b5d760 0x55ac48be5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.874+0000 7f8c4cc4f700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c30069610 0x7f8c3006bad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55ac48b02ce0 tx=0x7f8c4400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.874+0000 7f8c4dc51700 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55ac48b5d760 0x55ac48be5ba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c4dc51700 1 -- 172.21.15.5:0/3983432701 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f8c48001540 con 0x55ac48b5d760 2024-05-08T23:23:13.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55ac48b5d760 msgr2=0x55ac48be5ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55ac48b5d760 0x55ac48be5ba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c30069610 msgr2=0x7f8c3006bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c30069610 0x7f8c3006bad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55ac48b02ce0 tx=0x7f8c4400f000 comp rx=0 tx=0).stop 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 msgr2=0x55ac48b5e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 --2- 172.21.15.5:0/3983432701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac48b15780 0x55ac48b5e750 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c40018040 tx=0x7f8c40003020 comp rx=0 tx=0).stop 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c4dc51700 1 -- 172.21.15.5:0/3983432701 reap_dead start 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 shutdown_connections 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 >> 172.21.15.5:0/3983432701 conn(0x55ac48b3f2d0 msgr2=0x55ac48be9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 shutdown_connections 2024-05-08T23:23:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.875+0000 7f8c5737e080 1 -- 172.21.15.5:0/3983432701 wait complete. 2024-05-08T23:23:13.883 INFO:teuthology.orchestra.run.smithi005.stdout:tag123 2024-05-08T23:23:13.883 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000005.00000000 scrub_tag 2024-05-08T23:23:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.933+0000 7f4aaa06b080 1 Processor -- start 2024-05-08T23:23:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.933+0000 7f4aaa06b080 1 -- start start 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aaa06b080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330addd0 0x55a6330f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aaa06b080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330dc610 0x55a6330aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aaa06b080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330f4310 0x55a6330f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aaa06b080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a632fee6d0 con 0x55a6330dc610 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aaa06b080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a632fee370 con 0x55a6330f4310 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aaa06b080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a632fe5680 con 0x55a6330addd0 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330dc610 0x55a6330aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330dc610 0x55a6330aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38394/0 (socket says 172.21.15.5:38394) 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 -- 172.21.15.5:0/506993405 learned_addr learned my addr 172.21.15.5:0/506993405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aa093e700 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330f4310 0x55a6330f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 -- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330addd0 msgr2=0x55a6330f0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aa013d700 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330addd0 0x55a6330f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330addd0 0x55a6330f0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 -- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330f4310 msgr2=0x55a6330f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330f4310 0x55a6330f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 -- 172.21.15.5:0/506993405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a90005de0 con 0x55a6330dc610 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4aa093e700 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330f4310 0x55a6330f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.934+0000 7f4a9f93c700 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330dc610 0x55a6330aeaf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4a880098b0 tx=0x7f4a88005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2fca5e8e623ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4a9e93a700 1 -- 172.21.15.5:0/506993405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8800f040 con 0x55a6330dc610 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4a9e93a700 1 -- 172.21.15.5:0/506993405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a88008a30 con 0x55a6330dc610 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4a9e93a700 1 -- 172.21.15.5:0/506993405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88008de0 con 0x55a6330dc610 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 -- 172.21.15.5:0/506993405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330dc610 msgr2=0x55a6330aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330dc610 0x55a6330aeaf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4a880098b0 tx=0x7f4a88005a00 comp rx=0 tx=0).stop 2024-05-08T23:23:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 -- 172.21.15.5:0/506993405 shutdown_connections 2024-05-08T23:23:13.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330f4310 0x55a6330f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330addd0 0x55a6330f0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 --2- 172.21.15.5:0/506993405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330dc610 0x55a6330aeaf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 -- 172.21.15.5:0/506993405 >> 172.21.15.5:0/506993405 conn(0x55a6330acba0 msgr2=0x55a6330d8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 -- 172.21.15.5:0/506993405 shutdown_connections 2024-05-08T23:23:13.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.935+0000 7f4aaa06b080 1 -- 172.21.15.5:0/506993405 wait complete. 2024-05-08T23:23:13.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.936+0000 7f4aaa06b080 1 Processor -- start 2024-05-08T23:23:13.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.936+0000 7f4aaa06b080 1 -- start start 2024-05-08T23:23:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aaa06b080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330addd0 0x55a633173b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aaa06b080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330dc610 0x55a6330f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aaa06b080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330f4310 0x55a6330f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aaa06b080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a633178ed0 con 0x55a6330addd0 2024-05-08T23:23:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aaa06b080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a633174080 con 0x55a6330dc610 2024-05-08T23:23:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aaa06b080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a633174f00 con 0x55a6330f4310 2024-05-08T23:23:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330addd0 0x55a633173b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330addd0 0x55a633173b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38406/0 (socket says 172.21.15.5:38406) 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 -- 172.21.15.5:0/1341256714 learned_addr learned my addr 172.21.15.5:0/1341256714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4a9f93c700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330dc610 0x55a6330f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa093e700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330f4310 0x55a6330f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 -- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330f4310 msgr2=0x55a6330f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a6330f4310 0x55a6330f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 -- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330dc610 msgr2=0x55a6330f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6330dc610 0x55a6330f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 -- 172.21.15.5:0/1341256714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a88009510 con 0x55a6330addd0 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.937+0000 7f4aa013d700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330addd0 0x55a633173b40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4a90015040 tx=0x7f4a90002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.938+0000 7f4a9d938700 1 -- 172.21.15.5:0/1341256714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90008690 con 0x55a6330addd0 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.938+0000 7f4a9d938700 1 -- 172.21.15.5:0/1341256714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9001c620 con 0x55a6330addd0 2024-05-08T23:23:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.938+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a633017a40 con 0x55a6330addd0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.938+0000 7f4a9d938700 1 -- 172.21.15.5:0/1341256714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9001c9b0 con 0x55a6330addd0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.938+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a632ffd8a0 con 0x55a6330addd0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.939+0000 7f4a9d938700 1 -- 172.21.15.5:0/1341256714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a90019070 con 0x55a6330addd0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.939+0000 7f4a9d938700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a800695e0 0x7f4a8006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.940+0000 7f4a9d938700 1 -- 172.21.15.5:0/1341256714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a90088fe0 con 0x55a6330addd0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.940+0000 7f4a9f93c700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a800695e0 0x7f4a8006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.940+0000 7f4aaa06b080 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x55a63320ba20 0x55a63320be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.940+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 --> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] -- osd_op(unknown.0.0:1 4.9 4:9258b67d:::10000000005.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x55a63320c3c0 con 0x55a63320ba20 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.940+0000 7f4aa093e700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x55a63320ba20 0x55a63320be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.940+0000 7f4a9f93c700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a800695e0 0x7f4a8006baa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55a633179090 tx=0x7f4a8801f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.940+0000 7f4aa093e700 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x55a63320ba20 0x55a63320be80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aa093e700 1 -- 172.21.15.5:0/1341256714 <== osd.4 v2:172.21.15.49:6816/47400 1 ==== osd_op_reply(1 10000000005.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f4a98010640 con 0x55a63320ba20 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x55a63320ba20 msgr2=0x55a63320be80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x55a63320ba20 0x55a63320be80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a800695e0 msgr2=0x7f4a8006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a800695e0 0x7f4a8006baa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55a633179090 tx=0x7f4a8801f040 comp rx=0 tx=0).stop 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330addd0 msgr2=0x55a633173b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 --2- 172.21.15.5:0/1341256714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6330addd0 0x55a633173b40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4a90015040 tx=0x7f4a90002fb0 comp rx=0 tx=0).stop 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aa093e700 1 -- 172.21.15.5:0/1341256714 reap_dead start 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 shutdown_connections 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 >> 172.21.15.5:0/1341256714 conn(0x55a6330acba0 msgr2=0x55a6330f0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 shutdown_connections 2024-05-08T23:23:13.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.941+0000 7f4aaa06b080 1 -- 172.21.15.5:0/1341256714 wait complete. 2024-05-08T23:23:13.946 INFO:teuthology.orchestra.run.smithi005.stdout:tag123 2024-05-08T23:23:13.946 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000006.00000000 scrub_tag 2024-05-08T23:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.998+0000 7f16b3f9d080 1 Processor -- start 2024-05-08T23:23:13.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.998+0000 7f16b3f9d080 1 -- start start 2024-05-08T23:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16b3f9d080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca8b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16b3f9d080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca149c0 0x56325ca851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16b3f9d080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca8a620 0x56325ca85730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16b3f9d080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325c9741a0 con 0x56325c9f7be0 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16b3f9d080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325c90f6d0 con 0x56325ca8a620 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16b3f9d080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56325c90f370 con 0x56325ca149c0 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16aa06f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca8b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16aa06f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca8b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38424/0 (socket says 172.21.15.5:38424) 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16aa06f700 1 -- 172.21.15.5:0/3111365951 learned_addr learned my addr 172.21.15.5:0/3111365951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16aa870700 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca8a620 0x56325ca85730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:13.999+0000 7f16a986e700 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca149c0 0x56325ca851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16aa06f700 1 -- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca149c0 msgr2=0x56325ca851f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16aa06f700 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca149c0 0x56325ca851f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16aa06f700 1 -- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca8a620 msgr2=0x56325ca85730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16aa06f700 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca8a620 0x56325ca85730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16aa06f700 1 -- 172.21.15.5:0/3111365951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a400a1c0 con 0x56325c9f7be0 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16a986e700 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca149c0 0x56325ca851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16aa06f700 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca8b550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f169c009760 tx=0x7f169c009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3ded9bc67ee642 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16a886c700 1 -- 172.21.15.5:0/3111365951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00e070 con 0x56325c9f7be0 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16a886c700 1 -- 172.21.15.5:0/3111365951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c006440 con 0x56325c9f7be0 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.000+0000 7f16a886c700 1 -- 172.21.15.5:0/3111365951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0067b0 con 0x56325c9f7be0 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 -- 172.21.15.5:0/3111365951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 msgr2=0x56325ca8b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca8b550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f169c009760 tx=0x7f169c009a70 comp rx=0 tx=0).stop 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 -- 172.21.15.5:0/3111365951 shutdown_connections 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca8a620 0x56325ca85730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca8b550 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 --2- 172.21.15.5:0/3111365951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca149c0 0x56325ca851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 -- 172.21.15.5:0/3111365951 >> 172.21.15.5:0/3111365951 conn(0x56325c9f9d60 msgr2=0x56325ca89e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 -- 172.21.15.5:0/3111365951 shutdown_connections 2024-05-08T23:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.001+0000 7f16b3f9d080 1 -- 172.21.15.5:0/3111365951 wait complete. 2024-05-08T23:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.002+0000 7f16b3f9d080 1 Processor -- start 2024-05-08T23:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.002+0000 7f16b3f9d080 1 -- start start 2024-05-08T23:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16b3f9d080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca81740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16b3f9d080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca149c0 0x56325ca8b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16b3f9d080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca8a620 0x56325ca8ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16b3f9d080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325c906680 con 0x56325c9f7be0 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16b3f9d080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325caa2e00 con 0x56325ca149c0 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16b3f9d080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56325c9cf710 con 0x56325ca8a620 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca81740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca81740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38436/0 (socket says 172.21.15.5:38436) 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 -- 172.21.15.5:0/2874147099 learned_addr learned my addr 172.21.15.5:0/2874147099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16a986e700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca149c0 0x56325ca8b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa870700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca8a620 0x56325ca8ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 -- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca8a620 msgr2=0x56325ca8ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca8a620 0x56325ca8ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 -- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca149c0 msgr2=0x56325ca8b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56325ca149c0 0x56325ca8b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 -- 172.21.15.5:0/2874147099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c009410 con 0x56325c9f7be0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa870700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56325ca8a620 0x56325ca8ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f16aa06f700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca81740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f169c000c00 tx=0x7f169c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.003+0000 7f169b7fe700 1 -- 172.21.15.5:0/2874147099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c005d80 con 0x56325c9f7be0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.004+0000 7f169b7fe700 1 -- 172.21.15.5:0/2874147099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c01c440 con 0x56325c9f7be0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.004+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56325c91e8a0 con 0x56325c9f7be0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.004+0000 7f169b7fe700 1 -- 172.21.15.5:0/2874147099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c01c7f0 con 0x56325c9f7be0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.004+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56325c911ed0 con 0x56325c9f7be0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.005+0000 7f169b7fe700 1 -- 172.21.15.5:0/2874147099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f169c01cb60 con 0x56325c9f7be0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.005+0000 7f169b7fe700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f168c0695e0 0x7f168c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.005+0000 7f169b7fe700 1 -- 172.21.15.5:0/2874147099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f169c088eb0 con 0x56325c9f7be0 2024-05-08T23:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.006+0000 7f16b3f9d080 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x56325c9cca30 0x56325c9cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.006+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 4.5 4:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x56325c9cd3b0 con 0x56325c9cca30 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.006+0000 7f16a986e700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f168c0695e0 0x7f168c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.006+0000 7f16aa870700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x56325c9cca30 0x56325c9cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.006+0000 7f16a986e700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f168c0695e0 0x7f168c06baa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x56325ca84da0 tx=0x7f16a0003630 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.006+0000 7f16aa870700 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x56325c9cca30 0x56325c9cce70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16aa870700 1 -- 172.21.15.5:0/2874147099 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f16a400eaa0 con 0x56325c9cca30 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x56325c9cca30 msgr2=0x56325c9cce70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x56325c9cca30 0x56325c9cce70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f168c0695e0 msgr2=0x7f168c06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f168c0695e0 0x7f168c06baa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x56325ca84da0 tx=0x7f16a0003630 comp rx=0 tx=0).stop 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 msgr2=0x56325ca81740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 --2- 172.21.15.5:0/2874147099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325c9f7be0 0x56325ca81740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f169c000c00 tx=0x7f169c003020 comp rx=0 tx=0).stop 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16aa870700 1 -- 172.21.15.5:0/2874147099 reap_dead start 2024-05-08T23:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 shutdown_connections 2024-05-08T23:23:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 >> 172.21.15.5:0/2874147099 conn(0x56325c9f9d60 msgr2=0x56325caa2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 shutdown_connections 2024-05-08T23:23:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.007+0000 7f16b3f9d080 1 -- 172.21.15.5:0/2874147099 wait complete. 2024-05-08T23:23:14.012 INFO:teuthology.orchestra.run.smithi005.stdout:tag123 2024-05-08T23:23:14.012 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000007.00000000 scrub_tag 2024-05-08T23:23:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.061+0000 7f840f67f080 1 Processor -- start 2024-05-08T23:23:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.061+0000 7f840f67f080 1 -- start start 2024-05-08T23:23:14.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f840f67f080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 0x5621587bf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f840f67f080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621587d20c0 0x56215885c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f840f67f080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562158863740 0x5621587db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f840f67f080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621587381a0 con 0x562158863740 2024-05-08T23:23:14.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f840f67f080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621586d36d0 con 0x5621587d20c0 2024-05-08T23:23:14.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f840f67f080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621586d3370 con 0x562158794820 2024-05-08T23:23:14.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405f52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562158863740 0x5621587db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405f52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562158863740 0x5621587db4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38440/0 (socket says 172.21.15.5:38440) 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405f52700 1 -- 172.21.15.5:0/1780373443 learned_addr learned my addr 172.21.15.5:0/1780373443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8404f50700 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621587d20c0 0x56215885c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405751700 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 0x5621587bf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405f52700 1 -- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 msgr2=0x5621587bf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405f52700 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 0x5621587bf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405f52700 1 -- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621587d20c0 msgr2=0x56215885c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405f52700 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621587d20c0 0x56215885c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8405f52700 1 -- 172.21.15.5:0/1780373443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8005d20 con 0x562158863740 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.062+0000 7f8404f50700 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621587d20c0 0x56215885c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f8405f52700 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562158863740 0x5621587db4a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f840000abc0 tx=0x7f840000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7501f6db53e56e37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f83f77fe700 1 -- 172.21.15.5:0/1780373443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000e9e0 con 0x562158863740 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f83f77fe700 1 -- 172.21.15.5:0/1780373443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840000eb60 con 0x562158863740 2024-05-08T23:23:14.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f83f77fe700 1 -- 172.21.15.5:0/1780373443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000ef10 con 0x562158863740 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 -- 172.21.15.5:0/1780373443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562158863740 msgr2=0x5621587db4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562158863740 0x5621587db4a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f840000abc0 tx=0x7f840000e000 comp rx=0 tx=0).stop 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 -- 172.21.15.5:0/1780373443 shutdown_connections 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562158863740 0x5621587db4a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 0x5621587bf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 --2- 172.21.15.5:0/1780373443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5621587d20c0 0x56215885c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 -- 172.21.15.5:0/1780373443 >> 172.21.15.5:0/1780373443 conn(0x5621587bdd60 msgr2=0x562158794110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 -- 172.21.15.5:0/1780373443 shutdown_connections 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.063+0000 7f840f67f080 1 -- 172.21.15.5:0/1780373443 wait complete. 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.064+0000 7f840f67f080 1 Processor -- start 2024-05-08T23:23:14.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.064+0000 7f840f67f080 1 -- start start 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f840f67f080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 0x562158860380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f840f67f080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621587d20c0 0x56215885e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f840f67f080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562158863740 0x56215885eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f840f67f080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621586ca680 con 0x5621587d20c0 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f840f67f080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562158860160 con 0x562158863740 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f840f67f080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621587c13b0 con 0x562158794820 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8404f50700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621587d20c0 0x56215885e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8404f50700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621587d20c0 0x56215885e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38454/0 (socket says 172.21.15.5:38454) 2024-05-08T23:23:14.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8404f50700 1 -- 172.21.15.5:0/3680437119 learned_addr learned my addr 172.21.15.5:0/3680437119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8405f52700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562158863740 0x56215885eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8405751700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 0x562158860380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8404f50700 1 -- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 msgr2=0x562158860380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8404f50700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562158794820 0x562158860380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8404f50700 1 -- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562158863740 msgr2=0x56215885eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8404f50700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562158863740 0x56215885eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.065+0000 7f8404f50700 1 -- 172.21.15.5:0/3680437119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621586fca40 con 0x5621587d20c0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.066+0000 7f8404f50700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621587d20c0 0x56215885e990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00cf40 tx=0x7f83fc00a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.066+0000 7f83f67fc700 1 -- 172.21.15.5:0/3680437119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00aac0 con 0x5621587d20c0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.066+0000 7f83f67fc700 1 -- 172.21.15.5:0/3680437119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc00ac20 con 0x5621587d20c0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.066+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840000a1c0 con 0x5621587d20c0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.066+0000 7f83f67fc700 1 -- 172.21.15.5:0/3680437119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00f400 con 0x5621587d20c0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.066+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621586e28a0 con 0x5621587d20c0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.067+0000 7f83f67fc700 1 -- 172.21.15.5:0/3680437119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f83fc00d040 con 0x5621587d20c0 2024-05-08T23:23:14.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.068+0000 7f83f67fc700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83e80695e0 0x7f83e806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.068+0000 7f83f67fc700 1 -- 172.21.15.5:0/3680437119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83fc087770 con 0x5621587d20c0 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.068+0000 7f840f67f080 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x5621587d6690 0x56215890a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.068+0000 7f8405751700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83e80695e0 0x7f83e806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.068+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 --> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] -- osd_op(unknown.0.0:1 4.1d 4:bbf5850a:::10000000007.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x56215885f900 con 0x5621587d6690 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.068+0000 7f8405f52700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x5621587d6690 0x56215890a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.068+0000 7f8405f52700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x5621587d6690 0x56215890a8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.068+0000 7f8405751700 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83e80695e0 0x7f83e806baa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f83f8005630 tx=0x7f83f8005a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f8405f52700 1 -- 172.21.15.5:0/3680437119 <== osd.3 v2:172.21.15.5:6808/47142 1 ==== osd_op_reply(1 10000000007.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f840002b040 con 0x5621587d6690 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x5621587d6690 msgr2=0x56215890a8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x5621587d6690 0x56215890a8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83e80695e0 msgr2=0x7f83e806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f83e80695e0 0x7f83e806baa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f83f8005630 tx=0x7f83f8005a00 comp rx=0 tx=0).stop 2024-05-08T23:23:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621587d20c0 msgr2=0x56215885e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 --2- 172.21.15.5:0/3680437119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621587d20c0 0x56215885e990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00cf40 tx=0x7f83fc00a4e0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f8405f52700 1 -- 172.21.15.5:0/3680437119 reap_dead start 2024-05-08T23:23:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 shutdown_connections 2024-05-08T23:23:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 >> 172.21.15.5:0/3680437119 conn(0x5621587bdd60 msgr2=0x562158794110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 shutdown_connections 2024-05-08T23:23:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.069+0000 7f840f67f080 1 -- 172.21.15.5:0/3680437119 wait complete. 2024-05-08T23:23:14.075 INFO:teuthology.orchestra.run.smithi005.stdout:tag123 2024-05-08T23:23:14.075 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:23:14.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:23:14.118 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:23:14.169 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:23:14.170 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:23:14.170 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:14.192 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:23:14.192 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:14.255 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:14.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.438+0000 7f9eb29fa700 1 Processor -- start 2024-05-08T23:23:14.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.438+0000 7f9eb29fa700 1 -- start start 2024-05-08T23:23:14.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eb29fa700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9eac161030 0x7f9eac099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eb29fa700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9eac0b2480 0x7f9eac099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eb29fa700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac09a4d0 0x7f9eac0cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eb29fa700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac06ccc0 con 0x7f9eac161030 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eb29fa700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac0d0840 con 0x7f9eac0b2480 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eb29fa700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac06c600 con 0x7f9eac09a4d0 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eabfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9eac161030 0x7f9eac099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eabfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9eac161030 0x7f9eac099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38456/0 (socket says 172.21.15.5:38456) 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eabfff700 1 -- 172.21.15.5:0/3871527974 learned_addr learned my addr 172.21.15.5:0/3871527974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eb0f97700 1 --2- 172.21.15.5:0/3871527974 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac09a4d0 0x7f9eac0cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.439+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/3871527974 >> v2:172.21.15.49:3300/0 conn(0x7f9eac0b2480 0x7f9eac099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eabfff700 1 -- 172.21.15.5:0/3871527974 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac09a4d0 msgr2=0x7f9eac0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eabfff700 1 --2- 172.21.15.5:0/3871527974 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac09a4d0 0x7f9eac0cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eabfff700 1 -- 172.21.15.5:0/3871527974 >> v2:172.21.15.49:3300/0 conn(0x7f9eac0b2480 msgr2=0x7f9eac099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eabfff700 1 --2- 172.21.15.5:0/3871527974 >> v2:172.21.15.49:3300/0 conn(0x7f9eac0b2480 0x7f9eac099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eabfff700 1 -- 172.21.15.5:0/3871527974 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac06c8e0 con 0x7f9eac161030 2024-05-08T23:23:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eabfff700 1 --2- 172.21.15.5:0/3871527974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac099a50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0b2430 tx=0x7f9e9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f7f6eaab08a886 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eaaffd700 1 -- 172.21.15.5:0/3871527974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c011830 con 0x7f9eac161030 2024-05-08T23:23:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eaaffd700 1 -- 172.21.15.5:0/3871527974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c011990 con 0x7f9eac161030 2024-05-08T23:23:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.440+0000 7f9eaaffd700 1 -- 172.21.15.5:0/3871527974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c011d20 con 0x7f9eac161030 2024-05-08T23:23:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 -- 172.21.15.5:0/3871527974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 msgr2=0x7f9eac099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/3871527974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac099a50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0b2430 tx=0x7f9e9c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 -- 172.21.15.5:0/3871527974 shutdown_connections 2024-05-08T23:23:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/3871527974 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac09a4d0 0x7f9eac0cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/3871527974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac099a50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/3871527974 >> v2:172.21.15.49:3300/0 conn(0x7f9eac0b2480 0x7f9eac099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 -- 172.21.15.5:0/3871527974 >> 172.21.15.5:0/3871527974 conn(0x7f9eac0b2f20 msgr2=0x7f9eac0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 -- 172.21.15.5:0/3871527974 shutdown_connections 2024-05-08T23:23:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.441+0000 7f9eb29fa700 1 -- 172.21.15.5:0/3871527974 wait complete. 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.442+0000 7f9eb29fa700 1 Processor -- start 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.443+0000 7f9eb29fa700 1 -- start start 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb29fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eac09a4d0 0x7f9eac1f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb29fa700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac0b2480 0x7f9eac0c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb29fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb29fa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0c9150 con 0x7f9eac161030 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb29fa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0c93f0 con 0x7f9eac09a4d0 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb29fa700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0c96d0 con 0x7f9eac0b2480 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38466/0 (socket says 172.21.15.5:38466) 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 -- 172.21.15.5:0/1819507900 learned_addr learned my addr 172.21.15.5:0/1819507900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eabfff700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eac09a4d0 0x7f9eac1f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac0b2480 0x7f9eac0c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 -- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac0b2480 msgr2=0x7f9eac0c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac0b2480 0x7f9eac0c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 -- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eac09a4d0 msgr2=0x7f9eac1f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eac09a4d0 0x7f9eac1f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 -- 172.21.15.5:0/1819507900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c009550 con 0x7f9eac161030 2024-05-08T23:23:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac0b2480 0x7f9eac0c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.444+0000 7f9eb0f97700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac152690 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400eed0 tx=0x7f9ea400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.445+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1819507900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4010040 con 0x7f9eac161030 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.445+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1819507900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea400bd30 con 0x7f9eac161030 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.445+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0c99b0 con 0x7f9eac161030 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.445+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1819507900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4009ea0 con 0x7f9eac161030 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.445+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac0c8650 con 0x7f9eac161030 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.446+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac0c9ed0 con 0x7f9eac161030 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.446+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1819507900 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9ea4014640 con 0x7f9eac161030 2024-05-08T23:23:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.447+0000 7f9ea8ff9700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e9806ac90 0x7f9e9806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.447+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1819507900 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ea4089990 con 0x7f9eac161030 2024-05-08T23:23:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.447+0000 7f9eabfff700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e9806ac90 0x7f9e9806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.448+0000 7f9eabfff700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e9806ac90 0x7f9e9806d150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c014040 tx=0x7f9e9c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.452+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1819507900 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9ea4054b60 con 0x7f9eac161030 2024-05-08T23:23:14.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.672+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9eac068000 con 0x7f9eac161030 2024-05-08T23:23:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.673+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1819507900 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16005 (secure 0 0 0) 0x7f9ea4018090 con 0x7f9eac161030 2024-05-08T23:23:14.673 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:14.673 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:22:08.768203+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:22:05.437680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:22:06.733995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:14.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e9806ac90 msgr2=0x7f9e9806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e9806ac90 0x7f9e9806d150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c014040 tx=0x7f9e9c008030 comp rx=0 tx=0).stop 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 msgr2=0x7f9eac152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac152690 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400eed0 tx=0x7f9ea400b390 comp rx=0 tx=0).stop 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 shutdown_connections 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e9806ac90 0x7f9e9806d150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac161030 0x7f9eac152690 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eac09a4d0 0x7f9eac1f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 --2- 172.21.15.5:0/1819507900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9eac0b2480 0x7f9eac0c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 >> 172.21.15.5:0/1819507900 conn(0x7f9eac0b2f20 msgr2=0x7f9eac06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 shutdown_connections 2024-05-08T23:23:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.675+0000 7f9eb29fa700 1 -- 172.21.15.5:0/1819507900 wait complete. 2024-05-08T23:23:14.691 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:22:05.437680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:22:06.733995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:23:14.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:23:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.830+0000 7f16e8e81700 1 Processor -- start 2024-05-08T23:23:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.830+0000 7f16e8e81700 1 -- start start 2024-05-08T23:23:14.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.831+0000 7f16e8e81700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f16e40cdc90 0x7f16e40b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.831+0000 7f16e8e81700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e40b1200 con 0x7f16e40d13a0 2024-05-08T23:23:14.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.831+0000 7f16e8e81700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e40ce5c0 con 0x7f16e406bd60 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.831+0000 7f16e8e81700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e40ce8a0 con 0x7f16e40cdc90 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e259c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f16e40d13a0 0x7f16e40d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35900/0 (socket says 172.21.15.5:35900) 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e259c700 1 -- 172.21.15.5:0/2174693699 learned_addr learned my addr 172.21.15.5:0/2174693699 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e2d9d700 1 --2- 172.21.15.5:0/2174693699 >> v2:172.21.15.49:3300/0 conn(0x7f16e40cdc90 0x7f16e40b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370888021 0 0) 0x7f16e40b1200 con 0x7f16e40d13a0 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16cc0035b0 con 0x7f16e40d13a0 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4020923189 0 0) 0x7f16cc0035b0 con 0x7f16e40d13a0 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 >> v2:172.21.15.49:3300/0 conn(0x7f16e40cdc90 msgr2=0x7f16e40b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 --2- 172.21.15.5:0/2174693699 >> v2:172.21.15.49:3300/0 conn(0x7f16e40cdc90 0x7f16e40b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 >> v1:172.21.15.49:6789/0 conn(0x7f16e406bd60 legacy=0x7f16e40cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40ceb80 con 0x7f16e40d13a0 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f16d4001670 con 0x7f16e40d13a0 2024-05-08T23:23:14.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 >> v1:172.21.15.5:6789/0 conn(0x7f16e40d13a0 legacy=0x7f16e40d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc003a00 0x7f16cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc004850 0x7f16cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc005210 0x7f16cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40ce8a0 con 0x7f16cc003a00 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc00c0b0 con 0x7f16cc005210 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc00c270 con 0x7f16cc004850 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e2d9d700 1 --2- 172.21.15.5:0/2174693699 >> v2:172.21.15.49:3300/0 conn(0x7f16e40cdc90 0x7f16e40b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.832+0000 7f16e259c700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc003a00 0x7f16cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e1d9b700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc004850 0x7f16cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e2d9d700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc005210 0x7f16cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e259c700 1 -- 172.21.15.5:0/2174693699 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc004850 msgr2=0x7f16cc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e259c700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc004850 0x7f16cc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e259c700 1 -- 172.21.15.5:0/2174693699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc005210 msgr2=0x7f16cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e259c700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc005210 0x7f16cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e2d9d700 1 -- 172.21.15.5:0/2174693699 reap_dead start 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e259c700 1 -- 172.21.15.5:0/2174693699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0035b0 con 0x7f16cc003a00 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e259c700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc003a00 0x7f16cc003e20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f16d400c240 tx=0x7f16cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61960a22fc908376 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4007c10 con 0x7f16cc003a00 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d400cd80 con 0x7f16cc003a00 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.833+0000 7f16e159a700 1 -- 172.21.15.5:0/2174693699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4005ba0 con 0x7f16cc003a00 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.834+0000 7f16e8e81700 1 -- 172.21.15.5:0/2174693699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc003a00 msgr2=0x7f16cc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.834+0000 7f16e8e81700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc003a00 0x7f16cc003e20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f16d400c240 tx=0x7f16cc005880 comp rx=0 tx=0).stop 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.834+0000 7f16e8e81700 1 -- 172.21.15.5:0/2174693699 shutdown_connections 2024-05-08T23:23:14.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.834+0000 7f16e8e81700 1 --2- 172.21.15.5:0/2174693699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc003a00 0x7f16cc003e20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.834+0000 7f16e8e81700 1 -- 172.21.15.5:0/2174693699 >> 172.21.15.5:0/2174693699 conn(0x7f16e408afe0 msgr2=0x7f16e408b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:14.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.834+0000 7f16e8e81700 1 -- 172.21.15.5:0/2174693699 shutdown_connections 2024-05-08T23:23:14.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.834+0000 7f16e8e81700 1 -- 172.21.15.5:0/2174693699 wait complete. 2024-05-08T23:23:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.835+0000 7f16e8e81700 1 Processor -- start 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.836+0000 7f16e8e81700 1 -- start start 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.836+0000 7f16e8e81700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc003a00 0x7f16e40c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.836+0000 7f16e8e81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e4159550 0x7f16e4158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.836+0000 7f16e8e81700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c84e0 0x7f16e40af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.836+0000 7f16e8e81700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40ca310 con 0x7f16e4159550 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.836+0000 7f16e8e81700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41593b0 con 0x7f16e40c84e0 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.836+0000 7f16e8e81700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40c8d30 con 0x7f16cc003a00 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.836+0000 7f16e1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e4159550 0x7f16e4158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e4159550 0x7f16e4158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38474/0 (socket says 172.21.15.5:38474) 2024-05-08T23:23:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e1d9b700 1 -- 172.21.15.5:0/2509006073 learned_addr learned my addr 172.21.15.5:0/2509006073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e2d9d700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c84e0 0x7f16e40af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e259c700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc003a00 0x7f16e40c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e1d9b700 1 -- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc003a00 msgr2=0x7f16e40c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e1d9b700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc003a00 0x7f16e40c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e1d9b700 1 -- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c84e0 msgr2=0x7f16e40af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e1d9b700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c84e0 0x7f16e40af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e1d9b700 1 -- 172.21.15.5:0/2509006073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40c9010 con 0x7f16e4159550 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e1d9b700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e4159550 0x7f16e4158e70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f16d800f850 tx=0x7f16d800fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16d2ffd700 1 -- 172.21.15.5:0/2509006073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d80052e0 con 0x7f16e4159550 2024-05-08T23:23:14.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16d2ffd700 1 -- 172.21.15.5:0/2509006073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8005440 con 0x7f16e4159550 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e40c92f0 con 0x7f16e4159550 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16d2ffd700 1 -- 172.21.15.5:0/2509006073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d80038a0 con 0x7f16e4159550 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.837+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e40c98b0 con 0x7f16e4159550 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.838+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e4074bc0 con 0x7f16e4159550 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.839+0000 7f16d2ffd700 1 -- 172.21.15.5:0/2509006073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f16d8003a00 con 0x7f16e4159550 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.840+0000 7f16d2ffd700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c406ac90 0x7f16c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.840+0000 7f16d2ffd700 1 -- 172.21.15.5:0/2509006073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16d8089950 con 0x7f16e4159550 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.840+0000 7f16e259c700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c406ac90 0x7f16c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.840+0000 7f16e259c700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c406ac90 0x7f16c406d150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16d4000c00 tx=0x7f16d4007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:14.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:14.844+0000 7f16d2ffd700 1 -- 172.21.15.5:0/2509006073 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f16d8054c10 con 0x7f16e4159550 2024-05-08T23:23:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.061+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16e406c260 con 0x7f16e4159550 2024-05-08T23:23:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.061+0000 7f16d2ffd700 1 -- 172.21.15.5:0/2509006073 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16d8003cb0 con 0x7f16e4159550 2024-05-08T23:23:15.062 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:23:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.063+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c406ac90 msgr2=0x7f16c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.063+0000 7f16e8e81700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c406ac90 0x7f16c406d150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16d4000c00 tx=0x7f16d4007000 comp rx=0 tx=0).stop 2024-05-08T23:23:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.063+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e4159550 msgr2=0x7f16e4158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.063+0000 7f16e8e81700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e4159550 0x7f16e4158e70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f16d800f850 tx=0x7f16d800fb60 comp rx=0 tx=0).stop 2024-05-08T23:23:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.064+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 shutdown_connections 2024-05-08T23:23:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.064+0000 7f16e8e81700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f16c406ac90 0x7f16c406d150 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.064+0000 7f16e8e81700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16e40c84e0 0x7f16e40af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.064+0000 7f16e8e81700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f16cc003a00 0x7f16e40c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.064+0000 7f16e8e81700 1 --2- 172.21.15.5:0/2509006073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e4159550 0x7f16e4158e70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.064+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 >> 172.21.15.5:0/2509006073 conn(0x7f16e408afe0 msgr2=0x7f16e4070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.064+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 shutdown_connections 2024-05-08T23:23:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.064+0000 7f16e8e81700 1 -- 172.21.15.5:0/2509006073 wait complete. 2024-05-08T23:23:15.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.217+0000 7f2ec0566700 1 Processor -- start 2024-05-08T23:23:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.217+0000 7f2ec0566700 1 -- start start 2024-05-08T23:23:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ec0566700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d1e50 0x7f2eb80ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ec0566700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2eb80d2230 0x7f2eb8160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ec0566700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb80cf560 con 0x7f2eb80b2610 2024-05-08T23:23:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ec0566700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb806cae0 con 0x7f2eb80d2230 2024-05-08T23:23:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ec0566700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80ce950 con 0x7f2eb80d1e50 2024-05-08T23:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ebdb01700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2eb80d2230 0x7f2eb8160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ebe302700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d1e50 0x7f2eb80ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ebeb03700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2eb80b2610 0x7f2eb80d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:40440/0 (socket says 172.21.15.5:40440) 2024-05-08T23:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ebeb03700 1 -- 172.21.15.5:0/2269336703 learned_addr learned my addr 172.21.15.5:0/2269336703 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.218+0000 7f2ebdb01700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2eb80d2230 0x7f2eb8160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33732/0 (socket says 172.21.15.5:33732) 2024-05-08T23:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.219+0000 7f2ebdb01700 1 -- 172.21.15.5:0/2269336703 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d1e50 msgr2=0x7f2eb80ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.219+0000 7f2ebdb01700 1 --2- 172.21.15.5:0/2269336703 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d1e50 0x7f2eb80ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.219+0000 7f2ebdb01700 1 -- 172.21.15.5:0/2269336703 >> v1:172.21.15.49:6789/0 conn(0x7f2eb80b2610 legacy=0x7f2eb80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.219+0000 7f2ebdb01700 1 -- 172.21.15.5:0/2269336703 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80cec30 con 0x7f2eb80d2230 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.219+0000 7f2ebdb01700 1 --2- 172.21.15.5:0/2269336703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d2230 0x7f2eb8160c10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009840 tx=0x7f2eb4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5aa4a7a8790522 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.219+0000 7f2ebd300700 1 -- 172.21.15.5:0/2269336703 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4011830 con 0x7f2eb80d2230 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.219+0000 7f2ebd300700 1 -- 172.21.15.5:0/2269336703 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4011990 con 0x7f2eb80d2230 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.219+0000 7f2ebd300700 1 -- 172.21.15.5:0/2269336703 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4011d20 con 0x7f2eb80d2230 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.220+0000 7f2ec0566700 1 -- 172.21.15.5:0/2269336703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d2230 msgr2=0x7f2eb8160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.220+0000 7f2ec0566700 1 --2- 172.21.15.5:0/2269336703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d2230 0x7f2eb8160c10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009840 tx=0x7f2eb4009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.220+0000 7f2ec0566700 1 -- 172.21.15.5:0/2269336703 shutdown_connections 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.220+0000 7f2ec0566700 1 --2- 172.21.15.5:0/2269336703 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d1e50 0x7f2eb80ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.220+0000 7f2ec0566700 1 --2- 172.21.15.5:0/2269336703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d2230 0x7f2eb8160c10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.220+0000 7f2ec0566700 1 -- 172.21.15.5:0/2269336703 >> 172.21.15.5:0/2269336703 conn(0x7f2eb808afe0 msgr2=0x7f2eb80d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.220+0000 7f2ec0566700 1 -- 172.21.15.5:0/2269336703 shutdown_connections 2024-05-08T23:23:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.220+0000 7f2ec0566700 1 -- 172.21.15.5:0/2269336703 wait complete. 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.221+0000 7f2ec0566700 1 Processor -- start 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ec0566700 1 -- start start 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ec0566700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb80b2610 0x7f2eb80ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ec0566700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d1e50 0x7f2eb80c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ec0566700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d2230 0x7f2eb8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ec0566700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80c9ba0 con 0x7f2eb80b2610 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ec0566700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb8155070 con 0x7f2eb80d1e50 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ec0566700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80c8f90 con 0x7f2eb80d2230 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ebe302700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb80b2610 0x7f2eb80ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ebe302700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb80b2610 0x7f2eb80ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38486/0 (socket says 172.21.15.5:38486) 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.222+0000 7f2ebe302700 1 -- 172.21.15.5:0/799966570 learned_addr learned my addr 172.21.15.5:0/799966570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ebeb03700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d2230 0x7f2eb8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ebdb01700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d1e50 0x7f2eb80c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ebe302700 1 -- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d2230 msgr2=0x7f2eb8154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ebe302700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d2230 0x7f2eb8154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ebe302700 1 -- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d1e50 msgr2=0x7f2eb80c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ebe302700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d1e50 0x7f2eb80c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ebe302700 1 -- 172.21.15.5:0/799966570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40094a0 con 0x7f2eb80b2610 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ebe302700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb80b2610 0x7f2eb80ca030 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800e9b0 tx=0x7f2ea800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2eaeffd700 1 -- 172.21.15.5:0/799966570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8009710 con 0x7f2eb80b2610 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2eaeffd700 1 -- 172.21.15.5:0/799966570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea800ee50 con 0x7f2eb80b2610 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb80c9270 con 0x7f2eb80b2610 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2eaeffd700 1 -- 172.21.15.5:0/799966570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800ca50 con 0x7f2eb80b2610 2024-05-08T23:23:15.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.223+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb80c9830 con 0x7f2eb80b2610 2024-05-08T23:23:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.224+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb80c9e80 con 0x7f2eb80b2610 2024-05-08T23:23:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.225+0000 7f2eaeffd700 1 -- 172.21.15.5:0/799966570 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2ea800cbb0 con 0x7f2eb80b2610 2024-05-08T23:23:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.225+0000 7f2eaeffd700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2e9c06ac90 0x7f2e9c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.225+0000 7f2eaeffd700 1 -- 172.21.15.5:0/799966570 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ea8089620 con 0x7f2eb80b2610 2024-05-08T23:23:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.226+0000 7f2ebdb01700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2e9c06ac90 0x7f2e9c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.226+0000 7f2ebdb01700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2e9c06ac90 0x7f2e9c06d150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000c00 tx=0x7f2eb401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:15.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.230+0000 7f2eaeffd700 1 -- 172.21.15.5:0/799966570 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2ea8054890 con 0x7f2eb80b2610 2024-05-08T23:23:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.449+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2eb8074bc0 con 0x7f2eb80b2610 2024-05-08T23:23:15.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.450+0000 7f2eaeffd700 1 -- 172.21.15.5:0/799966570 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ea800ce60 con 0x7f2eb80b2610 2024-05-08T23:23:15.450 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2e9c06ac90 msgr2=0x7f2e9c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2e9c06ac90 0x7f2e9c06d150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000c00 tx=0x7f2eb401c040 comp rx=0 tx=0).stop 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb80b2610 msgr2=0x7f2eb80ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb80b2610 0x7f2eb80ca030 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800e9b0 tx=0x7f2ea800ecc0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 shutdown_connections 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2e9c06ac90 0x7f2e9c06d150 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2eb80d2230 0x7f2eb8154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb80b2610 0x7f2eb80ca030 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 --2- 172.21.15.5:0/799966570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eb80d1e50 0x7f2eb80c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 >> 172.21.15.5:0/799966570 conn(0x7f2eb808afe0 msgr2=0x7f2eb815cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 shutdown_connections 2024-05-08T23:23:15.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.452+0000 7f2ec0566700 1 -- 172.21.15.5:0/799966570 wait complete. 2024-05-08T23:23:15.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:23:15.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.604+0000 7facea575700 1 Processor -- start 2024-05-08T23:23:15.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.604+0000 7facea575700 1 -- start start 2024-05-08T23:23:15.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.605+0000 7facea575700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7face4099a50 0x7face40d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.605+0000 7facea575700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7face40cba90 0x7face406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.605+0000 7facea575700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face406f930 con 0x7face4099e30 2024-05-08T23:23:15.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.605+0000 7facea575700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face40cfdf0 con 0x7face40cba90 2024-05-08T23:23:15.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.605+0000 7facea575700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face40d00d0 con 0x7face4099a50 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face8b12700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7face40cba90 0x7face406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face8b12700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7face40cba90 0x7face406fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38490/0 (socket says 172.21.15.5:38490) 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face8b12700 1 -- 172.21.15.5:0/371965928 learned_addr learned my addr 172.21.15.5:0/371965928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face3fff700 1 --2- 172.21.15.5:0/371965928 >> v2:172.21.15.49:3300/0 conn(0x7face4099a50 0x7face40d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face8b12700 1 -- 172.21.15.5:0/371965928 >> v2:172.21.15.49:3300/0 conn(0x7face4099a50 msgr2=0x7face40d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face8b12700 1 --2- 172.21.15.5:0/371965928 >> v2:172.21.15.49:3300/0 conn(0x7face4099a50 0x7face40d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face8b12700 1 -- 172.21.15.5:0/371965928 >> v1:172.21.15.49:6789/0 conn(0x7face4099e30 legacy=0x7face409a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face8b12700 1 -- 172.21.15.5:0/371965928 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face40d03b0 con 0x7face40cba90 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face8b12700 1 --2- 172.21.15.5:0/371965928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face406fdc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7face406eb80 tx=0x7facdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2786ececf83e6f5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face2ffd700 1 -- 172.21.15.5:0/371965928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc00e070 con 0x7face40cba90 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face2ffd700 1 -- 172.21.15.5:0/371965928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc00b910 con 0x7face40cba90 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.606+0000 7face2ffd700 1 -- 172.21.15.5:0/371965928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc00bcc0 con 0x7face40cba90 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.607+0000 7facea575700 1 -- 172.21.15.5:0/371965928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 msgr2=0x7face406fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.607+0000 7facea575700 1 --2- 172.21.15.5:0/371965928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face406fdc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7face406eb80 tx=0x7facdc00b330 comp rx=0 tx=0).stop 2024-05-08T23:23:15.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.607+0000 7facea575700 1 -- 172.21.15.5:0/371965928 shutdown_connections 2024-05-08T23:23:15.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.607+0000 7facea575700 1 --2- 172.21.15.5:0/371965928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face406fdc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.607+0000 7facea575700 1 --2- 172.21.15.5:0/371965928 >> v2:172.21.15.49:3300/0 conn(0x7face4099a50 0x7face40d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.607+0000 7facea575700 1 -- 172.21.15.5:0/371965928 >> 172.21.15.5:0/371965928 conn(0x7face40b2f30 msgr2=0x7face40b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:15.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.607+0000 7facea575700 1 -- 172.21.15.5:0/371965928 shutdown_connections 2024-05-08T23:23:15.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.607+0000 7facea575700 1 -- 172.21.15.5:0/371965928 wait complete. 2024-05-08T23:23:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.608+0000 7facea575700 1 Processor -- start 2024-05-08T23:23:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7facea575700 1 -- start start 2024-05-08T23:23:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7facea575700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face4099a50 0x7face415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7facea575700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face4099e30 0x7face40b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7facea575700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face40c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7facea575700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40c6d30 con 0x7face40cba90 2024-05-08T23:23:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7facea575700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40b4c70 con 0x7face4099a50 2024-05-08T23:23:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7facea575700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face409f130 con 0x7face4099e30 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7face8b12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face40c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7face8b12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face40c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38496/0 (socket says 172.21.15.5:38496) 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.609+0000 7face8b12700 1 -- 172.21.15.5:0/2031604908 learned_addr learned my addr 172.21.15.5:0/2031604908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face3fff700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face4099a50 0x7face415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face37fe700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face4099e30 0x7face40b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face8b12700 1 -- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face4099e30 msgr2=0x7face40b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face8b12700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face4099e30 0x7face40b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face8b12700 1 -- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face4099a50 msgr2=0x7face415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face8b12700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face4099a50 0x7face415c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face8b12700 1 -- 172.21.15.5:0/2031604908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc009b30 con 0x7face40cba90 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face8b12700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face40c9c60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7face40745f0 tx=0x7facdc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face0ff9700 1 -- 172.21.15.5:0/2031604908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc0099b0 con 0x7face40cba90 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face0ff9700 1 -- 172.21.15.5:0/2031604908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc0092e0 con 0x7face40cba90 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face40b4ea0 con 0x7face40cba90 2024-05-08T23:23:15.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7face0ff9700 1 -- 172.21.15.5:0/2031604908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc01c640 con 0x7face40cba90 2024-05-08T23:23:15.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.610+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face409ee50 con 0x7face40cba90 2024-05-08T23:23:15.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.611+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face40d1240 con 0x7face40cba90 2024-05-08T23:23:15.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.612+0000 7face0ff9700 1 -- 172.21.15.5:0/2031604908 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7facdc01c7a0 con 0x7face40cba90 2024-05-08T23:23:15.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.612+0000 7face0ff9700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7facc406ac90 0x7facc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.612+0000 7face0ff9700 1 -- 172.21.15.5:0/2031604908 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facdc08aeb0 con 0x7face40cba90 2024-05-08T23:23:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.612+0000 7face3fff700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7facc406ac90 0x7facc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.613+0000 7face3fff700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7facc406ac90 0x7facc406d150 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7facd4009b70 tx=0x7facd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:15.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.617+0000 7face0ff9700 1 -- 172.21.15.5:0/2031604908 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7facdc056120 con 0x7face40cba90 2024-05-08T23:23:15.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.833+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7face4074bc0 con 0x7face40cba90 2024-05-08T23:23:15.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.833+0000 7face0ff9700 1 -- 172.21.15.5:0/2031604908 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7facdc059740 con 0x7face40cba90 2024-05-08T23:23:15.833 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:23:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.835+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7facc406ac90 msgr2=0x7facc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.835+0000 7facea575700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7facc406ac90 0x7facc406d150 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7facd4009b70 tx=0x7facd4008040 comp rx=0 tx=0).stop 2024-05-08T23:23:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 msgr2=0x7face40c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face40c9c60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7face40745f0 tx=0x7facdc004ad0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 shutdown_connections 2024-05-08T23:23:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7facc406ac90 0x7facc406d150 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face40cba90 0x7face40c9c60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face4099a50 0x7face415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 --2- 172.21.15.5:0/2031604908 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face4099e30 0x7face40b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 >> 172.21.15.5:0/2031604908 conn(0x7face40b2f30 msgr2=0x7face40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 shutdown_connections 2024-05-08T23:23:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.836+0000 7facea575700 1 -- 172.21.15.5:0/2031604908 wait complete. 2024-05-08T23:23:15.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:23:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.987+0000 7fa7b5111700 1 Processor -- start 2024-05-08T23:23:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.987+0000 7fa7b5111700 1 -- start start 2024-05-08T23:23:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7b5111700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b01614f0 0x7fa7b0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7b5111700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa7b00d0650 0x7fa7b01618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7b5111700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa7b015d0a0 0x7fa7b015d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7b5111700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b0161e10 con 0x7fa7b00d0650 2024-05-08T23:23:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7b5111700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b009a030 con 0x7fa7b015d0a0 2024-05-08T23:23:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7b5111700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b009a310 con 0x7fa7b01614f0 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7ae59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa7b00d0650 0x7fa7b01618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7ae59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa7b00d0650 0x7fa7b01618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38510/0 (socket says 172.21.15.5:38510) 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7ae59c700 1 -- 172.21.15.5:0/3911881747 learned_addr learned my addr 172.21.15.5:0/3911881747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7af59e700 1 --2- 172.21.15.5:0/3911881747 >> v2:172.21.15.49:3300/0 conn(0x7fa7b015d0a0 0x7fa7b015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.988+0000 7fa7aed9d700 1 --2- 172.21.15.5:0/3911881747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b01614f0 0x7fa7b0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7ae59c700 1 -- 172.21.15.5:0/3911881747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b01614f0 msgr2=0x7fa7b0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7ae59c700 1 --2- 172.21.15.5:0/3911881747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b01614f0 0x7fa7b0099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7ae59c700 1 -- 172.21.15.5:0/3911881747 >> v2:172.21.15.49:3300/0 conn(0x7fa7b015d0a0 msgr2=0x7fa7b015d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7ae59c700 1 --2- 172.21.15.5:0/3911881747 >> v2:172.21.15.49:3300/0 conn(0x7fa7b015d0a0 0x7fa7b015d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7ae59c700 1 -- 172.21.15.5:0/3911881747 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b009a5f0 con 0x7fa7b00d0650 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7af59e700 1 --2- 172.21.15.5:0/3911881747 >> v2:172.21.15.49:3300/0 conn(0x7fa7b015d0a0 0x7fa7b015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7ae59c700 1 --2- 172.21.15.5:0/3911881747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b00d0650 0x7fa7b01618d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0072870 tx=0x7fa7a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21b0fdab4f31eb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7add9b700 1 -- 172.21.15.5:0/3911881747 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400b870 con 0x7fa7b00d0650 2024-05-08T23:23:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7add9b700 1 -- 172.21.15.5:0/3911881747 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a400b9d0 con 0x7fa7b00d0650 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7add9b700 1 -- 172.21.15.5:0/3911881747 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400bd60 con 0x7fa7b00d0650 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7b5111700 1 -- 172.21.15.5:0/3911881747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b00d0650 msgr2=0x7fa7b01618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.989+0000 7fa7b5111700 1 --2- 172.21.15.5:0/3911881747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b00d0650 0x7fa7b01618d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0072870 tx=0x7fa7a4009bd0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.990+0000 7fa7b5111700 1 -- 172.21.15.5:0/3911881747 shutdown_connections 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.990+0000 7fa7b5111700 1 --2- 172.21.15.5:0/3911881747 >> v2:172.21.15.49:3300/0 conn(0x7fa7b015d0a0 0x7fa7b015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.990+0000 7fa7b5111700 1 --2- 172.21.15.5:0/3911881747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b01614f0 0x7fa7b0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.990+0000 7fa7b5111700 1 --2- 172.21.15.5:0/3911881747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b00d0650 0x7fa7b01618d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.990+0000 7fa7b5111700 1 -- 172.21.15.5:0/3911881747 >> 172.21.15.5:0/3911881747 conn(0x7fa7b00b2f20 msgr2=0x7fa7b00b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.990+0000 7fa7b5111700 1 -- 172.21.15.5:0/3911881747 shutdown_connections 2024-05-08T23:23:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.990+0000 7fa7b5111700 1 -- 172.21.15.5:0/3911881747 wait complete. 2024-05-08T23:23:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.991+0000 7fa7b5111700 1 Processor -- start 2024-05-08T23:23:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.991+0000 7fa7b5111700 1 -- start start 2024-05-08T23:23:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7b5111700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b00d0650 0x7fa7b0156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7b5111700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b015d0a0 0x7fa7b00c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7b5111700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b01614f0 0x7fa7b01e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7b5111700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b01e7120 con 0x7fa7b015d0a0 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7b5111700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b01e7400 con 0x7fa7b01614f0 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7b5111700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b01e76e0 con 0x7fa7b00d0650 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b015d0a0 0x7fa7b00c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b015d0a0 0x7fa7b00c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38514/0 (socket says 172.21.15.5:38514) 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.992+0000 7fa7ae59c700 1 -- 172.21.15.5:0/1639690935 learned_addr learned my addr 172.21.15.5:0/1639690935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7aed9d700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b00d0650 0x7fa7b0156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7af59e700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b01614f0 0x7fa7b01e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7ae59c700 1 -- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b00d0650 msgr2=0x7fa7b0156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7ae59c700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b00d0650 0x7fa7b0156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7ae59c700 1 -- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b01614f0 msgr2=0x7fa7b01e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7ae59c700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b01614f0 0x7fa7b01e81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7ae59c700 1 -- 172.21.15.5:0/1639690935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4009550 con 0x7fa7b015d0a0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7ae59c700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b015d0a0 0x7fa7b00c77e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4000c00 tx=0x7fa7a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7977fe700 1 -- 172.21.15.5:0/1639690935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4004540 con 0x7fa7b015d0a0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7977fe700 1 -- 172.21.15.5:0/1639690935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a40036a0 con 0x7fa7b015d0a0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b01e79c0 con 0x7fa7b015d0a0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7977fe700 1 -- 172.21.15.5:0/1639690935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4003a50 con 0x7fa7b015d0a0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.993+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b00c7d20 con 0x7fa7b015d0a0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.994+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b01e7ee0 con 0x7fa7b015d0a0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.995+0000 7fa7977fe700 1 -- 172.21.15.5:0/1639690935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa7a4021070 con 0x7fa7b015d0a0 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.995+0000 7fa7977fe700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa79c06ac90 0x7fa79c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.995+0000 7fa7977fe700 1 -- 172.21.15.5:0/1639690935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7a408aad0 con 0x7fa7b015d0a0 2024-05-08T23:23:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.995+0000 7fa7aed9d700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa79c06ac90 0x7fa79c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:15.996+0000 7fa7aed9d700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa79c06ac90 0x7fa79c06d150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00096f0 tx=0x7fa7a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.000+0000 7fa7977fe700 1 -- 172.21.15.5:0/1639690935 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa7b01e7ee0 con 0x7fa7b015d0a0 2024-05-08T23:23:16.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.238+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa7b01e7ee0 con 0x7fa7b015d0a0 2024-05-08T23:23:16.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.238+0000 7fa7977fe700 1 -- 172.21.15.5:0/1639690935 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa7b01e7ee0 con 0x7fa7b015d0a0 2024-05-08T23:23:16.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa79c06ac90 msgr2=0x7fa79c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa79c06ac90 0x7fa79c06d150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00096f0 tx=0x7fa7a0008040 comp rx=0 tx=0).stop 2024-05-08T23:23:16.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b015d0a0 msgr2=0x7fa7b00c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b015d0a0 0x7fa7b00c77e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4000c00 tx=0x7fa7a4004750 comp rx=0 tx=0).stop 2024-05-08T23:23:16.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 shutdown_connections 2024-05-08T23:23:16.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa79c06ac90 0x7fa79c06d150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b01614f0 0x7fa7b01e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7b00d0650 0x7fa7b0156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 --2- 172.21.15.5:0/1639690935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7b015d0a0 0x7fa7b00c77e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 >> 172.21.15.5:0/1639690935 conn(0x7fa7b00b2f20 msgr2=0x7fa7b006f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:16.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 shutdown_connections 2024-05-08T23:23:16.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.241+0000 7fa7b5111700 1 -- 172.21.15.5:0/1639690935 wait complete. 2024-05-08T23:23:16.244 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:23:16.255 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:23:16.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:16.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T23:23:16.308 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:23:16.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:16.308 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:23:16.340 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:23:16.340 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:16.363 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:23:16.363 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:23:16.364 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:16.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:23:16.403 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:23:16.403 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:16.435 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:23:16.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:16.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.594+0000 7fcdc50eb700 1 Processor -- start 2024-05-08T23:23:16.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.594+0000 7fcdc50eb700 1 -- start start 2024-05-08T23:23:16.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.595+0000 7fcdc50eb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcdc006f670 0x7fcdc0165420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:16.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.595+0000 7fcdc50eb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc00b2b10 con 0x7fcdc015a380 2024-05-08T23:23:16.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.595+0000 7fcdc50eb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc00ce340 con 0x7fcdc00cf3c0 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.595+0000 7fcdc50eb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc00ceba0 con 0x7fcdc006f670 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.595+0000 7fcdbed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcdc015a380 0x7fcdc00cbca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35916/0 (socket says 172.21.15.5:35916) 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.595+0000 7fcdbed9d700 1 -- 172.21.15.5:0/1103117710 learned_addr learned my addr 172.21.15.5:0/1103117710 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbf59e700 1 --2- 172.21.15.5:0/1103117710 >> v2:172.21.15.49:3300/0 conn(0x7fcdc006f670 0x7fcdc0165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbdd9b700 1 -- 172.21.15.5:0/1103117710 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281700015 0 0) 0x7fcdc00b2b10 con 0x7fcdc015a380 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbdd9b700 1 -- 172.21.15.5:0/1103117710 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcda80035b0 con 0x7fcdc015a380 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbdd9b700 1 -- 172.21.15.5:0/1103117710 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649668233 0 0) 0x7fcdc00ce340 con 0x7fcdc00cf3c0 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbdd9b700 1 -- 172.21.15.5:0/1103117710 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc00b2b10 con 0x7fcdc00cf3c0 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbf59e700 1 -- 172.21.15.5:0/1103117710 >> v1:172.21.15.49:6789/0 conn(0x7fcdc00cf3c0 legacy=0x7fcdc006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbf59e700 1 -- 172.21.15.5:0/1103117710 >> v1:172.21.15.5:6789/0 conn(0x7fcdc015a380 legacy=0x7fcdc00cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbf59e700 1 -- 172.21.15.5:0/1103117710 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00ce5e0 con 0x7fcdc006f670 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbf59e700 1 --2- 172.21.15.5:0/1103117710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc006f670 0x7fcdc0165420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009d90 tx=0x7fcdb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d6fa607a452425 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbdd9b700 1 -- 172.21.15.5:0/1103117710 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800e070 con 0x7fcdc006f670 2024-05-08T23:23:16.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.596+0000 7fcdbdd9b700 1 -- 172.21.15.5:0/1103117710 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb800bb20 con 0x7fcdc006f670 2024-05-08T23:23:16.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.597+0000 7fcdbdd9b700 1 -- 172.21.15.5:0/1103117710 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8012430 con 0x7fcdc006f670 2024-05-08T23:23:16.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.597+0000 7fcdc50eb700 1 -- 172.21.15.5:0/1103117710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc006f670 msgr2=0x7fcdc0165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.597+0000 7fcdc50eb700 1 --2- 172.21.15.5:0/1103117710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc006f670 0x7fcdc0165420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009d90 tx=0x7fcdb800b330 comp rx=0 tx=0).stop 2024-05-08T23:23:16.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.597+0000 7fcdc50eb700 1 -- 172.21.15.5:0/1103117710 shutdown_connections 2024-05-08T23:23:16.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.597+0000 7fcdc50eb700 1 --2- 172.21.15.5:0/1103117710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc006f670 0x7fcdc0165420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.597+0000 7fcdc50eb700 1 -- 172.21.15.5:0/1103117710 >> 172.21.15.5:0/1103117710 conn(0x7fcdc00cfce0 msgr2=0x7fcdc00d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:16.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.597+0000 7fcdc50eb700 1 -- 172.21.15.5:0/1103117710 shutdown_connections 2024-05-08T23:23:16.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.597+0000 7fcdc50eb700 1 -- 172.21.15.5:0/1103117710 wait complete. 2024-05-08T23:23:16.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.599+0000 7fcdc50eb700 1 Processor -- start 2024-05-08T23:23:16.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.599+0000 7fcdc50eb700 1 -- start start 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdc50eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc006f670 0x7fcdc014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdc50eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcdc00cf3c0 0x7fcdc0158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdc50eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc015a380 0x7fcdc00af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdc50eb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc014dad0 con 0x7fcdc006f670 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdc50eb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00adfe0 con 0x7fcdc015a380 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdc50eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00ae2c0 con 0x7fcdc00cf3c0 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc006f670 0x7fcdc014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc006f670 0x7fcdc014d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38518/0 (socket says 172.21.15.5:38518) 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 -- 172.21.15.5:0/322048018 learned_addr learned my addr 172.21.15.5:0/322048018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbf59e700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc015a380 0x7fcdc00af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbe59c700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcdc00cf3c0 0x7fcdc0158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 -- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcdc00cf3c0 msgr2=0x7fcdc0158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcdc00cf3c0 0x7fcdc0158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 -- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc015a380 msgr2=0x7fcdc00af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc015a380 0x7fcdc00af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 -- 172.21.15.5:0/322048018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8009a70 con 0x7fcdc006f670 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdbed9d700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc006f670 0x7fcdc014d590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000c700 tx=0x7fcdb000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcda77fe700 1 -- 172.21.15.5:0/322048018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0005d70 con 0x7fcdc006f670 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcda77fe700 1 -- 172.21.15.5:0/322048018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb00070d0 con 0x7fcdc006f670 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.600+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc006ea40 con 0x7fcdc006f670 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.601+0000 7fcda77fe700 1 -- 172.21.15.5:0/322048018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0007460 con 0x7fcdc006f670 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.601+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc00ae880 con 0x7fcdc006f670 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.601+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0068000 con 0x7fcdc006f670 2024-05-08T23:23:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.602+0000 7fcda77fe700 1 -- 172.21.15.5:0/322048018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcdb00075c0 con 0x7fcdc006f670 2024-05-08T23:23:16.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.603+0000 7fcda77fe700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcdac06ac90 0x7fcdac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:16.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.603+0000 7fcda77fe700 1 -- 172.21.15.5:0/322048018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdb00886d0 con 0x7fcdc006f670 2024-05-08T23:23:16.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.603+0000 7fcdbe59c700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcdac06ac90 0x7fcdac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:16.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.604+0000 7fcdbe59c700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcdac06ac90 0x7fcdac06d150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40011e0 tx=0x7fcdb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.608+0000 7fcda77fe700 1 -- 172.21.15.5:0/322048018 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcdb0053940 con 0x7fcdc006f670 2024-05-08T23:23:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.873+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdc006fbb0 con 0x7fcdc006f670 2024-05-08T23:23:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.873+0000 7fcda77fe700 1 -- 172.21.15.5:0/322048018 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4753 (secure 0 0 0) 0x7fcdb0017090 con 0x7fcdc006f670 2024-05-08T23:23:16.873 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:16.874 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:44.866299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4975,"mds_1":4745,"mds_2":4742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4742":{"gid":4742,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4745":{"gid":4745,"name":"c","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcdac06ac90 msgr2=0x7fcdac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcdac06ac90 0x7fcdac06d150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40011e0 tx=0x7fcdb4009040 comp rx=0 tx=0).stop 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc006f670 msgr2=0x7fcdc014d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc006f670 0x7fcdc014d590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000c700 tx=0x7fcdb000ca10 comp rx=0 tx=0).stop 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 shutdown_connections 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcdac06ac90 0x7fcdac06d150 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc015a380 0x7fcdc00af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc006f670 0x7fcdc014d590 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 --2- 172.21.15.5:0/322048018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcdc00cf3c0 0x7fcdc0158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 >> 172.21.15.5:0/322048018 conn(0x7fcdc00cfce0 msgr2=0x7fcdc0156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 shutdown_connections 2024-05-08T23:23:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:16.876+0000 7fcdc50eb700 1 -- 172.21.15.5:0/322048018 wait complete. 2024-05-08T23:23:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-05-08T23:23:16.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.027+0000 7faf0248f700 1 Processor -- start 2024-05-08T23:23:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.027+0000 7faf0248f700 1 -- start start 2024-05-08T23:23:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.027+0000 7faf0248f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faefc15ef60 0x7faefc0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faf0248f700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faefc0cfb70 0x7faefc0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faf0248f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc09ad40 con 0x7faefc0cf000 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faf0248f700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc0cd400 con 0x7faefc15ef60 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faf0248f700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc0ccb60 con 0x7faefc0cfb70 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefbfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faefc15ef60 0x7faefc0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefb7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faefc0cf000 0x7faefc0cf460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35920/0 (socket says 172.21.15.5:35920) 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefb7fe700 1 -- 172.21.15.5:0/2961891702 learned_addr learned my addr 172.21.15.5:0/2961891702 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faf00a2c700 1 --2- 172.21.15.5:0/2961891702 >> v2:172.21.15.49:3300/0 conn(0x7faefc0cfb70 0x7faefc0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefbfff700 1 -- 172.21.15.5:0/2961891702 >> v2:172.21.15.49:3300/0 conn(0x7faefc0cfb70 msgr2=0x7faefc0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefbfff700 1 --2- 172.21.15.5:0/2961891702 >> v2:172.21.15.49:3300/0 conn(0x7faefc0cfb70 0x7faefc0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefbfff700 1 -- 172.21.15.5:0/2961891702 >> v1:172.21.15.5:6789/0 conn(0x7faefc0cf000 legacy=0x7faefc0cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefbfff700 1 -- 172.21.15.5:0/2961891702 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0cce40 con 0x7faefc15ef60 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefbfff700 1 --2- 172.21.15.5:0/2961891702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0b2340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faefc072340 tx=0x7faeec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1063b403c229974 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.028+0000 7faefaffd700 1 -- 172.21.15.5:0/2961891702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec011830 con 0x7faefc15ef60 2024-05-08T23:23:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faefaffd700 1 -- 172.21.15.5:0/2961891702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec011990 con 0x7faefc15ef60 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faefaffd700 1 -- 172.21.15.5:0/2961891702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec011d20 con 0x7faefc15ef60 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faf0248f700 1 -- 172.21.15.5:0/2961891702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 msgr2=0x7faefc0b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faf0248f700 1 --2- 172.21.15.5:0/2961891702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0b2340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faefc072340 tx=0x7faeec009b20 comp rx=0 tx=0).stop 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faf0248f700 1 -- 172.21.15.5:0/2961891702 shutdown_connections 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faf0248f700 1 --2- 172.21.15.5:0/2961891702 >> v2:172.21.15.49:3300/0 conn(0x7faefc0cfb70 0x7faefc0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faf0248f700 1 --2- 172.21.15.5:0/2961891702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0b2340 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faf0248f700 1 -- 172.21.15.5:0/2961891702 >> 172.21.15.5:0/2961891702 conn(0x7faefc0b2f20 msgr2=0x7faefc0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faf0248f700 1 -- 172.21.15.5:0/2961891702 shutdown_connections 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.029+0000 7faf0248f700 1 -- 172.21.15.5:0/2961891702 wait complete. 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.031+0000 7faf0248f700 1 Processor -- start 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.031+0000 7faf0248f700 1 -- start start 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf0248f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faefc0cf000 0x7faefc158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf0248f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faefc0cfb70 0x7faefc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf0248f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf0248f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc1592a0 con 0x7faefc15ef60 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf0248f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc0cb070 con 0x7faefc0cfb70 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf0248f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc0ca460 con 0x7faefc0cf000 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38542/0 (socket says 172.21.15.5:38542) 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faefbfff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faefc0cf000 0x7faefc158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 -- 172.21.15.5:0/634306289 learned_addr learned my addr 172.21.15.5:0/634306289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faefb7fe700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faefc0cfb70 0x7faefc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 -- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faefc0cf000 msgr2=0x7faefc158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faefc0cf000 0x7faefc158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 -- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faefc0cfb70 msgr2=0x7faefc06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faefc0cfb70 0x7faefc06d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 -- 172.21.15.5:0/634306289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec0094a0 con 0x7faefc15ef60 2024-05-08T23:23:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.032+0000 7faf00a2c700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0cd540 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faef4009e30 tx=0x7faef400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.033+0000 7faef8ff9700 1 -- 172.21.15.5:0/634306289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4009ca0 con 0x7faefc15ef60 2024-05-08T23:23:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.033+0000 7faef8ff9700 1 -- 172.21.15.5:0/634306289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef400f040 con 0x7faefc15ef60 2024-05-08T23:23:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.033+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc0ca740 con 0x7faefc15ef60 2024-05-08T23:23:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.033+0000 7faef8ff9700 1 -- 172.21.15.5:0/634306289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4014480 con 0x7faefc15ef60 2024-05-08T23:23:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.033+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc0cad00 con 0x7faefc15ef60 2024-05-08T23:23:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.034+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc0cb200 con 0x7faefc15ef60 2024-05-08T23:23:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.034+0000 7faef8ff9700 1 -- 172.21.15.5:0/634306289 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7faef40145e0 con 0x7faefc15ef60 2024-05-08T23:23:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.035+0000 7faef8ff9700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faee806ac90 0x7faee806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.035+0000 7faef8ff9700 1 -- 172.21.15.5:0/634306289 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faef4089e20 con 0x7faefc15ef60 2024-05-08T23:23:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.035+0000 7faefbfff700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faee806ac90 0x7faee806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.036+0000 7faefbfff700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faee806ac90 0x7faee806d150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faeec0090b0 tx=0x7faeec01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.040+0000 7faef8ff9700 1 -- 172.21.15.5:0/634306289 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7faef4055000 con 0x7faefc15ef60 2024-05-08T23:23:17.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.260+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faefc068000 con 0x7faefc15ef60 2024-05-08T23:23:17.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.261+0000 7faef8ff9700 1 -- 172.21.15.5:0/634306289 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16005 (secure 0 0 0) 0x7faef4018020 con 0x7faefc15ef60 2024-05-08T23:23:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:17.261 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:22:08.768203+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:22:05.437680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:22:06.733995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.263+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faee806ac90 msgr2=0x7faee806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faee806ac90 0x7faee806d150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faeec0090b0 tx=0x7faeec01c040 comp rx=0 tx=0).stop 2024-05-08T23:23:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 msgr2=0x7faefc0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0cd540 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faef4009e30 tx=0x7faef400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 shutdown_connections 2024-05-08T23:23:17.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faee806ac90 0x7faee806d150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faefc15ef60 0x7faefc0cd540 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7faefc0cf000 0x7faefc158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 --2- 172.21.15.5:0/634306289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faefc0cfb70 0x7faefc06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 >> 172.21.15.5:0/634306289 conn(0x7faefc0b2f20 msgr2=0x7faefc0b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:17.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 shutdown_connections 2024-05-08T23:23:17.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.264+0000 7faf0248f700 1 -- 172.21.15.5:0/634306289 wait complete. 2024-05-08T23:23:17.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:22:05.437680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:22:06.733995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:23:17.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:23:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.420+0000 7f196eeb3700 1 Processor -- start 2024-05-08T23:23:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.420+0000 7f196eeb3700 1 -- start start 2024-05-08T23:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.421+0000 7f196eeb3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680cff30 0x7f19680d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f196eeb3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680d0850 con 0x7f19680d3d20 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f196eeb3700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680cf610 con 0x7f19680b4a90 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f196eeb3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19680d1fc0 con 0x7f19680cff30 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f1967fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f19680d3d20 0x7f19680ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51188/0 (socket says 172.21.15.5:51188) 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f1967fff700 1 -- 172.21.15.5:0/824231460 learned_addr learned my addr 172.21.15.5:0/824231460 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f196cc4f700 1 --2- 172.21.15.5:0/824231460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680cff30 0x7f19680d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f19677fe700 1 -- 172.21.15.5:0/824231460 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423035928 0 0) 0x7f19680d0850 con 0x7f19680d3d20 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f19677fe700 1 -- 172.21.15.5:0/824231460 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19500035b0 con 0x7f19680d3d20 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f19677fe700 1 -- 172.21.15.5:0/824231460 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534042966 0 0) 0x7f19680cf610 con 0x7f19680b4a90 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f19677fe700 1 -- 172.21.15.5:0/824231460 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19680d0850 con 0x7f19680b4a90 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f196cc4f700 1 -- 172.21.15.5:0/824231460 >> v1:172.21.15.49:6789/0 conn(0x7f19680b4a90 legacy=0x7f19680ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f196cc4f700 1 -- 172.21.15.5:0/824231460 >> v1:172.21.15.5:6789/0 conn(0x7f19680d3d20 legacy=0x7f19680ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f196cc4f700 1 -- 172.21.15.5:0/824231460 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680d22a0 con 0x7f19680cff30 2024-05-08T23:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.422+0000 7f196cc4f700 1 --2- 172.21.15.5:0/824231460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680cff30 0x7f19680d0310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1958009870 tx=0x7f1958009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab0634b51cc697f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f19677fe700 1 -- 172.21.15.5:0/824231460 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958011830 con 0x7f19680cff30 2024-05-08T23:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f19677fe700 1 -- 172.21.15.5:0/824231460 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1958011990 con 0x7f19680cff30 2024-05-08T23:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f19677fe700 1 -- 172.21.15.5:0/824231460 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958011d20 con 0x7f19680cff30 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f196eeb3700 1 -- 172.21.15.5:0/824231460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680cff30 msgr2=0x7f19680d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f196eeb3700 1 --2- 172.21.15.5:0/824231460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680cff30 0x7f19680d0310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1958009870 tx=0x7f1958009b80 comp rx=0 tx=0).stop 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f196eeb3700 1 -- 172.21.15.5:0/824231460 shutdown_connections 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f196eeb3700 1 --2- 172.21.15.5:0/824231460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680cff30 0x7f19680d0310 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f196eeb3700 1 -- 172.21.15.5:0/824231460 >> 172.21.15.5:0/824231460 conn(0x7f196808afe0 msgr2=0x7f19680d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f196eeb3700 1 -- 172.21.15.5:0/824231460 shutdown_connections 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.423+0000 7f196eeb3700 1 -- 172.21.15.5:0/824231460 wait complete. 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.425+0000 7f196eeb3700 1 Processor -- start 2024-05-08T23:23:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.425+0000 7f196eeb3700 1 -- start start 2024-05-08T23:23:17.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196eeb3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19680b4a90 0x7f1968157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196eeb3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19680cff30 0x7f19680b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196eeb3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680d3d20 0x7f19680b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196eeb3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19681f4a50 con 0x7f19680b4a90 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196eeb3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19681f3dd0 con 0x7f19680cff30 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196eeb3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19681f40b0 con 0x7f19680d3d20 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19680b4a90 0x7f1968157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19680b4a90 0x7f1968157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55330/0 (socket says 172.21.15.5:55330) 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 -- 172.21.15.5:0/3451747948 learned_addr learned my addr 172.21.15.5:0/3451747948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f1967fff700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19680cff30 0x7f19680b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196d450700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680d3d20 0x7f19680b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 -- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680d3d20 msgr2=0x7f19680b8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680d3d20 0x7f19680b8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 -- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19680cff30 msgr2=0x7f19680b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19680cff30 0x7f19680b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 -- 172.21.15.5:0/3451747948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19580094a0 con 0x7f19680b4a90 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196d450700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680d3d20 0x7f19680b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f1967fff700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19680cff30 0x7f19680b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196cc4f700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19680b4a90 0x7f1968157850 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f19580090b0 tx=0x7f1958004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f19657fa700 1 -- 172.21.15.5:0/3451747948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195801a860 con 0x7f19680b4a90 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f19657fa700 1 -- 172.21.15.5:0/3451747948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195801a9c0 con 0x7f19680b4a90 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19681f4390 con 0x7f19680b4a90 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.426+0000 7f19657fa700 1 -- 172.21.15.5:0/3451747948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195801ad30 con 0x7f19680b4a90 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.427+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1968157d90 con 0x7f19680b4a90 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.428+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968067a00 con 0x7f19680b4a90 2024-05-08T23:23:17.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.428+0000 7f19657fa700 1 -- 172.21.15.5:0/3451747948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1958022070 con 0x7f19680b4a90 2024-05-08T23:23:17.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.429+0000 7f19657fa700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f195406ac90 0x7f195406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.429+0000 7f19657fa700 1 -- 172.21.15.5:0/3451747948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f195808a430 con 0x7f19680b4a90 2024-05-08T23:23:17.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.429+0000 7f1967fff700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f195406ac90 0x7f195406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.430+0000 7f1967fff700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f195406ac90 0x7f195406d150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f196815bc00 tx=0x7f195c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.434+0000 7f19657fa700 1 -- 172.21.15.5:0/3451747948 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f19580556a0 con 0x7f19680b4a90 2024-05-08T23:23:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.649+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19680d0fa0 con 0x7f19680b4a90 2024-05-08T23:23:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.650+0000 7f19657fa700 1 -- 172.21.15.5:0/3451747948 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19580150a0 con 0x7f19680b4a90 2024-05-08T23:23:17.650 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:23:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f195406ac90 msgr2=0x7f195406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f195406ac90 0x7f195406d150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f196815bc00 tx=0x7f195c009040 comp rx=0 tx=0).stop 2024-05-08T23:23:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19680b4a90 msgr2=0x7f1968157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19680b4a90 0x7f1968157850 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f19580090b0 tx=0x7f1958004750 comp rx=0 tx=0).stop 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 shutdown_connections 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f195406ac90 0x7f195406d150 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19680d3d20 0x7f19680b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19680b4a90 0x7f1968157850 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 --2- 172.21.15.5:0/3451747948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19680cff30 0x7f19680b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 >> 172.21.15.5:0/3451747948 conn(0x7f196808afe0 msgr2=0x7f19680b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 shutdown_connections 2024-05-08T23:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.652+0000 7f196eeb3700 1 -- 172.21.15.5:0/3451747948 wait complete. 2024-05-08T23:23:17.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:23:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.805+0000 7f9beb7b4700 1 Processor -- start 2024-05-08T23:23:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.805+0000 7f9beb7b4700 1 -- start start 2024-05-08T23:23:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9beb7b4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9be4154df0 0x7f9be40d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9beb7b4700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9be40d0890 0x7f9be40b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9beb7b4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 0x7f9be40ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9beb7b4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be415abe0 con 0x7f9be4154df0 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9beb7b4700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be40d0d20 con 0x7f9be40d0890 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9beb7b4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be415add0 con 0x7f9be40b43b0 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9550700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9be4154df0 0x7f9be40d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9550700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9be4154df0 0x7f9be40d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55336/0 (socket says 172.21.15.5:55336) 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9550700 1 -- 172.21.15.5:0/2183330196 learned_addr learned my addr 172.21.15.5:0/2183330196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be8d4f700 1 --2- 172.21.15.5:0/2183330196 >> v2:172.21.15.49:3300/0 conn(0x7f9be40d0890 0x7f9be40b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9d51700 1 --2- 172.21.15.5:0/2183330196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 0x7f9be40ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9550700 1 -- 172.21.15.5:0/2183330196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 msgr2=0x7f9be40ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9550700 1 --2- 172.21.15.5:0/2183330196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 0x7f9be40ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9550700 1 -- 172.21.15.5:0/2183330196 >> v2:172.21.15.49:3300/0 conn(0x7f9be40d0890 msgr2=0x7f9be40b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9550700 1 --2- 172.21.15.5:0/2183330196 >> v2:172.21.15.49:3300/0 conn(0x7f9be40d0890 0x7f9be40b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be9550700 1 -- 172.21.15.5:0/2183330196 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be415b070 con 0x7f9be4154df0 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.806+0000 7f9be8d4f700 1 --2- 172.21.15.5:0/2183330196 >> v2:172.21.15.49:3300/0 conn(0x7f9be40d0890 0x7f9be40b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9be9550700 1 --2- 172.21.15.5:0/2183330196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be40d0350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9be40714a0 tx=0x7f9be0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ceb00c1dd5af7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9bdbfff700 1 -- 172.21.15.5:0/2183330196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0011830 con 0x7f9be4154df0 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9bdbfff700 1 -- 172.21.15.5:0/2183330196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be0011990 con 0x7f9be4154df0 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9bdbfff700 1 -- 172.21.15.5:0/2183330196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0011d20 con 0x7f9be4154df0 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9beb7b4700 1 -- 172.21.15.5:0/2183330196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 msgr2=0x7f9be40d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/2183330196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be40d0350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9be40714a0 tx=0x7f9be0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9beb7b4700 1 -- 172.21.15.5:0/2183330196 shutdown_connections 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/2183330196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 0x7f9be40ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/2183330196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be40d0350 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/2183330196 >> v2:172.21.15.49:3300/0 conn(0x7f9be40d0890 0x7f9be40b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.807+0000 7f9beb7b4700 1 -- 172.21.15.5:0/2183330196 >> 172.21.15.5:0/2183330196 conn(0x7f9be4099a50 msgr2=0x7f9be40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.808+0000 7f9beb7b4700 1 -- 172.21.15.5:0/2183330196 shutdown_connections 2024-05-08T23:23:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.808+0000 7f9beb7b4700 1 -- 172.21.15.5:0/2183330196 wait complete. 2024-05-08T23:23:17.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.809+0000 7f9beb7b4700 1 Processor -- start 2024-05-08T23:23:17.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.809+0000 7f9beb7b4700 1 -- start start 2024-05-08T23:23:17.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9beb7b4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 0x7f9be409e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9beb7b4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be40d0890 0x7f9be409ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9beb7b4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9beb7b4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be409bf50 con 0x7f9be4154df0 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9beb7b4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be409c1f0 con 0x7f9be40d0890 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9beb7b4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be409c4d0 con 0x7f9be40b43b0 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be406f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55352/0 (socket says 172.21.15.5:55352) 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 -- 172.21.15.5:0/1970290041 learned_addr learned my addr 172.21.15.5:0/1970290041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be8d4f700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be40d0890 0x7f9be409ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9550700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 0x7f9be409e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 -- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 msgr2=0x7f9be409e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 0x7f9be409e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 -- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be40d0890 msgr2=0x7f9be409ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be40d0890 0x7f9be409ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 -- 172.21.15.5:0/1970290041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd40094a0 con 0x7f9be4154df0 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9be9d51700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be406f1f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00eed0 tx=0x7f9bdc00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.810+0000 7f9bd9ffb700 1 -- 172.21.15.5:0/1970290041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc010040 con 0x7f9be4154df0 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.811+0000 7f9bd9ffb700 1 -- 172.21.15.5:0/1970290041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00b9f0 con 0x7f9be4154df0 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.811+0000 7f9bd9ffb700 1 -- 172.21.15.5:0/1970290041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00bda0 con 0x7f9be4154df0 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.811+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be0009550 con 0x7f9be4154df0 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.811+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be409ca90 con 0x7f9be4154df0 2024-05-08T23:23:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.812+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be4074bc0 con 0x7f9be4154df0 2024-05-08T23:23:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.812+0000 7f9bd9ffb700 1 -- 172.21.15.5:0/1970290041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9bdc014400 con 0x7f9be4154df0 2024-05-08T23:23:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.813+0000 7f9bd9ffb700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bc806acc0 0x7f9bc806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.813+0000 7f9bd9ffb700 1 -- 172.21.15.5:0/1970290041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bdc089610 con 0x7f9be4154df0 2024-05-08T23:23:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.813+0000 7f9be9550700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bc806acc0 0x7f9bc806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.814+0000 7f9be9550700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bc806acc0 0x7f9bc806d180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009c10 tx=0x7f9be0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:17.818+0000 7f9bd9ffb700 1 -- 172.21.15.5:0/1970290041 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9bdc0547f0 con 0x7f9be4154df0 2024-05-08T23:23:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.037+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9be415a430 con 0x7f9be4154df0 2024-05-08T23:23:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.037+0000 7f9bd9ffb700 1 -- 172.21.15.5:0/1970290041 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9bdc057e10 con 0x7f9be4154df0 2024-05-08T23:23:18.037 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:23:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bc806acc0 msgr2=0x7f9bc806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bc806acc0 0x7f9bc806d180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009c10 tx=0x7f9be0008030 comp rx=0 tx=0).stop 2024-05-08T23:23:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 msgr2=0x7f9be406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be406f1f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00eed0 tx=0x7f9bdc00ef00 comp rx=0 tx=0).stop 2024-05-08T23:23:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 shutdown_connections 2024-05-08T23:23:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bc806acc0 0x7f9bc806d180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9be4154df0 0x7f9be406f1f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9be40b43b0 0x7f9be409e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 --2- 172.21.15.5:0/1970290041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9be40d0890 0x7f9be409ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.039+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 >> 172.21.15.5:0/1970290041 conn(0x7f9be4099a50 msgr2=0x7f9be415ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.040+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 shutdown_connections 2024-05-08T23:23:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.040+0000 7f9beb7b4700 1 -- 172.21.15.5:0/1970290041 wait complete. 2024-05-08T23:23:18.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:23:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.192+0000 7f0248fe4700 1 Processor -- start 2024-05-08T23:23:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.192+0000 7f0248fe4700 1 -- start start 2024-05-08T23:23:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.193+0000 7f0248fe4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02440b2960 0x7f02440af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.193+0000 7f0248fe4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02440cf410 con 0x7f024415db20 2024-05-08T23:23:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.193+0000 7f0248fe4700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02440ad0f0 con 0x7f024406b070 2024-05-08T23:23:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.193+0000 7f0248fe4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024415df00 con 0x7f02440b2960 2024-05-08T23:23:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.193+0000 7f0242d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02440b2960 0x7f02440af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.193+0000 7f024259c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f024415db20 0x7f024406a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51192/0 (socket says 172.21.15.5:51192) 2024-05-08T23:23:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.193+0000 7f024259c700 1 -- 172.21.15.5:0/216158074 learned_addr learned my addr 172.21.15.5:0/216158074 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.193+0000 7f0242d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02440b2960 0x7f02440af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55360/0 (socket says 172.21.15.5:55360) 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.194+0000 7f0242d9d700 1 -- 172.21.15.5:0/216158074 >> v1:172.21.15.49:6789/0 conn(0x7f024406b070 legacy=0x7f02440ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.194+0000 7f0242d9d700 1 -- 172.21.15.5:0/216158074 >> v1:172.21.15.5:6789/0 conn(0x7f024415db20 legacy=0x7f024406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.194+0000 7f0242d9d700 1 -- 172.21.15.5:0/216158074 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440cf640 con 0x7f02440b2960 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.194+0000 7f024159a700 1 -- 172.21.15.5:0/216158074 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075814187 0 0) 0x7f02440cf410 con 0x7f024415db20 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.194+0000 7f0242d9d700 1 --2- 172.21.15.5:0/216158074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440b2960 0x7f02440af640 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f023c009d90 tx=0x7f023c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290ae67a7117fc4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.194+0000 7f024159a700 1 -- 172.21.15.5:0/216158074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c00e070 con 0x7f02440b2960 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.194+0000 7f024159a700 1 -- 172.21.15.5:0/216158074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c00bb90 con 0x7f02440b2960 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.194+0000 7f024159a700 1 -- 172.21.15.5:0/216158074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c003a20 con 0x7f02440b2960 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.195+0000 7f0248fe4700 1 -- 172.21.15.5:0/216158074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440b2960 msgr2=0x7f02440af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.195+0000 7f0248fe4700 1 --2- 172.21.15.5:0/216158074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440b2960 0x7f02440af640 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f023c009d90 tx=0x7f023c00b330 comp rx=0 tx=0).stop 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.195+0000 7f0248fe4700 1 -- 172.21.15.5:0/216158074 shutdown_connections 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.195+0000 7f0248fe4700 1 --2- 172.21.15.5:0/216158074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440b2960 0x7f02440af640 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.195+0000 7f0248fe4700 1 -- 172.21.15.5:0/216158074 >> 172.21.15.5:0/216158074 conn(0x7f0244099a50 msgr2=0x7f02440b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.195+0000 7f0248fe4700 1 -- 172.21.15.5:0/216158074 shutdown_connections 2024-05-08T23:23:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.195+0000 7f0248fe4700 1 -- 172.21.15.5:0/216158074 wait complete. 2024-05-08T23:23:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.196+0000 7f0248fe4700 1 Processor -- start 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.196+0000 7f0248fe4700 1 -- start start 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0248fe4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024406b070 0x7f024406aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0248fe4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02440b2960 0x7f0244146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0248fe4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024415db20 0x7f0244146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0248fe4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02440c7de0 con 0x7f024415db20 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0248fe4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02440cacf0 con 0x7f02440b2960 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0248fe4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02440cafd0 con 0x7f024406b070 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0242d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024415db20 0x7f0244146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0242d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024415db20 0x7f0244146660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55366/0 (socket says 172.21.15.5:55366) 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.197+0000 7f0242d9d700 1 -- 172.21.15.5:0/3405797130 learned_addr learned my addr 172.21.15.5:0/3405797130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f024259c700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024406b070 0x7f024406aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0241d9b700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02440b2960 0x7f0244146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0242d9d700 1 -- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024406b070 msgr2=0x7f024406aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0242d9d700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024406b070 0x7f024406aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0242d9d700 1 -- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02440b2960 msgr2=0x7f0244146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0242d9d700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02440b2960 0x7f0244146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0242d9d700 1 -- 172.21.15.5:0/3405797130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c009a70 con 0x7f024415db20 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0242d9d700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024415db20 0x7f0244146660 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f023c00b730 tx=0x7f023c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f023affd700 1 -- 172.21.15.5:0/3405797130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c012630 con 0x7f024415db20 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f023affd700 1 -- 172.21.15.5:0/3405797130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c012790 con 0x7f024415db20 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02440c8080 con 0x7f024415db20 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f023affd700 1 -- 172.21.15.5:0/3405797130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c012b40 con 0x7f024415db20 2024-05-08T23:23:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.198+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02440ca930 con 0x7f024415db20 2024-05-08T23:23:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.199+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02440caac0 con 0x7f024415db20 2024-05-08T23:23:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.200+0000 7f023affd700 1 -- 172.21.15.5:0/3405797130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f023c012ca0 con 0x7f024415db20 2024-05-08T23:23:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.200+0000 7f023affd700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022406ac90 0x7f022406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.200+0000 7f023affd700 1 -- 172.21.15.5:0/3405797130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f023c08b4f0 con 0x7f024415db20 2024-05-08T23:23:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.200+0000 7f024259c700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022406ac90 0x7f022406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.201+0000 7f024259c700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022406ac90 0x7f022406d150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0234007b10 tx=0x7f0234007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:18.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.205+0000 7f023affd700 1 -- 172.21.15.5:0/3405797130 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f023c0566d0 con 0x7f024415db20 2024-05-08T23:23:18.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.422+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02440c85a0 con 0x7f024415db20 2024-05-08T23:23:18.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.422+0000 7f023affd700 1 -- 172.21.15.5:0/3405797130 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02440c85a0 con 0x7f024415db20 2024-05-08T23:23:18.423 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022406ac90 msgr2=0x7f022406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022406ac90 0x7f022406d150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0234007b10 tx=0x7f0234007a60 comp rx=0 tx=0).stop 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024415db20 msgr2=0x7f0244146660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024415db20 0x7f0244146660 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f023c00b730 tx=0x7f023c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 shutdown_connections 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022406ac90 0x7f022406d150 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024415db20 0x7f0244146660 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024406b070 0x7f024406aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 --2- 172.21.15.5:0/3405797130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02440b2960 0x7f0244146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 >> 172.21.15.5:0/3405797130 conn(0x7f0244099a50 msgr2=0x7f02440adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.424+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 shutdown_connections 2024-05-08T23:23:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.425+0000 7f0248fe4700 1 -- 172.21.15.5:0/3405797130 wait complete. 2024-05-08T23:23:18.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.574+0000 7fded882b700 1 Processor -- start 2024-05-08T23:23:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.574+0000 7fded882b700 1 -- start start 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded882b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded40aef20 0x7fded415c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded882b700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded40cd3b0 con 0x7fded415f230 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded882b700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded40af300 con 0x7fded40ccc40 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded882b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded4141df0 con 0x7fded40aef20 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded259c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded40aef20 0x7fded415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded1d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fded415f230 0x7fded40cc530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51194/0 (socket says 172.21.15.5:51194) 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded1d9b700 1 -- 172.21.15.5:0/1285764605 learned_addr learned my addr 172.21.15.5:0/1285764605 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded259c700 1 --2- 172.21.15.5:0/1285764605 >> v2:172.21.15.5:3300/0 conn(0x7fded40aef20 0x7fded415c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55372/0 (socket says 172.21.15.5:55372) 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded259c700 1 -- 172.21.15.5:0/1285764605 >> v1:172.21.15.49:6789/0 conn(0x7fded40ccc40 legacy=0x7fded406ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded259c700 1 -- 172.21.15.5:0/1285764605 >> v1:172.21.15.5:6789/0 conn(0x7fded415f230 legacy=0x7fded40cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded259c700 1 -- 172.21.15.5:0/1285764605 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded415cbb0 con 0x7fded40aef20 2024-05-08T23:23:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.575+0000 7fded159a700 1 -- 172.21.15.5:0/1285764605 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792647294 0 0) 0x7fded40cd3b0 con 0x7fded415f230 2024-05-08T23:23:18.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded259c700 1 --2- 172.21.15.5:0/1285764605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40aef20 0x7fded415c670 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009840 tx=0x7fdec4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6928f61c5f1a3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:18.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded159a700 1 -- 172.21.15.5:0/1285764605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4011830 con 0x7fded40aef20 2024-05-08T23:23:18.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded159a700 1 -- 172.21.15.5:0/1285764605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec4011990 con 0x7fded40aef20 2024-05-08T23:23:18.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded159a700 1 -- 172.21.15.5:0/1285764605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4011d20 con 0x7fded40aef20 2024-05-08T23:23:18.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded882b700 1 -- 172.21.15.5:0/1285764605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40aef20 msgr2=0x7fded415c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded882b700 1 --2- 172.21.15.5:0/1285764605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40aef20 0x7fded415c670 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009840 tx=0x7fdec4009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:18.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded882b700 1 -- 172.21.15.5:0/1285764605 shutdown_connections 2024-05-08T23:23:18.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded882b700 1 --2- 172.21.15.5:0/1285764605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40aef20 0x7fded415c670 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded882b700 1 -- 172.21.15.5:0/1285764605 >> 172.21.15.5:0/1285764605 conn(0x7fded40cfce0 msgr2=0x7fded40d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:18.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.576+0000 7fded882b700 1 -- 172.21.15.5:0/1285764605 shutdown_connections 2024-05-08T23:23:18.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.577+0000 7fded882b700 1 -- 172.21.15.5:0/1285764605 wait complete. 2024-05-08T23:23:18.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.578+0000 7fded882b700 1 Processor -- start 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.578+0000 7fded882b700 1 -- start start 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded882b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded40aef20 0x7fded41556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded882b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fded40ccc40 0x7fded4139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded882b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded415f230 0x7fded4137270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded882b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded40b4c80 con 0x7fded415f230 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded882b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded4139a50 con 0x7fded40aef20 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded882b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded41f75f0 con 0x7fded40ccc40 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded415f230 0x7fded4137270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded415f230 0x7fded4137270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55386/0 (socket says 172.21.15.5:55386) 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded2d9d700 1 -- 172.21.15.5:0/106343648 learned_addr learned my addr 172.21.15.5:0/106343648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded1d9b700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fded40ccc40 0x7fded4139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded259c700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded40aef20 0x7fded41556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded2d9d700 1 -- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fded40ccc40 msgr2=0x7fded4139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded2d9d700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fded40ccc40 0x7fded4139010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded2d9d700 1 -- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded40aef20 msgr2=0x7fded41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded2d9d700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded40aef20 0x7fded41556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.579+0000 7fded2d9d700 1 -- 172.21.15.5:0/106343648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40094a0 con 0x7fded415f230 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.580+0000 7fded2d9d700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded415f230 0x7fded4137270 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00cc50 tx=0x7fdecc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.580+0000 7fdec2ffd700 1 -- 172.21.15.5:0/106343648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc003c00 con 0x7fded415f230 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.580+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded41f78d0 con 0x7fded415f230 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.580+0000 7fdec2ffd700 1 -- 172.21.15.5:0/106343648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc003d60 con 0x7fded415f230 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.580+0000 7fdec2ffd700 1 -- 172.21.15.5:0/106343648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00aaf0 con 0x7fded415f230 2024-05-08T23:23:18.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.580+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded4154ee0 con 0x7fded415f230 2024-05-08T23:23:18.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.581+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded4068000 con 0x7fded415f230 2024-05-08T23:23:18.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.581+0000 7fdec2ffd700 1 -- 172.21.15.5:0/106343648 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdecc00ac50 con 0x7fded415f230 2024-05-08T23:23:18.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.582+0000 7fdec2ffd700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdeb406ac90 0x7fdeb406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:18.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.582+0000 7fdec2ffd700 1 -- 172.21.15.5:0/106343648 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdecc089270 con 0x7fded415f230 2024-05-08T23:23:18.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.582+0000 7fded259c700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdeb406ac90 0x7fdeb406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:18.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.583+0000 7fded259c700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdeb406ac90 0x7fdeb406d150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdec40090b0 tx=0x7fdec401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:18.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.587+0000 7fdec2ffd700 1 -- 172.21.15.5:0/106343648 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdecc0544e0 con 0x7fded415f230 2024-05-08T23:23:18.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.853+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fded406ea40 con 0x7fded415f230 2024-05-08T23:23:18.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.853+0000 7fdec2ffd700 1 -- 172.21.15.5:0/106343648 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4753 (secure 0 0 0) 0x7fdecc057b00 con 0x7fded415f230 2024-05-08T23:23:18.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:18.853 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:44.866299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4975,"mds_1":4745,"mds_2":4742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4742":{"gid":4742,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4745":{"gid":4745,"name":"c","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:23:18.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdeb406ac90 msgr2=0x7fdeb406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdeb406ac90 0x7fdeb406d150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdec40090b0 tx=0x7fdec401c040 comp rx=0 tx=0).stop 2024-05-08T23:23:18.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded415f230 msgr2=0x7fded4137270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:18.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded415f230 0x7fded4137270 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00cc50 tx=0x7fdecc00cf60 comp rx=0 tx=0).stop 2024-05-08T23:23:18.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 shutdown_connections 2024-05-08T23:23:18.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdeb406ac90 0x7fdeb406d150 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded415f230 0x7fded4137270 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded40aef20 0x7fded41556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 --2- 172.21.15.5:0/106343648 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fded40ccc40 0x7fded4139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 >> 172.21.15.5:0/106343648 conn(0x7fded40cfce0 msgr2=0x7fded406b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 shutdown_connections 2024-05-08T23:23:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:18.856+0000 7fded882b700 1 -- 172.21.15.5:0/106343648 wait complete. 2024-05-08T23:23:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-05-08T23:23:18.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.007+0000 7fe0707df700 1 Processor -- start 2024-05-08T23:23:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.007+0000 7fe0707df700 1 -- start start 2024-05-08T23:23:19.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe0707df700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0680b2480 0x7fe06809a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe0707df700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680cbbc0 0x7fe0680cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe0707df700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0680d0840 con 0x7fe068161030 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe0707df700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06806ccc0 con 0x7fe0680b2480 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe0707df700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680d0660 con 0x7fe0680cbbc0 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06dd7a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0680b2480 0x7fe06809a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06dd7a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0680b2480 0x7fe06809a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55402/0 (socket says 172.21.15.5:55402) 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06dd7a700 1 -- 172.21.15.5:0/57949865 learned_addr learned my addr 172.21.15.5:0/57949865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06ed7c700 1 --2- 172.21.15.5:0/57949865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680cbbc0 0x7fe0680cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06dd7a700 1 -- 172.21.15.5:0/57949865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680cbbc0 msgr2=0x7fe0680cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06dd7a700 1 --2- 172.21.15.5:0/57949865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680cbbc0 0x7fe0680cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06dd7a700 1 -- 172.21.15.5:0/57949865 >> v1:172.21.15.49:6789/0 conn(0x7fe068161030 legacy=0x7fe068099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06dd7a700 1 -- 172.21.15.5:0/57949865 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06806c600 con 0x7fe0680b2480 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06ed7c700 1 --2- 172.21.15.5:0/57949865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680cbbc0 0x7fe0680cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.008+0000 7fe06dd7a700 1 --2- 172.21.15.5:0/57949865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680b2480 0x7fe06809a160 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe0580098f0 tx=0x7fe058009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d5512e5036a413 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe06d579700 1 -- 172.21.15.5:0/57949865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05800b8a0 con 0x7fe0680b2480 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe06d579700 1 -- 172.21.15.5:0/57949865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05800ba00 con 0x7fe0680b2480 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe06d579700 1 -- 172.21.15.5:0/57949865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05800bd90 con 0x7fe0680b2480 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe0707df700 1 -- 172.21.15.5:0/57949865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680b2480 msgr2=0x7fe06809a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe0707df700 1 --2- 172.21.15.5:0/57949865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680b2480 0x7fe06809a160 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe0580098f0 tx=0x7fe058009c00 comp rx=0 tx=0).stop 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe0707df700 1 -- 172.21.15.5:0/57949865 shutdown_connections 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe0707df700 1 --2- 172.21.15.5:0/57949865 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680cbbc0 0x7fe0680cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe0707df700 1 --2- 172.21.15.5:0/57949865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680b2480 0x7fe06809a160 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe0707df700 1 -- 172.21.15.5:0/57949865 >> 172.21.15.5:0/57949865 conn(0x7fe0680b2f20 msgr2=0x7fe0680b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe0707df700 1 -- 172.21.15.5:0/57949865 shutdown_connections 2024-05-08T23:23:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.009+0000 7fe0707df700 1 -- 172.21.15.5:0/57949865 wait complete. 2024-05-08T23:23:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.011+0000 7fe0707df700 1 Processor -- start 2024-05-08T23:23:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.011+0000 7fe0707df700 1 -- start start 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.011+0000 7fe0707df700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680b2480 0x7fe0680b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.011+0000 7fe0707df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680cbbc0 0x7fe0681f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.011+0000 7fe0707df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe068161030 0x7fe06815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.011+0000 7fe0707df700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681f7260 con 0x7fe0680cbbc0 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.011+0000 7fe0707df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681f7540 con 0x7fe068161030 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.011+0000 7fe0707df700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681f7820 con 0x7fe0680b2480 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680cbbc0 0x7fe0681f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680cbbc0 0x7fe0681f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55406/0 (socket says 172.21.15.5:55406) 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 -- 172.21.15.5:0/4087247720 learned_addr learned my addr 172.21.15.5:0/4087247720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06ed7c700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe068161030 0x7fe06815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06e57b700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680b2480 0x7fe0680b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 -- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680b2480 msgr2=0x7fe0680b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680b2480 0x7fe0680b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 -- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe068161030 msgr2=0x7fe06815ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe068161030 0x7fe06815ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 -- 172.21.15.5:0/4087247720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe058009550 con 0x7fe0680cbbc0 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe06dd7a700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680cbbc0 0x7fe0681f47a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe058009c40 tx=0x7fe058004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe05effd700 1 -- 172.21.15.5:0/4087247720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058004450 con 0x7fe0680cbbc0 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe05effd700 1 -- 172.21.15.5:0/4087247720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05801e070 con 0x7fe0680cbbc0 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0681f7b00 con 0x7fe0680cbbc0 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe05effd700 1 -- 172.21.15.5:0/4087247720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0580038e0 con 0x7fe0680cbbc0 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.012+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe068160500 con 0x7fe0680cbbc0 2024-05-08T23:23:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.014+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0681f8020 con 0x7fe0680cbbc0 2024-05-08T23:23:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.014+0000 7fe05effd700 1 -- 172.21.15.5:0/4087247720 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe058003a40 con 0x7fe0680cbbc0 2024-05-08T23:23:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.014+0000 7fe05effd700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe04c06ac90 0x7fe04c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.015+0000 7fe05effd700 1 -- 172.21.15.5:0/4087247720 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe058020070 con 0x7fe0680cbbc0 2024-05-08T23:23:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.015+0000 7fe06e57b700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe04c06ac90 0x7fe04c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.015+0000 7fe06e57b700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe04c06ac90 0x7fe04c06d150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe0640018a0 tx=0x7fe064009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.019+0000 7fe05effd700 1 -- 172.21.15.5:0/4087247720 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe0681f8020 con 0x7fe0680cbbc0 2024-05-08T23:23:19.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.239+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0681f8020 con 0x7fe0680cbbc0 2024-05-08T23:23:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.241+0000 7fe05effd700 1 -- 172.21.15.5:0/4087247720 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16005 (secure 0 0 0) 0x7fe0681f8020 con 0x7fe0680cbbc0 2024-05-08T23:23:19.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:19.241 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:22:08.768203+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:22:05.437680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:22:06.733995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe04c06ac90 msgr2=0x7fe04c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe04c06ac90 0x7fe04c06d150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe0640018a0 tx=0x7fe064009040 comp rx=0 tx=0).stop 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680cbbc0 msgr2=0x7fe0681f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680cbbc0 0x7fe0681f47a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe058009c40 tx=0x7fe058004750 comp rx=0 tx=0).stop 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 shutdown_connections 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe04c06ac90 0x7fe04c06d150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe068161030 0x7fe06815ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0680b2480 0x7fe0680b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 --2- 172.21.15.5:0/4087247720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0680cbbc0 0x7fe0681f47a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 >> 172.21.15.5:0/4087247720 conn(0x7fe0680b2f20 msgr2=0x7fe06806fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:19.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 shutdown_connections 2024-05-08T23:23:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.243+0000 7fe0707df700 1 -- 172.21.15.5:0/4087247720 wait complete. 2024-05-08T23:23:19.257 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:23:19.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.395+0000 7f1ce5596700 1 Processor -- start 2024-05-08T23:23:19.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.395+0000 7f1ce5596700 1 -- start start 2024-05-08T23:23:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.396+0000 7f1ce5596700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1ce015db00 0x7f1ce00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.396+0000 7f1ce5596700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ce006f990 0x7f1ce006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.396+0000 7f1ce5596700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce006ad20 con 0x7f1ce00caa50 2024-05-08T23:23:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.396+0000 7f1ce5596700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce015dee0 con 0x7f1ce006f990 2024-05-08T23:23:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.396+0000 7f1ce5596700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce00ccad0 con 0x7f1ce015db00 2024-05-08T23:23:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.396+0000 7f1cdf7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ce006f990 0x7f1ce006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.396+0000 7f1cdf7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ce006f990 0x7f1ce006e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55418/0 (socket says 172.21.15.5:55418) 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.396+0000 7f1cdf7fe700 1 -- 172.21.15.5:0/3675108331 learned_addr learned my addr 172.21.15.5:0/3675108331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cdeffd700 1 --2- 172.21.15.5:0/3675108331 >> v2:172.21.15.49:3300/0 conn(0x7f1ce015db00 0x7f1ce00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cdf7fe700 1 -- 172.21.15.5:0/3675108331 >> v2:172.21.15.49:3300/0 conn(0x7f1ce015db00 msgr2=0x7f1ce00cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cdf7fe700 1 --2- 172.21.15.5:0/3675108331 >> v2:172.21.15.49:3300/0 conn(0x7f1ce015db00 0x7f1ce00cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cdf7fe700 1 -- 172.21.15.5:0/3675108331 >> v1:172.21.15.49:6789/0 conn(0x7f1ce00caa50 legacy=0x7f1ce006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cdf7fe700 1 -- 172.21.15.5:0/3675108331 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce006b000 con 0x7f1ce006f990 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cdf7fe700 1 --2- 172.21.15.5:0/3675108331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce006f990 0x7f1ce006e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00cdf50 tx=0x7f1cd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a42dc122e1a68d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cddffb700 1 -- 172.21.15.5:0/3675108331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800e070 con 0x7f1ce006f990 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cddffb700 1 -- 172.21.15.5:0/3675108331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd8003a20 con 0x7f1ce006f990 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.397+0000 7f1cddffb700 1 -- 172.21.15.5:0/3675108331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8003dd0 con 0x7f1ce006f990 2024-05-08T23:23:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.398+0000 7f1ce5596700 1 -- 172.21.15.5:0/3675108331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce006f990 msgr2=0x7f1ce006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.398+0000 7f1ce5596700 1 --2- 172.21.15.5:0/3675108331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce006f990 0x7f1ce006e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00cdf50 tx=0x7f1cd800b330 comp rx=0 tx=0).stop 2024-05-08T23:23:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.398+0000 7f1ce5596700 1 -- 172.21.15.5:0/3675108331 shutdown_connections 2024-05-08T23:23:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.398+0000 7f1ce5596700 1 --2- 172.21.15.5:0/3675108331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce006f990 0x7f1ce006e320 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.398+0000 7f1ce5596700 1 --2- 172.21.15.5:0/3675108331 >> v2:172.21.15.49:3300/0 conn(0x7f1ce015db00 0x7f1ce00cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.398+0000 7f1ce5596700 1 -- 172.21.15.5:0/3675108331 >> 172.21.15.5:0/3675108331 conn(0x7f1ce0099a50 msgr2=0x7f1ce00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.398+0000 7f1ce5596700 1 -- 172.21.15.5:0/3675108331 shutdown_connections 2024-05-08T23:23:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.398+0000 7f1ce5596700 1 -- 172.21.15.5:0/3675108331 wait complete. 2024-05-08T23:23:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.399+0000 7f1ce5596700 1 Processor -- start 2024-05-08T23:23:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.399+0000 7f1ce5596700 1 -- start start 2024-05-08T23:23:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1ce5596700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce006f990 0x7f1ce0159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1ce5596700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce00caa50 0x7f1ce00c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1ce5596700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1ce015db00 0x7f1ce01f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1ce5596700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00c94b0 con 0x7f1ce00caa50 2024-05-08T23:23:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1ce5596700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce01f7900 con 0x7f1ce006f990 2024-05-08T23:23:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1ce5596700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce01f7be0 con 0x7f1ce015db00 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1cde7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce00caa50 0x7f1ce00c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1cde7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce00caa50 0x7f1ce00c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55430/0 (socket says 172.21.15.5:55430) 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1cde7fc700 1 -- 172.21.15.5:0/505189681 learned_addr learned my addr 172.21.15.5:0/505189681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1cdeffd700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce006f990 0x7f1ce0159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.400+0000 7f1cdf7fe700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1ce015db00 0x7f1ce01f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cde7fc700 1 -- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1ce015db00 msgr2=0x7f1ce01f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cde7fc700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1ce015db00 0x7f1ce01f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cde7fc700 1 -- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce006f990 msgr2=0x7f1ce0159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cde7fc700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce006f990 0x7f1ce0159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cde7fc700 1 -- 172.21.15.5:0/505189681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd8009b30 con 0x7f1ce00caa50 2024-05-08T23:23:19.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cdeffd700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce006f990 0x7f1ce0159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cde7fc700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce00caa50 0x7f1ce00c9f00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c4d0 tx=0x7f1cd400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cc77fe700 1 -- 172.21.15.5:0/505189681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4011790 con 0x7f1ce00caa50 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cc77fe700 1 -- 172.21.15.5:0/505189681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd40118f0 con 0x7f1ce00caa50 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce00c9700 con 0x7f1ce00caa50 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1cc77fe700 1 -- 172.21.15.5:0/505189681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4011ca0 con 0x7f1ce00caa50 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.401+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce01f3b60 con 0x7f1ce00caa50 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.402+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce00c9c20 con 0x7f1ce00caa50 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.402+0000 7f1cc77fe700 1 -- 172.21.15.5:0/505189681 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1cd401c460 con 0x7f1ce00caa50 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.403+0000 7f1cc77fe700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ccc06ac90 0x7f1ccc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.403+0000 7f1cc77fe700 1 -- 172.21.15.5:0/505189681 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cd4089190 con 0x7f1ce00caa50 2024-05-08T23:23:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.403+0000 7f1cdeffd700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ccc06ac90 0x7f1ccc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.404+0000 7f1cdeffd700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ccc06ac90 0x7f1ccc06d150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0007a50 tx=0x7f1cd0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.408+0000 7f1cc77fe700 1 -- 172.21.15.5:0/505189681 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1ce00c9c20 con 0x7f1ce00caa50 2024-05-08T23:23:19.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.676+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ce00c9c20 con 0x7f1ce00caa50 2024-05-08T23:23:19.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.677+0000 7f1cc77fe700 1 -- 172.21.15.5:0/505189681 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4753 (secure 0 0 0) 0x7f1ce00c9c20 con 0x7f1ce00caa50 2024-05-08T23:23:19.677 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:19.677 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:44.866299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4975,"mds_1":4745,"mds_2":4742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4742":{"gid":4742,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4745":{"gid":4745,"name":"c","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:23:19.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ccc06ac90 msgr2=0x7f1ccc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ccc06ac90 0x7f1ccc06d150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0007a50 tx=0x7f1cd0008040 comp rx=0 tx=0).stop 2024-05-08T23:23:19.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce00caa50 msgr2=0x7f1ce00c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce00caa50 0x7f1ce00c9f00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c4d0 tx=0x7f1cd400c890 comp rx=0 tx=0).stop 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 shutdown_connections 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ccc06ac90 0x7f1ccc06d150 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1ce015db00 0x7f1ce01f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce006f990 0x7f1ce0159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 --2- 172.21.15.5:0/505189681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ce00caa50 0x7f1ce00c9f00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 >> 172.21.15.5:0/505189681 conn(0x7f1ce0099a50 msgr2=0x7f1ce00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 shutdown_connections 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.679+0000 7f1ce5596700 1 -- 172.21.15.5:0/505189681 wait complete. 2024-05-08T23:23:19.680 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-05-08T23:23:19.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:19.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.831+0000 7f4fa8720700 1 Processor -- start 2024-05-08T23:23:19.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.831+0000 7f4fa8720700 1 -- start start 2024-05-08T23:23:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa8720700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4fa015f230 0x7f4fa00cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa8720700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4fa00cca70 0x7f4fa006ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa8720700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa00cd3b0 con 0x7f4fa00aef20 2024-05-08T23:23:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa8720700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa0141df0 con 0x7f4fa00cca70 2024-05-08T23:23:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa8720700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa00cce90 con 0x7f4fa015f230 2024-05-08T23:23:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa6cbd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4fa00cca70 0x7f4fa006ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa6cbd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4fa00cca70 0x7f4fa006ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55432/0 (socket says 172.21.15.5:55432) 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa6cbd700 1 -- 172.21.15.5:0/1533030775 learned_addr learned my addr 172.21.15.5:0/1533030775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.832+0000 7f4fa5cbb700 1 --2- 172.21.15.5:0/1533030775 >> v2:172.21.15.49:3300/0 conn(0x7f4fa015f230 0x7f4fa00cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.833+0000 7f4fa6cbd700 1 -- 172.21.15.5:0/1533030775 >> v2:172.21.15.49:3300/0 conn(0x7f4fa015f230 msgr2=0x7f4fa00cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.833+0000 7f4fa6cbd700 1 --2- 172.21.15.5:0/1533030775 >> v2:172.21.15.49:3300/0 conn(0x7f4fa015f230 0x7f4fa00cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.833+0000 7f4fa6cbd700 1 -- 172.21.15.5:0/1533030775 >> v1:172.21.15.49:6789/0 conn(0x7f4fa00aef20 legacy=0x7f4fa015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.833+0000 7f4fa6cbd700 1 -- 172.21.15.5:0/1533030775 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00af300 con 0x7f4fa00cca70 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.833+0000 7f4fa6cbd700 1 --2- 172.21.15.5:0/1533030775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa00cca70 0x7f4fa006ad40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00cfa90 tx=0x7f4f9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe0e05869d1eb46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.833+0000 7f4fa54ba700 1 -- 172.21.15.5:0/1533030775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800e070 con 0x7f4fa00cca70 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.833+0000 7f4fa54ba700 1 -- 172.21.15.5:0/1533030775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9800b9c0 con 0x7f4fa00cca70 2024-05-08T23:23:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.833+0000 7f4fa54ba700 1 -- 172.21.15.5:0/1533030775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800bd70 con 0x7f4fa00cca70 2024-05-08T23:23:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.834+0000 7f4fa8720700 1 -- 172.21.15.5:0/1533030775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa00cca70 msgr2=0x7f4fa006ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.834+0000 7f4fa8720700 1 --2- 172.21.15.5:0/1533030775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa00cca70 0x7f4fa006ad40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00cfa90 tx=0x7f4f9800b330 comp rx=0 tx=0).stop 2024-05-08T23:23:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.834+0000 7f4fa8720700 1 -- 172.21.15.5:0/1533030775 shutdown_connections 2024-05-08T23:23:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.834+0000 7f4fa8720700 1 --2- 172.21.15.5:0/1533030775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa00cca70 0x7f4fa006ad40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.834+0000 7f4fa8720700 1 --2- 172.21.15.5:0/1533030775 >> v2:172.21.15.49:3300/0 conn(0x7f4fa015f230 0x7f4fa00cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.834+0000 7f4fa8720700 1 -- 172.21.15.5:0/1533030775 >> 172.21.15.5:0/1533030775 conn(0x7f4fa00cfce0 msgr2=0x7f4fa00d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.834+0000 7f4fa8720700 1 -- 172.21.15.5:0/1533030775 shutdown_connections 2024-05-08T23:23:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.834+0000 7f4fa8720700 1 -- 172.21.15.5:0/1533030775 wait complete. 2024-05-08T23:23:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.835+0000 7f4fa8720700 1 Processor -- start 2024-05-08T23:23:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.836+0000 7f4fa8720700 1 -- start start 2024-05-08T23:23:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa8720700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa00aef20 0x7f4fa013a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa8720700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4fa00cca70 0x7f4fa015ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa8720700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa015f230 0x7f4fa0154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa8720700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa013aa80 con 0x7f4fa015f230 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa8720700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa013d5e0 con 0x7f4fa00aef20 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa8720700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa015ef90 con 0x7f4fa00cca70 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa015f230 0x7f4fa0154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa015f230 0x7f4fa0154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55438/0 (socket says 172.21.15.5:55438) 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 -- 172.21.15.5:0/2935863433 learned_addr learned my addr 172.21.15.5:0/2935863433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa64bc700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa00aef20 0x7f4fa013a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa5cbb700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4fa00cca70 0x7f4fa015ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 -- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4fa00cca70 msgr2=0x7f4fa015ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4fa00cca70 0x7f4fa015ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 -- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa00aef20 msgr2=0x7f4fa013a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa00aef20 0x7f4fa013a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 -- 172.21.15.5:0/2935863433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f98009b30 con 0x7f4fa015f230 2024-05-08T23:23:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4fa6cbd700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa015f230 0x7f4fa0154e70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00724a0 tx=0x7f4f98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.837+0000 7f4f96ffd700 1 -- 172.21.15.5:0/2935863433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98004880 con 0x7f4fa015f230 2024-05-08T23:23:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.838+0000 7f4f96ffd700 1 -- 172.21.15.5:0/2935863433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f980099b0 con 0x7f4fa015f230 2024-05-08T23:23:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.838+0000 7f4f96ffd700 1 -- 172.21.15.5:0/2935863433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9801c680 con 0x7f4fa015f230 2024-05-08T23:23:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.838+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa013ccb0 con 0x7f4fa015f230 2024-05-08T23:23:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.838+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa013d270 con 0x7f4fa015f230 2024-05-08T23:23:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.838+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa013d790 con 0x7f4fa015f230 2024-05-08T23:23:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.839+0000 7f4f96ffd700 1 -- 172.21.15.5:0/2935863433 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4f9801c7e0 con 0x7f4fa015f230 2024-05-08T23:23:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.840+0000 7f4f96ffd700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f8406ac90 0x7f4f8406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.840+0000 7f4f96ffd700 1 -- 172.21.15.5:0/2935863433 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f9808aeb0 con 0x7f4fa015f230 2024-05-08T23:23:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.840+0000 7f4fa64bc700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f8406ac90 0x7f4f8406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.840+0000 7f4fa64bc700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f8406ac90 0x7f4f8406d150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c007fd0 tx=0x7f4f9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:19.844+0000 7f4f96ffd700 1 -- 172.21.15.5:0/2935863433 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4f98056120 con 0x7f4fa015f230 2024-05-08T23:23:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.112+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fa0068000 con 0x7f4fa015f230 2024-05-08T23:23:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.113+0000 7f4f96ffd700 1 -- 172.21.15.5:0/2935863433 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4753 (secure 0 0 0) 0x7f4f98059740 con 0x7f4fa015f230 2024-05-08T23:23:20.113 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:20.113 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:22:07.766519+0000","modified":"2024-05-08T23:22:44.866299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4975,"mds_1":4745,"mds_2":4742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4742":{"gid":4742,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6834/295306227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":295306227},{"type":"v1","addr":"172.21.15.5:6834","nonce":295306227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4745":{"gid":4745,"name":"c","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6835/146081129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":146081129},{"type":"v1","addr":"172.21.15.5:6835","nonce":146081129}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4975":{"gid":4975,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6837/886789668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":886789668},{"type":"v1","addr":"172.21.15.49:6837","nonce":886789668}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:23:20.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f8406ac90 msgr2=0x7f4f8406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f8406ac90 0x7f4f8406d150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c007fd0 tx=0x7f4f9c009040 comp rx=0 tx=0).stop 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa015f230 msgr2=0x7f4fa0154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa015f230 0x7f4fa0154e70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00724a0 tx=0x7f4f98004ad0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 shutdown_connections 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f8406ac90 0x7f4f8406d150 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa015f230 0x7f4fa0154e70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa00aef20 0x7f4fa013a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 --2- 172.21.15.5:0/2935863433 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4fa00cca70 0x7f4fa015ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 >> 172.21.15.5:0/2935863433 conn(0x7f4fa00cfce0 msgr2=0x7f4fa01414f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 shutdown_connections 2024-05-08T23:23:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.115+0000 7f4fa8720700 1 -- 172.21.15.5:0/2935863433 wait complete. 2024-05-08T23:23:20.117 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-05-08T23:23:20.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.265+0000 7fafd2661700 1 Processor -- start 2024-05-08T23:23:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.265+0000 7fafd2661700 1 -- start start 2024-05-08T23:23:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafd2661700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc15b7b0 0x7fafcc0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafd2661700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fafcc0b3430 0x7fafcc0ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafd2661700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc0cb910 con 0x7fafcc0ac1c0 2024-05-08T23:23:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafd2661700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc1600f0 con 0x7fafcc0b3430 2024-05-08T23:23:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafd2661700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0acb80 con 0x7fafcc15b7b0 2024-05-08T23:23:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafcbfff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc15b7b0 0x7fafcc0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafd0bfe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fafcc0b3430 0x7fafcc0ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafcbfff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc15b7b0 0x7fafcc0cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:37340/0 (socket says 172.21.15.5:37340) 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafd0bfe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fafcc0b3430 0x7fafcc0ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:53026/0 (socket says 172.21.15.5:53026) 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.266+0000 7fafcbfff700 1 -- 172.21.15.5:0/1059757569 learned_addr learned my addr 172.21.15.5:0/1059757569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafd0bfe700 1 -- 172.21.15.5:0/1059757569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc15b7b0 msgr2=0x7fafcc0cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafd0bfe700 1 --2- 172.21.15.5:0/1059757569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc15b7b0 0x7fafcc0cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafd0bfe700 1 -- 172.21.15.5:0/1059757569 >> v1:172.21.15.49:6789/0 conn(0x7fafcc0ac1c0 legacy=0x7fafcc06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafd0bfe700 1 -- 172.21.15.5:0/1059757569 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc15f190 con 0x7fafcc0b3430 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafcaffd700 1 -- 172.21.15.5:0/1059757569 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054876069 0 0) 0x7fafcc0cb910 con 0x7fafcc0ac1c0 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafcbfff700 1 --2- 172.21.15.5:0/1059757569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc15b7b0 0x7fafcc0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafd0bfe700 1 --2- 172.21.15.5:0/1059757569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 0x7fafcc0ac640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009d90 tx=0x7fafc400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222a227fefd5ddb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafcaffd700 1 -- 172.21.15.5:0/1059757569 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc400e070 con 0x7fafcc0b3430 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafcaffd700 1 -- 172.21.15.5:0/1059757569 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc400bb20 con 0x7fafcc0b3430 2024-05-08T23:23:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.267+0000 7fafcaffd700 1 -- 172.21.15.5:0/1059757569 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4012430 con 0x7fafcc0b3430 2024-05-08T23:23:20.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.268+0000 7fafd2661700 1 -- 172.21.15.5:0/1059757569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 msgr2=0x7fafcc0ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.268+0000 7fafd2661700 1 --2- 172.21.15.5:0/1059757569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 0x7fafcc0ac640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009d90 tx=0x7fafc400b330 comp rx=0 tx=0).stop 2024-05-08T23:23:20.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.268+0000 7fafd2661700 1 -- 172.21.15.5:0/1059757569 shutdown_connections 2024-05-08T23:23:20.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.268+0000 7fafd2661700 1 --2- 172.21.15.5:0/1059757569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 0x7fafcc0ac640 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.268+0000 7fafd2661700 1 --2- 172.21.15.5:0/1059757569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc15b7b0 0x7fafcc0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.268+0000 7fafd2661700 1 -- 172.21.15.5:0/1059757569 >> 172.21.15.5:0/1059757569 conn(0x7fafcc0cf270 msgr2=0x7fafcc0cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.268+0000 7fafd2661700 1 -- 172.21.15.5:0/1059757569 shutdown_connections 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.268+0000 7fafd2661700 1 -- 172.21.15.5:0/1059757569 wait complete. 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.269+0000 7fafd2661700 1 Processor -- start 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.269+0000 7fafd2661700 1 -- start start 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd2661700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc0ac1c0 0x7fafcc15c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd2661700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 0x7fafcc1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd2661700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafcc0cd3d0 0x7fafcc0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd2661700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc15df30 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd2661700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc15daa0 con 0x7fafcc0b3430 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd2661700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc15ce90 con 0x7fafcc0ac1c0 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd0bfe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafcc0cd3d0 0x7fafcc0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd0bfe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafcc0cd3d0 0x7fafcc0cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55440/0 (socket says 172.21.15.5:55440) 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.270+0000 7fafd0bfe700 1 -- 172.21.15.5:0/2375233288 learned_addr learned my addr 172.21.15.5:0/2375233288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafcb7fe700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 0x7fafcc1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafcbfff700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc0ac1c0 0x7fafcc15c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafd0bfe700 1 -- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc0ac1c0 msgr2=0x7fafcc15c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafd0bfe700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc0ac1c0 0x7fafcc15c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafd0bfe700 1 -- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 msgr2=0x7fafcc1f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafd0bfe700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 0x7fafcc1f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafd0bfe700 1 -- 172.21.15.5:0/2375233288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc009550 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafd0bfe700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafcc0cd3d0 0x7fafcc0cdad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fafcc070580 tx=0x7fafc4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafc8ff9700 1 -- 172.21.15.5:0/2375233288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4012430 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafc8ff9700 1 -- 172.21.15.5:0/2375233288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc4020070 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafc8ff9700 1 -- 172.21.15.5:0/2375233288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4004280 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc4009a70 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.271+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc15d450 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.272+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc0678f0 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.273+0000 7fafc8ff9700 1 -- 172.21.15.5:0/2375233288 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fafc40043e0 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.273+0000 7fafc8ff9700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fafac06ac90 0x7fafac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.273+0000 7fafc8ff9700 1 -- 172.21.15.5:0/2375233288 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafc4022070 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.274+0000 7fafcbfff700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fafac06ac90 0x7fafac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.274+0000 7fafcbfff700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fafac06ac90 0x7fafac06d150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006d30 tx=0x7fafbc0092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.278+0000 7fafc8ff9700 1 -- 172.21.15.5:0/2375233288 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fafc40560c0 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.495+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafcc06eb30 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.497+0000 7fafc8ff9700 1 -- 172.21.15.5:0/2375233288 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16005 (secure 0 0 0) 0x7fafc402c090 con 0x7fafcc0cd3d0 2024-05-08T23:23:20.497 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:20.497 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:22:08.768203+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:22:05.437680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:22:06.733995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fafac06ac90 msgr2=0x7fafac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fafac06ac90 0x7fafac06d150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006d30 tx=0x7fafbc0092c0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafcc0cd3d0 msgr2=0x7fafcc0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafcc0cd3d0 0x7fafcc0cdad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fafcc070580 tx=0x7fafc4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 shutdown_connections 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fafac06ac90 0x7fafac06d150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafcc0cd3d0 0x7fafcc0cdad0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fafcc0ac1c0 0x7fafcc15c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 --2- 172.21.15.5:0/2375233288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafcc0b3430 0x7fafcc1f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 >> 172.21.15.5:0/2375233288 conn(0x7fafcc0cf270 msgr2=0x7fafcc06dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 shutdown_connections 2024-05-08T23:23:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.499+0000 7fafd2661700 1 -- 172.21.15.5:0/2375233288 wait complete. 2024-05-08T23:23:20.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:23:20.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.651+0000 7f334f8aa700 1 Processor -- start 2024-05-08T23:23:20.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.651+0000 7f334f8aa700 1 -- start start 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334f8aa700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f334806f9f0 0x7f33480cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334f8aa700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f334815ded0 0x7f334806ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334f8aa700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33480cee80 con 0x7f33480ce6a0 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334f8aa700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33480cbd70 con 0x7f334815ded0 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334f8aa700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334806fdd0 con 0x7f334806f9f0 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334de47700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f334815ded0 0x7f334806ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334de47700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f334815ded0 0x7f334806ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55448/0 (socket says 172.21.15.5:55448) 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334de47700 1 -- 172.21.15.5:0/3248129555 learned_addr learned my addr 172.21.15.5:0/3248129555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.652+0000 7f334d646700 1 --2- 172.21.15.5:0/3248129555 >> v2:172.21.15.49:3300/0 conn(0x7f334806f9f0 0x7f33480cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.653+0000 7f334de47700 1 -- 172.21.15.5:0/3248129555 >> v2:172.21.15.49:3300/0 conn(0x7f334806f9f0 msgr2=0x7f33480cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.653+0000 7f334de47700 1 --2- 172.21.15.5:0/3248129555 >> v2:172.21.15.49:3300/0 conn(0x7f334806f9f0 0x7f33480cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.653+0000 7f334de47700 1 -- 172.21.15.5:0/3248129555 >> v1:172.21.15.49:6789/0 conn(0x7f33480ce6a0 legacy=0x7f334815d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.653+0000 7f334de47700 1 -- 172.21.15.5:0/3248129555 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480cf160 con 0x7f334815ded0 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.653+0000 7f334de47700 1 --2- 172.21.15.5:0/3248129555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f334815ded0 0x7f334806ad40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f33480cdef0 tx=0x7f334000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f208e2dc82c5d88 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.653+0000 7f333bfff700 1 -- 172.21.15.5:0/3248129555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334000e070 con 0x7f334815ded0 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.653+0000 7f333bfff700 1 -- 172.21.15.5:0/3248129555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334000b980 con 0x7f334815ded0 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.653+0000 7f333bfff700 1 -- 172.21.15.5:0/3248129555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334000bd30 con 0x7f334815ded0 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.654+0000 7f334f8aa700 1 -- 172.21.15.5:0/3248129555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f334815ded0 msgr2=0x7f334806ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.654+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3248129555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f334815ded0 0x7f334806ad40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f33480cdef0 tx=0x7f334000b330 comp rx=0 tx=0).stop 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.654+0000 7f334f8aa700 1 -- 172.21.15.5:0/3248129555 shutdown_connections 2024-05-08T23:23:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.654+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3248129555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f334815ded0 0x7f334806ad40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.654+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3248129555 >> v2:172.21.15.49:3300/0 conn(0x7f334806f9f0 0x7f33480cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.654+0000 7f334f8aa700 1 -- 172.21.15.5:0/3248129555 >> 172.21.15.5:0/3248129555 conn(0x7f33480cc580 msgr2=0x7f33480cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.654+0000 7f334f8aa700 1 -- 172.21.15.5:0/3248129555 shutdown_connections 2024-05-08T23:23:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.654+0000 7f334f8aa700 1 -- 172.21.15.5:0/3248129555 wait complete. 2024-05-08T23:23:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.655+0000 7f334f8aa700 1 Processor -- start 2024-05-08T23:23:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.655+0000 7f334f8aa700 1 -- start start 2024-05-08T23:23:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334f8aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f334806f9f0 0x7f334814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334f8aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33480ce6a0 0x7f33481423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334f8aa700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f334815ded0 0x7f334806e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334f8aa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334814c3d0 con 0x7f33480ce6a0 2024-05-08T23:23:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334f8aa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3348149b00 con 0x7f334806f9f0 2024-05-08T23:23:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334f8aa700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334814b7c0 con 0x7f334815ded0 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33480ce6a0 0x7f33481423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33480ce6a0 0x7f33481423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55452/0 (socket says 172.21.15.5:55452) 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 -- 172.21.15.5:0/3442212434 learned_addr learned my addr 172.21.15.5:0/3442212434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334d646700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f334806f9f0 0x7f334814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334de47700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f334815ded0 0x7f334806e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 -- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f334815ded0 msgr2=0x7f334806e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f334815ded0 0x7f334806e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 -- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f334806f9f0 msgr2=0x7f334814af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f334806f9f0 0x7f334814af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 -- 172.21.15.5:0/3442212434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340009b30 con 0x7f33480ce6a0 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334de47700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f334815ded0 0x7f334806e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f334ce45700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33480ce6a0 0x7f33481423b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c490 tx=0x7f333c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.656+0000 7f3339ffb700 1 -- 172.21.15.5:0/3442212434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c0052d0 con 0x7f33480ce6a0 2024-05-08T23:23:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.657+0000 7f3339ffb700 1 -- 172.21.15.5:0/3442212434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c005450 con 0x7f33480ce6a0 2024-05-08T23:23:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.657+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334814baa0 con 0x7f33480ce6a0 2024-05-08T23:23:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.657+0000 7f3339ffb700 1 -- 172.21.15.5:0/3442212434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c006aa0 con 0x7f33480ce6a0 2024-05-08T23:23:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.657+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334814c060 con 0x7f33480ce6a0 2024-05-08T23:23:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.658+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334814c540 con 0x7f33480ce6a0 2024-05-08T23:23:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.658+0000 7f3339ffb700 1 -- 172.21.15.5:0/3442212434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f333c003680 con 0x7f33480ce6a0 2024-05-08T23:23:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.658+0000 7f3339ffb700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f332c06ac90 0x7f332c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.659+0000 7f3339ffb700 1 -- 172.21.15.5:0/3442212434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f333c088570 con 0x7f33480ce6a0 2024-05-08T23:23:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.659+0000 7f334d646700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f332c06ac90 0x7f332c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.659+0000 7f334d646700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f332c06ac90 0x7f332c06d150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f334814cbf0 tx=0x7f3344008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.663+0000 7f3339ffb700 1 -- 172.21.15.5:0/3442212434 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f333c053750 con 0x7f33480ce6a0 2024-05-08T23:23:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.931+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3348068000 con 0x7f33480ce6a0 2024-05-08T23:23:20.932 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:23:20.932+0000 7f8a49d06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:23:20.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.939+0000 7f3339ffb700 1 -- 172.21.15.5:0/3442212434 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f333c056d70 con 0x7f33480ce6a0 2024-05-08T23:23:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f332c06ac90 msgr2=0x7f332c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f332c06ac90 0x7f332c06d150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f334814cbf0 tx=0x7f3344008040 comp rx=0 tx=0).stop 2024-05-08T23:23:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33480ce6a0 msgr2=0x7f33481423b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33480ce6a0 0x7f33481423b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c490 tx=0x7f333c00c7a0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 shutdown_connections 2024-05-08T23:23:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f332c06ac90 0x7f332c06d150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f334815ded0 0x7f334806e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f334806f9f0 0x7f334814af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 --2- 172.21.15.5:0/3442212434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33480ce6a0 0x7f33481423b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 >> 172.21.15.5:0/3442212434 conn(0x7f33480cc580 msgr2=0x7f3348154020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 shutdown_connections 2024-05-08T23:23:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:20.942+0000 7f334f8aa700 1 -- 172.21.15.5:0/3442212434 wait complete. 2024-05-08T23:23:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:23:20.958 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:23:20.976 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:23:20.976 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:23:20.984 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:23:21.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.099+0000 7f0889e15700 1 Processor -- start 2024-05-08T23:23:21.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.099+0000 7f0889e15700 1 -- start start 2024-05-08T23:23:21.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f0889e15700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f088415b4e0 0x7f08840b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f0889e15700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f088415b8c0 0x7f08840d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f0889e15700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840cf560 con 0x7f08840d1740 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f0889e15700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08840cefe0 con 0x7f088415b4e0 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f0889e15700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08840ccff0 con 0x7f088415b8c0 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f08837fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f088415b4e0 0x7f08840b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f0883fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f08840d1740 0x7f08840cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51208/0 (socket says 172.21.15.5:51208) 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f0883fff700 1 -- 172.21.15.5:0/2529236677 learned_addr learned my addr 172.21.15.5:0/2529236677 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f0882ffd700 1 --2- 172.21.15.5:0/2529236677 >> v2:172.21.15.49:3300/0 conn(0x7f088415b8c0 0x7f08840d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f08837fe700 1 -- 172.21.15.5:0/2529236677 >> v2:172.21.15.49:3300/0 conn(0x7f088415b8c0 msgr2=0x7f08840d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f08837fe700 1 --2- 172.21.15.5:0/2529236677 >> v2:172.21.15.49:3300/0 conn(0x7f088415b8c0 0x7f08840d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f08837fe700 1 -- 172.21.15.5:0/2529236677 >> v1:172.21.15.5:6789/0 conn(0x7f08840d1740 legacy=0x7f08840cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:21.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f08827fc700 1 -- 172.21.15.5:0/2529236677 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472142895 0 0) 0x7f08840cf560 con 0x7f08840d1740 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.100+0000 7f08837fe700 1 -- 172.21.15.5:0/2529236677 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840cd2d0 con 0x7f088415b4e0 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f08837fe700 1 --2- 172.21.15.5:0/2529236677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f088415b4e0 0x7f08840b1200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f088415f1e0 tx=0x7f0874009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb58dd0520cd880c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f08827fc700 1 -- 172.21.15.5:0/2529236677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874011830 con 0x7f088415b4e0 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f08827fc700 1 -- 172.21.15.5:0/2529236677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0874011990 con 0x7f088415b4e0 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f08827fc700 1 -- 172.21.15.5:0/2529236677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874011d20 con 0x7f088415b4e0 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f0889e15700 1 -- 172.21.15.5:0/2529236677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f088415b4e0 msgr2=0x7f08840b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f0889e15700 1 --2- 172.21.15.5:0/2529236677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f088415b4e0 0x7f08840b1200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f088415f1e0 tx=0x7f0874009b20 comp rx=0 tx=0).stop 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f0889e15700 1 -- 172.21.15.5:0/2529236677 shutdown_connections 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f0889e15700 1 --2- 172.21.15.5:0/2529236677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f088415b4e0 0x7f08840b1200 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f0889e15700 1 --2- 172.21.15.5:0/2529236677 >> v2:172.21.15.49:3300/0 conn(0x7f088415b8c0 0x7f08840d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f0889e15700 1 -- 172.21.15.5:0/2529236677 >> 172.21.15.5:0/2529236677 conn(0x7f088408afe0 msgr2=0x7f088408b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f0889e15700 1 -- 172.21.15.5:0/2529236677 shutdown_connections 2024-05-08T23:23:21.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.101+0000 7f0889e15700 1 -- 172.21.15.5:0/2529236677 wait complete. 2024-05-08T23:23:21.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.103+0000 7f0889e15700 1 Processor -- start 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.103+0000 7f0889e15700 1 -- start start 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0889e15700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08840d1740 0x7f08840c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0889e15700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f088415b4e0 0x7f088406d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0889e15700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f088415b8c0 0x7f0884152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0889e15700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088406da60 con 0x7f08840d1740 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0889e15700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08840c6fb0 con 0x7f088415b8c0 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0889e15700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08840c7290 con 0x7f088415b4e0 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08840d1740 0x7f08840c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08840d1740 0x7f08840c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55496/0 (socket says 172.21.15.5:55496) 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 -- 172.21.15.5:0/4075924730 learned_addr learned my addr 172.21.15.5:0/4075924730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0883fff700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f088415b8c0 0x7f0884152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0882ffd700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f088415b4e0 0x7f088406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 -- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f088415b4e0 msgr2=0x7f088406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f088415b4e0 0x7f088406d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 -- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f088415b8c0 msgr2=0x7f0884152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f088415b8c0 0x7f0884152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 -- 172.21.15.5:0/4075924730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740094a0 con 0x7f08840d1740 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f0883fff700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f088415b8c0 0x7f0884152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f08837fe700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08840d1740 0x7f08840c69d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f08740090b0 tx=0x7f0874004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.104+0000 7f086bfff700 1 -- 172.21.15.5:0/4075924730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08740038a0 con 0x7f08840d1740 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.105+0000 7f086bfff700 1 -- 172.21.15.5:0/4075924730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087401f070 con 0x7f08840d1740 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.105+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088406bb10 con 0x7f08840d1740 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.105+0000 7f086bfff700 1 -- 172.21.15.5:0/4075924730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874003c20 con 0x7f08840d1740 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.105+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08840c7570 con 0x7f08840d1740 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.106+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088406c030 con 0x7f08840d1740 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.106+0000 7f086bfff700 1 -- 172.21.15.5:0/4075924730 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f087401a430 con 0x7f08840d1740 2024-05-08T23:23:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.107+0000 7f086bfff700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f087006ac90 0x7f087006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.107+0000 7f086bfff700 1 -- 172.21.15.5:0/4075924730 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0874021070 con 0x7f08840d1740 2024-05-08T23:23:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.107+0000 7f0882ffd700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f087006ac90 0x7f087006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.107+0000 7f0882ffd700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f087006ac90 0x7f087006d150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f08841f6e80 tx=0x7f0878008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.111+0000 7f086bfff700 1 -- 172.21.15.5:0/4075924730 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f088406c030 con 0x7f08840d1740 2024-05-08T23:23:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.380+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f088406c030 con 0x7f08840d1740 2024-05-08T23:23:21.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.949+0000 7f086bfff700 1 -- 172.21.15.5:0/4075924730 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f08740150a0 con 0x7f08840d1740 2024-05-08T23:23:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f087006ac90 msgr2=0x7f087006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f087006ac90 0x7f087006d150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f08841f6e80 tx=0x7f0878008040 comp rx=0 tx=0).stop 2024-05-08T23:23:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08840d1740 msgr2=0x7f08840c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08840d1740 0x7f08840c69d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f08740090b0 tx=0x7f0874004750 comp rx=0 tx=0).stop 2024-05-08T23:23:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 shutdown_connections 2024-05-08T23:23:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f087006ac90 0x7f087006d150 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f088415b8c0 0x7f0884152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08840d1740 0x7f08840c69d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 --2- 172.21.15.5:0/4075924730 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f088415b4e0 0x7f088406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 >> 172.21.15.5:0/4075924730 conn(0x7f088408afe0 msgr2=0x7f08840b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 shutdown_connections 2024-05-08T23:23:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:21.952+0000 7f0889e15700 1 -- 172.21.15.5:0/4075924730 wait complete. 2024-05-08T23:23:21.969 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.159+0000 7fcd7bc99700 1 Processor -- start 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.159+0000 7fcd7bc99700 1 -- start start 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd7bc99700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcd740aea70 0x7fcd740b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd7bc99700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7415add0 con 0x7fcd74154df0 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd7bc99700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7415b070 con 0x7fcd740d0a60 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd7bc99700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7415abe0 con 0x7fcd740aea70 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd79234700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcd740aea70 0x7fcd740b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd79234700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcd740aea70 0x7fcd740b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55506/0 (socket says 172.21.15.5:55506) 2024-05-08T23:23:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd79234700 1 -- 172.21.15.5:0/2945454082 learned_addr learned my addr 172.21.15.5:0/2945454082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd79a35700 1 --1- 172.21.15.5:0/2945454082 >> v1:172.21.15.5:6789/0 conn(0x7fcd74154df0 0x7fcd740d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51214/0 (socket says 172.21.15.5:51214) 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd79234700 1 -- 172.21.15.5:0/2945454082 >> v1:172.21.15.49:6789/0 conn(0x7fcd740d0a60 legacy=0x7fcd740b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd79234700 1 -- 172.21.15.5:0/2945454082 >> v1:172.21.15.5:6789/0 conn(0x7fcd74154df0 legacy=0x7fcd740d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd79234700 1 -- 172.21.15.5:0/2945454082 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7415b350 con 0x7fcd740aea70 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.160+0000 7fcd78a33700 1 -- 172.21.15.5:0/2945454082 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568646416 0 0) 0x7fcd7415add0 con 0x7fcd74154df0 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd79234700 1 --2- 172.21.15.5:0/2945454082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd740aea70 0x7fcd740b3e70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400bb90 tx=0x7fcd6400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2619046dd0a7ba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd78a33700 1 -- 172.21.15.5:0/2945454082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400dd50 con 0x7fcd740aea70 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd78a33700 1 -- 172.21.15.5:0/2945454082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd64010470 con 0x7fcd740aea70 2024-05-08T23:23:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd78a33700 1 -- 172.21.15.5:0/2945454082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64010800 con 0x7fcd740aea70 2024-05-08T23:23:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd7bc99700 1 -- 172.21.15.5:0/2945454082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd740aea70 msgr2=0x7fcd740b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd7bc99700 1 --2- 172.21.15.5:0/2945454082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd740aea70 0x7fcd740b3e70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400bb90 tx=0x7fcd6400bea0 comp rx=0 tx=0).stop 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.161+0000 7fcd7bc99700 1 -- 172.21.15.5:0/2945454082 shutdown_connections 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.162+0000 7fcd7bc99700 1 --2- 172.21.15.5:0/2945454082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd740aea70 0x7fcd740b3e70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.162+0000 7fcd7bc99700 1 -- 172.21.15.5:0/2945454082 >> 172.21.15.5:0/2945454082 conn(0x7fcd74099a50 msgr2=0x7fcd740b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.162+0000 7fcd7bc99700 1 -- 172.21.15.5:0/2945454082 shutdown_connections 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.162+0000 7fcd7bc99700 1 -- 172.21.15.5:0/2945454082 wait complete. 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.163+0000 7fcd7bc99700 1 Processor -- start 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.163+0000 7fcd7bc99700 1 -- start start 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7bc99700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd740aea70 0x7fcd740cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7bc99700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd740d0a60 0x7fcd741ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7bc99700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd74154df0 0x7fcd741eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7bc99700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740b0a40 con 0x7fcd74154df0 2024-05-08T23:23:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7bc99700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740cc720 con 0x7fcd740aea70 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7bc99700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740cc9c0 con 0x7fcd740d0a60 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd74154df0 0x7fcd741eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd74154df0 0x7fcd741eefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55514/0 (socket says 172.21.15.5:55514) 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 -- 172.21.15.5:0/132738775 learned_addr learned my addr 172.21.15.5:0/132738775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd79a35700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd740aea70 0x7fcd740cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd79234700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd740d0a60 0x7fcd741ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 -- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd740d0a60 msgr2=0x7fcd741ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd740d0a60 0x7fcd741ec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 -- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd740aea70 msgr2=0x7fcd740cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd740aea70 0x7fcd740cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 -- 172.21.15.5:0/132738775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6400b7f0 con 0x7fcd74154df0 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd7a236700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd74154df0 0x7fcd741eefc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00cc50 tx=0x7fcd6c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.164+0000 7fcd6a7fc700 1 -- 172.21.15.5:0/132738775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c003c00 con 0x7fcd74154df0 2024-05-08T23:23:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.165+0000 7fcd6a7fc700 1 -- 172.21.15.5:0/132738775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c003d60 con 0x7fcd74154df0 2024-05-08T23:23:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.165+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd741eca10 con 0x7fcd74154df0 2024-05-08T23:23:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.165+0000 7fcd6a7fc700 1 -- 172.21.15.5:0/132738775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00aab0 con 0x7fcd74154df0 2024-05-08T23:23:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.165+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd741efae0 con 0x7fcd74154df0 2024-05-08T23:23:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.166+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd740ccb30 con 0x7fcd74154df0 2024-05-08T23:23:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.166+0000 7fcd6a7fc700 1 -- 172.21.15.5:0/132738775 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcd6c00ac10 con 0x7fcd74154df0 2024-05-08T23:23:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.167+0000 7fcd6a7fc700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcd5806ac90 0x7fcd5806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.167+0000 7fcd6a7fc700 1 -- 172.21.15.5:0/132738775 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fcd6c089400 con 0x7fcd74154df0 2024-05-08T23:23:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.167+0000 7fcd79a35700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcd5806ac90 0x7fcd5806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.167+0000 7fcd79a35700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcd5806ac90 0x7fcd5806d150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd70000f60 tx=0x7fcd70004a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.171+0000 7fcd6a7fc700 1 -- 172.21.15.5:0/132738775 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcd6c0544e0 con 0x7fcd74154df0 2024-05-08T23:23:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.390+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd74074bc0 con 0x7fcd74154df0 2024-05-08T23:23:22.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.954+0000 7fcd6a7fc700 1 -- 172.21.15.5:0/132738775 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcd6c057b00 con 0x7fcd74154df0 2024-05-08T23:23:22.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.960+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcd5806ac90 msgr2=0x7fcd5806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.960+0000 7fcd7bc99700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcd5806ac90 0x7fcd5806d150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd70000f60 tx=0x7fcd70004a50 comp rx=0 tx=0).stop 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.960+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd74154df0 msgr2=0x7fcd741eefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.960+0000 7fcd7bc99700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd74154df0 0x7fcd741eefc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00cc50 tx=0x7fcd6c00cf60 comp rx=0 tx=0).stop 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.961+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 shutdown_connections 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.961+0000 7fcd7bc99700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcd5806ac90 0x7fcd5806d150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.961+0000 7fcd7bc99700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd74154df0 0x7fcd741eefc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.961+0000 7fcd7bc99700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd740aea70 0x7fcd740cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.961+0000 7fcd7bc99700 1 --2- 172.21.15.5:0/132738775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd740d0a60 0x7fcd741ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.961+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 >> 172.21.15.5:0/132738775 conn(0x7fcd74099a50 msgr2=0x7fcd74152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.961+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 shutdown_connections 2024-05-08T23:23:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:22.961+0000 7fcd7bc99700 1 -- 172.21.15.5:0/132738775 wait complete. 2024-05-08T23:23:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:23:22.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:23:23.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.122+0000 7fbfb91fa700 1 Processor -- start 2024-05-08T23:23:23.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.122+0000 7fbfb91fa700 1 -- start start 2024-05-08T23:23:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb91fa700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb40cdc90 0x7fbfb40b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb91fa700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb40b1200 con 0x7fbfb406bd60 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb91fa700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb40ce5c0 con 0x7fbfb40d13a0 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb91fa700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40ce8a0 con 0x7fbfb40cdc90 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb259c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbfb406bd60 0x7fbfb40cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51218/0 (socket says 172.21.15.5:51218) 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb259c700 1 -- 172.21.15.5:0/2583283079 learned_addr learned my addr 172.21.15.5:0/2583283079 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb359e700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb40cdc90 0x7fbfb40b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208305903 0 0) 0x7fbfb40b1200 con 0x7fbfb406bd60 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa00035b0 con 0x7fbfb406bd60 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115776704 0 0) 0x7fbfb40ce5c0 con 0x7fbfb40d13a0 2024-05-08T23:23:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb40b1200 con 0x7fbfb40d13a0 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.123+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1487900998 0 0) 0x7fbfa00035b0 con 0x7fbfb406bd60 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb40cdc90 msgr2=0x7fbfb40b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb40cdc90 0x7fbfb40b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 >> v1:172.21.15.49:6789/0 conn(0x7fbfb40d13a0 legacy=0x7fbfb40d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40ceb80 con 0x7fbfb406bd60 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fbf9c001670 con 0x7fbfb406bd60 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 >> v1:172.21.15.5:6789/0 conn(0x7fbfb406bd60 legacy=0x7fbfb40cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0003bc0 0x7fbfa0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfa0004850 0x7fbfa0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb40cdc90 0x7fbfa0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40ce8a0 con 0x7fbfa0004850 2024-05-08T23:23:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa000baf0 con 0x7fbfa0003bc0 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa000bcb0 con 0x7fbfb40cdc90 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb259c700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfa0004850 0x7fbfa0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb2d9d700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0003bc0 0x7fbfa0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb359e700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb40cdc90 0x7fbfa0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb259c700 1 -- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb40cdc90 msgr2=0x7fbfa0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb259c700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb40cdc90 0x7fbfa0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb259c700 1 -- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0003bc0 msgr2=0x7fbfa0003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb259c700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0003bc0 0x7fbfa0003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb259c700 1 -- 172.21.15.5:0/2583283079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa00035b0 con 0x7fbfa0004850 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb259c700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfa0004850 0x7fbfa0004cd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00c110 tx=0x7fbfa0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afdb676a1b600bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00ccb0 con 0x7fbfa0004850 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9c00ce10 con 0x7fbfa0004850 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.124+0000 7fbfb1d9b700 1 -- 172.21.15.5:0/2583283079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c005ba0 con 0x7fbfa0004850 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.125+0000 7fbfb91fa700 1 -- 172.21.15.5:0/2583283079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfa0004850 msgr2=0x7fbfa0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.125+0000 7fbfb91fa700 1 --2- 172.21.15.5:0/2583283079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfa0004850 0x7fbfa0004cd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00c110 tx=0x7fbfa0005880 comp rx=0 tx=0).stop 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.125+0000 7fbfb359e700 1 -- 172.21.15.5:0/2583283079 reap_dead start 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.125+0000 7fbfb91fa700 1 -- 172.21.15.5:0/2583283079 shutdown_connections 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.125+0000 7fbfb91fa700 1 -- 172.21.15.5:0/2583283079 >> 172.21.15.5:0/2583283079 conn(0x7fbfb408afe0 msgr2=0x7fbfb408b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.125+0000 7fbfb91fa700 1 -- 172.21.15.5:0/2583283079 shutdown_connections 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.125+0000 7fbfb91fa700 1 -- 172.21.15.5:0/2583283079 wait complete. 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.127+0000 7fbfb91fa700 1 Processor -- start 2024-05-08T23:23:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.127+0000 7fbfb91fa700 1 -- start start 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb91fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0004850 0x7fbfb40b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb91fa700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb4156b70 0x7fbfb4156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb91fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb40c8810 0x7fbfb40c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb91fa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40b6770 con 0x7fbfb40c8810 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb91fa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4157530 con 0x7fbfa0004850 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb91fa700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40b5b60 con 0x7fbfb4156b70 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb40c8810 0x7fbfb40c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb40c8810 0x7fbfb40c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55524/0 (socket says 172.21.15.5:55524) 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 -- 172.21.15.5:0/4281419201 learned_addr learned my addr 172.21.15.5:0/4281419201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb2d9d700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0004850 0x7fbfb40b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb259c700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb4156b70 0x7fbfb4156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 -- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb4156b70 msgr2=0x7fbfb4156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb4156b70 0x7fbfb4156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 -- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0004850 msgr2=0x7fbfb40b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0004850 0x7fbfb40b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 -- 172.21.15.5:0/4281419201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40b5e40 con 0x7fbfb40c8810 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb259c700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb4156b70 0x7fbfb4156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb2d9d700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0004850 0x7fbfb40b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfb359e700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb40c8810 0x7fbfb40c8c90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40cf210 tx=0x7fbfac0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.128+0000 7fbfab7fe700 1 -- 172.21.15.5:0/4281419201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00e070 con 0x7fbfb40c8810 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.129+0000 7fbfab7fe700 1 -- 172.21.15.5:0/4281419201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac003cf0 con 0x7fbfb40c8810 2024-05-08T23:23:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.129+0000 7fbfab7fe700 1 -- 172.21.15.5:0/4281419201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac003ea0 con 0x7fbfb40c8810 2024-05-08T23:23:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.129+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb40b6120 con 0x7fbfb40c8810 2024-05-08T23:23:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.129+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb40b5740 con 0x7fbfb40c8810 2024-05-08T23:23:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.129+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb40b5900 con 0x7fbfb40c8810 2024-05-08T23:23:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.130+0000 7fbfab7fe700 1 -- 172.21.15.5:0/4281419201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbfac011a10 con 0x7fbfb40c8810 2024-05-08T23:23:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.131+0000 7fbfab7fe700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf9406ac90 0x7fbf9406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.131+0000 7fbfab7fe700 1 -- 172.21.15.5:0/4281419201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5529+0+0 (secure 0 0 0) 0x7fbfac089600 con 0x7fbfb40c8810 2024-05-08T23:23:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.131+0000 7fbfb2d9d700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf9406ac90 0x7fbf9406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.131+0000 7fbfb2d9d700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf9406ac90 0x7fbf9406d150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4007fd0 tx=0x7fbfa4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.135+0000 7fbfab7fe700 1 -- 172.21.15.5:0/4281419201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbfac054920 con 0x7fbfb40c8810 2024-05-08T23:23:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.351+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbfb4074bc0 con 0x7fbfb40c8810 2024-05-08T23:23:23.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.958+0000 7fbfab7fe700 1 -- 172.21.15.5:0/4281419201 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbfac057f40 con 0x7fbfb40c8810 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf9406ac90 msgr2=0x7fbf9406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf9406ac90 0x7fbf9406d150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4007fd0 tx=0x7fbfa4009040 comp rx=0 tx=0).stop 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb40c8810 msgr2=0x7fbfb40c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb40c8810 0x7fbfb40c8c90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40cf210 tx=0x7fbfac0114b0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 shutdown_connections 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf9406ac90 0x7fbf9406d150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb40c8810 0x7fbfb40c8c90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa0004850 0x7fbfb40b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 --2- 172.21.15.5:0/4281419201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbfb4156b70 0x7fbfb4156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:23.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 >> 172.21.15.5:0/4281419201 conn(0x7fbfb408afe0 msgr2=0x7fbfb40cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:23.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.965+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 shutdown_connections 2024-05-08T23:23:23.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:23.966+0000 7fbfb91fa700 1 -- 172.21.15.5:0/4281419201 wait complete. 2024-05-08T23:23:23.968 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-08T23:23:23.984 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T23:23:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.126+0000 7fbb4a234700 1 Processor -- start 2024-05-08T23:23:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.126+0000 7fbb4a234700 1 -- start start 2024-05-08T23:23:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb4a234700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb440ae800 0x7fbb4415a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb4a234700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb44071780 0x7fbb440b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb4a234700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb44071c10 0x7fbb440b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb4a234700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb44072ae0 con 0x7fbb44071780 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb4a234700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4406d9f0 con 0x7fbb440ae800 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb4a234700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4406cb90 con 0x7fbb44071c10 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb42ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb44071780 0x7fbb440b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb42ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb44071780 0x7fbb440b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55528/0 (socket says 172.21.15.5:55528) 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.127+0000 7fbb42ffd700 1 -- 172.21.15.5:0/3790825607 learned_addr learned my addr 172.21.15.5:0/3790825607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb43fff700 1 --2- 172.21.15.5:0/3790825607 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb44071c10 0x7fbb440b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb437fe700 1 --2- 172.21.15.5:0/3790825607 >> v2:172.21.15.49:3300/0 conn(0x7fbb440ae800 0x7fbb4415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb42ffd700 1 -- 172.21.15.5:0/3790825607 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb44071c10 msgr2=0x7fbb440b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb42ffd700 1 --2- 172.21.15.5:0/3790825607 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb44071c10 0x7fbb440b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb42ffd700 1 -- 172.21.15.5:0/3790825607 >> v2:172.21.15.49:3300/0 conn(0x7fbb440ae800 msgr2=0x7fbb4415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb42ffd700 1 --2- 172.21.15.5:0/3790825607 >> v2:172.21.15.49:3300/0 conn(0x7fbb440ae800 0x7fbb4415a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb42ffd700 1 -- 172.21.15.5:0/3790825607 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4406ce70 con 0x7fbb44071780 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb43fff700 1 --2- 172.21.15.5:0/3790825607 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb44071c10 0x7fbb440b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb437fe700 1 --2- 172.21.15.5:0/3790825607 >> v2:172.21.15.49:3300/0 conn(0x7fbb440ae800 0x7fbb4415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb42ffd700 1 --2- 172.21.15.5:0/3790825607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071780 0x7fbb440b3460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbb440712b0 tx=0x7fbb3800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea07c1299e51bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb427fc700 1 -- 172.21.15.5:0/3790825607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800dd30 con 0x7fbb44071780 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb427fc700 1 -- 172.21.15.5:0/3790825607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3800de90 con 0x7fbb44071780 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.128+0000 7fbb427fc700 1 -- 172.21.15.5:0/3790825607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38010690 con 0x7fbb44071780 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 -- 172.21.15.5:0/3790825607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071780 msgr2=0x7fbb440b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 --2- 172.21.15.5:0/3790825607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071780 0x7fbb440b3460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbb440712b0 tx=0x7fbb3800be70 comp rx=0 tx=0).stop 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 -- 172.21.15.5:0/3790825607 shutdown_connections 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 --2- 172.21.15.5:0/3790825607 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb44071c10 0x7fbb440b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 --2- 172.21.15.5:0/3790825607 >> v2:172.21.15.49:3300/0 conn(0x7fbb440ae800 0x7fbb4415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 --2- 172.21.15.5:0/3790825607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071780 0x7fbb440b3460 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 -- 172.21.15.5:0/3790825607 >> 172.21.15.5:0/3790825607 conn(0x7fbb440b10c0 msgr2=0x7fbb440b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 -- 172.21.15.5:0/3790825607 shutdown_connections 2024-05-08T23:23:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.129+0000 7fbb4a234700 1 -- 172.21.15.5:0/3790825607 wait complete. 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.131+0000 7fbb4a234700 1 Processor -- start 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.131+0000 7fbb4a234700 1 -- start start 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb4a234700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44071780 0x7fbb441f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb4a234700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071c10 0x7fbb44144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb4a234700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb440ae800 0x7fbb440ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb4a234700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44145d10 con 0x7fbb44071c10 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb4a234700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44145100 con 0x7fbb44071780 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb4a234700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441453e0 con 0x7fbb440ae800 2024-05-08T23:23:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071c10 0x7fbb44144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071c10 0x7fbb44144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55544/0 (socket says 172.21.15.5:55544) 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb42ffd700 1 -- 172.21.15.5:0/1923289452 learned_addr learned my addr 172.21.15.5:0/1923289452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb437fe700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44071780 0x7fbb441f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb43fff700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb440ae800 0x7fbb440ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb42ffd700 1 -- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb440ae800 msgr2=0x7fbb440ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb42ffd700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb440ae800 0x7fbb440ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb42ffd700 1 -- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44071780 msgr2=0x7fbb441f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb42ffd700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44071780 0x7fbb441f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb42ffd700 1 -- 172.21.15.5:0/1923289452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3800b7f0 con 0x7fbb44071c10 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.132+0000 7fbb437fe700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44071780 0x7fbb441f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.133+0000 7fbb42ffd700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071c10 0x7fbb44144640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800beb0 tx=0x7fbb38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.133+0000 7fbb2bfff700 1 -- 172.21.15.5:0/1923289452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38003da0 con 0x7fbb44071c10 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.133+0000 7fbb2bfff700 1 -- 172.21.15.5:0/1923289452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb38010470 con 0x7fbb44071c10 2024-05-08T23:23:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.133+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb441456c0 con 0x7fbb44071c10 2024-05-08T23:23:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.133+0000 7fbb2bfff700 1 -- 172.21.15.5:0/1923289452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38010820 con 0x7fbb44071c10 2024-05-08T23:23:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.133+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb44099500 con 0x7fbb44071c10 2024-05-08T23:23:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.134+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb44074b50 con 0x7fbb44071c10 2024-05-08T23:23:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.134+0000 7fbb2bfff700 1 -- 172.21.15.5:0/1923289452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbb38019400 con 0x7fbb44071c10 2024-05-08T23:23:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.135+0000 7fbb2bfff700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb3006ac90 0x7fbb3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.135+0000 7fbb2bfff700 1 -- 172.21.15.5:0/1923289452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbb3808a540 con 0x7fbb44071c10 2024-05-08T23:23:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.135+0000 7fbb437fe700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb3006ac90 0x7fbb3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.136+0000 7fbb437fe700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb3006ac90 0x7fbb3006d150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb441f6920 tx=0x7fbb34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:24.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.140+0000 7fbb2bfff700 1 -- 172.21.15.5:0/1923289452 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbb38055a00 con 0x7fbb44071c10 2024-05-08T23:23:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.363+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbb440b2600 con 0x7fbb44071c10 2024-05-08T23:23:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.367+0000 7fbb2bfff700 1 -- 172.21.15.5:0/1923289452 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb38059020 con 0x7fbb44071c10 2024-05-08T23:23:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb3006ac90 msgr2=0x7fbb3006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb3006ac90 0x7fbb3006d150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb441f6920 tx=0x7fbb34008040 comp rx=0 tx=0).stop 2024-05-08T23:23:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071c10 msgr2=0x7fbb44144640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071c10 0x7fbb44144640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800beb0 tx=0x7fbb38004750 comp rx=0 tx=0).stop 2024-05-08T23:23:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 shutdown_connections 2024-05-08T23:23:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb3006ac90 0x7fbb3006d150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb440ae800 0x7fbb440ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44071780 0x7fbb441f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 --2- 172.21.15.5:0/1923289452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb44071c10 0x7fbb44144640 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 >> 172.21.15.5:0/1923289452 conn(0x7fbb440b10c0 msgr2=0x7fbb440b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 shutdown_connections 2024-05-08T23:23:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.369+0000 7fbb4a234700 1 -- 172.21.15.5:0/1923289452 wait complete. 2024-05-08T23:23:24.383 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-05-08T23:23:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.519+0000 7fd484a80700 1 Processor -- start 2024-05-08T23:23:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.519+0000 7fd484a80700 1 -- start start 2024-05-08T23:23:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd484a80700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd4800ce120 0x7fd48006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd484a80700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4800cb2a0 0x7fd4800cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd484a80700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48006fbf0 con 0x7fd48015db00 2024-05-08T23:23:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd484a80700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48006edb0 con 0x7fd4800ce120 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd484a80700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48006f090 con 0x7fd4800cb2a0 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47dd9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd4800ce120 0x7fd48006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47e59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd48015db00 0x7fd48006dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51220/0 (socket says 172.21.15.5:51220) 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47e59c700 1 -- 172.21.15.5:0/3715865798 learned_addr learned my addr 172.21.15.5:0/3715865798 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47ed9d700 1 --2- 172.21.15.5:0/3715865798 >> v2:172.21.15.49:3300/0 conn(0x7fd4800cb2a0 0x7fd4800cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47dd9b700 1 -- 172.21.15.5:0/3715865798 >> v2:172.21.15.49:3300/0 conn(0x7fd4800cb2a0 msgr2=0x7fd4800cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47dd9b700 1 --2- 172.21.15.5:0/3715865798 >> v2:172.21.15.49:3300/0 conn(0x7fd4800cb2a0 0x7fd4800cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47dd9b700 1 -- 172.21.15.5:0/3715865798 >> v1:172.21.15.5:6789/0 conn(0x7fd48015db00 legacy=0x7fd48006dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47d59a700 1 -- 172.21.15.5:0/3715865798 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409246070 0 0) 0x7fd48006fbf0 con 0x7fd48015db00 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47dd9b700 1 -- 172.21.15.5:0/3715865798 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48006f370 con 0x7fd4800ce120 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.520+0000 7fd47dd9b700 1 --2- 172.21.15.5:0/3715865798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd48006e390 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd4800727d0 tx=0x7fd468009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaafecd47d39149 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd47d59a700 1 -- 172.21.15.5:0/3715865798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468011830 con 0x7fd4800ce120 2024-05-08T23:23:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd47d59a700 1 -- 172.21.15.5:0/3715865798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468011990 con 0x7fd4800ce120 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd47d59a700 1 -- 172.21.15.5:0/3715865798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468011d20 con 0x7fd4800ce120 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd484a80700 1 -- 172.21.15.5:0/3715865798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 msgr2=0x7fd48006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd484a80700 1 --2- 172.21.15.5:0/3715865798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd48006e390 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd4800727d0 tx=0x7fd468009b20 comp rx=0 tx=0).stop 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd484a80700 1 -- 172.21.15.5:0/3715865798 shutdown_connections 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd484a80700 1 --2- 172.21.15.5:0/3715865798 >> v2:172.21.15.49:3300/0 conn(0x7fd4800cb2a0 0x7fd4800cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd484a80700 1 --2- 172.21.15.5:0/3715865798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd48006e390 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd484a80700 1 -- 172.21.15.5:0/3715865798 >> 172.21.15.5:0/3715865798 conn(0x7fd480099a50 msgr2=0x7fd4800d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd484a80700 1 -- 172.21.15.5:0/3715865798 shutdown_connections 2024-05-08T23:23:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.521+0000 7fd484a80700 1 -- 172.21.15.5:0/3715865798 wait complete. 2024-05-08T23:23:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.523+0000 7fd484a80700 1 Processor -- start 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.523+0000 7fd484a80700 1 -- start start 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd484a80700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4800cb2a0 0x7fd4801f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd484a80700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd4800c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd484a80700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd48015db00 0x7fd4800ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd484a80700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801520b0 con 0x7fd4800ce120 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd484a80700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800c4470 con 0x7fd4800cb2a0 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd484a80700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801514a0 con 0x7fd48015db00 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd4800c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd4800c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55558/0 (socket says 172.21.15.5:55558) 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 -- 172.21.15.5:0/1092796382 learned_addr learned my addr 172.21.15.5:0/1092796382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47e59c700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4800cb2a0 0x7fd4801f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47ed9d700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd48015db00 0x7fd4800ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 -- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd48015db00 msgr2=0x7fd4800ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd48015db00 0x7fd4800ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 -- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4800cb2a0 msgr2=0x7fd4801f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4800cb2a0 0x7fd4801f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 -- 172.21.15.5:0/1092796382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680094a0 con 0x7fd4800ce120 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47e59c700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4800cb2a0 0x7fd4801f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd47dd9b700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd4800c3f30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd4680090b0 tx=0x7fd468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.524+0000 7fd476ffd700 1 -- 172.21.15.5:0/1092796382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680038a0 con 0x7fd4800ce120 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.525+0000 7fd476ffd700 1 -- 172.21.15.5:0/1092796382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46801f070 con 0x7fd4800ce120 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.525+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd480151780 con 0x7fd4800ce120 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.525+0000 7fd476ffd700 1 -- 172.21.15.5:0/1092796382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468003c20 con 0x7fd4800ce120 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.525+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd480151d40 con 0x7fd4800ce120 2024-05-08T23:23:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.526+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd480151ed0 con 0x7fd4800ce120 2024-05-08T23:23:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.526+0000 7fd476ffd700 1 -- 172.21.15.5:0/1092796382 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd46801a430 con 0x7fd4800ce120 2024-05-08T23:23:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.527+0000 7fd476ffd700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd46006ac90 0x7fd46006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.527+0000 7fd476ffd700 1 -- 172.21.15.5:0/1092796382 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fd468021070 con 0x7fd4800ce120 2024-05-08T23:23:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.527+0000 7fd47e59c700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd46006ac90 0x7fd46006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.527+0000 7fd47e59c700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd46006ac90 0x7fd46006d150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd470000f60 tx=0x7fd470009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.531+0000 7fd476ffd700 1 -- 172.21.15.5:0/1092796382 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd468055ef0 con 0x7fd4800ce120 2024-05-08T23:23:24.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.841+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) v1 -- 0x7fd480068000 con 0x7fd4800ce120 2024-05-08T23:23:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.965+0000 7fd476ffd700 1 -- 172.21.15.5:0/1092796382 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v136) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd4680150c0 con 0x7fd4800ce120 2024-05-08T23:23:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd46006ac90 msgr2=0x7fd46006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd46006ac90 0x7fd46006d150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd470000f60 tx=0x7fd470009040 comp rx=0 tx=0).stop 2024-05-08T23:23:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 msgr2=0x7fd4800c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd4800c3f30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd4680090b0 tx=0x7fd468004750 comp rx=0 tx=0).stop 2024-05-08T23:23:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 shutdown_connections 2024-05-08T23:23:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd46006ac90 0x7fd46006d150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd48015db00 0x7fd4800ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4800cb2a0 0x7fd4801f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 --2- 172.21.15.5:0/1092796382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4800ce120 0x7fd4800c3f30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 >> 172.21.15.5:0/1092796382 conn(0x7fd480099a50 msgr2=0x7fd4800d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 shutdown_connections 2024-05-08T23:23:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:24.968+0000 7fd484a80700 1 -- 172.21.15.5:0/1092796382 wait complete. 2024-05-08T23:23:24.983 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-05-08T23:23:24.983 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-05-08T23:23:24.984 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.122+0000 7f6c325f7700 1 Processor -- start 2024-05-08T23:23:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.122+0000 7f6c325f7700 1 -- start start 2024-05-08T23:23:25.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c325f7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c15e830 0x7f6c2c0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c325f7700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6c2c06b0e0 0x7f6c2c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c325f7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0d0a90 con 0x7f6c2c0d0f20 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c325f7700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c06b720 con 0x7f6c2c06b0e0 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c325f7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0d0440 con 0x7f6c2c15e830 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c2b7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6c2c0d0f20 0x7f6c2c06a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51230/0 (socket says 172.21.15.5:51230) 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c2b7fe700 1 -- 172.21.15.5:0/2089619360 learned_addr learned my addr 172.21.15.5:0/2089619360 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2089619360 >> v2:172.21.15.49:3300/0 conn(0x7f6c2c06b0e0 0x7f6c2c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c2bfff700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c15e830 0x7f6c2c0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162079630 0 0) 0x7f6c2c0d0a90 con 0x7f6c2c0d0f20 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.123+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c140035b0 con 0x7f6c2c0d0f20 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132076715 0 0) 0x7f6c140035b0 con 0x7f6c2c0d0f20 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c15e830 msgr2=0x7f6c2c0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:25.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c15e830 0x7f6c2c0ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 >> v2:172.21.15.49:3300/0 conn(0x7f6c2c06b0e0 msgr2=0x7f6c2c0cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 --2- 172.21.15.5:0/2089619360 >> v2:172.21.15.49:3300/0 conn(0x7f6c2c06b0e0 0x7f6c2c0cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0d0720 con 0x7f6c2c0d0f20 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f6c20001670 con 0x7f6c2c0d0f20 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2089619360 >> v2:172.21.15.49:3300/0 conn(0x7f6c2c06b0e0 0x7f6c2c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 >> v1:172.21.15.5:6789/0 conn(0x7f6c2c0d0f20 legacy=0x7f6c2c06a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c15e830 0x7f6c14003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14003f80 0x7f6c14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c14004d90 0x7f6c14005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c06b720 con 0x7f6c14004d90 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0d0440 con 0x7f6c14003f80 2024-05-08T23:23:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1400ba10 con 0x7f6c2c15e830 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c14004d90 0x7f6c14005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2bfff700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c15e830 0x7f6c14003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c2b7fe700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14003f80 0x7f6c14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c30b94700 1 -- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c15e830 msgr2=0x7f6c14003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c15e830 0x7f6c14003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c30b94700 1 -- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14003f80 msgr2=0x7f6c14004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14003f80 0x7f6c14004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.124+0000 7f6c30b94700 1 -- 172.21.15.5:0/2089619360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c140035b0 con 0x7f6c14004d90 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c14004d90 0x7f6c14005210 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400e350 tx=0x7f6c14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bd08c6842274d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400ed90 con 0x7f6c14004d90 2024-05-08T23:23:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2400f040 con 0x7f6c14004d90 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c2affd700 1 -- 172.21.15.5:0/2089619360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240146a0 con 0x7f6c14004d90 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c325f7700 1 -- 172.21.15.5:0/2089619360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c14004d90 msgr2=0x7f6c14005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c325f7700 1 --2- 172.21.15.5:0/2089619360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c14004d90 0x7f6c14005210 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400e350 tx=0x7f6c14005880 comp rx=0 tx=0).stop 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c30b94700 1 -- 172.21.15.5:0/2089619360 reap_dead start 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c325f7700 1 -- 172.21.15.5:0/2089619360 shutdown_connections 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c325f7700 1 -- 172.21.15.5:0/2089619360 >> 172.21.15.5:0/2089619360 conn(0x7f6c2c0b5810 msgr2=0x7f6c2c0b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c325f7700 1 -- 172.21.15.5:0/2089619360 shutdown_connections 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.125+0000 7f6c325f7700 1 -- 172.21.15.5:0/2089619360 wait complete. 2024-05-08T23:23:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.127+0000 7f6c325f7700 1 Processor -- start 2024-05-08T23:23:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.127+0000 7f6c325f7700 1 -- start start 2024-05-08T23:23:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c325f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14004d90 0x7f6c2c149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c325f7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c0b0f20 0x7f6c2c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c325f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2c15c3b0 0x7f6c2c15c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c325f7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0cb8b0 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c325f7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c15cd70 con 0x7f6c14004d90 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c325f7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0b19a0 con 0x7f6c2c0b0f20 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2c15c3b0 0x7f6c2c15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2c15c3b0 0x7f6c2c15c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55588/0 (socket says 172.21.15.5:55588) 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 -- 172.21.15.5:0/2575487059 learned_addr learned my addr 172.21.15.5:0/2575487059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c2bfff700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14004d90 0x7f6c2c149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c2b7fe700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c0b0f20 0x7f6c2c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 -- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c0b0f20 msgr2=0x7f6c2c0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c0b0f20 0x7f6c2c0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 -- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14004d90 msgr2=0x7f6c2c149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14004d90 0x7f6c2c149e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 -- 172.21.15.5:0/2575487059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0caf10 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.128+0000 7f6c30b94700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2c15c3b0 0x7f6c2c15c830 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6c24017040 tx=0x7f6c24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.129+0000 7f6c28ff9700 1 -- 172.21.15.5:0/2575487059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24004450 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.129+0000 7f6c28ff9700 1 -- 172.21.15.5:0/2575487059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c240045d0 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.129+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c0cb1f0 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.129+0000 7f6c28ff9700 1 -- 172.21.15.5:0/2575487059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24009410 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.129+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c0b0c50 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.130+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c068000 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.130+0000 7f6c28ff9700 1 -- 172.21.15.5:0/2575487059 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6c24024070 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.131+0000 7f6c28ff9700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6c1806ac90 0x7f6c1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.131+0000 7f6c28ff9700 1 -- 172.21.15.5:0/2575487059 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6c2408a5f0 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.131+0000 7f6c2bfff700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6c1806ac90 0x7f6c1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.131+0000 7f6c2bfff700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6c1806ac90 0x7f6c1806d150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009ba0 tx=0x7f6c1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.135+0000 7f6c28ff9700 1 -- 172.21.15.5:0/2575487059 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6c24055b30 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.354+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c2c0cdf40 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.354+0000 7f6c28ff9700 1 -- 172.21.15.5:0/2575487059 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12672 (secure 0 0 0) 0x7f6c24059150 con 0x7f6c2c15c3b0 2024-05-08T23:23:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:25.355 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:23.953299+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/295306227":"2024-05-09T23:23:20.932703+0000","172.21.15.5:6832/295306227":"2024-05-09T23:23:20.932703+0000","172.21.15.5:6835/146081129":"2024-05-09T23:23:20.932719+0000","172.21.15.5:6833/146081129":"2024-05-09T23:23:20.932719+0000","172.21.15.49:6835/886789668":"2024-05-09T23:23:20.932724+0000","172.21.15.49:6837/886789668":"2024-05-09T23:23:20.932724+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6c1806ac90 msgr2=0x7f6c1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6c1806ac90 0x7f6c1806d150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009ba0 tx=0x7f6c1c008040 comp rx=0 tx=0).stop 2024-05-08T23:23:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2c15c3b0 msgr2=0x7f6c2c15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2c15c3b0 0x7f6c2c15c830 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6c24017040 tx=0x7f6c24009750 comp rx=0 tx=0).stop 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 shutdown_connections 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6c1806ac90 0x7f6c1806d150 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2c15c3b0 0x7f6c2c15c830 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14004d90 0x7f6c2c149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 --2- 172.21.15.5:0/2575487059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6c2c0b0f20 0x7f6c2c0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 >> 172.21.15.5:0/2575487059 conn(0x7f6c2c0b5810 msgr2=0x7f6c2c0af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 shutdown_connections 2024-05-08T23:23:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.357+0000 7f6c325f7700 1 -- 172.21.15.5:0/2575487059 wait complete. 2024-05-08T23:23:25.371 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:23:25.372 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:23:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.511+0000 7ff49cb22700 1 Processor -- start 2024-05-08T23:23:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.511+0000 7ff49cb22700 1 -- start start 2024-05-08T23:23:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.512+0000 7ff49cb22700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff49806bd60 0x7ff4980cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.512+0000 7ff49cb22700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4980cdc90 0x7ff4980b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.512+0000 7ff49cb22700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980b1200 con 0x7ff4980d13a0 2024-05-08T23:23:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.512+0000 7ff49cb22700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4980cfcc0 con 0x7ff49806bd60 2024-05-08T23:23:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.512+0000 7ff49cb22700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4980ce5c0 con 0x7ff4980cdc90 2024-05-08T23:23:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff495d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff49806bd60 0x7ff4980cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff49659c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff4980d13a0 0x7ff4980d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51246/0 (socket says 172.21.15.5:51246) 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff49659c700 1 -- 172.21.15.5:0/2084636573 learned_addr learned my addr 172.21.15.5:0/2084636573 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff496d9d700 1 --2- 172.21.15.5:0/2084636573 >> v2:172.21.15.49:3300/0 conn(0x7ff4980cdc90 0x7ff4980b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff495d9b700 1 -- 172.21.15.5:0/2084636573 >> v2:172.21.15.49:3300/0 conn(0x7ff4980cdc90 msgr2=0x7ff4980b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff495d9b700 1 --2- 172.21.15.5:0/2084636573 >> v2:172.21.15.49:3300/0 conn(0x7ff4980cdc90 0x7ff4980b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff495d9b700 1 -- 172.21.15.5:0/2084636573 >> v1:172.21.15.5:6789/0 conn(0x7ff4980d13a0 legacy=0x7ff4980d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff495d9b700 1 -- 172.21.15.5:0/2084636573 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980ce8a0 con 0x7ff49806bd60 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff49559a700 1 -- 172.21.15.5:0/2084636573 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902360307 0 0) 0x7ff4980b1200 con 0x7ff4980d13a0 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff495d9b700 1 --2- 172.21.15.5:0/2084636573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49806bd60 0x7ff4980cf780 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff4980b2eb0 tx=0x7ff48c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edfc417ee075842 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff49559a700 1 -- 172.21.15.5:0/2084636573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c011830 con 0x7ff49806bd60 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff49559a700 1 -- 172.21.15.5:0/2084636573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c011990 con 0x7ff49806bd60 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.513+0000 7ff49559a700 1 -- 172.21.15.5:0/2084636573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c011d20 con 0x7ff49806bd60 2024-05-08T23:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.514+0000 7ff49cb22700 1 -- 172.21.15.5:0/2084636573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49806bd60 msgr2=0x7ff4980cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.514+0000 7ff49cb22700 1 --2- 172.21.15.5:0/2084636573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49806bd60 0x7ff4980cf780 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff4980b2eb0 tx=0x7ff48c009b20 comp rx=0 tx=0).stop 2024-05-08T23:23:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.514+0000 7ff49cb22700 1 -- 172.21.15.5:0/2084636573 shutdown_connections 2024-05-08T23:23:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.514+0000 7ff49cb22700 1 --2- 172.21.15.5:0/2084636573 >> v2:172.21.15.49:3300/0 conn(0x7ff4980cdc90 0x7ff4980b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.514+0000 7ff49cb22700 1 --2- 172.21.15.5:0/2084636573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49806bd60 0x7ff4980cf780 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.514+0000 7ff49cb22700 1 -- 172.21.15.5:0/2084636573 >> 172.21.15.5:0/2084636573 conn(0x7ff49808afe0 msgr2=0x7ff49808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.514+0000 7ff49cb22700 1 -- 172.21.15.5:0/2084636573 shutdown_connections 2024-05-08T23:23:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.514+0000 7ff49cb22700 1 -- 172.21.15.5:0/2084636573 wait complete. 2024-05-08T23:23:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.516+0000 7ff49cb22700 1 Processor -- start 2024-05-08T23:23:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.516+0000 7ff49cb22700 1 -- start start 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.516+0000 7ff49cb22700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49806bd60 0x7ff4980cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.516+0000 7ff49cb22700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4980cdc90 0x7ff4980ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.516+0000 7ff49cb22700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4980d13a0 0x7ff49815be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.516+0000 7ff49cb22700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980ca630 con 0x7ff4980d13a0 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.516+0000 7ff49cb22700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980ca890 con 0x7ff49806bd60 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.516+0000 7ff49cb22700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981f33c0 con 0x7ff4980cdc90 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4980d13a0 0x7ff49815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4980d13a0 0x7ff49815be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55604/0 (socket says 172.21.15.5:55604) 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 -- 172.21.15.5:0/3437620969 learned_addr learned my addr 172.21.15.5:0/3437620969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff49659c700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49806bd60 0x7ff4980cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff495d9b700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4980cdc90 0x7ff4980ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 -- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4980cdc90 msgr2=0x7ff4980ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4980cdc90 0x7ff4980ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 -- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49806bd60 msgr2=0x7ff4980cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49806bd60 0x7ff4980cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 -- 172.21.15.5:0/3437620969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c0094a0 con 0x7ff4980d13a0 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff496d9d700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4980d13a0 0x7ff49815be80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff49000efc0 tx=0x7ff49000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff486ffd700 1 -- 172.21.15.5:0/3437620969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490009ea0 con 0x7ff4980d13a0 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff486ffd700 1 -- 172.21.15.5:0/3437620969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49000f040 con 0x7ff4980d13a0 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4981f36a0 con 0x7ff4980d13a0 2024-05-08T23:23:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff486ffd700 1 -- 172.21.15.5:0/3437620969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49000ced0 con 0x7ff4980d13a0 2024-05-08T23:23:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.517+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4981f3c60 con 0x7ff4980d13a0 2024-05-08T23:23:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.518+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4981f3e50 con 0x7ff4980d13a0 2024-05-08T23:23:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.519+0000 7ff486ffd700 1 -- 172.21.15.5:0/3437620969 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff490014480 con 0x7ff4980d13a0 2024-05-08T23:23:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.519+0000 7ff486ffd700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff47806ac90 0x7ff47806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.520+0000 7ff486ffd700 1 -- 172.21.15.5:0/3437620969 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff490089890 con 0x7ff4980d13a0 2024-05-08T23:23:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.520+0000 7ff49659c700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff47806ac90 0x7ff47806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.520+0000 7ff49659c700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff47806ac90 0x7ff47806d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff488007fd0 tx=0x7ff488007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.524+0000 7ff486ffd700 1 -- 172.21.15.5:0/3437620969 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff490054cc0 con 0x7ff4980d13a0 2024-05-08T23:23:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.739+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff498074bc0 con 0x7ff4980d13a0 2024-05-08T23:23:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.739+0000 7ff486ffd700 1 -- 172.21.15.5:0/3437620969 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4900180a0 con 0x7ff4980d13a0 2024-05-08T23:23:25.740 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.741+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff47806ac90 msgr2=0x7ff47806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.741+0000 7ff49cb22700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff47806ac90 0x7ff47806d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff488007fd0 tx=0x7ff488007de0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.741+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4980d13a0 msgr2=0x7ff49815be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.741+0000 7ff49cb22700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4980d13a0 0x7ff49815be80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff49000efc0 tx=0x7ff49000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.742+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 shutdown_connections 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.742+0000 7ff49cb22700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff47806ac90 0x7ff47806d150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.742+0000 7ff49cb22700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4980d13a0 0x7ff49815be80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.742+0000 7ff49cb22700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49806bd60 0x7ff4980cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.742+0000 7ff49cb22700 1 --2- 172.21.15.5:0/3437620969 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff4980cdc90 0x7ff4980ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.742+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 >> 172.21.15.5:0/3437620969 conn(0x7ff49808afe0 msgr2=0x7ff49806fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.742+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 shutdown_connections 2024-05-08T23:23:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.742+0000 7ff49cb22700 1 -- 172.21.15.5:0/3437620969 wait complete. 2024-05-08T23:23:25.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-05-08T23:23:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.891+0000 7f3933088700 1 Processor -- start 2024-05-08T23:23:25.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.891+0000 7f3933088700 1 -- start start 2024-05-08T23:23:25.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.892+0000 7f3933088700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.892+0000 7f3933088700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f392c0cf000 0x7f392c0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.892+0000 7f3933088700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392c0cfe50 con 0x7f392c0cf9c0 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.892+0000 7f3933088700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c0cd400 con 0x7f392c0cf000 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.892+0000 7f3933088700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c09ad40 con 0x7f392c15ef60 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f3930e24700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f3930e24700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:37536/0 (socket says 172.21.15.5:37536) 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f392bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f392c0cf000 0x7f392c0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f3930e24700 1 -- 172.21.15.5:0/4025476316 learned_addr learned my addr 172.21.15.5:0/4025476316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f392bfff700 1 -- 172.21.15.5:0/4025476316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 msgr2=0x7f392c0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f392bfff700 1 --2- 172.21.15.5:0/4025476316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f392bfff700 1 -- 172.21.15.5:0/4025476316 >> v1:172.21.15.49:6789/0 conn(0x7f392c0cf9c0 legacy=0x7f392c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f392bfff700 1 -- 172.21.15.5:0/4025476316 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0ccb60 con 0x7f392c0cf000 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f3930e24700 1 --2- 172.21.15.5:0/4025476316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.893+0000 7f392bfff700 1 --2- 172.21.15.5:0/4025476316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 0x7f392c0cf480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3914009840 tx=0x7f3914009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee3df46b91ff7016 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f392b7fe700 1 -- 172.21.15.5:0/4025476316 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914011830 con 0x7f392c0cf000 2024-05-08T23:23:25.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f392b7fe700 1 -- 172.21.15.5:0/4025476316 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3914011990 con 0x7f392c0cf000 2024-05-08T23:23:25.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f392b7fe700 1 -- 172.21.15.5:0/4025476316 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914011d20 con 0x7f392c0cf000 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f3933088700 1 -- 172.21.15.5:0/4025476316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 msgr2=0x7f392c0cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f3933088700 1 --2- 172.21.15.5:0/4025476316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 0x7f392c0cf480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3914009840 tx=0x7f3914009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f3933088700 1 -- 172.21.15.5:0/4025476316 shutdown_connections 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f3933088700 1 --2- 172.21.15.5:0/4025476316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f3933088700 1 --2- 172.21.15.5:0/4025476316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 0x7f392c0cf480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f3933088700 1 -- 172.21.15.5:0/4025476316 >> 172.21.15.5:0/4025476316 conn(0x7f392c0b2f20 msgr2=0x7f392c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f3933088700 1 -- 172.21.15.5:0/4025476316 shutdown_connections 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.894+0000 7f3933088700 1 -- 172.21.15.5:0/4025476316 wait complete. 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.896+0000 7f3933088700 1 Processor -- start 2024-05-08T23:23:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.896+0000 7f3933088700 1 -- start start 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f3933088700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 0x7f392c0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f3933088700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c0cf9c0 0x7f392c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f3933088700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f3933088700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c09e0d0 con 0x7f392c0cf9c0 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f3933088700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c0c5770 con 0x7f392c0cf000 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f3933088700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c0c5a50 con 0x7f392c15ef60 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c0cf9c0 0x7f392c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c0cf9c0 0x7f392c15a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55614/0 (socket says 172.21.15.5:55614) 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 -- 172.21.15.5:0/3114749230 learned_addr learned my addr 172.21.15.5:0/3114749230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f3930e24700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 0x7f392c0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f3931625700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 -- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 msgr2=0x7f392c0c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 -- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 msgr2=0x7f392c0c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 0x7f392c0c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 -- 172.21.15.5:0/3114749230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c009550 con 0x7f392c0cf9c0 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.897+0000 7f392bfff700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c0cf9c0 0x7f392c15a9d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3914009fd0 tx=0x7f3914004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.898+0000 7f39297fa700 1 -- 172.21.15.5:0/3114749230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140036a0 con 0x7f392c0cf9c0 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.898+0000 7f39297fa700 1 -- 172.21.15.5:0/3114749230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391401e070 con 0x7f392c0cf9c0 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.898+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39140094a0 con 0x7f392c0cf9c0 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.898+0000 7f39297fa700 1 -- 172.21.15.5:0/3114749230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914003a40 con 0x7f392c0cf9c0 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.898+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c15af10 con 0x7f392c0cf9c0 2024-05-08T23:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.899+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c09dba0 con 0x7f392c0cf9c0 2024-05-08T23:23:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.900+0000 7f39297fa700 1 -- 172.21.15.5:0/3114749230 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3914003ba0 con 0x7f392c0cf9c0 2024-05-08T23:23:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.900+0000 7f39297fa700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f390c06ac90 0x7f390c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.900+0000 7f39297fa700 1 -- 172.21.15.5:0/3114749230 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f391408a1f0 con 0x7f392c0cf9c0 2024-05-08T23:23:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.901+0000 7f3930e24700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f390c06ac90 0x7f390c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.901+0000 7f3930e24700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f390c06ac90 0x7f390c06d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f392c0c5e50 tx=0x7f391c0092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:25.906+0000 7f39297fa700 1 -- 172.21.15.5:0/3114749230 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3914055790 con 0x7f392c0cf9c0 2024-05-08T23:23:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.213+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) v1 -- 0x7f392c068000 con 0x7f392c0cf9c0 2024-05-08T23:23:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.972+0000 7f39297fa700 1 -- 172.21.15.5:0/3114749230 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v138) v1 ==== 146+0+0 (secure 0 0 0) 0x7f3914058db0 con 0x7f392c0cf9c0 2024-05-08T23:23:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f390c06ac90 msgr2=0x7f390c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f390c06ac90 0x7f390c06d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f392c0c5e50 tx=0x7f391c0092c0 comp rx=0 tx=0).stop 2024-05-08T23:23:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c0cf9c0 msgr2=0x7f392c15a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c0cf9c0 0x7f392c15a9d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3914009fd0 tx=0x7f3914004750 comp rx=0 tx=0).stop 2024-05-08T23:23:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 shutdown_connections 2024-05-08T23:23:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f390c06ac90 0x7f390c06d150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f392c15ef60 0x7f392c0c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f392c0cf000 0x7f392c0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 --2- 172.21.15.5:0/3114749230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392c0cf9c0 0x7f392c15a9d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 >> 172.21.15.5:0/3114749230 conn(0x7f392c0b2f20 msgr2=0x7f392c15df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 shutdown_connections 2024-05-08T23:23:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:26.975+0000 7f3933088700 1 -- 172.21.15.5:0/3114749230 wait complete. 2024-05-08T23:23:26.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T23:23:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.135+0000 7f146c1e8700 1 Processor -- start 2024-05-08T23:23:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.135+0000 7f146c1e8700 1 -- start start 2024-05-08T23:23:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f146c1e8700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f146415bb20 0x7f146406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f146c1e8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f1464159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f146c1e8700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14640cc980 con 0x7f14640cfd10 2024-05-08T23:23:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f146c1e8700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146406f0b0 con 0x7f146415bb20 2024-05-08T23:23:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f146c1e8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14640d00f0 con 0x7f146406ec20 2024-05-08T23:23:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f146a785700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f1464159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f1469783700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f146415bb20 0x7f146406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f1469783700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f146415bb20 0x7f146406e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:53268/0 (socket says 172.21.15.5:53268) 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f146a785700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f1464159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:37556/0 (socket says 172.21.15.5:37556) 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f1469783700 1 -- 172.21.15.5:0/2453774963 learned_addr learned my addr 172.21.15.5:0/2453774963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.136+0000 7f1469f84700 1 --1- 172.21.15.5:0/2453774963 >> v1:172.21.15.49:6789/0 conn(0x7f14640cfd10 0x7f146406dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:58038/0 (socket says 172.21.15.5:58038) 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f1469783700 1 -- 172.21.15.5:0/2453774963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 msgr2=0x7f1464159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f1469783700 1 --2- 172.21.15.5:0/2453774963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f1464159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f1469783700 1 -- 172.21.15.5:0/2453774963 >> v1:172.21.15.49:6789/0 conn(0x7f14640cfd10 legacy=0x7f146406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f1469783700 1 -- 172.21.15.5:0/2453774963 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640ccbd0 con 0x7f146415bb20 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f146a785700 1 --2- 172.21.15.5:0/2453774963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f1464159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f1469783700 1 --2- 172.21.15.5:0/2453774963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146415bb20 0x7f146406e6e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f146000bb90 tx=0x7f146000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d87b9b35d42aee28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f1468f82700 1 -- 172.21.15.5:0/2453774963 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146000dd30 con 0x7f146415bb20 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f1468f82700 1 -- 172.21.15.5:0/2453774963 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146000de90 con 0x7f146415bb20 2024-05-08T23:23:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.137+0000 7f1468f82700 1 -- 172.21.15.5:0/2453774963 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460010690 con 0x7f146415bb20 2024-05-08T23:23:27.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.138+0000 7f146c1e8700 1 -- 172.21.15.5:0/2453774963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146415bb20 msgr2=0x7f146406e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.138+0000 7f146c1e8700 1 --2- 172.21.15.5:0/2453774963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146415bb20 0x7f146406e6e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f146000bb90 tx=0x7f146000bea0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.138+0000 7f146c1e8700 1 -- 172.21.15.5:0/2453774963 shutdown_connections 2024-05-08T23:23:27.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.138+0000 7f146c1e8700 1 --2- 172.21.15.5:0/2453774963 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f1464159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.138+0000 7f146c1e8700 1 --2- 172.21.15.5:0/2453774963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146415bb20 0x7f146406e6e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.138+0000 7f146c1e8700 1 -- 172.21.15.5:0/2453774963 >> 172.21.15.5:0/2453774963 conn(0x7f1464099a50 msgr2=0x7f14640d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.138+0000 7f146c1e8700 1 -- 172.21.15.5:0/2453774963 shutdown_connections 2024-05-08T23:23:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.138+0000 7f146c1e8700 1 -- 172.21.15.5:0/2453774963 wait complete. 2024-05-08T23:23:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.139+0000 7f146c1e8700 1 Processor -- start 2024-05-08T23:23:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.139+0000 7f146c1e8700 1 -- start start 2024-05-08T23:23:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146c1e8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f14640c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146c1e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14640cfd10 0x7f14640b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146c1e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146415bb20 0x7f14640b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146c1e8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14640c5100 con 0x7f146415bb20 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146c1e8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14640b5bb0 con 0x7f14640cfd10 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146c1e8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14640b3ad0 con 0x7f146406ec20 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146a785700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146415bb20 0x7f14640b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146a785700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146415bb20 0x7f14640b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55622/0 (socket says 172.21.15.5:55622) 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146a785700 1 -- 172.21.15.5:0/1095426 learned_addr learned my addr 172.21.15.5:0/1095426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f1469783700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14640cfd10 0x7f14640b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f1469f84700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f14640c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146a785700 1 -- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 msgr2=0x7f14640c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146a785700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f14640c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146a785700 1 -- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14640cfd10 msgr2=0x7f14640b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146a785700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14640cfd10 0x7f14640b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.140+0000 7f146a785700 1 -- 172.21.15.5:0/1095426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c009a70 con 0x7f146415bb20 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.141+0000 7f146a785700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146415bb20 0x7f14640b3430 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f145c00efc0 tx=0x7f145c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.141+0000 7f145a7fc700 1 -- 172.21.15.5:0/1095426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00cb40 con 0x7f146415bb20 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.141+0000 7f145a7fc700 1 -- 172.21.15.5:0/1095426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c00f040 con 0x7f146415bb20 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.141+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146000b7f0 con 0x7f146415bb20 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.141+0000 7f145a7fc700 1 -- 172.21.15.5:0/1095426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c003a20 con 0x7f146415bb20 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.141+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14640b4090 con 0x7f146415bb20 2024-05-08T23:23:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.142+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1464074bc0 con 0x7f146415bb20 2024-05-08T23:23:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.142+0000 7f145a7fc700 1 -- 172.21.15.5:0/1095426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f145c014480 con 0x7f146415bb20 2024-05-08T23:23:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.143+0000 7f145a7fc700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f144806ac90 0x7f144806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.143+0000 7f145a7fc700 1 -- 172.21.15.5:0/1095426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f145c0899b0 con 0x7f146415bb20 2024-05-08T23:23:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.145+0000 7f1469f84700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f144806ac90 0x7f144806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.145+0000 7f1469f84700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f144806ac90 0x7f144806d150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1454000f60 tx=0x7f1454007d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:27.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.148+0000 7f145a7fc700 1 -- 172.21.15.5:0/1095426 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f145c054e70 con 0x7f146415bb20 2024-05-08T23:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.368+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f14640af970 con 0x7f146415bb20 2024-05-08T23:23:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.371+0000 7f145a7fc700 1 -- 172.21.15.5:0/1095426 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f145c0180c0 con 0x7f146415bb20 2024-05-08T23:23:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f144806ac90 msgr2=0x7f144806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f144806ac90 0x7f144806d150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1454000f60 tx=0x7f1454007d10 comp rx=0 tx=0).stop 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146415bb20 msgr2=0x7f14640b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146415bb20 0x7f14640b3430 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f145c00efc0 tx=0x7f145c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 shutdown_connections 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f144806ac90 0x7f144806d150 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146415bb20 0x7f14640b3430 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f146406ec20 0x7f14640c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 --2- 172.21.15.5:0/1095426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14640cfd10 0x7f14640b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 >> 172.21.15.5:0/1095426 conn(0x7f1464099a50 msgr2=0x7f1464158420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 shutdown_connections 2024-05-08T23:23:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.374+0000 7f146c1e8700 1 -- 172.21.15.5:0/1095426 wait complete. 2024-05-08T23:23:27.387 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:23:27.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:23:27.430 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:23:27.483 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:23:27.483 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:23:27.483 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:27.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:23:27.506 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:23:27.506 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:27.528 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:23:27.529 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:23:27.529 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:27.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:23:27.569 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:23:27.569 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:27.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:23:27.590 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T23:23:27.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.754+0000 7f55725ed700 1 Processor -- start 2024-05-08T23:23:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.754+0000 7f55725ed700 1 -- start start 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f55725ed700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f556c06bd80 0x7f556c0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f55725ed700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c06c3d0 0x7f556c06c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f55725ed700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c0cf990 con 0x7f556c15da10 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f55725ed700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c09ab40 con 0x7f556c06bd80 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f55725ed700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c06cda0 con 0x7f556c06c3d0 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f556b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f556c06bd80 0x7f556c0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f556bfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f556c15da10 0x7f556c09a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33668/0 (socket says 172.21.15.5:33668) 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f556bfff700 1 -- 172.21.15.5:0/3272791369 learned_addr learned my addr 172.21.15.5:0/3272791369 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f5570b8a700 1 --2- 172.21.15.5:0/3272791369 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c06c3d0 0x7f556c06c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f556b7fe700 1 -- 172.21.15.5:0/3272791369 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c06c3d0 msgr2=0x7f556c06c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f556b7fe700 1 --2- 172.21.15.5:0/3272791369 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c06c3d0 0x7f556c06c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f556b7fe700 1 -- 172.21.15.5:0/3272791369 >> v1:172.21.15.5:6789/0 conn(0x7f556c15da10 legacy=0x7f556c09a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f556b7fe700 1 -- 172.21.15.5:0/3272791369 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c0d04b0 con 0x7f556c06bd80 2024-05-08T23:23:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f5570b8a700 1 --2- 172.21.15.5:0/3272791369 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c06c3d0 0x7f556c06c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.755+0000 7f556affd700 1 -- 172.21.15.5:0/3272791369 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901572876 0 0) 0x7f556c0cf990 con 0x7f556c15da10 2024-05-08T23:23:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f556b7fe700 1 --2- 172.21.15.5:0/3272791369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0d0940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5554009870 tx=0x7f5554009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de5952f3fb5e58e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f556affd700 1 -- 172.21.15.5:0/3272791369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554011830 con 0x7f556c06bd80 2024-05-08T23:23:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f556affd700 1 -- 172.21.15.5:0/3272791369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5554011990 con 0x7f556c06bd80 2024-05-08T23:23:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f556affd700 1 -- 172.21.15.5:0/3272791369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554011d20 con 0x7f556c06bd80 2024-05-08T23:23:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f55725ed700 1 -- 172.21.15.5:0/3272791369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 msgr2=0x7f556c0d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f55725ed700 1 --2- 172.21.15.5:0/3272791369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0d0940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5554009870 tx=0x7f5554009b80 comp rx=0 tx=0).stop 2024-05-08T23:23:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f55725ed700 1 -- 172.21.15.5:0/3272791369 shutdown_connections 2024-05-08T23:23:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f55725ed700 1 --2- 172.21.15.5:0/3272791369 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c06c3d0 0x7f556c06c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f55725ed700 1 --2- 172.21.15.5:0/3272791369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0d0940 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f55725ed700 1 -- 172.21.15.5:0/3272791369 >> 172.21.15.5:0/3272791369 conn(0x7f556c0b3010 msgr2=0x7f556c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f55725ed700 1 -- 172.21.15.5:0/3272791369 shutdown_connections 2024-05-08T23:23:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.756+0000 7f55725ed700 1 -- 172.21.15.5:0/3272791369 wait complete. 2024-05-08T23:23:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.758+0000 7f55725ed700 1 Processor -- start 2024-05-08T23:23:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.758+0000 7f55725ed700 1 -- start start 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f55725ed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f55725ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f556c06c3d0 0x7f556c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f55725ed700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c15da10 0x7f556c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f55725ed700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1f4340 con 0x7f556c06bd80 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f55725ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1f4670 con 0x7f556c06c3d0 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f55725ed700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1f4950 con 0x7f556c15da10 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51570/0 (socket says 172.21.15.5:51570) 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 -- 172.21.15.5:0/1681705226 learned_addr learned my addr 172.21.15.5:0/1681705226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f5570b8a700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c15da10 0x7f556c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556b7fe700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f556c06c3d0 0x7f556c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 -- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c15da10 msgr2=0x7f556c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c15da10 0x7f556c159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 -- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f556c06c3d0 msgr2=0x7f556c1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f556c06c3d0 0x7f556c1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 -- 172.21.15.5:0/1681705226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55540094a0 con 0x7f556c06bd80 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556b7fe700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f556c06c3d0 0x7f556c1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.759+0000 7f556bfff700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0c6600 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f555c0011e0 tx=0x7f555c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.760+0000 7f5568ff9700 1 -- 172.21.15.5:0/1681705226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c004c80 con 0x7f556c06bd80 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.760+0000 7f5568ff9700 1 -- 172.21.15.5:0/1681705226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c004e00 con 0x7f556c06bd80 2024-05-08T23:23:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.760+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c1f4c30 con 0x7f556c06bd80 2024-05-08T23:23:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.760+0000 7f5568ff9700 1 -- 172.21.15.5:0/1681705226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c007310 con 0x7f556c06bd80 2024-05-08T23:23:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.760+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c0c6b40 con 0x7f556c06bd80 2024-05-08T23:23:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.761+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c1f5150 con 0x7f556c06bd80 2024-05-08T23:23:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.761+0000 7f5568ff9700 1 -- 172.21.15.5:0/1681705226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f555c007470 con 0x7f556c06bd80 2024-05-08T23:23:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.762+0000 7f5568ff9700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f554c06ac90 0x7f554c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.762+0000 7f5568ff9700 1 -- 172.21.15.5:0/1681705226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f555c088e60 con 0x7f556c06bd80 2024-05-08T23:23:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.762+0000 7f556b7fe700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f554c06ac90 0x7f554c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.762+0000 7f556b7fe700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f554c06ac90 0x7f554c06d150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f556c06c1c0 tx=0x7f5554008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:27.766+0000 7f5568ff9700 1 -- 172.21.15.5:0/1681705226 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f556c1f5150 con 0x7f556c06bd80 2024-05-08T23:23:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.034+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f556c068000 con 0x7f556c06bd80 2024-05-08T23:23:28.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.034+0000 7f5568ff9700 1 -- 172.21.15.5:0/1681705226 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f555c054290 con 0x7f556c06bd80 2024-05-08T23:23:28.033 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:28.033 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3272550127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3272550127},{"type":"v1","addr":"172.21.15.49:6836","nonce":3272550127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/3818707907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3818707907},{"type":"v1","addr":"172.21.15.5:6834","nonce":3818707907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1346502189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1346502189},{"type":"v1","addr":"172.21.15.49:6837","nonce":1346502189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/715514954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":715514954},{"type":"v1","addr":"172.21.15.5:6835","nonce":715514954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f554c06ac90 msgr2=0x7f554c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f554c06ac90 0x7f554c06d150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f556c06c1c0 tx=0x7f5554008030 comp rx=0 tx=0).stop 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 msgr2=0x7f556c0c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0c6600 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f555c0011e0 tx=0x7f555c00cae0 comp rx=0 tx=0).stop 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 shutdown_connections 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f554c06ac90 0x7f554c06d150 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f556c15da10 0x7f556c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f556c06bd80 0x7f556c0c6600 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 --2- 172.21.15.5:0/1681705226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f556c06c3d0 0x7f556c1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 >> 172.21.15.5:0/1681705226 conn(0x7f556c0b3010 msgr2=0x7f556c0cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 shutdown_connections 2024-05-08T23:23:28.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:28.036+0000 7f55725ed700 1 -- 172.21.15.5:0/1681705226 wait complete. 2024-05-08T23:23:28.037 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-05-08T23:23:28.049 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:23:28.050 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T23:23:28.050 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:23:28.050 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:23:28.050 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:23:28.050 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T23:23:28.050 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:23:28.050 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:23:28.051 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:23:28.051 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T23:23:28.051 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:23:28.051 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:23:28.052 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:23:28.052 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T23:23:28.052 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:23:28.052 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:23:28.060 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:23:28.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:23:28.061 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:23:28.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:23:28.065 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:23:28.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:23:28.104 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:23:28.104 DEBUG:teuthology.parallel:result is None 2024-05-08T23:23:28.105 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:23:28.105 DEBUG:teuthology.parallel:result is None 2024-05-08T23:23:28.107 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:23:28.107 DEBUG:teuthology.parallel:result is None 2024-05-08T23:23:28.193 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:23:28.193 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:23:28.202 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:23:29.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-05-08T23:23:34.064 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:23:34.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:23:34.068 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:23:34.068 DEBUG:teuthology.parallel:result is None 2024-05-08T23:23:34.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T23:23:34.160 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.218+0000 7fe40ce26700 1 Processor -- start 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.218+0000 7fe40ce26700 1 -- start start 2024-05-08T23:23:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.218+0000 7fe40ce26700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe40806b140 0x7fe4080ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40ce26700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 0x7fe40815bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40ce26700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe40815c1c0 0x7fe4080cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40ce26700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4080cdbb0 con 0x7fe40806b140 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40ce26700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4080cd310 con 0x7fe40815c1c0 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40ce26700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080cd5f0 con 0x7fe40806a960 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe40806b140 0x7fe4080ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe40806b140 0x7fe4080ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51606/0 (socket says 172.21.15.5:51606) 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 -- 172.21.15.5:0/298242676 learned_addr learned my addr 172.21.15.5:0/298242676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe3fffff700 1 --2- 172.21.15.5:0/298242676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 0x7fe40815bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe406d9d700 1 --2- 172.21.15.5:0/298242676 >> v2:172.21.15.49:3300/0 conn(0x7fe40815c1c0 0x7fe4080cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 -- 172.21.15.5:0/298242676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 msgr2=0x7fe40815bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 --2- 172.21.15.5:0/298242676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 0x7fe40815bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 -- 172.21.15.5:0/298242676 >> v2:172.21.15.49:3300/0 conn(0x7fe40815c1c0 msgr2=0x7fe4080cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 --2- 172.21.15.5:0/298242676 >> v2:172.21.15.49:3300/0 conn(0x7fe40815c1c0 0x7fe4080cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 -- 172.21.15.5:0/298242676 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4080cd8d0 con 0x7fe40806b140 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe40659c700 1 --2- 172.21.15.5:0/298242676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40806b140 0x7fe4080ccd30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe40806d750 tx=0x7fe3f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194e57a0e77d1326 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.219+0000 7fe405d9b700 1 -- 172.21.15.5:0/298242676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8011830 con 0x7fe40806b140 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe405d9b700 1 -- 172.21.15.5:0/298242676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8011990 con 0x7fe40806b140 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe405d9b700 1 -- 172.21.15.5:0/298242676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8011d20 con 0x7fe40806b140 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 -- 172.21.15.5:0/298242676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40806b140 msgr2=0x7fe4080ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 --2- 172.21.15.5:0/298242676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40806b140 0x7fe4080ccd30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe40806d750 tx=0x7fe3f8009bd0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 -- 172.21.15.5:0/298242676 shutdown_connections 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 --2- 172.21.15.5:0/298242676 >> v2:172.21.15.49:3300/0 conn(0x7fe40815c1c0 0x7fe4080cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 --2- 172.21.15.5:0/298242676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40806b140 0x7fe4080ccd30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 --2- 172.21.15.5:0/298242676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 0x7fe40815bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 -- 172.21.15.5:0/298242676 >> 172.21.15.5:0/298242676 conn(0x7fe4080d0320 msgr2=0x7fe4080d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 -- 172.21.15.5:0/298242676 shutdown_connections 2024-05-08T23:23:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.220+0000 7fe40ce26700 1 -- 172.21.15.5:0/298242676 wait complete. 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.222+0000 7fe40ce26700 1 Processor -- start 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.222+0000 7fe40ce26700 1 -- start start 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe40ce26700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 0x7fe408149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe40ce26700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe40806b140 0x7fe4080b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe40ce26700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40815c1c0 0x7fe40815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe40ce26700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080b3850 con 0x7fe40815c1c0 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe40ce26700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080b5230 con 0x7fe40806b140 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe40ce26700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080b5510 con 0x7fe40806a960 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40815c1c0 0x7fe40815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe40806b140 0x7fe4080b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40815c1c0 0x7fe40815a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51620/0 (socket says 172.21.15.5:51620) 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe40659c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 0x7fe408149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 -- 172.21.15.5:0/1102222395 learned_addr learned my addr 172.21.15.5:0/1102222395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 -- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 msgr2=0x7fe408149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 0x7fe408149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 -- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe40806b140 msgr2=0x7fe4080b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe40806b140 0x7fe4080b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 -- 172.21.15.5:0/1102222395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8009550 con 0x7fe40815c1c0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe406d9d700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40815c1c0 0x7fe40815a4d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe40000efc0 tx=0x7fe40000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe3feffd700 1 -- 172.21.15.5:0/1102222395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400009ea0 con 0x7fe40815c1c0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe3feffd700 1 -- 172.21.15.5:0/1102222395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40000f040 con 0x7fe40815c1c0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4080c3cc0 con 0x7fe40815c1c0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.223+0000 7fe3feffd700 1 -- 172.21.15.5:0/1102222395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40000ced0 con 0x7fe40815c1c0 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.224+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4080b5d10 con 0x7fe40815c1c0 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.224+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4080b5a30 con 0x7fe40815c1c0 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.225+0000 7fe3feffd700 1 -- 172.21.15.5:0/1102222395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe400014480 con 0x7fe40815c1c0 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.226+0000 7fe3feffd700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3e806ac90 0x7fe3e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.226+0000 7fe3feffd700 1 -- 172.21.15.5:0/1102222395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe4000899e0 con 0x7fe40815c1c0 2024-05-08T23:23:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.226+0000 7fe40659c700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3e806ac90 0x7fe3e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.226+0000 7fe40659c700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3e806ac90 0x7fe3e806d150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8014040 tx=0x7fe3f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.230+0000 7fe3feffd700 1 -- 172.21.15.5:0/1102222395 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe4080b5a30 con 0x7fe40815c1c0 2024-05-08T23:23:34.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:34.464+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe4080b5a30 con 0x7fe40815c1c0 2024-05-08T23:23:35.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.184+0000 7fe3feffd700 1 -- 172.21.15.5:0/1102222395 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe4080b5a30 con 0x7fe40815c1c0 2024-05-08T23:23:35.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3e806ac90 msgr2=0x7fe3e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3e806ac90 0x7fe3e806d150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8014040 tx=0x7fe3f8008030 comp rx=0 tx=0).stop 2024-05-08T23:23:35.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40815c1c0 msgr2=0x7fe40815a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40815c1c0 0x7fe40815a4d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe40000efc0 tx=0x7fe40000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 shutdown_connections 2024-05-08T23:23:35.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3e806ac90 0x7fe3e806d150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40815c1c0 0x7fe40815a4d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe40806a960 0x7fe408149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 --2- 172.21.15.5:0/1102222395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe40806b140 0x7fe4080b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 >> 172.21.15.5:0/1102222395 conn(0x7fe4080d0320 msgr2=0x7fe4080d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 shutdown_connections 2024-05-08T23:23:35.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.187+0000 7fe40ce26700 1 -- 172.21.15.5:0/1102222395 wait complete. 2024-05-08T23:23:35.189 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-08T23:23:35.203 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T23:23:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.345+0000 7fe566834700 1 Processor -- start 2024-05-08T23:23:35.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.345+0000 7fe566834700 1 -- start start 2024-05-08T23:23:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.346+0000 7fe566834700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5600ccbd0 0x7fe560168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.346+0000 7fe566834700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56009a870 con 0x7fe5600b2480 2024-05-08T23:23:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.346+0000 7fe566834700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5600d0840 con 0x7fe560161030 2024-05-08T23:23:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.346+0000 7fe566834700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56006ccc0 con 0x7fe5600ccbd0 2024-05-08T23:23:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.346+0000 7fe55f7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe5600b2480 0x7fe56009a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33692/0 (socket says 172.21.15.5:33692) 2024-05-08T23:23:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.346+0000 7fe55f7fe700 1 -- 172.21.15.5:0/584111381 learned_addr learned my addr 172.21.15.5:0/584111381 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.346+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570502809 0 0) 0x7fe56009a870 con 0x7fe5600b2480 2024-05-08T23:23:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5480035b0 con 0x7fe5600b2480 2024-05-08T23:23:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe564dd1700 1 --2- 172.21.15.5:0/584111381 >> v2:172.21.15.49:3300/0 conn(0x7fe5600ccbd0 0x7fe560168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2924948749 0 0) 0x7fe5480035b0 con 0x7fe5600b2480 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 >> v2:172.21.15.49:3300/0 conn(0x7fe5600ccbd0 msgr2=0x7fe560168580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 --2- 172.21.15.5:0/584111381 >> v2:172.21.15.49:3300/0 conn(0x7fe5600ccbd0 0x7fe560168580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 >> v1:172.21.15.49:6789/0 conn(0x7fe560161030 legacy=0x7fe560099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56006c600 con 0x7fe5600b2480 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fe554001670 con 0x7fe5600b2480 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 >> v1:172.21.15.5:6789/0 conn(0x7fe5600b2480 legacy=0x7fe56009a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe548003a50 0x7fe548003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe548004850 0x7fe548004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548005210 0x7fe54800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56006ccc0 con 0x7fe548004850 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54800c110 con 0x7fe548003a50 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54800c2d0 con 0x7fe548005210 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55f7fe700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe548004850 0x7fe548004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55ffff700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe548003a50 0x7fe548003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe564dd1700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548005210 0x7fe54800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55f7fe700 1 -- 172.21.15.5:0/584111381 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548005210 msgr2=0x7fe54800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55f7fe700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548005210 0x7fe54800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55f7fe700 1 -- 172.21.15.5:0/584111381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe548003a50 msgr2=0x7fe548003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55f7fe700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe548003a50 0x7fe548003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe55f7fe700 1 -- 172.21.15.5:0/584111381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480035b0 con 0x7fe548004850 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.347+0000 7fe564dd1700 1 -- 172.21.15.5:0/584111381 reap_dead start 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe55f7fe700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe548004850 0x7fe548004cd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe55400c160 tx=0x7fe548005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b811e3865ce910 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554007b70 con 0x7fe548004850 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55400cc80 con 0x7fe548004850 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe55effd700 1 -- 172.21.15.5:0/584111381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554005960 con 0x7fe548004850 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe566834700 1 -- 172.21.15.5:0/584111381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe548004850 msgr2=0x7fe548004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe566834700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe548004850 0x7fe548004cd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe55400c160 tx=0x7fe548005880 comp rx=0 tx=0).stop 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe566834700 1 -- 172.21.15.5:0/584111381 shutdown_connections 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe566834700 1 --2- 172.21.15.5:0/584111381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe548004850 0x7fe548004cd0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe566834700 1 -- 172.21.15.5:0/584111381 >> 172.21.15.5:0/584111381 conn(0x7fe5600b2f20 msgr2=0x7fe5600b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe566834700 1 -- 172.21.15.5:0/584111381 shutdown_connections 2024-05-08T23:23:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.348+0000 7fe566834700 1 -- 172.21.15.5:0/584111381 wait complete. 2024-05-08T23:23:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.350+0000 7fe566834700 1 Processor -- start 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.350+0000 7fe566834700 1 -- start start 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.351+0000 7fe566834700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548004850 0x7fe560152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.351+0000 7fe566834700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe56015f5d0 0x7fe56015fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.351+0000 7fe566834700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5600b55c0 0x7fe5600b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.351+0000 7fe566834700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600d0660 con 0x7fe56015f5d0 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.351+0000 7fe566834700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56015ff90 con 0x7fe5600b55c0 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.351+0000 7fe566834700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601f6e90 con 0x7fe548004850 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe56015f5d0 0x7fe56015fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe56015f5d0 0x7fe56015fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51634/0 (socket says 172.21.15.5:51634) 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 -- 172.21.15.5:0/2432112942 learned_addr learned my addr 172.21.15.5:0/2432112942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55ffff700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548004850 0x7fe560152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe564dd1700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5600b55c0 0x7fe5600b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 -- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548004850 msgr2=0x7fe560152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548004850 0x7fe560152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 -- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5600b55c0 msgr2=0x7fe5600b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5600b55c0 0x7fe5600b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 -- 172.21.15.5:0/2432112942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5601529d0 con 0x7fe56015f5d0 2024-05-08T23:23:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55f7fe700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe56015f5d0 0x7fe56015fa50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c0e0 tx=0x7fe554001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55cff9700 1 -- 172.21.15.5:0/2432112942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554004c80 con 0x7fe56015f5d0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55cff9700 1 -- 172.21.15.5:0/2432112942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe554004e20 con 0x7fe56015f5d0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5600b62b0 con 0x7fe56015f5d0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe55cff9700 1 -- 172.21.15.5:0/2432112942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554007310 con 0x7fe56015f5d0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.352+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5600c53e0 con 0x7fe56015f5d0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.353+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe560068000 con 0x7fe56015f5d0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.354+0000 7fe55cff9700 1 -- 172.21.15.5:0/2432112942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe554007470 con 0x7fe56015f5d0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.354+0000 7fe55cff9700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe54c06ac90 0x7fe54c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.354+0000 7fe55cff9700 1 -- 172.21.15.5:0/2432112942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe554089810 con 0x7fe56015f5d0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.355+0000 7fe55ffff700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe54c06ac90 0x7fe54c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.355+0000 7fe55ffff700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe54c06ac90 0x7fe54c06d150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe560069190 tx=0x7fe550009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.359+0000 7fe55cff9700 1 -- 172.21.15.5:0/2432112942 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe554054dd0 con 0x7fe56015f5d0 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.660+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe5600d0f50 con 0x7fe56015f5d0 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.661+0000 7fe55cff9700 1 -- 172.21.15.5:0/2432112942 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe5540583f0 con 0x7fe56015f5d0 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm4+KLORAAJGma8zcoGLift7K5wyvWlA==", 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm5iMNDBAANT+UrtTUBpYyQ9dzJyS01Q==", 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmRixkBhAAk9s24ERPJ/6nCJY3g7QppA==", 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmPmurExAAxIzDzbLZ3s35XVJrvYiVCA==", 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:23:35.664 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCrCDxmI4/bNBAAzNXcuLQCS9gaAJ0r4iYKlA==", 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCuCDxmMS6HNhAAcTP6nStED06bdM4MtbEPjQ==", 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCxCDxmDGv5NRAAljFQZXLGWwt+UXZ+HCdx0Q==", 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-08T23:23:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC0CDxmL+5LNRAAVPySIUtICBwIokJL9fHFtw==", 2024-05-08T23:23:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:23:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:23:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC5CDxm/F15BhAAEx9mKA9izSmb+xyxjEi9FA==", 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC8CDxm0zAYBhAAjHT+TZFqk5MWvd0P5LmwfA==", 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC/CDxmPcbiBRAAOmWNgaQGi6YGKTp8SJekUg==", 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDCCDxm9FsDBRAA/lfmrpmL4UO3s/tJJ4o6yQ==", 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm9tLAGBAAK8Lq89gJ902XqzXYAbYfkg==", 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmCStoIBAAhraObJ5LS9LtF97+hIcTFQ==", 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm2NzHHBAAUbeD58MntIoiwT8LekjIlw==", 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmyvveIxAATEn7nOPHqtqSbdfQid25HA==", 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm3ZIxDxAAM8M5YUfK9G2PkTzU4wY/+w==", 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxmntlgNRAA7JOIS+0NL1KmSSZhZjXlVQ==", 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm1dPQMBAAhu0CK6DcucQteuTo7qN2IQ==", 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe54c06ac90 msgr2=0x7fe54c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe54c06ac90 0x7fe54c06d150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe560069190 tx=0x7fe550009040 comp rx=0 tx=0).stop 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe56015f5d0 msgr2=0x7fe56015fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe56015f5d0 0x7fe56015fa50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c0e0 tx=0x7fe554001930 comp rx=0 tx=0).stop 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 shutdown_connections 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe54c06ac90 0x7fe54c06d150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5600b55c0 0x7fe5600b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe548004850 0x7fe560152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 --2- 172.21.15.5:0/2432112942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe56015f5d0 0x7fe56015fa50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 >> 172.21.15.5:0/2432112942 conn(0x7fe5600b2f20 msgr2=0x7fe56006fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.663+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 shutdown_connections 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.664+0000 7fe566834700 1 -- 172.21.15.5:0/2432112942 wait complete. 2024-05-08T23:23:35.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:35.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.820+0000 7fc4f24ae700 1 Processor -- start 2024-05-08T23:23:35.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.820+0000 7fc4f24ae700 1 -- start start 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4f24ae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 0x7fc4ec0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4f24ae700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc4ec0cf000 0x7fc4ec0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4f24ae700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ec0cfe50 con 0x7fc4ec0cf9c0 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4f24ae700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ec0cd400 con 0x7fc4ec0cf000 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4f24ae700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec09ad40 con 0x7fc4ec15ef60 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4eb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc4ec0cf000 0x7fc4ec0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4eb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc4ec0cf000 0x7fc4ec0cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51650/0 (socket says 172.21.15.5:51650) 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4eb7fe700 1 -- 172.21.15.5:0/1378801822 learned_addr learned my addr 172.21.15.5:0/1378801822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4ebfff700 1 --2- 172.21.15.5:0/1378801822 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 0x7fc4ec0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4eb7fe700 1 -- 172.21.15.5:0/1378801822 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 msgr2=0x7fc4ec0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4eb7fe700 1 --2- 172.21.15.5:0/1378801822 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 0x7fc4ec0b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4eb7fe700 1 -- 172.21.15.5:0/1378801822 >> v1:172.21.15.49:6789/0 conn(0x7fc4ec0cf9c0 legacy=0x7fc4ec099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.821+0000 7fc4eb7fe700 1 -- 172.21.15.5:0/1378801822 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec0ccb60 con 0x7fc4ec0cf000 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4eb7fe700 1 --2- 172.21.15.5:0/1378801822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec0cf480 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00098f0 tx=0x7fc4e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79b5161ed8e6c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4eaffd700 1 -- 172.21.15.5:0/1378801822 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000b8a0 con 0x7fc4ec0cf000 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4eaffd700 1 -- 172.21.15.5:0/1378801822 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e000ba00 con 0x7fc4ec0cf000 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4eaffd700 1 -- 172.21.15.5:0/1378801822 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000bd90 con 0x7fc4ec0cf000 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4f24ae700 1 -- 172.21.15.5:0/1378801822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 msgr2=0x7fc4ec0cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/1378801822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec0cf480 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00098f0 tx=0x7fc4e0009c00 comp rx=0 tx=0).stop 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4f24ae700 1 -- 172.21.15.5:0/1378801822 shutdown_connections 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/1378801822 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 0x7fc4ec0b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/1378801822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec0cf480 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.822+0000 7fc4f24ae700 1 -- 172.21.15.5:0/1378801822 >> 172.21.15.5:0/1378801822 conn(0x7fc4ec0b2f20 msgr2=0x7fc4ec0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.823+0000 7fc4f24ae700 1 -- 172.21.15.5:0/1378801822 shutdown_connections 2024-05-08T23:23:35.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.823+0000 7fc4f24ae700 1 -- 172.21.15.5:0/1378801822 wait complete. 2024-05-08T23:23:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.824+0000 7fc4f24ae700 1 Processor -- start 2024-05-08T23:23:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.824+0000 7fc4f24ae700 1 -- start start 2024-05-08T23:23:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4f24ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4f24ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4ec0cf9c0 0x7fc4ec1f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4f24ae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 0x7fc4ec0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4f24ae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec1537b0 con 0x7fc4ec0cf000 2024-05-08T23:23:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4f24ae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec15a860 con 0x7fc4ec0cf9c0 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4f24ae700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec1f5610 con 0x7fc4ec15ef60 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51662/0 (socket says 172.21.15.5:51662) 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4ebfff700 1 -- 172.21.15.5:0/905759045 learned_addr learned my addr 172.21.15.5:0/905759045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4eb7fe700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4ec0cf9c0 0x7fc4ec1f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4f0a4b700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 0x7fc4ec0cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4ebfff700 1 -- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 msgr2=0x7fc4ec0cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4ebfff700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 0x7fc4ec0cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4ebfff700 1 -- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4ec0cf9c0 msgr2=0x7fc4ec1f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4ebfff700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4ec0cf9c0 0x7fc4ec1f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.825+0000 7fc4ebfff700 1 -- 172.21.15.5:0/905759045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e0009550 con 0x7fc4ec0cf000 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.826+0000 7fc4ebfff700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec153270 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00c7f0 tx=0x7fc4dc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.826+0000 7fc4e8ff9700 1 -- 172.21.15.5:0/905759045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc011800 con 0x7fc4ec0cf000 2024-05-08T23:23:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.826+0000 7fc4e8ff9700 1 -- 172.21.15.5:0/905759045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc011960 con 0x7fc4ec0cf000 2024-05-08T23:23:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.826+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec1f0f90 con 0x7fc4ec0cf000 2024-05-08T23:23:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.826+0000 7fc4e8ff9700 1 -- 172.21.15.5:0/905759045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc011cf0 con 0x7fc4ec0cf000 2024-05-08T23:23:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.826+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec159fc0 con 0x7fc4ec0cf000 2024-05-08T23:23:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.827+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec068000 con 0x7fc4ec0cf000 2024-05-08T23:23:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.827+0000 7fc4e8ff9700 1 -- 172.21.15.5:0/905759045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc4dc018360 con 0x7fc4ec0cf000 2024-05-08T23:23:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.828+0000 7fc4e8ff9700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc4d806ac90 0x7fc4d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.828+0000 7fc4e8ff9700 1 -- 172.21.15.5:0/905759045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4dc088ed0 con 0x7fc4ec0cf000 2024-05-08T23:23:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.828+0000 7fc4eb7fe700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc4d806ac90 0x7fc4d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.829+0000 7fc4eb7fe700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc4d806ac90 0x7fc4d806d150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0014040 tx=0x7fc4e0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:35.833+0000 7fc4e8ff9700 1 -- 172.21.15.5:0/905759045 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc4dc054490 con 0x7fc4ec0cf000 2024-05-08T23:23:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.050+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4ec0b63a0 con 0x7fc4ec0cf000 2024-05-08T23:23:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.051+0000 7fc4e8ff9700 1 -- 172.21.15.5:0/905759045 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12293 (secure 0 0 0) 0x7fc4dc057ab0 con 0x7fc4ec0cf000 2024-05-08T23:23:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:36.051 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:35.181950+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc4d806ac90 msgr2=0x7fc4d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc4d806ac90 0x7fc4d806d150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0014040 tx=0x7fc4e0008030 comp rx=0 tx=0).stop 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 msgr2=0x7fc4ec153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec153270 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00c7f0 tx=0x7fc4dc00cbb0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 shutdown_connections 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc4d806ac90 0x7fc4d806d150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4ec15ef60 0x7fc4ec0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4ec0cf000 0x7fc4ec153270 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 --2- 172.21.15.5:0/905759045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4ec0cf9c0 0x7fc4ec1f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 >> 172.21.15.5:0/905759045 conn(0x7fc4ec0b2f20 msgr2=0x7fc4ec0b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 shutdown_connections 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.053+0000 7fc4f24ae700 1 -- 172.21.15.5:0/905759045 wait complete. 2024-05-08T23:23:36.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:23:36.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:23:36.091 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-05-08T23:23:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.207+0000 7ff8a321d700 1 Processor -- start 2024-05-08T23:23:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.208+0000 7ff8a321d700 1 -- start start 2024-05-08T23:23:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.208+0000 7ff8a321d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff89c0cfd10 0x7ff89c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.208+0000 7ff8a321d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c15bb20 0x7ff89c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.208+0000 7ff8a321d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c0cc980 con 0x7ff89c06ec20 2024-05-08T23:23:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.208+0000 7ff8a321d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c06f0d0 con 0x7ff89c0cfd10 2024-05-08T23:23:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.208+0000 7ff8a321d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0d00f0 con 0x7ff89c15bb20 2024-05-08T23:23:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a0fb9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff89c0cfd10 0x7ff89c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a17ba700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff89c06ec20 0x7ff89c159f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33696/0 (socket says 172.21.15.5:33696) 2024-05-08T23:23:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a17ba700 1 -- 172.21.15.5:0/3117816376 learned_addr learned my addr 172.21.15.5:0/3117816376 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a0fb9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff89c0cfd10 0x7ff89c06dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51672/0 (socket says 172.21.15.5:51672) 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff89bfff700 1 --2- 172.21.15.5:0/3117816376 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c15bb20 0x7ff89c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a0fb9700 1 -- 172.21.15.5:0/3117816376 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c15bb20 msgr2=0x7ff89c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a0fb9700 1 --2- 172.21.15.5:0/3117816376 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c15bb20 0x7ff89c06e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a0fb9700 1 -- 172.21.15.5:0/3117816376 >> v1:172.21.15.5:6789/0 conn(0x7ff89c06ec20 legacy=0x7ff89c159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a0fb9700 1 -- 172.21.15.5:0/3117816376 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c06e420 con 0x7ff89c0cfd10 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff8a0fb9700 1 --2- 172.21.15.5:0/3117816376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c0cfd10 0x7ff89c06dee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009840 tx=0x7ff88c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ecac151478d222 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff89b7fe700 1 -- 172.21.15.5:0/3117816376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c011830 con 0x7ff89c0cfd10 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff89b7fe700 1 -- 172.21.15.5:0/3117816376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c011990 con 0x7ff89c0cfd10 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.209+0000 7ff89b7fe700 1 -- 172.21.15.5:0/3117816376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c011d20 con 0x7ff89c0cfd10 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.210+0000 7ff8a321d700 1 -- 172.21.15.5:0/3117816376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c0cfd10 msgr2=0x7ff89c06dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.210+0000 7ff8a321d700 1 --2- 172.21.15.5:0/3117816376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c0cfd10 0x7ff89c06dee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009840 tx=0x7ff88c009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.210+0000 7ff8a321d700 1 -- 172.21.15.5:0/3117816376 shutdown_connections 2024-05-08T23:23:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.210+0000 7ff8a321d700 1 --2- 172.21.15.5:0/3117816376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c0cfd10 0x7ff89c06dee0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.210+0000 7ff8a321d700 1 --2- 172.21.15.5:0/3117816376 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c15bb20 0x7ff89c06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.210+0000 7ff8a321d700 1 -- 172.21.15.5:0/3117816376 >> 172.21.15.5:0/3117816376 conn(0x7ff89c099a50 msgr2=0x7ff89c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.210+0000 7ff8a321d700 1 -- 172.21.15.5:0/3117816376 shutdown_connections 2024-05-08T23:23:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.210+0000 7ff8a321d700 1 -- 172.21.15.5:0/3117816376 wait complete. 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.211+0000 7ff8a321d700 1 Processor -- start 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.211+0000 7ff8a321d700 1 -- start start 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.212+0000 7ff8a321d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c06ec20 0x7ff89c14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.212+0000 7ff8a321d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c0cfd10 0x7ff89c0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.212+0000 7ff8a321d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c15bb20 0x7ff89c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.212+0000 7ff8a321d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0b4bb0 con 0x7ff89c15bb20 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.212+0000 7ff8a321d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0b4e50 con 0x7ff89c06ec20 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.212+0000 7ff8a321d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0b50f0 con 0x7ff89c0cfd10 2024-05-08T23:23:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.212+0000 7ff8a17ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c15bb20 0x7ff89c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.212+0000 7ff8a17ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c15bb20 0x7ff89c0c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51686/0 (socket says 172.21.15.5:51686) 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a17ba700 1 -- 172.21.15.5:0/2668776094 learned_addr learned my addr 172.21.15.5:0/2668776094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a0fb9700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c06ec20 0x7ff89c14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff89bfff700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c0cfd10 0x7ff89c0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a17ba700 1 -- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c0cfd10 msgr2=0x7ff89c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a17ba700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c0cfd10 0x7ff89c0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a17ba700 1 -- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c06ec20 msgr2=0x7ff89c14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a17ba700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c06ec20 0x7ff89c14ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a17ba700 1 -- 172.21.15.5:0/2668776094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c0094a0 con 0x7ff89c15bb20 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a17ba700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c15bb20 0x7ff89c0c5fd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff89400cbc0 tx=0x7ff89400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8997fa700 1 -- 172.21.15.5:0/2668776094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89400a9e0 con 0x7ff89c15bb20 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8997fa700 1 -- 172.21.15.5:0/2668776094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89400ab40 con 0x7ff89c15bb20 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89c0b53d0 con 0x7ff89c15bb20 2024-05-08T23:23:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8997fa700 1 -- 172.21.15.5:0/2668776094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894012400 con 0x7ff89c15bb20 2024-05-08T23:23:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.213+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c0b5990 con 0x7ff89c15bb20 2024-05-08T23:23:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.214+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89c0b5bb0 con 0x7ff89c15bb20 2024-05-08T23:23:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.215+0000 7ff8997fa700 1 -- 172.21.15.5:0/2668776094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff89401b3b0 con 0x7ff89c15bb20 2024-05-08T23:23:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.215+0000 7ff8997fa700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff87c06ac90 0x7ff87c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.215+0000 7ff8997fa700 1 -- 172.21.15.5:0/2668776094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff894088a80 con 0x7ff89c15bb20 2024-05-08T23:23:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.215+0000 7ff8a0fb9700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff87c06ac90 0x7ff87c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.216+0000 7ff8a0fb9700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff87c06ac90 0x7ff87c06d150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0090b0 tx=0x7ff88c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.220+0000 7ff8997fa700 1 -- 172.21.15.5:0/2668776094 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff894054050 con 0x7ff89c15bb20 2024-05-08T23:23:36.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.439+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff89c074bc0 con 0x7ff89c15bb20 2024-05-08T23:23:36.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.439+0000 7ff8997fa700 1 -- 172.21.15.5:0/2668776094 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8940170a0 con 0x7ff89c15bb20 2024-05-08T23:23:36.439 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:23:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff87c06ac90 msgr2=0x7ff87c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff87c06ac90 0x7ff87c06d150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0090b0 tx=0x7ff88c008030 comp rx=0 tx=0).stop 2024-05-08T23:23:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c15bb20 msgr2=0x7ff89c0c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c15bb20 0x7ff89c0c5fd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff89400cbc0 tx=0x7ff89400ced0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 shutdown_connections 2024-05-08T23:23:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff87c06ac90 0x7ff87c06d150 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff89c15bb20 0x7ff89c0c5fd0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c06ec20 0x7ff89c14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 --2- 172.21.15.5:0/2668776094 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff89c0cfd10 0x7ff89c0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 >> 172.21.15.5:0/2668776094 conn(0x7ff89c099a50 msgr2=0x7ff89c073930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 shutdown_connections 2024-05-08T23:23:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.442+0000 7ff8a321d700 1 -- 172.21.15.5:0/2668776094 wait complete. 2024-05-08T23:23:36.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:23:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.591+0000 7fdd1759e700 1 Processor -- start 2024-05-08T23:23:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.592+0000 7fdd1759e700 1 -- start start 2024-05-08T23:23:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1759e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd180d12b0 0x7fdd1815df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1759e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdd18161ae0 0x7fdd18161f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1759e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1806ce00 con 0x7fdd1806c950 2024-05-08T23:23:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1759e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd18162470 con 0x7fdd180d12b0 2024-05-08T23:23:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1759e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1806d920 con 0x7fdd18161ae0 2024-05-08T23:23:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1659c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd180d12b0 0x7fdd1815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1659c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd180d12b0 0x7fdd1815df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51694/0 (socket says 172.21.15.5:51694) 2024-05-08T23:23:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1659c700 1 -- 172.21.15.5:0/1415898659 learned_addr learned my addr 172.21.15.5:0/1415898659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd16d9d700 1 --2- 172.21.15.5:0/1415898659 >> v2:172.21.15.49:3300/0 conn(0x7fdd18161ae0 0x7fdd18161f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1659c700 1 -- 172.21.15.5:0/1415898659 >> v2:172.21.15.49:3300/0 conn(0x7fdd18161ae0 msgr2=0x7fdd18161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1659c700 1 --2- 172.21.15.5:0/1415898659 >> v2:172.21.15.49:3300/0 conn(0x7fdd18161ae0 0x7fdd18161f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1659c700 1 -- 172.21.15.5:0/1415898659 >> v1:172.21.15.49:6789/0 conn(0x7fdd1806c950 legacy=0x7fdd180d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1659c700 1 -- 172.21.15.5:0/1415898659 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd18161600 con 0x7fdd180d12b0 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1659c700 1 --2- 172.21.15.5:0/1415898659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd180d12b0 0x7fdd1815df70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdd180d1e70 tx=0x7fdd08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac8890a7c1ab8e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.593+0000 7fdd1559a700 1 -- 172.21.15.5:0/1415898659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08011830 con 0x7fdd180d12b0 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1559a700 1 -- 172.21.15.5:0/1415898659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd08011990 con 0x7fdd180d12b0 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1559a700 1 -- 172.21.15.5:0/1415898659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08011d20 con 0x7fdd180d12b0 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1759e700 1 -- 172.21.15.5:0/1415898659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd180d12b0 msgr2=0x7fdd1815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1759e700 1 --2- 172.21.15.5:0/1415898659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd180d12b0 0x7fdd1815df70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdd180d1e70 tx=0x7fdd08009bd0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1759e700 1 -- 172.21.15.5:0/1415898659 shutdown_connections 2024-05-08T23:23:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1759e700 1 --2- 172.21.15.5:0/1415898659 >> v2:172.21.15.49:3300/0 conn(0x7fdd18161ae0 0x7fdd18161f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1759e700 1 --2- 172.21.15.5:0/1415898659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd180d12b0 0x7fdd1815df70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1759e700 1 -- 172.21.15.5:0/1415898659 >> 172.21.15.5:0/1415898659 conn(0x7fdd180b2f30 msgr2=0x7fdd180b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1759e700 1 -- 172.21.15.5:0/1415898659 shutdown_connections 2024-05-08T23:23:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.594+0000 7fdd1759e700 1 -- 172.21.15.5:0/1415898659 wait complete. 2024-05-08T23:23:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.596+0000 7fdd1759e700 1 Processor -- start 2024-05-08T23:23:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.596+0000 7fdd1759e700 1 -- start start 2024-05-08T23:23:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd1759e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd1806c950 0x7fdd180c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd1759e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd180d12b0 0x7fdd180b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd1759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd18161ae0 0x7fdd180ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd1759e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180c6a40 con 0x7fdd18161ae0 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd1759e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180c6d20 con 0x7fdd180d12b0 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd1759e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180c7000 con 0x7fdd1806c950 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd18161ae0 0x7fdd180ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd18161ae0 0x7fdd180ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51702/0 (socket says 172.21.15.5:51702) 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 -- 172.21.15.5:0/4177909160 learned_addr learned my addr 172.21.15.5:0/4177909160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd15d9b700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd180d12b0 0x7fdd180b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd1659c700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd1806c950 0x7fdd180c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 -- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd1806c950 msgr2=0x7fdd180c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd1806c950 0x7fdd180c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 -- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd180d12b0 msgr2=0x7fdd180b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd180d12b0 0x7fdd180b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 -- 172.21.15.5:0/4177909160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08009550 con 0x7fdd18161ae0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.597+0000 7fdd16d9d700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd18161ae0 0x7fdd180ca850 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000efc0 tx=0x7fdd1000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.598+0000 7fdd0effd700 1 -- 172.21.15.5:0/4177909160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000cad0 con 0x7fdd18161ae0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.598+0000 7fdd0effd700 1 -- 172.21.15.5:0/4177909160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1000f040 con 0x7fdd18161ae0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.598+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd180c72e0 con 0x7fdd18161ae0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.598+0000 7fdd0effd700 1 -- 172.21.15.5:0/4177909160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000ce70 con 0x7fdd18161ae0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.598+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd180b5b30 con 0x7fdd18161ae0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.599+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd18074b50 con 0x7fdd18161ae0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.599+0000 7fdd0effd700 1 -- 172.21.15.5:0/4177909160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdd10014480 con 0x7fdd18161ae0 2024-05-08T23:23:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.600+0000 7fdd0effd700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdcf806ac90 0x7fdcf806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.600+0000 7fdd0effd700 1 -- 172.21.15.5:0/4177909160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdd10089680 con 0x7fdd18161ae0 2024-05-08T23:23:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.600+0000 7fdd1659c700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdcf806ac90 0x7fdcf806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.601+0000 7fdd1659c700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdcf806ac90 0x7fdcf806d150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009c10 tx=0x7fdd08008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.605+0000 7fdd0effd700 1 -- 172.21.15.5:0/4177909160 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdd10054c40 con 0x7fdd18161ae0 2024-05-08T23:23:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.828+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdd180b3d30 con 0x7fdd18161ae0 2024-05-08T23:23:36.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.828+0000 7fdd0effd700 1 -- 172.21.15.5:0/4177909160 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdd10018060 con 0x7fdd18161ae0 2024-05-08T23:23:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:36.828 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:23:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:23:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-08T23:23:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-08T23:23:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:23:36.829 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.830+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdcf806ac90 msgr2=0x7fdcf806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.830+0000 7fdd1759e700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdcf806ac90 0x7fdcf806d150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009c10 tx=0x7fdd08008030 comp rx=0 tx=0).stop 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.830+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd18161ae0 msgr2=0x7fdd180ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.830+0000 7fdd1759e700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd18161ae0 0x7fdd180ca850 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000efc0 tx=0x7fdd1000c0c0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.831+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 shutdown_connections 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.831+0000 7fdd1759e700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdcf806ac90 0x7fdcf806d150 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.831+0000 7fdd1759e700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd18161ae0 0x7fdd180ca850 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.831+0000 7fdd1759e700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd1806c950 0x7fdd180c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.831+0000 7fdd1759e700 1 --2- 172.21.15.5:0/4177909160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd180d12b0 0x7fdd180b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.831+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 >> 172.21.15.5:0/4177909160 conn(0x7fdd180b2f30 msgr2=0x7fdd180b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.831+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 shutdown_connections 2024-05-08T23:23:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.831+0000 7fdd1759e700 1 -- 172.21.15.5:0/4177909160 wait complete. 2024-05-08T23:23:36.844 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:23:36.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:23:36.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.985+0000 7fa963f25700 1 Processor -- start 2024-05-08T23:23:36.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.985+0000 7fa963f25700 1 -- start start 2024-05-08T23:23:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa963f25700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa95c0aea70 0x7fa95c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa963f25700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa95c0d0a60 0x7fa95c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa963f25700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c15add0 con 0x7fa95c154df0 2024-05-08T23:23:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa963f25700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c15abe0 con 0x7fa95c0aea70 2024-05-08T23:23:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa963f25700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c15b070 con 0x7fa95c0d0a60 2024-05-08T23:23:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa9614c0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa95c0aea70 0x7fa95c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa9614c0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa95c0aea70 0x7fa95c0b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51708/0 (socket says 172.21.15.5:51708) 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa9614c0700 1 -- 172.21.15.5:0/3987624280 learned_addr learned my addr 172.21.15.5:0/3987624280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.986+0000 7fa9624c2700 1 --2- 172.21.15.5:0/3987624280 >> v2:172.21.15.49:3300/0 conn(0x7fa95c0d0a60 0x7fa95c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.987+0000 7fa9614c0700 1 -- 172.21.15.5:0/3987624280 >> v2:172.21.15.49:3300/0 conn(0x7fa95c0d0a60 msgr2=0x7fa95c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.987+0000 7fa9614c0700 1 --2- 172.21.15.5:0/3987624280 >> v2:172.21.15.49:3300/0 conn(0x7fa95c0d0a60 0x7fa95c0b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.987+0000 7fa9614c0700 1 -- 172.21.15.5:0/3987624280 >> v1:172.21.15.49:6789/0 conn(0x7fa95c154df0 legacy=0x7fa95c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.987+0000 7fa9614c0700 1 -- 172.21.15.5:0/3987624280 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c15b350 con 0x7fa95c0aea70 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.987+0000 7fa9614c0700 1 --2- 172.21.15.5:0/3987624280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c0aea70 0x7fa95c0b3e70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0cc4a0 tx=0x7fa94c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03e1af539c2f262 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.987+0000 7fa960cbf700 1 -- 172.21.15.5:0/3987624280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c00dcf0 con 0x7fa95c0aea70 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.987+0000 7fa960cbf700 1 -- 172.21.15.5:0/3987624280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c00de50 con 0x7fa95c0aea70 2024-05-08T23:23:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.987+0000 7fa960cbf700 1 -- 172.21.15.5:0/3987624280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004250 con 0x7fa95c0aea70 2024-05-08T23:23:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.988+0000 7fa963f25700 1 -- 172.21.15.5:0/3987624280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c0aea70 msgr2=0x7fa95c0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.988+0000 7fa963f25700 1 --2- 172.21.15.5:0/3987624280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c0aea70 0x7fa95c0b3e70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0cc4a0 tx=0x7fa94c00be70 comp rx=0 tx=0).stop 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.988+0000 7fa963f25700 1 -- 172.21.15.5:0/3987624280 shutdown_connections 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.988+0000 7fa963f25700 1 --2- 172.21.15.5:0/3987624280 >> v2:172.21.15.49:3300/0 conn(0x7fa95c0d0a60 0x7fa95c0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.988+0000 7fa963f25700 1 --2- 172.21.15.5:0/3987624280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c0aea70 0x7fa95c0b3e70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.988+0000 7fa963f25700 1 -- 172.21.15.5:0/3987624280 >> 172.21.15.5:0/3987624280 conn(0x7fa95c099a50 msgr2=0x7fa95c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.988+0000 7fa963f25700 1 -- 172.21.15.5:0/3987624280 shutdown_connections 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.988+0000 7fa963f25700 1 -- 172.21.15.5:0/3987624280 wait complete. 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.989+0000 7fa963f25700 1 Processor -- start 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.989+0000 7fa963f25700 1 -- start start 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa963f25700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa95c0aea70 0x7fa95c15c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa963f25700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa95c0d0a60 0x7fa95c1487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa963f25700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c154df0 0x7fa95c148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa963f25700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1f3120 con 0x7fa95c154df0 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa963f25700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1e5140 con 0x7fa95c0aea70 2024-05-08T23:23:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa963f25700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1e40a0 con 0x7fa95c0d0a60 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9624c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c154df0 0x7fa95c148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9624c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c154df0 0x7fa95c148ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51712/0 (socket says 172.21.15.5:51712) 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9624c2700 1 -- 172.21.15.5:0/3324428297 learned_addr learned my addr 172.21.15.5:0/3324428297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa961cc1700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa95c0aea70 0x7fa95c15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9614c0700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa95c0d0a60 0x7fa95c1487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9624c2700 1 -- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa95c0d0a60 msgr2=0x7fa95c1487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9624c2700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa95c0d0a60 0x7fa95c1487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9624c2700 1 -- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa95c0aea70 msgr2=0x7fa95c15c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9624c2700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa95c0aea70 0x7fa95c15c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.990+0000 7fa9624c2700 1 -- 172.21.15.5:0/3324428297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c00b7f0 con 0x7fa95c154df0 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.991+0000 7fa9624c2700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c154df0 0x7fa95c148ce0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa954007e40 tx=0x7fa95400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.991+0000 7fa9527fc700 1 -- 172.21.15.5:0/3324428297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95400ed30 con 0x7fa95c154df0 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.991+0000 7fa9527fc700 1 -- 172.21.15.5:0/3324428297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954014420 con 0x7fa95c154df0 2024-05-08T23:23:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.991+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c1e4380 con 0x7fa95c154df0 2024-05-08T23:23:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.991+0000 7fa9527fc700 1 -- 172.21.15.5:0/3324428297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9540147d0 con 0x7fa95c154df0 2024-05-08T23:23:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.991+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c1e4940 con 0x7fa95c154df0 2024-05-08T23:23:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.992+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c0b48f0 con 0x7fa95c154df0 2024-05-08T23:23:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.992+0000 7fa9527fc700 1 -- 172.21.15.5:0/3324428297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa954014930 con 0x7fa95c154df0 2024-05-08T23:23:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.993+0000 7fa9527fc700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa94006ac90 0x7fa94006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.993+0000 7fa9527fc700 1 -- 172.21.15.5:0/3324428297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa954089690 con 0x7fa95c154df0 2024-05-08T23:23:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.993+0000 7fa961cc1700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa94006ac90 0x7fa94006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.994+0000 7fa961cc1700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa94006ac90 0x7fa94006d150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa958009960 tx=0x7fa9580090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:36.998+0000 7fa9527fc700 1 -- 172.21.15.5:0/3324428297 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa954054bc0 con 0x7fa95c154df0 2024-05-08T23:23:37.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:37.216+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa95c074bc0 con 0x7fa95c154df0 2024-05-08T23:23:37.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:23:37.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:23:38.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.007+0000 7fa9527fc700 1 -- 172.21.15.5:0/3324428297 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa9540581e0 con 0x7fa95c154df0 2024-05-08T23:23:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.012+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa94006ac90 msgr2=0x7fa94006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.012+0000 7fa963f25700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa94006ac90 0x7fa94006d150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa958009960 tx=0x7fa9580090e0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.012+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c154df0 msgr2=0x7fa95c148ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.012+0000 7fa963f25700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c154df0 0x7fa95c148ce0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa954007e40 tx=0x7fa95400e3c0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.013+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 shutdown_connections 2024-05-08T23:23:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.013+0000 7fa963f25700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa94006ac90 0x7fa94006d150 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.013+0000 7fa963f25700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa95c154df0 0x7fa95c148ce0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.013+0000 7fa963f25700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa95c0aea70 0x7fa95c15c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.013+0000 7fa963f25700 1 --2- 172.21.15.5:0/3324428297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa95c0d0a60 0x7fa95c1487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.013+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 >> 172.21.15.5:0/3324428297 conn(0x7fa95c099a50 msgr2=0x7fa95c0ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.013+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 shutdown_connections 2024-05-08T23:23:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.013+0000 7fa963f25700 1 -- 172.21.15.5:0/3324428297 wait complete. 2024-05-08T23:23:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-08T23:23:38.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:23:38.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.176+0000 7f421b8ac700 1 Processor -- start 2024-05-08T23:23:38.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.176+0000 7f421b8ac700 1 -- start start 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.177+0000 7f421b8ac700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f421409a790 0x7f4214099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.177+0000 7f421b8ac700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421406d010 con 0x7f421415da60 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.177+0000 7f421b8ac700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421415de40 con 0x7f4214099f90 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.177+0000 7f421b8ac700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421409a420 con 0x7f421409a790 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.177+0000 7f4218e47700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f421409a790 0x7f4214099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4219648700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f421415da60 0x7f42140d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50726/0 (socket says 172.21.15.5:50726) 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4219648700 1 -- 172.21.15.5:0/3136915880 learned_addr learned my addr 172.21.15.5:0/3136915880 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4218e47700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f421409a790 0x7f4214099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35570/0 (socket says 172.21.15.5:35570) 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4218e47700 1 -- 172.21.15.5:0/3136915880 >> v1:172.21.15.49:6789/0 conn(0x7f4214099f90 legacy=0x7f42140cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4218e47700 1 -- 172.21.15.5:0/3136915880 >> v1:172.21.15.5:6789/0 conn(0x7f421415da60 legacy=0x7f42140d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4218e47700 1 -- 172.21.15.5:0/3136915880 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140d0b20 con 0x7f421409a790 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4207fff700 1 -- 172.21.15.5:0/3136915880 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046756077 0 0) 0x7f421406d010 con 0x7f421415da60 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4218e47700 1 --2- 172.21.15.5:0/3136915880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f421409a790 0x7f4214099a50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4208009840 tx=0x7f4208009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9113a669fdd8b8ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4207fff700 1 -- 172.21.15.5:0/3136915880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208011830 con 0x7f421409a790 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4207fff700 1 -- 172.21.15.5:0/3136915880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4208011990 con 0x7f421409a790 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.178+0000 7f4207fff700 1 -- 172.21.15.5:0/3136915880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208011d20 con 0x7f421409a790 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.179+0000 7f421b8ac700 1 -- 172.21.15.5:0/3136915880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f421409a790 msgr2=0x7f4214099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.179+0000 7f421b8ac700 1 --2- 172.21.15.5:0/3136915880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f421409a790 0x7f4214099a50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4208009840 tx=0x7f4208009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.179+0000 7f421b8ac700 1 -- 172.21.15.5:0/3136915880 shutdown_connections 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.179+0000 7f421b8ac700 1 --2- 172.21.15.5:0/3136915880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f421409a790 0x7f4214099a50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.179+0000 7f421b8ac700 1 -- 172.21.15.5:0/3136915880 >> 172.21.15.5:0/3136915880 conn(0x7f42140b2f60 msgr2=0x7f42140b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.179+0000 7f421b8ac700 1 -- 172.21.15.5:0/3136915880 shutdown_connections 2024-05-08T23:23:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.179+0000 7f421b8ac700 1 -- 172.21.15.5:0/3136915880 wait complete. 2024-05-08T23:23:38.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.180+0000 7f421b8ac700 1 Processor -- start 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.180+0000 7f421b8ac700 1 -- start start 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f421b8ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4214099f90 0x7f421406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f421b8ac700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f421409a790 0x7f4214160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f421b8ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f421415da60 0x7f4214160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f421b8ac700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42141540c0 con 0x7f4214099f90 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f421b8ac700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421415e0c0 con 0x7f421415da60 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f421b8ac700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42140c7000 con 0x7f421409a790 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f4219648700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4214099f90 0x7f421406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f4219648700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4214099f90 0x7f421406bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35582/0 (socket says 172.21.15.5:35582) 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f4219648700 1 -- 172.21.15.5:0/3495333888 learned_addr learned my addr 172.21.15.5:0/3495333888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f4219e49700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f421415da60 0x7f4214160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.181+0000 7f4218e47700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f421409a790 0x7f4214160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4219648700 1 -- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f421409a790 msgr2=0x7f4214160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4219648700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f421409a790 0x7f4214160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4219648700 1 -- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f421415da60 msgr2=0x7f4214160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4219648700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f421415da60 0x7f4214160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4219648700 1 -- 172.21.15.5:0/3495333888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080094a0 con 0x7f4214099f90 2024-05-08T23:23:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4219e49700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f421415da60 0x7f4214160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4218e47700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f421409a790 0x7f4214160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:38.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4219648700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4214099f90 0x7f421406bca0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f421000c520 tx=0x7f421000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:38.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4205ffb700 1 -- 172.21.15.5:0/3495333888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210004c80 con 0x7f4214099f90 2024-05-08T23:23:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4205ffb700 1 -- 172.21.15.5:0/3495333888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4210004e20 con 0x7f4214099f90 2024-05-08T23:23:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42140c72e0 con 0x7f4214099f90 2024-05-08T23:23:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f4205ffb700 1 -- 172.21.15.5:0/3495333888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210007310 con 0x7f4214099f90 2024-05-08T23:23:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.182+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42140c78a0 con 0x7f4214099f90 2024-05-08T23:23:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.183+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4214074bc0 con 0x7f4214099f90 2024-05-08T23:23:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.184+0000 7f4205ffb700 1 -- 172.21.15.5:0/3495333888 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4210007470 con 0x7f4214099f90 2024-05-08T23:23:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.184+0000 7f4205ffb700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f41f806ac90 0x7f41f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.184+0000 7f4205ffb700 1 -- 172.21.15.5:0/3495333888 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4210088e40 con 0x7f4214099f90 2024-05-08T23:23:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.185+0000 7f4218e47700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f41f806ac90 0x7f41f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.185+0000 7f4218e47700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f41f806ac90 0x7f41f806d150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f421406c670 tx=0x7f420801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:38.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.189+0000 7f4205ffb700 1 -- 172.21.15.5:0/3495333888 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4210054200 con 0x7f4214099f90 2024-05-08T23:23:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:38.407+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f42140cb170 con 0x7f4214099f90 2024-05-08T23:23:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.011+0000 7f4205ffb700 1 -- 172.21.15.5:0/3495333888 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4210057820 con 0x7f4214099f90 2024-05-08T23:23:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f41f806ac90 msgr2=0x7f41f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f41f806ac90 0x7f41f806d150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f421406c670 tx=0x7f420801c040 comp rx=0 tx=0).stop 2024-05-08T23:23:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4214099f90 msgr2=0x7f421406bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4214099f90 0x7f421406bca0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f421000c520 tx=0x7f421000c8e0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 shutdown_connections 2024-05-08T23:23:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f41f806ac90 0x7f41f806d150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f421415da60 0x7f4214160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4214099f90 0x7f421406bca0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 --2- 172.21.15.5:0/3495333888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f421409a790 0x7f4214160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.024+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 >> 172.21.15.5:0/3495333888 conn(0x7f42140b2f60 msgr2=0x7f4214070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.026+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 shutdown_connections 2024-05-08T23:23:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.026+0000 7f421b8ac700 1 -- 172.21.15.5:0/3495333888 wait complete. 2024-05-08T23:23:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-08T23:23:39.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:23:39.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.187+0000 7f2f5c4e2700 1 Processor -- start 2024-05-08T23:23:39.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.187+0000 7f2f5c4e2700 1 -- start start 2024-05-08T23:23:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.188+0000 7f2f5c4e2700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f5415db00 0x7f2f540cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.188+0000 7f2f5c4e2700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5406ad20 con 0x7f2f5406f990 2024-05-08T23:23:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.188+0000 7f2f5c4e2700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5406b000 con 0x7f2f540caa50 2024-05-08T23:23:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.188+0000 7f2f5c4e2700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5415dee0 con 0x7f2f5415db00 2024-05-08T23:23:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5a27e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f5415db00 0x7f2f540cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5aa7f700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2f5406f990 0x7f2f5406e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50736/0 (socket says 172.21.15.5:50736) 2024-05-08T23:23:39.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5aa7f700 1 -- 172.21.15.5:0/2901946231 learned_addr learned my addr 172.21.15.5:0/2901946231 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:39.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5a27e700 1 --2- 172.21.15.5:0/2901946231 >> v2:172.21.15.5:3300/0 conn(0x7f2f5415db00 0x7f2f540cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35590/0 (socket says 172.21.15.5:35590) 2024-05-08T23:23:39.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5a27e700 1 -- 172.21.15.5:0/2901946231 >> v1:172.21.15.49:6789/0 conn(0x7f2f540caa50 legacy=0x7f2f5406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5a27e700 1 -- 172.21.15.5:0/2901946231 >> v1:172.21.15.5:6789/0 conn(0x7f2f5406f990 legacy=0x7f2f5406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5a27e700 1 -- 172.21.15.5:0/2901946231 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5406b2e0 con 0x7f2f5415db00 2024-05-08T23:23:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5a27e700 1 --2- 172.21.15.5:0/2901946231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f5415db00 0x7f2f540cc590 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009840 tx=0x7f2f50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657d289961f5b792 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5927c700 1 -- 172.21.15.5:0/2901946231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50011830 con 0x7f2f5415db00 2024-05-08T23:23:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5927c700 1 -- 172.21.15.5:0/2901946231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f50011990 con 0x7f2f5415db00 2024-05-08T23:23:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.189+0000 7f2f5927c700 1 -- 172.21.15.5:0/2901946231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50011d20 con 0x7f2f5415db00 2024-05-08T23:23:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.190+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/2901946231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f5415db00 msgr2=0x7f2f540cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.190+0000 7f2f5c4e2700 1 --2- 172.21.15.5:0/2901946231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f5415db00 0x7f2f540cc590 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009840 tx=0x7f2f50009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.190+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/2901946231 shutdown_connections 2024-05-08T23:23:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.190+0000 7f2f5c4e2700 1 --2- 172.21.15.5:0/2901946231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f5415db00 0x7f2f540cc590 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.190+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/2901946231 >> 172.21.15.5:0/2901946231 conn(0x7f2f54099a50 msgr2=0x7f2f540d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.190+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/2901946231 shutdown_connections 2024-05-08T23:23:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.190+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/2901946231 wait complete. 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.191+0000 7f2f5c4e2700 1 Processor -- start 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.191+0000 7f2f5c4e2700 1 -- start start 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f5c4e2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f5406f990 0x7f2f540c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f5c4e2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540caa50 0x7f2f540b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f5c4e2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f5415db00 0x7f2f540c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f5c4e2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540b4c70 con 0x7f2f540caa50 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f5c4e2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540b5fb0 con 0x7f2f5415db00 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f5c4e2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540b4f10 con 0x7f2f5406f990 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f59a7d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540caa50 0x7f2f540b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f59a7d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540caa50 0x7f2f540b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35598/0 (socket says 172.21.15.5:35598) 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f59a7d700 1 -- 172.21.15.5:0/1145333535 learned_addr learned my addr 172.21.15.5:0/1145333535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f5aa7f700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f5415db00 0x7f2f540c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.192+0000 7f2f5a27e700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f5406f990 0x7f2f540c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f59a7d700 1 -- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f5406f990 msgr2=0x7f2f540c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f59a7d700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f5406f990 0x7f2f540c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f59a7d700 1 -- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f5415db00 msgr2=0x7f2f540c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f59a7d700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f5415db00 0x7f2f540c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f59a7d700 1 -- 172.21.15.5:0/1145333535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500094a0 con 0x7f2f540caa50 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f5a27e700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f5406f990 0x7f2f540c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f59a7d700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540caa50 0x7f2f540b4730 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c4e0 tx=0x7f2f4400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f4affd700 1 -- 172.21.15.5:0/1145333535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440052d0 con 0x7f2f540caa50 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f4affd700 1 -- 172.21.15.5:0/1145333535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44005430 con 0x7f2f540caa50 2024-05-08T23:23:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f540b51f0 con 0x7f2f540caa50 2024-05-08T23:23:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f4affd700 1 -- 172.21.15.5:0/1145333535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440038c0 con 0x7f2f540caa50 2024-05-08T23:23:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.193+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f540b57b0 con 0x7f2f540caa50 2024-05-08T23:23:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.194+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f54074bc0 con 0x7f2f540caa50 2024-05-08T23:23:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.195+0000 7f2f4affd700 1 -- 172.21.15.5:0/1145333535 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2f44003a20 con 0x7f2f540caa50 2024-05-08T23:23:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.195+0000 7f2f4affd700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3806ac90 0x7f2f3806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.195+0000 7f2f4affd700 1 -- 172.21.15.5:0/1145333535 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2f44088710 con 0x7f2f540caa50 2024-05-08T23:23:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.195+0000 7f2f5a27e700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3806ac90 0x7f2f3806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.196+0000 7f2f5a27e700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3806ac90 0x7f2f3806d150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2f500090b0 tx=0x7f2f5001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:39.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.200+0000 7f2f4affd700 1 -- 172.21.15.5:0/1145333535 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2f44053950 con 0x7f2f540caa50 2024-05-08T23:23:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.464+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2f540cca20 con 0x7f2f540caa50 2024-05-08T23:23:39.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.478+0000 7f2f4affd700 1 -- 172.21.15.5:0/1145333535 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2f44056f70 con 0x7f2f540caa50 2024-05-08T23:23:39.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.483+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3806ac90 msgr2=0x7f2f3806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.483+0000 7f2f5c4e2700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3806ac90 0x7f2f3806d150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2f500090b0 tx=0x7f2f5001c040 comp rx=0 tx=0).stop 2024-05-08T23:23:39.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.483+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540caa50 msgr2=0x7f2f540b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.483+0000 7f2f5c4e2700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540caa50 0x7f2f540b4730 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c4e0 tx=0x7f2f4400c7f0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.491+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 shutdown_connections 2024-05-08T23:23:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.491+0000 7f2f5c4e2700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2f3806ac90 0x7f2f3806d150 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.491+0000 7f2f5c4e2700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f5415db00 0x7f2f540c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.491+0000 7f2f5c4e2700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f5406f990 0x7f2f540c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.491+0000 7f2f5c4e2700 1 --2- 172.21.15.5:0/1145333535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540caa50 0x7f2f540b4730 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.491+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 >> 172.21.15.5:0/1145333535 conn(0x7f2f54099a50 msgr2=0x7f2f540cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.491+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 shutdown_connections 2024-05-08T23:23:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.491+0000 7f2f5c4e2700 1 -- 172.21.15.5:0/1145333535 wait complete. 2024-05-08T23:23:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 6 and data pool 7 2024-05-08T23:23:39.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:39.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.648+0000 7fd7e2de8700 1 Processor -- start 2024-05-08T23:23:39.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.648+0000 7fd7e2de8700 1 -- start start 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7e2de8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7dc15e200 0x7fd7dc15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7e2de8700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7dc0ce740 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7e2de8700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7dc15eb90 con 0x7fd7dc15a630 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7e2de8700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7dc0cb670 con 0x7fd7dc15e200 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7e1385700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7dc15e200 0x7fd7dc15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7dbfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd7dc0cdef0 0x7fd7dc0cbb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50744/0 (socket says 172.21.15.5:50744) 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7dbfff700 1 -- 172.21.15.5:0/2837262304 learned_addr learned my addr 172.21.15.5:0/2837262304 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7e1385700 1 -- 172.21.15.5:0/2837262304 >> v1:172.21.15.49:6789/0 conn(0x7fd7dc15a630 legacy=0x7fd7dc0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7e1385700 1 -- 172.21.15.5:0/2837262304 >> v1:172.21.15.5:6789/0 conn(0x7fd7dc0cdef0 legacy=0x7fd7dc0cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7e1385700 1 -- 172.21.15.5:0/2837262304 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc0cadc0 con 0x7fd7dc15e200 2024-05-08T23:23:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.649+0000 7fd7db7fe700 1 -- 172.21.15.5:0/2837262304 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022568525 0 0) 0x7fd7c4001110 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7e1385700 1 --2- 172.21.15.5:0/2837262304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc15e200 0x7fd7dc15e650 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009d90 tx=0x7fd7d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28e739062551a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7db7fe700 1 -- 172.21.15.5:0/2837262304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d400e070 con 0x7fd7dc15e200 2024-05-08T23:23:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7db7fe700 1 -- 172.21.15.5:0/2837262304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d400bbd0 con 0x7fd7dc15e200 2024-05-08T23:23:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7db7fe700 1 -- 172.21.15.5:0/2837262304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4012430 con 0x7fd7dc15e200 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7e2de8700 1 -- 172.21.15.5:0/2837262304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc15e200 msgr2=0x7fd7dc15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7e2de8700 1 --2- 172.21.15.5:0/2837262304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc15e200 0x7fd7dc15e650 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009d90 tx=0x7fd7d400b330 comp rx=0 tx=0).stop 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7e2de8700 1 -- 172.21.15.5:0/2837262304 shutdown_connections 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7e2de8700 1 --2- 172.21.15.5:0/2837262304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc15e200 0x7fd7dc15e650 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.650+0000 7fd7e2de8700 1 -- 172.21.15.5:0/2837262304 >> 172.21.15.5:0/2837262304 conn(0x7fd7dc099a50 msgr2=0x7fd7dc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.651+0000 7fd7e2de8700 1 -- 172.21.15.5:0/2837262304 shutdown_connections 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.651+0000 7fd7e2de8700 1 -- 172.21.15.5:0/2837262304 wait complete. 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.652+0000 7fd7e2de8700 1 Processor -- start 2024-05-08T23:23:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.652+0000 7fd7e2de8700 1 -- start start 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e2de8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc0cdef0 0x7fd7dc0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e2de8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7dc15a630 0x7fd7dc0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e2de8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc15e200 0x7fd7dc1f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e2de8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc14e2c0 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e2de8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0b5b30 con 0x7fd7dc15e200 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e2de8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0b5d80 con 0x7fd7dc15a630 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e0b84700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc0cdef0 0x7fd7dc0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e0b84700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc0cdef0 0x7fd7dc0b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35614/0 (socket says 172.21.15.5:35614) 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e0b84700 1 -- 172.21.15.5:0/164293363 learned_addr learned my addr 172.21.15.5:0/164293363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e1385700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc15e200 0x7fd7dc1f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7dbfff700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7dc15a630 0x7fd7dc0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e0b84700 1 -- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7dc15a630 msgr2=0x7fd7dc0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e0b84700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7dc15a630 0x7fd7dc0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e0b84700 1 -- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc15e200 msgr2=0x7fd7dc1f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e0b84700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc15e200 0x7fd7dc1f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e0b84700 1 -- 172.21.15.5:0/164293363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d4009a70 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.653+0000 7fd7e1385700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc15e200 0x7fd7dc1f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.654+0000 7fd7e0b84700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc0cdef0 0x7fd7dc0b55f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00c4e0 tx=0x7fd7cc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.654+0000 7fd7d97fa700 1 -- 172.21.15.5:0/164293363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0052d0 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.654+0000 7fd7d97fa700 1 -- 172.21.15.5:0/164293363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc005430 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.654+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc0b3a40 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.654+0000 7fd7d97fa700 1 -- 172.21.15.5:0/164293363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0038c0 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.654+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc14c880 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.655+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7dc068000 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.655+0000 7fd7d97fa700 1 -- 172.21.15.5:0/164293363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd7cc003a20 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.656+0000 7fd7d97fa700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7bc06ac90 0x7fd7bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.656+0000 7fd7d97fa700 1 -- 172.21.15.5:0/164293363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7cc088660 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.656+0000 7fd7dbfff700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7bc06ac90 0x7fd7bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.656+0000 7fd7dbfff700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7bc06ac90 0x7fd7bc06d150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4007fd0 tx=0x7fd7c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.660+0000 7fd7d97fa700 1 -- 172.21.15.5:0/164293363 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd7cc053840 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.882+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7dc15eb90 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.883+0000 7fd7d97fa700 1 -- 172.21.15.5:0/164293363 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15962 (secure 0 0 0) 0x7fd7dc15eb90 con 0x7fd7dc0cdef0 2024-05-08T23:23:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:39.884 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":45,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:39.466285+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:23:37.249225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T23:23:38.439923+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7bc06ac90 msgr2=0x7fd7bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7bc06ac90 0x7fd7bc06d150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4007fd0 tx=0x7fd7c4009040 comp rx=0 tx=0).stop 2024-05-08T23:23:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc0cdef0 msgr2=0x7fd7dc0b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc0cdef0 0x7fd7dc0b55f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00c4e0 tx=0x7fd7cc00c8a0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 shutdown_connections 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd7bc06ac90 0x7fd7bc06d150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7dc15e200 0x7fd7dc1f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7dc0cdef0 0x7fd7dc0b55f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 --2- 172.21.15.5:0/164293363 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd7dc15a630 0x7fd7dc0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 >> 172.21.15.5:0/164293363 conn(0x7fd7dc099a50 msgr2=0x7fd7dc06f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 shutdown_connections 2024-05-08T23:23:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:39.886+0000 7fd7e2de8700 1 -- 172.21.15.5:0/164293363 wait complete. 2024-05-08T23:23:39.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.037+0000 7f36518fd700 1 Processor -- start 2024-05-08T23:23:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.038+0000 7f36518fd700 1 -- start start 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.038+0000 7f36518fd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c099cc0 0x7f364c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.038+0000 7f36518fd700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c06c820 con 0x7f364c09a0a0 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.038+0000 7f36518fd700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c06bc10 con 0x7f364c0d0720 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.038+0000 7f36518fd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c06bef0 con 0x7f364c099cc0 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f364a7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f364c09a0a0 0x7f364c15d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50752/0 (socket says 172.21.15.5:50752) 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f364a7fc700 1 -- 172.21.15.5:0/2008209311 learned_addr learned my addr 172.21.15.5:0/2008209311 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f364affd700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c099cc0 0x7f364c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146495691 0 0) 0x7f364c06c820 con 0x7f364c09a0a0 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36340035b0 con 0x7f364c09a0a0 2024-05-08T23:23:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345768927 0 0) 0x7f364c06bc10 con 0x7f364c0d0720 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f364c06c820 con 0x7f364c0d0720 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3144434411 0 0) 0x7f36340035b0 con 0x7f364c09a0a0 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c099cc0 msgr2=0x7f364c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c099cc0 0x7f364c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 >> v1:172.21.15.49:6789/0 conn(0x7f364c0d0720 legacy=0x7f364c167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c06c1d0 con 0x7f364c09a0a0 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f3640001670 con 0x7f364c09a0a0 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 >> v1:172.21.15.5:6789/0 conn(0x7f364c09a0a0 legacy=0x7f364c15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 0x7f3634003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3634004850 0x7f3634004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c099cc0 0x7f3634005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c06bef0 con 0x7f3634004850 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363400bac0 con 0x7f3634003bc0 2024-05-08T23:23:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.039+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363400bc50 con 0x7f364c099cc0 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364affd700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 0x7f3634003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364a7fc700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3634004850 0x7f3634004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364b7fe700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c099cc0 0x7f3634005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364affd700 1 -- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c099cc0 msgr2=0x7f3634005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364affd700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c099cc0 0x7f3634005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364affd700 1 -- 172.21.15.5:0/2008209311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3634004850 msgr2=0x7f3634004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364affd700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3634004850 0x7f3634004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364affd700 1 -- 172.21.15.5:0/2008209311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340035b0 con 0x7f3634003bc0 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364a7fc700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3634004850 0x7f3634004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f364affd700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 0x7f3634003fc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f363c009b10 tx=0x7f3634005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dfba8da5227f540 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00e6c0 con 0x7f3634003bc0 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c00e820 con 0x7f3634003bc0 2024-05-08T23:23:40.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.040+0000 7f3649ffb700 1 -- 172.21.15.5:0/2008209311 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00ebd0 con 0x7f3634003bc0 2024-05-08T23:23:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.041+0000 7f36518fd700 1 -- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 msgr2=0x7f3634003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.041+0000 7f36518fd700 1 --2- 172.21.15.5:0/2008209311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 0x7f3634003fc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f363c009b10 tx=0x7f3634005880 comp rx=0 tx=0).stop 2024-05-08T23:23:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.041+0000 7f364b7fe700 1 -- 172.21.15.5:0/2008209311 reap_dead start 2024-05-08T23:23:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.041+0000 7f36518fd700 1 -- 172.21.15.5:0/2008209311 shutdown_connections 2024-05-08T23:23:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.041+0000 7f36518fd700 1 -- 172.21.15.5:0/2008209311 >> 172.21.15.5:0/2008209311 conn(0x7f364c0b2f80 msgr2=0x7f364c0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.041+0000 7f36518fd700 1 -- 172.21.15.5:0/2008209311 shutdown_connections 2024-05-08T23:23:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.041+0000 7f36518fd700 1 -- 172.21.15.5:0/2008209311 wait complete. 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.043+0000 7f36518fd700 1 Processor -- start 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.043+0000 7f36518fd700 1 -- start start 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f36518fd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 0x7f364c15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f36518fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0c60d0 0x7f364c1e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f36518fd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0c7630 0x7f364c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f36518fd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c15aef0 con 0x7f364c0c60d0 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f36518fd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0c6590 con 0x7f3634003bc0 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f36518fd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0c6870 con 0x7f364c0c7630 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0c60d0 0x7f364c1e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0c60d0 0x7f364c1e4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35636/0 (socket says 172.21.15.5:35636) 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364a7fc700 1 -- 172.21.15.5:0/2017751489 learned_addr learned my addr 172.21.15.5:0/2017751489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364affd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 0x7f364c15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364a7fc700 1 -- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0c7630 msgr2=0x7f364c0b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364a7fc700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0c7630 0x7f364c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364a7fc700 1 -- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 msgr2=0x7f364c15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364a7fc700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 0x7f364c15a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.044+0000 7f364a7fc700 1 -- 172.21.15.5:0/2017751489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c0c6b50 con 0x7f364c0c60d0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.045+0000 7f364a7fc700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0c60d0 0x7f364c1e4bd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3640007310 tx=0x7f36400076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.045+0000 7f3633fff700 1 -- 172.21.15.5:0/2017751489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000f7f0 con 0x7f364c0c60d0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.045+0000 7f3633fff700 1 -- 172.21.15.5:0/2017751489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364000f950 con 0x7f364c0c60d0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.045+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c0c6e30 con 0x7f364c0c60d0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.045+0000 7f3633fff700 1 -- 172.21.15.5:0/2017751489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000fcc0 con 0x7f364c0c60d0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.045+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c15a920 con 0x7f364c0c60d0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.046+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c0c7350 con 0x7f364c0c60d0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.046+0000 7f3633fff700 1 -- 172.21.15.5:0/2017751489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f364001d460 con 0x7f364c0c60d0 2024-05-08T23:23:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.047+0000 7f3633fff700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363806ac90 0x7f363806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.047+0000 7f3633fff700 1 -- 172.21.15.5:0/2017751489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36400893e0 con 0x7f364c0c60d0 2024-05-08T23:23:40.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.047+0000 7f364affd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363806ac90 0x7f363806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.048+0000 7f364affd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363806ac90 0x7f363806d150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f364c074020 tx=0x7f363c01e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:40.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.052+0000 7f3633fff700 1 -- 172.21.15.5:0/2017751489 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f364c0c7350 con 0x7f364c0c60d0 2024-05-08T23:23:40.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.274+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f364c0c7350 con 0x7f364c0c60d0 2024-05-08T23:23:40.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.275+0000 7f3633fff700 1 -- 172.21.15.5:0/2017751489 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15962 (secure 0 0 0) 0x7f364c0c7350 con 0x7f364c0c60d0 2024-05-08T23:23:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:40.276 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":45,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:39.466285+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:23:37.249225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T23:23:38.439923+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:40.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363806ac90 msgr2=0x7f363806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363806ac90 0x7f363806d150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f364c074020 tx=0x7f363c01e040 comp rx=0 tx=0).stop 2024-05-08T23:23:40.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0c60d0 msgr2=0x7f364c1e4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0c60d0 0x7f364c1e4bd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3640007310 tx=0x7f36400076d0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 shutdown_connections 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363806ac90 0x7f363806d150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0c7630 0x7f364c0b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3634003bc0 0x7f364c15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 --2- 172.21.15.5:0/2017751489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0c60d0 0x7f364c1e4bd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 >> 172.21.15.5:0/2017751489 conn(0x7f364c0b2f80 msgr2=0x7f364c15c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 shutdown_connections 2024-05-08T23:23:40.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.278+0000 7f36518fd700 1 -- 172.21.15.5:0/2017751489 wait complete. 2024-05-08T23:23:40.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:23:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.432+0000 7f52f8931700 1 Processor -- start 2024-05-08T23:23:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.432+0000 7f52f8931700 1 -- start start 2024-05-08T23:23:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.433+0000 7f52f8931700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f40b0d80 0x7f52f40b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.433+0000 7f52f8931700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f52f40b4d50 0x7f52f406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.433+0000 7f52f8931700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f4145040 con 0x7f52f406aea0 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.433+0000 7f52f8931700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f406b330 con 0x7f52f40b4d50 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.433+0000 7f52f8931700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40b38c0 con 0x7f52f40b0d80 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f2d9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f52f406aea0 0x7f52f40d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50754/0 (socket says 172.21.15.5:50754) 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f2d9d700 1 -- 172.21.15.5:0/1823179872 learned_addr learned my addr 172.21.15.5:0/1823179872 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f1d9b700 1 --2- 172.21.15.5:0/1823179872 >> v2:172.21.15.49:3300/0 conn(0x7f52f40b4d50 0x7f52f406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f259c700 1 --2- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f40b0d80 0x7f52f40b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f159a700 1 -- 172.21.15.5:0/1823179872 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640285778 0 0) 0x7f52f4145040 con 0x7f52f406aea0 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f159a700 1 -- 172.21.15.5:0/1823179872 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e00035b0 con 0x7f52f406aea0 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f1d9b700 1 -- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f40b0d80 msgr2=0x7f52f40b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f1d9b700 1 --2- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f40b0d80 0x7f52f40b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f1d9b700 1 -- 172.21.15.5:0/1823179872 >> v1:172.21.15.5:6789/0 conn(0x7f52f406aea0 legacy=0x7f52f40d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f1d9b700 1 -- 172.21.15.5:0/1823179872 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40b16e0 con 0x7f52f40b4d50 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f259c700 1 --2- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f40b0d80 0x7f52f40b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.434+0000 7f52f1d9b700 1 --2- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b4d50 0x7f52f406a960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009840 tx=0x7f52dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c3bbf573c66139 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f159a700 1 -- 172.21.15.5:0/1823179872 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc011830 con 0x7f52f40b4d50 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f159a700 1 -- 172.21.15.5:0/1823179872 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc011990 con 0x7f52f40b4d50 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f159a700 1 -- 172.21.15.5:0/1823179872 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc011d20 con 0x7f52f40b4d50 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f8931700 1 -- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b4d50 msgr2=0x7f52f406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f8931700 1 --2- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b4d50 0x7f52f406a960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009840 tx=0x7f52dc009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f8931700 1 -- 172.21.15.5:0/1823179872 shutdown_connections 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f8931700 1 --2- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f40b0d80 0x7f52f40b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f8931700 1 --2- 172.21.15.5:0/1823179872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b4d50 0x7f52f406a960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f8931700 1 -- 172.21.15.5:0/1823179872 >> 172.21.15.5:0/1823179872 conn(0x7f52f4099a50 msgr2=0x7f52f40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f8931700 1 -- 172.21.15.5:0/1823179872 shutdown_connections 2024-05-08T23:23:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.435+0000 7f52f8931700 1 -- 172.21.15.5:0/1823179872 wait complete. 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.437+0000 7f52f8931700 1 Processor -- start 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.437+0000 7f52f8931700 1 -- start start 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f8931700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f406aea0 0x7f52f409d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f8931700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b0d80 0x7f52f4132740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f8931700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f40b4d50 0x7f52f415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f8931700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f41f7500 con 0x7f52f40b4d50 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f8931700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40aed80 con 0x7f52f40b0d80 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f8931700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40ae450 con 0x7f52f406aea0 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f40b4d50 0x7f52f415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f40b4d50 0x7f52f415cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35646/0 (socket says 172.21.15.5:35646) 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 -- 172.21.15.5:0/719929875 learned_addr learned my addr 172.21.15.5:0/719929875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f1d9b700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b0d80 0x7f52f4132740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f259c700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f406aea0 0x7f52f409d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 -- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f406aea0 msgr2=0x7f52f409d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f406aea0 0x7f52f409d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 -- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b0d80 msgr2=0x7f52f4132740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b0d80 0x7f52f4132740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 -- 172.21.15.5:0/719929875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e40094a0 con 0x7f52f40b4d50 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.438+0000 7f52f2d9d700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f40b4d50 0x7f52f415cbd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00cb40 tx=0x7f52ec00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.439+0000 7f52eaffd700 1 -- 172.21.15.5:0/719929875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00a9e0 con 0x7f52f40b4d50 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.439+0000 7f52eaffd700 1 -- 172.21.15.5:0/719929875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec00ab40 con 0x7f52f40b4d50 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.439+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc0094a0 con 0x7f52f40b4d50 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.439+0000 7f52eaffd700 1 -- 172.21.15.5:0/719929875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec012400 con 0x7f52f40b4d50 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.439+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f40aea10 con 0x7f52f40b4d50 2024-05-08T23:23:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.440+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f40aeef0 con 0x7f52f40b4d50 2024-05-08T23:23:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.440+0000 7f52eaffd700 1 -- 172.21.15.5:0/719929875 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f52ec01b3b0 con 0x7f52f40b4d50 2024-05-08T23:23:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.441+0000 7f52eaffd700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f52d406ac90 0x7f52d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.441+0000 7f52eaffd700 1 -- 172.21.15.5:0/719929875 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52ec088f10 con 0x7f52f40b4d50 2024-05-08T23:23:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.441+0000 7f52f259c700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f52d406ac90 0x7f52d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.442+0000 7f52f259c700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f52d406ac90 0x7f52d406d150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009d70 tx=0x7f52e4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.446+0000 7f52eaffd700 1 -- 172.21.15.5:0/719929875 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f52ec054100 con 0x7f52f40b4d50 2024-05-08T23:23:40.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:40.718+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f52f4074bc0 con 0x7f52f40b4d50 2024-05-08T23:23:41.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.471+0000 7f52eaffd700 1 -- 172.21.15.5:0/719929875 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f52ec0170b0 con 0x7f52f40b4d50 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f52d406ac90 msgr2=0x7f52d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f52d406ac90 0x7f52d406d150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009d70 tx=0x7f52e4009210 comp rx=0 tx=0).stop 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f40b4d50 msgr2=0x7f52f415cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f40b4d50 0x7f52f415cbd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00cb40 tx=0x7f52ec00ce50 comp rx=0 tx=0).stop 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 shutdown_connections 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f52d406ac90 0x7f52d406d150 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f40b4d50 0x7f52f415cbd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f52f406aea0 0x7f52f409d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 --2- 172.21.15.5:0/719929875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52f40b0d80 0x7f52f4132740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 >> 172.21.15.5:0/719929875 conn(0x7f52f4099a50 msgr2=0x7f52f41606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:41.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.474+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 shutdown_connections 2024-05-08T23:23:41.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.475+0000 7f52f8931700 1 -- 172.21.15.5:0/719929875 wait complete. 2024-05-08T23:23:41.490 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:23:41.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:23:41.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.634+0000 7fd9eacc9700 1 Processor -- start 2024-05-08T23:23:41.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.634+0000 7fd9eacc9700 1 -- start start 2024-05-08T23:23:41.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9eacc9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40aea70 0x7fd9e40b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:41.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9eacc9700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e415add0 con 0x7fd9e40d0a60 2024-05-08T23:23:41.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9eacc9700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e415b070 con 0x7fd9e4154df0 2024-05-08T23:23:41.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9eacc9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e415abe0 con 0x7fd9e40aea70 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e9266700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd9e40d0a60 0x7fd9e40b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50766/0 (socket says 172.21.15.5:50766) 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e9266700 1 -- 172.21.15.5:0/2870467447 learned_addr learned my addr 172.21.15.5:0/2870467447 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e3fff700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40aea70 0x7fd9e40b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948361127 0 0) 0x7fd9e415add0 con 0x7fd9e40d0a60 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9cc0035b0 con 0x7fd9e40d0a60 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 299429747 0 0) 0x7fd9cc0035b0 con 0x7fd9e40d0a60 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40aea70 msgr2=0x7fd9e40b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40aea70 0x7fd9e40b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 >> v1:172.21.15.49:6789/0 conn(0x7fd9e4154df0 legacy=0x7fd9e40d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e415b350 con 0x7fd9e40d0a60 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fd9dc0019f0 con 0x7fd9e40d0a60 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 >> v1:172.21.15.5:6789/0 conn(0x7fd9e40d0a60 legacy=0x7fd9e40b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.636+0000 7fd9e37fe700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e37fe700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9cc0043e0 0x7fd9cc0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e37fe700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40aea70 0x7fd9cc005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc00ba10 con 0x7fd9cc003a30 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc00bb80 con 0x7fd9cc0043e0 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc00bd40 con 0x7fd9e40aea70 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e8a65700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e3fff700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9cc0043e0 0x7fd9cc0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e9266700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40aea70 0x7fd9cc005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e8a65700 1 -- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40aea70 msgr2=0x7fd9cc005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e8a65700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40aea70 0x7fd9cc005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e8a65700 1 -- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9cc0043e0 msgr2=0x7fd9cc0048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e8a65700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9cc0043e0 0x7fd9cc0048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e8a65700 1 -- 172.21.15.5:0/2870467447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0035b0 con 0x7fd9cc003a30 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e8a65700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9cc003e10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40018d0 tx=0x7fd9cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5631701936a8d2e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400c690 con 0x7fd9cc003a30 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d400c810 con 0x7fd9cc003a30 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.637+0000 7fd9e37fe700 1 -- 172.21.15.5:0/2870467447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400cbc0 con 0x7fd9cc003a30 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.638+0000 7fd9eacc9700 1 -- 172.21.15.5:0/2870467447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 msgr2=0x7fd9cc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.638+0000 7fd9eacc9700 1 --2- 172.21.15.5:0/2870467447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9cc003e10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40018d0 tx=0x7fd9cc005880 comp rx=0 tx=0).stop 2024-05-08T23:23:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.638+0000 7fd9e9266700 1 -- 172.21.15.5:0/2870467447 reap_dead start 2024-05-08T23:23:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.638+0000 7fd9eacc9700 1 -- 172.21.15.5:0/2870467447 shutdown_connections 2024-05-08T23:23:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.638+0000 7fd9eacc9700 1 -- 172.21.15.5:0/2870467447 >> 172.21.15.5:0/2870467447 conn(0x7fd9e4099a50 msgr2=0x7fd9e40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.638+0000 7fd9eacc9700 1 -- 172.21.15.5:0/2870467447 shutdown_connections 2024-05-08T23:23:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.638+0000 7fd9eacc9700 1 -- 172.21.15.5:0/2870467447 wait complete. 2024-05-08T23:23:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.639+0000 7fd9eacc9700 1 Processor -- start 2024-05-08T23:23:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.639+0000 7fd9eacc9700 1 -- start start 2024-05-08T23:23:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9eacc9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9e414eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9eacc9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40b2a40 0x7fd9e4151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9eacc9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e414ce50 0x7fd9e40cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9eacc9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e414f640 con 0x7fd9cc003a30 2024-05-08T23:23:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9eacc9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e414c0f0 con 0x7fd9e414ce50 2024-05-08T23:23:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9eacc9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e414f850 con 0x7fd9e40b2a40 2024-05-08T23:23:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9e8a65700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9e414eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9e8a65700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9e414eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35664/0 (socket says 172.21.15.5:35664) 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.640+0000 7fd9e8a65700 1 -- 172.21.15.5:0/3976178097 learned_addr learned my addr 172.21.15.5:0/3976178097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e3fff700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40b2a40 0x7fd9e4151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e9266700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e414ce50 0x7fd9e40cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e8a65700 1 -- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40b2a40 msgr2=0x7fd9e4151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e8a65700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40b2a40 0x7fd9e4151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e8a65700 1 -- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e414ce50 msgr2=0x7fd9e40cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e8a65700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e414ce50 0x7fd9e40cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e8a65700 1 -- 172.21.15.5:0/3976178097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e414fb30 con 0x7fd9cc003a30 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e8a65700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9e414eec0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00bb50 tx=0x7fd9d4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e17fa700 1 -- 172.21.15.5:0/3976178097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400cb80 con 0x7fd9cc003a30 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e17fa700 1 -- 172.21.15.5:0/3976178097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4021070 con 0x7fd9cc003a30 2024-05-08T23:23:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e415c830 con 0x7fd9cc003a30 2024-05-08T23:23:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9e17fa700 1 -- 172.21.15.5:0/3976178097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400cee0 con 0x7fd9cc003a30 2024-05-08T23:23:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.641+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e414d4b0 con 0x7fd9cc003a30 2024-05-08T23:23:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.642+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e415cba0 con 0x7fd9cc003a30 2024-05-08T23:23:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.643+0000 7fd9e17fa700 1 -- 172.21.15.5:0/3976178097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd9d4023030 con 0x7fd9cc003a30 2024-05-08T23:23:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.643+0000 7fd9e17fa700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9d006ac20 0x7fd9d006d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.644+0000 7fd9e17fa700 1 -- 172.21.15.5:0/3976178097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9d408a490 con 0x7fd9cc003a30 2024-05-08T23:23:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.644+0000 7fd9e3fff700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9d006ac20 0x7fd9d006d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.644+0000 7fd9e3fff700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9d006ac20 0x7fd9d006d0e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4154720 tx=0x7fd9d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.648+0000 7fd9e17fa700 1 -- 172.21.15.5:0/3976178097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd9d4055660 con 0x7fd9cc003a30 2024-05-08T23:23:41.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:41.915+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd9e414fd70 con 0x7fd9cc003a30 2024-05-08T23:23:42.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.473+0000 7fd9e17fa700 1 -- 172.21.15.5:0/3976178097 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd9e414fd70 con 0x7fd9cc003a30 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9d006ac20 msgr2=0x7fd9d006d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9d006ac20 0x7fd9d006d0e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4154720 tx=0x7fd9d8008040 comp rx=0 tx=0).stop 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 msgr2=0x7fd9e414eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9e414eec0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00bb50 tx=0x7fd9d4001230 comp rx=0 tx=0).stop 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 shutdown_connections 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9d006ac20 0x7fd9d006d0e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9e414ce50 0x7fd9e40cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9cc003a30 0x7fd9e414eec0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 --2- 172.21.15.5:0/3976178097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd9e40b2a40 0x7fd9e4151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.476+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 >> 172.21.15.5:0/3976178097 conn(0x7fd9e4099a50 msgr2=0x7fd9e406aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.477+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 shutdown_connections 2024-05-08T23:23:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.477+0000 7fd9eacc9700 1 -- 172.21.15.5:0/3976178097 wait complete. 2024-05-08T23:23:42.492 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:23:42.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.633+0000 7fea9a057700 1 Processor -- start 2024-05-08T23:23:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.633+0000 7fea9a057700 1 -- start start 2024-05-08T23:23:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.634+0000 7fea9a057700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fea9415d7a0 0x7fea94161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.634+0000 7fea9a057700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea94162450 con 0x7fea940d3f40 2024-05-08T23:23:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.634+0000 7fea9a057700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea940cee00 con 0x7fea9415db80 2024-05-08T23:23:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.634+0000 7fea9a057700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea940b27e0 con 0x7fea9415d7a0 2024-05-08T23:23:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.634+0000 7fea937fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fea9415d7a0 0x7fea94161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.634+0000 7fea92ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fea940d3f40 0x7fea940b0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50772/0 (socket says 172.21.15.5:50772) 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.634+0000 7fea92ffd700 1 -- 172.21.15.5:0/3128299672 learned_addr learned my addr 172.21.15.5:0/3128299672 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.635+0000 7fea937fe700 1 -- 172.21.15.5:0/3128299672 >> v1:172.21.15.49:6789/0 conn(0x7fea9415db80 legacy=0x7fea940cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.635+0000 7fea937fe700 1 -- 172.21.15.5:0/3128299672 >> v1:172.21.15.5:6789/0 conn(0x7fea940d3f40 legacy=0x7fea940b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.635+0000 7fea927fc700 1 -- 172.21.15.5:0/3128299672 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943707211 0 0) 0x7fea88001290 con 0x7fea940d3f40 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.635+0000 7fea937fe700 1 -- 172.21.15.5:0/3128299672 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea940d4400 con 0x7fea9415d7a0 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.635+0000 7fea937fe700 1 --2- 172.21.15.5:0/3128299672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415d7a0 0x7fea94161f10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fea84009840 tx=0x7fea84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c8e16541692566 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.635+0000 7fea927fc700 1 -- 172.21.15.5:0/3128299672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84011830 con 0x7fea9415d7a0 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.635+0000 7fea927fc700 1 -- 172.21.15.5:0/3128299672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea84011990 con 0x7fea9415d7a0 2024-05-08T23:23:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.635+0000 7fea927fc700 1 -- 172.21.15.5:0/3128299672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84011d20 con 0x7fea9415d7a0 2024-05-08T23:23:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.636+0000 7fea9a057700 1 -- 172.21.15.5:0/3128299672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415d7a0 msgr2=0x7fea94161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.636+0000 7fea9a057700 1 --2- 172.21.15.5:0/3128299672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415d7a0 0x7fea94161f10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fea84009840 tx=0x7fea84009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.636+0000 7fea9a057700 1 -- 172.21.15.5:0/3128299672 shutdown_connections 2024-05-08T23:23:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.636+0000 7fea9a057700 1 --2- 172.21.15.5:0/3128299672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415d7a0 0x7fea94161f10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.636+0000 7fea9a057700 1 -- 172.21.15.5:0/3128299672 >> 172.21.15.5:0/3128299672 conn(0x7fea9408afe0 msgr2=0x7fea940d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.636+0000 7fea9a057700 1 -- 172.21.15.5:0/3128299672 shutdown_connections 2024-05-08T23:23:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.636+0000 7fea9a057700 1 -- 172.21.15.5:0/3128299672 wait complete. 2024-05-08T23:23:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.637+0000 7fea9a057700 1 Processor -- start 2024-05-08T23:23:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.637+0000 7fea9a057700 1 -- start start 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea9a057700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fea940d3f40 0x7fea940c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea9a057700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea9415d7a0 0x7fea940ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea9a057700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415db80 0x7fea940cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea9a057700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9415a990 con 0x7fea9415db80 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea9a057700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940cd310 con 0x7fea9415d7a0 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea9a057700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea940cd5f0 con 0x7fea940d3f40 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea93fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415db80 0x7fea940cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea93fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415db80 0x7fea940cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35684/0 (socket says 172.21.15.5:35684) 2024-05-08T23:23:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea93fff700 1 -- 172.21.15.5:0/2732712076 learned_addr learned my addr 172.21.15.5:0/2732712076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea92ffd700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea9415d7a0 0x7fea940ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.638+0000 7fea937fe700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fea940d3f40 0x7fea940c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea93fff700 1 -- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fea940d3f40 msgr2=0x7fea940c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea93fff700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fea940d3f40 0x7fea940c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea93fff700 1 -- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea9415d7a0 msgr2=0x7fea940ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea93fff700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea9415d7a0 0x7fea940ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea93fff700 1 -- 172.21.15.5:0/2732712076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840094a0 con 0x7fea9415db80 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea93fff700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415db80 0x7fea940cac30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00cb50 tx=0x7fea8c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea7bfff700 1 -- 172.21.15.5:0/2732712076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c003c00 con 0x7fea9415db80 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea7bfff700 1 -- 172.21.15.5:0/2732712076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c003d60 con 0x7fea9415db80 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea940cd8d0 con 0x7fea9415db80 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea7bfff700 1 -- 172.21.15.5:0/2732712076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00ab70 con 0x7fea9415db80 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.639+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea940c6700 con 0x7fea9415db80 2024-05-08T23:23:42.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.640+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea940cdb10 con 0x7fea9415db80 2024-05-08T23:23:42.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.641+0000 7fea7bfff700 1 -- 172.21.15.5:0/2732712076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fea8c00acd0 con 0x7fea9415db80 2024-05-08T23:23:42.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.641+0000 7fea7bfff700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fea8006ac90 0x7fea8006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:42.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.641+0000 7fea7bfff700 1 -- 172.21.15.5:0/2732712076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea8c089380 con 0x7fea9415db80 2024-05-08T23:23:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.642+0000 7fea937fe700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fea8006ac90 0x7fea8006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.642+0000 7fea937fe700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fea8006ac90 0x7fea8006d150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fea84000c00 tx=0x7fea8401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:42.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.646+0000 7fea7bfff700 1 -- 172.21.15.5:0/2732712076 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fea940cdb10 con 0x7fea9415db80 2024-05-08T23:23:42.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.911+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea9415f030 con 0x7fea9415db80 2024-05-08T23:23:42.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.911+0000 7fea7bfff700 1 -- 172.21.15.5:0/2732712076 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4722 (secure 0 0 0) 0x7fea9415f030 con 0x7fea9415db80 2024-05-08T23:23:42.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:42.912 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:23:42.470597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:23:42.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fea8006ac90 msgr2=0x7fea8006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fea8006ac90 0x7fea8006d150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fea84000c00 tx=0x7fea8401c040 comp rx=0 tx=0).stop 2024-05-08T23:23:42.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415db80 msgr2=0x7fea940cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415db80 0x7fea940cac30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00cb50 tx=0x7fea8c00cf10 comp rx=0 tx=0).stop 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 shutdown_connections 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fea8006ac90 0x7fea8006d150 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9415db80 0x7fea940cac30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fea940d3f40 0x7fea940c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 --2- 172.21.15.5:0/2732712076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea9415d7a0 0x7fea940ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 >> 172.21.15.5:0/2732712076 conn(0x7fea9408afe0 msgr2=0x7fea940b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 shutdown_connections 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:42.914+0000 7fea9a057700 1 -- 172.21.15.5:0/2732712076 wait complete. 2024-05-08T23:23:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-05-08T23:23:42.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:23:43.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.065+0000 7ff160396700 1 Processor -- start 2024-05-08T23:23:43.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.065+0000 7ff160396700 1 -- start start 2024-05-08T23:23:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.066+0000 7ff160396700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff15806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.066+0000 7ff160396700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15806d5a0 con 0x7ff15806b970 2024-05-08T23:23:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.066+0000 7ff160396700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15806f3f0 con 0x7ff158161dc0 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.066+0000 7ff160396700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15806f6d0 con 0x7ff15815d800 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.066+0000 7ff15e933700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff15806b970 0x7ff15806dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50784/0 (socket says 172.21.15.5:50784) 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.066+0000 7ff15e933700 1 -- 172.21.15.5:0/1809948014 learned_addr learned my addr 172.21.15.5:0/1809948014 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15e132700 1 --2- 172.21.15.5:0/1809948014 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff15806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15d130700 1 -- 172.21.15.5:0/1809948014 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378713447 0 0) 0x7ff15806d5a0 con 0x7ff15806b970 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15d130700 1 -- 172.21.15.5:0/1809948014 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1440035b0 con 0x7ff15806b970 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15d130700 1 -- 172.21.15.5:0/1809948014 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308943994 0 0) 0x7ff15806f3f0 con 0x7ff158161dc0 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15d130700 1 -- 172.21.15.5:0/1809948014 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff15806d5a0 con 0x7ff158161dc0 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15d130700 1 -- 172.21.15.5:0/1809948014 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2109048060 0 0) 0x7ff1440035b0 con 0x7ff15806b970 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15e132700 1 -- 172.21.15.5:0/1809948014 >> v1:172.21.15.49:6789/0 conn(0x7ff158161dc0 legacy=0x7ff1580ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15e132700 1 -- 172.21.15.5:0/1809948014 >> v1:172.21.15.5:6789/0 conn(0x7ff15806b970 legacy=0x7ff15806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15e132700 1 -- 172.21.15.5:0/1809948014 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15806d810 con 0x7ff15815d800 2024-05-08T23:23:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15e132700 1 --2- 172.21.15.5:0/1809948014 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff15806ed80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009870 tx=0x7ff14c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bcdf95c5026838a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.067+0000 7ff15d130700 1 -- 172.21.15.5:0/1809948014 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1500019f0 con 0x7ff15815d800 2024-05-08T23:23:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff15d130700 1 -- 172.21.15.5:0/1809948014 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c004030 con 0x7ff15815d800 2024-05-08T23:23:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff15d130700 1 -- 172.21.15.5:0/1809948014 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0043c0 con 0x7ff15815d800 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff160396700 1 -- 172.21.15.5:0/1809948014 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 msgr2=0x7ff15806ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff160396700 1 --2- 172.21.15.5:0/1809948014 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff15806ed80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009870 tx=0x7ff14c009b80 comp rx=0 tx=0).stop 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff160396700 1 -- 172.21.15.5:0/1809948014 shutdown_connections 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff160396700 1 --2- 172.21.15.5:0/1809948014 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff15806ed80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff160396700 1 -- 172.21.15.5:0/1809948014 >> 172.21.15.5:0/1809948014 conn(0x7ff1580713c0 msgr2=0x7ff1580717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff160396700 1 -- 172.21.15.5:0/1809948014 shutdown_connections 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.068+0000 7ff160396700 1 -- 172.21.15.5:0/1809948014 wait complete. 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff160396700 1 Processor -- start 2024-05-08T23:23:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff160396700 1 -- start start 2024-05-08T23:23:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff160396700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff15806b970 0x7ff1580b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff160396700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff1580c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff160396700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff158161dc0 0x7ff1580c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff160396700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580c5b30 con 0x7ff15806b970 2024-05-08T23:23:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff160396700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580ca000 con 0x7ff158161dc0 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff160396700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580b4ea0 con 0x7ff15815d800 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.070+0000 7ff15e132700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff15806b970 0x7ff1580b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e132700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff15806b970 0x7ff1580b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35700/0 (socket says 172.21.15.5:35700) 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e132700 1 -- 172.21.15.5:0/3262296502 learned_addr learned my addr 172.21.15.5:0/3262296502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e933700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff158161dc0 0x7ff1580c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15d931700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff1580c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e132700 1 -- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 msgr2=0x7ff1580c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e132700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff1580c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e132700 1 -- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff158161dc0 msgr2=0x7ff1580c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e132700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff158161dc0 0x7ff1580c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e132700 1 -- 172.21.15.5:0/3262296502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c0094a0 con 0x7ff15806b970 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15d931700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff1580c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e933700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff158161dc0 0x7ff1580c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff15e132700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff15806b970 0x7ff1580b48c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff14c000c00 tx=0x7ff14c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff14affd700 1 -- 172.21.15.5:0/3262296502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c011630 con 0x7ff15806b970 2024-05-08T23:23:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff14affd700 1 -- 172.21.15.5:0/3262296502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c01f070 con 0x7ff15806b970 2024-05-08T23:23:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1580b5180 con 0x7ff15806b970 2024-05-08T23:23:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff14affd700 1 -- 172.21.15.5:0/3262296502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0119f0 con 0x7ff15806b970 2024-05-08T23:23:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.071+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1580b5740 con 0x7ff15806b970 2024-05-08T23:23:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.072+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1580b58d0 con 0x7ff15806b970 2024-05-08T23:23:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.073+0000 7ff14affd700 1 -- 172.21.15.5:0/3262296502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff14c011b50 con 0x7ff15806b970 2024-05-08T23:23:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.073+0000 7ff14affd700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff13c06ac90 0x7ff13c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.073+0000 7ff14affd700 1 -- 172.21.15.5:0/3262296502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff14c021070 con 0x7ff15806b970 2024-05-08T23:23:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.074+0000 7ff15d931700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff13c06ac90 0x7ff13c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.074+0000 7ff15d931700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff13c06ac90 0x7ff13c06d150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff1581f6ac0 tx=0x7ff154009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.078+0000 7ff14affd700 1 -- 172.21.15.5:0/3262296502 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff14c055f40 con 0x7ff15806b970 2024-05-08T23:23:43.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.296+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1580ca170 con 0x7ff15806b970 2024-05-08T23:23:43.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.297+0000 7ff14affd700 1 -- 172.21.15.5:0/3262296502 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15949 (secure 0 0 0) 0x7ff1580ca170 con 0x7ff15806b970 2024-05-08T23:23:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:43.297 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":46,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:40.468034+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:23:37.249225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T23:23:38.439923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:23:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff13c06ac90 msgr2=0x7ff13c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff13c06ac90 0x7ff13c06d150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff1581f6ac0 tx=0x7ff154009040 comp rx=0 tx=0).stop 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff15806b970 msgr2=0x7ff1580b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff15806b970 0x7ff1580b48c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff14c000c00 tx=0x7ff14c004750 comp rx=0 tx=0).stop 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 shutdown_connections 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff13c06ac90 0x7ff13c06d150 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff158161dc0 0x7ff1580c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff15806b970 0x7ff1580b48c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 --2- 172.21.15.5:0/3262296502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff15815d800 0x7ff1580c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 >> 172.21.15.5:0/3262296502 conn(0x7ff1580713c0 msgr2=0x7ff15815fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 shutdown_connections 2024-05-08T23:23:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.299+0000 7ff160396700 1 -- 172.21.15.5:0/3262296502 wait complete. 2024-05-08T23:23:43.314 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:23:43.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:23:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.452+0000 7f743eca0700 1 Processor -- start 2024-05-08T23:23:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.452+0000 7f743eca0700 1 -- start start 2024-05-08T23:23:43.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.453+0000 7f743eca0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7438161030 0x7f7438099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.453+0000 7f743eca0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74380cbbc0 0x7f74380cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.453+0000 7f743eca0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74380d0840 con 0x7f74380b2480 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.453+0000 7f743eca0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743806ccc0 con 0x7f74380cbbc0 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.453+0000 7f743eca0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380d0660 con 0x7f7438161030 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743ca3c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7438161030 0x7f7438099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743d23d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74380cbbc0 0x7f74380cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743ca3c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7438161030 0x7f7438099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:47912/0 (socket says 172.21.15.5:47912) 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743ca3c700 1 -- 172.21.15.5:0/3637017316 learned_addr learned my addr 172.21.15.5:0/3637017316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f7437fff700 1 --1- 172.21.15.5:0/3637017316 >> v1:172.21.15.49:6789/0 conn(0x7f74380b2480 0x7f7438099f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:52226/0 (socket says 172.21.15.5:52226) 2024-05-08T23:23:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743ca3c700 1 -- 172.21.15.5:0/3637017316 >> v2:172.21.15.49:3300/0 conn(0x7f74380cbbc0 msgr2=0x7f74380cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743ca3c700 1 --2- 172.21.15.5:0/3637017316 >> v2:172.21.15.49:3300/0 conn(0x7f74380cbbc0 0x7f74380cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743ca3c700 1 -- 172.21.15.5:0/3637017316 >> v1:172.21.15.49:6789/0 conn(0x7f74380b2480 legacy=0x7f7438099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743ca3c700 1 -- 172.21.15.5:0/3637017316 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743806c600 con 0x7f7438161030 2024-05-08T23:23:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f74377fe700 1 -- 172.21.15.5:0/3637017316 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283397413 0 0) 0x7f74380d0840 con 0x7f74380b2480 2024-05-08T23:23:43.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743d23d700 1 --2- 172.21.15.5:0/3637017316 >> v2:172.21.15.49:3300/0 conn(0x7f74380cbbc0 0x7f74380cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:43.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f743ca3c700 1 --2- 172.21.15.5:0/3637017316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7438161030 0x7f7438099a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74280098f0 tx=0x7f7428009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e0f73fcf8d93ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.454+0000 7f74377fe700 1 -- 172.21.15.5:0/3637017316 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428011830 con 0x7f7438161030 2024-05-08T23:23:43.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f74377fe700 1 -- 172.21.15.5:0/3637017316 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428011990 con 0x7f7438161030 2024-05-08T23:23:43.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f74377fe700 1 -- 172.21.15.5:0/3637017316 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428011d20 con 0x7f7438161030 2024-05-08T23:23:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f743eca0700 1 -- 172.21.15.5:0/3637017316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7438161030 msgr2=0x7f7438099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f743eca0700 1 --2- 172.21.15.5:0/3637017316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7438161030 0x7f7438099a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74280098f0 tx=0x7f7428009c00 comp rx=0 tx=0).stop 2024-05-08T23:23:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f743eca0700 1 -- 172.21.15.5:0/3637017316 shutdown_connections 2024-05-08T23:23:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f743eca0700 1 --2- 172.21.15.5:0/3637017316 >> v2:172.21.15.49:3300/0 conn(0x7f74380cbbc0 0x7f74380cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f743eca0700 1 --2- 172.21.15.5:0/3637017316 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7438161030 0x7f7438099a50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f743eca0700 1 -- 172.21.15.5:0/3637017316 >> 172.21.15.5:0/3637017316 conn(0x7f74380b2f20 msgr2=0x7f74380b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f743eca0700 1 -- 172.21.15.5:0/3637017316 shutdown_connections 2024-05-08T23:23:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.455+0000 7f743eca0700 1 -- 172.21.15.5:0/3637017316 wait complete. 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.457+0000 7f743eca0700 1 Processor -- start 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.457+0000 7f743eca0700 1 -- start start 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743eca0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74380b2480 0x7f743814caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743eca0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74380cbbc0 0x7f74380c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743eca0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7438161030 0x7f74380c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743eca0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743814d500 con 0x7f74380b2480 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743eca0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743814d030 con 0x7f7438161030 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743eca0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743815c6a0 con 0x7f74380cbbc0 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743ca3c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74380b2480 0x7f743814caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743ca3c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74380b2480 0x7f743814caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35704/0 (socket says 172.21.15.5:35704) 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743ca3c700 1 -- 172.21.15.5:0/1279594726 learned_addr learned my addr 172.21.15.5:0/1279594726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f743d23d700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7438161030 0x7f74380c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.458+0000 7f7437fff700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74380cbbc0 0x7f74380c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743ca3c700 1 -- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74380cbbc0 msgr2=0x7f74380c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743ca3c700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74380cbbc0 0x7f74380c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743ca3c700 1 -- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7438161030 msgr2=0x7f74380c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743ca3c700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7438161030 0x7f74380c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743ca3c700 1 -- 172.21.15.5:0/1279594726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430009a70 con 0x7f74380b2480 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743d23d700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7438161030 0x7f74380c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743ca3c700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74380b2480 0x7f743814caf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7438072340 tx=0x7f7428004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f74357fa700 1 -- 172.21.15.5:0/1279594726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428004480 con 0x7f74380b2480 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f74357fa700 1 -- 172.21.15.5:0/1279594726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742801f070 con 0x7f74380b2480 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7428009550 con 0x7f74380b2480 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f74357fa700 1 -- 172.21.15.5:0/1279594726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280038a0 con 0x7f74380b2480 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.459+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74380c7780 con 0x7f74380b2480 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.460+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743806d8f0 con 0x7f74380b2480 2024-05-08T23:23:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.461+0000 7f74357fa700 1 -- 172.21.15.5:0/1279594726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7428021030 con 0x7f74380b2480 2024-05-08T23:23:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.461+0000 7f74357fa700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f742406ac90 0x7f742406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.461+0000 7f74357fa700 1 -- 172.21.15.5:0/1279594726 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f742808a7d0 con 0x7f74380b2480 2024-05-08T23:23:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.461+0000 7f7437fff700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f742406ac90 0x7f742406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.462+0000 7f7437fff700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f742406ac90 0x7f742406d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f74381f6c90 tx=0x7f742c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.466+0000 7f74357fa700 1 -- 172.21.15.5:0/1279594726 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7428055ae0 con 0x7f74380b2480 2024-05-08T23:23:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.675+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f743809ac90 con 0x7f742406ac90 2024-05-08T23:23:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.684+0000 7f74357fa700 1 -- 172.21.15.5:0/1279594726 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289559 (secure 0 0 0) 0x7f742c0070d0 con 0x7f742406ac90 2024-05-08T23:23:43.691 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f742406ac90 msgr2=0x7f742406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f742406ac90 0x7f742406d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f74381f6c90 tx=0x7f742c009040 comp rx=0 tx=0).stop 2024-05-08T23:23:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74380b2480 msgr2=0x7f743814caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74380b2480 0x7f743814caf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7438072340 tx=0x7f7428004750 comp rx=0 tx=0).stop 2024-05-08T23:23:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 shutdown_connections 2024-05-08T23:23:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f742406ac90 0x7f742406d150 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7438161030 0x7f74380c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74380b2480 0x7f743814caf0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 --2- 172.21.15.5:0/1279594726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74380cbbc0 0x7f74380c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 >> 172.21.15.5:0/1279594726 conn(0x7f74380b2f20 msgr2=0x7f743806ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 shutdown_connections 2024-05-08T23:23:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.688+0000 7f743eca0700 1 -- 172.21.15.5:0/1279594726 wait complete. 2024-05-08T23:23:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:23:43.702 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-05-08T23:23:43.332322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221608,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753112,"statfs":{"total":773094113280,"available":772867186688,"internally_reserved":0,"allocated":5578752,"data_stored":2962164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.326161"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147742+0000","last_change":"2024-05-08T23:23:39.490895+0000","last_active":"2024-05-08T23:23:41.147742+0000","last_peered":"2024-05-08T23:23:41.147742+0000","last_clean":"2024-05-08T23:23:41.147742+0000","last_became_active":"2024-05-08T23:23:39.490566+0000","last_became_peered":"2024-05-08T23:23:39.490566+0000","last_unstale":"2024-05-08T23:23:41.147742+0000","last_undegraded":"2024-05-08T23:23:41.147742+0000","last_fullsized":"2024-05-08T23:23:41.147742+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:30.727813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473093+0000","last_change":"2024-05-08T23:23:39.492452+0000","last_active":"2024-05-08T23:23:40.473093+0000","last_peered":"2024-05-08T23:23:40.473093+0000","last_clean":"2024-05-08T23:23:40.473093+0000","last_became_active":"2024-05-08T23:23:39.492113+0000","last_became_peered":"2024-05-08T23:23:39.492113+0000","last_unstale":"2024-05-08T23:23:40.473093+0000","last_undegraded":"2024-05-08T23:23:40.473093+0000","last_fullsized":"2024-05-08T23:23:40.473093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:54.440871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472301+0000","last_change":"2024-05-08T23:23:39.493541+0000","last_active":"2024-05-08T23:23:40.472301+0000","last_peered":"2024-05-08T23:23:40.472301+0000","last_clean":"2024-05-08T23:23:40.472301+0000","last_became_active":"2024-05-08T23:23:39.493248+0000","last_became_peered":"2024-05-08T23:23:39.493248+0000","last_unstale":"2024-05-08T23:23:40.472301+0000","last_undegraded":"2024-05-08T23:23:40.472301+0000","last_fullsized":"2024-05-08T23:23:40.472301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:10.500447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009609+0000","last_change":"2024-05-08T23:23:39.491285+0000","last_active":"2024-05-08T23:23:41.009609+0000","last_peered":"2024-05-08T23:23:41.009609+0000","last_clean":"2024-05-08T23:23:41.009609+0000","last_became_active":"2024-05-08T23:23:39.491004+0000","last_became_peered":"2024-05-08T23:23:39.491004+0000","last_unstale":"2024-05-08T23:23:41.009609+0000","last_undegraded":"2024-05-08T23:23:41.009609+0000","last_fullsized":"2024-05-08T23:23:41.009609+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:37.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472980+0000","last_change":"2024-05-08T23:23:39.492169+0000","last_active":"2024-05-08T23:23:40.472980+0000","last_peered":"2024-05-08T23:23:40.472980+0000","last_clean":"2024-05-08T23:23:40.472980+0000","last_became_active":"2024-05-08T23:23:39.491823+0000","last_became_peered":"2024-05-08T23:23:39.491823+0000","last_unstale":"2024-05-08T23:23:40.472980+0000","last_undegraded":"2024-05-08T23:23:40.472980+0000","last_fullsized":"2024-05-08T23:23:40.472980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:29.828185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471433+0000","last_change":"2024-05-08T23:23:39.485976+0000","last_active":"2024-05-08T23:23:40.471433+0000","last_peered":"2024-05-08T23:23:40.471433+0000","last_clean":"2024-05-08T23:23:40.471433+0000","last_became_active":"2024-05-08T23:23:39.485676+0000","last_became_peered":"2024-05-08T23:23:39.485676+0000","last_unstale":"2024-05-08T23:23:40.471433+0000","last_undegraded":"2024-05-08T23:23:40.471433+0000","last_fullsized":"2024-05-08T23:23:40.471433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:56.158345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147741+0000","last_change":"2024-05-08T23:23:39.497134+0000","last_active":"2024-05-08T23:23:41.147741+0000","last_peered":"2024-05-08T23:23:41.147741+0000","last_clean":"2024-05-08T23:23:41.147741+0000","last_became_active":"2024-05-08T23:23:39.496842+0000","last_became_peered":"2024-05-08T23:23:39.496842+0000","last_unstale":"2024-05-08T23:23:41.147741+0000","last_undegraded":"2024-05-08T23:23:41.147741+0000","last_fullsized":"2024-05-08T23:23:41.147741+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:17.076074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009163+0000","last_change":"2024-05-08T23:23:39.491247+0000","last_active":"2024-05-08T23:23:41.009163+0000","last_peered":"2024-05-08T23:23:41.009163+0000","last_clean":"2024-05-08T23:23:41.009163+0000","last_became_active":"2024-05-08T23:23:39.490867+0000","last_became_peered":"2024-05-08T23:23:39.490867+0000","last_unstale":"2024-05-08T23:23:41.009163+0000","last_undegraded":"2024-05-08T23:23:41.009163+0000","last_fullsized":"2024-05-08T23:23:41.009163+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:05.858704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009830+0000","last_change":"2024-05-08T23:23:39.495008+0000","last_active":"2024-05-08T23:23:41.009830+0000","last_peered":"2024-05-08T23:23:41.009830+0000","last_clean":"2024-05-08T23:23:41.009830+0000","last_became_active":"2024-05-08T23:23:39.490557+0000","last_became_peered":"2024-05-08T23:23:39.490557+0000","last_unstale":"2024-05-08T23:23:41.009830+0000","last_undegraded":"2024-05-08T23:23:41.009830+0000","last_fullsized":"2024-05-08T23:23:41.009830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:41.299209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009588+0000","last_change":"2024-05-08T23:23:39.493878+0000","last_active":"2024-05-08T23:23:41.009588+0000","last_peered":"2024-05-08T23:23:41.009588+0000","last_clean":"2024-05-08T23:23:41.009588+0000","last_became_active":"2024-05-08T23:23:39.490794+0000","last_became_peered":"2024-05-08T23:23:39.490794+0000","last_unstale":"2024-05-08T23:23:41.009588+0000","last_undegraded":"2024-05-08T23:23:41.009588+0000","last_fullsized":"2024-05-08T23:23:41.009588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:22.791683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009491+0000","last_change":"2024-05-08T23:23:39.501623+0000","last_active":"2024-05-08T23:23:41.009491+0000","last_peered":"2024-05-08T23:23:41.009491+0000","last_clean":"2024-05-08T23:23:41.009491+0000","last_became_active":"2024-05-08T23:23:39.501282+0000","last_became_peered":"2024-05-08T23:23:39.501282+0000","last_unstale":"2024-05-08T23:23:41.009491+0000","last_undegraded":"2024-05-08T23:23:41.009491+0000","last_fullsized":"2024-05-08T23:23:41.009491+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:34.367633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473165+0000","last_change":"2024-05-08T23:23:39.493054+0000","last_active":"2024-05-08T23:23:40.473165+0000","last_peered":"2024-05-08T23:23:40.473165+0000","last_clean":"2024-05-08T23:23:40.473165+0000","last_became_active":"2024-05-08T23:23:39.492087+0000","last_became_peered":"2024-05-08T23:23:39.492087+0000","last_unstale":"2024-05-08T23:23:40.473165+0000","last_undegraded":"2024-05-08T23:23:40.473165+0000","last_fullsized":"2024-05-08T23:23:40.473165+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.768894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646460+0000","last_change":"2024-05-08T23:23:39.493024+0000","last_active":"2024-05-08T23:23:40.646460+0000","last_peered":"2024-05-08T23:23:40.646460+0000","last_clean":"2024-05-08T23:23:40.646460+0000","last_became_active":"2024-05-08T23:23:39.492772+0000","last_became_peered":"2024-05-08T23:23:39.492772+0000","last_unstale":"2024-05-08T23:23:40.646460+0000","last_undegraded":"2024-05-08T23:23:40.646460+0000","last_fullsized":"2024-05-08T23:23:40.646460+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:49.200126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009425+0000","last_change":"2024-05-08T23:23:39.494669+0000","last_active":"2024-05-08T23:23:41.009425+0000","last_peered":"2024-05-08T23:23:41.009425+0000","last_clean":"2024-05-08T23:23:41.009425+0000","last_became_active":"2024-05-08T23:23:39.494231+0000","last_became_peered":"2024-05-08T23:23:39.494231+0000","last_unstale":"2024-05-08T23:23:41.009425+0000","last_undegraded":"2024-05-08T23:23:41.009425+0000","last_fullsized":"2024-05-08T23:23:41.009425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:01.672017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471304+0000","last_change":"2024-05-08T23:23:39.476475+0000","last_active":"2024-05-08T23:23:40.471304+0000","last_peered":"2024-05-08T23:23:40.471304+0000","last_clean":"2024-05-08T23:23:40.471304+0000","last_became_active":"2024-05-08T23:23:39.476208+0000","last_became_peered":"2024-05-08T23:23:39.476208+0000","last_unstale":"2024-05-08T23:23:40.471304+0000","last_undegraded":"2024-05-08T23:23:40.471304+0000","last_fullsized":"2024-05-08T23:23:40.471304+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:30.372964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148606+0000","last_change":"2024-05-08T23:23:39.493063+0000","last_active":"2024-05-08T23:23:41.148606+0000","last_peered":"2024-05-08T23:23:41.148606+0000","last_clean":"2024-05-08T23:23:41.148606+0000","last_became_active":"2024-05-08T23:23:39.492784+0000","last_became_peered":"2024-05-08T23:23:39.492784+0000","last_unstale":"2024-05-08T23:23:41.148606+0000","last_undegraded":"2024-05-08T23:23:41.148606+0000","last_fullsized":"2024-05-08T23:23:41.148606+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:19.521622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472262+0000","last_change":"2024-05-08T23:23:39.475820+0000","last_active":"2024-05-08T23:23:40.472262+0000","last_peered":"2024-05-08T23:23:40.472262+0000","last_clean":"2024-05-08T23:23:40.472262+0000","last_became_active":"2024-05-08T23:23:39.475518+0000","last_became_peered":"2024-05-08T23:23:39.475518+0000","last_unstale":"2024-05-08T23:23:40.472262+0000","last_undegraded":"2024-05-08T23:23:40.472262+0000","last_fullsized":"2024-05-08T23:23:40.472262+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:54.950277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472254+0000","last_change":"2024-05-08T23:23:39.493653+0000","last_active":"2024-05-08T23:23:40.472254+0000","last_peered":"2024-05-08T23:23:40.472254+0000","last_clean":"2024-05-08T23:23:40.472254+0000","last_became_active":"2024-05-08T23:23:39.493349+0000","last_became_peered":"2024-05-08T23:23:39.493349+0000","last_unstale":"2024-05-08T23:23:40.472254+0000","last_undegraded":"2024-05-08T23:23:40.472254+0000","last_fullsized":"2024-05-08T23:23:40.472254+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:22.791683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148995+0000","last_change":"2024-05-08T23:23:39.498212+0000","last_active":"2024-05-08T23:23:41.148995+0000","last_peered":"2024-05-08T23:23:41.148995+0000","last_clean":"2024-05-08T23:23:41.148995+0000","last_became_active":"2024-05-08T23:23:39.497906+0000","last_became_peered":"2024-05-08T23:23:39.497906+0000","last_unstale":"2024-05-08T23:23:41.148995+0000","last_undegraded":"2024-05-08T23:23:41.148995+0000","last_fullsized":"2024-05-08T23:23:41.148995+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:50:18.129675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646609+0000","last_change":"2024-05-08T23:23:39.495322+0000","last_active":"2024-05-08T23:23:40.646609+0000","last_peered":"2024-05-08T23:23:40.646609+0000","last_clean":"2024-05-08T23:23:40.646609+0000","last_became_active":"2024-05-08T23:23:39.495075+0000","last_became_peered":"2024-05-08T23:23:39.495075+0000","last_unstale":"2024-05-08T23:23:40.646609+0000","last_undegraded":"2024-05-08T23:23:40.646609+0000","last_fullsized":"2024-05-08T23:23:40.646609+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:03.099478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471445+0000","last_change":"2024-05-08T23:23:39.484588+0000","last_active":"2024-05-08T23:23:40.471445+0000","last_peered":"2024-05-08T23:23:40.471445+0000","last_clean":"2024-05-08T23:23:40.471445+0000","last_became_active":"2024-05-08T23:23:39.483740+0000","last_became_peered":"2024-05-08T23:23:39.483740+0000","last_unstale":"2024-05-08T23:23:40.471445+0000","last_undegraded":"2024-05-08T23:23:40.471445+0000","last_fullsized":"2024-05-08T23:23:40.471445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:28.984936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009440+0000","last_change":"2024-05-08T23:23:39.488733+0000","last_active":"2024-05-08T23:23:41.009440+0000","last_peered":"2024-05-08T23:23:41.009440+0000","last_clean":"2024-05-08T23:23:41.009440+0000","last_became_active":"2024-05-08T23:23:39.488420+0000","last_became_peered":"2024-05-08T23:23:39.488420+0000","last_unstale":"2024-05-08T23:23:41.009440+0000","last_undegraded":"2024-05-08T23:23:41.009440+0000","last_fullsized":"2024-05-08T23:23:41.009440+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:30.727813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148615+0000","last_change":"2024-05-08T23:23:39.486406+0000","last_active":"2024-05-08T23:23:41.148615+0000","last_peered":"2024-05-08T23:23:41.148615+0000","last_clean":"2024-05-08T23:23:41.148615+0000","last_became_active":"2024-05-08T23:23:39.486062+0000","last_became_peered":"2024-05-08T23:23:39.486062+0000","last_unstale":"2024-05-08T23:23:41.148615+0000","last_undegraded":"2024-05-08T23:23:41.148615+0000","last_fullsized":"2024-05-08T23:23:41.148615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:16.785894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148433+0000","last_change":"2024-05-08T23:23:39.485950+0000","last_active":"2024-05-08T23:23:41.148433+0000","last_peered":"2024-05-08T23:23:41.148433+0000","last_clean":"2024-05-08T23:23:41.148433+0000","last_became_active":"2024-05-08T23:23:39.485649+0000","last_became_peered":"2024-05-08T23:23:39.485649+0000","last_unstale":"2024-05-08T23:23:41.148433+0000","last_undegraded":"2024-05-08T23:23:41.148433+0000","last_fullsized":"2024-05-08T23:23:41.148433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:25.353577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.149124+0000","last_change":"2024-05-08T23:23:39.488452+0000","last_active":"2024-05-08T23:23:41.149124+0000","last_peered":"2024-05-08T23:23:41.149124+0000","last_clean":"2024-05-08T23:23:41.149124+0000","last_became_active":"2024-05-08T23:23:39.488141+0000","last_became_peered":"2024-05-08T23:23:39.488141+0000","last_unstale":"2024-05-08T23:23:41.149124+0000","last_undegraded":"2024-05-08T23:23:41.149124+0000","last_fullsized":"2024-05-08T23:23:41.149124+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:56.655941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646603+0000","last_change":"2024-05-08T23:23:39.488603+0000","last_active":"2024-05-08T23:23:40.646603+0000","last_peered":"2024-05-08T23:23:40.646603+0000","last_clean":"2024-05-08T23:23:40.646603+0000","last_became_active":"2024-05-08T23:23:39.488166+0000","last_became_peered":"2024-05-08T23:23:39.488166+0000","last_unstale":"2024-05-08T23:23:40.646603+0000","last_undegraded":"2024-05-08T23:23:40.646603+0000","last_fullsized":"2024-05-08T23:23:40.646603+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:38:50.581266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472419+0000","last_change":"2024-05-08T23:23:39.475898+0000","last_active":"2024-05-08T23:23:40.472419+0000","last_peered":"2024-05-08T23:23:40.472419+0000","last_clean":"2024-05-08T23:23:40.472419+0000","last_became_active":"2024-05-08T23:23:39.475601+0000","last_became_peered":"2024-05-08T23:23:39.475601+0000","last_unstale":"2024-05-08T23:23:40.472419+0000","last_undegraded":"2024-05-08T23:23:40.472419+0000","last_fullsized":"2024-05-08T23:23:40.472419+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:32.042955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472895+0000","last_change":"2024-05-08T23:23:39.496005+0000","last_active":"2024-05-08T23:23:40.472895+0000","last_peered":"2024-05-08T23:23:40.472895+0000","last_clean":"2024-05-08T23:23:40.472895+0000","last_became_active":"2024-05-08T23:23:39.495685+0000","last_became_peered":"2024-05-08T23:23:39.495685+0000","last_unstale":"2024-05-08T23:23:40.472895+0000","last_undegraded":"2024-05-08T23:23:40.472895+0000","last_fullsized":"2024-05-08T23:23:40.472895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:59.158101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148721+0000","last_change":"2024-05-08T23:23:39.497512+0000","last_active":"2024-05-08T23:23:41.148721+0000","last_peered":"2024-05-08T23:23:41.148721+0000","last_clean":"2024-05-08T23:23:41.148721+0000","last_became_active":"2024-05-08T23:23:39.497222+0000","last_became_peered":"2024-05-08T23:23:39.497222+0000","last_unstale":"2024-05-08T23:23:41.148721+0000","last_undegraded":"2024-05-08T23:23:41.148721+0000","last_fullsized":"2024-05-08T23:23:41.148721+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:07.789294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472114+0000","last_change":"2024-05-08T23:23:39.487156+0000","last_active":"2024-05-08T23:23:40.472114+0000","last_peered":"2024-05-08T23:23:40.472114+0000","last_clean":"2024-05-08T23:23:40.472114+0000","last_became_active":"2024-05-08T23:23:39.486861+0000","last_became_peered":"2024-05-08T23:23:39.486861+0000","last_unstale":"2024-05-08T23:23:40.472114+0000","last_undegraded":"2024-05-08T23:23:40.472114+0000","last_fullsized":"2024-05-08T23:23:40.472114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.768894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471372+0000","last_change":"2024-05-08T23:23:39.475937+0000","last_active":"2024-05-08T23:23:40.471372+0000","last_peered":"2024-05-08T23:23:40.471372+0000","last_clean":"2024-05-08T23:23:40.471372+0000","last_became_active":"2024-05-08T23:23:39.475672+0000","last_became_peered":"2024-05-08T23:23:39.475672+0000","last_unstale":"2024-05-08T23:23:40.471372+0000","last_undegraded":"2024-05-08T23:23:40.471372+0000","last_fullsized":"2024-05-08T23:23:40.471372+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:28.786132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646394+0000","last_change":"2024-05-08T23:23:39.498230+0000","last_active":"2024-05-08T23:23:40.646394+0000","last_peered":"2024-05-08T23:23:40.646394+0000","last_clean":"2024-05-08T23:23:40.646394+0000","last_became_active":"2024-05-08T23:23:39.497859+0000","last_became_peered":"2024-05-08T23:23:39.497859+0000","last_unstale":"2024-05-08T23:23:40.646394+0000","last_undegraded":"2024-05-08T23:23:40.646394+0000","last_fullsized":"2024-05-08T23:23:40.646394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:35.068859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688592+0000","last_change":"2024-05-08T23:23:39.489075+0000","last_active":"2024-05-08T23:23:40.688592+0000","last_peered":"2024-05-08T23:23:40.688592+0000","last_clean":"2024-05-08T23:23:40.688592+0000","last_became_active":"2024-05-08T23:23:39.488806+0000","last_became_peered":"2024-05-08T23:23:39.488806+0000","last_unstale":"2024-05-08T23:23:40.688592+0000","last_undegraded":"2024-05-08T23:23:40.688592+0000","last_fullsized":"2024-05-08T23:23:40.688592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:33.959563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471976+0000","last_change":"2024-05-08T23:23:39.017929+0000","last_active":"2024-05-08T23:23:40.471976+0000","last_peered":"2024-05-08T23:23:40.471976+0000","last_clean":"2024-05-08T23:23:40.471976+0000","last_became_active":"2024-05-08T23:23:39.017608+0000","last_became_peered":"2024-05-08T23:23:39.017608+0000","last_unstale":"2024-05-08T23:23:40.471976+0000","last_undegraded":"2024-05-08T23:23:40.471976+0000","last_fullsized":"2024-05-08T23:23:40.471976+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:36.357850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148994+0000","last_change":"2024-05-08T23:23:39.484978+0000","last_active":"2024-05-08T23:23:41.148994+0000","last_peered":"2024-05-08T23:23:41.148994+0000","last_clean":"2024-05-08T23:23:41.148994+0000","last_became_active":"2024-05-08T23:23:39.484683+0000","last_became_peered":"2024-05-08T23:23:39.484683+0000","last_unstale":"2024-05-08T23:23:41.148994+0000","last_undegraded":"2024-05-08T23:23:41.148994+0000","last_fullsized":"2024-05-08T23:23:41.148994+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:07.161103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472673+0000","last_change":"2024-05-08T23:23:39.026823+0000","last_active":"2024-05-08T23:23:40.472673+0000","last_peered":"2024-05-08T23:23:40.472673+0000","last_clean":"2024-05-08T23:23:40.472673+0000","last_became_active":"2024-05-08T23:23:39.026546+0000","last_became_peered":"2024-05-08T23:23:39.026546+0000","last_unstale":"2024-05-08T23:23:40.472673+0000","last_undegraded":"2024-05-08T23:23:40.472673+0000","last_fullsized":"2024-05-08T23:23:40.472673+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:10.901466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473026+0000","last_change":"2024-05-08T23:23:39.478089+0000","last_active":"2024-05-08T23:23:40.473026+0000","last_peered":"2024-05-08T23:23:40.473026+0000","last_clean":"2024-05-08T23:23:40.473026+0000","last_became_active":"2024-05-08T23:23:39.477835+0000","last_became_peered":"2024-05-08T23:23:39.477835+0000","last_unstale":"2024-05-08T23:23:40.473026+0000","last_undegraded":"2024-05-08T23:23:40.473026+0000","last_fullsized":"2024-05-08T23:23:40.473026+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:22.791683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688684+0000","last_change":"2024-05-08T23:23:39.028361+0000","last_active":"2024-05-08T23:23:40.688684+0000","last_peered":"2024-05-08T23:23:40.688684+0000","last_clean":"2024-05-08T23:23:40.688684+0000","last_became_active":"2024-05-08T23:23:39.028090+0000","last_became_peered":"2024-05-08T23:23:39.028090+0000","last_unstale":"2024-05-08T23:23:40.688684+0000","last_undegraded":"2024-05-08T23:23:40.688684+0000","last_fullsized":"2024-05-08T23:23:40.688684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:34.063977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688553+0000","last_change":"2024-05-08T23:23:39.497907+0000","last_active":"2024-05-08T23:23:40.688553+0000","last_peered":"2024-05-08T23:23:40.688553+0000","last_clean":"2024-05-08T23:23:40.688553+0000","last_became_active":"2024-05-08T23:23:39.494619+0000","last_became_peered":"2024-05-08T23:23:39.494619+0000","last_unstale":"2024-05-08T23:23:40.688553+0000","last_undegraded":"2024-05-08T23:23:40.688553+0000","last_fullsized":"2024-05-08T23:23:40.688553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:07.161103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147559+0000","last_change":"2024-05-08T23:23:39.028246+0000","last_active":"2024-05-08T23:23:41.147559+0000","last_peered":"2024-05-08T23:23:41.147559+0000","last_clean":"2024-05-08T23:23:41.147559+0000","last_became_active":"2024-05-08T23:23:39.027931+0000","last_became_peered":"2024-05-08T23:23:39.027931+0000","last_unstale":"2024-05-08T23:23:41.147559+0000","last_undegraded":"2024-05-08T23:23:41.147559+0000","last_fullsized":"2024-05-08T23:23:41.147559+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:43.383813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472220+0000","last_change":"2024-05-08T23:23:39.483343+0000","last_active":"2024-05-08T23:23:40.472220+0000","last_peered":"2024-05-08T23:23:40.472220+0000","last_clean":"2024-05-08T23:23:40.472220+0000","last_became_active":"2024-05-08T23:23:39.483034+0000","last_became_peered":"2024-05-08T23:23:39.483034+0000","last_unstale":"2024-05-08T23:23:40.472220+0000","last_undegraded":"2024-05-08T23:23:40.472220+0000","last_fullsized":"2024-05-08T23:23:40.472220+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:29.828185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471631+0000","last_change":"2024-05-08T23:23:39.029649+0000","last_active":"2024-05-08T23:23:40.471631+0000","last_peered":"2024-05-08T23:23:40.471631+0000","last_clean":"2024-05-08T23:23:40.471631+0000","last_became_active":"2024-05-08T23:23:39.028984+0000","last_became_peered":"2024-05-08T23:23:39.028984+0000","last_unstale":"2024-05-08T23:23:40.471631+0000","last_undegraded":"2024-05-08T23:23:40.471631+0000","last_fullsized":"2024-05-08T23:23:40.471631+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:07.682195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009657+0000","last_change":"2024-05-08T23:23:39.487414+0000","last_active":"2024-05-08T23:23:41.009657+0000","last_peered":"2024-05-08T23:23:41.009657+0000","last_clean":"2024-05-08T23:23:41.009657+0000","last_became_active":"2024-05-08T23:23:39.487140+0000","last_became_peered":"2024-05-08T23:23:39.487140+0000","last_unstale":"2024-05-08T23:23:41.009657+0000","last_undegraded":"2024-05-08T23:23:41.009657+0000","last_fullsized":"2024-05-08T23:23:41.009657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:54.440871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472345+0000","last_change":"2024-05-08T23:23:39.027821+0000","last_active":"2024-05-08T23:23:40.472345+0000","last_peered":"2024-05-08T23:23:40.472345+0000","last_clean":"2024-05-08T23:23:40.472345+0000","last_became_active":"2024-05-08T23:23:39.027505+0000","last_became_peered":"2024-05-08T23:23:39.027505+0000","last_unstale":"2024-05-08T23:23:40.472345+0000","last_undegraded":"2024-05-08T23:23:40.472345+0000","last_fullsized":"2024-05-08T23:23:40.472345+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:24.348695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009788+0000","last_change":"2024-05-08T23:23:39.491086+0000","last_active":"2024-05-08T23:23:41.009788+0000","last_peered":"2024-05-08T23:23:41.009788+0000","last_clean":"2024-05-08T23:23:41.009788+0000","last_became_active":"2024-05-08T23:23:39.490748+0000","last_became_peered":"2024-05-08T23:23:39.490748+0000","last_unstale":"2024-05-08T23:23:41.009788+0000","last_undegraded":"2024-05-08T23:23:41.009788+0000","last_fullsized":"2024-05-08T23:23:41.009788+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:28.729823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472813+0000","last_change":"2024-05-08T23:23:39.027215+0000","last_active":"2024-05-08T23:23:40.472813+0000","last_peered":"2024-05-08T23:23:40.472813+0000","last_clean":"2024-05-08T23:23:40.472813+0000","last_became_active":"2024-05-08T23:23:39.026917+0000","last_became_peered":"2024-05-08T23:23:39.026917+0000","last_unstale":"2024-05-08T23:23:40.472813+0000","last_undegraded":"2024-05-08T23:23:40.472813+0000","last_fullsized":"2024-05-08T23:23:40.472813+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:55.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148664+0000","last_change":"2024-05-08T23:23:39.486146+0000","last_active":"2024-05-08T23:23:41.148664+0000","last_peered":"2024-05-08T23:23:41.148664+0000","last_clean":"2024-05-08T23:23:41.148664+0000","last_became_active":"2024-05-08T23:23:39.485849+0000","last_became_peered":"2024-05-08T23:23:39.485849+0000","last_unstale":"2024-05-08T23:23:41.148664+0000","last_undegraded":"2024-05-08T23:23:41.148664+0000","last_fullsized":"2024-05-08T23:23:41.148664+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:11.906348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009420+0000","last_change":"2024-05-08T23:23:39.030521+0000","last_active":"2024-05-08T23:23:41.009420+0000","last_peered":"2024-05-08T23:23:41.009420+0000","last_clean":"2024-05-08T23:23:41.009420+0000","last_became_active":"2024-05-08T23:23:39.030248+0000","last_became_peered":"2024-05-08T23:23:39.030248+0000","last_unstale":"2024-05-08T23:23:41.009420+0000","last_undegraded":"2024-05-08T23:23:41.009420+0000","last_fullsized":"2024-05-08T23:23:41.009420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:40.828807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472568+0000","last_change":"2024-05-08T23:23:39.491842+0000","last_active":"2024-05-08T23:23:40.472568+0000","last_peered":"2024-05-08T23:23:40.472568+0000","last_clean":"2024-05-08T23:23:40.472568+0000","last_became_active":"2024-05-08T23:23:39.491557+0000","last_became_peered":"2024-05-08T23:23:39.491557+0000","last_unstale":"2024-05-08T23:23:40.472568+0000","last_undegraded":"2024-05-08T23:23:40.472568+0000","last_fullsized":"2024-05-08T23:23:40.472568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:54.440871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472575+0000","last_change":"2024-05-08T23:23:39.027185+0000","last_active":"2024-05-08T23:23:40.472575+0000","last_peered":"2024-05-08T23:23:40.472575+0000","last_clean":"2024-05-08T23:23:40.472575+0000","last_became_active":"2024-05-08T23:23:39.026893+0000","last_became_peered":"2024-05-08T23:23:39.026893+0000","last_unstale":"2024-05-08T23:23:40.472575+0000","last_undegraded":"2024-05-08T23:23:40.472575+0000","last_fullsized":"2024-05-08T23:23:40.472575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:08:55.318230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147884+0000","last_change":"2024-05-08T23:23:39.490848+0000","last_active":"2024-05-08T23:23:41.147884+0000","last_peered":"2024-05-08T23:23:41.147884+0000","last_clean":"2024-05-08T23:23:41.147884+0000","last_became_active":"2024-05-08T23:23:39.490543+0000","last_became_peered":"2024-05-08T23:23:39.490543+0000","last_unstale":"2024-05-08T23:23:41.147884+0000","last_undegraded":"2024-05-08T23:23:41.147884+0000","last_fullsized":"2024-05-08T23:23:41.147884+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:08.687077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009859+0000","last_change":"2024-05-08T23:23:39.031385+0000","last_active":"2024-05-08T23:23:41.009859+0000","last_peered":"2024-05-08T23:23:41.009859+0000","last_clean":"2024-05-08T23:23:41.009859+0000","last_became_active":"2024-05-08T23:23:39.031089+0000","last_became_peered":"2024-05-08T23:23:39.031089+0000","last_unstale":"2024-05-08T23:23:41.009859+0000","last_undegraded":"2024-05-08T23:23:41.009859+0000","last_fullsized":"2024-05-08T23:23:41.009859+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:15.129335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471766+0000","last_change":"2024-05-08T23:23:39.492475+0000","last_active":"2024-05-08T23:23:40.471766+0000","last_peered":"2024-05-08T23:23:40.471766+0000","last_clean":"2024-05-08T23:23:40.471766+0000","last_became_active":"2024-05-08T23:23:39.492169+0000","last_became_peered":"2024-05-08T23:23:39.492169+0000","last_unstale":"2024-05-08T23:23:40.471766+0000","last_undegraded":"2024-05-08T23:23:40.471766+0000","last_fullsized":"2024-05-08T23:23:40.471766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:59.343711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147749+0000","last_change":"2024-05-08T23:23:39.028065+0000","last_active":"2024-05-08T23:23:41.147749+0000","last_peered":"2024-05-08T23:23:41.147749+0000","last_clean":"2024-05-08T23:23:41.147749+0000","last_became_active":"2024-05-08T23:23:39.027786+0000","last_became_peered":"2024-05-08T23:23:39.027786+0000","last_unstale":"2024-05-08T23:23:41.147749+0000","last_undegraded":"2024-05-08T23:23:41.147749+0000","last_fullsized":"2024-05-08T23:23:41.147749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:53.435989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147630+0000","last_change":"2024-05-08T23:23:39.500371+0000","last_active":"2024-05-08T23:23:41.147630+0000","last_peered":"2024-05-08T23:23:41.147630+0000","last_clean":"2024-05-08T23:23:41.147630+0000","last_became_active":"2024-05-08T23:23:39.499995+0000","last_became_peered":"2024-05-08T23:23:39.499995+0000","last_unstale":"2024-05-08T23:23:41.147630+0000","last_undegraded":"2024-05-08T23:23:41.147630+0000","last_fullsized":"2024-05-08T23:23:41.147630+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:27.581561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646658+0000","last_change":"2024-05-08T23:23:39.027079+0000","last_active":"2024-05-08T23:23:40.646658+0000","last_peered":"2024-05-08T23:23:40.646658+0000","last_clean":"2024-05-08T23:23:40.646658+0000","last_became_active":"2024-05-08T23:23:39.026789+0000","last_became_peered":"2024-05-08T23:23:39.026789+0000","last_unstale":"2024-05-08T23:23:40.646658+0000","last_undegraded":"2024-05-08T23:23:40.646658+0000","last_fullsized":"2024-05-08T23:23:40.646658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:03.713283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148758+0000","last_change":"2024-05-08T23:23:39.498249+0000","last_active":"2024-05-08T23:23:41.148758+0000","last_peered":"2024-05-08T23:23:41.148758+0000","last_clean":"2024-05-08T23:23:41.148758+0000","last_became_active":"2024-05-08T23:23:39.497963+0000","last_became_peered":"2024-05-08T23:23:39.497963+0000","last_unstale":"2024-05-08T23:23:41.148758+0000","last_undegraded":"2024-05-08T23:23:41.148758+0000","last_fullsized":"2024-05-08T23:23:41.148758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:06.330086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471741+0000","last_change":"2024-05-08T23:23:39.025640+0000","last_active":"2024-05-08T23:23:40.471741+0000","last_peered":"2024-05-08T23:23:40.471741+0000","last_clean":"2024-05-08T23:23:40.471741+0000","last_became_active":"2024-05-08T23:23:39.025390+0000","last_became_peered":"2024-05-08T23:23:39.025390+0000","last_unstale":"2024-05-08T23:23:40.471741+0000","last_undegraded":"2024-05-08T23:23:40.471741+0000","last_fullsized":"2024-05-08T23:23:40.471741+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:04.853822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009747+0000","last_change":"2024-05-08T23:23:39.495146+0000","last_active":"2024-05-08T23:23:41.009747+0000","last_peered":"2024-05-08T23:23:41.009747+0000","last_clean":"2024-05-08T23:23:41.009747+0000","last_became_active":"2024-05-08T23:23:39.494844+0000","last_became_peered":"2024-05-08T23:23:39.494844+0000","last_unstale":"2024-05-08T23:23:41.009747+0000","last_undegraded":"2024-05-08T23:23:41.009747+0000","last_fullsized":"2024-05-08T23:23:41.009747+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:59.158101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147553+0000","last_change":"2024-05-08T23:23:39.028638+0000","last_active":"2024-05-08T23:23:41.147553+0000","last_peered":"2024-05-08T23:23:41.147553+0000","last_clean":"2024-05-08T23:23:41.147553+0000","last_became_active":"2024-05-08T23:23:39.028360+0000","last_became_peered":"2024-05-08T23:23:39.028360+0000","last_unstale":"2024-05-08T23:23:41.147553+0000","last_undegraded":"2024-05-08T23:23:41.147553+0000","last_fullsized":"2024-05-08T23:23:41.147553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:28.823303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471271+0000","last_change":"2024-05-08T23:23:39.493388+0000","last_active":"2024-05-08T23:23:40.471271+0000","last_peered":"2024-05-08T23:23:40.471271+0000","last_clean":"2024-05-08T23:23:40.471271+0000","last_became_active":"2024-05-08T23:23:39.493091+0000","last_became_peered":"2024-05-08T23:23:39.493091+0000","last_unstale":"2024-05-08T23:23:40.471271+0000","last_undegraded":"2024-05-08T23:23:40.471271+0000","last_fullsized":"2024-05-08T23:23:40.471271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:05.335477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471413+0000","last_change":"2024-05-08T23:23:39.025294+0000","last_active":"2024-05-08T23:23:40.471413+0000","last_peered":"2024-05-08T23:23:40.471413+0000","last_clean":"2024-05-08T23:23:40.471413+0000","last_became_active":"2024-05-08T23:23:39.024959+0000","last_became_peered":"2024-05-08T23:23:39.024959+0000","last_unstale":"2024-05-08T23:23:40.471413+0000","last_undegraded":"2024-05-08T23:23:40.471413+0000","last_fullsized":"2024-05-08T23:23:40.471413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:29.722931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147662+0000","last_change":"2024-05-08T23:23:39.491344+0000","last_active":"2024-05-08T23:23:41.147662+0000","last_peered":"2024-05-08T23:23:41.147662+0000","last_clean":"2024-05-08T23:23:41.147662+0000","last_became_active":"2024-05-08T23:23:39.487190+0000","last_became_peered":"2024-05-08T23:23:39.487190+0000","last_unstale":"2024-05-08T23:23:41.147662+0000","last_undegraded":"2024-05-08T23:23:41.147662+0000","last_fullsized":"2024-05-08T23:23:41.147662+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:54.680549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009305+0000","last_change":"2024-05-08T23:23:39.031326+0000","last_active":"2024-05-08T23:23:41.009305+0000","last_peered":"2024-05-08T23:23:41.009305+0000","last_clean":"2024-05-08T23:23:41.009305+0000","last_became_active":"2024-05-08T23:23:39.030967+0000","last_became_peered":"2024-05-08T23:23:39.030967+0000","last_unstale":"2024-05-08T23:23:41.009305+0000","last_undegraded":"2024-05-08T23:23:41.009305+0000","last_fullsized":"2024-05-08T23:23:41.009305+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:07.056231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646653+0000","last_change":"2024-05-08T23:23:39.488942+0000","last_active":"2024-05-08T23:23:40.646653+0000","last_peered":"2024-05-08T23:23:40.646653+0000","last_clean":"2024-05-08T23:23:40.646653+0000","last_became_active":"2024-05-08T23:23:39.488621+0000","last_became_peered":"2024-05-08T23:23:39.488621+0000","last_unstale":"2024-05-08T23:23:40.646653+0000","last_undegraded":"2024-05-08T23:23:40.646653+0000","last_fullsized":"2024-05-08T23:23:40.646653+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:06.309720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646661+0000","last_change":"2024-05-08T23:23:39.015694+0000","last_active":"2024-05-08T23:23:40.646661+0000","last_peered":"2024-05-08T23:23:40.646661+0000","last_clean":"2024-05-08T23:23:40.646661+0000","last_became_active":"2024-05-08T23:23:39.015375+0000","last_became_peered":"2024-05-08T23:23:39.015375+0000","last_unstale":"2024-05-08T23:23:40.646661+0000","last_undegraded":"2024-05-08T23:23:40.646661+0000","last_fullsized":"2024-05-08T23:23:40.646661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:13.429089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009872+0000","last_change":"2024-05-08T23:23:39.480382+0000","last_active":"2024-05-08T23:23:41.009872+0000","last_peered":"2024-05-08T23:23:41.009872+0000","last_clean":"2024-05-08T23:23:41.009872+0000","last_became_active":"2024-05-08T23:23:39.480099+0000","last_became_peered":"2024-05-08T23:23:39.480099+0000","last_unstale":"2024-05-08T23:23:41.009872+0000","last_undegraded":"2024-05-08T23:23:41.009872+0000","last_fullsized":"2024-05-08T23:23:41.009872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:29.828185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472463+0000","last_change":"2024-05-08T23:23:39.030569+0000","last_active":"2024-05-08T23:23:40.472463+0000","last_peered":"2024-05-08T23:23:40.472463+0000","last_clean":"2024-05-08T23:23:40.472463+0000","last_became_active":"2024-05-08T23:23:39.030285+0000","last_became_peered":"2024-05-08T23:23:39.030285+0000","last_unstale":"2024-05-08T23:23:40.472463+0000","last_undegraded":"2024-05-08T23:23:40.472463+0000","last_fullsized":"2024-05-08T23:23:40.472463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:40.828807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148798+0000","last_change":"2024-05-08T23:23:39.487383+0000","last_active":"2024-05-08T23:23:41.148798+0000","last_peered":"2024-05-08T23:23:41.148798+0000","last_clean":"2024-05-08T23:23:41.148798+0000","last_became_active":"2024-05-08T23:23:39.486493+0000","last_became_peered":"2024-05-08T23:23:39.486493+0000","last_unstale":"2024-05-08T23:23:41.148798+0000","last_undegraded":"2024-05-08T23:23:41.148798+0000","last_fullsized":"2024-05-08T23:23:41.148798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:41.833689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473067+0000","last_change":"2024-05-08T23:23:39.028725+0000","last_active":"2024-05-08T23:23:40.473067+0000","last_peered":"2024-05-08T23:23:40.473067+0000","last_clean":"2024-05-08T23:23:40.473067+0000","last_became_active":"2024-05-08T23:23:39.027186+0000","last_became_peered":"2024-05-08T23:23:39.027186+0000","last_unstale":"2024-05-08T23:23:40.473067+0000","last_undegraded":"2024-05-08T23:23:40.473067+0000","last_fullsized":"2024-05-08T23:23:40.473067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:50:17.124793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.149047+0000","last_change":"2024-05-08T23:23:39.500401+0000","last_active":"2024-05-08T23:23:41.149047+0000","last_peered":"2024-05-08T23:23:41.149047+0000","last_clean":"2024-05-08T23:23:41.149047+0000","last_became_active":"2024-05-08T23:23:39.500066+0000","last_became_peered":"2024-05-08T23:23:39.500066+0000","last_unstale":"2024-05-08T23:23:41.149047+0000","last_undegraded":"2024-05-08T23:23:41.149047+0000","last_fullsized":"2024-05-08T23:23:41.149047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:33.959563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471776+0000","last_change":"2024-05-08T23:23:39.017107+0000","last_active":"2024-05-08T23:23:40.471776+0000","last_peered":"2024-05-08T23:23:40.471776+0000","last_clean":"2024-05-08T23:23:40.471776+0000","last_became_active":"2024-05-08T23:23:39.016802+0000","last_became_peered":"2024-05-08T23:23:39.016802+0000","last_unstale":"2024-05-08T23:23:40.471776+0000","last_undegraded":"2024-05-08T23:23:40.471776+0000","last_fullsized":"2024-05-08T23:23:40.471776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:33.362751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471431+0000","last_change":"2024-05-08T23:23:39.492574+0000","last_active":"2024-05-08T23:23:40.471431+0000","last_peered":"2024-05-08T23:23:40.471431+0000","last_clean":"2024-05-08T23:23:40.471431+0000","last_became_active":"2024-05-08T23:23:39.492292+0000","last_became_peered":"2024-05-08T23:23:39.492292+0000","last_unstale":"2024-05-08T23:23:40.471431+0000","last_undegraded":"2024-05-08T23:23:40.471431+0000","last_fullsized":"2024-05-08T23:23:40.471431+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:22.022575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471771+0000","last_change":"2024-05-08T23:23:39.029842+0000","last_active":"2024-05-08T23:23:40.471771+0000","last_peered":"2024-05-08T23:23:40.471771+0000","last_clean":"2024-05-08T23:23:40.471771+0000","last_became_active":"2024-05-08T23:23:39.029365+0000","last_became_peered":"2024-05-08T23:23:39.029365+0000","last_unstale":"2024-05-08T23:23:40.471771+0000","last_undegraded":"2024-05-08T23:23:40.471771+0000","last_fullsized":"2024-05-08T23:23:40.471771+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:53.675667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688491+0000","last_change":"2024-05-08T23:23:39.494374+0000","last_active":"2024-05-08T23:23:40.688491+0000","last_peered":"2024-05-08T23:23:40.688491+0000","last_clean":"2024-05-08T23:23:40.688491+0000","last_became_active":"2024-05-08T23:23:39.494009+0000","last_became_peered":"2024-05-08T23:23:39.494009+0000","last_unstale":"2024-05-08T23:23:40.688491+0000","last_undegraded":"2024-05-08T23:23:40.688491+0000","last_fullsized":"2024-05-08T23:23:40.688491+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:11.906348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472932+0000","last_change":"2024-05-08T23:23:39.023650+0000","last_active":"2024-05-08T23:23:40.472932+0000","last_peered":"2024-05-08T23:23:40.472932+0000","last_clean":"2024-05-08T23:23:40.472932+0000","last_became_active":"2024-05-08T23:23:39.023351+0000","last_became_peered":"2024-05-08T23:23:39.023351+0000","last_unstale":"2024-05-08T23:23:40.472932+0000","last_undegraded":"2024-05-08T23:23:40.472932+0000","last_fullsized":"2024-05-08T23:23:40.472932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:06.784412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646394+0000","last_change":"2024-05-08T23:23:39.491812+0000","last_active":"2024-05-08T23:23:40.646394+0000","last_peered":"2024-05-08T23:23:40.646394+0000","last_clean":"2024-05-08T23:23:40.646394+0000","last_became_active":"2024-05-08T23:23:39.491543+0000","last_became_peered":"2024-05-08T23:23:39.491543+0000","last_unstale":"2024-05-08T23:23:40.646394+0000","last_undegraded":"2024-05-08T23:23:40.646394+0000","last_fullsized":"2024-05-08T23:23:40.646394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.146424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009253+0000","last_change":"2024-05-08T23:23:39.031430+0000","last_active":"2024-05-08T23:23:41.009253+0000","last_peered":"2024-05-08T23:23:41.009253+0000","last_clean":"2024-05-08T23:23:41.009253+0000","last_became_active":"2024-05-08T23:23:39.031071+0000","last_became_peered":"2024-05-08T23:23:39.031071+0000","last_unstale":"2024-05-08T23:23:41.009253+0000","last_undegraded":"2024-05-08T23:23:41.009253+0000","last_fullsized":"2024-05-08T23:23:41.009253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:34.063977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009236+0000","last_change":"2024-05-08T23:23:39.488731+0000","last_active":"2024-05-08T23:23:41.009236+0000","last_peered":"2024-05-08T23:23:41.009236+0000","last_clean":"2024-05-08T23:23:41.009236+0000","last_became_active":"2024-05-08T23:23:39.488437+0000","last_became_peered":"2024-05-08T23:23:39.488437+0000","last_unstale":"2024-05-08T23:23:41.009236+0000","last_undegraded":"2024-05-08T23:23:41.009236+0000","last_fullsized":"2024-05-08T23:23:41.009236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:54.680549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147844+0000","last_change":"2024-05-08T23:23:39.027238+0000","last_active":"2024-05-08T23:23:41.147844+0000","last_peered":"2024-05-08T23:23:41.147844+0000","last_clean":"2024-05-08T23:23:41.147844+0000","last_became_active":"2024-05-08T23:23:39.026975+0000","last_became_peered":"2024-05-08T23:23:39.026975+0000","last_unstale":"2024-05-08T23:23:41.147844+0000","last_undegraded":"2024-05-08T23:23:41.147844+0000","last_fullsized":"2024-05-08T23:23:41.147844+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.945395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.149069+0000","last_change":"2024-05-08T23:23:39.498459+0000","last_active":"2024-05-08T23:23:41.149069+0000","last_peered":"2024-05-08T23:23:41.149069+0000","last_clean":"2024-05-08T23:23:41.149069+0000","last_became_active":"2024-05-08T23:23:39.498160+0000","last_became_peered":"2024-05-08T23:23:39.498160+0000","last_unstale":"2024-05-08T23:23:41.149069+0000","last_undegraded":"2024-05-08T23:23:41.149069+0000","last_fullsized":"2024-05-08T23:23:41.149069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:27:52.210690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471938+0000","last_change":"2024-05-08T23:23:39.030477+0000","last_active":"2024-05-08T23:23:40.471938+0000","last_peered":"2024-05-08T23:23:40.471938+0000","last_clean":"2024-05-08T23:23:40.471938+0000","last_became_active":"2024-05-08T23:23:39.030192+0000","last_became_peered":"2024-05-08T23:23:39.030192+0000","last_unstale":"2024-05-08T23:23:40.471938+0000","last_undegraded":"2024-05-08T23:23:40.471938+0000","last_fullsized":"2024-05-08T23:23:40.471938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:40.294327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009662+0000","last_change":"2024-05-08T23:23:39.480489+0000","last_active":"2024-05-08T23:23:41.009662+0000","last_peered":"2024-05-08T23:23:41.009662+0000","last_clean":"2024-05-08T23:23:41.009662+0000","last_became_active":"2024-05-08T23:23:39.480187+0000","last_became_peered":"2024-05-08T23:23:39.480187+0000","last_unstale":"2024-05-08T23:23:41.009662+0000","last_undegraded":"2024-05-08T23:23:41.009662+0000","last_fullsized":"2024-05-08T23:23:41.009662+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.768894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147855+0000","last_change":"2024-05-08T23:23:39.030698+0000","last_active":"2024-05-08T23:23:41.147855+0000","last_peered":"2024-05-08T23:23:41.147855+0000","last_clean":"2024-05-08T23:23:41.147855+0000","last_became_active":"2024-05-08T23:23:39.030407+0000","last_became_peered":"2024-05-08T23:23:39.030407+0000","last_unstale":"2024-05-08T23:23:41.147855+0000","last_undegraded":"2024-05-08T23:23:41.147855+0000","last_fullsized":"2024-05-08T23:23:41.147855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:09.029707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147649+0000","last_change":"2024-05-08T23:23:39.492682+0000","last_active":"2024-05-08T23:23:41.147649+0000","last_peered":"2024-05-08T23:23:41.147649+0000","last_clean":"2024-05-08T23:23:41.147649+0000","last_became_active":"2024-05-08T23:23:39.492390+0000","last_became_peered":"2024-05-08T23:23:39.492390+0000","last_unstale":"2024-05-08T23:23:41.147649+0000","last_undegraded":"2024-05-08T23:23:41.147649+0000","last_fullsized":"2024-05-08T23:23:41.147649+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:05.858704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688405+0000","last_change":"2024-05-08T23:23:39.030159+0000","last_active":"2024-05-08T23:23:40.688405+0000","last_peered":"2024-05-08T23:23:40.688405+0000","last_clean":"2024-05-08T23:23:40.688405+0000","last_became_active":"2024-05-08T23:23:39.029872+0000","last_became_peered":"2024-05-08T23:23:39.029872+0000","last_unstale":"2024-05-08T23:23:40.688405+0000","last_undegraded":"2024-05-08T23:23:40.688405+0000","last_fullsized":"2024-05-08T23:23:40.688405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:05.304838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471510+0000","last_change":"2024-05-08T23:23:39.483949+0000","last_active":"2024-05-08T23:23:40.471510+0000","last_peered":"2024-05-08T23:23:40.471510+0000","last_clean":"2024-05-08T23:23:40.471510+0000","last_became_active":"2024-05-08T23:23:39.483767+0000","last_became_peered":"2024-05-08T23:23:39.483767+0000","last_unstale":"2024-05-08T23:23:40.471510+0000","last_undegraded":"2024-05-08T23:23:40.471510+0000","last_fullsized":"2024-05-08T23:23:40.471510+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:45.808539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147887+0000","last_change":"2024-05-08T23:23:39.030805+0000","last_active":"2024-05-08T23:23:41.147887+0000","last_peered":"2024-05-08T23:23:41.147887+0000","last_clean":"2024-05-08T23:23:41.147887+0000","last_became_active":"2024-05-08T23:23:39.030392+0000","last_became_peered":"2024-05-08T23:23:39.030392+0000","last_unstale":"2024-05-08T23:23:41.147887+0000","last_undegraded":"2024-05-08T23:23:41.147887+0000","last_fullsized":"2024-05-08T23:23:41.147887+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:44.764012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471635+0000","last_change":"2024-05-08T23:23:39.491511+0000","last_active":"2024-05-08T23:23:40.471635+0000","last_peered":"2024-05-08T23:23:40.471635+0000","last_clean":"2024-05-08T23:23:40.471635+0000","last_became_active":"2024-05-08T23:23:39.490866+0000","last_became_peered":"2024-05-08T23:23:39.490866+0000","last_unstale":"2024-05-08T23:23:40.471635+0000","last_undegraded":"2024-05-08T23:23:40.471635+0000","last_fullsized":"2024-05-08T23:23:40.471635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:59:59.952098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472482+0000","last_change":"2024-05-08T23:23:39.030262+0000","last_active":"2024-05-08T23:23:40.472482+0000","last_peered":"2024-05-08T23:23:40.472482+0000","last_clean":"2024-05-08T23:23:40.472482+0000","last_became_active":"2024-05-08T23:23:39.029990+0000","last_became_peered":"2024-05-08T23:23:39.029990+0000","last_unstale":"2024-05-08T23:23:40.472482+0000","last_undegraded":"2024-05-08T23:23:40.472482+0000","last_fullsized":"2024-05-08T23:23:40.472482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:15.781012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472096+0000","last_change":"2024-05-08T23:23:39.497366+0000","last_active":"2024-05-08T23:23:40.472096+0000","last_peered":"2024-05-08T23:23:40.472096+0000","last_clean":"2024-05-08T23:23:40.472096+0000","last_became_active":"2024-05-08T23:23:39.497135+0000","last_became_peered":"2024-05-08T23:23:39.497135+0000","last_unstale":"2024-05-08T23:23:40.472096+0000","last_undegraded":"2024-05-08T23:23:40.472096+0000","last_fullsized":"2024-05-08T23:23:40.472096+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:10.034589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688398+0000","last_change":"2024-05-08T23:23:39.031397+0000","last_active":"2024-05-08T23:23:40.688398+0000","last_peered":"2024-05-08T23:23:40.688398+0000","last_clean":"2024-05-08T23:23:40.688398+0000","last_became_active":"2024-05-08T23:23:39.031064+0000","last_became_peered":"2024-05-08T23:23:39.031064+0000","last_unstale":"2024-05-08T23:23:40.688398+0000","last_undegraded":"2024-05-08T23:23:40.688398+0000","last_fullsized":"2024-05-08T23:23:40.688398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:02.094596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472095+0000","last_change":"2024-05-08T23:23:39.481783+0000","last_active":"2024-05-08T23:23:40.472095+0000","last_peered":"2024-05-08T23:23:40.472095+0000","last_clean":"2024-05-08T23:23:40.472095+0000","last_became_active":"2024-05-08T23:23:39.481548+0000","last_became_peered":"2024-05-08T23:23:39.481548+0000","last_unstale":"2024-05-08T23:23:40.472095+0000","last_undegraded":"2024-05-08T23:23:40.472095+0000","last_fullsized":"2024-05-08T23:23:40.472095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:06.330086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009380+0000","last_change":"2024-05-08T23:23:39.028102+0000","last_active":"2024-05-08T23:23:41.009380+0000","last_peered":"2024-05-08T23:23:41.009380+0000","last_clean":"2024-05-08T23:23:41.009380+0000","last_became_active":"2024-05-08T23:23:39.027806+0000","last_became_peered":"2024-05-08T23:23:39.027806+0000","last_unstale":"2024-05-08T23:23:41.009380+0000","last_undegraded":"2024-05-08T23:23:41.009380+0000","last_fullsized":"2024-05-08T23:23:41.009380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:08:55.318230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009752+0000","last_change":"2024-05-08T23:23:39.494242+0000","last_active":"2024-05-08T23:23:41.009752+0000","last_peered":"2024-05-08T23:23:41.009752+0000","last_clean":"2024-05-08T23:23:41.009752+0000","last_became_active":"2024-05-08T23:23:39.493821+0000","last_became_peered":"2024-05-08T23:23:39.493821+0000","last_unstale":"2024-05-08T23:23:41.009752+0000","last_undegraded":"2024-05-08T23:23:41.009752+0000","last_fullsized":"2024-05-08T23:23:41.009752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:04.635681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473237+0000","last_change":"2024-05-08T23:23:39.028452+0000","last_active":"2024-05-08T23:23:40.473237+0000","last_peered":"2024-05-08T23:23:40.473237+0000","last_clean":"2024-05-08T23:23:40.473237+0000","last_became_active":"2024-05-08T23:23:39.028154+0000","last_became_peered":"2024-05-08T23:23:39.028154+0000","last_unstale":"2024-05-08T23:23:40.473237+0000","last_undegraded":"2024-05-08T23:23:40.473237+0000","last_fullsized":"2024-05-08T23:23:40.473237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:32.954681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T23:23:39.476250+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:23:39.476250+0000","last_peered":"2024-05-08T23:23:39.476250+0000","last_clean":"2024-05-08T23:23:39.476250+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:23:39.476250+0000","last_undegraded":"2024-05-08T23:23:39.476250+0000","last_fullsized":"2024-05-08T23:23:39.476250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":812546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.48299999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63200000000000001}]},{"osd":2,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.52700000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.622},{"interface":"front","average":{"1min":0.628,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68899999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71099999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.38900000000000001}]},{"osd":5,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.40100000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70399999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.433},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78300000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":548864,"data_stored":222760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70799999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76700000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.443,"5min":0.438,"15min":0.438},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73599999999999999,"5min":1.115,"15min":1.115},"last":0.62}]},{"osd":3,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.622,"15min":0.626},"min":{"1min":0.495,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75800000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54700000000000004,"15min":0.55600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.432},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.55700000000000005,"15min":0.56799999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36299999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56000000000000005,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69399999999999995,"5min":1.052,"15min":1.052},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73899999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63600000000000001}]},{"osd":7,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.755,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.496}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":222158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.61599999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.495,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.27300000000000002}]},{"osd":2,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61099999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60299999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.503,"15min":0.50700000000000001},"min":{"1min":0.372,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65800000000000003,"5min":1.01,"15min":1.01},"last":0.437},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.57999999999999996,"5min":1.028,"15min":1.028},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.48999999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.004,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.53900000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85299999999999998,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.53500000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51900000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81299999999999994,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.58499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92500000000000004,"5min":1.105,"15min":1.105},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.55500000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89200000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.54300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95399999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":812490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.75700000000000001,"15min":0.81699999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.94099999999999995,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.74299999999999999,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.81699999999999995,"15min":0.879},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.371},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.77800000000000002,"15min":0.83099999999999996},"min":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71399999999999997,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.47199999999999998,"15min":0.45700000000000002},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.84399999999999997,"15min":0.88400000000000001},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.9770000000000001,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.96499999999999997,"15min":1.0269999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.94599999999999995,"15min":0.98799999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.069,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.94599999999999995,"15min":0.98599999999999999},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.0350000000000001,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":1.0429999999999999,"15min":1.103},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2150000000000001,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":1.008,"15min":1.0660000000000001},"min":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1759999999999999,"5min":7.734,"15min":7.734},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.98799999999999999,"15min":1.044},"min":{"1min":0.503,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83099999999999996,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.88}]},{"osd":7,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":1,"15min":1.048},"min":{"1min":0.53100000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5369999999999999,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.96699999999999997,"15min":0.999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.1139999999999999,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":223674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29199999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.437,"5min":0.40000000000000002,"15min":0.39400000000000002},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.621,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.42399999999999999,"15min":0.41999999999999998},"min":{"1min":0.33000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.41999999999999998},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59599999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.45000000000000001}]},{"osd":4,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.504,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73899999999999999}]},{"osd":5,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97899999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.86399999999999999,"5min":1.159,"15min":1.159},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.56200000000000006,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.997,"5min":1.194,"15min":1.194},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":552960,"data_stored":222726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54100000000000004,"5min":1.46,"15min":1.46},"last":0.432},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57099999999999995,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.311}]},{"osd":1,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60499999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46999999999999997,"15min":0.47399999999999998},"min":{"1min":0.316,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59799999999999998,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.88200000000000001,"15min":0.95199999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65500000000000003,"5min":11.964,"15min":11.964},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65600000000000003,"5min":1.593,"15min":1.593},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82599999999999996,"5min":1.375,"15min":1.375},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.73199999999999998,"15min":0.746},"min":{"1min":0.48299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84399999999999997,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84099999999999997,"5min":1.536,"15min":1.536},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68500000000000005,"15min":0.69099999999999995},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84999999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.84899999999999998,"5min":1.204,"15min":1.204},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82599999999999996,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.627}]},{"osd":7,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.53200000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82499999999999996,"5min":1.339,"15min":1.339},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.53900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89400000000000002,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.71299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":552960,"data_stored":223652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70299999999999996,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.73599999999999999,"15min":0.76800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70099999999999996,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.67900000000000005,"15min":0.69699999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71999999999999997,"5min":2.145,"15min":2.145},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70399999999999996,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.69799999999999995,"15min":0.71699999999999997},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.72899999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.80100000000000005,"15min":0.83599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.751,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.68500000000000005}]},{"osd":3,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.70999999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.1739999999999999,"15min":1.268},"min":{"1min":0.49399999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.76000000000000001,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80200000000000005,"5min":0.877,"15min":0.877},"last":0.497},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45800000000000002}]},{"osd":6,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57599999999999996}]},{"osd":7,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.69999999999999996,"15min":0.71999999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85199999999999998,"5min":1.24,"15min":1.24},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":222158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.72599999999999998,"15min":0.746},"min":{"1min":0.46000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.96099999999999997,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.92500000000000004,"15min":0.97699999999999998},"min":{"1min":0.41099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78500000000000003,"5min":8.798,"15min":8.798},"last":0.624}]},{"osd":1,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":4.71,"15min":4.71},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.62,"5min":1.0029999999999999,"15min":1.0669999999999999},"min":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0189999999999999,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.497}]},{"osd":2,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.76900000000000002,"15min":0.78700000000000003},"min":{"1min":0.436,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.002,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.80800000000000005,"15min":0.83399999999999996},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.96099999999999997,"15min":1.0169999999999999},"min":{"1min":0.47699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89300000000000002,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":1.048,"15min":1.1140000000000001},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.214,"5min":10.427,"15min":10.427},"last":0.58499999999999996}]},{"osd":4,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54900000000000004,"15min":0.55800000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74399999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90800000000000003,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.623,"15min":0.628},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.44400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83399999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:23:43.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:23:43.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.882+0000 7f0f48b52700 1 Processor -- start 2024-05-08T23:23:43.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.882+0000 7f0f48b52700 1 -- start start 2024-05-08T23:23:43.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f48b52700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f440cf840 0x7f0f440af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f48b52700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0f440d0eb0 0x7f0f44161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f48b52700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f440cd980 con 0x7f0f4406a960 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f48b52700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f440ccdb0 con 0x7f0f440cf840 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f48b52700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f440cd050 con 0x7f0f440d0eb0 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f4259c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f440cf840 0x7f0f440af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f41d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0f4406a960 0x7f0f4415d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50786/0 (socket says 172.21.15.5:50786) 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f41d9b700 1 -- 172.21.15.5:0/2733771519 learned_addr learned my addr 172.21.15.5:0/2733771519 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f42d9d700 1 --2- 172.21.15.5:0/2733771519 >> v2:172.21.15.49:3300/0 conn(0x7f0f440d0eb0 0x7f0f44161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f4259c700 1 -- 172.21.15.5:0/2733771519 >> v2:172.21.15.49:3300/0 conn(0x7f0f440d0eb0 msgr2=0x7f0f44161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f4259c700 1 --2- 172.21.15.5:0/2733771519 >> v2:172.21.15.49:3300/0 conn(0x7f0f440d0eb0 0x7f0f44161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f4259c700 1 -- 172.21.15.5:0/2733771519 >> v1:172.21.15.5:6789/0 conn(0x7f0f4406a960 legacy=0x7f0f4415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f4259c700 1 -- 172.21.15.5:0/2733771519 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440cd330 con 0x7f0f440cf840 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.883+0000 7f0f4159a700 1 -- 172.21.15.5:0/2733771519 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27649158 0 0) 0x7f0f440cd980 con 0x7f0f4406a960 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f4259c700 1 --2- 172.21.15.5:0/2733771519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f440cf840 0x7f0f440af440 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0f440cd8f0 tx=0x7f0f34009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4545a658aa77c20 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f4159a700 1 -- 172.21.15.5:0/2733771519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34011830 con 0x7f0f440cf840 2024-05-08T23:23:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f4159a700 1 -- 172.21.15.5:0/2733771519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f34011990 con 0x7f0f440cf840 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f4159a700 1 -- 172.21.15.5:0/2733771519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34011d20 con 0x7f0f440cf840 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f48b52700 1 -- 172.21.15.5:0/2733771519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f440cf840 msgr2=0x7f0f440af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f48b52700 1 --2- 172.21.15.5:0/2733771519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f440cf840 0x7f0f440af440 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0f440cd8f0 tx=0x7f0f34009b20 comp rx=0 tx=0).stop 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f48b52700 1 -- 172.21.15.5:0/2733771519 shutdown_connections 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f48b52700 1 --2- 172.21.15.5:0/2733771519 >> v2:172.21.15.49:3300/0 conn(0x7f0f440d0eb0 0x7f0f44161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f48b52700 1 --2- 172.21.15.5:0/2733771519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f440cf840 0x7f0f440af440 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f48b52700 1 -- 172.21.15.5:0/2733771519 >> 172.21.15.5:0/2733771519 conn(0x7f0f440d0320 msgr2=0x7f0f440d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f48b52700 1 -- 172.21.15.5:0/2733771519 shutdown_connections 2024-05-08T23:23:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.884+0000 7f0f48b52700 1 -- 172.21.15.5:0/2733771519 wait complete. 2024-05-08T23:23:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.886+0000 7f0f48b52700 1 Processor -- start 2024-05-08T23:23:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.886+0000 7f0f48b52700 1 -- start start 2024-05-08T23:23:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f48b52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4406a960 0x7f0f4409dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f48b52700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f440cf840 0x7f0f441f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f48b52700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f440d0eb0 0x7f0f440b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f48b52700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440b3b60 con 0x7f0f4406a960 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f48b52700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440c6cb0 con 0x7f0f440cf840 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f48b52700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440b2f20 con 0x7f0f440d0eb0 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4406a960 0x7f0f4409dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4406a960 0x7f0f4409dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35726/0 (socket says 172.21.15.5:35726) 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 -- 172.21.15.5:0/1955047115 learned_addr learned my addr 172.21.15.5:0/1955047115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f41d9b700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f440cf840 0x7f0f441f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 -- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f440d0eb0 msgr2=0x7f0f440b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f42d9d700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f440d0eb0 0x7f0f440b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f440d0eb0 0x7f0f440b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 -- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f440cf840 msgr2=0x7f0f441f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f440cf840 0x7f0f441f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 -- 172.21.15.5:0/1955047115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f340094a0 con 0x7f0f4406a960 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.887+0000 7f0f4259c700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4406a960 0x7f0f4409dc80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0f340090b0 tx=0x7f0f34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.888+0000 7f0f32ffd700 1 -- 172.21.15.5:0/1955047115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340038a0 con 0x7f0f4406a960 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.888+0000 7f0f32ffd700 1 -- 172.21.15.5:0/1955047115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3401f070 con 0x7f0f4406a960 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.888+0000 7f0f32ffd700 1 -- 172.21.15.5:0/1955047115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34003c20 con 0x7f0f4406a960 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.888+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f440b31c0 con 0x7f0f4406a960 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.888+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f440b3780 con 0x7f0f4406a960 2024-05-08T23:23:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.889+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4406f220 con 0x7f0f4406a960 2024-05-08T23:23:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.889+0000 7f0f32ffd700 1 -- 172.21.15.5:0/1955047115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0f34026480 con 0x7f0f4406a960 2024-05-08T23:23:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.890+0000 7f0f32ffd700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f2406ac90 0x7f0f2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.890+0000 7f0f32ffd700 1 -- 172.21.15.5:0/1955047115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f34021070 con 0x7f0f4406a960 2024-05-08T23:23:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.890+0000 7f0f41d9b700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f2406ac90 0x7f0f2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.890+0000 7f0f41d9b700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f2406ac90 0x7f0f2406d150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0f440b58e0 tx=0x7f0f38007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:43.894+0000 7f0f32ffd700 1 -- 172.21.15.5:0/1955047115 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0f34055fa0 con 0x7f0f4406a960 2024-05-08T23:23:43.985 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:23:43.985+0000 7f5d502e2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:23:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.107+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f440b3ca0 con 0x7f0f2406ac90 2024-05-08T23:23:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.116+0000 7f0f32ffd700 1 -- 172.21.15.5:0/1955047115 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289559 (secure 0 0 0) 0x7f0f440b3ca0 con 0x7f0f2406ac90 2024-05-08T23:23:44.119 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f2406ac90 msgr2=0x7f0f2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f2406ac90 0x7f0f2406d150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0f440b58e0 tx=0x7f0f38007c90 comp rx=0 tx=0).stop 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4406a960 msgr2=0x7f0f4409dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4406a960 0x7f0f4409dc80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0f340090b0 tx=0x7f0f34004750 comp rx=0 tx=0).stop 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 shutdown_connections 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f2406ac90 0x7f0f2406d150 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f440d0eb0 0x7f0f440b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4406a960 0x7f0f4409dc80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 --2- 172.21.15.5:0/1955047115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f440cf840 0x7f0f441f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 >> 172.21.15.5:0/1955047115 conn(0x7f0f440d0320 msgr2=0x7f0f4406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 shutdown_connections 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.119+0000 7f0f48b52700 1 -- 172.21.15.5:0/1955047115 wait complete. 2024-05-08T23:23:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:23:44.133 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-05-08T23:23:43.332322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221608,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753112,"statfs":{"total":773094113280,"available":772867186688,"internally_reserved":0,"allocated":5578752,"data_stored":2962164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.326161"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147742+0000","last_change":"2024-05-08T23:23:39.490895+0000","last_active":"2024-05-08T23:23:41.147742+0000","last_peered":"2024-05-08T23:23:41.147742+0000","last_clean":"2024-05-08T23:23:41.147742+0000","last_became_active":"2024-05-08T23:23:39.490566+0000","last_became_peered":"2024-05-08T23:23:39.490566+0000","last_unstale":"2024-05-08T23:23:41.147742+0000","last_undegraded":"2024-05-08T23:23:41.147742+0000","last_fullsized":"2024-05-08T23:23:41.147742+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:30.727813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473093+0000","last_change":"2024-05-08T23:23:39.492452+0000","last_active":"2024-05-08T23:23:40.473093+0000","last_peered":"2024-05-08T23:23:40.473093+0000","last_clean":"2024-05-08T23:23:40.473093+0000","last_became_active":"2024-05-08T23:23:39.492113+0000","last_became_peered":"2024-05-08T23:23:39.492113+0000","last_unstale":"2024-05-08T23:23:40.473093+0000","last_undegraded":"2024-05-08T23:23:40.473093+0000","last_fullsized":"2024-05-08T23:23:40.473093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:54.440871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472301+0000","last_change":"2024-05-08T23:23:39.493541+0000","last_active":"2024-05-08T23:23:40.472301+0000","last_peered":"2024-05-08T23:23:40.472301+0000","last_clean":"2024-05-08T23:23:40.472301+0000","last_became_active":"2024-05-08T23:23:39.493248+0000","last_became_peered":"2024-05-08T23:23:39.493248+0000","last_unstale":"2024-05-08T23:23:40.472301+0000","last_undegraded":"2024-05-08T23:23:40.472301+0000","last_fullsized":"2024-05-08T23:23:40.472301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:10.500447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009609+0000","last_change":"2024-05-08T23:23:39.491285+0000","last_active":"2024-05-08T23:23:41.009609+0000","last_peered":"2024-05-08T23:23:41.009609+0000","last_clean":"2024-05-08T23:23:41.009609+0000","last_became_active":"2024-05-08T23:23:39.491004+0000","last_became_peered":"2024-05-08T23:23:39.491004+0000","last_unstale":"2024-05-08T23:23:41.009609+0000","last_undegraded":"2024-05-08T23:23:41.009609+0000","last_fullsized":"2024-05-08T23:23:41.009609+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:37.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472980+0000","last_change":"2024-05-08T23:23:39.492169+0000","last_active":"2024-05-08T23:23:40.472980+0000","last_peered":"2024-05-08T23:23:40.472980+0000","last_clean":"2024-05-08T23:23:40.472980+0000","last_became_active":"2024-05-08T23:23:39.491823+0000","last_became_peered":"2024-05-08T23:23:39.491823+0000","last_unstale":"2024-05-08T23:23:40.472980+0000","last_undegraded":"2024-05-08T23:23:40.472980+0000","last_fullsized":"2024-05-08T23:23:40.472980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:29.828185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471433+0000","last_change":"2024-05-08T23:23:39.485976+0000","last_active":"2024-05-08T23:23:40.471433+0000","last_peered":"2024-05-08T23:23:40.471433+0000","last_clean":"2024-05-08T23:23:40.471433+0000","last_became_active":"2024-05-08T23:23:39.485676+0000","last_became_peered":"2024-05-08T23:23:39.485676+0000","last_unstale":"2024-05-08T23:23:40.471433+0000","last_undegraded":"2024-05-08T23:23:40.471433+0000","last_fullsized":"2024-05-08T23:23:40.471433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:56.158345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147741+0000","last_change":"2024-05-08T23:23:39.497134+0000","last_active":"2024-05-08T23:23:41.147741+0000","last_peered":"2024-05-08T23:23:41.147741+0000","last_clean":"2024-05-08T23:23:41.147741+0000","last_became_active":"2024-05-08T23:23:39.496842+0000","last_became_peered":"2024-05-08T23:23:39.496842+0000","last_unstale":"2024-05-08T23:23:41.147741+0000","last_undegraded":"2024-05-08T23:23:41.147741+0000","last_fullsized":"2024-05-08T23:23:41.147741+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:17.076074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009163+0000","last_change":"2024-05-08T23:23:39.491247+0000","last_active":"2024-05-08T23:23:41.009163+0000","last_peered":"2024-05-08T23:23:41.009163+0000","last_clean":"2024-05-08T23:23:41.009163+0000","last_became_active":"2024-05-08T23:23:39.490867+0000","last_became_peered":"2024-05-08T23:23:39.490867+0000","last_unstale":"2024-05-08T23:23:41.009163+0000","last_undegraded":"2024-05-08T23:23:41.009163+0000","last_fullsized":"2024-05-08T23:23:41.009163+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:05.858704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009830+0000","last_change":"2024-05-08T23:23:39.495008+0000","last_active":"2024-05-08T23:23:41.009830+0000","last_peered":"2024-05-08T23:23:41.009830+0000","last_clean":"2024-05-08T23:23:41.009830+0000","last_became_active":"2024-05-08T23:23:39.490557+0000","last_became_peered":"2024-05-08T23:23:39.490557+0000","last_unstale":"2024-05-08T23:23:41.009830+0000","last_undegraded":"2024-05-08T23:23:41.009830+0000","last_fullsized":"2024-05-08T23:23:41.009830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:41.299209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009588+0000","last_change":"2024-05-08T23:23:39.493878+0000","last_active":"2024-05-08T23:23:41.009588+0000","last_peered":"2024-05-08T23:23:41.009588+0000","last_clean":"2024-05-08T23:23:41.009588+0000","last_became_active":"2024-05-08T23:23:39.490794+0000","last_became_peered":"2024-05-08T23:23:39.490794+0000","last_unstale":"2024-05-08T23:23:41.009588+0000","last_undegraded":"2024-05-08T23:23:41.009588+0000","last_fullsized":"2024-05-08T23:23:41.009588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:22.791683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009491+0000","last_change":"2024-05-08T23:23:39.501623+0000","last_active":"2024-05-08T23:23:41.009491+0000","last_peered":"2024-05-08T23:23:41.009491+0000","last_clean":"2024-05-08T23:23:41.009491+0000","last_became_active":"2024-05-08T23:23:39.501282+0000","last_became_peered":"2024-05-08T23:23:39.501282+0000","last_unstale":"2024-05-08T23:23:41.009491+0000","last_undegraded":"2024-05-08T23:23:41.009491+0000","last_fullsized":"2024-05-08T23:23:41.009491+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:34.367633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473165+0000","last_change":"2024-05-08T23:23:39.493054+0000","last_active":"2024-05-08T23:23:40.473165+0000","last_peered":"2024-05-08T23:23:40.473165+0000","last_clean":"2024-05-08T23:23:40.473165+0000","last_became_active":"2024-05-08T23:23:39.492087+0000","last_became_peered":"2024-05-08T23:23:39.492087+0000","last_unstale":"2024-05-08T23:23:40.473165+0000","last_undegraded":"2024-05-08T23:23:40.473165+0000","last_fullsized":"2024-05-08T23:23:40.473165+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.768894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646460+0000","last_change":"2024-05-08T23:23:39.493024+0000","last_active":"2024-05-08T23:23:40.646460+0000","last_peered":"2024-05-08T23:23:40.646460+0000","last_clean":"2024-05-08T23:23:40.646460+0000","last_became_active":"2024-05-08T23:23:39.492772+0000","last_became_peered":"2024-05-08T23:23:39.492772+0000","last_unstale":"2024-05-08T23:23:40.646460+0000","last_undegraded":"2024-05-08T23:23:40.646460+0000","last_fullsized":"2024-05-08T23:23:40.646460+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:49.200126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009425+0000","last_change":"2024-05-08T23:23:39.494669+0000","last_active":"2024-05-08T23:23:41.009425+0000","last_peered":"2024-05-08T23:23:41.009425+0000","last_clean":"2024-05-08T23:23:41.009425+0000","last_became_active":"2024-05-08T23:23:39.494231+0000","last_became_peered":"2024-05-08T23:23:39.494231+0000","last_unstale":"2024-05-08T23:23:41.009425+0000","last_undegraded":"2024-05-08T23:23:41.009425+0000","last_fullsized":"2024-05-08T23:23:41.009425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:01.672017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471304+0000","last_change":"2024-05-08T23:23:39.476475+0000","last_active":"2024-05-08T23:23:40.471304+0000","last_peered":"2024-05-08T23:23:40.471304+0000","last_clean":"2024-05-08T23:23:40.471304+0000","last_became_active":"2024-05-08T23:23:39.476208+0000","last_became_peered":"2024-05-08T23:23:39.476208+0000","last_unstale":"2024-05-08T23:23:40.471304+0000","last_undegraded":"2024-05-08T23:23:40.471304+0000","last_fullsized":"2024-05-08T23:23:40.471304+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:30.372964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148606+0000","last_change":"2024-05-08T23:23:39.493063+0000","last_active":"2024-05-08T23:23:41.148606+0000","last_peered":"2024-05-08T23:23:41.148606+0000","last_clean":"2024-05-08T23:23:41.148606+0000","last_became_active":"2024-05-08T23:23:39.492784+0000","last_became_peered":"2024-05-08T23:23:39.492784+0000","last_unstale":"2024-05-08T23:23:41.148606+0000","last_undegraded":"2024-05-08T23:23:41.148606+0000","last_fullsized":"2024-05-08T23:23:41.148606+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:19.521622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472262+0000","last_change":"2024-05-08T23:23:39.475820+0000","last_active":"2024-05-08T23:23:40.472262+0000","last_peered":"2024-05-08T23:23:40.472262+0000","last_clean":"2024-05-08T23:23:40.472262+0000","last_became_active":"2024-05-08T23:23:39.475518+0000","last_became_peered":"2024-05-08T23:23:39.475518+0000","last_unstale":"2024-05-08T23:23:40.472262+0000","last_undegraded":"2024-05-08T23:23:40.472262+0000","last_fullsized":"2024-05-08T23:23:40.472262+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:54.950277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472254+0000","last_change":"2024-05-08T23:23:39.493653+0000","last_active":"2024-05-08T23:23:40.472254+0000","last_peered":"2024-05-08T23:23:40.472254+0000","last_clean":"2024-05-08T23:23:40.472254+0000","last_became_active":"2024-05-08T23:23:39.493349+0000","last_became_peered":"2024-05-08T23:23:39.493349+0000","last_unstale":"2024-05-08T23:23:40.472254+0000","last_undegraded":"2024-05-08T23:23:40.472254+0000","last_fullsized":"2024-05-08T23:23:40.472254+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:22.791683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148995+0000","last_change":"2024-05-08T23:23:39.498212+0000","last_active":"2024-05-08T23:23:41.148995+0000","last_peered":"2024-05-08T23:23:41.148995+0000","last_clean":"2024-05-08T23:23:41.148995+0000","last_became_active":"2024-05-08T23:23:39.497906+0000","last_became_peered":"2024-05-08T23:23:39.497906+0000","last_unstale":"2024-05-08T23:23:41.148995+0000","last_undegraded":"2024-05-08T23:23:41.148995+0000","last_fullsized":"2024-05-08T23:23:41.148995+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:50:18.129675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646609+0000","last_change":"2024-05-08T23:23:39.495322+0000","last_active":"2024-05-08T23:23:40.646609+0000","last_peered":"2024-05-08T23:23:40.646609+0000","last_clean":"2024-05-08T23:23:40.646609+0000","last_became_active":"2024-05-08T23:23:39.495075+0000","last_became_peered":"2024-05-08T23:23:39.495075+0000","last_unstale":"2024-05-08T23:23:40.646609+0000","last_undegraded":"2024-05-08T23:23:40.646609+0000","last_fullsized":"2024-05-08T23:23:40.646609+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:03.099478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471445+0000","last_change":"2024-05-08T23:23:39.484588+0000","last_active":"2024-05-08T23:23:40.471445+0000","last_peered":"2024-05-08T23:23:40.471445+0000","last_clean":"2024-05-08T23:23:40.471445+0000","last_became_active":"2024-05-08T23:23:39.483740+0000","last_became_peered":"2024-05-08T23:23:39.483740+0000","last_unstale":"2024-05-08T23:23:40.471445+0000","last_undegraded":"2024-05-08T23:23:40.471445+0000","last_fullsized":"2024-05-08T23:23:40.471445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:28.984936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009440+0000","last_change":"2024-05-08T23:23:39.488733+0000","last_active":"2024-05-08T23:23:41.009440+0000","last_peered":"2024-05-08T23:23:41.009440+0000","last_clean":"2024-05-08T23:23:41.009440+0000","last_became_active":"2024-05-08T23:23:39.488420+0000","last_became_peered":"2024-05-08T23:23:39.488420+0000","last_unstale":"2024-05-08T23:23:41.009440+0000","last_undegraded":"2024-05-08T23:23:41.009440+0000","last_fullsized":"2024-05-08T23:23:41.009440+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:30.727813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148615+0000","last_change":"2024-05-08T23:23:39.486406+0000","last_active":"2024-05-08T23:23:41.148615+0000","last_peered":"2024-05-08T23:23:41.148615+0000","last_clean":"2024-05-08T23:23:41.148615+0000","last_became_active":"2024-05-08T23:23:39.486062+0000","last_became_peered":"2024-05-08T23:23:39.486062+0000","last_unstale":"2024-05-08T23:23:41.148615+0000","last_undegraded":"2024-05-08T23:23:41.148615+0000","last_fullsized":"2024-05-08T23:23:41.148615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:16.785894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148433+0000","last_change":"2024-05-08T23:23:39.485950+0000","last_active":"2024-05-08T23:23:41.148433+0000","last_peered":"2024-05-08T23:23:41.148433+0000","last_clean":"2024-05-08T23:23:41.148433+0000","last_became_active":"2024-05-08T23:23:39.485649+0000","last_became_peered":"2024-05-08T23:23:39.485649+0000","last_unstale":"2024-05-08T23:23:41.148433+0000","last_undegraded":"2024-05-08T23:23:41.148433+0000","last_fullsized":"2024-05-08T23:23:41.148433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:25.353577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.149124+0000","last_change":"2024-05-08T23:23:39.488452+0000","last_active":"2024-05-08T23:23:41.149124+0000","last_peered":"2024-05-08T23:23:41.149124+0000","last_clean":"2024-05-08T23:23:41.149124+0000","last_became_active":"2024-05-08T23:23:39.488141+0000","last_became_peered":"2024-05-08T23:23:39.488141+0000","last_unstale":"2024-05-08T23:23:41.149124+0000","last_undegraded":"2024-05-08T23:23:41.149124+0000","last_fullsized":"2024-05-08T23:23:41.149124+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:56.655941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646603+0000","last_change":"2024-05-08T23:23:39.488603+0000","last_active":"2024-05-08T23:23:40.646603+0000","last_peered":"2024-05-08T23:23:40.646603+0000","last_clean":"2024-05-08T23:23:40.646603+0000","last_became_active":"2024-05-08T23:23:39.488166+0000","last_became_peered":"2024-05-08T23:23:39.488166+0000","last_unstale":"2024-05-08T23:23:40.646603+0000","last_undegraded":"2024-05-08T23:23:40.646603+0000","last_fullsized":"2024-05-08T23:23:40.646603+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:38:50.581266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472419+0000","last_change":"2024-05-08T23:23:39.475898+0000","last_active":"2024-05-08T23:23:40.472419+0000","last_peered":"2024-05-08T23:23:40.472419+0000","last_clean":"2024-05-08T23:23:40.472419+0000","last_became_active":"2024-05-08T23:23:39.475601+0000","last_became_peered":"2024-05-08T23:23:39.475601+0000","last_unstale":"2024-05-08T23:23:40.472419+0000","last_undegraded":"2024-05-08T23:23:40.472419+0000","last_fullsized":"2024-05-08T23:23:40.472419+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:32.042955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472895+0000","last_change":"2024-05-08T23:23:39.496005+0000","last_active":"2024-05-08T23:23:40.472895+0000","last_peered":"2024-05-08T23:23:40.472895+0000","last_clean":"2024-05-08T23:23:40.472895+0000","last_became_active":"2024-05-08T23:23:39.495685+0000","last_became_peered":"2024-05-08T23:23:39.495685+0000","last_unstale":"2024-05-08T23:23:40.472895+0000","last_undegraded":"2024-05-08T23:23:40.472895+0000","last_fullsized":"2024-05-08T23:23:40.472895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:59.158101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148721+0000","last_change":"2024-05-08T23:23:39.497512+0000","last_active":"2024-05-08T23:23:41.148721+0000","last_peered":"2024-05-08T23:23:41.148721+0000","last_clean":"2024-05-08T23:23:41.148721+0000","last_became_active":"2024-05-08T23:23:39.497222+0000","last_became_peered":"2024-05-08T23:23:39.497222+0000","last_unstale":"2024-05-08T23:23:41.148721+0000","last_undegraded":"2024-05-08T23:23:41.148721+0000","last_fullsized":"2024-05-08T23:23:41.148721+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:07.789294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472114+0000","last_change":"2024-05-08T23:23:39.487156+0000","last_active":"2024-05-08T23:23:40.472114+0000","last_peered":"2024-05-08T23:23:40.472114+0000","last_clean":"2024-05-08T23:23:40.472114+0000","last_became_active":"2024-05-08T23:23:39.486861+0000","last_became_peered":"2024-05-08T23:23:39.486861+0000","last_unstale":"2024-05-08T23:23:40.472114+0000","last_undegraded":"2024-05-08T23:23:40.472114+0000","last_fullsized":"2024-05-08T23:23:40.472114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.768894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471372+0000","last_change":"2024-05-08T23:23:39.475937+0000","last_active":"2024-05-08T23:23:40.471372+0000","last_peered":"2024-05-08T23:23:40.471372+0000","last_clean":"2024-05-08T23:23:40.471372+0000","last_became_active":"2024-05-08T23:23:39.475672+0000","last_became_peered":"2024-05-08T23:23:39.475672+0000","last_unstale":"2024-05-08T23:23:40.471372+0000","last_undegraded":"2024-05-08T23:23:40.471372+0000","last_fullsized":"2024-05-08T23:23:40.471372+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:28.786132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646394+0000","last_change":"2024-05-08T23:23:39.498230+0000","last_active":"2024-05-08T23:23:40.646394+0000","last_peered":"2024-05-08T23:23:40.646394+0000","last_clean":"2024-05-08T23:23:40.646394+0000","last_became_active":"2024-05-08T23:23:39.497859+0000","last_became_peered":"2024-05-08T23:23:39.497859+0000","last_unstale":"2024-05-08T23:23:40.646394+0000","last_undegraded":"2024-05-08T23:23:40.646394+0000","last_fullsized":"2024-05-08T23:23:40.646394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:35.068859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688592+0000","last_change":"2024-05-08T23:23:39.489075+0000","last_active":"2024-05-08T23:23:40.688592+0000","last_peered":"2024-05-08T23:23:40.688592+0000","last_clean":"2024-05-08T23:23:40.688592+0000","last_became_active":"2024-05-08T23:23:39.488806+0000","last_became_peered":"2024-05-08T23:23:39.488806+0000","last_unstale":"2024-05-08T23:23:40.688592+0000","last_undegraded":"2024-05-08T23:23:40.688592+0000","last_fullsized":"2024-05-08T23:23:40.688592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:33.959563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471976+0000","last_change":"2024-05-08T23:23:39.017929+0000","last_active":"2024-05-08T23:23:40.471976+0000","last_peered":"2024-05-08T23:23:40.471976+0000","last_clean":"2024-05-08T23:23:40.471976+0000","last_became_active":"2024-05-08T23:23:39.017608+0000","last_became_peered":"2024-05-08T23:23:39.017608+0000","last_unstale":"2024-05-08T23:23:40.471976+0000","last_undegraded":"2024-05-08T23:23:40.471976+0000","last_fullsized":"2024-05-08T23:23:40.471976+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:36.357850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148994+0000","last_change":"2024-05-08T23:23:39.484978+0000","last_active":"2024-05-08T23:23:41.148994+0000","last_peered":"2024-05-08T23:23:41.148994+0000","last_clean":"2024-05-08T23:23:41.148994+0000","last_became_active":"2024-05-08T23:23:39.484683+0000","last_became_peered":"2024-05-08T23:23:39.484683+0000","last_unstale":"2024-05-08T23:23:41.148994+0000","last_undegraded":"2024-05-08T23:23:41.148994+0000","last_fullsized":"2024-05-08T23:23:41.148994+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:07.161103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472673+0000","last_change":"2024-05-08T23:23:39.026823+0000","last_active":"2024-05-08T23:23:40.472673+0000","last_peered":"2024-05-08T23:23:40.472673+0000","last_clean":"2024-05-08T23:23:40.472673+0000","last_became_active":"2024-05-08T23:23:39.026546+0000","last_became_peered":"2024-05-08T23:23:39.026546+0000","last_unstale":"2024-05-08T23:23:40.472673+0000","last_undegraded":"2024-05-08T23:23:40.472673+0000","last_fullsized":"2024-05-08T23:23:40.472673+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:10.901466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473026+0000","last_change":"2024-05-08T23:23:39.478089+0000","last_active":"2024-05-08T23:23:40.473026+0000","last_peered":"2024-05-08T23:23:40.473026+0000","last_clean":"2024-05-08T23:23:40.473026+0000","last_became_active":"2024-05-08T23:23:39.477835+0000","last_became_peered":"2024-05-08T23:23:39.477835+0000","last_unstale":"2024-05-08T23:23:40.473026+0000","last_undegraded":"2024-05-08T23:23:40.473026+0000","last_fullsized":"2024-05-08T23:23:40.473026+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:22.791683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688684+0000","last_change":"2024-05-08T23:23:39.028361+0000","last_active":"2024-05-08T23:23:40.688684+0000","last_peered":"2024-05-08T23:23:40.688684+0000","last_clean":"2024-05-08T23:23:40.688684+0000","last_became_active":"2024-05-08T23:23:39.028090+0000","last_became_peered":"2024-05-08T23:23:39.028090+0000","last_unstale":"2024-05-08T23:23:40.688684+0000","last_undegraded":"2024-05-08T23:23:40.688684+0000","last_fullsized":"2024-05-08T23:23:40.688684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:34.063977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688553+0000","last_change":"2024-05-08T23:23:39.497907+0000","last_active":"2024-05-08T23:23:40.688553+0000","last_peered":"2024-05-08T23:23:40.688553+0000","last_clean":"2024-05-08T23:23:40.688553+0000","last_became_active":"2024-05-08T23:23:39.494619+0000","last_became_peered":"2024-05-08T23:23:39.494619+0000","last_unstale":"2024-05-08T23:23:40.688553+0000","last_undegraded":"2024-05-08T23:23:40.688553+0000","last_fullsized":"2024-05-08T23:23:40.688553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:07.161103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147559+0000","last_change":"2024-05-08T23:23:39.028246+0000","last_active":"2024-05-08T23:23:41.147559+0000","last_peered":"2024-05-08T23:23:41.147559+0000","last_clean":"2024-05-08T23:23:41.147559+0000","last_became_active":"2024-05-08T23:23:39.027931+0000","last_became_peered":"2024-05-08T23:23:39.027931+0000","last_unstale":"2024-05-08T23:23:41.147559+0000","last_undegraded":"2024-05-08T23:23:41.147559+0000","last_fullsized":"2024-05-08T23:23:41.147559+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:43.383813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472220+0000","last_change":"2024-05-08T23:23:39.483343+0000","last_active":"2024-05-08T23:23:40.472220+0000","last_peered":"2024-05-08T23:23:40.472220+0000","last_clean":"2024-05-08T23:23:40.472220+0000","last_became_active":"2024-05-08T23:23:39.483034+0000","last_became_peered":"2024-05-08T23:23:39.483034+0000","last_unstale":"2024-05-08T23:23:40.472220+0000","last_undegraded":"2024-05-08T23:23:40.472220+0000","last_fullsized":"2024-05-08T23:23:40.472220+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:29.828185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471631+0000","last_change":"2024-05-08T23:23:39.029649+0000","last_active":"2024-05-08T23:23:40.471631+0000","last_peered":"2024-05-08T23:23:40.471631+0000","last_clean":"2024-05-08T23:23:40.471631+0000","last_became_active":"2024-05-08T23:23:39.028984+0000","last_became_peered":"2024-05-08T23:23:39.028984+0000","last_unstale":"2024-05-08T23:23:40.471631+0000","last_undegraded":"2024-05-08T23:23:40.471631+0000","last_fullsized":"2024-05-08T23:23:40.471631+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:07.682195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009657+0000","last_change":"2024-05-08T23:23:39.487414+0000","last_active":"2024-05-08T23:23:41.009657+0000","last_peered":"2024-05-08T23:23:41.009657+0000","last_clean":"2024-05-08T23:23:41.009657+0000","last_became_active":"2024-05-08T23:23:39.487140+0000","last_became_peered":"2024-05-08T23:23:39.487140+0000","last_unstale":"2024-05-08T23:23:41.009657+0000","last_undegraded":"2024-05-08T23:23:41.009657+0000","last_fullsized":"2024-05-08T23:23:41.009657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:54.440871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472345+0000","last_change":"2024-05-08T23:23:39.027821+0000","last_active":"2024-05-08T23:23:40.472345+0000","last_peered":"2024-05-08T23:23:40.472345+0000","last_clean":"2024-05-08T23:23:40.472345+0000","last_became_active":"2024-05-08T23:23:39.027505+0000","last_became_peered":"2024-05-08T23:23:39.027505+0000","last_unstale":"2024-05-08T23:23:40.472345+0000","last_undegraded":"2024-05-08T23:23:40.472345+0000","last_fullsized":"2024-05-08T23:23:40.472345+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:24.348695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009788+0000","last_change":"2024-05-08T23:23:39.491086+0000","last_active":"2024-05-08T23:23:41.009788+0000","last_peered":"2024-05-08T23:23:41.009788+0000","last_clean":"2024-05-08T23:23:41.009788+0000","last_became_active":"2024-05-08T23:23:39.490748+0000","last_became_peered":"2024-05-08T23:23:39.490748+0000","last_unstale":"2024-05-08T23:23:41.009788+0000","last_undegraded":"2024-05-08T23:23:41.009788+0000","last_fullsized":"2024-05-08T23:23:41.009788+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:28.729823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472813+0000","last_change":"2024-05-08T23:23:39.027215+0000","last_active":"2024-05-08T23:23:40.472813+0000","last_peered":"2024-05-08T23:23:40.472813+0000","last_clean":"2024-05-08T23:23:40.472813+0000","last_became_active":"2024-05-08T23:23:39.026917+0000","last_became_peered":"2024-05-08T23:23:39.026917+0000","last_unstale":"2024-05-08T23:23:40.472813+0000","last_undegraded":"2024-05-08T23:23:40.472813+0000","last_fullsized":"2024-05-08T23:23:40.472813+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:55.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148664+0000","last_change":"2024-05-08T23:23:39.486146+0000","last_active":"2024-05-08T23:23:41.148664+0000","last_peered":"2024-05-08T23:23:41.148664+0000","last_clean":"2024-05-08T23:23:41.148664+0000","last_became_active":"2024-05-08T23:23:39.485849+0000","last_became_peered":"2024-05-08T23:23:39.485849+0000","last_unstale":"2024-05-08T23:23:41.148664+0000","last_undegraded":"2024-05-08T23:23:41.148664+0000","last_fullsized":"2024-05-08T23:23:41.148664+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:11.906348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009420+0000","last_change":"2024-05-08T23:23:39.030521+0000","last_active":"2024-05-08T23:23:41.009420+0000","last_peered":"2024-05-08T23:23:41.009420+0000","last_clean":"2024-05-08T23:23:41.009420+0000","last_became_active":"2024-05-08T23:23:39.030248+0000","last_became_peered":"2024-05-08T23:23:39.030248+0000","last_unstale":"2024-05-08T23:23:41.009420+0000","last_undegraded":"2024-05-08T23:23:41.009420+0000","last_fullsized":"2024-05-08T23:23:41.009420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:40.828807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472568+0000","last_change":"2024-05-08T23:23:39.491842+0000","last_active":"2024-05-08T23:23:40.472568+0000","last_peered":"2024-05-08T23:23:40.472568+0000","last_clean":"2024-05-08T23:23:40.472568+0000","last_became_active":"2024-05-08T23:23:39.491557+0000","last_became_peered":"2024-05-08T23:23:39.491557+0000","last_unstale":"2024-05-08T23:23:40.472568+0000","last_undegraded":"2024-05-08T23:23:40.472568+0000","last_fullsized":"2024-05-08T23:23:40.472568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:54.440871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472575+0000","last_change":"2024-05-08T23:23:39.027185+0000","last_active":"2024-05-08T23:23:40.472575+0000","last_peered":"2024-05-08T23:23:40.472575+0000","last_clean":"2024-05-08T23:23:40.472575+0000","last_became_active":"2024-05-08T23:23:39.026893+0000","last_became_peered":"2024-05-08T23:23:39.026893+0000","last_unstale":"2024-05-08T23:23:40.472575+0000","last_undegraded":"2024-05-08T23:23:40.472575+0000","last_fullsized":"2024-05-08T23:23:40.472575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:08:55.318230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147884+0000","last_change":"2024-05-08T23:23:39.490848+0000","last_active":"2024-05-08T23:23:41.147884+0000","last_peered":"2024-05-08T23:23:41.147884+0000","last_clean":"2024-05-08T23:23:41.147884+0000","last_became_active":"2024-05-08T23:23:39.490543+0000","last_became_peered":"2024-05-08T23:23:39.490543+0000","last_unstale":"2024-05-08T23:23:41.147884+0000","last_undegraded":"2024-05-08T23:23:41.147884+0000","last_fullsized":"2024-05-08T23:23:41.147884+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:08.687077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009859+0000","last_change":"2024-05-08T23:23:39.031385+0000","last_active":"2024-05-08T23:23:41.009859+0000","last_peered":"2024-05-08T23:23:41.009859+0000","last_clean":"2024-05-08T23:23:41.009859+0000","last_became_active":"2024-05-08T23:23:39.031089+0000","last_became_peered":"2024-05-08T23:23:39.031089+0000","last_unstale":"2024-05-08T23:23:41.009859+0000","last_undegraded":"2024-05-08T23:23:41.009859+0000","last_fullsized":"2024-05-08T23:23:41.009859+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:15.129335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471766+0000","last_change":"2024-05-08T23:23:39.492475+0000","last_active":"2024-05-08T23:23:40.471766+0000","last_peered":"2024-05-08T23:23:40.471766+0000","last_clean":"2024-05-08T23:23:40.471766+0000","last_became_active":"2024-05-08T23:23:39.492169+0000","last_became_peered":"2024-05-08T23:23:39.492169+0000","last_unstale":"2024-05-08T23:23:40.471766+0000","last_undegraded":"2024-05-08T23:23:40.471766+0000","last_fullsized":"2024-05-08T23:23:40.471766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:59.343711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147749+0000","last_change":"2024-05-08T23:23:39.028065+0000","last_active":"2024-05-08T23:23:41.147749+0000","last_peered":"2024-05-08T23:23:41.147749+0000","last_clean":"2024-05-08T23:23:41.147749+0000","last_became_active":"2024-05-08T23:23:39.027786+0000","last_became_peered":"2024-05-08T23:23:39.027786+0000","last_unstale":"2024-05-08T23:23:41.147749+0000","last_undegraded":"2024-05-08T23:23:41.147749+0000","last_fullsized":"2024-05-08T23:23:41.147749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:53.435989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147630+0000","last_change":"2024-05-08T23:23:39.500371+0000","last_active":"2024-05-08T23:23:41.147630+0000","last_peered":"2024-05-08T23:23:41.147630+0000","last_clean":"2024-05-08T23:23:41.147630+0000","last_became_active":"2024-05-08T23:23:39.499995+0000","last_became_peered":"2024-05-08T23:23:39.499995+0000","last_unstale":"2024-05-08T23:23:41.147630+0000","last_undegraded":"2024-05-08T23:23:41.147630+0000","last_fullsized":"2024-05-08T23:23:41.147630+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:27.581561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646658+0000","last_change":"2024-05-08T23:23:39.027079+0000","last_active":"2024-05-08T23:23:40.646658+0000","last_peered":"2024-05-08T23:23:40.646658+0000","last_clean":"2024-05-08T23:23:40.646658+0000","last_became_active":"2024-05-08T23:23:39.026789+0000","last_became_peered":"2024-05-08T23:23:39.026789+0000","last_unstale":"2024-05-08T23:23:40.646658+0000","last_undegraded":"2024-05-08T23:23:40.646658+0000","last_fullsized":"2024-05-08T23:23:40.646658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:03.713283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148758+0000","last_change":"2024-05-08T23:23:39.498249+0000","last_active":"2024-05-08T23:23:41.148758+0000","last_peered":"2024-05-08T23:23:41.148758+0000","last_clean":"2024-05-08T23:23:41.148758+0000","last_became_active":"2024-05-08T23:23:39.497963+0000","last_became_peered":"2024-05-08T23:23:39.497963+0000","last_unstale":"2024-05-08T23:23:41.148758+0000","last_undegraded":"2024-05-08T23:23:41.148758+0000","last_fullsized":"2024-05-08T23:23:41.148758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:06.330086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471741+0000","last_change":"2024-05-08T23:23:39.025640+0000","last_active":"2024-05-08T23:23:40.471741+0000","last_peered":"2024-05-08T23:23:40.471741+0000","last_clean":"2024-05-08T23:23:40.471741+0000","last_became_active":"2024-05-08T23:23:39.025390+0000","last_became_peered":"2024-05-08T23:23:39.025390+0000","last_unstale":"2024-05-08T23:23:40.471741+0000","last_undegraded":"2024-05-08T23:23:40.471741+0000","last_fullsized":"2024-05-08T23:23:40.471741+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:04.853822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009747+0000","last_change":"2024-05-08T23:23:39.495146+0000","last_active":"2024-05-08T23:23:41.009747+0000","last_peered":"2024-05-08T23:23:41.009747+0000","last_clean":"2024-05-08T23:23:41.009747+0000","last_became_active":"2024-05-08T23:23:39.494844+0000","last_became_peered":"2024-05-08T23:23:39.494844+0000","last_unstale":"2024-05-08T23:23:41.009747+0000","last_undegraded":"2024-05-08T23:23:41.009747+0000","last_fullsized":"2024-05-08T23:23:41.009747+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:59.158101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147553+0000","last_change":"2024-05-08T23:23:39.028638+0000","last_active":"2024-05-08T23:23:41.147553+0000","last_peered":"2024-05-08T23:23:41.147553+0000","last_clean":"2024-05-08T23:23:41.147553+0000","last_became_active":"2024-05-08T23:23:39.028360+0000","last_became_peered":"2024-05-08T23:23:39.028360+0000","last_unstale":"2024-05-08T23:23:41.147553+0000","last_undegraded":"2024-05-08T23:23:41.147553+0000","last_fullsized":"2024-05-08T23:23:41.147553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:28.823303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471271+0000","last_change":"2024-05-08T23:23:39.493388+0000","last_active":"2024-05-08T23:23:40.471271+0000","last_peered":"2024-05-08T23:23:40.471271+0000","last_clean":"2024-05-08T23:23:40.471271+0000","last_became_active":"2024-05-08T23:23:39.493091+0000","last_became_peered":"2024-05-08T23:23:39.493091+0000","last_unstale":"2024-05-08T23:23:40.471271+0000","last_undegraded":"2024-05-08T23:23:40.471271+0000","last_fullsized":"2024-05-08T23:23:40.471271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:05.335477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471413+0000","last_change":"2024-05-08T23:23:39.025294+0000","last_active":"2024-05-08T23:23:40.471413+0000","last_peered":"2024-05-08T23:23:40.471413+0000","last_clean":"2024-05-08T23:23:40.471413+0000","last_became_active":"2024-05-08T23:23:39.024959+0000","last_became_peered":"2024-05-08T23:23:39.024959+0000","last_unstale":"2024-05-08T23:23:40.471413+0000","last_undegraded":"2024-05-08T23:23:40.471413+0000","last_fullsized":"2024-05-08T23:23:40.471413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:29.722931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147662+0000","last_change":"2024-05-08T23:23:39.491344+0000","last_active":"2024-05-08T23:23:41.147662+0000","last_peered":"2024-05-08T23:23:41.147662+0000","last_clean":"2024-05-08T23:23:41.147662+0000","last_became_active":"2024-05-08T23:23:39.487190+0000","last_became_peered":"2024-05-08T23:23:39.487190+0000","last_unstale":"2024-05-08T23:23:41.147662+0000","last_undegraded":"2024-05-08T23:23:41.147662+0000","last_fullsized":"2024-05-08T23:23:41.147662+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:54.680549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009305+0000","last_change":"2024-05-08T23:23:39.031326+0000","last_active":"2024-05-08T23:23:41.009305+0000","last_peered":"2024-05-08T23:23:41.009305+0000","last_clean":"2024-05-08T23:23:41.009305+0000","last_became_active":"2024-05-08T23:23:39.030967+0000","last_became_peered":"2024-05-08T23:23:39.030967+0000","last_unstale":"2024-05-08T23:23:41.009305+0000","last_undegraded":"2024-05-08T23:23:41.009305+0000","last_fullsized":"2024-05-08T23:23:41.009305+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:07.056231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646653+0000","last_change":"2024-05-08T23:23:39.488942+0000","last_active":"2024-05-08T23:23:40.646653+0000","last_peered":"2024-05-08T23:23:40.646653+0000","last_clean":"2024-05-08T23:23:40.646653+0000","last_became_active":"2024-05-08T23:23:39.488621+0000","last_became_peered":"2024-05-08T23:23:39.488621+0000","last_unstale":"2024-05-08T23:23:40.646653+0000","last_undegraded":"2024-05-08T23:23:40.646653+0000","last_fullsized":"2024-05-08T23:23:40.646653+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:06.309720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646661+0000","last_change":"2024-05-08T23:23:39.015694+0000","last_active":"2024-05-08T23:23:40.646661+0000","last_peered":"2024-05-08T23:23:40.646661+0000","last_clean":"2024-05-08T23:23:40.646661+0000","last_became_active":"2024-05-08T23:23:39.015375+0000","last_became_peered":"2024-05-08T23:23:39.015375+0000","last_unstale":"2024-05-08T23:23:40.646661+0000","last_undegraded":"2024-05-08T23:23:40.646661+0000","last_fullsized":"2024-05-08T23:23:40.646661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:13.429089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009872+0000","last_change":"2024-05-08T23:23:39.480382+0000","last_active":"2024-05-08T23:23:41.009872+0000","last_peered":"2024-05-08T23:23:41.009872+0000","last_clean":"2024-05-08T23:23:41.009872+0000","last_became_active":"2024-05-08T23:23:39.480099+0000","last_became_peered":"2024-05-08T23:23:39.480099+0000","last_unstale":"2024-05-08T23:23:41.009872+0000","last_undegraded":"2024-05-08T23:23:41.009872+0000","last_fullsized":"2024-05-08T23:23:41.009872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:29.828185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472463+0000","last_change":"2024-05-08T23:23:39.030569+0000","last_active":"2024-05-08T23:23:40.472463+0000","last_peered":"2024-05-08T23:23:40.472463+0000","last_clean":"2024-05-08T23:23:40.472463+0000","last_became_active":"2024-05-08T23:23:39.030285+0000","last_became_peered":"2024-05-08T23:23:39.030285+0000","last_unstale":"2024-05-08T23:23:40.472463+0000","last_undegraded":"2024-05-08T23:23:40.472463+0000","last_fullsized":"2024-05-08T23:23:40.472463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:40.828807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.148798+0000","last_change":"2024-05-08T23:23:39.487383+0000","last_active":"2024-05-08T23:23:41.148798+0000","last_peered":"2024-05-08T23:23:41.148798+0000","last_clean":"2024-05-08T23:23:41.148798+0000","last_became_active":"2024-05-08T23:23:39.486493+0000","last_became_peered":"2024-05-08T23:23:39.486493+0000","last_unstale":"2024-05-08T23:23:41.148798+0000","last_undegraded":"2024-05-08T23:23:41.148798+0000","last_fullsized":"2024-05-08T23:23:41.148798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:41.833689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473067+0000","last_change":"2024-05-08T23:23:39.028725+0000","last_active":"2024-05-08T23:23:40.473067+0000","last_peered":"2024-05-08T23:23:40.473067+0000","last_clean":"2024-05-08T23:23:40.473067+0000","last_became_active":"2024-05-08T23:23:39.027186+0000","last_became_peered":"2024-05-08T23:23:39.027186+0000","last_unstale":"2024-05-08T23:23:40.473067+0000","last_undegraded":"2024-05-08T23:23:40.473067+0000","last_fullsized":"2024-05-08T23:23:40.473067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:50:17.124793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.149047+0000","last_change":"2024-05-08T23:23:39.500401+0000","last_active":"2024-05-08T23:23:41.149047+0000","last_peered":"2024-05-08T23:23:41.149047+0000","last_clean":"2024-05-08T23:23:41.149047+0000","last_became_active":"2024-05-08T23:23:39.500066+0000","last_became_peered":"2024-05-08T23:23:39.500066+0000","last_unstale":"2024-05-08T23:23:41.149047+0000","last_undegraded":"2024-05-08T23:23:41.149047+0000","last_fullsized":"2024-05-08T23:23:41.149047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:33.959563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471776+0000","last_change":"2024-05-08T23:23:39.017107+0000","last_active":"2024-05-08T23:23:40.471776+0000","last_peered":"2024-05-08T23:23:40.471776+0000","last_clean":"2024-05-08T23:23:40.471776+0000","last_became_active":"2024-05-08T23:23:39.016802+0000","last_became_peered":"2024-05-08T23:23:39.016802+0000","last_unstale":"2024-05-08T23:23:40.471776+0000","last_undegraded":"2024-05-08T23:23:40.471776+0000","last_fullsized":"2024-05-08T23:23:40.471776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:33.362751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471431+0000","last_change":"2024-05-08T23:23:39.492574+0000","last_active":"2024-05-08T23:23:40.471431+0000","last_peered":"2024-05-08T23:23:40.471431+0000","last_clean":"2024-05-08T23:23:40.471431+0000","last_became_active":"2024-05-08T23:23:39.492292+0000","last_became_peered":"2024-05-08T23:23:39.492292+0000","last_unstale":"2024-05-08T23:23:40.471431+0000","last_undegraded":"2024-05-08T23:23:40.471431+0000","last_fullsized":"2024-05-08T23:23:40.471431+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:22.022575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471771+0000","last_change":"2024-05-08T23:23:39.029842+0000","last_active":"2024-05-08T23:23:40.471771+0000","last_peered":"2024-05-08T23:23:40.471771+0000","last_clean":"2024-05-08T23:23:40.471771+0000","last_became_active":"2024-05-08T23:23:39.029365+0000","last_became_peered":"2024-05-08T23:23:39.029365+0000","last_unstale":"2024-05-08T23:23:40.471771+0000","last_undegraded":"2024-05-08T23:23:40.471771+0000","last_fullsized":"2024-05-08T23:23:40.471771+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:53.675667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688491+0000","last_change":"2024-05-08T23:23:39.494374+0000","last_active":"2024-05-08T23:23:40.688491+0000","last_peered":"2024-05-08T23:23:40.688491+0000","last_clean":"2024-05-08T23:23:40.688491+0000","last_became_active":"2024-05-08T23:23:39.494009+0000","last_became_peered":"2024-05-08T23:23:39.494009+0000","last_unstale":"2024-05-08T23:23:40.688491+0000","last_undegraded":"2024-05-08T23:23:40.688491+0000","last_fullsized":"2024-05-08T23:23:40.688491+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:11.906348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472932+0000","last_change":"2024-05-08T23:23:39.023650+0000","last_active":"2024-05-08T23:23:40.472932+0000","last_peered":"2024-05-08T23:23:40.472932+0000","last_clean":"2024-05-08T23:23:40.472932+0000","last_became_active":"2024-05-08T23:23:39.023351+0000","last_became_peered":"2024-05-08T23:23:39.023351+0000","last_unstale":"2024-05-08T23:23:40.472932+0000","last_undegraded":"2024-05-08T23:23:40.472932+0000","last_fullsized":"2024-05-08T23:23:40.472932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:06.784412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.646394+0000","last_change":"2024-05-08T23:23:39.491812+0000","last_active":"2024-05-08T23:23:40.646394+0000","last_peered":"2024-05-08T23:23:40.646394+0000","last_clean":"2024-05-08T23:23:40.646394+0000","last_became_active":"2024-05-08T23:23:39.491543+0000","last_became_peered":"2024-05-08T23:23:39.491543+0000","last_unstale":"2024-05-08T23:23:40.646394+0000","last_undegraded":"2024-05-08T23:23:40.646394+0000","last_fullsized":"2024-05-08T23:23:40.646394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.146424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009253+0000","last_change":"2024-05-08T23:23:39.031430+0000","last_active":"2024-05-08T23:23:41.009253+0000","last_peered":"2024-05-08T23:23:41.009253+0000","last_clean":"2024-05-08T23:23:41.009253+0000","last_became_active":"2024-05-08T23:23:39.031071+0000","last_became_peered":"2024-05-08T23:23:39.031071+0000","last_unstale":"2024-05-08T23:23:41.009253+0000","last_undegraded":"2024-05-08T23:23:41.009253+0000","last_fullsized":"2024-05-08T23:23:41.009253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:34.063977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009236+0000","last_change":"2024-05-08T23:23:39.488731+0000","last_active":"2024-05-08T23:23:41.009236+0000","last_peered":"2024-05-08T23:23:41.009236+0000","last_clean":"2024-05-08T23:23:41.009236+0000","last_became_active":"2024-05-08T23:23:39.488437+0000","last_became_peered":"2024-05-08T23:23:39.488437+0000","last_unstale":"2024-05-08T23:23:41.009236+0000","last_undegraded":"2024-05-08T23:23:41.009236+0000","last_fullsized":"2024-05-08T23:23:41.009236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:54.680549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147844+0000","last_change":"2024-05-08T23:23:39.027238+0000","last_active":"2024-05-08T23:23:41.147844+0000","last_peered":"2024-05-08T23:23:41.147844+0000","last_clean":"2024-05-08T23:23:41.147844+0000","last_became_active":"2024-05-08T23:23:39.026975+0000","last_became_peered":"2024-05-08T23:23:39.026975+0000","last_unstale":"2024-05-08T23:23:41.147844+0000","last_undegraded":"2024-05-08T23:23:41.147844+0000","last_fullsized":"2024-05-08T23:23:41.147844+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.945395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.149069+0000","last_change":"2024-05-08T23:23:39.498459+0000","last_active":"2024-05-08T23:23:41.149069+0000","last_peered":"2024-05-08T23:23:41.149069+0000","last_clean":"2024-05-08T23:23:41.149069+0000","last_became_active":"2024-05-08T23:23:39.498160+0000","last_became_peered":"2024-05-08T23:23:39.498160+0000","last_unstale":"2024-05-08T23:23:41.149069+0000","last_undegraded":"2024-05-08T23:23:41.149069+0000","last_fullsized":"2024-05-08T23:23:41.149069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:27:52.210690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471938+0000","last_change":"2024-05-08T23:23:39.030477+0000","last_active":"2024-05-08T23:23:40.471938+0000","last_peered":"2024-05-08T23:23:40.471938+0000","last_clean":"2024-05-08T23:23:40.471938+0000","last_became_active":"2024-05-08T23:23:39.030192+0000","last_became_peered":"2024-05-08T23:23:39.030192+0000","last_unstale":"2024-05-08T23:23:40.471938+0000","last_undegraded":"2024-05-08T23:23:40.471938+0000","last_fullsized":"2024-05-08T23:23:40.471938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:40.294327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009662+0000","last_change":"2024-05-08T23:23:39.480489+0000","last_active":"2024-05-08T23:23:41.009662+0000","last_peered":"2024-05-08T23:23:41.009662+0000","last_clean":"2024-05-08T23:23:41.009662+0000","last_became_active":"2024-05-08T23:23:39.480187+0000","last_became_peered":"2024-05-08T23:23:39.480187+0000","last_unstale":"2024-05-08T23:23:41.009662+0000","last_undegraded":"2024-05-08T23:23:41.009662+0000","last_fullsized":"2024-05-08T23:23:41.009662+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.768894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147855+0000","last_change":"2024-05-08T23:23:39.030698+0000","last_active":"2024-05-08T23:23:41.147855+0000","last_peered":"2024-05-08T23:23:41.147855+0000","last_clean":"2024-05-08T23:23:41.147855+0000","last_became_active":"2024-05-08T23:23:39.030407+0000","last_became_peered":"2024-05-08T23:23:39.030407+0000","last_unstale":"2024-05-08T23:23:41.147855+0000","last_undegraded":"2024-05-08T23:23:41.147855+0000","last_fullsized":"2024-05-08T23:23:41.147855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:09.029707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147649+0000","last_change":"2024-05-08T23:23:39.492682+0000","last_active":"2024-05-08T23:23:41.147649+0000","last_peered":"2024-05-08T23:23:41.147649+0000","last_clean":"2024-05-08T23:23:41.147649+0000","last_became_active":"2024-05-08T23:23:39.492390+0000","last_became_peered":"2024-05-08T23:23:39.492390+0000","last_unstale":"2024-05-08T23:23:41.147649+0000","last_undegraded":"2024-05-08T23:23:41.147649+0000","last_fullsized":"2024-05-08T23:23:41.147649+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:05.858704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688405+0000","last_change":"2024-05-08T23:23:39.030159+0000","last_active":"2024-05-08T23:23:40.688405+0000","last_peered":"2024-05-08T23:23:40.688405+0000","last_clean":"2024-05-08T23:23:40.688405+0000","last_became_active":"2024-05-08T23:23:39.029872+0000","last_became_peered":"2024-05-08T23:23:39.029872+0000","last_unstale":"2024-05-08T23:23:40.688405+0000","last_undegraded":"2024-05-08T23:23:40.688405+0000","last_fullsized":"2024-05-08T23:23:40.688405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:05.304838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471510+0000","last_change":"2024-05-08T23:23:39.483949+0000","last_active":"2024-05-08T23:23:40.471510+0000","last_peered":"2024-05-08T23:23:40.471510+0000","last_clean":"2024-05-08T23:23:40.471510+0000","last_became_active":"2024-05-08T23:23:39.483767+0000","last_became_peered":"2024-05-08T23:23:39.483767+0000","last_unstale":"2024-05-08T23:23:40.471510+0000","last_undegraded":"2024-05-08T23:23:40.471510+0000","last_fullsized":"2024-05-08T23:23:40.471510+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:45.808539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.147887+0000","last_change":"2024-05-08T23:23:39.030805+0000","last_active":"2024-05-08T23:23:41.147887+0000","last_peered":"2024-05-08T23:23:41.147887+0000","last_clean":"2024-05-08T23:23:41.147887+0000","last_became_active":"2024-05-08T23:23:39.030392+0000","last_became_peered":"2024-05-08T23:23:39.030392+0000","last_unstale":"2024-05-08T23:23:41.147887+0000","last_undegraded":"2024-05-08T23:23:41.147887+0000","last_fullsized":"2024-05-08T23:23:41.147887+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:44.764012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.471635+0000","last_change":"2024-05-08T23:23:39.491511+0000","last_active":"2024-05-08T23:23:40.471635+0000","last_peered":"2024-05-08T23:23:40.471635+0000","last_clean":"2024-05-08T23:23:40.471635+0000","last_became_active":"2024-05-08T23:23:39.490866+0000","last_became_peered":"2024-05-08T23:23:39.490866+0000","last_unstale":"2024-05-08T23:23:40.471635+0000","last_undegraded":"2024-05-08T23:23:40.471635+0000","last_fullsized":"2024-05-08T23:23:40.471635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:59:59.952098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472482+0000","last_change":"2024-05-08T23:23:39.030262+0000","last_active":"2024-05-08T23:23:40.472482+0000","last_peered":"2024-05-08T23:23:40.472482+0000","last_clean":"2024-05-08T23:23:40.472482+0000","last_became_active":"2024-05-08T23:23:39.029990+0000","last_became_peered":"2024-05-08T23:23:39.029990+0000","last_unstale":"2024-05-08T23:23:40.472482+0000","last_undegraded":"2024-05-08T23:23:40.472482+0000","last_fullsized":"2024-05-08T23:23:40.472482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:15.781012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472096+0000","last_change":"2024-05-08T23:23:39.497366+0000","last_active":"2024-05-08T23:23:40.472096+0000","last_peered":"2024-05-08T23:23:40.472096+0000","last_clean":"2024-05-08T23:23:40.472096+0000","last_became_active":"2024-05-08T23:23:39.497135+0000","last_became_peered":"2024-05-08T23:23:39.497135+0000","last_unstale":"2024-05-08T23:23:40.472096+0000","last_undegraded":"2024-05-08T23:23:40.472096+0000","last_fullsized":"2024-05-08T23:23:40.472096+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:10.034589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.688398+0000","last_change":"2024-05-08T23:23:39.031397+0000","last_active":"2024-05-08T23:23:40.688398+0000","last_peered":"2024-05-08T23:23:40.688398+0000","last_clean":"2024-05-08T23:23:40.688398+0000","last_became_active":"2024-05-08T23:23:39.031064+0000","last_became_peered":"2024-05-08T23:23:39.031064+0000","last_unstale":"2024-05-08T23:23:40.688398+0000","last_undegraded":"2024-05-08T23:23:40.688398+0000","last_fullsized":"2024-05-08T23:23:40.688398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:02.094596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.472095+0000","last_change":"2024-05-08T23:23:39.481783+0000","last_active":"2024-05-08T23:23:40.472095+0000","last_peered":"2024-05-08T23:23:40.472095+0000","last_clean":"2024-05-08T23:23:40.472095+0000","last_became_active":"2024-05-08T23:23:39.481548+0000","last_became_peered":"2024-05-08T23:23:39.481548+0000","last_unstale":"2024-05-08T23:23:40.472095+0000","last_undegraded":"2024-05-08T23:23:40.472095+0000","last_fullsized":"2024-05-08T23:23:40.472095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:06.330086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009380+0000","last_change":"2024-05-08T23:23:39.028102+0000","last_active":"2024-05-08T23:23:41.009380+0000","last_peered":"2024-05-08T23:23:41.009380+0000","last_clean":"2024-05-08T23:23:41.009380+0000","last_became_active":"2024-05-08T23:23:39.027806+0000","last_became_peered":"2024-05-08T23:23:39.027806+0000","last_unstale":"2024-05-08T23:23:41.009380+0000","last_undegraded":"2024-05-08T23:23:41.009380+0000","last_fullsized":"2024-05-08T23:23:41.009380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:08:55.318230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:41.009752+0000","last_change":"2024-05-08T23:23:39.494242+0000","last_active":"2024-05-08T23:23:41.009752+0000","last_peered":"2024-05-08T23:23:41.009752+0000","last_clean":"2024-05-08T23:23:41.009752+0000","last_became_active":"2024-05-08T23:23:39.493821+0000","last_became_peered":"2024-05-08T23:23:39.493821+0000","last_unstale":"2024-05-08T23:23:41.009752+0000","last_undegraded":"2024-05-08T23:23:41.009752+0000","last_fullsized":"2024-05-08T23:23:41.009752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:39.005316+0000","last_clean_scrub_stamp":"2024-05-08T23:23:39.005316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:04.635681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T23:23:40.473237+0000","last_change":"2024-05-08T23:23:39.028452+0000","last_active":"2024-05-08T23:23:40.473237+0000","last_peered":"2024-05-08T23:23:40.473237+0000","last_clean":"2024-05-08T23:23:40.473237+0000","last_became_active":"2024-05-08T23:23:39.028154+0000","last_became_peered":"2024-05-08T23:23:39.028154+0000","last_unstale":"2024-05-08T23:23:40.473237+0000","last_undegraded":"2024-05-08T23:23:40.473237+0000","last_fullsized":"2024-05-08T23:23:40.473237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:23:38.000434+0000","last_clean_scrub_stamp":"2024-05-08T23:23:38.000434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:32.954681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T23:23:39.476250+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:23:39.476250+0000","last_peered":"2024-05-08T23:23:39.476250+0000","last_clean":"2024-05-08T23:23:39.476250+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:23:39.476250+0000","last_undegraded":"2024-05-08T23:23:39.476250+0000","last_fullsized":"2024-05-08T23:23:39.476250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":812546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.48299999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63200000000000001}]},{"osd":2,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.52700000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.622},{"interface":"front","average":{"1min":0.628,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68899999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71099999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.38900000000000001}]},{"osd":5,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.40100000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70399999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.433},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed May 8 23:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78300000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":548864,"data_stored":222760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70799999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76700000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.443,"5min":0.438,"15min":0.438},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73599999999999999,"5min":1.115,"15min":1.115},"last":0.62}]},{"osd":3,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.622,"15min":0.626},"min":{"1min":0.495,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75800000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54700000000000004,"15min":0.55600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.432},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.55700000000000005,"15min":0.56799999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36299999999999999}]},{"osd":5,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56000000000000005,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69399999999999995,"5min":1.052,"15min":1.052},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73899999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63600000000000001}]},{"osd":7,"last update":"Wed May 8 23:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.755,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.496}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":222158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.61599999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.495,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.27300000000000002}]},{"osd":2,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61099999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60299999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.503,"15min":0.50700000000000001},"min":{"1min":0.372,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65800000000000003,"5min":1.01,"15min":1.01},"last":0.437},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.57999999999999996,"5min":1.028,"15min":1.028},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.48999999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.004,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.53900000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85299999999999998,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.53500000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51900000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81299999999999994,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.58499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92500000000000004,"5min":1.105,"15min":1.105},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.55500000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89200000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.54300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95399999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":812490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.75700000000000001,"15min":0.81699999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.94099999999999995,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.74299999999999999,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59899999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.81699999999999995,"15min":0.879},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.371},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.77800000000000002,"15min":0.83099999999999996},"min":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71399999999999997,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.47199999999999998,"15min":0.45700000000000002},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.84399999999999997,"15min":0.88400000000000001},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.9770000000000001,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.96499999999999997,"15min":1.0269999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.94599999999999995,"15min":0.98799999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.069,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.94599999999999995,"15min":0.98599999999999999},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.0350000000000001,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":1.0429999999999999,"15min":1.103},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2150000000000001,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":1.008,"15min":1.0660000000000001},"min":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1759999999999999,"5min":7.734,"15min":7.734},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.98799999999999999,"15min":1.044},"min":{"1min":0.503,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83099999999999996,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.88}]},{"osd":7,"last update":"Wed May 8 23:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":1,"15min":1.048},"min":{"1min":0.53100000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5369999999999999,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.96699999999999997,"15min":0.999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.1139999999999999,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":223674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29199999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.437,"5min":0.40000000000000002,"15min":0.39400000000000002},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.621,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.42399999999999999,"15min":0.41999999999999998},"min":{"1min":0.33000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.41999999999999998},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59599999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.45000000000000001}]},{"osd":4,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.504,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73899999999999999}]},{"osd":5,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97899999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.86399999999999999,"5min":1.159,"15min":1.159},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.56200000000000006,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed May 8 23:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.997,"5min":1.194,"15min":1.194},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":552960,"data_stored":222726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54100000000000004,"5min":1.46,"15min":1.46},"last":0.432},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57099999999999995,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.311}]},{"osd":1,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60499999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46999999999999997,"15min":0.47399999999999998},"min":{"1min":0.316,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59799999999999998,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.88200000000000001,"15min":0.95199999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65500000000000003,"5min":11.964,"15min":11.964},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65600000000000003,"5min":1.593,"15min":1.593},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82599999999999996,"5min":1.375,"15min":1.375},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.73199999999999998,"15min":0.746},"min":{"1min":0.48299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84399999999999997,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84099999999999997,"5min":1.536,"15min":1.536},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68500000000000005,"15min":0.69099999999999995},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84999999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.84899999999999998,"5min":1.204,"15min":1.204},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82599999999999996,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.627}]},{"osd":7,"last update":"Wed May 8 23:23:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.53200000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82499999999999996,"5min":1.339,"15min":1.339},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.53900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89400000000000002,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.71299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":552960,"data_stored":223652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70299999999999996,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.73599999999999999,"15min":0.76800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70099999999999996,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.67900000000000005,"15min":0.69699999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71999999999999997,"5min":2.145,"15min":2.145},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70399999999999996,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.69799999999999995,"15min":0.71699999999999997},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.72899999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.80100000000000005,"15min":0.83599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.751,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.68500000000000005}]},{"osd":3,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.70999999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.1739999999999999,"15min":1.268},"min":{"1min":0.49399999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.76000000000000001,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80200000000000005,"5min":0.877,"15min":0.877},"last":0.497},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45800000000000002}]},{"osd":6,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57599999999999996}]},{"osd":7,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.69999999999999996,"15min":0.71999999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85199999999999998,"5min":1.24,"15min":1.24},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":222158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.72599999999999998,"15min":0.746},"min":{"1min":0.46000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.96099999999999997,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.92500000000000004,"15min":0.97699999999999998},"min":{"1min":0.41099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78500000000000003,"5min":8.798,"15min":8.798},"last":0.624}]},{"osd":1,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":4.71,"15min":4.71},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.62,"5min":1.0029999999999999,"15min":1.0669999999999999},"min":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0189999999999999,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.497}]},{"osd":2,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.76900000000000002,"15min":0.78700000000000003},"min":{"1min":0.436,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.002,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.80800000000000005,"15min":0.83399999999999996},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.96099999999999997,"15min":1.0169999999999999},"min":{"1min":0.47699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89300000000000002,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":1.048,"15min":1.1140000000000001},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.214,"5min":10.427,"15min":10.427},"last":0.58499999999999996}]},{"osd":4,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54900000000000004,"15min":0.55800000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74399999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90800000000000003,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.623,"15min":0.628},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed May 8 23:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.44400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83399999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:23:44.137 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:23:44.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T23:23:44.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.320+0000 7fe9a5215700 1 Processor -- start 2024-05-08T23:23:44.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.320+0000 7fe9a5215700 1 -- start start 2024-05-08T23:23:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.321+0000 7fe9a5215700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9a00b5550 0x7fe9a01618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.321+0000 7fe9a5215700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a0099ce0 con 0x7fe9a015d8c0 2024-05-08T23:23:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.321+0000 7fe9a5215700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a006f260 con 0x7fe9a00b50d0 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.321+0000 7fe9a5215700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a006fde0 con 0x7fe9a00b5550 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.321+0000 7fe99ed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe9a015d8c0 0x7fe9a00ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50788/0 (socket says 172.21.15.5:50788) 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.321+0000 7fe99ed9d700 1 -- 172.21.15.5:0/1396381301 learned_addr learned my addr 172.21.15.5:0/1396381301 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99f59e700 1 --2- 172.21.15.5:0/1396381301 >> v2:172.21.15.49:3300/0 conn(0x7fe9a00b5550 0x7fe9a01618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674895878 0 0) 0x7fe9a0099ce0 con 0x7fe9a015d8c0 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9880035b0 con 0x7fe9a015d8c0 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 846739269 0 0) 0x7fe9880035b0 con 0x7fe9a015d8c0 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 >> v2:172.21.15.49:3300/0 conn(0x7fe9a00b5550 msgr2=0x7fe9a01618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 --2- 172.21.15.5:0/1396381301 >> v2:172.21.15.49:3300/0 conn(0x7fe9a00b5550 0x7fe9a01618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 >> v1:172.21.15.49:6789/0 conn(0x7fe9a00b50d0 legacy=0x7fe9a006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a006f540 con 0x7fe9a015d8c0 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fe990001670 con 0x7fe9a015d8c0 2024-05-08T23:23:44.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 >> v1:172.21.15.5:6789/0 conn(0x7fe9a015d8c0 legacy=0x7fe9a00ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe988003a50 0x7fe988003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988004850 0x7fe988004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988005210 0x7fe98800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a006fde0 con 0x7fe988004850 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98800c110 con 0x7fe988003a50 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98800c2d0 con 0x7fe988005210 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99ed9d700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe988003a50 0x7fe988003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.322+0000 7fe99e59c700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988004850 0x7fe988004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99f59e700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988005210 0x7fe98800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99e59c700 1 -- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988005210 msgr2=0x7fe98800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99e59c700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988005210 0x7fe98800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99e59c700 1 -- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe988003a50 msgr2=0x7fe988003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99e59c700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe988003a50 0x7fe988003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99e59c700 1 -- 172.21.15.5:0/1396381301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880035b0 con 0x7fe988004850 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99ed9d700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe988003a50 0x7fe988003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99f59e700 1 -- 172.21.15.5:0/1396381301 reap_dead start 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99e59c700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988004850 0x7fe988004cd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe9940017c0 tx=0x7fe988005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668f069cd5bbb02e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940052d0 con 0x7fe988004850 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994005430 con 0x7fe988004850 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.323+0000 7fe99dd9b700 1 -- 172.21.15.5:0/1396381301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940038c0 con 0x7fe988004850 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.324+0000 7fe9a5215700 1 -- 172.21.15.5:0/1396381301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988004850 msgr2=0x7fe988004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.324+0000 7fe9a5215700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988004850 0x7fe988004cd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe9940017c0 tx=0x7fe988005880 comp rx=0 tx=0).stop 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.324+0000 7fe9a5215700 1 -- 172.21.15.5:0/1396381301 shutdown_connections 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.324+0000 7fe9a5215700 1 --2- 172.21.15.5:0/1396381301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988004850 0x7fe988004cd0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.324+0000 7fe9a5215700 1 -- 172.21.15.5:0/1396381301 >> 172.21.15.5:0/1396381301 conn(0x7fe9a00b5a30 msgr2=0x7fe9a00b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.324+0000 7fe9a5215700 1 -- 172.21.15.5:0/1396381301 shutdown_connections 2024-05-08T23:23:44.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.324+0000 7fe9a5215700 1 -- 172.21.15.5:0/1396381301 wait complete. 2024-05-08T23:23:44.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.325+0000 7fe9a5215700 1 Processor -- start 2024-05-08T23:23:44.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe9a5215700 1 -- start start 2024-05-08T23:23:44.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe9a5215700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988004850 0x7fe9a0154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe9a5215700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9a0159d10 0x7fe9a015a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe9a5215700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a009adf0 0x7fe9a009b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe9a5215700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a015a6d0 con 0x7fe9a009adf0 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe9a5215700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a009c3c0 con 0x7fe9a0159d10 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe9a5215700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a009bb20 con 0x7fe988004850 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe99f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a009adf0 0x7fe9a009b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe99f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a009adf0 0x7fe9a009b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35752/0 (socket says 172.21.15.5:35752) 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.326+0000 7fe99f59e700 1 -- 172.21.15.5:0/2728915201 learned_addr learned my addr 172.21.15.5:0/2728915201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99ed9d700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988004850 0x7fe9a0154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99e59c700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9a0159d10 0x7fe9a015a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99f59e700 1 -- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988004850 msgr2=0x7fe9a0154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99f59e700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988004850 0x7fe9a0154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99f59e700 1 -- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9a0159d10 msgr2=0x7fe9a015a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99f59e700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9a0159d10 0x7fe9a015a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99f59e700 1 -- 172.21.15.5:0/2728915201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a009be00 con 0x7fe9a009adf0 2024-05-08T23:23:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99ed9d700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988004850 0x7fe9a0154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe99f59e700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a009adf0 0x7fe9a009b2a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00b30b0 tx=0x7fe998011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe9877fe700 1 -- 172.21.15.5:0/2728915201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99800e040 con 0x7fe9a009adf0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe9877fe700 1 -- 172.21.15.5:0/2728915201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99800f040 con 0x7fe9a009adf0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a009c0e0 con 0x7fe9a009adf0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe9877fe700 1 -- 172.21.15.5:0/2728915201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99800fd50 con 0x7fe9a009adf0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.327+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a009b7e0 con 0x7fe9a009adf0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.328+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a0074bc0 con 0x7fe9a009adf0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.329+0000 7fe9877fe700 1 -- 172.21.15.5:0/2728915201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe998015430 con 0x7fe9a009adf0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.329+0000 7fe9877fe700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe98c06ac90 0x7fe98c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.329+0000 7fe9877fe700 1 -- 172.21.15.5:0/2728915201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe998089990 con 0x7fe9a009adf0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.329+0000 7fe99ed9d700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe98c06ac90 0x7fe98c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.330+0000 7fe99ed9d700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe98c06ac90 0x7fe98c06d150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe990000c00 tx=0x7fe990009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.334+0000 7fe9877fe700 1 -- 172.21.15.5:0/2728915201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe998054b70 con 0x7fe9a009adf0 2024-05-08T23:23:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.638+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe9a0068210 con 0x7fe9a009adf0 2024-05-08T23:23:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.642+0000 7fe9877fe700 1 -- 172.21.15.5:0/2728915201 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe998058190 con 0x7fe9a009adf0 2024-05-08T23:23:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe98c06ac90 msgr2=0x7fe98c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe98c06ac90 0x7fe98c06d150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe990000c00 tx=0x7fe990009040 comp rx=0 tx=0).stop 2024-05-08T23:23:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a009adf0 msgr2=0x7fe9a009b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a009adf0 0x7fe9a009b2a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00b30b0 tx=0x7fe998011f80 comp rx=0 tx=0).stop 2024-05-08T23:23:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 shutdown_connections 2024-05-08T23:23:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe98c06ac90 0x7fe98c06d150 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a009adf0 0x7fe9a009b2a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe988004850 0x7fe9a0154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 --2- 172.21.15.5:0/2728915201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9a0159d10 0x7fe9a015a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 >> 172.21.15.5:0/2728915201 conn(0x7fe9a00b5a30 msgr2=0x7fe9a01608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 shutdown_connections 2024-05-08T23:23:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.645+0000 7fe9a5215700 1 -- 172.21.15.5:0/2728915201 wait complete. 2024-05-08T23:23:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-08T23:23:44.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.795+0000 7f2b0f2d0700 1 Processor -- start 2024-05-08T23:23:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.795+0000 7f2b0f2d0700 1 -- start start 2024-05-08T23:23:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.796+0000 7f2b0f2d0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b08152470 0x7f2b0815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.796+0000 7f2b0f2d0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b08152850 0x7f2b080cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.796+0000 7f2b0f2d0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b080ce100 con 0x7f2b080d0d90 2024-05-08T23:23:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.796+0000 7f2b0f2d0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0815c220 con 0x7f2b08152850 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.796+0000 7f2b0f2d0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0815c500 con 0x7f2b08152470 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.796+0000 7f2b0d86d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2b080d0d90 0x7f2b0806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50792/0 (socket says 172.21.15.5:50792) 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.796+0000 7f2b0d86d700 1 -- 172.21.15.5:0/3553707481 learned_addr learned my addr 172.21.15.5:0/3553707481 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.796+0000 7f2b0c86b700 1 --2- 172.21.15.5:0/3553707481 >> v2:172.21.15.49:3300/0 conn(0x7f2b08152850 0x7f2b080cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0d06c700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b08152470 0x7f2b0815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874317591 0 0) 0x7f2b080ce100 con 0x7f2b080d0d90 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af40035b0 con 0x7f2b080d0d90 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3342344150 0 0) 0x7f2af40035b0 con 0x7f2b080d0d90 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b08152470 msgr2=0x7f2b0815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b08152470 0x7f2b0815bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 >> v2:172.21.15.49:3300/0 conn(0x7f2b08152850 msgr2=0x7f2b080cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 --2- 172.21.15.5:0/3553707481 >> v2:172.21.15.49:3300/0 conn(0x7f2b08152850 0x7f2b080cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0815c7e0 con 0x7f2b080d0d90 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f2b000019f0 con 0x7f2b080d0d90 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0c86b700 1 --2- 172.21.15.5:0/3553707481 >> v2:172.21.15.49:3300/0 conn(0x7f2b08152850 0x7f2b080cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 >> v1:172.21.15.5:6789/0 conn(0x7f2b080d0d90 legacy=0x7f2b0806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003a00 0x7f2af4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4004850 0x7f2af4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b08152470 0x7f2af4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0815c220 con 0x7f2af4003a00 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0815c500 con 0x7f2af4004850 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af400ba10 con 0x7f2b08152470 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0d06c700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003a00 0x7f2af4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0c86b700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4004850 0x7f2af4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0d86d700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b08152470 0x7f2af4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0d06c700 1 -- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b08152470 msgr2=0x7f2af4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0d06c700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b08152470 0x7f2af4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0d06c700 1 -- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4004850 msgr2=0x7f2af4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0d06c700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4004850 0x7f2af4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.797+0000 7f2b0d06c700 1 -- 172.21.15.5:0/3553707481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40035b0 con 0x7f2af4003a00 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2b0d06c700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003a00 0x7f2af4003e20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009b60 tx=0x7f2af4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf549d93bb1b6c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400e6c0 con 0x7f2af4003a00 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0400e820 con 0x7f2af4003a00 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2afffff700 1 -- 172.21.15.5:0/3553707481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400eb90 con 0x7f2af4003a00 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/3553707481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003a00 msgr2=0x7f2af4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2b0f2d0700 1 --2- 172.21.15.5:0/3553707481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af4003a00 0x7f2af4003e20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009b60 tx=0x7f2af4005880 comp rx=0 tx=0).stop 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2b0d86d700 1 -- 172.21.15.5:0/3553707481 reap_dead start 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/3553707481 shutdown_connections 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.798+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/3553707481 >> 172.21.15.5:0/3553707481 conn(0x7f2b08099a50 msgr2=0x7f2b080d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.799+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/3553707481 shutdown_connections 2024-05-08T23:23:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.799+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/3553707481 wait complete. 2024-05-08T23:23:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.800+0000 7f2b0f2d0700 1 Processor -- start 2024-05-08T23:23:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.800+0000 7f2b0f2d0700 1 -- start start 2024-05-08T23:23:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0f2d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4003a00 0x7f2b080c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0f2d0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b080b5c30 0x7f2b080b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0f2d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080c5fe0 0x7f2b081511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0f2d0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080b4bb0 con 0x7f2b080c5fe0 2024-05-08T23:23:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0f2d0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080b3f40 con 0x7f2af4003a00 2024-05-08T23:23:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0f2d0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080b4e70 con 0x7f2b080b5c30 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080c5fe0 0x7f2b081511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080c5fe0 0x7f2b081511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35762/0 (socket says 172.21.15.5:35762) 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 -- 172.21.15.5:0/4082557456 learned_addr learned my addr 172.21.15.5:0/4082557456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0c86b700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b080b5c30 0x7f2b080b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d06c700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4003a00 0x7f2b080c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 -- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b080b5c30 msgr2=0x7f2b080b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b080b5c30 0x7f2b080b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 -- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4003a00 msgr2=0x7f2b080c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4003a00 0x7f2b080c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 -- 172.21.15.5:0/4082557456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080d13a0 con 0x7f2b080c5fe0 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2b0d86d700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080c5fe0 0x7f2b081511c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2b00000f80 tx=0x7f2b0000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2afdffb700 1 -- 172.21.15.5:0/4082557456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000ec40 con 0x7f2b080c5fe0 2024-05-08T23:23:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.801+0000 7f2afdffb700 1 -- 172.21.15.5:0/4082557456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0000eda0 con 0x7f2b080c5fe0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.802+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b080b5150 con 0x7f2b080c5fe0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.802+0000 7f2afdffb700 1 -- 172.21.15.5:0/4082557456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00013660 con 0x7f2b080c5fe0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.802+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b080b5710 con 0x7f2b080c5fe0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.803+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b080b5930 con 0x7f2b080c5fe0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.803+0000 7f2afdffb700 1 -- 172.21.15.5:0/4082557456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2b000137c0 con 0x7f2b080c5fe0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.804+0000 7f2afdffb700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2aec06acc0 0x7f2aec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.804+0000 7f2afdffb700 1 -- 172.21.15.5:0/4082557456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b00089820 con 0x7f2b080c5fe0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.804+0000 7f2b0d06c700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2aec06acc0 0x7f2aec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.805+0000 7f2b0d06c700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2aec06acc0 0x7f2aec06d180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b040090b0 tx=0x7f2b0401f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:44.809+0000 7f2afdffb700 1 -- 172.21.15.5:0/4082557456 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2b00054970 con 0x7f2b080c5fe0 2024-05-08T23:23:45.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.075+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b080b5930 con 0x7f2b080c5fe0 2024-05-08T23:23:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.076+0000 7f2afdffb700 1 -- 172.21.15.5:0/4082557456 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4722 (secure 0 0 0) 0x7f2b00089020 con 0x7f2b080c5fe0 2024-05-08T23:23:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:45.076 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:23:42.470597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2aec06acc0 msgr2=0x7f2aec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2aec06acc0 0x7f2aec06d180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b040090b0 tx=0x7f2b0401f040 comp rx=0 tx=0).stop 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080c5fe0 msgr2=0x7f2b081511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080c5fe0 0x7f2b081511c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2b00000f80 tx=0x7f2b0000e440 comp rx=0 tx=0).stop 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 shutdown_connections 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2aec06acc0 0x7f2aec06d180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b080c5fe0 0x7f2b081511c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af4003a00 0x7f2b080c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 --2- 172.21.15.5:0/4082557456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2b080b5c30 0x7f2b080b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 >> 172.21.15.5:0/4082557456 conn(0x7f2b08099a50 msgr2=0x7f2b080aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 shutdown_connections 2024-05-08T23:23:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.078+0000 7f2b0f2d0700 1 -- 172.21.15.5:0/4082557456 wait complete. 2024-05-08T23:23:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-05-08T23:23:45.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:23:39.466617+0000', 'modified': '2024-05-08T23:23:42.470597+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5778}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5778': {'gid': 5778, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6835/3359342067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 3359342067}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3359342067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:23:45.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:23:45.496 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:23:45.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.642+0000 7f6d9c3ab700 1 Processor -- start 2024-05-08T23:23:45.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.642+0000 7f6d9c3ab700 1 -- start start 2024-05-08T23:23:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d9c3ab700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d9406a960 0x7f6d9406adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d9c3ab700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d940ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d9c3ab700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9406b300 con 0x7f6d940cc060 2024-05-08T23:23:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d9c3ab700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d940ccc80 con 0x7f6d9406a960 2024-05-08T23:23:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d9c3ab700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940cd7c0 con 0x7f6d940ce230 2024-05-08T23:23:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d99946700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d9406a960 0x7f6d9406adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d99946700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d9406a960 0x7f6d9406adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35776/0 (socket says 172.21.15.5:35776) 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d99946700 1 -- 172.21.15.5:0/3753885158 learned_addr learned my addr 172.21.15.5:0/3753885158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.643+0000 7f6d9a948700 1 --2- 172.21.15.5:0/3753885158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d940ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.644+0000 7f6d99946700 1 -- 172.21.15.5:0/3753885158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 msgr2=0x7f6d940ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.644+0000 7f6d99946700 1 --2- 172.21.15.5:0/3753885158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d940ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.644+0000 7f6d99946700 1 -- 172.21.15.5:0/3753885158 >> v1:172.21.15.49:6789/0 conn(0x7f6d940cc060 legacy=0x7f6d940cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.644+0000 7f6d99946700 1 -- 172.21.15.5:0/3753885158 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940cffb0 con 0x7f6d9406a960 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.644+0000 7f6d99946700 1 --2- 172.21.15.5:0/3753885158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d9406adc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6d840098f0 tx=0x7f6d84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbb6bcdd65a51eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.644+0000 7f6d99145700 1 -- 172.21.15.5:0/3753885158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84011830 con 0x7f6d9406a960 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.644+0000 7f6d99145700 1 -- 172.21.15.5:0/3753885158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84011990 con 0x7f6d9406a960 2024-05-08T23:23:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.644+0000 7f6d99145700 1 -- 172.21.15.5:0/3753885158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84011d20 con 0x7f6d9406a960 2024-05-08T23:23:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.645+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/3753885158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 msgr2=0x7f6d9406adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.645+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/3753885158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d9406adc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6d840098f0 tx=0x7f6d84009c00 comp rx=0 tx=0).stop 2024-05-08T23:23:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.645+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/3753885158 shutdown_connections 2024-05-08T23:23:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.645+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/3753885158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d940ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.645+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/3753885158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d9406adc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.645+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/3753885158 >> 172.21.15.5:0/3753885158 conn(0x7f6d94099a50 msgr2=0x7f6d940d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.645+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/3753885158 shutdown_connections 2024-05-08T23:23:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.645+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/3753885158 wait complete. 2024-05-08T23:23:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.646+0000 7f6d9c3ab700 1 Processor -- start 2024-05-08T23:23:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.646+0000 7f6d9c3ab700 1 -- start start 2024-05-08T23:23:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9c3ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d94156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9c3ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d940cc060 0x7f6d940b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9c3ab700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d9409d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9c3ab700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c4b60 con 0x7f6d9406a960 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9c3ab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d94156a40 con 0x7f6d940cc060 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9c3ab700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940b1490 con 0x7f6d940ce230 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9a147700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d94156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9a147700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d94156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35792/0 (socket says 172.21.15.5:35792) 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9a147700 1 -- 172.21.15.5:0/2681993694 learned_addr learned my addr 172.21.15.5:0/2681993694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d99946700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d940cc060 0x7f6d940b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.647+0000 7f6d9a948700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d9409d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9a147700 1 -- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 msgr2=0x7f6d9409d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d9409d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9a147700 1 -- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d940cc060 msgr2=0x7f6d940b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d940cc060 0x7f6d940b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9a147700 1 -- 172.21.15.5:0/2681993694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d84009550 con 0x7f6d9406a960 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9a948700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d9409d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d94156500 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c4e0 tx=0x7f6d9000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d8affd700 1 -- 172.21.15.5:0/2681993694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d900052d0 con 0x7f6d9406a960 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d8affd700 1 -- 172.21.15.5:0/2681993694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d90005430 con 0x7f6d9406a960 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/2681993694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d940b1760 con 0x7f6d9406a960 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d8affd700 1 -- 172.21.15.5:0/2681993694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d900038c0 con 0x7f6d9406a960 2024-05-08T23:23:45.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.648+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/2681993694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d940b1d20 con 0x7f6d9406a960 2024-05-08T23:23:45.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.649+0000 7f6d8affd700 1 -- 172.21.15.5:0/2681993694 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d90003a20 con 0x7f6d9406a960 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.650+0000 7f6d8affd700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d7806ac90 0x7f6d7806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.650+0000 7f6d8affd700 1 -- 172.21.15.5:0/2681993694 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d90088790 con 0x7f6d9406a960 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.650+0000 7f6d99946700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d7806ac90 0x7f6d7806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.651+0000 7f6d99946700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d7806ac90 0x7f6d7806d150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009c40 tx=0x7f6d84008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.652+0000 7f6da8565b80 1 Processor -- start 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6da8565b80 1 -- start start 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bd950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bd97320 0x55dc7bd9a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6da8565b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bcf0100 con 0x55dc7bd97320 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6da8565b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bcedac0 con 0x55dc7bdc3d80 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6da8565b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bce93b0 con 0x55dc7bdbc750 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6d99946700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bd97320 0x55dc7bd9a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6d99946700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bd97320 0x55dc7bd9a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35804/0 (socket says 172.21.15.5:35804) 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.653+0000 7f6d99946700 1 -- 172.21.15.5:0/2822212465 learned_addr learned my addr 172.21.15.5:0/2822212465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d9a948700 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bd950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d99946700 1 -- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 msgr2=0x55dc7bd950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d99946700 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bd950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d99946700 1 -- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdc3d80 msgr2=0x55dc7bdc4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d99946700 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d99946700 1 -- 172.21.15.5:0/2822212465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc7bb6a360 con 0x55dc7bd97320 2024-05-08T23:23:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bd950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d99946700 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bd97320 0x55dc7bd9a050 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009f50 tx=0x7f6d84010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdc5d97bb98459b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d77fff700 1 -- 172.21.15.5:0/2822212465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d840038a0 con 0x55dc7bd97320 2024-05-08T23:23:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d77fff700 1 -- 172.21.15.5:0/2822212465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84003a00 con 0x55dc7bd97320 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.654+0000 7f6d77fff700 1 -- 172.21.15.5:0/2822212465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84003d70 con 0x55dc7bd97320 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 -- 172.21.15.5:0/2822212465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bd97320 msgr2=0x55dc7bd9a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bd97320 0x55dc7bd9a050 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009f50 tx=0x7f6d84010000 comp rx=0 tx=0).stop 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 -- 172.21.15.5:0/2822212465 shutdown_connections 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bd950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2822212465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bd97320 0x55dc7bd9a050 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 -- 172.21.15.5:0/2822212465 >> 172.21.15.5:0/2822212465 conn(0x55dc7bcf71c0 msgr2=0x55dc7bdded80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 -- 172.21.15.5:0/2822212465 shutdown_connections 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.655+0000 7f6da8565b80 1 -- 172.21.15.5:0/2822212465 wait complete. 2024-05-08T23:23:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.656+0000 7f6da8565b80 1 -- start start 2024-05-08T23:23:45.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.656+0000 7f6da8565b80 1 Processor -- start 2024-05-08T23:23:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.657+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bd97320 0x55dc7bdc6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.657+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bde2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bde2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6da8565b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bc85510 con 0x55dc7bdc3d80 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6da8565b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bc851b0 con 0x55dc7bd97320 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6da8565b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bb6c650 con 0x55dc7bdbc750 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6da8565b80 10 client.0 _ll_register_callbacks cb 0x55dc7bc8eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bde2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a147700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bd97320 0x55dc7bdc6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bde2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35818/0 (socket says 172.21.15.5:35818) 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 -- 172.21.15.5:0/3403942191 learned_addr learned my addr 172.21.15.5:0/3403942191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d99946700 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bde2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 -- 172.21.15.5:0/3403942191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 msgr2=0x55dc7bde2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bde2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 -- 172.21.15.5:0/3403942191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bd97320 msgr2=0x55dc7bdc6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bd97320 0x55dc7bdc6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 -- 172.21.15.5:0/3403942191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d940cffb0 con 0x55dc7bdc3d80 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d9a948700 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bde2a70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c007dd0 tx=0x7f6d8c012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14217fa77601f338 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d757fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d757fa700 1 -- 172.21.15.5:0/3403942191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c012bb0 con 0x55dc7bdc3d80 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6d757fa700 1 -- 172.21.15.5:0/3403942191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d8c012d10 con 0x55dc7bdc3d80 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.658+0000 7f6da8565b80 1 -- 172.21.15.5:0/3403942191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc7bc42220 con 0x55dc7bdc3d80 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6d757fa700 1 -- 172.21.15.5:0/3403942191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8c018470 con 0x55dc7bdc3d80 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6da8565b80 10 client.5928 fetch_fsmap learned FSMap version 31 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6da8565b80 1 -- 172.21.15.5:0/3403942191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55dc7bb16b10 con 0x55dc7bdc3d80 2024-05-08T23:23:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6d757fa700 1 -- 172.21.15.5:0/3403942191 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc7bc42220 con 0x55dc7bdc3d80 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6d757fa700 1 -- 172.21.15.5:0/3403942191 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6d8c014040 con 0x55dc7bdc3d80 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6da8565b80 10 client.5928 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6da8565b80 10 client.5928 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6da8565b80 20 client.5928 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6da8565b80 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x55dc7bde1540 0x55dc7bdc4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6da8565b80 4 client.5928 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6da8565b80 1 -- 172.21.15.5:0/3403942191 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc7bd68790 con 0x55dc7bde1540 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6d9a147700 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x55dc7bde1540 0x55dc7bdc4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6d757fa700 10 client.5928 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.659+0000 7f6d9a147700 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x55dc7bde1540 0x55dc7bdc4f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b971258900ecc002 server_cookie=2f62e7824f0f6721 in_seq=0 out_seq=0 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.660+0000 7f6d757fa700 1 -- 172.21.15.5:0/3403942191 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc7bd68790 con 0x55dc7bde1540 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.660+0000 7f6d757fa700 10 client.5928 handle_command_reply: tid=0 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.660+0000 7f6da8565b80 1 client.5928 shutdown 2024-05-08T23:23:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.660+0000 7f6da8565b80 1 -- 172.21.15.5:0/3403942191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 msgr2=0x55dc7bde2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.660+0000 7f6da8565b80 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bde2a70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c007dd0 tx=0x7f6d8c012520 comp rx=0 tx=0).stop 2024-05-08T23:23:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.660+0000 7f6da8565b80 1 -- 172.21.15.5:0/3403942191 shutdown_connections 2024-05-08T23:23:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6da8565b80 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bde2a70 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6da8565b80 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bd97320 0x55dc7bdc6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6da8565b80 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x55dc7bde1540 0x55dc7bdc4f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6da8565b80 1 --2- 172.21.15.5:0/3403942191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bdbc750 0x55dc7bde2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6d757fa700 0 client.5928 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:23:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6da8565b80 1 -- 172.21.15.5:0/3403942191 >> 172.21.15.5:0/3403942191 conn(0x55dc7bcf71c0 msgr2=0x55dc7bdc3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:45.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6da8565b80 1 -- 172.21.15.5:0/3403942191 shutdown_connections 2024-05-08T23:23:45.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6da8565b80 1 -- 172.21.15.5:0/3403942191 wait complete. 2024-05-08T23:23:45.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.661+0000 7f6da8565b80 20 client.5928 trim_cache size 0 max 16384 2024-05-08T23:23:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.680+0000 7f6da8565b80 1 Processor -- start 2024-05-08T23:23:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.680+0000 7f6da8565b80 1 -- start start 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bd9a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7bd95130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6da8565b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bd9b520 con 0x55dc7bdc3d80 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6da8565b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bdbbe50 con 0x55dc7bdbc750 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6da8565b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bde1cc0 con 0x55dc7bd97320 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a948700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a948700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35820/0 (socket says 172.21.15.5:35820) 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a948700 1 -- 172.21.15.5:0/1496809788 learned_addr learned my addr 172.21.15.5:0/1496809788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d99946700 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7bd95130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a147700 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bd9a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a948700 1 -- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 msgr2=0x55dc7bd9a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a948700 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bd9a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a948700 1 -- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 msgr2=0x55dc7bd95130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a948700 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7bd95130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d9a948700 1 -- 172.21.15.5:0/1496809788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc7bcaf510 con 0x55dc7bdc3d80 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.681+0000 7f6d99946700 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7bd95130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6d9a948700 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc6470 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c002230 tx=0x7f6d8c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23393325f3b5e26a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6d88da4700 1 -- 172.21.15.5:0/1496809788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c018470 con 0x55dc7bdc3d80 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6d88da4700 1 -- 172.21.15.5:0/1496809788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c00f040 con 0x55dc7bdc3d80 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6d88da4700 1 -- 172.21.15.5:0/1496809788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c0185f0 con 0x55dc7bdc3d80 2024-05-08T23:23:45.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 -- 172.21.15.5:0/1496809788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 msgr2=0x55dc7bdc6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc6470 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c002230 tx=0x7f6d8c018000 comp rx=0 tx=0).stop 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 -- 172.21.15.5:0/1496809788 shutdown_connections 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc6470 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bd9a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 --2- 172.21.15.5:0/1496809788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7bd95130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 -- 172.21.15.5:0/1496809788 >> 172.21.15.5:0/1496809788 conn(0x55dc7bcf71c0 msgr2=0x55dc7bdc3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 -- 172.21.15.5:0/1496809788 shutdown_connections 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.682+0000 7f6da8565b80 1 -- 172.21.15.5:0/1496809788 wait complete. 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.683+0000 7f6da8565b80 1 -- start start 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.683+0000 7f6da8565b80 1 Processor -- start 2024-05-08T23:23:45.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bdbff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7be47100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6da8565b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6da8565b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bd92fb0 con 0x55dc7bdc3d80 2024-05-08T23:23:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6da8565b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7be47640 con 0x55dc7bdbc750 2024-05-08T23:23:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6da8565b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc7bd93200 con 0x55dc7bd97320 2024-05-08T23:23:45.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6da8565b80 10 client.0 _ll_register_callbacks cb 0x7f6d8c0076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6d9a948700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6d9a948700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35824/0 (socket says 172.21.15.5:35824) 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.684+0000 7f6d9a948700 1 -- 172.21.15.5:0/2190466284 learned_addr learned my addr 172.21.15.5:0/2190466284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bdbff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d99946700 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7be47100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d9a948700 1 -- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 msgr2=0x55dc7bdbff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d9a948700 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bdbff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d9a948700 1 -- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 msgr2=0x55dc7be47100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d9a948700 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7be47100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d9a948700 1 -- 172.21.15.5:0/2190466284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc7bb16b10 con 0x55dc7bdc3d80 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bdbff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d9a948700 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc7600 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c02d040 tx=0x7f6d8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9599520cb3777cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d757fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d757fa700 1 -- 172.21.15.5:0/2190466284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c012db0 con 0x55dc7bdc3d80 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d757fa700 1 -- 172.21.15.5:0/2190466284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d8c018470 con 0x55dc7bdc3d80 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6da8565b80 1 -- 172.21.15.5:0/2190466284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d84003a00 con 0x55dc7bdc3d80 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d757fa700 1 -- 172.21.15.5:0/2190466284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8c018650 con 0x55dc7bdc3d80 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6da8565b80 10 client.5934 fetch_fsmap learned FSMap version 31 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6da8565b80 1 -- 172.21.15.5:0/2190466284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55dc7bb6ec80 con 0x55dc7bdc3d80 2024-05-08T23:23:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d757fa700 1 -- 172.21.15.5:0/2190466284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84003a00 con 0x55dc7bdc3d80 2024-05-08T23:23:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6d757fa700 1 -- 172.21.15.5:0/2190466284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6d8c012240 con 0x55dc7bdc3d80 2024-05-08T23:23:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6da8565b80 10 client.5934 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:23:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6da8565b80 10 client.5934 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:23:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6da8565b80 20 client.5934 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.685+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x55dc7bd98ee0 0x55dc7bd992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 4 client.5934 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 -- 172.21.15.5:0/2190466284 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dc7bce1e70 con 0x55dc7bd98ee0 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x55dc7bd98ee0 0x55dc7bd992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6d9a147700 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x55dc7bd98ee0 0x55dc7bd992c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0d86dc33ec9016 server_cookie=1131457e2ecde3b5 in_seq=0 out_seq=0 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6d757fa700 10 client.5934 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6d757fa700 1 -- 172.21.15.5:0/2190466284 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+316 (crc 0 0 0) 0x55dc7bce1e70 con 0x55dc7bd98ee0 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6d757fa700 10 client.5934 handle_command_reply: tid=0 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 client.5934 shutdown 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 -- 172.21.15.5:0/2190466284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 msgr2=0x55dc7bdc7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc7600 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c02d040 tx=0x7f6d8c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 -- 172.21.15.5:0/2190466284 shutdown_connections 2024-05-08T23:23:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7bdc3d80 0x55dc7bdc7600 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x55dc7bd98ee0 0x55dc7bd992c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dc7bd97320 0x55dc7bdbff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 --2- 172.21.15.5:0/2190466284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc7bdbc750 0x55dc7be47100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6d757fa700 0 client.5934 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:23:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.686+0000 7f6da8565b80 1 -- 172.21.15.5:0/2190466284 >> 172.21.15.5:0/2190466284 conn(0x55dc7bcf71c0 msgr2=0x55dc7bdc6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6da8565b80 1 -- 172.21.15.5:0/2190466284 shutdown_connections 2024-05-08T23:23:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6da8565b80 1 -- 172.21.15.5:0/2190466284 wait complete. 2024-05-08T23:23:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6da8565b80 20 client.5934 trim_cache size 0 max 16384 2024-05-08T23:23:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:23:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:23:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:23:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5778, 2024-05-08T23:23:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:23:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:23:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:23:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.21912602, 2024-05-08T23:23:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 31, 2024-05-08T23:23:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 46, 2024-05-08T23:23:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 46, 2024-05-08T23:23:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 17.49113367 2024-05-08T23:23:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:23:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d7806ac90 msgr2=0x7f6d7806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d7806ac90 0x7f6d7806d150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009c40 tx=0x7f6d84008030 comp rx=0 tx=0).stop 2024-05-08T23:23:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/2681993694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 msgr2=0x7f6d94156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d94156500 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c4e0 tx=0x7f6d9000c8a0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/2681993694 shutdown_connections 2024-05-08T23:23:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d7806ac90 0x7f6d7806d150 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d940ce230 0x7f6d9409d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9406a960 0x7f6d94156500 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 --2- 172.21.15.5:0/2681993694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d940cc060 0x7f6d940b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/2681993694 >> 172.21.15.5:0/2681993694 conn(0x7f6d94099a50 msgr2=0x7f6d940b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/2681993694 shutdown_connections 2024-05-08T23:23:45.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.687+0000 7f6d9c3ab700 1 -- 172.21.15.5:0/2681993694 wait complete. 2024-05-08T23:23:45.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:23:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.839+0000 7f2a281f5700 1 Processor -- start 2024-05-08T23:23:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.839+0000 7f2a281f5700 1 -- start start 2024-05-08T23:23:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.840+0000 7f2a281f5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a2015b810 0x7f2a200cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.840+0000 7f2a281f5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a200d0520 0x7f2a20099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.840+0000 7f2a281f5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2009aca0 con 0x7f2a20099f90 2024-05-08T23:23:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.840+0000 7f2a281f5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2009a440 con 0x7f2a200d0520 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.840+0000 7f2a281f5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200cd9e0 con 0x7f2a2015b810 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.840+0000 7f2a25790700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a200d0520 0x7f2a20099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.840+0000 7f2a26792700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a20099f90 0x7f2a200ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50802/0 (socket says 172.21.15.5:50802) 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.840+0000 7f2a26792700 1 -- 172.21.15.5:0/1930037646 learned_addr learned my addr 172.21.15.5:0/1930037646 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a25f91700 1 --2- 172.21.15.5:0/1930037646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a2015b810 0x7f2a200cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a25790700 1 -- 172.21.15.5:0/1930037646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a2015b810 msgr2=0x7f2a200cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a25790700 1 --2- 172.21.15.5:0/1930037646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a2015b810 0x7f2a200cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a25790700 1 -- 172.21.15.5:0/1930037646 >> v1:172.21.15.5:6789/0 conn(0x7f2a20099f90 legacy=0x7f2a200ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a25790700 1 -- 172.21.15.5:0/1930037646 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a200cf220 con 0x7f2a200d0520 2024-05-08T23:23:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a25f91700 1 --2- 172.21.15.5:0/1930037646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a2015b810 0x7f2a200cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a24f8f700 1 -- 172.21.15.5:0/1930037646 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864357585 0 0) 0x7f2a2009aca0 con 0x7f2a20099f90 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a25790700 1 --2- 172.21.15.5:0/1930037646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a200d0520 0x7f2a20099a50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009840 tx=0x7f2a10009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47833cae53fde43f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a24f8f700 1 -- 172.21.15.5:0/1930037646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10011830 con 0x7f2a200d0520 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a24f8f700 1 -- 172.21.15.5:0/1930037646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a10011990 con 0x7f2a200d0520 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.841+0000 7f2a24f8f700 1 -- 172.21.15.5:0/1930037646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10011d20 con 0x7f2a200d0520 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.842+0000 7f2a281f5700 1 -- 172.21.15.5:0/1930037646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a200d0520 msgr2=0x7f2a20099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.842+0000 7f2a281f5700 1 --2- 172.21.15.5:0/1930037646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a200d0520 0x7f2a20099a50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009840 tx=0x7f2a10009b50 comp rx=0 tx=0).stop 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.842+0000 7f2a281f5700 1 -- 172.21.15.5:0/1930037646 shutdown_connections 2024-05-08T23:23:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.842+0000 7f2a281f5700 1 --2- 172.21.15.5:0/1930037646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a2015b810 0x7f2a200cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.842+0000 7f2a281f5700 1 --2- 172.21.15.5:0/1930037646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a200d0520 0x7f2a20099a50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.842+0000 7f2a281f5700 1 -- 172.21.15.5:0/1930037646 >> 172.21.15.5:0/1930037646 conn(0x7f2a200b3010 msgr2=0x7f2a200b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.842+0000 7f2a281f5700 1 -- 172.21.15.5:0/1930037646 shutdown_connections 2024-05-08T23:23:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.842+0000 7f2a281f5700 1 -- 172.21.15.5:0/1930037646 wait complete. 2024-05-08T23:23:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.843+0000 7f2a281f5700 1 Processor -- start 2024-05-08T23:23:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.843+0000 7f2a281f5700 1 -- start start 2024-05-08T23:23:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a281f5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a20099f90 0x7f2a201589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a281f5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a200d0520 0x7f2a2006c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a281f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a2015b810 0x7f2a2006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a281f5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200b4140 con 0x7f2a2015b810 2024-05-08T23:23:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a281f5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2006d4e0 con 0x7f2a20099f90 2024-05-08T23:23:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a281f5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2009c3f0 con 0x7f2a200d0520 2024-05-08T23:23:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a26792700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a2015b810 0x7f2a2006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a26792700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a2015b810 0x7f2a2006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35834/0 (socket says 172.21.15.5:35834) 2024-05-08T23:23:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a26792700 1 -- 172.21.15.5:0/3910012714 learned_addr learned my addr 172.21.15.5:0/3910012714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a25f91700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a20099f90 0x7f2a201589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.844+0000 7f2a25790700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a200d0520 0x7f2a2006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a26792700 1 -- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a200d0520 msgr2=0x7f2a2006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a26792700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a200d0520 0x7f2a2006c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a26792700 1 -- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a20099f90 msgr2=0x7f2a201589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a26792700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a20099f90 0x7f2a201589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a26792700 1 -- 172.21.15.5:0/3910012714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a100094a0 con 0x7f2a2015b810 2024-05-08T23:23:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a26792700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a2015b810 0x7f2a2006f2c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800cba0 tx=0x7f2a1800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a167fc700 1 -- 172.21.15.5:0/3910012714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800aaa0 con 0x7f2a2015b810 2024-05-08T23:23:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a167fc700 1 -- 172.21.15.5:0/3910012714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1800ac20 con 0x7f2a2015b810 2024-05-08T23:23:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2009c6d0 con 0x7f2a2015b810 2024-05-08T23:23:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a167fc700 1 -- 172.21.15.5:0/3910012714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a180070d0 con 0x7f2a2015b810 2024-05-08T23:23:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.845+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2009cc90 con 0x7f2a2015b810 2024-05-08T23:23:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.846+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a20068000 con 0x7f2a2015b810 2024-05-08T23:23:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.847+0000 7f2a167fc700 1 -- 172.21.15.5:0/3910012714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2a18007230 con 0x7f2a2015b810 2024-05-08T23:23:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.847+0000 7f2a167fc700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a0406acc0 0x7f2a0406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.847+0000 7f2a167fc700 1 -- 172.21.15.5:0/3910012714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a18088d00 con 0x7f2a2015b810 2024-05-08T23:23:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.847+0000 7f2a25f91700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a0406acc0 0x7f2a0406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.848+0000 7f2a25f91700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a0406acc0 0x7f2a0406d180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0099d0 tx=0x7f2a1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:45.852+0000 7f2a167fc700 1 -- 172.21.15.5:0/3910012714 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2a18053f70 con 0x7f2a2015b810 2024-05-08T23:23:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.119+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2a200b3c70 con 0x7f2a2015b810 2024-05-08T23:23:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.119+0000 7f2a167fc700 1 -- 172.21.15.5:0/3910012714 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2a18018070 con 0x7f2a2015b810 2024-05-08T23:23:46.119 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:23:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a0406acc0 msgr2=0x7f2a0406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a0406acc0 0x7f2a0406d180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0099d0 tx=0x7f2a1c008040 comp rx=0 tx=0).stop 2024-05-08T23:23:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a2015b810 msgr2=0x7f2a2006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a2015b810 0x7f2a2006f2c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800cba0 tx=0x7f2a1800ceb0 comp rx=0 tx=0).stop 2024-05-08T23:23:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 shutdown_connections 2024-05-08T23:23:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a0406acc0 0x7f2a0406d180 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a2015b810 0x7f2a2006f2c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a20099f90 0x7f2a201589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 --2- 172.21.15.5:0/3910012714 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a200d0520 0x7f2a2006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 >> 172.21.15.5:0/3910012714 conn(0x7f2a200b3010 msgr2=0x7f2a20159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 shutdown_connections 2024-05-08T23:23:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:46.122+0000 7f2a281f5700 1 -- 172.21.15.5:0/3910012714 wait complete. 2024-05-08T23:23:46.135 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T23:23:46.135 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:23:46.135 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-08T23:23:46.136 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:23:46.136 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:23:46.136 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T23:23:46.136 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T23:23:46.136 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:23:46.136 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:23:46.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.136 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-08T23:23:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42091sec preferred_lft 42091sec 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:23:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::1c2e:b3ff:fe6b:56f/64 scope link 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::84a4:2aff:fe7c:ce1a/64 scope link 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:23:46.159 INFO:teuthology.orchestra.run.smithi005.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:23:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether e2:3e:2c:7a:5d:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:23:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::e03e:2cff:fe7a:5db1/64 scope link 2024-05-08T23:23:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:23:46.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.161 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:23:46.228 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-08T23:23:46.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.231 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-08T23:23:46.296 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-08T23:23:46.296 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:23:46.296 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:23:46.296 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:23:46.296 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:23:46.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.297 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:23:46.297 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:23:46.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:23:46.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:23:46.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:23:46.298 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:23:46.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.420 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:23:46.441 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:23:46.441 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:23:46.441 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:23:46.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-08T23:23:46.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:23:46.518 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:23:46.581 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:23:46.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:23:46.581 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:23:46.643 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T23:23:46.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:23:46.742 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:23:46.742 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:23:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:23:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:23:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:23:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:23:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:23:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:23:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:23:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:23:46.782 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:23:46.782 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:23:46.782 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:23:46.782 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:23:46.783 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:23:46.783 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:23:46.783 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:23:46.783 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:23:46.783 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:23:46.783 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:23:46.783 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:23:46.785 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:23:46.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:23:46.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:46.818 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:23:46.912 INFO:teuthology.orchestra.run.smithi005.stdout:client5255 2024-05-08T23:23:46.912 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:23:46.912 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:23:46.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T23:23:47.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.124+0000 7fd8fc6f1700 1 Processor -- start 2024-05-08T23:23:47.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.124+0000 7fd8fc6f1700 1 -- start start 2024-05-08T23:23:47.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8fc6f1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd8f40d1e50 0x7fd8f40ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8fc6f1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8f40d2230 0x7fd8f4160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8fc6f1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f4161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8fc6f1700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f40cf560 con 0x7fd8f40d2230 2024-05-08T23:23:47.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8fc6f1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f40ce950 con 0x7fd8f40d1e50 2024-05-08T23:23:47.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8fc6f1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f40cec30 con 0x7fd8f40b2610 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8f9c8c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8f40d2230 0x7fd8f4160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8f9c8c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8f40d2230 0x7fd8f4160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35836/0 (socket says 172.21.15.5:35836) 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.125+0000 7fd8f9c8c700 1 -- 172.21.15.5:0/105521069 learned_addr learned my addr 172.21.15.5:0/105521069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8fac8e700 1 --2- 172.21.15.5:0/105521069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f4161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8fa48d700 1 --2- 172.21.15.5:0/105521069 >> v2:172.21.15.49:3300/0 conn(0x7fd8f40d1e50 0x7fd8f40ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f9c8c700 1 -- 172.21.15.5:0/105521069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 msgr2=0x7fd8f4161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f9c8c700 1 --2- 172.21.15.5:0/105521069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f4161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f9c8c700 1 -- 172.21.15.5:0/105521069 >> v2:172.21.15.49:3300/0 conn(0x7fd8f40d1e50 msgr2=0x7fd8f40ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f9c8c700 1 --2- 172.21.15.5:0/105521069 >> v2:172.21.15.49:3300/0 conn(0x7fd8f40d1e50 0x7fd8f40ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f9c8c700 1 -- 172.21.15.5:0/105521069 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f40cef10 con 0x7fd8f40d2230 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8fac8e700 1 --2- 172.21.15.5:0/105521069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f4161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f9c8c700 1 --2- 172.21.15.5:0/105521069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d2230 0x7fd8f4160c10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd8f408d7c0 tx=0x7fd8e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e409198292b2e344 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f948b700 1 -- 172.21.15.5:0/105521069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4011830 con 0x7fd8f40d2230 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f948b700 1 -- 172.21.15.5:0/105521069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e4011990 con 0x7fd8f40d2230 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.126+0000 7fd8f948b700 1 -- 172.21.15.5:0/105521069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4011d20 con 0x7fd8f40d2230 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/105521069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d2230 msgr2=0x7fd8f4160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/105521069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d2230 0x7fd8f4160c10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd8f408d7c0 tx=0x7fd8e4009bd0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/105521069 shutdown_connections 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/105521069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f4161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/105521069 >> v2:172.21.15.49:3300/0 conn(0x7fd8f40d1e50 0x7fd8f40ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/105521069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d2230 0x7fd8f4160c10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/105521069 >> 172.21.15.5:0/105521069 conn(0x7fd8f408afe0 msgr2=0x7fd8f40d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/105521069 shutdown_connections 2024-05-08T23:23:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.127+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/105521069 wait complete. 2024-05-08T23:23:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.128+0000 7fd8fc6f1700 1 Processor -- start 2024-05-08T23:23:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.128+0000 7fd8fc6f1700 1 -- start start 2024-05-08T23:23:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8fc6f1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8fc6f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d1e50 0x7fd8f415cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8fc6f1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f40d2230 0x7fd8f40c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8fc6f1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f40c8c30 con 0x7fd8f40d1e50 2024-05-08T23:23:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8fc6f1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f40c8f10 con 0x7fd8f40d2230 2024-05-08T23:23:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8fc6f1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f40c91f0 con 0x7fd8f40b2610 2024-05-08T23:23:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8f9c8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d1e50 0x7fd8f415cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8f9c8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d1e50 0x7fd8f415cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35844/0 (socket says 172.21.15.5:35844) 2024-05-08T23:23:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.129+0000 7fd8f9c8c700 1 -- 172.21.15.5:0/4211575289 learned_addr learned my addr 172.21.15.5:0/4211575289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8fa48d700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8fac8e700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f40d2230 0x7fd8f40c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8f9c8c700 1 -- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 msgr2=0x7fd8f40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8f9c8c700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f40c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8f9c8c700 1 -- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f40d2230 msgr2=0x7fd8f40c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8f9c8c700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f40d2230 0x7fd8f40c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8f9c8c700 1 -- 172.21.15.5:0/4211575289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4009550 con 0x7fd8f40d1e50 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8f9c8c700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d1e50 0x7fd8f415cfe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009c10 tx=0x7fd8e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8eaffd700 1 -- 172.21.15.5:0/4211575289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4004450 con 0x7fd8f40d1e50 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8eaffd700 1 -- 172.21.15.5:0/4211575289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e401e070 con 0x7fd8f40d1e50 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f40c94d0 con 0x7fd8f40d1e50 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8eaffd700 1 -- 172.21.15.5:0/4211575289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40038e0 con 0x7fd8f40d1e50 2024-05-08T23:23:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.130+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f415d520 con 0x7fd8f40d1e50 2024-05-08T23:23:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.131+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f40cff80 con 0x7fd8f40d1e50 2024-05-08T23:23:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.132+0000 7fd8eaffd700 1 -- 172.21.15.5:0/4211575289 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd8e4003a40 con 0x7fd8f40d1e50 2024-05-08T23:23:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.132+0000 7fd8eaffd700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8d806acc0 0x7fd8d806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.132+0000 7fd8eaffd700 1 -- 172.21.15.5:0/4211575289 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8e408a0b0 con 0x7fd8f40d1e50 2024-05-08T23:23:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.133+0000 7fd8fa48d700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8d806acc0 0x7fd8d806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.133+0000 7fd8fa48d700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8d806acc0 0x7fd8d806d180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009be0 tx=0x7fd8f0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:47.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.137+0000 7fd8eaffd700 1 -- 172.21.15.5:0/4211575289 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd8e4055280 con 0x7fd8f40d1e50 2024-05-08T23:23:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.404+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fd8f4074bc0 con 0x7fd8f40d1e50 2024-05-08T23:23:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.649+0000 7fd8eaffd700 1 -- 172.21.15.5:0/4211575289 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v32) v1 ==== 105+0+0 (secure 0 0 0) 0x7fd8e40588a0 con 0x7fd8f40d1e50 2024-05-08T23:23:47.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8d806acc0 msgr2=0x7fd8d806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8d806acc0 0x7fd8d806d180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009be0 tx=0x7fd8f0009210 comp rx=0 tx=0).stop 2024-05-08T23:23:47.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d1e50 msgr2=0x7fd8f415cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d1e50 0x7fd8f415cfe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009c10 tx=0x7fd8e4004750 comp rx=0 tx=0).stop 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 shutdown_connections 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd8d806acc0 0x7fd8d806d180 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f40d2230 0x7fd8f40c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd8f40b2610 0x7fd8f40c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 --2- 172.21.15.5:0/4211575289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8f40d1e50 0x7fd8f415cfe0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 >> 172.21.15.5:0/4211575289 conn(0x7fd8f408afe0 msgr2=0x7fd8f40b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 shutdown_connections 2024-05-08T23:23:47.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.653+0000 7fd8fc6f1700 1 -- 172.21.15.5:0/4211575289 wait complete. 2024-05-08T23:23:47.673 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:47.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.812+0000 7fadd4f3e700 1 Processor -- start 2024-05-08T23:23:47.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.812+0000 7fadd4f3e700 1 -- start start 2024-05-08T23:23:47.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadd4f3e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadd4f3e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fadd00cbfc0 0x7fadd00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadd4f3e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd00cca10 con 0x7fadd00b1260 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadd4f3e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd015c740 con 0x7fadd00cbfc0 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadd4f3e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd015c080 con 0x7fadd00d1070 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadce59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fadd00b1260 0x7fadd00b1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55030/0 (socket says 172.21.15.5:55030) 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadce59c700 1 -- 172.21.15.5:0/1618888192 learned_addr learned my addr 172.21.15.5:0/1618888192 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadced9d700 1 --2- 172.21.15.5:0/1618888192 >> v2:172.21.15.49:3300/0 conn(0x7fadd00cbfc0 0x7fadd00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.813+0000 7fadcdd9b700 1 --2- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadcd59a700 1 -- 172.21.15.5:0/1618888192 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531444610 0 0) 0x7fadd00cca10 con 0x7fadd00b1260 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadcd59a700 1 -- 172.21.15.5:0/1618888192 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadb80035b0 con 0x7fadd00b1260 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadced9d700 1 -- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 msgr2=0x7fadd015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadced9d700 1 --2- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadced9d700 1 -- 172.21.15.5:0/1618888192 >> v1:172.21.15.5:6789/0 conn(0x7fadd00b1260 legacy=0x7fadd00b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadced9d700 1 -- 172.21.15.5:0/1618888192 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd015c360 con 0x7fadd00cbfc0 2024-05-08T23:23:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadcdd9b700 1 --2- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadced9d700 1 --2- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 0x7fadd00cc430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009d90 tx=0x7fadc800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eccd3cea79836052 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadcd59a700 1 -- 172.21.15.5:0/1618888192 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800e070 con 0x7fadd00cbfc0 2024-05-08T23:23:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadcd59a700 1 -- 172.21.15.5:0/1618888192 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc800bb20 con 0x7fadd00cbfc0 2024-05-08T23:23:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.814+0000 7fadcd59a700 1 -- 172.21.15.5:0/1618888192 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8012430 con 0x7fadd00cbfc0 2024-05-08T23:23:47.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.815+0000 7fadd4f3e700 1 -- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 msgr2=0x7fadd00cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.815+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 0x7fadd00cc430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009d90 tx=0x7fadc800b330 comp rx=0 tx=0).stop 2024-05-08T23:23:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.815+0000 7fadd4f3e700 1 -- 172.21.15.5:0/1618888192 shutdown_connections 2024-05-08T23:23:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.815+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 0x7fadd00cc430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.815+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/1618888192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.815+0000 7fadd4f3e700 1 -- 172.21.15.5:0/1618888192 >> 172.21.15.5:0/1618888192 conn(0x7fadd0099a50 msgr2=0x7fadd00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.815+0000 7fadd4f3e700 1 -- 172.21.15.5:0/1618888192 shutdown_connections 2024-05-08T23:23:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.815+0000 7fadd4f3e700 1 -- 172.21.15.5:0/1618888192 wait complete. 2024-05-08T23:23:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.817+0000 7fadd4f3e700 1 Processor -- start 2024-05-08T23:23:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.817+0000 7fadd4f3e700 1 -- start start 2024-05-08T23:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadd4f3e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd00b1260 0x7fadd0150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadd4f3e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 0x7fadd00c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadd4f3e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd00c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadd4f3e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd00c6d10 con 0x7fadd00b1260 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadd4f3e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd00cf680 con 0x7fadd00cbfc0 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadd4f3e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd00c6100 con 0x7fadd00d1070 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd00b1260 0x7fadd0150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd00b1260 0x7fadd0150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59504/0 (socket says 172.21.15.5:59504) 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadce59c700 1 -- 172.21.15.5:0/2915942618 learned_addr learned my addr 172.21.15.5:0/2915942618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.818+0000 7fadcdd9b700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 0x7fadd00c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadced9d700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd00c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadce59c700 1 -- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 msgr2=0x7fadd00c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadce59c700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd00c5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadce59c700 1 -- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 msgr2=0x7fadd00c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadce59c700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 0x7fadd00c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadce59c700 1 -- 172.21.15.5:0/2915942618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc40094a0 con 0x7fadd00b1260 2024-05-08T23:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadce59c700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd00b1260 0x7fadd0150ca0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c700 tx=0x7fadc000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadbeffd700 1 -- 172.21.15.5:0/2915942618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0005d70 con 0x7fadd00b1260 2024-05-08T23:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadbeffd700 1 -- 172.21.15.5:0/2915942618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc00070d0 con 0x7fadd00b1260 2024-05-08T23:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc8009a70 con 0x7fadd00b1260 2024-05-08T23:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadbeffd700 1 -- 172.21.15.5:0/2915942618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0007460 con 0x7fadd00b1260 2024-05-08T23:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.819+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd00c66c0 con 0x7fadd00b1260 2024-05-08T23:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.820+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd0068000 con 0x7fadd00b1260 2024-05-08T23:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.821+0000 7fadbeffd700 1 -- 172.21.15.5:0/2915942618 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fadc00075c0 con 0x7fadd00b1260 2024-05-08T23:23:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.821+0000 7fadbeffd700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fadb006ac90 0x7fadb006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.821+0000 7fadbeffd700 1 -- 172.21.15.5:0/2915942618 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadc0088950 con 0x7fadd00b1260 2024-05-08T23:23:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.821+0000 7fadcdd9b700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fadb006ac90 0x7fadb006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.822+0000 7fadcdd9b700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fadb006ac90 0x7fadb006d150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fadd00cf340 tx=0x7fadc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:47.826+0000 7fadbeffd700 1 -- 172.21.15.5:0/2915942618 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fadc0053b40 con 0x7fadd00b1260 2024-05-08T23:23:48.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.094+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadd006b050 con 0x7fadd00b1260 2024-05-08T23:23:48.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.095+0000 7fadbeffd700 1 -- 172.21.15.5:0/2915942618 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fadc0057160 con 0x7fadd00b1260 2024-05-08T23:23:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:48.095 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:23:47.650788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5778,"mds_1":5566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5566":{"gid":5566,"name":"b","rank":1,"incarnation":33,"state":"up:creating","state_seq":1,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:23:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fadb006ac90 msgr2=0x7fadb006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fadb006ac90 0x7fadb006d150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fadd00cf340 tx=0x7fadc4008040 comp rx=0 tx=0).stop 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd00b1260 msgr2=0x7fadd0150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd00b1260 0x7fadd0150ca0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c700 tx=0x7fadc000cac0 comp rx=0 tx=0).stop 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 shutdown_connections 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fadb006ac90 0x7fadb006d150 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fadd00d1070 0x7fadd00c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd00b1260 0x7fadd0150ca0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 --2- 172.21.15.5:0/2915942618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadd00cbfc0 0x7fadd00c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 >> 172.21.15.5:0/2915942618 conn(0x7fadd0099a50 msgr2=0x7fadd015dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 shutdown_connections 2024-05-08T23:23:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:48.097+0000 7fadd4f3e700 1 -- 172.21.15.5:0/2915942618 wait complete. 2024-05-08T23:23:48.099 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-05-08T23:23:48.514 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:23:39.466617+0000', 'modified': '2024-05-08T23:23:47.650788+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5778, 'mds_1': 5566}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5566': {'gid': 5566, 'name': 'b', 'rank': 1, 'incarnation': 33, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.49:6835/962044603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 962044603}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 962044603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5778': {'gid': 5778, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6835/3359342067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 3359342067}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3359342067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:23:48.515 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5566:up:creating 2024-05-08T23:23:48.986 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:23:48.986+0000 7f5d502e2700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:23:49.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.663+0000 7f773dccc700 1 Processor -- start 2024-05-08T23:23:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.663+0000 7f773dccc700 1 -- start start 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.664+0000 7f773dccc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f773815db00 0x7f773806dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.664+0000 7f773dccc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77380cb2a0 0x7f77380cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.664+0000 7f773dccc700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773806fbf0 con 0x7f77380ce120 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.664+0000 7f773dccc700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773806edb0 con 0x7f77380cb2a0 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.664+0000 7f773dccc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773806f090 con 0x7f773815db00 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.664+0000 7f7736ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f77380ce120 0x7f773806e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55046/0 (socket says 172.21.15.5:55046) 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.664+0000 7f7736ffd700 1 -- 172.21.15.5:0/3903519278 learned_addr learned my addr 172.21.15.5:0/3903519278 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77377fe700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f773815db00 0x7f773806dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f7737fff700 1 --2- 172.21.15.5:0/3903519278 >> v2:172.21.15.49:3300/0 conn(0x7f77380cb2a0 0x7f77380cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547825137 0 0) 0x7f773806fbf0 con 0x7f77380ce120 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77200035b0 con 0x7f77380ce120 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 10136849 0 0) 0x7f77200035b0 con 0x7f77380ce120 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f773815db00 msgr2=0x7f773806dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f773815db00 0x7f773806dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 >> v2:172.21.15.49:3300/0 conn(0x7f77380cb2a0 msgr2=0x7f77380cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 --2- 172.21.15.5:0/3903519278 >> v2:172.21.15.49:3300/0 conn(0x7f77380cb2a0 0x7f77380cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773806f370 con 0x7f77380ce120 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f772c001670 con 0x7f77380ce120 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 >> v1:172.21.15.5:6789/0 conn(0x7f77380ce120 legacy=0x7f773806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7720003a00 0x7f7720003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7720004850 0x7f7720004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f773815db00 0x7f7720005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773806edb0 con 0x7f7720004850 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773806f090 con 0x7f7720003a00 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772000ba10 con 0x7f773815db00 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f7736ffd700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7720004850 0x7f7720004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f77377fe700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7720003a00 0x7f7720003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f7737fff700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f773815db00 0x7f7720005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f7736ffd700 1 -- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f773815db00 msgr2=0x7f7720005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f7736ffd700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f773815db00 0x7f7720005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f7736ffd700 1 -- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7720003a00 msgr2=0x7f7720003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f7736ffd700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7720003a00 0x7f7720003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.665+0000 7f7736ffd700 1 -- 172.21.15.5:0/3903519278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200035b0 con 0x7f7720004850 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.666+0000 7f7736ffd700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7720004850 0x7f7720004cd0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c190 tx=0x7f7720005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c424e61d946ef6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.666+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c007b70 con 0x7f7720004850 2024-05-08T23:23:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.666+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c00ccd0 con 0x7f7720004850 2024-05-08T23:23:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.666+0000 7f77367fc700 1 -- 172.21.15.5:0/3903519278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c005ba0 con 0x7f7720004850 2024-05-08T23:23:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.666+0000 7f773dccc700 1 -- 172.21.15.5:0/3903519278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7720004850 msgr2=0x7f7720004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.666+0000 7f773dccc700 1 --2- 172.21.15.5:0/3903519278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7720004850 0x7f7720004cd0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c190 tx=0x7f7720005880 comp rx=0 tx=0).stop 2024-05-08T23:23:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.666+0000 7f7737fff700 1 -- 172.21.15.5:0/3903519278 reap_dead start 2024-05-08T23:23:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.666+0000 7f773dccc700 1 -- 172.21.15.5:0/3903519278 shutdown_connections 2024-05-08T23:23:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.667+0000 7f773dccc700 1 -- 172.21.15.5:0/3903519278 >> 172.21.15.5:0/3903519278 conn(0x7f7738099a50 msgr2=0x7f77380d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.667+0000 7f773dccc700 1 -- 172.21.15.5:0/3903519278 shutdown_connections 2024-05-08T23:23:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.667+0000 7f773dccc700 1 -- 172.21.15.5:0/3903519278 wait complete. 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.668+0000 7f773dccc700 1 Processor -- start 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.668+0000 7f773dccc700 1 -- start start 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f773dccc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7720004850 0x7f7738150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f773dccc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77380c86f0 0x7f77380d0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f773dccc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77380c8b70 0x7f77380d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f773dccc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77380ca1c0 con 0x7f77380c8b70 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f773dccc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77380c9120 con 0x7f77380c86f0 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f773dccc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77380c9400 con 0x7f7720004850 2024-05-08T23:23:49.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7737fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77380c8b70 0x7f77380d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7737fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77380c8b70 0x7f77380d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59518/0 (socket says 172.21.15.5:59518) 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7737fff700 1 -- 172.21.15.5:0/3560588722 learned_addr learned my addr 172.21.15.5:0/3560588722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7736ffd700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77380c86f0 0x7f77380d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f77377fe700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7720004850 0x7f7738150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7737fff700 1 -- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7720004850 msgr2=0x7f7738150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7737fff700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7720004850 0x7f7738150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7737fff700 1 -- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77380c86f0 msgr2=0x7f77380d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7737fff700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77380c86f0 0x7f77380d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7737fff700 1 -- 172.21.15.5:0/3560588722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77380c96e0 con 0x7f77380c8b70 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.669+0000 7f7736ffd700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77380c86f0 0x7f77380d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.670+0000 7f7737fff700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77380c8b70 0x7f77380d0dc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7738070650 tx=0x7f773000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.670+0000 7f771ffff700 1 -- 172.21.15.5:0/3560588722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773000e040 con 0x7f77380c8b70 2024-05-08T23:23:49.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.670+0000 7f771ffff700 1 -- 172.21.15.5:0/3560588722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773000f040 con 0x7f77380c8b70 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.670+0000 7f771ffff700 1 -- 172.21.15.5:0/3560588722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730015430 con 0x7f77380c8b70 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.670+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77380c99c0 con 0x7f77380c8b70 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.670+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7738150d70 con 0x7f77380c8b70 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.671+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77380c9ee0 con 0x7f77380c8b70 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.671+0000 7f771ffff700 1 -- 172.21.15.5:0/3560588722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7730015590 con 0x7f77380c8b70 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.672+0000 7f771ffff700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f772406ac90 0x7f772406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.672+0000 7f771ffff700 1 -- 172.21.15.5:0/3560588722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7730089840 con 0x7f77380c8b70 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.672+0000 7f77377fe700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f772406ac90 0x7f772406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.672+0000 7f77377fe700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f772406ac90 0x7f772406d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7728009940 tx=0x7f7728008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.676+0000 7f771ffff700 1 -- 172.21.15.5:0/3560588722 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f77380c9ee0 con 0x7f77380c8b70 2024-05-08T23:23:49.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.944+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7738071f00 con 0x7f77380c8b70 2024-05-08T23:23:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.945+0000 7f771ffff700 1 -- 172.21.15.5:0/3560588722 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4750 (secure 0 0 0) 0x7f7730054ab0 con 0x7f77380c8b70 2024-05-08T23:23:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:23:49.656410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5778,"mds_1":5566,"mds_2":5180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5180":{"gid":5180,"name":"d","rank":2,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5566":{"gid":5566,"name":"b","rank":1,"incarnation":33,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:23:49.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f772406ac90 msgr2=0x7f772406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:49.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f772406ac90 0x7f772406d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7728009940 tx=0x7f7728008040 comp rx=0 tx=0).stop 2024-05-08T23:23:49.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77380c8b70 msgr2=0x7f77380d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:49.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77380c8b70 0x7f77380d0dc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7738070650 tx=0x7f773000f490 comp rx=0 tx=0).stop 2024-05-08T23:23:49.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 shutdown_connections 2024-05-08T23:23:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f772406ac90 0x7f772406d150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77380c8b70 0x7f77380d0dc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7720004850 0x7f7738150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 --2- 172.21.15.5:0/3560588722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77380c86f0 0x7f77380d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 >> 172.21.15.5:0/3560588722 conn(0x7f7738099a50 msgr2=0x7f77380ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 shutdown_connections 2024-05-08T23:23:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:49.947+0000 7f773dccc700 1 -- 172.21.15.5:0/3560588722 wait complete. 2024-05-08T23:23:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-05-08T23:23:50.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:23:39.466617+0000', 'modified': '2024-05-08T23:23:49.656410+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5778, 'mds_1': 5566, 'mds_2': 5180}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5180': {'gid': 5180, 'name': 'd', 'rank': 2, 'incarnation': 35, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.49:6837/350829837', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 350829837}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 350829837}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5566': {'gid': 5566, 'name': 'b', 'rank': 1, 'incarnation': 33, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6835/962044603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 962044603}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 962044603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_5778': {'gid': 5778, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6835/3359342067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 3359342067}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3359342067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:23:50.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:23:50.364 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T23:23:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.509+0000 7f70a5fac700 1 Processor -- start 2024-05-08T23:23:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.509+0000 7f70a5fac700 1 -- start start 2024-05-08T23:23:50.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f70a5fac700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70a009a160 0x7f70a00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f70a5fac700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a00b4280 con 0x7f70a015ba00 2024-05-08T23:23:50.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f70a5fac700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a00b4560 con 0x7f70a00b2510 2024-05-08T23:23:50.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f70a5fac700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a00cccf0 con 0x7f70a009a160 2024-05-08T23:23:50.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f709effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70a009a160 0x7f70a00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f709effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70a009a160 0x7f70a00b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59522/0 (socket says 172.21.15.5:59522) 2024-05-08T23:23:50.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f709f7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f70a015ba00 0x7f70a0099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55056/0 (socket says 172.21.15.5:55056) 2024-05-08T23:23:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f709effd700 1 -- 172.21.15.5:0/4149210545 learned_addr learned my addr 172.21.15.5:0/4149210545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f709effd700 1 -- 172.21.15.5:0/4149210545 >> v1:172.21.15.49:6789/0 conn(0x7f70a00b2510 legacy=0x7f70a00b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f709effd700 1 -- 172.21.15.5:0/4149210545 >> v1:172.21.15.5:6789/0 conn(0x7f70a015ba00 legacy=0x7f70a0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f709effd700 1 -- 172.21.15.5:0/4149210545 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00ccfd0 con 0x7f70a009a160 2024-05-08T23:23:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.510+0000 7f709e7fc700 1 -- 172.21.15.5:0/4149210545 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615836700 0 0) 0x7f70a00b4280 con 0x7f70a015ba00 2024-05-08T23:23:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.511+0000 7f709effd700 1 --2- 172.21.15.5:0/4149210545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a00b3a40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f70940098f0 tx=0x7f7094009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aec006dec98763 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.511+0000 7f709e7fc700 1 -- 172.21.15.5:0/4149210545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709400b8a0 con 0x7f70a009a160 2024-05-08T23:23:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.511+0000 7f709e7fc700 1 -- 172.21.15.5:0/4149210545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709400ba00 con 0x7f70a009a160 2024-05-08T23:23:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.511+0000 7f709e7fc700 1 -- 172.21.15.5:0/4149210545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709400bd90 con 0x7f70a009a160 2024-05-08T23:23:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.511+0000 7f70a5fac700 1 -- 172.21.15.5:0/4149210545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 msgr2=0x7f70a00b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.511+0000 7f70a5fac700 1 --2- 172.21.15.5:0/4149210545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a00b3a40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f70940098f0 tx=0x7f7094009c00 comp rx=0 tx=0).stop 2024-05-08T23:23:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.511+0000 7f70a5fac700 1 -- 172.21.15.5:0/4149210545 shutdown_connections 2024-05-08T23:23:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.511+0000 7f70a5fac700 1 --2- 172.21.15.5:0/4149210545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a00b3a40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.512+0000 7f70a5fac700 1 -- 172.21.15.5:0/4149210545 >> 172.21.15.5:0/4149210545 conn(0x7f70a00b2f20 msgr2=0x7f70a00b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.512+0000 7f70a5fac700 1 -- 172.21.15.5:0/4149210545 shutdown_connections 2024-05-08T23:23:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.512+0000 7f70a5fac700 1 -- 172.21.15.5:0/4149210545 wait complete. 2024-05-08T23:23:50.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.513+0000 7f70a5fac700 1 Processor -- start 2024-05-08T23:23:50.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.513+0000 7f70a5fac700 1 -- start start 2024-05-08T23:23:50.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f70a5fac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a009b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f70a5fac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a00b2510 0x7f70a009db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f70a5fac700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a015ba00 0x7f70a009e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f70a5fac700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a014a110 con 0x7f70a009a160 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f70a5fac700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a014a3f0 con 0x7f70a00b2510 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f70a5fac700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a014a6d0 con 0x7f70a015ba00 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a009b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a009b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59530/0 (socket says 172.21.15.5:59530) 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 -- 172.21.15.5:0/3050537373 learned_addr learned my addr 172.21.15.5:0/3050537373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709effd700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a00b2510 0x7f70a009db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709ffff700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a015ba00 0x7f70a009e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 -- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a015ba00 msgr2=0x7f70a009e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a015ba00 0x7f70a009e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 -- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a00b2510 msgr2=0x7f70a009db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a00b2510 0x7f70a009db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 -- 172.21.15.5:0/3050537373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009550 con 0x7f70a009a160 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.514+0000 7f709f7fe700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a009b250 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f709000c520 tx=0x7f709000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.515+0000 7f7087fff700 1 -- 172.21.15.5:0/3050537373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090004c80 con 0x7f70a009a160 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.515+0000 7f7087fff700 1 -- 172.21.15.5:0/3050537373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090004e20 con 0x7f70a009a160 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.515+0000 7f70a5fac700 1 -- 172.21.15.5:0/3050537373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a014a9b0 con 0x7f70a009a160 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.515+0000 7f7087fff700 1 -- 172.21.15.5:0/3050537373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090007310 con 0x7f70a009a160 2024-05-08T23:23:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.515+0000 7f70a5fac700 1 -- 172.21.15.5:0/3050537373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a014b1b0 con 0x7f70a009a160 2024-05-08T23:23:50.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.516+0000 7f7087fff700 1 -- 172.21.15.5:0/3050537373 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7090007470 con 0x7f70a009a160 2024-05-08T23:23:50.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.517+0000 7f7087fff700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f708c06ac90 0x7f708c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.517+0000 7f7087fff700 1 -- 172.21.15.5:0/3050537373 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7090089110 con 0x7f70a009a160 2024-05-08T23:23:50.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.517+0000 7f709effd700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f708c06ac90 0x7f708c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.517+0000 7f709effd700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f708c06ac90 0x7f708c06d150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7094009840 tx=0x7f7094008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.519+0000 7f70b2166b80 1 Processor -- start 2024-05-08T23:23:50.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.519+0000 7f70b2166b80 1 -- start start 2024-05-08T23:23:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af2858f320 0x55af28592050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af285bbd80 0x55af285bc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f70b2166b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af284e8100 con 0x55af285bbd80 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f70b2166b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af284e5ac0 con 0x55af2858f320 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f70b2166b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af284e13b0 con 0x55af285b4750 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af285bbd80 0x55af285bc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af285bbd80 0x55af285bc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59542/0 (socket says 172.21.15.5:59542) 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709ffff700 1 -- 172.21.15.5:0/584200763 learned_addr learned my addr 172.21.15.5:0/584200763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709effd700 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af2858f320 0x55af28592050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709f7fe700 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709ffff700 1 -- 172.21.15.5:0/584200763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 msgr2=0x55af2858d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709ffff700 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709ffff700 1 -- 172.21.15.5:0/584200763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af2858f320 msgr2=0x55af28592050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709ffff700 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af2858f320 0x55af28592050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.520+0000 7f709ffff700 1 -- 172.21.15.5:0/584200763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af28362360 con 0x55af285bbd80 2024-05-08T23:23:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.521+0000 7f709ffff700 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af285bbd80 0x55af285bc200 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f70980020b0 tx=0x7f709800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c3101f49a5380d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.521+0000 7f70857fa700 1 -- 172.21.15.5:0/584200763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709800ec60 con 0x55af285bbd80 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.521+0000 7f70857fa700 1 -- 172.21.15.5:0/584200763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709800edc0 con 0x55af285bbd80 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.521+0000 7f70857fa700 1 -- 172.21.15.5:0/584200763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098003c40 con 0x55af285bbd80 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.521+0000 7f70b2166b80 1 -- 172.21.15.5:0/584200763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af285bbd80 msgr2=0x55af285bc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.521+0000 7f70b2166b80 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af285bbd80 0x55af285bc200 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f70980020b0 tx=0x7f709800e440 comp rx=0 tx=0).stop 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 -- 172.21.15.5:0/584200763 shutdown_connections 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af285bbd80 0x55af285bc200 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 --2- 172.21.15.5:0/584200763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af2858f320 0x55af28592050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 -- 172.21.15.5:0/584200763 >> 172.21.15.5:0/584200763 conn(0x55af284ef1c0 msgr2=0x55af285d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 -- 172.21.15.5:0/584200763 shutdown_connections 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 -- 172.21.15.5:0/584200763 wait complete. 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 -- start start 2024-05-08T23:23:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.522+0000 7f70b2166b80 1 Processor -- start 2024-05-08T23:23:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285b4750 0x55af285da010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285bbd80 0x55af285bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f70b2166b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af2847d510 con 0x55af2858f320 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f70b2166b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af2847d1b0 con 0x55af285b4750 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f70b2166b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af28364650 con 0x55af285bbd80 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f70b2166b80 10 client.0 _ll_register_callbacks cb 0x55af28486ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f709f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f709f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59548/0 (socket says 172.21.15.5:59548) 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.523+0000 7f709f7fe700 1 -- 172.21.15.5:0/2046017937 learned_addr learned my addr 172.21.15.5:0/2046017937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f709effd700 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285b4750 0x55af285da010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f709ffff700 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285bbd80 0x55af285bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f709f7fe700 1 -- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285bbd80 msgr2=0x55af285bc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f709f7fe700 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285bbd80 0x55af285bc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f709f7fe700 1 -- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285b4750 msgr2=0x55af285da010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f709f7fe700 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285b4750 0x55af285da010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f709f7fe700 1 -- 172.21.15.5:0/2046017937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709800f040 con 0x55af2858f320 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f709f7fe700 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285b3470 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f709000cce0 tx=0x7f7090057a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c598aa4c3d64a7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f707e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f707e7fc700 1 -- 172.21.15.5:0/2046017937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090058370 con 0x55af2858f320 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f707e7fc700 1 -- 172.21.15.5:0/2046017937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7090057f10 con 0x55af2858f320 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f70b2166b80 1 -- 172.21.15.5:0/2046017937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af2843a220 con 0x55af2858f320 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f707e7fc700 1 -- 172.21.15.5:0/2046017937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709005bd40 con 0x55af2858f320 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f70b2166b80 10 client.5976 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f70b2166b80 1 -- 172.21.15.5:0/2046017937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55af2830eb10 con 0x55af2858f320 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.524+0000 7f707e7fc700 1 -- 172.21.15.5:0/2046017937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af2843a220 con 0x55af2858f320 2024-05-08T23:23:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f707e7fc700 1 -- 172.21.15.5:0/2046017937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f7090057400 con 0x55af2858f320 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f70b2166b80 10 client.5976 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f70b2166b80 10 client.5976 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f70b2166b80 20 client.5976 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f70b2166b80 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55af285d8940 0x55af285f3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f70b2166b80 4 client.5976 mds_command: new command op to 5180 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f70b2166b80 1 -- 172.21.15.5:0/2046017937 --> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af28560790 con 0x55af285d8940 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f709effd700 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55af285d8940 0x55af285f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f707e7fc700 10 client.5976 ms_handle_connect on v2:172.21.15.49:6836/350829837 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.525+0000 7f709effd700 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55af285d8940 0x55af285f3980 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a17cce070b83d5dc server_cookie=1f9e9d0ace366844 in_seq=0 out_seq=0 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f707e7fc700 1 -- 172.21.15.5:0/2046017937 <== mds.2 v2:172.21.15.49:6836/350829837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af28560790 con 0x55af285d8940 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f707e7fc700 10 client.5976 handle_command_reply: tid=0 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 client.5976 shutdown 2024-05-08T23:23:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 -- 172.21.15.5:0/2046017937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 msgr2=0x55af285b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285b3470 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f709000cce0 tx=0x7f7090057a50 comp rx=0 tx=0).stop 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 -- 172.21.15.5:0/2046017937 shutdown_connections 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285bbd80 0x55af285bc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285b3470 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55af285d8940 0x55af285f3980 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 --2- 172.21.15.5:0/2046017937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285b4750 0x55af285da010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f707e7fc700 0 client.5976 ms_handle_reset on v2:172.21.15.49:6836/350829837 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.526+0000 7f70b2166b80 1 -- 172.21.15.5:0/2046017937 >> 172.21.15.5:0/2046017937 conn(0x55af284ef1c0 msgr2=0x55af285912c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.527+0000 7f70b2166b80 1 -- 172.21.15.5:0/2046017937 shutdown_connections 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.527+0000 7f70b2166b80 1 -- 172.21.15.5:0/2046017937 wait complete. 2024-05-08T23:23:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.527+0000 7f70b2166b80 20 client.5976 trim_cache size 0 max 16384 2024-05-08T23:23:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.546+0000 7f70b2166b80 1 Processor -- start 2024-05-08T23:23:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.546+0000 7f70b2166b80 1 -- start start 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285efb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 0x55af2858b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f70b2166b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af285bc1d0 con 0x55af2858f320 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f70b2166b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af285f0290 con 0x55af285bbd80 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f70b2166b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af285d7990 con 0x55af285b4750 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59562/0 (socket says 172.21.15.5:59562) 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 -- 172.21.15.5:0/984553514 learned_addr learned my addr 172.21.15.5:0/984553514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709ffff700 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 0x55af2858b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709effd700 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 -- 172.21.15.5:0/984553514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 msgr2=0x55af2858ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 -- 172.21.15.5:0/984553514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 msgr2=0x55af2858b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 0x55af2858b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 -- 172.21.15.5:0/984553514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af284a7510 con 0x55af2858f320 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.547+0000 7f709f7fe700 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285efb60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7090054930 tx=0x7f709001a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32e34e7b0712a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f7085ffb700 1 -- 172.21.15.5:0/984553514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000e040 con 0x55af2858f320 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f7085ffb700 1 -- 172.21.15.5:0/984553514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090057a50 con 0x55af2858f320 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f7085ffb700 1 -- 172.21.15.5:0/984553514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090057f10 con 0x55af2858f320 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 -- 172.21.15.5:0/984553514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 msgr2=0x55af285efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285efb60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7090054930 tx=0x7f709001a610 comp rx=0 tx=0).stop 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 -- 172.21.15.5:0/984553514 shutdown_connections 2024-05-08T23:23:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 0x55af2858b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285efb60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 --2- 172.21.15.5:0/984553514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 -- 172.21.15.5:0/984553514 >> 172.21.15.5:0/984553514 conn(0x55af284ef1c0 msgr2=0x55af285b88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 -- 172.21.15.5:0/984553514 shutdown_connections 2024-05-08T23:23:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.548+0000 7f70b2166b80 1 -- 172.21.15.5:0/984553514 wait complete. 2024-05-08T23:23:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.549+0000 7f70b2166b80 1 -- start start 2024-05-08T23:23:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.549+0000 7f70b2166b80 1 Processor -- start 2024-05-08T23:23:50.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f70b2166b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 0x55af2858b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f70b2166b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af286480a0 con 0x55af2858f320 2024-05-08T23:23:50.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f70b2166b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af285b3b40 con 0x55af285bbd80 2024-05-08T23:23:50.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f70b2166b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af285bf090 con 0x55af285b4750 2024-05-08T23:23:50.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f70b2166b80 10 client.0 _ll_register_callbacks cb 0x55af28486ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:50.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59574/0 (socket says 172.21.15.5:59574) 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 -- 172.21.15.5:0/1782717080 learned_addr learned my addr 172.21.15.5:0/1782717080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709ffff700 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 0x55af2858b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709effd700 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 -- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 msgr2=0x55af2858ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 -- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 msgr2=0x55af2858b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 0x55af2858b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 -- 172.21.15.5:0/1782717080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af2830eb10 con 0x55af2858f320 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f709f7fe700 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285f79a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7090058570 tx=0x7f7090087470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def328705ffeb733 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.550+0000 7f707e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f707e7fc700 1 -- 172.21.15.5:0/1782717080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000e040 con 0x55af2858f320 2024-05-08T23:23:50.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f707e7fc700 1 -- 172.21.15.5:0/1782717080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f709005bd40 con 0x55af2858f320 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 1 -- 172.21.15.5:0/1782717080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f709800edc0 con 0x55af2858f320 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f707e7fc700 1 -- 172.21.15.5:0/1782717080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7090060300 con 0x55af2858f320 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 10 client.5982 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 1 -- 172.21.15.5:0/1782717080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55af28366c80 con 0x55af2858f320 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f707e7fc700 1 -- 172.21.15.5:0/1782717080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709800edc0 con 0x55af2858f320 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f707e7fc700 1 -- 172.21.15.5:0/1782717080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f7090053e20 con 0x55af2858f320 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 10 client.5982 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 10 client.5982 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 20 client.5982 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55af28643760 0x55af28647110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 4 client.5982 mds_command: new command op to 5180 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f70b2166b80 1 -- 172.21.15.5:0/1782717080 --> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55af284d9e70 con 0x55af28643760 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.551+0000 7f709effd700 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55af28643760 0x55af28647110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.552+0000 7f707e7fc700 10 client.5982 ms_handle_connect on v2:172.21.15.49:6836/350829837 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.552+0000 7f709effd700 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55af28643760 0x55af28647110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc944982247e79f9 server_cookie=5edb03cf7e43b060 in_seq=0 out_seq=0 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.552+0000 7f707e7fc700 1 -- 172.21.15.5:0/1782717080 <== mds.2 v2:172.21.15.49:6836/350829837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55af284d9e70 con 0x55af28643760 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.552+0000 7f707e7fc700 10 client.5982 handle_command_reply: tid=0 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.552+0000 7f70b2166b80 1 client.5982 shutdown 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 -- 172.21.15.5:0/1782717080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 msgr2=0x55af285f79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285f79a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7090058570 tx=0x7f7090087470 comp rx=0 tx=0).stop 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 -- 172.21.15.5:0/1782717080 shutdown_connections 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55af28643760 0x55af28647110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af285bbd80 0x55af2858b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af2858f320 0x55af285f79a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 --2- 172.21.15.5:0/1782717080 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55af285b4750 0x55af2858ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f707e7fc700 0 client.5982 ms_handle_reset on v2:172.21.15.49:6836/350829837 2024-05-08T23:23:50.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 -- 172.21.15.5:0/1782717080 >> 172.21.15.5:0/1782717080 conn(0x55af284ef1c0 msgr2=0x55af28591d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 -- 172.21.15.5:0/1782717080 shutdown_connections 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 1 -- 172.21.15.5:0/1782717080 wait complete. 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70b2166b80 20 client.5982 trim_cache size 0 max 16384 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5180, 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:23:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:23:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:23:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 1.896450779, 2024-05-08T23:23:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 36, 2024-05-08T23:23:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 46, 2024-05-08T23:23:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 46, 2024-05-08T23:23:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 16.390050380000002 2024-05-08T23:23:50.560 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:23:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 -- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f708c06ac90 msgr2=0x7f708c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f708c06ac90 0x7f708c06d150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7094009840 tx=0x7f7094008030 comp rx=0 tx=0).stop 2024-05-08T23:23:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 -- 172.21.15.5:0/3050537373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 msgr2=0x7f70a009b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a009b250 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f709000c520 tx=0x7f709000c8e0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 -- 172.21.15.5:0/3050537373 shutdown_connections 2024-05-08T23:23:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f708c06ac90 0x7f708c06d150 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a015ba00 0x7f70a009e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a009a160 0x7f70a009b250 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.553+0000 7f70a5fac700 1 --2- 172.21.15.5:0/3050537373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a00b2510 0x7f70a009db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.554+0000 7f70a5fac700 1 -- 172.21.15.5:0/3050537373 >> 172.21.15.5:0/3050537373 conn(0x7f70a00b2f20 msgr2=0x7f70a00b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.554+0000 7f70a5fac700 1 -- 172.21.15.5:0/3050537373 shutdown_connections 2024-05-08T23:23:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.554+0000 7f70a5fac700 1 -- 172.21.15.5:0/3050537373 wait complete. 2024-05-08T23:23:50.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:23:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.705+0000 7f358f52d700 1 Processor -- start 2024-05-08T23:23:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.706+0000 7f358f52d700 1 -- start start 2024-05-08T23:23:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358f52d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f358806ed30 0x7f35880ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358f52d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f35880cd1b0 0x7f35880ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358f52d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358815abe0 con 0x7f358815a800 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358f52d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358806e8a0 con 0x7f358806ed30 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358f52d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35880cd5c0 con 0x7f35880cd1b0 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358cac8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f358806ed30 0x7f35880ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358d2c9700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f358815a800 0x7f35880cba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55064/0 (socket says 172.21.15.5:55064) 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358d2c9700 1 -- 172.21.15.5:0/4237878917 learned_addr learned my addr 172.21.15.5:0/4237878917 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358daca700 1 --2- 172.21.15.5:0/4237878917 >> v2:172.21.15.49:3300/0 conn(0x7f35880cd1b0 0x7f35880ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358cac8700 1 -- 172.21.15.5:0/4237878917 >> v2:172.21.15.49:3300/0 conn(0x7f35880cd1b0 msgr2=0x7f35880ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358cac8700 1 --2- 172.21.15.5:0/4237878917 >> v2:172.21.15.49:3300/0 conn(0x7f35880cd1b0 0x7f35880ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358cac8700 1 -- 172.21.15.5:0/4237878917 >> v1:172.21.15.5:6789/0 conn(0x7f358815a800 legacy=0x7f35880cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358cac8700 1 -- 172.21.15.5:0/4237878917 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358806dc90 con 0x7f358806ed30 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f357ffff700 1 -- 172.21.15.5:0/4237878917 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528046794 0 0) 0x7f3584001110 con 0x7f358815a800 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.707+0000 7f358cac8700 1 --2- 172.21.15.5:0/4237878917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358806ed30 0x7f35880ccc70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3588072c10 tx=0x7f357800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1aca32013633a47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f357ffff700 1 -- 172.21.15.5:0/4237878917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357800dc70 con 0x7f358806ed30 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f357ffff700 1 -- 172.21.15.5:0/4237878917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357800ddd0 con 0x7f358806ed30 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f357ffff700 1 -- 172.21.15.5:0/4237878917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578010690 con 0x7f358806ed30 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f358f52d700 1 -- 172.21.15.5:0/4237878917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358806ed30 msgr2=0x7f35880ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f358f52d700 1 --2- 172.21.15.5:0/4237878917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358806ed30 0x7f35880ccc70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3588072c10 tx=0x7f357800bdc0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f358f52d700 1 -- 172.21.15.5:0/4237878917 shutdown_connections 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f358f52d700 1 --2- 172.21.15.5:0/4237878917 >> v2:172.21.15.49:3300/0 conn(0x7f35880cd1b0 0x7f35880ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f358f52d700 1 --2- 172.21.15.5:0/4237878917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358806ed30 0x7f35880ccc70 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f358f52d700 1 -- 172.21.15.5:0/4237878917 >> 172.21.15.5:0/4237878917 conn(0x7f3588099a50 msgr2=0x7f35880d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f358f52d700 1 -- 172.21.15.5:0/4237878917 shutdown_connections 2024-05-08T23:23:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.708+0000 7f358f52d700 1 -- 172.21.15.5:0/4237878917 wait complete. 2024-05-08T23:23:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358f52d700 1 Processor -- start 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358f52d700 1 -- start start 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358f52d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f358806ed30 0x7f358815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358f52d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f35880cd1b0 0x7f35880cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358f52d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358815a800 0x7f3588152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358f52d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3588154ba0 con 0x7f358815a800 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358f52d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3588153f20 con 0x7f358806ed30 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358f52d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3588154200 con 0x7f35880cd1b0 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.710+0000 7f358daca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358815a800 0x7f3588152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358daca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358815a800 0x7f3588152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59592/0 (socket says 172.21.15.5:59592) 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358daca700 1 -- 172.21.15.5:0/888281535 learned_addr learned my addr 172.21.15.5:0/888281535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358d2c9700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f358806ed30 0x7f358815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358cac8700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f35880cd1b0 0x7f35880cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358daca700 1 -- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f35880cd1b0 msgr2=0x7f35880cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358daca700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f35880cd1b0 0x7f35880cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358daca700 1 -- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f358806ed30 msgr2=0x7f358815c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358daca700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f358806ed30 0x7f358815c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358daca700 1 -- 172.21.15.5:0/888281535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357800b740 con 0x7f358815a800 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358daca700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358815a800 0x7f3588152350 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f358000efc0 tx=0x7f358000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f357dffb700 1 -- 172.21.15.5:0/888281535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580009ea0 con 0x7f358815a800 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f357dffb700 1 -- 172.21.15.5:0/888281535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358000f040 con 0x7f358815a800 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f358f52d700 1 -- 172.21.15.5:0/888281535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35881544e0 con 0x7f358815a800 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.711+0000 7f357dffb700 1 -- 172.21.15.5:0/888281535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358000ced0 con 0x7f358815a800 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.712+0000 7f358f52d700 1 -- 172.21.15.5:0/888281535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358815ce10 con 0x7f358815a800 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.713+0000 7f357dffb700 1 -- 172.21.15.5:0/888281535 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3580014480 con 0x7f358815a800 2024-05-08T23:23:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.714+0000 7f357dffb700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f356c06ac90 0x7f356c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.714+0000 7f357dffb700 1 -- 172.21.15.5:0/888281535 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3580089e10 con 0x7f358815a800 2024-05-08T23:23:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.714+0000 7f358d2c9700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f356c06ac90 0x7f356c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.714+0000 7f358d2c9700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f356c06ac90 0x7f356c06d150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3584000f60 tx=0x7f3584007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f359b6e7b80 1 Processor -- start 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f359b6e7b80 1 -- start start 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x5592750bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 0x559275091150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559274fe8100 con 0x5592750bbf70 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559274fe5ac0 con 0x55927508f320 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559274fe13b0 con 0x5592750b9380 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.716+0000 7f358d2c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x5592750bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358d2c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x5592750bc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59602/0 (socket says 172.21.15.5:59602) 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358d2c9700 1 -- 172.21.15.5:0/3310883292 learned_addr learned my addr 172.21.15.5:0/3310883292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358cac8700 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358daca700 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 0x559275091150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358d2c9700 1 -- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 msgr2=0x559275091150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358d2c9700 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 0x559275091150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358d2c9700 1 -- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 msgr2=0x5592750d8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358d2c9700 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750d8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358d2c9700 1 -- 172.21.15.5:0/3310883292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559274e62360 con 0x5592750bbf70 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358cac8700 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750d8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f358d2c9700 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x5592750bc350 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f35840138a0 tx=0x7f3584013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcacd8ff7d1847b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f3572ffd700 1 -- 172.21.15.5:0/3310883292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584013e10 con 0x5592750bbf70 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f3572ffd700 1 -- 172.21.15.5:0/3310883292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840014a0 con 0x5592750bbf70 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.717+0000 7f3572ffd700 1 -- 172.21.15.5:0/3310883292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584001810 con 0x5592750bbf70 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3310883292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 msgr2=0x5592750bc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x5592750bc350 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f35840138a0 tx=0x7f3584013000 comp rx=0 tx=0).stop 2024-05-08T23:23:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3310883292 shutdown_connections 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 0x559275091150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x5592750bc350 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3310883292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750d8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3310883292 >> 172.21.15.5:0/3310883292 conn(0x559274fef1c0 msgr2=0x5592750d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3310883292 shutdown_connections 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.718+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3310883292 wait complete. 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.719+0000 7f359b6e7b80 1 -- start start 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.719+0000 7f359b6e7b80 1 Processor -- start 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927508f320 0x5592750a8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592750b9380 0x5592750b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559274f7d510 con 0x55927508f320 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559274f7d1b0 con 0x5592750b9380 2024-05-08T23:23:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559274e64650 con 0x5592750bbf70 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358d2c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927508f320 0x5592750a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f359b6e7b80 10 client.0 _ll_register_callbacks cb 0x559274f86ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358d2c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927508f320 0x5592750a8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59608/0 (socket says 172.21.15.5:59608) 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358d2c9700 1 -- 172.21.15.5:0/3391716331 learned_addr learned my addr 172.21.15.5:0/3391716331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358cac8700 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592750b9380 0x5592750b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358daca700 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358d2c9700 1 -- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 msgr2=0x5592750b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358d2c9700 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358d2c9700 1 -- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592750b9380 msgr2=0x5592750b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358d2c9700 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592750b9380 0x5592750b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358d2c9700 1 -- 172.21.15.5:0/3391716331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3584013550 con 0x55927508f320 2024-05-08T23:23:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.720+0000 7f358daca700 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f358d2c9700 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927508f320 0x5592750a8880 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3584001c10 tx=0x7f3584000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e46c3d6b6e8247 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f3567fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f3567fff700 1 -- 172.21.15.5:0/3391716331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840133f0 con 0x55927508f320 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f3567fff700 1 -- 172.21.15.5:0/3391716331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f358401b050 con 0x55927508f320 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3391716331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559274f3a220 con 0x55927508f320 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f3567fff700 1 -- 172.21.15.5:0/3391716331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3584013e10 con 0x55927508f320 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 10 client.5997 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3391716331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x559274e0eb10 con 0x55927508f320 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f3567fff700 1 -- 172.21.15.5:0/3391716331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559274f3a220 con 0x55927508f320 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f3567fff700 1 -- 172.21.15.5:0/3391716331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f3584020630 con 0x55927508f320 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 10 client.5997 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 10 client.5997 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:23:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 20 client.5997 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5592750afb70 0x5592750aff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 4 client.5997 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3391716331 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559275060790 con 0x5592750afb70 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.721+0000 7f358cac8700 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5592750afb70 0x5592750aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.722+0000 7f358cac8700 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5592750afb70 0x5592750aff50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3217571c6c8c26c server_cookie=348491c604eb2e67 in_seq=0 out_seq=0 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.722+0000 7f3567fff700 10 client.5997 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f3567fff700 1 -- 172.21.15.5:0/3391716331 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559275060790 con 0x5592750afb70 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f3567fff700 10 client.5997 handle_command_reply: tid=0 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 client.5997 shutdown 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3391716331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927508f320 msgr2=0x5592750a8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927508f320 0x5592750a8880 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3584001c10 tx=0x7f3584000ed0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3391716331 shutdown_connections 2024-05-08T23:23:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5592750afb70 0x5592750aff50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927508f320 0x5592750a8880 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f3567fff700 0 client.5997 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:23:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/3391716331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592750b9380 0x5592750b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3391716331 >> 172.21.15.5:0/3391716331 conn(0x559274fef1c0 msgr2=0x5592750910c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3391716331 shutdown_connections 2024-05-08T23:23:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 1 -- 172.21.15.5:0/3391716331 wait complete. 2024-05-08T23:23:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.723+0000 7f359b6e7b80 20 client.5997 trim_cache size 0 max 16384 2024-05-08T23:23:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.742+0000 7f359b6e7b80 1 Processor -- start 2024-05-08T23:23:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.742+0000 7f359b6e7b80 1 -- start start 2024-05-08T23:23:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750b9380 0x5592750b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592750fd510 con 0x5592750b9380 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592750883e0 con 0x55927508f320 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592750fce80 con 0x5592750bbf70 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358cac8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750b9380 0x5592750b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358cac8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750b9380 0x5592750b8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59614/0 (socket says 172.21.15.5:59614) 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358cac8700 1 -- 172.21.15.5:0/1648735780 learned_addr learned my addr 172.21.15.5:0/1648735780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358d2c9700 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358daca700 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358cac8700 1 -- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 msgr2=0x5592750b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358cac8700 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358cac8700 1 -- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 msgr2=0x5592750e9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.743+0000 7f358cac8700 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750e9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f358cac8700 1 -- 172.21.15.5:0/1648735780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559274fa7510 con 0x5592750b9380 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f358d2c9700 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750e9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f358cac8700 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750b9380 0x5592750b8bc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3578015000 tx=0x7f3578013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b096708c08bf99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f35737fe700 1 -- 172.21.15.5:0/1648735780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578004300 con 0x5592750b9380 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f35737fe700 1 -- 172.21.15.5:0/1648735780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3578020040 con 0x5592750b9380 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f35737fe700 1 -- 172.21.15.5:0/1648735780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35780036a0 con 0x5592750b9380 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1648735780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750b9380 msgr2=0x5592750b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750b9380 0x5592750b8bc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3578015000 tx=0x7f3578013000 comp rx=0 tx=0).stop 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1648735780 shutdown_connections 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750bbf70 0x5592750b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x5592750e9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1648735780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750b9380 0x5592750b8bc0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1648735780 >> 172.21.15.5:0/1648735780 conn(0x559274fef1c0 msgr2=0x55927508bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.744+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1648735780 shutdown_connections 2024-05-08T23:23:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.745+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1648735780 wait complete. 2024-05-08T23:23:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.745+0000 7f359b6e7b80 1 -- start start 2024-05-08T23:23:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.745+0000 7f359b6e7b80 1 Processor -- start 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x559275146620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 0x5592750ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f359b6e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x55927508b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592750b4180 con 0x5592750bbf70 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592750ccb00 con 0x55927508f320 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f359b6e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592750ccf60 con 0x5592750b9380 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f359b6e7b80 10 client.0 _ll_register_callbacks cb 0x559274f86ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f358daca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x55927508b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f358daca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x55927508b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59622/0 (socket says 172.21.15.5:59622) 2024-05-08T23:23:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f358daca700 1 -- 172.21.15.5:0/1466527131 learned_addr learned my addr 172.21.15.5:0/1466527131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f358d2c9700 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x559275146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.746+0000 7f358cac8700 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 0x5592750ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f358daca700 1 -- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 msgr2=0x5592750ed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f358daca700 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 0x5592750ed030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f358daca700 1 -- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 msgr2=0x559275146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f358daca700 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x559275146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f358daca700 1 -- 172.21.15.5:0/1466527131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358806dc90 con 0x5592750bbf70 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f358daca700 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x55927508b3e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f35800551b0 tx=0x7f3580058db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0b62e158e4ab30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f3567fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f3567fff700 1 -- 172.21.15.5:0/1466527131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580059110 con 0x5592750bbf70 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f3567fff700 1 -- 172.21.15.5:0/1466527131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35800610a0 con 0x5592750bbf70 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1466527131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35840014a0 con 0x5592750bbf70 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f3567fff700 1 -- 172.21.15.5:0/1466527131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3580088080 con 0x5592750bbf70 2024-05-08T23:23:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 10 client.6003 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1466527131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x559274e66c80 con 0x5592750bbf70 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f3567fff700 1 -- 172.21.15.5:0/1466527131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3580074070 con 0x5592750bbf70 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f3567fff700 1 -- 172.21.15.5:0/1466527131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f35800581a0 con 0x5592750bbf70 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 10 client.6003 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 10 client.6003 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 20 client.6003 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5592750b7700 0x5592750b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 4 client.6003 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.747+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1466527131 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559274fd9e70 con 0x5592750b7700 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.748+0000 7f358d2c9700 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5592750b7700 0x5592750b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.748+0000 7f3567fff700 10 client.6003 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.748+0000 7f358d2c9700 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5592750b7700 0x5592750b7ae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3efe0515b2087e46 server_cookie=53410f755bb8f476 in_seq=0 out_seq=0 2024-05-08T23:23:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.748+0000 7f3567fff700 1 -- 172.21.15.5:0/1466527131 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559274fd9e70 con 0x5592750b7700 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.748+0000 7f3567fff700 10 client.6003 handle_command_reply: tid=0 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.748+0000 7f359b6e7b80 1 client.6003 shutdown 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.748+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1466527131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 msgr2=0x55927508b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.748+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x55927508b3e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f35800551b0 tx=0x7f3580058db0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1466527131 shutdown_connections 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5592750b7700 0x5592750b7ae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592750bbf70 0x55927508b3e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927508f320 0x559275146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f3567fff700 0 client.6003 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 1 --2- 172.21.15.5:0/1466527131 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5592750b9380 0x5592750ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1466527131 >> 172.21.15.5:0/1466527131 conn(0x559274fef1c0 msgr2=0x5592750e98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1466527131 shutdown_connections 2024-05-08T23:23:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 1 -- 172.21.15.5:0/1466527131 wait complete. 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f359b6e7b80 20 client.6003 trim_cache size 0 max 16384 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5566, 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 3.0971200909999999, 2024-05-08T23:23:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 36, 2024-05-08T23:23:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 46, 2024-05-08T23:23:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 46, 2024-05-08T23:23:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 22.547066589 2024-05-08T23:23:50.757 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 -- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f356c06ac90 msgr2=0x7f356c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f356c06ac90 0x7f356c06d150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3584000f60 tx=0x7f3584007e70 comp rx=0 tx=0).stop 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 -- 172.21.15.5:0/888281535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358815a800 msgr2=0x7f3588152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358815a800 0x7f3588152350 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f358000efc0 tx=0x7f358000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 -- 172.21.15.5:0/888281535 shutdown_connections 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f356c06ac90 0x7f356c06d150 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f358815a800 0x7f3588152350 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f358806ed30 0x7f358815c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 --2- 172.21.15.5:0/888281535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f35880cd1b0 0x7f35880cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.749+0000 7f358f52d700 1 -- 172.21.15.5:0/888281535 >> 172.21.15.5:0/888281535 conn(0x7f3588099a50 msgr2=0x7f358815e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.750+0000 7f358f52d700 1 -- 172.21.15.5:0/888281535 shutdown_connections 2024-05-08T23:23:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.750+0000 7f358f52d700 1 -- 172.21.15.5:0/888281535 wait complete. 2024-05-08T23:23:50.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:23:50.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.900+0000 7f053468d700 1 Processor -- start 2024-05-08T23:23:50.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.900+0000 7f053468d700 1 -- start start 2024-05-08T23:23:50.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f053468d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 0x7f052c06e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f053468d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f052c06d4f0 0x7f052c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f053468d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f052c15f660 0x7f052c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f053468d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c06dcb0 con 0x7f052c06d4f0 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f053468d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c06e740 con 0x7f052c15f660 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f053468d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c15fac0 con 0x7f052c155da0 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0531c28700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f052c06d4f0 0x7f052c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0531c28700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f052c06d4f0 0x7f052c15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59632/0 (socket says 172.21.15.5:59632) 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0531c28700 1 -- 172.21.15.5:0/2944621325 learned_addr learned my addr 172.21.15.5:0/2944621325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0532c2a700 1 --2- 172.21.15.5:0/2944621325 >> v2:172.21.15.49:3300/0 conn(0x7f052c15f660 0x7f052c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0531c28700 1 -- 172.21.15.5:0/2944621325 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 msgr2=0x7f052c06e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0532429700 1 --2- 172.21.15.5:0/2944621325 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 0x7f052c06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0531c28700 1 --2- 172.21.15.5:0/2944621325 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 0x7f052c06e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0531c28700 1 -- 172.21.15.5:0/2944621325 >> v2:172.21.15.49:3300/0 conn(0x7f052c15f660 msgr2=0x7f052c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0531c28700 1 --2- 172.21.15.5:0/2944621325 >> v2:172.21.15.49:3300/0 conn(0x7f052c15f660 0x7f052c15bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0531c28700 1 -- 172.21.15.5:0/2944621325 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c06be40 con 0x7f052c06d4f0 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.901+0000 7f0532c2a700 1 --2- 172.21.15.5:0/2944621325 >> v2:172.21.15.49:3300/0 conn(0x7f052c15f660 0x7f052c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f0531c28700 1 --2- 172.21.15.5:0/2944621325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c15f120 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f052c0ca100 tx=0x7f051c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6a4bd4db88abd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f0531427700 1 -- 172.21.15.5:0/2944621325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00b870 con 0x7f052c06d4f0 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f0531427700 1 -- 172.21.15.5:0/2944621325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c00b9d0 con 0x7f052c06d4f0 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f0531427700 1 -- 172.21.15.5:0/2944621325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00bd60 con 0x7f052c06d4f0 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f053468d700 1 -- 172.21.15.5:0/2944621325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 msgr2=0x7f052c15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f053468d700 1 --2- 172.21.15.5:0/2944621325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c15f120 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f052c0ca100 tx=0x7f051c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f053468d700 1 -- 172.21.15.5:0/2944621325 shutdown_connections 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f053468d700 1 --2- 172.21.15.5:0/2944621325 >> v2:172.21.15.49:3300/0 conn(0x7f052c15f660 0x7f052c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f053468d700 1 --2- 172.21.15.5:0/2944621325 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 0x7f052c06e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f053468d700 1 --2- 172.21.15.5:0/2944621325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c15f120 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.902+0000 7f053468d700 1 -- 172.21.15.5:0/2944621325 >> 172.21.15.5:0/2944621325 conn(0x7f052c0cc580 msgr2=0x7f052c0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.903+0000 7f053468d700 1 -- 172.21.15.5:0/2944621325 shutdown_connections 2024-05-08T23:23:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.903+0000 7f053468d700 1 -- 172.21.15.5:0/2944621325 wait complete. 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.904+0000 7f053468d700 1 Processor -- start 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.904+0000 7f053468d700 1 -- start start 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f053468d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c14a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f053468d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 0x7f052c0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f053468d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f052c15f660 0x7f052c0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f053468d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0c7d70 con 0x7f052c06d4f0 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f053468d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c14ab10 con 0x7f052c15f660 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f053468d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0c7fa0 con 0x7f052c155da0 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c14a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c14a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59636/0 (socket says 172.21.15.5:59636) 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 -- 172.21.15.5:0/3762766491 learned_addr learned my addr 172.21.15.5:0/3762766491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532c2a700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f052c15f660 0x7f052c0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0531c28700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 0x7f052c0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 -- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 msgr2=0x7f052c0c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 0x7f052c0c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 -- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f052c15f660 msgr2=0x7f052c0b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f052c15f660 0x7f052c0b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 -- 172.21.15.5:0/3762766491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c009550 con 0x7f052c06d4f0 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0532429700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c14a5d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f052800c7f0 tx=0x7f052800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0522ffd700 1 -- 172.21.15.5:0/3762766491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528004220 con 0x7f052c06d4f0 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.905+0000 7f0522ffd700 1 -- 172.21.15.5:0/3762766491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0528004380 con 0x7f052c06d4f0 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.906+0000 7f053468d700 1 -- 172.21.15.5:0/3762766491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c0c8240 con 0x7f052c06d4f0 2024-05-08T23:23:50.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.906+0000 7f0522ffd700 1 -- 172.21.15.5:0/3762766491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05280036a0 con 0x7f052c06d4f0 2024-05-08T23:23:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.906+0000 7f053468d700 1 -- 172.21.15.5:0/3762766491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c0c85d0 con 0x7f052c06d4f0 2024-05-08T23:23:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.907+0000 7f0522ffd700 1 -- 172.21.15.5:0/3762766491 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0528003800 con 0x7f052c06d4f0 2024-05-08T23:23:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.907+0000 7f0522ffd700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f051006ac90 0x7f051006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.908+0000 7f0522ffd700 1 -- 172.21.15.5:0/3762766491 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0528088d30 con 0x7f052c06d4f0 2024-05-08T23:23:50.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.908+0000 7f0531c28700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f051006ac90 0x7f051006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.908+0000 7f0531c28700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f051006ac90 0x7f051006d150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f051c000c00 tx=0x7f051c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.909+0000 7f0540847b80 1 Processor -- start 2024-05-08T23:23:50.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.910+0000 7f0540847b80 1 -- start start 2024-05-08T23:23:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.910+0000 7f0540847b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48eae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.910+0000 7f0540847b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48e81320 0x556b48eca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.910+0000 7f0540847b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eab380 0x556b48e83150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.910+0000 7f0540847b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b48dda100 con 0x556b48e81320 2024-05-08T23:23:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.910+0000 7f0540847b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b48dd7ac0 con 0x556b48eab380 2024-05-08T23:23:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.910+0000 7f0540847b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b48dd33b0 con 0x556b48eadf70 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48e81320 0x556b48eca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48e81320 0x556b48eca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59642/0 (socket says 172.21.15.5:59642) 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 -- 172.21.15.5:0/587301679 learned_addr learned my addr 172.21.15.5:0/587301679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0532c2a700 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eab380 0x556b48e83150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0532429700 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48eae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 -- 172.21.15.5:0/587301679 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 msgr2=0x556b48eae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48eae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 -- 172.21.15.5:0/587301679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eab380 msgr2=0x556b48e83150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eab380 0x556b48e83150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 -- 172.21.15.5:0/587301679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b48c54360 con 0x556b48e81320 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f0531c28700 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48e81320 0x556b48eca200 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f051c00be30 tx=0x7f051c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cea560dc5c76ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f050ffff700 1 -- 172.21.15.5:0/587301679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c020070 con 0x556b48e81320 2024-05-08T23:23:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f050ffff700 1 -- 172.21.15.5:0/587301679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c0036a0 con 0x556b48e81320 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.911+0000 7f050ffff700 1 -- 172.21.15.5:0/587301679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c003a30 con 0x556b48e81320 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 -- 172.21.15.5:0/587301679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48e81320 msgr2=0x556b48eca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48e81320 0x556b48eca200 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f051c00be30 tx=0x7f051c010000 comp rx=0 tx=0).stop 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 -- 172.21.15.5:0/587301679 shutdown_connections 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eab380 0x556b48e83150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48eae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 --2- 172.21.15.5:0/587301679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48e81320 0x556b48eca200 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 -- 172.21.15.5:0/587301679 >> 172.21.15.5:0/587301679 conn(0x556b48de11c0 msgr2=0x556b48ec8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 -- 172.21.15.5:0/587301679 shutdown_connections 2024-05-08T23:23:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.912+0000 7f0540847b80 1 -- 172.21.15.5:0/587301679 wait complete. 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.913+0000 7f0540847b80 1 -- start start 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.913+0000 7f0540847b80 1 Processor -- start 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0540847b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48ec9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0540847b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eab380 0x556b48ea2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0540847b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eadf70 0x556b48ea0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0540847b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b48d6f510 con 0x556b48eadf70 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0540847b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b48d6f1b0 con 0x556b48e81320 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0540847b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b48c56650 con 0x556b48eab380 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0540847b80 10 client.0 _ll_register_callbacks cb 0x556b48d78ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0532c2a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eadf70 0x556b48ea0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0532c2a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eadf70 0x556b48ea0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59650/0 (socket says 172.21.15.5:59650) 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0532c2a700 1 -- 172.21.15.5:0/1918870430 learned_addr learned my addr 172.21.15.5:0/1918870430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0532429700 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48ec9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.914+0000 7f0531c28700 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eab380 0x556b48ea2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0532c2a700 1 -- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eab380 msgr2=0x556b48ea2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0532c2a700 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eab380 0x556b48ea2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0532c2a700 1 -- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 msgr2=0x556b48ec9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0532c2a700 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48ec9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0532c2a700 1 -- 172.21.15.5:0/1918870430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c06be40 con 0x556b48eadf70 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0532429700 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48ec9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0532c2a700 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eadf70 0x556b48ea0f00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f052400ff30 tx=0x7f052400d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384ff56531b09ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f050d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f050d7fa700 1 -- 172.21.15.5:0/1918870430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524014070 con 0x556b48eadf70 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f050d7fa700 1 -- 172.21.15.5:0/1918870430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f052400dd90 con 0x556b48eadf70 2024-05-08T23:23:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 1 -- 172.21.15.5:0/1918870430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b48d2c220 con 0x556b48eadf70 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f050d7fa700 1 -- 172.21.15.5:0/1918870430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0524007c00 con 0x556b48eadf70 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 10 client.6015 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 1 -- 172.21.15.5:0/1918870430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x556b48c00b10 con 0x556b48eadf70 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f050d7fa700 1 -- 172.21.15.5:0/1918870430 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b48d2c220 con 0x556b48eadf70 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f050d7fa700 1 -- 172.21.15.5:0/1918870430 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f0524018410 con 0x556b48eadf70 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 10 client.6015 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 10 client.6015 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 20 client.6015 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x556b48ea1440 0x556b48ea9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 4 client.6015 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.915+0000 7f0540847b80 1 -- 172.21.15.5:0/1918870430 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b48e52790 con 0x556b48ea1440 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.916+0000 7f0532429700 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x556b48ea1440 0x556b48ea9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.916+0000 7f050d7fa700 10 client.6015 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.916+0000 7f0532429700 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x556b48ea1440 0x556b48ea9ac0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb9e0c6b5c44273 server_cookie=88fd786ecde7b380 in_seq=0 out_seq=0 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f050d7fa700 1 -- 172.21.15.5:0/1918870430 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b48e52790 con 0x556b48ea1440 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f050d7fa700 10 client.6015 handle_command_reply: tid=0 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 client.6015 shutdown 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 -- 172.21.15.5:0/1918870430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eadf70 msgr2=0x556b48ea0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eadf70 0x556b48ea0f00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f052400ff30 tx=0x7f052400d650 comp rx=0 tx=0).stop 2024-05-08T23:23:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 -- 172.21.15.5:0/1918870430 shutdown_connections 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eadf70 0x556b48ea0f00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48ec9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x556b48ea1440 0x556b48ea9ac0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 --2- 172.21.15.5:0/1918870430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eab380 0x556b48ea2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 -- 172.21.15.5:0/1918870430 >> 172.21.15.5:0/1918870430 conn(0x556b48de11c0 msgr2=0x556b48e830c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f050d7fa700 0 client.6015 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 -- 172.21.15.5:0/1918870430 shutdown_connections 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 1 -- 172.21.15.5:0/1918870430 wait complete. 2024-05-08T23:23:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.917+0000 7f0540847b80 20 client.6015 trim_cache size 0 max 16384 2024-05-08T23:23:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.936+0000 7f0540847b80 1 Processor -- start 2024-05-08T23:23:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0540847b80 1 -- start start 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0540847b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48e81320 0x556b48ead380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0540847b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48ea1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0540847b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eadf70 0x556b48e7e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0540847b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b48ed5280 con 0x556b48eab380 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0540847b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b48ea12f0 con 0x556b48eadf70 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0540847b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b48e7fb20 con 0x556b48e81320 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0531c28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48ea1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0531c28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48ea1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59658/0 (socket says 172.21.15.5:59658) 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.937+0000 7f0531c28700 1 -- 172.21.15.5:0/475527397 learned_addr learned my addr 172.21.15.5:0/475527397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0532c2a700 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eadf70 0x556b48e7e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0532429700 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48e81320 0x556b48ead380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0531c28700 1 -- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48e81320 msgr2=0x556b48ead380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0531c28700 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48e81320 0x556b48ead380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0531c28700 1 -- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eadf70 msgr2=0x556b48e7e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0531c28700 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eadf70 0x556b48e7e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0531c28700 1 -- 172.21.15.5:0/475527397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b48d99510 con 0x556b48eab380 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0532429700 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48e81320 0x556b48ead380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0531c28700 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48ea1ea0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f051c00be00 tx=0x7f051c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd15191a73f4a04d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0520ff9700 1 -- 172.21.15.5:0/475527397 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c037070 con 0x556b48eab380 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0520ff9700 1 -- 172.21.15.5:0/475527397 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c006bd0 con 0x556b48eab380 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.938+0000 7f0520ff9700 1 -- 172.21.15.5:0/475527397 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c003a10 con 0x556b48eab380 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 -- 172.21.15.5:0/475527397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 msgr2=0x556b48ea1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48ea1ea0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f051c00be00 tx=0x7f051c020000 comp rx=0 tx=0).stop 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 -- 172.21.15.5:0/475527397 shutdown_connections 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48eadf70 0x556b48e7e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48e81320 0x556b48ead380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 --2- 172.21.15.5:0/475527397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48ea1ea0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 -- 172.21.15.5:0/475527397 >> 172.21.15.5:0/475527397 conn(0x556b48de11c0 msgr2=0x556b48e7cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 -- 172.21.15.5:0/475527397 shutdown_connections 2024-05-08T23:23:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 -- 172.21.15.5:0/475527397 wait complete. 2024-05-08T23:23:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.939+0000 7f0540847b80 1 -- start start 2024-05-08T23:23:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.940+0000 7f0540847b80 1 Processor -- start 2024-05-08T23:23:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.940+0000 7f0540847b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48eca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0540847b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48e7cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0540847b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48ea8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0540847b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b48eca740 con 0x556b48eab380 2024-05-08T23:23:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0540847b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b48ecab70 con 0x556b48e81320 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0540847b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b48ea40d0 con 0x556b48eadf70 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0540847b80 10 client.0 _ll_register_callbacks cb 0x556b48d78ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48e7cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48e7cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59670/0 (socket says 172.21.15.5:59670) 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 -- 172.21.15.5:0/3197857729 learned_addr learned my addr 172.21.15.5:0/3197857729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0532429700 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48eca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0532c2a700 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48ea8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 -- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 msgr2=0x556b48ea8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48ea8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 -- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 msgr2=0x556b48eca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48eca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 -- 172.21.15.5:0/3197857729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b48c54360 con 0x556b48eab380 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0532c2a700 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48ea8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0531c28700 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48e7cf20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f051c00be30 tx=0x7f051c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fd56795e7e458d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f050d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f050d7fa700 1 -- 172.21.15.5:0/3197857729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c020040 con 0x556b48eab380 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f050d7fa700 1 -- 172.21.15.5:0/3197857729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f051c03c660 con 0x556b48eab380 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.941+0000 7f0540847b80 1 -- 172.21.15.5:0/3197857729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f051c0036a0 con 0x556b48eab380 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f050d7fa700 1 -- 172.21.15.5:0/3197857729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f051c03c820 con 0x556b48eab380 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0540847b80 10 client.6021 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0540847b80 1 -- 172.21.15.5:0/3197857729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x556b48c58c80 con 0x556b48eab380 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f050d7fa700 1 -- 172.21.15.5:0/3197857729 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c0036a0 con 0x556b48eab380 2024-05-08T23:23:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f050d7fa700 1 -- 172.21.15.5:0/3197857729 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f051c005a20 con 0x556b48eab380 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0540847b80 10 client.6021 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0540847b80 10 client.6021 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0540847b80 20 client.6021 populate_metadata read hostname 'smithi005' 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0540847b80 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x556b48f35b40 0x556b48e7dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0540847b80 4 client.6021 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0540847b80 1 -- 172.21.15.5:0/3197857729 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556b48dcbe70 con 0x556b48f35b40 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0532429700 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x556b48f35b40 0x556b48e7dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f050d7fa700 10 client.6021 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.942+0000 7f0532429700 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x556b48f35b40 0x556b48e7dd50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef630aa0e13cfcc server_cookie=2cc85e55fa31f818 in_seq=0 out_seq=0 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f050d7fa700 1 -- 172.21.15.5:0/3197857729 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x556b48dcbe70 con 0x556b48f35b40 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f050d7fa700 10 client.6021 handle_command_reply: tid=0 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 client.6021 shutdown 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 -- 172.21.15.5:0/3197857729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 msgr2=0x556b48e7cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48e7cf20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f051c00be30 tx=0x7f051c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 -- 172.21.15.5:0/3197857729 shutdown_connections 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x556b48f35b40 0x556b48e7dd50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556b48eadf70 0x556b48ea8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f050d7fa700 0 client.6021 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556b48e81320 0x556b48eca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 --2- 172.21.15.5:0/3197857729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b48eab380 0x556b48e7cf20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 -- 172.21.15.5:0/3197857729 >> 172.21.15.5:0/3197857729 conn(0x556b48de11c0 msgr2=0x556b48ea2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 -- 172.21.15.5:0/3197857729 shutdown_connections 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 1 -- 172.21.15.5:0/3197857729 wait complete. 2024-05-08T23:23:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f0540847b80 20 client.6021 trim_cache size 0 max 16384 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5778, 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 11.47548579, 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 36, 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 46, 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 46, 2024-05-08T23:23:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 22.747494369000002 2024-05-08T23:23:50.951 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:23:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f053468d700 1 -- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f051006ac90 msgr2=0x7f051006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f053468d700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f051006ac90 0x7f051006d150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f051c000c00 tx=0x7f051c008030 comp rx=0 tx=0).stop 2024-05-08T23:23:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f053468d700 1 -- 172.21.15.5:0/3762766491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 msgr2=0x7f052c14a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.943+0000 7f053468d700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c14a5d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f052800c7f0 tx=0x7f052800cb00 comp rx=0 tx=0).stop 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.944+0000 7f053468d700 1 -- 172.21.15.5:0/3762766491 shutdown_connections 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.944+0000 7f053468d700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f051006ac90 0x7f051006d150 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.944+0000 7f053468d700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f052c15f660 0x7f052c0b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.944+0000 7f053468d700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c06d4f0 0x7f052c14a5d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.944+0000 7f053468d700 1 --2- 172.21.15.5:0/3762766491 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f052c155da0 0x7f052c0c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.944+0000 7f053468d700 1 -- 172.21.15.5:0/3762766491 >> 172.21.15.5:0/3762766491 conn(0x7f052c0cc580 msgr2=0x7f052c0b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.944+0000 7f053468d700 1 -- 172.21.15.5:0/3762766491 shutdown_connections 2024-05-08T23:23:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:50.944+0000 7f053468d700 1 -- 172.21.15.5:0/3762766491 wait complete. 2024-05-08T23:23:50.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.094+0000 7fbf057c3700 1 Processor -- start 2024-05-08T23:23:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.094+0000 7fbf057c3700 1 -- start start 2024-05-08T23:23:51.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.094+0000 7fbf057c3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbf00161900 0x7fbf0015e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:51.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.094+0000 7fbf057c3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf00070fa0 0x7fbf00071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:51.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.094+0000 7fbf057c3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf00071e90 con 0x7fbf0015d590 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbf057c3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf00071950 con 0x7fbf00070fa0 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbf057c3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf00162090 con 0x7fbf00161900 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbeff7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf00070fa0 0x7fbf00071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbefeffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf0015d590 0x7fbf001611f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55070/0 (socket says 172.21.15.5:55070) 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbefeffd700 1 -- 172.21.15.5:0/2611196139 learned_addr learned my addr 172.21.15.5:0/2611196139 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbefe7fc700 1 --2- 172.21.15.5:0/2611196139 >> v2:172.21.15.49:3300/0 conn(0x7fbf00161900 0x7fbf0015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbefdffb700 1 -- 172.21.15.5:0/2611196139 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146171126 0 0) 0x7fbf00071e90 con 0x7fbf0015d590 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbefdffb700 1 -- 172.21.15.5:0/2611196139 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee80035b0 con 0x7fbf0015d590 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbeff7fe700 1 -- 172.21.15.5:0/2611196139 >> v2:172.21.15.49:3300/0 conn(0x7fbf00161900 msgr2=0x7fbf0015e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbeff7fe700 1 --2- 172.21.15.5:0/2611196139 >> v2:172.21.15.49:3300/0 conn(0x7fbf00161900 0x7fbf0015e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbeff7fe700 1 -- 172.21.15.5:0/2611196139 >> v1:172.21.15.5:6789/0 conn(0x7fbf0015d590 legacy=0x7fbf001611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbeff7fe700 1 -- 172.21.15.5:0/2611196139 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf00161db0 con 0x7fbf00070fa0 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbeff7fe700 1 --2- 172.21.15.5:0/2611196139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00070fa0 0x7fbf00071410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbf000747a0 tx=0x7fbef800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d6f6f68e84d811 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbefdffb700 1 -- 172.21.15.5:0/2611196139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef800e070 con 0x7fbf00070fa0 2024-05-08T23:23:51.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbefdffb700 1 -- 172.21.15.5:0/2611196139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef800bb20 con 0x7fbf00070fa0 2024-05-08T23:23:51.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.095+0000 7fbefdffb700 1 -- 172.21.15.5:0/2611196139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8012430 con 0x7fbf00070fa0 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.096+0000 7fbf057c3700 1 -- 172.21.15.5:0/2611196139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00070fa0 msgr2=0x7fbf00071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.096+0000 7fbf057c3700 1 --2- 172.21.15.5:0/2611196139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00070fa0 0x7fbf00071410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbf000747a0 tx=0x7fbef800b330 comp rx=0 tx=0).stop 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.096+0000 7fbf057c3700 1 -- 172.21.15.5:0/2611196139 shutdown_connections 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.096+0000 7fbf057c3700 1 --2- 172.21.15.5:0/2611196139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00070fa0 0x7fbf00071410 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.096+0000 7fbf057c3700 1 --2- 172.21.15.5:0/2611196139 >> v2:172.21.15.49:3300/0 conn(0x7fbf00161900 0x7fbf0015e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.096+0000 7fbf057c3700 1 -- 172.21.15.5:0/2611196139 >> 172.21.15.5:0/2611196139 conn(0x7fbf000b1470 msgr2=0x7fbf000b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.096+0000 7fbf057c3700 1 -- 172.21.15.5:0/2611196139 shutdown_connections 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.096+0000 7fbf057c3700 1 -- 172.21.15.5:0/2611196139 wait complete. 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.098+0000 7fbf057c3700 1 Processor -- start 2024-05-08T23:23:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.098+0000 7fbf057c3700 1 -- start start 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.098+0000 7fbf057c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf00070fa0 0x7fbf00153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.098+0000 7fbf057c3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf0015d590 0x7fbf00150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.098+0000 7fbf057c3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00161900 0x7fbf00156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.098+0000 7fbf057c3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf001f6a60 con 0x7fbf00161900 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.098+0000 7fbf057c3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf001f6150 con 0x7fbf00070fa0 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.098+0000 7fbf057c3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf001507c0 con 0x7fbf0015d590 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00161900 0x7fbf00156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00161900 0x7fbf00156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59682/0 (socket says 172.21.15.5:59682) 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 -- 172.21.15.5:0/789132702 learned_addr learned my addr 172.21.15.5:0/789132702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbefeffd700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf00070fa0 0x7fbf00153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbefe7fc700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf0015d590 0x7fbf00150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 -- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf0015d590 msgr2=0x7fbf00150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf0015d590 0x7fbf00150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 -- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf00070fa0 msgr2=0x7fbf00153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf00070fa0 0x7fbf00153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 -- 172.21.15.5:0/789132702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef8009a70 con 0x7fbf00161900 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbeff7fe700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00161900 0x7fbf00156a50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbef8014040 tx=0x7fbef8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbee77fe700 1 -- 172.21.15.5:0/789132702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8012430 con 0x7fbf00161900 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbee77fe700 1 -- 172.21.15.5:0/789132702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8020070 con 0x7fbf00161900 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf00157360 con 0x7fbf00161900 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbee77fe700 1 -- 172.21.15.5:0/789132702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80098f0 con 0x7fbf00161900 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.099+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf001f6680 con 0x7fbf00161900 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.101+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf000678f0 con 0x7fbf00161900 2024-05-08T23:23:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.101+0000 7fbee77fe700 1 -- 172.21.15.5:0/789132702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbef8004040 con 0x7fbf00161900 2024-05-08T23:23:51.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.102+0000 7fbee77fe700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbeec06ac90 0x7fbeec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:51.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.102+0000 7fbee77fe700 1 -- 172.21.15.5:0/789132702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbef8022070 con 0x7fbf00161900 2024-05-08T23:23:51.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.102+0000 7fbefeffd700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbeec06ac90 0x7fbeec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:51.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.102+0000 7fbefeffd700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbeec06ac90 0x7fbeec06d150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbef0007fd0 tx=0x7fbef0007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:51.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.107+0000 7fbee77fe700 1 -- 172.21.15.5:0/789132702 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbef8091080 con 0x7fbf00161900 2024-05-08T23:23:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.371+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf000b2b10 con 0x7fbf00161900 2024-05-08T23:23:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.372+0000 7fbee77fe700 1 -- 172.21.15.5:0/789132702 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4750 (secure 0 0 0) 0x7fbef802c020 con 0x7fbf00161900 2024-05-08T23:23:51.372 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:23:51.372 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:23:49.656410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5778,"mds_1":5566,"mds_2":5180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5180":{"gid":5180,"name":"d","rank":2,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5566":{"gid":5566,"name":"b","rank":1,"incarnation":33,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:23:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbeec06ac90 msgr2=0x7fbeec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbeec06ac90 0x7fbeec06d150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbef0007fd0 tx=0x7fbef0007dc0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00161900 msgr2=0x7fbf00156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00161900 0x7fbf00156a50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbef8014040 tx=0x7fbef8004ad0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 shutdown_connections 2024-05-08T23:23:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbeec06ac90 0x7fbeec06d150 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf00161900 0x7fbf00156a50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf00070fa0 0x7fbf00153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 --2- 172.21.15.5:0/789132702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf0015d590 0x7fbf00150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 >> 172.21.15.5:0/789132702 conn(0x7fbf000b1470 msgr2=0x7fbf000b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 shutdown_connections 2024-05-08T23:23:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:23:51.374+0000 7fbf057c3700 1 -- 172.21.15.5:0/789132702 wait complete. 2024-05-08T23:23:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-05-08T23:23:51.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:51.388 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:23:51.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:51.417 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:23:51.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:51.483 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-05-08T23:23:51.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:51.547 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-05-08T23:23:51.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:23:51.609 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-05-08T23:23:51.673 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:23:51.879 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:23:51.879 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:23:51.879 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:23:53.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:23:54.044 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:23:54.045 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:23:54.045 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:23:56.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:23:56.210 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:23:56.210 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:23:56.211 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:23:58.211 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:23:58.376 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-08T23:23:58.377 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:23:58.377 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:24:00.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:24:00.543 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-05-08T23:24:00.544 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:24:00.544 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:24:00.544 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-08T23:24:00.703 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-05-08T23:24:00.704 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:24:00.704 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:24:02.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-08T23:24:02.863 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-05-08T23:24:02.864 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:24:02.864 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:24:04.866 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-08T23:24:05.022 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-08T23:24:05.023 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:24:05.023 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:24:05.024 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:24:05.167 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:24:05.168 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:24:05.168 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:24:05.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-05-08T23:24:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.316+0000 7f8f53a85700 1 Processor -- start 2024-05-08T23:24:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.316+0000 7f8f53a85700 1 -- start start 2024-05-08T23:24:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f53a85700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f4c154df0 0x7f8f4c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f53a85700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0d0890 0x7f8f4c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f53a85700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c0d0cf0 con 0x7f8f4c0b43b0 2024-05-08T23:24:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f53a85700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4c15abe0 con 0x7f8f4c154df0 2024-05-08T23:24:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f53a85700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c15add0 con 0x7f8f4c0d0890 2024-05-08T23:24:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f51020700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0d0890 0x7f8f4c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f51821700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f4c154df0 0x7f8f4c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f52022700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8f4c0b43b0 0x7f8f4c0ada60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:50252/0 (socket says 172.21.15.5:50252) 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.317+0000 7f8f52022700 1 -- 172.21.15.5:0/2125322788 learned_addr learned my addr 172.21.15.5:0/2125322788 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f51020700 1 -- 172.21.15.5:0/2125322788 >> v2:172.21.15.49:3300/0 conn(0x7f8f4c154df0 msgr2=0x7f8f4c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f51020700 1 --2- 172.21.15.5:0/2125322788 >> v2:172.21.15.49:3300/0 conn(0x7f8f4c154df0 0x7f8f4c0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f51020700 1 -- 172.21.15.5:0/2125322788 >> v1:172.21.15.49:6789/0 conn(0x7f8f4c0b43b0 legacy=0x7f8f4c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f51020700 1 -- 172.21.15.5:0/2125322788 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c15b070 con 0x7f8f4c0d0890 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f5081f700 1 -- 172.21.15.5:0/2125322788 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340161589 0 0) 0x7f8f4c0d0cf0 con 0x7f8f4c0b43b0 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f51821700 1 --2- 172.21.15.5:0/2125322788 >> v2:172.21.15.49:3300/0 conn(0x7f8f4c154df0 0x7f8f4c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f51020700 1 --2- 172.21.15.5:0/2125322788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0d0890 0x7f8f4c0b3e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009840 tx=0x7f8f3c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c49d76936bed57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f5081f700 1 -- 172.21.15.5:0/2125322788 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c011830 con 0x7f8f4c0d0890 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f5081f700 1 -- 172.21.15.5:0/2125322788 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c011990 con 0x7f8f4c0d0890 2024-05-08T23:24:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.318+0000 7f8f5081f700 1 -- 172.21.15.5:0/2125322788 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c011d20 con 0x7f8f4c0d0890 2024-05-08T23:24:05.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.319+0000 7f8f53a85700 1 -- 172.21.15.5:0/2125322788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0d0890 msgr2=0x7f8f4c0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.319+0000 7f8f53a85700 1 --2- 172.21.15.5:0/2125322788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0d0890 0x7f8f4c0b3e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009840 tx=0x7f8f3c009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:05.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.319+0000 7f8f53a85700 1 -- 172.21.15.5:0/2125322788 shutdown_connections 2024-05-08T23:24:05.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.319+0000 7f8f53a85700 1 --2- 172.21.15.5:0/2125322788 >> v2:172.21.15.49:3300/0 conn(0x7f8f4c154df0 0x7f8f4c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.319+0000 7f8f53a85700 1 --2- 172.21.15.5:0/2125322788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0d0890 0x7f8f4c0b3e70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.319+0000 7f8f53a85700 1 -- 172.21.15.5:0/2125322788 >> 172.21.15.5:0/2125322788 conn(0x7f8f4c099a50 msgr2=0x7f8f4c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.319+0000 7f8f53a85700 1 -- 172.21.15.5:0/2125322788 shutdown_connections 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.319+0000 7f8f53a85700 1 -- 172.21.15.5:0/2125322788 wait complete. 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.320+0000 7f8f53a85700 1 Processor -- start 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.320+0000 7f8f53a85700 1 -- start start 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f53a85700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0b43b0 0x7f8f4c148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f53a85700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f4c0d0890 0x7f8f4c150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f53a85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f4c154df0 0x7f8f4c1520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f53a85700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1f0870 con 0x7f8f4c0d0890 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f53a85700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1eff40 con 0x7f8f4c154df0 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f53a85700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1f0220 con 0x7f8f4c0b43b0 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f4c0d0890 0x7f8f4c150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f4c0d0890 0x7f8f4c150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60320/0 (socket says 172.21.15.5:60320) 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51020700 1 -- 172.21.15.5:0/3090514763 learned_addr learned my addr 172.21.15.5:0/3090514763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f52022700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f4c154df0 0x7f8f4c1520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51821700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0b43b0 0x7f8f4c148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51020700 1 -- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0b43b0 msgr2=0x7f8f4c148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51020700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0b43b0 0x7f8f4c148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51020700 1 -- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f4c154df0 msgr2=0x7f8f4c1520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51020700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f4c154df0 0x7f8f4c1520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.321+0000 7f8f51020700 1 -- 172.21.15.5:0/3090514763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480094a0 con 0x7f8f4c0d0890 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.322+0000 7f8f51020700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f4c0d0890 0x7f8f4c150870 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009fd0 tx=0x7f8f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.322+0000 7f8f427fc700 1 -- 172.21.15.5:0/3090514763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0036a0 con 0x7f8f4c0d0890 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.322+0000 7f8f427fc700 1 -- 172.21.15.5:0/3090514763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c01e070 con 0x7f8f4c0d0890 2024-05-08T23:24:05.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.322+0000 7f8f53a85700 1 -- 172.21.15.5:0/3090514763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c0094a0 con 0x7f8f4c0d0890 2024-05-08T23:24:05.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.322+0000 7f8f427fc700 1 -- 172.21.15.5:0/3090514763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c003a40 con 0x7f8f4c0d0890 2024-05-08T23:24:05.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.322+0000 7f8f53a85700 1 -- 172.21.15.5:0/3090514763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c1efc20 con 0x7f8f4c0d0890 2024-05-08T23:24:05.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.324+0000 7f8f427fc700 1 -- 172.21.15.5:0/3090514763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8f3c003ba0 con 0x7f8f4c0d0890 2024-05-08T23:24:05.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.324+0000 7f8f427fc700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f3006ac90 0x7f8f3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.324+0000 7f8f427fc700 1 -- 172.21.15.5:0/3090514763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8f3c08a420 con 0x7f8f4c0d0890 2024-05-08T23:24:05.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.325+0000 7f8f51821700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f3006ac90 0x7f8f3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.325+0000 7f8f51821700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f3006ac90 0x7f8f3006d150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c151920 tx=0x7f8f48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.326+0000 7f8f5fc3fb80 1 Processor -- start 2024-05-08T23:24:05.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.326+0000 7f8f5fc3fb80 1 -- start start 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94ddb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94df7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 0x564e94db01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94d07150 con 0x564e94dae370 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94d04b10 con 0x564e94ddafc0 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e94d00400 con 0x564e94dd83d0 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f51020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94df7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f51020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94df7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60330/0 (socket says 172.21.15.5:60330) 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f51020700 1 -- 172.21.15.5:0/3766829275 learned_addr learned my addr 172.21.15.5:0/3766829275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f51821700 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94ddb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.327+0000 7f8f52022700 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 0x564e94db01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f51020700 1 -- 172.21.15.5:0/3766829275 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 msgr2=0x564e94db01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f51020700 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 0x564e94db01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f51020700 1 -- 172.21.15.5:0/3766829275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 msgr2=0x564e94ddb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f51020700 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94ddb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f51020700 1 -- 172.21.15.5:0/3766829275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e94b813b0 con 0x564e94dae370 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f51020700 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94df7250 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c061130 tx=0x7f8f3c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3d801ba257865 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/3766829275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c059270 con 0x564e94dae370 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/3766829275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c05d0a0 con 0x564e94dae370 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.328+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/3766829275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c061660 con 0x564e94dae370 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/3766829275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 msgr2=0x564e94df7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94df7250 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c061130 tx=0x7f8f3c009ee0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/3766829275 shutdown_connections 2024-05-08T23:24:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 0x564e94db01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94ddb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/3766829275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94df7250 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/3766829275 >> 172.21.15.5:0/3766829275 conn(0x564e94d0e210 msgr2=0x564e94df5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/3766829275 shutdown_connections 2024-05-08T23:24:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.329+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/3766829275 wait complete. 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.330+0000 7f8f5fc3fb80 1 -- start start 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.330+0000 7f8f5fc3fb80 1 Processor -- start 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94dd05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 0x564e94df6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94dd0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94c9c560 con 0x564e94dae370 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94c9c200 con 0x564e94ddafc0 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e94b836a0 con 0x564e94dd83d0 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94dd05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f5fc3fb80 10 client.0 _ll_register_callbacks cb 0x564e94ca5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94dd05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60334/0 (socket says 172.21.15.5:60334) 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 -- 172.21.15.5:0/1553151342 learned_addr learned my addr 172.21.15.5:0/1553151342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51020700 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 0x564e94df6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f52022700 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94dd0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 -- 172.21.15.5:0/1553151342 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 msgr2=0x564e94df6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 0x564e94df6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 -- 172.21.15.5:0/1553151342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 msgr2=0x564e94dd0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94dd0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 -- 172.21.15.5:0/1553151342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c1efc20 con 0x564e94dae370 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f51821700 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94dd05a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8f48015890 tx=0x7f8f48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d5d404b34f4366 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.331+0000 7f8f2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f2cff9700 1 -- 172.21.15.5:0/1553151342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48014520 con 0x564e94dae370 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f2cff9700 1 -- 172.21.15.5:0/1553151342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8f48014680 con 0x564e94dae370 2024-05-08T23:24:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/1553151342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e94c59270 con 0x564e94dae370 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f2cff9700 1 -- 172.21.15.5:0/1553151342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f48005a20 con 0x564e94dae370 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 10 client.6039 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/1553151342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x564e94b2db60 con 0x564e94dae370 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f2cff9700 1 -- 172.21.15.5:0/1553151342 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e94c59270 con 0x564e94dae370 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f2cff9700 1 -- 172.21.15.5:0/1553151342 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f8f48028020 con 0x564e94dae370 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 10 client.6039 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 10 client.6039 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 20 client.6039 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x564e94dddcf0 0x564e94dd9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 4 client.6039 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/1553151342 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e94d7f7e0 con 0x564e94dddcf0 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.332+0000 7f8f51020700 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x564e94dddcf0 0x564e94dd9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f2cff9700 10 client.6039 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f51020700 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x564e94dddcf0 0x564e94dd9460 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e5219c9c7125ba server_cookie=8741f56c59a98b80 in_seq=0 out_seq=0 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f2cff9700 1 -- 172.21.15.5:0/1553151342 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e94d7f7e0 con 0x564e94dddcf0 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f2cff9700 10 client.6039 handle_command_reply: tid=0 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 client.6039 shutdown 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/1553151342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 msgr2=0x564e94dd05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94dd05a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8f48015890 tx=0x7f8f48004750 comp rx=0 tx=0).stop 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/1553151342 shutdown_connections 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94dd0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dae370 0x564e94dd05a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x564e94dddcf0 0x564e94dd9460 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/1553151342 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dd83d0 0x564e94df6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/1553151342 >> 172.21.15.5:0/1553151342 conn(0x564e94d0e210 msgr2=0x564e94df8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f2cff9700 0 client.6039 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.333+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/1553151342 shutdown_connections 2024-05-08T23:24:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.334+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/1553151342 wait complete. 2024-05-08T23:24:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.334+0000 7f8f5fc3fb80 20 client.6039 trim_cache size 0 max 16384 2024-05-08T23:24:05.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.350+0000 7f8f5fc3fb80 1 Processor -- start 2024-05-08T23:24:05.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.350+0000 7f8f5fc3fb80 1 -- start start 2024-05-08T23:24:05.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.350+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 0x564e94dd0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.350+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.350+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94df6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.350+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94daba00 con 0x564e94dd83d0 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.350+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94dabdd0 con 0x564e94ddafc0 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e94daded0 con 0x564e94dae370 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e085a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60350/0 (socket says 172.21.15.5:60350) 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 -- 172.21.15.5:0/488715737 learned_addr learned my addr 172.21.15.5:0/488715737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f52022700 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94df6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51821700 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 0x564e94dd0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 -- 172.21.15.5:0/488715737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 msgr2=0x564e94dd0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 0x564e94dd0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 -- 172.21.15.5:0/488715737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 msgr2=0x564e94df6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94df6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 -- 172.21.15.5:0/488715737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e94cc6560 con 0x564e94dd83d0 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f51020700 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e085a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c058f70 tx=0x7f8f3c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19606ca3ad797ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f2ffff700 1 -- 172.21.15.5:0/488715737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c076040 con 0x564e94dd83d0 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f2ffff700 1 -- 172.21.15.5:0/488715737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c0616e0 con 0x564e94dd83d0 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.351+0000 7f8f2ffff700 1 -- 172.21.15.5:0/488715737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c059270 con 0x564e94dd83d0 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/488715737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 msgr2=0x564e94e085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e085a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c058f70 tx=0x7f8f3c020000 comp rx=0 tx=0).stop 2024-05-08T23:24:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/488715737 shutdown_connections 2024-05-08T23:24:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94df6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 0x564e94dd0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/488715737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e085a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/488715737 >> 172.21.15.5:0/488715737 conn(0x564e94d0e210 msgr2=0x564e94da71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/488715737 shutdown_connections 2024-05-08T23:24:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.352+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/488715737 wait complete. 2024-05-08T23:24:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.353+0000 7f8f5fc3fb80 1 -- start start 2024-05-08T23:24:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.353+0000 7f8f5fc3fb80 1 Processor -- start 2024-05-08T23:24:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 0x564e94e0b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e4bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f5fc3fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94e4c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94df6c10 con 0x564e94dd83d0 2024-05-08T23:24:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e94e0bbc0 con 0x564e94ddafc0 2024-05-08T23:24:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f5fc3fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e94debe00 con 0x564e94dae370 2024-05-08T23:24:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f5fc3fb80 10 client.0 _ll_register_callbacks cb 0x564e94ca5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e4bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e4bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60362/0 (socket says 172.21.15.5:60362) 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 -- 172.21.15.5:0/2924045737 learned_addr learned my addr 172.21.15.5:0/2924045737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51821700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 0x564e94e0b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f52022700 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94e4c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 -- 172.21.15.5:0/2924045737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 msgr2=0x564e94e0b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 0x564e94e0b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 -- 172.21.15.5:0/2924045737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 msgr2=0x564e94e4c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94e4c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 -- 172.21.15.5:0/2924045737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e94b813b0 con 0x564e94dd83d0 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f51020700 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e4bc90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009b90 tx=0x7f8f3c009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e15a66a266cb5ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f2cff9700 1 -- 172.21.15.5:0/2924045737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c076070 con 0x564e94dd83d0 2024-05-08T23:24:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.354+0000 7f8f2cff9700 1 -- 172.21.15.5:0/2924045737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8f3c07d070 con 0x564e94dd83d0 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/2924045737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f3c05d0a0 con 0x564e94dd83d0 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f2cff9700 1 -- 172.21.15.5:0/2924045737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f3c059040 con 0x564e94dd83d0 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 10 client.6045 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/2924045737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x564e94b85cd0 con 0x564e94dd83d0 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f2cff9700 1 -- 172.21.15.5:0/2924045737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c05d0a0 con 0x564e94dd83d0 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f2cff9700 1 -- 172.21.15.5:0/2924045737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f8f3c08a0e0 con 0x564e94dd83d0 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 10 client.6045 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 10 client.6045 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 20 client.6045 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x564e94e48fc0 0x564e94e12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 4 client.6045 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/2924045737 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564e94cf8ec0 con 0x564e94e48fc0 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f51821700 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x564e94e48fc0 0x564e94e12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f2cff9700 10 client.6045 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:05.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.355+0000 7f8f51821700 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x564e94e48fc0 0x564e94e12880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbfe5154fb24cf98 server_cookie=76586aa8f425117d in_seq=0 out_seq=0 2024-05-08T23:24:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.367+0000 7f8f2cff9700 1 -- 172.21.15.5:0/2924045737 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564e94cf8ec0 con 0x564e94e48fc0 2024-05-08T23:24:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.367+0000 7f8f2cff9700 10 client.6045 handle_command_reply: tid=0 2024-05-08T23:24:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.367+0000 7f8f5fc3fb80 1 client.6045 shutdown 2024-05-08T23:24:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/2924045737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 msgr2=0x564e94e4bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e4bc90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009b90 tx=0x7f8f3c009210 comp rx=0 tx=0).stop 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/2924045737 shutdown_connections 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e94ddafc0 0x564e94e4c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x564e94e48fc0 0x564e94e12880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564e94dae370 0x564e94e0b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 --2- 172.21.15.5:0/2924045737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e94dd83d0 0x564e94e4bc90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f2cff9700 0 client.6045 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/2924045737 >> 172.21.15.5:0/2924045737 conn(0x564e94d0e210 msgr2=0x564e94e086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/2924045737 shutdown_connections 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 1 -- 172.21.15.5:0/2924045737 wait complete. 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.368+0000 7f8f5fc3fb80 20 client.6045 trim_cache size 0 max 16384 2024-05-08T23:24:05.369 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:05.370 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:24:05.370 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:24:05.370 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 -- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f3006ac90 msgr2=0x7f8f3006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f3006ac90 0x7f8f3006d150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c151920 tx=0x7f8f48008040 comp rx=0 tx=0).stop 2024-05-08T23:24:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 -- 172.21.15.5:0/3090514763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f4c0d0890 msgr2=0x7f8f4c150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f4c0d0890 0x7f8f4c150870 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009fd0 tx=0x7f8f3c004750 comp rx=0 tx=0).stop 2024-05-08T23:24:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 -- 172.21.15.5:0/3090514763 shutdown_connections 2024-05-08T23:24:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8f3006ac90 0x7f8f3006d150 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f4c154df0 0x7f8f4c1520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8f4c0b43b0 0x7f8f4c148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 --2- 172.21.15.5:0/3090514763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f4c0d0890 0x7f8f4c150870 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 -- 172.21.15.5:0/3090514763 >> 172.21.15.5:0/3090514763 conn(0x7f8f4c099a50 msgr2=0x7f8f4c06f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 -- 172.21.15.5:0/3090514763 shutdown_connections 2024-05-08T23:24:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.369+0000 7f8f53a85700 1 -- 172.21.15.5:0/3090514763 wait complete. 2024-05-08T23:24:05.383 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 flush journal 2024-05-08T23:24:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.521+0000 7fec705df700 1 Processor -- start 2024-05-08T23:24:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.521+0000 7fec705df700 1 -- start start 2024-05-08T23:24:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec705df700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fec68161320 0x7fec680ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec705df700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d2230 0x7fec6806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec705df700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec680cf560 con 0x7fec680d1e50 2024-05-08T23:24:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec705df700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec680ce950 con 0x7fec68161320 2024-05-08T23:24:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec705df700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec680cec30 con 0x7fec680d2230 2024-05-08T23:24:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec6db7a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fec68161320 0x7fec680ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec6e37b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fec680d1e50 0x7fec68160c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:50264/0 (socket says 172.21.15.5:50264) 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec6eb7c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d2230 0x7fec6806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.522+0000 7fec6e37b700 1 -- 172.21.15.5:0/3656570489 learned_addr learned my addr 172.21.15.5:0/3656570489 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6d379700 1 -- 172.21.15.5:0/3656570489 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279596818 0 0) 0x7fec680cf560 con 0x7fec680d1e50 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6eb7c700 1 -- 172.21.15.5:0/3656570489 >> v2:172.21.15.49:3300/0 conn(0x7fec68161320 msgr2=0x7fec680ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/3656570489 >> v2:172.21.15.49:3300/0 conn(0x7fec68161320 0x7fec680ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6eb7c700 1 -- 172.21.15.5:0/3656570489 >> v1:172.21.15.49:6789/0 conn(0x7fec680d1e50 legacy=0x7fec68160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6eb7c700 1 -- 172.21.15.5:0/3656570489 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680cef10 con 0x7fec680d2230 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/3656570489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d2230 0x7fec6806bad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec60009d90 tx=0x7fec6000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92d92e4b95b24de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6d379700 1 -- 172.21.15.5:0/3656570489 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000e070 con 0x7fec680d2230 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6d379700 1 -- 172.21.15.5:0/3656570489 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6000bb20 con 0x7fec680d2230 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.523+0000 7fec6d379700 1 -- 172.21.15.5:0/3656570489 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60012430 con 0x7fec680d2230 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.524+0000 7fec705df700 1 -- 172.21.15.5:0/3656570489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d2230 msgr2=0x7fec6806bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.524+0000 7fec705df700 1 --2- 172.21.15.5:0/3656570489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d2230 0x7fec6806bad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec60009d90 tx=0x7fec6000b330 comp rx=0 tx=0).stop 2024-05-08T23:24:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.524+0000 7fec705df700 1 -- 172.21.15.5:0/3656570489 shutdown_connections 2024-05-08T23:24:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.524+0000 7fec705df700 1 --2- 172.21.15.5:0/3656570489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d2230 0x7fec6806bad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.524+0000 7fec705df700 1 --2- 172.21.15.5:0/3656570489 >> v2:172.21.15.49:3300/0 conn(0x7fec68161320 0x7fec680ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.524+0000 7fec705df700 1 -- 172.21.15.5:0/3656570489 >> 172.21.15.5:0/3656570489 conn(0x7fec6808afe0 msgr2=0x7fec680d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.524+0000 7fec705df700 1 -- 172.21.15.5:0/3656570489 shutdown_connections 2024-05-08T23:24:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.524+0000 7fec705df700 1 -- 172.21.15.5:0/3656570489 wait complete. 2024-05-08T23:24:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.526+0000 7fec705df700 1 Processor -- start 2024-05-08T23:24:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.526+0000 7fec705df700 1 -- start start 2024-05-08T23:24:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec705df700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d1e50 0x7fec680cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec705df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec680d2230 0x7fec680b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec705df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec68161320 0x7fec680b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec705df700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec680cd080 con 0x7fec68161320 2024-05-08T23:24:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec705df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec681f73e0 con 0x7fec680d2230 2024-05-08T23:24:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec705df700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec680cc9c0 con 0x7fec680d1e50 2024-05-08T23:24:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec68161320 0x7fec680b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec68161320 0x7fec680b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60370/0 (socket says 172.21.15.5:60370) 2024-05-08T23:24:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 -- 172.21.15.5:0/339381137 learned_addr learned my addr 172.21.15.5:0/339381137 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6e37b700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d1e50 0x7fec680cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6db7a700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec680d2230 0x7fec680b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 -- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d1e50 msgr2=0x7fec680cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d1e50 0x7fec680cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 -- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec680d2230 msgr2=0x7fec680b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec680d2230 0x7fec680b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 -- 172.21.15.5:0/339381137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60009a70 con 0x7fec68161320 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.527+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec68161320 0x7fec680b0ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fec60014040 tx=0x7fec60004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.528+0000 7fec5effd700 1 -- 172.21.15.5:0/339381137 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60012430 con 0x7fec68161320 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.528+0000 7fec5effd700 1 -- 172.21.15.5:0/339381137 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec60020070 con 0x7fec68161320 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.528+0000 7fec705df700 1 -- 172.21.15.5:0/339381137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec680ccca0 con 0x7fec68161320 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.528+0000 7fec5effd700 1 -- 172.21.15.5:0/339381137 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60004280 con 0x7fec68161320 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.528+0000 7fec705df700 1 -- 172.21.15.5:0/339381137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec680cb0a0 con 0x7fec68161320 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.529+0000 7fec5effd700 1 -- 172.21.15.5:0/339381137 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fec600043e0 con 0x7fec68161320 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.530+0000 7fec5effd700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec4c06ac90 0x7fec4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.530+0000 7fec5effd700 1 -- 172.21.15.5:0/339381137 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fec60022070 con 0x7fec68161320 2024-05-08T23:24:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.530+0000 7fec6e37b700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec4c06ac90 0x7fec4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.530+0000 7fec6e37b700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec4c06ac90 0x7fec4c06d150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fec64007fd0 tx=0x7fec64007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.532+0000 7fec7c799b80 1 Processor -- start 2024-05-08T23:24:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.532+0000 7fec7c799b80 1 -- start start 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd89e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec7c799b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd8939150 con 0x563dd8a057a0 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec7c799b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd8936b10 con 0x563dd8a0cdd0 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec7c799b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563dd8932400 con 0x563dd89e0370 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec6e37b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec6e37b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89de110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60382/0 (socket says 172.21.15.5:60382) 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec6e37b700 1 -- 172.21.15.5:0/121168582 learned_addr learned my addr 172.21.15.5:0/121168582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec6db7a700 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd89e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.533+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec6e37b700 1 -- 172.21.15.5:0/121168582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 msgr2=0x563dd89e30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec6e37b700 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd89e30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec6e37b700 1 -- 172.21.15.5:0/121168582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 msgr2=0x563dd8a0d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec6e37b700 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec6e37b700 1 -- 172.21.15.5:0/121168582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dd87b33b0 con 0x563dd8a057a0 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec6e37b700 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89de110 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fec640016f0 tx=0x7fec64004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff89ef93738658a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec4bfff700 1 -- 172.21.15.5:0/121168582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64021450 con 0x563dd8a057a0 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec4bfff700 1 -- 172.21.15.5:0/121168582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec640215b0 con 0x563dd8a057a0 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.534+0000 7fec4bfff700 1 -- 172.21.15.5:0/121168582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64021920 con 0x563dd8a057a0 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 -- 172.21.15.5:0/121168582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 msgr2=0x563dd89de110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89de110 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fec640016f0 tx=0x7fec64004f90 comp rx=0 tx=0).stop 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 -- 172.21.15.5:0/121168582 shutdown_connections 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89de110 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 --2- 172.21.15.5:0/121168582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd89e30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 -- 172.21.15.5:0/121168582 >> 172.21.15.5:0/121168582 conn(0x563dd8940210 msgr2=0x563dd8a27dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 -- 172.21.15.5:0/121168582 shutdown_connections 2024-05-08T23:24:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.535+0000 7fec7c799b80 1 -- 172.21.15.5:0/121168582 wait complete. 2024-05-08T23:24:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.536+0000 7fec7c799b80 1 -- start start 2024-05-08T23:24:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.536+0000 7fec7c799b80 1 Processor -- start 2024-05-08T23:24:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd89e0370 0x563dd89f4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89f4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd8a0cdd0 0x563dd89f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec7c799b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd88ce560 con 0x563dd8a057a0 2024-05-08T23:24:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec7c799b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd88ce200 con 0x563dd89e0370 2024-05-08T23:24:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec7c799b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563dd87b56a0 con 0x563dd8a0cdd0 2024-05-08T23:24:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec7c799b80 10 client.0 _ll_register_callbacks cb 0x563dd88d7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec6db7a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.537+0000 7fec6db7a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89f4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60386/0 (socket says 172.21.15.5:60386) 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6db7a700 1 -- 172.21.15.5:0/1902610559 learned_addr learned my addr 172.21.15.5:0/1902610559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6e37b700 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd89e0370 0x563dd89f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd8a0cdd0 0x563dd89f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6db7a700 1 -- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd8a0cdd0 msgr2=0x563dd89f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6db7a700 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd8a0cdd0 0x563dd89f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6db7a700 1 -- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd89e0370 msgr2=0x563dd89f4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6db7a700 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd89e0370 0x563dd89f4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6db7a700 1 -- 172.21.15.5:0/1902610559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec64001430 con 0x563dd8a057a0 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec6db7a700 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89f4990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fec580149d0 tx=0x7fec58014a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a764ca551320 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec497fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec497fa700 1 -- 172.21.15.5:0/1902610559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58015630 con 0x563dd8a057a0 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec497fa700 1 -- 172.21.15.5:0/1902610559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fec58015790 con 0x563dd8a057a0 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec7c799b80 1 -- 172.21.15.5:0/1902610559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563dd888b270 con 0x563dd8a057a0 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec497fa700 1 -- 172.21.15.5:0/1902610559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec58015950 con 0x563dd8a057a0 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec7c799b80 10 client.6054 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.538+0000 7fec7c799b80 1 -- 172.21.15.5:0/1902610559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x563dd875fb60 con 0x563dd8a057a0 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec497fa700 1 -- 172.21.15.5:0/1902610559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563dd888b270 con 0x563dd8a057a0 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec497fa700 1 -- 172.21.15.5:0/1902610559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fec58015ab0 con 0x563dd8a057a0 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec7c799b80 10 client.6054 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec7c799b80 10 client.6054 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec7c799b80 20 client.6054 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec7c799b80 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x563dd89f8c70 0x563dd89f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec7c799b80 4 client.6054 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec7c799b80 1 -- 172.21.15.5:0/1902610559 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563dd89b17e0 con 0x563dd89f8c70 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec6e37b700 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x563dd89f8c70 0x563dd89f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec497fa700 10 client.6054 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.539+0000 7fec6e37b700 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x563dd89f8c70 0x563dd89f7e00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f7e5476bfbeb7f3 server_cookie=65df295324a91bbb in_seq=0 out_seq=0 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec497fa700 1 -- 172.21.15.5:0/1902610559 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563dd89b17e0 con 0x563dd89f8c70 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec497fa700 10 client.6054 handle_command_reply: tid=0 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 client.6054 shutdown 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 -- 172.21.15.5:0/1902610559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 msgr2=0x563dd89f4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89f4990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fec580149d0 tx=0x7fec58014a00 comp rx=0 tx=0).stop 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 -- 172.21.15.5:0/1902610559 shutdown_connections 2024-05-08T23:24:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x563dd89f8c70 0x563dd89f7e00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd8a0cdd0 0x563dd89f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd89e0370 0x563dd89f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec497fa700 0 client.6054 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 --2- 172.21.15.5:0/1902610559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89f4990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 -- 172.21.15.5:0/1902610559 >> 172.21.15.5:0/1902610559 conn(0x563dd8940210 msgr2=0x563dd8a280c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.540+0000 7fec7c799b80 1 -- 172.21.15.5:0/1902610559 shutdown_connections 2024-05-08T23:24:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.541+0000 7fec7c799b80 1 -- 172.21.15.5:0/1902610559 wait complete. 2024-05-08T23:24:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.541+0000 7fec7c799b80 20 client.6054 trim_cache size 0 max 16384 2024-05-08T23:24:05.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.557+0000 7fec7c799b80 1 Processor -- start 2024-05-08T23:24:05.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.557+0000 7fec7c799b80 1 -- start start 2024-05-08T23:24:05.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd89e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 0x563dd89f4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec7c799b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd89f3310 con 0x563dd8a057a0 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec7c799b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd89f62b0 con 0x563dd8a0cdd0 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec7c799b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563dd89df850 con 0x563dd89e0370 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89e1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60392/0 (socket says 172.21.15.5:60392) 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 -- 172.21.15.5:0/2005012354 learned_addr learned my addr 172.21.15.5:0/2005012354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6e37b700 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd89e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 0x563dd89f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 -- 172.21.15.5:0/2005012354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 msgr2=0x563dd89e1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd89e1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 -- 172.21.15.5:0/2005012354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 msgr2=0x563dd89f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 0x563dd89f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 -- 172.21.15.5:0/2005012354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dd88f8560 con 0x563dd8a057a0 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec6db7a700 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89e1ca0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fec580090b0 tx=0x7fec58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb238641e6130a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec5cff9700 1 -- 172.21.15.5:0/2005012354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58005ce0 con 0x563dd8a057a0 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec5cff9700 1 -- 172.21.15.5:0/2005012354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec58005e60 con 0x563dd8a057a0 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.558+0000 7fec5cff9700 1 -- 172.21.15.5:0/2005012354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5801ccc0 con 0x563dd8a057a0 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 -- 172.21.15.5:0/2005012354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 msgr2=0x563dd89e1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89e1ca0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fec580090b0 tx=0x7fec58017000 comp rx=0 tx=0).stop 2024-05-08T23:24:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 -- 172.21.15.5:0/2005012354 shutdown_connections 2024-05-08T23:24:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a0cdd0 0x563dd89f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd89e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 --2- 172.21.15.5:0/2005012354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a057a0 0x563dd89e1ca0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 -- 172.21.15.5:0/2005012354 >> 172.21.15.5:0/2005012354 conn(0x563dd8940210 msgr2=0x563dd89d91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 -- 172.21.15.5:0/2005012354 shutdown_connections 2024-05-08T23:24:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.559+0000 7fec7c799b80 1 -- 172.21.15.5:0/2005012354 wait complete. 2024-05-08T23:24:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.560+0000 7fec7c799b80 1 -- start start 2024-05-08T23:24:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.560+0000 7fec7c799b80 1 Processor -- start 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd8a90300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a057a0 0x563dd8a94060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec7c799b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec7c799b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd8a93c40 con 0x563dd8a0cdd0 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec7c799b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd8a90d80 con 0x563dd8a057a0 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec7c799b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563dd8a938f0 con 0x563dd89e0370 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec7c799b80 10 client.0 _ll_register_callbacks cb 0x7fec58009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec6eb7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec6eb7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60402/0 (socket says 172.21.15.5:60402) 2024-05-08T23:24:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.561+0000 7fec6eb7c700 1 -- 172.21.15.5:0/987745333 learned_addr learned my addr 172.21.15.5:0/987745333 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec6db7a700 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a057a0 0x563dd8a94060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec6e37b700 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd8a90300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec6eb7c700 1 -- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 msgr2=0x563dd8a90300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd8a90300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec6eb7c700 1 -- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a057a0 msgr2=0x563dd8a94060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a057a0 0x563dd8a94060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec6eb7c700 1 -- 172.21.15.5:0/987745333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563dd875fb60 con 0x563dd8a0cdd0 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec6eb7c700 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d1b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fec600562b0 tx=0x7fec6005a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e77b3ca2f9b095a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec497fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec497fa700 1 -- 172.21.15.5:0/987745333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60089200 con 0x563dd8a0cdd0 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec497fa700 1 -- 172.21.15.5:0/987745333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fec6005dc30 con 0x563dd8a0cdd0 2024-05-08T23:24:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec7c799b80 1 -- 172.21.15.5:0/987745333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563dd89f6890 con 0x563dd8a0cdd0 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec497fa700 1 -- 172.21.15.5:0/987745333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec600621f0 con 0x563dd8a0cdd0 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec7c799b80 10 client.6060 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec7c799b80 1 -- 172.21.15.5:0/987745333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x563dd87b7cd0 con 0x563dd8a0cdd0 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.562+0000 7fec497fa700 1 -- 172.21.15.5:0/987745333 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563dd89f6890 con 0x563dd8a0cdd0 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec497fa700 1 -- 172.21.15.5:0/987745333 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fec60004690 con 0x563dd8a0cdd0 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec7c799b80 10 client.6060 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec7c799b80 10 client.6060 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec7c799b80 20 client.6060 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec7c799b80 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x563dd8a945a0 0x563dd8a0d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec7c799b80 4 client.6060 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec7c799b80 1 -- 172.21.15.5:0/987745333 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fec640215b0 con 0x563dd8a945a0 2024-05-08T23:24:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec6e37b700 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x563dd8a945a0 0x563dd8a0d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec497fa700 10 client.6060 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.563+0000 7fec6e37b700 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x563dd8a945a0 0x563dd8a0d6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c52f37b6292b975c server_cookie=f30789109b54dbb1 in_seq=0 out_seq=0 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec497fa700 1 -- 172.21.15.5:0/987745333 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fec640215b0 con 0x563dd8a945a0 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec497fa700 10 client.6060 handle_command_reply: tid=0 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 client.6060 shutdown 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 -- 172.21.15.5:0/987745333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a0cdd0 msgr2=0x563dd8a0d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d1b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fec600562b0 tx=0x7fec6005a340 comp rx=0 tx=0).stop 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 -- 172.21.15.5:0/987745333 shutdown_connections 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x563dd8a945a0 0x563dd8a0d6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563dd8a0cdd0 0x563dd8a0d1b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563dd89e0370 0x563dd8a90300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec497fa700 0 client.6060 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 --2- 172.21.15.5:0/987745333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563dd8a057a0 0x563dd8a94060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 -- 172.21.15.5:0/987745333 >> 172.21.15.5:0/987745333 conn(0x563dd8940210 msgr2=0x563dd89dbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 -- 172.21.15.5:0/987745333 shutdown_connections 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.571+0000 7fec7c799b80 1 -- 172.21.15.5:0/987745333 wait complete. 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec7c799b80 20 client.6060 trim_cache size 0 max 16384 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:24:05.573 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 -- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec4c06ac90 msgr2=0x7fec4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec4c06ac90 0x7fec4c06d150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fec64007fd0 tx=0x7fec64007de0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 -- 172.21.15.5:0/339381137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec68161320 msgr2=0x7fec680b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec68161320 0x7fec680b0ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fec60014040 tx=0x7fec60004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 -- 172.21.15.5:0/339381137 shutdown_connections 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec4c06ac90 0x7fec4c06d150 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec68161320 0x7fec680b0ad0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fec680d1e50 0x7fec680cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 --2- 172.21.15.5:0/339381137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec680d2230 0x7fec680b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 -- 172.21.15.5:0/339381137 >> 172.21.15.5:0/339381137 conn(0x7fec6808afe0 msgr2=0x7fec680d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 -- 172.21.15.5:0/339381137 shutdown_connections 2024-05-08T23:24:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.572+0000 7fec705df700 1 -- 172.21.15.5:0/339381137 wait complete. 2024-05-08T23:24:05.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 flush journal 2024-05-08T23:24:05.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.724+0000 7f6daaa43700 1 Processor -- start 2024-05-08T23:24:05.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.724+0000 7f6daaa43700 1 -- start start 2024-05-08T23:24:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.725+0000 7f6daaa43700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6da415a7c0 0x7f6da40cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.725+0000 7f6daaa43700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da40cd7c0 con 0x7f6da40727b0 2024-05-08T23:24:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.725+0000 7f6daaa43700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da40cd1b0 con 0x7f6da40ca130 2024-05-08T23:24:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.725+0000 7f6daaa43700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da415aba0 con 0x7f6da415a7c0 2024-05-08T23:24:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.725+0000 7f6da37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6da40727b0 0x7f6da40c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45904/0 (socket says 172.21.15.5:45904) 2024-05-08T23:24:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.725+0000 7f6da37fe700 1 -- 172.21.15.5:0/624321530 learned_addr learned my addr 172.21.15.5:0/624321530 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da3fff700 1 --2- 172.21.15.5:0/624321530 >> v2:172.21.15.49:3300/0 conn(0x7f6da415a7c0 0x7f6da40cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480836401 0 0) 0x7f6da40cd7c0 con 0x7f6da40727b0 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d8c0035b0 con 0x7f6da40727b0 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 715797830 0 0) 0x7f6d8c0035b0 con 0x7f6da40727b0 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 >> v2:172.21.15.49:3300/0 conn(0x7f6da415a7c0 msgr2=0x7f6da40cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 --2- 172.21.15.5:0/624321530 >> v2:172.21.15.49:3300/0 conn(0x7f6da415a7c0 0x7f6da40cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 >> v1:172.21.15.49:6789/0 conn(0x7f6da40ca130 legacy=0x7f6da40b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da40b3e70 con 0x7f6da40727b0 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f6d98001670 con 0x7f6da40727b0 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 >> v1:172.21.15.5:6789/0 conn(0x7f6da40727b0 legacy=0x7f6da40c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c003a00 0x7f6d8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d8c0043b0 0x7f6d8c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c005110 0x7f6d8c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da415aba0 con 0x7f6d8c0043b0 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c00c060 con 0x7f6d8c003a00 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c00c220 con 0x7f6d8c005110 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da37fe700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d8c0043b0 0x7f6d8c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.726+0000 7f6da3fff700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c003a00 0x7f6d8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da8fe0700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c005110 0x7f6d8c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da37fe700 1 -- 172.21.15.5:0/624321530 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c005110 msgr2=0x7f6d8c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da37fe700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c005110 0x7f6d8c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da37fe700 1 -- 172.21.15.5:0/624321530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c003a00 msgr2=0x7f6d8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da37fe700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c003a00 0x7f6d8c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da37fe700 1 -- 172.21.15.5:0/624321530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0035b0 con 0x7f6d8c0043b0 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da8fe0700 1 -- 172.21.15.5:0/624321530 reap_dead start 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da37fe700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d8c0043b0 0x7f6d8c004870 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6da4072c30 tx=0x7f6d8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f15c65400caae98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98005b60 con 0x7f6d8c0043b0 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98005cc0 con 0x7f6d8c0043b0 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.727+0000 7f6da2ffd700 1 -- 172.21.15.5:0/624321530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98008320 con 0x7f6d8c0043b0 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.728+0000 7f6daaa43700 1 -- 172.21.15.5:0/624321530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d8c0043b0 msgr2=0x7f6d8c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.728+0000 7f6daaa43700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d8c0043b0 0x7f6d8c004870 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6da4072c30 tx=0x7f6d8c005880 comp rx=0 tx=0).stop 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.728+0000 7f6daaa43700 1 -- 172.21.15.5:0/624321530 shutdown_connections 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.728+0000 7f6daaa43700 1 --2- 172.21.15.5:0/624321530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d8c0043b0 0x7f6d8c004870 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.728+0000 7f6daaa43700 1 -- 172.21.15.5:0/624321530 >> 172.21.15.5:0/624321530 conn(0x7f6da406a960 msgr2=0x7f6da406ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.728+0000 7f6daaa43700 1 -- 172.21.15.5:0/624321530 shutdown_connections 2024-05-08T23:24:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.728+0000 7f6daaa43700 1 -- 172.21.15.5:0/624321530 wait complete. 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.729+0000 7f6daaa43700 1 Processor -- start 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.729+0000 7f6daaa43700 1 -- start start 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6daaa43700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c0043b0 0x7f6da415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6daaa43700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6da415da10 0x7f6da4151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6daaa43700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6da415ce10 0x7f6da4152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6daaa43700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da415bad0 con 0x7f6da415da10 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6daaa43700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da415bd70 con 0x7f6da415ce10 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6daaa43700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da415c050 con 0x7f6d8c0043b0 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6da37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6da415da10 0x7f6da4151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6da37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6da415da10 0x7f6da4151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60416/0 (socket says 172.21.15.5:60416) 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6da37fe700 1 -- 172.21.15.5:0/634182355 learned_addr learned my addr 172.21.15.5:0/634182355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6da8fe0700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6da415ce10 0x7f6da4152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6da3fff700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c0043b0 0x7f6da415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6da37fe700 1 -- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c0043b0 msgr2=0x7f6da415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6da37fe700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c0043b0 0x7f6da415d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.730+0000 7f6da37fe700 1 -- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6da415ce10 msgr2=0x7f6da4152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.731+0000 7f6da37fe700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6da415ce10 0x7f6da4152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.731+0000 7f6da37fe700 1 -- 172.21.15.5:0/634182355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da415c330 con 0x7f6da415da10 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.731+0000 7f6da37fe700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6da415da10 0x7f6da4151c10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c030 tx=0x7f6d98001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.731+0000 7f6da0ff9700 1 -- 172.21.15.5:0/634182355 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98001630 con 0x7f6da415da10 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.731+0000 7f6da0ff9700 1 -- 172.21.15.5:0/634182355 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d980018e0 con 0x7f6da415da10 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.731+0000 7f6daaa43700 1 -- 172.21.15.5:0/634182355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da415c610 con 0x7f6da415da10 2024-05-08T23:24:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.731+0000 7f6da0ff9700 1 -- 172.21.15.5:0/634182355 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98008320 con 0x7f6da415da10 2024-05-08T23:24:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.731+0000 7f6daaa43700 1 -- 172.21.15.5:0/634182355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da414f340 con 0x7f6da415da10 2024-05-08T23:24:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.733+0000 7f6da0ff9700 1 -- 172.21.15.5:0/634182355 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d98008480 con 0x7f6da415da10 2024-05-08T23:24:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.733+0000 7f6da0ff9700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d9006ac90 0x7f6d9006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.733+0000 7f6da0ff9700 1 -- 172.21.15.5:0/634182355 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6d98089a50 con 0x7f6da415da10 2024-05-08T23:24:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.733+0000 7f6da3fff700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d9006ac90 0x7f6d9006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.734+0000 7f6da3fff700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d9006ac90 0x7f6d9006d150 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6da40680b0 tx=0x7f6d94009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.735+0000 7f6db6bfdb80 1 Processor -- start 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.735+0000 7f6db6bfdb80 1 -- start start 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb53c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53f3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb531f150 con 0x55cdb53f2dd0 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb531cb10 con 0x55cdb53eb7a0 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdb5318400 con 0x55cdb53c6370 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6da37fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb53c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6da8fe0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6da3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6da37fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb53c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:46176/0 (socket says 172.21.15.5:46176) 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6da37fe700 1 -- 172.21.15.5:0/3904206462 learned_addr learned my addr 172.21.15.5:0/3904206462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.736+0000 7f6da3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47414/0 (socket says 172.21.15.5:47414) 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6da8fe0700 1 -- 172.21.15.5:0/3904206462 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 msgr2=0x55cdb53c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6da8fe0700 1 --2- 172.21.15.5:0/3904206462 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb53c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6da8fe0700 1 -- 172.21.15.5:0/3904206462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 msgr2=0x55cdb53c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6da8fe0700 1 --2- 172.21.15.5:0/3904206462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6da8fe0700 1 -- 172.21.15.5:0/3904206462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdb51993b0 con 0x55cdb53f2dd0 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6da37fe700 1 --2- 172.21.15.5:0/3904206462 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb53c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6da8fe0700 1 --2- 172.21.15.5:0/3904206462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53f3250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6da406b910 tx=0x7f6d9c011250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd5885e70827714 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6d89ffb700 1 -- 172.21.15.5:0/3904206462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c003c10 con 0x55cdb53f2dd0 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6d89ffb700 1 -- 172.21.15.5:0/3904206462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c003d70 con 0x55cdb53f2dd0 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.737+0000 7f6d89ffb700 1 -- 172.21.15.5:0/3904206462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0119e0 con 0x55cdb53f2dd0 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/3904206462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 msgr2=0x55cdb53f3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/3904206462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53f3250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6da406b910 tx=0x7f6d9c011250 comp rx=0 tx=0).stop 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/3904206462 shutdown_connections 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/3904206462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53f3250 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/3904206462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/3904206462 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb53c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/3904206462 >> 172.21.15.5:0/3904206462 conn(0x55cdb5326210 msgr2=0x55cdb540ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/3904206462 shutdown_connections 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/3904206462 wait complete. 2024-05-08T23:24:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.738+0000 7f6db6bfdb80 1 -- start start 2024-05-08T23:24:05.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.739+0000 7f6db6bfdb80 1 Processor -- start 2024-05-08T23:24:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53c6370 0x55cdb541dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53eb7a0 0x55cdb53f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 0x55cdb53f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb52b4560 con 0x55cdb53eb7a0 2024-05-08T23:24:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb52b4200 con 0x55cdb53c6370 2024-05-08T23:24:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdb519b6a0 con 0x55cdb53f2dd0 2024-05-08T23:24:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6db6bfdb80 10 client.0 _ll_register_callbacks cb 0x55cdb52bdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53eb7a0 0x55cdb53f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53eb7a0 0x55cdb53f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60424/0 (socket says 172.21.15.5:60424) 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 -- 172.21.15.5:0/743466345 learned_addr learned my addr 172.21.15.5:0/743466345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da3fff700 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53c6370 0x55cdb541dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da8fe0700 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 0x55cdb53f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 -- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 msgr2=0x55cdb53f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 0x55cdb53f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 -- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53c6370 msgr2=0x55cdb541dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53c6370 0x55cdb541dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 -- 172.21.15.5:0/743466345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da414f340 con 0x55cdb53eb7a0 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6da37fe700 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53eb7a0 0x55cdb53f64a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6d98087f40 tx=0x7f6d98004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2521cf2309531 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6d82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6d82ffd700 1 -- 172.21.15.5:0/743466345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9805c830 con 0x55cdb53eb7a0 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6d82ffd700 1 -- 172.21.15.5:0/743466345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6d98060df0 con 0x55cdb53eb7a0 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/743466345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdb5271270 con 0x55cdb53eb7a0 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.740+0000 7f6d82ffd700 1 -- 172.21.15.5:0/743466345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d98057ef0 con 0x55cdb53eb7a0 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6db6bfdb80 10 client.6072 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/743466345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x7f6d9c00f040 con 0x55cdb53eb7a0 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6d82ffd700 1 -- 172.21.15.5:0/743466345 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cdb5271270 con 0x55cdb53eb7a0 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6d82ffd700 1 -- 172.21.15.5:0/743466345 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f6d980771f0 con 0x55cdb53eb7a0 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6db6bfdb80 10 client.6072 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6db6bfdb80 10 client.6072 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6db6bfdb80 20 client.6072 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55cdb5411a30 0x55cdb541e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6db6bfdb80 4 client.6072 mds_command: new command op to 5180 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/743466345 --> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdb53977e0 con 0x55cdb5411a30 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6da3fff700 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55cdb5411a30 0x55cdb541e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6d82ffd700 10 client.6072 ms_handle_connect on v2:172.21.15.49:6836/350829837 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.741+0000 7f6da3fff700 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55cdb5411a30 0x55cdb541e280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10ee345249a1aa5e server_cookie=873f5d8e85334cf7 in_seq=0 out_seq=0 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6d82ffd700 1 -- 172.21.15.5:0/743466345 <== mds.2 v2:172.21.15.49:6836/350829837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cdb53977e0 con 0x55cdb5411a30 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6d82ffd700 10 client.6072 handle_command_reply: tid=0 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 client.6072 shutdown 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/743466345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53eb7a0 msgr2=0x55cdb53f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53eb7a0 0x55cdb53f64a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6d98087f40 tx=0x7f6d98004e60 comp rx=0 tx=0).stop 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/743466345 shutdown_connections 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 0x55cdb53f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55cdb5411a30 0x55cdb541e280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53c6370 0x55cdb541dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/743466345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53eb7a0 0x55cdb53f64a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6d82ffd700 0 client.6072 ms_handle_reset on v2:172.21.15.49:6836/350829837 2024-05-08T23:24:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/743466345 >> 172.21.15.5:0/743466345 conn(0x55cdb5326210 msgr2=0x55cdb53c84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/743466345 shutdown_connections 2024-05-08T23:24:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.742+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/743466345 wait complete. 2024-05-08T23:24:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.743+0000 7f6db6bfdb80 20 client.6072 trim_cache size 0 max 16384 2024-05-08T23:24:05.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.758+0000 7f6db6bfdb80 1 Processor -- start 2024-05-08T23:24:05.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.758+0000 7f6db6bfdb80 1 -- start start 2024-05-08T23:24:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53c6370 0x55cdb53f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 0x55cdb53eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb53c1d80 con 0x55cdb53c6370 2024-05-08T23:24:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb53c4af0 con 0x55cdb53eb7a0 2024-05-08T23:24:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdb53f6460 con 0x55cdb53f2dd0 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53c6370 0x55cdb53f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53c6370 0x55cdb53f31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60438/0 (socket says 172.21.15.5:60438) 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da3fff700 1 -- 172.21.15.5:0/1382288171 learned_addr learned my addr 172.21.15.5:0/1382288171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da37fe700 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da8fe0700 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 0x55cdb53eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da3fff700 1 -- 172.21.15.5:0/1382288171 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 msgr2=0x55cdb53eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da3fff700 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 0x55cdb53eedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da3fff700 1 -- 172.21.15.5:0/1382288171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 msgr2=0x55cdb53ee8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.759+0000 7f6da3fff700 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53ee8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6da3fff700 1 -- 172.21.15.5:0/1382288171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdb5145b60 con 0x55cdb53c6370 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6da3fff700 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53c6370 0x55cdb53f31b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009470 tx=0x7f6d94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ec2bf864c798c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6d8a7fc700 1 -- 172.21.15.5:0/1382288171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9402a070 con 0x55cdb53c6370 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6d8a7fc700 1 -- 172.21.15.5:0/1382288171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d94019070 con 0x55cdb53c6370 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6d8a7fc700 1 -- 172.21.15.5:0/1382288171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94014c40 con 0x55cdb53c6370 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1382288171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53c6370 msgr2=0x55cdb53f31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53c6370 0x55cdb53f31b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009470 tx=0x7f6d94017000 comp rx=0 tx=0).stop 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1382288171 shutdown_connections 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53f2dd0 0x55cdb53eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53c6370 0x55cdb53f31b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1382288171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb53ee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1382288171 >> 172.21.15.5:0/1382288171 conn(0x55cdb5326210 msgr2=0x55cdb53bf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.760+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1382288171 shutdown_connections 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.761+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1382288171 wait complete. 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.761+0000 7f6db6bfdb80 1 -- start start 2024-05-08T23:24:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.761+0000 7f6db6bfdb80 1 Processor -- start 2024-05-08T23:24:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb5414b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb5476200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6db6bfdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb53c8a40 con 0x55cdb53f2dd0 2024-05-08T23:24:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb5403760 con 0x55cdb53eb7a0 2024-05-08T23:24:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6db6bfdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdb53c8c50 con 0x55cdb53c6370 2024-05-08T23:24:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6db6bfdb80 10 client.0 _ll_register_callbacks cb 0x55cdb52bdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6da8fe0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6da37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb5476200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6da3fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb5414b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6da37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb5476200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47428/0 (socket says 172.21.15.5:47428) 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6da8fe0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53ee8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60444/0 (socket says 172.21.15.5:60444) 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.762+0000 7f6da37fe700 1 -- 172.21.15.5:0/1128719411 learned_addr learned my addr 172.21.15.5:0/1128719411 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6da3fff700 1 -- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 msgr2=0x55cdb5476200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6da3fff700 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb5476200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6da3fff700 1 -- 172.21.15.5:0/1128719411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 msgr2=0x55cdb53ee8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6da3fff700 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53ee8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6da3fff700 1 -- 172.21.15.5:0/1128719411 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d94004090 con 0x55cdb53c6370 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6da8fe0700 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53ee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6da3fff700 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb5414b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d94024000 tx=0x7f6d94004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cefea12222286c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6d82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3301/0 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6d82ffd700 1 -- 172.21.15.5:0/1128719411 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94005a20 con 0x55cdb53c6370 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6d82ffd700 1 -- 172.21.15.5:0/1128719411 <== mon.2 v2:172.21.15.49:3301/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6d94005b80 con 0x55cdb53c6370 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1128719411 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d9c003d70 con 0x55cdb53c6370 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6d82ffd700 1 -- 172.21.15.5:0/1128719411 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d94005d60 con 0x55cdb53c6370 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6db6bfdb80 10 client.5357 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1128719411 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55cdb52de560 con 0x55cdb53c6370 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.763+0000 7f6d82ffd700 1 -- 172.21.15.5:0/1128719411 <== mon.2 v2:172.21.15.49:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c003d70 con 0x55cdb53c6370 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6d82ffd700 1 -- 172.21.15.5:0/1128719411 <== mon.2 v2:172.21.15.49:3301/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f6d94032070 con 0x55cdb53c6370 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6db6bfdb80 10 client.5357 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6db6bfdb80 10 client.5357 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6db6bfdb80 20 client.5357 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55cdb53c4440 0x55cdb53eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6db6bfdb80 4 client.5357 mds_command: new command op to 5180 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:24:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1128719411 --> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cdb5310ec0 con 0x55cdb53c4440 2024-05-08T23:24:05.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6da37fe700 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55cdb53c4440 0x55cdb53eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:05.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6d82ffd700 10 client.5357 ms_handle_connect on v2:172.21.15.49:6836/350829837 2024-05-08T23:24:05.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.764+0000 7f6da37fe700 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55cdb53c4440 0x55cdb53eedf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c84cac1fa2413d8d server_cookie=41240228418460fa in_seq=0 out_seq=0 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.770+0000 7f6d82ffd700 1 -- 172.21.15.5:0/1128719411 <== mds.2 v2:172.21.15.49:6836/350829837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cdb5310ec0 con 0x55cdb53c4440 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.770+0000 7f6d82ffd700 10 client.5357 handle_command_reply: tid=0 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.770+0000 7f6db6bfdb80 1 client.5357 shutdown 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 msgr2=0x55cdb5414b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb5414b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d94024000 tx=0x7f6d94004750 comp rx=0 tx=0).stop 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1128719411 shutdown_connections 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb53f2dd0 0x55cdb53ee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cdb53c6370 0x55cdb5414b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55cdb53c4440 0x55cdb53eedf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 --2- 172.21.15.5:0/1128719411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cdb53eb7a0 0x55cdb5476200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1128719411 >> 172.21.15.5:0/1128719411 conn(0x55cdb5326210 msgr2=0x55cdb53ead90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6d82ffd700 0 client.5357 ms_handle_reset on v2:172.21.15.49:6836/350829837 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1128719411 shutdown_connections 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 1 -- 172.21.15.5:0/1128719411 wait complete. 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6db6bfdb80 20 client.5357 trim_cache size 0 max 16384 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:24:05.772 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 -- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d9006ac90 msgr2=0x7f6d9006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d9006ac90 0x7f6d9006d150 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6da40680b0 tx=0x7f6d94009210 comp rx=0 tx=0).stop 2024-05-08T23:24:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 -- 172.21.15.5:0/634182355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6da415da10 msgr2=0x7f6da4151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6da415da10 0x7f6da4151c10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c030 tx=0x7f6d98001e30 comp rx=0 tx=0).stop 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 -- 172.21.15.5:0/634182355 shutdown_connections 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d9006ac90 0x7f6d9006d150 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6da415ce10 0x7f6da4152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d8c0043b0 0x7f6da415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 --2- 172.21.15.5:0/634182355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6da415da10 0x7f6da4151c10 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.771+0000 7f6daaa43700 1 -- 172.21.15.5:0/634182355 >> 172.21.15.5:0/634182355 conn(0x7f6da406a960 msgr2=0x7f6da40713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.772+0000 7f6daaa43700 1 -- 172.21.15.5:0/634182355 shutdown_connections 2024-05-08T23:24:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:05.772+0000 7f6daaa43700 1 -- 172.21.15.5:0/634182355 wait complete. 2024-05-08T23:24:05.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:05.785 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-05-08T23:24:05.812 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/ 2024-05-08T23:24:05.812 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4 2024-05-08T23:24:05.812 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5 2024-05-08T23:24:05.813 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6 2024-05-08T23:24:05.813 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7 2024-05-08T23:24:05.813 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-05-08T23:24:05.813 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:24:05.813 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:24:05.813 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:24:05.813 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:24:05.813 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-05-08T23:24:05.813 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:24:05.931 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-08T23:24:06.120 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:24:06.120 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:24:06.121 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:24:06.121 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:24:06.121 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-05-08T23:24:06.121 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:24:06.243 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-05-08T23:24:06.431 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:24:06.431 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:24:06.431 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:24:06.431 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:24:06.431 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-05-08T23:24:06.431 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:24:06.553 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-05-08T23:24:06.741 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:24:06.741 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:24:06.741 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:24:06.741 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:24:06.742 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-05-08T23:24:06.742 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:24:06.864 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627781 2024-05-08T23:24:07.052 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:24:07.052 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:24:07.052 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:24:07.052 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:24:07.052 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-05-08T23:24:07.052 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:24:07.173 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627782 2024-05-08T23:24:07.362 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:24:07.363 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:24:07.363 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:24:07.363 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:24:07.363 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-05-08T23:24:07.363 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:24:07.484 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627783 2024-05-08T23:24:07.672 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-05-08T23:24:07.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:24:07.717 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:24:07.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.768+0000 7ff0ac0b4080 1 Processor -- start 2024-05-08T23:24:07.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.768+0000 7ff0ac0b4080 1 -- start start 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0ac0b4080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1047efa0 0x55ac10555970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0ac0b4080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac1054f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0ac0b4080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac10551e10 0x55ac105501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0ac0b4080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac104241a0 con 0x55ac104c7dc0 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0ac0b4080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac103bf6d0 con 0x55ac1047efa0 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0ac0b4080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac103bf370 con 0x55ac10551e10 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0a1985700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac1054f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0a1985700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac1054f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46778/0 (socket says 172.21.15.5:46778) 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0a1985700 1 -- 172.21.15.5:0/2031675163 learned_addr learned my addr 172.21.15.5:0/2031675163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:07.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.769+0000 7ff0a2186700 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1047efa0 0x55ac10555970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a2987700 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac10551e10 0x55ac105501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a1985700 1 -- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac10551e10 msgr2=0x55ac105501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a1985700 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac10551e10 0x55ac105501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a1985700 1 -- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1047efa0 msgr2=0x55ac10555970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a1985700 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1047efa0 0x55ac10555970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a1985700 1 -- 172.21.15.5:0/2031675163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c00a100 con 0x55ac104c7dc0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a2987700 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac10551e10 0x55ac105501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a1985700 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac1054f580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff09800a7a0 tx=0x7ff09800aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f096453070db9a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a0983700 1 -- 172.21.15.5:0/2031675163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800f070 con 0x55ac104c7dc0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a0983700 1 -- 172.21.15.5:0/2031675163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff098008580 con 0x55ac104c7dc0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.770+0000 7ff0a0983700 1 -- 172.21.15.5:0/2031675163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980088f0 con 0x55ac104c7dc0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/2031675163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 msgr2=0x55ac1054f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac1054f580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff09800a7a0 tx=0x7ff09800aab0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/2031675163 shutdown_connections 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac10551e10 0x55ac105501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1047efa0 0x55ac10555970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 --2- 172.21.15.5:0/2031675163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac1054f580 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/2031675163 >> 172.21.15.5:0/2031675163 conn(0x55ac1047dba0 msgr2=0x55ac104c6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/2031675163 shutdown_connections 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.771+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/2031675163 wait complete. 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.772+0000 7ff0ac0b4080 1 Processor -- start 2024-05-08T23:24:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.772+0000 7ff0ac0b4080 1 -- start start 2024-05-08T23:24:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0ac0b4080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1047efa0 0x55ac1053dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0ac0b4080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac104c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0ac0b4080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac10551e10 0x55ac1054d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0ac0b4080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac103b6680 con 0x55ac104c7dc0 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0ac0b4080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac10544980 con 0x55ac10551e10 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0ac0b4080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac104c4de0 con 0x55ac1047efa0 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a1985700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac104c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a1985700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac104c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46794/0 (socket says 172.21.15.5:46794) 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a1985700 1 -- 172.21.15.5:0/3398943114 learned_addr learned my addr 172.21.15.5:0/3398943114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a2987700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac10551e10 0x55ac1054d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a2186700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1047efa0 0x55ac1053dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a1985700 1 -- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1047efa0 msgr2=0x55ac1053dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a1985700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1047efa0 0x55ac1053dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a1985700 1 -- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac10551e10 msgr2=0x55ac1054d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a1985700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac10551e10 0x55ac1054d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.773+0000 7ff0a1985700 1 -- 172.21.15.5:0/3398943114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09800a420 con 0x55ac104c7dc0 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.774+0000 7ff0a1985700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac104c48a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff098014000 tx=0x7ff098003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.774+0000 7ff0937fe700 1 -- 172.21.15.5:0/3398943114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098015630 con 0x55ac104c7dc0 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.774+0000 7ff0937fe700 1 -- 172.21.15.5:0/3398943114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0980157b0 con 0x55ac104c7dc0 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.774+0000 7ff0937fe700 1 -- 172.21.15.5:0/3398943114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098015b60 con 0x55ac104c7dc0 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.774+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac103ce8a0 con 0x55ac104c7dc0 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.774+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac103c1ed0 con 0x55ac104c7dc0 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.775+0000 7ff0937fe700 1 -- 172.21.15.5:0/3398943114 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff098013070 con 0x55ac104c7dc0 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.776+0000 7ff0937fe700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0840695e0 0x7ff08406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.776+0000 7ff0937fe700 1 -- 172.21.15.5:0/3398943114 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff098089270 con 0x55ac104c7dc0 2024-05-08T23:24:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.776+0000 7ff0a2186700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0840695e0 0x7ff08406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.776+0000 7ff0ac0b4080 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55ac10540390 0x55ac10540770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.776+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 --> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v8 -- 0x55ac10540cb0 con 0x55ac10540390 2024-05-08T23:24:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.776+0000 7ff0a2987700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55ac10540390 0x55ac10540770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.777+0000 7ff0a2186700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0840695e0 0x7ff08406baa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff094000f60 tx=0x7ff094006850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.777+0000 7ff0a2987700 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55ac10540390 0x55ac10540770 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0a2987700 1 -- 172.21.15.5:0/3398943114 <== osd.7 v2:172.21.15.49:6822/47408 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v48'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff09c00fa60 con 0x55ac10540390 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55ac10540390 msgr2=0x55ac10540770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0ac0b4080 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55ac10540390 0x55ac10540770 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0840695e0 msgr2=0x7ff08406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0ac0b4080 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff0840695e0 0x7ff08406baa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff094000f60 tx=0x7ff094006850 comp rx=0 tx=0).stop 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 msgr2=0x55ac104c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0ac0b4080 1 --2- 172.21.15.5:0/3398943114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac104c7dc0 0x55ac104c48a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff098014000 tx=0x7ff098003020 comp rx=0 tx=0).stop 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0a2987700 1 -- 172.21.15.5:0/3398943114 reap_dead start 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 shutdown_connections 2024-05-08T23:24:07.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.778+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 >> 172.21.15.5:0/3398943114 conn(0x55ac1047dba0 msgr2=0x55ac1054f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.779+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 shutdown_connections 2024-05-08T23:24:07.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.779+0000 7ff0ac0b4080 1 -- 172.21.15.5:0/3398943114 wait complete. 2024-05-08T23:24:07.784 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-05-08T23:24:07.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.877+0000 7fa259c67080 1 Processor -- start 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.878+0000 7fa259c67080 1 -- start start 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.878+0000 7fa259c67080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f7e8d80 0x56245f888ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.878+0000 7fa259c67080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 0x56245f88d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.878+0000 7fa259c67080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f893150 0x56245f88d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.878+0000 7fa259c67080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245f7698b0 con 0x56245f893150 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.878+0000 7fa259c67080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245f7651a0 con 0x56245f7e8d80 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.878+0000 7fa259c67080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56245f7006d0 con 0x56245f807480 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f893150 0x56245f88d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f893150 0x56245f88d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46804/0 (socket says 172.21.15.5:46804) 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 -- 172.21.15.5:0/3517193145 learned_addr learned my addr 172.21.15.5:0/3517193145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa24fd39700 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f7e8d80 0x56245f888ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa24f538700 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 0x56245f88d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 -- 172.21.15.5:0/3517193145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 msgr2=0x56245f88d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 0x56245f88d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 -- 172.21.15.5:0/3517193145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f7e8d80 msgr2=0x56245f888ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f7e8d80 0x56245f888ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 -- 172.21.15.5:0/3517193145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244005d20 con 0x56245f893150 2024-05-08T23:24:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa25053a700 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f893150 0x56245f88d6e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa248006f80 tx=0x7fa248001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e57e414bf1f0dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa24e536700 1 -- 172.21.15.5:0/3517193145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248012600 con 0x56245f893150 2024-05-08T23:24:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa24e536700 1 -- 172.21.15.5:0/3517193145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa248012760 con 0x56245f893150 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.879+0000 7fa24e536700 1 -- 172.21.15.5:0/3517193145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248012b10 con 0x56245f893150 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 -- 172.21.15.5:0/3517193145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f893150 msgr2=0x56245f88d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f893150 0x56245f88d6e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa248006f80 tx=0x7fa248001820 comp rx=0 tx=0).stop 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 -- 172.21.15.5:0/3517193145 shutdown_connections 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f893150 0x56245f88d6e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f7e8d80 0x56245f888ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 --2- 172.21.15.5:0/3517193145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 0x56245f88d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 -- 172.21.15.5:0/3517193145 >> 172.21.15.5:0/3517193145 conn(0x56245f7ead60 msgr2=0x56245f890cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 -- 172.21.15.5:0/3517193145 shutdown_connections 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.880+0000 7fa259c67080 1 -- 172.21.15.5:0/3517193145 wait complete. 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.881+0000 7fa259c67080 1 Processor -- start 2024-05-08T23:24:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.881+0000 7fa259c67080 1 -- start start 2024-05-08T23:24:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.881+0000 7fa259c67080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f7e8d80 0x56245f887550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.881+0000 7fa259c67080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 0x56245f88de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa259c67080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f893150 0x56245f886ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa259c67080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245f700370 con 0x56245f7e8d80 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa259c67080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245f6f7680 con 0x56245f893150 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa259c67080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56245f887a90 con 0x56245f807480 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f7e8d80 0x56245f887550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f7e8d80 0x56245f887550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46806/0 (socket says 172.21.15.5:46806) 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 -- 172.21.15.5:0/607890893 learned_addr learned my addr 172.21.15.5:0/607890893 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24f538700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 0x56245f88de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa25053a700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f893150 0x56245f886ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 -- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 msgr2=0x56245f88de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56245f807480 0x56245f88de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 -- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f893150 msgr2=0x56245f886ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56245f893150 0x56245f886ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 -- 172.21.15.5:0/607890893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56245f729a40 con 0x56245f7e8d80 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24fd39700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f7e8d80 0x56245f887550 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c500 tx=0x7fa24000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.882+0000 7fa24d534700 1 -- 172.21.15.5:0/607890893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240016030 con 0x56245f7e8d80 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.883+0000 7fa24d534700 1 -- 172.21.15.5:0/607890893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa240005070 con 0x56245f7e8d80 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.883+0000 7fa24d534700 1 -- 172.21.15.5:0/607890893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2400116c0 con 0x56245f7e8d80 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.883+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56245f702ed0 con 0x56245f7e8d80 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.883+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56245f702930 con 0x56245f7e8d80 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.884+0000 7fa24d534700 1 -- 172.21.15.5:0/607890893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa240019070 con 0x56245f7e8d80 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.885+0000 7fa24d534700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa23c0695e0 0x7fa23c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.885+0000 7fa24d534700 1 -- 172.21.15.5:0/607890893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa240017070 con 0x56245f7e8d80 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.885+0000 7fa259c67080 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x56245f88e3c0 0x56245f885860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.885+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 --> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] -- osd_op(unknown.0.0:1 6.12 6:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v8 -- 0x56245f887000 con 0x56245f88e3c0 2024-05-08T23:24:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.885+0000 7fa24f538700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa23c0695e0 0x7fa23c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.885+0000 7fa25053a700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x56245f88e3c0 0x56245f885860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.886+0000 7fa24f538700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa23c0695e0 0x7fa23c06baa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa244015040 tx=0x7fa24400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.886+0000 7fa25053a700 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x56245f88e3c0 0x56245f885860 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa25053a700 1 -- 172.21.15.5:0/607890893 <== osd.7 v2:172.21.15.49:6822/47408 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v48'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa248012400 con 0x56245f88e3c0 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x56245f88e3c0 msgr2=0x56245f885860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x56245f88e3c0 0x56245f885860 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa23c0695e0 msgr2=0x7fa23c06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa23c0695e0 0x7fa23c06baa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa244015040 tx=0x7fa24400f000 comp rx=0 tx=0).stop 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f7e8d80 msgr2=0x56245f887550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 --2- 172.21.15.5:0/607890893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56245f7e8d80 0x56245f887550 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c500 tx=0x7fa24000c810 comp rx=0 tx=0).stop 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa25053a700 1 -- 172.21.15.5:0/607890893 reap_dead start 2024-05-08T23:24:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 shutdown_connections 2024-05-08T23:24:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 >> 172.21.15.5:0/607890893 conn(0x56245f7ead60 msgr2=0x56245f888ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 shutdown_connections 2024-05-08T23:24:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.887+0000 7fa259c67080 1 -- 172.21.15.5:0/607890893 wait complete. 2024-05-08T23:24:07.892 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-05-08T23:24:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.943+0000 7f7f118d2080 1 Processor -- start 2024-05-08T23:24:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.943+0000 7f7f118d2080 1 -- start start 2024-05-08T23:24:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f118d2080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 0x565130910260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f118d2080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912230 0x5651309126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f118d2080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912bf0 0x5651309a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f118d2080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651307e88b0 con 0x565130912bf0 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f118d2080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651307e41a0 con 0x565130912230 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f118d2080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56513077f6d0 con 0x7f7ef0002260 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f081a5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912bf0 0x5651309a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f081a5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912bf0 0x5651309a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46826/0 (socket says 172.21.15.5:46826) 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f081a5700 1 -- 172.21.15.5:0/834298046 learned_addr learned my addr 172.21.15.5:0/834298046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f079a4700 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 0x565130910260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f071a3700 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912230 0x5651309126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f081a5700 1 -- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 msgr2=0x565130910260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f081a5700 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 0x565130910260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f081a5700 1 -- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912230 msgr2=0x5651309126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f081a5700 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912230 0x5651309126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f081a5700 1 -- 172.21.15.5:0/834298046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565130781ed0 con 0x565130912bf0 2024-05-08T23:24:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.944+0000 7f7f071a3700 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912230 0x5651309126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f081a5700 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912bf0 0x5651309a4d60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x56513086e8e0 tx=0x7f7f00004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cea22b56b71aabf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f061a1700 1 -- 172.21.15.5:0/834298046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000f450 con 0x565130912bf0 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f061a1700 1 -- 172.21.15.5:0/834298046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0000f5d0 con 0x565130912bf0 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f061a1700 1 -- 172.21.15.5:0/834298046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000f980 con 0x565130912bf0 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 -- 172.21.15.5:0/834298046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912bf0 msgr2=0x5651309a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912bf0 0x5651309a4d60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x56513086e8e0 tx=0x7f7f00004740 comp rx=0 tx=0).stop 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 -- 172.21.15.5:0/834298046 shutdown_connections 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912bf0 0x5651309a4d60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 0x565130910260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 --2- 172.21.15.5:0/834298046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912230 0x5651309126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 -- 172.21.15.5:0/834298046 >> 172.21.15.5:0/834298046 conn(0x56513083dba0 msgr2=0x56513083ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 -- 172.21.15.5:0/834298046 shutdown_connections 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.945+0000 7f7f118d2080 1 -- 172.21.15.5:0/834298046 wait complete. 2024-05-08T23:24:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.946+0000 7f7f118d2080 1 Processor -- start 2024-05-08T23:24:07.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.946+0000 7f7f118d2080 1 -- start start 2024-05-08T23:24:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f118d2080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912230 0x5651309aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f118d2080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912bf0 0x565130906270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f118d2080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 0x5651309067b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f118d2080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513077f370 con 0x565130912230 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f118d2080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x565130776680 con 0x565130912bf0 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f118d2080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56513086d870 con 0x7f7ef0002260 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f079a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912230 0x5651309aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f079a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912230 0x5651309aacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46834/0 (socket says 172.21.15.5:46834) 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f071a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912bf0 0x565130906270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f079a4700 1 -- 172.21.15.5:0/612787190 learned_addr learned my addr 172.21.15.5:0/612787190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f081a5700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 0x5651309067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f079a4700 1 -- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 msgr2=0x5651309067b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f079a4700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7ef0002260 0x5651309067b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f079a4700 1 -- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912bf0 msgr2=0x565130906270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f079a4700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565130912bf0 0x565130906270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.947+0000 7f7f079a4700 1 -- 172.21.15.5:0/612787190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0000e040 con 0x565130912230 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.948+0000 7f7f079a4700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912230 0x5651309aacb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000a860 tx=0x7f7ef8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.948+0000 7f7f0519f700 1 -- 172.21.15.5:0/612787190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800f040 con 0x565130912230 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.948+0000 7f7f0519f700 1 -- 172.21.15.5:0/612787190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef80054c0 con 0x565130912230 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.948+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565130781930 con 0x565130912230 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.948+0000 7f7f0519f700 1 -- 172.21.15.5:0/612787190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8027660 con 0x565130912230 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.948+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651309ab1f0 con 0x565130912230 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.949+0000 7f7f0519f700 1 -- 172.21.15.5:0/612787190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7ef8005040 con 0x565130912230 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.950+0000 7f7f0519f700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ef40695e0 0x7f7ef406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.950+0000 7f7f0519f700 1 -- 172.21.15.5:0/612787190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7ef80893c0 con 0x565130912230 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.950+0000 7f7f118d2080 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x565130999ee0 0x565130903980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.950+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 --> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] -- osd_op(unknown.0.0:1 6.5 6:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v8 -- 0x56513086d120 con 0x565130999ee0 2024-05-08T23:24:07.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.950+0000 7f7f071a3700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ef40695e0 0x7f7ef406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.950+0000 7f7f081a5700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x565130999ee0 0x565130903980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.951+0000 7f7f081a5700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x565130999ee0 0x565130903980 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.951+0000 7f7f071a3700 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ef40695e0 0x7f7ef406baa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0010e0 tx=0x7f7efc001460 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.952+0000 7f7f081a5700 1 -- 172.21.15.5:0/612787190 <== osd.7 v2:172.21.15.49:6822/47408 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v48'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7f0000f7c0 con 0x565130999ee0 2024-05-08T23:24:07.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.952+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x565130999ee0 msgr2=0x565130903980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.952+0000 7f7f118d2080 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x565130999ee0 0x565130903980 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.952+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ef40695e0 msgr2=0x7f7ef406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.952+0000 7f7f118d2080 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7ef40695e0 0x7f7ef406baa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0010e0 tx=0x7f7efc001460 comp rx=0 tx=0).stop 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.952+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912230 msgr2=0x5651309aacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.952+0000 7f7f118d2080 1 --2- 172.21.15.5:0/612787190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565130912230 0x5651309aacb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000a860 tx=0x7f7ef8005a00 comp rx=0 tx=0).stop 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.952+0000 7f7f081a5700 1 -- 172.21.15.5:0/612787190 reap_dead start 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.953+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 shutdown_connections 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.953+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 >> 172.21.15.5:0/612787190 conn(0x56513083dba0 msgr2=0x565130910260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.953+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 shutdown_connections 2024-05-08T23:24:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:07.953+0000 7f7f118d2080 1 -- 172.21.15.5:0/612787190 wait complete. 2024-05-08T23:24:07.957 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.006+0000 7f77d025d080 1 Processor -- start 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.006+0000 7f77d025d080 1 -- start start 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77d025d080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f4374fa70 0x563f437dff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77d025d080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77d025d080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f43752450 0x563f4374ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77d025d080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f436af1a0 con 0x563f43752070 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77d025d080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4364a6d0 con 0x563f43752450 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77d025d080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f4364a370 con 0x563f4374fa70 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46850/0 (socket says 172.21.15.5:46850) 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 -- 172.21.15.5:0/2198406440 learned_addr learned my addr 172.21.15.5:0/2198406440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c6b30700 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f43752450 0x563f4374ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c632f700 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f4374fa70 0x563f437dff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 -- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f4374fa70 msgr2=0x563f437dff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f4374fa70 0x563f437dff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 -- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f43752450 msgr2=0x563f4374ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f43752450 0x563f4374ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 -- 172.21.15.5:0/2198406440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8005de0 con 0x563f43752070 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c6b30700 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f43752450 0x563f4374ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c5b2e700 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437c3650 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00a770 tx=0x7f77bc00aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359508b2c088d701 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.007+0000 7f77c4b2c700 1 -- 172.21.15.5:0/2198406440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc00f070 con 0x563f43752070 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77c4b2c700 1 -- 172.21.15.5:0/2198406440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc0084c0 con 0x563f43752070 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77c4b2c700 1 -- 172.21.15.5:0/2198406440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc008850 con 0x563f43752070 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 -- 172.21.15.5:0/2198406440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 msgr2=0x563f437c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437c3650 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00a770 tx=0x7f77bc00aa80 comp rx=0 tx=0).stop 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 -- 172.21.15.5:0/2198406440 shutdown_connections 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f43752450 0x563f4374ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f4374fa70 0x563f437dff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 --2- 172.21.15.5:0/2198406440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437c3650 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 -- 172.21.15.5:0/2198406440 >> 172.21.15.5:0/2198406440 conn(0x563f43707ac0 msgr2=0x563f43736710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 -- 172.21.15.5:0/2198406440 shutdown_connections 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.008+0000 7f77d025d080 1 -- 172.21.15.5:0/2198406440 wait complete. 2024-05-08T23:24:08.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.009+0000 7f77d025d080 1 Processor -- start 2024-05-08T23:24:08.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.009+0000 7f77d025d080 1 -- start start 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77d025d080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f4374fa70 0x563f4374ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77d025d080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437dd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77d025d080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f43752450 0x563f437ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77d025d080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f43641680 con 0x563f43752070 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77d025d080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f437c1920 con 0x563f4374fa70 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77d025d080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f437de160 con 0x563f43752450 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77c5b2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437dd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77c5b2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437dd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46854/0 (socket says 172.21.15.5:46854) 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.010+0000 7f77c5b2e700 1 -- 172.21.15.5:0/4081561844 learned_addr learned my addr 172.21.15.5:0/4081561844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77c632f700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f4374fa70 0x563f4374ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77c6b30700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f43752450 0x563f437ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77c5b2e700 1 -- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f43752450 msgr2=0x563f437ddc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77c5b2e700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f43752450 0x563f437ddc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77c5b2e700 1 -- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f4374fa70 msgr2=0x563f4374ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77c5b2e700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f4374fa70 0x563f4374ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77c5b2e700 1 -- 172.21.15.5:0/4081561844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc00a420 con 0x563f43752070 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77c5b2e700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437dd6e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f77bc000c00 tx=0x7f77bc003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77b77fe700 1 -- 172.21.15.5:0/4081561844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc020850 con 0x563f43752070 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77b77fe700 1 -- 172.21.15.5:0/4081561844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc0209d0 con 0x563f43752070 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563f436598a0 con 0x563f43752070 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77b77fe700 1 -- 172.21.15.5:0/4081561844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc020d80 con 0x563f43752070 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.011+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f4364ced0 con 0x563f43752070 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.013+0000 7f77b77fe700 1 -- 172.21.15.5:0/4081561844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f77bc01c040 con 0x563f43752070 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.013+0000 7f77b77fe700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f77a80695e0 0x7f77a806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.013+0000 7f77b77fe700 1 -- 172.21.15.5:0/4081561844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f77bc089400 con 0x563f43752070 2024-05-08T23:24:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.013+0000 7f77c632f700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f77a80695e0 0x7f77a806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.013+0000 7f77d025d080 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x563f4374f370 0x563f4386f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.013+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- osd_op(unknown.0.0:1 6.9 6:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v8 -- 0x563f437def00 con 0x563f4374f370 2024-05-08T23:24:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.014+0000 7f77c6b30700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x563f4374f370 0x563f4386f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.014+0000 7f77c632f700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f77a80695e0 0x7f77a806baa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f77b800a4f0 tx=0x7f77b800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.014+0000 7f77c6b30700 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x563f4374f370 0x563f4386f2b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.015+0000 7f77c6b30700 1 -- 172.21.15.5:0/4081561844 <== osd.0 v2:172.21.15.5:6824/47134 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v48'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f77c0003040 con 0x563f4374f370 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.015+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x563f4374f370 msgr2=0x563f4386f2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.015+0000 7f77d025d080 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x563f4374f370 0x563f4386f2b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.015+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f77a80695e0 msgr2=0x7f77a806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.015+0000 7f77d025d080 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f77a80695e0 0x7f77a806baa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f77b800a4f0 tx=0x7f77b800f000 comp rx=0 tx=0).stop 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.015+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 msgr2=0x563f437dd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.015+0000 7f77d025d080 1 --2- 172.21.15.5:0/4081561844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f43752070 0x563f437dd6e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f77bc000c00 tx=0x7f77bc003020 comp rx=0 tx=0).stop 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.015+0000 7f77c6b30700 1 -- 172.21.15.5:0/4081561844 reap_dead start 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.016+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 shutdown_connections 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.016+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 >> 172.21.15.5:0/4081561844 conn(0x563f43707ac0 msgr2=0x563f43736710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.016+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 shutdown_connections 2024-05-08T23:24:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.016+0000 7f77d025d080 1 -- 172.21.15.5:0/4081561844 wait complete. 2024-05-08T23:24:08.020 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-05-08T23:24:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.069+0000 7ff516e54080 1 Processor -- start 2024-05-08T23:24:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.069+0000 7ff516e54080 1 -- start start 2024-05-08T23:24:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff516e54080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b231fa0 0x55772b2fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff516e54080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b27a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff516e54080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b309350 0x55772b392240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff516e54080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772b1db8b0 con 0x55772b278310 2024-05-08T23:24:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff516e54080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772b1d71a0 con 0x55772b231fa0 2024-05-08T23:24:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff516e54080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55772b1726d0 con 0x55772b309350 2024-05-08T23:24:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b27a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b27a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46882/0 (socket says 172.21.15.5:46882) 2024-05-08T23:24:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 -- 172.21.15.5:0/3660356044 learned_addr learned my addr 172.21.15.5:0/3660356044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50cf26700 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b231fa0 0x55772b2fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50d727700 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b309350 0x55772b392240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 -- 172.21.15.5:0/3660356044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b309350 msgr2=0x55772b392240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b309350 0x55772b392240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 -- 172.21.15.5:0/3660356044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b231fa0 msgr2=0x55772b2fddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b231fa0 0x55772b2fddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 -- 172.21.15.5:0/3660356044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50400a100 con 0x55772b278310 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50c725700 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b27a1b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff500001d50 tx=0x7ff50000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c27239fca8ca1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50b723700 1 -- 172.21.15.5:0/3660356044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500014070 con 0x55772b278310 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.070+0000 7ff50b723700 1 -- 172.21.15.5:0/3660356044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff500008030 con 0x55772b278310 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff50b723700 1 -- 172.21.15.5:0/3660356044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500001ea0 con 0x55772b278310 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 -- 172.21.15.5:0/3660356044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 msgr2=0x55772b27a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b27a1b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff500001d50 tx=0x7ff50000b4c0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 -- 172.21.15.5:0/3660356044 shutdown_connections 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b309350 0x55772b392240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b231fa0 0x55772b2fddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 --2- 172.21.15.5:0/3660356044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b27a1b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 -- 172.21.15.5:0/3660356044 >> 172.21.15.5:0/3660356044 conn(0x55772b230ba0 msgr2=0x55772b274a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 -- 172.21.15.5:0/3660356044 shutdown_connections 2024-05-08T23:24:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.071+0000 7ff516e54080 1 -- 172.21.15.5:0/3660356044 wait complete. 2024-05-08T23:24:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.072+0000 7ff516e54080 1 Processor -- start 2024-05-08T23:24:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.072+0000 7ff516e54080 1 -- start start 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff516e54080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b231fa0 0x55772b390410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff516e54080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b2f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff516e54080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b309350 0x55772b25feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff516e54080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772b172370 con 0x55772b278310 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff516e54080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772b169680 con 0x55772b309350 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff516e54080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55772b2fe440 con 0x55772b231fa0 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b2f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b2f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46894/0 (socket says 172.21.15.5:46894) 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 -- 172.21.15.5:0/3095651987 learned_addr learned my addr 172.21.15.5:0/3095651987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50d727700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b309350 0x55772b25feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50cf26700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b231fa0 0x55772b390410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 -- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b231fa0 msgr2=0x55772b390410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55772b231fa0 0x55772b390410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 -- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b309350 msgr2=0x55772b25feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55772b309350 0x55772b25feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 -- 172.21.15.5:0/3095651987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50000a040 con 0x55772b278310 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.073+0000 7ff50c725700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b2f4ba0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff500002000 tx=0x7ff500000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.074+0000 7ff50a721700 1 -- 172.21.15.5:0/3095651987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500014040 con 0x55772b278310 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.074+0000 7ff50a721700 1 -- 172.21.15.5:0/3095651987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50000fe20 con 0x55772b278310 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.074+0000 7ff50a721700 1 -- 172.21.15.5:0/3095651987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500001ea0 con 0x55772b278310 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.074+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55772b174ed0 con 0x55772b278310 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.074+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55772b174930 con 0x55772b278310 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.075+0000 7ff50a721700 1 -- 172.21.15.5:0/3095651987 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff50001b650 con 0x55772b278310 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.076+0000 7ff50a721700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff4f80695e0 0x7ff4f806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.076+0000 7ff50a721700 1 -- 172.21.15.5:0/3095651987 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff500089290 con 0x55772b278310 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.076+0000 7ff516e54080 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55772b391400 0x55772b2fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.076+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 --> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] -- osd_op(unknown.0.0:1 6.5 6:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v8 -- 0x55772b2fd0d0 con 0x55772b391400 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.076+0000 7ff50cf26700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff4f80695e0 0x7ff4f806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.076+0000 7ff50d727700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55772b391400 0x55772b2fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.076+0000 7ff50cf26700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff4f80695e0 0x7ff4f806baa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55772b2fea40 tx=0x7ff4fc0016f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.077+0000 7ff50d727700 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55772b391400 0x55772b2fcb90 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff50d727700 1 -- 172.21.15.5:0/3095651987 <== osd.7 v2:172.21.15.49:6822/47408 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v48'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff50400fba0 con 0x55772b391400 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55772b391400 msgr2=0x55772b2fcb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x55772b391400 0x55772b2fcb90 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff4f80695e0 msgr2=0x7ff4f806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff4f80695e0 0x7ff4f806baa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55772b2fea40 tx=0x7ff4fc0016f0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 msgr2=0x55772b2f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 --2- 172.21.15.5:0/3095651987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55772b278310 0x55772b2f4ba0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff500002000 tx=0x7ff500000bc0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff50d727700 1 -- 172.21.15.5:0/3095651987 reap_dead start 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 shutdown_connections 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 >> 172.21.15.5:0/3095651987 conn(0x55772b230ba0 msgr2=0x55772b274a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 shutdown_connections 2024-05-08T23:24:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.078+0000 7ff516e54080 1 -- 172.21.15.5:0/3095651987 wait complete. 2024-05-08T23:24:08.083 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-05-08T23:24:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.133+0000 7fd6bcf33080 1 Processor -- start 2024-05-08T23:24:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.133+0000 7fd6bcf33080 1 -- start start 2024-05-08T23:24:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.133+0000 7fd6bcf33080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 0x5561234461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.133+0000 7fd6bcf33080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561233d38e0 0x5561234576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.133+0000 7fd6bcf33080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556123462980 0x556123460b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.133+0000 7fd6bcf33080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561233348b0 con 0x5561233d38e0 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.133+0000 7fd6bcf33080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561233301a0 con 0x55612338d7b0 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.133+0000 7fd6bcf33080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561232cb6d0 con 0x556123462980 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561233d38e0 0x5561234576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561233d38e0 0x5561234576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46912/0 (socket says 172.21.15.5:46912) 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 -- 172.21.15.5:0/2372037637 learned_addr learned my addr 172.21.15.5:0/2372037637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b3005700 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 0x5561234461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b3806700 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556123462980 0x556123460b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 -- 172.21.15.5:0/2372037637 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556123462980 msgr2=0x556123460b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556123462980 0x556123460b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 -- 172.21.15.5:0/2372037637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 msgr2=0x5561234461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 0x5561234461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 -- 172.21.15.5:0/2372037637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c005d20 con 0x5561233d38e0 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b2804700 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561233d38e0 0x5561234576a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x5561233b8430 tx=0x7fd69c00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b658893803eab8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b1802700 1 -- 172.21.15.5:0/2372037637 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c008c00 con 0x5561233d38e0 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b1802700 1 -- 172.21.15.5:0/2372037637 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c008d60 con 0x5561233d38e0 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.134+0000 7fd6b1802700 1 -- 172.21.15.5:0/2372037637 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0136a0 con 0x5561233d38e0 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 -- 172.21.15.5:0/2372037637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561233d38e0 msgr2=0x5561234576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561233d38e0 0x5561234576a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x5561233b8430 tx=0x7fd69c00d000 comp rx=0 tx=0).stop 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 -- 172.21.15.5:0/2372037637 shutdown_connections 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x556123462980 0x556123460b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 0x5561234461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 --2- 172.21.15.5:0/2372037637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561233d38e0 0x5561234576a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 -- 172.21.15.5:0/2372037637 >> 172.21.15.5:0/2372037637 conn(0x5561233b7370 msgr2=0x5561233bb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 -- 172.21.15.5:0/2372037637 shutdown_connections 2024-05-08T23:24:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.135+0000 7fd6bcf33080 1 -- 172.21.15.5:0/2372037637 wait complete. 2024-05-08T23:24:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6bcf33080 1 Processor -- start 2024-05-08T23:24:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6bcf33080 1 -- start start 2024-05-08T23:24:08.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6bcf33080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 0x556123445120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6bcf33080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561233d38e0 0x5561233b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6bcf33080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556123462980 0x5561233b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6bcf33080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561232cb370 con 0x556123462980 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6bcf33080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561232c2680 con 0x55612338d7b0 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6bcf33080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561234599b0 con 0x5561233d38e0 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.136+0000 7fd6b3806700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556123462980 0x5561233b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3806700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556123462980 0x5561233b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46928/0 (socket says 172.21.15.5:46928) 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3806700 1 -- 172.21.15.5:0/1332206365 learned_addr learned my addr 172.21.15.5:0/1332206365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3005700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 0x556123445120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b2804700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561233d38e0 0x5561233b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3806700 1 -- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561233d38e0 msgr2=0x5561233b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3806700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561233d38e0 0x5561233b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3806700 1 -- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 msgr2=0x556123445120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3806700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55612338d7b0 0x556123445120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3806700 1 -- 172.21.15.5:0/1332206365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561232da8a0 con 0x556123462980 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b3806700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556123462980 0x5561233b7c90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac003f20 tx=0x7fd6ac001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b0800700 1 -- 172.21.15.5:0/1332206365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0020c0 con 0x556123462980 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b0800700 1 -- 172.21.15.5:0/1332206365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac011350 con 0x556123462980 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5561232cded0 con 0x556123462980 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6b0800700 1 -- 172.21.15.5:0/1332206365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0116e0 con 0x556123462980 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.137+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561232cd930 con 0x556123462980 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.139+0000 7fd6b0800700 1 -- 172.21.15.5:0/1332206365 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd6ac011840 con 0x556123462980 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.139+0000 7fd6b0800700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd6940695e0 0x7fd69406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.139+0000 7fd6b0800700 1 -- 172.21.15.5:0/1332206365 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd6ac087330 con 0x556123462980 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.139+0000 7fd6bcf33080 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x5561234489d0 0x556123502d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.139+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 --> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] -- osd_op(unknown.0.0:1 6.1d 6:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v8 -- 0x556123503290 con 0x5561234489d0 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.139+0000 7fd6b3005700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd6940695e0 0x7fd69406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.140+0000 7fd6b2804700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x5561234489d0 0x556123502d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.140+0000 7fd6b3005700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd6940695e0 0x7fd69406baa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000f60 tx=0x7fd6a4002ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.140+0000 7fd6b2804700 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x5561234489d0 0x556123502d50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6b2804700 1 -- 172.21.15.5:0/1332206365 <== osd.1 v2:172.21.15.5:6814/47133 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v48'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd69c008a80 con 0x5561234489d0 2024-05-08T23:24:08.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x5561234489d0 msgr2=0x556123502d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x5561234489d0 0x556123502d50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd6940695e0 msgr2=0x7fd69406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd6940695e0 0x7fd69406baa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000f60 tx=0x7fd6a4002ab0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556123462980 msgr2=0x5561233b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 --2- 172.21.15.5:0/1332206365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556123462980 0x5561233b7c90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac003f20 tx=0x7fd6ac001910 comp rx=0 tx=0).stop 2024-05-08T23:24:08.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6b3806700 1 -- 172.21.15.5:0/1332206365 reap_dead start 2024-05-08T23:24:08.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 shutdown_connections 2024-05-08T23:24:08.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 >> 172.21.15.5:0/1332206365 conn(0x5561233b7370 msgr2=0x5561233b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 shutdown_connections 2024-05-08T23:24:08.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.142+0000 7fd6bcf33080 1 -- 172.21.15.5:0/1332206365 wait complete. 2024-05-08T23:24:08.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /d1/d2/d3 recursive,force 2024-05-08T23:24:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.288+0000 7f3690e95700 1 Processor -- start 2024-05-08T23:24:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.288+0000 7f3690e95700 1 -- start start 2024-05-08T23:24:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.288+0000 7f3690e95700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f368c0d3d20 0x7f368c0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.288+0000 7f3690e95700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 0x7f368c0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.288+0000 7f3690e95700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0cf610 con 0x7f368c0cff30 2024-05-08T23:24:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.288+0000 7f3690e95700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c0d1fc0 con 0x7f368c0d3d20 2024-05-08T23:24:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3690e95700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0d22a0 con 0x7f368c0b3fc0 2024-05-08T23:24:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3681d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f368c0d3d20 0x7f368c0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3681d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f368c0d3d20 0x7f368c0b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34276/0 (socket says 172.21.15.5:34276) 2024-05-08T23:24:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3681d9b700 1 -- 172.21.15.5:0/2215861122 learned_addr learned my addr 172.21.15.5:0/2215861122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f368ad9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 0x7f368c0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3681d9b700 1 -- 172.21.15.5:0/2215861122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 msgr2=0x7f368c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3681d9b700 1 --2- 172.21.15.5:0/2215861122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 0x7f368c0b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3681d9b700 1 -- 172.21.15.5:0/2215861122 >> v1:172.21.15.49:6789/0 conn(0x7f368c0cff30 legacy=0x7f368c0d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3681d9b700 1 -- 172.21.15.5:0/2215861122 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0d2580 con 0x7f368c0d3d20 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.289+0000 7f3681d9b700 1 --2- 172.21.15.5:0/2215861122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0d3d20 0x7f368c0b3a80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f36740098f0 tx=0x7f3674009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b72135eaee54f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3689d9b700 1 -- 172.21.15.5:0/2215861122 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674011830 con 0x7f368c0d3d20 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3689d9b700 1 -- 172.21.15.5:0/2215861122 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674011990 con 0x7f368c0d3d20 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3689d9b700 1 -- 172.21.15.5:0/2215861122 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674011d20 con 0x7f368c0d3d20 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3690e95700 1 -- 172.21.15.5:0/2215861122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0d3d20 msgr2=0x7f368c0b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3690e95700 1 --2- 172.21.15.5:0/2215861122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0d3d20 0x7f368c0b3a80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f36740098f0 tx=0x7f3674009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3690e95700 1 -- 172.21.15.5:0/2215861122 shutdown_connections 2024-05-08T23:24:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3690e95700 1 --2- 172.21.15.5:0/2215861122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 0x7f368c0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3690e95700 1 --2- 172.21.15.5:0/2215861122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0d3d20 0x7f368c0b3a80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3690e95700 1 -- 172.21.15.5:0/2215861122 >> 172.21.15.5:0/2215861122 conn(0x7f368c08afe0 msgr2=0x7f368c0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3690e95700 1 -- 172.21.15.5:0/2215861122 shutdown_connections 2024-05-08T23:24:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.290+0000 7f3690e95700 1 -- 172.21.15.5:0/2215861122 wait complete. 2024-05-08T23:24:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.292+0000 7f3690e95700 1 Processor -- start 2024-05-08T23:24:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.292+0000 7f3690e95700 1 -- start start 2024-05-08T23:24:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f3690e95700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 0x7f368c1e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f3690e95700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0cff30 0x7f368c0b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f3690e95700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d3d20 0x7f368c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f3690e95700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0c8bf0 con 0x7f368c0d3d20 2024-05-08T23:24:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f3690e95700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0c85e0 con 0x7f368c0cff30 2024-05-08T23:24:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f3690e95700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0b9280 con 0x7f368c0b3fc0 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d3d20 0x7f368c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d3d20 0x7f368c0b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46938/0 (socket says 172.21.15.5:46938) 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368ad9d700 1 -- 172.21.15.5:0/2706615902 learned_addr learned my addr 172.21.15.5:0/2706615902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f3681d9b700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0cff30 0x7f368c0b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368a59c700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 0x7f368c1e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368ad9d700 1 -- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 msgr2=0x7f368c1e6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368ad9d700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 0x7f368c1e6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368ad9d700 1 -- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0cff30 msgr2=0x7f368c0b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368ad9d700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0cff30 0x7f368c0b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.293+0000 7f368ad9d700 1 -- 172.21.15.5:0/2706615902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674009550 con 0x7f368c0d3d20 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.294+0000 7f368ad9d700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d3d20 0x7f368c0b0ac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f368400efc0 tx=0x7f368400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.294+0000 7f36837fe700 1 -- 172.21.15.5:0/2706615902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368400cd40 con 0x7f368c0d3d20 2024-05-08T23:24:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.294+0000 7f36837fe700 1 -- 172.21.15.5:0/2706615902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368400f040 con 0x7f368c0d3d20 2024-05-08T23:24:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.294+0000 7f3690e95700 1 -- 172.21.15.5:0/2706615902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c0b71e0 con 0x7f368c0d3d20 2024-05-08T23:24:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.294+0000 7f36837fe700 1 -- 172.21.15.5:0/2706615902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36840146c0 con 0x7f368c0d3d20 2024-05-08T23:24:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.294+0000 7f3690e95700 1 -- 172.21.15.5:0/2706615902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c0b74c0 con 0x7f368c0d3d20 2024-05-08T23:24:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.297+0000 7f36837fe700 1 -- 172.21.15.5:0/2706615902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3684014820 con 0x7f368c0d3d20 2024-05-08T23:24:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.297+0000 7f36837fe700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f366c06acc0 0x7f366c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.297+0000 7f36837fe700 1 -- 172.21.15.5:0/2706615902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3684089ca0 con 0x7f368c0d3d20 2024-05-08T23:24:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.297+0000 7f368a59c700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f366c06acc0 0x7f366c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.298+0000 7f369d04fb80 1 Processor -- start 2024-05-08T23:24:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.298+0000 7f368a59c700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f366c06acc0 0x7f366c06d180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f367c000f60 tx=0x7f367c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.298+0000 7f369d04fb80 1 -- start start 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fbf830 0x558d10f981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 0x558d10f9d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10fc72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f369d04fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d10ef31e0 con 0x558d10fc6e60 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f369d04fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d10ef0ba0 con 0x558d10fbf830 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f369d04fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d10eec490 con 0x558d10f9a400 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f368ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10fc72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f368ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10fc72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46952/0 (socket says 172.21.15.5:46952) 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f368ad9d700 1 -- 172.21.15.5:0/970082115 learned_addr learned my addr 172.21.15.5:0/970082115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f368a59c700 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fbf830 0x558d10f981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f3681d9b700 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 0x558d10f9d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f368ad9d700 1 -- 172.21.15.5:0/970082115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 msgr2=0x558d10f9d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.299+0000 7f368ad9d700 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 0x558d10f9d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.300+0000 7f368ad9d700 1 -- 172.21.15.5:0/970082115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fbf830 msgr2=0x558d10f981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.300+0000 7f368ad9d700 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fbf830 0x558d10f981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.300+0000 7f368ad9d700 1 -- 172.21.15.5:0/970082115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d10d6d440 con 0x558d10fc6e60 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.300+0000 7f368ad9d700 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10fc72e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f368400c7a0 tx=0x7f368400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea6059170dad77d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.300+0000 7f366bfff700 1 -- 172.21.15.5:0/970082115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684058af0 con 0x558d10fc6e60 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.300+0000 7f366bfff700 1 -- 172.21.15.5:0/970082115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368405c920 con 0x558d10fc6e60 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.300+0000 7f366bfff700 1 -- 172.21.15.5:0/970082115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684060ee0 con 0x558d10fc6e60 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 -- 172.21.15.5:0/970082115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 msgr2=0x558d10fc72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10fc72e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f368400c7a0 tx=0x7f368400cea0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 -- 172.21.15.5:0/970082115 shutdown_connections 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10fc72e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fbf830 0x558d10f981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 --2- 172.21.15.5:0/970082115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 0x558d10f9d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 -- 172.21.15.5:0/970082115 >> 172.21.15.5:0/970082115 conn(0x558d10efa2a0 msgr2=0x558d10fe1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 -- 172.21.15.5:0/970082115 shutdown_connections 2024-05-08T23:24:08.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 -- 172.21.15.5:0/970082115 wait complete. 2024-05-08T23:24:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 -- start start 2024-05-08T23:24:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.301+0000 7f369d04fb80 1 Processor -- start 2024-05-08T23:24:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.302+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10f9a400 0x558d10fca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.302+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 0x558d10ff1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.302+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 0x558d10fbe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.302+0000 7f369d04fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d10e885f0 con 0x558d10f9a400 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.302+0000 7f369d04fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d10e88290 con 0x558d10fc6e60 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.302+0000 7f369d04fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d10d6f730 con 0x558d10fbf830 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f369d04fb80 10 client.0 _ll_register_callbacks cb 0x558d10e91bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10f9a400 0x558d10fca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10f9a400 0x558d10fca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46968/0 (socket says 172.21.15.5:46968) 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 -- 172.21.15.5:0/509694702 learned_addr learned my addr 172.21.15.5:0/509694702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368ad9d700 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 0x558d10fbe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f3681d9b700 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 0x558d10ff1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 -- 172.21.15.5:0/509694702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 msgr2=0x558d10ff1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 0x558d10ff1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 -- 172.21.15.5:0/509694702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 msgr2=0x558d10fbe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 0x558d10fbe4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 -- 172.21.15.5:0/509694702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c0b74c0 con 0x558d10f9a400 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f368a59c700 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10f9a400 0x558d10fca7c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f367c003c30 tx=0x7f367c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca9e839faf59d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f36697fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f36697fa700 1 -- 172.21.15.5:0/509694702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c025070 con 0x558d10f9a400 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f36697fa700 1 -- 172.21.15.5:0/509694702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f367c0054b0 con 0x558d10f9a400 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.303+0000 7f369d04fb80 1 -- 172.21.15.5:0/509694702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d10e45300 con 0x558d10f9a400 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f36697fa700 1 -- 172.21.15.5:0/509694702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f367c014ad0 con 0x558d10f9a400 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f369d04fb80 10 client.6153 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f369d04fb80 1 -- 172.21.15.5:0/509694702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x558d10d19bf0 con 0x558d10f9a400 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f36697fa700 1 -- 172.21.15.5:0/509694702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558d10e45300 con 0x558d10f9a400 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f36697fa700 1 -- 172.21.15.5:0/509694702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f367c028020 con 0x558d10f9a400 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f369d04fb80 10 client.6153 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f369d04fb80 10 client.6153 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-08T23:24:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f369d04fb80 20 client.6153 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f369d04fb80 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x558d10f96470 0x558d10fe4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f369d04fb80 4 client.6153 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f369d04fb80 1 -- 172.21.15.5:0/509694702 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d10f6b870 con 0x558d10f96470 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f3681d9b700 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x558d10f96470 0x558d10fe4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f3681d9b700 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x558d10f96470 0x558d10fe4660 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3932fbc79fcab80 server_cookie=a2a247938d8bdf26 in_seq=0 out_seq=0 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.304+0000 7f36697fa700 10 client.6153 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f36697fa700 1 -- 172.21.15.5:0/509694702 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d10f6b870 con 0x558d10f96470 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f36697fa700 10 client.6153 handle_command_reply: tid=0 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 client.6153 shutdown 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 -- 172.21.15.5:0/509694702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10f9a400 msgr2=0x558d10fca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10f9a400 0x558d10fca7c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f367c003c30 tx=0x7f367c001120 comp rx=0 tx=0).stop 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 -- 172.21.15.5:0/509694702 shutdown_connections 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 0x558d10fbe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10f9a400 0x558d10fca7c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x558d10f96470 0x558d10fe4660 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 --2- 172.21.15.5:0/509694702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 0x558d10ff1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 -- 172.21.15.5:0/509694702 >> 172.21.15.5:0/509694702 conn(0x558d10efa2a0 msgr2=0x558d10f9c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f36697fa700 0 client.6153 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 -- 172.21.15.5:0/509694702 shutdown_connections 2024-05-08T23:24:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.305+0000 7f369d04fb80 1 -- 172.21.15.5:0/509694702 wait complete. 2024-05-08T23:24:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.306+0000 7f369d04fb80 20 client.6153 trim_cache size 0 max 16384 2024-05-08T23:24:08.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.324+0000 7f369d04fb80 1 Processor -- start 2024-05-08T23:24:08.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.324+0000 7f369d04fb80 1 -- start start 2024-05-08T23:24:08.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 0x558d11004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fbf830 0x558d10fc9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 0x558d10fe32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f369d04fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d10fca2f0 con 0x558d10fbf830 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f369d04fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d11041870 con 0x558d10fc6e60 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f369d04fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d10f9c020 con 0x558d10f9a400 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fbf830 0x558d10fc9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fbf830 0x558d10fc9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46984/0 (socket says 172.21.15.5:46984) 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 -- 172.21.15.5:0/1557706358 learned_addr learned my addr 172.21.15.5:0/1557706358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f368a59c700 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 0x558d11004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f368ad9d700 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 0x558d10fe32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 -- 172.21.15.5:0/1557706358 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 msgr2=0x558d11004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 0x558d11004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 -- 172.21.15.5:0/1557706358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 msgr2=0x558d10fe32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 0x558d10fe32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 -- 172.21.15.5:0/1557706358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d10eb25f0 con 0x558d10fbf830 2024-05-08T23:24:08.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3681d9b700 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fbf830 0x558d10fc9340 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3674013000 tx=0x7f3674010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98849d2172bff82c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.325+0000 7f3680d99700 1 -- 172.21.15.5:0/1557706358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674021070 con 0x558d10fbf830 2024-05-08T23:24:08.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f3680d99700 1 -- 172.21.15.5:0/1557706358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674022040 con 0x558d10fbf830 2024-05-08T23:24:08.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f3680d99700 1 -- 172.21.15.5:0/1557706358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740044d0 con 0x558d10fbf830 2024-05-08T23:24:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 -- 172.21.15.5:0/1557706358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fbf830 msgr2=0x558d10fc9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fbf830 0x558d10fc9340 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3674013000 tx=0x7f3674010000 comp rx=0 tx=0).stop 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 -- 172.21.15.5:0/1557706358 shutdown_connections 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10fc6e60 0x558d10fe32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10f9a400 0x558d11004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 --2- 172.21.15.5:0/1557706358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fbf830 0x558d10fc9340 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 -- 172.21.15.5:0/1557706358 >> 172.21.15.5:0/1557706358 conn(0x558d10efa2a0 msgr2=0x558d10f93270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 -- 172.21.15.5:0/1557706358 shutdown_connections 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.326+0000 7f369d04fb80 1 -- 172.21.15.5:0/1557706358 wait complete. 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.327+0000 7f369d04fb80 1 -- start start 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.327+0000 7f369d04fb80 1 Processor -- start 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10f9a400 0x558d1103deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 0x558d10f9c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f369d04fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10f9caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f369d04fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d11040e60 con 0x558d10fc6e60 2024-05-08T23:24:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f369d04fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d10f9b550 con 0x558d10f9a400 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f369d04fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d1103e3f0 con 0x558d10fbf830 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f369d04fb80 10 client.0 _ll_register_callbacks cb 0x558d10e91bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10f9caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10f9caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46994/0 (socket says 172.21.15.5:46994) 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 -- 172.21.15.5:0/2695147401 learned_addr learned my addr 172.21.15.5:0/2695147401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368a59c700 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10f9a400 0x558d1103deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f3681d9b700 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 0x558d10f9c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 -- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 msgr2=0x558d10f9c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 0x558d10f9c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 -- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10f9a400 msgr2=0x558d1103deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10f9a400 0x558d1103deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 -- 172.21.15.5:0/2695147401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c0d2580 con 0x558d10fc6e60 2024-05-08T23:24:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368a59c700 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10f9a400 0x558d1103deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f368ad9d700 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10f9caf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f368400d010 tx=0x7f368401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364a8d57d7e5359 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.328+0000 7f36697fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f36697fa700 1 -- 172.21.15.5:0/2695147401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684075070 con 0x558d10fc6e60 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f36697fa700 1 -- 172.21.15.5:0/2695147401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3684060ee0 con 0x558d10fc6e60 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 1 -- 172.21.15.5:0/2695147401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f368405c920 con 0x558d10fc6e60 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f36697fa700 1 -- 172.21.15.5:0/2695147401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3684058af0 con 0x558d10fc6e60 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 10 client.6159 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 1 -- 172.21.15.5:0/2695147401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x558d10d71d60 con 0x558d10fc6e60 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f36697fa700 1 -- 172.21.15.5:0/2695147401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368405c920 con 0x558d10fc6e60 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f36697fa700 1 -- 172.21.15.5:0/2695147401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f3684077310 con 0x558d10fc6e60 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 10 client.6159 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 10 client.6159 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 20 client.6159 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x558d10fe1870 0x558d110439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 4 client.6159 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f369d04fb80 1 -- 172.21.15.5:0/2695147401 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) v1 -- 0x558d10ee4f50 con 0x558d10fe1870 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.329+0000 7f368a59c700 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x558d10fe1870 0x558d110439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.330+0000 7f36697fa700 10 client.6159 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.330+0000 7f368a59c700 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x558d10fe1870 0x558d110439c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13adcc629fd3d1cf server_cookie=1bd80ebd1bff997e in_seq=0 out_seq=0 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.330+0000 7f36697fa700 1 -- 172.21.15.5:0/2695147401 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558d10ee4f50 con 0x558d10fe1870 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.330+0000 7f36697fa700 10 client.6159 handle_command_reply: tid=0 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.330+0000 7f369d04fb80 1 client.6159 shutdown 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.330+0000 7f369d04fb80 1 -- 172.21.15.5:0/2695147401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 msgr2=0x558d10f9caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.330+0000 7f369d04fb80 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10f9caf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f368400d010 tx=0x7f368401f040 comp rx=0 tx=0).stop 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 1 -- 172.21.15.5:0/2695147401 shutdown_connections 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d10fc6e60 0x558d10f9caf0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d10f9a400 0x558d1103deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x558d10fe1870 0x558d110439c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 1 --2- 172.21.15.5:0/2695147401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d10fbf830 0x558d10f9c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 1 -- 172.21.15.5:0/2695147401 >> 172.21.15.5:0/2695147401 conn(0x558d10efa2a0 msgr2=0x558d10fe32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f36697fa700 0 client.6159 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 1 -- 172.21.15.5:0/2695147401 shutdown_connections 2024-05-08T23:24:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 1 -- 172.21.15.5:0/2695147401 wait complete. 2024-05-08T23:24:08.339 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f5d4cadb700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1) n(v3 rc2024-05-08T23:23:51.671503+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={5255=pAsLsXsFsx/pAsLsXsFsx@5},l=5255 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56331a5da580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.671503+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-05-08T23:23:51.671503+0000 6=0+6)","error_str":""},"return_code":-61} 2024-05-08T23:24:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f369d04fb80 20 client.6159 trim_cache size 0 max 16384 2024-05-08T23:24:08.340 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-08T23:24:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "cc35a3fa-d857-42f3-b1d7-b82b25291fa7", 2024-05-08T23:24:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-08T23:24:08.340 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:08.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 -- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f366c06acc0 msgr2=0x7f366c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f366c06acc0 0x7f366c06d180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f367c000f60 tx=0x7f367c009040 comp rx=0 tx=0).stop 2024-05-08T23:24:08.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 -- 172.21.15.5:0/2706615902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d3d20 msgr2=0x7f368c0b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d3d20 0x7f368c0b0ac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f368400efc0 tx=0x7f368400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 -- 172.21.15.5:0/2706615902 shutdown_connections 2024-05-08T23:24:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f366c06acc0 0x7f366c06d180 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d3d20 0x7f368c0b0ac0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f368c0b3fc0 0x7f368c1e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.331+0000 7f3690e95700 1 --2- 172.21.15.5:0/2706615902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c0cff30 0x7f368c0b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.332+0000 7f3690e95700 1 -- 172.21.15.5:0/2706615902 >> 172.21.15.5:0/2706615902 conn(0x7f368c08afe0 msgr2=0x7f368c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.332+0000 7f3690e95700 1 -- 172.21.15.5:0/2706615902 shutdown_connections 2024-05-08T23:24:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.332+0000 7f3690e95700 1 -- 172.21.15.5:0/2706615902 wait complete. 2024-05-08T23:24:08.342 INFO:tasks.ceph.mds.b.smithi049.stderr:2024-05-08T23:24:08.331+0000 7ff4ccee9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-05-08T23:23:51.414509+0000 1=0+1) n(v2 rc2024-05-08T23:23:51.671503+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={5255=pAsLsXsFsx/pAsLsXsFsx@4},l=5255 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55e7715ad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.414509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.671503+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.414509+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-05-08T23:23:51.671503+0000 4=0+4)","error_str":""},"return_code":-61} 2024-05-08T23:24:08.343 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:24:08.332+0000 7f5d52ae7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1) n(v4 rc2024-05-08T23:23:51.671503+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={5255=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x56331a5dab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.671503+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-05-08T23:23:51.671503+0000 5=0+5)","error_str":""},"return_code":-61} 2024-05-08T23:24:08.343 INFO:tasks.ceph.mds.d.smithi049.stderr:2024-05-08T23:24:08.332+0000 7f15a3868700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1) n(v0 rc2024-05-08T23:23:51.415509+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={5255=pAsLsXsFsx/pAsLsXsFsx@4},l=5255 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5618a3dfb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.415509+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-05-08T23:23:51.415509+0000 2=0+2)","error_str":""},"return_code":-61} 2024-05-08T23:24:08.343 INFO:tasks.ceph.mds.d.smithi049.stderr:2024-05-08T23:24:08.332+0000 7f15a3868700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={5255=pAsLsXsFsx/AsLsXsFsx@2},l=5255 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5618a3dfb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-05-08T23:24:08.343 INFO:tasks.ceph.mds.b.smithi049.stderr:2024-05-08T23:24:08.333+0000 7ff4d2ef5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1) n(v1 rc2024-05-08T23:23:51.671503+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={5255=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55e7715ad700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.415509+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-05-08T23:23:51.671503+0000 3=0+3)","error_str":""},"return_code":-61} 2024-05-08T23:24:08.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub abort 2024-05-08T23:24:08.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.484+0000 7f4b21412700 1 Processor -- start 2024-05-08T23:24:08.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.484+0000 7f4b21412700 1 -- start start 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b21412700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b1c0d0a60 0x7f4b1c0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b21412700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c15fbb0 0x7f4b1c0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b21412700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c0af6e0 con 0x7f4b1c145040 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b21412700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1c0b6460 con 0x7f4b1c0d0a60 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b21412700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0aeb50 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b1a7fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b1c0d0a60 0x7f4b1c0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b1a7fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b1c0d0a60 0x7f4b1c0b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47004/0 (socket says 172.21.15.5:47004) 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/4137879558 learned_addr learned my addr 172.21.15.5:0/4137879558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.485+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/4137879558 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c15fbb0 0x7f4b1c0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/4137879558 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c15fbb0 msgr2=0x7f4b1c0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/4137879558 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c15fbb0 0x7f4b1c0afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/4137879558 >> v1:172.21.15.49:6789/0 conn(0x7f4b1c145040 legacy=0x7f4b1c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/4137879558 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0aedd0 con 0x7f4b1c0d0a60 2024-05-08T23:24:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/4137879558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c0d0a60 0x7f4b1c0b5f20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0098f0 tx=0x7f4b0c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b269417f6f06a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b19ffb700 1 -- 172.21.15.5:0/4137879558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011830 con 0x7f4b1c0d0a60 2024-05-08T23:24:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b19ffb700 1 -- 172.21.15.5:0/4137879558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c011990 con 0x7f4b1c0d0a60 2024-05-08T23:24:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b19ffb700 1 -- 172.21.15.5:0/4137879558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011d20 con 0x7f4b1c0d0a60 2024-05-08T23:24:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b21412700 1 -- 172.21.15.5:0/4137879558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c0d0a60 msgr2=0x7f4b1c0b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.486+0000 7f4b21412700 1 --2- 172.21.15.5:0/4137879558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c0d0a60 0x7f4b1c0b5f20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0098f0 tx=0x7f4b0c009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.487+0000 7f4b21412700 1 -- 172.21.15.5:0/4137879558 shutdown_connections 2024-05-08T23:24:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.487+0000 7f4b21412700 1 --2- 172.21.15.5:0/4137879558 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c15fbb0 0x7f4b1c0afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.487+0000 7f4b21412700 1 --2- 172.21.15.5:0/4137879558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c0d0a60 0x7f4b1c0b5f20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.487+0000 7f4b21412700 1 -- 172.21.15.5:0/4137879558 >> 172.21.15.5:0/4137879558 conn(0x7f4b1c099a50 msgr2=0x7f4b1c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.487+0000 7f4b21412700 1 -- 172.21.15.5:0/4137879558 shutdown_connections 2024-05-08T23:24:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.487+0000 7f4b21412700 1 -- 172.21.15.5:0/4137879558 wait complete. 2024-05-08T23:24:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.488+0000 7f4b21412700 1 Processor -- start 2024-05-08T23:24:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.489+0000 7f4b21412700 1 -- start start 2024-05-08T23:24:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.489+0000 7f4b21412700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c0d0a60 0x7f4b1c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b21412700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c145040 0x7f4b1c13aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b21412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c15fbb0 0x7f4b1c1f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b21412700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c1f1770 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b21412700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c1f1a10 con 0x7f4b1c145040 2024-05-08T23:24:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b21412700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c1f1cf0 con 0x7f4b1c0d0a60 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c15fbb0 0x7f4b1c1f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c145040 0x7f4b1c13aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1affd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c0d0a60 0x7f4b1c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c15fbb0 0x7f4b1c1f1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47014/0 (socket says 172.21.15.5:47014) 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/676829164 learned_addr learned my addr 172.21.15.5:0/676829164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c0d0a60 msgr2=0x7f4b1c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c0d0a60 0x7f4b1c15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c145040 msgr2=0x7f4b1c13aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c145040 0x7f4b1c13aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/676829164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c009550 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c15fbb0 0x7f4b1c1f1230 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400efc0 tx=0x7f4b1400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b137fe700 1 -- 172.21.15.5:0/676829164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400cad0 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b137fe700 1 -- 172.21.15.5:0/676829164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1400f040 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.490+0000 7f4b21412700 1 -- 172.21.15.5:0/676829164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c1f1fd0 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.491+0000 7f4b137fe700 1 -- 172.21.15.5:0/676829164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400ce70 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.491+0000 7f4b21412700 1 -- 172.21.15.5:0/676829164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c13c470 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.493+0000 7f4b137fe700 1 -- 172.21.15.5:0/676829164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4b14014480 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.493+0000 7f4b137fe700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4afc06ac90 0x7f4afc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.493+0000 7f4b137fe700 1 -- 172.21.15.5:0/676829164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4b14089aa0 con 0x7f4b1c15fbb0 2024-05-08T23:24:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.494+0000 7f4b1affd700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4afc06ac90 0x7f4afc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.494+0000 7f4b1affd700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4afc06ac90 0x7f4afc06d150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4b04007fd0 tx=0x7f4b04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.495+0000 7f4b2d5ccb80 1 Processor -- start 2024-05-08T23:24:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.495+0000 7f4b2d5ccb80 1 -- start start 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e067c790 0x5639e0655100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0657360 0x5639e065a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0683dc0 0x5639e0684240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e05b0140 con 0x5639e0657360 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e05adb00 con 0x5639e067c790 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639e05a93f0 con 0x5639e0683dc0 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0657360 0x5639e065a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e067c790 0x5639e0655100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0657360 0x5639e065a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47030/0 (socket says 172.21.15.5:47030) 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0683dc0 0x5639e0684240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/3371873877 learned_addr learned my addr 172.21.15.5:0/3371873877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1affd700 1 -- 172.21.15.5:0/3371873877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0683dc0 msgr2=0x5639e0684240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1affd700 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0683dc0 0x5639e0684240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1affd700 1 -- 172.21.15.5:0/3371873877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0657360 msgr2=0x5639e065a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1affd700 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0657360 0x5639e065a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1affd700 1 -- 172.21.15.5:0/3371873877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639e042a3a0 con 0x5639e067c790 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0683dc0 0x5639e0684240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.496+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0657360 0x5639e065a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.497+0000 7f4b1affd700 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e067c790 0x5639e0655100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400f6c0 tx=0x7f4b04012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c41f07009198112 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.497+0000 7f4b10ff9700 1 -- 172.21.15.5:0/3371873877 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0400fd90 con 0x5639e067c790 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.497+0000 7f4b10ff9700 1 -- 172.21.15.5:0/3371873877 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b04014450 con 0x5639e067c790 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.497+0000 7f4b10ff9700 1 -- 172.21.15.5:0/3371873877 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04014800 con 0x5639e067c790 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3371873877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e067c790 msgr2=0x5639e0655100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e067c790 0x5639e0655100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400f6c0 tx=0x7f4b04012000 comp rx=0 tx=0).stop 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3371873877 shutdown_connections 2024-05-08T23:24:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0683dc0 0x5639e0684240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e067c790 0x5639e0655100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/3371873877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0657360 0x5639e065a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3371873877 >> 172.21.15.5:0/3371873877 conn(0x5639e05b7200 msgr2=0x5639e069edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3371873877 shutdown_connections 2024-05-08T23:24:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3371873877 wait complete. 2024-05-08T23:24:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.498+0000 7f4b2d5ccb80 1 -- start start 2024-05-08T23:24:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.499+0000 7f4b2d5ccb80 1 Processor -- start 2024-05-08T23:24:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.499+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.499+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e06a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.499+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e067a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.499+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e0545550 con 0x5639e0683dc0 2024-05-08T23:24:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.499+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e05451f0 con 0x5639e0657360 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.499+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639e042c690 con 0x5639e067c790 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.499+0000 7f4b2d5ccb80 10 client.0 _ll_register_callbacks cb 0x5639e054eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e067a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e067a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47032/0 (socket says 172.21.15.5:47032) 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/2058027283 learned_addr learned my addr 172.21.15.5:0/2058027283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1affd700 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e06a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 msgr2=0x5639e06a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e06a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 msgr2=0x5639e067bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/2058027283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c13c470 con 0x5639e0683dc0 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e06a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e067a4a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400ef90 tx=0x7f4b14058e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb840e01c759e01d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4af9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4af9ffb700 1 -- 172.21.15.5:0/2058027283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14060860 con 0x5639e0683dc0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4af9ffb700 1 -- 172.21.15.5:0/2058027283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4b140609c0 con 0x5639e0683dc0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/2058027283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639e0502260 con 0x5639e0683dc0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4af9ffb700 1 -- 172.21.15.5:0/2058027283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b140546f0 con 0x5639e0683dc0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b2d5ccb80 10 client.6171 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/2058027283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x7f4b1c0af6e0 con 0x5639e0683dc0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.500+0000 7f4af9ffb700 1 -- 172.21.15.5:0/2058027283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639e0502260 con 0x5639e0683dc0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4af9ffb700 1 -- 172.21.15.5:0/2058027283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f4b14054850 con 0x5639e0683dc0 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4b2d5ccb80 10 client.6171 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4b2d5ccb80 10 client.6171 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4b2d5ccb80 20 client.6171 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x5639e067eae0 0x5639e067eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4b2d5ccb80 4 client.6171 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/2058027283 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639e06287d0 con 0x5639e067eae0 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4b1affd700 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x5639e067eae0 0x5639e067eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4af9ffb700 10 client.6171 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.501+0000 7f4b1affd700 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x5639e067eae0 0x5639e067eef0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3a8b21fc1eda952 server_cookie=a2e8269662dc9da6 in_seq=0 out_seq=0 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4af9ffb700 1 -- 172.21.15.5:0/2058027283 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639e06287d0 con 0x5639e067eae0 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4af9ffb700 10 client.6171 handle_command_reply: tid=0 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 client.6171 shutdown 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/2058027283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 msgr2=0x5639e067a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e067a4a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400ef90 tx=0x7f4b14058e40 comp rx=0 tx=0).stop 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/2058027283 shutdown_connections 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e067a4a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x5639e067eae0 0x5639e067eef0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/2058027283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e06a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4af9ffb700 0 client.6171 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/2058027283 >> 172.21.15.5:0/2058027283 conn(0x5639e05b7200 msgr2=0x5639e06592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/2058027283 shutdown_connections 2024-05-08T23:24:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/2058027283 wait complete. 2024-05-08T23:24:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.502+0000 7f4b2d5ccb80 20 client.6171 trim_cache size 0 max 16384 2024-05-08T23:24:08.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.520+0000 7f4b2d5ccb80 1 Processor -- start 2024-05-08T23:24:08.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.520+0000 7f4b2d5ccb80 1 -- start start 2024-05-08T23:24:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e067b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e0686980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e06b1560 con 0x5639e0683dc0 2024-05-08T23:24:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e0652cd0 con 0x5639e0657360 2024-05-08T23:24:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639e0653550 con 0x5639e067c790 2024-05-08T23:24:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e0686980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e0686980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47036/0 (socket says 172.21.15.5:47036) 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/3605430320 learned_addr learned my addr 172.21.15.5:0/3605430320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1affd700 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e067b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 msgr2=0x5639e067b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e067b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 msgr2=0x5639e067aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 -- 172.21.15.5:0/3605430320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639e03d6b50 con 0x5639e0683dc0 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1affd700 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.521+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e0686980 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4b14055320 tx=0x7f4b1405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184efe73730b0279 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b117fa700 1 -- 172.21.15.5:0/3605430320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14058440 con 0x5639e0683dc0 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b117fa700 1 -- 172.21.15.5:0/3605430320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14081e90 con 0x5639e0683dc0 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b117fa700 1 -- 172.21.15.5:0/3605430320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14060a60 con 0x5639e0683dc0 2024-05-08T23:24:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3605430320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 msgr2=0x5639e0686980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e0686980 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4b14055320 tx=0x7f4b1405b000 comp rx=0 tx=0).stop 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3605430320 shutdown_connections 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e0683dc0 0x5639e0686980 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0657360 0x5639e067aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/3605430320 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e067c790 0x5639e067b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3605430320 >> 172.21.15.5:0/3605430320 conn(0x5639e05b7200 msgr2=0x5639e06592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3605430320 shutdown_connections 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.522+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/3605430320 wait complete. 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.523+0000 7f4b2d5ccb80 1 -- start start 2024-05-08T23:24:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.523+0000 7f4b2d5ccb80 1 Processor -- start 2024-05-08T23:24:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0657360 0x5639e07080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e067c790 0x5639e0686980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b2d5ccb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0683dc0 0x5639e0686ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e06a2910 con 0x5639e067c790 2024-05-08T23:24:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e0699ea0 con 0x5639e0683dc0 2024-05-08T23:24:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b2d5ccb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639e0684900 con 0x5639e0657360 2024-05-08T23:24:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b2d5ccb80 10 client.0 _ll_register_callbacks cb 0x5639e054eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e067c790 0x5639e0686980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e067c790 0x5639e0686980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47042/0 (socket says 172.21.15.5:47042) 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/561055698 learned_addr learned my addr 172.21.15.5:0/561055698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1affd700 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0657360 0x5639e07080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1b7fe700 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0683dc0 0x5639e0686ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/561055698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0657360 msgr2=0x5639e07080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0657360 0x5639e07080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/561055698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0683dc0 msgr2=0x5639e0686ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0683dc0 0x5639e0686ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 -- 172.21.15.5:0/561055698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c0af6e0 con 0x5639e067c790 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4b1a7fc700 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e067c790 0x5639e0686980 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c013000 tx=0x7f4b0c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be521b3487aaccbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.524+0000 7f4af9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4af9ffb700 1 -- 172.21.15.5:0/561055698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c022040 con 0x5639e067c790 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4af9ffb700 1 -- 172.21.15.5:0/561055698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4b0c004450 con 0x5639e067c790 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/561055698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b04014450 con 0x5639e067c790 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4af9ffb700 1 -- 172.21.15.5:0/561055698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b0c0093d0 con 0x5639e067c790 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 10 client.6177 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/561055698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x5639e056f550 con 0x5639e067c790 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4af9ffb700 1 -- 172.21.15.5:0/561055698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b04014450 con 0x5639e067c790 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4af9ffb700 1 -- 172.21.15.5:0/561055698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f4b0c025030 con 0x5639e067c790 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 10 client.6177 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 10 client.6177 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 20 client.6177 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x5639e0656350 0x5639e0698c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 4 client.6177 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/561055698 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5639e05a1eb0 con 0x5639e0656350 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.525+0000 7f4b1affd700 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x5639e0656350 0x5639e0698c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b1affd700 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x5639e0656350 0x5639e0698c00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4998e939058ce791 server_cookie=e3e6115a9b660f90 in_seq=0 out_seq=0 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4af9ffb700 10 client.6177 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4af9ffb700 1 -- 172.21.15.5:0/561055698 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5639e05a1eb0 con 0x5639e0656350 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4af9ffb700 10 client.6177 handle_command_reply: tid=0 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 client.6177 shutdown 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/561055698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e067c790 msgr2=0x5639e0686980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e067c790 0x5639e0686980 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c013000 tx=0x7f4b0c00e000 comp rx=0 tx=0).stop 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/561055698 shutdown_connections 2024-05-08T23:24:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639e0683dc0 0x5639e0686ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x5639e0656350 0x5639e0698c00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5639e0657360 0x5639e07080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 --2- 172.21.15.5:0/561055698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639e067c790 0x5639e0686980 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4af9ffb700 0 client.6177 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/561055698 >> 172.21.15.5:0/561055698 conn(0x5639e05b7200 msgr2=0x5639e067e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/561055698 shutdown_connections 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 1 -- 172.21.15.5:0/561055698 wait complete. 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.526+0000 7f4b2d5ccb80 20 client.6177 trim_cache size 0 max 16384 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:24:08.533 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 -- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4afc06ac90 msgr2=0x7f4afc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4afc06ac90 0x7f4afc06d150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4b04007fd0 tx=0x7f4b04009040 comp rx=0 tx=0).stop 2024-05-08T23:24:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 -- 172.21.15.5:0/676829164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c15fbb0 msgr2=0x7f4b1c1f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c15fbb0 0x7f4b1c1f1230 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400efc0 tx=0x7f4b1400c0c0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 -- 172.21.15.5:0/676829164 shutdown_connections 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4afc06ac90 0x7f4afc06d150 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c15fbb0 0x7f4b1c1f1230 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b1c0d0a60 0x7f4b1c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 --2- 172.21.15.5:0/676829164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c145040 0x7f4b1c13aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 -- 172.21.15.5:0/676829164 >> 172.21.15.5:0/676829164 conn(0x7f4b1c099a50 msgr2=0x7f4b1c15ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 -- 172.21.15.5:0/676829164 shutdown_connections 2024-05-08T23:24:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.527+0000 7f4b21412700 1 -- 172.21.15.5:0/676829164 wait complete. 2024-05-08T23:24:08.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 scrub status 2024-05-08T23:24:08.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.681+0000 7fab2026c700 1 Processor -- start 2024-05-08T23:24:08.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.681+0000 7fab2026c700 1 -- start start 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab2026c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab181436f0 0x7fab180cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab2026c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180ce660 con 0x7fab180cf700 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab2026c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180ce940 con 0x7fab1815de80 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab2026c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180cec20 con 0x7fab181436f0 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab1d807700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fab180cf700 0x7fab1815d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49412/0 (socket says 172.21.15.5:49412) 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab1d807700 1 -- 172.21.15.5:0/2381874221 learned_addr learned my addr 172.21.15.5:0/2381874221 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab1e008700 1 --2- 172.21.15.5:0/2381874221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab181436f0 0x7fab180cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab1d006700 1 -- 172.21.15.5:0/2381874221 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469040541 0 0) 0x7fab180ce660 con 0x7fab180cf700 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.682+0000 7fab1d006700 1 -- 172.21.15.5:0/2381874221 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab040035b0 con 0x7fab180cf700 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1d006700 1 -- 172.21.15.5:0/2381874221 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267863650 0 0) 0x7fab180ce940 con 0x7fab1815de80 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1d006700 1 -- 172.21.15.5:0/2381874221 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab180ce660 con 0x7fab1815de80 2024-05-08T23:24:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1e008700 1 -- 172.21.15.5:0/2381874221 >> v1:172.21.15.49:6789/0 conn(0x7fab1815de80 legacy=0x7fab1806d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1e008700 1 -- 172.21.15.5:0/2381874221 >> v1:172.21.15.5:6789/0 conn(0x7fab180cf700 legacy=0x7fab1815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1d006700 1 -- 172.21.15.5:0/2381874221 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1776716674 0 0) 0x7fab040035b0 con 0x7fab180cf700 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1e008700 1 -- 172.21.15.5:0/2381874221 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180cef00 con 0x7fab181436f0 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1e008700 1 --2- 172.21.15.5:0/2381874221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab181436f0 0x7fab180cdff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009870 tx=0x7fab0c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55f3fcef32f70a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1d006700 1 -- 172.21.15.5:0/2381874221 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14001670 con 0x7fab181436f0 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1d006700 1 -- 172.21.15.5:0/2381874221 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c004030 con 0x7fab181436f0 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.683+0000 7fab1d006700 1 -- 172.21.15.5:0/2381874221 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0043c0 con 0x7fab181436f0 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.684+0000 7fab2026c700 1 -- 172.21.15.5:0/2381874221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab181436f0 msgr2=0x7fab180cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.684+0000 7fab2026c700 1 --2- 172.21.15.5:0/2381874221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab181436f0 0x7fab180cdff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009870 tx=0x7fab0c009b80 comp rx=0 tx=0).stop 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.684+0000 7fab2026c700 1 -- 172.21.15.5:0/2381874221 shutdown_connections 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.684+0000 7fab2026c700 1 --2- 172.21.15.5:0/2381874221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab181436f0 0x7fab180cdff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.684+0000 7fab2026c700 1 -- 172.21.15.5:0/2381874221 >> 172.21.15.5:0/2381874221 conn(0x7fab180722d0 msgr2=0x7fab1806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.684+0000 7fab2026c700 1 -- 172.21.15.5:0/2381874221 shutdown_connections 2024-05-08T23:24:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.684+0000 7fab2026c700 1 -- 172.21.15.5:0/2381874221 wait complete. 2024-05-08T23:24:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.685+0000 7fab2026c700 1 Processor -- start 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.685+0000 7fab2026c700 1 -- start start 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab2026c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab180cf700 0x7fab1813d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab2026c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab181436f0 0x7fab1813b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab2026c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab1815de80 0x7fab1815c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab2026c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1815a330 con 0x7fab181436f0 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab2026c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1813b810 con 0x7fab180cf700 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab2026c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1813ba20 con 0x7fab1815de80 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1d807700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab181436f0 0x7fab1813b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1d807700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab181436f0 0x7fab1813b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47044/0 (socket says 172.21.15.5:47044) 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1d807700 1 -- 172.21.15.5:0/1927391331 learned_addr learned my addr 172.21.15.5:0/1927391331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1e008700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab180cf700 0x7fab1813d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1e809700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab1815de80 0x7fab1815c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1d807700 1 -- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab1815de80 msgr2=0x7fab1815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1d807700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab1815de80 0x7fab1815c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1d807700 1 -- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab180cf700 msgr2=0x7fab1813d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1d807700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab180cf700 0x7fab1813d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.686+0000 7fab1d807700 1 -- 172.21.15.5:0/1927391331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c0094a0 con 0x7fab181436f0 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.687+0000 7fab1d807700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab181436f0 0x7fab1813b2d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fab1400c750 tx=0x7fab1400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.687+0000 7fab0affd700 1 -- 172.21.15.5:0/1927391331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14004c80 con 0x7fab181436f0 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.687+0000 7fab0affd700 1 -- 172.21.15.5:0/1927391331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab14004e00 con 0x7fab181436f0 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.687+0000 7fab2026c700 1 -- 172.21.15.5:0/1927391331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1813bd00 con 0x7fab181436f0 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.687+0000 7fab0affd700 1 -- 172.21.15.5:0/1927391331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14007310 con 0x7fab181436f0 2024-05-08T23:24:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.687+0000 7fab2026c700 1 -- 172.21.15.5:0/1927391331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1813c2c0 con 0x7fab181436f0 2024-05-08T23:24:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.688+0000 7fab0affd700 1 -- 172.21.15.5:0/1927391331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fab14007470 con 0x7fab181436f0 2024-05-08T23:24:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.689+0000 7fab0affd700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faafc06ac90 0x7faafc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.689+0000 7fab0affd700 1 -- 172.21.15.5:0/1927391331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fab140890e0 con 0x7fab181436f0 2024-05-08T23:24:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.689+0000 7fab1e008700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faafc06ac90 0x7faafc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.690+0000 7fab1e008700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faafc06ac90 0x7faafc06d150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0090b0 tx=0x7fab0c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.691+0000 7fab2c426b80 1 Processor -- start 2024-05-08T23:24:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.691+0000 7fab2c426b80 1 -- start start 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261948bdd0 0x56261948c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261945e720 0x562619461970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x562619461eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab2c426b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626193b8150 con 0x56261948bdd0 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab2c426b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626193b5b10 con 0x562619489570 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab2c426b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626193b1400 con 0x56261945e720 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab1e008700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261948bdd0 0x56261948c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab1e008700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261948bdd0 0x56261948c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47050/0 (socket says 172.21.15.5:47050) 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab1e008700 1 -- 172.21.15.5:0/556497298 learned_addr learned my addr 172.21.15.5:0/556497298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab1e809700 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x562619461eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab1d807700 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261945e720 0x562619461970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.692+0000 7fab1e008700 1 -- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261945e720 msgr2=0x562619461970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7fab1e008700 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261945e720 0x562619461970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7fab1e008700 1 -- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 msgr2=0x562619461eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7fab1e008700 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x562619461eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7fab1e008700 1 -- 172.21.15.5:0/556497298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626192323b0 con 0x56261948bdd0 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7fab1e809700 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x562619461eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7fab1e008700 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261948bdd0 0x56261948c1b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006f90 tx=0x7fab0c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e45269e62c1087d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7faafbfff700 1 -- 172.21.15.5:0/556497298 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c02b070 con 0x56261948bdd0 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7faafbfff700 1 -- 172.21.15.5:0/556497298 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c003b30 con 0x56261948bdd0 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.693+0000 7faafbfff700 1 -- 172.21.15.5:0/556497298 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c020070 con 0x56261948bdd0 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 -- 172.21.15.5:0/556497298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261948bdd0 msgr2=0x56261948c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261948bdd0 0x56261948c1b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006f90 tx=0x7fab0c010000 comp rx=0 tx=0).stop 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 -- 172.21.15.5:0/556497298 shutdown_connections 2024-05-08T23:24:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x562619461eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261948bdd0 0x56261948c1b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 --2- 172.21.15.5:0/556497298 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261945e720 0x562619461970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 -- 172.21.15.5:0/556497298 >> 172.21.15.5:0/556497298 conn(0x5626193bf210 msgr2=0x5626194a6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 -- 172.21.15.5:0/556497298 shutdown_connections 2024-05-08T23:24:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 -- 172.21.15.5:0/556497298 wait complete. 2024-05-08T23:24:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.694+0000 7fab2c426b80 1 -- start start 2024-05-08T23:24:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.695+0000 7fab2c426b80 1 Processor -- start 2024-05-08T23:24:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261945e720 0x562619485570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562619489570 0x56261948f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 0x5626194a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab2c426b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261934d560 con 0x562619489570 2024-05-08T23:24:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab2c426b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261934d200 con 0x56261945e720 2024-05-08T23:24:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab2c426b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626192346a0 con 0x56261948bdd0 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab2c426b80 10 client.0 _ll_register_callbacks cb 0x562619356b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562619489570 0x56261948f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562619489570 0x56261948f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47056/0 (socket says 172.21.15.5:47056) 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 -- 172.21.15.5:0/859745290 learned_addr learned my addr 172.21.15.5:0/859745290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1e008700 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261945e720 0x562619485570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1e809700 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 0x5626194a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 -- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 msgr2=0x5626194a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 0x5626194a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 -- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261945e720 msgr2=0x562619485570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261945e720 0x562619485570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 -- 172.21.15.5:0/859745290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab180cef00 con 0x562619489570 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab1d807700 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562619489570 0x56261948f830 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fab1400cdb0 tx=0x7fab14057800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5043cd18e58e6d28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7faaf97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7faaf97fa700 1 -- 172.21.15.5:0/859745290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14058300 con 0x562619489570 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7faaf97fa700 1 -- 172.21.15.5:0/859745290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fab14057ea0 con 0x562619489570 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.696+0000 7fab2c426b80 1 -- 172.21.15.5:0/859745290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56261930a270 con 0x562619489570 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7faaf97fa700 1 -- 172.21.15.5:0/859745290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab1405bcd0 con 0x562619489570 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab2c426b80 10 client.6189 fetch_fsmap learned FSMap version 38 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab2c426b80 1 -- 172.21.15.5:0/859745290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x5626191deb60 con 0x562619489570 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7faaf97fa700 1 -- 172.21.15.5:0/859745290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56261930a270 con 0x562619489570 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7faaf97fa700 1 -- 172.21.15.5:0/859745290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fab14057390 con 0x562619489570 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab2c426b80 10 client.6189 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab2c426b80 10 client.6189 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab2c426b80 20 client.6189 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab2c426b80 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x562619488400 0x5626194a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab2c426b80 4 client.6189 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab2c426b80 1 -- 172.21.15.5:0/859745290 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626194307e0 con 0x562619488400 2024-05-08T23:24:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.697+0000 7fab1e008700 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x562619488400 0x5626194a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.698+0000 7fab1e008700 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x562619488400 0x5626194a4980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f04975b48c7b2c49 server_cookie=47da5f6983f3e14c in_seq=0 out_seq=0 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.698+0000 7faaf97fa700 10 client.6189 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.698+0000 7faaf97fa700 1 -- 172.21.15.5:0/859745290 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626194307e0 con 0x562619488400 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.698+0000 7faaf97fa700 10 client.6189 handle_command_reply: tid=0 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.698+0000 7fab2c426b80 1 client.6189 shutdown 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.698+0000 7fab2c426b80 1 -- 172.21.15.5:0/859745290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562619489570 msgr2=0x56261948f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562619489570 0x56261948f830 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fab1400cdb0 tx=0x7fab14057800 comp rx=0 tx=0).stop 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 -- 172.21.15.5:0/859745290 shutdown_connections 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x562619488400 0x5626194a4980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 0x5626194a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261945e720 0x562619485570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 --2- 172.21.15.5:0/859745290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562619489570 0x56261948f830 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7faaf97fa700 0 client.6189 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 -- 172.21.15.5:0/859745290 >> 172.21.15.5:0/859745290 conn(0x5626193bf210 msgr2=0x56261945ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 -- 172.21.15.5:0/859745290 shutdown_connections 2024-05-08T23:24:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 1 -- 172.21.15.5:0/859745290 wait complete. 2024-05-08T23:24:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.699+0000 7fab2c426b80 20 client.6189 trim_cache size 0 max 16384 2024-05-08T23:24:08.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.718+0000 7fab2c426b80 1 Processor -- start 2024-05-08T23:24:08.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.718+0000 7fab2c426b80 1 -- start start 2024-05-08T23:24:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x5626194d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562619489570 0x56261948f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261948bdd0 0x5626194a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab2c426b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626194a72d0 con 0x56261945e720 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab2c426b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562619499250 con 0x56261948bdd0 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab2c426b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562619456a60 con 0x562619489570 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x5626194d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x5626194d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47062/0 (socket says 172.21.15.5:47062) 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 -- 172.21.15.5:0/2888154272 learned_addr learned my addr 172.21.15.5:0/2888154272 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e809700 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261948bdd0 0x5626194a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1d807700 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562619489570 0x56261948f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 -- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562619489570 msgr2=0x56261948f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562619489570 0x56261948f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 -- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261948bdd0 msgr2=0x5626194a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261948bdd0 0x5626194a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 -- 172.21.15.5:0/2888154272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562619377560 con 0x56261945e720 2024-05-08T23:24:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1d807700 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562619489570 0x56261948f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e809700 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261948bdd0 0x5626194a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.719+0000 7fab1e008700 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x5626194d1770 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006f90 tx=0x7fab0c009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec87a9cdcb1f02a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab08da4700 1 -- 172.21.15.5:0/2888154272 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c02b040 con 0x56261945e720 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab08da4700 1 -- 172.21.15.5:0/2888154272 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c040370 con 0x56261945e720 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab08da4700 1 -- 172.21.15.5:0/2888154272 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0406e0 con 0x56261945e720 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 -- 172.21.15.5:0/2888154272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 msgr2=0x5626194d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x5626194d1770 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fab0c006f90 tx=0x7fab0c009210 comp rx=0 tx=0).stop 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 -- 172.21.15.5:0/2888154272 shutdown_connections 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56261948bdd0 0x5626194a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x5626194d1770 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 --2- 172.21.15.5:0/2888154272 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562619489570 0x56261948f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 -- 172.21.15.5:0/2888154272 >> 172.21.15.5:0/2888154272 conn(0x5626193bf210 msgr2=0x562619485380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 -- 172.21.15.5:0/2888154272 shutdown_connections 2024-05-08T23:24:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.720+0000 7fab2c426b80 1 -- 172.21.15.5:0/2888154272 wait complete. 2024-05-08T23:24:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.721+0000 7fab2c426b80 1 -- start start 2024-05-08T23:24:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.721+0000 7fab2c426b80 1 Processor -- start 2024-05-08T23:24:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x562619460380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x5626195186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab2c426b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 0x56261949bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab2c426b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626194c7f60 con 0x56261945e720 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab2c426b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626194c1090 con 0x562619489570 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab2c426b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626194d1700 con 0x56261948bdd0 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab2c426b80 10 client.0 _ll_register_callbacks cb 0x562619356b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab1e008700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x562619460380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab1e008700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x562619460380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47072/0 (socket says 172.21.15.5:47072) 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab1e008700 1 -- 172.21.15.5:0/3671960175 learned_addr learned my addr 172.21.15.5:0/3671960175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab1d807700 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x5626195186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.722+0000 7fab1e809700 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 0x56261949bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab1e008700 1 -- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 msgr2=0x56261949bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab1e008700 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 0x56261949bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab1e008700 1 -- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 msgr2=0x5626195186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab1e008700 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x5626195186f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab1e008700 1 -- 172.21.15.5:0/3671960175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626192323b0 con 0x56261945e720 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab1d807700 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x5626195186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab1e008700 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x562619460380 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fab0c040040 tx=0x7fab0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9356b29e74791ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7faaf97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7faaf97fa700 1 -- 172.21.15.5:0/3671960175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c02b050 con 0x56261945e720 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7faaf97fa700 1 -- 172.21.15.5:0/3671960175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fab0c0406e0 con 0x56261945e720 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab2c426b80 1 -- 172.21.15.5:0/3671960175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab0c003b30 con 0x56261945e720 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7faaf97fa700 1 -- 172.21.15.5:0/3671960175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab0c0408a0 con 0x56261945e720 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab2c426b80 10 client.6195 fetch_fsmap learned FSMap version 39 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab2c426b80 1 -- 172.21.15.5:0/3671960175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x562619236cd0 con 0x56261945e720 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7faaf97fa700 1 -- 172.21.15.5:0/3671960175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c003b30 con 0x56261945e720 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7faaf97fa700 1 -- 172.21.15.5:0/3671960175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fab0c011630 con 0x56261945e720 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab2c426b80 10 client.6195 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab2c426b80 10 client.6195 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.723+0000 7fab2c426b80 20 client.6195 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7fab2c426b80 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5626194c0920 0x56261949c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7fab2c426b80 4 client.6195 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7fab2c426b80 1 -- 172.21.15.5:0/3671960175 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5626193a9ec0 con 0x5626194c0920 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7fab1d807700 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5626194c0920 0x56261949c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7faaf97fa700 10 client.6195 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7fab1d807700 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5626194c0920 0x56261949c040 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=923a0705766039a0 server_cookie=9c5a7ee7811b9157 in_seq=0 out_seq=0 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7faaf97fa700 1 -- 172.21.15.5:0/3671960175 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5626193a9ec0 con 0x5626194c0920 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7faaf97fa700 10 client.6195 handle_command_reply: tid=0 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.724+0000 7fab2c426b80 1 client.6195 shutdown 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 -- 172.21.15.5:0/3671960175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 msgr2=0x562619460380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x562619460380 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fab0c040040 tx=0x7fab0c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 -- 172.21.15.5:0/3671960175 shutdown_connections 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5626194c0920 0x56261949c040 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56261948bdd0 0x56261949bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56261945e720 0x562619460380 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7faaf97fa700 0 client.6195 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 --2- 172.21.15.5:0/3671960175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562619489570 0x5626195186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 -- 172.21.15.5:0/3671960175 >> 172.21.15.5:0/3671960175 conn(0x5626193bf210 msgr2=0x56261945c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 -- 172.21.15.5:0/3671960175 shutdown_connections 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 1 -- 172.21.15.5:0/3671960175 wait complete. 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2c426b80 20 client.6195 trim_cache size 0 max 16384 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:24:08.730 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:24:08.731 INFO:teuthology.orchestra.run.smithi005.stdout: "cc35a3fa-d857-42f3-b1d7-b82b25291fa7": { 2024-05-08T23:24:08.731 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:24:08.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "cc35a3fa-d857-42f3-b1d7-b82b25291fa7", 2024-05-08T23:24:08.731 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:24:08.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:08.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:08.731 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 -- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faafc06ac90 msgr2=0x7faafc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faafc06ac90 0x7faafc06d150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0090b0 tx=0x7fab0c008030 comp rx=0 tx=0).stop 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 -- 172.21.15.5:0/1927391331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab181436f0 msgr2=0x7fab1813b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab181436f0 0x7fab1813b2d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fab1400c750 tx=0x7fab1400ca60 comp rx=0 tx=0).stop 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 -- 172.21.15.5:0/1927391331 shutdown_connections 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faafc06ac90 0x7faafc06d150 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab1815de80 0x7fab1815c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab180cf700 0x7fab1813d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 --2- 172.21.15.5:0/1927391331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab181436f0 0x7fab1813b2d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.725+0000 7fab2026c700 1 -- 172.21.15.5:0/1927391331 >> 172.21.15.5:0/1927391331 conn(0x7fab180722d0 msgr2=0x7fab1813de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.726+0000 7fab2026c700 1 -- 172.21.15.5:0/1927391331 shutdown_connections 2024-05-08T23:24:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:08.726+0000 7fab2026c700 1 -- 172.21.15.5:0/1927391331 wait complete. 2024-05-08T23:24:12.718 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:24:12.719+0000 7f8a4c50b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T23:24:13.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 scrub status 2024-05-08T23:24:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.886+0000 7ff7445d6700 1 Processor -- start 2024-05-08T23:24:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.886+0000 7ff7445d6700 1 -- start start 2024-05-08T23:24:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.887+0000 7ff7445d6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff73c0d1e50 0x7ff73c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.887+0000 7ff7445d6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c161320 0x7ff73c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.887+0000 7ff7445d6700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c0cf560 con 0x7ff73c0d2230 2024-05-08T23:24:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.887+0000 7ff7445d6700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c0ce950 con 0x7ff73c0d1e50 2024-05-08T23:24:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.887+0000 7ff7445d6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0cec30 con 0x7ff73c161320 2024-05-08T23:24:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742b73700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c161320 0x7ff73c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742372700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff73c0d1e50 0x7ff73c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742b73700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c161320 0x7ff73c06bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:55272/0 (socket says 172.21.15.5:55272) 2024-05-08T23:24:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742b73700 1 -- 172.21.15.5:0/2378165078 learned_addr learned my addr 172.21.15.5:0/2378165078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff741b71700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff73c0d2230 0x7ff73c160c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:56746/0 (socket says 172.21.15.5:56746) 2024-05-08T23:24:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff741370700 1 -- 172.21.15.5:0/2378165078 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861271780 0 0) 0x7ff73c0cf560 con 0x7ff73c0d2230 2024-05-08T23:24:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff741370700 1 -- 172.21.15.5:0/2378165078 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7280035b0 con 0x7ff73c0d2230 2024-05-08T23:24:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742372700 1 -- 172.21.15.5:0/2378165078 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c161320 msgr2=0x7ff73c06bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742372700 1 --2- 172.21.15.5:0/2378165078 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c161320 0x7ff73c06bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742372700 1 -- 172.21.15.5:0/2378165078 >> v1:172.21.15.49:6789/0 conn(0x7ff73c0d2230 legacy=0x7ff73c160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742372700 1 -- 172.21.15.5:0/2378165078 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0cef10 con 0x7ff73c0d1e50 2024-05-08T23:24:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742b73700 1 --2- 172.21.15.5:0/2378165078 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c161320 0x7ff73c06bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.888+0000 7ff742372700 1 --2- 172.21.15.5:0/2378165078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d1e50 0x7ff73c0ce2b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff738009840 tx=0x7ff738009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83669229789d59e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff741370700 1 -- 172.21.15.5:0/2378165078 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738011830 con 0x7ff73c0d1e50 2024-05-08T23:24:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff741370700 1 -- 172.21.15.5:0/2378165078 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff738011990 con 0x7ff73c0d1e50 2024-05-08T23:24:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff741370700 1 -- 172.21.15.5:0/2378165078 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738011d20 con 0x7ff73c0d1e50 2024-05-08T23:24:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff7445d6700 1 -- 172.21.15.5:0/2378165078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d1e50 msgr2=0x7ff73c0ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff7445d6700 1 --2- 172.21.15.5:0/2378165078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d1e50 0x7ff73c0ce2b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff738009840 tx=0x7ff738009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:13.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff7445d6700 1 -- 172.21.15.5:0/2378165078 shutdown_connections 2024-05-08T23:24:13.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff7445d6700 1 --2- 172.21.15.5:0/2378165078 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c161320 0x7ff73c06bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff7445d6700 1 --2- 172.21.15.5:0/2378165078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d1e50 0x7ff73c0ce2b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff7445d6700 1 -- 172.21.15.5:0/2378165078 >> 172.21.15.5:0/2378165078 conn(0x7ff73c08afe0 msgr2=0x7ff73c0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:13.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff7445d6700 1 -- 172.21.15.5:0/2378165078 shutdown_connections 2024-05-08T23:24:13.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.889+0000 7ff7445d6700 1 -- 172.21.15.5:0/2378165078 wait complete. 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.891+0000 7ff7445d6700 1 Processor -- start 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.891+0000 7ff7445d6700 1 -- start start 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.891+0000 7ff7445d6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c0d1e50 0x7ff73c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.891+0000 7ff7445d6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d2230 0x7ff73c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.891+0000 7ff7445d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff73c161320 0x7ff73c15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.891+0000 7ff7445d6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c1f6900 con 0x7ff73c161320 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.891+0000 7ff7445d6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c1f6ba0 con 0x7ff73c0d2230 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.891+0000 7ff7445d6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c1f6e80 con 0x7ff73c0d1e50 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff73c161320 0x7ff73c15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff73c161320 0x7ff73c15af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47080/0 (socket says 172.21.15.5:47080) 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 -- 172.21.15.5:0/3703652042 learned_addr learned my addr 172.21.15.5:0/3703652042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742372700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c0d1e50 0x7ff73c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff741b71700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d2230 0x7ff73c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 -- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c0d1e50 msgr2=0x7ff73c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c0d1e50 0x7ff73c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 -- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d2230 msgr2=0x7ff73c15aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d2230 0x7ff73c15aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 -- 172.21.15.5:0/3703652042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734009b30 con 0x7ff73c161320 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742372700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c0d1e50 0x7ff73c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff742b73700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff73c161320 0x7ff73c15af60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff73400ef90 tx=0x7ff73400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff732ffd700 1 -- 172.21.15.5:0/3703652042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734003c20 con 0x7ff73c161320 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff732ffd700 1 -- 172.21.15.5:0/3703652042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73400f040 con 0x7ff73c161320 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff7445d6700 1 -- 172.21.15.5:0/3703652042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7380094a0 con 0x7ff73c161320 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff732ffd700 1 -- 172.21.15.5:0/3703652042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73400c880 con 0x7ff73c161320 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.892+0000 7ff7445d6700 1 -- 172.21.15.5:0/3703652042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c15b4a0 con 0x7ff73c161320 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.894+0000 7ff732ffd700 1 -- 172.21.15.5:0/3703652042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff73400c9e0 con 0x7ff73c161320 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.894+0000 7ff732ffd700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff72006ac90 0x7ff72006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.895+0000 7ff732ffd700 1 -- 172.21.15.5:0/3703652042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff734089670 con 0x7ff73c161320 2024-05-08T23:24:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.895+0000 7ff742372700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff72006ac90 0x7ff72006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.895+0000 7ff742372700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff72006ac90 0x7ff72006d150 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff738000c00 tx=0x7ff738008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.897+0000 7ff750790b80 1 Processor -- start 2024-05-08T23:24:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.897+0000 7ff750790b80 1 -- start start 2024-05-08T23:24:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff750790b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x55861563a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff750790b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x55861563f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff750790b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615669250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff750790b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558615595150 con 0x558615668dd0 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff750790b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558615592b10 con 0x5586156617a0 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff750790b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55861558e400 con 0x55861563c370 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615669250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742372700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x55861563a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615669250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47092/0 (socket says 172.21.15.5:47092) 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 -- 172.21.15.5:0/914455967 learned_addr learned my addr 172.21.15.5:0/914455967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff741b71700 1 --2- 172.21.15.5:0/914455967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x55861563f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 -- 172.21.15.5:0/914455967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 msgr2=0x55861563f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 --2- 172.21.15.5:0/914455967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x55861563f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 -- 172.21.15.5:0/914455967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 msgr2=0x55861563a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 --2- 172.21.15.5:0/914455967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x55861563a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 -- 172.21.15.5:0/914455967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55861540f3b0 con 0x558615668dd0 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff742b73700 1 --2- 172.21.15.5:0/914455967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615669250 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff7340549d0 tx=0x7ff734057eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79638a7506d94f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff71ffff700 1 -- 172.21.15.5:0/914455967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734058c30 con 0x558615668dd0 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.898+0000 7ff71ffff700 1 -- 172.21.15.5:0/914455967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7340584d0 con 0x558615668dd0 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff71ffff700 1 -- 172.21.15.5:0/914455967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73405c300 con 0x558615668dd0 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 -- 172.21.15.5:0/914455967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 msgr2=0x558615669250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 --2- 172.21.15.5:0/914455967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615669250 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff7340549d0 tx=0x7ff734057eb0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 -- 172.21.15.5:0/914455967 shutdown_connections 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 --2- 172.21.15.5:0/914455967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615669250 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 --2- 172.21.15.5:0/914455967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x55861563a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 --2- 172.21.15.5:0/914455967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x55861563f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 -- 172.21.15.5:0/914455967 >> 172.21.15.5:0/914455967 conn(0x55861559c210 msgr2=0x558615683dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 -- 172.21.15.5:0/914455967 shutdown_connections 2024-05-08T23:24:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.899+0000 7ff750790b80 1 -- 172.21.15.5:0/914455967 wait complete. 2024-05-08T23:24:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.900+0000 7ff750790b80 1 -- start start 2024-05-08T23:24:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.900+0000 7ff750790b80 1 Processor -- start 2024-05-08T23:24:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff750790b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x558615660b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff750790b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615681980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff750790b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615686e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff750790b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861552a560 con 0x558615668dd0 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff750790b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861552a200 con 0x5586156617a0 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff750790b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586154116a0 con 0x55861563c370 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff750790b80 10 client.0 _ll_register_callbacks cb 0x558615533b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615686e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615686e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47104/0 (socket says 172.21.15.5:47104) 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff742b73700 1 -- 172.21.15.5:0/887392163 learned_addr learned my addr 172.21.15.5:0/887392163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff741b71700 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615681980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.901+0000 7ff742372700 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x558615660b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff742b73700 1 -- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 msgr2=0x558615660b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff742b73700 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x558615660b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff742b73700 1 -- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 msgr2=0x558615681980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff742b73700 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615681980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff742b73700 1 -- 172.21.15.5:0/887392163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c15b4a0 con 0x558615668dd0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff742b73700 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615686e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff7340549d0 tx=0x7ff73405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5f6f04dd6876e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff71d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff71d7fa700 1 -- 172.21.15.5:0/887392163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734058c30 con 0x558615668dd0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff71d7fa700 1 -- 172.21.15.5:0/887392163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff7340608c0 con 0x558615668dd0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 1 -- 172.21.15.5:0/887392163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586154e7270 con 0x558615668dd0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff71d7fa700 1 -- 172.21.15.5:0/887392163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff73405c300 con 0x558615668dd0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 10 client.6204 fetch_fsmap learned FSMap version 40 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 1 -- 172.21.15.5:0/887392163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5586153bbb60 con 0x558615668dd0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff71d7fa700 1 -- 172.21.15.5:0/887392163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586154e7270 con 0x558615668dd0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff71d7fa700 1 -- 172.21.15.5:0/887392163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) v1 ==== 3154+0+0 (secure 0 0 0) 0x7ff734058730 con 0x558615668dd0 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 10 client.6204 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 10 client.6204 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 20 client.6204 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x558615687350 0x558615685250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 4 client.6204 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.902+0000 7ff750790b80 1 -- 172.21.15.5:0/887392163 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55861560d7e0 con 0x558615687350 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.903+0000 7ff742372700 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x558615687350 0x558615685250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.903+0000 7ff742372700 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x558615687350 0x558615685250 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a60ac98584404899 server_cookie=661cb4e534df2a2e in_seq=0 out_seq=0 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.903+0000 7ff71d7fa700 10 client.6204 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff71d7fa700 1 -- 172.21.15.5:0/887392163 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55861560d7e0 con 0x558615687350 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff71d7fa700 10 client.6204 handle_command_reply: tid=0 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 client.6204 shutdown 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 -- 172.21.15.5:0/887392163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 msgr2=0x558615686e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615686e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff7340549d0 tx=0x7ff73405b040 comp rx=0 tx=0).stop 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 -- 172.21.15.5:0/887392163 shutdown_connections 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x558615687350 0x558615685250 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615686e10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x558615660b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff71d7fa700 0 client.6204 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 --2- 172.21.15.5:0/887392163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615681980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 -- 172.21.15.5:0/887392163 >> 172.21.15.5:0/887392163 conn(0x55861559c210 msgr2=0x55861563e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 -- 172.21.15.5:0/887392163 shutdown_connections 2024-05-08T23:24:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 1 -- 172.21.15.5:0/887392163 wait complete. 2024-05-08T23:24:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.904+0000 7ff750790b80 20 client.6204 trim_cache size 0 max 16384 2024-05-08T23:24:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.922+0000 7ff750790b80 1 Processor -- start 2024-05-08T23:24:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.922+0000 7ff750790b80 1 -- start start 2024-05-08T23:24:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff750790b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x55861563ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff750790b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615638970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff750790b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615683bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff750790b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586156605a0 con 0x558615668dd0 2024-05-08T23:24:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff750790b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861569d5a0 con 0x5586156617a0 2024-05-08T23:24:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff750790b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55861563f530 con 0x55861563c370 2024-05-08T23:24:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615683bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615683bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47108/0 (socket says 172.21.15.5:47108) 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff742b73700 1 -- 172.21.15.5:0/1414092077 learned_addr learned my addr 172.21.15.5:0/1414092077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff741b71700 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615638970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.923+0000 7ff742372700 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x55861563ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff742b73700 1 -- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 msgr2=0x55861563ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff742b73700 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x55861563ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff742b73700 1 -- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 msgr2=0x558615638970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff742b73700 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615638970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff742b73700 1 -- 172.21.15.5:0/1414092077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558615554560 con 0x558615668dd0 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff741b71700 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615638970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff742b73700 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615683bc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff734054550 tx=0x7ff734083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff017b27f876e945 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff730ff9700 1 -- 172.21.15.5:0/1414092077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73405c300 con 0x558615668dd0 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff730ff9700 1 -- 172.21.15.5:0/1414092077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff734058c30 con 0x558615668dd0 2024-05-08T23:24:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff730ff9700 1 -- 172.21.15.5:0/1414092077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340608c0 con 0x558615668dd0 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff750790b80 1 -- 172.21.15.5:0/1414092077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 msgr2=0x558615683bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.924+0000 7ff750790b80 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615683bc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff734054550 tx=0x7ff734083040 comp rx=0 tx=0).stop 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 -- 172.21.15.5:0/1414092077 shutdown_connections 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615683bc0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55861563c370 0x55861563ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 --2- 172.21.15.5:0/1414092077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5586156617a0 0x558615638970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 -- 172.21.15.5:0/1414092077 >> 172.21.15.5:0/1414092077 conn(0x55861559c210 msgr2=0x558615686bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 -- 172.21.15.5:0/1414092077 shutdown_connections 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 -- 172.21.15.5:0/1414092077 wait complete. 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 -- start start 2024-05-08T23:24:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.925+0000 7ff750790b80 1 Processor -- start 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff750790b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55861563c370 0x5586156ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff750790b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5586156617a0 0x55861565f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff750790b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615679760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff750790b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55861567a560 con 0x558615668dd0 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff750790b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586156854e0 con 0x55861563c370 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff750790b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55861566bb10 con 0x5586156617a0 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff750790b80 10 client.0 _ll_register_callbacks cb 0x558615533b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615679760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff742372700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55861563c370 0x5586156ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff742b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615679760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47112/0 (socket says 172.21.15.5:47112) 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff742b73700 1 -- 172.21.15.5:0/3401537664 learned_addr learned my addr 172.21.15.5:0/3401537664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.926+0000 7ff741b71700 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5586156617a0 0x55861565f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff742b73700 1 -- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5586156617a0 msgr2=0x55861565f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff742b73700 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5586156617a0 0x55861565f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff742b73700 1 -- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55861563c370 msgr2=0x5586156ebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff742b73700 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55861563c370 0x5586156ebc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff742b73700 1 -- 172.21.15.5:0/3401537664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586153bbb60 con 0x558615668dd0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff742b73700 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615679760 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff734070050 tx=0x7ff734054810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369d9bf5f05eb26e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff71d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff71d7fa700 1 -- 172.21.15.5:0/3401537664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73405c300 con 0x558615668dd0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff71d7fa700 1 -- 172.21.15.5:0/3401537664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff7340608c0 con 0x558615668dd0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff750790b80 1 -- 172.21.15.5:0/3401537664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558615586ec0 con 0x558615668dd0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff71d7fa700 1 -- 172.21.15.5:0/3401537664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff734058c30 con 0x558615668dd0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff750790b80 10 client.6210 fetch_fsmap learned FSMap version 40 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff750790b80 1 -- 172.21.15.5:0/3401537664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x558615413cd0 con 0x558615668dd0 2024-05-08T23:24:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff71d7fa700 1 -- 172.21.15.5:0/3401537664 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558615586ec0 con 0x558615668dd0 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.927+0000 7ff71d7fa700 1 -- 172.21.15.5:0/3401537664 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) v1 ==== 3154+0+0 (secure 0 0 0) 0x7ff734058730 con 0x558615668dd0 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff750790b80 10 client.6210 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff750790b80 10 client.6210 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff750790b80 20 client.6210 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff750790b80 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x55861566c840 0x558615679ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff750790b80 4 client.6210 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff750790b80 1 -- 172.21.15.5:0/3401537664 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586153dccf0 con 0x55861566c840 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff742372700 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x55861566c840 0x558615679ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff71d7fa700 10 client.6210 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff742372700 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x55861566c840 0x558615679ca0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=981a798983dc6fc server_cookie=868570d3ab512d2a in_seq=0 out_seq=0 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff71d7fa700 1 -- 172.21.15.5:0/3401537664 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5586153dccf0 con 0x55861566c840 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff71d7fa700 10 client.6210 handle_command_reply: tid=0 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.928+0000 7ff750790b80 1 client.6210 shutdown 2024-05-08T23:24:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 -- 172.21.15.5:0/3401537664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 msgr2=0x558615679760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615679760 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff734070050 tx=0x7ff734054810 comp rx=0 tx=0).stop 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 -- 172.21.15.5:0/3401537664 shutdown_connections 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x55861566c840 0x558615679ca0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558615668dd0 0x558615679760 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55861563c370 0x5586156ebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 --2- 172.21.15.5:0/3401537664 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5586156617a0 0x55861565f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff71d7fa700 0 client.6210 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 -- 172.21.15.5:0/3401537664 >> 172.21.15.5:0/3401537664 conn(0x55861559c210 msgr2=0x558615638970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 -- 172.21.15.5:0/3401537664 shutdown_connections 2024-05-08T23:24:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 1 -- 172.21.15.5:0/3401537664 wait complete. 2024-05-08T23:24:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff750790b80 20 client.6210 trim_cache size 0 max 16384 2024-05-08T23:24:13.934 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:24:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:24:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "cc35a3fa-d857-42f3-b1d7-b82b25291fa7": { 2024-05-08T23:24:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:24:13.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "cc35a3fa-d857-42f3-b1d7-b82b25291fa7", 2024-05-08T23:24:13.935 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:24:13.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:13.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:13.935 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 -- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff72006ac90 msgr2=0x7ff72006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff72006ac90 0x7ff72006d150 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff738000c00 tx=0x7ff738008030 comp rx=0 tx=0).stop 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 -- 172.21.15.5:0/3703652042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff73c161320 msgr2=0x7ff73c15af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff73c161320 0x7ff73c15af60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff73400ef90 tx=0x7ff73400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 -- 172.21.15.5:0/3703652042 shutdown_connections 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff72006ac90 0x7ff72006d150 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff73c161320 0x7ff73c15af60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff73c0d1e50 0x7ff73c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 --2- 172.21.15.5:0/3703652042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff73c0d2230 0x7ff73c15aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.929+0000 7ff7445d6700 1 -- 172.21.15.5:0/3703652042 >> 172.21.15.5:0/3703652042 conn(0x7ff73c08afe0 msgr2=0x7ff73c15e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.930+0000 7ff7445d6700 1 -- 172.21.15.5:0/3703652042 shutdown_connections 2024-05-08T23:24:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:13.930+0000 7ff7445d6700 1 -- 172.21.15.5:0/3703652042 wait complete. 2024-05-08T23:24:18.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 scrub status 2024-05-08T23:24:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.092+0000 7fa2c6d27700 1 Processor -- start 2024-05-08T23:24:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.092+0000 7fa2c6d27700 1 -- start start 2024-05-08T23:24:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.093+0000 7fa2c6d27700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa2c00aea70 0x7fa2c00b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.093+0000 7fa2c6d27700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa2c00d0a60 0x7fa2c00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.093+0000 7fa2c6d27700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c015add0 con 0x7fa2c0154df0 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.093+0000 7fa2c6d27700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c015abe0 con 0x7fa2c00d0a60 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.093+0000 7fa2c6d27700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c015b070 con 0x7fa2c00aea70 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.093+0000 7fa2c52c4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa2c00d0a60 0x7fa2c00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.093+0000 7fa2c52c4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa2c00d0a60 0x7fa2c00b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53132/0 (socket says 172.21.15.5:53132) 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.093+0000 7fa2c52c4700 1 -- 172.21.15.5:0/1222383874 learned_addr learned my addr 172.21.15.5:0/1222383874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2bffff700 1 --2- 172.21.15.5:0/1222383874 >> v2:172.21.15.49:3300/0 conn(0x7fa2c00aea70 0x7fa2c00b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2c52c4700 1 -- 172.21.15.5:0/1222383874 >> v2:172.21.15.49:3300/0 conn(0x7fa2c00aea70 msgr2=0x7fa2c00b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/1222383874 >> v2:172.21.15.49:3300/0 conn(0x7fa2c00aea70 0x7fa2c00b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2c52c4700 1 -- 172.21.15.5:0/1222383874 >> v1:172.21.15.49:6789/0 conn(0x7fa2c0154df0 legacy=0x7fa2c00d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2c52c4700 1 -- 172.21.15.5:0/1222383874 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c015b350 con 0x7fa2c00d0a60 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/1222383874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00b43b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0072c00 tx=0x7fa2b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbacae319d16e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2bf7fe700 1 -- 172.21.15.5:0/1222383874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800e070 con 0x7fa2c00d0a60 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2bf7fe700 1 -- 172.21.15.5:0/1222383874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b8003a20 con 0x7fa2c00d0a60 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.094+0000 7fa2bf7fe700 1 -- 172.21.15.5:0/1222383874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003dd0 con 0x7fa2c00d0a60 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.095+0000 7fa2c6d27700 1 -- 172.21.15.5:0/1222383874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 msgr2=0x7fa2c00b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.095+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/1222383874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00b43b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0072c00 tx=0x7fa2b800b330 comp rx=0 tx=0).stop 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.095+0000 7fa2c6d27700 1 -- 172.21.15.5:0/1222383874 shutdown_connections 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.095+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/1222383874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00b43b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.095+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/1222383874 >> v2:172.21.15.49:3300/0 conn(0x7fa2c00aea70 0x7fa2c00b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.095+0000 7fa2c6d27700 1 -- 172.21.15.5:0/1222383874 >> 172.21.15.5:0/1222383874 conn(0x7fa2c0099a50 msgr2=0x7fa2c00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.095+0000 7fa2c6d27700 1 -- 172.21.15.5:0/1222383874 shutdown_connections 2024-05-08T23:24:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.095+0000 7fa2c6d27700 1 -- 172.21.15.5:0/1222383874 wait complete. 2024-05-08T23:24:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.096+0000 7fa2c6d27700 1 Processor -- start 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.097+0000 7fa2c6d27700 1 -- start start 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c6d27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c00aea70 0x7fa2c00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c6d27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c6d27700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c0154df0 0x7fa2c006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c6d27700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c009e370 con 0x7fa2c00d0a60 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c6d27700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00cd4f0 con 0x7fa2c00aea70 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c6d27700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00cd7d0 con 0x7fa2c0154df0 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53136/0 (socket says 172.21.15.5:53136) 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 -- 172.21.15.5:0/3062635894 learned_addr learned my addr 172.21.15.5:0/3062635894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c0154df0 0x7fa2c006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c00aea70 0x7fa2c00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 -- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c0154df0 msgr2=0x7fa2c006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c0154df0 0x7fa2c006e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 -- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c00aea70 msgr2=0x7fa2c00ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c00aea70 0x7fa2c00ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 -- 172.21.15.5:0/3062635894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8009b30 con 0x7fa2c00d0a60 2024-05-08T23:24:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c00aea70 0x7fa2c00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.098+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00aff80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e810 tx=0x7fa2a800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.099+0000 7fa2bd7fa700 1 -- 172.21.15.5:0/3062635894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8013610 con 0x7fa2c00d0a60 2024-05-08T23:24:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.099+0000 7fa2bd7fa700 1 -- 172.21.15.5:0/3062635894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8013770 con 0x7fa2c00d0a60 2024-05-08T23:24:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.099+0000 7fa2c6d27700 1 -- 172.21.15.5:0/3062635894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c00cdab0 con 0x7fa2c00d0a60 2024-05-08T23:24:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.099+0000 7fa2bd7fa700 1 -- 172.21.15.5:0/3062635894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8013b20 con 0x7fa2c00d0a60 2024-05-08T23:24:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.099+0000 7fa2c6d27700 1 -- 172.21.15.5:0/3062635894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c00ce070 con 0x7fa2c00d0a60 2024-05-08T23:24:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.100+0000 7fa2bd7fa700 1 -- 172.21.15.5:0/3062635894 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa2a8013c80 con 0x7fa2c00d0a60 2024-05-08T23:24:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.101+0000 7fa2bd7fa700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2a006ac90 0x7fa2a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.101+0000 7fa2bd7fa700 1 -- 172.21.15.5:0/3062635894 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa2a8089060 con 0x7fa2c00d0a60 2024-05-08T23:24:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.101+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2a006ac90 0x7fa2a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.102+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2a006ac90 0x7fa2a006d150 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0006fd0 tx=0x7fa2b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.103+0000 7fa2d2ee1b80 1 Processor -- start 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.103+0000 7fa2d2ee1b80 1 -- start start 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975d7e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975d830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975dacdd0 0x557975dad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557975cd9150 con 0x557975da57a0 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557975cd6b10 con 0x557975dacdd0 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557975cd2400 con 0x557975d80370 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975d7e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975d7e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53144/0 (socket says 172.21.15.5:53144) 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 -- 172.21.15.5:0/3537097314 learned_addr learned my addr 172.21.15.5:0/3537097314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975d830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975dacdd0 0x557975dad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 -- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 msgr2=0x557975d830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975d830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 -- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975dacdd0 msgr2=0x557975dad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975dacdd0 0x557975dad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 -- 172.21.15.5:0/3537097314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557975b533b0 con 0x557975da57a0 2024-05-08T23:24:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975dacdd0 0x557975dad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975d7e110 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c9a0 tx=0x7fa2b000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc38d86abf30c250 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.104+0000 7fa2b67fc700 1 -- 172.21.15.5:0/3537097314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000ade0 con 0x557975da57a0 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2b67fc700 1 -- 172.21.15.5:0/3537097314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b00136a0 con 0x557975da57a0 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2b67fc700 1 -- 172.21.15.5:0/3537097314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0013a50 con 0x557975da57a0 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3537097314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 msgr2=0x557975d7e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975d7e110 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c9a0 tx=0x7fa2b000f000 comp rx=0 tx=0).stop 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3537097314 shutdown_connections 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975dacdd0 0x557975dad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975d7e110 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3537097314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975d830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3537097314 >> 172.21.15.5:0/3537097314 conn(0x557975ce0210 msgr2=0x557975dc7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:19.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3537097314 shutdown_connections 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.105+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3537097314 wait complete. 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.106+0000 7fa2d2ee1b80 1 -- start start 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.106+0000 7fa2d2ee1b80 1 Processor -- start 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975daf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975d7c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975dd9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557975c6e560 con 0x557975dacdd0 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557975c6e200 con 0x557975da57a0 2024-05-08T23:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557975b556a0 con 0x557975d80370 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2d2ee1b80 10 client.0 _ll_register_callbacks cb 0x557975c77b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2c52c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975dd9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2c52c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975dd9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53158/0 (socket says 172.21.15.5:53158) 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2c52c4700 1 -- 172.21.15.5:0/1521306205 learned_addr learned my addr 172.21.15.5:0/1521306205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2bffff700 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975d7c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975daf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2c52c4700 1 -- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 msgr2=0x557975daf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975daf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2c52c4700 1 -- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 msgr2=0x557975d7c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.107+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975d7c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2c52c4700 1 -- 172.21.15.5:0/1521306205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b000c680 con 0x557975dacdd0 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975daf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975dd9b30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00676a0 tx=0x7fa2b8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721f02a55b5331a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa29f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa29f7fe700 1 -- 172.21.15.5:0/1521306205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004230 con 0x557975dacdd0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa29f7fe700 1 -- 172.21.15.5:0/1521306205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa2b8004390 con 0x557975dacdd0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/1521306205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557975c2b270 con 0x557975dacdd0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa29f7fe700 1 -- 172.21.15.5:0/1521306205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b8009530 con 0x557975dacdd0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 10 client.6222 fetch_fsmap learned FSMap version 41 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/1521306205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x557975affb60 con 0x557975dacdd0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa29f7fe700 1 -- 172.21.15.5:0/1521306205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557975c2b270 con 0x557975dacdd0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa29f7fe700 1 -- 172.21.15.5:0/1521306205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fa2b8004760 con 0x557975dacdd0 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 10 client.6222 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 10 client.6222 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 20 client.6222 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x557975dca650 0x557975dcaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 4 client.6222 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.108+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/1521306205 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557975d517e0 con 0x557975dca650 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.109+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x557975dca650 0x557975dcaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.109+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x557975dca650 0x557975dcaa60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a59b55585ba8e7d server_cookie=703f220dc0aebd6e in_seq=0 out_seq=0 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.109+0000 7fa29f7fe700 10 client.6222 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa29f7fe700 1 -- 172.21.15.5:0/1521306205 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557975d517e0 con 0x557975dca650 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa29f7fe700 10 client.6222 handle_command_reply: tid=0 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 client.6222 shutdown 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/1521306205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 msgr2=0x557975dd9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975dd9b30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00676a0 tx=0x7fa2b8004b40 comp rx=0 tx=0).stop 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/1521306205 shutdown_connections 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x557975dca650 0x557975dcaa60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975dd9b30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975daf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/1521306205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975d7c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa29f7fe700 0 client.6222 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/1521306205 >> 172.21.15.5:0/1521306205 conn(0x557975ce0210 msgr2=0x557975dad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/1521306205 shutdown_connections 2024-05-08T23:24:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/1521306205 wait complete. 2024-05-08T23:24:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.110+0000 7fa2d2ee1b80 20 client.6222 trim_cache size 0 max 16384 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.128+0000 7fa2d2ee1b80 1 Processor -- start 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.128+0000 7fa2d2ee1b80 1 -- start start 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975d80370 0x557975de1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975da4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975dacdd0 0x557975dc6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557975d7e0c0 con 0x557975da57a0 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557975d82290 con 0x557975d80370 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557975dc8b60 con 0x557975dacdd0 2024-05-08T23:24:19.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975da4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975da4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53160/0 (socket says 172.21.15.5:53160) 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2bffff700 1 -- 172.21.15.5:0/3977673242 learned_addr learned my addr 172.21.15.5:0/3977673242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975dacdd0 0x557975dc6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975d80370 0x557975de1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2bffff700 1 -- 172.21.15.5:0/3977673242 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975dacdd0 msgr2=0x557975dc6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975dacdd0 0x557975dc6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2bffff700 1 -- 172.21.15.5:0/3977673242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975d80370 msgr2=0x557975de1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975d80370 0x557975de1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.129+0000 7fa2bffff700 1 -- 172.21.15.5:0/3977673242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557975dd6770 con 0x557975da57a0 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975da4b30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80543d0 tx=0x7fa2a8054340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7366ff6e107cde6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2b6ffd700 1 -- 172.21.15.5:0/3977673242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80602c0 con 0x557975da57a0 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2b6ffd700 1 -- 172.21.15.5:0/3977673242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8056270 con 0x557975da57a0 2024-05-08T23:24:19.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2b6ffd700 1 -- 172.21.15.5:0/3977673242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80575c0 con 0x557975da57a0 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3977673242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 msgr2=0x557975da4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975da4b30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80543d0 tx=0x7fa2a8054340 comp rx=0 tx=0).stop 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3977673242 shutdown_connections 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975dacdd0 0x557975dc6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975d80370 0x557975de1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3977673242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975da57a0 0x557975da4b30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3977673242 >> 172.21.15.5:0/3977673242 conn(0x557975ce0210 msgr2=0x557975d791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3977673242 shutdown_connections 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.130+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3977673242 wait complete. 2024-05-08T23:24:19.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.131+0000 7fa2d2ee1b80 1 -- start start 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.131+0000 7fa2d2ee1b80 1 Processor -- start 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975dc6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975dc7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2d2ee1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975e174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557975d81660 con 0x557975dacdd0 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557975ddb600 con 0x557975da57a0 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2d2ee1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557975e18710 con 0x557975d80370 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2d2ee1b80 10 client.0 _ll_register_callbacks cb 0x557975c77b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975e174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975e174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53164/0 (socket says 172.21.15.5:53164) 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 -- 172.21.15.5:0/3260496987 learned_addr learned my addr 172.21.15.5:0/3260496987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975dc6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975dc7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 -- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 msgr2=0x557975dc6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975dc6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 -- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 msgr2=0x557975dc7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975dc7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 -- 172.21.15.5:0/3260496987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c00ce070 con 0x557975dacdd0 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2bffff700 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975dc7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa2c52c4700 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975e174e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800b8a0 tx=0x7fa2b8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9d94db66ba0e17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.133+0000 7fa29f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa29f7fe700 1 -- 172.21.15.5:0/3260496987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800e040 con 0x557975dacdd0 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa29f7fe700 1 -- 172.21.15.5:0/3260496987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa2b801d4d0 con 0x557975dacdd0 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3260496987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2b00136a0 con 0x557975dacdd0 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa29f7fe700 1 -- 172.21.15.5:0/3260496987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b801d6e0 con 0x557975dacdd0 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 10 client.6228 fetch_fsmap learned FSMap version 41 2024-05-08T23:24:19.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3260496987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x557975c98560 con 0x557975dacdd0 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa29f7fe700 1 -- 172.21.15.5:0/3260496987 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b00136a0 con 0x557975dacdd0 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa29f7fe700 1 -- 172.21.15.5:0/3260496987 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fa2b801d840 con 0x557975dacdd0 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 10 client.6228 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 10 client.6228 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 20 client.6228 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x557975e16ed0 0x557975e17a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 4 client.6228 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3260496987 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557975ccaec0 con 0x557975e16ed0 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.134+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x557975e16ed0 0x557975e17a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa29f7fe700 10 client.6228 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2c4ac3700 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x557975e16ed0 0x557975e17a20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a3d2ba4630634ce server_cookie=2f02e15e1028f2b1 in_seq=0 out_seq=0 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa29f7fe700 1 -- 172.21.15.5:0/3260496987 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x557975ccaec0 con 0x557975e16ed0 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa29f7fe700 10 client.6228 handle_command_reply: tid=0 2024-05-08T23:24:19.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 client.6228 shutdown 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3260496987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 msgr2=0x557975e174e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975e174e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800b8a0 tx=0x7fa2b8009810 comp rx=0 tx=0).stop 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3260496987 shutdown_connections 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x557975e16ed0 0x557975e17a20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557975dacdd0 0x557975e174e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557975d80370 0x557975dc6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa29f7fe700 0 client.6228 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 --2- 172.21.15.5:0/3260496987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557975da57a0 0x557975dc7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3260496987 >> 172.21.15.5:0/3260496987 conn(0x557975ce0210 msgr2=0x557975d791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3260496987 shutdown_connections 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 1 -- 172.21.15.5:0/3260496987 wait complete. 2024-05-08T23:24:19.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.135+0000 7fa2d2ee1b80 20 client.6228 trim_cache size 0 max 16384 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout: "cc35a3fa-d857-42f3-b1d7-b82b25291fa7": { 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "cc35a3fa-d857-42f3-b1d7-b82b25291fa7", 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:19.338 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 -- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2a006ac90 msgr2=0x7fa2a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2a006ac90 0x7fa2a006d150 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0006fd0 tx=0x7fa2b0008040 comp rx=0 tx=0).stop 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 -- 172.21.15.5:0/3062635894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 msgr2=0x7fa2c00aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00aff80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e810 tx=0x7fa2a800ebd0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 -- 172.21.15.5:0/3062635894 shutdown_connections 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2a006ac90 0x7fa2a006d150 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c0154df0 0x7fa2c006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c00aea70 0x7fa2c00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 --2- 172.21.15.5:0/3062635894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c00d0a60 0x7fa2c00aff80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 -- 172.21.15.5:0/3062635894 >> 172.21.15.5:0/3062635894 conn(0x7fa2c0099a50 msgr2=0x7fa2c01510c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:19.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 -- 172.21.15.5:0/3062635894 shutdown_connections 2024-05-08T23:24:19.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:19.136+0000 7fa2c6d27700 1 -- 172.21.15.5:0/3062635894 wait complete. 2024-05-08T23:24:20.094 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:24:20.094+0000 7f8a4c50b700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T23:24:24.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 scrub status 2024-05-08T23:24:24.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.489+0000 7f01ea074700 1 Processor -- start 2024-05-08T23:24:24.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.489+0000 7f01ea074700 1 -- start start 2024-05-08T23:24:24.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01ea074700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01e40b4d50 0x7f01e406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01ea074700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e406aea0 0x7f01e41454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01ea074700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e4145040 con 0x7f01e40b0d80 2024-05-08T23:24:24.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01ea074700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e406b330 con 0x7f01e40b4d50 2024-05-08T23:24:24.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01ea074700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40b16e0 con 0x7f01e406aea0 2024-05-08T23:24:24.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01e2ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01e40b4d50 0x7f01e406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01e37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f01e40b0d80 0x7f01e40b3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57178/0 (socket says 172.21.15.5:57178) 2024-05-08T23:24:24.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01e2ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01e40b4d50 0x7f01e406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53170/0 (socket says 172.21.15.5:53170) 2024-05-08T23:24:24.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01e37fe700 1 -- 172.21.15.5:0/3614792360 learned_addr learned my addr 172.21.15.5:0/3614792360 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:24.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01e3fff700 1 --2- 172.21.15.5:0/3614792360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e406aea0 0x7f01e41454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01e2ffd700 1 -- 172.21.15.5:0/3614792360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e406aea0 msgr2=0x7f01e41454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/3614792360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e406aea0 0x7f01e41454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.490+0000 7f01e2ffd700 1 -- 172.21.15.5:0/3614792360 >> v1:172.21.15.5:6789/0 conn(0x7f01e40b0d80 legacy=0x7f01e40b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.491+0000 7f01e2ffd700 1 -- 172.21.15.5:0/3614792360 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40b19c0 con 0x7f01e40b4d50 2024-05-08T23:24:24.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.491+0000 7f01e3fff700 1 --2- 172.21.15.5:0/3614792360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e406aea0 0x7f01e41454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:24.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.491+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/3614792360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e406a960 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009840 tx=0x7f01d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3639fd4f59677b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.491+0000 7f01e27fc700 1 -- 172.21.15.5:0/3614792360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8011830 con 0x7f01e40b4d50 2024-05-08T23:24:24.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.491+0000 7f01e27fc700 1 -- 172.21.15.5:0/3614792360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8011990 con 0x7f01e40b4d50 2024-05-08T23:24:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.491+0000 7f01e27fc700 1 -- 172.21.15.5:0/3614792360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8011d20 con 0x7f01e40b4d50 2024-05-08T23:24:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.491+0000 7f01ea074700 1 -- 172.21.15.5:0/3614792360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 msgr2=0x7f01e406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.491+0000 7f01ea074700 1 --2- 172.21.15.5:0/3614792360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e406a960 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009840 tx=0x7f01d8009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.492+0000 7f01ea074700 1 -- 172.21.15.5:0/3614792360 shutdown_connections 2024-05-08T23:24:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.492+0000 7f01ea074700 1 --2- 172.21.15.5:0/3614792360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e406aea0 0x7f01e41454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.492+0000 7f01ea074700 1 --2- 172.21.15.5:0/3614792360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e406a960 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.492+0000 7f01ea074700 1 -- 172.21.15.5:0/3614792360 >> 172.21.15.5:0/3614792360 conn(0x7f01e4099a50 msgr2=0x7f01e40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.492+0000 7f01ea074700 1 -- 172.21.15.5:0/3614792360 shutdown_connections 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.492+0000 7f01ea074700 1 -- 172.21.15.5:0/3614792360 wait complete. 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.493+0000 7f01ea074700 1 Processor -- start 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.493+0000 7f01ea074700 1 -- start start 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01ea074700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e406aea0 0x7f01e415d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01ea074700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e40b0d80 0x7f01e415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01ea074700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01ea074700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e415d5c0 con 0x7f01e40b4d50 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01ea074700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e409a480 con 0x7f01e406aea0 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01ea074700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e409a760 con 0x7f01e40b0d80 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e415f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53176/0 (socket says 172.21.15.5:53176) 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e3fff700 1 -- 172.21.15.5:0/1768502728 learned_addr learned my addr 172.21.15.5:0/1768502728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e40b0d80 0x7f01e415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e37fe700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e406aea0 0x7f01e415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e3fff700 1 -- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e40b0d80 msgr2=0x7f01e415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e3fff700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e40b0d80 0x7f01e415f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e3fff700 1 -- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e406aea0 msgr2=0x7f01e415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e3fff700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e406aea0 0x7f01e415d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.494+0000 7f01e3fff700 1 -- 172.21.15.5:0/1768502728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80094a0 con 0x7f01e40b4d50 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.495+0000 7f01e3fff700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e415f9a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00cef0 tx=0x7f01dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.495+0000 7f01cbfff700 1 -- 172.21.15.5:0/1768502728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00ae50 con 0x7f01e40b4d50 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.495+0000 7f01cbfff700 1 -- 172.21.15.5:0/1768502728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc015440 con 0x7f01e40b4d50 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.495+0000 7f01ea074700 1 -- 172.21.15.5:0/1768502728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e409aa40 con 0x7f01e40b4d50 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.495+0000 7f01cbfff700 1 -- 172.21.15.5:0/1768502728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0157f0 con 0x7f01e40b4d50 2024-05-08T23:24:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.495+0000 7f01ea074700 1 -- 172.21.15.5:0/1768502728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e409b000 con 0x7f01e40b4d50 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.496+0000 7f01cbfff700 1 -- 172.21.15.5:0/1768502728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f01dc00d430 con 0x7f01e40b4d50 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.497+0000 7f01cbfff700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01d006ac90 0x7f01d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.497+0000 7f01cbfff700 1 -- 172.21.15.5:0/1768502728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01dc089920 con 0x7f01e40b4d50 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.497+0000 7f01e37fe700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01d006ac90 0x7f01d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.498+0000 7f01e37fe700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01d006ac90 0x7f01d006d150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f01d4005910 tx=0x7f01d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.499+0000 7f01f622eb80 1 Processor -- start 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.499+0000 7f01f622eb80 1 -- start start 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.500+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa19f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.500+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1a1370 0x5651aa1a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.500+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa1ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.500+0000 7f01f622eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651aa0fa150 con 0x5651aa1a1370 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.500+0000 7f01f622eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651aa0f7b10 con 0x5651aa1c67a0 2024-05-08T23:24:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.500+0000 7f01f622eb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651aa0f3400 con 0x5651aa1cddd0 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1a1370 0x5651aa1a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa1ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57134/0 (socket says 172.21.15.5:57134) 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e37fe700 1 -- 172.21.15.5:0/3210000993 learned_addr learned my addr 172.21.15.5:0/3210000993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa1ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:56436/0 (socket says 172.21.15.5:56436) 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1a1370 0x5651aa1a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53182/0 (socket says 172.21.15.5:53182) 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e2ffd700 1 -- 172.21.15.5:0/3210000993 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 msgr2=0x5651aa1ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa1ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e2ffd700 1 -- 172.21.15.5:0/3210000993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 msgr2=0x5651aa19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e2ffd700 1 -- 172.21.15.5:0/3210000993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651a9f743b0 con 0x5651aa1a1370 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e37fe700 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e3fff700 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa1ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1a1370 0x5651aa1a40a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f01d8016000 tx=0x7f01d8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e75f8c22c94fff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01c97fa700 1 -- 172.21.15.5:0/3210000993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d801f070 con 0x5651aa1a1370 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01c97fa700 1 -- 172.21.15.5:0/3210000993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8011b00 con 0x5651aa1a1370 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.501+0000 7f01c97fa700 1 -- 172.21.15.5:0/3210000993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8004040 con 0x5651aa1a1370 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 -- 172.21.15.5:0/3210000993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1a1370 msgr2=0x5651aa1a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1a1370 0x5651aa1a40a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f01d8016000 tx=0x7f01d8010000 comp rx=0 tx=0).stop 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 -- 172.21.15.5:0/3210000993 shutdown_connections 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa1ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 --2- 172.21.15.5:0/3210000993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1a1370 0x5651aa1a40a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 -- 172.21.15.5:0/3210000993 >> 172.21.15.5:0/3210000993 conn(0x5651aa101210 msgr2=0x5651aa1e8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 -- 172.21.15.5:0/3210000993 shutdown_connections 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.502+0000 7f01f622eb80 1 -- 172.21.15.5:0/3210000993 wait complete. 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.503+0000 7f01f622eb80 1 -- start start 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.503+0000 7f01f622eb80 1 Processor -- start 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa1d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1ee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01f622eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651aa08f560 con 0x5651aa1cddd0 2024-05-08T23:24:24.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01f622eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651aa08f200 con 0x5651aa1c67a0 2024-05-08T23:24:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01f622eb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651a9f766a0 con 0x5651aa1a1370 2024-05-08T23:24:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01f622eb80 10 client.0 _ll_register_callbacks cb 0x5651aa098b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1ee300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53196/0 (socket says 172.21.15.5:53196) 2024-05-08T23:24:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e3fff700 1 -- 172.21.15.5:0/109373393 learned_addr learned my addr 172.21.15.5:0/109373393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa1d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e37fe700 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e3fff700 1 -- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 msgr2=0x5651aa1f0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e3fff700 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1f0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e3fff700 1 -- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 msgr2=0x5651aa1d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e3fff700 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa1d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e3fff700 1 -- 172.21.15.5:0/109373393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e409b000 con 0x5651aa1cddd0 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.504+0000 7f01e37fe700 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01e3fff700 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1ee300 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0606d0 tx=0x7f01dc058d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9de7e8b15208e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01c27fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01c27fc700 1 -- 172.21.15.5:0/109373393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc070070 con 0x5651aa1cddd0 2024-05-08T23:24:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01c27fc700 1 -- 172.21.15.5:0/109373393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01dc0701d0 con 0x5651aa1cddd0 2024-05-08T23:24:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 1 -- 172.21.15.5:0/109373393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651aa04c270 con 0x5651aa1cddd0 2024-05-08T23:24:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01c27fc700 1 -- 172.21.15.5:0/109373393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01dc05c3b0 con 0x5651aa1cddd0 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 10 client.6243 fetch_fsmap learned FSMap version 41 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 1 -- 172.21.15.5:0/109373393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x7f01e40b19c0 con 0x5651aa1cddd0 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01c27fc700 1 -- 172.21.15.5:0/109373393 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651aa04c270 con 0x5651aa1cddd0 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01c27fc700 1 -- 172.21.15.5:0/109373393 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f01dc060770 con 0x5651aa1cddd0 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 10 client.6243 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 10 client.6243 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 20 client.6243 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5651aa1c4710 0x5651aa1ee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 4 client.6243 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.505+0000 7f01f622eb80 1 -- 172.21.15.5:0/109373393 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651aa1727e0 con 0x5651aa1c4710 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.506+0000 7f01e37fe700 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5651aa1c4710 0x5651aa1ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.506+0000 7f01e37fe700 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5651aa1c4710 0x5651aa1ee840 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c14cdad2b4344f1d server_cookie=274b7c26fc5a83e1 in_seq=0 out_seq=0 2024-05-08T23:24:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.506+0000 7f01c27fc700 10 client.6243 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01c27fc700 1 -- 172.21.15.5:0/109373393 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651aa1727e0 con 0x5651aa1c4710 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01c27fc700 10 client.6243 handle_command_reply: tid=0 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 client.6243 shutdown 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 -- 172.21.15.5:0/109373393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 msgr2=0x5651aa1ee300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1ee300 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0606d0 tx=0x7f01dc058d50 comp rx=0 tx=0).stop 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 -- 172.21.15.5:0/109373393 shutdown_connections 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5651aa1c4710 0x5651aa1ee840 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1ee300 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 --2- 172.21.15.5:0/109373393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa1d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01c27fc700 0 client.6243 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 -- 172.21.15.5:0/109373393 >> 172.21.15.5:0/109373393 conn(0x5651aa101210 msgr2=0x5651aa1ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 -- 172.21.15.5:0/109373393 shutdown_connections 2024-05-08T23:24:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 1 -- 172.21.15.5:0/109373393 wait complete. 2024-05-08T23:24:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.507+0000 7f01f622eb80 20 client.6243 trim_cache size 0 max 16384 2024-05-08T23:24:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.525+0000 7f01f622eb80 1 Processor -- start 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.525+0000 7f01f622eb80 1 -- start start 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa1cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01f622eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651aa1e8280 con 0x5651aa1cddd0 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01f622eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651aa209c70 con 0x5651aa1c67a0 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01f622eb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651aa1cba00 con 0x5651aa1a1370 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53198/0 (socket says 172.21.15.5:53198) 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01e3fff700 1 -- 172.21.15.5:0/2365933576 learned_addr learned my addr 172.21.15.5:0/2365933576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa1cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.526+0000 7f01e37fe700 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01e3fff700 1 -- 172.21.15.5:0/2365933576 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 msgr2=0x5651aa1e9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01e3fff700 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1e9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01e3fff700 1 -- 172.21.15.5:0/2365933576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 msgr2=0x5651aa1cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01e3fff700 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa1cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01e3fff700 1 -- 172.21.15.5:0/2365933576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651a9f20b60 con 0x5651aa1cddd0 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01e3fff700 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1cf4b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f01dc089c00 tx=0x7f01dc05a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804465aa13cdb041 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01c9ffb700 1 -- 172.21.15.5:0/2365933576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc070040 con 0x5651aa1cddd0 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01c9ffb700 1 -- 172.21.15.5:0/2365933576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc007a00 con 0x5651aa1cddd0 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.527+0000 7f01c9ffb700 1 -- 172.21.15.5:0/2365933576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00a090 con 0x5651aa1cddd0 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 -- 172.21.15.5:0/2365933576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 msgr2=0x5651aa1cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1cf4b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f01dc089c00 tx=0x7f01dc05a000 comp rx=0 tx=0).stop 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 -- 172.21.15.5:0/2365933576 shutdown_connections 2024-05-08T23:24:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1cddd0 0x5651aa1cf4b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1a1370 0x5651aa1e9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 --2- 172.21.15.5:0/2365933576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1c67a0 0x5651aa1cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 -- 172.21.15.5:0/2365933576 >> 172.21.15.5:0/2365933576 conn(0x5651aa101210 msgr2=0x5651aa19e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 -- 172.21.15.5:0/2365933576 shutdown_connections 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 -- 172.21.15.5:0/2365933576 wait complete. 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 -- start start 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.528+0000 7f01f622eb80 1 Processor -- start 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1a1370 0x5651aa1de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1c67a0 0x5651aa2509f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01f622eb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa19d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01f622eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651aa1ec6e0 con 0x5651aa1c67a0 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01f622eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651aa1decc0 con 0x5651aa1a1370 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01f622eb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651aa1d0860 con 0x5651aa1cddd0 2024-05-08T23:24:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01f622eb80 10 client.0 _ll_register_callbacks cb 0x5651aa098b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1c67a0 0x5651aa2509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1c67a0 0x5651aa2509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53208/0 (socket says 172.21.15.5:53208) 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e2ffd700 1 -- 172.21.15.5:0/4119439010 learned_addr learned my addr 172.21.15.5:0/4119439010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e37fe700 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1a1370 0x5651aa1de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e3fff700 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e2ffd700 1 -- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 msgr2=0x5651aa19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e2ffd700 1 -- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1a1370 msgr2=0x5651aa1de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1a1370 0x5651aa1de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e2ffd700 1 -- 172.21.15.5:0/4119439010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e40b19c0 con 0x5651aa1c67a0 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.530+0000 7f01e3fff700 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01e2ffd700 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1c67a0 0x5651aa2509f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f01d8014040 tx=0x7f01d800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18bb0662da7e261 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01c27fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01c27fc700 1 -- 172.21.15.5:0/4119439010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d801f040 con 0x5651aa1c67a0 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01c27fc700 1 -- 172.21.15.5:0/4119439010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01d8004450 con 0x5651aa1c67a0 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 1 -- 172.21.15.5:0/4119439010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651aa0ebec0 con 0x5651aa1c67a0 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01c27fc700 1 -- 172.21.15.5:0/4119439010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d80036a0 con 0x5651aa1c67a0 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 10 client.6249 fetch_fsmap learned FSMap version 41 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 1 -- 172.21.15.5:0/4119439010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5651aa0b9560 con 0x5651aa1c67a0 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01c27fc700 1 -- 172.21.15.5:0/4119439010 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651aa0ebec0 con 0x5651aa1c67a0 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01c27fc700 1 -- 172.21.15.5:0/4119439010 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f01d8003800 con 0x5651aa1c67a0 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 10 client.6249 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 10 client.6249 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.b 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 20 client.6249 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5651aa1ed1a0 0x5651aa19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 4 client.6249 mds_command: new command op to 5566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01f622eb80 1 -- 172.21.15.5:0/4119439010 --> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5651a9f41cf0 con 0x5651aa1ed1a0 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.531+0000 7f01e37fe700 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5651aa1ed1a0 0x5651aa19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01e37fe700 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5651aa1ed1a0 0x5651aa19dab0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93c6a535ddf8c6bc server_cookie=e4b01dfaba43969f in_seq=0 out_seq=0 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01c27fc700 10 client.6249 ms_handle_connect on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01c27fc700 1 -- 172.21.15.5:0/4119439010 <== mds.1 v2:172.21.15.49:6834/962044603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5651a9f41cf0 con 0x5651aa1ed1a0 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01c27fc700 10 client.6249 handle_command_reply: tid=0 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 client.6249 shutdown 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 -- 172.21.15.5:0/4119439010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1c67a0 msgr2=0x5651aa2509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1c67a0 0x5651aa2509f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f01d8014040 tx=0x7f01d800e000 comp rx=0 tx=0).stop 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 -- 172.21.15.5:0/4119439010 shutdown_connections 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:6834/962044603,v1:172.21.15.49:6835/962044603] conn(0x5651aa1ed1a0 0x5651aa19dab0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5651aa1cddd0 0x5651aa19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651aa1a1370 0x5651aa1de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 --2- 172.21.15.5:0/4119439010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651aa1c67a0 0x5651aa2509f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01c27fc700 0 client.6249 ms_handle_reset on v2:172.21.15.49:6834/962044603 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 -- 172.21.15.5:0/4119439010 >> 172.21.15.5:0/4119439010 conn(0x5651aa101210 msgr2=0x5651aa1cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 -- 172.21.15.5:0/4119439010 shutdown_connections 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.532+0000 7f01f622eb80 1 -- 172.21.15.5:0/4119439010 wait complete. 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01f622eb80 20 client.6249 trim_cache size 0 max 16384 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-08T23:24:24.544 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 -- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01d006ac90 msgr2=0x7f01d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01d006ac90 0x7f01d006d150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f01d4005910 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 -- 172.21.15.5:0/1768502728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 msgr2=0x7f01e415f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e415f9a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00cef0 tx=0x7f01dc00a340 comp rx=0 tx=0).stop 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 -- 172.21.15.5:0/1768502728 shutdown_connections 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f01d006ac90 0x7f01d006d150 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01e40b4d50 0x7f01e415f9a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e406aea0 0x7f01e415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 --2- 172.21.15.5:0/1768502728 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f01e40b0d80 0x7f01e415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 -- 172.21.15.5:0/1768502728 >> 172.21.15.5:0/1768502728 conn(0x7f01e4099a50 msgr2=0x7f01e406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 -- 172.21.15.5:0/1768502728 shutdown_connections 2024-05-08T23:24:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.533+0000 7f01ea074700 1 -- 172.21.15.5:0/1768502728 wait complete. 2024-05-08T23:24:24.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:24:24.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 scrub status 2024-05-08T23:24:24.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.684+0000 7fb59c873700 1 Processor -- start 2024-05-08T23:24:24.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.684+0000 7fb59c873700 1 -- start start 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.685+0000 7fb59c873700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5980d13a0 0x7fb5980d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.685+0000 7fb59c873700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.685+0000 7fb59c873700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5980b1200 con 0x7fb59806bd60 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.685+0000 7fb59c873700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5980d1cc0 con 0x7fb5980d13a0 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.685+0000 7fb59c873700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5980ce5c0 con 0x7fb5980cdc90 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb59659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5980d13a0 0x7fb5980d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:56476/0 (socket says 172.21.15.5:56476) 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb596d9d700 1 -- 172.21.15.5:0/483937759 learned_addr learned my addr 172.21.15.5:0/483937759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb59659c700 1 --2- 172.21.15.5:0/483937759 >> v2:172.21.15.49:3300/0 conn(0x7fb5980d13a0 0x7fb5980d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57176/0 (socket says 172.21.15.5:57176) 2024-05-08T23:24:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb596d9d700 1 -- 172.21.15.5:0/483937759 >> v2:172.21.15.49:3300/0 conn(0x7fb5980d13a0 msgr2=0x7fb5980d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb596d9d700 1 --2- 172.21.15.5:0/483937759 >> v2:172.21.15.49:3300/0 conn(0x7fb5980d13a0 0x7fb5980d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb596d9d700 1 -- 172.21.15.5:0/483937759 >> v1:172.21.15.49:6789/0 conn(0x7fb59806bd60 legacy=0x7fb5980cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb596d9d700 1 -- 172.21.15.5:0/483937759 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980ce8a0 con 0x7fb5980cdc90 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb59659c700 1 --2- 172.21.15.5:0/483937759 >> v2:172.21.15.49:3300/0 conn(0x7fb5980d13a0 0x7fb5980d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.686+0000 7fb596d9d700 1 --2- 172.21.15.5:0/483937759 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b19c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb590009e50 tx=0x7fb59000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960d24ac999344d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59559a700 1 -- 172.21.15.5:0/483937759 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59000e070 con 0x7fb5980cdc90 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59559a700 1 -- 172.21.15.5:0/483937759 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59000b910 con 0x7fb5980cdc90 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59559a700 1 -- 172.21.15.5:0/483937759 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59000bcc0 con 0x7fb5980cdc90 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59c873700 1 -- 172.21.15.5:0/483937759 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 msgr2=0x7fb5980b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59c873700 1 --2- 172.21.15.5:0/483937759 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b19c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb590009e50 tx=0x7fb59000b330 comp rx=0 tx=0).stop 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59c873700 1 -- 172.21.15.5:0/483937759 shutdown_connections 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59c873700 1 --2- 172.21.15.5:0/483937759 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b19c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59c873700 1 --2- 172.21.15.5:0/483937759 >> v2:172.21.15.49:3300/0 conn(0x7fb5980d13a0 0x7fb5980d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59c873700 1 -- 172.21.15.5:0/483937759 >> 172.21.15.5:0/483937759 conn(0x7fb59808afe0 msgr2=0x7fb59808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59c873700 1 -- 172.21.15.5:0/483937759 shutdown_connections 2024-05-08T23:24:24.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.687+0000 7fb59c873700 1 -- 172.21.15.5:0/483937759 wait complete. 2024-05-08T23:24:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.689+0000 7fb59c873700 1 Processor -- start 2024-05-08T23:24:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.689+0000 7fb59c873700 1 -- start start 2024-05-08T23:24:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb59c873700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59806bd60 0x7fb5980b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb59c873700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb59c873700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5980d13a0 0x7fb59815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb59c873700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59815cad0 con 0x7fb5980d13a0 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb59c873700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5980c4f20 con 0x7fb59806bd60 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb59c873700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5980b4540 con 0x7fb5980cdc90 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5980d13a0 0x7fb59815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5980d13a0 0x7fb59815bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53224/0 (socket says 172.21.15.5:53224) 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 -- 172.21.15.5:0/1049406881 learned_addr learned my addr 172.21.15.5:0/1049406881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb59659c700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59806bd60 0x7fb5980b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb595d9b700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 -- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 msgr2=0x7fb5980b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 -- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59806bd60 msgr2=0x7fb5980b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59806bd60 0x7fb5980b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 -- 172.21.15.5:0/1049406881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5880094a0 con 0x7fb5980d13a0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.690+0000 7fb596d9d700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5980d13a0 0x7fb59815bf10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb5980ce700 tx=0x7fb590004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.691+0000 7fb586ffd700 1 -- 172.21.15.5:0/1049406881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590004800 con 0x7fb5980d13a0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.691+0000 7fb586ffd700 1 -- 172.21.15.5:0/1049406881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590004960 con 0x7fb5980d13a0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.691+0000 7fb59c873700 1 -- 172.21.15.5:0/1049406881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb590009b30 con 0x7fb5980d13a0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.691+0000 7fb586ffd700 1 -- 172.21.15.5:0/1049406881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59001c680 con 0x7fb5980d13a0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.691+0000 7fb59c873700 1 -- 172.21.15.5:0/1049406881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5980b4b00 con 0x7fb5980d13a0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.692+0000 7fb586ffd700 1 -- 172.21.15.5:0/1049406881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb59001c7e0 con 0x7fb5980d13a0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.693+0000 7fb586ffd700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb57806ac90 0x7fb57806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.693+0000 7fb586ffd700 1 -- 172.21.15.5:0/1049406881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb59008af40 con 0x7fb5980d13a0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.693+0000 7fb59659c700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb57806ac90 0x7fb57806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.693+0000 7fb59659c700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb57806ac90 0x7fb57806d150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5980b0170 tx=0x7fb588009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.695+0000 7fb5a8a2db80 1 Processor -- start 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.695+0000 7fb5a8a2db80 1 -- start start 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c2dd0 0x55e13d3c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d395720 0x55e13d398970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 0x55e13d398eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13d2ef150 con 0x55e13d395720 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13d2ecb10 con 0x55e13d3c2dd0 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e13d2e8400 con 0x55e13d3c0570 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb595d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d395720 0x55e13d398970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb595d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d395720 0x55e13d398970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53226/0 (socket says 172.21.15.5:53226) 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb595d9b700 1 -- 172.21.15.5:0/2326779314 learned_addr learned my addr 172.21.15.5:0/2326779314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb59659c700 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c2dd0 0x55e13d3c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb596d9d700 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 0x55e13d398eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb595d9b700 1 -- 172.21.15.5:0/2326779314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 msgr2=0x55e13d398eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb595d9b700 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 0x55e13d398eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb595d9b700 1 -- 172.21.15.5:0/2326779314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c2dd0 msgr2=0x55e13d3c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb595d9b700 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c2dd0 0x55e13d3c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.696+0000 7fb595d9b700 1 -- 172.21.15.5:0/2326779314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e13d1693b0 con 0x55e13d395720 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.697+0000 7fb595d9b700 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d395720 0x55e13d398970 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0016a0 tx=0x7fb58c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1319e702c6b3a51d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.697+0000 7fb577fff700 1 -- 172.21.15.5:0/2326779314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c005320 con 0x55e13d395720 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.697+0000 7fb577fff700 1 -- 172.21.15.5:0/2326779314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c005480 con 0x55e13d395720 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.697+0000 7fb577fff700 1 -- 172.21.15.5:0/2326779314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c003880 con 0x55e13d395720 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.697+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/2326779314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d395720 msgr2=0x55e13d398970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.697+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d395720 0x55e13d398970 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0016a0 tx=0x7fb58c007ec0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/2326779314 shutdown_connections 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 0x55e13d398eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c2dd0 0x55e13d3c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/2326779314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d395720 0x55e13d398970 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/2326779314 >> 172.21.15.5:0/2326779314 conn(0x55e13d2f6210 msgr2=0x55e13d3dddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/2326779314 shutdown_connections 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/2326779314 wait complete. 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 -- start start 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.698+0000 7fb5a8a2db80 1 Processor -- start 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.699+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d395720 0x55e13d3d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.699+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 0x55e13d3d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.699+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.699+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13d284560 con 0x55e13d3c2dd0 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.699+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13d284200 con 0x55e13d395720 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.699+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e13d16b6a0 con 0x55e13d3c0570 2024-05-08T23:24:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.699+0000 7fb5a8a2db80 10 client.0 _ll_register_callbacks cb 0x55e13d28db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d395720 0x55e13d3d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb595d9b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 0x55e13d3d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d395720 0x55e13d3d3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57202/0 (socket says 172.21.15.5:57202) 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 -- 172.21.15.5:0/1927642643 learned_addr learned my addr 172.21.15.5:0/1927642643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 -- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 msgr2=0x55e13d3d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 0x55e13d3d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 -- 172.21.15.5:0/1927642643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 msgr2=0x55e13d3bcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 -- 172.21.15.5:0/1927642643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5980b1200 con 0x55e13d395720 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb596d9d700 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb59659c700 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d395720 0x55e13d3d3be0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb588014b90 tx=0x7fb588004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1285c6b8cf5c7121 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb5757fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-08T23:24:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb5757fa700 1 -- 172.21.15.5:0/1927642643 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588013a40 con 0x55e13d395720 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/1927642643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e13d241270 con 0x55e13d395720 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.700+0000 7fb5757fa700 1 -- 172.21.15.5:0/1927642643 <== mon.1 v2:172.21.15.49:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb58801c040 con 0x55e13d395720 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5757fa700 1 -- 172.21.15.5:0/1927642643 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb588013ba0 con 0x55e13d395720 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5a8a2db80 10 client.5983 fetch_fsmap learned FSMap version 41 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/1927642643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55e13d115b60 con 0x55e13d395720 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5757fa700 1 -- 172.21.15.5:0/1927642643 <== mon.1 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e13d241270 con 0x55e13d395720 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5757fa700 1 -- 172.21.15.5:0/1927642643 <== mon.1 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fb588013d00 con 0x55e13d395720 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5a8a2db80 10 client.5983 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5a8a2db80 10 client.5983 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5a8a2db80 20 client.5983 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55e13d3d2430 0x55e13d3c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5a8a2db80 4 client.5983 mds_command: new command op to 5180 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/1927642643 --> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e13d3677e0 con 0x55e13d3d2430 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.701+0000 7fb595d9b700 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55e13d3d2430 0x55e13d3c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5757fa700 10 client.5983 ms_handle_connect on v2:172.21.15.49:6836/350829837 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb595d9b700 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55e13d3d2430 0x55e13d3c6140 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb39261afd0835c0 server_cookie=e63888a47a476b56 in_seq=0 out_seq=0 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5757fa700 1 -- 172.21.15.5:0/1927642643 <== mds.2 v2:172.21.15.49:6836/350829837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e13d3677e0 con 0x55e13d3d2430 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5757fa700 10 client.5983 handle_command_reply: tid=0 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 client.5983 shutdown 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d395720 msgr2=0x55e13d3d3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d395720 0x55e13d3d3be0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb588014b90 tx=0x7fb588004750 comp rx=0 tx=0).stop 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/1927642643 shutdown_connections 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55e13d3d2430 0x55e13d3c6140 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d395720 0x55e13d3d3be0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/1927642643 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d3c0570 0x55e13d3d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/1927642643 >> 172.21.15.5:0/1927642643 conn(0x55e13d2f6210 msgr2=0x55e13d3df250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.702+0000 7fb5757fa700 0 client.5983 ms_handle_reset on v2:172.21.15.49:6836/350829837 2024-05-08T23:24:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.703+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/1927642643 shutdown_connections 2024-05-08T23:24:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.703+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/1927642643 wait complete. 2024-05-08T23:24:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.703+0000 7fb5a8a2db80 20 client.5983 trim_cache size 0 max 16384 2024-05-08T23:24:24.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.721+0000 7fb5a8a2db80 1 Processor -- start 2024-05-08T23:24:24.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.721+0000 7fb5a8a2db80 1 -- start start 2024-05-08T23:24:24.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.721+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 0x55e13d3cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.721+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 0x55e13d3bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.721+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.721+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13d395f70 con 0x55e13d3c2dd0 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.721+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13d3bc7b0 con 0x55e13d3c0570 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.721+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e13d3bc510 con 0x55e13d395720 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53250/0 (socket says 172.21.15.5:53250) 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb595d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 0x55e13d3bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 -- 172.21.15.5:0/159095148 learned_addr learned my addr 172.21.15.5:0/159095148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb59659c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 0x55e13d3cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 -- 172.21.15.5:0/159095148 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 msgr2=0x55e13d3cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 --2- 172.21.15.5:0/159095148 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 0x55e13d3cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 -- 172.21.15.5:0/159095148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 msgr2=0x55e13d3bfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 --2- 172.21.15.5:0/159095148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 0x55e13d3bfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 -- 172.21.15.5:0/159095148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980c9360 con 0x55e13d3c2dd0 2024-05-08T23:24:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb596d9d700 1 --2- 172.21.15.5:0/159095148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bbfd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb59000b370 tx=0x7fb590009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd580bc1c97b8215 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb584ff9700 1 -- 172.21.15.5:0/159095148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590089180 con 0x55e13d3c2dd0 2024-05-08T23:24:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb584ff9700 1 -- 172.21.15.5:0/159095148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590059d90 con 0x55e13d3c2dd0 2024-05-08T23:24:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.722+0000 7fb584ff9700 1 -- 172.21.15.5:0/159095148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59005dbc0 con 0x55e13d3c2dd0 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/159095148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 msgr2=0x55e13d3bbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/159095148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bbfd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb59000b370 tx=0x7fb590009a30 comp rx=0 tx=0).stop 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/159095148 shutdown_connections 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/159095148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3bbfd0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/159095148 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 0x55e13d3cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/159095148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 0x55e13d3bfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/159095148 >> 172.21.15.5:0/159095148 conn(0x55e13d2f6210 msgr2=0x55e13d38f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/159095148 shutdown_connections 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/159095148 wait complete. 2024-05-08T23:24:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.723+0000 7fb5a8a2db80 1 -- start start 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.724+0000 7fb5a8a2db80 1 Processor -- start 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 0x55e13d3dd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 0x55e13d3d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5a8a2db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13d435f30 con 0x55e13d3c2dd0 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13d3be150 con 0x55e13d3c0570 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5a8a2db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e13d3be3a0 con 0x55e13d395720 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5a8a2db80 10 client.0 _ll_register_callbacks cb 0x55e13d28db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53256/0 (socket says 172.21.15.5:53256) 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 -- 172.21.15.5:0/4264326977 learned_addr learned my addr 172.21.15.5:0/4264326977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb595d9b700 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 0x55e13d3d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb59659c700 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 0x55e13d3dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 -- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 msgr2=0x55e13d3dd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 0x55e13d3dd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 -- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 msgr2=0x55e13d3d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 0x55e13d3d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 -- 172.21.15.5:0/4264326977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5980b4b00 con 0x55e13d3c2dd0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5757fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb596d9d700 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3d0bf0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb590056290 tx=0x7fb590009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f6ea90bd6ac9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5757fa700 1 -- 172.21.15.5:0/4264326977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590059480 con 0x55e13d3c2dd0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5757fa700 1 -- 172.21.15.5:0/4264326977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb590062180 con 0x55e13d3c2dd0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.725+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/4264326977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb58c005480 con 0x55e13d3c2dd0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5757fa700 1 -- 172.21.15.5:0/4264326977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb590089180 con 0x55e13d3c2dd0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5a8a2db80 10 client.6264 fetch_fsmap learned FSMap version 41 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/4264326977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55e13d2ae560 con 0x55e13d3c2dd0 2024-05-08T23:24:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5757fa700 1 -- 172.21.15.5:0/4264326977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c005480 con 0x55e13d3c2dd0 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5757fa700 1 -- 172.21.15.5:0/4264326977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fb59006dbd0 con 0x55e13d3c2dd0 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5a8a2db80 10 client.6264 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5a8a2db80 10 client.6264 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.d 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5a8a2db80 20 client.6264 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55e13d3bdca0 0x55e13d398250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5a8a2db80 4 client.6264 mds_command: new command op to 5180 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/4264326977 --> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e13d2e0ec0 con 0x55e13d3bdca0 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.726+0000 7fb59659c700 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55e13d3bdca0 0x55e13d398250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb59659c700 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55e13d3bdca0 0x55e13d398250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f79d4997105f9f server_cookie=138d2506e0bfe07 in_seq=0 out_seq=0 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5757fa700 10 client.6264 ms_handle_connect on v2:172.21.15.49:6836/350829837 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5757fa700 1 -- 172.21.15.5:0/4264326977 <== mds.2 v2:172.21.15.49:6836/350829837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e13d2e0ec0 con 0x55e13d3bdca0 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5757fa700 10 client.6264 handle_command_reply: tid=0 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 client.6264 shutdown 2024-05-08T23:24:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/4264326977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 msgr2=0x55e13d3d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3d0bf0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb590056290 tx=0x7fb590009810 comp rx=0 tx=0).stop 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/4264326977 shutdown_connections 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13d3c2dd0 0x55e13d3d0bf0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e13d395720 0x55e13d3dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:6836/350829837,v1:172.21.15.49:6837/350829837] conn(0x55e13d3bdca0 0x55e13d398250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 --2- 172.21.15.5:0/4264326977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e13d3c0570 0x55e13d3d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/4264326977 >> 172.21.15.5:0/4264326977 conn(0x55e13d2f6210 msgr2=0x55e13d3bcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5757fa700 0 client.6264 ms_handle_reset on v2:172.21.15.49:6836/350829837 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/4264326977 shutdown_connections 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 1 -- 172.21.15.5:0/4264326977 wait complete. 2024-05-08T23:24:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.727+0000 7fb5a8a2db80 20 client.6264 trim_cache size 0 max 16384 2024-05-08T23:24:24.734 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:24.734 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-08T23:24:24.734 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-08T23:24:24.734 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 -- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb57806ac90 msgr2=0x7fb57806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb57806ac90 0x7fb57806d150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5980b0170 tx=0x7fb588009210 comp rx=0 tx=0).stop 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 -- 172.21.15.5:0/1049406881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5980d13a0 msgr2=0x7fb59815bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5980d13a0 0x7fb59815bf10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb5980ce700 tx=0x7fb590004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 -- 172.21.15.5:0/1049406881 shutdown_connections 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb57806ac90 0x7fb57806d150 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5980d13a0 0x7fb59815bf10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59806bd60 0x7fb5980b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 --2- 172.21.15.5:0/1049406881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb5980cdc90 0x7fb5980b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 -- 172.21.15.5:0/1049406881 >> 172.21.15.5:0/1049406881 conn(0x7fb59808afe0 msgr2=0x7fb5980cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 -- 172.21.15.5:0/1049406881 shutdown_connections 2024-05-08T23:24:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.728+0000 7fb59c873700 1 -- 172.21.15.5:0/1049406881 wait complete. 2024-05-08T23:24:24.741 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:24:24.741 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-08T23:24:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.879+0000 7f636b4b0700 1 Processor -- start 2024-05-08T23:24:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.879+0000 7f636b4b0700 1 -- start start 2024-05-08T23:24:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.880+0000 7f636b4b0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6364099a50 0x7f63640d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.880+0000 7f636b4b0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636406f930 con 0x7f6364099e30 2024-05-08T23:24:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.880+0000 7f636b4b0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63640cfdf0 con 0x7f63640cba90 2024-05-08T23:24:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.880+0000 7f636b4b0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63640d00d0 con 0x7f6364099a50 2024-05-08T23:24:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.880+0000 7f636924c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6364099a50 0x7f63640d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.880+0000 7f6368a4b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6364099e30 0x7f636409a790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57186/0 (socket says 172.21.15.5:57186) 2024-05-08T23:24:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.880+0000 7f6368a4b700 1 -- 172.21.15.5:0/83410729 learned_addr learned my addr 172.21.15.5:0/83410729 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.881+0000 7f636924c700 1 -- 172.21.15.5:0/83410729 >> v1:172.21.15.49:6789/0 conn(0x7f63640cba90 legacy=0x7f636406fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.881+0000 7f636924c700 1 -- 172.21.15.5:0/83410729 >> v1:172.21.15.5:6789/0 conn(0x7f6364099e30 legacy=0x7f636409a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.881+0000 7f636924c700 1 -- 172.21.15.5:0/83410729 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63640d03b0 con 0x7f6364099a50 2024-05-08T23:24:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.881+0000 7f636924c700 1 --2- 172.21.15.5:0/83410729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364099a50 0x7f63640d0690 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6360009840 tx=0x7f6360009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386c28be281dffbe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.881+0000 7f635bfff700 1 -- 172.21.15.5:0/83410729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360011830 con 0x7f6364099a50 2024-05-08T23:24:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.881+0000 7f635bfff700 1 -- 172.21.15.5:0/83410729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6360011990 con 0x7f6364099a50 2024-05-08T23:24:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.881+0000 7f635bfff700 1 -- 172.21.15.5:0/83410729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360011d20 con 0x7f6364099a50 2024-05-08T23:24:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.882+0000 7f636b4b0700 1 -- 172.21.15.5:0/83410729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364099a50 msgr2=0x7f63640d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.882+0000 7f636b4b0700 1 --2- 172.21.15.5:0/83410729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364099a50 0x7f63640d0690 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6360009840 tx=0x7f6360009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.882+0000 7f636b4b0700 1 -- 172.21.15.5:0/83410729 shutdown_connections 2024-05-08T23:24:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.882+0000 7f636b4b0700 1 --2- 172.21.15.5:0/83410729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364099a50 0x7f63640d0690 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.882+0000 7f636b4b0700 1 -- 172.21.15.5:0/83410729 >> 172.21.15.5:0/83410729 conn(0x7f63640b2f30 msgr2=0x7f63640b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.882+0000 7f636b4b0700 1 -- 172.21.15.5:0/83410729 shutdown_connections 2024-05-08T23:24:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.882+0000 7f636b4b0700 1 -- 172.21.15.5:0/83410729 wait complete. 2024-05-08T23:24:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.883+0000 7f636b4b0700 1 Processor -- start 2024-05-08T23:24:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.883+0000 7f636b4b0700 1 -- start start 2024-05-08T23:24:24.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f636b4b0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6364099a50 0x7f63640b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f636b4b0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6364099e30 0x7f636415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f636b4b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63640cba90 0x7f63640c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f636b4b0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636415a7f0 con 0x7f63640cba90 2024-05-08T23:24:24.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f636b4b0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63640c6ed0 con 0x7f6364099a50 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f636b4b0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63640cac50 con 0x7f6364099e30 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6369a4d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63640cba90 0x7f63640c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6369a4d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63640cba90 0x7f63640c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53270/0 (socket says 172.21.15.5:53270) 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6369a4d700 1 -- 172.21.15.5:0/2682973651 learned_addr learned my addr 172.21.15.5:0/2682973651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f636924c700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6364099a50 0x7f63640b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6368a4b700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6364099e30 0x7f636415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6369a4d700 1 -- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6364099e30 msgr2=0x7f636415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6369a4d700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6364099e30 0x7f636415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6369a4d700 1 -- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6364099a50 msgr2=0x7f63640b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6369a4d700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6364099a50 0x7f63640b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6369a4d700 1 -- 172.21.15.5:0/2682973651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600094a0 con 0x7f63640cba90 2024-05-08T23:24:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.884+0000 7f6368a4b700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6364099e30 0x7f636415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.885+0000 7f6369a4d700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63640cba90 0x7f63640c6990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f635c00cb50 tx=0x7f635c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.885+0000 7f6359ffb700 1 -- 172.21.15.5:0/2682973651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c003c00 con 0x7f63640cba90 2024-05-08T23:24:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.885+0000 7f6359ffb700 1 -- 172.21.15.5:0/2682973651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c003d60 con 0x7f63640cba90 2024-05-08T23:24:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.885+0000 7f636b4b0700 1 -- 172.21.15.5:0/2682973651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63640c5eb0 con 0x7f63640cba90 2024-05-08T23:24:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.885+0000 7f6359ffb700 1 -- 172.21.15.5:0/2682973651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c00aac0 con 0x7f63640cba90 2024-05-08T23:24:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.885+0000 7f636b4b0700 1 -- 172.21.15.5:0/2682973651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63640c6470 con 0x7f63640cba90 2024-05-08T23:24:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.887+0000 7f6359ffb700 1 -- 172.21.15.5:0/2682973651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f635c00ac20 con 0x7f63640cba90 2024-05-08T23:24:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.887+0000 7f6359ffb700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f634806ac90 0x7f634806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.887+0000 7f6359ffb700 1 -- 172.21.15.5:0/2682973651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f635c0892d0 con 0x7f63640cba90 2024-05-08T23:24:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.887+0000 7f636924c700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f634806ac90 0x7f634806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.888+0000 7f636924c700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f634806ac90 0x7f634806d150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f63600090b0 tx=0x7f6360008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.889+0000 7f637766ab80 1 Processor -- start 2024-05-08T23:24:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.889+0000 7f637766ab80 1 -- start start 2024-05-08T23:24:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f637766ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f8057a0 0x56018f7de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f637766ab80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f7e0370 0x56018f7e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f637766ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f637766ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018f739150 con 0x56018f8057a0 2024-05-08T23:24:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f637766ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018f736b10 con 0x56018f80cdd0 2024-05-08T23:24:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f637766ab80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56018f732400 con 0x56018f7e0370 2024-05-08T23:24:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f636924c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f8057a0 0x56018f7de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f636924c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f8057a0 0x56018f7de110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53272/0 (socket says 172.21.15.5:53272) 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f636924c700 1 -- 172.21.15.5:0/3067450277 learned_addr learned my addr 172.21.15.5:0/3067450277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f6369a4d700 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.890+0000 7f6368a4b700 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f7e0370 0x56018f7e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f636924c700 1 -- 172.21.15.5:0/3067450277 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f7e0370 msgr2=0x56018f7e30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f636924c700 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f7e0370 0x56018f7e30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f636924c700 1 -- 172.21.15.5:0/3067450277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 msgr2=0x56018f80d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f636924c700 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f636924c700 1 -- 172.21.15.5:0/3067450277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56018f5b33b0 con 0x56018f8057a0 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f636924c700 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f8057a0 0x56018f7de110 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f63640745f0 tx=0x7f6360010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d77d837984af122 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f634effd700 1 -- 172.21.15.5:0/3067450277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636002c070 con 0x56018f8057a0 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f634effd700 1 -- 172.21.15.5:0/3067450277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63600036a0 con 0x56018f8057a0 2024-05-08T23:24:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.891+0000 7f634effd700 1 -- 172.21.15.5:0/3067450277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360020070 con 0x56018f8057a0 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 -- 172.21.15.5:0/3067450277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f8057a0 msgr2=0x56018f7de110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f8057a0 0x56018f7de110 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f63640745f0 tx=0x7f6360010000 comp rx=0 tx=0).stop 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 -- 172.21.15.5:0/3067450277 shutdown_connections 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f8057a0 0x56018f7de110 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 --2- 172.21.15.5:0/3067450277 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f7e0370 0x56018f7e30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 -- 172.21.15.5:0/3067450277 >> 172.21.15.5:0/3067450277 conn(0x56018f740210 msgr2=0x56018f827dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 -- 172.21.15.5:0/3067450277 shutdown_connections 2024-05-08T23:24:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 -- 172.21.15.5:0/3067450277 wait complete. 2024-05-08T23:24:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.892+0000 7f637766ab80 1 -- start start 2024-05-08T23:24:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.893+0000 7f637766ab80 1 Processor -- start 2024-05-08T23:24:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f637766ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f7dc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f637766ab80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f82aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f637766ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f7ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f637766ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018f6ce560 con 0x56018f7e0370 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f637766ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018f6ce200 con 0x56018f80cdd0 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f637766ab80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56018f5b56a0 con 0x56018f8057a0 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f637766ab80 10 client.0 _ll_register_callbacks cb 0x56018f6d7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f7dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f7dc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53282/0 (socket says 172.21.15.5:53282) 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 -- 172.21.15.5:0/1198661763 learned_addr learned my addr 172.21.15.5:0/1198661763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f6368a4b700 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f82aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f6369a4d700 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f7ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 -- 172.21.15.5:0/1198661763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 msgr2=0x56018f82aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f82aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 -- 172.21.15.5:0/1198661763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 msgr2=0x56018f7ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f7ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 -- 172.21.15.5:0/1198661763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63640d03b0 con 0x56018f7e0370 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f636924c700 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f7dc180 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6360009840 tx=0x7f63600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ad24619babb9d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f6343fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f6343fff700 1 -- 172.21.15.5:0/1198661763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636002c040 con 0x56018f7e0370 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f6343fff700 1 -- 172.21.15.5:0/1198661763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6360003de0 con 0x56018f7e0370 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.894+0000 7f637766ab80 1 -- 172.21.15.5:0/1198661763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56018f68b270 con 0x56018f7e0370 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f6343fff700 1 -- 172.21.15.5:0/1198661763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6360031470 con 0x56018f7e0370 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f637766ab80 10 client.6279 fetch_fsmap learned FSMap version 41 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f637766ab80 1 -- 172.21.15.5:0/1198661763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x56018f55fb60 con 0x56018f7e0370 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f6343fff700 1 -- 172.21.15.5:0/1198661763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56018f68b270 con 0x56018f7e0370 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f6343fff700 1 -- 172.21.15.5:0/1198661763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f6360038030 con 0x56018f7e0370 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f637766ab80 10 client.6279 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f637766ab80 10 client.6279 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f637766ab80 20 client.6279 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f637766ab80 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x56018f7ffe10 0x56018f80df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f637766ab80 4 client.6279 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f637766ab80 1 -- 172.21.15.5:0/1198661763 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56018f7b17e0 con 0x56018f7ffe10 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f6368a4b700 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x56018f7ffe10 0x56018f80df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f6343fff700 10 client.6279 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.895+0000 7f6368a4b700 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x56018f7ffe10 0x56018f80df70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f72a6ebc28f6621 server_cookie=5984ce12d5d8acea in_seq=0 out_seq=0 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f6343fff700 1 -- 172.21.15.5:0/1198661763 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56018f7b17e0 con 0x56018f7ffe10 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f6343fff700 10 client.6279 handle_command_reply: tid=0 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 client.6279 shutdown 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 -- 172.21.15.5:0/1198661763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 msgr2=0x56018f7dc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f7dc180 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6360009840 tx=0x7f63600047c0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 -- 172.21.15.5:0/1198661763 shutdown_connections 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f7ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f7dc180 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x56018f7ffe10 0x56018f80df70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 --2- 172.21.15.5:0/1198661763 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f82aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 -- 172.21.15.5:0/1198661763 >> 172.21.15.5:0/1198661763 conn(0x56018f740210 msgr2=0x56018f7dcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f6343fff700 0 client.6279 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 -- 172.21.15.5:0/1198661763 shutdown_connections 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.896+0000 7f637766ab80 1 -- 172.21.15.5:0/1198661763 wait complete. 2024-05-08T23:24:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.897+0000 7f637766ab80 20 client.6279 trim_cache size 0 max 16384 2024-05-08T23:24:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.915+0000 7f637766ab80 1 Processor -- start 2024-05-08T23:24:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.915+0000 7f637766ab80 1 -- start start 2024-05-08T23:24:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f637766ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f841230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f637766ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f8057a0 0x56018f829250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f637766ab80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f80cdd0 0x56018f7ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f637766ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018f828da0 con 0x56018f7e0370 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f637766ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018f82ad90 con 0x56018f8057a0 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f637766ab80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56018f7de7c0 con 0x56018f80cdd0 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f841230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f841230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53288/0 (socket says 172.21.15.5:53288) 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 -- 172.21.15.5:0/2888890156 learned_addr learned my addr 172.21.15.5:0/2888890156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 -- 172.21.15.5:0/2888890156 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f80cdd0 msgr2=0x56018f7ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 --2- 172.21.15.5:0/2888890156 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f80cdd0 0x56018f7ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 -- 172.21.15.5:0/2888890156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f8057a0 msgr2=0x56018f829250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 --2- 172.21.15.5:0/2888890156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f8057a0 0x56018f829250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 -- 172.21.15.5:0/2888890156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56018f6f8560 con 0x56018f7e0370 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f636924c700 1 --2- 172.21.15.5:0/2888890156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f841230 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6360011870 tx=0x7f636000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd697a14f9f97cac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.916+0000 7f634f7fe700 1 -- 172.21.15.5:0/2888890156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636002c070 con 0x56018f7e0370 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f634f7fe700 1 -- 172.21.15.5:0/2888890156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6360031470 con 0x56018f7e0370 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f634f7fe700 1 -- 172.21.15.5:0/2888890156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360003e00 con 0x56018f7e0370 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 -- 172.21.15.5:0/2888890156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 msgr2=0x56018f841230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 --2- 172.21.15.5:0/2888890156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f841230 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6360011870 tx=0x7f636000b050 comp rx=0 tx=0).stop 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 -- 172.21.15.5:0/2888890156 shutdown_connections 2024-05-08T23:24:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 --2- 172.21.15.5:0/2888890156 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f80cdd0 0x56018f7ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 --2- 172.21.15.5:0/2888890156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f841230 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 --2- 172.21.15.5:0/2888890156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f8057a0 0x56018f829250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 -- 172.21.15.5:0/2888890156 >> 172.21.15.5:0/2888890156 conn(0x56018f740210 msgr2=0x56018f826ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 -- 172.21.15.5:0/2888890156 shutdown_connections 2024-05-08T23:24:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.917+0000 7f637766ab80 1 -- 172.21.15.5:0/2888890156 wait complete. 2024-05-08T23:24:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.918+0000 7f637766ab80 1 -- start start 2024-05-08T23:24:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.918+0000 7f637766ab80 1 Processor -- start 2024-05-08T23:24:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f637766ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f81dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f637766ab80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f80e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f637766ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f637766ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018f803790 con 0x56018f7e0370 2024-05-08T23:24:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f637766ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018f895280 con 0x56018f80cdd0 2024-05-08T23:24:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f637766ab80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56018f80f0b0 con 0x56018f8057a0 2024-05-08T23:24:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f637766ab80 10 client.0 _ll_register_callbacks cb 0x56018f6d7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f81dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f81dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53292/0 (socket says 172.21.15.5:53292) 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 -- 172.21.15.5:0/1667220353 learned_addr learned my addr 172.21.15.5:0/1667220353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f6368a4b700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f80e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f6369a4d700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 -- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 msgr2=0x56018f80e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f80e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 -- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 msgr2=0x56018f80e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 -- 172.21.15.5:0/1667220353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56018f55fb60 con 0x56018f7e0370 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f6368a4b700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f80e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f6369a4d700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f636924c700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f81dfe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6360009840 tx=0x7f6360029070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71787665f16a281f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.919+0000 7f6343fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f6343fff700 1 -- 172.21.15.5:0/1667220353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636002c070 con 0x56018f7e0370 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f6343fff700 1 -- 172.21.15.5:0/1667220353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6360032e70 con 0x56018f7e0370 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 1 -- 172.21.15.5:0/1667220353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63600036a0 con 0x56018f7e0370 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f6343fff700 1 -- 172.21.15.5:0/1667220353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6360020040 con 0x56018f7e0370 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 10 client.6285 fetch_fsmap learned FSMap version 41 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 1 -- 172.21.15.5:0/1667220353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x56018f5b7cd0 con 0x56018f7e0370 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f6343fff700 1 -- 172.21.15.5:0/1667220353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63600036a0 con 0x56018f7e0370 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f6343fff700 1 -- 172.21.15.5:0/1667220353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 41) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f6360038050 con 0x56018f7e0370 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 10 client.6285 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 10 client.6285 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 20 client.6285 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x56018f894450 0x56018f894830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 4 client.6285 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f637766ab80 1 -- 172.21.15.5:0/1667220353 --> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56018f72aec0 con 0x56018f894450 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.920+0000 7f6368a4b700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x56018f894450 0x56018f894830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f6343fff700 10 client.6285 ms_handle_connect on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f6368a4b700 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x56018f894450 0x56018f894830 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb290205b5e6c6f8 server_cookie=1a77fc528ccd8426 in_seq=0 out_seq=0 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f6343fff700 1 -- 172.21.15.5:0/1667220353 <== mds.0 v2:172.21.15.5:6833/3359342067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56018f72aec0 con 0x56018f894450 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f6343fff700 10 client.6285 handle_command_reply: tid=0 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 client.6285 shutdown 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 -- 172.21.15.5:0/1667220353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 msgr2=0x56018f81dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f81dfe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6360009840 tx=0x7f6360029070 comp rx=0 tx=0).stop 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 -- 172.21.15.5:0/1667220353 shutdown_connections 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56018f80cdd0 0x56018f80e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56018f7e0370 0x56018f81dfe0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.5:6833/3359342067,v1:172.21.15.5:6835/3359342067] conn(0x56018f894450 0x56018f894830 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 --2- 172.21.15.5:0/1667220353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56018f8057a0 0x56018f80e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f6343fff700 0 client.6285 ms_handle_reset on v2:172.21.15.5:6833/3359342067 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 -- 172.21.15.5:0/1667220353 >> 172.21.15.5:0/1667220353 conn(0x56018f740210 msgr2=0x56018f840d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 -- 172.21.15.5:0/1667220353 shutdown_connections 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 1 -- 172.21.15.5:0/1667220353 wait complete. 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.921+0000 7f637766ab80 20 client.6285 trim_cache size 0 max 16384 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-08T23:24:24.929 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-08T23:24:24.930 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 -- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f634806ac90 msgr2=0x7f634806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f634806ac90 0x7f634806d150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f63600090b0 tx=0x7f6360008030 comp rx=0 tx=0).stop 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 -- 172.21.15.5:0/2682973651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63640cba90 msgr2=0x7f63640c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63640cba90 0x7f63640c6990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f635c00cb50 tx=0x7f635c00ce60 comp rx=0 tx=0).stop 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 -- 172.21.15.5:0/2682973651 shutdown_connections 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f634806ac90 0x7f634806d150 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63640cba90 0x7f63640c6990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6364099a50 0x7f63640b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 --2- 172.21.15.5:0/2682973651 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6364099e30 0x7f636415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 -- 172.21.15.5:0/2682973651 >> 172.21.15.5:0/2682973651 conn(0x7f63640b2f30 msgr2=0x7f63640cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 -- 172.21.15.5:0/2682973651 shutdown_connections 2024-05-08T23:24:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:24.922+0000 7f636b4b0700 1 -- 172.21.15.5:0/2682973651 wait complete. 2024-05-08T23:24:24.935 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:24:24.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.071+0000 7fee84d33700 1 Processor -- start 2024-05-08T23:24:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.071+0000 7fee84d33700 1 -- start start 2024-05-08T23:24:25.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee84d33700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fee800cf4f0 0x7fee80166910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee84d33700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8015bd70 con 0x7fee800cf9d0 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee84d33700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8015c050 con 0x7fee800cf040 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee84d33700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8015c330 con 0x7fee800cf4f0 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee7ed9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fee800cf4f0 0x7fee80166910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee7e59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fee800cf9d0 0x7fee800cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57188/0 (socket says 172.21.15.5:57188) 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee7e59c700 1 -- 172.21.15.5:0/2341289531 learned_addr learned my addr 172.21.15.5:0/2341289531 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee7ed9d700 1 -- 172.21.15.5:0/2341289531 >> v1:172.21.15.49:6789/0 conn(0x7fee800cf040 legacy=0x7fee801631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee7ed9d700 1 -- 172.21.15.5:0/2341289531 >> v1:172.21.15.5:6789/0 conn(0x7fee800cf9d0 legacy=0x7fee800cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee7ed9d700 1 -- 172.21.15.5:0/2341289531 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8015c610 con 0x7fee800cf4f0 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.072+0000 7fee7ed9d700 1 --2- 172.21.15.5:0/2341289531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf4f0 0x7fee80166910 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fee78009dc0 tx=0x7fee7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c305cd1e57aa945 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee7d59a700 1 -- 172.21.15.5:0/2341289531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7800e070 con 0x7fee800cf4f0 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee7d59a700 1 -- 172.21.15.5:0/2341289531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7800bb90 con 0x7fee800cf4f0 2024-05-08T23:24:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee7d59a700 1 -- 172.21.15.5:0/2341289531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78012430 con 0x7fee800cf4f0 2024-05-08T23:24:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee84d33700 1 -- 172.21.15.5:0/2341289531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf4f0 msgr2=0x7fee80166910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee84d33700 1 --2- 172.21.15.5:0/2341289531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf4f0 0x7fee80166910 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fee78009dc0 tx=0x7fee7800b330 comp rx=0 tx=0).stop 2024-05-08T23:24:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee84d33700 1 -- 172.21.15.5:0/2341289531 shutdown_connections 2024-05-08T23:24:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee84d33700 1 --2- 172.21.15.5:0/2341289531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf4f0 0x7fee80166910 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee84d33700 1 -- 172.21.15.5:0/2341289531 >> 172.21.15.5:0/2341289531 conn(0x7fee800b2f60 msgr2=0x7fee800b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee84d33700 1 -- 172.21.15.5:0/2341289531 shutdown_connections 2024-05-08T23:24:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.073+0000 7fee84d33700 1 -- 172.21.15.5:0/2341289531 wait complete. 2024-05-08T23:24:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.075+0000 7fee84d33700 1 Processor -- start 2024-05-08T23:24:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.075+0000 7fee84d33700 1 -- start start 2024-05-08T23:24:25.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee84d33700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee800cf040 0x7fee8015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee84d33700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee800cf4f0 0x7fee8006d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee84d33700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf9d0 0x7fee801f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee84d33700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee80143d30 con 0x7fee800cf9d0 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee84d33700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee80156f90 con 0x7fee800cf4f0 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee84d33700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee80157270 con 0x7fee800cf040 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf9d0 0x7fee801f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf9d0 0x7fee801f6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53316/0 (socket says 172.21.15.5:53316) 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7ed9d700 1 -- 172.21.15.5:0/2369913361 learned_addr learned my addr 172.21.15.5:0/2369913361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7dd9b700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee800cf4f0 0x7fee8006d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7e59c700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee800cf040 0x7fee8015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7ed9d700 1 -- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee800cf040 msgr2=0x7fee8015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7ed9d700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee800cf040 0x7fee8015e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7ed9d700 1 -- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee800cf4f0 msgr2=0x7fee8006d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7ed9d700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee800cf4f0 0x7fee8006d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.076+0000 7fee7ed9d700 1 -- 172.21.15.5:0/2369913361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78009a70 con 0x7fee800cf9d0 2024-05-08T23:24:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.077+0000 7fee7ed9d700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf9d0 0x7fee801f6220 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee78014040 tx=0x7fee78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.077+0000 7fee6effd700 1 -- 172.21.15.5:0/2369913361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78009910 con 0x7fee800cf9d0 2024-05-08T23:24:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.077+0000 7fee6effd700 1 -- 172.21.15.5:0/2369913361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee78020070 con 0x7fee800cf9d0 2024-05-08T23:24:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.077+0000 7fee6effd700 1 -- 172.21.15.5:0/2369913361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee780276c0 con 0x7fee800cf9d0 2024-05-08T23:24:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.077+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee80157550 con 0x7fee800cf9d0 2024-05-08T23:24:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.077+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee80157b10 con 0x7fee800cf9d0 2024-05-08T23:24:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.078+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee80068000 con 0x7fee800cf9d0 2024-05-08T23:24:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.078+0000 7fee6effd700 1 -- 172.21.15.5:0/2369913361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fee78012430 con 0x7fee800cf9d0 2024-05-08T23:24:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.079+0000 7fee6effd700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee6006ac90 0x7fee6006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.079+0000 7fee6effd700 1 -- 172.21.15.5:0/2369913361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fee78022070 con 0x7fee800cf9d0 2024-05-08T23:24:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.079+0000 7fee7e59c700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee6006ac90 0x7fee6006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.080+0000 7fee7e59c700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee6006ac90 0x7fee6006d150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee70005910 tx=0x7fee70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.084+0000 7fee6effd700 1 -- 172.21.15.5:0/2369913361 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fee780567f0 con 0x7fee800cf9d0 2024-05-08T23:24:25.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.352+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee80157cd0 con 0x7fee800cf9d0 2024-05-08T23:24:25.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.353+0000 7fee6effd700 1 -- 172.21.15.5:0/2369913361 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4759 (secure 0 0 0) 0x7fee78059e10 con 0x7fee800cf9d0 2024-05-08T23:24:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:25.353 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:24:15.731387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5778,"mds_1":5566,"mds_2":5180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5180":{"gid":5180,"name":"d","rank":2,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5566":{"gid":5566,"name":"b","rank":1,"incarnation":33,"state":"up:active","state_seq":13,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:24:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee6006ac90 msgr2=0x7fee6006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee6006ac90 0x7fee6006d150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee70005910 tx=0x7fee70009040 comp rx=0 tx=0).stop 2024-05-08T23:24:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf9d0 msgr2=0x7fee801f6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf9d0 0x7fee801f6220 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee78014040 tx=0x7fee78004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 shutdown_connections 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee6006ac90 0x7fee6006d150 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee800cf9d0 0x7fee801f6220 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee800cf040 0x7fee8015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 --2- 172.21.15.5:0/2369913361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee800cf4f0 0x7fee8006d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 >> 172.21.15.5:0/2369913361 conn(0x7fee800b2f60 msgr2=0x7fee800c8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 shutdown_connections 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.355+0000 7fee84d33700 1 -- 172.21.15.5:0/2369913361 wait complete. 2024-05-08T23:24:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 41 2024-05-08T23:24:25.369 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'b', 'd'] 2024-05-08T23:24:25.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:24:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.504+0000 7f3745926700 1 Processor -- start 2024-05-08T23:24:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.504+0000 7f3745926700 1 -- start start 2024-05-08T23:24:25.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f3745926700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f374006ee90 0x7f374015b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f3745926700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37400cd910 con 0x7f374006d510 2024-05-08T23:24:25.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f3745926700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37400cdbf0 con 0x7f374006f270 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f3745926700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37400cded0 con 0x7f374006ee90 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f374006ee90 0x7f374015b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373f7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f374006d510 0x7f374006d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57198/0 (socket says 172.21.15.5:57198) 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373f7fe700 1 -- 172.21.15.5:0/687735304 learned_addr learned my addr 172.21.15.5:0/687735304 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373effd700 1 -- 172.21.15.5:0/687735304 >> v1:172.21.15.49:6789/0 conn(0x7f374006f270 legacy=0x7f37400cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373effd700 1 -- 172.21.15.5:0/687735304 >> v1:172.21.15.5:6789/0 conn(0x7f374006d510 legacy=0x7f374006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373effd700 1 -- 172.21.15.5:0/687735304 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400ce1b0 con 0x7f374006ee90 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373dffb700 1 -- 172.21.15.5:0/687735304 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645948143 0 0) 0x7f37400cd910 con 0x7f374006d510 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373effd700 1 --2- 172.21.15.5:0/687735304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006ee90 0x7f374015b520 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3728009840 tx=0x7f3728009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53067e725296805c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.506+0000 7f373dffb700 1 -- 172.21.15.5:0/687735304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728011830 con 0x7f374006ee90 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f373dffb700 1 -- 172.21.15.5:0/687735304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3728011990 con 0x7f374006ee90 2024-05-08T23:24:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f373dffb700 1 -- 172.21.15.5:0/687735304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728011d20 con 0x7f374006ee90 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f3745926700 1 -- 172.21.15.5:0/687735304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006ee90 msgr2=0x7f374015b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f3745926700 1 --2- 172.21.15.5:0/687735304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006ee90 0x7f374015b520 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3728009840 tx=0x7f3728009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f3745926700 1 -- 172.21.15.5:0/687735304 shutdown_connections 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f3745926700 1 --2- 172.21.15.5:0/687735304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006ee90 0x7f374015b520 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f3745926700 1 -- 172.21.15.5:0/687735304 >> 172.21.15.5:0/687735304 conn(0x7f37400cfce0 msgr2=0x7f37400d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f3745926700 1 -- 172.21.15.5:0/687735304 shutdown_connections 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.507+0000 7f3745926700 1 -- 172.21.15.5:0/687735304 wait complete. 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.509+0000 7f3745926700 1 Processor -- start 2024-05-08T23:24:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.509+0000 7f3745926700 1 -- start start 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3745926700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374006d510 0x7f3740153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3745926700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f374006ee90 0x7f37401565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3745926700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006f270 0x7f37400acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3745926700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3740156f70 con 0x7f374006f270 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3745926700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3740157af0 con 0x7f374006d510 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3745926700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3740156b20 con 0x7f374006ee90 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006f270 0x7f37400acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006f270 0x7f37400acb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53340/0 (socket says 172.21.15.5:53340) 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 -- 172.21.15.5:0/2975004916 learned_addr learned my addr 172.21.15.5:0/2975004916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373effd700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374006d510 0x7f3740153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373e7fc700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f374006ee90 0x7f37401565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 -- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f374006ee90 msgr2=0x7f37401565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f374006ee90 0x7f37401565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 -- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374006d510 msgr2=0x7f3740153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374006d510 0x7f3740153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 -- 172.21.15.5:0/2975004916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37280094a0 con 0x7f374006f270 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f373f7fe700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006f270 0x7f37400acb30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f373800cb60 tx=0x7f373800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3737fff700 1 -- 172.21.15.5:0/2975004916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373800ab50 con 0x7f374006f270 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3737fff700 1 -- 172.21.15.5:0/2975004916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373800acb0 con 0x7f374006f270 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.510+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3740157150 con 0x7f374006f270 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.511+0000 7f3737fff700 1 -- 172.21.15.5:0/2975004916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738007310 con 0x7f374006f270 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.511+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3740157710 con 0x7f374006f270 2024-05-08T23:24:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.512+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3740068000 con 0x7f374006f270 2024-05-08T23:24:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.512+0000 7f3737fff700 1 -- 172.21.15.5:0/2975004916 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3738007470 con 0x7f374006f270 2024-05-08T23:24:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.513+0000 7f3737fff700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f372006ac90 0x7f372006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.513+0000 7f3737fff700 1 -- 172.21.15.5:0/2975004916 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3738088e50 con 0x7f374006f270 2024-05-08T23:24:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.513+0000 7f373effd700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f372006ac90 0x7f372006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.513+0000 7f373effd700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f372006ac90 0x7f372006d150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37280090b0 tx=0x7f3728008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.517+0000 7f3737fff700 1 -- 172.21.15.5:0/2975004916 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3738054080 con 0x7f374006f270 2024-05-08T23:24:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.830+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f37400cc310 con 0x7f374006f270 2024-05-08T23:24:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.830+0000 7f3737fff700 1 -- 172.21.15.5:0/2975004916 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1822 (secure 0 0 0) 0x7f3738018020 con 0x7f374006f270 2024-05-08T23:24:25.831 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:25.831 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":220,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":705854,"bytes_used":227401728,"bytes_avail":772866711552,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":41,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"d","status":"up:active","gid":5180},{"filesystem_id":3,"rank":1,"name":"b","status":"up:active","gid":5566},{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5778}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-05-08T23:24:25.342782+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:24:25.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f372006ac90 msgr2=0x7f372006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f372006ac90 0x7f372006d150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37280090b0 tx=0x7f3728008030 comp rx=0 tx=0).stop 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006f270 msgr2=0x7f37400acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006f270 0x7f37400acb30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f373800cb60 tx=0x7f373800cf20 comp rx=0 tx=0).stop 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 shutdown_connections 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f372006ac90 0x7f372006d150 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f374006f270 0x7f37400acb30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f374006d510 0x7f3740153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 --2- 172.21.15.5:0/2975004916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f374006ee90 0x7f37401565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 >> 172.21.15.5:0/2975004916 conn(0x7f37400cfce0 msgr2=0x7f37400d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 shutdown_connections 2024-05-08T23:24:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:25.833+0000 7f3745926700 1 -- 172.21.15.5:0/2975004916 wait complete. 2024-05-08T23:24:25.847 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-05-08T23:24:25.847 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:24:25.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:24:25.891 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:24:25.940 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:24:25.941 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:24:25.941 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:25.961 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:24:25.962 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:26.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:26.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.209+0000 7fae38257700 1 Processor -- start 2024-05-08T23:24:26.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.210+0000 7fae38257700 1 -- start start 2024-05-08T23:24:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae38257700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3015d7a0 0x7fae30161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae38257700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae30162450 con 0x7fae3015db80 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae38257700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae300cee00 con 0x7fae300d3f40 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae38257700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae300b27e0 con 0x7fae3015d7a0 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae367f4700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fae3015db80 0x7fae300cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57202/0 (socket says 172.21.15.5:57202) 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae367f4700 1 -- 172.21.15.5:0/2493956192 learned_addr learned my addr 172.21.15.5:0/2493956192 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae35ff3700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3015d7a0 0x7fae30161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508014967 0 0) 0x7fae28001490 con 0x7fae3015db80 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae1c0035b0 con 0x7fae3015db80 2024-05-08T23:24:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 99353453 0 0) 0x7fae1c0035b0 con 0x7fae3015db80 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3015d7a0 msgr2=0x7fae30161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3015d7a0 0x7fae30161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 >> v1:172.21.15.49:6789/0 conn(0x7fae300d3f40 legacy=0x7fae300b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae300d4400 con 0x7fae3015db80 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fae28001b80 con 0x7fae3015db80 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 >> v1:172.21.15.5:6789/0 conn(0x7fae3015db80 legacy=0x7fae300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.211+0000 7fae34ff1700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c003a30 0x7fae1c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae34ff1700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3015d7a0 0x7fae1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae34ff1700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c004d90 0x7fae1c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae300b27e0 con 0x7fae1c003a30 2024-05-08T23:24:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c00bac0 con 0x7fae1c004d90 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c00bc30 con 0x7fae3015d7a0 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae35ff3700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c003a30 0x7fae1c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae367f4700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c004d90 0x7fae1c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae357f2700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3015d7a0 0x7fae1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae35ff3700 1 -- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3015d7a0 msgr2=0x7fae1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae35ff3700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3015d7a0 0x7fae1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae35ff3700 1 -- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c004d90 msgr2=0x7fae1c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae35ff3700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c004d90 0x7fae1c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae35ff3700 1 -- 172.21.15.5:0/2493956192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c0035b0 con 0x7fae1c003a30 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae367f4700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c004d90 0x7fae1c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae35ff3700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c003a30 0x7fae1c003e10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009b60 tx=0x7fae1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51ae115faf608e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00e6c0 con 0x7fae1c003a30 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c00e820 con 0x7fae1c003a30 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.212+0000 7fae34ff1700 1 -- 172.21.15.5:0/2493956192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00eb90 con 0x7fae1c003a30 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.213+0000 7fae38257700 1 -- 172.21.15.5:0/2493956192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c003a30 msgr2=0x7fae1c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.213+0000 7fae38257700 1 --2- 172.21.15.5:0/2493956192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c003a30 0x7fae1c003e10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009b60 tx=0x7fae1c005880 comp rx=0 tx=0).stop 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.213+0000 7fae367f4700 1 -- 172.21.15.5:0/2493956192 reap_dead start 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.213+0000 7fae38257700 1 -- 172.21.15.5:0/2493956192 shutdown_connections 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.213+0000 7fae38257700 1 -- 172.21.15.5:0/2493956192 >> 172.21.15.5:0/2493956192 conn(0x7fae3008afe0 msgr2=0x7fae300d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.213+0000 7fae38257700 1 -- 172.21.15.5:0/2493956192 shutdown_connections 2024-05-08T23:24:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.213+0000 7fae38257700 1 -- 172.21.15.5:0/2493956192 wait complete. 2024-05-08T23:24:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.214+0000 7fae38257700 1 Processor -- start 2024-05-08T23:24:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae38257700 1 -- start start 2024-05-08T23:24:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae38257700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae1c003a30 0x7fae300c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae38257700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae300c97a0 0x7fae300c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae38257700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3015b830 0x7fae3015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae38257700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3014b250 con 0x7fae300c97a0 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae38257700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3015c1f0 con 0x7fae3015b830 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae38257700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3015e270 con 0x7fae1c003a30 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae357f2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae300c97a0 0x7fae300c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae357f2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae300c97a0 0x7fae300c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53356/0 (socket says 172.21.15.5:53356) 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.215+0000 7fae357f2700 1 -- 172.21.15.5:0/3569930259 learned_addr learned my addr 172.21.15.5:0/3569930259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae367f4700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3015b830 0x7fae3015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae35ff3700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae1c003a30 0x7fae300c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae357f2700 1 -- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae1c003a30 msgr2=0x7fae300c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae357f2700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae1c003a30 0x7fae300c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae357f2700 1 -- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3015b830 msgr2=0x7fae3015bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae357f2700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3015b830 0x7fae3015bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae357f2700 1 -- 172.21.15.5:0/3569930259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3015e550 con 0x7fae300c97a0 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae357f2700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae300c97a0 0x7fae300c9c20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fae2400e650 tx=0x7fae2400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae227fc700 1 -- 172.21.15.5:0/3569930259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240052e0 con 0x7fae300c97a0 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae227fc700 1 -- 172.21.15.5:0/3569930259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24005440 con 0x7fae300c97a0 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3015e830 con 0x7fae300c97a0 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae227fc700 1 -- 172.21.15.5:0/3569930259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24003910 con 0x7fae300c97a0 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.216+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3015edf0 con 0x7fae300c97a0 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.217+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae30068000 con 0x7fae300c97a0 2024-05-08T23:24:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.218+0000 7fae227fc700 1 -- 172.21.15.5:0/3569930259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fae24003a70 con 0x7fae300c97a0 2024-05-08T23:24:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.218+0000 7fae227fc700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae1406ac90 0x7fae1406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.218+0000 7fae227fc700 1 -- 172.21.15.5:0/3569930259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fae24088ba0 con 0x7fae300c97a0 2024-05-08T23:24:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.219+0000 7fae35ff3700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae1406ac90 0x7fae1406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.219+0000 7fae35ff3700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae1406ac90 0x7fae1406d150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0090b0 tx=0x7fae2c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.223+0000 7fae227fc700 1 -- 172.21.15.5:0/3569930259 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fae24053d40 con 0x7fae300c97a0 2024-05-08T23:24:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.443+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae300d0e30 con 0x7fae300c97a0 2024-05-08T23:24:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.444+0000 7fae227fc700 1 -- 172.21.15.5:0/3569930259 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16100 (secure 0 0 0) 0x7fae24019070 con 0x7fae300c97a0 2024-05-08T23:24:26.444 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:26.444 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":48,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:53.673469+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:23:37.249225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T23:23:38.439923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.2d","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:26.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae1406ac90 msgr2=0x7fae1406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae1406ac90 0x7fae1406d150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0090b0 tx=0x7fae2c008030 comp rx=0 tx=0).stop 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae300c97a0 msgr2=0x7fae300c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae300c97a0 0x7fae300c9c20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fae2400e650 tx=0x7fae2400e960 comp rx=0 tx=0).stop 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 shutdown_connections 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae1406ac90 0x7fae1406d150 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3015b830 0x7fae3015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae1c003a30 0x7fae300c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 --2- 172.21.15.5:0/3569930259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae300c97a0 0x7fae300c9c20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 >> 172.21.15.5:0/3569930259 conn(0x7fae3008afe0 msgr2=0x7fae300d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 shutdown_connections 2024-05-08T23:24:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.446+0000 7fae38257700 1 -- 172.21.15.5:0/3569930259 wait complete. 2024-05-08T23:24:26.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:23:37.249225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:23:38.439923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:24:26.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:24:26.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.599+0000 7f9625d97700 1 Processor -- start 2024-05-08T23:24:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.599+0000 7f9625d97700 1 -- start start 2024-05-08T23:24:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.600+0000 7f9625d97700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9620099a50 0x7f96200d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.600+0000 7f9625d97700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962006f930 con 0x7f96200cba90 2024-05-08T23:24:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.600+0000 7f9625d97700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96200cfdf0 con 0x7f9620099e30 2024-05-08T23:24:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.600+0000 7f9625d97700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96200d00d0 con 0x7f9620099a50 2024-05-08T23:24:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.600+0000 7f961f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9620099a50 0x7f96200d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.600+0000 7f961ffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f96200cba90 0x7f962006fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57206/0 (socket says 172.21.15.5:57206) 2024-05-08T23:24:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.600+0000 7f961f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9620099a50 0x7f96200d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53358/0 (socket says 172.21.15.5:53358) 2024-05-08T23:24:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.600+0000 7f961f7fe700 1 -- 172.21.15.5:0/1164998134 learned_addr learned my addr 172.21.15.5:0/1164998134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.601+0000 7f961f7fe700 1 -- 172.21.15.5:0/1164998134 >> v1:172.21.15.49:6789/0 conn(0x7f9620099e30 legacy=0x7f962009a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.601+0000 7f961f7fe700 1 -- 172.21.15.5:0/1164998134 >> v1:172.21.15.5:6789/0 conn(0x7f96200cba90 legacy=0x7f962006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.601+0000 7f961f7fe700 1 -- 172.21.15.5:0/1164998134 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96200d03b0 con 0x7f9620099a50 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.601+0000 7f961e7fc700 1 -- 172.21.15.5:0/1164998134 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316528959 0 0) 0x7f962006f930 con 0x7f96200cba90 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.601+0000 7f961f7fe700 1 --2- 172.21.15.5:0/1164998134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099a50 0x7f96200d0690 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f96100098f0 tx=0x7f9610009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2234272e132083ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.601+0000 7f961e7fc700 1 -- 172.21.15.5:0/1164998134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961000b8a0 con 0x7f9620099a50 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.601+0000 7f961e7fc700 1 -- 172.21.15.5:0/1164998134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961000ba00 con 0x7f9620099a50 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.601+0000 7f961e7fc700 1 -- 172.21.15.5:0/1164998134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961000bd90 con 0x7f9620099a50 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.602+0000 7f9625d97700 1 -- 172.21.15.5:0/1164998134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099a50 msgr2=0x7f96200d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.602+0000 7f9625d97700 1 --2- 172.21.15.5:0/1164998134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099a50 0x7f96200d0690 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f96100098f0 tx=0x7f9610009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.602+0000 7f9625d97700 1 -- 172.21.15.5:0/1164998134 shutdown_connections 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.602+0000 7f9625d97700 1 --2- 172.21.15.5:0/1164998134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099a50 0x7f96200d0690 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.602+0000 7f9625d97700 1 -- 172.21.15.5:0/1164998134 >> 172.21.15.5:0/1164998134 conn(0x7f96200b2f30 msgr2=0x7f96200b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.602+0000 7f9625d97700 1 -- 172.21.15.5:0/1164998134 shutdown_connections 2024-05-08T23:24:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.602+0000 7f9625d97700 1 -- 172.21.15.5:0/1164998134 wait complete. 2024-05-08T23:24:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.603+0000 7f9625d97700 1 Processor -- start 2024-05-08T23:24:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.603+0000 7f9625d97700 1 -- start start 2024-05-08T23:24:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f9625d97700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9620099a50 0x7f96200af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f9625d97700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099e30 0x7f96200c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f9625d97700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96200cba90 0x7f96201534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f9625d97700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96200c9120 con 0x7f9620099e30 2024-05-08T23:24:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f9625d97700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96200c8510 con 0x7f9620099a50 2024-05-08T23:24:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f9625d97700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96200c87f0 con 0x7f96200cba90 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f961effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099e30 0x7f96200c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f961effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099e30 0x7f96200c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53374/0 (socket says 172.21.15.5:53374) 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.604+0000 7f961effd700 1 -- 172.21.15.5:0/1807385113 learned_addr learned my addr 172.21.15.5:0/1807385113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f961f7fe700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9620099a50 0x7f96200af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f961ffff700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96200cba90 0x7f96201534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f961effd700 1 -- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96200cba90 msgr2=0x7f96201534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f961effd700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96200cba90 0x7f96201534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f961effd700 1 -- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9620099a50 msgr2=0x7f96200af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f961effd700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9620099a50 0x7f96200af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f961effd700 1 -- 172.21.15.5:0/1807385113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610009550 con 0x7f9620099e30 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f961effd700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099e30 0x7f96200c6b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f961400c4e0 tx=0x7f961400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f9607fff700 1 -- 172.21.15.5:0/1807385113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140052d0 con 0x7f9620099e30 2024-05-08T23:24:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f9607fff700 1 -- 172.21.15.5:0/1807385113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9614005430 con 0x7f9620099e30 2024-05-08T23:24:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f9607fff700 1 -- 172.21.15.5:0/1807385113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140038c0 con 0x7f9620099e30 2024-05-08T23:24:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96200c8ad0 con 0x7f9620099e30 2024-05-08T23:24:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.605+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962006d540 con 0x7f9620099e30 2024-05-08T23:24:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.606+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96200d1240 con 0x7f9620099e30 2024-05-08T23:24:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.607+0000 7f9607fff700 1 -- 172.21.15.5:0/1807385113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9614003a20 con 0x7f9620099e30 2024-05-08T23:24:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.607+0000 7f9607fff700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f960c06acc0 0x7f960c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.608+0000 7f9607fff700 1 -- 172.21.15.5:0/1807385113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9614088820 con 0x7f9620099e30 2024-05-08T23:24:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.608+0000 7f961f7fe700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f960c06acc0 0x7f960c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.608+0000 7f961f7fe700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f960c06acc0 0x7f960c06d180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9610009130 tx=0x7f9610008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.612+0000 7f9607fff700 1 -- 172.21.15.5:0/1807385113 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f96140539c0 con 0x7f9620099e30 2024-05-08T23:24:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.834+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9620074bc0 con 0x7f9620099e30 2024-05-08T23:24:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.834+0000 7f9607fff700 1 -- 172.21.15.5:0/1807385113 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96140170a0 con 0x7f9620099e30 2024-05-08T23:24:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:24:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f960c06acc0 msgr2=0x7f960c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f960c06acc0 0x7f960c06d180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9610009130 tx=0x7f9610008030 comp rx=0 tx=0).stop 2024-05-08T23:24:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099e30 msgr2=0x7f96200c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099e30 0x7f96200c6b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f961400c4e0 tx=0x7f961400c8a0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 shutdown_connections 2024-05-08T23:24:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f960c06acc0 0x7f960c06d180 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96200cba90 0x7f96201534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9620099a50 0x7f96200af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 --2- 172.21.15.5:0/1807385113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9620099e30 0x7f96200c6b50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 >> 172.21.15.5:0/1807385113 conn(0x7f96200b2f30 msgr2=0x7f96200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 shutdown_connections 2024-05-08T23:24:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.837+0000 7f9625d97700 1 -- 172.21.15.5:0/1807385113 wait complete. 2024-05-08T23:24:26.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:24:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.990+0000 7f024f8b1700 1 Processor -- start 2024-05-08T23:24:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.990+0000 7f024f8b1700 1 -- start start 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024f8b1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02480b5500 0x7f0248099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024f8b1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02480aea70 0x7f02480aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024f8b1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024815c920 con 0x7f024806a960 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024f8b1700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480ad6b0 con 0x7f02480b5500 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024f8b1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480b2700 con 0x7f02480aea70 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024ce4c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02480b5500 0x7f0248099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024d64d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f024806a960 0x7f024806ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57218/0 (socket says 172.21.15.5:57218) 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024d64d700 1 -- 172.21.15.5:0/2188231787 learned_addr learned my addr 172.21.15.5:0/2188231787 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024de4e700 1 --2- 172.21.15.5:0/2188231787 >> v2:172.21.15.49:3300/0 conn(0x7f02480aea70 0x7f02480aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024ce4c700 1 -- 172.21.15.5:0/2188231787 >> v2:172.21.15.49:3300/0 conn(0x7f02480aea70 msgr2=0x7f02480aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024ce4c700 1 --2- 172.21.15.5:0/2188231787 >> v2:172.21.15.49:3300/0 conn(0x7f02480aea70 0x7f02480aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024ce4c700 1 -- 172.21.15.5:0/2188231787 >> v1:172.21.15.5:6789/0 conn(0x7f024806a960 legacy=0x7f024806ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f023ffff700 1 -- 172.21.15.5:0/2188231787 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421866228 0 0) 0x7f024815c920 con 0x7f024806a960 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.991+0000 7f024ce4c700 1 -- 172.21.15.5:0/2188231787 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480b2910 con 0x7f02480b5500 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024ce4c700 1 --2- 172.21.15.5:0/2188231787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480b5500 0x7f0248099a50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0238009840 tx=0x7f0238009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc42fc98b2a21d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f023ffff700 1 -- 172.21.15.5:0/2188231787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238011830 con 0x7f02480b5500 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f023ffff700 1 -- 172.21.15.5:0/2188231787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0238011990 con 0x7f02480b5500 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f023ffff700 1 -- 172.21.15.5:0/2188231787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238011d20 con 0x7f02480b5500 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024f8b1700 1 -- 172.21.15.5:0/2188231787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480b5500 msgr2=0x7f0248099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024f8b1700 1 --2- 172.21.15.5:0/2188231787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480b5500 0x7f0248099a50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0238009840 tx=0x7f0238009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024f8b1700 1 -- 172.21.15.5:0/2188231787 shutdown_connections 2024-05-08T23:24:26.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024f8b1700 1 --2- 172.21.15.5:0/2188231787 >> v2:172.21.15.49:3300/0 conn(0x7f02480aea70 0x7f02480aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024f8b1700 1 --2- 172.21.15.5:0/2188231787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480b5500 0x7f0248099a50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024f8b1700 1 -- 172.21.15.5:0/2188231787 >> 172.21.15.5:0/2188231787 conn(0x7f02480b5a30 msgr2=0x7f02480b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:26.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024f8b1700 1 -- 172.21.15.5:0/2188231787 shutdown_connections 2024-05-08T23:24:26.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.992+0000 7f024f8b1700 1 -- 172.21.15.5:0/2188231787 wait complete. 2024-05-08T23:24:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.994+0000 7f024f8b1700 1 Processor -- start 2024-05-08T23:24:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.994+0000 7f024f8b1700 1 -- start start 2024-05-08T23:24:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.994+0000 7f024f8b1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024806a960 0x7f024813e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.994+0000 7f024f8b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480aea70 0x7f02481533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.994+0000 7f024f8b1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02480b5500 0x7f024813efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.994+0000 7f024f8b1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248141fe0 con 0x7f02480aea70 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.994+0000 7f024f8b1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248152630 con 0x7f02480b5500 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.994+0000 7f024f8b1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024813eca0 con 0x7f024806a960 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480aea70 0x7f02481533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480aea70 0x7f02481533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53386/0 (socket says 172.21.15.5:53386) 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 -- 172.21.15.5:0/4171138413 learned_addr learned my addr 172.21.15.5:0/4171138413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024d64d700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024806a960 0x7f024813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024de4e700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02480b5500 0x7f024813efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 -- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024806a960 msgr2=0x7f024813e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024806a960 0x7f024813e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 -- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02480b5500 msgr2=0x7f024813efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02480b5500 0x7f024813efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 -- 172.21.15.5:0/4171138413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02380094a0 con 0x7f02480aea70 2024-05-08T23:24:26.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024de4e700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02480b5500 0x7f024813efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024ce4c700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480aea70 0x7f02481533b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f02380090b0 tx=0x7f0238004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f023dffb700 1 -- 172.21.15.5:0/4171138413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023801aa10 con 0x7f02480aea70 2024-05-08T23:24:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f023dffb700 1 -- 172.21.15.5:0/4171138413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023801ab70 con 0x7f02480aea70 2024-05-08T23:24:27.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0248141d00 con 0x7f02480aea70 2024-05-08T23:24:27.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.995+0000 7f023dffb700 1 -- 172.21.15.5:0/4171138413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02380243f0 con 0x7f02480aea70 2024-05-08T23:24:27.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.996+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0248152bb0 con 0x7f02480aea70 2024-05-08T23:24:27.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.997+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0248074bc0 con 0x7f02480aea70 2024-05-08T23:24:27.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.997+0000 7f023dffb700 1 -- 172.21.15.5:0/4171138413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f023802d450 con 0x7f02480aea70 2024-05-08T23:24:27.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.998+0000 7f023dffb700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022c06ac90 0x7f022c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.998+0000 7f023dffb700 1 -- 172.21.15.5:0/4171138413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f023808a8d0 con 0x7f02480aea70 2024-05-08T23:24:27.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.998+0000 7f024d64d700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022c06ac90 0x7f022c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:26.999+0000 7f024d64d700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022c06ac90 0x7f022c06d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0244000f60 tx=0x7f0244007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.003+0000 7f023dffb700 1 -- 172.21.15.5:0/4171138413 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0238055a60 con 0x7f02480aea70 2024-05-08T23:24:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.220+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f024806b590 con 0x7f02480aea70 2024-05-08T23:24:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.221+0000 7f023dffb700 1 -- 172.21.15.5:0/4171138413 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02380150a0 con 0x7f02480aea70 2024-05-08T23:24:27.221 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:24:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022c06ac90 msgr2=0x7f022c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022c06ac90 0x7f022c06d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0244000f60 tx=0x7f0244007be0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480aea70 msgr2=0x7f02481533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480aea70 0x7f02481533b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f02380090b0 tx=0x7f0238004750 comp rx=0 tx=0).stop 2024-05-08T23:24:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 shutdown_connections 2024-05-08T23:24:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f022c06ac90 0x7f022c06d150 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02480b5500 0x7f024813efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f024806a960 0x7f024813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 --2- 172.21.15.5:0/4171138413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02480aea70 0x7f02481533b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 >> 172.21.15.5:0/4171138413 conn(0x7f02480b5a30 msgr2=0x7f02480b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 shutdown_connections 2024-05-08T23:24:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.223+0000 7f024f8b1700 1 -- 172.21.15.5:0/4171138413 wait complete. 2024-05-08T23:24:27.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:24:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.376+0000 7f351eb85700 1 Processor -- start 2024-05-08T23:24:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.376+0000 7f351eb85700 1 -- start start 2024-05-08T23:24:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.377+0000 7f351eb85700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f351806b070 0x7f3518168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.377+0000 7f351eb85700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3518145040 con 0x7f35180b4d50 2024-05-08T23:24:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.377+0000 7f351eb85700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35180b16e0 con 0x7f35180b0d80 2024-05-08T23:24:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.377+0000 7f351eb85700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35180b19c0 con 0x7f351806b070 2024-05-08T23:24:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.377+0000 7f3517fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f35180b4d50 0x7f351806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57234/0 (socket says 172.21.15.5:57234) 2024-05-08T23:24:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.377+0000 7f3517fff700 1 -- 172.21.15.5:0/3317768106 learned_addr learned my addr 172.21.15.5:0/3317768106 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.377+0000 7f351d122700 1 --2- 172.21.15.5:0/3317768106 >> v2:172.21.15.49:3300/0 conn(0x7f351806b070 0x7f3518168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.377+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212749216 0 0) 0x7f3518145040 con 0x7f35180b4d50 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35000035b0 con 0x7f35180b4d50 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2857771675 0 0) 0x7f35000035b0 con 0x7f35180b4d50 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 >> v2:172.21.15.49:3300/0 conn(0x7f351806b070 msgr2=0x7f3518168490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 --2- 172.21.15.5:0/3317768106 >> v2:172.21.15.49:3300/0 conn(0x7f351806b070 0x7f3518168490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 >> v1:172.21.15.49:6789/0 conn(0x7f35180b0d80 legacy=0x7f35180b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180b1ca0 con 0x7f35180b4d50 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f350c001670 con 0x7f35180b4d50 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 >> v1:172.21.15.5:6789/0 conn(0x7f35180b4d50 legacy=0x7f351806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500003a00 0x7f3500003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3500004850 0x7f3500004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3500005210 0x7f350000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35180b19c0 con 0x7f3500005210 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350000c0b0 con 0x7f3500003a00 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350000c270 con 0x7f3500004850 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f351c921700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500003a00 0x7f3500003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f351d122700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3500005210 0x7f350000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f3517fff700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3500004850 0x7f3500004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f351d122700 1 -- 172.21.15.5:0/3317768106 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3500004850 msgr2=0x7f3500004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f351d122700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3500004850 0x7f3500004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f351d122700 1 -- 172.21.15.5:0/3317768106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500003a00 msgr2=0x7f3500003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f351d122700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500003a00 0x7f3500003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.378+0000 7f351d122700 1 -- 172.21.15.5:0/3317768106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000035b0 con 0x7f3500005210 2024-05-08T23:24:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351d122700 1 -- 172.21.15.5:0/3317768106 reap_dead start 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351d122700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3500005210 0x7f350000ba10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3510009f90 tx=0x7f3500005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe58d9eefd411cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510003a20 con 0x7f3500005210 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351000f040 con 0x7f3500005210 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f35177fe700 1 -- 172.21.15.5:0/3317768106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510003de0 con 0x7f3500005210 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351eb85700 1 -- 172.21.15.5:0/3317768106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3500005210 msgr2=0x7f350000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351eb85700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3500005210 0x7f350000ba10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3510009f90 tx=0x7f3500005880 comp rx=0 tx=0).stop 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351eb85700 1 -- 172.21.15.5:0/3317768106 shutdown_connections 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351eb85700 1 --2- 172.21.15.5:0/3317768106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3500005210 0x7f350000ba10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351eb85700 1 -- 172.21.15.5:0/3317768106 >> 172.21.15.5:0/3317768106 conn(0x7f3518099a50 msgr2=0x7f35180b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351eb85700 1 -- 172.21.15.5:0/3317768106 shutdown_connections 2024-05-08T23:24:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.379+0000 7f351eb85700 1 -- 172.21.15.5:0/3317768106 wait complete. 2024-05-08T23:24:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.381+0000 7f351eb85700 1 Processor -- start 2024-05-08T23:24:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.381+0000 7f351eb85700 1 -- start start 2024-05-08T23:24:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351eb85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500005210 0x7f351815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351eb85700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35180ae610 0x7f3518130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351eb85700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3518130aa0 0x7f351815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351eb85700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518130f50 con 0x7f35180ae610 2024-05-08T23:24:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351eb85700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351812e650 con 0x7f3500005210 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351eb85700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351812e910 con 0x7f3518130aa0 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35180ae610 0x7f3518130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35180ae610 0x7f3518130560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53398/0 (socket says 172.21.15.5:53398) 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 -- 172.21.15.5:0/266418206 learned_addr learned my addr 172.21.15.5:0/266418206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351d122700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3518130aa0 0x7f351815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351c921700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500005210 0x7f351815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 -- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3518130aa0 msgr2=0x7f351815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3518130aa0 0x7f351815f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 -- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500005210 msgr2=0x7f351815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500005210 0x7f351815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 -- 172.21.15.5:0/266418206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351812ebf0 con 0x7f35180ae610 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351d122700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3518130aa0 0x7f351815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f351c921700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500005210 0x7f351815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f3517fff700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35180ae610 0x7f3518130560 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f350c007290 tx=0x7f350c0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.382+0000 7f35157fa700 1 -- 172.21.15.5:0/266418206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00f660 con 0x7f35180ae610 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.383+0000 7f35157fa700 1 -- 172.21.15.5:0/266418206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c00f7c0 con 0x7f35180ae610 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.383+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351812eed0 con 0x7f35180ae610 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.383+0000 7f35157fa700 1 -- 172.21.15.5:0/266418206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00fb70 con 0x7f35180ae610 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.383+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351815ff10 con 0x7f35180ae610 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.384+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3518074bc0 con 0x7f35180ae610 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.384+0000 7f35157fa700 1 -- 172.21.15.5:0/266418206 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f350c00fcd0 con 0x7f35180ae610 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.384+0000 7f35157fa700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f350406acc0 0x7f350406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.385+0000 7f35157fa700 1 -- 172.21.15.5:0/266418206 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f350c089280 con 0x7f35180ae610 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.385+0000 7f351c921700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f350406acc0 0x7f350406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.385+0000 7f351c921700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f350406acc0 0x7f350406d180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3508000c00 tx=0x7f3508009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:27.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.389+0000 7f35157fa700 1 -- 172.21.15.5:0/266418206 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f350c0544b0 con 0x7f35180ae610 2024-05-08T23:24:27.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.607+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f351806ed60 con 0x7f35180ae610 2024-05-08T23:24:27.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.608+0000 7f35157fa700 1 -- 172.21.15.5:0/266418206 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f350c057ad0 con 0x7f35180ae610 2024-05-08T23:24:27.609 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:24:27.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.610+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f350406acc0 msgr2=0x7f350406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.610+0000 7f351eb85700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f350406acc0 0x7f350406d180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3508000c00 tx=0x7f3508009040 comp rx=0 tx=0).stop 2024-05-08T23:24:27.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.610+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35180ae610 msgr2=0x7f3518130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.610+0000 7f351eb85700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35180ae610 0x7f3518130560 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f350c007290 tx=0x7f350c0075a0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.611+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 shutdown_connections 2024-05-08T23:24:27.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.611+0000 7f351eb85700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f350406acc0 0x7f350406d180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.611+0000 7f351eb85700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3518130aa0 0x7f351815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.611+0000 7f351eb85700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3500005210 0x7f351815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.611+0000 7f351eb85700 1 --2- 172.21.15.5:0/266418206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35180ae610 0x7f3518130560 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.611+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 >> 172.21.15.5:0/266418206 conn(0x7f3518099a50 msgr2=0x7f351806f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:27.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.611+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 shutdown_connections 2024-05-08T23:24:27.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.611+0000 7f351eb85700 1 -- 172.21.15.5:0/266418206 wait complete. 2024-05-08T23:24:27.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:24:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.762+0000 7f66aeef9700 1 Processor -- start 2024-05-08T23:24:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.762+0000 7f66aeef9700 1 -- start start 2024-05-08T23:24:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66aeef9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f66a815b780 0x7f66a80b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66aeef9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66aeef9700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f66a8099a50 0x7f66a8099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66aeef9700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a80cf600 con 0x7f66a815b780 2024-05-08T23:24:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66aeef9700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a809ae40 con 0x7f66a8099a50 2024-05-08T23:24:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66aeef9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a809a3e0 con 0x7f66a80b0390 2024-05-08T23:24:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66acc95700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f66a815b780 0x7f66a80b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66acc95700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f66a815b780 0x7f66a80b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41422/0 (socket says 172.21.15.5:41422) 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66acc95700 1 -- 172.21.15.5:0/2878039533 learned_addr learned my addr 172.21.15.5:0/2878039533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.763+0000 7f66ad496700 1 --2- 172.21.15.5:0/2878039533 >> v2:172.21.15.49:3300/0 conn(0x7f66a8099a50 0x7f66a8099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66a7fff700 1 --2- 172.21.15.5:0/2878039533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66acc95700 1 -- 172.21.15.5:0/2878039533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 msgr2=0x7f66a806ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66acc95700 1 --2- 172.21.15.5:0/2878039533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a806ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66acc95700 1 -- 172.21.15.5:0/2878039533 >> v2:172.21.15.49:3300/0 conn(0x7f66a8099a50 msgr2=0x7f66a8099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66acc95700 1 --2- 172.21.15.5:0/2878039533 >> v2:172.21.15.49:3300/0 conn(0x7f66a8099a50 0x7f66a8099ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66acc95700 1 -- 172.21.15.5:0/2878039533 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a809a8c0 con 0x7f66a815b780 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66acc95700 1 --2- 172.21.15.5:0/2878039533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a815b780 0x7f66a80b08e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f66a80729e0 tx=0x7f669c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2065b86f6aff1f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66a77fe700 1 -- 172.21.15.5:0/2878039533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c011830 con 0x7f66a815b780 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66a77fe700 1 -- 172.21.15.5:0/2878039533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c011990 con 0x7f66a815b780 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.764+0000 7f66a77fe700 1 -- 172.21.15.5:0/2878039533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c011d20 con 0x7f66a815b780 2024-05-08T23:24:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 -- 172.21.15.5:0/2878039533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a815b780 msgr2=0x7f66a80b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 --2- 172.21.15.5:0/2878039533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a815b780 0x7f66a80b08e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f66a80729e0 tx=0x7f669c009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 -- 172.21.15.5:0/2878039533 shutdown_connections 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 --2- 172.21.15.5:0/2878039533 >> v2:172.21.15.49:3300/0 conn(0x7f66a8099a50 0x7f66a8099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 --2- 172.21.15.5:0/2878039533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a815b780 0x7f66a80b08e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 --2- 172.21.15.5:0/2878039533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a806ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 -- 172.21.15.5:0/2878039533 >> 172.21.15.5:0/2878039533 conn(0x7f66a80b2f60 msgr2=0x7f66a80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 -- 172.21.15.5:0/2878039533 shutdown_connections 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.765+0000 7f66aeef9700 1 -- 172.21.15.5:0/2878039533 wait complete. 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.766+0000 7f66aeef9700 1 Processor -- start 2024-05-08T23:24:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.766+0000 7f66aeef9700 1 -- start start 2024-05-08T23:24:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66aeef9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a8099a50 0x7f66a8159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66aeef9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a80b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66aeef9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a815b780 0x7f66a80b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66aeef9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a80cb100 con 0x7f66a8099a50 2024-05-08T23:24:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66aeef9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a80c92c0 con 0x7f66a815b780 2024-05-08T23:24:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66aeef9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a80cea80 con 0x7f66a80b0390 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66acc95700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a8099a50 0x7f66a8159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66acc95700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a8099a50 0x7f66a8159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41424/0 (socket says 172.21.15.5:41424) 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.767+0000 7f66acc95700 1 -- 172.21.15.5:0/239009185 learned_addr learned my addr 172.21.15.5:0/239009185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66a7fff700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a80b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66ad496700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a815b780 0x7f66a80b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66acc95700 1 -- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 msgr2=0x7f66a80b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66acc95700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a80b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66acc95700 1 -- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a815b780 msgr2=0x7f66a80b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66acc95700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a815b780 0x7f66a80b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66acc95700 1 -- 172.21.15.5:0/239009185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c009550 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66a7fff700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a80b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66ad496700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a815b780 0x7f66a80b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66acc95700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a8099a50 0x7f66a8159bb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f669c0098f0 tx=0x7f669c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66a57fa700 1 -- 172.21.15.5:0/239009185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c004510 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66a57fa700 1 -- 172.21.15.5:0/239009185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c01e070 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66a57fa700 1 -- 172.21.15.5:0/239009185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c0038a0 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a80caa40 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.768+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a80cb520 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.769+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a8068000 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.770+0000 7f66a57fa700 1 -- 172.21.15.5:0/239009185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f669c003a00 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.770+0000 7f66a57fa700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f668c06ac90 0x7f668c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.770+0000 7f66a57fa700 1 -- 172.21.15.5:0/239009185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f669c08a170 con 0x7f66a8099a50 2024-05-08T23:24:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.771+0000 7f66a7fff700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f668c06ac90 0x7f668c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.771+0000 7f66a7fff700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f668c06ac90 0x7f668c06d150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f66a81f6a80 tx=0x7f6694008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:27.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:27.775+0000 7f66a57fa700 1 -- 172.21.15.5:0/239009185 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f669c0553a0 con 0x7f66a8099a50 2024-05-08T23:24:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.011+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f66a80cb9a0 con 0x7f66a8099a50 2024-05-08T23:24:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.012+0000 7f66a57fa700 1 -- 172.21.15.5:0/239009185 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7f669c003cb0 con 0x7f66a8099a50 2024-05-08T23:24:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f668c06ac90 msgr2=0x7f668c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f668c06ac90 0x7f668c06d150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f66a81f6a80 tx=0x7f6694008040 comp rx=0 tx=0).stop 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a8099a50 msgr2=0x7f66a8159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a8099a50 0x7f66a8159bb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f669c0098f0 tx=0x7f669c004750 comp rx=0 tx=0).stop 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 shutdown_connections 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f668c06ac90 0x7f668c06d150 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a815b780 0x7f66a80b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66a8099a50 0x7f66a8159bb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 --2- 172.21.15.5:0/239009185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f66a80b0390 0x7f66a80b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 >> 172.21.15.5:0/239009185 conn(0x7f66a80b2f60 msgr2=0x7f66a80c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 shutdown_connections 2024-05-08T23:24:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.014+0000 7f66aeef9700 1 -- 172.21.15.5:0/239009185 wait complete. 2024-05-08T23:24:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:24:28.028 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:24:28.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:28.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T23:24:28.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:24:28.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:28.088 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:24:28.108 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:24:28.108 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:28.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:24:28.130 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:24:28.130 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:28.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:24:28.169 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:24:28.169 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:28.191 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:24:28.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:28.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.355+0000 7f1c47b6e700 1 Processor -- start 2024-05-08T23:24:28.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.355+0000 7f1c47b6e700 1 -- start start 2024-05-08T23:24:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c47b6e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1c4015e830 0x7f1c400ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c47b6e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4006b0e0 0x7f1c400cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c47b6e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400d0a90 con 0x7f1c400d0f20 2024-05-08T23:24:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c47b6e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4006b720 con 0x7f1c4015e830 2024-05-08T23:24:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c47b6e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400d0440 con 0x7f1c4006b0e0 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c4610b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4006b0e0 0x7f1c400cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c4590a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1c4015e830 0x7f1c400ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c4610b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4006b0e0 0x7f1c400cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:54622/0 (socket says 172.21.15.5:54622) 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c45109700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1c400d0f20 0x7f1c4006a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:58662/0 (socket says 172.21.15.5:58662) 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.356+0000 7f1c4610b700 1 -- 172.21.15.5:0/978936119 learned_addr learned my addr 172.21.15.5:0/978936119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c44908700 1 -- 172.21.15.5:0/978936119 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703417510 0 0) 0x7f1c400d0a90 con 0x7f1c400d0f20 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c44908700 1 -- 172.21.15.5:0/978936119 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c2c0035b0 con 0x7f1c400d0f20 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c4590a700 1 -- 172.21.15.5:0/978936119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4006b0e0 msgr2=0x7f1c400cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c4590a700 1 --2- 172.21.15.5:0/978936119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4006b0e0 0x7f1c400cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c4590a700 1 -- 172.21.15.5:0/978936119 >> v1:172.21.15.49:6789/0 conn(0x7f1c400d0f20 legacy=0x7f1c4006a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c4590a700 1 -- 172.21.15.5:0/978936119 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400d0720 con 0x7f1c4015e830 2024-05-08T23:24:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c4610b700 1 --2- 172.21.15.5:0/978936119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4006b0e0 0x7f1c400cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c4590a700 1 --2- 172.21.15.5:0/978936119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4015e830 0x7f1c400ceb50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009840 tx=0x7f1c3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48324cab9bc73cba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c44908700 1 -- 172.21.15.5:0/978936119 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c011830 con 0x7f1c4015e830 2024-05-08T23:24:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c44908700 1 -- 172.21.15.5:0/978936119 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c011990 con 0x7f1c4015e830 2024-05-08T23:24:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.357+0000 7f1c44908700 1 -- 172.21.15.5:0/978936119 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c011d20 con 0x7f1c4015e830 2024-05-08T23:24:28.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.358+0000 7f1c47b6e700 1 -- 172.21.15.5:0/978936119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4015e830 msgr2=0x7f1c400ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.358+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/978936119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4015e830 0x7f1c400ceb50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009840 tx=0x7f1c3c009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:28.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.358+0000 7f1c47b6e700 1 -- 172.21.15.5:0/978936119 shutdown_connections 2024-05-08T23:24:28.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.358+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/978936119 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4006b0e0 0x7f1c400cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.358+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/978936119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4015e830 0x7f1c400ceb50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.358+0000 7f1c47b6e700 1 -- 172.21.15.5:0/978936119 >> 172.21.15.5:0/978936119 conn(0x7f1c400b5810 msgr2=0x7f1c400b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.358+0000 7f1c47b6e700 1 -- 172.21.15.5:0/978936119 shutdown_connections 2024-05-08T23:24:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.358+0000 7f1c47b6e700 1 -- 172.21.15.5:0/978936119 wait complete. 2024-05-08T23:24:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.359+0000 7f1c47b6e700 1 Processor -- start 2024-05-08T23:24:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.359+0000 7f1c47b6e700 1 -- start start 2024-05-08T23:24:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c47b6e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4006b0e0 0x7f1c400cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c47b6e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d0f20 0x7f1c400ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c47b6e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4015e830 0x7f1c40154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c47b6e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c401f32b0 con 0x7f1c400d0f20 2024-05-08T23:24:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c47b6e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40155490 con 0x7f1c4006b0e0 2024-05-08T23:24:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c47b6e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400cb9f0 con 0x7f1c4015e830 2024-05-08T23:24:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c45109700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d0f20 0x7f1c400ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c45109700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d0f20 0x7f1c400ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41434/0 (socket says 172.21.15.5:41434) 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.360+0000 7f1c45109700 1 -- 172.21.15.5:0/105864869 learned_addr learned my addr 172.21.15.5:0/105864869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c4590a700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4006b0e0 0x7f1c400cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c4610b700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4015e830 0x7f1c40154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c45109700 1 -- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4015e830 msgr2=0x7f1c40154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c45109700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4015e830 0x7f1c40154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c45109700 1 -- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4006b0e0 msgr2=0x7f1c400cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c45109700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4006b0e0 0x7f1c400cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c45109700 1 -- 172.21.15.5:0/105864869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c38009b30 con 0x7f1c400d0f20 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c45109700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d0f20 0x7f1c400ca160 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c700 tx=0x7f1c3000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c367fc700 1 -- 172.21.15.5:0/105864869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30005cb0 con 0x7f1c400d0f20 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c367fc700 1 -- 172.21.15.5:0/105864869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c30005e10 con 0x7f1c400d0f20 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c0094a0 con 0x7f1c400d0f20 2024-05-08T23:24:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c367fc700 1 -- 172.21.15.5:0/105864869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300072d0 con 0x7f1c400d0f20 2024-05-08T23:24:28.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.361+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c400cbfb0 con 0x7f1c400d0f20 2024-05-08T23:24:28.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.362+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c401f3420 con 0x7f1c400d0f20 2024-05-08T23:24:28.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.363+0000 7f1c367fc700 1 -- 172.21.15.5:0/105864869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1c30007430 con 0x7f1c400d0f20 2024-05-08T23:24:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.363+0000 7f1c367fc700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1c2406ac90 0x7f1c2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.363+0000 7f1c367fc700 1 -- 172.21.15.5:0/105864869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1c300887b0 con 0x7f1c400d0f20 2024-05-08T23:24:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.363+0000 7f1c4590a700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1c2406ac90 0x7f1c2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.364+0000 7f1c4590a700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1c2406ac90 0x7f1c2406d150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c000c00 tx=0x7f1c3c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:28.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.368+0000 7f1c367fc700 1 -- 172.21.15.5:0/105864869 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1c401f3420 con 0x7f1c400d0f20 2024-05-08T23:24:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.638+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c401f33f0 con 0x7f1c400d0f20 2024-05-08T23:24:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.638+0000 7f1c367fc700 1 -- 172.21.15.5:0/105864869 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4759 (secure 0 0 0) 0x7f1c30053a10 con 0x7f1c400d0f20 2024-05-08T23:24:28.638 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:28.638 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:24:15.731387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5778,"mds_1":5566,"mds_2":5180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5180":{"gid":5180,"name":"d","rank":2,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5566":{"gid":5566,"name":"b","rank":1,"incarnation":33,"state":"up:active","state_seq":13,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1c2406ac90 msgr2=0x7f1c2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1c2406ac90 0x7f1c2406d150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c000c00 tx=0x7f1c3c01c040 comp rx=0 tx=0).stop 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d0f20 msgr2=0x7f1c400ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d0f20 0x7f1c400ca160 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c700 tx=0x7f1c3000cac0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 shutdown_connections 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1c2406ac90 0x7f1c2406d150 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1c4015e830 0x7f1c40154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c4006b0e0 0x7f1c400cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 --2- 172.21.15.5:0/105864869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d0f20 0x7f1c400ca160 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 >> 172.21.15.5:0/105864869 conn(0x7f1c400b5810 msgr2=0x7f1c4015fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.640+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 shutdown_connections 2024-05-08T23:24:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.641+0000 7f1c47b6e700 1 -- 172.21.15.5:0/105864869 wait complete. 2024-05-08T23:24:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 41 2024-05-08T23:24:28.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.791+0000 7f53f31ee700 1 Processor -- start 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.791+0000 7f53f31ee700 1 -- start start 2024-05-08T23:24:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.792+0000 7f53f31ee700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53ec15dc70 0x7f53ec06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.792+0000 7f53f31ee700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec06d440 con 0x7f53ec09a160 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.792+0000 7f53f31ee700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec06d720 con 0x7f53ec0b0750 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.792+0000 7f53f31ee700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0cba40 con 0x7f53ec15dc70 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.792+0000 7f53f178b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f53ec09a160 0x7f53ec0cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34890/0 (socket says 172.21.15.5:34890) 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.792+0000 7f53f178b700 1 -- 172.21.15.5:0/732191461 learned_addr learned my addr 172.21.15.5:0/732191461 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53f0f8a700 1 --2- 172.21.15.5:0/732191461 >> v2:172.21.15.49:3300/0 conn(0x7f53ec15dc70 0x7f53ec06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547019499 0 0) 0x7f53ec06d440 con 0x7f53ec09a160 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d40035b0 con 0x7f53ec09a160 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056534691 0 0) 0x7f53ec06d720 con 0x7f53ec0b0750 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53ec06d440 con 0x7f53ec0b0750 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533482663 0 0) 0x7f53d40035b0 con 0x7f53ec09a160 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 >> v2:172.21.15.49:3300/0 conn(0x7f53ec15dc70 msgr2=0x7f53ec06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 --2- 172.21.15.5:0/732191461 >> v2:172.21.15.49:3300/0 conn(0x7f53ec15dc70 0x7f53ec06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 >> v1:172.21.15.49:6789/0 conn(0x7f53ec0b0750 legacy=0x7f53ec099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0cbd20 con 0x7f53ec09a160 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f53e40019f0 con 0x7f53ec09a160 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 >> v1:172.21.15.5:6789/0 conn(0x7f53ec09a160 legacy=0x7f53ec0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4003b90 0x7f53d4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4004850 0x7f53d4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53d4005210 0x7f53d400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0cba40 con 0x7f53d4004850 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d400c140 con 0x7f53d4003b90 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d400c300 con 0x7f53d4005210 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53f0f8a700 1 --2- 172.21.15.5:0/732191461 >> v2:172.21.15.49:3300/0 conn(0x7f53ec15dc70 0x7f53ec06cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53ebfff700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4004850 0x7f53d4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53f178b700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53d4005210 0x7f53d400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.793+0000 7f53f0f8a700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4003b90 0x7f53d4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53ebfff700 1 -- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53d4005210 msgr2=0x7f53d400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53ebfff700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53d4005210 0x7f53d400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53ebfff700 1 -- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4003b90 msgr2=0x7f53d4003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53ebfff700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4003b90 0x7f53d4003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53ebfff700 1 -- 172.21.15.5:0/732191461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40035b0 con 0x7f53d4004850 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53f178b700 1 -- 172.21.15.5:0/732191461 reap_dead start 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53f0f8a700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4003b90 0x7f53d4003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53ebfff700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4004850 0x7f53d4004cd0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f53e0001930 tx=0x7f53d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a68bc1c9ab6e78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00052d0 con 0x7f53d4004850 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0005430 con 0x7f53d4004850 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53eb7fe700 1 -- 172.21.15.5:0/732191461 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00038a0 con 0x7f53d4004850 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53f31ee700 1 -- 172.21.15.5:0/732191461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4004850 msgr2=0x7f53d4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.794+0000 7f53f31ee700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4004850 0x7f53d4004cd0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f53e0001930 tx=0x7f53d4005880 comp rx=0 tx=0).stop 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.795+0000 7f53f31ee700 1 -- 172.21.15.5:0/732191461 shutdown_connections 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.795+0000 7f53f31ee700 1 --2- 172.21.15.5:0/732191461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d4004850 0x7f53d4004cd0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.795+0000 7f53f31ee700 1 -- 172.21.15.5:0/732191461 >> 172.21.15.5:0/732191461 conn(0x7f53ec0b2ee0 msgr2=0x7f53ec0b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.795+0000 7f53f31ee700 1 -- 172.21.15.5:0/732191461 shutdown_connections 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.795+0000 7f53f31ee700 1 -- 172.21.15.5:0/732191461 wait complete. 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.796+0000 7f53f31ee700 1 Processor -- start 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.796+0000 7f53f31ee700 1 -- start start 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f31ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4004850 0x7f53ec0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f31ee700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53ec0cab80 0x7f53ec1f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f31ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ec0cb000 0x7f53ec1f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f31ee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1f3ab0 con 0x7f53ec0cb000 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f31ee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1f7b70 con 0x7f53d4004850 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f31ee700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0cd9b0 con 0x7f53ec0cab80 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ec0cb000 0x7f53ec1f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ec0cb000 0x7f53ec1f71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41450/0 (socket says 172.21.15.5:41450) 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 -- 172.21.15.5:0/2713829145 learned_addr learned my addr 172.21.15.5:0/2713829145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f0f8a700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4004850 0x7f53ec0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53ebfff700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53ec0cab80 0x7f53ec1f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 -- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53ec0cab80 msgr2=0x7f53ec1f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53ec0cab80 0x7f53ec1f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 -- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4004850 msgr2=0x7f53ec0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4004850 0x7f53ec0cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 -- 172.21.15.5:0/2713829145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec1f7890 con 0x7f53ec0cb000 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f178b700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ec0cb000 0x7f53ec1f71c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0b0be0 tx=0x7f53e400a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53e97fa700 1 -- 172.21.15.5:0/2713829145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4001410 con 0x7f53ec0cb000 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53e97fa700 1 -- 172.21.15.5:0/2713829145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e400aca0 con 0x7f53ec0cb000 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec0b4710 con 0x7f53ec0cb000 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.798+0000 7f53e97fa700 1 -- 172.21.15.5:0/2713829145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4013630 con 0x7f53ec0cb000 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.799+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec0b4cd0 con 0x7f53ec0cb000 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.800+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec068000 con 0x7f53ec0cb000 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.800+0000 7f53e97fa700 1 -- 172.21.15.5:0/2713829145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f53e4013790 con 0x7f53ec0cb000 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.800+0000 7f53e97fa700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f53d806ac90 0x7f53d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.801+0000 7f53e97fa700 1 -- 172.21.15.5:0/2713829145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f53e4089b60 con 0x7f53ec0cb000 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.801+0000 7f53f0f8a700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f53d806ac90 0x7f53d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.801+0000 7f53f0f8a700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f53d806ac90 0x7f53d806d150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f53ec069190 tx=0x7f53dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:28.805+0000 7f53e97fa700 1 -- 172.21.15.5:0/2713829145 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f53e4054d00 con 0x7f53ec0cb000 2024-05-08T23:24:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.026+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53ec09a940 con 0x7f53ec0cb000 2024-05-08T23:24:29.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.027+0000 7f53e97fa700 1 -- 172.21.15.5:0/2713829145 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16100 (secure 0 0 0) 0x7f53e4058320 con 0x7f53ec0cb000 2024-05-08T23:24:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:29.027 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":48,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:53.673469+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:23:37.249225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T23:23:38.439923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.2d","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f53d806ac90 msgr2=0x7f53d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f53d806ac90 0x7f53d806d150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f53ec069190 tx=0x7f53dc008040 comp rx=0 tx=0).stop 2024-05-08T23:24:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ec0cb000 msgr2=0x7f53ec1f71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ec0cb000 0x7f53ec1f71c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0b0be0 tx=0x7f53e400a5b0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 shutdown_connections 2024-05-08T23:24:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f53d806ac90 0x7f53d806d150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ec0cb000 0x7f53ec1f71c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d4004850 0x7f53ec0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 --2- 172.21.15.5:0/2713829145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53ec0cab80 0x7f53ec1f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 >> 172.21.15.5:0/2713829145 conn(0x7f53ec0b2ee0 msgr2=0x7f53ec15b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.029+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 shutdown_connections 2024-05-08T23:24:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.030+0000 7f53f31ee700 1 -- 172.21.15.5:0/2713829145 wait complete. 2024-05-08T23:24:29.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:23:37.249225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:23:38.439923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:24:29.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:24:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.182+0000 7face57f5700 1 Processor -- start 2024-05-08T23:24:29.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.182+0000 7face57f5700 1 -- start start 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.183+0000 7face57f5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face00d0a60 0x7face00b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.183+0000 7face57f5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7face015fbb0 0x7face00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.183+0000 7face57f5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face00af6e0 con 0x7face0145040 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.183+0000 7face57f5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face00b6460 con 0x7face015fbb0 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.183+0000 7face57f5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face00aeb50 con 0x7face00d0a60 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.183+0000 7facdf7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7face015fbb0 0x7face00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.183+0000 7facdf7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7face015fbb0 0x7face00afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41460/0 (socket says 172.21.15.5:41460) 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.183+0000 7facdf7fe700 1 -- 172.21.15.5:0/1646413869 learned_addr learned my addr 172.21.15.5:0/1646413869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facde7fc700 1 --2- 172.21.15.5:0/1646413869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face00d0a60 0x7face00b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facdf7fe700 1 -- 172.21.15.5:0/1646413869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face00d0a60 msgr2=0x7face00b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facdf7fe700 1 --2- 172.21.15.5:0/1646413869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face00d0a60 0x7face00b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facdf7fe700 1 -- 172.21.15.5:0/1646413869 >> v1:172.21.15.49:6789/0 conn(0x7face0145040 legacy=0x7face00d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facdf7fe700 1 -- 172.21.15.5:0/1646413869 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face00aedd0 con 0x7face015fbb0 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facdf7fe700 1 --2- 172.21.15.5:0/1646413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face00afb70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7facd8009e50 tx=0x7facd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4ae82a8277cb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facddffb700 1 -- 172.21.15.5:0/1646413869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800e070 con 0x7face015fbb0 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facddffb700 1 -- 172.21.15.5:0/1646413869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800b9c0 con 0x7face015fbb0 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.184+0000 7facddffb700 1 -- 172.21.15.5:0/1646413869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800bd70 con 0x7face015fbb0 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.185+0000 7face57f5700 1 -- 172.21.15.5:0/1646413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 msgr2=0x7face00afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.185+0000 7face57f5700 1 --2- 172.21.15.5:0/1646413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face00afb70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7facd8009e50 tx=0x7facd800b330 comp rx=0 tx=0).stop 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.185+0000 7face57f5700 1 -- 172.21.15.5:0/1646413869 shutdown_connections 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.185+0000 7face57f5700 1 --2- 172.21.15.5:0/1646413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face00afb70 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.185+0000 7face57f5700 1 --2- 172.21.15.5:0/1646413869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face00d0a60 0x7face00b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.185+0000 7face57f5700 1 -- 172.21.15.5:0/1646413869 >> 172.21.15.5:0/1646413869 conn(0x7face0099a50 msgr2=0x7face00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.185+0000 7face57f5700 1 -- 172.21.15.5:0/1646413869 shutdown_connections 2024-05-08T23:24:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.185+0000 7face57f5700 1 -- 172.21.15.5:0/1646413869 wait complete. 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.186+0000 7face57f5700 1 Processor -- start 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.186+0000 7face57f5700 1 -- start start 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7face57f5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face00d0a60 0x7face015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7face57f5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face0145040 0x7face014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7face57f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7face57f5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face015e2f0 con 0x7face015fbb0 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7face57f5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face015d6e0 con 0x7face00d0a60 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7face57f5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face015d9c0 con 0x7face0145040 2024-05-08T23:24:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7facdf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7facdf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face014d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41474/0 (socket says 172.21.15.5:41474) 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7facdf7fe700 1 -- 172.21.15.5:0/422003962 learned_addr learned my addr 172.21.15.5:0/422003962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7facdeffd700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face00d0a60 0x7face015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.187+0000 7facde7fc700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face0145040 0x7face014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facdf7fe700 1 -- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face0145040 msgr2=0x7face014cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facdf7fe700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face0145040 0x7face014cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facdf7fe700 1 -- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face00d0a60 msgr2=0x7face015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facdf7fe700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face00d0a60 0x7face015cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facdf7fe700 1 -- 172.21.15.5:0/422003962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd8009b30 con 0x7face015fbb0 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facdeffd700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face00d0a60 0x7face015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facde7fc700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face0145040 0x7face014cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facdf7fe700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face014d1b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7face00b5790 tx=0x7facd8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facc77fe700 1 -- 172.21.15.5:0/422003962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd80099b0 con 0x7face015fbb0 2024-05-08T23:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facc77fe700 1 -- 172.21.15.5:0/422003962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd80092e0 con 0x7face015fbb0 2024-05-08T23:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face015dca0 con 0x7face015fbb0 2024-05-08T23:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7facc77fe700 1 -- 172.21.15.5:0/422003962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd801c640 con 0x7face015fbb0 2024-05-08T23:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.188+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face014d960 con 0x7face015fbb0 2024-05-08T23:24:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.189+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face015e4a0 con 0x7face015fbb0 2024-05-08T23:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.189+0000 7facc77fe700 1 -- 172.21.15.5:0/422003962 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7facd801c7a0 con 0x7face015fbb0 2024-05-08T23:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.190+0000 7facc77fe700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faccc06ac90 0x7faccc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.190+0000 7facc77fe700 1 -- 172.21.15.5:0/422003962 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7facd808afc0 con 0x7face015fbb0 2024-05-08T23:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.190+0000 7facdeffd700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faccc06ac90 0x7faccc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.191+0000 7facdeffd700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faccc06ac90 0x7faccc06d150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7facd0000f60 tx=0x7facd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.195+0000 7facc77fe700 1 -- 172.21.15.5:0/422003962 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7face015e4a0 con 0x7face015fbb0 2024-05-08T23:24:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.410+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7face0074bc0 con 0x7face015fbb0 2024-05-08T23:24:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.411+0000 7facc77fe700 1 -- 172.21.15.5:0/422003962 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7facd80561f0 con 0x7face015fbb0 2024-05-08T23:24:29.411 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:24:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faccc06ac90 msgr2=0x7faccc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faccc06ac90 0x7faccc06d150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7facd0000f60 tx=0x7facd0009040 comp rx=0 tx=0).stop 2024-05-08T23:24:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 msgr2=0x7face014d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face014d1b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7face00b5790 tx=0x7facd8004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 shutdown_connections 2024-05-08T23:24:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7faccc06ac90 0x7faccc06d150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face015fbb0 0x7face014d1b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7face00d0a60 0x7face015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 --2- 172.21.15.5:0/422003962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7face0145040 0x7face014cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 >> 172.21.15.5:0/422003962 conn(0x7face0099a50 msgr2=0x7face006f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 shutdown_connections 2024-05-08T23:24:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.413+0000 7face57f5700 1 -- 172.21.15.5:0/422003962 wait complete. 2024-05-08T23:24:29.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:24:29.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.566+0000 7f480bd6b700 1 Processor -- start 2024-05-08T23:24:29.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.567+0000 7f480bd6b700 1 -- start start 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f480bd6b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f480415b710 0x7f4804099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f480bd6b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cff30 0x7f4804099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f480bd6b700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48040d04b0 con 0x7f48040cf040 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f480bd6b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040b0910 con 0x7f480415b710 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f480bd6b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040b0bf0 con 0x7f48040cff30 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4809306700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cff30 0x7f4804099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4809306700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cff30 0x7f4804099f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:54678/0 (socket says 172.21.15.5:54678) 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4809306700 1 -- 172.21.15.5:0/2637307465 learned_addr learned my addr 172.21.15.5:0/2637307465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4809b07700 1 --2- 172.21.15.5:0/2637307465 >> v2:172.21.15.49:3300/0 conn(0x7f480415b710 0x7f4804099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4809306700 1 -- 172.21.15.5:0/2637307465 >> v2:172.21.15.49:3300/0 conn(0x7f480415b710 msgr2=0x7f4804099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4808b05700 1 -- 172.21.15.5:0/2637307465 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345269704 0 0) 0x7f48040d04b0 con 0x7f48040cf040 2024-05-08T23:24:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4809306700 1 --2- 172.21.15.5:0/2637307465 >> v2:172.21.15.49:3300/0 conn(0x7f480415b710 0x7f4804099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4809306700 1 -- 172.21.15.5:0/2637307465 >> v1:172.21.15.49:6789/0 conn(0x7f48040cf040 legacy=0x7f48040cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.568+0000 7f4809306700 1 -- 172.21.15.5:0/2637307465 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040b0ed0 con 0x7f48040cff30 2024-05-08T23:24:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.569+0000 7f4809306700 1 --2- 172.21.15.5:0/2637307465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cff30 0x7f4804099f90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b11933071681397 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.569+0000 7f4808b05700 1 -- 172.21.15.5:0/2637307465 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800011830 con 0x7f48040cff30 2024-05-08T23:24:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.569+0000 7f4808b05700 1 -- 172.21.15.5:0/2637307465 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4800011990 con 0x7f48040cff30 2024-05-08T23:24:29.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.569+0000 7f4808b05700 1 -- 172.21.15.5:0/2637307465 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800011d20 con 0x7f48040cff30 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.569+0000 7f480bd6b700 1 -- 172.21.15.5:0/2637307465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cff30 msgr2=0x7f4804099f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.569+0000 7f480bd6b700 1 --2- 172.21.15.5:0/2637307465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cff30 0x7f4804099f90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.570+0000 7f480bd6b700 1 -- 172.21.15.5:0/2637307465 shutdown_connections 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.570+0000 7f480bd6b700 1 --2- 172.21.15.5:0/2637307465 >> v2:172.21.15.49:3300/0 conn(0x7f480415b710 0x7f4804099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.570+0000 7f480bd6b700 1 --2- 172.21.15.5:0/2637307465 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cff30 0x7f4804099f90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.570+0000 7f480bd6b700 1 -- 172.21.15.5:0/2637307465 >> 172.21.15.5:0/2637307465 conn(0x7f48040b2f60 msgr2=0x7f48040b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.570+0000 7f480bd6b700 1 -- 172.21.15.5:0/2637307465 shutdown_connections 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.570+0000 7f480bd6b700 1 -- 172.21.15.5:0/2637307465 wait complete. 2024-05-08T23:24:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.571+0000 7f480bd6b700 1 Processor -- start 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.571+0000 7f480bd6b700 1 -- start start 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f480bd6b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cf040 0x7f4804157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f480bd6b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48040cff30 0x7f48041f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f480bd6b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f480415b710 0x7f48040b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f480bd6b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48041f1cf0 con 0x7f48040cff30 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f480bd6b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48040b55c0 con 0x7f480415b710 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f480bd6b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040b4520 con 0x7f48040cf040 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809306700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48040cff30 0x7f48041f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809306700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48040cff30 0x7f48041f17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41480/0 (socket says 172.21.15.5:41480) 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809306700 1 -- 172.21.15.5:0/1299593774 learned_addr learned my addr 172.21.15.5:0/1299593774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809b07700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cf040 0x7f4804157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f480a308700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f480415b710 0x7f48040b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809306700 1 -- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cf040 msgr2=0x7f4804157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809306700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cf040 0x7f4804157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809306700 1 -- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f480415b710 msgr2=0x7f48040b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809306700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f480415b710 0x7f48040b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f4809306700 1 -- 172.21.15.5:0/1299593774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800009550 con 0x7f48040cff30 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.572+0000 7f480a308700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f480415b710 0x7f48040b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.573+0000 7f4809306700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48040cff30 0x7f48041f17b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c40 tx=0x7f4800004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.573+0000 7f47fa7fc700 1 -- 172.21.15.5:0/1299593774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000045d0 con 0x7f48040cff30 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.573+0000 7f47fa7fc700 1 -- 172.21.15.5:0/1299593774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480001f070 con 0x7f48040cff30 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.573+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48040b4800 con 0x7f48040cff30 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.573+0000 7f47fa7fc700 1 -- 172.21.15.5:0/1299593774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000038a0 con 0x7f48040cff30 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.573+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48040b4dc0 con 0x7f48040cff30 2024-05-08T23:24:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.574+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4804074bc0 con 0x7f48040cff30 2024-05-08T23:24:29.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.575+0000 7f47fa7fc700 1 -- 172.21.15.5:0/1299593774 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4800003a00 con 0x7f48040cff30 2024-05-08T23:24:29.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.575+0000 7f47fa7fc700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47e806ac90 0x7f47e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.575+0000 7f47fa7fc700 1 -- 172.21.15.5:0/1299593774 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4800021070 con 0x7f48040cff30 2024-05-08T23:24:29.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.576+0000 7f4809b07700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47e806ac90 0x7f47e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.576+0000 7f4809b07700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47e806ac90 0x7f47e806d150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f48040cbbd0 tx=0x7f47f4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.579+0000 7f47fa7fc700 1 -- 172.21.15.5:0/1299593774 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4800056260 con 0x7f48040cff30 2024-05-08T23:24:29.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.794+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48040b02a0 con 0x7f48040cff30 2024-05-08T23:24:29.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.794+0000 7f47fa7fc700 1 -- 172.21.15.5:0/1299593774 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4800003cb0 con 0x7f48040cff30 2024-05-08T23:24:29.795 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:24:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47e806ac90 msgr2=0x7f47e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47e806ac90 0x7f47e806d150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f48040cbbd0 tx=0x7f47f4009210 comp rx=0 tx=0).stop 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48040cff30 msgr2=0x7f48041f17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48040cff30 0x7f48041f17b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c40 tx=0x7f4800004750 comp rx=0 tx=0).stop 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 shutdown_connections 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47e806ac90 0x7f47e806d150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f480415b710 0x7f48040b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48040cf040 0x7f4804157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 --2- 172.21.15.5:0/1299593774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48040cff30 0x7f48041f17b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 >> 172.21.15.5:0/1299593774 conn(0x7f48040b2f60 msgr2=0x7f48040ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 shutdown_connections 2024-05-08T23:24:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.797+0000 7f480bd6b700 1 -- 172.21.15.5:0/1299593774 wait complete. 2024-05-08T23:24:29.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:24:29.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.956+0000 7f8e40880700 1 Processor -- start 2024-05-08T23:24:29.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.956+0000 7f8e40880700 1 -- start start 2024-05-08T23:24:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e40880700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c15db00 0x7f8e3c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e40880700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e3c06f990 0x7f8e3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e40880700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c06ad20 con 0x7f8e3c0caa50 2024-05-08T23:24:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e40880700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3c15dee0 con 0x7f8e3c06f990 2024-05-08T23:24:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e40880700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0ccad0 con 0x7f8e3c15db00 2024-05-08T23:24:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e39d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8e3c0caa50 0x7f8e3c06f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34896/0 (socket says 172.21.15.5:34896) 2024-05-08T23:24:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e39d9b700 1 -- 172.21.15.5:0/152806584 learned_addr learned my addr 172.21.15.5:0/152806584 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e3ad9d700 1 --2- 172.21.15.5:0/152806584 >> v2:172.21.15.49:3300/0 conn(0x7f8e3c06f990 0x7f8e3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.957+0000 7f8e3a59c700 1 --2- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c15db00 0x7f8e3c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3959a700 1 -- 172.21.15.5:0/152806584 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245891918 0 0) 0x7f8e3c06ad20 con 0x7f8e3c0caa50 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3959a700 1 -- 172.21.15.5:0/152806584 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e240035b0 con 0x7f8e3c0caa50 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3ad9d700 1 -- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c15db00 msgr2=0x7f8e3c0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3ad9d700 1 --2- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c15db00 0x7f8e3c0cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3ad9d700 1 -- 172.21.15.5:0/152806584 >> v1:172.21.15.5:6789/0 conn(0x7f8e3c0caa50 legacy=0x7f8e3c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3ad9d700 1 -- 172.21.15.5:0/152806584 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c06b000 con 0x7f8e3c06f990 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3a59c700 1 --2- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c15db00 0x7f8e3c0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3ad9d700 1 --2- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c06f990 0x7f8e3c06e320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009dc0 tx=0x7f8e3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=defd65099ff6c81d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3959a700 1 -- 172.21.15.5:0/152806584 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400e070 con 0x7f8e3c06f990 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3959a700 1 -- 172.21.15.5:0/152806584 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3400bae0 con 0x7f8e3c06f990 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.958+0000 7f8e3959a700 1 -- 172.21.15.5:0/152806584 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400be90 con 0x7f8e3c06f990 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.959+0000 7f8e40880700 1 -- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c06f990 msgr2=0x7f8e3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.959+0000 7f8e40880700 1 --2- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c06f990 0x7f8e3c06e320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009dc0 tx=0x7f8e3400b330 comp rx=0 tx=0).stop 2024-05-08T23:24:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.959+0000 7f8e40880700 1 -- 172.21.15.5:0/152806584 shutdown_connections 2024-05-08T23:24:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.959+0000 7f8e40880700 1 --2- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c06f990 0x7f8e3c06e320 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.959+0000 7f8e40880700 1 --2- 172.21.15.5:0/152806584 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c15db00 0x7f8e3c0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.959+0000 7f8e40880700 1 -- 172.21.15.5:0/152806584 >> 172.21.15.5:0/152806584 conn(0x7f8e3c099a50 msgr2=0x7f8e3c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.959+0000 7f8e40880700 1 -- 172.21.15.5:0/152806584 shutdown_connections 2024-05-08T23:24:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.959+0000 7f8e40880700 1 -- 172.21.15.5:0/152806584 wait complete. 2024-05-08T23:24:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.960+0000 7f8e40880700 1 Processor -- start 2024-05-08T23:24:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e40880700 1 -- start start 2024-05-08T23:24:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e40880700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c06f990 0x7f8e3c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e40880700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c0caa50 0x7f8e3c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e40880700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c15db00 0x7f8e3c1e7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e40880700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c155c60 con 0x7f8e3c06f990 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e40880700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1e5290 con 0x7f8e3c15db00 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e40880700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1e5570 con 0x7f8e3c0caa50 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e3a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c06f990 0x7f8e3c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e3a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c06f990 0x7f8e3c1545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41494/0 (socket says 172.21.15.5:41494) 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.961+0000 7f8e3a59c700 1 -- 172.21.15.5:0/3394718936 learned_addr learned my addr 172.21.15.5:0/3394718936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e3ad9d700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c15db00 0x7f8e3c1e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e39d9b700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c0caa50 0x7f8e3c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e3a59c700 1 -- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c0caa50 msgr2=0x7f8e3c0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e3a59c700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c0caa50 0x7f8e3c0b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e3a59c700 1 -- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c15db00 msgr2=0x7f8e3c1e7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e3a59c700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c15db00 0x7f8e3c1e7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e3a59c700 1 -- 172.21.15.5:0/3394718936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c0094a0 con 0x7f8e3c06f990 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e3a59c700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c06f990 0x7f8e3c1545b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00ebe0 tx=0x7f8e2c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e2affd700 1 -- 172.21.15.5:0/3394718936 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c009760 con 0x7f8e3c06f990 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e2affd700 1 -- 172.21.15.5:0/3394718936 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c00c850 con 0x7f8e3c06f990 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e34009a70 con 0x7f8e3c06f990 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e2affd700 1 -- 172.21.15.5:0/3394718936 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00cbe0 con 0x7f8e3c06f990 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.962+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c155500 con 0x7f8e3c06f990 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.963+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c1e5bc0 con 0x7f8e3c06f990 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.964+0000 7f8e2affd700 1 -- 172.21.15.5:0/3394718936 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8e2c00cd40 con 0x7f8e3c06f990 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.964+0000 7f8e2affd700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8e1c06ac90 0x7f8e1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.964+0000 7f8e2affd700 1 -- 172.21.15.5:0/3394718936 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8e2c089800 con 0x7f8e3c06f990 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.965+0000 7f8e39d9b700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8e1c06ac90 0x7f8e1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.965+0000 7f8e39d9b700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8e1c06ac90 0x7f8e1c06d150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0c8590 tx=0x7f8e30007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:29.969+0000 7f8e2affd700 1 -- 172.21.15.5:0/3394718936 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8e2c054890 con 0x7f8e3c06f990 2024-05-08T23:24:30.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.186+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e3c074bc0 con 0x7f8e3c06f990 2024-05-08T23:24:30.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.186+0000 7f8e2affd700 1 -- 172.21.15.5:0/3394718936 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e2c057eb0 con 0x7f8e3c06f990 2024-05-08T23:24:30.186 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:24:30.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.188+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8e1c06ac90 msgr2=0x7f8e1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.188+0000 7f8e40880700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8e1c06ac90 0x7f8e1c06d150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0c8590 tx=0x7f8e30007dc0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.188+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c06f990 msgr2=0x7f8e3c1545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.188+0000 7f8e40880700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c06f990 0x7f8e3c1545b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00ebe0 tx=0x7f8e2c00eef0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.189+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 shutdown_connections 2024-05-08T23:24:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.189+0000 7f8e40880700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8e1c06ac90 0x7f8e1c06d150 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.189+0000 7f8e40880700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e3c15db00 0x7f8e3c1e7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.189+0000 7f8e40880700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c06f990 0x7f8e3c1545b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.189+0000 7f8e40880700 1 --2- 172.21.15.5:0/3394718936 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e3c0caa50 0x7f8e3c0b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.189+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 >> 172.21.15.5:0/3394718936 conn(0x7f8e3c099a50 msgr2=0x7f8e3c156130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.189+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 shutdown_connections 2024-05-08T23:24:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.189+0000 7f8e40880700 1 -- 172.21.15.5:0/3394718936 wait complete. 2024-05-08T23:24:30.202 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.339+0000 7f76b1a06700 1 Processor -- start 2024-05-08T23:24:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.339+0000 7f76b1a06700 1 -- start start 2024-05-08T23:24:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76b1a06700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f76ac15d780 0x7f76ac0b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76b1a06700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76ac0d13e0 0x7f76ac0d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76b1a06700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ac0b4430 con 0x7f76ac15db60 2024-05-08T23:24:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76b1a06700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ac0d45c0 con 0x7f76ac0d13e0 2024-05-08T23:24:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76b1a06700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ac0b3820 con 0x7f76ac15d780 2024-05-08T23:24:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76aa7fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76ac0d13e0 0x7f76ac0d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76ab7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f76ac15db60 0x7f76ac0ce3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34900/0 (socket says 172.21.15.5:34900) 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76ab7fe700 1 -- 172.21.15.5:0/2671163528 learned_addr learned my addr 172.21.15.5:0/2671163528 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.340+0000 7f76aaffd700 1 --2- 172.21.15.5:0/2671163528 >> v2:172.21.15.49:3300/0 conn(0x7f76ac15d780 0x7f76ac0b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76aa7fc700 1 -- 172.21.15.5:0/2671163528 >> v2:172.21.15.49:3300/0 conn(0x7f76ac15d780 msgr2=0x7f76ac0b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76aa7fc700 1 --2- 172.21.15.5:0/2671163528 >> v2:172.21.15.49:3300/0 conn(0x7f76ac15d780 0x7f76ac0b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76aa7fc700 1 -- 172.21.15.5:0/2671163528 >> v1:172.21.15.5:6789/0 conn(0x7f76ac15db60 legacy=0x7f76ac0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76aa7fc700 1 -- 172.21.15.5:0/2671163528 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac0b3b00 con 0x7f76ac0d13e0 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76a9ffb700 1 -- 172.21.15.5:0/2671163528 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765233836 0 0) 0x7f76ac0b4430 con 0x7f76ac15db60 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76aa7fc700 1 --2- 172.21.15.5:0/2671163528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac0d1be0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f76ac06b640 tx=0x7f76a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65a8877ea039284 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76a9ffb700 1 -- 172.21.15.5:0/2671163528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011830 con 0x7f76ac0d13e0 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76a9ffb700 1 -- 172.21.15.5:0/2671163528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a0011990 con 0x7f76ac0d13e0 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.341+0000 7f76a9ffb700 1 -- 172.21.15.5:0/2671163528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011d20 con 0x7f76ac0d13e0 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.342+0000 7f76b1a06700 1 -- 172.21.15.5:0/2671163528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 msgr2=0x7f76ac0d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.342+0000 7f76b1a06700 1 --2- 172.21.15.5:0/2671163528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac0d1be0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f76ac06b640 tx=0x7f76a0009b20 comp rx=0 tx=0).stop 2024-05-08T23:24:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.342+0000 7f76b1a06700 1 -- 172.21.15.5:0/2671163528 shutdown_connections 2024-05-08T23:24:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.342+0000 7f76b1a06700 1 --2- 172.21.15.5:0/2671163528 >> v2:172.21.15.49:3300/0 conn(0x7f76ac15d780 0x7f76ac0b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.342+0000 7f76b1a06700 1 --2- 172.21.15.5:0/2671163528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac0d1be0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.342+0000 7f76b1a06700 1 -- 172.21.15.5:0/2671163528 >> 172.21.15.5:0/2671163528 conn(0x7f76ac08afe0 msgr2=0x7f76ac0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.342+0000 7f76b1a06700 1 -- 172.21.15.5:0/2671163528 shutdown_connections 2024-05-08T23:24:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.342+0000 7f76b1a06700 1 -- 172.21.15.5:0/2671163528 wait complete. 2024-05-08T23:24:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.343+0000 7f76b1a06700 1 Processor -- start 2024-05-08T23:24:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.343+0000 7f76b1a06700 1 -- start start 2024-05-08T23:24:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76b1a06700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76b1a06700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f76ac15d780 0x7f76ac0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76b1a06700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ac15db60 0x7f76ac0c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76b1a06700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0b91e0 con 0x7f76ac0d13e0 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76b1a06700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0c9f50 con 0x7f76ac15db60 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76b1a06700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac06d480 con 0x7f76ac15d780 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76aaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76aaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41516/0 (socket says 172.21.15.5:41516) 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76aaffd700 1 -- 172.21.15.5:0/3215201705 learned_addr learned my addr 172.21.15.5:0/3215201705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76ab7fe700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ac15db60 0x7f76ac0c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76aaffd700 1 -- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f76ac15d780 msgr2=0x7f76ac0b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76aaffd700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f76ac15d780 0x7f76ac0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76aaffd700 1 -- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ac15db60 msgr2=0x7f76ac0c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76aaffd700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ac15db60 0x7f76ac0c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.344+0000 7f76aaffd700 1 -- 172.21.15.5:0/3215201705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00094a0 con 0x7f76ac0d13e0 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.345+0000 7f76ab7fe700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ac15db60 0x7f76ac0c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:30.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.345+0000 7f76aaffd700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac157050 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e730 tx=0x7f769c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:30.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.345+0000 7f7693fff700 1 -- 172.21.15.5:0/3215201705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c013610 con 0x7f76ac0d13e0 2024-05-08T23:24:30.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.345+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ac0c9990 con 0x7f76ac0d13e0 2024-05-08T23:24:30.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.345+0000 7f7693fff700 1 -- 172.21.15.5:0/3215201705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c013770 con 0x7f76ac0d13e0 2024-05-08T23:24:30.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.345+0000 7f7693fff700 1 -- 172.21.15.5:0/3215201705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c013b20 con 0x7f76ac0d13e0 2024-05-08T23:24:30.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.345+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac0b6b30 con 0x7f76ac0d13e0 2024-05-08T23:24:30.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.346+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac0d1a50 con 0x7f76ac0d13e0 2024-05-08T23:24:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.347+0000 7f7693fff700 1 -- 172.21.15.5:0/3215201705 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f769c013c80 con 0x7f76ac0d13e0 2024-05-08T23:24:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.347+0000 7f7693fff700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f769806ac90 0x7f769806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.347+0000 7f7693fff700 1 -- 172.21.15.5:0/3215201705 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f769c089030 con 0x7f76ac0d13e0 2024-05-08T23:24:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.348+0000 7f76aa7fc700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f769806ac90 0x7f769806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.348+0000 7f76aa7fc700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f769806ac90 0x7f769806d150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0c6f80 tx=0x7f76a0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.352+0000 7f7693fff700 1 -- 172.21.15.5:0/3215201705 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f769c0541e0 con 0x7f76ac0d13e0 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.622+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76ac0678f0 con 0x7f76ac0d13e0 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.622+0000 7f7693fff700 1 -- 172.21.15.5:0/3215201705 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4759 (secure 0 0 0) 0x7f769c057800 con 0x7f76ac0d13e0 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:24:15.731387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5778,"mds_1":5566,"mds_2":5180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5180":{"gid":5180,"name":"d","rank":2,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5566":{"gid":5566,"name":"b","rank":1,"incarnation":33,"state":"up:active","state_seq":13,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f769806ac90 msgr2=0x7f769806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f769806ac90 0x7f769806d150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0c6f80 tx=0x7f76a0010000 comp rx=0 tx=0).stop 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 msgr2=0x7f76ac157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac157050 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e730 tx=0x7f769c00eaf0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 shutdown_connections 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f769806ac90 0x7f769806d150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76ac15db60 0x7f76ac0c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac0d13e0 0x7f76ac157050 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 --2- 172.21.15.5:0/3215201705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f76ac15d780 0x7f76ac0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 >> 172.21.15.5:0/3215201705 conn(0x7f76ac08afe0 msgr2=0x7f76ac0b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 shutdown_connections 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.625+0000 7f76b1a06700 1 -- 172.21.15.5:0/3215201705 wait complete. 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 41 2024-05-08T23:24:30.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.774+0000 7fe6bd247700 1 Processor -- start 2024-05-08T23:24:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.774+0000 7fe6bd247700 1 -- start start 2024-05-08T23:24:30.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.775+0000 7fe6bd247700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b815ef60 0x7fe6b80b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.775+0000 7fe6bd247700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b809ad40 con 0x7fe6b80cf000 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.775+0000 7fe6bd247700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b80ccb60 con 0x7fe6b80cfb90 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.775+0000 7fe6bd247700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80cd400 con 0x7fe6b815ef60 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.775+0000 7fe6b659c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe6b80cf000 0x7fe6b80cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34902/0 (socket says 172.21.15.5:34902) 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.775+0000 7fe6b659c700 1 -- 172.21.15.5:0/209052390 learned_addr learned my addr 172.21.15.5:0/209052390 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b6d9d700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b815ef60 0x7fe6b80b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791331662 0 0) 0x7fe6b809ad40 con 0x7fe6b80cf000 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a40035b0 con 0x7fe6b80cf000 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895581604 0 0) 0x7fe6a40035b0 con 0x7fe6b80cf000 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b815ef60 msgr2=0x7fe6b80b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b815ef60 0x7fe6b80b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 >> v1:172.21.15.49:6789/0 conn(0x7fe6b80cfb90 legacy=0x7fe6b80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80cce40 con 0x7fe6b80cf000 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fe6a0001670 con 0x7fe6b80cf000 2024-05-08T23:24:30.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 >> v1:172.21.15.5:6789/0 conn(0x7fe6b80cf000 legacy=0x7fe6b80cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a4003a30 0x7fe6a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a40043e0 0x7fe6a40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b815ef60 0x7fe6a40051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80cd400 con 0x7fe6a4003a30 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a400bac0 con 0x7fe6a40043e0 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a400bc80 con 0x7fe6b815ef60 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b6d9d700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a4003a30 0x7fe6a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b659c700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a40043e0 0x7fe6a40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.776+0000 7fe6b759e700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b815ef60 0x7fe6a40051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b6d9d700 1 -- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b815ef60 msgr2=0x7fe6a40051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b6d9d700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b815ef60 0x7fe6a40051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b6d9d700 1 -- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a40043e0 msgr2=0x7fe6a40048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b6d9d700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a40043e0 0x7fe6a40048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b6d9d700 1 -- 172.21.15.5:0/209052390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40035b0 con 0x7fe6a4003a30 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b659c700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a40043e0 0x7fe6a40048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b6d9d700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a4003a30 0x7fe6a4003e10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009b60 tx=0x7fe6a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd69307c143b8f90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800e6c0 con 0x7fe6a4003a30 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a800e820 con 0x7fe6a4003a30 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b5d9b700 1 -- 172.21.15.5:0/209052390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800eb90 con 0x7fe6a4003a30 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6bd247700 1 -- 172.21.15.5:0/209052390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a4003a30 msgr2=0x7fe6a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6bd247700 1 --2- 172.21.15.5:0/209052390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a4003a30 0x7fe6a4003e10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009b60 tx=0x7fe6a4005880 comp rx=0 tx=0).stop 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6b759e700 1 -- 172.21.15.5:0/209052390 reap_dead start 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6bd247700 1 -- 172.21.15.5:0/209052390 shutdown_connections 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6bd247700 1 -- 172.21.15.5:0/209052390 >> 172.21.15.5:0/209052390 conn(0x7fe6b80b2f20 msgr2=0x7fe6b80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.777+0000 7fe6bd247700 1 -- 172.21.15.5:0/209052390 shutdown_connections 2024-05-08T23:24:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.778+0000 7fe6bd247700 1 -- 172.21.15.5:0/209052390 wait complete. 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.779+0000 7fe6bd247700 1 Processor -- start 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.780+0000 7fe6bd247700 1 -- start start 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6bd247700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4003a30 0x7fe6b8159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6bd247700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b80c7c80 0x7fe6b815d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6bd247700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b80c5610 0x7fe6b815d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6bd247700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80c9e40 con 0x7fe6b80c5610 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6bd247700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80c5af0 con 0x7fe6a4003a30 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6bd247700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80c97f0 con 0x7fe6b80c7c80 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b80c5610 0x7fe6b815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b80c5610 0x7fe6b815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41540/0 (socket says 172.21.15.5:41540) 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 -- 172.21.15.5:0/1982379169 learned_addr learned my addr 172.21.15.5:0/1982379169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b659c700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b80c7c80 0x7fe6b815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b6d9d700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4003a30 0x7fe6b8159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 -- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b80c7c80 msgr2=0x7fe6b815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b80c7c80 0x7fe6b815d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 -- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4003a30 msgr2=0x7fe6b8159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4003a30 0x7fe6b8159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 -- 172.21.15.5:0/1982379169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80c9ad0 con 0x7fe6b80c5610 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6b759e700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b80c5610 0x7fe6b815d950 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80724e0 tx=0x7fe6b000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6af7fe700 1 -- 172.21.15.5:0/1982379169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0003c10 con 0x7fe6b80c5610 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6af7fe700 1 -- 172.21.15.5:0/1982379169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0003d90 con 0x7fe6b80c5610 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b80c4d20 con 0x7fe6b80c5610 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.781+0000 7fe6af7fe700 1 -- 172.21.15.5:0/1982379169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0005310 con 0x7fe6b80c5610 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.782+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b80c8500 con 0x7fe6b80c5610 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.782+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b8068000 con 0x7fe6b80c5610 2024-05-08T23:24:30.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.783+0000 7fe6af7fe700 1 -- 172.21.15.5:0/1982379169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe6b0005470 con 0x7fe6b80c5610 2024-05-08T23:24:30.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.783+0000 7fe6af7fe700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe69806ac90 0x7fe69806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:30.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.784+0000 7fe6af7fe700 1 -- 172.21.15.5:0/1982379169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe6b0089430 con 0x7fe6b80c5610 2024-05-08T23:24:30.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.784+0000 7fe6b6d9d700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe69806ac90 0x7fe69806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:30.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.784+0000 7fe6b6d9d700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe69806ac90 0x7fe69806d150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009a80 tx=0x7fe6a8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:30.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:30.788+0000 7fe6af7fe700 1 -- 172.21.15.5:0/1982379169 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe6b00545c0 con 0x7fe6b80c5610 2024-05-08T23:24:31.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.005+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6b806ba70 con 0x7fe6b80c5610 2024-05-08T23:24:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.006+0000 7fe6af7fe700 1 -- 172.21.15.5:0/1982379169 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16100 (secure 0 0 0) 0x7fe6b0057be0 con 0x7fe6b80c5610 2024-05-08T23:24:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:31.006 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":48,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:53.673469+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:23:37.249225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T23:23:38.439923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.2d","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:31.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe69806ac90 msgr2=0x7fe69806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe69806ac90 0x7fe69806d150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009a80 tx=0x7fe6a8008030 comp rx=0 tx=0).stop 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b80c5610 msgr2=0x7fe6b815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b80c5610 0x7fe6b815d950 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80724e0 tx=0x7fe6b000ef20 comp rx=0 tx=0).stop 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 shutdown_connections 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe69806ac90 0x7fe69806d150 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b80c5610 0x7fe6b815d950 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4003a30 0x7fe6b8159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 --2- 172.21.15.5:0/1982379169 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe6b80c7c80 0x7fe6b815d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 >> 172.21.15.5:0/1982379169 conn(0x7fe6b80b2f20 msgr2=0x7fe6b815bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 shutdown_connections 2024-05-08T23:24:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.008+0000 7fe6bd247700 1 -- 172.21.15.5:0/1982379169 wait complete. 2024-05-08T23:24:31.022 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:24:31.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:31.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.159+0000 7fb92e513700 1 Processor -- start 2024-05-08T23:24:31.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.159+0000 7fb92e513700 1 -- start start 2024-05-08T23:24:31.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb92e513700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb92815ef60 0x7fb9280b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb92e513700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9280cf000 0x7fb9280cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb92e513700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9280cfe10 con 0x7fb9280cf9a0 2024-05-08T23:24:31.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb92e513700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280cd400 con 0x7fb9280cf000 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb92e513700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92809ad40 con 0x7fb92815ef60 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb9277fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9280cf000 0x7fb9280cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb92cab0700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb9280cf9a0 0x7fb928099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34910/0 (socket says 172.21.15.5:34910) 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb92cab0700 1 -- 172.21.15.5:0/1140827093 learned_addr learned my addr 172.21.15.5:0/1140827093 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb927fff700 1 --2- 172.21.15.5:0/1140827093 >> v2:172.21.15.49:3300/0 conn(0x7fb92815ef60 0x7fb9280b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb9277fe700 1 -- 172.21.15.5:0/1140827093 >> v2:172.21.15.49:3300/0 conn(0x7fb92815ef60 msgr2=0x7fb9280b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb9277fe700 1 --2- 172.21.15.5:0/1140827093 >> v2:172.21.15.49:3300/0 conn(0x7fb92815ef60 0x7fb9280b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb9277fe700 1 -- 172.21.15.5:0/1140827093 >> v1:172.21.15.5:6789/0 conn(0x7fb9280cf9a0 legacy=0x7fb928099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb9277fe700 1 -- 172.21.15.5:0/1140827093 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280ccb60 con 0x7fb9280cf000 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb9277fe700 1 --2- 172.21.15.5:0/1140827093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb9280cf460 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb92806e820 tx=0x7fb91c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31691b811175ac45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.160+0000 7fb926ffd700 1 -- 172.21.15.5:0/1140827093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c011830 con 0x7fb9280cf000 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb926ffd700 1 -- 172.21.15.5:0/1140827093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91c011990 con 0x7fb9280cf000 2024-05-08T23:24:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb926ffd700 1 -- 172.21.15.5:0/1140827093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c011d20 con 0x7fb9280cf000 2024-05-08T23:24:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb92e513700 1 -- 172.21.15.5:0/1140827093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 msgr2=0x7fb9280cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb92e513700 1 --2- 172.21.15.5:0/1140827093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb9280cf460 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb92806e820 tx=0x7fb91c009b20 comp rx=0 tx=0).stop 2024-05-08T23:24:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb92e513700 1 -- 172.21.15.5:0/1140827093 shutdown_connections 2024-05-08T23:24:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb92e513700 1 --2- 172.21.15.5:0/1140827093 >> v2:172.21.15.49:3300/0 conn(0x7fb92815ef60 0x7fb9280b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb92e513700 1 --2- 172.21.15.5:0/1140827093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb9280cf460 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb92e513700 1 -- 172.21.15.5:0/1140827093 >> 172.21.15.5:0/1140827093 conn(0x7fb9280b2f20 msgr2=0x7fb9280b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb92e513700 1 -- 172.21.15.5:0/1140827093 shutdown_connections 2024-05-08T23:24:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.161+0000 7fb92e513700 1 -- 172.21.15.5:0/1140827093 wait complete. 2024-05-08T23:24:31.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.163+0000 7fb92e513700 1 Processor -- start 2024-05-08T23:24:31.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.163+0000 7fb92e513700 1 -- start start 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb92e513700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb928150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb92e513700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9280cf9a0 0x7fb92809d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb92e513700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb92815ef60 0x7fb92809ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb92e513700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9281512c0 con 0x7fb9280cf000 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb92e513700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92809e3f0 con 0x7fb9280cf9a0 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb92e513700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92809e6d0 con 0x7fb92815ef60 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb928150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb928150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41564/0 (socket says 172.21.15.5:41564) 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 -- 172.21.15.5:0/3476851009 learned_addr learned my addr 172.21.15.5:0/3476851009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb9277fe700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9280cf9a0 0x7fb92809d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb92cab0700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb92815ef60 0x7fb92809ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 -- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb92815ef60 msgr2=0x7fb92809ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb92815ef60 0x7fb92809ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 -- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9280cf9a0 msgr2=0x7fb92809d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9280cf9a0 0x7fb92809d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 -- 172.21.15.5:0/3476851009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c0094a0 con 0x7fb9280cf000 2024-05-08T23:24:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb927fff700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb928150d80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb91800e810 tx=0x7fb91800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb924ff9700 1 -- 172.21.15.5:0/3476851009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91800c070 con 0x7fb9280cf000 2024-05-08T23:24:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb924ff9700 1 -- 172.21.15.5:0/3476851009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb918013610 con 0x7fb9280cf000 2024-05-08T23:24:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb924ff9700 1 -- 172.21.15.5:0/3476851009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180139c0 con 0x7fb9280cf000 2024-05-08T23:24:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.164+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92809e9b0 con 0x7fb9280cf000 2024-05-08T23:24:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.165+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92809ef70 con 0x7fb9280cf000 2024-05-08T23:24:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.166+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9280b4420 con 0x7fb9280cf000 2024-05-08T23:24:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.166+0000 7fb924ff9700 1 -- 172.21.15.5:0/3476851009 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb918013b20 con 0x7fb9280cf000 2024-05-08T23:24:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.166+0000 7fb924ff9700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb91406ac90 0x7fb91406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.167+0000 7fb924ff9700 1 -- 172.21.15.5:0/3476851009 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb918089100 con 0x7fb9280cf000 2024-05-08T23:24:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.167+0000 7fb9277fe700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb91406ac90 0x7fb91406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.167+0000 7fb9277fe700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb91406ac90 0x7fb91406d150 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0090b0 tx=0x7fb91c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.171+0000 7fb924ff9700 1 -- 172.21.15.5:0/3476851009 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb918054220 con 0x7fb9280cf000 2024-05-08T23:24:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.438+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb928068000 con 0x7fb9280cf000 2024-05-08T23:24:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.439+0000 7fb924ff9700 1 -- 172.21.15.5:0/3476851009 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4759 (secure 0 0 0) 0x7fb918057840 con 0x7fb9280cf000 2024-05-08T23:24:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:31.439 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:24:15.731387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5778,"mds_1":5566,"mds_2":5180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5180":{"gid":5180,"name":"d","rank":2,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5566":{"gid":5566,"name":"b","rank":1,"incarnation":33,"state":"up:active","state_seq":13,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:24:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.441+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb91406ac90 msgr2=0x7fb91406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.441+0000 7fb92e513700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb91406ac90 0x7fb91406d150 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0090b0 tx=0x7fb91c01c040 comp rx=0 tx=0).stop 2024-05-08T23:24:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.441+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 msgr2=0x7fb928150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.441+0000 7fb92e513700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb928150d80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb91800e810 tx=0x7fb91800eb20 comp rx=0 tx=0).stop 2024-05-08T23:24:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.442+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 shutdown_connections 2024-05-08T23:24:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.442+0000 7fb92e513700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb91406ac90 0x7fb91406d150 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.442+0000 7fb92e513700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb92815ef60 0x7fb92809ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.442+0000 7fb92e513700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9280cf000 0x7fb928150d80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.442+0000 7fb92e513700 1 --2- 172.21.15.5:0/3476851009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9280cf9a0 0x7fb92809d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.442+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 >> 172.21.15.5:0/3476851009 conn(0x7fb9280b2f20 msgr2=0x7fb92815e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:31.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.442+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 shutdown_connections 2024-05-08T23:24:31.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.442+0000 7fb92e513700 1 -- 172.21.15.5:0/3476851009 wait complete. 2024-05-08T23:24:31.443 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 41 2024-05-08T23:24:31.456 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.592+0000 7f7d8ee94700 1 Processor -- start 2024-05-08T23:24:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.592+0000 7f7d8ee94700 1 -- start start 2024-05-08T23:24:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8ee94700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 0x7f7d880cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8ee94700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d8809a160 0x7f7d880ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8ee94700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8809aca0 con 0x7f7d880d0520 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8ee94700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880cd9e0 con 0x7f7d8809a160 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8ee94700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880cf220 con 0x7f7d8815b810 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8d431700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d8809a160 0x7f7d880ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8d431700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d8809a160 0x7f7d880ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41574/0 (socket says 172.21.15.5:41574) 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8d431700 1 -- 172.21.15.5:0/3377312336 learned_addr learned my addr 172.21.15.5:0/3377312336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8cc30700 1 --2- 172.21.15.5:0/3377312336 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 0x7f7d880cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8d431700 1 -- 172.21.15.5:0/3377312336 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 msgr2=0x7f7d880cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8d431700 1 --2- 172.21.15.5:0/3377312336 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 0x7f7d880cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8d431700 1 -- 172.21.15.5:0/3377312336 >> v1:172.21.15.49:6789/0 conn(0x7f7d880d0520 legacy=0x7f7d88099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8d431700 1 -- 172.21.15.5:0/3377312336 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d880cf500 con 0x7f7d8809a160 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d7f7fe700 1 -- 172.21.15.5:0/3377312336 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083385591 0 0) 0x7f7d8809aca0 con 0x7f7d880d0520 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.593+0000 7f7d8d431700 1 --2- 172.21.15.5:0/3377312336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d8809a160 0x7f7d880ccdd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009e80 tx=0x7f7d8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ff55656c857269 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d7f7fe700 1 -- 172.21.15.5:0/3377312336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000e070 con 0x7f7d8809a160 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d7f7fe700 1 -- 172.21.15.5:0/3377312336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000b8d0 con 0x7f7d8809a160 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d7f7fe700 1 -- 172.21.15.5:0/3377312336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000bc80 con 0x7f7d8809a160 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3377312336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d8809a160 msgr2=0x7f7d880ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3377312336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d8809a160 0x7f7d880ccdd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009e80 tx=0x7f7d8000b330 comp rx=0 tx=0).stop 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3377312336 shutdown_connections 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3377312336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d8809a160 0x7f7d880ccdd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3377312336 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 0x7f7d880cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3377312336 >> 172.21.15.5:0/3377312336 conn(0x7f7d880b3010 msgr2=0x7f7d880b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3377312336 shutdown_connections 2024-05-08T23:24:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.594+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3377312336 wait complete. 2024-05-08T23:24:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.596+0000 7f7d8ee94700 1 Processor -- start 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.596+0000 7f7d8ee94700 1 -- start start 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d8ee94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8809a160 0x7f7d8809bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d8ee94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880d0520 0x7f7d8809c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d8ee94700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 0x7f7d8809b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d8ee94700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b4140 con 0x7f7d880d0520 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d8ee94700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b4fa0 con 0x7f7d8809a160 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d8ee94700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b4390 con 0x7f7d8815b810 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d7ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880d0520 0x7f7d8809c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d7ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880d0520 0x7f7d8809c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41580/0 (socket says 172.21.15.5:41580) 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.597+0000 7f7d7ffff700 1 -- 172.21.15.5:0/3285058035 learned_addr learned my addr 172.21.15.5:0/3285058035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d8cc30700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8809a160 0x7f7d8809bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d8d431700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 0x7f7d8809b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7ffff700 1 -- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 msgr2=0x7f7d8809b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7ffff700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 0x7f7d8809b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7ffff700 1 -- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8809a160 msgr2=0x7f7d8809bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7ffff700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8809a160 0x7f7d8809bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7ffff700 1 -- 172.21.15.5:0/3285058035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80009b30 con 0x7f7d880d0520 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7ffff700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880d0520 0x7f7d8809c530 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800c520 tx=0x7f7d7800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7d7fa700 1 -- 172.21.15.5:0/3285058035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780054e0 con 0x7f7d880d0520 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7d7fa700 1 -- 172.21.15.5:0/3285058035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d78003680 con 0x7f7d880d0520 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d7d7fa700 1 -- 172.21.15.5:0/3285058035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78003a30 con 0x7f7d880d0520 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d880b4670 con 0x7f7d880d0520 2024-05-08T23:24:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.598+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d880b4c30 con 0x7f7d880d0520 2024-05-08T23:24:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.599+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d880b5110 con 0x7f7d880d0520 2024-05-08T23:24:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.600+0000 7f7d7d7fa700 1 -- 172.21.15.5:0/3285058035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7d78003b90 con 0x7f7d880d0520 2024-05-08T23:24:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.600+0000 7f7d7d7fa700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d7406ac90 0x7f7d7406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.600+0000 7f7d7d7fa700 1 -- 172.21.15.5:0/3285058035 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7d780887e0 con 0x7f7d880d0520 2024-05-08T23:24:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.601+0000 7f7d8cc30700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d7406ac90 0x7f7d7406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:31.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.601+0000 7f7d8cc30700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d7406ac90 0x7f7d7406d150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009b40 tx=0x7f7d84009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.605+0000 7f7d7d7fa700 1 -- 172.21.15.5:0/3285058035 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7d78053970 con 0x7f7d880d0520 2024-05-08T23:24:31.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.871+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d880aeb40 con 0x7f7d880d0520 2024-05-08T23:24:31.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.872+0000 7f7d7d7fa700 1 -- 172.21.15.5:0/3285058035 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4759 (secure 0 0 0) 0x7f7d78017090 con 0x7f7d880d0520 2024-05-08T23:24:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:31.872 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:23:39.466617+0000","modified":"2024-05-08T23:24:15.731387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5778,"mds_1":5566,"mds_2":5180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5180":{"gid":5180,"name":"d","rank":2,"incarnation":35,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/350829837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":350829837},{"type":"v1","addr":"172.21.15.49:6837","nonce":350829837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5566":{"gid":5566,"name":"b","rank":1,"incarnation":33,"state":"up:active","state_seq":13,"addr":"172.21.15.49:6835/962044603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":962044603},{"type":"v1","addr":"172.21.15.49:6835","nonce":962044603}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.5:6835/3359342067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3359342067},{"type":"v1","addr":"172.21.15.5:6835","nonce":3359342067}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:24:31.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d7406ac90 msgr2=0x7f7d7406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d7406ac90 0x7f7d7406d150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009b40 tx=0x7f7d84009210 comp rx=0 tx=0).stop 2024-05-08T23:24:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880d0520 msgr2=0x7f7d8809c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880d0520 0x7f7d8809c530 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800c520 tx=0x7f7d7800c830 comp rx=0 tx=0).stop 2024-05-08T23:24:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 shutdown_connections 2024-05-08T23:24:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d7406ac90 0x7f7d7406d150 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d8815b810 0x7f7d8809b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d8809a160 0x7f7d8809bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 --2- 172.21.15.5:0/3285058035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d880d0520 0x7f7d8809c530 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 >> 172.21.15.5:0/3285058035 conn(0x7f7d880b3010 msgr2=0x7f7d880ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 shutdown_connections 2024-05-08T23:24:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:31.874+0000 7f7d8ee94700 1 -- 172.21.15.5:0/3285058035 wait complete. 2024-05-08T23:24:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 41 2024-05-08T23:24:31.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.025+0000 7f87226a3700 1 Processor -- start 2024-05-08T23:24:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.025+0000 7f87226a3700 1 -- start start 2024-05-08T23:24:32.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.025+0000 7f87226a3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f871c0cd4e0 0x7f871c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.025+0000 7f87226a3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f871c06f130 0x7f871c0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f87226a3700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c15c280 con 0x7f871c0ceb20 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f87226a3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c0cefb0 con 0x7f871c0cd4e0 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f87226a3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c06e780 con 0x7f871c06f130 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871bfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f871c0cd4e0 0x7f871c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871bfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f871c0cd4e0 0x7f871c06f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41596/0 (socket says 172.21.15.5:41596) 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871bfff700 1 -- 172.21.15.5:0/4272992988 learned_addr learned my addr 172.21.15.5:0/4272992988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871b7fe700 1 --2- 172.21.15.5:0/4272992988 >> v2:172.21.15.49:3300/0 conn(0x7f871c06f130 0x7f871c0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871bfff700 1 -- 172.21.15.5:0/4272992988 >> v2:172.21.15.49:3300/0 conn(0x7f871c06f130 msgr2=0x7f871c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871bfff700 1 --2- 172.21.15.5:0/4272992988 >> v2:172.21.15.49:3300/0 conn(0x7f871c06f130 0x7f871c0ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871bfff700 1 -- 172.21.15.5:0/4272992988 >> v1:172.21.15.49:6789/0 conn(0x7f871c0ceb20 legacy=0x7f871c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871bfff700 1 -- 172.21.15.5:0/4272992988 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c06ea60 con 0x7f871c0cd4e0 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871b7fe700 1 --2- 172.21.15.5:0/4272992988 >> v2:172.21.15.49:3300/0 conn(0x7f871c06f130 0x7f871c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871bfff700 1 --2- 172.21.15.5:0/4272992988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0cd4e0 0x7f871c06f910 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f871c0c3e20 tx=0x7f870c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d3752bfa192ffc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871affd700 1 -- 172.21.15.5:0/4272992988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00f8a0 con 0x7f871c0cd4e0 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871affd700 1 -- 172.21.15.5:0/4272992988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c00fa00 con 0x7f871c0cd4e0 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.026+0000 7f871affd700 1 -- 172.21.15.5:0/4272992988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00fd90 con 0x7f871c0cd4e0 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.027+0000 7f87226a3700 1 -- 172.21.15.5:0/4272992988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0cd4e0 msgr2=0x7f871c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.027+0000 7f87226a3700 1 --2- 172.21.15.5:0/4272992988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0cd4e0 0x7f871c06f910 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f871c0c3e20 tx=0x7f870c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.027+0000 7f87226a3700 1 -- 172.21.15.5:0/4272992988 shutdown_connections 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.027+0000 7f87226a3700 1 --2- 172.21.15.5:0/4272992988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0cd4e0 0x7f871c06f910 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.027+0000 7f87226a3700 1 --2- 172.21.15.5:0/4272992988 >> v2:172.21.15.49:3300/0 conn(0x7f871c06f130 0x7f871c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.027+0000 7f87226a3700 1 -- 172.21.15.5:0/4272992988 >> 172.21.15.5:0/4272992988 conn(0x7f871c0d0320 msgr2=0x7f871c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.027+0000 7f87226a3700 1 -- 172.21.15.5:0/4272992988 shutdown_connections 2024-05-08T23:24:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.027+0000 7f87226a3700 1 -- 172.21.15.5:0/4272992988 wait complete. 2024-05-08T23:24:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.028+0000 7f87226a3700 1 Processor -- start 2024-05-08T23:24:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.029+0000 7f87226a3700 1 -- start start 2024-05-08T23:24:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.029+0000 7f87226a3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f871c06f130 0x7f871c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.029+0000 7f87226a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f871c0cd4e0 0x7f871c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.029+0000 7f87226a3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0ceb20 0x7f871c0cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.029+0000 7f87226a3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1f4490 con 0x7f871c0ceb20 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.029+0000 7f87226a3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1f3810 con 0x7f871c0cd4e0 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.029+0000 7f87226a3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1f3af0 con 0x7f871c06f130 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0ceb20 0x7f871c0cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0ceb20 0x7f871c0cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41610/0 (socket says 172.21.15.5:41610) 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 -- 172.21.15.5:0/655247725 learned_addr learned my addr 172.21.15.5:0/655247725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f871b7fe700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f871c0cd4e0 0x7f871c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f871bfff700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f871c06f130 0x7f871c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 -- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f871c06f130 msgr2=0x7f871c156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f871c06f130 0x7f871c156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 -- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f871c0cd4e0 msgr2=0x7f871c0c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f871c0cd4e0 0x7f871c0c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 -- 172.21.15.5:0/655247725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c009550 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8720c40700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0ceb20 0x7f871c0cacc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f871400cb50 tx=0x7f871400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8718ff9700 1 -- 172.21.15.5:0/655247725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714003c00 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8718ff9700 1 -- 172.21.15.5:0/655247725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8714003d60 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c1f3dd0 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f8718ff9700 1 -- 172.21.15.5:0/655247725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871400ab70 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.030+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c157480 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.031+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c068000 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.032+0000 7f8718ff9700 1 -- 172.21.15.5:0/655247725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f871400acd0 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.032+0000 7f8718ff9700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870806ac90 0x7f870806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.032+0000 7f8718ff9700 1 -- 172.21.15.5:0/655247725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8714089320 con 0x7f871c0ceb20 2024-05-08T23:24:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.033+0000 7f871bfff700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870806ac90 0x7f870806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.033+0000 7f871bfff700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870806ac90 0x7f870806d150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f870c000c00 tx=0x7f870c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.037+0000 7f8718ff9700 1 -- 172.21.15.5:0/655247725 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8714054550 con 0x7f871c0ceb20 2024-05-08T23:24:32.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.254+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f871c06d440 con 0x7f871c0ceb20 2024-05-08T23:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.256+0000 7f8718ff9700 1 -- 172.21.15.5:0/655247725 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16100 (secure 0 0 0) 0x7f8714057b70 con 0x7f871c0ceb20 2024-05-08T23:24:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:32.256 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":48,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:23:53.673469+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:23:37.249225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T23:23:38.439923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.2d","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870806ac90 msgr2=0x7f870806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870806ac90 0x7f870806d150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f870c000c00 tx=0x7f870c008030 comp rx=0 tx=0).stop 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0ceb20 msgr2=0x7f871c0cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0ceb20 0x7f871c0cacc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f871400cb50 tx=0x7f871400cf10 comp rx=0 tx=0).stop 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 shutdown_connections 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870806ac90 0x7f870806d150 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c0ceb20 0x7f871c0cacc0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f871c06f130 0x7f871c156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 --2- 172.21.15.5:0/655247725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f871c0cd4e0 0x7f871c0c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 >> 172.21.15.5:0/655247725 conn(0x7f871c0d0320 msgr2=0x7f871c15b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 shutdown_connections 2024-05-08T23:24:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.258+0000 7f87226a3700 1 -- 172.21.15.5:0/655247725 wait complete. 2024-05-08T23:24:32.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:24:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.409+0000 7fda146b3700 1 Processor -- start 2024-05-08T23:24:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.409+0000 7fda146b3700 1 -- start start 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda146b3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fda0c06f130 0x7fda0c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda146b3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0c15c280 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda146b3700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0c06e780 con 0x7fda0c0cecf0 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda146b3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0c06ea60 con 0x7fda0c06f130 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda1244f700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fda0c0cd4e0 0x7fda0c0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34920/0 (socket says 172.21.15.5:34920) 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda1244f700 1 -- 172.21.15.5:0/1941804020 learned_addr learned my addr 172.21.15.5:0/1941804020 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda12c50700 1 --2- 172.21.15.5:0/1941804020 >> v2:172.21.15.49:3300/0 conn(0x7fda0c06f130 0x7fda0c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda1144d700 1 -- 172.21.15.5:0/1941804020 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243865003 0 0) 0x7fda0c15c280 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.410+0000 7fda1144d700 1 -- 172.21.15.5:0/1941804020 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f80035b0 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.411+0000 7fda12c50700 1 -- 172.21.15.5:0/1941804020 >> v1:172.21.15.49:6789/0 conn(0x7fda0c0cecf0 legacy=0x7fda0c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.411+0000 7fda12c50700 1 -- 172.21.15.5:0/1941804020 >> v1:172.21.15.5:6789/0 conn(0x7fda0c0cd4e0 legacy=0x7fda0c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.411+0000 7fda12c50700 1 -- 172.21.15.5:0/1941804020 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c06ed40 con 0x7fda0c06f130 2024-05-08T23:24:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.411+0000 7fda12c50700 1 --2- 172.21.15.5:0/1941804020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c06f130 0x7fda0c06f910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda04009d90 tx=0x7fda0400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a4c93f14a0d4d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.411+0000 7fda1144d700 1 -- 172.21.15.5:0/1941804020 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400e070 con 0x7fda0c06f130 2024-05-08T23:24:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.411+0000 7fda1144d700 1 -- 172.21.15.5:0/1941804020 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0400bb20 con 0x7fda0c06f130 2024-05-08T23:24:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.411+0000 7fda1144d700 1 -- 172.21.15.5:0/1941804020 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400be90 con 0x7fda0c06f130 2024-05-08T23:24:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.412+0000 7fda146b3700 1 -- 172.21.15.5:0/1941804020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c06f130 msgr2=0x7fda0c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.412+0000 7fda146b3700 1 --2- 172.21.15.5:0/1941804020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c06f130 0x7fda0c06f910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda04009d90 tx=0x7fda0400b330 comp rx=0 tx=0).stop 2024-05-08T23:24:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.412+0000 7fda146b3700 1 -- 172.21.15.5:0/1941804020 shutdown_connections 2024-05-08T23:24:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.412+0000 7fda146b3700 1 --2- 172.21.15.5:0/1941804020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c06f130 0x7fda0c06f910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.412+0000 7fda146b3700 1 -- 172.21.15.5:0/1941804020 >> 172.21.15.5:0/1941804020 conn(0x7fda0c0d0320 msgr2=0x7fda0c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.412+0000 7fda146b3700 1 -- 172.21.15.5:0/1941804020 shutdown_connections 2024-05-08T23:24:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.412+0000 7fda146b3700 1 -- 172.21.15.5:0/1941804020 wait complete. 2024-05-08T23:24:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.413+0000 7fda146b3700 1 Processor -- start 2024-05-08T23:24:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.413+0000 7fda146b3700 1 -- start start 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda146b3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda0c06f130 0x7fda0c14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda146b3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda0c0cd4e0 0x7fda0c0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda146b3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c0cecf0 0x7fda0c0c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda146b3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c1f0ef0 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda146b3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c14a620 con 0x7fda0c0cecf0 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda146b3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0c68b0 con 0x7fda0c06f130 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda11c4e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda0c0cd4e0 0x7fda0c0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda11c4e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda0c0cd4e0 0x7fda0c0c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41616/0 (socket says 172.21.15.5:41616) 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda11c4e700 1 -- 172.21.15.5:0/3469598120 learned_addr learned my addr 172.21.15.5:0/3469598120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda1244f700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda0c06f130 0x7fda0c14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.414+0000 7fda12c50700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c0cecf0 0x7fda0c0c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda11c4e700 1 -- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda0c06f130 msgr2=0x7fda0c14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda11c4e700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda0c06f130 0x7fda0c14a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda11c4e700 1 -- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c0cecf0 msgr2=0x7fda0c0c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda11c4e700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c0cecf0 0x7fda0c0c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda11c4e700 1 -- 172.21.15.5:0/3469598120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04009a70 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda11c4e700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda0c0cd4e0 0x7fda0c0c6370 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c480 tx=0x7fd9fc00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda02ffd700 1 -- 172.21.15.5:0/3469598120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0052d0 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda02ffd700 1 -- 172.21.15.5:0/3469598120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc005430 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda02ffd700 1 -- 172.21.15.5:0/3469598120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0038c0 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c0c52d0 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.415+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c0c5890 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.416+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0c068000 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.417+0000 7fda02ffd700 1 -- 172.21.15.5:0/3469598120 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd9fc003a20 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.417+0000 7fda02ffd700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9f006ac90 0x7fd9f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.417+0000 7fda02ffd700 1 -- 172.21.15.5:0/3469598120 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd9fc0885b0 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.417+0000 7fda1244f700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9f006ac90 0x7fd9f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.418+0000 7fda1244f700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9f006ac90 0x7fd9f006d150 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda08007fd0 tx=0x7fda08007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.422+0000 7fda02ffd700 1 -- 172.21.15.5:0/3469598120 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd9fc053760 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.693+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fda0c06d440 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.693 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:24:32.693+0000 7f8a49d06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:24:32.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.700+0000 7fda02ffd700 1 -- 172.21.15.5:0/3469598120 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd9fc05d020 con 0x7fda0c0cd4e0 2024-05-08T23:24:32.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9f006ac90 msgr2=0x7fd9f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9f006ac90 0x7fd9f006d150 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda08007fd0 tx=0x7fda08007dc0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda0c0cd4e0 msgr2=0x7fda0c0c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda0c0cd4e0 0x7fda0c0c6370 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c480 tx=0x7fd9fc00c790 comp rx=0 tx=0).stop 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 shutdown_connections 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd9f006ac90 0x7fd9f006d150 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda0c0cecf0 0x7fda0c0c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda0c06f130 0x7fda0c14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 --2- 172.21.15.5:0/3469598120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda0c0cd4e0 0x7fda0c0c6370 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 >> 172.21.15.5:0/3469598120 conn(0x7fda0c0d0320 msgr2=0x7fda0c15b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 shutdown_connections 2024-05-08T23:24:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.703+0000 7fda146b3700 1 -- 172.21.15.5:0/3469598120 wait complete. 2024-05-08T23:24:32.705 INFO:tasks.ceph.mds.c.smithi005.stderr: -9939> 2024-05-08T23:23:48.986+0000 7f5d502e2700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:24:32.705 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:24:32.711 INFO:tasks.ceph.mds.d.smithi049.stderr: -1010> 2024-05-08T23:24:08.332+0000 7f15a3868700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1) n(v0 rc2024-05-08T23:23:51.415509+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={5255=pAsLsXsFsx/pAsLsXsFsx@4},l=5255 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5618a3dfb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.415509+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-05-08T23:23:51.415509+0000 2=0+2)","error_str":""},"return_code":-61} 2024-05-08T23:24:32.711 INFO:tasks.ceph.mds.d.smithi049.stderr: -995> 2024-05-08T23:24:08.332+0000 7f15a3868700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={5255=pAsLsXsFsx/AsLsXsFsx@2},l=5255 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5618a3dfb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-05-08T23:24:32.713 INFO:tasks.ceph.mds.c.smithi005.stderr: -2367> 2024-05-08T23:24:08.331+0000 7f5d4cadb700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1) n(v3 rc2024-05-08T23:23:51.671503+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={5255=pAsLsXsFsx/pAsLsXsFsx@5},l=5255 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56331a5da580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.671503+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-05-08T23:23:51.671503+0000 6=0+6)","error_str":""},"return_code":-61} 2024-05-08T23:24:32.713 INFO:tasks.ceph.mds.c.smithi005.stderr: -2273> 2024-05-08T23:24:08.332+0000 7f5d52ae7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1) n(v4 rc2024-05-08T23:23:51.671503+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={5255=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x56331a5dab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.671503+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.413509+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-05-08T23:23:51.671503+0000 5=0+5)","error_str":""},"return_code":-61} 2024-05-08T23:24:32.714 INFO:tasks.ceph.mds.b.smithi049.stderr: -1192> 2024-05-08T23:24:08.331+0000 7ff4ccee9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-05-08T23:23:51.414509+0000 1=0+1) n(v2 rc2024-05-08T23:23:51.671503+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={5255=pAsLsXsFsx/pAsLsXsFsx@4},l=5255 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55e7715ad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.414509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.671503+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.414509+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-05-08T23:23:51.671503+0000 4=0+4)","error_str":""},"return_code":-61} 2024-05-08T23:24:32.714 INFO:tasks.ceph.mds.b.smithi049.stderr: -1072> 2024-05-08T23:24:08.333+0000 7ff4d2ef5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1) n(v1 rc2024-05-08T23:23:51.671503+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={5255=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55e7715ad700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:23:51.415509+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-05-08T23:23:51.415509+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-05-08T23:23:51.671503+0000 3=0+3)","error_str":""},"return_code":-61} 2024-05-08T23:24:32.718 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:24:32.742 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:24:32.743 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:24:32.743 INFO:tasks.ceph.mds.d.smithi049.stderr:2024-05-08T23:24:32.743+0000 7fbd196b5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-08T23:24:32.745 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:24:32.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.858+0000 7fa13544e700 1 Processor -- start 2024-05-08T23:24:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.858+0000 7fa13544e700 1 -- start start 2024-05-08T23:24:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa13544e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1300d1e50 0x7fa1300ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa13544e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 0x7fa13006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa13544e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1300cf560 con 0x7fa1300d2230 2024-05-08T23:24:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa13544e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1300ce950 con 0x7fa1300d1e50 2024-05-08T23:24:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa13544e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300cec30 con 0x7fa130161320 2024-05-08T23:24:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa12effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1300d1e50 0x7fa1300ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa12effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1300d1e50 0x7fa1300ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41642/0 (socket says 172.21.15.5:41642) 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa1267fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa1300d2230 0x7fa130160c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34930/0 (socket says 172.21.15.5:34930) 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa12effd700 1 -- 172.21.15.5:0/1962791006 learned_addr learned my addr 172.21.15.5:0/1962791006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.859+0000 7fa12f7fe700 1 --2- 172.21.15.5:0/1962791006 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 0x7fa13006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12effd700 1 -- 172.21.15.5:0/1962791006 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 msgr2=0x7fa13006bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12effd700 1 --2- 172.21.15.5:0/1962791006 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 0x7fa13006bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12effd700 1 -- 172.21.15.5:0/1962791006 >> v1:172.21.15.5:6789/0 conn(0x7fa1300d2230 legacy=0x7fa130160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12effd700 1 -- 172.21.15.5:0/1962791006 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300cef10 con 0x7fa1300d1e50 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12e7fc700 1 -- 172.21.15.5:0/1962791006 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076547094 0 0) 0x7fa1300cf560 con 0x7fa1300d2230 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12effd700 1 --2- 172.21.15.5:0/1962791006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d1e50 0x7fa1300ce2b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098f0 tx=0x7fa120009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb367d187fc78666 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12e7fc700 1 -- 172.21.15.5:0/1962791006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120011830 con 0x7fa1300d1e50 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12e7fc700 1 -- 172.21.15.5:0/1962791006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa120011990 con 0x7fa1300d1e50 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.860+0000 7fa12e7fc700 1 -- 172.21.15.5:0/1962791006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120011d20 con 0x7fa1300d1e50 2024-05-08T23:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.861+0000 7fa13544e700 1 -- 172.21.15.5:0/1962791006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d1e50 msgr2=0x7fa1300ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.861+0000 7fa13544e700 1 --2- 172.21.15.5:0/1962791006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d1e50 0x7fa1300ce2b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098f0 tx=0x7fa120009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.861+0000 7fa13544e700 1 -- 172.21.15.5:0/1962791006 shutdown_connections 2024-05-08T23:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.861+0000 7fa13544e700 1 --2- 172.21.15.5:0/1962791006 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 0x7fa13006bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.861+0000 7fa13544e700 1 --2- 172.21.15.5:0/1962791006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d1e50 0x7fa1300ce2b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.861+0000 7fa13544e700 1 -- 172.21.15.5:0/1962791006 >> 172.21.15.5:0/1962791006 conn(0x7fa13008afe0 msgr2=0x7fa1300d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.861+0000 7fa13544e700 1 -- 172.21.15.5:0/1962791006 shutdown_connections 2024-05-08T23:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.861+0000 7fa13544e700 1 -- 172.21.15.5:0/1962791006 wait complete. 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.862+0000 7fa13544e700 1 Processor -- start 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.862+0000 7fa13544e700 1 -- start start 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa13544e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1300d1e50 0x7fa1300a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa13544e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d2230 0x7fa1300b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa13544e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 0x7fa1300b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa13544e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1301f4430 con 0x7fa1300d2230 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa13544e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13015b5e0 con 0x7fa1300d1e50 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa13544e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300a1350 con 0x7fa130161320 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa1267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d2230 0x7fa1300b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa1267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d2230 0x7fa1300b8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41650/0 (socket says 172.21.15.5:41650) 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa1267fc700 1 -- 172.21.15.5:0/956449040 learned_addr learned my addr 172.21.15.5:0/956449040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa12f7fe700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 0x7fa1300b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.863+0000 7fa12effd700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1300d1e50 0x7fa1300a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa1267fc700 1 -- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 msgr2=0x7fa1300b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa1267fc700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 0x7fa1300b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa1267fc700 1 -- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1300d1e50 msgr2=0x7fa1300a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa1267fc700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1300d1e50 0x7fa1300a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa1267fc700 1 -- 172.21.15.5:0/956449040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa120009550 con 0x7fa1300d2230 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa1267fc700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d2230 0x7fa1300b8490 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa1180011e0 tx=0x7fa11800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa127fff700 1 -- 172.21.15.5:0/956449040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118004c80 con 0x7fa1300d2230 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa127fff700 1 -- 172.21.15.5:0/956449040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa118004e00 con 0x7fa1300d2230 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1301f37f0 con 0x7fa1300d2230 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa127fff700 1 -- 172.21.15.5:0/956449040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118007310 con 0x7fa1300d2230 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.864+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1301f3d70 con 0x7fa1300d2230 2024-05-08T23:24:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.865+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1300cf9d0 con 0x7fa1300d2230 2024-05-08T23:24:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.866+0000 7fa127fff700 1 -- 172.21.15.5:0/956449040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa118007470 con 0x7fa1300d2230 2024-05-08T23:24:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.866+0000 7fa127fff700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa11006ac90 0x7fa11006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.866+0000 7fa127fff700 1 -- 172.21.15.5:0/956449040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fa118089030 con 0x7fa1300d2230 2024-05-08T23:24:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.866+0000 7fa12effd700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa11006ac90 0x7fa11006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.867+0000 7fa12effd700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa11006ac90 0x7fa11006d150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa120009c40 tx=0x7fa120008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:32.871+0000 7fa127fff700 1 -- 172.21.15.5:0/956449040 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa1300cf9d0 con 0x7fa1300d2230 2024-05-08T23:24:33.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.139+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa1301f4570 con 0x7fa1300d2230 2024-05-08T23:24:33.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.702+0000 7fa127fff700 1 -- 172.21.15.5:0/956449040 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa1301f4570 con 0x7fa1300d2230 2024-05-08T23:24:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa11006ac90 msgr2=0x7fa11006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa11006ac90 0x7fa11006d150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa120009c40 tx=0x7fa120008030 comp rx=0 tx=0).stop 2024-05-08T23:24:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d2230 msgr2=0x7fa1300b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d2230 0x7fa1300b8490 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa1180011e0 tx=0x7fa11800ca30 comp rx=0 tx=0).stop 2024-05-08T23:24:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 shutdown_connections 2024-05-08T23:24:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa11006ac90 0x7fa11006d150 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa130161320 0x7fa1300b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1300d1e50 0x7fa1300a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 --2- 172.21.15.5:0/956449040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1300d2230 0x7fa1300b8490 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 >> 172.21.15.5:0/956449040 conn(0x7fa13008afe0 msgr2=0x7fa1300b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 shutdown_connections 2024-05-08T23:24:33.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.706+0000 7fa13544e700 1 -- 172.21.15.5:0/956449040 wait complete. 2024-05-08T23:24:33.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:24:33.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.911+0000 7fa2cf212700 1 Processor -- start 2024-05-08T23:24:33.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.911+0000 7fa2cf212700 1 -- start start 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2cf212700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa2c80d13a0 0x7fa2c80d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2cf212700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 0x7fa2c80cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2cf212700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c80b1200 con 0x7fa2c80cdc90 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2cf212700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c80d1cc0 con 0x7fa2c80d13a0 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2cf212700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c80cfcc0 con 0x7fa2c806bd60 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2cd7af700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa2c80cdc90 0x7fa2c80b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34932/0 (socket says 172.21.15.5:34932) 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2cd7af700 1 -- 172.21.15.5:0/1859241068 learned_addr learned my addr 172.21.15.5:0/1859241068 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2c7fff700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 0x7fa2c80cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/1859241068 >> v2:172.21.15.49:3300/0 conn(0x7fa2c80d13a0 0x7fa2c80d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718127672 0 0) 0x7fa2c80b1200 con 0x7fa2c80cdc90 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2b80035b0 con 0x7fa2c80cdc90 2024-05-08T23:24:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.912+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2292523589 0 0) 0x7fa2b80035b0 con 0x7fa2c80cdc90 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 msgr2=0x7fa2c80cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 0x7fa2c80cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 >> v2:172.21.15.49:3300/0 conn(0x7fa2c80d13a0 msgr2=0x7fa2c80d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 --2- 172.21.15.5:0/1859241068 >> v2:172.21.15.49:3300/0 conn(0x7fa2c80d13a0 0x7fa2c80d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c80ce5c0 con 0x7fa2c80cdc90 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fa2c00019f0 con 0x7fa2c80cdc90 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/1859241068 >> v2:172.21.15.49:3300/0 conn(0x7fa2c80d13a0 0x7fa2c80d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c7fff700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 0x7fa2c80cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 >> v1:172.21.15.5:6789/0 conn(0x7fa2c80cdc90 legacy=0x7fa2c80b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8003a00 0x7fa2b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b8004850 0x7fa2b8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 0x7fa2b8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c80d1cc0 con 0x7fa2b8003a00 2024-05-08T23:24:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c80cfcc0 con 0x7fa2b8004850 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b800ba10 con 0x7fa2c806bd60 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8003a00 0x7fa2b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2cd7af700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 0x7fa2b8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2c7fff700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b8004850 0x7fa2b8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2ccfae700 1 -- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 msgr2=0x7fa2b8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2c806bd60 0x7fa2b8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2ccfae700 1 -- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b8004850 msgr2=0x7fa2b8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b8004850 0x7fa2b8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2ccfae700 1 -- 172.21.15.5:0/1859241068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b40094a0 con 0x7fa2b8003a00 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.913+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8003a00 0x7fa2b8003e20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009d30 tx=0x7fa2b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab44856f921cb31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc009e80 con 0x7fa2b8003a00 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00e9a0 con 0x7fa2b8003a00 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2c77fe700 1 -- 172.21.15.5:0/1859241068 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00ed30 con 0x7fa2b8003a00 2024-05-08T23:24:33.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2cf212700 1 -- 172.21.15.5:0/1859241068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8003a00 msgr2=0x7fa2b8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2cf212700 1 --2- 172.21.15.5:0/1859241068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8003a00 0x7fa2b8003e20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009d30 tx=0x7fa2b8005880 comp rx=0 tx=0).stop 2024-05-08T23:24:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2cd7af700 1 -- 172.21.15.5:0/1859241068 reap_dead start 2024-05-08T23:24:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2cf212700 1 -- 172.21.15.5:0/1859241068 shutdown_connections 2024-05-08T23:24:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2cf212700 1 -- 172.21.15.5:0/1859241068 >> 172.21.15.5:0/1859241068 conn(0x7fa2c808afe0 msgr2=0x7fa2c808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2cf212700 1 -- 172.21.15.5:0/1859241068 shutdown_connections 2024-05-08T23:24:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.914+0000 7fa2cf212700 1 -- 172.21.15.5:0/1859241068 wait complete. 2024-05-08T23:24:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.916+0000 7fa2cf212700 1 Processor -- start 2024-05-08T23:24:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.916+0000 7fa2cf212700 1 -- start start 2024-05-08T23:24:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2cf212700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2b8003a00 0x7fa2c80c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2cf212700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c80af470 0x7fa2c80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2cf212700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c8159570 0x7fa2c8159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2cf212700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c8159f40 con 0x7fa2c80af470 2024-05-08T23:24:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2cf212700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c80b5fb0 con 0x7fa2c8159570 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2cf212700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c80b6290 con 0x7fa2b8003a00 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c80af470 0x7fa2c80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c80af470 0x7fa2c80af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41676/0 (socket says 172.21.15.5:41676) 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 -- 172.21.15.5:0/4059383725 learned_addr learned my addr 172.21.15.5:0/4059383725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2b8003a00 0x7fa2c80c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2cd7af700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c8159570 0x7fa2c8159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 -- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2b8003a00 msgr2=0x7fa2c80c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2b8003a00 0x7fa2c80c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 -- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c8159570 msgr2=0x7fa2c8159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c8159570 0x7fa2c8159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 -- 172.21.15.5:0/4059383725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c80b6570 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2cd7af700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c8159570 0x7fa2c8159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c7fff700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c80af470 0x7fa2c80af8d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4011ca0 tx=0x7fa2b4009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.917+0000 7fa2c57fa700 1 -- 172.21.15.5:0/4059383725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400fa00 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.918+0000 7fa2c57fa700 1 -- 172.21.15.5:0/4059383725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b400fb60 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.918+0000 7fa2c57fa700 1 -- 172.21.15.5:0/4059383725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4004020 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.918+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c80b6850 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.918+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c80b5c30 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.919+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c80b6d70 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.919+0000 7fa2c57fa700 1 -- 172.21.15.5:0/4059383725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa2b4004180 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.920+0000 7fa2c57fa700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2ac06ac90 0x7fa2ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.920+0000 7fa2c57fa700 1 -- 172.21.15.5:0/4059383725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fa2b40899b0 con 0x7fa2c80af470 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.920+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2ac06ac90 0x7fa2ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.921+0000 7fa2ccfae700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2ac06ac90 0x7fa2ac06d150 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:33.925+0000 7fa2c57fa700 1 -- 172.21.15.5:0/4059383725 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa2c80b6d70 con 0x7fa2c80af470 2024-05-08T23:24:34.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.140+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa2c80b6d70 con 0x7fa2c80af470 2024-05-08T23:24:34.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.783+0000 7fa2c57fa700 1 -- 172.21.15.5:0/4059383725 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa2c80b6d70 con 0x7fa2c80af470 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2ac06ac90 msgr2=0x7fa2ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2ac06ac90 0x7fa2ac06d150 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc008030 comp rx=0 tx=0).stop 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c80af470 msgr2=0x7fa2c80af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c80af470 0x7fa2c80af8d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4011ca0 tx=0x7fa2b4009e80 comp rx=0 tx=0).stop 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 shutdown_connections 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2ac06ac90 0x7fa2ac06d150 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2c8159570 0x7fa2c8159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2b8003a00 0x7fa2c80c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 --2- 172.21.15.5:0/4059383725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2c80af470 0x7fa2c80af8d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 >> 172.21.15.5:0/4059383725 conn(0x7fa2c808afe0 msgr2=0x7fa2c80b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 shutdown_connections 2024-05-08T23:24:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.788+0000 7fa2cf212700 1 -- 172.21.15.5:0/4059383725 wait complete. 2024-05-08T23:24:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:24:34.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:24:34.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.949+0000 7fb09b8c7700 1 Processor -- start 2024-05-08T23:24:34.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.949+0000 7fb09b8c7700 1 -- start start 2024-05-08T23:24:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.950+0000 7fb09b8c7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb09414b790 0x7fb0940c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.950+0000 7fb09b8c7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0940c4920 0x7fb0940ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.950+0000 7fb09b8c7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0940c4da0 con 0x7fb0940ae800 2024-05-08T23:24:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.950+0000 7fb09b8c7700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0940b00b0 con 0x7fb09414b790 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.950+0000 7fb09b8c7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940af010 con 0x7fb0940c4920 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.950+0000 7fb099663700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb09414b790 0x7fb0940c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.950+0000 7fb099e64700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb0940ae800 0x7fb094165930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34938/0 (socket says 172.21.15.5:34938) 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.950+0000 7fb099e64700 1 -- 172.21.15.5:0/3168048137 learned_addr learned my addr 172.21.15.5:0/3168048137 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb098e62700 1 --2- 172.21.15.5:0/3168048137 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0940c4920 0x7fb0940ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb099663700 1 -- 172.21.15.5:0/3168048137 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0940c4920 msgr2=0x7fb0940ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb099663700 1 --2- 172.21.15.5:0/3168048137 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0940c4920 0x7fb0940ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb099663700 1 -- 172.21.15.5:0/3168048137 >> v1:172.21.15.5:6789/0 conn(0x7fb0940ae800 legacy=0x7fb094165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb087fff700 1 -- 172.21.15.5:0/3168048137 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592668069 0 0) 0x7fb08c001490 con 0x7fb0940ae800 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb099663700 1 -- 172.21.15.5:0/3168048137 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0940af2f0 con 0x7fb09414b790 2024-05-08T23:24:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb099663700 1 --2- 172.21.15.5:0/3168048137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb09414b790 0x7fb0940c43e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb090009840 tx=0x7fb090009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0eafd1f637812dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb087fff700 1 -- 172.21.15.5:0/3168048137 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090011830 con 0x7fb09414b790 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb087fff700 1 -- 172.21.15.5:0/3168048137 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090011990 con 0x7fb09414b790 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.951+0000 7fb087fff700 1 -- 172.21.15.5:0/3168048137 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090011d20 con 0x7fb09414b790 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.952+0000 7fb09b8c7700 1 -- 172.21.15.5:0/3168048137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb09414b790 msgr2=0x7fb0940c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.952+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/3168048137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb09414b790 0x7fb0940c43e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb090009840 tx=0x7fb090009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.952+0000 7fb09b8c7700 1 -- 172.21.15.5:0/3168048137 shutdown_connections 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.952+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/3168048137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb09414b790 0x7fb0940c43e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.952+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/3168048137 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0940c4920 0x7fb0940ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.952+0000 7fb09b8c7700 1 -- 172.21.15.5:0/3168048137 >> 172.21.15.5:0/3168048137 conn(0x7fb09408afe0 msgr2=0x7fb09408b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.952+0000 7fb09b8c7700 1 -- 172.21.15.5:0/3168048137 shutdown_connections 2024-05-08T23:24:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.952+0000 7fb09b8c7700 1 -- 172.21.15.5:0/3168048137 wait complete. 2024-05-08T23:24:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.953+0000 7fb09b8c7700 1 Processor -- start 2024-05-08T23:24:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.953+0000 7fb09b8c7700 1 -- start start 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb09b8c7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0940ae800 0x7fb094160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb09b8c7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940c4920 0x7fb0940c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb09b8c7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb09414b790 0x7fb0940c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb09b8c7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940c6cf0 con 0x7fb0940ae800 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb09b8c7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094161580 con 0x7fb0940c4920 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb09b8c7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb094161060 con 0x7fb09414b790 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099663700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0940ae800 0x7fb094160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099663700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0940ae800 0x7fb094160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41694/0 (socket says 172.21.15.5:41694) 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099663700 1 -- 172.21.15.5:0/2596068804 learned_addr learned my addr 172.21.15.5:0/2596068804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099e64700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb09414b790 0x7fb0940c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb098e62700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940c4920 0x7fb0940c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099663700 1 -- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb09414b790 msgr2=0x7fb0940c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099663700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb09414b790 0x7fb0940c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099663700 1 -- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940c4920 msgr2=0x7fb0940c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099663700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940c4920 0x7fb0940c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb099663700 1 -- 172.21.15.5:0/2596068804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0900094a0 con 0x7fb0940ae800 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.954+0000 7fb098e62700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940c4920 0x7fb0940c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.955+0000 7fb099663700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0940ae800 0x7fb094160b20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb0900090b0 tx=0x7fb090004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.955+0000 7fb085ffb700 1 -- 172.21.15.5:0/2596068804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900038a0 con 0x7fb0940ae800 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.955+0000 7fb085ffb700 1 -- 172.21.15.5:0/2596068804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09001f070 con 0x7fb0940ae800 2024-05-08T23:24:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.955+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0940c6070 con 0x7fb0940ae800 2024-05-08T23:24:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.955+0000 7fb085ffb700 1 -- 172.21.15.5:0/2596068804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090003c00 con 0x7fb0940ae800 2024-05-08T23:24:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.955+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0940c6630 con 0x7fb0940ae800 2024-05-08T23:24:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.956+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb094074bc0 con 0x7fb0940ae800 2024-05-08T23:24:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.957+0000 7fb085ffb700 1 -- 172.21.15.5:0/2596068804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb09001a430 con 0x7fb0940ae800 2024-05-08T23:24:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.957+0000 7fb085ffb700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb07806ac90 0x7fb07806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.957+0000 7fb085ffb700 1 -- 172.21.15.5:0/2596068804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5587+0+0 (secure 0 0 0) 0x7fb090021070 con 0x7fb0940ae800 2024-05-08T23:24:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.958+0000 7fb098e62700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb07806ac90 0x7fb07806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.958+0000 7fb098e62700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb07806ac90 0x7fb07806d150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb0940ceb00 tx=0x7fb088008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:34.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:34.962+0000 7fb085ffb700 1 -- 172.21.15.5:0/2596068804 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb090055ff0 con 0x7fb0940ae800 2024-05-08T23:24:35.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.180+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0940bb9b0 con 0x7fb0940ae800 2024-05-08T23:24:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.796+0000 7fb085ffb700 1 -- 172.21.15.5:0/2596068804 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb090059610 con 0x7fb0940ae800 2024-05-08T23:24:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.801+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb07806ac90 msgr2=0x7fb07806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.801+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb07806ac90 0x7fb07806d150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb0940ceb00 tx=0x7fb088008040 comp rx=0 tx=0).stop 2024-05-08T23:24:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.801+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0940ae800 msgr2=0x7fb094160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.801+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0940ae800 0x7fb094160b20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb0900090b0 tx=0x7fb090004750 comp rx=0 tx=0).stop 2024-05-08T23:24:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.802+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 shutdown_connections 2024-05-08T23:24:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.802+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb07806ac90 0x7fb07806d150 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.802+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb09414b790 0x7fb0940c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.802+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0940ae800 0x7fb094160b20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.802+0000 7fb09b8c7700 1 --2- 172.21.15.5:0/2596068804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940c4920 0x7fb0940c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.802+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 >> 172.21.15.5:0/2596068804 conn(0x7fb09408afe0 msgr2=0x7fb09408b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:35.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.802+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 shutdown_connections 2024-05-08T23:24:35.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.802+0000 7fb09b8c7700 1 -- 172.21.15.5:0/2596068804 wait complete. 2024-05-08T23:24:35.803 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-08T23:24:35.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T23:24:35.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.960+0000 7f0f10f27700 1 Processor -- start 2024-05-08T23:24:35.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.960+0000 7f0f10f27700 1 -- start start 2024-05-08T23:24:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.961+0000 7f0f10f27700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0f0c0cc590 0x7f0f0c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.961+0000 7f0f10f27700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f0c06f7c0 0x7f0f0c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.961+0000 7f0f10f27700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c0cb160 con 0x7f0f0c15db00 2024-05-08T23:24:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.961+0000 7f0f10f27700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c06fc50 con 0x7f0f0c06f7c0 2024-05-08T23:24:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.961+0000 7f0f10f27700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c15dee0 con 0x7f0f0c0cc590 2024-05-08T23:24:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0ad9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f0c06f7c0 0x7f0f0c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0ad9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f0c06f7c0 0x7f0f0c06e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41710/0 (socket says 172.21.15.5:41710) 2024-05-08T23:24:35.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0ad9d700 1 -- 172.21.15.5:0/694951463 learned_addr learned my addr 172.21.15.5:0/694951463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f09d9b700 1 --2- 172.21.15.5:0/694951463 >> v2:172.21.15.49:3300/0 conn(0x7f0f0c0cc590 0x7f0f0c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0ad9d700 1 -- 172.21.15.5:0/694951463 >> v2:172.21.15.49:3300/0 conn(0x7f0f0c0cc590 msgr2=0x7f0f0c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0ad9d700 1 --2- 172.21.15.5:0/694951463 >> v2:172.21.15.49:3300/0 conn(0x7f0f0c0cc590 0x7f0f0c06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0ad9d700 1 -- 172.21.15.5:0/694951463 >> v1:172.21.15.49:6789/0 conn(0x7f0f0c15db00 legacy=0x7f0f0c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0ad9d700 1 -- 172.21.15.5:0/694951463 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c06ad20 con 0x7f0f0c06f7c0 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0ad9d700 1 --2- 172.21.15.5:0/694951463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c06f7c0 0x7f0f0c06e320 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0cdef0 tx=0x7f0f0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a1f9246b5166a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0959a700 1 -- 172.21.15.5:0/694951463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400e070 con 0x7f0f0c06f7c0 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0959a700 1 -- 172.21.15.5:0/694951463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f04003a20 con 0x7f0f0c06f7c0 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.962+0000 7f0f0959a700 1 -- 172.21.15.5:0/694951463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04003dd0 con 0x7f0f0c06f7c0 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.963+0000 7f0f10f27700 1 -- 172.21.15.5:0/694951463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c06f7c0 msgr2=0x7f0f0c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.963+0000 7f0f10f27700 1 --2- 172.21.15.5:0/694951463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c06f7c0 0x7f0f0c06e320 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0cdef0 tx=0x7f0f0400b330 comp rx=0 tx=0).stop 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.963+0000 7f0f10f27700 1 -- 172.21.15.5:0/694951463 shutdown_connections 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.963+0000 7f0f10f27700 1 --2- 172.21.15.5:0/694951463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c06f7c0 0x7f0f0c06e320 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.963+0000 7f0f10f27700 1 --2- 172.21.15.5:0/694951463 >> v2:172.21.15.49:3300/0 conn(0x7f0f0c0cc590 0x7f0f0c06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.963+0000 7f0f10f27700 1 -- 172.21.15.5:0/694951463 >> 172.21.15.5:0/694951463 conn(0x7f0f0c099a50 msgr2=0x7f0f0c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.963+0000 7f0f10f27700 1 -- 172.21.15.5:0/694951463 shutdown_connections 2024-05-08T23:24:35.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.963+0000 7f0f10f27700 1 -- 172.21.15.5:0/694951463 wait complete. 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.965+0000 7f0f10f27700 1 Processor -- start 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.965+0000 7f0f10f27700 1 -- start start 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.965+0000 7f0f10f27700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f0c06f7c0 0x7f0f0c0c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.965+0000 7f0f10f27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c0cc590 0x7f0f0c1f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.965+0000 7f0f10f27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c15db00 0x7f0f0c1f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.965+0000 7f0f10f27700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0ca680 con 0x7f0f0c0cc590 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.965+0000 7f0f10f27700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c7bd0 con 0x7f0f0c15db00 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.965+0000 7f0f10f27700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c9ce0 con 0x7f0f0c06f7c0 2024-05-08T23:24:35.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c0cc590 0x7f0f0c1f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c0cc590 0x7f0f0c1f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41720/0 (socket says 172.21.15.5:41720) 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 -- 172.21.15.5:0/2908593494 learned_addr learned my addr 172.21.15.5:0/2908593494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f0ad9d700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c15db00 0x7f0f0c1f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f0a59c700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f0c06f7c0 0x7f0f0c0c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 -- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f0c06f7c0 msgr2=0x7f0f0c0c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f0c06f7c0 0x7f0f0c0c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 -- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c15db00 msgr2=0x7f0f0c1f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c15db00 0x7f0f0c1f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 -- 172.21.15.5:0/2908593494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f04009b30 con 0x7f0f0c0cc590 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f09d9b700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c0cc590 0x7f0f0c1f6150 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c7f0 tx=0x7f0f0000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0efaffd700 1 -- 172.21.15.5:0/2908593494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00004220 con 0x7f0f0c0cc590 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0efaffd700 1 -- 172.21.15.5:0/2908593494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00004380 con 0x7f0f0c0cc590 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c0c9fc0 con 0x7f0f0c0cc590 2024-05-08T23:24:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0efaffd700 1 -- 172.21.15.5:0/2908593494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f000036a0 con 0x7f0f0c0cc590 2024-05-08T23:24:35.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.966+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c14e430 con 0x7f0f0c0cc590 2024-05-08T23:24:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.967+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c0ca7c0 con 0x7f0f0c0cc590 2024-05-08T23:24:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.968+0000 7f0efaffd700 1 -- 172.21.15.5:0/2908593494 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0f00003800 con 0x7f0f0c0cc590 2024-05-08T23:24:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.968+0000 7f0efaffd700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0eec06ac90 0x7f0eec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.968+0000 7f0efaffd700 1 -- 172.21.15.5:0/2908593494 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f0f00088b70 con 0x7f0f0c0cc590 2024-05-08T23:24:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.969+0000 7f0f0a59c700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0eec06ac90 0x7f0eec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:35.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.969+0000 7f0f0a59c700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0eec06ac90 0x7f0eec06d150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009960 tx=0x7f0efc0090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:35.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:35.973+0000 7f0efaffd700 1 -- 172.21.15.5:0/2908593494 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0f0c0ca7c0 con 0x7f0f0c0cc590 2024-05-08T23:24:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.204+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0f0c0ca7c0 con 0x7f0f0c0cc590 2024-05-08T23:24:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.208+0000 7f0efaffd700 1 -- 172.21.15.5:0/2908593494 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0f0c0ca7c0 con 0x7f0f0c0cc590 2024-05-08T23:24:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0eec06ac90 msgr2=0x7f0eec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0eec06ac90 0x7f0eec06d150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009960 tx=0x7f0efc0090e0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c0cc590 msgr2=0x7f0f0c1f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c0cc590 0x7f0f0c1f6150 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c7f0 tx=0x7f0f0000cb00 comp rx=0 tx=0).stop 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 shutdown_connections 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0eec06ac90 0x7f0eec06d150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c15db00 0x7f0f0c1f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f0c06f7c0 0x7f0f0c0c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 --2- 172.21.15.5:0/2908593494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f0c0cc590 0x7f0f0c1f6150 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 >> 172.21.15.5:0/2908593494 conn(0x7f0f0c099a50 msgr2=0x7f0f0c15c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 shutdown_connections 2024-05-08T23:24:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.210+0000 7f0f10f27700 1 -- 172.21.15.5:0/2908593494 wait complete. 2024-05-08T23:24:36.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-05-08T23:24:36.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.361+0000 7fbcfac92700 1 Processor -- start 2024-05-08T23:24:36.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.361+0000 7fbcfac92700 1 -- start start 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.362+0000 7fbcfac92700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbcf415a630 0x7fbcf40d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcfac92700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbcf40cdef0 0x7fbcf40cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcfac92700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf40ce740 con 0x7fbcf40cc040 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcfac92700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf40cb670 con 0x7fbcf415a630 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcfac92700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf40cadc0 con 0x7fbcf40cdef0 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf8a2e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbcf415a630 0x7fbcf40d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf8a2e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbcf415a630 0x7fbcf40d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41726/0 (socket says 172.21.15.5:41726) 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf8a2e700 1 -- 172.21.15.5:0/3362531830 learned_addr learned my addr 172.21.15.5:0/3362531830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf3fff700 1 --2- 172.21.15.5:0/3362531830 >> v2:172.21.15.49:3300/0 conn(0x7fbcf40cdef0 0x7fbcf40cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf8a2e700 1 -- 172.21.15.5:0/3362531830 >> v2:172.21.15.49:3300/0 conn(0x7fbcf40cdef0 msgr2=0x7fbcf40cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf8a2e700 1 --2- 172.21.15.5:0/3362531830 >> v2:172.21.15.49:3300/0 conn(0x7fbcf40cdef0 0x7fbcf40cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf8a2e700 1 -- 172.21.15.5:0/3362531830 >> v1:172.21.15.49:6789/0 conn(0x7fbcf40cc040 legacy=0x7fbcf415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf8a2e700 1 -- 172.21.15.5:0/3362531830 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40cb060 con 0x7fbcf415a630 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf3fff700 1 --2- 172.21.15.5:0/3362531830 >> v2:172.21.15.49:3300/0 conn(0x7fbcf40cdef0 0x7fbcf40cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf8a2e700 1 --2- 172.21.15.5:0/3362531830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf40d0b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4099ef0 tx=0x7fbce4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1536bcaf248ccc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf37fe700 1 -- 172.21.15.5:0/3362531830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce400b870 con 0x7fbcf415a630 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf37fe700 1 -- 172.21.15.5:0/3362531830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce400b9d0 con 0x7fbcf415a630 2024-05-08T23:24:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.363+0000 7fbcf37fe700 1 -- 172.21.15.5:0/3362531830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce400bd60 con 0x7fbcf415a630 2024-05-08T23:24:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.364+0000 7fbcfac92700 1 -- 172.21.15.5:0/3362531830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 msgr2=0x7fbcf40d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.364+0000 7fbcfac92700 1 --2- 172.21.15.5:0/3362531830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf40d0b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4099ef0 tx=0x7fbce4009bd0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.364+0000 7fbcfac92700 1 -- 172.21.15.5:0/3362531830 shutdown_connections 2024-05-08T23:24:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.364+0000 7fbcfac92700 1 --2- 172.21.15.5:0/3362531830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf40d0b50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.364+0000 7fbcfac92700 1 --2- 172.21.15.5:0/3362531830 >> v2:172.21.15.49:3300/0 conn(0x7fbcf40cdef0 0x7fbcf40cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.364+0000 7fbcfac92700 1 -- 172.21.15.5:0/3362531830 >> 172.21.15.5:0/3362531830 conn(0x7fbcf4099a50 msgr2=0x7fbcf40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.364+0000 7fbcfac92700 1 -- 172.21.15.5:0/3362531830 shutdown_connections 2024-05-08T23:24:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.364+0000 7fbcfac92700 1 -- 172.21.15.5:0/3362531830 wait complete. 2024-05-08T23:24:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.365+0000 7fbcfac92700 1 Processor -- start 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.365+0000 7fbcfac92700 1 -- start start 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.366+0000 7fbcfac92700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbcf40cc040 0x7fbcf414db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.366+0000 7fbcfac92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcf40cdef0 0x7fbcf415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.366+0000 7fbcfac92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.366+0000 7fbcfac92700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf414e060 con 0x7fbcf415a630 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.366+0000 7fbcfac92700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf415d8a0 con 0x7fbcf40cdef0 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.366+0000 7fbcfac92700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40b4f50 con 0x7fbcf40cc040 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf415d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41734/0 (socket says 172.21.15.5:41734) 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 -- 172.21.15.5:0/1044849915 learned_addr learned my addr 172.21.15.5:0/1044849915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf3fff700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcf40cdef0 0x7fbcf415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf8a2e700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbcf40cc040 0x7fbcf414db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 -- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbcf40cc040 msgr2=0x7fbcf414db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbcf40cc040 0x7fbcf414db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 -- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcf40cdef0 msgr2=0x7fbcf415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcf40cdef0 0x7fbcf415ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 -- 172.21.15.5:0/1044849915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce4009550 con 0x7fbcf415a630 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf3fff700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcf40cdef0 0x7fbcf415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf922f700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf415d360 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00cbc0 tx=0x7fbcec00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf17fa700 1 -- 172.21.15.5:0/1044849915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec003c00 con 0x7fbcf415a630 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf17fa700 1 -- 172.21.15.5:0/1044849915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec003d60 con 0x7fbcf415a630 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf406f130 con 0x7fbcf415a630 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcf17fa700 1 -- 172.21.15.5:0/1044849915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00ab90 con 0x7fbcf415a630 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.367+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf406ee50 con 0x7fbcf415a630 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.368+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf4068000 con 0x7fbcf415a630 2024-05-08T23:24:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.369+0000 7fbcf17fa700 1 -- 172.21.15.5:0/1044849915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbcec00acf0 con 0x7fbcf415a630 2024-05-08T23:24:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.369+0000 7fbcf17fa700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbce006ac90 0x7fbce006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.370+0000 7fbcf17fa700 1 -- 172.21.15.5:0/1044849915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbcec088f30 con 0x7fbcf415a630 2024-05-08T23:24:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.370+0000 7fbcf8a2e700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbce006ac90 0x7fbce006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.370+0000 7fbcf8a2e700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbce006ac90 0x7fbce006d150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbce4000c00 tx=0x7fbce4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.374+0000 7fbcf17fa700 1 -- 172.21.15.5:0/1044849915 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbcec054360 con 0x7fbcf415a630 2024-05-08T23:24:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.683+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) v1 -- 0x7fbcf40ce740 con 0x7fbcf415a630 2024-05-08T23:24:36.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.794+0000 7fbcf17fa700 1 -- 172.21.15.5:0/1044849915 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v194) v1 ==== 143+0+0 (secure 0 0 0) 0x7fbcf40ce740 con 0x7fbcf415a630 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbce006ac90 msgr2=0x7fbce006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbce006ac90 0x7fbce006d150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbce4000c00 tx=0x7fbce4008030 comp rx=0 tx=0).stop 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 msgr2=0x7fbcf415d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf415d360 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00cbc0 tx=0x7fbcec00cf80 comp rx=0 tx=0).stop 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 shutdown_connections 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbce006ac90 0x7fbce006d150 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf415a630 0x7fbcf415d360 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbcf40cc040 0x7fbcf414db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 --2- 172.21.15.5:0/1044849915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcf40cdef0 0x7fbcf415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 >> 172.21.15.5:0/1044849915 conn(0x7fbcf4099a50 msgr2=0x7fbcf40af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 shutdown_connections 2024-05-08T23:24:36.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.796+0000 7fbcfac92700 1 -- 172.21.15.5:0/1044849915 wait complete. 2024-05-08T23:24:36.811 INFO:tasks.cephfs_test_runner:test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-05-08T23:24:36.811 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-05-08T23:24:36.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:36.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.950+0000 7f321ec86700 1 Processor -- start 2024-05-08T23:24:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.950+0000 7f321ec86700 1 -- start start 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321ec86700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f321815a380 0x7f32180cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321ec86700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32180cf3c0 0x7f321806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321ec86700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f321806f4a0 0x7f321815e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321ec86700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32180ceba0 con 0x7f321806f4a0 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321ec86700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321806f950 con 0x7f321815a380 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321ec86700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32180b2b10 con 0x7f32180cf3c0 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321d223700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f321806f4a0 0x7f321815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321d223700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f321806f4a0 0x7f321815e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41738/0 (socket says 172.21.15.5:41738) 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.951+0000 7f321d223700 1 -- 172.21.15.5:0/986893414 learned_addr learned my addr 172.21.15.5:0/986893414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f321ca22700 1 --2- 172.21.15.5:0/986893414 >> v2:172.21.15.49:3300/0 conn(0x7f321815a380 0x7f32180cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f3217fff700 1 --2- 172.21.15.5:0/986893414 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32180cf3c0 0x7f321806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f321d223700 1 -- 172.21.15.5:0/986893414 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32180cf3c0 msgr2=0x7f321806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f321d223700 1 --2- 172.21.15.5:0/986893414 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32180cf3c0 0x7f321806ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f321d223700 1 -- 172.21.15.5:0/986893414 >> v2:172.21.15.49:3300/0 conn(0x7f321815a380 msgr2=0x7f32180cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f321d223700 1 --2- 172.21.15.5:0/986893414 >> v2:172.21.15.49:3300/0 conn(0x7f321815a380 0x7f32180cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f321d223700 1 -- 172.21.15.5:0/986893414 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32180ce340 con 0x7f321806f4a0 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f321d223700 1 --2- 172.21.15.5:0/986893414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321806f4a0 0x7f321815e090 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3218072710 tx=0x7f321000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1991bb6cf7095063 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f32177fe700 1 -- 172.21.15.5:0/986893414 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000e070 con 0x7f321806f4a0 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f32177fe700 1 -- 172.21.15.5:0/986893414 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321000b9c0 con 0x7f321806f4a0 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.952+0000 7f32177fe700 1 -- 172.21.15.5:0/986893414 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000bd70 con 0x7f321806f4a0 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 -- 172.21.15.5:0/986893414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321806f4a0 msgr2=0x7f321815e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 --2- 172.21.15.5:0/986893414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321806f4a0 0x7f321815e090 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3218072710 tx=0x7f321000b330 comp rx=0 tx=0).stop 2024-05-08T23:24:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 -- 172.21.15.5:0/986893414 shutdown_connections 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 --2- 172.21.15.5:0/986893414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321806f4a0 0x7f321815e090 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 --2- 172.21.15.5:0/986893414 >> v2:172.21.15.49:3300/0 conn(0x7f321815a380 0x7f32180cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 --2- 172.21.15.5:0/986893414 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f32180cf3c0 0x7f321806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 -- 172.21.15.5:0/986893414 >> 172.21.15.5:0/986893414 conn(0x7f32180cfce0 msgr2=0x7f32180d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 -- 172.21.15.5:0/986893414 shutdown_connections 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.953+0000 7f321ec86700 1 -- 172.21.15.5:0/986893414 wait complete. 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.954+0000 7f321ec86700 1 Processor -- start 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.954+0000 7f321ec86700 1 -- start start 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.955+0000 7f321ec86700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f321806f4a0 0x7f321806e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.955+0000 7f321ec86700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180cf3c0 0x7f321814d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.955+0000 7f321ec86700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321815a380 0x7f321814d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.955+0000 7f321ec86700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321806ec40 con 0x7f321815a380 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.955+0000 7f321ec86700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32181519b0 con 0x7f32180cf3c0 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.955+0000 7f321ec86700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3218151c90 con 0x7f321806f4a0 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.955+0000 7f321d223700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321815a380 0x7f321814d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321d223700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321815a380 0x7f321814d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41742/0 (socket says 172.21.15.5:41742) 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321d223700 1 -- 172.21.15.5:0/1650102955 learned_addr learned my addr 172.21.15.5:0/1650102955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f3217fff700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180cf3c0 0x7f321814d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321ca22700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f321806f4a0 0x7f321806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321d223700 1 -- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f321806f4a0 msgr2=0x7f321806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321d223700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f321806f4a0 0x7f321806e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321d223700 1 -- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180cf3c0 msgr2=0x7f321814d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321d223700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180cf3c0 0x7f321814d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321d223700 1 -- 172.21.15.5:0/1650102955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210009b30 con 0x7f321815a380 2024-05-08T23:24:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321ca22700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f321806f4a0 0x7f321806e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321d223700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321815a380 0x7f321814d540 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f32180667b0 tx=0x7f3210004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f32157fa700 1 -- 172.21.15.5:0/1650102955 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32100092e0 con 0x7f321815a380 2024-05-08T23:24:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f32157fa700 1 -- 172.21.15.5:0/1650102955 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3210004250 con 0x7f321815a380 2024-05-08T23:24:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3218151f70 con 0x7f321815a380 2024-05-08T23:24:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f32157fa700 1 -- 172.21.15.5:0/1650102955 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210004600 con 0x7f321815a380 2024-05-08T23:24:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.956+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3218152530 con 0x7f321815a380 2024-05-08T23:24:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.957+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3218068000 con 0x7f321815a380 2024-05-08T23:24:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.958+0000 7f32157fa700 1 -- 172.21.15.5:0/1650102955 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3210004760 con 0x7f321815a380 2024-05-08T23:24:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.958+0000 7f32157fa700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320406ac90 0x7f320406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.959+0000 7f32157fa700 1 -- 172.21.15.5:0/1650102955 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f321008a190 con 0x7f321815a380 2024-05-08T23:24:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.959+0000 7f321ca22700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320406ac90 0x7f320406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.959+0000 7f321ca22700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320406ac90 0x7f320406d150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f32180cbe80 tx=0x7f3208008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:36.963+0000 7f32157fa700 1 -- 172.21.15.5:0/1650102955 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3210055650 con 0x7f321815a380 2024-05-08T23:24:37.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.180+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f321814db70 con 0x7f321815a380 2024-05-08T23:24:37.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.181+0000 7f32157fa700 1 -- 172.21.15.5:0/1650102955 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+12676 (secure 0 0 0) 0x7f3210058c70 con 0x7f321815a380 2024-05-08T23:24:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:37.181 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:35.786421+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6833/3359342067":"2024-05-09T23:24:32.694410+0000","172.21.15.49:6834/962044603":"2024-05-09T23:24:32.694405+0000","172.21.15.49:6837/350829837":"2024-05-09T23:24:32.694391+0000","172.21.15.5:6835/3359342067":"2024-05-09T23:24:32.694410+0000","172.21.15.49:6835/962044603":"2024-05-09T23:24:32.694405+0000","172.21.15.49:6836/350829837":"2024-05-09T23:24:32.694391+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320406ac90 msgr2=0x7f320406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320406ac90 0x7f320406d150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f32180cbe80 tx=0x7f3208008040 comp rx=0 tx=0).stop 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321815a380 msgr2=0x7f321814d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321815a380 0x7f321814d540 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f32180667b0 tx=0x7f3210004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 shutdown_connections 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320406ac90 0x7f320406d150 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f321815a380 0x7f321814d540 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f321806f4a0 0x7f321806e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.183+0000 7f321ec86700 1 --2- 172.21.15.5:0/1650102955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32180cf3c0 0x7f321814d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.184+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 >> 172.21.15.5:0/1650102955 conn(0x7f32180cfce0 msgr2=0x7f32180d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.184+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 shutdown_connections 2024-05-08T23:24:37.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.184+0000 7f321ec86700 1 -- 172.21.15.5:0/1650102955 wait complete. 2024-05-08T23:24:37.198 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:24:37.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:24:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.338+0000 7f5f79db0700 1 Processor -- start 2024-05-08T23:24:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.338+0000 7f5f79db0700 1 -- start start 2024-05-08T23:24:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.339+0000 7f5f79db0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5f74099a50 0x7f5f740d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.339+0000 7f5f79db0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 0x7f5f7409a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.339+0000 7f5f79db0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7406f930 con 0x7f5f740cba90 2024-05-08T23:24:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.339+0000 7f5f79db0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f740cfdf0 con 0x7f5f74099a50 2024-05-08T23:24:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.339+0000 7f5f79db0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740d00d0 con 0x7f5f74099e30 2024-05-08T23:24:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.339+0000 7f5f737fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5f74099a50 0x7f5f740d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.339+0000 7f5f73fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5f740cba90 0x7f5f7406fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34940/0 (socket says 172.21.15.5:34940) 2024-05-08T23:24:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.339+0000 7f5f73fff700 1 -- 172.21.15.5:0/561865814 learned_addr learned my addr 172.21.15.5:0/561865814 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f72ffd700 1 --2- 172.21.15.5:0/561865814 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 0x7f5f7409a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f737fe700 1 -- 172.21.15.5:0/561865814 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 msgr2=0x7f5f7409a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f737fe700 1 --2- 172.21.15.5:0/561865814 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 0x7f5f7409a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f737fe700 1 -- 172.21.15.5:0/561865814 >> v1:172.21.15.5:6789/0 conn(0x7f5f740cba90 legacy=0x7f5f7406fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f737fe700 1 -- 172.21.15.5:0/561865814 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740d03b0 con 0x7f5f74099a50 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f727fc700 1 -- 172.21.15.5:0/561865814 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769107226 0 0) 0x7f5f7406f930 con 0x7f5f740cba90 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f737fe700 1 --2- 172.21.15.5:0/561865814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740d0690 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009840 tx=0x7f5f64009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbacff43c6baf47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f727fc700 1 -- 172.21.15.5:0/561865814 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64011830 con 0x7f5f74099a50 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f727fc700 1 -- 172.21.15.5:0/561865814 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f64011990 con 0x7f5f74099a50 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.340+0000 7f5f727fc700 1 -- 172.21.15.5:0/561865814 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64011d20 con 0x7f5f74099a50 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.341+0000 7f5f79db0700 1 -- 172.21.15.5:0/561865814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 msgr2=0x7f5f740d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.341+0000 7f5f79db0700 1 --2- 172.21.15.5:0/561865814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740d0690 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009840 tx=0x7f5f64009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.341+0000 7f5f79db0700 1 -- 172.21.15.5:0/561865814 shutdown_connections 2024-05-08T23:24:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.341+0000 7f5f79db0700 1 --2- 172.21.15.5:0/561865814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740d0690 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.341+0000 7f5f79db0700 1 --2- 172.21.15.5:0/561865814 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 0x7f5f7409a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.341+0000 7f5f79db0700 1 -- 172.21.15.5:0/561865814 >> 172.21.15.5:0/561865814 conn(0x7f5f740b2f30 msgr2=0x7f5f740b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.341+0000 7f5f79db0700 1 -- 172.21.15.5:0/561865814 shutdown_connections 2024-05-08T23:24:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.341+0000 7f5f79db0700 1 -- 172.21.15.5:0/561865814 wait complete. 2024-05-08T23:24:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.342+0000 7f5f79db0700 1 Processor -- start 2024-05-08T23:24:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.342+0000 7f5f79db0700 1 -- start start 2024-05-08T23:24:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f79db0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f79db0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 0x7f5f7415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f79db0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f740cba90 0x7f5f7415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f79db0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740b5650 con 0x7f5f74099a50 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f79db0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740b4a40 con 0x7f5f740cba90 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f79db0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740b4d20 con 0x7f5f74099e30 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41764/0 (socket says 172.21.15.5:41764) 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f737fe700 1 -- 172.21.15.5:0/2895767384 learned_addr learned my addr 172.21.15.5:0/2895767384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f72ffd700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 0x7f5f7415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.343+0000 7f5f73fff700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f740cba90 0x7f5f7415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f737fe700 1 -- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 msgr2=0x7f5f7415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f737fe700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 0x7f5f7415cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f737fe700 1 -- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f740cba90 msgr2=0x7f5f7415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f737fe700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f740cba90 0x7f5f7415d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f737fe700 1 -- 172.21.15.5:0/2895767384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f640094a0 con 0x7f5f74099a50 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f737fe700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740ca9e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009b90 tx=0x7f5f64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f5bfff700 1 -- 172.21.15.5:0/2895767384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f640036a0 con 0x7f5f74099a50 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f5bfff700 1 -- 172.21.15.5:0/2895767384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6401f070 con 0x7f5f74099a50 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f740b5000 con 0x7f5f74099a50 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f5bfff700 1 -- 172.21.15.5:0/2895767384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64003a40 con 0x7f5f74099a50 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.344+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f740b6bf0 con 0x7f5f74099a50 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.345+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f740d1240 con 0x7f5f74099a50 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.346+0000 7f5f5bfff700 1 -- 172.21.15.5:0/2895767384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f64003ba0 con 0x7f5f74099a50 2024-05-08T23:24:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.346+0000 7f5f5bfff700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f6006ac90 0x7f5f6006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.347+0000 7f5f5bfff700 1 -- 172.21.15.5:0/2895767384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f5f64021070 con 0x7f5f74099a50 2024-05-08T23:24:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.347+0000 7f5f72ffd700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f6006ac90 0x7f5f6006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.347+0000 7f5f72ffd700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f6006ac90 0x7f5f6006d150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f740b0900 tx=0x7f5f68009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.351+0000 7f5f5bfff700 1 -- 172.21.15.5:0/2895767384 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f64055d40 con 0x7f5f74099a50 2024-05-08T23:24:37.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.571+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f740b6d80 con 0x7f5f74099a50 2024-05-08T23:24:37.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.572+0000 7f5f5bfff700 1 -- 172.21.15.5:0/2895767384 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f64059360 con 0x7f5f74099a50 2024-05-08T23:24:37.572 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:24:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.574+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f6006ac90 msgr2=0x7f5f6006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.574+0000 7f5f79db0700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f6006ac90 0x7f5f6006d150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f740b0900 tx=0x7f5f68009210 comp rx=0 tx=0).stop 2024-05-08T23:24:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.574+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 msgr2=0x7f5f740ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.574+0000 7f5f79db0700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740ca9e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009b90 tx=0x7f5f64004750 comp rx=0 tx=0).stop 2024-05-08T23:24:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.575+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 shutdown_connections 2024-05-08T23:24:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.575+0000 7f5f79db0700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f6006ac90 0x7f5f6006d150 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.575+0000 7f5f79db0700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f740cba90 0x7f5f7415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.575+0000 7f5f79db0700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f74099a50 0x7f5f740ca9e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.575+0000 7f5f79db0700 1 --2- 172.21.15.5:0/2895767384 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f74099e30 0x7f5f7415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.575+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 >> 172.21.15.5:0/2895767384 conn(0x7f5f740b2f30 msgr2=0x7f5f740cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.575+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 shutdown_connections 2024-05-08T23:24:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.575+0000 7f5f79db0700 1 -- 172.21.15.5:0/2895767384 wait complete. 2024-05-08T23:24:37.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-05-08T23:24:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.725+0000 7fe3773d6700 1 Processor -- start 2024-05-08T23:24:37.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.725+0000 7fe3773d6700 1 -- start start 2024-05-08T23:24:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe3773d6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe3700cf840 0x7fe3700af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe3773d6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 0x7fe37015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe3773d6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3700d0eb0 0x7fe37015da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe3773d6700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3700cd980 con 0x7fe3700d0eb0 2024-05-08T23:24:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe3773d6700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3700ccdb0 con 0x7fe3700cf840 2024-05-08T23:24:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe3773d6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700cd050 con 0x7fe37006a960 2024-05-08T23:24:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe375973700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3700d0eb0 0x7fe37015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe375973700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3700d0eb0 0x7fe37015da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49522/0 (socket says 172.21.15.5:49522) 2024-05-08T23:24:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe375973700 1 -- 172.21.15.5:0/3021295708 learned_addr learned my addr 172.21.15.5:0/3021295708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe375172700 1 --2- 172.21.15.5:0/3021295708 >> v2:172.21.15.49:3300/0 conn(0x7fe3700cf840 0x7fe3700af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.726+0000 7fe374971700 1 --2- 172.21.15.5:0/3021295708 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 0x7fe37015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe375973700 1 -- 172.21.15.5:0/3021295708 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 msgr2=0x7fe37015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe375973700 1 --2- 172.21.15.5:0/3021295708 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 0x7fe37015d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe375973700 1 -- 172.21.15.5:0/3021295708 >> v2:172.21.15.49:3300/0 conn(0x7fe3700cf840 msgr2=0x7fe3700af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe375973700 1 --2- 172.21.15.5:0/3021295708 >> v2:172.21.15.49:3300/0 conn(0x7fe3700cf840 0x7fe3700af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe375973700 1 -- 172.21.15.5:0/3021295708 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3700cd330 con 0x7fe3700d0eb0 2024-05-08T23:24:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe375973700 1 --2- 172.21.15.5:0/3021295708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700d0eb0 0x7fe37015da30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe370071f10 tx=0x7fe36800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95516a0bd2cf302d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe363fff700 1 -- 172.21.15.5:0/3021295708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36800e070 con 0x7fe3700d0eb0 2024-05-08T23:24:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe363fff700 1 -- 172.21.15.5:0/3021295708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36800b910 con 0x7fe3700d0eb0 2024-05-08T23:24:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.727+0000 7fe363fff700 1 -- 172.21.15.5:0/3021295708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36800bcc0 con 0x7fe3700d0eb0 2024-05-08T23:24:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 -- 172.21.15.5:0/3021295708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700d0eb0 msgr2=0x7fe37015da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 --2- 172.21.15.5:0/3021295708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700d0eb0 0x7fe37015da30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe370071f10 tx=0x7fe36800b330 comp rx=0 tx=0).stop 2024-05-08T23:24:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 -- 172.21.15.5:0/3021295708 shutdown_connections 2024-05-08T23:24:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 --2- 172.21.15.5:0/3021295708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700d0eb0 0x7fe37015da30 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 --2- 172.21.15.5:0/3021295708 >> v2:172.21.15.49:3300/0 conn(0x7fe3700cf840 0x7fe3700af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 --2- 172.21.15.5:0/3021295708 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 0x7fe37015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 -- 172.21.15.5:0/3021295708 >> 172.21.15.5:0/3021295708 conn(0x7fe3700d0320 msgr2=0x7fe3700d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 -- 172.21.15.5:0/3021295708 shutdown_connections 2024-05-08T23:24:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.728+0000 7fe3773d6700 1 -- 172.21.15.5:0/3021295708 wait complete. 2024-05-08T23:24:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.729+0000 7fe3773d6700 1 Processor -- start 2024-05-08T23:24:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.729+0000 7fe3773d6700 1 -- start start 2024-05-08T23:24:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe3773d6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 0x7fe370153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe3773d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700cf840 0x7fe370160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe3773d6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3700d0eb0 0x7fe3701f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe3773d6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700b4dc0 con 0x7fe3700cf840 2024-05-08T23:24:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe3773d6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700b50a0 con 0x7fe3700d0eb0 2024-05-08T23:24:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe3773d6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700b5380 con 0x7fe37006a960 2024-05-08T23:24:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe374971700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700cf840 0x7fe370160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe374971700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700cf840 0x7fe370160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49536/0 (socket says 172.21.15.5:49536) 2024-05-08T23:24:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe374971700 1 -- 172.21.15.5:0/1718257239 learned_addr learned my addr 172.21.15.5:0/1718257239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe375973700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3700d0eb0 0x7fe3701f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe375172700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 0x7fe370153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe374971700 1 -- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 msgr2=0x7fe370153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe374971700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 0x7fe370153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe374971700 1 -- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3700d0eb0 msgr2=0x7fe3701f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe374971700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3700d0eb0 0x7fe3701f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe374971700 1 -- 172.21.15.5:0/1718257239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368009b30 con 0x7fe3700cf840 2024-05-08T23:24:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.730+0000 7fe375973700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3700d0eb0 0x7fe3701f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.731+0000 7fe374971700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700cf840 0x7fe370160a80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe36400e840 tx=0x7fe36400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.731+0000 7fe361ffb700 1 -- 172.21.15.5:0/1718257239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364013680 con 0x7fe3700cf840 2024-05-08T23:24:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.731+0000 7fe361ffb700 1 -- 172.21.15.5:0/1718257239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3640137e0 con 0x7fe3700cf840 2024-05-08T23:24:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.731+0000 7fe361ffb700 1 -- 172.21.15.5:0/1718257239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364013b70 con 0x7fe3700cf840 2024-05-08T23:24:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.731+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37015d360 con 0x7fe3700cf840 2024-05-08T23:24:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.731+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3701541f0 con 0x7fe3700cf840 2024-05-08T23:24:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.732+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37006fc40 con 0x7fe3700cf840 2024-05-08T23:24:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.733+0000 7fe361ffb700 1 -- 172.21.15.5:0/1718257239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe364013cd0 con 0x7fe3700cf840 2024-05-08T23:24:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.733+0000 7fe361ffb700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe35406ac90 0x7fe35406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.733+0000 7fe361ffb700 1 -- 172.21.15.5:0/1718257239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe364088c60 con 0x7fe3700cf840 2024-05-08T23:24:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.733+0000 7fe375172700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe35406ac90 0x7fe35406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.734+0000 7fe375172700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe35406ac90 0x7fe35406d150 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009ad0 tx=0x7fe36c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:37.738+0000 7fe361ffb700 1 -- 172.21.15.5:0/1718257239 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe3640540a0 con 0x7fe3700cf840 2024-05-08T23:24:37.788 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:24:37.791 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:24:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.049+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) v1 -- 0x7fe3700b5540 con 0x7fe3700cf840 2024-05-08T23:24:38.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.801+0000 7fe361ffb700 1 -- 172.21.15.5:0/1718257239 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v196) v1 ==== 145+0+0 (secure 0 0 0) 0x7fe3640576c0 con 0x7fe3700cf840 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe35406ac90 msgr2=0x7fe35406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe35406ac90 0x7fe35406d150 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009ad0 tx=0x7fe36c008040 comp rx=0 tx=0).stop 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700cf840 msgr2=0x7fe370160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700cf840 0x7fe370160a80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe36400e840 tx=0x7fe36400ec00 comp rx=0 tx=0).stop 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 shutdown_connections 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe35406ac90 0x7fe35406d150 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3700d0eb0 0x7fe3701f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe37006a960 0x7fe370153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 --2- 172.21.15.5:0/1718257239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700cf840 0x7fe370160a80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 >> 172.21.15.5:0/1718257239 conn(0x7fe3700d0320 msgr2=0x7fe3700ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 shutdown_connections 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.804+0000 7fe3773d6700 1 -- 172.21.15.5:0/1718257239 wait complete. 2024-05-08T23:24:38.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T23:24:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.963+0000 7fb6b355d700 1 Processor -- start 2024-05-08T23:24:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.964+0000 7fb6b355d700 1 -- start start 2024-05-08T23:24:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.964+0000 7fb6b355d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6ac15d9f0 0x7fb6ac0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.964+0000 7fb6b355d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac0b0b10 con 0x7fb6ac0af720 2024-05-08T23:24:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.964+0000 7fb6b355d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac0b29a0 con 0x7fb6ac0cd830 2024-05-08T23:24:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.964+0000 7fb6b355d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ac0b1d30 con 0x7fb6ac15d9f0 2024-05-08T23:24:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6b12f9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6ac15d9f0 0x7fb6ac0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6b0af8700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb6ac0af720 0x7fb6ac0afcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60470/0 (socket says 172.21.15.5:60470) 2024-05-08T23:24:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6b0af8700 1 -- 172.21.15.5:0/1514440341 learned_addr learned my addr 172.21.15.5:0/1514440341 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6b12f9700 1 -- 172.21.15.5:0/1514440341 >> v1:172.21.15.49:6789/0 conn(0x7fb6ac0cd830 legacy=0x7fb6ac0b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6b12f9700 1 -- 172.21.15.5:0/1514440341 >> v1:172.21.15.5:6789/0 conn(0x7fb6ac0af720 legacy=0x7fb6ac0afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6b12f9700 1 -- 172.21.15.5:0/1514440341 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac0b2c80 con 0x7fb6ac15d9f0 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6abfff700 1 -- 172.21.15.5:0/1514440341 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758743323 0 0) 0x7fb6ac0b0b10 con 0x7fb6ac0af720 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6b12f9700 1 --2- 172.21.15.5:0/1514440341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac15d9f0 0x7fb6ac0cf4a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009d90 tx=0x7fb69c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70eff7b9715dbc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6abfff700 1 -- 172.21.15.5:0/1514440341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00e070 con 0x7fb6ac15d9f0 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6abfff700 1 -- 172.21.15.5:0/1514440341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c00bbd0 con 0x7fb6ac15d9f0 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.965+0000 7fb6abfff700 1 -- 172.21.15.5:0/1514440341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c012430 con 0x7fb6ac15d9f0 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.966+0000 7fb6b355d700 1 -- 172.21.15.5:0/1514440341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac15d9f0 msgr2=0x7fb6ac0cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.966+0000 7fb6b355d700 1 --2- 172.21.15.5:0/1514440341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac15d9f0 0x7fb6ac0cf4a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009d90 tx=0x7fb69c00b330 comp rx=0 tx=0).stop 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.966+0000 7fb6b355d700 1 -- 172.21.15.5:0/1514440341 shutdown_connections 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.966+0000 7fb6b355d700 1 --2- 172.21.15.5:0/1514440341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac15d9f0 0x7fb6ac0cf4a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.966+0000 7fb6b355d700 1 -- 172.21.15.5:0/1514440341 >> 172.21.15.5:0/1514440341 conn(0x7fb6ac0b1e80 msgr2=0x7fb6ac0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.966+0000 7fb6b355d700 1 -- 172.21.15.5:0/1514440341 shutdown_connections 2024-05-08T23:24:38.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.966+0000 7fb6b355d700 1 -- 172.21.15.5:0/1514440341 wait complete. 2024-05-08T23:24:38.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.968+0000 7fb6b355d700 1 Processor -- start 2024-05-08T23:24:38.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.968+0000 7fb6b355d700 1 -- start start 2024-05-08T23:24:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b355d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6ac0af720 0x7fb6ac156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b355d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac0cd830 0x7fb6ac143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b355d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6ac15d9f0 0x7fb6ac15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b355d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac144ef0 con 0x7fb6ac0cd830 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b355d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0c90d0 con 0x7fb6ac15d9f0 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b355d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac144270 con 0x7fb6ac0af720 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b0af8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac0cd830 0x7fb6ac143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b0af8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac0cd830 0x7fb6ac143820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49560/0 (socket says 172.21.15.5:49560) 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b0af8700 1 -- 172.21.15.5:0/2781528937 learned_addr learned my addr 172.21.15.5:0/2781528937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b12f9700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6ac0af720 0x7fb6ac156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b1afa700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6ac15d9f0 0x7fb6ac15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b0af8700 1 -- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6ac0af720 msgr2=0x7fb6ac156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b0af8700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6ac0af720 0x7fb6ac156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b0af8700 1 -- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6ac15d9f0 msgr2=0x7fb6ac15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b0af8700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6ac15d9f0 0x7fb6ac15aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b0af8700 1 -- 172.21.15.5:0/2781528937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c009a70 con 0x7fb6ac0cd830 2024-05-08T23:24:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.969+0000 7fb6b12f9700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6ac0af720 0x7fb6ac156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.970+0000 7fb6b0af8700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac0cd830 0x7fb6ac143820 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c520 tx=0x7fb6a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.970+0000 7fb6a9ffb700 1 -- 172.21.15.5:0/2781528937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0004c80 con 0x7fb6ac0cd830 2024-05-08T23:24:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.970+0000 7fb6a9ffb700 1 -- 172.21.15.5:0/2781528937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0004de0 con 0x7fb6ac0cd830 2024-05-08T23:24:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.970+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac144550 con 0x7fb6ac0cd830 2024-05-08T23:24:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.970+0000 7fb6a9ffb700 1 -- 172.21.15.5:0/2781528937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0007310 con 0x7fb6ac0cd830 2024-05-08T23:24:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.970+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac144b10 con 0x7fb6ac0cd830 2024-05-08T23:24:38.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.971+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac074bc0 con 0x7fb6ac0cd830 2024-05-08T23:24:38.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.972+0000 7fb6a9ffb700 1 -- 172.21.15.5:0/2781528937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb6a0007470 con 0x7fb6ac0cd830 2024-05-08T23:24:38.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.972+0000 7fb6a9ffb700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb69806acc0 0x7fb69806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.972+0000 7fb6a9ffb700 1 -- 172.21.15.5:0/2781528937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb6a0088da0 con 0x7fb6ac0cd830 2024-05-08T23:24:38.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.973+0000 7fb6b12f9700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb69806acc0 0x7fb69806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.973+0000 7fb6b12f9700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb69806acc0 0x7fb69806d180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb69c014040 tx=0x7fb69c01d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:38.977+0000 7fb6a9ffb700 1 -- 172.21.15.5:0/2781528937 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb6a0054130 con 0x7fb6ac0cd830 2024-05-08T23:24:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.194+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb6ac0cd580 con 0x7fb6ac0cd830 2024-05-08T23:24:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.198+0000 7fb6a9ffb700 1 -- 172.21.15.5:0/2781528937 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb6a0057750 con 0x7fb6ac0cd830 2024-05-08T23:24:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb69806acc0 msgr2=0x7fb69806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb69806acc0 0x7fb69806d180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb69c014040 tx=0x7fb69c01d040 comp rx=0 tx=0).stop 2024-05-08T23:24:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac0cd830 msgr2=0x7fb6ac143820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac0cd830 0x7fb6ac143820 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c520 tx=0x7fb6a000c830 comp rx=0 tx=0).stop 2024-05-08T23:24:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 shutdown_connections 2024-05-08T23:24:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb69806acc0 0x7fb69806d180 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6ac15d9f0 0x7fb6ac15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6ac0af720 0x7fb6ac156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 --2- 172.21.15.5:0/2781528937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6ac0cd830 0x7fb6ac143820 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 >> 172.21.15.5:0/2781528937 conn(0x7fb6ac0b1e80 msgr2=0x7fb6ac06e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:39.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 shutdown_connections 2024-05-08T23:24:39.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.201+0000 7fb6b355d700 1 -- 172.21.15.5:0/2781528937 wait complete. 2024-05-08T23:24:39.214 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:24:39.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:24:39.255 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:24:39.304 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:24:39.305 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:24:39.305 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:39.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:24:39.325 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:24:39.326 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:39.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:24:39.346 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:24:39.346 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:39.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:24:39.388 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:24:39.388 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:39.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:24:39.409 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T23:24:39.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:39.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.573+0000 7f922f0ce700 1 Processor -- start 2024-05-08T23:24:39.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.573+0000 7f922f0ce700 1 -- start start 2024-05-08T23:24:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f922f0ce700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92280af030 0x7f922815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f922f0ce700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92280b4a80 con 0x7f922806e4d0 2024-05-08T23:24:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f922f0ce700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922806d800 con 0x7f92280b3990 2024-05-08T23:24:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f922f0ce700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922815ede0 con 0x7f92280af030 2024-05-08T23:24:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f922d66b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f922806e4d0 0x7f922806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60474/0 (socket says 172.21.15.5:60474) 2024-05-08T23:24:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f922d66b700 1 -- 172.21.15.5:0/3536017541 learned_addr learned my addr 172.21.15.5:0/3536017541 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f922ce6a700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92280af030 0x7f922815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985069193 0 0) 0x7f92280b4a80 con 0x7f922806e4d0 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92180035b0 con 0x7f922806e4d0 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953591837 0 0) 0x7f922806d800 con 0x7f92280b3990 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.574+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92280b4a80 con 0x7f92280b3990 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2524495571 0 0) 0x7f92180035b0 con 0x7f922806e4d0 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92280af030 msgr2=0x7f922815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92280af030 0x7f922815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 >> v1:172.21.15.49:6789/0 conn(0x7f92280b3990 legacy=0x7f922806ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280b3e70 con 0x7f922806e4d0 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f92200019f0 con 0x7f922806e4d0 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 >> v1:172.21.15.5:6789/0 conn(0x7f922806e4d0 legacy=0x7f922806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9218003bc0 0x7f9218003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9218004850 0x7f9218004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92280af030 0x7f9218005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922815ede0 con 0x7f9218003bc0 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921800bac0 con 0x7f9218004850 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921800bc50 con 0x7f92280af030 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f922ce6a700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9218003bc0 0x7f9218003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f9227fff700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9218004850 0x7f9218004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f922d66b700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92280af030 0x7f9218005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f922ce6a700 1 -- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92280af030 msgr2=0x7f9218005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f922ce6a700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92280af030 0x7f9218005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f922ce6a700 1 -- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9218004850 msgr2=0x7f9218004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f922ce6a700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9218004850 0x7f9218004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f922ce6a700 1 -- 172.21.15.5:0/3536017541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92180035b0 con 0x7f9218003bc0 2024-05-08T23:24:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.575+0000 7f922ce6a700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9218003bc0 0x7f9218003fc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f921c009b60 tx=0x7f9218005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faedb2372aebbe12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c00e6c0 con 0x7f9218003bc0 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c00e820 con 0x7f9218003bc0 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f92277fe700 1 -- 172.21.15.5:0/3536017541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c00eb90 con 0x7f9218003bc0 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f922f0ce700 1 -- 172.21.15.5:0/3536017541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9218003bc0 msgr2=0x7f9218003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f922f0ce700 1 --2- 172.21.15.5:0/3536017541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9218003bc0 0x7f9218003fc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f921c009b60 tx=0x7f9218005880 comp rx=0 tx=0).stop 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f922d66b700 1 -- 172.21.15.5:0/3536017541 reap_dead start 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f922f0ce700 1 -- 172.21.15.5:0/3536017541 shutdown_connections 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f922f0ce700 1 -- 172.21.15.5:0/3536017541 >> 172.21.15.5:0/3536017541 conn(0x7f92280cfce0 msgr2=0x7f92280d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f922f0ce700 1 -- 172.21.15.5:0/3536017541 shutdown_connections 2024-05-08T23:24:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.576+0000 7f922f0ce700 1 -- 172.21.15.5:0/3536017541 wait complete. 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.578+0000 7f922f0ce700 1 Processor -- start 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.578+0000 7f922f0ce700 1 -- start start 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922f0ce700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9218003bc0 0x7f9228145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922f0ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92280ce420 0x7f92280cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922f0ce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92280ad980 0x7f92280ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922f0ce700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280cd2f0 con 0x7f92280ad980 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922f0ce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280ce8a0 con 0x7f92280ce420 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922f0ce700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280ae340 con 0x7f9218003bc0 2024-05-08T23:24:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92280ad980 0x7f92280ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92280ad980 0x7f92280ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49590/0 (socket says 172.21.15.5:49590) 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 -- 172.21.15.5:0/2716371312 learned_addr learned my addr 172.21.15.5:0/2716371312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922ce6a700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9218003bc0 0x7f9228145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f9227fff700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92280ce420 0x7f92280cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 -- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9218003bc0 msgr2=0x7f9228145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9218003bc0 0x7f9228145710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 -- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92280ce420 msgr2=0x7f92280cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92280ce420 0x7f92280cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 -- 172.21.15.5:0/2716371312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92281f3380 con 0x7f92280ad980 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922d66b700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92280ad980 0x7f92280ade00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9228066ea0 tx=0x7f922000a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f92257fa700 1 -- 172.21.15.5:0/2716371312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922000acc0 con 0x7f92280ad980 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f92257fa700 1 -- 172.21.15.5:0/2716371312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922000ae40 con 0x7f92280ad980 2024-05-08T23:24:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.579+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92280cc770 con 0x7f92280ad980 2024-05-08T23:24:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.580+0000 7f92257fa700 1 -- 172.21.15.5:0/2716371312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220013630 con 0x7f92280ad980 2024-05-08T23:24:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.580+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92280ccd30 con 0x7f92280ad980 2024-05-08T23:24:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.580+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9228068000 con 0x7f92280ad980 2024-05-08T23:24:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.581+0000 7f92257fa700 1 -- 172.21.15.5:0/2716371312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9220013790 con 0x7f92280ad980 2024-05-08T23:24:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.582+0000 7f92257fa700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f920c06ac90 0x7f920c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.582+0000 7f92257fa700 1 -- 172.21.15.5:0/2716371312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9220089800 con 0x7f92280ad980 2024-05-08T23:24:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.582+0000 7f922ce6a700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f920c06ac90 0x7f920c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.583+0000 7f922ce6a700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f920c06ac90 0x7f920c06d150 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f921c0090b0 tx=0x7f921c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.587+0000 7f92257fa700 1 -- 172.21.15.5:0/2716371312 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9220054cc0 con 0x7f92280ad980 2024-05-08T23:24:39.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.853+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92280b3380 con 0x7f92280ad980 2024-05-08T23:24:39.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.854+0000 7f92257fa700 1 -- 172.21.15.5:0/2716371312 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f92200582e0 con 0x7f92280ad980 2024-05-08T23:24:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:39.854 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2344908233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2344908233},{"type":"v1","addr":"172.21.15.5:6834","nonce":2344908233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5585,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/1173846762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1173846762},{"type":"v1","addr":"172.21.15.49:6836","nonce":1173846762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6133,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2050539525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2050539525},{"type":"v1","addr":"172.21.15.49:6837","nonce":2050539525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4090421303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":4090421303},{"type":"v1","addr":"172.21.15.5:6835","nonce":4090421303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[]} 2024-05-08T23:24:39.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f920c06ac90 msgr2=0x7f920c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f920c06ac90 0x7f920c06d150 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f921c0090b0 tx=0x7f921c008030 comp rx=0 tx=0).stop 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92280ad980 msgr2=0x7f92280ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92280ad980 0x7f92280ade00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9228066ea0 tx=0x7f922000a5b0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 shutdown_connections 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f920c06ac90 0x7f920c06d150 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92280ad980 0x7f92280ade00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9218003bc0 0x7f9228145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 --2- 172.21.15.5:0/2716371312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92280ce420 0x7f92280cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 >> 172.21.15.5:0/2716371312 conn(0x7f92280cfce0 msgr2=0x7f9228154580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 shutdown_connections 2024-05-08T23:24:39.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:39.856+0000 7f922f0ce700 1 -- 172.21.15.5:0/2716371312 wait complete. 2024-05-08T23:24:39.858 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 43 2024-05-08T23:24:39.870 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:24:39.871 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T23:24:39.871 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:24:39.871 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:24:39.871 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:24:39.871 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T23:24:39.871 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:24:39.871 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:24:39.872 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:24:39.872 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T23:24:39.872 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:24:39.872 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:24:39.872 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:24:39.872 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T23:24:39.872 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:24:39.872 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:24:39.882 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:24:39.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:24:39.883 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:24:39.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:24:39.884 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:24:39.885 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:24:39.885 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:24:39.885 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:24:39.924 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:24:39.924 DEBUG:teuthology.parallel:result is None 2024-05-08T23:24:39.925 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:24:39.925 DEBUG:teuthology.parallel:result is None 2024-05-08T23:24:39.927 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:24:39.927 DEBUG:teuthology.parallel:result is None 2024-05-08T23:24:39.928 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:24:39.928 DEBUG:teuthology.parallel:result is None 2024-05-08T23:24:39.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T23:24:40.020 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:24:40.021 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:24:40.026 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:24:40.035 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:24:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.085+0000 7eff6bade700 1 Processor -- start 2024-05-08T23:24:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.085+0000 7eff6bade700 1 -- start start 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff6bade700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff6415b900 0x7eff640cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff6bade700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7eff64099a50 0x7eff64099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff6bade700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff640b1410 con 0x7eff6409a410 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff6bade700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6409a8a0 con 0x7eff64099a50 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff6bade700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff640b0e30 con 0x7eff6415b900 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff69079700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7eff64099a50 0x7eff64099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff6a07b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7eff6409a410 0x7eff640cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60500/0 (socket says 172.21.15.5:60500) 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff6a07b700 1 -- 172.21.15.5:0/1964500314 learned_addr learned my addr 172.21.15.5:0/1964500314 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.086+0000 7eff6987a700 1 --2- 172.21.15.5:0/1964500314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff6415b900 0x7eff640cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff69079700 1 -- 172.21.15.5:0/1964500314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff6415b900 msgr2=0x7eff640cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff69079700 1 --2- 172.21.15.5:0/1964500314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff6415b900 0x7eff640cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff69079700 1 -- 172.21.15.5:0/1964500314 >> v1:172.21.15.5:6789/0 conn(0x7eff6409a410 legacy=0x7eff640cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff69079700 1 -- 172.21.15.5:0/1964500314 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff640cffd0 con 0x7eff64099a50 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff68878700 1 -- 172.21.15.5:0/1964500314 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261206842 0 0) 0x7eff640b1410 con 0x7eff6409a410 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff6987a700 1 --2- 172.21.15.5:0/1964500314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff6415b900 0x7eff640cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff69079700 1 --2- 172.21.15.5:0/1964500314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff64099a50 0x7eff64099ed0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7eff54009840 tx=0x7eff54009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28a1fdc6a3e21a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff68878700 1 -- 172.21.15.5:0/1964500314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54011830 con 0x7eff64099a50 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff68878700 1 -- 172.21.15.5:0/1964500314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff54011990 con 0x7eff64099a50 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.087+0000 7eff68878700 1 -- 172.21.15.5:0/1964500314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54011d20 con 0x7eff64099a50 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.088+0000 7eff6bade700 1 -- 172.21.15.5:0/1964500314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff64099a50 msgr2=0x7eff64099ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.088+0000 7eff6bade700 1 --2- 172.21.15.5:0/1964500314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff64099a50 0x7eff64099ed0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7eff54009840 tx=0x7eff54009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.088+0000 7eff6bade700 1 -- 172.21.15.5:0/1964500314 shutdown_connections 2024-05-08T23:24:40.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.088+0000 7eff6bade700 1 --2- 172.21.15.5:0/1964500314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff6415b900 0x7eff640cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.088+0000 7eff6bade700 1 --2- 172.21.15.5:0/1964500314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff64099a50 0x7eff64099ed0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.088+0000 7eff6bade700 1 -- 172.21.15.5:0/1964500314 >> 172.21.15.5:0/1964500314 conn(0x7eff640b2f20 msgr2=0x7eff640b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.088+0000 7eff6bade700 1 -- 172.21.15.5:0/1964500314 shutdown_connections 2024-05-08T23:24:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.088+0000 7eff6bade700 1 -- 172.21.15.5:0/1964500314 wait complete. 2024-05-08T23:24:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.089+0000 7eff6bade700 1 Processor -- start 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.089+0000 7eff6bade700 1 -- start start 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6bade700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff64099a50 0x7eff6414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6bade700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6409a410 0x7eff6415e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6bade700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6415b900 0x7eff640c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6bade700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415e500 con 0x7eff6415b900 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6bade700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415de90 con 0x7eff6409a410 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6bade700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415cdd0 con 0x7eff64099a50 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6415b900 0x7eff640c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6415b900 0x7eff640c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49640/0 (socket says 172.21.15.5:49640) 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 -- 172.21.15.5:0/908335528 learned_addr learned my addr 172.21.15.5:0/908335528 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6987a700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff64099a50 0x7eff6414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff69079700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6409a410 0x7eff6415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 -- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff64099a50 msgr2=0x7eff6414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff64099a50 0x7eff6414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 -- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6409a410 msgr2=0x7eff6415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6409a410 0x7eff6415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 -- 172.21.15.5:0/908335528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff60009a70 con 0x7eff6415b900 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.090+0000 7eff6a07b700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6415b900 0x7eff640c8500 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c410 tx=0x7eff5c00c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.091+0000 7eff5a7fc700 1 -- 172.21.15.5:0/908335528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004c80 con 0x7eff6415b900 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.091+0000 7eff5a7fc700 1 -- 172.21.15.5:0/908335528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c004de0 con 0x7eff6415b900 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.091+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff540094a0 con 0x7eff6415b900 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.091+0000 7eff5a7fc700 1 -- 172.21.15.5:0/908335528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0072f0 con 0x7eff6415b900 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.091+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff640c9170 con 0x7eff6415b900 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.092+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff64068000 con 0x7eff6415b900 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.092+0000 7eff5a7fc700 1 -- 172.21.15.5:0/908335528 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7eff5c007450 con 0x7eff6415b900 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.093+0000 7eff5a7fc700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7eff4806ac90 0x7eff4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.093+0000 7eff5a7fc700 1 -- 172.21.15.5:0/908335528 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5115+0+0 (secure 0 0 0) 0x7eff5c088b00 con 0x7eff6415b900 2024-05-08T23:24:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.093+0000 7eff6987a700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7eff4806ac90 0x7eff4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.093+0000 7eff6987a700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7eff4806ac90 0x7eff4806d150 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7eff60009fd0 tx=0x7eff6000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.097+0000 7eff5a7fc700 1 -- 172.21.15.5:0/908335528 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7eff5c053fc0 con 0x7eff6415b900 2024-05-08T23:24:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:40.333+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7eff6415d2a0 con 0x7eff6415b900 2024-05-08T23:24:41.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.202+0000 7eff5a7fc700 1 -- 172.21.15.5:0/908335528 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) v1 ==== 97+0+0 (secure 0 0 0) 0x7eff5c0575e0 con 0x7eff6415b900 2024-05-08T23:24:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7eff4806ac90 msgr2=0x7eff4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7eff4806ac90 0x7eff4806d150 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7eff60009fd0 tx=0x7eff6000b040 comp rx=0 tx=0).stop 2024-05-08T23:24:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6415b900 msgr2=0x7eff640c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6415b900 0x7eff640c8500 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c410 tx=0x7eff5c00c720 comp rx=0 tx=0).stop 2024-05-08T23:24:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 shutdown_connections 2024-05-08T23:24:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7eff4806ac90 0x7eff4806d150 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6415b900 0x7eff640c8500 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7eff64099a50 0x7eff6414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 --2- 172.21.15.5:0/908335528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6409a410 0x7eff6415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 >> 172.21.15.5:0/908335528 conn(0x7eff640b2f20 msgr2=0x7eff64070410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 shutdown_connections 2024-05-08T23:24:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.205+0000 7eff6bade700 1 -- 172.21.15.5:0/908335528 wait complete. 2024-05-08T23:24:41.207 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-08T23:24:41.222 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T23:24:41.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.362+0000 7f6deb1e0700 1 Processor -- start 2024-05-08T23:24:41.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.362+0000 7f6deb1e0700 1 -- start start 2024-05-08T23:24:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.363+0000 7f6deb1e0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6de406b070 0x7f6de415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.363+0000 7f6deb1e0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de40ce100 con 0x7f6de4152470 2024-05-08T23:24:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.363+0000 7f6deb1e0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de415c220 con 0x7f6de4152850 2024-05-08T23:24:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.363+0000 7f6deb1e0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de415c500 con 0x7f6de406b070 2024-05-08T23:24:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.363+0000 7f6de977d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6de406b070 0x7f6de415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.363+0000 7f6de8f7c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6de4152470 0x7f6de40d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60502/0 (socket says 172.21.15.5:60502) 2024-05-08T23:24:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.363+0000 7f6de8f7c700 1 -- 172.21.15.5:0/1450133409 learned_addr learned my addr 172.21.15.5:0/1450133409 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.364+0000 7f6de977d700 1 -- 172.21.15.5:0/1450133409 >> v1:172.21.15.49:6789/0 conn(0x7f6de4152850 legacy=0x7f6de406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.364+0000 7f6de977d700 1 -- 172.21.15.5:0/1450133409 >> v1:172.21.15.5:6789/0 conn(0x7f6de4152470 legacy=0x7f6de40d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.364+0000 7f6de977d700 1 -- 172.21.15.5:0/1450133409 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de415c7e0 con 0x7f6de406b070 2024-05-08T23:24:41.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.364+0000 7f6de37fe700 1 -- 172.21.15.5:0/1450133409 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631064155 0 0) 0x7f6de40ce100 con 0x7f6de4152470 2024-05-08T23:24:41.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.364+0000 7f6de977d700 1 --2- 172.21.15.5:0/1450133409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de406b070 0x7f6de415bbf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009d90 tx=0x7f6ddc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcbcfa2682b9348 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.364+0000 7f6de37fe700 1 -- 172.21.15.5:0/1450133409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00e070 con 0x7f6de406b070 2024-05-08T23:24:41.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.364+0000 7f6de37fe700 1 -- 172.21.15.5:0/1450133409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00bbd0 con 0x7f6de406b070 2024-05-08T23:24:41.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.364+0000 7f6de37fe700 1 -- 172.21.15.5:0/1450133409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc012430 con 0x7f6de406b070 2024-05-08T23:24:41.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.365+0000 7f6deb1e0700 1 -- 172.21.15.5:0/1450133409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de406b070 msgr2=0x7f6de415bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.365+0000 7f6deb1e0700 1 --2- 172.21.15.5:0/1450133409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de406b070 0x7f6de415bbf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009d90 tx=0x7f6ddc00b330 comp rx=0 tx=0).stop 2024-05-08T23:24:41.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.365+0000 7f6deb1e0700 1 -- 172.21.15.5:0/1450133409 shutdown_connections 2024-05-08T23:24:41.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.365+0000 7f6deb1e0700 1 --2- 172.21.15.5:0/1450133409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de406b070 0x7f6de415bbf0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.365+0000 7f6deb1e0700 1 -- 172.21.15.5:0/1450133409 >> 172.21.15.5:0/1450133409 conn(0x7f6de4099a50 msgr2=0x7f6de40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.365+0000 7f6deb1e0700 1 -- 172.21.15.5:0/1450133409 shutdown_connections 2024-05-08T23:24:41.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.365+0000 7f6deb1e0700 1 -- 172.21.15.5:0/1450133409 wait complete. 2024-05-08T23:24:41.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.366+0000 7f6deb1e0700 1 Processor -- start 2024-05-08T23:24:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.366+0000 7f6deb1e0700 1 -- start start 2024-05-08T23:24:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6deb1e0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de406b070 0x7f6de4140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6deb1e0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de4152470 0x7f6de4140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6deb1e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de4152850 0x7f6de413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6deb1e0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40b4510 con 0x7f6de4152850 2024-05-08T23:24:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6deb1e0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40b3c40 con 0x7f6de4152470 2024-05-08T23:24:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6deb1e0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4155970 con 0x7f6de406b070 2024-05-08T23:24:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6de977d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de4152850 0x7f6de413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6de977d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de4152850 0x7f6de413e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49650/0 (socket says 172.21.15.5:49650) 2024-05-08T23:24:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6de977d700 1 -- 172.21.15.5:0/4198760644 learned_addr learned my addr 172.21.15.5:0/4198760644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6de8f7c700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de406b070 0x7f6de4140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.367+0000 7f6de3fff700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de4152470 0x7f6de4140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de977d700 1 -- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de406b070 msgr2=0x7f6de4140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de977d700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de406b070 0x7f6de4140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de977d700 1 -- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de4152470 msgr2=0x7f6de4140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de977d700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de4152470 0x7f6de4140fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de977d700 1 -- 172.21.15.5:0/4198760644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc009a70 con 0x7f6de4152850 2024-05-08T23:24:41.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de977d700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de4152850 0x7f6de413e9e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc014040 tx=0x7f6ddc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de17fa700 1 -- 172.21.15.5:0/4198760644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc01b810 con 0x7f6de4152850 2024-05-08T23:24:41.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de17fa700 1 -- 172.21.15.5:0/4198760644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc020070 con 0x7f6de4152850 2024-05-08T23:24:41.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de40b3ec0 con 0x7f6de4152850 2024-05-08T23:24:41.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6de17fa700 1 -- 172.21.15.5:0/4198760644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc01bbb0 con 0x7f6de4152850 2024-05-08T23:24:41.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.368+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de4155620 con 0x7f6de4152850 2024-05-08T23:24:41.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.369+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4068000 con 0x7f6de4152850 2024-05-08T23:24:41.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.370+0000 7f6de17fa700 1 -- 172.21.15.5:0/4198760644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6ddc01bd10 con 0x7f6de4152850 2024-05-08T23:24:41.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.370+0000 7f6de17fa700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dc406ac90 0x7f6dc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.370+0000 7f6de17fa700 1 -- 172.21.15.5:0/4198760644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6ddc022070 con 0x7f6de4152850 2024-05-08T23:24:41.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.370+0000 7f6de8f7c700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dc406ac90 0x7f6dc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.371+0000 7f6de8f7c700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dc406ac90 0x7f6dc406d150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4007b10 tx=0x7f6dd4007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.375+0000 7f6de17fa700 1 -- 172.21.15.5:0/4198760644 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6ddc056d10 con 0x7f6de4152850 2024-05-08T23:24:41.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.684+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6de40b4650 con 0x7f6de4152850 2024-05-08T23:24:41.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.685+0000 7f6de17fa700 1 -- 172.21.15.5:0/4198760644 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6ddc05a330 con 0x7f6de4152850 2024-05-08T23:24:41.685 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:41.685 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:41.685 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-08T23:24:41.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.685 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-08T23:24:41.686 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm4+KLORAAJGma8zcoGLift7K5wyvWlA==", 2024-05-08T23:24:41.686 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:24:41.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:24:41.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:24:41.686 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.687 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-08T23:24:41.687 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm5iMNDBAANT+UrtTUBpYyQ9dzJyS01Q==", 2024-05-08T23:24:41.687 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.687 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:24:41.687 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:24:41.688 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:24:41.688 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.688 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-08T23:24:41.688 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmRixkBhAAk9s24ERPJ/6nCJY3g7QppA==", 2024-05-08T23:24:41.688 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.689 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:24:41.689 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:24:41.689 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:24:41.689 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.690 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-08T23:24:41.690 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmPmurExAAxIzDzbLZ3s35XVJrvYiVCA==", 2024-05-08T23:24:41.690 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:24:41.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:24:41.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:24:41.690 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCrCDxmI4/bNBAAzNXcuLQCS9gaAJ0r4iYKlA==", 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:24:41.691 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.692 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-08T23:24:41.692 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCuCDxmMS6HNhAAcTP6nStED06bdM4MtbEPjQ==", 2024-05-08T23:24:41.692 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.692 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:24:41.692 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCxCDxmDGv5NRAAljFQZXLGWwt+UXZ+HCdx0Q==", 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:24:41.693 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC0CDxmL+5LNRAAVPySIUtICBwIokJL9fHFtw==", 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:24:41.694 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC5CDxm/F15BhAAEx9mKA9izSmb+xyxjEi9FA==", 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC8CDxm0zAYBhAAjHT+TZFqk5MWvd0P5LmwfA==", 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC/CDxmPcbiBRAAOmWNgaQGi6YGKTp8SJekUg==", 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.697 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDCCDxm9FsDBRAA/lfmrpmL4UO3s/tJJ4o6yQ==", 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm9tLAGBAAK8Lq89gJ902XqzXYAbYfkg==", 2024-05-08T23:24:41.698 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmCStoIBAAhraObJ5LS9LtF97+hIcTFQ==", 2024-05-08T23:24:41.699 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-08T23:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm2NzHHBAAUbeD58MntIoiwT8LekjIlw==", 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmyvveIxAATEn7nOPHqtqSbdfQid25HA==", 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:24:41.701 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm3ZIxDxAAM8M5YUfK9G2PkTzU4wY/+w==", 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxmntlgNRAA7JOIS+0NL1KmSSZhZjXlVQ==", 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm1dPQMBAAhu0CK6DcucQteuTo7qN2IQ==", 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:24:41.703 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:24:41.704 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:24:41.704 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.704 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:41.704 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-08T23:24:41.704 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dc406ac90 msgr2=0x7f6dc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dc406ac90 0x7f6dc406d150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4007b10 tx=0x7f6dd4007a60 comp rx=0 tx=0).stop 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de4152850 msgr2=0x7f6de413e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de4152850 0x7f6de413e9e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc014040 tx=0x7f6ddc004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 shutdown_connections 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6dc406ac90 0x7f6dc406d150 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6de4152850 0x7f6de413e9e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6de406b070 0x7f6de4140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 --2- 172.21.15.5:0/4198760644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de4152470 0x7f6de4140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 >> 172.21.15.5:0/4198760644 conn(0x7f6de4099a50 msgr2=0x7f6de415a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 shutdown_connections 2024-05-08T23:24:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.687+0000 7f6deb1e0700 1 -- 172.21.15.5:0/4198760644 wait complete. 2024-05-08T23:24:41.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.844+0000 7fdd639d1700 1 Processor -- start 2024-05-08T23:24:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.845+0000 7fdd639d1700 1 -- start start 2024-05-08T23:24:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd639d1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd5c0cecf0 0x7fdd5c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd639d1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdd5c06f130 0x7fdd5c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd639d1700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c15c280 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd639d1700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c06e780 con 0x7fdd5c0cecf0 2024-05-08T23:24:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd639d1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c06ea60 con 0x7fdd5c06f130 2024-05-08T23:24:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd60f6c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd5c0cecf0 0x7fdd5c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd60f6c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd5c0cecf0 0x7fdd5c06f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49662/0 (socket says 172.21.15.5:49662) 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd60f6c700 1 -- 172.21.15.5:0/2343667021 learned_addr learned my addr 172.21.15.5:0/2343667021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd61f6e700 1 --2- 172.21.15.5:0/2343667021 >> v2:172.21.15.49:3300/0 conn(0x7fdd5c06f130 0x7fdd5c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd60f6c700 1 -- 172.21.15.5:0/2343667021 >> v2:172.21.15.49:3300/0 conn(0x7fdd5c06f130 msgr2=0x7fdd5c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd60f6c700 1 --2- 172.21.15.5:0/2343667021 >> v2:172.21.15.49:3300/0 conn(0x7fdd5c06f130 0x7fdd5c15c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd60f6c700 1 -- 172.21.15.5:0/2343667021 >> v1:172.21.15.49:6789/0 conn(0x7fdd5c0cd4e0 legacy=0x7fdd5c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.846+0000 7fdd60f6c700 1 -- 172.21.15.5:0/2343667021 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c06ed40 con 0x7fdd5c0cecf0 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.847+0000 7fdd60f6c700 1 --2- 172.21.15.5:0/2343667021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cecf0 0x7fdd5c06f910 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c15f520 tx=0x7fdd4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a78a59ec3c19e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.847+0000 7fdd53fff700 1 -- 172.21.15.5:0/2343667021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004230 con 0x7fdd5c0cecf0 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.847+0000 7fdd53fff700 1 -- 172.21.15.5:0/2343667021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c004390 con 0x7fdd5c0cecf0 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.847+0000 7fdd53fff700 1 -- 172.21.15.5:0/2343667021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0036a0 con 0x7fdd5c0cecf0 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.847+0000 7fdd639d1700 1 -- 172.21.15.5:0/2343667021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cecf0 msgr2=0x7fdd5c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.847+0000 7fdd639d1700 1 --2- 172.21.15.5:0/2343667021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cecf0 0x7fdd5c06f910 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c15f520 tx=0x7fdd4c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.847+0000 7fdd639d1700 1 -- 172.21.15.5:0/2343667021 shutdown_connections 2024-05-08T23:24:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.847+0000 7fdd639d1700 1 --2- 172.21.15.5:0/2343667021 >> v2:172.21.15.49:3300/0 conn(0x7fdd5c06f130 0x7fdd5c15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.848+0000 7fdd639d1700 1 --2- 172.21.15.5:0/2343667021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cecf0 0x7fdd5c06f910 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.848+0000 7fdd639d1700 1 -- 172.21.15.5:0/2343667021 >> 172.21.15.5:0/2343667021 conn(0x7fdd5c0d0320 msgr2=0x7fdd5c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.848+0000 7fdd639d1700 1 -- 172.21.15.5:0/2343667021 shutdown_connections 2024-05-08T23:24:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.848+0000 7fdd639d1700 1 -- 172.21.15.5:0/2343667021 wait complete. 2024-05-08T23:24:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.849+0000 7fdd639d1700 1 Processor -- start 2024-05-08T23:24:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.849+0000 7fdd639d1700 1 -- start start 2024-05-08T23:24:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.850+0000 7fdd639d1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd5c06f130 0x7fdd5c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.850+0000 7fdd639d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cd4e0 0x7fdd5c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.850+0000 7fdd639d1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c0cecf0 0x7fdd5c0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.850+0000 7fdd639d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c157020 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.850+0000 7fdd639d1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0b3460 con 0x7fdd5c0cecf0 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.850+0000 7fdd639d1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0b3690 con 0x7fdd5c06f130 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cd4e0 0x7fdd5c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cd4e0 0x7fdd5c0b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49666/0 (socket says 172.21.15.5:49666) 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 -- 172.21.15.5:0/430935181 learned_addr learned my addr 172.21.15.5:0/430935181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd6176d700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd5c06f130 0x7fdd5c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd61f6e700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c0cecf0 0x7fdd5c0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 -- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd5c06f130 msgr2=0x7fdd5c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd5c06f130 0x7fdd5c0b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 -- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c0cecf0 msgr2=0x7fdd5c0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c0cecf0 0x7fdd5c0b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 -- 172.21.15.5:0/430935181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009550 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd60f6c700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cd4e0 0x7fdd5c0b4730 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009130 tx=0x7fdd4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd51ffb700 1 -- 172.21.15.5:0/430935181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c003d60 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd51ffb700 1 -- 172.21.15.5:0/430935181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c011630 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c0b3970 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.851+0000 7fdd51ffb700 1 -- 172.21.15.5:0/430935181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0119c0 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.852+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c0b3f30 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.853+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c068000 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.853+0000 7fdd51ffb700 1 -- 172.21.15.5:0/430935181 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdd4c011b20 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.854+0000 7fdd51ffb700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdd4006ac90 0x7fdd4006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.854+0000 7fdd51ffb700 1 -- 172.21.15.5:0/430935181 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdd4c08a470 con 0x7fdd5c0cd4e0 2024-05-08T23:24:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.854+0000 7fdd6176d700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdd4006ac90 0x7fdd4006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.855+0000 7fdd6176d700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdd4006ac90 0x7fdd4006d150 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdd58007fd0 tx=0x7fdd58007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:41.859+0000 7fdd51ffb700 1 -- 172.21.15.5:0/430935181 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdd4c0559b0 con 0x7fdd5c0cd4e0 2024-05-08T23:24:42.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.080+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5c06d440 con 0x7fdd5c0cd4e0 2024-05-08T23:24:42.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.081+0000 7fdd51ffb700 1 -- 172.21.15.5:0/430935181 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12293 (secure 0 0 0) 0x7fdd4c058fd0 con 0x7fdd5c0cd4e0 2024-05-08T23:24:42.081 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:42.081 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":56,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:41.198667+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdd4006ac90 msgr2=0x7fdd4006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdd4006ac90 0x7fdd4006d150 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdd58007fd0 tx=0x7fdd58007ec0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cd4e0 msgr2=0x7fdd5c0b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cd4e0 0x7fdd5c0b4730 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009130 tx=0x7fdd4c004750 comp rx=0 tx=0).stop 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 shutdown_connections 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdd4006ac90 0x7fdd4006d150 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c0cecf0 0x7fdd5c0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdd5c06f130 0x7fdd5c0b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 --2- 172.21.15.5:0/430935181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd5c0cd4e0 0x7fdd5c0b4730 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.083+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 >> 172.21.15.5:0/430935181 conn(0x7fdd5c0d0320 msgr2=0x7fdd5c15da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.084+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 shutdown_connections 2024-05-08T23:24:42.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.084+0000 7fdd639d1700 1 -- 172.21.15.5:0/430935181 wait complete. 2024-05-08T23:24:42.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:24:42.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:24:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.239+0000 7ff734cb8700 1 Processor -- start 2024-05-08T23:24:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.239+0000 7ff734cb8700 1 -- start start 2024-05-08T23:24:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff734cb8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff7300cbf50 0x7ff730161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff734cb8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff730162240 0x7ff73015e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff734cb8700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7300cf970 con 0x7ff7300d0f10 2024-05-08T23:24:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff734cb8700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7300cfc50 con 0x7ff730162240 2024-05-08T23:24:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff734cb8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300cff30 con 0x7ff7300cbf50 2024-05-08T23:24:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72dd9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff730162240 0x7ff73015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72dd9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff730162240 0x7ff73015e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49674/0 (socket says 172.21.15.5:49674) 2024-05-08T23:24:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72dd9b700 1 -- 172.21.15.5:0/3006809520 learned_addr learned my addr 172.21.15.5:0/3006809520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72e59c700 1 --2- 172.21.15.5:0/3006809520 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff7300cbf50 0x7ff730161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72dd9b700 1 -- 172.21.15.5:0/3006809520 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff7300cbf50 msgr2=0x7ff730161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72dd9b700 1 --2- 172.21.15.5:0/3006809520 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff7300cbf50 0x7ff730161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72dd9b700 1 -- 172.21.15.5:0/3006809520 >> v1:172.21.15.49:6789/0 conn(0x7ff7300d0f10 legacy=0x7ff73006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72dd9b700 1 -- 172.21.15.5:0/3006809520 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7301617e0 con 0x7ff730162240 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.240+0000 7ff72dd9b700 1 --2- 172.21.15.5:0/3006809520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff730162240 0x7ff73015e050 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff7240098f0 tx=0x7ff724009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9c2cf606299e54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff72d59a700 1 -- 172.21.15.5:0/3006809520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400b8a0 con 0x7ff730162240 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff72d59a700 1 -- 172.21.15.5:0/3006809520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72400ba00 con 0x7ff730162240 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff72d59a700 1 -- 172.21.15.5:0/3006809520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400bd90 con 0x7ff730162240 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff734cb8700 1 -- 172.21.15.5:0/3006809520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff730162240 msgr2=0x7ff73015e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3006809520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff730162240 0x7ff73015e050 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff7240098f0 tx=0x7ff724009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff734cb8700 1 -- 172.21.15.5:0/3006809520 shutdown_connections 2024-05-08T23:24:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3006809520 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff7300cbf50 0x7ff730161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3006809520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff730162240 0x7ff73015e050 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff734cb8700 1 -- 172.21.15.5:0/3006809520 >> 172.21.15.5:0/3006809520 conn(0x7ff730099a50 msgr2=0x7ff7300d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff734cb8700 1 -- 172.21.15.5:0/3006809520 shutdown_connections 2024-05-08T23:24:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.241+0000 7ff734cb8700 1 -- 172.21.15.5:0/3006809520 wait complete. 2024-05-08T23:24:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.243+0000 7ff734cb8700 1 Processor -- start 2024-05-08T23:24:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.243+0000 7ff734cb8700 1 -- start start 2024-05-08T23:24:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff734cb8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7300cbf50 0x7ff730151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff734cb8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7300d0f10 0x7ff7301579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff734cb8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff730162240 0x7ff7300b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff734cb8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730157ef0 con 0x7ff7300d0f10 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff734cb8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730154860 con 0x7ff7300cbf50 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff734cb8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff730154250 con 0x7ff730162240 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7300d0f10 0x7ff7301579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7300d0f10 0x7ff7301579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49680/0 (socket says 172.21.15.5:49680) 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 -- 172.21.15.5:0/3350544145 learned_addr learned my addr 172.21.15.5:0/3350544145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72e59c700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7300cbf50 0x7ff730151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72ed9d700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff730162240 0x7ff7300b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 -- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff730162240 msgr2=0x7ff7300b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff730162240 0x7ff7300b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 -- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7300cbf50 msgr2=0x7ff730151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7300cbf50 0x7ff730151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 -- 172.21.15.5:0/3350544145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724009550 con 0x7ff7300d0f10 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72ed9d700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff730162240 0x7ff7300b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.244+0000 7ff72dd9b700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7300d0f10 0x7ff7301579b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff724009c40 tx=0x7ff724004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.245+0000 7ff71effd700 1 -- 172.21.15.5:0/3350544145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724004590 con 0x7ff7300d0f10 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.245+0000 7ff71effd700 1 -- 172.21.15.5:0/3350544145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72401e070 con 0x7ff7300d0f10 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.245+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7300b5da0 con 0x7ff7300d0f10 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.245+0000 7ff71effd700 1 -- 172.21.15.5:0/3350544145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7240038e0 con 0x7ff7300d0f10 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.245+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7301544f0 con 0x7ff7300d0f10 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.246+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff730074bc0 con 0x7ff7300d0f10 2024-05-08T23:24:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.246+0000 7ff71effd700 1 -- 172.21.15.5:0/3350544145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff724003a40 con 0x7ff7300d0f10 2024-05-08T23:24:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.247+0000 7ff71effd700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff71006acc0 0x7ff71006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.247+0000 7ff71effd700 1 -- 172.21.15.5:0/3350544145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff724020070 con 0x7ff7300d0f10 2024-05-08T23:24:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.247+0000 7ff72e59c700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff71006acc0 0x7ff71006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.248+0000 7ff72e59c700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff71006acc0 0x7ff71006d180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7200099d0 tx=0x7ff720008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.251+0000 7ff71effd700 1 -- 172.21.15.5:0/3350544145 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff724055b10 con 0x7ff7300d0f10 2024-05-08T23:24:42.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.471+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7300b62b0 con 0x7ff7300d0f10 2024-05-08T23:24:42.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.471+0000 7ff71effd700 1 -- 172.21.15.5:0/3350544145 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff724059130 con 0x7ff7300d0f10 2024-05-08T23:24:42.472 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff71006acc0 msgr2=0x7ff71006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff71006acc0 0x7ff71006d180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7200099d0 tx=0x7ff720008040 comp rx=0 tx=0).stop 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7300d0f10 msgr2=0x7ff7301579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7300d0f10 0x7ff7301579b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff724009c40 tx=0x7ff724004750 comp rx=0 tx=0).stop 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 shutdown_connections 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff71006acc0 0x7ff71006d180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff730162240 0x7ff7300b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7300cbf50 0x7ff730151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 --2- 172.21.15.5:0/3350544145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7300d0f10 0x7ff7301579b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 >> 172.21.15.5:0/3350544145 conn(0x7ff730099a50 msgr2=0x7ff7300cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 shutdown_connections 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.473+0000 7ff734cb8700 1 -- 172.21.15.5:0/3350544145 wait complete. 2024-05-08T23:24:42.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.625+0000 7f961cc81700 1 Processor -- start 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.625+0000 7f961cc81700 1 -- start start 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f961cc81700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f961815d9c0 0x7f96180cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f961cc81700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96180cf7a0 0x7f96180d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f961cc81700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96180d0880 con 0x7f961806b810 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f961cc81700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96180d0b60 con 0x7f96180cf7a0 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f961cc81700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96180ce5b0 con 0x7f961815d9c0 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f9616d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96180cf7a0 0x7f96180d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f9616d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96180cf7a0 0x7f96180d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49692/0 (socket says 172.21.15.5:49692) 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f9616d9d700 1 -- 172.21.15.5:0/1268718163 learned_addr learned my addr 172.21.15.5:0/1268718163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.626+0000 7f961659c700 1 --2- 172.21.15.5:0/1268718163 >> v2:172.21.15.49:3300/0 conn(0x7f961815d9c0 0x7f96180cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.627+0000 7f9616d9d700 1 -- 172.21.15.5:0/1268718163 >> v2:172.21.15.49:3300/0 conn(0x7f961815d9c0 msgr2=0x7f96180cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.627+0000 7f9616d9d700 1 --2- 172.21.15.5:0/1268718163 >> v2:172.21.15.49:3300/0 conn(0x7f961815d9c0 0x7f96180cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.627+0000 7f9616d9d700 1 -- 172.21.15.5:0/1268718163 >> v1:172.21.15.49:6789/0 conn(0x7f961806b810 legacy=0x7f96180cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.627+0000 7f9616d9d700 1 -- 172.21.15.5:0/1268718163 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96180ce890 con 0x7f96180cf7a0 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.627+0000 7f9616d9d700 1 --2- 172.21.15.5:0/1268718163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96180cf7a0 0x7f96180d16e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f961808b780 tx=0x7f961000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1831b0fa6c22860 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.627+0000 7f961559a700 1 -- 172.21.15.5:0/1268718163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961000e070 con 0x7f96180cf7a0 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.627+0000 7f961559a700 1 -- 172.21.15.5:0/1268718163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961000b910 con 0x7f96180cf7a0 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.627+0000 7f961559a700 1 -- 172.21.15.5:0/1268718163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961000bcc0 con 0x7f96180cf7a0 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.628+0000 7f961cc81700 1 -- 172.21.15.5:0/1268718163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96180cf7a0 msgr2=0x7f96180d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.628+0000 7f961cc81700 1 --2- 172.21.15.5:0/1268718163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96180cf7a0 0x7f96180d16e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f961808b780 tx=0x7f961000b330 comp rx=0 tx=0).stop 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.628+0000 7f961cc81700 1 -- 172.21.15.5:0/1268718163 shutdown_connections 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.628+0000 7f961cc81700 1 --2- 172.21.15.5:0/1268718163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96180cf7a0 0x7f96180d16e0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.628+0000 7f961cc81700 1 --2- 172.21.15.5:0/1268718163 >> v2:172.21.15.49:3300/0 conn(0x7f961815d9c0 0x7f96180cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.628+0000 7f961cc81700 1 -- 172.21.15.5:0/1268718163 >> 172.21.15.5:0/1268718163 conn(0x7f961808afe0 msgr2=0x7f96180d3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.628+0000 7f961cc81700 1 -- 172.21.15.5:0/1268718163 shutdown_connections 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.628+0000 7f961cc81700 1 -- 172.21.15.5:0/1268718163 wait complete. 2024-05-08T23:24:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.630+0000 7f961cc81700 1 Processor -- start 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.630+0000 7f961cc81700 1 -- start start 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961cc81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f961806b810 0x7f96181e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961cc81700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96180cf7a0 0x7f961815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961cc81700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f961815d9c0 0x7f96180b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961cc81700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180cb3c0 con 0x7f961806b810 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961cc81700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180ca7b0 con 0x7f96180cf7a0 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961cc81700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96180caa90 con 0x7f961815d9c0 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f961806b810 0x7f96181e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f961806b810 0x7f96181e6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49704/0 (socket says 172.21.15.5:49704) 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 -- 172.21.15.5:0/3757733840 learned_addr learned my addr 172.21.15.5:0/3757733840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f9616d9d700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f961815d9c0 0x7f96180b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f9615d9b700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96180cf7a0 0x7f961815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 -- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f961815d9c0 msgr2=0x7f96180b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f961815d9c0 0x7f96180b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 -- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96180cf7a0 msgr2=0x7f961815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96180cf7a0 0x7f961815bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 -- 172.21.15.5:0/3757733840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610009b30 con 0x7f961806b810 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f9615d9b700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96180cf7a0 0x7f961815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.631+0000 7f961659c700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f961806b810 0x7f96181e6ef0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f960800e870 tx=0x7f960800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.632+0000 7f9606ffd700 1 -- 172.21.15.5:0/3757733840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608009590 con 0x7f961806b810 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.632+0000 7f9606ffd700 1 -- 172.21.15.5:0/3757733840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9608013680 con 0x7f961806b810 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.632+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96180cad70 con 0x7f961806b810 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.632+0000 7f9606ffd700 1 -- 172.21.15.5:0/3757733840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96080139f0 con 0x7f961806b810 2024-05-08T23:24:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.632+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96181e7430 con 0x7f961806b810 2024-05-08T23:24:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.633+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96180678f0 con 0x7f961806b810 2024-05-08T23:24:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.633+0000 7f9606ffd700 1 -- 172.21.15.5:0/3757733840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9608013b50 con 0x7f961806b810 2024-05-08T23:24:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.634+0000 7f9606ffd700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f806ac90 0x7f95f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.634+0000 7f9606ffd700 1 -- 172.21.15.5:0/3757733840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9608088ff0 con 0x7f961806b810 2024-05-08T23:24:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.634+0000 7f9615d9b700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f806ac90 0x7f95f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.635+0000 7f9615d9b700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f806ac90 0x7f95f806d150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f96180b49e0 tx=0x7f960c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.639+0000 7f9606ffd700 1 -- 172.21.15.5:0/3757733840 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f96080bc900 con 0x7f961806b810 2024-05-08T23:24:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.857+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f96180d2740 con 0x7f961806b810 2024-05-08T23:24:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.858+0000 7f9606ffd700 1 -- 172.21.15.5:0/3757733840 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+65 (secure 0 0 0) 0x7f96080544a0 con 0x7f961806b810 2024-05-08T23:24:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:42.858 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:24:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:24:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-08T23:24:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-08T23:24:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:24:42.859 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f806ac90 msgr2=0x7f95f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f806ac90 0x7f95f806d150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f96180b49e0 tx=0x7f960c009040 comp rx=0 tx=0).stop 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f961806b810 msgr2=0x7f96181e6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f961806b810 0x7f96181e6ef0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f960800e870 tx=0x7f960800eb80 comp rx=0 tx=0).stop 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 shutdown_connections 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f95f806ac90 0x7f95f806d150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f961815d9c0 0x7f96180b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f961806b810 0x7f96181e6ef0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 --2- 172.21.15.5:0/3757733840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96180cf7a0 0x7f961815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 >> 172.21.15.5:0/3757733840 conn(0x7f961808afe0 msgr2=0x7f961806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 shutdown_connections 2024-05-08T23:24:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:42.860+0000 7f961cc81700 1 -- 172.21.15.5:0/3757733840 wait complete. 2024-05-08T23:24:42.874 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:24:42.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:24:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.015+0000 7fecb6679700 1 Processor -- start 2024-05-08T23:24:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.015+0000 7fecb6679700 1 -- start start 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecb6679700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fecb015b4e0 0x7fecb00b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecb6679700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb00cf560 con 0x7fecb00d1910 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecb6679700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb00ccff0 con 0x7fecb015b8c0 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecb6679700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb00cefe0 con 0x7fecb015b4e0 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecaffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fecb015b4e0 0x7fecb00b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecb4c16700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fecb00d1910 0x7fecb00cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60510/0 (socket says 172.21.15.5:60510) 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecb4c16700 1 -- 172.21.15.5:0/1300321175 learned_addr learned my addr 172.21.15.5:0/1300321175 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecaffff700 1 -- 172.21.15.5:0/1300321175 >> v1:172.21.15.49:6789/0 conn(0x7fecb015b8c0 legacy=0x7fecb00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecaffff700 1 -- 172.21.15.5:0/1300321175 >> v1:172.21.15.5:6789/0 conn(0x7fecb00d1910 legacy=0x7fecb00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecaffff700 1 -- 172.21.15.5:0/1300321175 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00cd2d0 con 0x7fecb015b4e0 2024-05-08T23:24:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.016+0000 7fecaeffd700 1 -- 172.21.15.5:0/1300321175 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252900456 0 0) 0x7fecb00cf560 con 0x7fecb00d1910 2024-05-08T23:24:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecaffff700 1 --2- 172.21.15.5:0/1300321175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb015b4e0 0x7fecb00b1200 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7feca0009840 tx=0x7feca0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adac28c54cad15f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecaeffd700 1 -- 172.21.15.5:0/1300321175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0011830 con 0x7fecb015b4e0 2024-05-08T23:24:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecaeffd700 1 -- 172.21.15.5:0/1300321175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca0011990 con 0x7fecb015b4e0 2024-05-08T23:24:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecaeffd700 1 -- 172.21.15.5:0/1300321175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0011d20 con 0x7fecb015b4e0 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecb6679700 1 -- 172.21.15.5:0/1300321175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb015b4e0 msgr2=0x7fecb00b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecb6679700 1 --2- 172.21.15.5:0/1300321175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb015b4e0 0x7fecb00b1200 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7feca0009840 tx=0x7feca0009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecb6679700 1 -- 172.21.15.5:0/1300321175 shutdown_connections 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecb6679700 1 --2- 172.21.15.5:0/1300321175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb015b4e0 0x7fecb00b1200 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecb6679700 1 -- 172.21.15.5:0/1300321175 >> 172.21.15.5:0/1300321175 conn(0x7fecb008afe0 msgr2=0x7fecb008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.017+0000 7fecb6679700 1 -- 172.21.15.5:0/1300321175 shutdown_connections 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.018+0000 7fecb6679700 1 -- 172.21.15.5:0/1300321175 wait complete. 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.019+0000 7fecb6679700 1 Processor -- start 2024-05-08T23:24:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.019+0000 7fecb6679700 1 -- start start 2024-05-08T23:24:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecb6679700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb00d1910 0x7fecb00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecb6679700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fecb015b4e0 0x7fecb0150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecb6679700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecb015b8c0 0x7fecb0151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecb6679700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cb3c0 con 0x7fecb00d1910 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecb6679700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00c8cd0 con 0x7fecb015b8c0 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecb6679700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cb0e0 con 0x7fecb015b4e0 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb00d1910 0x7fecb00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb00d1910 0x7fecb00ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49732/0 (socket says 172.21.15.5:49732) 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 -- 172.21.15.5:0/3639028115 learned_addr learned my addr 172.21.15.5:0/3639028115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecb4c16700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecb015b8c0 0x7fecb0151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaf7fe700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fecb015b4e0 0x7fecb0150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 -- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fecb015b4e0 msgr2=0x7fecb0150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fecb015b4e0 0x7fecb0150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 -- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecb015b8c0 msgr2=0x7fecb0151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecb015b8c0 0x7fecb0151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 -- 172.21.15.5:0/3639028115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca00094a0 con 0x7fecb00d1910 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaf7fe700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fecb015b4e0 0x7fecb0150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.020+0000 7fecaffff700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb00d1910 0x7fecb00ca770 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7feca00090b0 tx=0x7feca0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.021+0000 7fecacff9700 1 -- 172.21.15.5:0/3639028115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca001a900 con 0x7fecb00d1910 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.021+0000 7fecacff9700 1 -- 172.21.15.5:0/3639028115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca001aa60 con 0x7fecb00d1910 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.021+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb00c8680 con 0x7fecb00d1910 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.021+0000 7fecacff9700 1 -- 172.21.15.5:0/3639028115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca001add0 con 0x7fecb00d1910 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.021+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb00cacb0 con 0x7fecb00d1910 2024-05-08T23:24:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.022+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb00caea0 con 0x7fecb00d1910 2024-05-08T23:24:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.022+0000 7fecacff9700 1 -- 172.21.15.5:0/3639028115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feca0022070 con 0x7fecb00d1910 2024-05-08T23:24:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.023+0000 7fecacff9700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec9c06ac90 0x7fec9c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.023+0000 7fecacff9700 1 -- 172.21.15.5:0/3639028115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feca008a180 con 0x7fecb00d1910 2024-05-08T23:24:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.023+0000 7fecaf7fe700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec9c06ac90 0x7fec9c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.023+0000 7fecaf7fe700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec9c06ac90 0x7fec9c06d150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fecb015ad60 tx=0x7feca4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.028+0000 7fecacff9700 1 -- 172.21.15.5:0/3639028115 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fecb00caea0 con 0x7fecb00d1910 2024-05-08T23:24:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.244+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fecb00caea0 con 0x7fecb00d1910 2024-05-08T23:24:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.817+0000 7fecacff9700 1 -- 172.21.15.5:0/3639028115 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7fecb00caea0 con 0x7fecb00d1910 2024-05-08T23:24:43.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec9c06ac90 msgr2=0x7fec9c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec9c06ac90 0x7fec9c06d150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fecb015ad60 tx=0x7feca4009040 comp rx=0 tx=0).stop 2024-05-08T23:24:43.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb00d1910 msgr2=0x7fecb00ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb00d1910 0x7fecb00ca770 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7feca00090b0 tx=0x7feca0004750 comp rx=0 tx=0).stop 2024-05-08T23:24:43.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 shutdown_connections 2024-05-08T23:24:43.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fec9c06ac90 0x7fec9c06d150 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecb015b8c0 0x7fecb0151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecb00d1910 0x7fecb00ca770 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 --2- 172.21.15.5:0/3639028115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fecb015b4e0 0x7fecb0150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 >> 172.21.15.5:0/3639028115 conn(0x7fecb008afe0 msgr2=0x7fecb0155ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:43.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 shutdown_connections 2024-05-08T23:24:43.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.823+0000 7fecb6679700 1 -- 172.21.15.5:0/3639028115 wait complete. 2024-05-08T23:24:43.824 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-08T23:24:43.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.987+0000 7f86dd68a700 1 Processor -- start 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.987+0000 7f86dd68a700 1 -- start start 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86dd68a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d815d700 0x7f86d80b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86dd68a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f86d80b38b0 0x7f86d81614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86dd68a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d80b49b0 con 0x7f86d815dae0 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86dd68a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d80b1150 con 0x7f86d80b38b0 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86dd68a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80b3da0 con 0x7f86d815d700 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86d77fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f86d815dae0 0x7f86d80d18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60512/0 (socket says 172.21.15.5:60512) 2024-05-08T23:24:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86d77fe700 1 -- 172.21.15.5:0/1451462777 learned_addr learned my addr 172.21.15.5:0/1451462777 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86d6ffd700 1 --2- 172.21.15.5:0/1451462777 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d815d700 0x7f86d80b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86d67fc700 1 --2- 172.21.15.5:0/1451462777 >> v2:172.21.15.49:3300/0 conn(0x7f86d80b38b0 0x7f86d81614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86d5ffb700 1 -- 172.21.15.5:0/1451462777 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006547838 0 0) 0x7f86d80b49b0 con 0x7f86d815dae0 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.988+0000 7f86d5ffb700 1 -- 172.21.15.5:0/1451462777 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c00035b0 con 0x7f86d815dae0 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d5ffb700 1 -- 172.21.15.5:0/1451462777 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 717218546 0 0) 0x7f86c00035b0 con 0x7f86d815dae0 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d6ffd700 1 -- 172.21.15.5:0/1451462777 >> v2:172.21.15.49:3300/0 conn(0x7f86d80b38b0 msgr2=0x7f86d81614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d6ffd700 1 --2- 172.21.15.5:0/1451462777 >> v2:172.21.15.49:3300/0 conn(0x7f86d80b38b0 0x7f86d81614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d6ffd700 1 -- 172.21.15.5:0/1451462777 >> v1:172.21.15.5:6789/0 conn(0x7f86d815dae0 legacy=0x7f86d80d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d6ffd700 1 -- 172.21.15.5:0/1451462777 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80b4080 con 0x7f86d815d700 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d67fc700 1 --2- 172.21.15.5:0/1451462777 >> v2:172.21.15.49:3300/0 conn(0x7f86d80b38b0 0x7f86d81614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d6ffd700 1 --2- 172.21.15.5:0/1451462777 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d815d700 0x7f86d80b3370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009820 tx=0x7f86c8009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea918a3647cf585a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d5ffb700 1 -- 172.21.15.5:0/1451462777 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d00019f0 con 0x7f86d815d700 2024-05-08T23:24:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d5ffb700 1 -- 172.21.15.5:0/1451462777 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c8005c30 con 0x7f86d815d700 2024-05-08T23:24:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.989+0000 7f86d5ffb700 1 -- 172.21.15.5:0/1451462777 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8004030 con 0x7f86d815d700 2024-05-08T23:24:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.990+0000 7f86dd68a700 1 -- 172.21.15.5:0/1451462777 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d815d700 msgr2=0x7f86d80b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.990+0000 7f86dd68a700 1 --2- 172.21.15.5:0/1451462777 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d815d700 0x7f86d80b3370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009820 tx=0x7f86c8009b30 comp rx=0 tx=0).stop 2024-05-08T23:24:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.990+0000 7f86dd68a700 1 -- 172.21.15.5:0/1451462777 shutdown_connections 2024-05-08T23:24:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.990+0000 7f86dd68a700 1 --2- 172.21.15.5:0/1451462777 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d815d700 0x7f86d80b3370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.990+0000 7f86dd68a700 1 --2- 172.21.15.5:0/1451462777 >> v2:172.21.15.49:3300/0 conn(0x7f86d80b38b0 0x7f86d81614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.990+0000 7f86dd68a700 1 -- 172.21.15.5:0/1451462777 >> 172.21.15.5:0/1451462777 conn(0x7f86d808afe0 msgr2=0x7f86d80d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.990+0000 7f86dd68a700 1 -- 172.21.15.5:0/1451462777 shutdown_connections 2024-05-08T23:24:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.990+0000 7f86dd68a700 1 -- 172.21.15.5:0/1451462777 wait complete. 2024-05-08T23:24:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.991+0000 7f86dd68a700 1 Processor -- start 2024-05-08T23:24:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.991+0000 7f86dd68a700 1 -- start start 2024-05-08T23:24:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86dd68a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d80b38b0 0x7f86d8151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86dd68a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d815d700 0x7f86d80b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86dd68a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d815dae0 0x7f86d80b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86dd68a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d81521f0 con 0x7f86d815d700 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86dd68a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80b8f10 con 0x7f86d815dae0 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86dd68a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80b7000 con 0x7f86d80b38b0 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86d67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d815d700 0x7f86d80b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86d67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d815d700 0x7f86d80b8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49742/0 (socket says 172.21.15.5:49742) 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.992+0000 7f86d67fc700 1 -- 172.21.15.5:0/2152566097 learned_addr learned my addr 172.21.15.5:0/2152566097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d77fe700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d815dae0 0x7f86d80b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d6ffd700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d80b38b0 0x7f86d8151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d67fc700 1 -- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d80b38b0 msgr2=0x7f86d8151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d67fc700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d80b38b0 0x7f86d8151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d67fc700 1 -- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d815dae0 msgr2=0x7f86d80b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d67fc700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d815dae0 0x7f86d80b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d67fc700 1 -- 172.21.15.5:0/2152566097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc0094a0 con 0x7f86d815d700 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d77fe700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d815dae0 0x7f86d80b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86d67fc700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d815d700 0x7f86d80b8490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00e9b0 tx=0x7f86cc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86bf7fe700 1 -- 172.21.15.5:0/2152566097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc009700 con 0x7f86d815d700 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86bf7fe700 1 -- 172.21.15.5:0/2152566097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc00c850 con 0x7f86d815d700 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c8009430 con 0x7f86d815d700 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86bf7fe700 1 -- 172.21.15.5:0/2152566097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00cc00 con 0x7f86d815d700 2024-05-08T23:24:44.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.993+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d80c7e50 con 0x7f86d815d700 2024-05-08T23:24:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.994+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d8074bc0 con 0x7f86d815d700 2024-05-08T23:24:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.995+0000 7f86bf7fe700 1 -- 172.21.15.5:0/2152566097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f86cc013470 con 0x7f86d815d700 2024-05-08T23:24:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.995+0000 7f86bf7fe700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c406ac90 0x7f86c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.996+0000 7f86bf7fe700 1 -- 172.21.15.5:0/2152566097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f86cc089140 con 0x7f86d815d700 2024-05-08T23:24:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.996+0000 7f86d6ffd700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c406ac90 0x7f86c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:43.996+0000 7f86d6ffd700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c406ac90 0x7f86c406d150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009b70 tx=0x7f86c8008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.001+0000 7f86bf7fe700 1 -- 172.21.15.5:0/2152566097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f86cc054590 con 0x7f86d815d700 2024-05-08T23:24:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.222+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f86d80d0e00 con 0x7f86d815d700 2024-05-08T23:24:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.818+0000 7f86bf7fe700 1 -- 172.21.15.5:0/2152566097 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7f86cc057bb0 con 0x7f86d815d700 2024-05-08T23:24:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.832+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c406ac90 msgr2=0x7f86c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.832+0000 7f86dd68a700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c406ac90 0x7f86c406d150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009b70 tx=0x7f86c8008070 comp rx=0 tx=0).stop 2024-05-08T23:24:44.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.832+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d815d700 msgr2=0x7f86d80b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.832+0000 7f86dd68a700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d815d700 0x7f86d80b8490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00e9b0 tx=0x7f86cc00ecc0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.833+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 shutdown_connections 2024-05-08T23:24:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.833+0000 7f86dd68a700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c406ac90 0x7f86c406d150 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.833+0000 7f86dd68a700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86d815dae0 0x7f86d80b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.833+0000 7f86dd68a700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86d80b38b0 0x7f86d8151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.833+0000 7f86dd68a700 1 --2- 172.21.15.5:0/2152566097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d815d700 0x7f86d80b8490 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.833+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 >> 172.21.15.5:0/2152566097 conn(0x7f86d808afe0 msgr2=0x7f86d80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:44.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.833+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 shutdown_connections 2024-05-08T23:24:44.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.833+0000 7f86dd68a700 1 -- 172.21.15.5:0/2152566097 wait complete. 2024-05-08T23:24:44.836 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-08T23:24:44.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:24:44.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.995+0000 7fb997053700 1 Processor -- start 2024-05-08T23:24:44.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.995+0000 7fb997053700 1 -- start start 2024-05-08T23:24:44.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.996+0000 7fb997053700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9900d13a0 0x7fb9900d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.996+0000 7fb997053700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb99006bd60 0x7fb9900cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.996+0000 7fb997053700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9900b1200 con 0x7fb9900cdc90 2024-05-08T23:24:44.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.996+0000 7fb997053700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9900d1cc0 con 0x7fb99006bd60 2024-05-08T23:24:44.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.996+0000 7fb997053700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9900cfcc0 con 0x7fb9900d13a0 2024-05-08T23:24:44.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.996+0000 7fb98ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb99006bd60 0x7fb9900cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.996+0000 7fb994def700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9900d13a0 0x7fb9900d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb9955f0700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb9900cdc90 0x7fb9900b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:48828/0 (socket says 172.21.15.5:48828) 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb99006bd60 0x7fb9900cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41808/0 (socket says 172.21.15.5:41808) 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb9955f0700 1 -- 172.21.15.5:0/2444339667 learned_addr learned my addr 172.21.15.5:0/2444339667 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98ffff700 1 -- 172.21.15.5:0/2444339667 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9900d13a0 msgr2=0x7fb9900d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98ffff700 1 --2- 172.21.15.5:0/2444339667 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9900d13a0 0x7fb9900d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98ffff700 1 -- 172.21.15.5:0/2444339667 >> v1:172.21.15.49:6789/0 conn(0x7fb9900cdc90 legacy=0x7fb9900b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98ffff700 1 -- 172.21.15.5:0/2444339667 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9900ce5c0 con 0x7fb99006bd60 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98f7fe700 1 -- 172.21.15.5:0/2444339667 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491117089 0 0) 0x7fb9900b1200 con 0x7fb9900cdc90 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb994def700 1 --2- 172.21.15.5:0/2444339667 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9900d13a0 0x7fb9900d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98ffff700 1 --2- 172.21.15.5:0/2444339667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99006bd60 0x7fb9900cf780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb980009840 tx=0x7fb980009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b178112fbd9be96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98f7fe700 1 -- 172.21.15.5:0/2444339667 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980011830 con 0x7fb99006bd60 2024-05-08T23:24:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98f7fe700 1 -- 172.21.15.5:0/2444339667 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb980011990 con 0x7fb99006bd60 2024-05-08T23:24:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.997+0000 7fb98f7fe700 1 -- 172.21.15.5:0/2444339667 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980011d20 con 0x7fb99006bd60 2024-05-08T23:24:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.998+0000 7fb997053700 1 -- 172.21.15.5:0/2444339667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99006bd60 msgr2=0x7fb9900cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.998+0000 7fb997053700 1 --2- 172.21.15.5:0/2444339667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99006bd60 0x7fb9900cf780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb980009840 tx=0x7fb980009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.998+0000 7fb997053700 1 -- 172.21.15.5:0/2444339667 shutdown_connections 2024-05-08T23:24:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.998+0000 7fb997053700 1 --2- 172.21.15.5:0/2444339667 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9900d13a0 0x7fb9900d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.998+0000 7fb997053700 1 --2- 172.21.15.5:0/2444339667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99006bd60 0x7fb9900cf780 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.998+0000 7fb997053700 1 -- 172.21.15.5:0/2444339667 >> 172.21.15.5:0/2444339667 conn(0x7fb99008afe0 msgr2=0x7fb99008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.998+0000 7fb997053700 1 -- 172.21.15.5:0/2444339667 shutdown_connections 2024-05-08T23:24:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.998+0000 7fb997053700 1 -- 172.21.15.5:0/2444339667 wait complete. 2024-05-08T23:24:45.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.999+0000 7fb997053700 1 Processor -- start 2024-05-08T23:24:45.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:44.999+0000 7fb997053700 1 -- start start 2024-05-08T23:24:45.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.000+0000 7fb997053700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb99006bd60 0x7fb9901f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.000+0000 7fb997053700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9900cdc90 0x7fb9900c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.000+0000 7fb997053700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9900d13a0 0x7fb9900ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.000+0000 7fb997053700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990157cb0 con 0x7fb9900d13a0 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.000+0000 7fb997053700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9900af3c0 con 0x7fb9900cdc90 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.000+0000 7fb997053700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9900aead0 con 0x7fb99006bd60 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9900d13a0 0x7fb9900ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9900d13a0 0x7fb9900ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49752/0 (socket says 172.21.15.5:49752) 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 -- 172.21.15.5:0/2847513246 learned_addr learned my addr 172.21.15.5:0/2847513246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb98ffff700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9900cdc90 0x7fb9900c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb994def700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb99006bd60 0x7fb9901f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 -- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb99006bd60 msgr2=0x7fb9901f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb99006bd60 0x7fb9901f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 -- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9900cdc90 msgr2=0x7fb9900c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9900cdc90 0x7fb9900c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 -- 172.21.15.5:0/2847513246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9780094a0 con 0x7fb9900d13a0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb994def700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb99006bd60 0x7fb9901f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb9955f0700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9900d13a0 0x7fb9900ca510 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb988002230 tx=0x7fb98800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb98d7fa700 1 -- 172.21.15.5:0/2847513246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98800ab70 con 0x7fb9900d13a0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb98d7fa700 1 -- 172.21.15.5:0/2847513246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98800acf0 con 0x7fb9900d13a0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9800094a0 con 0x7fb9900d13a0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.001+0000 7fb98d7fa700 1 -- 172.21.15.5:0/2847513246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988007310 con 0x7fb9900d13a0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.002+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9900af050 con 0x7fb9900d13a0 2024-05-08T23:24:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.003+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9900af530 con 0x7fb9900d13a0 2024-05-08T23:24:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.003+0000 7fb98d7fa700 1 -- 172.21.15.5:0/2847513246 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb988007470 con 0x7fb9900d13a0 2024-05-08T23:24:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.003+0000 7fb98d7fa700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97006ac90 0x7fb97006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.004+0000 7fb98d7fa700 1 -- 172.21.15.5:0/2847513246 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb988089140 con 0x7fb9900d13a0 2024-05-08T23:24:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.004+0000 7fb994def700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97006ac90 0x7fb97006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.004+0000 7fb994def700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97006ac90 0x7fb97006d150 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb978009d20 tx=0x7fb978008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.009+0000 7fb98d7fa700 1 -- 172.21.15.5:0/2847513246 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb988054380 con 0x7fb9900d13a0 2024-05-08T23:24:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.281+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb990074bc0 con 0x7fb9900d13a0 2024-05-08T23:24:45.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.287+0000 7fb98d7fa700 1 -- 172.21.15.5:0/2847513246 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v45) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb98800ae60 con 0x7fb9900d13a0 2024-05-08T23:24:45.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97006ac90 msgr2=0x7fb97006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97006ac90 0x7fb97006d150 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb978009d20 tx=0x7fb978008040 comp rx=0 tx=0).stop 2024-05-08T23:24:45.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9900d13a0 msgr2=0x7fb9900ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9900d13a0 0x7fb9900ca510 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb988002230 tx=0x7fb98800cf50 comp rx=0 tx=0).stop 2024-05-08T23:24:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 shutdown_connections 2024-05-08T23:24:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97006ac90 0x7fb97006d150 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9900d13a0 0x7fb9900ca510 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb99006bd60 0x7fb9901f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 --2- 172.21.15.5:0/2847513246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9900cdc90 0x7fb9900c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 >> 172.21.15.5:0/2847513246 conn(0x7fb99008afe0 msgr2=0x7fb99008b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 shutdown_connections 2024-05-08T23:24:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.314+0000 7fb997053700 1 -- 172.21.15.5:0/2847513246 wait complete. 2024-05-08T23:24:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 8 and data pool 9 2024-05-08T23:24:45.330 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.467+0000 7f582282e700 1 Processor -- start 2024-05-08T23:24:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.467+0000 7f582282e700 1 -- start start 2024-05-08T23:24:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f582282e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f581c0ce120 0x7f581c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f582282e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 0x7f581c0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f582282e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c06fbf0 con 0x7f581c15db00 2024-05-08T23:24:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f582282e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c06edb0 con 0x7f581c0ce120 2024-05-08T23:24:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f582282e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c06f090 con 0x7f581c0cb2a0 2024-05-08T23:24:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f581b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f581c0ce120 0x7f581c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f581b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f581c0ce120 0x7f581c06e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49768/0 (socket says 172.21.15.5:49768) 2024-05-08T23:24:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f581b7fe700 1 -- 172.21.15.5:0/2860107552 learned_addr learned my addr 172.21.15.5:0/2860107552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f5820dcb700 1 --2- 172.21.15.5:0/2860107552 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 0x7f581c0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f581b7fe700 1 -- 172.21.15.5:0/2860107552 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 msgr2=0x7f581c0cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f581b7fe700 1 --2- 172.21.15.5:0/2860107552 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 0x7f581c0cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f581b7fe700 1 -- 172.21.15.5:0/2860107552 >> v1:172.21.15.49:6789/0 conn(0x7f581c15db00 legacy=0x7f581c06dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f581b7fe700 1 -- 172.21.15.5:0/2860107552 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c06f370 con 0x7f581c0ce120 2024-05-08T23:24:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.468+0000 7f581b7fe700 1 --2- 172.21.15.5:0/2860107552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c06e390 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f581000bb90 tx=0x7f581000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b77ae4eaecc3442 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f581affd700 1 -- 172.21.15.5:0/2860107552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581000dd30 con 0x7f581c0ce120 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f581affd700 1 -- 172.21.15.5:0/2860107552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581000de90 con 0x7f581c0ce120 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f581affd700 1 -- 172.21.15.5:0/2860107552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810010690 con 0x7f581c0ce120 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f582282e700 1 -- 172.21.15.5:0/2860107552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 msgr2=0x7f581c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f582282e700 1 --2- 172.21.15.5:0/2860107552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c06e390 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f581000bb90 tx=0x7f581000bea0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f582282e700 1 -- 172.21.15.5:0/2860107552 shutdown_connections 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f582282e700 1 --2- 172.21.15.5:0/2860107552 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 0x7f581c0cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f582282e700 1 --2- 172.21.15.5:0/2860107552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c06e390 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f582282e700 1 -- 172.21.15.5:0/2860107552 >> 172.21.15.5:0/2860107552 conn(0x7f581c099a50 msgr2=0x7f581c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f582282e700 1 -- 172.21.15.5:0/2860107552 shutdown_connections 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.469+0000 7f582282e700 1 -- 172.21.15.5:0/2860107552 wait complete. 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.471+0000 7f582282e700 1 Processor -- start 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.471+0000 7f582282e700 1 -- start start 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f582282e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 0x7f581c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f582282e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f582282e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581c15db00 0x7f581c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f582282e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c3f30 con 0x7f581c0ce120 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f582282e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c4d90 con 0x7f581c15db00 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f582282e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c14ee50 con 0x7f581c0cb2a0 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c15bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49782/0 (socket says 172.21.15.5:49782) 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 -- 172.21.15.5:0/1782834709 learned_addr learned my addr 172.21.15.5:0/1782834709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581bfff700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 0x7f581c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f5820dcb700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581c15db00 0x7f581c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 -- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 msgr2=0x7f581c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 0x7f581c14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 -- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581c15db00 msgr2=0x7f581c1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581c15db00 0x7f581c1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 -- 172.21.15.5:0/1782834709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581000b7f0 con 0x7f581c0ce120 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.472+0000 7f581b7fe700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c15bb00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5810000c00 tx=0x7f5810004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.473+0000 7f5818ff9700 1 -- 172.21.15.5:0/1782834709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810003b70 con 0x7f581c0ce120 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.473+0000 7f5818ff9700 1 -- 172.21.15.5:0/1782834709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810003cd0 con 0x7f581c0ce120 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.473+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c15c040 con 0x7f581c0ce120 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.473+0000 7f5818ff9700 1 -- 172.21.15.5:0/1782834709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100106b0 con 0x7f581c0ce120 2024-05-08T23:24:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.473+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c0c4460 con 0x7f581c0ce120 2024-05-08T23:24:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.474+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c0c4f00 con 0x7f581c0ce120 2024-05-08T23:24:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.474+0000 7f5818ff9700 1 -- 172.21.15.5:0/1782834709 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5810019400 con 0x7f581c0ce120 2024-05-08T23:24:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.475+0000 7f5818ff9700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f580806ac90 0x7f580806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.475+0000 7f5818ff9700 1 -- 172.21.15.5:0/1782834709 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f581008a540 con 0x7f581c0ce120 2024-05-08T23:24:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.475+0000 7f581bfff700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f580806ac90 0x7f580806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.475+0000 7f581bfff700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f580806ac90 0x7f580806d150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f581c0ca920 tx=0x7f580c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.480+0000 7f5818ff9700 1 -- 172.21.15.5:0/1782834709 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f58100557b0 con 0x7f581c0ce120 2024-05-08T23:24:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.696+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f581c068000 con 0x7f581c0ce120 2024-05-08T23:24:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.697+0000 7f5818ff9700 1 -- 172.21.15.5:0/1782834709 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16014 (secure 0 0 0) 0x7f5810058dd0 con 0x7f581c0ce120 2024-05-08T23:24:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":59,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:45.283156+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:24:43.276676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T23:24:44.254746+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f580806ac90 msgr2=0x7f580806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f580806ac90 0x7f580806d150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f581c0ca920 tx=0x7f580c009040 comp rx=0 tx=0).stop 2024-05-08T23:24:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 msgr2=0x7f581c15bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c15bb00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5810000c00 tx=0x7f5810004750 comp rx=0 tx=0).stop 2024-05-08T23:24:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 shutdown_connections 2024-05-08T23:24:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f580806ac90 0x7f580806d150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581c15db00 0x7f581c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f581c0cb2a0 0x7f581c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 --2- 172.21.15.5:0/1782834709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581c0ce120 0x7f581c15bb00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 >> 172.21.15.5:0/1782834709 conn(0x7f581c099a50 msgr2=0x7f581c0ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 shutdown_connections 2024-05-08T23:24:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.700+0000 7f582282e700 1 -- 172.21.15.5:0/1782834709 wait complete. 2024-05-08T23:24:45.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.850+0000 7fe00cd5e700 1 Processor -- start 2024-05-08T23:24:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.850+0000 7fe00cd5e700 1 -- start start 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00cd5e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe00815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00cd5e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0080b3990 0x7fe00806ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00cd5e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe00806e300 0x7fe008161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00cd5e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00806e790 con 0x7fe0080b3990 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00cd5e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00815ede0 con 0x7fe00806e300 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00cd5e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080b4a80 con 0x7fe0080af030 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0080b3990 0x7fe00806ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0080b3990 0x7fe00806ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49786/0 (socket says 172.21.15.5:49786) 2024-05-08T23:24:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 -- 172.21.15.5:0/3984200401 learned_addr learned my addr 172.21.15.5:0/3984200401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe006d9d700 1 --2- 172.21.15.5:0/3984200401 >> v2:172.21.15.49:3300/0 conn(0x7fe00806e300 0x7fe008161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00659c700 1 --2- 172.21.15.5:0/3984200401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe00815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 -- 172.21.15.5:0/3984200401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 msgr2=0x7fe00815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 --2- 172.21.15.5:0/3984200401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe00815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 -- 172.21.15.5:0/3984200401 >> v2:172.21.15.49:3300/0 conn(0x7fe00806e300 msgr2=0x7fe008161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 --2- 172.21.15.5:0/3984200401 >> v2:172.21.15.49:3300/0 conn(0x7fe00806e300 0x7fe008161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 -- 172.21.15.5:0/3984200401 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00806d800 con 0x7fe0080b3990 2024-05-08T23:24:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe006d9d700 1 --2- 172.21.15.5:0/3984200401 >> v2:172.21.15.49:3300/0 conn(0x7fe00806e300 0x7fe008161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00659c700 1 --2- 172.21.15.5:0/3984200401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe00815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe005d9b700 1 --2- 172.21.15.5:0/3984200401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0080b3990 0x7fe00806ddc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe00806dc00 tx=0x7fdffc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b194ddb0027be838 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00559a700 1 -- 172.21.15.5:0/3984200401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00b870 con 0x7fe0080b3990 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.852+0000 7fe00559a700 1 -- 172.21.15.5:0/3984200401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc00b9d0 con 0x7fe0080b3990 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00559a700 1 -- 172.21.15.5:0/3984200401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00bd60 con 0x7fe0080b3990 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 -- 172.21.15.5:0/3984200401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0080b3990 msgr2=0x7fe00806ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/3984200401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0080b3990 0x7fe00806ddc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe00806dc00 tx=0x7fdffc009bd0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 -- 172.21.15.5:0/3984200401 shutdown_connections 2024-05-08T23:24:45.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/3984200401 >> v2:172.21.15.49:3300/0 conn(0x7fe00806e300 0x7fe008161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/3984200401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe00815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/3984200401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0080b3990 0x7fe00806ddc0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 -- 172.21.15.5:0/3984200401 >> 172.21.15.5:0/3984200401 conn(0x7fe0080cfce0 msgr2=0x7fe0080d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 -- 172.21.15.5:0/3984200401 shutdown_connections 2024-05-08T23:24:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.853+0000 7fe00cd5e700 1 -- 172.21.15.5:0/3984200401 wait complete. 2024-05-08T23:24:45.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.855+0000 7fe00cd5e700 1 Processor -- start 2024-05-08T23:24:45.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.855+0000 7fe00cd5e700 1 -- start start 2024-05-08T23:24:45.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.855+0000 7fe00cd5e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00806e300 0x7fe00814ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00cd5e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe0081e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00cd5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0080b3990 0x7fe00814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00cd5e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080cbf40 con 0x7fe00806e300 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00cd5e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00814f490 con 0x7fe0080b3990 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00cd5e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080cc190 con 0x7fe0080af030 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00806e300 0x7fe00814ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00806e300 0x7fe00814ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49796/0 (socket says 172.21.15.5:49796) 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 -- 172.21.15.5:0/1427570464 learned_addr learned my addr 172.21.15.5:0/1427570464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe006d9d700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0080b3990 0x7fe00814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe005d9b700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe0081e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 -- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 msgr2=0x7fe0081e68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe0081e68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 -- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0080b3990 msgr2=0x7fe00814e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0080b3990 0x7fe00814e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 -- 172.21.15.5:0/1427570464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc009550 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe006d9d700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0080b3990 0x7fe00814e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00659c700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00806e300 0x7fe00814ef50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fdff800e810 tx=0x7fdff800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fdff6ffd700 1 -- 172.21.15.5:0/1427570464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8004230 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fdff6ffd700 1 -- 172.21.15.5:0/1427570464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff80043b0 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0080cc470 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fdff6ffd700 1 -- 172.21.15.5:0/1427570464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff80038e0 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.856+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0080cca30 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.857+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00806f2a0 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.858+0000 7fdff6ffd700 1 -- 172.21.15.5:0/1427570464 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdff8003a40 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.858+0000 7fdff6ffd700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfe806ac90 0x7fdfe806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.859+0000 7fdff6ffd700 1 -- 172.21.15.5:0/1427570464 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdff8088d30 con 0x7fe00806e300 2024-05-08T23:24:45.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.859+0000 7fe005d9b700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfe806ac90 0x7fdfe806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.859+0000 7fe005d9b700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfe806ac90 0x7fdfe806d150 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdffc014040 tx=0x7fdffc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:45.863+0000 7fdff6ffd700 1 -- 172.21.15.5:0/1427570464 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdff8053f10 con 0x7fe00806e300 2024-05-08T23:24:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.084+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe008068000 con 0x7fe00806e300 2024-05-08T23:24:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.085+0000 7fdff6ffd700 1 -- 172.21.15.5:0/1427570464 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16014 (secure 0 0 0) 0x7fdff80094c0 con 0x7fe00806e300 2024-05-08T23:24:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:46.086 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":59,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:45.283156+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:24:43.276676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T23:24:44.254746+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfe806ac90 msgr2=0x7fdfe806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfe806ac90 0x7fdfe806d150 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdffc014040 tx=0x7fdffc008030 comp rx=0 tx=0).stop 2024-05-08T23:24:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00806e300 msgr2=0x7fe00814ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00806e300 0x7fe00814ef50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fdff800e810 tx=0x7fdff800eb20 comp rx=0 tx=0).stop 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 shutdown_connections 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfe806ac90 0x7fdfe806d150 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0080b3990 0x7fe00814e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00806e300 0x7fe00814ef50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 --2- 172.21.15.5:0/1427570464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0080af030 0x7fe0081e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 >> 172.21.15.5:0/1427570464 conn(0x7fe0080cfce0 msgr2=0x7fe0080ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 shutdown_connections 2024-05-08T23:24:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.088+0000 7fe00cd5e700 1 -- 172.21.15.5:0/1427570464 wait complete. 2024-05-08T23:24:46.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:24:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.241+0000 7f175e442700 1 Processor -- start 2024-05-08T23:24:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.241+0000 7f175e442700 1 -- start start 2024-05-08T23:24:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.242+0000 7f175e442700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f175815b4e0 0x7f17580b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.242+0000 7f175e442700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17580d1910 0x7f17580cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.242+0000 7f175e442700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17580cf560 con 0x7f175815b8c0 2024-05-08T23:24:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.242+0000 7f175e442700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17580cefe0 con 0x7f17580d1910 2024-05-08T23:24:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.242+0000 7f175e442700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17580ccff0 con 0x7f175815b4e0 2024-05-08T23:24:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.242+0000 7f175c9df700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17580d1910 0x7f17580cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.242+0000 7f175c9df700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17580d1910 0x7f17580cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49798/0 (socket says 172.21.15.5:49798) 2024-05-08T23:24:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.242+0000 7f175c9df700 1 -- 172.21.15.5:0/4107891748 learned_addr learned my addr 172.21.15.5:0/4107891748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f1757fff700 1 --2- 172.21.15.5:0/4107891748 >> v2:172.21.15.49:3300/0 conn(0x7f175815b4e0 0x7f17580b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f175c9df700 1 -- 172.21.15.5:0/4107891748 >> v2:172.21.15.49:3300/0 conn(0x7f175815b4e0 msgr2=0x7f17580b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f175c9df700 1 --2- 172.21.15.5:0/4107891748 >> v2:172.21.15.49:3300/0 conn(0x7f175815b4e0 0x7f17580b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f175c9df700 1 -- 172.21.15.5:0/4107891748 >> v1:172.21.15.49:6789/0 conn(0x7f175815b8c0 legacy=0x7f17580d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f175c9df700 1 -- 172.21.15.5:0/4107891748 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580cd2d0 con 0x7f17580d1910 2024-05-08T23:24:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f175c9df700 1 --2- 172.21.15.5:0/4107891748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580d1910 0x7f17580cc2c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f17580b0f90 tx=0x7f175000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987ce1227b50236e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f1756ffd700 1 -- 172.21.15.5:0/4107891748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000e070 con 0x7f17580d1910 2024-05-08T23:24:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f1756ffd700 1 -- 172.21.15.5:0/4107891748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175000b910 con 0x7f17580d1910 2024-05-08T23:24:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.243+0000 7f1756ffd700 1 -- 172.21.15.5:0/4107891748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000bcc0 con 0x7f17580d1910 2024-05-08T23:24:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.244+0000 7f175e442700 1 -- 172.21.15.5:0/4107891748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580d1910 msgr2=0x7f17580cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.244+0000 7f175e442700 1 --2- 172.21.15.5:0/4107891748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580d1910 0x7f17580cc2c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f17580b0f90 tx=0x7f175000b330 comp rx=0 tx=0).stop 2024-05-08T23:24:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.244+0000 7f175e442700 1 -- 172.21.15.5:0/4107891748 shutdown_connections 2024-05-08T23:24:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.244+0000 7f175e442700 1 --2- 172.21.15.5:0/4107891748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580d1910 0x7f17580cc2c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.244+0000 7f175e442700 1 --2- 172.21.15.5:0/4107891748 >> v2:172.21.15.49:3300/0 conn(0x7f175815b4e0 0x7f17580b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.244+0000 7f175e442700 1 -- 172.21.15.5:0/4107891748 >> 172.21.15.5:0/4107891748 conn(0x7f175808afe0 msgr2=0x7f175808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.244+0000 7f175e442700 1 -- 172.21.15.5:0/4107891748 shutdown_connections 2024-05-08T23:24:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.244+0000 7f175e442700 1 -- 172.21.15.5:0/4107891748 wait complete. 2024-05-08T23:24:46.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.245+0000 7f175e442700 1 Processor -- start 2024-05-08T23:24:46.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.245+0000 7f175e442700 1 -- start start 2024-05-08T23:24:46.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175e442700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580d1910 0x7f17580cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175e442700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f175815b4e0 0x7f17580c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175e442700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f175815b8c0 0x7f17581f0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175e442700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17581541c0 con 0x7f175815b8c0 2024-05-08T23:24:46.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175e442700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175815cb40 con 0x7f17580d1910 2024-05-08T23:24:46.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175e442700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175815ce20 con 0x7f175815b4e0 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175c9df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f175815b8c0 0x7f17581f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175c9df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f175815b8c0 0x7f17581f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49814/0 (socket says 172.21.15.5:49814) 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f175c9df700 1 -- 172.21.15.5:0/978776687 learned_addr learned my addr 172.21.15.5:0/978776687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f1757fff700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580d1910 0x7f17580cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.246+0000 7f17577fe700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f175815b4e0 0x7f17580c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f175c9df700 1 -- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f175815b4e0 msgr2=0x7f17580c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f175c9df700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f175815b4e0 0x7f17580c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f175c9df700 1 -- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580d1910 msgr2=0x7f17580cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f175c9df700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580d1910 0x7f17580cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f175c9df700 1 -- 172.21.15.5:0/978776687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1750009b30 con 0x7f175815b8c0 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f175c9df700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f175815b8c0 0x7f17581f0360 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f17580cef90 tx=0x7f1750004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f1754ff9700 1 -- 172.21.15.5:0/978776687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750004880 con 0x7f175815b8c0 2024-05-08T23:24:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f1754ff9700 1 -- 172.21.15.5:0/978776687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17500099b0 con 0x7f175815b8c0 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175815d100 con 0x7f175815b8c0 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f1754ff9700 1 -- 172.21.15.5:0/978776687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175001c680 con 0x7f175815b8c0 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.247+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17580cb4e0 con 0x7f175815b8c0 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.248+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175815c9a0 con 0x7f175815b8c0 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.249+0000 7f1754ff9700 1 -- 172.21.15.5:0/978776687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f175001c7e0 con 0x7f175815b8c0 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.249+0000 7f1754ff9700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f173806ac90 0x7f173806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.249+0000 7f1754ff9700 1 -- 172.21.15.5:0/978776687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f175008a670 con 0x7f175815b8c0 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.249+0000 7f1757fff700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f173806ac90 0x7f173806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.250+0000 7f1757fff700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f173806ac90 0x7f173806d150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1748009be0 tx=0x7f1748009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:46.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.254+0000 7f1754ff9700 1 -- 172.21.15.5:0/978776687 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f175815c9a0 con 0x7f175815b8c0 2024-05-08T23:24:46.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:46.519+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f17580cb4e0 con 0x7f175815b8c0 2024-05-08T23:24:47.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.296+0000 7f1754ff9700 1 -- 172.21.15.5:0/978776687 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7f17580cb4e0 con 0x7f175815b8c0 2024-05-08T23:24:47.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f173806ac90 msgr2=0x7f173806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f173806ac90 0x7f173806d150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1748009be0 tx=0x7f1748009210 comp rx=0 tx=0).stop 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f175815b8c0 msgr2=0x7f17581f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f175815b8c0 0x7f17581f0360 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f17580cef90 tx=0x7f1750004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 shutdown_connections 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f173806ac90 0x7f173806d150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f175815b8c0 0x7f17581f0360 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17580d1910 0x7f17580cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 --2- 172.21.15.5:0/978776687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f175815b4e0 0x7f17580c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 >> 172.21.15.5:0/978776687 conn(0x7f175808afe0 msgr2=0x7f17580ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 shutdown_connections 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.299+0000 7f175e442700 1 -- 172.21.15.5:0/978776687 wait complete. 2024-05-08T23:24:47.315 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:24:47.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:24:47.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.458+0000 7fdb54ed5700 1 Processor -- start 2024-05-08T23:24:47.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.458+0000 7fdb54ed5700 1 -- start start 2024-05-08T23:24:47.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.459+0000 7fdb54ed5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb500d13a0 0x7fdb500d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb54ed5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb500b1200 con 0x7fdb500cdc90 2024-05-08T23:24:47.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb54ed5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb500d1cc0 con 0x7fdb5006bd60 2024-05-08T23:24:47.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb54ed5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb500ce5c0 con 0x7fdb500d13a0 2024-05-08T23:24:47.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4ed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdb500cdc90 0x7fdb500b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36100/0 (socket says 172.21.15.5:36100) 2024-05-08T23:24:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4ed9d700 1 -- 172.21.15.5:0/167636148 learned_addr learned my addr 172.21.15.5:0/167636148 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4e59c700 1 --2- 172.21.15.5:0/167636148 >> v2:172.21.15.49:3300/0 conn(0x7fdb500d13a0 0x7fdb500d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4d59a700 1 -- 172.21.15.5:0/167636148 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901216103 0 0) 0x7fdb500b1200 con 0x7fdb500cdc90 2024-05-08T23:24:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4d59a700 1 -- 172.21.15.5:0/167636148 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb380035b0 con 0x7fdb500cdc90 2024-05-08T23:24:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4e59c700 1 -- 172.21.15.5:0/167636148 >> v1:172.21.15.49:6789/0 conn(0x7fdb5006bd60 legacy=0x7fdb500cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4e59c700 1 -- 172.21.15.5:0/167636148 >> v1:172.21.15.5:6789/0 conn(0x7fdb500cdc90 legacy=0x7fdb500b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4e59c700 1 -- 172.21.15.5:0/167636148 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb500ce8a0 con 0x7fdb500d13a0 2024-05-08T23:24:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.460+0000 7fdb4e59c700 1 --2- 172.21.15.5:0/167636148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500d13a0 0x7fdb500d1780 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009840 tx=0x7fdb40009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67548e5aa34b27e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:47.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb4d59a700 1 -- 172.21.15.5:0/167636148 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40011830 con 0x7fdb500d13a0 2024-05-08T23:24:47.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb4d59a700 1 -- 172.21.15.5:0/167636148 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb40011990 con 0x7fdb500d13a0 2024-05-08T23:24:47.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb4d59a700 1 -- 172.21.15.5:0/167636148 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40011d20 con 0x7fdb500d13a0 2024-05-08T23:24:47.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb54ed5700 1 -- 172.21.15.5:0/167636148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500d13a0 msgr2=0x7fdb500d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb54ed5700 1 --2- 172.21.15.5:0/167636148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500d13a0 0x7fdb500d1780 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009840 tx=0x7fdb40009b50 comp rx=0 tx=0).stop 2024-05-08T23:24:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb54ed5700 1 -- 172.21.15.5:0/167636148 shutdown_connections 2024-05-08T23:24:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb54ed5700 1 --2- 172.21.15.5:0/167636148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500d13a0 0x7fdb500d1780 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb54ed5700 1 -- 172.21.15.5:0/167636148 >> 172.21.15.5:0/167636148 conn(0x7fdb5008afe0 msgr2=0x7fdb5008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb54ed5700 1 -- 172.21.15.5:0/167636148 shutdown_connections 2024-05-08T23:24:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.461+0000 7fdb54ed5700 1 -- 172.21.15.5:0/167636148 wait complete. 2024-05-08T23:24:47.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.463+0000 7fdb54ed5700 1 Processor -- start 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.463+0000 7fdb54ed5700 1 -- start start 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb54ed5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb5006bd60 0x7fdb5015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb54ed5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500cdc90 0x7fdb5009bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb54ed5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb500d13a0 0x7fdb500ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb54ed5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5009efc0 con 0x7fdb5006bd60 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb54ed5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5009e0d0 con 0x7fdb500cdc90 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb54ed5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5009e3b0 con 0x7fdb500d13a0 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb5006bd60 0x7fdb5015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb5006bd60 0x7fdb5015be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57790/0 (socket says 172.21.15.5:57790) 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 -- 172.21.15.5:0/235930356 learned_addr learned my addr 172.21.15.5:0/235930356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4dd9b700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500cdc90 0x7fdb5009bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4ed9d700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb500d13a0 0x7fdb500ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 -- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb500d13a0 msgr2=0x7fdb500ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb500d13a0 0x7fdb500ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 -- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500cdc90 msgr2=0x7fdb5009bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500cdc90 0x7fdb5009bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 -- 172.21.15.5:0/235930356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400094a0 con 0x7fdb5006bd60 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4ed9d700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb500d13a0 0x7fdb500ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.464+0000 7fdb4e59c700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb5006bd60 0x7fdb5015be10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009fd0 tx=0x7fdb40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.465+0000 7fdb3effd700 1 -- 172.21.15.5:0/235930356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb400038a0 con 0x7fdb5006bd60 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.465+0000 7fdb3effd700 1 -- 172.21.15.5:0/235930356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4001f070 con 0x7fdb5006bd60 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.465+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5009e690 con 0x7fdb5006bd60 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.465+0000 7fdb3effd700 1 -- 172.21.15.5:0/235930356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40003c40 con 0x7fdb5006bd60 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.465+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5009ec50 con 0x7fdb5006bd60 2024-05-08T23:24:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.466+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb50074bc0 con 0x7fdb5006bd60 2024-05-08T23:24:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.466+0000 7fdb3effd700 1 -- 172.21.15.5:0/235930356 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdb40026480 con 0x7fdb5006bd60 2024-05-08T23:24:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.467+0000 7fdb3effd700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb3006ac90 0x7fdb3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.467+0000 7fdb3effd700 1 -- 172.21.15.5:0/235930356 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb40021070 con 0x7fdb5006bd60 2024-05-08T23:24:47.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.467+0000 7fdb4dd9b700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb3006ac90 0x7fdb3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.467+0000 7fdb4dd9b700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb3006ac90 0x7fdb3006d150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdb500af4a0 tx=0x7fdb44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:47.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.471+0000 7fdb3effd700 1 -- 172.21.15.5:0/235930356 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdb40055f20 con 0x7fdb5006bd60 2024-05-08T23:24:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:47.740+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdb50068400 con 0x7fdb5006bd60 2024-05-08T23:24:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.300+0000 7fdb3effd700 1 -- 172.21.15.5:0/235930356 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdb40015020 con 0x7fdb5006bd60 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb3006ac90 msgr2=0x7fdb3006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb3006ac90 0x7fdb3006d150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdb500af4a0 tx=0x7fdb44009040 comp rx=0 tx=0).stop 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb5006bd60 msgr2=0x7fdb5015be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb5006bd60 0x7fdb5015be10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009fd0 tx=0x7fdb40004750 comp rx=0 tx=0).stop 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 shutdown_connections 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb3006ac90 0x7fdb3006d150 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb500d13a0 0x7fdb500ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb5006bd60 0x7fdb5015be10 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.303+0000 7fdb54ed5700 1 --2- 172.21.15.5:0/235930356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb500cdc90 0x7fdb5009bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.304+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 >> 172.21.15.5:0/235930356 conn(0x7fdb5008afe0 msgr2=0x7fdb500cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.304+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 shutdown_connections 2024-05-08T23:24:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.304+0000 7fdb54ed5700 1 -- 172.21.15.5:0/235930356 wait complete. 2024-05-08T23:24:48.319 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:24:48.320 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.460+0000 7ff83c92c700 1 Processor -- start 2024-05-08T23:24:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.460+0000 7ff83c92c700 1 -- start start 2024-05-08T23:24:48.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.461+0000 7ff83c92c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8380d3f40 0x7ff838161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.461+0000 7ff83c92c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 0x7ff8380cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.461+0000 7ff83c92c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff838162450 con 0x7ff83815d7a0 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.461+0000 7ff83c92c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8380cee00 con 0x7ff8380d3f40 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.461+0000 7ff83c92c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380b27e0 con 0x7ff83815db80 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff835d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8380d3f40 0x7ff838161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff835d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8380d3f40 0x7ff838161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57796/0 (socket says 172.21.15.5:57796) 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff835d9b700 1 -- 172.21.15.5:0/504218736 learned_addr learned my addr 172.21.15.5:0/504218736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff836d9d700 1 --2- 172.21.15.5:0/504218736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 0x7ff8380cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff835d9b700 1 -- 172.21.15.5:0/504218736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 msgr2=0x7ff8380cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff835d9b700 1 --2- 172.21.15.5:0/504218736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 0x7ff8380cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff835d9b700 1 -- 172.21.15.5:0/504218736 >> v1:172.21.15.49:6789/0 conn(0x7ff83815d7a0 legacy=0x7ff8380b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff835d9b700 1 -- 172.21.15.5:0/504218736 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380d4400 con 0x7ff8380d3f40 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff83559a700 1 -- 172.21.15.5:0/504218736 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101847346 0 0) 0x7ff828001110 con 0x7ff83815d7a0 2024-05-08T23:24:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff835d9b700 1 --2- 172.21.15.5:0/504218736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8380d3f40 0x7ff838161f10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff82c006fd0 tx=0x7ff82c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c689c2d5e2a2519 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff83559a700 1 -- 172.21.15.5:0/504218736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c005e30 con 0x7ff8380d3f40 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff83559a700 1 -- 172.21.15.5:0/504218736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c004030 con 0x7ff8380d3f40 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.462+0000 7ff83559a700 1 -- 172.21.15.5:0/504218736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0043e0 con 0x7ff8380d3f40 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.463+0000 7ff83c92c700 1 -- 172.21.15.5:0/504218736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8380d3f40 msgr2=0x7ff838161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.463+0000 7ff83c92c700 1 --2- 172.21.15.5:0/504218736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8380d3f40 0x7ff838161f10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff82c006fd0 tx=0x7ff82c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.463+0000 7ff83c92c700 1 -- 172.21.15.5:0/504218736 shutdown_connections 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.463+0000 7ff83c92c700 1 --2- 172.21.15.5:0/504218736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 0x7ff8380cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.463+0000 7ff83c92c700 1 --2- 172.21.15.5:0/504218736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8380d3f40 0x7ff838161f10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.463+0000 7ff83c92c700 1 -- 172.21.15.5:0/504218736 >> 172.21.15.5:0/504218736 conn(0x7ff83808afe0 msgr2=0x7ff8380d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.463+0000 7ff83c92c700 1 -- 172.21.15.5:0/504218736 shutdown_connections 2024-05-08T23:24:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.463+0000 7ff83c92c700 1 -- 172.21.15.5:0/504218736 wait complete. 2024-05-08T23:24:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.465+0000 7ff83c92c700 1 Processor -- start 2024-05-08T23:24:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.465+0000 7ff83c92c700 1 -- start start 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.465+0000 7ff83c92c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8380d3f40 0x7ff83814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.465+0000 7ff83c92c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83815d7a0 0x7ff8380c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.465+0000 7ff83c92c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 0x7ff8380c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.465+0000 7ff83c92c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83814d7f0 con 0x7ff83815d7a0 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.465+0000 7ff83c92c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83814d340 con 0x7ff8380d3f40 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.465+0000 7ff83c92c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83814da00 con 0x7ff83815db80 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83815d7a0 0x7ff8380c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83815d7a0 0x7ff8380c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57802/0 (socket says 172.21.15.5:57802) 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 -- 172.21.15.5:0/3589743330 learned_addr learned my addr 172.21.15.5:0/3589743330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff836d9d700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 0x7ff8380c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff83659c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8380d3f40 0x7ff83814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 -- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 msgr2=0x7ff8380c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 0x7ff8380c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 -- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8380d3f40 msgr2=0x7ff83814ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8380d3f40 0x7ff83814ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 -- 172.21.15.5:0/3589743330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c009500 con 0x7ff83815d7a0 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff835d9b700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83815d7a0 0x7ff8380c96c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff82c004000 tx=0x7ff82c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff826ffd700 1 -- 172.21.15.5:0/3589743330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c003c30 con 0x7ff83815d7a0 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff826ffd700 1 -- 172.21.15.5:0/3589743330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c003d90 con 0x7ff83815d7a0 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff826ffd700 1 -- 172.21.15.5:0/3589743330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c01c640 con 0x7ff83815d7a0 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83814dce0 con 0x7ff83815d7a0 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.466+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83814e2a0 con 0x7ff83815d7a0 2024-05-08T23:24:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.467+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff838068000 con 0x7ff83815d7a0 2024-05-08T23:24:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.468+0000 7ff826ffd700 1 -- 172.21.15.5:0/3589743330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff82c01c7a0 con 0x7ff83815d7a0 2024-05-08T23:24:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.468+0000 7ff826ffd700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff81806ac90 0x7ff81806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.469+0000 7ff826ffd700 1 -- 172.21.15.5:0/3589743330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff82c08a240 con 0x7ff83815d7a0 2024-05-08T23:24:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.469+0000 7ff83659c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff81806ac90 0x7ff81806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.469+0000 7ff83659c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff81806ac90 0x7ff81806d150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff828000f60 tx=0x7ff828009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:48.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.473+0000 7ff826ffd700 1 -- 172.21.15.5:0/3589743330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff82c0554b0 con 0x7ff83815d7a0 2024-05-08T23:24:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.743+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff838073f10 con 0x7ff83815d7a0 2024-05-08T23:24:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.743+0000 7ff826ffd700 1 -- 172.21.15.5:0/3589743330 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4728 (secure 0 0 0) 0x7ff82c058ad0 con 0x7ff83815d7a0 2024-05-08T23:24:48.743 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:48.744 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6516,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:24:48.295616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6525},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:24:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff81806ac90 msgr2=0x7ff81806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff81806ac90 0x7ff81806d150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff828000f60 tx=0x7ff828009040 comp rx=0 tx=0).stop 2024-05-08T23:24:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83815d7a0 msgr2=0x7ff8380c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83815d7a0 0x7ff8380c96c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff82c004000 tx=0x7ff82c004750 comp rx=0 tx=0).stop 2024-05-08T23:24:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 shutdown_connections 2024-05-08T23:24:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff81806ac90 0x7ff81806d150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff83815db80 0x7ff8380c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8380d3f40 0x7ff83814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 --2- 172.21.15.5:0/3589743330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83815d7a0 0x7ff8380c96c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.745+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 >> 172.21.15.5:0/3589743330 conn(0x7ff83808afe0 msgr2=0x7ff83815ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.746+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 shutdown_connections 2024-05-08T23:24:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.746+0000 7ff83c92c700 1 -- 172.21.15.5:0/3589743330 wait complete. 2024-05-08T23:24:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 49 2024-05-08T23:24:48.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.896+0000 7f686915a700 1 Processor -- start 2024-05-08T23:24:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.896+0000 7f686915a700 1 -- start start 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f686915a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68640ce120 0x7f686406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f686915a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f686915a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686406fbf0 con 0x7f686415db00 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f686915a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686406edb0 con 0x7f68640ce120 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f686915a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686406f090 con 0x7f68640cb2a0 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f6862d9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f686415db00 0x7f686406dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36112/0 (socket says 172.21.15.5:36112) 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f6862d9d700 1 -- 172.21.15.5:0/2299692073 learned_addr learned my addr 172.21.15.5:0/2299692073 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f686259c700 1 --2- 172.21.15.5:0/2299692073 >> v2:172.21.15.49:3300/0 conn(0x7f68640ce120 0x7f686406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f686359e700 1 --2- 172.21.15.5:0/2299692073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.897+0000 7f6861d9b700 1 -- 172.21.15.5:0/2299692073 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192174319 0 0) 0x7f686406fbf0 con 0x7f686415db00 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f6861d9b700 1 -- 172.21.15.5:0/2299692073 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f684c0035b0 con 0x7f686415db00 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f686359e700 1 -- 172.21.15.5:0/2299692073 >> v2:172.21.15.49:3300/0 conn(0x7f68640ce120 msgr2=0x7f686406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f686359e700 1 --2- 172.21.15.5:0/2299692073 >> v2:172.21.15.49:3300/0 conn(0x7f68640ce120 0x7f686406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f686359e700 1 -- 172.21.15.5:0/2299692073 >> v1:172.21.15.5:6789/0 conn(0x7f686415db00 legacy=0x7f686406dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f686359e700 1 -- 172.21.15.5:0/2299692073 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686406f370 con 0x7f68640cb2a0 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f686359e700 1 --2- 172.21.15.5:0/2299692073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640cb730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f685c009d90 tx=0x7f685c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ecc037bacc28b9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f6861d9b700 1 -- 172.21.15.5:0/2299692073 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c00e070 con 0x7f68640cb2a0 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f6861d9b700 1 -- 172.21.15.5:0/2299692073 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685c00bb20 con 0x7f68640cb2a0 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.898+0000 7f6861d9b700 1 -- 172.21.15.5:0/2299692073 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c012430 con 0x7f68640cb2a0 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.899+0000 7f686915a700 1 -- 172.21.15.5:0/2299692073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 msgr2=0x7f68640cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.899+0000 7f686915a700 1 --2- 172.21.15.5:0/2299692073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640cb730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f685c009d90 tx=0x7f685c00b330 comp rx=0 tx=0).stop 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.899+0000 7f686915a700 1 -- 172.21.15.5:0/2299692073 shutdown_connections 2024-05-08T23:24:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.899+0000 7f686915a700 1 --2- 172.21.15.5:0/2299692073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640cb730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.899+0000 7f686915a700 1 --2- 172.21.15.5:0/2299692073 >> v2:172.21.15.49:3300/0 conn(0x7f68640ce120 0x7f686406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.899+0000 7f686915a700 1 -- 172.21.15.5:0/2299692073 >> 172.21.15.5:0/2299692073 conn(0x7f6864099a50 msgr2=0x7f68640d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.899+0000 7f686915a700 1 -- 172.21.15.5:0/2299692073 shutdown_connections 2024-05-08T23:24:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.899+0000 7f686915a700 1 -- 172.21.15.5:0/2299692073 wait complete. 2024-05-08T23:24:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.900+0000 7f686915a700 1 Processor -- start 2024-05-08T23:24:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686915a700 1 -- start start 2024-05-08T23:24:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686915a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686915a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68640ce120 0x7f68640b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686915a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f686415db00 0x7f68640c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686915a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68640b0380 con 0x7f68640ce120 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686915a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68640c5560 con 0x7f686415db00 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686915a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68640c5840 con 0x7f68640cb2a0 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68640ce120 0x7f68640b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68640ce120 0x7f68640b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57806/0 (socket says 172.21.15.5:57806) 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.901+0000 7f686259c700 1 -- 172.21.15.5:0/2217433380 learned_addr learned my addr 172.21.15.5:0/2217433380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686359e700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f686415db00 0x7f68640c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f6862d9d700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686259c700 1 -- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 msgr2=0x7f68640afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686259c700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686259c700 1 -- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f686415db00 msgr2=0x7f68640c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686259c700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f686415db00 0x7f68640c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686259c700 1 -- 172.21.15.5:0/2217433380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c009a70 con 0x7f68640ce120 2024-05-08T23:24:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686259c700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68640ce120 0x7f68640b11b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f685800e9b0 tx=0x7f685800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f684b7fe700 1 -- 172.21.15.5:0/2217433380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858009710 con 0x7f68640ce120 2024-05-08T23:24:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f684b7fe700 1 -- 172.21.15.5:0/2217433380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685800ee50 con 0x7f68640ce120 2024-05-08T23:24:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68640c5b20 con 0x7f68640ce120 2024-05-08T23:24:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f684b7fe700 1 -- 172.21.15.5:0/2217433380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800ca50 con 0x7f68640ce120 2024-05-08T23:24:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.902+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68640c60e0 con 0x7f68640ce120 2024-05-08T23:24:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.903+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6864068000 con 0x7f68640ce120 2024-05-08T23:24:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.904+0000 7f684b7fe700 1 -- 172.21.15.5:0/2217433380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f685800cbb0 con 0x7f68640ce120 2024-05-08T23:24:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.904+0000 7f684b7fe700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f685006ac90 0x7f685006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.904+0000 7f684b7fe700 1 -- 172.21.15.5:0/2217433380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68580892d0 con 0x7f68640ce120 2024-05-08T23:24:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.905+0000 7f6862d9d700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f685006ac90 0x7f685006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.905+0000 7f6862d9d700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f685006ac90 0x7f685006d150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6854007fd0 tx=0x7f6854007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:48.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:48.909+0000 7f684b7fe700 1 -- 172.21.15.5:0/2217433380 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f68580544c0 con 0x7f68640ce120 2024-05-08T23:24:49.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.130+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68640cda40 con 0x7f68640ce120 2024-05-08T23:24:49.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.131+0000 7f684b7fe700 1 -- 172.21.15.5:0/2217433380 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16001 (secure 0 0 0) 0x7f685800ce60 con 0x7f68640ce120 2024-05-08T23:24:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:46.285230+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:24:43.276676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T23:24:44.254746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f685006ac90 msgr2=0x7f685006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f685006ac90 0x7f685006d150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6854007fd0 tx=0x7f6854007dc0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68640ce120 msgr2=0x7f68640b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68640ce120 0x7f68640b11b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f685800e9b0 tx=0x7f685800ecc0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 shutdown_connections 2024-05-08T23:24:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f685006ac90 0x7f685006d150 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f686415db00 0x7f68640c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f68640cb2a0 0x7f68640afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 --2- 172.21.15.5:0/2217433380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68640ce120 0x7f68640b11b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 >> 172.21.15.5:0/2217433380 conn(0x7f6864099a50 msgr2=0x7f68640cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 shutdown_connections 2024-05-08T23:24:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.134+0000 7f686915a700 1 -- 172.21.15.5:0/2217433380 wait complete. 2024-05-08T23:24:49.148 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:24:49.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:24:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.285+0000 7f29ab870700 1 Processor -- start 2024-05-08T23:24:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.285+0000 7f29ab870700 1 -- start start 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29ab870700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f29a415f230 0x7f29a40cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29ab870700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f29a40cca70 0x7f29a406ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29ab870700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a40cd3b0 con 0x7f29a40aef20 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29ab870700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a4141df0 con 0x7f29a415f230 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29ab870700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a40cce90 con 0x7f29a40cca70 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29a8e0b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f29a415f230 0x7f29a40cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29a960c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f29a40aef20 0x7f29a415c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36122/0 (socket says 172.21.15.5:36122) 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29a960c700 1 -- 172.21.15.5:0/373430924 learned_addr learned my addr 172.21.15.5:0/373430924 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29a9e0d700 1 --2- 172.21.15.5:0/373430924 >> v2:172.21.15.49:3300/0 conn(0x7f29a40cca70 0x7f29a406ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29a8e0b700 1 -- 172.21.15.5:0/373430924 >> v2:172.21.15.49:3300/0 conn(0x7f29a40cca70 msgr2=0x7f29a406ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29a8e0b700 1 --2- 172.21.15.5:0/373430924 >> v2:172.21.15.49:3300/0 conn(0x7f29a40cca70 0x7f29a406ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29a8e0b700 1 -- 172.21.15.5:0/373430924 >> v1:172.21.15.5:6789/0 conn(0x7f29a40aef20 legacy=0x7f29a415c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f299bfff700 1 -- 172.21.15.5:0/373430924 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477601385 0 0) 0x7f29a40cd3b0 con 0x7f29a40aef20 2024-05-08T23:24:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.286+0000 7f29a8e0b700 1 -- 172.21.15.5:0/373430924 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40af300 con 0x7f29a415f230 2024-05-08T23:24:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.287+0000 7f29a8e0b700 1 --2- 172.21.15.5:0/373430924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a415f230 0x7f29a40cc530 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f29a415cd80 tx=0x7f2994009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6f26168c9fdd0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.287+0000 7f299bfff700 1 -- 172.21.15.5:0/373430924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994011830 con 0x7f29a415f230 2024-05-08T23:24:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.287+0000 7f299bfff700 1 -- 172.21.15.5:0/373430924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2994011990 con 0x7f29a415f230 2024-05-08T23:24:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.287+0000 7f299bfff700 1 -- 172.21.15.5:0/373430924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994011d20 con 0x7f29a415f230 2024-05-08T23:24:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.287+0000 7f29ab870700 1 -- 172.21.15.5:0/373430924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a415f230 msgr2=0x7f29a40cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.287+0000 7f29ab870700 1 --2- 172.21.15.5:0/373430924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a415f230 0x7f29a40cc530 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f29a415cd80 tx=0x7f2994009b20 comp rx=0 tx=0).stop 2024-05-08T23:24:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.288+0000 7f29ab870700 1 -- 172.21.15.5:0/373430924 shutdown_connections 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.288+0000 7f29ab870700 1 --2- 172.21.15.5:0/373430924 >> v2:172.21.15.49:3300/0 conn(0x7f29a40cca70 0x7f29a406ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.288+0000 7f29ab870700 1 --2- 172.21.15.5:0/373430924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a415f230 0x7f29a40cc530 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.288+0000 7f29ab870700 1 -- 172.21.15.5:0/373430924 >> 172.21.15.5:0/373430924 conn(0x7f29a40cfce0 msgr2=0x7f29a40d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.288+0000 7f29ab870700 1 -- 172.21.15.5:0/373430924 shutdown_connections 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.288+0000 7f29ab870700 1 -- 172.21.15.5:0/373430924 wait complete. 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.289+0000 7f29ab870700 1 Processor -- start 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.289+0000 7f29ab870700 1 -- start start 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29ab870700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a40aef20 0x7f29a4136360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29ab870700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a40cca70 0x7f29a4147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29ab870700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f29a415f230 0x7f29a413f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29ab870700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a4148330 con 0x7f29a40cca70 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29ab870700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a413eec0 con 0x7f29a40aef20 2024-05-08T23:24:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29ab870700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a4147d70 con 0x7f29a415f230 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a8e0b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a40cca70 0x7f29a4147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a8e0b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a40cca70 0x7f29a4147390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57820/0 (socket says 172.21.15.5:57820) 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a8e0b700 1 -- 172.21.15.5:0/194466982 learned_addr learned my addr 172.21.15.5:0/194466982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a960c700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a40aef20 0x7f29a4136360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a9e0d700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f29a415f230 0x7f29a413f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a8e0b700 1 -- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f29a415f230 msgr2=0x7f29a413f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a8e0b700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f29a415f230 0x7f29a413f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a8e0b700 1 -- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a40aef20 msgr2=0x7f29a4136360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a8e0b700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a40aef20 0x7f29a4136360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.290+0000 7f29a8e0b700 1 -- 172.21.15.5:0/194466982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29940094a0 con 0x7f29a40cca70 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.291+0000 7f29a8e0b700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a40cca70 0x7f29a4147390 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f29940090b0 tx=0x7f2994004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.291+0000 7f2999ffb700 1 -- 172.21.15.5:0/194466982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994004590 con 0x7f29a40cca70 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.291+0000 7f2999ffb700 1 -- 172.21.15.5:0/194466982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29940036a0 con 0x7f29a40cca70 2024-05-08T23:24:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.291+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a4148050 con 0x7f29a40cca70 2024-05-08T23:24:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.291+0000 7f2999ffb700 1 -- 172.21.15.5:0/194466982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994003a10 con 0x7f29a40cca70 2024-05-08T23:24:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.291+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a41368a0 con 0x7f29a40cca70 2024-05-08T23:24:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.292+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a413e9b0 con 0x7f29a40cca70 2024-05-08T23:24:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.293+0000 7f2999ffb700 1 -- 172.21.15.5:0/194466982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2994003b70 con 0x7f29a40cca70 2024-05-08T23:24:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.293+0000 7f2999ffb700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f298806ac90 0x7f298806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.293+0000 7f2999ffb700 1 -- 172.21.15.5:0/194466982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f299408a6f0 con 0x7f29a40cca70 2024-05-08T23:24:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.293+0000 7f29a960c700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f298806ac90 0x7f298806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.294+0000 7f29a960c700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f298806ac90 0x7f298806d150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29a0000f60 tx=0x7f29a0007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.298+0000 7f2999ffb700 1 -- 172.21.15.5:0/194466982 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f29a413e9b0 con 0x7f29a40cca70 2024-05-08T23:24:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.511+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29a415fcb0 con 0x7f298806ac90 2024-05-08T23:24:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.520+0000 7f2999ffb700 1 -- 172.21.15.5:0/194466982 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289529 (secure 0 0 0) 0x7f29a0001290 con 0x7f298806ac90 2024-05-08T23:24:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f298806ac90 msgr2=0x7f298806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f298806ac90 0x7f298806d150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29a0000f60 tx=0x7f29a0007e70 comp rx=0 tx=0).stop 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a40cca70 msgr2=0x7f29a4147390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a40cca70 0x7f29a4147390 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f29940090b0 tx=0x7f2994004750 comp rx=0 tx=0).stop 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 shutdown_connections 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f298806ac90 0x7f298806d150 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f29a415f230 0x7f29a413f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29a40aef20 0x7f29a4136360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 --2- 172.21.15.5:0/194466982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29a40cca70 0x7f29a4147390 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 >> 172.21.15.5:0/194466982 conn(0x7f29a40cfce0 msgr2=0x7f29a413fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 shutdown_connections 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.524+0000 7f29ab870700 1 -- 172.21.15.5:0/194466982 wait complete. 2024-05-08T23:24:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:24:49.538 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-05-08T23:24:49.348024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223040,"kb_used_data":6820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751680,"statfs":{"total":773094113280,"available":772865720320,"internally_reserved":0,"allocated":6983680,"data_stored":3600712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.533517"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292222+0000","last_change":"2024-05-08T23:24:45.311564+0000","last_active":"2024-05-08T23:24:46.292222+0000","last_peered":"2024-05-08T23:24:46.292222+0000","last_clean":"2024-05-08T23:24:46.292222+0000","last_became_active":"2024-05-08T23:24:45.311261+0000","last_became_peered":"2024-05-08T23:24:45.311261+0000","last_unstale":"2024-05-08T23:24:46.292222+0000","last_undegraded":"2024-05-08T23:24:46.292222+0000","last_fullsized":"2024-05-08T23:24:46.292222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:29.592077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292349+0000","last_change":"2024-05-08T23:24:45.317600+0000","last_active":"2024-05-08T23:24:46.292349+0000","last_peered":"2024-05-08T23:24:46.292349+0000","last_clean":"2024-05-08T23:24:46.292349+0000","last_became_active":"2024-05-08T23:24:45.317327+0000","last_became_peered":"2024-05-08T23:24:45.317327+0000","last_unstale":"2024-05-08T23:24:46.292349+0000","last_undegraded":"2024-05-08T23:24:46.292349+0000","last_fullsized":"2024-05-08T23:24:46.292349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:10.154887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991333+0000","last_change":"2024-05-08T23:24:45.313722+0000","last_active":"2024-05-08T23:24:46.991333+0000","last_peered":"2024-05-08T23:24:46.991333+0000","last_clean":"2024-05-08T23:24:46.991333+0000","last_became_active":"2024-05-08T23:24:45.310587+0000","last_became_peered":"2024-05-08T23:24:45.310587+0000","last_unstale":"2024-05-08T23:24:46.991333+0000","last_undegraded":"2024-05-08T23:24:46.991333+0000","last_fullsized":"2024-05-08T23:24:46.991333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:43.893324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292404+0000","last_change":"2024-05-08T23:24:45.306531+0000","last_active":"2024-05-08T23:24:46.292404+0000","last_peered":"2024-05-08T23:24:46.292404+0000","last_clean":"2024-05-08T23:24:46.292404+0000","last_became_active":"2024-05-08T23:24:45.306250+0000","last_became_peered":"2024-05-08T23:24:45.306250+0000","last_unstale":"2024-05-08T23:24:46.292404+0000","last_undegraded":"2024-05-08T23:24:46.292404+0000","last_fullsized":"2024-05-08T23:24:46.292404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:03.702310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291733+0000","last_change":"2024-05-08T23:24:45.305983+0000","last_active":"2024-05-08T23:24:46.291733+0000","last_peered":"2024-05-08T23:24:46.291733+0000","last_clean":"2024-05-08T23:24:46.291733+0000","last_became_active":"2024-05-08T23:24:45.305099+0000","last_became_peered":"2024-05-08T23:24:45.305099+0000","last_unstale":"2024-05-08T23:24:46.291733+0000","last_undegraded":"2024-05-08T23:24:46.291733+0000","last_fullsized":"2024-05-08T23:24:46.291733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:01.505793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990693+0000","last_change":"2024-05-08T23:24:45.305791+0000","last_active":"2024-05-08T23:24:46.990693+0000","last_peered":"2024-05-08T23:24:46.990693+0000","last_clean":"2024-05-08T23:24:46.990693+0000","last_became_active":"2024-05-08T23:24:45.305571+0000","last_became_peered":"2024-05-08T23:24:45.305571+0000","last_unstale":"2024-05-08T23:24:46.990693+0000","last_undegraded":"2024-05-08T23:24:46.990693+0000","last_fullsized":"2024-05-08T23:24:46.990693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:43.893324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991529+0000","last_change":"2024-05-08T23:24:45.297285+0000","last_active":"2024-05-08T23:24:46.991529+0000","last_peered":"2024-05-08T23:24:46.991529+0000","last_clean":"2024-05-08T23:24:46.991529+0000","last_became_active":"2024-05-08T23:24:45.296999+0000","last_became_peered":"2024-05-08T23:24:45.296999+0000","last_unstale":"2024-05-08T23:24:46.991529+0000","last_undegraded":"2024-05-08T23:24:46.991529+0000","last_fullsized":"2024-05-08T23:24:46.991529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:24:39.096348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291769+0000","last_change":"2024-05-08T23:24:45.301330+0000","last_active":"2024-05-08T23:24:46.291769+0000","last_peered":"2024-05-08T23:24:46.291769+0000","last_clean":"2024-05-08T23:24:46.291769+0000","last_became_active":"2024-05-08T23:24:45.300996+0000","last_became_peered":"2024-05-08T23:24:45.300996+0000","last_unstale":"2024-05-08T23:24:46.291769+0000","last_undegraded":"2024-05-08T23:24:46.291769+0000","last_fullsized":"2024-05-08T23:24:46.291769+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:26.419489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992117+0000","last_change":"2024-05-08T23:24:45.292468+0000","last_active":"2024-05-08T23:24:46.992117+0000","last_peered":"2024-05-08T23:24:46.992117+0000","last_clean":"2024-05-08T23:24:46.992117+0000","last_became_active":"2024-05-08T23:24:45.292169+0000","last_became_peered":"2024-05-08T23:24:45.292169+0000","last_unstale":"2024-05-08T23:24:46.992117+0000","last_undegraded":"2024-05-08T23:24:46.992117+0000","last_fullsized":"2024-05-08T23:24:46.992117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:59.820193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991170+0000","last_change":"2024-05-08T23:24:45.301478+0000","last_active":"2024-05-08T23:24:46.991170+0000","last_peered":"2024-05-08T23:24:46.991170+0000","last_clean":"2024-05-08T23:24:46.991170+0000","last_became_active":"2024-05-08T23:24:45.301203+0000","last_became_peered":"2024-05-08T23:24:45.301203+0000","last_unstale":"2024-05-08T23:24:46.991170+0000","last_undegraded":"2024-05-08T23:24:46.991170+0000","last_fullsized":"2024-05-08T23:24:46.991170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:25.697853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991032+0000","last_change":"2024-05-08T23:24:45.314582+0000","last_active":"2024-05-08T23:24:46.991032+0000","last_peered":"2024-05-08T23:24:46.991032+0000","last_clean":"2024-05-08T23:24:46.991032+0000","last_became_active":"2024-05-08T23:24:45.314355+0000","last_became_peered":"2024-05-08T23:24:45.314355+0000","last_unstale":"2024-05-08T23:24:46.991032+0000","last_undegraded":"2024-05-08T23:24:46.991032+0000","last_fullsized":"2024-05-08T23:24:46.991032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:33.599117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292529+0000","last_change":"2024-05-08T23:24:45.301823+0000","last_active":"2024-05-08T23:24:46.292529+0000","last_peered":"2024-05-08T23:24:46.292529+0000","last_clean":"2024-05-08T23:24:46.292529+0000","last_became_active":"2024-05-08T23:24:45.301313+0000","last_became_peered":"2024-05-08T23:24:45.301313+0000","last_unstale":"2024-05-08T23:24:46.292529+0000","last_undegraded":"2024-05-08T23:24:46.292529+0000","last_fullsized":"2024-05-08T23:24:46.292529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:53.159272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991471+0000","last_change":"2024-05-08T23:24:45.295795+0000","last_active":"2024-05-08T23:24:46.991471+0000","last_peered":"2024-05-08T23:24:46.991471+0000","last_clean":"2024-05-08T23:24:46.991471+0000","last_became_active":"2024-05-08T23:24:45.295263+0000","last_became_peered":"2024-05-08T23:24:45.295263+0000","last_unstale":"2024-05-08T23:24:46.991471+0000","last_undegraded":"2024-05-08T23:24:46.991471+0000","last_fullsized":"2024-05-08T23:24:46.991471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:10.130820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991602+0000","last_change":"2024-05-08T23:24:45.305903+0000","last_active":"2024-05-08T23:24:46.991602+0000","last_peered":"2024-05-08T23:24:46.991602+0000","last_clean":"2024-05-08T23:24:46.991602+0000","last_became_active":"2024-05-08T23:24:45.305473+0000","last_became_peered":"2024-05-08T23:24:45.305473+0000","last_unstale":"2024-05-08T23:24:46.991602+0000","last_undegraded":"2024-05-08T23:24:46.991602+0000","last_fullsized":"2024-05-08T23:24:46.991602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:41.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991645+0000","last_change":"2024-05-08T23:24:45.305318+0000","last_active":"2024-05-08T23:24:46.991645+0000","last_peered":"2024-05-08T23:24:46.991645+0000","last_clean":"2024-05-08T23:24:46.991645+0000","last_became_active":"2024-05-08T23:24:45.305004+0000","last_became_peered":"2024-05-08T23:24:45.305004+0000","last_unstale":"2024-05-08T23:24:46.991645+0000","last_undegraded":"2024-05-08T23:24:46.991645+0000","last_fullsized":"2024-05-08T23:24:46.991645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:32.833314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991118+0000","last_change":"2024-05-08T23:24:45.302122+0000","last_active":"2024-05-08T23:24:46.991118+0000","last_peered":"2024-05-08T23:24:46.991118+0000","last_clean":"2024-05-08T23:24:46.991118+0000","last_became_active":"2024-05-08T23:24:45.301855+0000","last_became_peered":"2024-05-08T23:24:45.301855+0000","last_unstale":"2024-05-08T23:24:46.991118+0000","last_undegraded":"2024-05-08T23:24:46.991118+0000","last_fullsized":"2024-05-08T23:24:46.991118+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:26.099058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292335+0000","last_change":"2024-05-08T23:24:45.310506+0000","last_active":"2024-05-08T23:24:46.292335+0000","last_peered":"2024-05-08T23:24:46.292335+0000","last_clean":"2024-05-08T23:24:46.292335+0000","last_became_active":"2024-05-08T23:24:45.309405+0000","last_became_peered":"2024-05-08T23:24:45.309405+0000","last_unstale":"2024-05-08T23:24:46.292335+0000","last_undegraded":"2024-05-08T23:24:46.292335+0000","last_fullsized":"2024-05-08T23:24:46.292335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:49.588979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991910+0000","last_change":"2024-05-08T23:24:45.313266+0000","last_active":"2024-05-08T23:24:46.991910+0000","last_peered":"2024-05-08T23:24:46.991910+0000","last_clean":"2024-05-08T23:24:46.991910+0000","last_became_active":"2024-05-08T23:24:45.312991+0000","last_became_peered":"2024-05-08T23:24:45.312991+0000","last_unstale":"2024-05-08T23:24:46.991910+0000","last_undegraded":"2024-05-08T23:24:46.991910+0000","last_fullsized":"2024-05-08T23:24:46.991910+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:40.712218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292132+0000","last_change":"2024-05-08T23:24:45.308974+0000","last_active":"2024-05-08T23:24:46.292132+0000","last_peered":"2024-05-08T23:24:46.292132+0000","last_clean":"2024-05-08T23:24:46.292132+0000","last_became_active":"2024-05-08T23:24:45.308691+0000","last_became_peered":"2024-05-08T23:24:45.308691+0000","last_unstale":"2024-05-08T23:24:46.292132+0000","last_undegraded":"2024-05-08T23:24:46.292132+0000","last_fullsized":"2024-05-08T23:24:46.292132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:42.812590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292754+0000","last_change":"2024-05-08T23:24:45.309185+0000","last_active":"2024-05-08T23:24:46.292754+0000","last_peered":"2024-05-08T23:24:46.292754+0000","last_clean":"2024-05-08T23:24:46.292754+0000","last_became_active":"2024-05-08T23:24:45.308135+0000","last_became_peered":"2024-05-08T23:24:45.308135+0000","last_unstale":"2024-05-08T23:24:46.292754+0000","last_undegraded":"2024-05-08T23:24:46.292754+0000","last_fullsized":"2024-05-08T23:24:46.292754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:29.431050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991773+0000","last_change":"2024-05-08T23:24:45.309924+0000","last_active":"2024-05-08T23:24:46.991773+0000","last_peered":"2024-05-08T23:24:46.991773+0000","last_clean":"2024-05-08T23:24:46.991773+0000","last_became_active":"2024-05-08T23:24:45.309608+0000","last_became_peered":"2024-05-08T23:24:45.309608+0000","last_unstale":"2024-05-08T23:24:46.991773+0000","last_undegraded":"2024-05-08T23:24:46.991773+0000","last_fullsized":"2024-05-08T23:24:46.991773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:02.893488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291882+0000","last_change":"2024-05-08T23:24:45.308180+0000","last_active":"2024-05-08T23:24:46.291882+0000","last_peered":"2024-05-08T23:24:46.291882+0000","last_clean":"2024-05-08T23:24:46.291882+0000","last_became_active":"2024-05-08T23:24:45.307850+0000","last_became_peered":"2024-05-08T23:24:45.307850+0000","last_unstale":"2024-05-08T23:24:46.291882+0000","last_undegraded":"2024-05-08T23:24:46.291882+0000","last_fullsized":"2024-05-08T23:24:46.291882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:59.384496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992227+0000","last_change":"2024-05-08T23:24:45.314435+0000","last_active":"2024-05-08T23:24:46.992227+0000","last_peered":"2024-05-08T23:24:46.992227+0000","last_clean":"2024-05-08T23:24:46.992227+0000","last_became_active":"2024-05-08T23:24:45.314144+0000","last_became_peered":"2024-05-08T23:24:45.314144+0000","last_unstale":"2024-05-08T23:24:46.992227+0000","last_undegraded":"2024-05-08T23:24:46.992227+0000","last_fullsized":"2024-05-08T23:24:46.992227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:51.926879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991272+0000","last_change":"2024-05-08T23:24:45.304612+0000","last_active":"2024-05-08T23:24:46.991272+0000","last_peered":"2024-05-08T23:24:46.991272+0000","last_clean":"2024-05-08T23:24:46.991272+0000","last_became_active":"2024-05-08T23:24:45.304237+0000","last_became_peered":"2024-05-08T23:24:45.304237+0000","last_unstale":"2024-05-08T23:24:46.991272+0000","last_undegraded":"2024-05-08T23:24:46.991272+0000","last_fullsized":"2024-05-08T23:24:46.991272+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:21.608208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292304+0000","last_change":"2024-05-08T23:24:45.317893+0000","last_active":"2024-05-08T23:24:46.292304+0000","last_peered":"2024-05-08T23:24:46.292304+0000","last_clean":"2024-05-08T23:24:46.292304+0000","last_became_active":"2024-05-08T23:24:45.317611+0000","last_became_peered":"2024-05-08T23:24:45.317611+0000","last_unstale":"2024-05-08T23:24:46.292304+0000","last_undegraded":"2024-05-08T23:24:46.292304+0000","last_fullsized":"2024-05-08T23:24:46.292304+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:35.959761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292480+0000","last_change":"2024-05-08T23:24:45.296165+0000","last_active":"2024-05-08T23:24:46.292480+0000","last_peered":"2024-05-08T23:24:46.292480+0000","last_clean":"2024-05-08T23:24:46.292480+0000","last_became_active":"2024-05-08T23:24:45.295870+0000","last_became_peered":"2024-05-08T23:24:45.295870+0000","last_unstale":"2024-05-08T23:24:46.292480+0000","last_undegraded":"2024-05-08T23:24:46.292480+0000","last_fullsized":"2024-05-08T23:24:46.292480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:41.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990841+0000","last_change":"2024-05-08T23:24:45.298086+0000","last_active":"2024-05-08T23:24:46.990841+0000","last_peered":"2024-05-08T23:24:46.990841+0000","last_clean":"2024-05-08T23:24:46.990841+0000","last_became_active":"2024-05-08T23:24:45.297804+0000","last_became_peered":"2024-05-08T23:24:45.297804+0000","last_unstale":"2024-05-08T23:24:46.990841+0000","last_undegraded":"2024-05-08T23:24:46.990841+0000","last_fullsized":"2024-05-08T23:24:46.990841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:47.956169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292024+0000","last_change":"2024-05-08T23:24:45.303621+0000","last_active":"2024-05-08T23:24:46.292024+0000","last_peered":"2024-05-08T23:24:46.292024+0000","last_clean":"2024-05-08T23:24:46.292024+0000","last_became_active":"2024-05-08T23:24:45.303356+0000","last_became_peered":"2024-05-08T23:24:45.303356+0000","last_unstale":"2024-05-08T23:24:46.292024+0000","last_undegraded":"2024-05-08T23:24:46.292024+0000","last_fullsized":"2024-05-08T23:24:46.292024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.786227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292972+0000","last_change":"2024-05-08T23:24:45.300248+0000","last_active":"2024-05-08T23:24:46.292972+0000","last_peered":"2024-05-08T23:24:46.292972+0000","last_clean":"2024-05-08T23:24:46.292972+0000","last_became_active":"2024-05-08T23:24:45.299876+0000","last_became_peered":"2024-05-08T23:24:45.299876+0000","last_unstale":"2024-05-08T23:24:46.292972+0000","last_undegraded":"2024-05-08T23:24:46.292972+0000","last_fullsized":"2024-05-08T23:24:46.292972+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:00.499202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991025+0000","last_change":"2024-05-08T23:24:45.310126+0000","last_active":"2024-05-08T23:24:46.991025+0000","last_peered":"2024-05-08T23:24:46.991025+0000","last_clean":"2024-05-08T23:24:46.991025+0000","last_became_active":"2024-05-08T23:24:45.309811+0000","last_became_peered":"2024-05-08T23:24:45.309811+0000","last_unstale":"2024-05-08T23:24:46.991025+0000","last_undegraded":"2024-05-08T23:24:46.991025+0000","last_fullsized":"2024-05-08T23:24:46.991025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:47.956169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291552+0000","last_change":"2024-05-08T23:24:45.316988+0000","last_active":"2024-05-08T23:24:46.291552+0000","last_peered":"2024-05-08T23:24:46.291552+0000","last_clean":"2024-05-08T23:24:46.291552+0000","last_became_active":"2024-05-08T23:24:45.316658+0000","last_became_peered":"2024-05-08T23:24:45.316658+0000","last_unstale":"2024-05-08T23:24:46.291552+0000","last_undegraded":"2024-05-08T23:24:46.291552+0000","last_fullsized":"2024-05-08T23:24:46.291552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:53.801148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991376+0000","last_change":"2024-05-08T23:24:45.302835+0000","last_active":"2024-05-08T23:24:46.991376+0000","last_peered":"2024-05-08T23:24:46.991376+0000","last_clean":"2024-05-08T23:24:46.991376+0000","last_became_active":"2024-05-08T23:24:45.302546+0000","last_became_peered":"2024-05-08T23:24:45.302546+0000","last_unstale":"2024-05-08T23:24:46.991376+0000","last_undegraded":"2024-05-08T23:24:46.991376+0000","last_fullsized":"2024-05-08T23:24:46.991376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:53.356137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991468+0000","last_change":"2024-05-08T23:24:45.310626+0000","last_active":"2024-05-08T23:24:46.991468+0000","last_peered":"2024-05-08T23:24:46.991468+0000","last_clean":"2024-05-08T23:24:46.991468+0000","last_became_active":"2024-05-08T23:24:45.310331+0000","last_became_peered":"2024-05-08T23:24:45.310331+0000","last_unstale":"2024-05-08T23:24:46.991468+0000","last_undegraded":"2024-05-08T23:24:46.991468+0000","last_fullsized":"2024-05-08T23:24:46.991468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:02.893488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991563+0000","last_change":"2024-05-08T23:24:44.833160+0000","last_active":"2024-05-08T23:24:46.991563+0000","last_peered":"2024-05-08T23:24:46.991563+0000","last_clean":"2024-05-08T23:24:46.991563+0000","last_became_active":"2024-05-08T23:24:44.832856+0000","last_became_peered":"2024-05-08T23:24:44.832856+0000","last_unstale":"2024-05-08T23:24:46.991563+0000","last_undegraded":"2024-05-08T23:24:46.991563+0000","last_fullsized":"2024-05-08T23:24:46.991563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:28.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991446+0000","last_change":"2024-05-08T23:24:45.315842+0000","last_active":"2024-05-08T23:24:46.991446+0000","last_peered":"2024-05-08T23:24:46.991446+0000","last_clean":"2024-05-08T23:24:46.991446+0000","last_became_active":"2024-05-08T23:24:45.315588+0000","last_became_peered":"2024-05-08T23:24:45.315588+0000","last_unstale":"2024-05-08T23:24:46.991446+0000","last_undegraded":"2024-05-08T23:24:46.991446+0000","last_fullsized":"2024-05-08T23:24:46.991446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:13.847361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292436+0000","last_change":"2024-05-08T23:24:44.834221+0000","last_active":"2024-05-08T23:24:46.292436+0000","last_peered":"2024-05-08T23:24:46.292436+0000","last_clean":"2024-05-08T23:24:46.292436+0000","last_became_active":"2024-05-08T23:24:44.834012+0000","last_became_peered":"2024-05-08T23:24:44.834012+0000","last_unstale":"2024-05-08T23:24:46.292436+0000","last_undegraded":"2024-05-08T23:24:46.292436+0000","last_fullsized":"2024-05-08T23:24:46.292436+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:45.857534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991245+0000","last_change":"2024-05-08T23:24:45.305306+0000","last_active":"2024-05-08T23:24:46.991245+0000","last_peered":"2024-05-08T23:24:46.991245+0000","last_clean":"2024-05-08T23:24:46.991245+0000","last_became_active":"2024-05-08T23:24:45.304956+0000","last_became_peered":"2024-05-08T23:24:45.304956+0000","last_unstale":"2024-05-08T23:24:46.991245+0000","last_undegraded":"2024-05-08T23:24:46.991245+0000","last_fullsized":"2024-05-08T23:24:46.991245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:33.903726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991086+0000","last_change":"2024-05-08T23:24:44.838890+0000","last_active":"2024-05-08T23:24:46.991086+0000","last_peered":"2024-05-08T23:24:46.991086+0000","last_clean":"2024-05-08T23:24:46.991086+0000","last_became_active":"2024-05-08T23:24:44.838553+0000","last_became_peered":"2024-05-08T23:24:44.838553+0000","last_unstale":"2024-05-08T23:24:46.991086+0000","last_undegraded":"2024-05-08T23:24:46.991086+0000","last_fullsized":"2024-05-08T23:24:46.991086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:31.828404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990969+0000","last_change":"2024-05-08T23:24:45.297758+0000","last_active":"2024-05-08T23:24:46.990969+0000","last_peered":"2024-05-08T23:24:46.990969+0000","last_clean":"2024-05-08T23:24:46.990969+0000","last_became_active":"2024-05-08T23:24:45.297386+0000","last_became_peered":"2024-05-08T23:24:45.297386+0000","last_unstale":"2024-05-08T23:24:46.990969+0000","last_undegraded":"2024-05-08T23:24:46.990969+0000","last_fullsized":"2024-05-08T23:24:46.990969+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:22.639106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292653+0000","last_change":"2024-05-08T23:24:44.835229+0000","last_active":"2024-05-08T23:24:46.292653+0000","last_peered":"2024-05-08T23:24:46.292653+0000","last_clean":"2024-05-08T23:24:46.292653+0000","last_became_active":"2024-05-08T23:24:44.834835+0000","last_became_peered":"2024-05-08T23:24:44.834835+0000","last_unstale":"2024-05-08T23:24:46.292653+0000","last_undegraded":"2024-05-08T23:24:46.292653+0000","last_fullsized":"2024-05-08T23:24:46.292653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:34.271517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291985+0000","last_change":"2024-05-08T23:24:45.308260+0000","last_active":"2024-05-08T23:24:46.291985+0000","last_peered":"2024-05-08T23:24:46.291985+0000","last_clean":"2024-05-08T23:24:46.291985+0000","last_became_active":"2024-05-08T23:24:45.307438+0000","last_became_peered":"2024-05-08T23:24:45.307438+0000","last_unstale":"2024-05-08T23:24:46.291985+0000","last_undegraded":"2024-05-08T23:24:46.291985+0000","last_fullsized":"2024-05-08T23:24:46.291985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:33.532590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991710+0000","last_change":"2024-05-08T23:24:44.838162+0000","last_active":"2024-05-08T23:24:46.991710+0000","last_peered":"2024-05-08T23:24:46.991710+0000","last_clean":"2024-05-08T23:24:46.991710+0000","last_became_active":"2024-05-08T23:24:44.837809+0000","last_became_peered":"2024-05-08T23:24:44.837809+0000","last_unstale":"2024-05-08T23:24:46.991710+0000","last_undegraded":"2024-05-08T23:24:46.991710+0000","last_fullsized":"2024-05-08T23:24:46.991710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.715143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.293077+0000","last_change":"2024-05-08T23:24:45.310436+0000","last_active":"2024-05-08T23:24:46.293077+0000","last_peered":"2024-05-08T23:24:46.293077+0000","last_clean":"2024-05-08T23:24:46.293077+0000","last_became_active":"2024-05-08T23:24:45.307492+0000","last_became_peered":"2024-05-08T23:24:45.307492+0000","last_unstale":"2024-05-08T23:24:46.293077+0000","last_undegraded":"2024-05-08T23:24:46.293077+0000","last_fullsized":"2024-05-08T23:24:46.293077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:27.414348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990881+0000","last_change":"2024-05-08T23:24:44.833118+0000","last_active":"2024-05-08T23:24:46.990881+0000","last_peered":"2024-05-08T23:24:46.990881+0000","last_clean":"2024-05-08T23:24:46.990881+0000","last_became_active":"2024-05-08T23:24:44.832818+0000","last_became_peered":"2024-05-08T23:24:44.832818+0000","last_unstale":"2024-05-08T23:24:46.990881+0000","last_undegraded":"2024-05-08T23:24:46.990881+0000","last_fullsized":"2024-05-08T23:24:46.990881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:58.815283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991380+0000","last_change":"2024-05-08T23:24:45.300276+0000","last_active":"2024-05-08T23:24:46.991380+0000","last_peered":"2024-05-08T23:24:46.991380+0000","last_clean":"2024-05-08T23:24:46.991380+0000","last_became_active":"2024-05-08T23:24:45.299977+0000","last_became_peered":"2024-05-08T23:24:45.299977+0000","last_unstale":"2024-05-08T23:24:46.991380+0000","last_undegraded":"2024-05-08T23:24:46.991380+0000","last_fullsized":"2024-05-08T23:24:46.991380+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:47.994864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292168+0000","last_change":"2024-05-08T23:24:44.831992+0000","last_active":"2024-05-08T23:24:46.292168+0000","last_peered":"2024-05-08T23:24:46.292168+0000","last_clean":"2024-05-08T23:24:46.292168+0000","last_became_active":"2024-05-08T23:24:44.831730+0000","last_became_peered":"2024-05-08T23:24:44.831730+0000","last_unstale":"2024-05-08T23:24:46.292168+0000","last_undegraded":"2024-05-08T23:24:46.292168+0000","last_fullsized":"2024-05-08T23:24:46.292168+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.715143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292261+0000","last_change":"2024-05-08T23:24:45.302068+0000","last_active":"2024-05-08T23:24:46.292261+0000","last_peered":"2024-05-08T23:24:46.292261+0000","last_clean":"2024-05-08T23:24:46.292261+0000","last_became_active":"2024-05-08T23:24:45.301737+0000","last_became_peered":"2024-05-08T23:24:45.301737+0000","last_unstale":"2024-05-08T23:24:46.292261+0000","last_undegraded":"2024-05-08T23:24:46.292261+0000","last_fullsized":"2024-05-08T23:24:46.292261+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:05.224848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992159+0000","last_change":"2024-05-08T23:24:44.829538+0000","last_active":"2024-05-08T23:24:46.992159+0000","last_peered":"2024-05-08T23:24:46.992159+0000","last_clean":"2024-05-08T23:24:46.992159+0000","last_became_active":"2024-05-08T23:24:44.829223+0000","last_became_peered":"2024-05-08T23:24:44.829223+0000","last_unstale":"2024-05-08T23:24:46.992159+0000","last_undegraded":"2024-05-08T23:24:46.992159+0000","last_fullsized":"2024-05-08T23:24:46.992159+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:33.819270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991987+0000","last_change":"2024-05-08T23:24:45.300174+0000","last_active":"2024-05-08T23:24:46.991987+0000","last_peered":"2024-05-08T23:24:46.991987+0000","last_clean":"2024-05-08T23:24:46.991987+0000","last_became_active":"2024-05-08T23:24:45.299879+0000","last_became_peered":"2024-05-08T23:24:45.299879+0000","last_unstale":"2024-05-08T23:24:46.991987+0000","last_undegraded":"2024-05-08T23:24:46.991987+0000","last_fullsized":"2024-05-08T23:24:46.991987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:58.502436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991846+0000","last_change":"2024-05-08T23:24:44.838767+0000","last_active":"2024-05-08T23:24:46.991846+0000","last_peered":"2024-05-08T23:24:46.991846+0000","last_clean":"2024-05-08T23:24:46.991846+0000","last_became_active":"2024-05-08T23:24:44.838479+0000","last_became_peered":"2024-05-08T23:24:44.838479+0000","last_unstale":"2024-05-08T23:24:46.991846+0000","last_undegraded":"2024-05-08T23:24:46.991846+0000","last_fullsized":"2024-05-08T23:24:46.991846+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:55.904063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992111+0000","last_change":"2024-05-08T23:24:45.317860+0000","last_active":"2024-05-08T23:24:46.992111+0000","last_peered":"2024-05-08T23:24:46.992111+0000","last_clean":"2024-05-08T23:24:46.992111+0000","last_became_active":"2024-05-08T23:24:45.317614+0000","last_became_peered":"2024-05-08T23:24:45.317614+0000","last_unstale":"2024-05-08T23:24:46.992111+0000","last_undegraded":"2024-05-08T23:24:46.992111+0000","last_fullsized":"2024-05-08T23:24:46.992111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:53.605503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292496+0000","last_change":"2024-05-08T23:24:44.838156+0000","last_active":"2024-05-08T23:24:46.292496+0000","last_peered":"2024-05-08T23:24:46.292496+0000","last_clean":"2024-05-08T23:24:46.292496+0000","last_became_active":"2024-05-08T23:24:44.837932+0000","last_became_peered":"2024-05-08T23:24:44.837932+0000","last_unstale":"2024-05-08T23:24:46.292496+0000","last_undegraded":"2024-05-08T23:24:46.292496+0000","last_fullsized":"2024-05-08T23:24:46.292496+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:42.888414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990742+0000","last_change":"2024-05-08T23:24:45.314187+0000","last_active":"2024-05-08T23:24:46.990742+0000","last_peered":"2024-05-08T23:24:46.990742+0000","last_clean":"2024-05-08T23:24:46.990742+0000","last_became_active":"2024-05-08T23:24:45.309482+0000","last_became_peered":"2024-05-08T23:24:45.309482+0000","last_unstale":"2024-05-08T23:24:46.990742+0000","last_undegraded":"2024-05-08T23:24:46.990742+0000","last_fullsized":"2024-05-08T23:24:46.990742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:26.006592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991638+0000","last_change":"2024-05-08T23:24:44.835902+0000","last_active":"2024-05-08T23:24:46.991638+0000","last_peered":"2024-05-08T23:24:46.991638+0000","last_clean":"2024-05-08T23:24:46.991638+0000","last_became_active":"2024-05-08T23:24:44.835287+0000","last_became_peered":"2024-05-08T23:24:44.835287+0000","last_unstale":"2024-05-08T23:24:46.991638+0000","last_undegraded":"2024-05-08T23:24:46.991638+0000","last_fullsized":"2024-05-08T23:24:46.991638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:42.806659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991433+0000","last_change":"2024-05-08T23:24:45.317494+0000","last_active":"2024-05-08T23:24:46.991433+0000","last_peered":"2024-05-08T23:24:46.991433+0000","last_clean":"2024-05-08T23:24:46.991433+0000","last_became_active":"2024-05-08T23:24:45.317199+0000","last_became_peered":"2024-05-08T23:24:45.317199+0000","last_unstale":"2024-05-08T23:24:46.991433+0000","last_undegraded":"2024-05-08T23:24:46.991433+0000","last_fullsized":"2024-05-08T23:24:46.991433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:53.159272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992084+0000","last_change":"2024-05-08T23:24:44.831798+0000","last_active":"2024-05-08T23:24:46.992084+0000","last_peered":"2024-05-08T23:24:46.992084+0000","last_clean":"2024-05-08T23:24:46.992084+0000","last_became_active":"2024-05-08T23:24:44.831506+0000","last_became_peered":"2024-05-08T23:24:44.831506+0000","last_unstale":"2024-05-08T23:24:46.992084+0000","last_undegraded":"2024-05-08T23:24:46.992084+0000","last_fullsized":"2024-05-08T23:24:46.992084+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:46.989954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292026+0000","last_change":"2024-05-08T23:24:45.303385+0000","last_active":"2024-05-08T23:24:46.292026+0000","last_peered":"2024-05-08T23:24:46.292026+0000","last_clean":"2024-05-08T23:24:46.292026+0000","last_became_active":"2024-05-08T23:24:45.303116+0000","last_became_peered":"2024-05-08T23:24:45.303116+0000","last_unstale":"2024-05-08T23:24:46.292026+0000","last_undegraded":"2024-05-08T23:24:46.292026+0000","last_fullsized":"2024-05-08T23:24:46.292026+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:47:45.766129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.293123+0000","last_change":"2024-05-08T23:24:44.838808+0000","last_active":"2024-05-08T23:24:46.293123+0000","last_peered":"2024-05-08T23:24:46.293123+0000","last_clean":"2024-05-08T23:24:46.293123+0000","last_became_active":"2024-05-08T23:24:44.838527+0000","last_became_peered":"2024-05-08T23:24:44.838527+0000","last_unstale":"2024-05-08T23:24:46.293123+0000","last_undegraded":"2024-05-08T23:24:46.293123+0000","last_fullsized":"2024-05-08T23:24:46.293123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:46.951259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991883+0000","last_change":"2024-05-08T23:24:45.309437+0000","last_active":"2024-05-08T23:24:46.991883+0000","last_peered":"2024-05-08T23:24:46.991883+0000","last_clean":"2024-05-08T23:24:46.991883+0000","last_became_active":"2024-05-08T23:24:45.309124+0000","last_became_peered":"2024-05-08T23:24:45.309124+0000","last_unstale":"2024-05-08T23:24:46.991883+0000","last_undegraded":"2024-05-08T23:24:46.991883+0000","last_fullsized":"2024-05-08T23:24:46.991883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:03.702310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991073+0000","last_change":"2024-05-08T23:24:44.831589+0000","last_active":"2024-05-08T23:24:46.991073+0000","last_peered":"2024-05-08T23:24:46.991073+0000","last_clean":"2024-05-08T23:24:46.991073+0000","last_became_active":"2024-05-08T23:24:44.831249+0000","last_became_peered":"2024-05-08T23:24:44.831249+0000","last_unstale":"2024-05-08T23:24:46.991073+0000","last_undegraded":"2024-05-08T23:24:46.991073+0000","last_fullsized":"2024-05-08T23:24:46.991073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:25.001682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990883+0000","last_change":"2024-05-08T23:24:45.311852+0000","last_active":"2024-05-08T23:24:46.990883+0000","last_peered":"2024-05-08T23:24:46.990883+0000","last_clean":"2024-05-08T23:24:46.990883+0000","last_became_active":"2024-05-08T23:24:45.311581+0000","last_became_peered":"2024-05-08T23:24:45.311581+0000","last_unstale":"2024-05-08T23:24:46.990883+0000","last_undegraded":"2024-05-08T23:24:46.990883+0000","last_fullsized":"2024-05-08T23:24:46.990883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:58:16.684695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292078+0000","last_change":"2024-05-08T23:24:44.835564+0000","last_active":"2024-05-08T23:24:46.292078+0000","last_peered":"2024-05-08T23:24:46.292078+0000","last_clean":"2024-05-08T23:24:46.292078+0000","last_became_active":"2024-05-08T23:24:44.834783+0000","last_became_peered":"2024-05-08T23:24:44.834783+0000","last_unstale":"2024-05-08T23:24:46.292078+0000","last_undegraded":"2024-05-08T23:24:46.292078+0000","last_fullsized":"2024-05-08T23:24:46.292078+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:42.806659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992013+0000","last_change":"2024-05-08T23:24:45.313273+0000","last_active":"2024-05-08T23:24:46.992013+0000","last_peered":"2024-05-08T23:24:46.992013+0000","last_clean":"2024-05-08T23:24:46.992013+0000","last_became_active":"2024-05-08T23:24:45.313015+0000","last_became_peered":"2024-05-08T23:24:45.313015+0000","last_unstale":"2024-05-08T23:24:46.992013+0000","last_undegraded":"2024-05-08T23:24:46.992013+0000","last_fullsized":"2024-05-08T23:24:46.992013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:34.043367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991066+0000","last_change":"2024-05-08T23:24:44.837004+0000","last_active":"2024-05-08T23:24:46.991066+0000","last_peered":"2024-05-08T23:24:46.991066+0000","last_clean":"2024-05-08T23:24:46.991066+0000","last_became_active":"2024-05-08T23:24:44.836660+0000","last_became_peered":"2024-05-08T23:24:44.836660+0000","last_unstale":"2024-05-08T23:24:46.991066+0000","last_undegraded":"2024-05-08T23:24:46.991066+0000","last_fullsized":"2024-05-08T23:24:46.991066+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:39.535915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992033+0000","last_change":"2024-05-08T23:24:45.310715+0000","last_active":"2024-05-08T23:24:46.992033+0000","last_peered":"2024-05-08T23:24:46.992033+0000","last_clean":"2024-05-08T23:24:46.992033+0000","last_became_active":"2024-05-08T23:24:45.310427+0000","last_became_peered":"2024-05-08T23:24:45.310427+0000","last_unstale":"2024-05-08T23:24:46.992033+0000","last_undegraded":"2024-05-08T23:24:46.992033+0000","last_fullsized":"2024-05-08T23:24:46.992033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:26.099058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292430+0000","last_change":"2024-05-08T23:24:44.839874+0000","last_active":"2024-05-08T23:24:46.292430+0000","last_peered":"2024-05-08T23:24:46.292430+0000","last_clean":"2024-05-08T23:24:46.292430+0000","last_became_active":"2024-05-08T23:24:44.839590+0000","last_became_peered":"2024-05-08T23:24:44.839590+0000","last_unstale":"2024-05-08T23:24:46.292430+0000","last_undegraded":"2024-05-08T23:24:46.292430+0000","last_fullsized":"2024-05-08T23:24:46.292430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:33.038457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292699+0000","last_change":"2024-05-08T23:24:45.310961+0000","last_active":"2024-05-08T23:24:46.292699+0000","last_peered":"2024-05-08T23:24:46.292699+0000","last_clean":"2024-05-08T23:24:46.292699+0000","last_became_active":"2024-05-08T23:24:45.310668+0000","last_became_peered":"2024-05-08T23:24:45.310668+0000","last_unstale":"2024-05-08T23:24:46.292699+0000","last_undegraded":"2024-05-08T23:24:46.292699+0000","last_fullsized":"2024-05-08T23:24:46.292699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:26.599427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991558+0000","last_change":"2024-05-08T23:24:44.836375+0000","last_active":"2024-05-08T23:24:46.991558+0000","last_peered":"2024-05-08T23:24:46.991558+0000","last_clean":"2024-05-08T23:24:46.991558+0000","last_became_active":"2024-05-08T23:24:44.836083+0000","last_became_peered":"2024-05-08T23:24:44.836083+0000","last_unstale":"2024-05-08T23:24:46.991558+0000","last_undegraded":"2024-05-08T23:24:46.991558+0000","last_fullsized":"2024-05-08T23:24:46.991558+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:57.497526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292849+0000","last_change":"2024-05-08T23:24:45.300637+0000","last_active":"2024-05-08T23:24:46.292849+0000","last_peered":"2024-05-08T23:24:46.292849+0000","last_clean":"2024-05-08T23:24:46.292849+0000","last_became_active":"2024-05-08T23:24:45.300276+0000","last_became_peered":"2024-05-08T23:24:45.300276+0000","last_unstale":"2024-05-08T23:24:46.292849+0000","last_undegraded":"2024-05-08T23:24:46.292849+0000","last_fullsized":"2024-05-08T23:24:46.292849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:50.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292486+0000","last_change":"2024-05-08T23:24:44.834762+0000","last_active":"2024-05-08T23:24:46.292486+0000","last_peered":"2024-05-08T23:24:46.292486+0000","last_clean":"2024-05-08T23:24:46.292486+0000","last_became_active":"2024-05-08T23:24:44.834456+0000","last_became_peered":"2024-05-08T23:24:44.834456+0000","last_unstale":"2024-05-08T23:24:46.292486+0000","last_undegraded":"2024-05-08T23:24:46.292486+0000","last_fullsized":"2024-05-08T23:24:46.292486+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:55.904063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991530+0000","last_change":"2024-05-08T23:24:45.317531+0000","last_active":"2024-05-08T23:24:46.991530+0000","last_peered":"2024-05-08T23:24:46.991530+0000","last_clean":"2024-05-08T23:24:46.991530+0000","last_became_active":"2024-05-08T23:24:45.317315+0000","last_became_peered":"2024-05-08T23:24:45.317315+0000","last_unstale":"2024-05-08T23:24:46.991530+0000","last_undegraded":"2024-05-08T23:24:46.991530+0000","last_fullsized":"2024-05-08T23:24:46.991530+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:10.734065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991483+0000","last_change":"2024-05-08T23:24:44.839061+0000","last_active":"2024-05-08T23:24:46.991483+0000","last_peered":"2024-05-08T23:24:46.991483+0000","last_clean":"2024-05-08T23:24:46.991483+0000","last_became_active":"2024-05-08T23:24:44.838787+0000","last_became_peered":"2024-05-08T23:24:44.838787+0000","last_unstale":"2024-05-08T23:24:46.991483+0000","last_undegraded":"2024-05-08T23:24:46.991483+0000","last_fullsized":"2024-05-08T23:24:46.991483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:40.805471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292134+0000","last_change":"2024-05-08T23:24:45.303366+0000","last_active":"2024-05-08T23:24:46.292134+0000","last_peered":"2024-05-08T23:24:46.292134+0000","last_clean":"2024-05-08T23:24:46.292134+0000","last_became_active":"2024-05-08T23:24:45.303065+0000","last_became_peered":"2024-05-08T23:24:45.303065+0000","last_unstale":"2024-05-08T23:24:46.292134+0000","last_undegraded":"2024-05-08T23:24:46.292134+0000","last_fullsized":"2024-05-08T23:24:46.292134+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:52.546124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.293184+0000","last_change":"2024-05-08T23:24:44.827652+0000","last_active":"2024-05-08T23:24:46.293184+0000","last_peered":"2024-05-08T23:24:46.293184+0000","last_clean":"2024-05-08T23:24:46.293184+0000","last_became_active":"2024-05-08T23:24:44.827329+0000","last_became_peered":"2024-05-08T23:24:44.827329+0000","last_unstale":"2024-05-08T23:24:46.293184+0000","last_undegraded":"2024-05-08T23:24:46.293184+0000","last_fullsized":"2024-05-08T23:24:46.293184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:01.888578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991030+0000","last_change":"2024-05-08T23:24:45.295718+0000","last_active":"2024-05-08T23:24:46.991030+0000","last_peered":"2024-05-08T23:24:46.991030+0000","last_clean":"2024-05-08T23:24:46.991030+0000","last_became_active":"2024-05-08T23:24:45.295437+0000","last_became_peered":"2024-05-08T23:24:45.295437+0000","last_unstale":"2024-05-08T23:24:46.991030+0000","last_undegraded":"2024-05-08T23:24:46.991030+0000","last_fullsized":"2024-05-08T23:24:46.991030+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:40.712218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990846+0000","last_change":"2024-05-08T23:24:44.838175+0000","last_active":"2024-05-08T23:24:46.990846+0000","last_peered":"2024-05-08T23:24:46.990846+0000","last_clean":"2024-05-08T23:24:46.990846+0000","last_became_active":"2024-05-08T23:24:44.837861+0000","last_became_peered":"2024-05-08T23:24:44.837861+0000","last_unstale":"2024-05-08T23:24:46.990846+0000","last_undegraded":"2024-05-08T23:24:46.990846+0000","last_fullsized":"2024-05-08T23:24:46.990846+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:53.781316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991228+0000","last_change":"2024-05-08T23:24:45.305029+0000","last_active":"2024-05-08T23:24:46.991228+0000","last_peered":"2024-05-08T23:24:46.991228+0000","last_clean":"2024-05-08T23:24:46.991228+0000","last_became_active":"2024-05-08T23:24:45.304732+0000","last_became_peered":"2024-05-08T23:24:45.304732+0000","last_unstale":"2024-05-08T23:24:46.991228+0000","last_undegraded":"2024-05-08T23:24:46.991228+0000","last_fullsized":"2024-05-08T23:24:46.991228+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:46.862444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291968+0000","last_change":"2024-05-08T23:24:44.837412+0000","last_active":"2024-05-08T23:24:46.291968+0000","last_peered":"2024-05-08T23:24:46.291968+0000","last_clean":"2024-05-08T23:24:46.291968+0000","last_became_active":"2024-05-08T23:24:44.837113+0000","last_became_peered":"2024-05-08T23:24:44.837113+0000","last_unstale":"2024-05-08T23:24:46.291968+0000","last_undegraded":"2024-05-08T23:24:46.291968+0000","last_fullsized":"2024-05-08T23:24:46.291968+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:20.603298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292518+0000","last_change":"2024-05-08T23:24:45.306133+0000","last_active":"2024-05-08T23:24:46.292518+0000","last_peered":"2024-05-08T23:24:46.292518+0000","last_clean":"2024-05-08T23:24:46.292518+0000","last_became_active":"2024-05-08T23:24:45.305799+0000","last_became_peered":"2024-05-08T23:24:45.305799+0000","last_unstale":"2024-05-08T23:24:46.292518+0000","last_undegraded":"2024-05-08T23:24:46.292518+0000","last_fullsized":"2024-05-08T23:24:46.292518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:33.665957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991554+0000","last_change":"2024-05-08T23:24:44.837193+0000","last_active":"2024-05-08T23:24:46.991554+0000","last_peered":"2024-05-08T23:24:46.991554+0000","last_clean":"2024-05-08T23:24:46.991554+0000","last_became_active":"2024-05-08T23:24:44.836894+0000","last_became_peered":"2024-05-08T23:24:44.836894+0000","last_unstale":"2024-05-08T23:24:46.991554+0000","last_undegraded":"2024-05-08T23:24:46.991554+0000","last_fullsized":"2024-05-08T23:24:46.991554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:00.500883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292727+0000","last_change":"2024-05-08T23:24:45.308069+0000","last_active":"2024-05-08T23:24:46.292727+0000","last_peered":"2024-05-08T23:24:46.292727+0000","last_clean":"2024-05-08T23:24:46.292727+0000","last_became_active":"2024-05-08T23:24:45.307241+0000","last_became_peered":"2024-05-08T23:24:45.307241+0000","last_unstale":"2024-05-08T23:24:46.292727+0000","last_undegraded":"2024-05-08T23:24:46.292727+0000","last_fullsized":"2024-05-08T23:24:46.292727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:10.734065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991627+0000","last_change":"2024-05-08T23:24:44.833012+0000","last_active":"2024-05-08T23:24:46.991627+0000","last_peered":"2024-05-08T23:24:46.991627+0000","last_clean":"2024-05-08T23:24:46.991627+0000","last_became_active":"2024-05-08T23:24:44.832691+0000","last_became_peered":"2024-05-08T23:24:44.832691+0000","last_unstale":"2024-05-08T23:24:46.991627+0000","last_undegraded":"2024-05-08T23:24:46.991627+0000","last_fullsized":"2024-05-08T23:24:46.991627+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:24:38.091438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990861+0000","last_change":"2024-05-08T23:24:45.309846+0000","last_active":"2024-05-08T23:24:46.990861+0000","last_peered":"2024-05-08T23:24:46.990861+0000","last_clean":"2024-05-08T23:24:46.990861+0000","last_became_active":"2024-05-08T23:24:45.309092+0000","last_became_peered":"2024-05-08T23:24:45.309092+0000","last_unstale":"2024-05-08T23:24:46.990861+0000","last_undegraded":"2024-05-08T23:24:46.990861+0000","last_fullsized":"2024-05-08T23:24:46.990861+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:32:34.716931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991539+0000","last_change":"2024-05-08T23:24:44.829257+0000","last_active":"2024-05-08T23:24:46.991539+0000","last_peered":"2024-05-08T23:24:46.991539+0000","last_clean":"2024-05-08T23:24:46.991539+0000","last_became_active":"2024-05-08T23:24:44.827685+0000","last_became_peered":"2024-05-08T23:24:44.827685+0000","last_unstale":"2024-05-08T23:24:46.991539+0000","last_undegraded":"2024-05-08T23:24:46.991539+0000","last_fullsized":"2024-05-08T23:24:46.991539+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:03.302183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991236+0000","last_change":"2024-05-08T23:24:45.296534+0000","last_active":"2024-05-08T23:24:46.991236+0000","last_peered":"2024-05-08T23:24:46.991236+0000","last_clean":"2024-05-08T23:24:46.991236+0000","last_became_active":"2024-05-08T23:24:45.294854+0000","last_became_peered":"2024-05-08T23:24:45.294854+0000","last_unstale":"2024-05-08T23:24:46.991236+0000","last_undegraded":"2024-05-08T23:24:46.991236+0000","last_fullsized":"2024-05-08T23:24:46.991236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:34.043367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292590+0000","last_change":"2024-05-08T23:24:44.830521+0000","last_active":"2024-05-08T23:24:46.292590+0000","last_peered":"2024-05-08T23:24:46.292590+0000","last_clean":"2024-05-08T23:24:46.292590+0000","last_became_active":"2024-05-08T23:24:44.830227+0000","last_became_peered":"2024-05-08T23:24:44.830227+0000","last_unstale":"2024-05-08T23:24:46.292590+0000","last_undegraded":"2024-05-08T23:24:46.292590+0000","last_fullsized":"2024-05-08T23:24:46.292590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:33.819270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292084+0000","last_change":"2024-05-08T23:24:45.316904+0000","last_active":"2024-05-08T23:24:46.292084+0000","last_peered":"2024-05-08T23:24:46.292084+0000","last_clean":"2024-05-08T23:24:46.292084+0000","last_became_active":"2024-05-08T23:24:45.315651+0000","last_became_peered":"2024-05-08T23:24:45.315651+0000","last_unstale":"2024-05-08T23:24:46.292084+0000","last_undegraded":"2024-05-08T23:24:46.292084+0000","last_fullsized":"2024-05-08T23:24:46.292084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:01.809393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991628+0000","last_change":"2024-05-08T23:24:44.835050+0000","last_active":"2024-05-08T23:24:46.991628+0000","last_peered":"2024-05-08T23:24:46.991628+0000","last_clean":"2024-05-08T23:24:46.991628+0000","last_became_active":"2024-05-08T23:24:44.834622+0000","last_became_peered":"2024-05-08T23:24:44.834622+0000","last_unstale":"2024-05-08T23:24:46.991628+0000","last_undegraded":"2024-05-08T23:24:46.991628+0000","last_fullsized":"2024-05-08T23:24:46.991628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:10.795747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992007+0000","last_change":"2024-05-08T23:24:45.314120+0000","last_active":"2024-05-08T23:24:46.992007+0000","last_peered":"2024-05-08T23:24:46.992007+0000","last_clean":"2024-05-08T23:24:46.992007+0000","last_became_active":"2024-05-08T23:24:45.313833+0000","last_became_peered":"2024-05-08T23:24:45.313833+0000","last_unstale":"2024-05-08T23:24:46.992007+0000","last_undegraded":"2024-05-08T23:24:46.992007+0000","last_fullsized":"2024-05-08T23:24:46.992007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:22.639106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991711+0000","last_change":"2024-05-08T23:24:44.831697+0000","last_active":"2024-05-08T23:24:46.991711+0000","last_peered":"2024-05-08T23:24:46.991711+0000","last_clean":"2024-05-08T23:24:46.991711+0000","last_became_active":"2024-05-08T23:24:44.831423+0000","last_became_peered":"2024-05-08T23:24:44.831423+0000","last_unstale":"2024-05-08T23:24:46.991711+0000","last_undegraded":"2024-05-08T23:24:46.991711+0000","last_fullsized":"2024-05-08T23:24:46.991711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:20.603298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990520+0000","last_change":"2024-05-08T23:24:45.302536+0000","last_active":"2024-05-08T23:24:46.990520+0000","last_peered":"2024-05-08T23:24:46.990520+0000","last_clean":"2024-05-08T23:24:46.990520+0000","last_became_active":"2024-05-08T23:24:45.302274+0000","last_became_peered":"2024-05-08T23:24:45.302274+0000","last_unstale":"2024-05-08T23:24:46.990520+0000","last_undegraded":"2024-05-08T23:24:46.990520+0000","last_fullsized":"2024-05-08T23:24:46.990520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:04.307093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990619+0000","last_change":"2024-05-08T23:24:44.835544+0000","last_active":"2024-05-08T23:24:46.990619+0000","last_peered":"2024-05-08T23:24:46.990619+0000","last_clean":"2024-05-08T23:24:46.990619+0000","last_became_active":"2024-05-08T23:24:44.835254+0000","last_became_peered":"2024-05-08T23:24:44.835254+0000","last_unstale":"2024-05-08T23:24:46.990619+0000","last_undegraded":"2024-05-08T23:24:46.990619+0000","last_fullsized":"2024-05-08T23:24:46.990619+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:26.656445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292358+0000","last_change":"2024-05-08T23:24:45.300158+0000","last_active":"2024-05-08T23:24:46.292358+0000","last_peered":"2024-05-08T23:24:46.292358+0000","last_clean":"2024-05-08T23:24:46.292358+0000","last_became_active":"2024-05-08T23:24:45.299858+0000","last_became_peered":"2024-05-08T23:24:45.299858+0000","last_unstale":"2024-05-08T23:24:46.292358+0000","last_undegraded":"2024-05-08T23:24:46.292358+0000","last_fullsized":"2024-05-08T23:24:46.292358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:33.393325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291973+0000","last_change":"2024-05-08T23:24:44.832373+0000","last_active":"2024-05-08T23:24:46.291973+0000","last_peered":"2024-05-08T23:24:46.291973+0000","last_clean":"2024-05-08T23:24:46.291973+0000","last_became_active":"2024-05-08T23:24:44.832091+0000","last_became_peered":"2024-05-08T23:24:44.832091+0000","last_unstale":"2024-05-08T23:24:46.291973+0000","last_undegraded":"2024-05-08T23:24:46.291973+0000","last_fullsized":"2024-05-08T23:24:46.291973+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:39.707308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291881+0000","last_change":"2024-05-08T23:24:45.308475+0000","last_active":"2024-05-08T23:24:46.291881+0000","last_peered":"2024-05-08T23:24:46.291881+0000","last_clean":"2024-05-08T23:24:46.291881+0000","last_became_active":"2024-05-08T23:24:45.308121+0000","last_became_peered":"2024-05-08T23:24:45.308121+0000","last_unstale":"2024-05-08T23:24:46.291881+0000","last_undegraded":"2024-05-08T23:24:46.291881+0000","last_fullsized":"2024-05-08T23:24:46.291881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:40.669776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991915+0000","last_change":"2024-05-08T23:24:44.833194+0000","last_active":"2024-05-08T23:24:46.991915+0000","last_peered":"2024-05-08T23:24:46.991915+0000","last_clean":"2024-05-08T23:24:46.991915+0000","last_became_active":"2024-05-08T23:24:44.832811+0000","last_became_peered":"2024-05-08T23:24:44.832811+0000","last_unstale":"2024-05-08T23:24:46.991915+0000","last_undegraded":"2024-05-08T23:24:46.991915+0000","last_fullsized":"2024-05-08T23:24:46.991915+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:32:33.712021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-05-08T23:24:45.305307+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:24:45.305307+0000","last_peered":"2024-05-08T23:24:45.305307+0000","last_clean":"2024-05-08T23:24:45.305307+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:24:45.305307+0000","last_undegraded":"2024-05-08T23:24:45.305307+0000","last_fullsized":"2024-05-08T23:24:45.305307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1323008,"data_stored":894797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.69099999999999995}]},{"osd":1,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.56899999999999995,"15min":0.54300000000000004},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81000000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.43099999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.753}]},{"osd":3,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78100000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84899999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.79000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":297969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88200000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.40400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81599999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85699999999999998,"5min":1.115,"15min":1.115},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79800000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.498},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98899999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.43099999999999999}]},{"osd":5,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69499999999999995,"5min":1.052,"15min":1.052},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48499999999999999}]},{"osd":7,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.36899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":298059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.39000000000000001,"15min":0.40000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.48099999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.41799999999999998,"15min":0.42799999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54200000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.378}]},{"osd":2,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41799999999999998,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52400000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.28399999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.48899999999999999,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65000000000000002,"5min":1.01,"15min":1.01},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":1.028,"15min":1.028},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.438,"5min":0.432,"15min":0.432},"max":{"1min":0.84299999999999997,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.87,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86699999999999999,"5min":1.115,"15min":1.115},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68999999999999995,"15min":0.70199999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84599999999999997,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91700000000000004,"5min":1.105,"15min":1.105},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69999999999999996,"15min":0.70499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90800000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1318912,"data_stored":890483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.66700000000000004,"15min":0.78700000000000003},"min":{"1min":0.255,"5min":0.254,"15min":0.254},"max":{"1min":0.63400000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.65900000000000003,"15min":0.76900000000000002},"min":{"1min":0.254,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65800000000000003,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.71699999999999997,"15min":0.84499999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58599999999999997,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.433,"5min":0.69299999999999995,"15min":0.80300000000000005},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69899999999999995,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.44900000000000001}]},{"osd":3,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46899999999999997,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59899999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.85499999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79800000000000004,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.86699999999999999,"15min":0.99399999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.68400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.86499999999999999,"15min":0.96099999999999997},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.73399999999999999,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.871,"15min":0.96099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89800000000000002,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.94599999999999995,"15min":1.0700000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.72899999999999998}]},{"osd":6,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.91400000000000003,"15min":1.034},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95099999999999996,"5min":7.734,"15min":7.734},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.90300000000000002,"15min":1.0149999999999999},"min":{"1min":0.48399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.91400000000000003,"15min":1.02},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79000000000000004,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.88800000000000001,"15min":0.97199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81699999999999995,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":304325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61299999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41499999999999998,"15min":0.40200000000000002},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54600000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.42099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.55600000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.58099999999999996}]},{"osd":3,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.38900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.747,"15min":0.747},"last":0.748}]},{"osd":4,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93799999999999994,"5min":1.024,"15min":1.024},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82799999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.76700000000000002}]},{"osd":5,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85699999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.82799999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.878}]},{"osd":6,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":1.159,"15min":1.159},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91300000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.91900000000000004}]},{"osd":7,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.872,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92000000000000004,"5min":1.194,"15min":1.194},"last":0.89800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":304325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26200000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.49099999999999999,"5min":1.46,"15min":1.46},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.28799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51500000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.442}]},{"osd":1,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54100000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.46300000000000002,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64600000000000002,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.76800000000000002,"15min":0.91400000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.63300000000000001,"5min":11.964,"15min":11.964},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":1.593,"15min":1.593},"last":0.67300000000000004}]},{"osd":4,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64800000000000002,"15min":0.66500000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93500000000000005,"5min":1.375,"15min":1.375},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.70299999999999996,"15min":0.73699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86899999999999999,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95099999999999996,"5min":1.536,"15min":1.536},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.47199999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84199999999999997,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67900000000000005,"15min":0.68899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.82699999999999996,"5min":1.204,"15min":1.204},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.71299999999999997,"15min":0.73499999999999999},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92000000000000004,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88800000000000001,"5min":1.339,"15min":1.339},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.52700000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.90800000000000003,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.69399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":305833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.375,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73899999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.68600000000000005,"15min":0.751},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73099999999999998,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.64700000000000002,"15min":0.68700000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":2.145,"15min":2.145},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67200000000000004,"15min":0.70799999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79300000000000004,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.66800000000000004,"15min":0.70699999999999996},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.73999999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.75600000000000001,"15min":0.82099999999999995},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.51400000000000001}]},{"osd":3,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.44400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":1.03,"15min":1.22},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.72399999999999998,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.67200000000000004}]},{"osd":5,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.59899999999999998},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75,"5min":0.877,"15min":0.877},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56599999999999995,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67100000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61099999999999999,"15min":0.63700000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81499999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68700000000000006}]},{"osd":7,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.63700000000000001},"min":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67700000000000005,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72599999999999998,"5min":1.24,"15min":1.24},"last":0.746}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":304921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.69499999999999995,"15min":0.73499999999999999},"min":{"1min":0.39900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.88700000000000001,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.83899999999999997,"15min":0.94899999999999995},"min":{"1min":0.379,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93400000000000005,"5min":8.798,"15min":8.798},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.73899999999999999,"15min":0.81000000000000005},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.70499999999999996,"5min":4.71,"15min":4.71},"last":0.622},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.90300000000000002,"15min":1.0329999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0109999999999999,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.72999999999999998}]},{"osd":2,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.73599999999999999,"15min":0.77600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98899999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.76500000000000001,"15min":0.81899999999999995},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.86099999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.747}]},{"osd":3,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.875,"15min":0.98899999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91800000000000004,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.94099999999999995,"15min":1.079},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.746,"5min":10.427,"15min":10.427},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.874,"15min":0.874},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.40899999999999997}]},{"osd":6,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50900000000000001}]},{"osd":7,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.377,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85599999999999998,"5min":1.135,"15min":1.135},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:24:49.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:24:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.718+0000 7fb0cb3eb700 1 Processor -- start 2024-05-08T23:24:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.718+0000 7fb0cb3eb700 1 -- start start 2024-05-08T23:24:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.719+0000 7fb0cb3eb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0c4155cd0 0x7fb0c415fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.719+0000 7fb0cb3eb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb0c406f970 0x7fb0c415c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.719+0000 7fb0cb3eb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c415f810 con 0x7fb0c40cc530 2024-05-08T23:24:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.719+0000 7fb0cb3eb700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c41560b0 con 0x7fb0c4155cd0 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.719+0000 7fb0cb3eb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c415cb90 con 0x7fb0c406f970 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.719+0000 7fb0c9187700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0c4155cd0 0x7fb0c415fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.719+0000 7fb0c9187700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0c4155cd0 0x7fb0c415fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57832/0 (socket says 172.21.15.5:57832) 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.719+0000 7fb0c9187700 1 -- 172.21.15.5:0/1029539246 learned_addr learned my addr 172.21.15.5:0/1029539246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0c8986700 1 --2- 172.21.15.5:0/1029539246 >> v2:172.21.15.49:3300/0 conn(0x7fb0c406f970 0x7fb0c415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0c9187700 1 -- 172.21.15.5:0/1029539246 >> v2:172.21.15.49:3300/0 conn(0x7fb0c406f970 msgr2=0x7fb0c415c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0c9187700 1 --2- 172.21.15.5:0/1029539246 >> v2:172.21.15.49:3300/0 conn(0x7fb0c406f970 0x7fb0c415c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0c9187700 1 -- 172.21.15.5:0/1029539246 >> v1:172.21.15.49:6789/0 conn(0x7fb0c40cc530 legacy=0x7fb0c40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0c9187700 1 -- 172.21.15.5:0/1029539246 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c40cbea0 con 0x7fb0c4155cd0 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0c9187700 1 --2- 172.21.15.5:0/1029539246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c4155cd0 0x7fb0c415fca0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40af6f0 tx=0x7fb0b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfc0a79303ab62c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0bbfff700 1 -- 172.21.15.5:0/1029539246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4011830 con 0x7fb0c4155cd0 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0bbfff700 1 -- 172.21.15.5:0/1029539246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b4011990 con 0x7fb0c4155cd0 2024-05-08T23:24:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.720+0000 7fb0bbfff700 1 -- 172.21.15.5:0/1029539246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4011d20 con 0x7fb0c4155cd0 2024-05-08T23:24:49.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.721+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/1029539246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c4155cd0 msgr2=0x7fb0c415fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.721+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/1029539246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c4155cd0 0x7fb0c415fca0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40af6f0 tx=0x7fb0b4009bd0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.721+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/1029539246 shutdown_connections 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.721+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/1029539246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c4155cd0 0x7fb0c415fca0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.721+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/1029539246 >> v2:172.21.15.49:3300/0 conn(0x7fb0c406f970 0x7fb0c415c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.721+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/1029539246 >> 172.21.15.5:0/1029539246 conn(0x7fb0c40cfce0 msgr2=0x7fb0c40d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.721+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/1029539246 shutdown_connections 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.721+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/1029539246 wait complete. 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.723+0000 7fb0cb3eb700 1 Processor -- start 2024-05-08T23:24:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.723+0000 7fb0cb3eb700 1 -- start start 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0cb3eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c406f970 0x7fb0c414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0cb3eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0c40cc530 0x7fb0c415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0cb3eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0c4155cd0 0x7fb0c41461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0cb3eb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4158090 con 0x7fb0c406f970 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0cb3eb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4158370 con 0x7fb0c4155cd0 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0cb3eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4158650 con 0x7fb0c40cc530 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c406f970 0x7fb0c414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c406f970 0x7fb0c414f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57842/0 (socket says 172.21.15.5:57842) 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 -- 172.21.15.5:0/279620315 learned_addr learned my addr 172.21.15.5:0/279620315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c8986700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0c40cc530 0x7fb0c415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9988700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0c4155cd0 0x7fb0c41461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 -- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0c40cc530 msgr2=0x7fb0c415aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0c40cc530 0x7fb0c415aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 -- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0c4155cd0 msgr2=0x7fb0c41461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0c4155cd0 0x7fb0c41461f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 -- 172.21.15.5:0/279620315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4009550 con 0x7fb0c406f970 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c8986700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0c40cc530 0x7fb0c415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.724+0000 7fb0c9187700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c406f970 0x7fb0c414f090 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009c10 tx=0x7fb0b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.725+0000 7fb0b9ffb700 1 -- 172.21.15.5:0/279620315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4004590 con 0x7fb0c406f970 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.725+0000 7fb0b9ffb700 1 -- 172.21.15.5:0/279620315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b401e070 con 0x7fb0c406f970 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.725+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c4158930 con 0x7fb0c406f970 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.725+0000 7fb0b9ffb700 1 -- 172.21.15.5:0/279620315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40038e0 con 0x7fb0c406f970 2024-05-08T23:24:49.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.725+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c414f5d0 con 0x7fb0c406f970 2024-05-08T23:24:49.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.726+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c4068000 con 0x7fb0c406f970 2024-05-08T23:24:49.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.726+0000 7fb0b9ffb700 1 -- 172.21.15.5:0/279620315 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb0b4003a40 con 0x7fb0c406f970 2024-05-08T23:24:49.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.727+0000 7fb0b9ffb700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0a806ac90 0x7fb0a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:49.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.727+0000 7fb0b9ffb700 1 -- 172.21.15.5:0/279620315 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0b408a280 con 0x7fb0c406f970 2024-05-08T23:24:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.727+0000 7fb0c8986700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0a806ac90 0x7fb0a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.728+0000 7fb0c8986700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0a806ac90 0x7fb0a806d150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb0c41573c0 tx=0x7fb0c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.731+0000 7fb0b9ffb700 1 -- 172.21.15.5:0/279620315 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb0b4055450 con 0x7fb0c406f970 2024-05-08T23:24:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.943+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0c415f4a0 con 0x7fb0a806ac90 2024-05-08T23:24:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.952+0000 7fb0b9ffb700 1 -- 172.21.15.5:0/279620315 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289529 (secure 0 0 0) 0x7fb0c0004030 con 0x7fb0a806ac90 2024-05-08T23:24:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0a806ac90 msgr2=0x7fb0a806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0a806ac90 0x7fb0a806d150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb0c41573c0 tx=0x7fb0c0008040 comp rx=0 tx=0).stop 2024-05-08T23:24:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c406f970 msgr2=0x7fb0c414f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c406f970 0x7fb0c414f090 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009c10 tx=0x7fb0b4004750 comp rx=0 tx=0).stop 2024-05-08T23:24:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 shutdown_connections 2024-05-08T23:24:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0a806ac90 0x7fb0a806d150 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0c4155cd0 0x7fb0c41461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c406f970 0x7fb0c414f090 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 --2- 172.21.15.5:0/279620315 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0c40cc530 0x7fb0c415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:49.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.955+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 >> 172.21.15.5:0/279620315 conn(0x7fb0c40cfce0 msgr2=0x7fb0c406ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:49.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.956+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 shutdown_connections 2024-05-08T23:24:49.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:49.956+0000 7fb0cb3eb700 1 -- 172.21.15.5:0/279620315 wait complete. 2024-05-08T23:24:49.960 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:24:49.969 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-05-08T23:24:49.348024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223040,"kb_used_data":6820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751680,"statfs":{"total":773094113280,"available":772865720320,"internally_reserved":0,"allocated":6983680,"data_stored":3600712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":221236789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.533517"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292222+0000","last_change":"2024-05-08T23:24:45.311564+0000","last_active":"2024-05-08T23:24:46.292222+0000","last_peered":"2024-05-08T23:24:46.292222+0000","last_clean":"2024-05-08T23:24:46.292222+0000","last_became_active":"2024-05-08T23:24:45.311261+0000","last_became_peered":"2024-05-08T23:24:45.311261+0000","last_unstale":"2024-05-08T23:24:46.292222+0000","last_undegraded":"2024-05-08T23:24:46.292222+0000","last_fullsized":"2024-05-08T23:24:46.292222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:29.592077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292349+0000","last_change":"2024-05-08T23:24:45.317600+0000","last_active":"2024-05-08T23:24:46.292349+0000","last_peered":"2024-05-08T23:24:46.292349+0000","last_clean":"2024-05-08T23:24:46.292349+0000","last_became_active":"2024-05-08T23:24:45.317327+0000","last_became_peered":"2024-05-08T23:24:45.317327+0000","last_unstale":"2024-05-08T23:24:46.292349+0000","last_undegraded":"2024-05-08T23:24:46.292349+0000","last_fullsized":"2024-05-08T23:24:46.292349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:10.154887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991333+0000","last_change":"2024-05-08T23:24:45.313722+0000","last_active":"2024-05-08T23:24:46.991333+0000","last_peered":"2024-05-08T23:24:46.991333+0000","last_clean":"2024-05-08T23:24:46.991333+0000","last_became_active":"2024-05-08T23:24:45.310587+0000","last_became_peered":"2024-05-08T23:24:45.310587+0000","last_unstale":"2024-05-08T23:24:46.991333+0000","last_undegraded":"2024-05-08T23:24:46.991333+0000","last_fullsized":"2024-05-08T23:24:46.991333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:43.893324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292404+0000","last_change":"2024-05-08T23:24:45.306531+0000","last_active":"2024-05-08T23:24:46.292404+0000","last_peered":"2024-05-08T23:24:46.292404+0000","last_clean":"2024-05-08T23:24:46.292404+0000","last_became_active":"2024-05-08T23:24:45.306250+0000","last_became_peered":"2024-05-08T23:24:45.306250+0000","last_unstale":"2024-05-08T23:24:46.292404+0000","last_undegraded":"2024-05-08T23:24:46.292404+0000","last_fullsized":"2024-05-08T23:24:46.292404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:03.702310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291733+0000","last_change":"2024-05-08T23:24:45.305983+0000","last_active":"2024-05-08T23:24:46.291733+0000","last_peered":"2024-05-08T23:24:46.291733+0000","last_clean":"2024-05-08T23:24:46.291733+0000","last_became_active":"2024-05-08T23:24:45.305099+0000","last_became_peered":"2024-05-08T23:24:45.305099+0000","last_unstale":"2024-05-08T23:24:46.291733+0000","last_undegraded":"2024-05-08T23:24:46.291733+0000","last_fullsized":"2024-05-08T23:24:46.291733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:01.505793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990693+0000","last_change":"2024-05-08T23:24:45.305791+0000","last_active":"2024-05-08T23:24:46.990693+0000","last_peered":"2024-05-08T23:24:46.990693+0000","last_clean":"2024-05-08T23:24:46.990693+0000","last_became_active":"2024-05-08T23:24:45.305571+0000","last_became_peered":"2024-05-08T23:24:45.305571+0000","last_unstale":"2024-05-08T23:24:46.990693+0000","last_undegraded":"2024-05-08T23:24:46.990693+0000","last_fullsized":"2024-05-08T23:24:46.990693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:43.893324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991529+0000","last_change":"2024-05-08T23:24:45.297285+0000","last_active":"2024-05-08T23:24:46.991529+0000","last_peered":"2024-05-08T23:24:46.991529+0000","last_clean":"2024-05-08T23:24:46.991529+0000","last_became_active":"2024-05-08T23:24:45.296999+0000","last_became_peered":"2024-05-08T23:24:45.296999+0000","last_unstale":"2024-05-08T23:24:46.991529+0000","last_undegraded":"2024-05-08T23:24:46.991529+0000","last_fullsized":"2024-05-08T23:24:46.991529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:24:39.096348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291769+0000","last_change":"2024-05-08T23:24:45.301330+0000","last_active":"2024-05-08T23:24:46.291769+0000","last_peered":"2024-05-08T23:24:46.291769+0000","last_clean":"2024-05-08T23:24:46.291769+0000","last_became_active":"2024-05-08T23:24:45.300996+0000","last_became_peered":"2024-05-08T23:24:45.300996+0000","last_unstale":"2024-05-08T23:24:46.291769+0000","last_undegraded":"2024-05-08T23:24:46.291769+0000","last_fullsized":"2024-05-08T23:24:46.291769+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:26.419489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992117+0000","last_change":"2024-05-08T23:24:45.292468+0000","last_active":"2024-05-08T23:24:46.992117+0000","last_peered":"2024-05-08T23:24:46.992117+0000","last_clean":"2024-05-08T23:24:46.992117+0000","last_became_active":"2024-05-08T23:24:45.292169+0000","last_became_peered":"2024-05-08T23:24:45.292169+0000","last_unstale":"2024-05-08T23:24:46.992117+0000","last_undegraded":"2024-05-08T23:24:46.992117+0000","last_fullsized":"2024-05-08T23:24:46.992117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:59.820193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991170+0000","last_change":"2024-05-08T23:24:45.301478+0000","last_active":"2024-05-08T23:24:46.991170+0000","last_peered":"2024-05-08T23:24:46.991170+0000","last_clean":"2024-05-08T23:24:46.991170+0000","last_became_active":"2024-05-08T23:24:45.301203+0000","last_became_peered":"2024-05-08T23:24:45.301203+0000","last_unstale":"2024-05-08T23:24:46.991170+0000","last_undegraded":"2024-05-08T23:24:46.991170+0000","last_fullsized":"2024-05-08T23:24:46.991170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:25.697853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991032+0000","last_change":"2024-05-08T23:24:45.314582+0000","last_active":"2024-05-08T23:24:46.991032+0000","last_peered":"2024-05-08T23:24:46.991032+0000","last_clean":"2024-05-08T23:24:46.991032+0000","last_became_active":"2024-05-08T23:24:45.314355+0000","last_became_peered":"2024-05-08T23:24:45.314355+0000","last_unstale":"2024-05-08T23:24:46.991032+0000","last_undegraded":"2024-05-08T23:24:46.991032+0000","last_fullsized":"2024-05-08T23:24:46.991032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:33.599117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292529+0000","last_change":"2024-05-08T23:24:45.301823+0000","last_active":"2024-05-08T23:24:46.292529+0000","last_peered":"2024-05-08T23:24:46.292529+0000","last_clean":"2024-05-08T23:24:46.292529+0000","last_became_active":"2024-05-08T23:24:45.301313+0000","last_became_peered":"2024-05-08T23:24:45.301313+0000","last_unstale":"2024-05-08T23:24:46.292529+0000","last_undegraded":"2024-05-08T23:24:46.292529+0000","last_fullsized":"2024-05-08T23:24:46.292529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:53.159272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991471+0000","last_change":"2024-05-08T23:24:45.295795+0000","last_active":"2024-05-08T23:24:46.991471+0000","last_peered":"2024-05-08T23:24:46.991471+0000","last_clean":"2024-05-08T23:24:46.991471+0000","last_became_active":"2024-05-08T23:24:45.295263+0000","last_became_peered":"2024-05-08T23:24:45.295263+0000","last_unstale":"2024-05-08T23:24:46.991471+0000","last_undegraded":"2024-05-08T23:24:46.991471+0000","last_fullsized":"2024-05-08T23:24:46.991471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:10.130820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991602+0000","last_change":"2024-05-08T23:24:45.305903+0000","last_active":"2024-05-08T23:24:46.991602+0000","last_peered":"2024-05-08T23:24:46.991602+0000","last_clean":"2024-05-08T23:24:46.991602+0000","last_became_active":"2024-05-08T23:24:45.305473+0000","last_became_peered":"2024-05-08T23:24:45.305473+0000","last_unstale":"2024-05-08T23:24:46.991602+0000","last_undegraded":"2024-05-08T23:24:46.991602+0000","last_fullsized":"2024-05-08T23:24:46.991602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:41.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991645+0000","last_change":"2024-05-08T23:24:45.305318+0000","last_active":"2024-05-08T23:24:46.991645+0000","last_peered":"2024-05-08T23:24:46.991645+0000","last_clean":"2024-05-08T23:24:46.991645+0000","last_became_active":"2024-05-08T23:24:45.305004+0000","last_became_peered":"2024-05-08T23:24:45.305004+0000","last_unstale":"2024-05-08T23:24:46.991645+0000","last_undegraded":"2024-05-08T23:24:46.991645+0000","last_fullsized":"2024-05-08T23:24:46.991645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:32.833314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991118+0000","last_change":"2024-05-08T23:24:45.302122+0000","last_active":"2024-05-08T23:24:46.991118+0000","last_peered":"2024-05-08T23:24:46.991118+0000","last_clean":"2024-05-08T23:24:46.991118+0000","last_became_active":"2024-05-08T23:24:45.301855+0000","last_became_peered":"2024-05-08T23:24:45.301855+0000","last_unstale":"2024-05-08T23:24:46.991118+0000","last_undegraded":"2024-05-08T23:24:46.991118+0000","last_fullsized":"2024-05-08T23:24:46.991118+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:26.099058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292335+0000","last_change":"2024-05-08T23:24:45.310506+0000","last_active":"2024-05-08T23:24:46.292335+0000","last_peered":"2024-05-08T23:24:46.292335+0000","last_clean":"2024-05-08T23:24:46.292335+0000","last_became_active":"2024-05-08T23:24:45.309405+0000","last_became_peered":"2024-05-08T23:24:45.309405+0000","last_unstale":"2024-05-08T23:24:46.292335+0000","last_undegraded":"2024-05-08T23:24:46.292335+0000","last_fullsized":"2024-05-08T23:24:46.292335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:49.588979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991910+0000","last_change":"2024-05-08T23:24:45.313266+0000","last_active":"2024-05-08T23:24:46.991910+0000","last_peered":"2024-05-08T23:24:46.991910+0000","last_clean":"2024-05-08T23:24:46.991910+0000","last_became_active":"2024-05-08T23:24:45.312991+0000","last_became_peered":"2024-05-08T23:24:45.312991+0000","last_unstale":"2024-05-08T23:24:46.991910+0000","last_undegraded":"2024-05-08T23:24:46.991910+0000","last_fullsized":"2024-05-08T23:24:46.991910+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:40.712218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292132+0000","last_change":"2024-05-08T23:24:45.308974+0000","last_active":"2024-05-08T23:24:46.292132+0000","last_peered":"2024-05-08T23:24:46.292132+0000","last_clean":"2024-05-08T23:24:46.292132+0000","last_became_active":"2024-05-08T23:24:45.308691+0000","last_became_peered":"2024-05-08T23:24:45.308691+0000","last_unstale":"2024-05-08T23:24:46.292132+0000","last_undegraded":"2024-05-08T23:24:46.292132+0000","last_fullsized":"2024-05-08T23:24:46.292132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:42.812590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292754+0000","last_change":"2024-05-08T23:24:45.309185+0000","last_active":"2024-05-08T23:24:46.292754+0000","last_peered":"2024-05-08T23:24:46.292754+0000","last_clean":"2024-05-08T23:24:46.292754+0000","last_became_active":"2024-05-08T23:24:45.308135+0000","last_became_peered":"2024-05-08T23:24:45.308135+0000","last_unstale":"2024-05-08T23:24:46.292754+0000","last_undegraded":"2024-05-08T23:24:46.292754+0000","last_fullsized":"2024-05-08T23:24:46.292754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:29.431050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991773+0000","last_change":"2024-05-08T23:24:45.309924+0000","last_active":"2024-05-08T23:24:46.991773+0000","last_peered":"2024-05-08T23:24:46.991773+0000","last_clean":"2024-05-08T23:24:46.991773+0000","last_became_active":"2024-05-08T23:24:45.309608+0000","last_became_peered":"2024-05-08T23:24:45.309608+0000","last_unstale":"2024-05-08T23:24:46.991773+0000","last_undegraded":"2024-05-08T23:24:46.991773+0000","last_fullsized":"2024-05-08T23:24:46.991773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:02.893488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291882+0000","last_change":"2024-05-08T23:24:45.308180+0000","last_active":"2024-05-08T23:24:46.291882+0000","last_peered":"2024-05-08T23:24:46.291882+0000","last_clean":"2024-05-08T23:24:46.291882+0000","last_became_active":"2024-05-08T23:24:45.307850+0000","last_became_peered":"2024-05-08T23:24:45.307850+0000","last_unstale":"2024-05-08T23:24:46.291882+0000","last_undegraded":"2024-05-08T23:24:46.291882+0000","last_fullsized":"2024-05-08T23:24:46.291882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:59.384496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992227+0000","last_change":"2024-05-08T23:24:45.314435+0000","last_active":"2024-05-08T23:24:46.992227+0000","last_peered":"2024-05-08T23:24:46.992227+0000","last_clean":"2024-05-08T23:24:46.992227+0000","last_became_active":"2024-05-08T23:24:45.314144+0000","last_became_peered":"2024-05-08T23:24:45.314144+0000","last_unstale":"2024-05-08T23:24:46.992227+0000","last_undegraded":"2024-05-08T23:24:46.992227+0000","last_fullsized":"2024-05-08T23:24:46.992227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:51.926879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991272+0000","last_change":"2024-05-08T23:24:45.304612+0000","last_active":"2024-05-08T23:24:46.991272+0000","last_peered":"2024-05-08T23:24:46.991272+0000","last_clean":"2024-05-08T23:24:46.991272+0000","last_became_active":"2024-05-08T23:24:45.304237+0000","last_became_peered":"2024-05-08T23:24:45.304237+0000","last_unstale":"2024-05-08T23:24:46.991272+0000","last_undegraded":"2024-05-08T23:24:46.991272+0000","last_fullsized":"2024-05-08T23:24:46.991272+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:21.608208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292304+0000","last_change":"2024-05-08T23:24:45.317893+0000","last_active":"2024-05-08T23:24:46.292304+0000","last_peered":"2024-05-08T23:24:46.292304+0000","last_clean":"2024-05-08T23:24:46.292304+0000","last_became_active":"2024-05-08T23:24:45.317611+0000","last_became_peered":"2024-05-08T23:24:45.317611+0000","last_unstale":"2024-05-08T23:24:46.292304+0000","last_undegraded":"2024-05-08T23:24:46.292304+0000","last_fullsized":"2024-05-08T23:24:46.292304+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:35.959761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292480+0000","last_change":"2024-05-08T23:24:45.296165+0000","last_active":"2024-05-08T23:24:46.292480+0000","last_peered":"2024-05-08T23:24:46.292480+0000","last_clean":"2024-05-08T23:24:46.292480+0000","last_became_active":"2024-05-08T23:24:45.295870+0000","last_became_peered":"2024-05-08T23:24:45.295870+0000","last_unstale":"2024-05-08T23:24:46.292480+0000","last_undegraded":"2024-05-08T23:24:46.292480+0000","last_fullsized":"2024-05-08T23:24:46.292480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:41.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990841+0000","last_change":"2024-05-08T23:24:45.298086+0000","last_active":"2024-05-08T23:24:46.990841+0000","last_peered":"2024-05-08T23:24:46.990841+0000","last_clean":"2024-05-08T23:24:46.990841+0000","last_became_active":"2024-05-08T23:24:45.297804+0000","last_became_peered":"2024-05-08T23:24:45.297804+0000","last_unstale":"2024-05-08T23:24:46.990841+0000","last_undegraded":"2024-05-08T23:24:46.990841+0000","last_fullsized":"2024-05-08T23:24:46.990841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:47.956169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292024+0000","last_change":"2024-05-08T23:24:45.303621+0000","last_active":"2024-05-08T23:24:46.292024+0000","last_peered":"2024-05-08T23:24:46.292024+0000","last_clean":"2024-05-08T23:24:46.292024+0000","last_became_active":"2024-05-08T23:24:45.303356+0000","last_became_peered":"2024-05-08T23:24:45.303356+0000","last_unstale":"2024-05-08T23:24:46.292024+0000","last_undegraded":"2024-05-08T23:24:46.292024+0000","last_fullsized":"2024-05-08T23:24:46.292024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.786227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292972+0000","last_change":"2024-05-08T23:24:45.300248+0000","last_active":"2024-05-08T23:24:46.292972+0000","last_peered":"2024-05-08T23:24:46.292972+0000","last_clean":"2024-05-08T23:24:46.292972+0000","last_became_active":"2024-05-08T23:24:45.299876+0000","last_became_peered":"2024-05-08T23:24:45.299876+0000","last_unstale":"2024-05-08T23:24:46.292972+0000","last_undegraded":"2024-05-08T23:24:46.292972+0000","last_fullsized":"2024-05-08T23:24:46.292972+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:00.499202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991025+0000","last_change":"2024-05-08T23:24:45.310126+0000","last_active":"2024-05-08T23:24:46.991025+0000","last_peered":"2024-05-08T23:24:46.991025+0000","last_clean":"2024-05-08T23:24:46.991025+0000","last_became_active":"2024-05-08T23:24:45.309811+0000","last_became_peered":"2024-05-08T23:24:45.309811+0000","last_unstale":"2024-05-08T23:24:46.991025+0000","last_undegraded":"2024-05-08T23:24:46.991025+0000","last_fullsized":"2024-05-08T23:24:46.991025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:47.956169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291552+0000","last_change":"2024-05-08T23:24:45.316988+0000","last_active":"2024-05-08T23:24:46.291552+0000","last_peered":"2024-05-08T23:24:46.291552+0000","last_clean":"2024-05-08T23:24:46.291552+0000","last_became_active":"2024-05-08T23:24:45.316658+0000","last_became_peered":"2024-05-08T23:24:45.316658+0000","last_unstale":"2024-05-08T23:24:46.291552+0000","last_undegraded":"2024-05-08T23:24:46.291552+0000","last_fullsized":"2024-05-08T23:24:46.291552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:53.801148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991376+0000","last_change":"2024-05-08T23:24:45.302835+0000","last_active":"2024-05-08T23:24:46.991376+0000","last_peered":"2024-05-08T23:24:46.991376+0000","last_clean":"2024-05-08T23:24:46.991376+0000","last_became_active":"2024-05-08T23:24:45.302546+0000","last_became_peered":"2024-05-08T23:24:45.302546+0000","last_unstale":"2024-05-08T23:24:46.991376+0000","last_undegraded":"2024-05-08T23:24:46.991376+0000","last_fullsized":"2024-05-08T23:24:46.991376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:53.356137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991468+0000","last_change":"2024-05-08T23:24:45.310626+0000","last_active":"2024-05-08T23:24:46.991468+0000","last_peered":"2024-05-08T23:24:46.991468+0000","last_clean":"2024-05-08T23:24:46.991468+0000","last_became_active":"2024-05-08T23:24:45.310331+0000","last_became_peered":"2024-05-08T23:24:45.310331+0000","last_unstale":"2024-05-08T23:24:46.991468+0000","last_undegraded":"2024-05-08T23:24:46.991468+0000","last_fullsized":"2024-05-08T23:24:46.991468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:02.893488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991563+0000","last_change":"2024-05-08T23:24:44.833160+0000","last_active":"2024-05-08T23:24:46.991563+0000","last_peered":"2024-05-08T23:24:46.991563+0000","last_clean":"2024-05-08T23:24:46.991563+0000","last_became_active":"2024-05-08T23:24:44.832856+0000","last_became_peered":"2024-05-08T23:24:44.832856+0000","last_unstale":"2024-05-08T23:24:46.991563+0000","last_undegraded":"2024-05-08T23:24:46.991563+0000","last_fullsized":"2024-05-08T23:24:46.991563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:28.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991446+0000","last_change":"2024-05-08T23:24:45.315842+0000","last_active":"2024-05-08T23:24:46.991446+0000","last_peered":"2024-05-08T23:24:46.991446+0000","last_clean":"2024-05-08T23:24:46.991446+0000","last_became_active":"2024-05-08T23:24:45.315588+0000","last_became_peered":"2024-05-08T23:24:45.315588+0000","last_unstale":"2024-05-08T23:24:46.991446+0000","last_undegraded":"2024-05-08T23:24:46.991446+0000","last_fullsized":"2024-05-08T23:24:46.991446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:13.847361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292436+0000","last_change":"2024-05-08T23:24:44.834221+0000","last_active":"2024-05-08T23:24:46.292436+0000","last_peered":"2024-05-08T23:24:46.292436+0000","last_clean":"2024-05-08T23:24:46.292436+0000","last_became_active":"2024-05-08T23:24:44.834012+0000","last_became_peered":"2024-05-08T23:24:44.834012+0000","last_unstale":"2024-05-08T23:24:46.292436+0000","last_undegraded":"2024-05-08T23:24:46.292436+0000","last_fullsized":"2024-05-08T23:24:46.292436+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:45.857534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991245+0000","last_change":"2024-05-08T23:24:45.305306+0000","last_active":"2024-05-08T23:24:46.991245+0000","last_peered":"2024-05-08T23:24:46.991245+0000","last_clean":"2024-05-08T23:24:46.991245+0000","last_became_active":"2024-05-08T23:24:45.304956+0000","last_became_peered":"2024-05-08T23:24:45.304956+0000","last_unstale":"2024-05-08T23:24:46.991245+0000","last_undegraded":"2024-05-08T23:24:46.991245+0000","last_fullsized":"2024-05-08T23:24:46.991245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:33.903726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991086+0000","last_change":"2024-05-08T23:24:44.838890+0000","last_active":"2024-05-08T23:24:46.991086+0000","last_peered":"2024-05-08T23:24:46.991086+0000","last_clean":"2024-05-08T23:24:46.991086+0000","last_became_active":"2024-05-08T23:24:44.838553+0000","last_became_peered":"2024-05-08T23:24:44.838553+0000","last_unstale":"2024-05-08T23:24:46.991086+0000","last_undegraded":"2024-05-08T23:24:46.991086+0000","last_fullsized":"2024-05-08T23:24:46.991086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:31.828404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990969+0000","last_change":"2024-05-08T23:24:45.297758+0000","last_active":"2024-05-08T23:24:46.990969+0000","last_peered":"2024-05-08T23:24:46.990969+0000","last_clean":"2024-05-08T23:24:46.990969+0000","last_became_active":"2024-05-08T23:24:45.297386+0000","last_became_peered":"2024-05-08T23:24:45.297386+0000","last_unstale":"2024-05-08T23:24:46.990969+0000","last_undegraded":"2024-05-08T23:24:46.990969+0000","last_fullsized":"2024-05-08T23:24:46.990969+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:22.639106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292653+0000","last_change":"2024-05-08T23:24:44.835229+0000","last_active":"2024-05-08T23:24:46.292653+0000","last_peered":"2024-05-08T23:24:46.292653+0000","last_clean":"2024-05-08T23:24:46.292653+0000","last_became_active":"2024-05-08T23:24:44.834835+0000","last_became_peered":"2024-05-08T23:24:44.834835+0000","last_unstale":"2024-05-08T23:24:46.292653+0000","last_undegraded":"2024-05-08T23:24:46.292653+0000","last_fullsized":"2024-05-08T23:24:46.292653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:34.271517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291985+0000","last_change":"2024-05-08T23:24:45.308260+0000","last_active":"2024-05-08T23:24:46.291985+0000","last_peered":"2024-05-08T23:24:46.291985+0000","last_clean":"2024-05-08T23:24:46.291985+0000","last_became_active":"2024-05-08T23:24:45.307438+0000","last_became_peered":"2024-05-08T23:24:45.307438+0000","last_unstale":"2024-05-08T23:24:46.291985+0000","last_undegraded":"2024-05-08T23:24:46.291985+0000","last_fullsized":"2024-05-08T23:24:46.291985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:33.532590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991710+0000","last_change":"2024-05-08T23:24:44.838162+0000","last_active":"2024-05-08T23:24:46.991710+0000","last_peered":"2024-05-08T23:24:46.991710+0000","last_clean":"2024-05-08T23:24:46.991710+0000","last_became_active":"2024-05-08T23:24:44.837809+0000","last_became_peered":"2024-05-08T23:24:44.837809+0000","last_unstale":"2024-05-08T23:24:46.991710+0000","last_undegraded":"2024-05-08T23:24:46.991710+0000","last_fullsized":"2024-05-08T23:24:46.991710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.715143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.293077+0000","last_change":"2024-05-08T23:24:45.310436+0000","last_active":"2024-05-08T23:24:46.293077+0000","last_peered":"2024-05-08T23:24:46.293077+0000","last_clean":"2024-05-08T23:24:46.293077+0000","last_became_active":"2024-05-08T23:24:45.307492+0000","last_became_peered":"2024-05-08T23:24:45.307492+0000","last_unstale":"2024-05-08T23:24:46.293077+0000","last_undegraded":"2024-05-08T23:24:46.293077+0000","last_fullsized":"2024-05-08T23:24:46.293077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:27.414348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990881+0000","last_change":"2024-05-08T23:24:44.833118+0000","last_active":"2024-05-08T23:24:46.990881+0000","last_peered":"2024-05-08T23:24:46.990881+0000","last_clean":"2024-05-08T23:24:46.990881+0000","last_became_active":"2024-05-08T23:24:44.832818+0000","last_became_peered":"2024-05-08T23:24:44.832818+0000","last_unstale":"2024-05-08T23:24:46.990881+0000","last_undegraded":"2024-05-08T23:24:46.990881+0000","last_fullsized":"2024-05-08T23:24:46.990881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:58.815283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991380+0000","last_change":"2024-05-08T23:24:45.300276+0000","last_active":"2024-05-08T23:24:46.991380+0000","last_peered":"2024-05-08T23:24:46.991380+0000","last_clean":"2024-05-08T23:24:46.991380+0000","last_became_active":"2024-05-08T23:24:45.299977+0000","last_became_peered":"2024-05-08T23:24:45.299977+0000","last_unstale":"2024-05-08T23:24:46.991380+0000","last_undegraded":"2024-05-08T23:24:46.991380+0000","last_fullsized":"2024-05-08T23:24:46.991380+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:47.994864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292168+0000","last_change":"2024-05-08T23:24:44.831992+0000","last_active":"2024-05-08T23:24:46.292168+0000","last_peered":"2024-05-08T23:24:46.292168+0000","last_clean":"2024-05-08T23:24:46.292168+0000","last_became_active":"2024-05-08T23:24:44.831730+0000","last_became_peered":"2024-05-08T23:24:44.831730+0000","last_unstale":"2024-05-08T23:24:46.292168+0000","last_undegraded":"2024-05-08T23:24:46.292168+0000","last_fullsized":"2024-05-08T23:24:46.292168+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.715143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292261+0000","last_change":"2024-05-08T23:24:45.302068+0000","last_active":"2024-05-08T23:24:46.292261+0000","last_peered":"2024-05-08T23:24:46.292261+0000","last_clean":"2024-05-08T23:24:46.292261+0000","last_became_active":"2024-05-08T23:24:45.301737+0000","last_became_peered":"2024-05-08T23:24:45.301737+0000","last_unstale":"2024-05-08T23:24:46.292261+0000","last_undegraded":"2024-05-08T23:24:46.292261+0000","last_fullsized":"2024-05-08T23:24:46.292261+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:05.224848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992159+0000","last_change":"2024-05-08T23:24:44.829538+0000","last_active":"2024-05-08T23:24:46.992159+0000","last_peered":"2024-05-08T23:24:46.992159+0000","last_clean":"2024-05-08T23:24:46.992159+0000","last_became_active":"2024-05-08T23:24:44.829223+0000","last_became_peered":"2024-05-08T23:24:44.829223+0000","last_unstale":"2024-05-08T23:24:46.992159+0000","last_undegraded":"2024-05-08T23:24:46.992159+0000","last_fullsized":"2024-05-08T23:24:46.992159+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:33.819270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991987+0000","last_change":"2024-05-08T23:24:45.300174+0000","last_active":"2024-05-08T23:24:46.991987+0000","last_peered":"2024-05-08T23:24:46.991987+0000","last_clean":"2024-05-08T23:24:46.991987+0000","last_became_active":"2024-05-08T23:24:45.299879+0000","last_became_peered":"2024-05-08T23:24:45.299879+0000","last_unstale":"2024-05-08T23:24:46.991987+0000","last_undegraded":"2024-05-08T23:24:46.991987+0000","last_fullsized":"2024-05-08T23:24:46.991987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:58.502436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991846+0000","last_change":"2024-05-08T23:24:44.838767+0000","last_active":"2024-05-08T23:24:46.991846+0000","last_peered":"2024-05-08T23:24:46.991846+0000","last_clean":"2024-05-08T23:24:46.991846+0000","last_became_active":"2024-05-08T23:24:44.838479+0000","last_became_peered":"2024-05-08T23:24:44.838479+0000","last_unstale":"2024-05-08T23:24:46.991846+0000","last_undegraded":"2024-05-08T23:24:46.991846+0000","last_fullsized":"2024-05-08T23:24:46.991846+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:55.904063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992111+0000","last_change":"2024-05-08T23:24:45.317860+0000","last_active":"2024-05-08T23:24:46.992111+0000","last_peered":"2024-05-08T23:24:46.992111+0000","last_clean":"2024-05-08T23:24:46.992111+0000","last_became_active":"2024-05-08T23:24:45.317614+0000","last_became_peered":"2024-05-08T23:24:45.317614+0000","last_unstale":"2024-05-08T23:24:46.992111+0000","last_undegraded":"2024-05-08T23:24:46.992111+0000","last_fullsized":"2024-05-08T23:24:46.992111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:53.605503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292496+0000","last_change":"2024-05-08T23:24:44.838156+0000","last_active":"2024-05-08T23:24:46.292496+0000","last_peered":"2024-05-08T23:24:46.292496+0000","last_clean":"2024-05-08T23:24:46.292496+0000","last_became_active":"2024-05-08T23:24:44.837932+0000","last_became_peered":"2024-05-08T23:24:44.837932+0000","last_unstale":"2024-05-08T23:24:46.292496+0000","last_undegraded":"2024-05-08T23:24:46.292496+0000","last_fullsized":"2024-05-08T23:24:46.292496+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:42.888414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990742+0000","last_change":"2024-05-08T23:24:45.314187+0000","last_active":"2024-05-08T23:24:46.990742+0000","last_peered":"2024-05-08T23:24:46.990742+0000","last_clean":"2024-05-08T23:24:46.990742+0000","last_became_active":"2024-05-08T23:24:45.309482+0000","last_became_peered":"2024-05-08T23:24:45.309482+0000","last_unstale":"2024-05-08T23:24:46.990742+0000","last_undegraded":"2024-05-08T23:24:46.990742+0000","last_fullsized":"2024-05-08T23:24:46.990742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:26.006592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991638+0000","last_change":"2024-05-08T23:24:44.835902+0000","last_active":"2024-05-08T23:24:46.991638+0000","last_peered":"2024-05-08T23:24:46.991638+0000","last_clean":"2024-05-08T23:24:46.991638+0000","last_became_active":"2024-05-08T23:24:44.835287+0000","last_became_peered":"2024-05-08T23:24:44.835287+0000","last_unstale":"2024-05-08T23:24:46.991638+0000","last_undegraded":"2024-05-08T23:24:46.991638+0000","last_fullsized":"2024-05-08T23:24:46.991638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:42.806659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991433+0000","last_change":"2024-05-08T23:24:45.317494+0000","last_active":"2024-05-08T23:24:46.991433+0000","last_peered":"2024-05-08T23:24:46.991433+0000","last_clean":"2024-05-08T23:24:46.991433+0000","last_became_active":"2024-05-08T23:24:45.317199+0000","last_became_peered":"2024-05-08T23:24:45.317199+0000","last_unstale":"2024-05-08T23:24:46.991433+0000","last_undegraded":"2024-05-08T23:24:46.991433+0000","last_fullsized":"2024-05-08T23:24:46.991433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:53.159272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992084+0000","last_change":"2024-05-08T23:24:44.831798+0000","last_active":"2024-05-08T23:24:46.992084+0000","last_peered":"2024-05-08T23:24:46.992084+0000","last_clean":"2024-05-08T23:24:46.992084+0000","last_became_active":"2024-05-08T23:24:44.831506+0000","last_became_peered":"2024-05-08T23:24:44.831506+0000","last_unstale":"2024-05-08T23:24:46.992084+0000","last_undegraded":"2024-05-08T23:24:46.992084+0000","last_fullsized":"2024-05-08T23:24:46.992084+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:46.989954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292026+0000","last_change":"2024-05-08T23:24:45.303385+0000","last_active":"2024-05-08T23:24:46.292026+0000","last_peered":"2024-05-08T23:24:46.292026+0000","last_clean":"2024-05-08T23:24:46.292026+0000","last_became_active":"2024-05-08T23:24:45.303116+0000","last_became_peered":"2024-05-08T23:24:45.303116+0000","last_unstale":"2024-05-08T23:24:46.292026+0000","last_undegraded":"2024-05-08T23:24:46.292026+0000","last_fullsized":"2024-05-08T23:24:46.292026+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:47:45.766129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.293123+0000","last_change":"2024-05-08T23:24:44.838808+0000","last_active":"2024-05-08T23:24:46.293123+0000","last_peered":"2024-05-08T23:24:46.293123+0000","last_clean":"2024-05-08T23:24:46.293123+0000","last_became_active":"2024-05-08T23:24:44.838527+0000","last_became_peered":"2024-05-08T23:24:44.838527+0000","last_unstale":"2024-05-08T23:24:46.293123+0000","last_undegraded":"2024-05-08T23:24:46.293123+0000","last_fullsized":"2024-05-08T23:24:46.293123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:46.951259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991883+0000","last_change":"2024-05-08T23:24:45.309437+0000","last_active":"2024-05-08T23:24:46.991883+0000","last_peered":"2024-05-08T23:24:46.991883+0000","last_clean":"2024-05-08T23:24:46.991883+0000","last_became_active":"2024-05-08T23:24:45.309124+0000","last_became_peered":"2024-05-08T23:24:45.309124+0000","last_unstale":"2024-05-08T23:24:46.991883+0000","last_undegraded":"2024-05-08T23:24:46.991883+0000","last_fullsized":"2024-05-08T23:24:46.991883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:03.702310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991073+0000","last_change":"2024-05-08T23:24:44.831589+0000","last_active":"2024-05-08T23:24:46.991073+0000","last_peered":"2024-05-08T23:24:46.991073+0000","last_clean":"2024-05-08T23:24:46.991073+0000","last_became_active":"2024-05-08T23:24:44.831249+0000","last_became_peered":"2024-05-08T23:24:44.831249+0000","last_unstale":"2024-05-08T23:24:46.991073+0000","last_undegraded":"2024-05-08T23:24:46.991073+0000","last_fullsized":"2024-05-08T23:24:46.991073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:25.001682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990883+0000","last_change":"2024-05-08T23:24:45.311852+0000","last_active":"2024-05-08T23:24:46.990883+0000","last_peered":"2024-05-08T23:24:46.990883+0000","last_clean":"2024-05-08T23:24:46.990883+0000","last_became_active":"2024-05-08T23:24:45.311581+0000","last_became_peered":"2024-05-08T23:24:45.311581+0000","last_unstale":"2024-05-08T23:24:46.990883+0000","last_undegraded":"2024-05-08T23:24:46.990883+0000","last_fullsized":"2024-05-08T23:24:46.990883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:58:16.684695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292078+0000","last_change":"2024-05-08T23:24:44.835564+0000","last_active":"2024-05-08T23:24:46.292078+0000","last_peered":"2024-05-08T23:24:46.292078+0000","last_clean":"2024-05-08T23:24:46.292078+0000","last_became_active":"2024-05-08T23:24:44.834783+0000","last_became_peered":"2024-05-08T23:24:44.834783+0000","last_unstale":"2024-05-08T23:24:46.292078+0000","last_undegraded":"2024-05-08T23:24:46.292078+0000","last_fullsized":"2024-05-08T23:24:46.292078+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:42.806659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992013+0000","last_change":"2024-05-08T23:24:45.313273+0000","last_active":"2024-05-08T23:24:46.992013+0000","last_peered":"2024-05-08T23:24:46.992013+0000","last_clean":"2024-05-08T23:24:46.992013+0000","last_became_active":"2024-05-08T23:24:45.313015+0000","last_became_peered":"2024-05-08T23:24:45.313015+0000","last_unstale":"2024-05-08T23:24:46.992013+0000","last_undegraded":"2024-05-08T23:24:46.992013+0000","last_fullsized":"2024-05-08T23:24:46.992013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:34.043367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991066+0000","last_change":"2024-05-08T23:24:44.837004+0000","last_active":"2024-05-08T23:24:46.991066+0000","last_peered":"2024-05-08T23:24:46.991066+0000","last_clean":"2024-05-08T23:24:46.991066+0000","last_became_active":"2024-05-08T23:24:44.836660+0000","last_became_peered":"2024-05-08T23:24:44.836660+0000","last_unstale":"2024-05-08T23:24:46.991066+0000","last_undegraded":"2024-05-08T23:24:46.991066+0000","last_fullsized":"2024-05-08T23:24:46.991066+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:39.535915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992033+0000","last_change":"2024-05-08T23:24:45.310715+0000","last_active":"2024-05-08T23:24:46.992033+0000","last_peered":"2024-05-08T23:24:46.992033+0000","last_clean":"2024-05-08T23:24:46.992033+0000","last_became_active":"2024-05-08T23:24:45.310427+0000","last_became_peered":"2024-05-08T23:24:45.310427+0000","last_unstale":"2024-05-08T23:24:46.992033+0000","last_undegraded":"2024-05-08T23:24:46.992033+0000","last_fullsized":"2024-05-08T23:24:46.992033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:26.099058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292430+0000","last_change":"2024-05-08T23:24:44.839874+0000","last_active":"2024-05-08T23:24:46.292430+0000","last_peered":"2024-05-08T23:24:46.292430+0000","last_clean":"2024-05-08T23:24:46.292430+0000","last_became_active":"2024-05-08T23:24:44.839590+0000","last_became_peered":"2024-05-08T23:24:44.839590+0000","last_unstale":"2024-05-08T23:24:46.292430+0000","last_undegraded":"2024-05-08T23:24:46.292430+0000","last_fullsized":"2024-05-08T23:24:46.292430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:33.038457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292699+0000","last_change":"2024-05-08T23:24:45.310961+0000","last_active":"2024-05-08T23:24:46.292699+0000","last_peered":"2024-05-08T23:24:46.292699+0000","last_clean":"2024-05-08T23:24:46.292699+0000","last_became_active":"2024-05-08T23:24:45.310668+0000","last_became_peered":"2024-05-08T23:24:45.310668+0000","last_unstale":"2024-05-08T23:24:46.292699+0000","last_undegraded":"2024-05-08T23:24:46.292699+0000","last_fullsized":"2024-05-08T23:24:46.292699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:26.599427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991558+0000","last_change":"2024-05-08T23:24:44.836375+0000","last_active":"2024-05-08T23:24:46.991558+0000","last_peered":"2024-05-08T23:24:46.991558+0000","last_clean":"2024-05-08T23:24:46.991558+0000","last_became_active":"2024-05-08T23:24:44.836083+0000","last_became_peered":"2024-05-08T23:24:44.836083+0000","last_unstale":"2024-05-08T23:24:46.991558+0000","last_undegraded":"2024-05-08T23:24:46.991558+0000","last_fullsized":"2024-05-08T23:24:46.991558+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:57.497526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292849+0000","last_change":"2024-05-08T23:24:45.300637+0000","last_active":"2024-05-08T23:24:46.292849+0000","last_peered":"2024-05-08T23:24:46.292849+0000","last_clean":"2024-05-08T23:24:46.292849+0000","last_became_active":"2024-05-08T23:24:45.300276+0000","last_became_peered":"2024-05-08T23:24:45.300276+0000","last_unstale":"2024-05-08T23:24:46.292849+0000","last_undegraded":"2024-05-08T23:24:46.292849+0000","last_fullsized":"2024-05-08T23:24:46.292849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:50.218413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292486+0000","last_change":"2024-05-08T23:24:44.834762+0000","last_active":"2024-05-08T23:24:46.292486+0000","last_peered":"2024-05-08T23:24:46.292486+0000","last_clean":"2024-05-08T23:24:46.292486+0000","last_became_active":"2024-05-08T23:24:44.834456+0000","last_became_peered":"2024-05-08T23:24:44.834456+0000","last_unstale":"2024-05-08T23:24:46.292486+0000","last_undegraded":"2024-05-08T23:24:46.292486+0000","last_fullsized":"2024-05-08T23:24:46.292486+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:55.904063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991530+0000","last_change":"2024-05-08T23:24:45.317531+0000","last_active":"2024-05-08T23:24:46.991530+0000","last_peered":"2024-05-08T23:24:46.991530+0000","last_clean":"2024-05-08T23:24:46.991530+0000","last_became_active":"2024-05-08T23:24:45.317315+0000","last_became_peered":"2024-05-08T23:24:45.317315+0000","last_unstale":"2024-05-08T23:24:46.991530+0000","last_undegraded":"2024-05-08T23:24:46.991530+0000","last_fullsized":"2024-05-08T23:24:46.991530+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:10.734065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991483+0000","last_change":"2024-05-08T23:24:44.839061+0000","last_active":"2024-05-08T23:24:46.991483+0000","last_peered":"2024-05-08T23:24:46.991483+0000","last_clean":"2024-05-08T23:24:46.991483+0000","last_became_active":"2024-05-08T23:24:44.838787+0000","last_became_peered":"2024-05-08T23:24:44.838787+0000","last_unstale":"2024-05-08T23:24:46.991483+0000","last_undegraded":"2024-05-08T23:24:46.991483+0000","last_fullsized":"2024-05-08T23:24:46.991483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:40.805471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292134+0000","last_change":"2024-05-08T23:24:45.303366+0000","last_active":"2024-05-08T23:24:46.292134+0000","last_peered":"2024-05-08T23:24:46.292134+0000","last_clean":"2024-05-08T23:24:46.292134+0000","last_became_active":"2024-05-08T23:24:45.303065+0000","last_became_peered":"2024-05-08T23:24:45.303065+0000","last_unstale":"2024-05-08T23:24:46.292134+0000","last_undegraded":"2024-05-08T23:24:46.292134+0000","last_fullsized":"2024-05-08T23:24:46.292134+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:52.546124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.293184+0000","last_change":"2024-05-08T23:24:44.827652+0000","last_active":"2024-05-08T23:24:46.293184+0000","last_peered":"2024-05-08T23:24:46.293184+0000","last_clean":"2024-05-08T23:24:46.293184+0000","last_became_active":"2024-05-08T23:24:44.827329+0000","last_became_peered":"2024-05-08T23:24:44.827329+0000","last_unstale":"2024-05-08T23:24:46.293184+0000","last_undegraded":"2024-05-08T23:24:46.293184+0000","last_fullsized":"2024-05-08T23:24:46.293184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:01.888578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991030+0000","last_change":"2024-05-08T23:24:45.295718+0000","last_active":"2024-05-08T23:24:46.991030+0000","last_peered":"2024-05-08T23:24:46.991030+0000","last_clean":"2024-05-08T23:24:46.991030+0000","last_became_active":"2024-05-08T23:24:45.295437+0000","last_became_peered":"2024-05-08T23:24:45.295437+0000","last_unstale":"2024-05-08T23:24:46.991030+0000","last_undegraded":"2024-05-08T23:24:46.991030+0000","last_fullsized":"2024-05-08T23:24:46.991030+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:40.712218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990846+0000","last_change":"2024-05-08T23:24:44.838175+0000","last_active":"2024-05-08T23:24:46.990846+0000","last_peered":"2024-05-08T23:24:46.990846+0000","last_clean":"2024-05-08T23:24:46.990846+0000","last_became_active":"2024-05-08T23:24:44.837861+0000","last_became_peered":"2024-05-08T23:24:44.837861+0000","last_unstale":"2024-05-08T23:24:46.990846+0000","last_undegraded":"2024-05-08T23:24:46.990846+0000","last_fullsized":"2024-05-08T23:24:46.990846+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:53.781316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991228+0000","last_change":"2024-05-08T23:24:45.305029+0000","last_active":"2024-05-08T23:24:46.991228+0000","last_peered":"2024-05-08T23:24:46.991228+0000","last_clean":"2024-05-08T23:24:46.991228+0000","last_became_active":"2024-05-08T23:24:45.304732+0000","last_became_peered":"2024-05-08T23:24:45.304732+0000","last_unstale":"2024-05-08T23:24:46.991228+0000","last_undegraded":"2024-05-08T23:24:46.991228+0000","last_fullsized":"2024-05-08T23:24:46.991228+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:46.862444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291968+0000","last_change":"2024-05-08T23:24:44.837412+0000","last_active":"2024-05-08T23:24:46.291968+0000","last_peered":"2024-05-08T23:24:46.291968+0000","last_clean":"2024-05-08T23:24:46.291968+0000","last_became_active":"2024-05-08T23:24:44.837113+0000","last_became_peered":"2024-05-08T23:24:44.837113+0000","last_unstale":"2024-05-08T23:24:46.291968+0000","last_undegraded":"2024-05-08T23:24:46.291968+0000","last_fullsized":"2024-05-08T23:24:46.291968+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:20.603298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292518+0000","last_change":"2024-05-08T23:24:45.306133+0000","last_active":"2024-05-08T23:24:46.292518+0000","last_peered":"2024-05-08T23:24:46.292518+0000","last_clean":"2024-05-08T23:24:46.292518+0000","last_became_active":"2024-05-08T23:24:45.305799+0000","last_became_peered":"2024-05-08T23:24:45.305799+0000","last_unstale":"2024-05-08T23:24:46.292518+0000","last_undegraded":"2024-05-08T23:24:46.292518+0000","last_fullsized":"2024-05-08T23:24:46.292518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:33.665957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991554+0000","last_change":"2024-05-08T23:24:44.837193+0000","last_active":"2024-05-08T23:24:46.991554+0000","last_peered":"2024-05-08T23:24:46.991554+0000","last_clean":"2024-05-08T23:24:46.991554+0000","last_became_active":"2024-05-08T23:24:44.836894+0000","last_became_peered":"2024-05-08T23:24:44.836894+0000","last_unstale":"2024-05-08T23:24:46.991554+0000","last_undegraded":"2024-05-08T23:24:46.991554+0000","last_fullsized":"2024-05-08T23:24:46.991554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:00.500883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292727+0000","last_change":"2024-05-08T23:24:45.308069+0000","last_active":"2024-05-08T23:24:46.292727+0000","last_peered":"2024-05-08T23:24:46.292727+0000","last_clean":"2024-05-08T23:24:46.292727+0000","last_became_active":"2024-05-08T23:24:45.307241+0000","last_became_peered":"2024-05-08T23:24:45.307241+0000","last_unstale":"2024-05-08T23:24:46.292727+0000","last_undegraded":"2024-05-08T23:24:46.292727+0000","last_fullsized":"2024-05-08T23:24:46.292727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:10.734065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991627+0000","last_change":"2024-05-08T23:24:44.833012+0000","last_active":"2024-05-08T23:24:46.991627+0000","last_peered":"2024-05-08T23:24:46.991627+0000","last_clean":"2024-05-08T23:24:46.991627+0000","last_became_active":"2024-05-08T23:24:44.832691+0000","last_became_peered":"2024-05-08T23:24:44.832691+0000","last_unstale":"2024-05-08T23:24:46.991627+0000","last_undegraded":"2024-05-08T23:24:46.991627+0000","last_fullsized":"2024-05-08T23:24:46.991627+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:24:38.091438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990861+0000","last_change":"2024-05-08T23:24:45.309846+0000","last_active":"2024-05-08T23:24:46.990861+0000","last_peered":"2024-05-08T23:24:46.990861+0000","last_clean":"2024-05-08T23:24:46.990861+0000","last_became_active":"2024-05-08T23:24:45.309092+0000","last_became_peered":"2024-05-08T23:24:45.309092+0000","last_unstale":"2024-05-08T23:24:46.990861+0000","last_undegraded":"2024-05-08T23:24:46.990861+0000","last_fullsized":"2024-05-08T23:24:46.990861+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:32:34.716931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991539+0000","last_change":"2024-05-08T23:24:44.829257+0000","last_active":"2024-05-08T23:24:46.991539+0000","last_peered":"2024-05-08T23:24:46.991539+0000","last_clean":"2024-05-08T23:24:46.991539+0000","last_became_active":"2024-05-08T23:24:44.827685+0000","last_became_peered":"2024-05-08T23:24:44.827685+0000","last_unstale":"2024-05-08T23:24:46.991539+0000","last_undegraded":"2024-05-08T23:24:46.991539+0000","last_fullsized":"2024-05-08T23:24:46.991539+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:03.302183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991236+0000","last_change":"2024-05-08T23:24:45.296534+0000","last_active":"2024-05-08T23:24:46.991236+0000","last_peered":"2024-05-08T23:24:46.991236+0000","last_clean":"2024-05-08T23:24:46.991236+0000","last_became_active":"2024-05-08T23:24:45.294854+0000","last_became_peered":"2024-05-08T23:24:45.294854+0000","last_unstale":"2024-05-08T23:24:46.991236+0000","last_undegraded":"2024-05-08T23:24:46.991236+0000","last_fullsized":"2024-05-08T23:24:46.991236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:34.043367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292590+0000","last_change":"2024-05-08T23:24:44.830521+0000","last_active":"2024-05-08T23:24:46.292590+0000","last_peered":"2024-05-08T23:24:46.292590+0000","last_clean":"2024-05-08T23:24:46.292590+0000","last_became_active":"2024-05-08T23:24:44.830227+0000","last_became_peered":"2024-05-08T23:24:44.830227+0000","last_unstale":"2024-05-08T23:24:46.292590+0000","last_undegraded":"2024-05-08T23:24:46.292590+0000","last_fullsized":"2024-05-08T23:24:46.292590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:33.819270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292084+0000","last_change":"2024-05-08T23:24:45.316904+0000","last_active":"2024-05-08T23:24:46.292084+0000","last_peered":"2024-05-08T23:24:46.292084+0000","last_clean":"2024-05-08T23:24:46.292084+0000","last_became_active":"2024-05-08T23:24:45.315651+0000","last_became_peered":"2024-05-08T23:24:45.315651+0000","last_unstale":"2024-05-08T23:24:46.292084+0000","last_undegraded":"2024-05-08T23:24:46.292084+0000","last_fullsized":"2024-05-08T23:24:46.292084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:01.809393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991628+0000","last_change":"2024-05-08T23:24:44.835050+0000","last_active":"2024-05-08T23:24:46.991628+0000","last_peered":"2024-05-08T23:24:46.991628+0000","last_clean":"2024-05-08T23:24:46.991628+0000","last_became_active":"2024-05-08T23:24:44.834622+0000","last_became_peered":"2024-05-08T23:24:44.834622+0000","last_unstale":"2024-05-08T23:24:46.991628+0000","last_undegraded":"2024-05-08T23:24:46.991628+0000","last_fullsized":"2024-05-08T23:24:46.991628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:10.795747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.992007+0000","last_change":"2024-05-08T23:24:45.314120+0000","last_active":"2024-05-08T23:24:46.992007+0000","last_peered":"2024-05-08T23:24:46.992007+0000","last_clean":"2024-05-08T23:24:46.992007+0000","last_became_active":"2024-05-08T23:24:45.313833+0000","last_became_peered":"2024-05-08T23:24:45.313833+0000","last_unstale":"2024-05-08T23:24:46.992007+0000","last_undegraded":"2024-05-08T23:24:46.992007+0000","last_fullsized":"2024-05-08T23:24:46.992007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:22.639106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991711+0000","last_change":"2024-05-08T23:24:44.831697+0000","last_active":"2024-05-08T23:24:46.991711+0000","last_peered":"2024-05-08T23:24:46.991711+0000","last_clean":"2024-05-08T23:24:46.991711+0000","last_became_active":"2024-05-08T23:24:44.831423+0000","last_became_peered":"2024-05-08T23:24:44.831423+0000","last_unstale":"2024-05-08T23:24:46.991711+0000","last_undegraded":"2024-05-08T23:24:46.991711+0000","last_fullsized":"2024-05-08T23:24:46.991711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:20.603298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990520+0000","last_change":"2024-05-08T23:24:45.302536+0000","last_active":"2024-05-08T23:24:46.990520+0000","last_peered":"2024-05-08T23:24:46.990520+0000","last_clean":"2024-05-08T23:24:46.990520+0000","last_became_active":"2024-05-08T23:24:45.302274+0000","last_became_peered":"2024-05-08T23:24:45.302274+0000","last_unstale":"2024-05-08T23:24:46.990520+0000","last_undegraded":"2024-05-08T23:24:46.990520+0000","last_fullsized":"2024-05-08T23:24:46.990520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:04.307093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.990619+0000","last_change":"2024-05-08T23:24:44.835544+0000","last_active":"2024-05-08T23:24:46.990619+0000","last_peered":"2024-05-08T23:24:46.990619+0000","last_clean":"2024-05-08T23:24:46.990619+0000","last_became_active":"2024-05-08T23:24:44.835254+0000","last_became_peered":"2024-05-08T23:24:44.835254+0000","last_unstale":"2024-05-08T23:24:46.990619+0000","last_undegraded":"2024-05-08T23:24:46.990619+0000","last_fullsized":"2024-05-08T23:24:46.990619+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:26.656445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.292358+0000","last_change":"2024-05-08T23:24:45.300158+0000","last_active":"2024-05-08T23:24:46.292358+0000","last_peered":"2024-05-08T23:24:46.292358+0000","last_clean":"2024-05-08T23:24:46.292358+0000","last_became_active":"2024-05-08T23:24:45.299858+0000","last_became_peered":"2024-05-08T23:24:45.299858+0000","last_unstale":"2024-05-08T23:24:46.292358+0000","last_undegraded":"2024-05-08T23:24:46.292358+0000","last_fullsized":"2024-05-08T23:24:46.292358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:33.393325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291973+0000","last_change":"2024-05-08T23:24:44.832373+0000","last_active":"2024-05-08T23:24:46.291973+0000","last_peered":"2024-05-08T23:24:46.291973+0000","last_clean":"2024-05-08T23:24:46.291973+0000","last_became_active":"2024-05-08T23:24:44.832091+0000","last_became_peered":"2024-05-08T23:24:44.832091+0000","last_unstale":"2024-05-08T23:24:46.291973+0000","last_undegraded":"2024-05-08T23:24:46.291973+0000","last_fullsized":"2024-05-08T23:24:46.291973+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:39.707308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.291881+0000","last_change":"2024-05-08T23:24:45.308475+0000","last_active":"2024-05-08T23:24:46.291881+0000","last_peered":"2024-05-08T23:24:46.291881+0000","last_clean":"2024-05-08T23:24:46.291881+0000","last_became_active":"2024-05-08T23:24:45.308121+0000","last_became_peered":"2024-05-08T23:24:45.308121+0000","last_unstale":"2024-05-08T23:24:46.291881+0000","last_undegraded":"2024-05-08T23:24:46.291881+0000","last_fullsized":"2024-05-08T23:24:46.291881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:44.813758+0000","last_clean_scrub_stamp":"2024-05-08T23:24:44.813758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:40.669776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T23:24:46.991915+0000","last_change":"2024-05-08T23:24:44.833194+0000","last_active":"2024-05-08T23:24:46.991915+0000","last_peered":"2024-05-08T23:24:46.991915+0000","last_clean":"2024-05-08T23:24:46.991915+0000","last_became_active":"2024-05-08T23:24:44.832811+0000","last_became_peered":"2024-05-08T23:24:44.832811+0000","last_unstale":"2024-05-08T23:24:46.991915+0000","last_undegraded":"2024-05-08T23:24:46.991915+0000","last_fullsized":"2024-05-08T23:24:46.991915+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:24:43.808848+0000","last_clean_scrub_stamp":"2024-05-08T23:24:43.808848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:32:33.712021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-05-08T23:24:45.305307+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:24:45.305307+0000","last_peered":"2024-05-08T23:24:45.305307+0000","last_clean":"2024-05-08T23:24:45.305307+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:24:45.305307+0000","last_undegraded":"2024-05-08T23:24:45.305307+0000","last_fullsized":"2024-05-08T23:24:45.305307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1323008,"data_stored":894797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.69099999999999995}]},{"osd":1,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.56899999999999995,"15min":0.54300000000000004},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81000000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.43099999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.753}]},{"osd":3,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78100000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84899999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed May 8 23:24:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.79000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":297969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88200000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.40400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81599999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85699999999999998,"5min":1.115,"15min":1.115},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79800000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.498},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98899999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.43099999999999999}]},{"osd":5,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69499999999999995,"5min":1.052,"15min":1.052},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48499999999999999}]},{"osd":7,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.36899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":298059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.39000000000000001,"15min":0.40000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.48099999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.41799999999999998,"15min":0.42799999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54200000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.378}]},{"osd":2,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41799999999999998,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52400000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.28399999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.48899999999999999,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65000000000000002,"5min":1.01,"15min":1.01},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":1.028,"15min":1.028},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.438,"5min":0.432,"15min":0.432},"max":{"1min":0.84299999999999997,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.87,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86699999999999999,"5min":1.115,"15min":1.115},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68999999999999995,"15min":0.70199999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84599999999999997,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91700000000000004,"5min":1.105,"15min":1.105},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed May 8 23:24:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69999999999999996,"15min":0.70499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90800000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1318912,"data_stored":890483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.66700000000000004,"15min":0.78700000000000003},"min":{"1min":0.255,"5min":0.254,"15min":0.254},"max":{"1min":0.63400000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.65900000000000003,"15min":0.76900000000000002},"min":{"1min":0.254,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65800000000000003,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.71699999999999997,"15min":0.84499999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58599999999999997,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.433,"5min":0.69299999999999995,"15min":0.80300000000000005},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69899999999999995,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.44900000000000001}]},{"osd":3,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46899999999999997,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59899999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.85499999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79800000000000004,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.86699999999999999,"15min":0.99399999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.68400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.86499999999999999,"15min":0.96099999999999997},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.73399999999999999,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.871,"15min":0.96099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89800000000000002,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.94599999999999995,"15min":1.0700000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.72899999999999998}]},{"osd":6,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.91400000000000003,"15min":1.034},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95099999999999996,"5min":7.734,"15min":7.734},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.90300000000000002,"15min":1.0149999999999999},"min":{"1min":0.48399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Wed May 8 23:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.91400000000000003,"15min":1.02},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79000000000000004,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.88800000000000001,"15min":0.97199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81699999999999995,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":304325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61299999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41499999999999998,"15min":0.40200000000000002},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54600000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.42099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.55600000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.58099999999999996}]},{"osd":3,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.38900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.747,"15min":0.747},"last":0.748}]},{"osd":4,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93799999999999994,"5min":1.024,"15min":1.024},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82799999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.76700000000000002}]},{"osd":5,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85699999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.82799999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.878}]},{"osd":6,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":1.159,"15min":1.159},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91300000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.91900000000000004}]},{"osd":7,"last update":"Wed May 8 23:24:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.872,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92000000000000004,"5min":1.194,"15min":1.194},"last":0.89800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":304325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26200000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.49099999999999999,"5min":1.46,"15min":1.46},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.28799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51500000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.442}]},{"osd":1,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54100000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.46300000000000002,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64600000000000002,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.76800000000000002,"15min":0.91400000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.63300000000000001,"5min":11.964,"15min":11.964},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":1.593,"15min":1.593},"last":0.67300000000000004}]},{"osd":4,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64800000000000002,"15min":0.66500000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93500000000000005,"5min":1.375,"15min":1.375},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.70299999999999996,"15min":0.73699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86899999999999999,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95099999999999996,"5min":1.536,"15min":1.536},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.47199999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84199999999999997,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67900000000000005,"15min":0.68899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.82699999999999996,"5min":1.204,"15min":1.204},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.71299999999999997,"15min":0.73499999999999999},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92000000000000004,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Wed May 8 23:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88800000000000001,"5min":1.339,"15min":1.339},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.52700000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.90800000000000003,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.69399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":305833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.375,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73899999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.68600000000000005,"15min":0.751},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73099999999999998,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.64700000000000002,"15min":0.68700000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":2.145,"15min":2.145},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67200000000000004,"15min":0.70799999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79300000000000004,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.66800000000000004,"15min":0.70699999999999996},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.73999999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.75600000000000001,"15min":0.82099999999999995},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.51400000000000001}]},{"osd":3,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.44400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":1.03,"15min":1.22},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.72399999999999998,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.67200000000000004}]},{"osd":5,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.59899999999999998},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75,"5min":0.877,"15min":0.877},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56599999999999995,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67100000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61099999999999999,"15min":0.63700000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81499999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68700000000000006}]},{"osd":7,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.63700000000000001},"min":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67700000000000005,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72599999999999998,"5min":1.24,"15min":1.24},"last":0.746}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":304921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.69499999999999995,"15min":0.73499999999999999},"min":{"1min":0.39900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.88700000000000001,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.83899999999999997,"15min":0.94899999999999995},"min":{"1min":0.379,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93400000000000005,"5min":8.798,"15min":8.798},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.73899999999999999,"15min":0.81000000000000005},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.70499999999999996,"5min":4.71,"15min":4.71},"last":0.622},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.90300000000000002,"15min":1.0329999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0109999999999999,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.72999999999999998}]},{"osd":2,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.73599999999999999,"15min":0.77600000000000002},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98899999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.76500000000000001,"15min":0.81899999999999995},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.86099999999999999,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.747}]},{"osd":3,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.875,"15min":0.98899999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91800000000000004,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.94099999999999995,"15min":1.079},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.746,"5min":10.427,"15min":10.427},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67500000000000004,"5min":0.874,"15min":0.874},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.40899999999999997}]},{"osd":6,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50900000000000001}]},{"osd":7,"last update":"Wed May 8 23:24:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.377,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85599999999999998,"5min":1.135,"15min":1.135},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:24:49.975 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:24:49.976 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T23:24:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.157+0000 7f3c657d7700 1 Processor -- start 2024-05-08T23:24:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.157+0000 7f3c657d7700 1 -- start start 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c657d7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c6015a7c0 0x7f3c600cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c657d7700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3c600727b0 0x7f3c600c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c657d7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c600cd7c0 con 0x7f3c60072260 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c657d7700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c600cd1b0 con 0x7f3c600727b0 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c657d7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6015aba0 con 0x7f3c6015a7c0 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c5f7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3c60072260 0x7f3c600b24f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36136/0 (socket says 172.21.15.5:36136) 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c5f7fe700 1 -- 172.21.15.5:0/416401949 learned_addr learned my addr 172.21.15.5:0/416401949 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c5e7fc700 1 --2- 172.21.15.5:0/416401949 >> v2:172.21.15.49:3300/0 conn(0x7f3c600727b0 0x7f3c600c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c5effd700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c6015a7c0 0x7f3c600cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.158+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038612622 0 0) 0x7f3c600cd7c0 con 0x7f3c60072260 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c480035b0 con 0x7f3c60072260 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3697484075 0 0) 0x7f3c480035b0 con 0x7f3c60072260 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c6015a7c0 msgr2=0x7f3c600cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c6015a7c0 0x7f3c600cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 >> v2:172.21.15.49:3300/0 conn(0x7f3c600727b0 msgr2=0x7f3c600c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 --2- 172.21.15.5:0/416401949 >> v2:172.21.15.49:3300/0 conn(0x7f3c600727b0 0x7f3c600c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600b3e70 con 0x7f3c60072260 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f3c580019f0 con 0x7f3c60072260 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5e7fc700 1 --2- 172.21.15.5:0/416401949 >> v2:172.21.15.49:3300/0 conn(0x7f3c600727b0 0x7f3c600c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 >> v1:172.21.15.5:6789/0 conn(0x7f3c60072260 legacy=0x7f3c600b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48003a00 0x7f3c48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48004850 0x7f3c48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c6015a7c0 0x7f3c48005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600cd1b0 con 0x7f3c48004850 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6015aba0 con 0x7f3c48003a00 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4800ba10 con 0x7f3c6015a7c0 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5e7fc700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48004850 0x7f3c48004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5effd700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48003a00 0x7f3c48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5f7fe700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c6015a7c0 0x7f3c48005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5e7fc700 1 -- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c6015a7c0 msgr2=0x7f3c48005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5e7fc700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c6015a7c0 0x7f3c48005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5e7fc700 1 -- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48003a00 msgr2=0x7f3c48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5e7fc700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48003a00 0x7f3c48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.159+0000 7f3c5e7fc700 1 -- 172.21.15.5:0/416401949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480035b0 con 0x7f3c48004850 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c5e7fc700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48004850 0x7f3c48004cd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009d30 tx=0x7f3c48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c8ad4be8b15882 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54009e80 con 0x7f3c48004850 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5400e9a0 con 0x7f3c48004850 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c5dffb700 1 -- 172.21.15.5:0/416401949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5400ed30 con 0x7f3c48004850 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c657d7700 1 -- 172.21.15.5:0/416401949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48004850 msgr2=0x7f3c48004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c657d7700 1 --2- 172.21.15.5:0/416401949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48004850 0x7f3c48004cd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009d30 tx=0x7f3c48005880 comp rx=0 tx=0).stop 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c5f7fe700 1 -- 172.21.15.5:0/416401949 reap_dead start 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c657d7700 1 -- 172.21.15.5:0/416401949 shutdown_connections 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c657d7700 1 -- 172.21.15.5:0/416401949 >> 172.21.15.5:0/416401949 conn(0x7f3c6006a960 msgr2=0x7f3c6006ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c657d7700 1 -- 172.21.15.5:0/416401949 shutdown_connections 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.160+0000 7f3c657d7700 1 -- 172.21.15.5:0/416401949 wait complete. 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.162+0000 7f3c657d7700 1 Processor -- start 2024-05-08T23:24:50.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.162+0000 7f3c657d7700 1 -- start start 2024-05-08T23:24:50.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c657d7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48004850 0x7f3c60146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c657d7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c600afe60 0x7f3c6015d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c657d7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c6015d8f0 0x7f3c6015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c657d7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6015bb40 con 0x7f3c6015d8f0 2024-05-08T23:24:50.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c657d7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c60146c50 con 0x7f3c48004850 2024-05-08T23:24:50.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c657d7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600ad8a0 con 0x7f3c600afe60 2024-05-08T23:24:50.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c6015d8f0 0x7f3c6015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c6015d8f0 0x7f3c6015ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57858/0 (socket says 172.21.15.5:57858) 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 -- 172.21.15.5:0/2702200767 learned_addr learned my addr 172.21.15.5:0/2702200767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5e7fc700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c600afe60 0x7f3c6015d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5effd700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48004850 0x7f3c60146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 -- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c600afe60 msgr2=0x7f3c6015d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c600afe60 0x7f3c6015d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 -- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48004850 msgr2=0x7f3c60146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48004850 0x7f3c60146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 -- 172.21.15.5:0/2702200767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600adb80 con 0x7f3c6015d8f0 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.163+0000 7f3c5f7fe700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c6015d8f0 0x7f3c6015ace0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3c58002230 tx=0x7f3c5800a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.164+0000 7f3c477fe700 1 -- 172.21.15.5:0/2702200767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5800ad50 con 0x7f3c6015d8f0 2024-05-08T23:24:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.164+0000 7f3c477fe700 1 -- 172.21.15.5:0/2702200767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c580133f0 con 0x7f3c6015d8f0 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.164+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c600af660 con 0x7f3c6015d8f0 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.164+0000 7f3c477fe700 1 -- 172.21.15.5:0/2702200767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580137a0 con 0x7f3c6015d8f0 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.164+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c600b02e0 con 0x7f3c6015d8f0 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.165+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c600af4c0 con 0x7f3c6015d8f0 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.165+0000 7f3c477fe700 1 -- 172.21.15.5:0/2702200767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3c58013900 con 0x7f3c6015d8f0 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.166+0000 7f3c477fe700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c4c06ac90 0x7f3c4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.166+0000 7f3c477fe700 1 -- 172.21.15.5:0/2702200767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c58089bb0 con 0x7f3c6015d8f0 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.166+0000 7f3c5effd700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c4c06ac90 0x7f3c4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.166+0000 7f3c5effd700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c4c06ac90 0x7f3c4c06d150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009b00 tx=0x7f3c50009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.170+0000 7f3c477fe700 1 -- 172.21.15.5:0/2702200767 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3c600af4c0 con 0x7f3c6015d8f0 2024-05-08T23:24:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.477+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3c600afb80 con 0x7f3c6015d8f0 2024-05-08T23:24:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.480+0000 7f3c477fe700 1 -- 172.21.15.5:0/2702200767 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3c600afb80 con 0x7f3c6015d8f0 2024-05-08T23:24:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c4c06ac90 msgr2=0x7f3c4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c4c06ac90 0x7f3c4c06d150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009b00 tx=0x7f3c50009210 comp rx=0 tx=0).stop 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c6015d8f0 msgr2=0x7f3c6015ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c6015d8f0 0x7f3c6015ace0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3c58002230 tx=0x7f3c5800a640 comp rx=0 tx=0).stop 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 shutdown_connections 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3c4c06ac90 0x7f3c4c06d150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c6015d8f0 0x7f3c6015ace0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c48004850 0x7f3c60146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 --2- 172.21.15.5:0/2702200767 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3c600afe60 0x7f3c6015d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 >> 172.21.15.5:0/2702200767 conn(0x7f3c6006a960 msgr2=0x7f3c600ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 shutdown_connections 2024-05-08T23:24:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.483+0000 7f3c657d7700 1 -- 172.21.15.5:0/2702200767 wait complete. 2024-05-08T23:24:50.485 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-08T23:24:50.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.634+0000 7f1afe562700 1 Processor -- start 2024-05-08T23:24:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.634+0000 7f1afe562700 1 -- start start 2024-05-08T23:24:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1afe562700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1af80c5e30 0x7f1af80d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1afe562700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af80cfbd0 con 0x7f1af815a750 2024-05-08T23:24:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1afe562700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af80cb780 con 0x7f1af80af860 2024-05-08T23:24:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1afe562700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af80cf440 con 0x7f1af80c5e30 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1af7fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1af815a750 0x7f1af80ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36138/0 (socket says 172.21.15.5:36138) 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1af7fff700 1 -- 172.21.15.5:0/803509703 learned_addr learned my addr 172.21.15.5:0/803509703 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1afcaff700 1 --2- 172.21.15.5:0/803509703 >> v2:172.21.15.49:3300/0 conn(0x7f1af80c5e30 0x7f1af80d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1af6ffd700 1 -- 172.21.15.5:0/803509703 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759402386 0 0) 0x7f1af80cfbd0 con 0x7f1af815a750 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.635+0000 7f1af6ffd700 1 -- 172.21.15.5:0/803509703 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae00035b0 con 0x7f1af815a750 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.636+0000 7f1afcaff700 1 -- 172.21.15.5:0/803509703 >> v1:172.21.15.49:6789/0 conn(0x7f1af80af860 legacy=0x7f1af80c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.636+0000 7f1af6ffd700 1 -- 172.21.15.5:0/803509703 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541905539 0 0) 0x7f1af80cb780 con 0x7f1af80af860 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.636+0000 7f1afcaff700 1 -- 172.21.15.5:0/803509703 >> v1:172.21.15.5:6789/0 conn(0x7f1af815a750 legacy=0x7f1af80ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.636+0000 7f1afcaff700 1 -- 172.21.15.5:0/803509703 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80cf650 con 0x7f1af80c5e30 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.636+0000 7f1afcaff700 1 --2- 172.21.15.5:0/803509703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 0x7f1af80d0ec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009d90 tx=0x7f1af000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ecb7cd9a7ca58f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.636+0000 7f1af6ffd700 1 -- 172.21.15.5:0/803509703 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af000e070 con 0x7f1af80c5e30 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.636+0000 7f1af6ffd700 1 -- 172.21.15.5:0/803509703 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af000bb20 con 0x7f1af80c5e30 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.636+0000 7f1af6ffd700 1 -- 172.21.15.5:0/803509703 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af000bed0 con 0x7f1af80c5e30 2024-05-08T23:24:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.637+0000 7f1afe562700 1 -- 172.21.15.5:0/803509703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 msgr2=0x7f1af80d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.637+0000 7f1afe562700 1 --2- 172.21.15.5:0/803509703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 0x7f1af80d0ec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009d90 tx=0x7f1af000b330 comp rx=0 tx=0).stop 2024-05-08T23:24:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.637+0000 7f1afe562700 1 -- 172.21.15.5:0/803509703 shutdown_connections 2024-05-08T23:24:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.637+0000 7f1afe562700 1 --2- 172.21.15.5:0/803509703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 0x7f1af80d0ec0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.637+0000 7f1afe562700 1 -- 172.21.15.5:0/803509703 >> 172.21.15.5:0/803509703 conn(0x7f1af80d03a0 msgr2=0x7f1af80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.637+0000 7f1afe562700 1 -- 172.21.15.5:0/803509703 shutdown_connections 2024-05-08T23:24:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.637+0000 7f1afe562700 1 -- 172.21.15.5:0/803509703 wait complete. 2024-05-08T23:24:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.638+0000 7f1afe562700 1 Processor -- start 2024-05-08T23:24:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.638+0000 7f1afe562700 1 -- start start 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1afe562700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af80af860 0x7f1af80b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1afe562700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 0x7f1af81e7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1afe562700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1af815a750 0x7f1af81e5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1afe562700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80b5b20 con 0x7f1af80af860 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1afe562700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80afdb0 con 0x7f1af80c5e30 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1afe562700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af81e7d80 con 0x7f1af815a750 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af80af860 0x7f1af80b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af80af860 0x7f1af80b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57872/0 (socket says 172.21.15.5:57872) 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af7fff700 1 -- 172.21.15.5:0/3196569190 learned_addr learned my addr 172.21.15.5:0/3196569190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af77fe700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 0x7f1af81e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1afcaff700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1af815a750 0x7f1af81e5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af7fff700 1 -- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1af815a750 msgr2=0x7f1af81e5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af7fff700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1af815a750 0x7f1af81e5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af7fff700 1 -- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 msgr2=0x7f1af81e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af7fff700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 0x7f1af81e7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.639+0000 7f1af7fff700 1 -- 172.21.15.5:0/3196569190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af0009a70 con 0x7f1af80af860 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.640+0000 7f1af7fff700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af80af860 0x7f1af80b55e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800c4d0 tx=0x7f1ae800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.640+0000 7f1af4ff9700 1 -- 172.21.15.5:0/3196569190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8005d70 con 0x7f1af80af860 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.640+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af81e8060 con 0x7f1af80af860 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.640+0000 7f1af4ff9700 1 -- 172.21.15.5:0/3196569190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae80070d0 con 0x7f1af80af860 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.640+0000 7f1af4ff9700 1 -- 172.21.15.5:0/3196569190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8007440 con 0x7f1af80af860 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.640+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af80b02e0 con 0x7f1af80af860 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.641+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af8068000 con 0x7f1af80af860 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.641+0000 7f1af4ff9700 1 -- 172.21.15.5:0/3196569190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1ae80075a0 con 0x7f1af80af860 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.642+0000 7f1af4ff9700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ae406ac90 0x7f1ae406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.642+0000 7f1af4ff9700 1 -- 172.21.15.5:0/3196569190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ae80887c0 con 0x7f1af80af860 2024-05-08T23:24:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.642+0000 7f1af77fe700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ae406ac90 0x7f1ae406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.643+0000 7f1af77fe700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ae406ac90 0x7f1ae406d150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1aec007fd0 tx=0x7f1aec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.647+0000 7f1af4ff9700 1 -- 172.21.15.5:0/3196569190 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1ae80539d0 con 0x7f1af80af860 2024-05-08T23:24:50.954 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:24:50.811+0000 7f8300507700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:24:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.912+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1af8071ed0 con 0x7f1af80af860 2024-05-08T23:24:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.913+0000 7f1af4ff9700 1 -- 172.21.15.5:0/3196569190 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4728 (secure 0 0 0) 0x7f1ae8056ff0 con 0x7f1af80af860 2024-05-08T23:24:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:50.955 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6516,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:24:48.295616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6525},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ae406ac90 msgr2=0x7f1ae406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ae406ac90 0x7f1ae406d150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1aec007fd0 tx=0x7f1aec009040 comp rx=0 tx=0).stop 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af80af860 msgr2=0x7f1af80b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af80af860 0x7f1af80b55e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800c4d0 tx=0x7f1ae800c890 comp rx=0 tx=0).stop 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 shutdown_connections 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1ae406ac90 0x7f1ae406d150 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1af815a750 0x7f1af81e5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af80af860 0x7f1af80b55e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 --2- 172.21.15.5:0/3196569190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1af80c5e30 0x7f1af81e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 >> 172.21.15.5:0/3196569190 conn(0x7f1af80d03a0 msgr2=0x7f1af80d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 shutdown_connections 2024-05-08T23:24:50.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:50.915+0000 7f1afe562700 1 -- 172.21.15.5:0/3196569190 wait complete. 2024-05-08T23:24:50.959 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 49 2024-05-08T23:24:51.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:24:45.283486+0000', 'modified': '2024-05-08T23:24:48.295616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6525}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6525': {'gid': 6525, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/1169758885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 1169758885}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1169758885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:24:51.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:24:51.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T23:24:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.508+0000 7f4b6db69700 1 Processor -- start 2024-05-08T23:24:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.508+0000 7f4b6db69700 1 -- start start 2024-05-08T23:24:51.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.509+0000 7f4b6db69700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b680af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.509+0000 7f4b6db69700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b680d0eb0 0x7f4b68161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.509+0000 7f4b6db69700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b680cd980 con 0x7f4b6806a960 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.509+0000 7f4b6db69700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b680ccdb0 con 0x7f4b680d0eb0 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.509+0000 7f4b6db69700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680cd050 con 0x7f4b680cf840 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.509+0000 7f4b67fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b680d0eb0 0x7f4b68161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.509+0000 7f4b67fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b680d0eb0 0x7f4b68161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57884/0 (socket says 172.21.15.5:57884) 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.509+0000 7f4b67fff700 1 -- 172.21.15.5:0/658931355 learned_addr learned my addr 172.21.15.5:0/658931355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b677fe700 1 --2- 172.21.15.5:0/658931355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b680af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b67fff700 1 -- 172.21.15.5:0/658931355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 msgr2=0x7f4b680af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b67fff700 1 --2- 172.21.15.5:0/658931355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b680af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b67fff700 1 -- 172.21.15.5:0/658931355 >> v1:172.21.15.49:6789/0 conn(0x7f4b6806a960 legacy=0x7f4b6815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b67fff700 1 -- 172.21.15.5:0/658931355 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b680cd330 con 0x7f4b680d0eb0 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b677fe700 1 --2- 172.21.15.5:0/658931355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b680af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b67fff700 1 --2- 172.21.15.5:0/658931355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b680d0eb0 0x7f4b68161620 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4b600098f0 tx=0x7f4b60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48e0be5212909b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b667fc700 1 -- 172.21.15.5:0/658931355 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000b8a0 con 0x7f4b680d0eb0 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b667fc700 1 -- 172.21.15.5:0/658931355 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6000ba00 con 0x7f4b680d0eb0 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.510+0000 7f4b667fc700 1 -- 172.21.15.5:0/658931355 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000bd90 con 0x7f4b680d0eb0 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.511+0000 7f4b6db69700 1 -- 172.21.15.5:0/658931355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b680d0eb0 msgr2=0x7f4b68161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.511+0000 7f4b6db69700 1 --2- 172.21.15.5:0/658931355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b680d0eb0 0x7f4b68161620 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4b600098f0 tx=0x7f4b60009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.511+0000 7f4b6db69700 1 -- 172.21.15.5:0/658931355 shutdown_connections 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.511+0000 7f4b6db69700 1 --2- 172.21.15.5:0/658931355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b680d0eb0 0x7f4b68161620 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.511+0000 7f4b6db69700 1 --2- 172.21.15.5:0/658931355 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b680af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.511+0000 7f4b6db69700 1 -- 172.21.15.5:0/658931355 >> 172.21.15.5:0/658931355 conn(0x7f4b680d0320 msgr2=0x7f4b680d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.511+0000 7f4b6db69700 1 -- 172.21.15.5:0/658931355 shutdown_connections 2024-05-08T23:24:51.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.511+0000 7f4b6db69700 1 -- 172.21.15.5:0/658931355 wait complete. 2024-05-08T23:24:51.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.512+0000 7f4b6db69700 1 Processor -- start 2024-05-08T23:24:51.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.512+0000 7f4b6db69700 1 -- start start 2024-05-08T23:24:51.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b6db69700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6806a960 0x7f4b68160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b6db69700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b6814d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b6db69700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b680d0eb0 0x7f4b680ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b6db69700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68160610 con 0x7f4b6806a960 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b6db69700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6815fa00 con 0x7f4b680d0eb0 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b6db69700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6815fce0 con 0x7f4b680cf840 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6806a960 0x7f4b68160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6806a960 0x7f4b68160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57892/0 (socket says 172.21.15.5:57892) 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.513+0000 7f4b677fe700 1 -- 172.21.15.5:0/1095104059 learned_addr learned my addr 172.21.15.5:0/1095104059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b67fff700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b680d0eb0 0x7f4b680ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b6814d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b677fe700 1 -- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 msgr2=0x7f4b6814d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b677fe700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b6814d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b677fe700 1 -- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b680d0eb0 msgr2=0x7f4b680ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b677fe700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b680d0eb0 0x7f4b680ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b677fe700 1 -- 172.21.15.5:0/1095104059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b60009550 con 0x7f4b6806a960 2024-05-08T23:24:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b677fe700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6806a960 0x7f4b68160aa0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009fc0 tx=0x7f4b5800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b5ffff700 1 -- 172.21.15.5:0/1095104059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58003a20 con 0x7f4b6806a960 2024-05-08T23:24:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b5ffff700 1 -- 172.21.15.5:0/1095104059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5800f040 con 0x7f4b6806a960 2024-05-08T23:24:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b6db69700 1 -- 172.21.15.5:0/1095104059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6815ffc0 con 0x7f4b6806a960 2024-05-08T23:24:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b5ffff700 1 -- 172.21.15.5:0/1095104059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58003de0 con 0x7f4b6806a960 2024-05-08T23:24:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.514+0000 7f4b6db69700 1 -- 172.21.15.5:0/1095104059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6814dd40 con 0x7f4b6806a960 2024-05-08T23:24:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.515+0000 7f4b5ffff700 1 -- 172.21.15.5:0/1095104059 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4b58004030 con 0x7f4b6806a960 2024-05-08T23:24:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.516+0000 7f4b5ffff700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4b4806ac90 0x7f4b4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.516+0000 7f4b5ffff700 1 -- 172.21.15.5:0/1095104059 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b580894b0 con 0x7f4b6806a960 2024-05-08T23:24:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.516+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4b4806ac90 0x7f4b4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.517+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4b4806ac90 0x7f4b4806d150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b680c41b0 tx=0x7f4b50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.518+0000 7f4b79d23b80 1 Processor -- start 2024-05-08T23:24:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.518+0000 7f4b79d23b80 1 -- start start 2024-05-08T23:24:51.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 0x5573114950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x55731149a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114c3d80 0x5573114c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573113f0100 con 0x557311497320 2024-05-08T23:24:51.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573113edac0 con 0x5573114c3d80 2024-05-08T23:24:51.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573113e93b0 con 0x5573114bc750 2024-05-08T23:24:51.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b66ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x55731149a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b66ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x55731149a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57900/0 (socket says 172.21.15.5:57900) 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b67fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114c3d80 0x5573114c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b66ffd700 1 -- 172.21.15.5:0/1043562310 learned_addr learned my addr 172.21.15.5:0/1043562310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.519+0000 7f4b677fe700 1 --2- 172.21.15.5:0/1043562310 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 0x5573114950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b66ffd700 1 -- 172.21.15.5:0/1043562310 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 msgr2=0x5573114950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/1043562310 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 0x5573114950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b66ffd700 1 -- 172.21.15.5:0/1043562310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114c3d80 msgr2=0x5573114c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/1043562310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114c3d80 0x5573114c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b66ffd700 1 -- 172.21.15.5:0/1043562310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55731126a360 con 0x557311497320 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/1043562310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x55731149a050 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4b6806ad80 tx=0x7f4b50011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e139d9b5b52d403 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b5d7fa700 1 -- 172.21.15.5:0/1043562310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50003d20 con 0x557311497320 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b5d7fa700 1 -- 172.21.15.5:0/1043562310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b50003e80 con 0x557311497320 2024-05-08T23:24:51.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.520+0000 7f4b5d7fa700 1 -- 172.21.15.5:0/1043562310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50021650 con 0x557311497320 2024-05-08T23:24:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 -- 172.21.15.5:0/1043562310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 msgr2=0x55731149a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/1043562310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x55731149a050 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4b6806ad80 tx=0x7f4b50011000 comp rx=0 tx=0).stop 2024-05-08T23:24:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 -- 172.21.15.5:0/1043562310 shutdown_connections 2024-05-08T23:24:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/1043562310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114c3d80 0x5573114c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/1043562310 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 0x5573114950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/1043562310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x55731149a050 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 -- 172.21.15.5:0/1043562310 >> 172.21.15.5:0/1043562310 conn(0x5573113f71c0 msgr2=0x5573114ded80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 -- 172.21.15.5:0/1043562310 shutdown_connections 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 -- 172.21.15.5:0/1043562310 wait complete. 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.521+0000 7f4b79d23b80 1 -- start start 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.522+0000 7f4b79d23b80 1 Processor -- start 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.522+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 0x5573114ed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.522+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114bc750 0x5573114c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.522+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.522+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311385510 con 0x5573114bc750 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.522+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573113851b0 con 0x557311497320 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.522+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55731126c650 con 0x5573114c3d80 2024-05-08T23:24:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b79d23b80 10 client.0 _ll_register_callbacks cb 0x55731138eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114bc750 0x5573114c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114bc750 0x5573114c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57904/0 (socket says 172.21.15.5:57904) 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 -- 172.21.15.5:0/4079507338 learned_addr learned my addr 172.21.15.5:0/4079507338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b677fe700 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 0x5573114ed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b67fff700 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114bb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 -- 172.21.15.5:0/4079507338 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 msgr2=0x5573114bb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114bb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 -- 172.21.15.5:0/4079507338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 msgr2=0x5573114ed9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 0x5573114ed9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 -- 172.21.15.5:0/4079507338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5000f040 con 0x5573114bc750 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114bc750 0x5573114c4f20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4b50003960 tx=0x7f4b50005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c296a5135c794 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b467fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b467fc700 1 -- 172.21.15.5:0/4079507338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50028040 con 0x5573114bc750 2024-05-08T23:24:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b467fc700 1 -- 172.21.15.5:0/4079507338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b50021b00 con 0x5573114bc750 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b79d23b80 1 -- 172.21.15.5:0/4079507338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557311468790 con 0x5573114bc750 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.523+0000 7f4b467fc700 1 -- 172.21.15.5:0/4079507338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b50001580 con 0x5573114bc750 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b79d23b80 10 client.6654 fetch_fsmap learned FSMap version 49 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b79d23b80 1 -- 172.21.15.5:0/4079507338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x557311216b10 con 0x5573114bc750 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b467fc700 1 -- 172.21.15.5:0/4079507338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557311468790 con 0x5573114bc750 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b467fc700 1 -- 172.21.15.5:0/4079507338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4b50021d00 con 0x5573114bc750 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b79d23b80 10 client.6654 fetch_fsmap finished waiting for FSMap version 49 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b79d23b80 10 client.6654 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b79d23b80 20 client.6654 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5573114e0600 0x5573114e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b79d23b80 4 client.6654 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b79d23b80 1 -- 172.21.15.5:0/4079507338 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573113e1e70 con 0x5573114e0600 2024-05-08T23:24:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b677fe700 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5573114e0600 0x5573114e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b467fc700 10 client.6654 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.524+0000 7f4b677fe700 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5573114e0600 0x5573114e09e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=affe0ff5619dff2e server_cookie=81f1a296b60e53d6 in_seq=0 out_seq=0 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b467fc700 1 -- 172.21.15.5:0/4079507338 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573113e1e70 con 0x5573114e0600 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b467fc700 10 client.6654 handle_command_reply: tid=0 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 client.6654 shutdown 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 -- 172.21.15.5:0/4079507338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114bc750 msgr2=0x5573114c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114bc750 0x5573114c4f20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4b50003960 tx=0x7f4b50005000 comp rx=0 tx=0).stop 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 -- 172.21.15.5:0/4079507338 shutdown_connections 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114bb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5573114e0600 0x5573114e09e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 0x5573114ed9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/4079507338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114bc750 0x5573114c4f20 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b467fc700 0 client.6654 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:24:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 -- 172.21.15.5:0/4079507338 >> 172.21.15.5:0/4079507338 conn(0x5573113f71c0 msgr2=0x5573114992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 -- 172.21.15.5:0/4079507338 shutdown_connections 2024-05-08T23:24:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 1 -- 172.21.15.5:0/4079507338 wait complete. 2024-05-08T23:24:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.525+0000 7f4b79d23b80 20 client.6654 trim_cache size 0 max 16384 2024-05-08T23:24:51.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.545+0000 7f4b79d23b80 1 Processor -- start 2024-05-08T23:24:51.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.545+0000 7f4b79d23b80 1 -- start start 2024-05-08T23:24:51.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.545+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x5573114ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114bc750 0x5573114c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557311495a10 con 0x557311497320 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573114c0fe0 con 0x5573114bc750 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573114ecf20 con 0x5573114c3d80 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x5573114ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x5573114ec9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57910/0 (socket says 172.21.15.5:57910) 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 -- 172.21.15.5:0/3454212117 learned_addr learned my addr 172.21.15.5:0/3454212117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 -- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 msgr2=0x5573114c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b67fff700 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114bc750 0x5573114c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 -- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114bc750 msgr2=0x5573114c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114bc750 0x5573114c0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 -- 172.21.15.5:0/3454212117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573113af510 con 0x557311497320 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114bc750 0x5573114c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b67fff700 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b677fe700 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x5573114ec9e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4b58054900 tx=0x7f4b580878b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e878f21c16de10b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b5dffb700 1 -- 172.21.15.5:0/3454212117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58087af0 con 0x557311497320 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b5dffb700 1 -- 172.21.15.5:0/3454212117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b58056540 con 0x557311497320 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.546+0000 7f4b5dffb700 1 -- 172.21.15.5:0/3454212117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b580890c0 con 0x557311497320 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3454212117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 msgr2=0x5573114ec9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x5573114ec9e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4b58054900 tx=0x7f4b580878b0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3454212117 shutdown_connections 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114c3d80 0x5573114c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557311497320 0x5573114ec9e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3454212117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5573114bc750 0x5573114c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3454212117 >> 172.21.15.5:0/3454212117 conn(0x5573113f71c0 msgr2=0x5573114defb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3454212117 shutdown_connections 2024-05-08T23:24:51.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.547+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3454212117 wait complete. 2024-05-08T23:24:51.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.548+0000 7f4b79d23b80 1 -- start start 2024-05-08T23:24:51.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.548+0000 7f4b79d23b80 1 Processor -- start 2024-05-08T23:24:51.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.549+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 0x5573114baf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.549+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 0x55731153eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.549+0000 7f4b79d23b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114c3d80 0x557311497700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.549+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573114e0fc0 con 0x5573114c3d80 2024-05-08T23:24:51.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.549+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731153f050 con 0x557311497320 2024-05-08T23:24:51.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.549+0000 7f4b79d23b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557311548fd0 con 0x5573114bc750 2024-05-08T23:24:51.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.549+0000 7f4b79d23b80 10 client.0 _ll_register_callbacks cb 0x55731138eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:51.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114c3d80 0x557311497700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114c3d80 0x557311497700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57912/0 (socket says 172.21.15.5:57912) 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 -- 172.21.15.5:0/3468140547 learned_addr learned my addr 172.21.15.5:0/3468140547 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b66ffd700 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 0x55731153eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b677fe700 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 0x5573114baf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 -- 172.21.15.5:0/3468140547 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 msgr2=0x55731153eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 0x55731153eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 -- 172.21.15.5:0/3468140547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 msgr2=0x5573114baf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 0x5573114baf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 -- 172.21.15.5:0/3468140547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6814dd40 con 0x5573114c3d80 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b67fff700 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114c3d80 0x557311497700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4b680704d0 tx=0x7f4b600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa8e7ea9582ba3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b467fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b467fc700 1 -- 172.21.15.5:0/3468140547 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60021050 con 0x5573114c3d80 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b467fc700 1 -- 172.21.15.5:0/3468140547 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b60004450 con 0x5573114c3d80 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3468140547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557311342220 con 0x5573114c3d80 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b467fc700 1 -- 172.21.15.5:0/3468140547 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6000be10 con 0x5573114c3d80 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b79d23b80 10 client.6660 fetch_fsmap learned FSMap version 49 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.550+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3468140547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55731126ec80 con 0x5573114c3d80 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b467fc700 1 -- 172.21.15.5:0/3468140547 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557311342220 con 0x5573114c3d80 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b467fc700 1 -- 172.21.15.5:0/3468140547 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4b600036a0 con 0x5573114c3d80 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b79d23b80 10 client.6660 fetch_fsmap finished waiting for FSMap version 49 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b79d23b80 10 client.6660 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b79d23b80 20 client.6660 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55731154b9c0 0x55731154bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b79d23b80 4 client.6660 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3468140547 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557311237ca0 con 0x55731154b9c0 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b677fe700 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55731154b9c0 0x55731154bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b467fc700 10 client.6660 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b677fe700 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55731154b9c0 0x55731154bda0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ce8cd169c6a15a server_cookie=dd57fd67021e684e in_seq=0 out_seq=0 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b467fc700 1 -- 172.21.15.5:0/3468140547 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557311237ca0 con 0x55731154b9c0 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b467fc700 10 client.6660 handle_command_reply: tid=0 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.551+0000 7f4b79d23b80 1 client.6660 shutdown 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3468140547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114c3d80 msgr2=0x557311497700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114c3d80 0x557311497700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4b680704d0 tx=0x7f4b600047c0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3468140547 shutdown_connections 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573114c3d80 0x557311497700 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55731154b9c0 0x55731154bda0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557311497320 0x5573114baf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 --2- 172.21.15.5:0/3468140547 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5573114bc750 0x55731153eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b467fc700 0 client.6660 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:24:51.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3468140547 >> 172.21.15.5:0/3468140547 conn(0x5573113f71c0 msgr2=0x5573114defb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:51.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3468140547 shutdown_connections 2024-05-08T23:24:51.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 1 -- 172.21.15.5:0/3468140547 wait complete. 2024-05-08T23:24:51.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b79d23b80 20 client.6660 trim_cache size 0 max 16384 2024-05-08T23:24:51.559 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:51.560 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:24:51.560 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:24:51.560 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6525, 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.2674653060000001, 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 49, 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 60, 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 60, 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 11.514750594000001 2024-05-08T23:24:51.561 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 -- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4b4806ac90 msgr2=0x7f4b4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4b4806ac90 0x7f4b4806d150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b680c41b0 tx=0x7f4b50009040 comp rx=0 tx=0).stop 2024-05-08T23:24:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 -- 172.21.15.5:0/1095104059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6806a960 msgr2=0x7f4b68160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6806a960 0x7f4b68160aa0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009fc0 tx=0x7f4b5800d460 comp rx=0 tx=0).stop 2024-05-08T23:24:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 -- 172.21.15.5:0/1095104059 shutdown_connections 2024-05-08T23:24:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4b4806ac90 0x7f4b4806d150 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b680d0eb0 0x7f4b680ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6806a960 0x7f4b68160aa0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 --2- 172.21.15.5:0/1095104059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4b680cf840 0x7f4b6814d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.552+0000 7f4b6db69700 1 -- 172.21.15.5:0/1095104059 >> 172.21.15.5:0/1095104059 conn(0x7f4b680d0320 msgr2=0x7f4b680c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:51.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.553+0000 7f4b6db69700 1 -- 172.21.15.5:0/1095104059 shutdown_connections 2024-05-08T23:24:51.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.553+0000 7f4b6db69700 1 -- 172.21.15.5:0/1095104059 wait complete. 2024-05-08T23:24:51.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.703+0000 7f4aaca59700 1 Processor -- start 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.703+0000 7f4aaca59700 1 -- start start 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.704+0000 7f4aaca59700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4aa806a960 0x7f4aa815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.704+0000 7f4aaca59700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4aa80d0eb0 0x7f4aa815da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.704+0000 7f4aaca59700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa80cd980 con 0x7f4aa80cf840 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.704+0000 7f4aaca59700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa80ccdb0 con 0x7f4aa806a960 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.704+0000 7f4aaca59700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa80cd050 con 0x7f4aa80d0eb0 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.704+0000 7f4aa5d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4aa806a960 0x7f4aa815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.704+0000 7f4aa5d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4aa806a960 0x7f4aa815d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57918/0 (socket says 172.21.15.5:57918) 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.704+0000 7f4aa5d9b700 1 -- 172.21.15.5:0/944406159 learned_addr learned my addr 172.21.15.5:0/944406159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa6d9d700 1 --2- 172.21.15.5:0/944406159 >> v2:172.21.15.49:3300/0 conn(0x7f4aa80d0eb0 0x7f4aa815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa5d9b700 1 -- 172.21.15.5:0/944406159 >> v2:172.21.15.49:3300/0 conn(0x7f4aa80d0eb0 msgr2=0x7f4aa815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa5d9b700 1 --2- 172.21.15.5:0/944406159 >> v2:172.21.15.49:3300/0 conn(0x7f4aa80d0eb0 0x7f4aa815da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa5d9b700 1 -- 172.21.15.5:0/944406159 >> v1:172.21.15.49:6789/0 conn(0x7f4aa80cf840 legacy=0x7f4aa80af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa5d9b700 1 -- 172.21.15.5:0/944406159 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80cd330 con 0x7f4aa806a960 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa5d9b700 1 --2- 172.21.15.5:0/944406159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa815d4f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80cd8f0 tx=0x7f4a9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a85a2336caac53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa559a700 1 -- 172.21.15.5:0/944406159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c011830 con 0x7f4aa806a960 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa559a700 1 -- 172.21.15.5:0/944406159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c011990 con 0x7f4aa806a960 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.705+0000 7f4aa559a700 1 -- 172.21.15.5:0/944406159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c011d20 con 0x7f4aa806a960 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.706+0000 7f4aaca59700 1 -- 172.21.15.5:0/944406159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 msgr2=0x7f4aa815d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.706+0000 7f4aaca59700 1 --2- 172.21.15.5:0/944406159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa815d4f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80cd8f0 tx=0x7f4a9c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.706+0000 7f4aaca59700 1 -- 172.21.15.5:0/944406159 shutdown_connections 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.706+0000 7f4aaca59700 1 --2- 172.21.15.5:0/944406159 >> v2:172.21.15.49:3300/0 conn(0x7f4aa80d0eb0 0x7f4aa815da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.706+0000 7f4aaca59700 1 --2- 172.21.15.5:0/944406159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa815d4f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.706+0000 7f4aaca59700 1 -- 172.21.15.5:0/944406159 >> 172.21.15.5:0/944406159 conn(0x7f4aa80d0320 msgr2=0x7f4aa80d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.706+0000 7f4aaca59700 1 -- 172.21.15.5:0/944406159 shutdown_connections 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.706+0000 7f4aaca59700 1 -- 172.21.15.5:0/944406159 wait complete. 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.707+0000 7f4aaca59700 1 Processor -- start 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aaca59700 1 -- start start 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aaca59700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa81477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aaca59700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4aa80cf840 0x7f4aa80b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aaca59700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4aa80d0eb0 0x7f4aa8159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aaca59700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80b49b0 con 0x7f4aa806a960 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aaca59700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa8147ce0 con 0x7f4aa80d0eb0 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aaca59700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80b43f0 con 0x7f4aa80cf840 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aa659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa81477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aa659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa81477a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57922/0 (socket says 172.21.15.5:57922) 2024-05-08T23:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.708+0000 7f4aa659c700 1 -- 172.21.15.5:0/3313477374 learned_addr learned my addr 172.21.15.5:0/3313477374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa6d9d700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4aa80d0eb0 0x7f4aa8159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa5d9b700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4aa80cf840 0x7f4aa80b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa659c700 1 -- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4aa80cf840 msgr2=0x7f4aa80b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa659c700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4aa80cf840 0x7f4aa80b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa659c700 1 -- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4aa80d0eb0 msgr2=0x7f4aa8159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa659c700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4aa80d0eb0 0x7f4aa8159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa659c700 1 -- 172.21.15.5:0/3313477374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c009550 con 0x7f4aa806a960 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa5d9b700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4aa80cf840 0x7f4aa80b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aa659c700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa81477a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800c4e0 tx=0x7f4a9800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4a96ffd700 1 -- 172.21.15.5:0/3313477374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a980052d0 con 0x7f4aa806a960 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4a96ffd700 1 -- 172.21.15.5:0/3313477374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a98005430 con 0x7f4aa806a960 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa80b46d0 con 0x7f4aa806a960 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4a96ffd700 1 -- 172.21.15.5:0/3313477374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a980038c0 con 0x7f4aa806a960 2024-05-08T23:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.709+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa8157df0 con 0x7f4aa806a960 2024-05-08T23:24:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.710+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa8157970 con 0x7f4aa806a960 2024-05-08T23:24:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.711+0000 7f4a96ffd700 1 -- 172.21.15.5:0/3313477374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a98003a20 con 0x7f4aa806a960 2024-05-08T23:24:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.711+0000 7f4a96ffd700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a8806ac90 0x7f4a8806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.711+0000 7f4a96ffd700 1 -- 172.21.15.5:0/3313477374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a98088780 con 0x7f4aa806a960 2024-05-08T23:24:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.711+0000 7f4aa5d9b700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a8806ac90 0x7f4a8806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.712+0000 7f4aa5d9b700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a8806ac90 0x7f4a8806d150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009c10 tx=0x7f4a9c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.716+0000 7f4a96ffd700 1 -- 172.21.15.5:0/3313477374 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4aa8157970 con 0x7f4aa806a960 2024-05-08T23:24:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.983+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4aa8157970 con 0x7f4aa806a960 2024-05-08T23:24:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.984+0000 7f4a96ffd700 1 -- 172.21.15.5:0/3313477374 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4a98017060 con 0x7f4aa806a960 2024-05-08T23:24:51.985 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:24:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a8806ac90 msgr2=0x7f4a8806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a8806ac90 0x7f4a8806d150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009c10 tx=0x7f4a9c008030 comp rx=0 tx=0).stop 2024-05-08T23:24:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 msgr2=0x7f4aa81477a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa81477a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800c4e0 tx=0x7f4a9800c7f0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 shutdown_connections 2024-05-08T23:24:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a8806ac90 0x7f4a8806d150 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4aa80d0eb0 0x7f4aa8159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa806a960 0x7f4aa81477a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 --2- 172.21.15.5:0/3313477374 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4aa80cf840 0x7f4aa80b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 >> 172.21.15.5:0/3313477374 conn(0x7f4aa80d0320 msgr2=0x7f4aa80b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 shutdown_connections 2024-05-08T23:24:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:51.986+0000 7f4aaca59700 1 -- 172.21.15.5:0/3313477374 wait complete. 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:24:52.000 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:24:52.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.001 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-08T23:24:52.021 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42025sec preferred_lft 42025sec 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-08T23:24:52.022 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::1c2e:b3ff:fe6b:56f/64 scope link 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::84a4:2aff:fe7c:ce1a/64 scope link 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether e2:3e:2c:7a:5d:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::e03e:2cff:fe7a:5db1/64 scope link 2024-05-08T23:24:52.023 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:24:52.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.024 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:24:52.092 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-08T23:24:52.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.093 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-08T23:24:52.157 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-08T23:24:52.157 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:24:52.157 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:24:52.157 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:24:52.158 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:24:52.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.158 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:24:52.158 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:24:52.158 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:24:52.158 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:24:52.158 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:24:52.158 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:24:52.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.283 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:24:52.303 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:24:52.303 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:24:52.303 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:24:52.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-08T23:24:52.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:24:52.379 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:24:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:24:52.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:24:52.441 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:24:52.503 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T23:24:52.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:24:52.602 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:24:52.603 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:24:52.623 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:24:52.624 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:24:52.624 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:24:52.624 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:24:52.624 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:24:52.624 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:24:52.624 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:24:52.624 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:24:52.642 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:24:52.642 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:24:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:24:52.645 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:24:52.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:24:52.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:52.678 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:24:52.774 INFO:teuthology.orchestra.run.smithi005.stdout:client6349 2024-05-08T23:24:52.775 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:24:52.775 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:52.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T23:24:52.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.985+0000 7f05ee483700 1 Processor -- start 2024-05-08T23:24:52.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.985+0000 7f05ee483700 1 -- start start 2024-05-08T23:24:52.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05ee483700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05e80d13a0 0x7f05e80d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05ee483700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e806bd60 0x7f05e80cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05ee483700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e80b1200 con 0x7f05e80cdc90 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05ee483700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e80d1cc0 con 0x7f05e80d13a0 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05ee483700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80cfcc0 con 0x7f05e806bd60 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05e7fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05e80d13a0 0x7f05e80d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05e7fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05e80d13a0 0x7f05e80d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57924/0 (socket says 172.21.15.5:57924) 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05e7fff700 1 -- 172.21.15.5:0/2763329460 learned_addr learned my addr 172.21.15.5:0/2763329460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05e77fe700 1 --2- 172.21.15.5:0/2763329460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e806bd60 0x7f05e80cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05e7fff700 1 -- 172.21.15.5:0/2763329460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e806bd60 msgr2=0x7f05e80cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05e7fff700 1 --2- 172.21.15.5:0/2763329460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e806bd60 0x7f05e80cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.986+0000 7f05e7fff700 1 -- 172.21.15.5:0/2763329460 >> v1:172.21.15.49:6789/0 conn(0x7f05e80cdc90 legacy=0x7f05e80b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.987+0000 7f05e7fff700 1 -- 172.21.15.5:0/2763329460 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80ce5c0 con 0x7f05e80d13a0 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.987+0000 7f05e7fff700 1 --2- 172.21.15.5:0/2763329460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e80d13a0 0x7f05e80d1780 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098f0 tx=0x7f05d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b47b36d338b5433 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.987+0000 7f05e6ffd700 1 -- 172.21.15.5:0/2763329460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800b8a0 con 0x7f05e80d13a0 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.987+0000 7f05e6ffd700 1 -- 172.21.15.5:0/2763329460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d800ba00 con 0x7f05e80d13a0 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.987+0000 7f05e6ffd700 1 -- 172.21.15.5:0/2763329460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800bd90 con 0x7f05e80d13a0 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.987+0000 7f05ee483700 1 -- 172.21.15.5:0/2763329460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e80d13a0 msgr2=0x7f05e80d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.987+0000 7f05ee483700 1 --2- 172.21.15.5:0/2763329460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e80d13a0 0x7f05e80d1780 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098f0 tx=0x7f05d8009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.988+0000 7f05ee483700 1 -- 172.21.15.5:0/2763329460 shutdown_connections 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.988+0000 7f05ee483700 1 --2- 172.21.15.5:0/2763329460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e80d13a0 0x7f05e80d1780 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.988+0000 7f05ee483700 1 --2- 172.21.15.5:0/2763329460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e806bd60 0x7f05e80cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.988+0000 7f05ee483700 1 -- 172.21.15.5:0/2763329460 >> 172.21.15.5:0/2763329460 conn(0x7f05e808afe0 msgr2=0x7f05e808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.988+0000 7f05ee483700 1 -- 172.21.15.5:0/2763329460 shutdown_connections 2024-05-08T23:24:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.988+0000 7f05ee483700 1 -- 172.21.15.5:0/2763329460 wait complete. 2024-05-08T23:24:52.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.989+0000 7f05ee483700 1 Processor -- start 2024-05-08T23:24:52.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.989+0000 7f05ee483700 1 -- start start 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05ee483700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e806bd60 0x7f05e815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05ee483700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e80cdc90 0x7f05e814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05ee483700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e80d13a0 0x7f05e81600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05ee483700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e814fd40 con 0x7f05e806bd60 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05ee483700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80b4030 con 0x7f05e80d13a0 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05ee483700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80b58e0 con 0x7f05e80cdc90 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e806bd60 0x7f05e815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e806bd60 0x7f05e815d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57940/0 (socket says 172.21.15.5:57940) 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 -- 172.21.15.5:0/3527604045 learned_addr learned my addr 172.21.15.5:0/3527604045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e77fe700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e80cdc90 0x7f05e814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05eca20700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e80d13a0 0x7f05e81600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:52.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 -- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e80cdc90 msgr2=0x7f05e814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e80cdc90 0x7f05e814f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 -- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e80d13a0 msgr2=0x7f05e81600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e80d13a0 0x7f05e81600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 -- 172.21.15.5:0/3527604045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d8009550 con 0x7f05e806bd60 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.990+0000 7f05e7fff700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e806bd60 0x7f05e815d180 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009c40 tx=0x7f05d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.991+0000 7f05e4ff9700 1 -- 172.21.15.5:0/3527604045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d80036a0 con 0x7f05e806bd60 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.991+0000 7f05e4ff9700 1 -- 172.21.15.5:0/3527604045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d801e070 con 0x7f05e806bd60 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.991+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e80b5bc0 con 0x7f05e806bd60 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.991+0000 7f05e4ff9700 1 -- 172.21.15.5:0/3527604045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8003a40 con 0x7f05e806bd60 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.991+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e80b42f0 con 0x7f05e806bd60 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.992+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e80b44e0 con 0x7f05e806bd60 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.992+0000 7f05e4ff9700 1 -- 172.21.15.5:0/3527604045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f05d8003ba0 con 0x7f05e806bd60 2024-05-08T23:24:52.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.993+0000 7f05e4ff9700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f05d406ac90 0x7f05d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:52.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.993+0000 7f05e4ff9700 1 -- 172.21.15.5:0/3527604045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d8020070 con 0x7f05e806bd60 2024-05-08T23:24:52.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.993+0000 7f05e77fe700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f05d406ac90 0x7f05d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:52.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.994+0000 7f05e77fe700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f05d406ac90 0x7f05d406d150 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f05e80c8450 tx=0x7f05dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:52.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:52.998+0000 7f05e4ff9700 1 -- 172.21.15.5:0/3527604045 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f05d80559f0 con 0x7f05e806bd60 2024-05-08T23:24:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.264+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f05e8074bc0 con 0x7f05e806bd60 2024-05-08T23:24:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.313+0000 7f05e4ff9700 1 -- 172.21.15.5:0/3527604045 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v50) v1 ==== 105+0+0 (secure 0 0 0) 0x7f05d8059010 con 0x7f05e806bd60 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f05d406ac90 msgr2=0x7f05d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f05d406ac90 0x7f05d406d150 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f05e80c8450 tx=0x7f05dc008040 comp rx=0 tx=0).stop 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e806bd60 msgr2=0x7f05e815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e806bd60 0x7f05e815d180 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009c40 tx=0x7f05d8004750 comp rx=0 tx=0).stop 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 shutdown_connections 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f05d406ac90 0x7f05d406d150 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e80d13a0 0x7f05e81600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e806bd60 0x7f05e815d180 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 --2- 172.21.15.5:0/3527604045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05e80cdc90 0x7f05e814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 >> 172.21.15.5:0/3527604045 conn(0x7f05e808afe0 msgr2=0x7f05e806ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 shutdown_connections 2024-05-08T23:24:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.317+0000 7f05ee483700 1 -- 172.21.15.5:0/3527604045 wait complete. 2024-05-08T23:24:53.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.471+0000 7fd09eec2700 1 Processor -- start 2024-05-08T23:24:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.471+0000 7fd09eec2700 1 -- start start 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.472+0000 7fd09eec2700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd09806bd80 0x7fd0980d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.472+0000 7fd09eec2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd09806c3d0 0x7fd09806c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.472+0000 7fd09eec2700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0980cf990 con 0x7fd09815da10 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.472+0000 7fd09eec2700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09809ab40 con 0x7fd09806bd80 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.472+0000 7fd09eec2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09806cda0 con 0x7fd09806c3d0 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.472+0000 7fd09cc5e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd09815da10 0x7fd09809a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36146/0 (socket says 172.21.15.5:36146) 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.472+0000 7fd09cc5e700 1 -- 172.21.15.5:0/1236045388 learned_addr learned my addr 172.21.15.5:0/1236045388 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd097fff700 1 --2- 172.21.15.5:0/1236045388 >> v2:172.21.15.49:3300/0 conn(0x7fd09806bd80 0x7fd0980d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd09d45f700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd09806c3d0 0x7fd09806c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254174534 0 0) 0x7fd0980cf990 con 0x7fd09815da10 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd07c0035b0 con 0x7fd09815da10 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 76098087 0 0) 0x7fd07c0035b0 con 0x7fd09815da10 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd09806c3d0 msgr2=0x7fd09806c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd09806c3d0 0x7fd09806c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 >> v2:172.21.15.49:3300/0 conn(0x7fd09806bd80 msgr2=0x7fd0980d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 --2- 172.21.15.5:0/1236045388 >> v2:172.21.15.49:3300/0 conn(0x7fd09806bd80 0x7fd0980d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980d04b0 con 0x7fd09815da10 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fd088001670 con 0x7fd09815da10 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 >> v1:172.21.15.5:6789/0 conn(0x7fd09815da10 legacy=0x7fd09809a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c003a30 0x7fd07c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c004850 0x7fd07c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd09806c3d0 0x7fd07c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09809ab40 con 0x7fd07c003a30 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09806cda0 con 0x7fd07c004850 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c00ba10 con 0x7fd09806c3d0 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd09cc5e700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c003a30 0x7fd07c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd097fff700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c004850 0x7fd07c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.473+0000 7fd09d45f700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd09806c3d0 0x7fd07c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd09cc5e700 1 -- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd09806c3d0 msgr2=0x7fd07c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd09cc5e700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd09806c3d0 0x7fd07c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd09cc5e700 1 -- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c004850 msgr2=0x7fd07c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd09cc5e700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c004850 0x7fd07c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd09cc5e700 1 -- 172.21.15.5:0/1236045388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c0035b0 con 0x7fd07c003a30 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd09cc5e700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c003a30 0x7fd07c003e50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd08800c240 tx=0x7fd07c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f47ec72b8d136 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088007c10 con 0x7fd07c003a30 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08800cde0 con 0x7fd07c003a30 2024-05-08T23:24:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.474+0000 7fd0977fe700 1 -- 172.21.15.5:0/1236045388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088005ba0 con 0x7fd07c003a30 2024-05-08T23:24:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.475+0000 7fd09eec2700 1 -- 172.21.15.5:0/1236045388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c003a30 msgr2=0x7fd07c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.475+0000 7fd09eec2700 1 --2- 172.21.15.5:0/1236045388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c003a30 0x7fd07c003e50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd08800c240 tx=0x7fd07c005880 comp rx=0 tx=0).stop 2024-05-08T23:24:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.475+0000 7fd09d45f700 1 -- 172.21.15.5:0/1236045388 reap_dead start 2024-05-08T23:24:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.475+0000 7fd09eec2700 1 -- 172.21.15.5:0/1236045388 shutdown_connections 2024-05-08T23:24:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.475+0000 7fd09eec2700 1 -- 172.21.15.5:0/1236045388 >> 172.21.15.5:0/1236045388 conn(0x7fd0980b3010 msgr2=0x7fd0980b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.475+0000 7fd09eec2700 1 -- 172.21.15.5:0/1236045388 shutdown_connections 2024-05-08T23:24:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.475+0000 7fd09eec2700 1 -- 172.21.15.5:0/1236045388 wait complete. 2024-05-08T23:24:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.476+0000 7fd09eec2700 1 Processor -- start 2024-05-08T23:24:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.476+0000 7fd09eec2700 1 -- start start 2024-05-08T23:24:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09eec2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd07c003a30 0x7fd0980b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09eec2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0980ae150 0x7fd0980b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09eec2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0980b58f0 0x7fd0980b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09eec2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980b4be0 con 0x7fd0980b58f0 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09eec2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980af500 con 0x7fd0980ae150 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09eec2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09814c140 con 0x7fd07c003a30 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09d45f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0980b58f0 0x7fd0980b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09d45f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0980b58f0 0x7fd0980b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57956/0 (socket says 172.21.15.5:57956) 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09d45f700 1 -- 172.21.15.5:0/2440001456 learned_addr learned my addr 172.21.15.5:0/2440001456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd09cc5e700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd07c003a30 0x7fd0980b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.477+0000 7fd097fff700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0980ae150 0x7fd0980b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd09d45f700 1 -- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd07c003a30 msgr2=0x7fd0980b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd09d45f700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd07c003a30 0x7fd0980b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd09d45f700 1 -- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0980ae150 msgr2=0x7fd0980b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd09d45f700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0980ae150 0x7fd0980b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd09d45f700 1 -- 172.21.15.5:0/2440001456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980b62b0 con 0x7fd0980b58f0 2024-05-08T23:24:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd09d45f700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0980b58f0 0x7fd0980b5d70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd0980cb940 tx=0x7fd090011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd0957fa700 1 -- 172.21.15.5:0/2440001456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090004300 con 0x7fd0980b58f0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd0957fa700 1 -- 172.21.15.5:0/2440001456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd090004460 con 0x7fd0980b58f0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0980ae8f0 con 0x7fd0980b58f0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd0957fa700 1 -- 172.21.15.5:0/2440001456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0900038e0 con 0x7fd0980b58f0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.478+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0980aeeb0 con 0x7fd0980b58f0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.479+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd098068000 con 0x7fd0980b58f0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.480+0000 7fd0957fa700 1 -- 172.21.15.5:0/2440001456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd090003a40 con 0x7fd0980b58f0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.480+0000 7fd0957fa700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd08006ac90 0x7fd08006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.480+0000 7fd0957fa700 1 -- 172.21.15.5:0/2440001456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0900895d0 con 0x7fd0980b58f0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.480+0000 7fd09cc5e700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd08006ac90 0x7fd08006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.481+0000 7fd09cc5e700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd08006ac90 0x7fd08006d150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd088000c00 tx=0x7fd088007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.485+0000 7fd0957fa700 1 -- 172.21.15.5:0/2440001456 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd090054840 con 0x7fd0980b58f0 2024-05-08T23:24:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.750+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0980af640 con 0x7fd0980b58f0 2024-05-08T23:24:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.750+0000 7fd0957fa700 1 -- 172.21.15.5:0/2440001456 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4744 (secure 0 0 0) 0x7fd090057e60 con 0x7fd0980b58f0 2024-05-08T23:24:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:53.750 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:24:53.314356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6525,"mds_1":6516},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6516":{"gid":6516,"name":"c","rank":1,"incarnation":51,"state":"up:creating","state_seq":1,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:24:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.752+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd08006ac90 msgr2=0x7fd08006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.752+0000 7fd09eec2700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd08006ac90 0x7fd08006d150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd088000c00 tx=0x7fd088007000 comp rx=0 tx=0).stop 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.752+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0980b58f0 msgr2=0x7fd0980b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.752+0000 7fd09eec2700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0980b58f0 0x7fd0980b5d70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd0980cb940 tx=0x7fd090011b40 comp rx=0 tx=0).stop 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.753+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 shutdown_connections 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.753+0000 7fd09eec2700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd08006ac90 0x7fd08006d150 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.753+0000 7fd09eec2700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0980b58f0 0x7fd0980b5d70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.753+0000 7fd09eec2700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd07c003a30 0x7fd0980b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.753+0000 7fd09eec2700 1 --2- 172.21.15.5:0/2440001456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0980ae150 0x7fd0980b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.753+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 >> 172.21.15.5:0/2440001456 conn(0x7fd0980b3010 msgr2=0x7fd0980b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.753+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 shutdown_connections 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:53.753+0000 7fd09eec2700 1 -- 172.21.15.5:0/2440001456 wait complete. 2024-05-08T23:24:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 51 2024-05-08T23:24:54.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:24:45.283486+0000', 'modified': '2024-05-08T23:24:53.314356+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6525, 'mds_1': 6516}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6516': {'gid': 6516, 'name': 'c', 'rank': 1, 'incarnation': 51, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.5:6833/2315721152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 2315721152}, {'type': 'v1', 'addr': '172.21.15.5:6833', 'nonce': 2315721152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_6525': {'gid': 6525, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/1169758885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 1169758885}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1169758885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:24:54.170 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6516:up:creating 2024-05-08T23:24:55.171 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.317+0000 7f8893ba5700 1 Processor -- start 2024-05-08T23:24:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.317+0000 7f8893ba5700 1 -- start start 2024-05-08T23:24:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8893ba5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f888c15ef60 0x7f888c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8893ba5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf000 0x7f888c0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8893ba5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888c0cfe30 con 0x7f888c0cf9a0 2024-05-08T23:24:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8893ba5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f888c0cd400 con 0x7f888c15ef60 2024-05-08T23:24:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8893ba5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888c09ad40 con 0x7f888c0cf000 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8891941700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f888c15ef60 0x7f888c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8891140700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf000 0x7f888c0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8891941700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f888c15ef60 0x7f888c0b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40368/0 (socket says 172.21.15.5:40368) 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8891941700 1 -- 172.21.15.5:0/2522838354 learned_addr learned my addr 172.21.15.5:0/2522838354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8891140700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf000 0x7f888c0cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:37782/0 (socket says 172.21.15.5:37782) 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.318+0000 7f8892142700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f888c0cf9a0 0x7f888c099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:45470/0 (socket says 172.21.15.5:45470) 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.319+0000 7f8891941700 1 -- 172.21.15.5:0/2522838354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf000 msgr2=0x7f888c0cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.319+0000 7f8891941700 1 --2- 172.21.15.5:0/2522838354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf000 0x7f888c0cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.319+0000 7f8891941700 1 -- 172.21.15.5:0/2522838354 >> v1:172.21.15.49:6789/0 conn(0x7f888c0cf9a0 legacy=0x7f888c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.319+0000 7f8891941700 1 -- 172.21.15.5:0/2522838354 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0ccb60 con 0x7f888c15ef60 2024-05-08T23:24:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.319+0000 7f8891941700 1 --2- 172.21.15.5:0/2522838354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c15ef60 0x7f888c0b2340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f88880098f0 tx=0x7f8888009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1414675e3f0273e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.322+0000 7f889093f700 1 -- 172.21.15.5:0/2522838354 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888011830 con 0x7f888c15ef60 2024-05-08T23:24:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.322+0000 7f889093f700 1 -- 172.21.15.5:0/2522838354 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8888011990 con 0x7f888c15ef60 2024-05-08T23:24:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.322+0000 7f889093f700 1 -- 172.21.15.5:0/2522838354 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888011d20 con 0x7f888c15ef60 2024-05-08T23:24:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.323+0000 7f8893ba5700 1 -- 172.21.15.5:0/2522838354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c15ef60 msgr2=0x7f888c0b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.323+0000 7f8893ba5700 1 --2- 172.21.15.5:0/2522838354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c15ef60 0x7f888c0b2340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f88880098f0 tx=0x7f8888009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.323+0000 7f8893ba5700 1 -- 172.21.15.5:0/2522838354 shutdown_connections 2024-05-08T23:24:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.323+0000 7f8893ba5700 1 --2- 172.21.15.5:0/2522838354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c15ef60 0x7f888c0b2340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.323+0000 7f8893ba5700 1 --2- 172.21.15.5:0/2522838354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf000 0x7f888c0cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.323+0000 7f8893ba5700 1 -- 172.21.15.5:0/2522838354 >> 172.21.15.5:0/2522838354 conn(0x7f888c0b2f20 msgr2=0x7f888c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.323+0000 7f8893ba5700 1 -- 172.21.15.5:0/2522838354 shutdown_connections 2024-05-08T23:24:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.323+0000 7f8893ba5700 1 -- 172.21.15.5:0/2522838354 wait complete. 2024-05-08T23:24:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.325+0000 7f8893ba5700 1 Processor -- start 2024-05-08T23:24:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.325+0000 7f8893ba5700 1 -- start start 2024-05-08T23:24:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.325+0000 7f8893ba5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c0cf000 0x7f888c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8893ba5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf9a0 0x7f888c158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8893ba5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c15ef60 0x7f888c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8893ba5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c159af0 con 0x7f888c15ef60 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8893ba5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c1f15d0 con 0x7f888c0cf000 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8893ba5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888c1f18b0 con 0x7f888c0cf9a0 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c15ef60 0x7f888c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c15ef60 0x7f888c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57972/0 (socket says 172.21.15.5:57972) 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 -- 172.21.15.5:0/1689050352 learned_addr learned my addr 172.21.15.5:0/1689050352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8891941700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c0cf000 0x7f888c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8891140700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf9a0 0x7f888c158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 -- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf9a0 msgr2=0x7f888c158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf9a0 0x7f888c158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 -- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c0cf000 msgr2=0x7f888c1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c0cf000 0x7f888c1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 -- 172.21.15.5:0/1689050352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8888009550 con 0x7f888c15ef60 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8892142700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c15ef60 0x7f888c0af3e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f888400cbc0 tx=0x7f888400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f88827fc700 1 -- 172.21.15.5:0/1689050352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888400a9e0 con 0x7f888c15ef60 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f88827fc700 1 -- 172.21.15.5:0/1689050352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888400ab40 con 0x7f888c15ef60 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c1f1b90 con 0x7f888c15ef60 2024-05-08T23:24:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f88827fc700 1 -- 172.21.15.5:0/1689050352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884007070 con 0x7f888c15ef60 2024-05-08T23:24:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.326+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c158fa0 con 0x7f888c15ef60 2024-05-08T23:24:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.328+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c068000 con 0x7f888c15ef60 2024-05-08T23:24:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.328+0000 7f88827fc700 1 -- 172.21.15.5:0/1689050352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f88840071d0 con 0x7f888c15ef60 2024-05-08T23:24:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.328+0000 7f88827fc700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f887006ac90 0x7f887006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.328+0000 7f88827fc700 1 -- 172.21.15.5:0/1689050352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8884088dc0 con 0x7f888c15ef60 2024-05-08T23:24:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.329+0000 7f8891941700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f887006ac90 0x7f887006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.329+0000 7f8891941700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f887006ac90 0x7f887006d150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8888000c00 tx=0x7f8888008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.333+0000 7f88827fc700 1 -- 172.21.15.5:0/1689050352 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f88840540e0 con 0x7f888c15ef60 2024-05-08T23:24:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.600+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f888c0cedc0 con 0x7f888c15ef60 2024-05-08T23:24:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.601+0000 7f88827fc700 1 -- 172.21.15.5:0/1689050352 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4756 (secure 0 0 0) 0x7f8884017070 con 0x7f888c15ef60 2024-05-08T23:24:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:55.601 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:24:55.322507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6525,"mds_1":6516,"mds_2":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":2,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6516":{"gid":6516,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:24:55.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f887006ac90 msgr2=0x7f887006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:55.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f887006ac90 0x7f887006d150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8888000c00 tx=0x7f8888008030 comp rx=0 tx=0).stop 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c15ef60 msgr2=0x7f888c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c15ef60 0x7f888c0af3e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f888400cbc0 tx=0x7f888400ced0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 shutdown_connections 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f887006ac90 0x7f887006d150 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c15ef60 0x7f888c0af3e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c0cf000 0x7f888c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 --2- 172.21.15.5:0/1689050352 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f888c0cf9a0 0x7f888c158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 >> 172.21.15.5:0/1689050352 conn(0x7f888c0b2f20 msgr2=0x7f888c06f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 shutdown_connections 2024-05-08T23:24:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:55.603+0000 7f8893ba5700 1 -- 172.21.15.5:0/1689050352 wait complete. 2024-05-08T23:24:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-05-08T23:24:55.812 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:24:55.812+0000 7f8300507700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:24:55.812 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:24:55.812+0000 7f8300507700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T23:24:56.020 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:24:45.283486+0000', 'modified': '2024-05-08T23:24:55.322507+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6525, 'mds_1': 6516, 'mds_2': 6193}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6193': {'gid': 6193, 'name': 'd', 'rank': 2, 'incarnation': 53, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.49:6836/3197377913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 3197377913}, {'type': 'v1', 'addr': '172.21.15.49:6836', 'nonce': 3197377913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_6516': {'gid': 6516, 'name': 'c', 'rank': 1, 'incarnation': 51, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.5:6833/2315721152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 2315721152}, {'type': 'v1', 'addr': '172.21.15.5:6833', 'nonce': 2315721152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_6525': {'gid': 6525, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/1169758885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 1169758885}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1169758885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:24:56.021 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:24:56.021 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T23:24:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.168+0000 7f3456232700 1 Processor -- start 2024-05-08T23:24:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.168+0000 7f3456232700 1 -- start start 2024-05-08T23:24:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f3456232700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f345015b810 0x7f34500cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f3456232700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34500ccf80 con 0x7f34500cfcd0 2024-05-08T23:24:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f3456232700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34500cd260 con 0x7f34500af730 2024-05-08T23:24:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f3456232700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34500cd540 con 0x7f345015b810 2024-05-08T23:24:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f344ffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f34500cfcd0 0x7f345006df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36148/0 (socket says 172.21.15.5:36148) 2024-05-08T23:24:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f344ffff700 1 -- 172.21.15.5:0/2015414601 learned_addr learned my addr 172.21.15.5:0/2015414601 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f344f7fe700 1 --2- 172.21.15.5:0/2015414601 >> v2:172.21.15.49:3300/0 conn(0x7f345015b810 0x7f34500cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795874220 0 0) 0x7f34500ccf80 con 0x7f34500cfcd0 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.169+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f343c0035b0 con 0x7f34500cfcd0 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3125589453 0 0) 0x7f343c0035b0 con 0x7f34500cfcd0 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 >> v2:172.21.15.49:3300/0 conn(0x7f345015b810 msgr2=0x7f34500cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 --2- 172.21.15.5:0/2015414601 >> v2:172.21.15.49:3300/0 conn(0x7f345015b810 0x7f34500cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 >> v1:172.21.15.49:6789/0 conn(0x7f34500af730 legacy=0x7f34500cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500cd820 con 0x7f34500cfcd0 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f34480019f0 con 0x7f34500cfcd0 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 >> v1:172.21.15.5:6789/0 conn(0x7f34500cfcd0 legacy=0x7f345006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c003a00 0x7f343c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f343c004400 0x7f343c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f343c0050d0 0x7f343c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500cd540 con 0x7f343c0050d0 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c00c030 con 0x7f343c003a00 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c00c1f0 con 0x7f343c004400 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344ffff700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f343c0050d0 0x7f343c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344f7fe700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c003a00 0x7f343c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344effd700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f343c004400 0x7f343c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344ffff700 1 -- 172.21.15.5:0/2015414601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f343c004400 msgr2=0x7f343c0048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344ffff700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f343c004400 0x7f343c0048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344ffff700 1 -- 172.21.15.5:0/2015414601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c003a00 msgr2=0x7f343c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344ffff700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c003a00 0x7f343c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344ffff700 1 -- 172.21.15.5:0/2015414601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0035b0 con 0x7f343c0050d0 2024-05-08T23:24:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344ffff700 1 -- 172.21.15.5:0/2015414601 reap_dead start 2024-05-08T23:24:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.170+0000 7f344ffff700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f343c0050d0 0x7f343c00ba10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f344800a830 tx=0x7f343c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf62e1478bf8770 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448012600 con 0x7f343c0050d0 2024-05-08T23:24:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448012780 con 0x7f343c0050d0 2024-05-08T23:24:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f344e7fc700 1 -- 172.21.15.5:0/2015414601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448012b30 con 0x7f343c0050d0 2024-05-08T23:24:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f3456232700 1 -- 172.21.15.5:0/2015414601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f343c0050d0 msgr2=0x7f343c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f3456232700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f343c0050d0 0x7f343c00ba10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f344800a830 tx=0x7f343c005880 comp rx=0 tx=0).stop 2024-05-08T23:24:56.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f3456232700 1 -- 172.21.15.5:0/2015414601 shutdown_connections 2024-05-08T23:24:56.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f3456232700 1 --2- 172.21.15.5:0/2015414601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f343c0050d0 0x7f343c00ba10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f3456232700 1 -- 172.21.15.5:0/2015414601 >> 172.21.15.5:0/2015414601 conn(0x7f34500b1f80 msgr2=0x7f34500b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f3456232700 1 -- 172.21.15.5:0/2015414601 shutdown_connections 2024-05-08T23:24:56.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.171+0000 7f3456232700 1 -- 172.21.15.5:0/2015414601 wait complete. 2024-05-08T23:24:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.173+0000 7f3456232700 1 Processor -- start 2024-05-08T23:24:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.173+0000 7f3456232700 1 -- start start 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3456232700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c0050d0 0x7f34500b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3456232700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34500b4890 0x7f34500b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3456232700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34500ae2b0 0x7f34500ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3456232700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500aeca0 con 0x7f34500b4890 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3456232700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500b3420 con 0x7f343c0050d0 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3456232700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345015a870 con 0x7f34500ae2b0 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34500b4890 0x7f34500b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34500b4890 0x7f34500b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57990/0 (socket says 172.21.15.5:57990) 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 -- 172.21.15.5:0/2983125617 learned_addr learned my addr 172.21.15.5:0/2983125617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344f7fe700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c0050d0 0x7f34500b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344ffff700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34500ae2b0 0x7f34500ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 -- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34500ae2b0 msgr2=0x7f34500ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34500ae2b0 0x7f34500ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 -- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c0050d0 msgr2=0x7f34500b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c0050d0 0x7f34500b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 -- 172.21.15.5:0/2983125617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500afd60 con 0x7f34500b4890 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344ffff700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34500ae2b0 0x7f34500ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f344effd700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34500b4890 0x7f34500b5700 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f343800f7f0 tx=0x7f343800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3447fff700 1 -- 172.21.15.5:0/2983125617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34380052e0 con 0x7f34500b4890 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3447fff700 1 -- 172.21.15.5:0/2983125617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3438005460 con 0x7f34500b4890 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3447fff700 1 -- 172.21.15.5:0/2983125617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34380038c0 con 0x7f34500b4890 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3456232700 1 -- 172.21.15.5:0/2983125617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34500c5da0 con 0x7f34500b4890 2024-05-08T23:24:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.174+0000 7f3456232700 1 -- 172.21.15.5:0/2983125617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3450158770 con 0x7f34500b4890 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.176+0000 7f3447fff700 1 -- 172.21.15.5:0/2983125617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3438003a20 con 0x7f34500b4890 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.177+0000 7f3447fff700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f343006ac90 0x7f343006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.177+0000 7f3447fff700 1 -- 172.21.15.5:0/2983125617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3438089100 con 0x7f34500b4890 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.177+0000 7f344f7fe700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f343006ac90 0x7f343006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.178+0000 7f344f7fe700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f343006ac90 0x7f343006d150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3440009be0 tx=0x7f3440009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.179+0000 7f34623ecb80 1 Processor -- start 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.179+0000 7f34623ecb80 1 -- start start 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 0x5644566460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x56445664b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x564456675200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f34623ecb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644565a1100 con 0x564456674d80 2024-05-08T23:24:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f34623ecb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445659eac0 con 0x56445666d750 2024-05-08T23:24:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f34623ecb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56445659a3b0 con 0x564456648320 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x564456675200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x564456675200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58006/0 (socket says 172.21.15.5:58006) 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344ffff700 1 -- 172.21.15.5:0/3686273646 learned_addr learned my addr 172.21.15.5:0/3686273646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344effd700 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x56445664b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344f7fe700 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 0x5644566460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344ffff700 1 -- 172.21.15.5:0/3686273646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 msgr2=0x56445664b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344ffff700 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x56445664b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344ffff700 1 -- 172.21.15.5:0/3686273646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 msgr2=0x5644566460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344ffff700 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 0x5644566460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.180+0000 7f344ffff700 1 -- 172.21.15.5:0/3686273646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56445641b360 con 0x564456674d80 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.181+0000 7f344ffff700 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x564456675200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f34500ce5b0 tx=0x7f3448009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26be9b3ebcbc04d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.181+0000 7f34457fa700 1 -- 172.21.15.5:0/3686273646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448012600 con 0x564456674d80 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.181+0000 7f34457fa700 1 -- 172.21.15.5:0/3686273646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448001250 con 0x564456674d80 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.181+0000 7f34457fa700 1 -- 172.21.15.5:0/3686273646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480019b0 con 0x564456674d80 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.181+0000 7f34623ecb80 1 -- 172.21.15.5:0/3686273646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 msgr2=0x564456675200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.181+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x564456675200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f34500ce5b0 tx=0x7f3448009030 comp rx=0 tx=0).stop 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 -- 172.21.15.5:0/3686273646 shutdown_connections 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x564456675200 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 0x5644566460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3686273646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x56445664b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 -- 172.21.15.5:0/3686273646 >> 172.21.15.5:0/3686273646 conn(0x5644565a81c0 msgr2=0x56445668fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 -- 172.21.15.5:0/3686273646 shutdown_connections 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 -- 172.21.15.5:0/3686273646 wait complete. 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 -- start start 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.182+0000 7f34623ecb80 1 Processor -- start 2024-05-08T23:24:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x56445666ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456695460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456677160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f34623ecb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564456536510 con 0x56445666d750 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f34623ecb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644565361b0 con 0x564456674d80 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f34623ecb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56445641d650 con 0x564456648320 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f34623ecb80 10 client.0 _ll_register_callbacks cb 0x56445653fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f344effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456695460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f344effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456695460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58022/0 (socket says 172.21.15.5:58022) 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f344effd700 1 -- 172.21.15.5:0/3689535596 learned_addr learned my addr 172.21.15.5:0/3689535596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f344f7fe700 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x56445666ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.183+0000 7f344ffff700 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456677160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f344effd700 1 -- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 msgr2=0x56445666ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f344effd700 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x56445666ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f344effd700 1 -- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 msgr2=0x564456677160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f344effd700 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456677160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f344effd700 1 -- 172.21.15.5:0/3689535596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c0035b0 con 0x56445666d750 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f344effd700 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456695460 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3438087620 tx=0x7f3438057fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d779d41120bf6d6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f342e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f342e7fc700 1 -- 172.21.15.5:0/3689535596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343805e070 con 0x56445666d750 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f342e7fc700 1 -- 172.21.15.5:0/3689535596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3438060300 con 0x56445666d750 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f34623ecb80 1 -- 172.21.15.5:0/3689535596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644564f3220 con 0x56445666d750 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f342e7fc700 1 -- 172.21.15.5:0/3689535596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3438058420 con 0x56445666d750 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f34623ecb80 10 client.6693 fetch_fsmap learned FSMap version 54 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f34623ecb80 1 -- 172.21.15.5:0/3689535596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x5644563c7b10 con 0x56445666d750 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f342e7fc700 1 -- 172.21.15.5:0/3689535596 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5644564f3220 con 0x56445666d750 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f342e7fc700 1 -- 172.21.15.5:0/3689535596 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f34380871f0 con 0x56445666d750 2024-05-08T23:24:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f34623ecb80 10 client.6693 fetch_fsmap finished waiting for FSMap version 54 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f34623ecb80 10 client.6693 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.184+0000 7f34623ecb80 20 client.6693 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.185+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x564456698040 0x5644566931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.185+0000 7f34623ecb80 4 client.6693 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.185+0000 7f34623ecb80 1 -- 172.21.15.5:0/3689535596 --> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564456619790 con 0x564456698040 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.185+0000 7f344f7fe700 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x564456698040 0x5644566931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.185+0000 7f342e7fc700 10 client.6693 ms_handle_connect on v2:172.21.15.49:6834/3197377913 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.185+0000 7f344f7fe700 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x564456698040 0x5644566931b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16a236c70cec0364 server_cookie=fdda52ba19752ff0 in_seq=0 out_seq=0 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f342e7fc700 1 -- 172.21.15.5:0/3689535596 <== mds.2 v2:172.21.15.49:6834/3197377913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564456619790 con 0x564456698040 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f342e7fc700 10 client.6693 handle_command_reply: tid=0 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 client.6693 shutdown 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 -- 172.21.15.5:0/3689535596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 msgr2=0x564456695460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456695460 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3438087620 tx=0x7f3438057fc0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 -- 172.21.15.5:0/3689535596 shutdown_connections 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x564456698040 0x5644566931b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456677160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x56445666ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3689535596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456695460 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 -- 172.21.15.5:0/3689535596 >> 172.21.15.5:0/3689535596 conn(0x5644565a81c0 msgr2=0x564456674650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f342e7fc700 0 client.6693 ms_handle_reset on v2:172.21.15.49:6834/3197377913 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 -- 172.21.15.5:0/3689535596 shutdown_connections 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 1 -- 172.21.15.5:0/3689535596 wait complete. 2024-05-08T23:24:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.186+0000 7f34623ecb80 20 client.6693 trim_cache size 0 max 16384 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.205+0000 7f34623ecb80 1 Processor -- start 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.205+0000 7f34623ecb80 1 -- start start 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x564456690950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456643c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456678540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f34623ecb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564456678330 con 0x56445666d750 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f34623ecb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564456690660 con 0x564456674d80 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f34623ecb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564456696fd0 con 0x564456648320 2024-05-08T23:24:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f344effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456643c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f344effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456643c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58028/0 (socket says 172.21.15.5:58028) 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.206+0000 7f344effd700 1 -- 172.21.15.5:0/3122366927 learned_addr learned my addr 172.21.15.5:0/3122366927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344f7fe700 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x564456690950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344ffff700 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456678540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344effd700 1 -- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 msgr2=0x564456690950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344effd700 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x564456690950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344effd700 1 -- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 msgr2=0x564456678540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344effd700 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456678540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344effd700 1 -- 172.21.15.5:0/3122366927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644566727d0 con 0x56445666d750 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344ffff700 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456678540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f344effd700 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456643c90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3438001160 tx=0x7f343806c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414a9f190efd11cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f3445ffb700 1 -- 172.21.15.5:0/3122366927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438053db0 con 0x56445666d750 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f3445ffb700 1 -- 172.21.15.5:0/3122366927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3438060300 con 0x56445666d750 2024-05-08T23:24:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.207+0000 7f3445ffb700 1 -- 172.21.15.5:0/3122366927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34380752f0 con 0x56445666d750 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 -- 172.21.15.5:0/3122366927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 msgr2=0x564456643c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456643c90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3438001160 tx=0x7f343806c000 comp rx=0 tx=0).stop 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 -- 172.21.15.5:0/3122366927 shutdown_connections 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564456674d80 0x564456678540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x564456690950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 --2- 172.21.15.5:0/3122366927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56445666d750 0x564456643c90 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 -- 172.21.15.5:0/3122366927 >> 172.21.15.5:0/3122366927 conn(0x5644565a81c0 msgr2=0x564456674650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 -- 172.21.15.5:0/3122366927 shutdown_connections 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 -- 172.21.15.5:0/3122366927 wait complete. 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.208+0000 7f34623ecb80 1 -- start start 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f34623ecb80 1 Processor -- start 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x5644566f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 0x564456675160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f34623ecb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x5644566756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f34623ecb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445664be20 con 0x564456674d80 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f34623ecb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564456693490 con 0x56445666d750 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f34623ecb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564456648830 con 0x564456648320 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f34623ecb80 10 client.0 _ll_register_callbacks cb 0x56445653fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f344ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x5644566756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.209+0000 7f344ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x5644566756a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58042/0 (socket says 172.21.15.5:58042) 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344ffff700 1 -- 172.21.15.5:0/1331729408 learned_addr learned my addr 172.21.15.5:0/1331729408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344f7fe700 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x5644566f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344effd700 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 0x564456675160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344ffff700 1 -- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 msgr2=0x5644566f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344ffff700 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x5644566f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344ffff700 1 -- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 msgr2=0x564456675160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344ffff700 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 0x564456675160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344ffff700 1 -- 172.21.15.5:0/1331729408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644563c7b10 con 0x564456674d80 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f344ffff700 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x5644566756a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3448012630 tx=0x7f3448007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c2f352b8e09493 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f342e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f342e7fc700 1 -- 172.21.15.5:0/1331729408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344802d070 con 0x564456674d80 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f342e7fc700 1 -- 172.21.15.5:0/1331729408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34480019c0 con 0x564456674d80 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f34623ecb80 1 -- 172.21.15.5:0/1331729408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3448001250 con 0x564456674d80 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f342e7fc700 1 -- 172.21.15.5:0/1331729408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3448001b80 con 0x564456674d80 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f34623ecb80 10 client.6699 fetch_fsmap learned FSMap version 54 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.210+0000 7f34623ecb80 1 -- 172.21.15.5:0/1331729408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x564456560510 con 0x564456674d80 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f342e7fc700 1 -- 172.21.15.5:0/1331729408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448001250 con 0x564456674d80 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f342e7fc700 1 -- 172.21.15.5:0/1331729408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f3448033070 con 0x564456674d80 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f34623ecb80 10 client.6699 fetch_fsmap finished waiting for FSMap version 54 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f34623ecb80 10 client.6699 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:24:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f34623ecb80 20 client.6699 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f34623ecb80 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x564456645120 0x564456645500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f34623ecb80 4 client.6699 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f34623ecb80 1 -- 172.21.15.5:0/1331729408 --> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564456592e70 con 0x564456645120 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f344f7fe700 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x564456645120 0x564456645500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f342e7fc700 10 client.6699 ms_handle_connect on v2:172.21.15.49:6834/3197377913 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.211+0000 7f344f7fe700 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x564456645120 0x564456645500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e2a1247df84d377 server_cookie=5803daf7ea2b9491 in_seq=0 out_seq=0 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f342e7fc700 1 -- 172.21.15.5:0/1331729408 <== mds.2 v2:172.21.15.49:6834/3197377913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x564456592e70 con 0x564456645120 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f342e7fc700 10 client.6699 handle_command_reply: tid=0 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 client.6699 shutdown 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 -- 172.21.15.5:0/1331729408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 msgr2=0x5644566756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x5644566756a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3448012630 tx=0x7f3448007000 comp rx=0 tx=0).stop 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 -- 172.21.15.5:0/1331729408 shutdown_connections 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x564456645120 0x564456645500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564456674d80 0x5644566756a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564456648320 0x5644566f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f342e7fc700 0 client.6699 ms_handle_reset on v2:172.21.15.49:6834/3197377913 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 --2- 172.21.15.5:0/1331729408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56445666d750 0x564456675160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 -- 172.21.15.5:0/1331729408 >> 172.21.15.5:0/1331729408 conn(0x5644565a81c0 msgr2=0x564456678540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 -- 172.21.15.5:0/1331729408 shutdown_connections 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 1 -- 172.21.15.5:0/1331729408 wait complete. 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f34623ecb80 20 client.6699 trim_cache size 0 max 16384 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:56.221 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6193, 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 1.890152061, 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 54, 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 60, 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 60, 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 16.189327714000001 2024-05-08T23:24:56.222 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 -- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f343006ac90 msgr2=0x7f343006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f343006ac90 0x7f343006d150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3440009be0 tx=0x7f3440009210 comp rx=0 tx=0).stop 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 -- 172.21.15.5:0/2983125617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34500b4890 msgr2=0x7f34500b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34500b4890 0x7f34500b5700 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f343800f7f0 tx=0x7f343800fbb0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 -- 172.21.15.5:0/2983125617 shutdown_connections 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f343006ac90 0x7f343006d150 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f34500ae2b0 0x7f34500ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f343c0050d0 0x7f34500b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.212+0000 7f3456232700 1 --2- 172.21.15.5:0/2983125617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34500b4890 0x7f34500b5700 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.213+0000 7f3456232700 1 -- 172.21.15.5:0/2983125617 >> 172.21.15.5:0/2983125617 conn(0x7f34500b1f80 msgr2=0x7f345006de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.213+0000 7f3456232700 1 -- 172.21.15.5:0/2983125617 shutdown_connections 2024-05-08T23:24:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.213+0000 7f3456232700 1 -- 172.21.15.5:0/2983125617 wait complete. 2024-05-08T23:24:56.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:24:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.365+0000 7f1fc8c08700 1 Processor -- start 2024-05-08T23:24:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.365+0000 7f1fc8c08700 1 -- start start 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.366+0000 7f1fc8c08700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fc40cf9e0 0x7f1fc409a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.366+0000 7f1fc8c08700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1fc40b18a0 0x7f1fc40b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.366+0000 7f1fc8c08700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc40b1410 con 0x7f1fc415b900 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc8c08700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc40b0e30 con 0x7f1fc40b18a0 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc8c08700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40cffd0 con 0x7f1fc40cf9e0 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc259c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1fc415b900 0x7f1fc4099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36156/0 (socket says 172.21.15.5:36156) 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc259c700 1 -- 172.21.15.5:0/317297076 learned_addr learned my addr 172.21.15.5:0/317297076 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fc40cf9e0 0x7f1fc409a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/317297076 >> v2:172.21.15.49:3300/0 conn(0x7f1fc40b18a0 0x7f1fc40b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632572116 0 0) 0x7f1fc40b1410 con 0x7f1fc415b900 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fac0035b0 con 0x7f1fc415b900 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 820171375 0 0) 0x7f1fac0035b0 con 0x7f1fc415b900 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fc40cf9e0 msgr2=0x7f1fc409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fc40cf9e0 0x7f1fc409a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 >> v2:172.21.15.49:3300/0 conn(0x7f1fc40b18a0 msgr2=0x7f1fc40b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 --2- 172.21.15.5:0/317297076 >> v2:172.21.15.49:3300/0 conn(0x7f1fc40b18a0 0x7f1fc40b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc40b10a0 con 0x7f1fc415b900 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f1fb4001670 con 0x7f1fc415b900 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 >> v1:172.21.15.5:6789/0 conn(0x7f1fc415b900 legacy=0x7f1fc4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fac003a00 0x7f1fac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fc40cf9e0 0x7f1fac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fac004d90 0x7f1fac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40b0e30 con 0x7f1fac004d90 2024-05-08T23:24:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40cffd0 con 0x7f1fac003a00 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac00ba10 con 0x7f1fc40cf9e0 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.367+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fac004d90 0x7f1fac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc259c700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fac003a00 0x7f1fac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fc40cf9e0 0x7f1fac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fc40cf9e0 msgr2=0x7f1fac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fc40cf9e0 0x7f1fac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fac003a00 msgr2=0x7f1fac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fac003a00 0x7f1fac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/317297076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac0035b0 con 0x7f1fac004d90 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fac004d90 0x7f1fac005210 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009fc0 tx=0x7f1fac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e0525792fc53ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00ec30 con 0x7f1fac004d90 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc00f040 con 0x7f1fac004d90 2024-05-08T23:24:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.368+0000 7f1fc159a700 1 -- 172.21.15.5:0/317297076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc014480 con 0x7f1fac004d90 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.369+0000 7f1fc8c08700 1 -- 172.21.15.5:0/317297076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fac004d90 msgr2=0x7f1fac005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.369+0000 7f1fc8c08700 1 --2- 172.21.15.5:0/317297076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fac004d90 0x7f1fac005210 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009fc0 tx=0x7f1fac005880 comp rx=0 tx=0).stop 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.369+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/317297076 reap_dead start 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.369+0000 7f1fc8c08700 1 -- 172.21.15.5:0/317297076 shutdown_connections 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.369+0000 7f1fc8c08700 1 -- 172.21.15.5:0/317297076 >> 172.21.15.5:0/317297076 conn(0x7f1fc40b2f20 msgr2=0x7f1fc40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.369+0000 7f1fc8c08700 1 -- 172.21.15.5:0/317297076 shutdown_connections 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.369+0000 7f1fc8c08700 1 -- 172.21.15.5:0/317297076 wait complete. 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.370+0000 7f1fc8c08700 1 Processor -- start 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.371+0000 7f1fc8c08700 1 -- start start 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.371+0000 7f1fc8c08700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fac004d90 0x7f1fc41f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.371+0000 7f1fc8c08700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc41f6db0 0x7f1fc40afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.371+0000 7f1fc8c08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc415e2c0 0x7f1fc415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.371+0000 7f1fc8c08700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40b55b0 con 0x7f1fc415e2c0 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.371+0000 7f1fc8c08700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41f7350 con 0x7f1fc41f6db0 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.371+0000 7f1fc8c08700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41f7630 con 0x7f1fac004d90 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.371+0000 7f1fc2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc415e2c0 0x7f1fc415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc415e2c0 0x7f1fc415e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58070/0 (socket says 172.21.15.5:58070) 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/3263429457 learned_addr learned my addr 172.21.15.5:0/3263429457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc259c700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fac004d90 0x7f1fc41f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc41f6db0 0x7f1fc40afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fac004d90 msgr2=0x7f1fc41f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fac004d90 0x7f1fc41f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc41f6db0 msgr2=0x7f1fc40afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc41f6db0 0x7f1fc40afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/3263429457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc41f7910 con 0x7f1fc415e2c0 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc415e2c0 0x7f1fc415e740 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc017040 tx=0x7f1fbc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fb2ffd700 1 -- 172.21.15.5:0/3263429457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc007000 con 0x7f1fc415e2c0 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fb2ffd700 1 -- 172.21.15.5:0/3263429457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc014c70 con 0x7f1fc415e2c0 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc8c08700 1 -- 172.21.15.5:0/3263429457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc41f7bf0 con 0x7f1fc415e2c0 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fb2ffd700 1 -- 172.21.15.5:0/3263429457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0276b0 con 0x7f1fc415e2c0 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.372+0000 7f1fc8c08700 1 -- 172.21.15.5:0/3263429457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc415ec80 con 0x7f1fc415e2c0 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.374+0000 7f1fb2ffd700 1 -- 172.21.15.5:0/3263429457 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1fbc024070 con 0x7f1fc415e2c0 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.374+0000 7f1fb2ffd700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1fa406ac90 0x7f1fa406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.374+0000 7f1fb2ffd700 1 -- 172.21.15.5:0/3263429457 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fbc08afb0 con 0x7f1fc415e2c0 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.375+0000 7f1fc259c700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1fa406ac90 0x7f1fa406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.375+0000 7f1fc259c700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1fa406ac90 0x7f1fa406d150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4000c00 tx=0x7f1fb4007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.376+0000 7f1fd4dc2b80 1 Processor -- start 2024-05-08T23:24:56.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.377+0000 7f1fd4dc2b80 1 -- start start 2024-05-08T23:24:56.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.377+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.377+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 0x55d3e6a48050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.377+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a71d80 0x55d3e6a72200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.377+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e699e100 con 0x55d3e6a6a750 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e699bac0 con 0x55d3e6a45320 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e69973b0 con 0x55d3e6a71d80 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58074/0 (socket says 172.21.15.5:58074) 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 -- 172.21.15.5:0/945550540 learned_addr learned my addr 172.21.15.5:0/945550540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a71d80 0x55d3e6a72200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 0x55d3e6a48050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 -- 172.21.15.5:0/945550540 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a71d80 msgr2=0x55d3e6a72200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a71d80 0x55d3e6a72200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 -- 172.21.15.5:0/945550540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 msgr2=0x55d3e6a48050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 0x55d3e6a48050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 -- 172.21.15.5:0/945550540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e6818360 con 0x55d3e6a6a750 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fc259c700 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a430c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4007440 tx=0x7f1fb400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30ea897a284f6bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fa3fff700 1 -- 172.21.15.5:0/945550540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb401a040 con 0x55d3e6a6a750 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fa3fff700 1 -- 172.21.15.5:0/945550540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb40158d0 con 0x55d3e6a6a750 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.378+0000 7f1fa3fff700 1 -- 172.21.15.5:0/945550540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4015c40 con 0x55d3e6a6a750 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/945550540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 msgr2=0x55d3e6a430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a430c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4007440 tx=0x7f1fb400c000 comp rx=0 tx=0).stop 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/945550540 shutdown_connections 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a71d80 0x55d3e6a72200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a430c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/945550540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 0x55d3e6a48050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/945550540 >> 172.21.15.5:0/945550540 conn(0x55d3e69a51c0 msgr2=0x55d3e6a8cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/945550540 shutdown_connections 2024-05-08T23:24:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.379+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/945550540 wait complete. 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.380+0000 7f1fd4dc2b80 1 -- start start 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.380+0000 7f1fd4dc2b80 1 Processor -- start 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6a92110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a6a750 0x55d3e6a755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a74060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6933510 con 0x55d3e6a71d80 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e69331b0 con 0x55d3e6a6a750 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e681a650 con 0x55d3e6a45320 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fd4dc2b80 10 client.0 _ll_register_callbacks cb 0x55d3e693cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fc259c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6a92110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fc1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a6a750 0x55d3e6a755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fc2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a74060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fc259c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6a92110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:37896/0 (socket says 172.21.15.5:37896) 2024-05-08T23:24:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fc1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a6a750 0x55d3e6a755c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40520/0 (socket says 172.21.15.5:40520) 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.381+0000 7f1fc259c700 1 -- 172.21.15.5:0/4068956939 learned_addr learned my addr 172.21.15.5:0/4068956939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fc259c700 1 -- 172.21.15.5:0/4068956939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a6a750 msgr2=0x55d3e6a755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fc259c700 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a6a750 0x55d3e6a755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fc259c700 1 -- 172.21.15.5:0/4068956939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 msgr2=0x55d3e6a74060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fc259c700 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a74060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fc259c700 1 -- 172.21.15.5:0/4068956939 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc40b10a0 con 0x55d3e6a45320 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a74060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fc259c700 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6a92110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4005fd0 tx=0x7f1fb4001a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f86114ba79af3e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3301/0 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fa17fa700 1 -- 172.21.15.5:0/4068956939 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40242e0 con 0x55d3e6a45320 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/4068956939 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3e68f0220 con 0x55d3e6a45320 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fa17fa700 1 -- 172.21.15.5:0/4068956939 <== mon.2 v2:172.21.15.49:3301/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fb40246c0 con 0x55d3e6a45320 2024-05-08T23:24:56.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fa17fa700 1 -- 172.21.15.5:0/4068956939 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb4024880 con 0x55d3e6a45320 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fd4dc2b80 10 client.5798 fetch_fsmap learned FSMap version 54 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.382+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/4068956939 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55d3e67c4b10 con 0x55d3e6a45320 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fa17fa700 1 -- 172.21.15.5:0/4068956939 <== mon.2 v2:172.21.15.49:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d3e68f0220 con 0x55d3e6a45320 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fa17fa700 1 -- 172.21.15.5:0/4068956939 <== mon.2 v2:172.21.15.49:3301/0 5 ==== fsmap(e 54) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f1fb4020040 con 0x55d3e6a45320 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fd4dc2b80 10 client.5798 fetch_fsmap finished waiting for FSMap version 54 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fd4dc2b80 10 client.5798 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fd4dc2b80 20 client.5798 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x55d3e6a8e200 0x55d3e6a8e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fd4dc2b80 4 client.5798 mds_command: new command op to 6516 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/4068956939 --> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3e6a16790 con 0x55d3e6a8e200 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x55d3e6a8e200 0x55d3e6a8e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fa17fa700 10 client.5798 ms_handle_connect on v2:172.21.15.5:6832/2315721152 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.383+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x55d3e6a8e200 0x55d3e6a8e5e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d0907f803b0951f server_cookie=b33438db863386f5 in_seq=0 out_seq=0 2024-05-08T23:24:56.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fa17fa700 1 -- 172.21.15.5:0/4068956939 <== mds.1 v2:172.21.15.5:6832/2315721152 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3e6a16790 con 0x55d3e6a8e200 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fa17fa700 10 client.5798 handle_command_reply: tid=0 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 client.5798 shutdown 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/4068956939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 msgr2=0x55d3e6a92110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6a92110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4005fd0 tx=0x7f1fb4001a00 comp rx=0 tx=0).stop 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/4068956939 shutdown_connections 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a74060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x55d3e6a8e200 0x55d3e6a8e5e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6a92110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/4068956939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a6a750 0x55d3e6a755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/4068956939 >> 172.21.15.5:0/4068956939 conn(0x55d3e69a51c0 msgr2=0x55d3e6a6f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fa17fa700 0 client.5798 ms_handle_reset on v2:172.21.15.5:6832/2315721152 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/4068956939 shutdown_connections 2024-05-08T23:24:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/4068956939 wait complete. 2024-05-08T23:24:56.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.384+0000 7f1fd4dc2b80 20 client.5798 trim_cache size 0 max 16384 2024-05-08T23:24:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.404+0000 7f1fd4dc2b80 1 Processor -- start 2024-05-08T23:24:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.404+0000 7f1fd4dc2b80 1 -- start start 2024-05-08T23:24:56.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 0x55d3e6a47fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a6a750 0x55d3e6a41660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6a43e20 con 0x55d3e6a71d80 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6a68db0 con 0x55d3e6a45320 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6a8a600 con 0x55d3e6a6a750 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58096/0 (socket says 172.21.15.5:58096) 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/1195571429 learned_addr learned my addr 172.21.15.5:0/1195571429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a6a750 0x55d3e6a41660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc259c700 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 0x55d3e6a47fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/1195571429 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a6a750 msgr2=0x55d3e6a41660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a6a750 0x55d3e6a41660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/1195571429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 msgr2=0x55d3e6a47fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 0x55d3e6a47fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.405+0000 7f1fc2d9d700 1 -- 172.21.15.5:0/1195571429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e695d510 con 0x55d3e6a71d80 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a753d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0600d0 tx=0x7f1fbc056180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847e7178009f8823 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fb0ff9700 1 -- 172.21.15.5:0/1195571429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc089210 con 0x55d3e6a71d80 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fb0ff9700 1 -- 172.21.15.5:0/1195571429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc059e40 con 0x55d3e6a71d80 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fb0ff9700 1 -- 172.21.15.5:0/1195571429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc05dc70 con 0x55d3e6a71d80 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/1195571429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 msgr2=0x55d3e6a753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a753d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0600d0 tx=0x7f1fbc056180 comp rx=0 tx=0).stop 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/1195571429 shutdown_connections 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a71d80 0x55d3e6a753d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a45320 0x55d3e6a47fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/1195571429 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a6a750 0x55d3e6a41660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/1195571429 >> 172.21.15.5:0/1195571429 conn(0x55d3e69a51c0 msgr2=0x55d3e6a3e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/1195571429 shutdown_connections 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.406+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/1195571429 wait complete. 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.407+0000 7f1fd4dc2b80 1 -- start start 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.407+0000 7f1fd4dc2b80 1 Processor -- start 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6aef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a45700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fd4dc2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a71d80 0x55d3e6a45c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6aaf250 con 0x55d3e6a6a750 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6a92340 con 0x55d3e6a71d80 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fd4dc2b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6a46c70 con 0x55d3e6a45320 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fd4dc2b80 10 client.0 _ll_register_callbacks cb 0x55d3e693cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fc1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a45700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fc1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a45700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58112/0 (socket says 172.21.15.5:58112) 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fc1d9b700 1 -- 172.21.15.5:0/2755413422 learned_addr learned my addr 172.21.15.5:0/2755413422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fc259c700 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6aef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.408+0000 7f1fc2d9d700 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a71d80 0x55d3e6a45c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fc1d9b700 1 -- 172.21.15.5:0/2755413422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 msgr2=0x55d3e6aef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6aef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fc1d9b700 1 -- 172.21.15.5:0/2755413422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a71d80 msgr2=0x55d3e6a45c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a71d80 0x55d3e6a45c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fc1d9b700 1 -- 172.21.15.5:0/2755413422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc415ec80 con 0x55d3e6a6a750 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fc1d9b700 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a45700 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x55d3e6a73b40 tx=0x7f1fb8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4065eef6c6c03dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fa17fa700 1 -- 172.21.15.5:0/2755413422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb801a030 con 0x55d3e6a6a750 2024-05-08T23:24:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fa17fa700 1 -- 172.21.15.5:0/2755413422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fb8003a30 con 0x55d3e6a6a750 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/2755413422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fb40158d0 con 0x55d3e6a6a750 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fa17fa700 1 -- 172.21.15.5:0/2755413422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb8003bf0 con 0x55d3e6a6a750 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 10 client.6717 fetch_fsmap learned FSMap version 54 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/2755413422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55d3e681cc80 con 0x55d3e6a6a750 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fa17fa700 1 -- 172.21.15.5:0/2755413422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb40158d0 con 0x55d3e6a6a750 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fa17fa700 1 -- 172.21.15.5:0/2755413422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f1fb80240f0 con 0x55d3e6a6a750 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 10 client.6717 fetch_fsmap finished waiting for FSMap version 54 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 10 client.6717 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 20 client.6717 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x55d3e6af8d50 0x55d3e6af9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 4 client.6717 mds_command: new command op to 6516 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.409+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/2755413422 --> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d3e698fe70 con 0x55d3e6af8d50 2024-05-08T23:24:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fc259c700 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x55d3e6af8d50 0x55d3e6af9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fa17fa700 10 client.6717 ms_handle_connect on v2:172.21.15.5:6832/2315721152 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fc259c700 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x55d3e6af8d50 0x55d3e6af9130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a663f9f941db165 server_cookie=bca730bd96481fa0 in_seq=0 out_seq=0 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fa17fa700 1 -- 172.21.15.5:0/2755413422 <== mds.1 v2:172.21.15.5:6832/2315721152 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d3e698fe70 con 0x55d3e6af8d50 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fa17fa700 10 client.6717 handle_command_reply: tid=0 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 client.6717 shutdown 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/2755413422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 msgr2=0x55d3e6a45700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a45700 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x55d3e6a73b40 tx=0x7f1fb8004270 comp rx=0 tx=0).stop 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/2755413422 shutdown_connections 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d3e6a71d80 0x55d3e6a45c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x55d3e6af8d50 0x55d3e6af9130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55d3e6a45320 0x55d3e6aef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 --2- 172.21.15.5:0/2755413422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e6a6a750 0x55d3e6a45700 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/2755413422 >> 172.21.15.5:0/2755413422 conn(0x55d3e69a51c0 msgr2=0x55d3e6a42f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fa17fa700 0 client.6717 ms_handle_reset on v2:172.21.15.5:6832/2315721152 2024-05-08T23:24:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.410+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/2755413422 shutdown_connections 2024-05-08T23:24:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fd4dc2b80 1 -- 172.21.15.5:0/2755413422 wait complete. 2024-05-08T23:24:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fd4dc2b80 20 client.6717 trim_cache size 0 max 16384 2024-05-08T23:24:56.421 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:56.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:24:56.421 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-08T23:24:56.421 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6516, 2024-05-08T23:24:56.421 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:24:56.421 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:24:56.422 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:24:56.422 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 3.0956002929999999, 2024-05-08T23:24:56.422 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 54, 2024-05-08T23:24:56.422 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 60, 2024-05-08T23:24:56.422 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 60, 2024-05-08T23:24:56.422 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 16.382224963999999 2024-05-08T23:24:56.422 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 -- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1fa406ac90 msgr2=0x7f1fa406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1fa406ac90 0x7f1fa406d150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4000c00 tx=0x7f1fb4007980 comp rx=0 tx=0).stop 2024-05-08T23:24:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 -- 172.21.15.5:0/3263429457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc415e2c0 msgr2=0x7f1fc415e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc415e2c0 0x7f1fc415e740 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc017040 tx=0x7f1fbc009750 comp rx=0 tx=0).stop 2024-05-08T23:24:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 -- 172.21.15.5:0/3263429457 shutdown_connections 2024-05-08T23:24:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1fa406ac90 0x7f1fa406d150 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc415e2c0 0x7f1fc415e740 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1fac004d90 0x7f1fc41f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 --2- 172.21.15.5:0/3263429457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc41f6db0 0x7f1fc40afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 -- 172.21.15.5:0/3263429457 >> 172.21.15.5:0/3263429457 conn(0x7f1fc40b2f20 msgr2=0x7f1fc406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 -- 172.21.15.5:0/3263429457 shutdown_connections 2024-05-08T23:24:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.411+0000 7f1fc8c08700 1 -- 172.21.15.5:0/3263429457 wait complete. 2024-05-08T23:24:56.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T23:24:56.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.567+0000 7f3679c0c700 1 Processor -- start 2024-05-08T23:24:56.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.567+0000 7f3679c0c700 1 -- start start 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.568+0000 7f3679c0c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740c6900 0x7f3674166260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.568+0000 7f3679c0c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36740c5130 con 0x7f367415b6c0 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.568+0000 7f3679c0c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36740cf790 con 0x7f36740cbee0 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.568+0000 7f3679c0c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36740d0380 con 0x7f36740c6900 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f36737fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f367415b6c0 0x7f36740cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36164/0 (socket says 172.21.15.5:36164) 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f36737fe700 1 -- 172.21.15.5:0/3593464953 learned_addr learned my addr 172.21.15.5:0/3593464953 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f3673fff700 1 --2- 172.21.15.5:0/3593464953 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740c6900 0x7f3674166260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f36727fc700 1 -- 172.21.15.5:0/3593464953 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214506865 0 0) 0x7f36740c5130 con 0x7f367415b6c0 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f36727fc700 1 -- 172.21.15.5:0/3593464953 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36600035b0 con 0x7f367415b6c0 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f36727fc700 1 -- 172.21.15.5:0/3593464953 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936472945 0 0) 0x7f36740cf790 con 0x7f36740cbee0 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f36727fc700 1 -- 172.21.15.5:0/3593464953 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36740c5130 con 0x7f36740cbee0 2024-05-08T23:24:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f3673fff700 1 -- 172.21.15.5:0/3593464953 >> v1:172.21.15.49:6789/0 conn(0x7f36740cbee0 legacy=0x7f36740cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f36727fc700 1 -- 172.21.15.5:0/3593464953 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 521721183 0 0) 0x7f36600035b0 con 0x7f367415b6c0 2024-05-08T23:24:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f3673fff700 1 -- 172.21.15.5:0/3593464953 >> v1:172.21.15.5:6789/0 conn(0x7f367415b6c0 legacy=0x7f36740cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f3673fff700 1 -- 172.21.15.5:0/3593464953 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36740cfdc0 con 0x7f36740c6900 2024-05-08T23:24:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.569+0000 7f3673fff700 1 --2- 172.21.15.5:0/3593464953 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740c6900 0x7f3674166260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f366c009dc0 tx=0x7f366c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc1c23b006718b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f36727fc700 1 -- 172.21.15.5:0/3593464953 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c00e070 con 0x7f36740c6900 2024-05-08T23:24:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f36727fc700 1 -- 172.21.15.5:0/3593464953 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c003a20 con 0x7f36740c6900 2024-05-08T23:24:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f36727fc700 1 -- 172.21.15.5:0/3593464953 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664001670 con 0x7f36740c6900 2024-05-08T23:24:56.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f3679c0c700 1 -- 172.21.15.5:0/3593464953 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740c6900 msgr2=0x7f3674166260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f3679c0c700 1 --2- 172.21.15.5:0/3593464953 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740c6900 0x7f3674166260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f366c009dc0 tx=0x7f366c00b330 comp rx=0 tx=0).stop 2024-05-08T23:24:56.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f3679c0c700 1 -- 172.21.15.5:0/3593464953 shutdown_connections 2024-05-08T23:24:56.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f3679c0c700 1 --2- 172.21.15.5:0/3593464953 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740c6900 0x7f3674166260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f3679c0c700 1 -- 172.21.15.5:0/3593464953 >> 172.21.15.5:0/3593464953 conn(0x7f36740b2f30 msgr2=0x7f36740b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f3679c0c700 1 -- 172.21.15.5:0/3593464953 shutdown_connections 2024-05-08T23:24:56.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.570+0000 7f3679c0c700 1 -- 172.21.15.5:0/3593464953 wait complete. 2024-05-08T23:24:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.572+0000 7f3679c0c700 1 Processor -- start 2024-05-08T23:24:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.572+0000 7f3679c0c700 1 -- start start 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f3679c0c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36740c6900 0x7f36740cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f3679c0c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740cbee0 0x7f36740b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f3679c0c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f367415b6c0 0x7f36740b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f3679c0c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740b4840 con 0x7f36740c6900 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f3679c0c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740c9f10 con 0x7f367415b6c0 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f3679c0c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36740cb220 con 0x7f36740cbee0 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36740c6900 0x7f36740cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36740c6900 0x7f36740cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58122/0 (socket says 172.21.15.5:58122) 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 -- 172.21.15.5:0/4051917036 learned_addr learned my addr 172.21.15.5:0/4051917036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f3672ffd700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740cbee0 0x7f36740b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f3673fff700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f367415b6c0 0x7f36740b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 -- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740cbee0 msgr2=0x7f36740b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740cbee0 0x7f36740b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 -- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f367415b6c0 msgr2=0x7f36740b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f367415b6c0 0x7f36740b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 -- 172.21.15.5:0/4051917036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c009a70 con 0x7f36740c6900 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f36737fe700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36740c6900 0x7f36740cab10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f366400c750 tx=0x7f366400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f366bfff700 1 -- 172.21.15.5:0/4051917036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664004d40 con 0x7f36740c6900 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.573+0000 7f366bfff700 1 -- 172.21.15.5:0/4051917036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36640070d0 con 0x7f36740c6900 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.574+0000 7f3679c0c700 1 -- 172.21.15.5:0/4051917036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36740c9150 con 0x7f36740c6900 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.574+0000 7f366bfff700 1 -- 172.21.15.5:0/4051917036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664004ea0 con 0x7f36740c6900 2024-05-08T23:24:56.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.574+0000 7f3679c0c700 1 -- 172.21.15.5:0/4051917036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36740c9710 con 0x7f36740c6900 2024-05-08T23:24:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.575+0000 7f366bfff700 1 -- 172.21.15.5:0/4051917036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3664007460 con 0x7f36740c6900 2024-05-08T23:24:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.576+0000 7f366bfff700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f365406ac90 0x7f365406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.576+0000 7f366bfff700 1 -- 172.21.15.5:0/4051917036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3664089100 con 0x7f36740c6900 2024-05-08T23:24:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.576+0000 7f3672ffd700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f365406ac90 0x7f365406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.576+0000 7f3672ffd700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f365406ac90 0x7f365406d150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f367406d990 tx=0x7f365c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.578+0000 7f3685dc6b80 1 Processor -- start 2024-05-08T23:24:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.578+0000 7f3685dc6b80 1 -- start start 2024-05-08T23:24:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 0x55b7c073b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 0x55b7c0740050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c076a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c0696100 con 0x55b7c0769d80 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c0693ac0 con 0x55b7c0762750 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7c068f3b0 con 0x55b7c073d320 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c076a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f36737fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 0x55b7c073b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c076a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58134/0 (socket says 172.21.15.5:58134) 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 -- 172.21.15.5:0/2762628633 learned_addr learned my addr 172.21.15.5:0/2762628633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3672ffd700 1 --2- 172.21.15.5:0/2762628633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 0x55b7c0740050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 -- 172.21.15.5:0/2762628633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 msgr2=0x55b7c0740050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 --2- 172.21.15.5:0/2762628633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 0x55b7c0740050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 -- 172.21.15.5:0/2762628633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 msgr2=0x55b7c073b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 --2- 172.21.15.5:0/2762628633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 0x55b7c073b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 -- 172.21.15.5:0/2762628633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7c0510360 con 0x55b7c0769d80 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f3673fff700 1 --2- 172.21.15.5:0/2762628633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c076a200 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f366c015040 tx=0x7f366c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed465cd197d9ce94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.579+0000 7f36697fa700 1 -- 172.21.15.5:0/2762628633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c00ba20 con 0x55b7c0769d80 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f36697fa700 1 -- 172.21.15.5:0/2762628633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c00bb80 con 0x55b7c0769d80 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f36697fa700 1 -- 172.21.15.5:0/2762628633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c004030 con 0x55b7c0769d80 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2762628633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 msgr2=0x55b7c076a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/2762628633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c076a200 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f366c015040 tx=0x7f366c010000 comp rx=0 tx=0).stop 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2762628633 shutdown_connections 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/2762628633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c076a200 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/2762628633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 0x55b7c073b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/2762628633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 0x55b7c0740050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2762628633 >> 172.21.15.5:0/2762628633 conn(0x55b7c069d1c0 msgr2=0x55b7c0784d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2762628633 shutdown_connections 2024-05-08T23:24:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.580+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2762628633 wait complete. 2024-05-08T23:24:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.581+0000 7f3685dc6b80 1 -- start start 2024-05-08T23:24:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.581+0000 7f3685dc6b80 1 Processor -- start 2024-05-08T23:24:56.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 0x55b7c07970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0762750 0x55b7c0786c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0769d80 0x55b7c07941c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c062b510 con 0x55b7c0762750 2024-05-08T23:24:56.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c062b1b0 con 0x55b7c0769d80 2024-05-08T23:24:56.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7c0512650 con 0x55b7c073d320 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3685dc6b80 10 client.0 _ll_register_callbacks cb 0x55b7c0634ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0762750 0x55b7c0786c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0762750 0x55b7c0786c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58150/0 (socket says 172.21.15.5:58150) 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 -- 172.21.15.5:0/4161746608 learned_addr learned my addr 172.21.15.5:0/4161746608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f36737fe700 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 0x55b7c07970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3673fff700 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0769d80 0x55b7c07941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 -- 172.21.15.5:0/4161746608 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 msgr2=0x55b7c07970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 0x55b7c07970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 -- 172.21.15.5:0/4161746608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0769d80 msgr2=0x55b7c07941c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0769d80 0x55b7c07941c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 -- 172.21.15.5:0/4161746608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36740cfdc0 con 0x55b7c0762750 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f3672ffd700 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0762750 0x55b7c0786c10 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f365c001e60 tx=0x7f365c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93b1356a5fb9548 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.583+0000 7f36527fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:56.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f36527fc700 1 -- 172.21.15.5:0/4161746608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c014af0 con 0x55b7c0762750 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f36527fc700 1 -- 172.21.15.5:0/4161746608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f365c01c040 con 0x55b7c0762750 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 1 -- 172.21.15.5:0/4161746608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7c05e8220 con 0x55b7c0762750 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f36527fc700 1 -- 172.21.15.5:0/4161746608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f365c014c50 con 0x55b7c0762750 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 10 client.6729 fetch_fsmap learned FSMap version 54 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 1 -- 172.21.15.5:0/4161746608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55b7c04bcb10 con 0x55b7c0762750 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f36527fc700 1 -- 172.21.15.5:0/4161746608 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b7c05e8220 con 0x55b7c0762750 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f36527fc700 1 -- 172.21.15.5:0/4161746608 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f365c021400 con 0x55b7c0762750 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 10 client.6729 fetch_fsmap finished waiting for FSMap version 54 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 10 client.6729 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 20 client.6729 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55b7c0787150 0x55b7c0783b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 4 client.6729 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f3685dc6b80 1 -- 172.21.15.5:0/4161746608 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7c070e790 con 0x55b7c0787150 2024-05-08T23:24:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.584+0000 7f36737fe700 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55b7c0787150 0x55b7c0783b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f36737fe700 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55b7c0787150 0x55b7c0783b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835cfad58d3e1481 server_cookie=225a966a49285f2d in_seq=0 out_seq=0 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f36527fc700 10 client.6729 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f36527fc700 1 -- 172.21.15.5:0/4161746608 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7c070e790 con 0x55b7c0787150 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f36527fc700 10 client.6729 handle_command_reply: tid=0 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 client.6729 shutdown 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 -- 172.21.15.5:0/4161746608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0762750 msgr2=0x55b7c0786c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0762750 0x55b7c0786c10 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f365c001e60 tx=0x7f365c001190 comp rx=0 tx=0).stop 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 -- 172.21.15.5:0/4161746608 shutdown_connections 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55b7c0787150 0x55b7c0783b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0769d80 0x55b7c07941c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c073d320 0x55b7c07970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f36527fc700 0 client.6729 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/4161746608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0762750 0x55b7c0786c10 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.585+0000 7f3685dc6b80 1 -- 172.21.15.5:0/4161746608 >> 172.21.15.5:0/4161746608 conn(0x55b7c069d1c0 msgr2=0x55b7c076a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.586+0000 7f3685dc6b80 1 -- 172.21.15.5:0/4161746608 shutdown_connections 2024-05-08T23:24:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.586+0000 7f3685dc6b80 1 -- 172.21.15.5:0/4161746608 wait complete. 2024-05-08T23:24:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.586+0000 7f3685dc6b80 20 client.6729 trim_cache size 0 max 16384 2024-05-08T23:24:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.605+0000 7f3685dc6b80 1 Processor -- start 2024-05-08T23:24:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.606+0000 7f3685dc6b80 1 -- start start 2024-05-08T23:24:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.606+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c073d320 0x55b7c073a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.606+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0762750 0x55b7c0783b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.606+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c07840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.606+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c0796af0 con 0x55b7c0769d80 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.606+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c0740430 con 0x55b7c073d320 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.606+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7c07856e0 con 0x55b7c0762750 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c07840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c07840b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58156/0 (socket says 172.21.15.5:58156) 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 -- 172.21.15.5:0/2883403525 learned_addr learned my addr 172.21.15.5:0/2883403525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f36737fe700 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c073d320 0x55b7c073a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3672ffd700 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0762750 0x55b7c0783b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 -- 172.21.15.5:0/2883403525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0762750 msgr2=0x55b7c0783b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0762750 0x55b7c0783b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 -- 172.21.15.5:0/2883403525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c073d320 msgr2=0x55b7c073a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c073d320 0x55b7c073a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 -- 172.21.15.5:0/2883403525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7c0655510 con 0x55b7c0769d80 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3673fff700 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c07840b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f366c004000 tx=0x7f366c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255f9a2739852de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3669ffb700 1 -- 172.21.15.5:0/2883403525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c009840 con 0x55b7c0769d80 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3669ffb700 1 -- 172.21.15.5:0/2883403525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c003d80 con 0x55b7c0769d80 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.607+0000 7f3669ffb700 1 -- 172.21.15.5:0/2883403525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c004270 con 0x55b7c0769d80 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2883403525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 msgr2=0x55b7c07840b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c07840b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f366c004000 tx=0x7f366c009750 comp rx=0 tx=0).stop 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2883403525 shutdown_connections 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c0769d80 0x55b7c07840b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c073d320 0x55b7c073a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/2883403525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0762750 0x55b7c0783b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2883403525 >> 172.21.15.5:0/2883403525 conn(0x55b7c069d1c0 msgr2=0x55b7c0736190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2883403525 shutdown_connections 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.608+0000 7f3685dc6b80 1 -- 172.21.15.5:0/2883403525 wait complete. 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.609+0000 7f3685dc6b80 1 -- start start 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.609+0000 7f3685dc6b80 1 Processor -- start 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c073d320 0x55b7c07ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 0x55b7c0739810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3685dc6b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0769d80 0x55b7c0739d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c0784520 con 0x55b7c073d320 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c07ee540 con 0x55b7c0762750 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3685dc6b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7c07ee6c0 con 0x55b7c0769d80 2024-05-08T23:24:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3685dc6b80 10 client.0 _ll_register_callbacks cb 0x55b7c0634ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c073d320 0x55b7c07ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c073d320 0x55b7c07ee000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58160/0 (socket says 172.21.15.5:58160) 2024-05-08T23:24:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 -- 172.21.15.5:0/984955583 learned_addr learned my addr 172.21.15.5:0/984955583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3673fff700 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0769d80 0x55b7c0739d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f3672ffd700 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 0x55b7c0739810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 -- 172.21.15.5:0/984955583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0769d80 msgr2=0x55b7c0739d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0769d80 0x55b7c0739d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 -- 172.21.15.5:0/984955583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 msgr2=0x55b7c0739810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 0x55b7c0739810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 -- 172.21.15.5:0/984955583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7c0510360 con 0x55b7c073d320 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36737fe700 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c073d320 0x55b7c07ee000 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3664087620 tx=0x7f366405a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2c91734750e8f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36527fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.610+0000 7f36527fc700 1 -- 172.21.15.5:0/984955583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664087320 con 0x55b7c073d320 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f36527fc700 1 -- 172.21.15.5:0/984955583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3664060380 con 0x55b7c073d320 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 1 -- 172.21.15.5:0/984955583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f366c00bb80 con 0x55b7c073d320 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f36527fc700 1 -- 172.21.15.5:0/984955583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36640581f0 con 0x55b7c073d320 2024-05-08T23:24:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 10 client.6735 fetch_fsmap learned FSMap version 54 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 1 -- 172.21.15.5:0/984955583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55b7c0514c80 con 0x55b7c073d320 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f36527fc700 1 -- 172.21.15.5:0/984955583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c00bb80 con 0x55b7c073d320 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f36527fc700 1 -- 172.21.15.5:0/984955583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f3664057480 con 0x55b7c073d320 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 10 client.6735 fetch_fsmap finished waiting for FSMap version 54 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 10 client.6735 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 20 client.6735 populate_metadata read hostname 'smithi005' 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55b7c0765690 0x55b7c0765a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 4 client.6735 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3685dc6b80 1 -- 172.21.15.5:0/984955583 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b7c0687e70 con 0x55b7c0765690 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3672ffd700 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55b7c0765690 0x55b7c0765a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f36527fc700 10 client.6735 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:24:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.611+0000 7f3672ffd700 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55b7c0765690 0x55b7c0765a70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cf2bf59d619b06 server_cookie=7c4ba166a1762484 in_seq=0 out_seq=0 2024-05-08T23:24:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f36527fc700 1 -- 172.21.15.5:0/984955583 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55b7c0687e70 con 0x55b7c0765690 2024-05-08T23:24:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f36527fc700 10 client.6735 handle_command_reply: tid=0 2024-05-08T23:24:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 client.6735 shutdown 2024-05-08T23:24:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 -- 172.21.15.5:0/984955583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c073d320 msgr2=0x55b7c07ee000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c073d320 0x55b7c07ee000 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3664087620 tx=0x7f366405a040 comp rx=0 tx=0).stop 2024-05-08T23:24:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 -- 172.21.15.5:0/984955583 shutdown_connections 2024-05-08T23:24:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b7c0769d80 0x55b7c0739d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c073d320 0x55b7c07ee000 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55b7c0765690 0x55b7c0765a70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 --2- 172.21.15.5:0/984955583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b7c0762750 0x55b7c0739810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 -- 172.21.15.5:0/984955583 >> 172.21.15.5:0/984955583 conn(0x55b7c069d1c0 msgr2=0x55b7c0783b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f36527fc700 0 client.6735 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:24:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 -- 172.21.15.5:0/984955583 shutdown_connections 2024-05-08T23:24:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 1 -- 172.21.15.5:0/984955583 wait complete. 2024-05-08T23:24:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.612+0000 7f3685dc6b80 20 client.6735 trim_cache size 0 max 16384 2024-05-08T23:24:56.629 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:24:56.629 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:24:56.629 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:24:56.629 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6525, 2024-05-08T23:24:56.629 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:24:56.629 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:24:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:24:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 11.327778645, 2024-05-08T23:24:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 54, 2024-05-08T23:24:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 60, 2024-05-08T23:24:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 60, 2024-05-08T23:24:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 16.575064028 2024-05-08T23:24:56.630 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 -- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f365406ac90 msgr2=0x7f365406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f365406ac90 0x7f365406d150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f367406d990 tx=0x7f365c009040 comp rx=0 tx=0).stop 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 -- 172.21.15.5:0/4051917036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36740c6900 msgr2=0x7f36740cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36740c6900 0x7f36740cab10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f366400c750 tx=0x7f366400ca60 comp rx=0 tx=0).stop 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 -- 172.21.15.5:0/4051917036 shutdown_connections 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f365406ac90 0x7f365406d150 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f367415b6c0 0x7f36740b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36740c6900 0x7f36740cab10 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 --2- 172.21.15.5:0/4051917036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36740cbee0 0x7f36740b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 -- 172.21.15.5:0/4051917036 >> 172.21.15.5:0/4051917036 conn(0x7f36740b2f30 msgr2=0x7f36740b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 -- 172.21.15.5:0/4051917036 shutdown_connections 2024-05-08T23:24:56.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.613+0000 7f3679c0c700 1 -- 172.21.15.5:0/4051917036 wait complete. 2024-05-08T23:24:56.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.774+0000 7fcc905eb700 1 Processor -- start 2024-05-08T23:24:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.774+0000 7fcc905eb700 1 -- start start 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.775+0000 7fcc905eb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcc8815a630 0x7fcc880d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.775+0000 7fcc905eb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcc880cdef0 0x7fcc880cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.775+0000 7fcc905eb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc880ce740 con 0x7fcc880cc040 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.775+0000 7fcc905eb700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc880cb670 con 0x7fcc880cdef0 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.775+0000 7fcc905eb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc880cadc0 con 0x7fcc8815a630 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.775+0000 7fcc8db86700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcc880cdef0 0x7fcc880cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.775+0000 7fcc8db86700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcc880cdef0 0x7fcc880cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58162/0 (socket says 172.21.15.5:58162) 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.775+0000 7fcc8db86700 1 -- 172.21.15.5:0/2173149515 learned_addr learned my addr 172.21.15.5:0/2173149515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8e387700 1 --2- 172.21.15.5:0/2173149515 >> v2:172.21.15.49:3300/0 conn(0x7fcc8815a630 0x7fcc880d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8db86700 1 -- 172.21.15.5:0/2173149515 >> v2:172.21.15.49:3300/0 conn(0x7fcc8815a630 msgr2=0x7fcc880d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8db86700 1 --2- 172.21.15.5:0/2173149515 >> v2:172.21.15.49:3300/0 conn(0x7fcc8815a630 0x7fcc880d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8db86700 1 -- 172.21.15.5:0/2173149515 >> v1:172.21.15.49:6789/0 conn(0x7fcc880cc040 legacy=0x7fcc8815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8db86700 1 -- 172.21.15.5:0/2173149515 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc880cb060 con 0x7fcc880cdef0 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8db86700 1 --2- 172.21.15.5:0/2173149515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc880cbb00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc88070670 tx=0x7fcc7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277c241ff05f7d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8d385700 1 -- 172.21.15.5:0/2173149515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c004230 con 0x7fcc880cdef0 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8d385700 1 -- 172.21.15.5:0/2173149515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c004390 con 0x7fcc880cdef0 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.776+0000 7fcc8d385700 1 -- 172.21.15.5:0/2173149515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c0036a0 con 0x7fcc880cdef0 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.777+0000 7fcc905eb700 1 -- 172.21.15.5:0/2173149515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 msgr2=0x7fcc880cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.777+0000 7fcc905eb700 1 --2- 172.21.15.5:0/2173149515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc880cbb00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc88070670 tx=0x7fcc7c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.777+0000 7fcc905eb700 1 -- 172.21.15.5:0/2173149515 shutdown_connections 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.777+0000 7fcc905eb700 1 --2- 172.21.15.5:0/2173149515 >> v2:172.21.15.49:3300/0 conn(0x7fcc8815a630 0x7fcc880d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.777+0000 7fcc905eb700 1 --2- 172.21.15.5:0/2173149515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc880cbb00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.777+0000 7fcc905eb700 1 -- 172.21.15.5:0/2173149515 >> 172.21.15.5:0/2173149515 conn(0x7fcc88099a50 msgr2=0x7fcc880d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.777+0000 7fcc905eb700 1 -- 172.21.15.5:0/2173149515 shutdown_connections 2024-05-08T23:24:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.777+0000 7fcc905eb700 1 -- 172.21.15.5:0/2173149515 wait complete. 2024-05-08T23:24:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.778+0000 7fcc905eb700 1 Processor -- start 2024-05-08T23:24:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.778+0000 7fcc905eb700 1 -- start start 2024-05-08T23:24:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc905eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc880cc040 0x7fcc880b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc905eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc8815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc905eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc8815a630 0x7fcc880c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc905eb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880b4e00 con 0x7fcc880cdef0 2024-05-08T23:24:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc905eb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880c4d80 con 0x7fcc8815a630 2024-05-08T23:24:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc905eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880c6ed0 con 0x7fcc880cc040 2024-05-08T23:24:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8db86700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc8815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8db86700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc8815ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58166/0 (socket says 172.21.15.5:58166) 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8db86700 1 -- 172.21.15.5:0/3714514895 learned_addr learned my addr 172.21.15.5:0/3714514895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8eb88700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc8815a630 0x7fcc880c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8e387700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc880cc040 0x7fcc880b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8db86700 1 -- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc880cc040 msgr2=0x7fcc880b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8db86700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc880cc040 0x7fcc880b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8db86700 1 -- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc8815a630 msgr2=0x7fcc880c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8db86700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc8815a630 0x7fcc880c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.779+0000 7fcc8db86700 1 -- 172.21.15.5:0/3714514895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c009550 con 0x7fcc880cdef0 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.780+0000 7fcc8db86700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc8815ab90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009130 tx=0x7fcc7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.780+0000 7fcc7affd700 1 -- 172.21.15.5:0/3714514895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c003b90 con 0x7fcc880cdef0 2024-05-08T23:24:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.780+0000 7fcc7affd700 1 -- 172.21.15.5:0/3714514895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c01f070 con 0x7fcc880cdef0 2024-05-08T23:24:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.780+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc880b4180 con 0x7fcc880cdef0 2024-05-08T23:24:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.780+0000 7fcc7affd700 1 -- 172.21.15.5:0/3714514895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c011630 con 0x7fcc880cdef0 2024-05-08T23:24:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.780+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc880b4740 con 0x7fcc880cdef0 2024-05-08T23:24:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.781+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc880b4f40 con 0x7fcc880cdef0 2024-05-08T23:24:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.781+0000 7fcc7affd700 1 -- 172.21.15.5:0/3714514895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcc7c021030 con 0x7fcc880cdef0 2024-05-08T23:24:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.782+0000 7fcc7affd700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcc6c06ac90 0x7fcc6c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.782+0000 7fcc7affd700 1 -- 172.21.15.5:0/3714514895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc7c08a6f0 con 0x7fcc880cdef0 2024-05-08T23:24:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.782+0000 7fcc8e387700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcc6c06ac90 0x7fcc6c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.782+0000 7fcc8e387700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcc6c06ac90 0x7fcc6c06d150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcc8809e3d0 tx=0x7fcc84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:56.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:56.787+0000 7fcc7affd700 1 -- 172.21.15.5:0/3714514895 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcc880b4f40 con 0x7fcc880cdef0 2024-05-08T23:24:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.055+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc880ce740 con 0x7fcc880cdef0 2024-05-08T23:24:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.055+0000 7fcc7affd700 1 -- 172.21.15.5:0/3714514895 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4756 (secure 0 0 0) 0x7fcc880ce740 con 0x7fcc880cdef0 2024-05-08T23:24:57.055 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:24:57.056 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:24:55.322507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6525,"mds_1":6516,"mds_2":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":2,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6516":{"gid":6516,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:24:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcc6c06ac90 msgr2=0x7fcc6c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcc6c06ac90 0x7fcc6c06d150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcc8809e3d0 tx=0x7fcc84008040 comp rx=0 tx=0).stop 2024-05-08T23:24:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 msgr2=0x7fcc8815ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc8815ab90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009130 tx=0x7fcc7c004750 comp rx=0 tx=0).stop 2024-05-08T23:24:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 shutdown_connections 2024-05-08T23:24:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcc6c06ac90 0x7fcc6c06d150 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc8815a630 0x7fcc880c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc880cc040 0x7fcc880b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 --2- 172.21.15.5:0/3714514895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc880cdef0 0x7fcc8815ab90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 >> 172.21.15.5:0/3714514895 conn(0x7fcc88099a50 msgr2=0x7fcc880d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 shutdown_connections 2024-05-08T23:24:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:24:57.058+0000 7fcc905eb700 1 -- 172.21.15.5:0/3714514895 wait complete. 2024-05-08T23:24:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-05-08T23:24:57.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:57.072 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:24:57.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:57.103 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:24:57.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:57.169 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-05-08T23:24:57.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:57.234 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-05-08T23:24:57.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:57.297 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-05-08T23:24:57.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:24:57.565 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:24:57.565 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:24:57.566 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:24:59.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:24:59.732 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:24:59.732 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:24:59.732 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:25:01.734 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:25:01.900 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:25:01.900 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:25:01.901 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:25:03.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:25:04.067 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:25:04.068 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:25:04.068 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:25:06.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:25:06.248 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-05-08T23:25:06.249 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:25:06.249 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:25:06.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:25:06.406 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:25:06.406 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:25:06.406 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:25:07.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:25:07.880 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:25:08.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:25:08.570 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:25:08.571 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:25:08.571 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:25:10.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:25:10.736 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-08T23:25:10.737 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:25:10.737 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:25:10.737 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:25:10.895 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:25:10.895 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:25:10.896 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:25:10.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-05-08T23:25:11.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.041+0000 7ff19f3cc700 1 Processor -- start 2024-05-08T23:25:11.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.041+0000 7ff19f3cc700 1 -- start start 2024-05-08T23:25:11.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.042+0000 7ff19f3cc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff1980b0d80 0x7ff1980b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.042+0000 7ff19f3cc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1980b4d50 0x7ff19806a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.042+0000 7ff19f3cc700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff198145040 con 0x7ff19806aea0 2024-05-08T23:25:11.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.042+0000 7ff19f3cc700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19806b330 con 0x7ff1980b0d80 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.042+0000 7ff19f3cc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980b38c0 con 0x7ff1980b4d50 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19d168700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff1980b0d80 0x7ff1980b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19d969700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff19806aea0 0x7ff1980d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38542/0 (socket says 172.21.15.5:38542) 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19d969700 1 -- 172.21.15.5:0/3316072239 learned_addr learned my addr 172.21.15.5:0/3316072239 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19c967700 1 --2- 172.21.15.5:0/3316072239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1980b4d50 0x7ff19806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19d168700 1 -- 172.21.15.5:0/3316072239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1980b4d50 msgr2=0x7ff19806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19d168700 1 --2- 172.21.15.5:0/3316072239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1980b4d50 0x7ff19806a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19d168700 1 -- 172.21.15.5:0/3316072239 >> v1:172.21.15.5:6789/0 conn(0x7ff19806aea0 legacy=0x7ff1980d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19d168700 1 -- 172.21.15.5:0/3316072239 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980b16e0 con 0x7ff1980b0d80 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff19d168700 1 --2- 172.21.15.5:0/3316072239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff1980b3380 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff194009840 tx=0x7ff194009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea55549368ab2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff18bfff700 1 -- 172.21.15.5:0/3316072239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194011830 con 0x7ff1980b0d80 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff18bfff700 1 -- 172.21.15.5:0/3316072239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194011990 con 0x7ff1980b0d80 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.043+0000 7ff18bfff700 1 -- 172.21.15.5:0/3316072239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194011d20 con 0x7ff1980b0d80 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.044+0000 7ff19f3cc700 1 -- 172.21.15.5:0/3316072239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 msgr2=0x7ff1980b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.044+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/3316072239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff1980b3380 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff194009840 tx=0x7ff194009b50 comp rx=0 tx=0).stop 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.044+0000 7ff19f3cc700 1 -- 172.21.15.5:0/3316072239 shutdown_connections 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.044+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/3316072239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff1980b3380 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.044+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/3316072239 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1980b4d50 0x7ff19806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.044+0000 7ff19f3cc700 1 -- 172.21.15.5:0/3316072239 >> 172.21.15.5:0/3316072239 conn(0x7ff198099a50 msgr2=0x7ff1980b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.044+0000 7ff19f3cc700 1 -- 172.21.15.5:0/3316072239 shutdown_connections 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.044+0000 7ff19f3cc700 1 -- 172.21.15.5:0/3316072239 wait complete. 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.045+0000 7ff19f3cc700 1 Processor -- start 2024-05-08T23:25:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.045+0000 7ff19f3cc700 1 -- start start 2024-05-08T23:25:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19f3cc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff19806aea0 0x7ff198140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19f3cc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff19809d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19f3cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1980b4d50 0x7ff19809d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19f3cc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980aeaf0 con 0x7ff1980b0d80 2024-05-08T23:25:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19f3cc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1981408b0 con 0x7ff1980b4d50 2024-05-08T23:25:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19f3cc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19809cea0 con 0x7ff19806aea0 2024-05-08T23:25:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19c967700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff19809d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19c967700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff19809d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33232/0 (socket says 172.21.15.5:33232) 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19c967700 1 -- 172.21.15.5:0/606369872 learned_addr learned my addr 172.21.15.5:0/606369872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19d168700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff19806aea0 0x7ff198140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.046+0000 7ff19d969700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1980b4d50 0x7ff19809d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19c967700 1 -- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff19806aea0 msgr2=0x7ff198140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19c967700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff19806aea0 0x7ff198140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19c967700 1 -- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1980b4d50 msgr2=0x7ff19809d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19c967700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1980b4d50 0x7ff19809d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19c967700 1 -- 172.21.15.5:0/606369872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940094a0 con 0x7ff1980b0d80 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19d969700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1980b4d50 0x7ff19809d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19d168700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff19806aea0 0x7ff198140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19c967700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff19809d260 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00d800 tx=0x7ff18c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff189ffb700 1 -- 172.21.15.5:0/606369872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0125f0 con 0x7ff1980b0d80 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff189ffb700 1 -- 172.21.15.5:0/606369872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c012750 con 0x7ff1980b0d80 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19f3cc700 1 -- 172.21.15.5:0/606369872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19809bea0 con 0x7ff1980b0d80 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff189ffb700 1 -- 172.21.15.5:0/606369872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c012b00 con 0x7ff1980b0d80 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.047+0000 7ff19f3cc700 1 -- 172.21.15.5:0/606369872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1980ae710 con 0x7ff1980b0d80 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.049+0000 7ff189ffb700 1 -- 172.21.15.5:0/606369872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff18c012c60 con 0x7ff1980b0d80 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.050+0000 7ff189ffb700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff17c06ac90 0x7ff17c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.050+0000 7ff189ffb700 1 -- 172.21.15.5:0/606369872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff18c0896c0 con 0x7ff1980b0d80 2024-05-08T23:25:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.050+0000 7ff19d168700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff17c06ac90 0x7ff17c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.051+0000 7ff19d168700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff17c06ac90 0x7ff17c06d150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff1940090b0 tx=0x7ff194008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.051+0000 7ff1ab586b80 1 Processor -- start 2024-05-08T23:25:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff1ab586b80 1 -- start start 2024-05-08T23:25:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x564981189110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 0x56498118e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 0x5649811b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649810e4150 con 0x5649811b07a0 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649810e1b10 con 0x5649811b7dd0 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649810dd400 con 0x56498118b370 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff19d168700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x564981189110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff19d168700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x564981189110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33242/0 (socket says 172.21.15.5:33242) 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.052+0000 7ff19d168700 1 -- 172.21.15.5:0/912795023 learned_addr learned my addr 172.21.15.5:0/912795023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff19d969700 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 0x5649811b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff19c967700 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 0x56498118e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff19d168700 1 -- 172.21.15.5:0/912795023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 msgr2=0x56498118e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff19d168700 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 0x56498118e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff19d168700 1 -- 172.21.15.5:0/912795023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 msgr2=0x5649811b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff19d168700 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 0x5649811b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff19d168700 1 -- 172.21.15.5:0/912795023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564980f5e3b0 con 0x5649811b07a0 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff19d168700 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x564981189110 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff194009f50 tx=0x7ff194010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0a7092dea12673 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff182ffd700 1 -- 172.21.15.5:0/912795023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940038a0 con 0x5649811b07a0 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff182ffd700 1 -- 172.21.15.5:0/912795023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19402b070 con 0x5649811b07a0 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.053+0000 7ff182ffd700 1 -- 172.21.15.5:0/912795023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194003c00 con 0x5649811b07a0 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 -- 172.21.15.5:0/912795023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 msgr2=0x564981189110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x564981189110 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff194009f50 tx=0x7ff194010000 comp rx=0 tx=0).stop 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 -- 172.21.15.5:0/912795023 shutdown_connections 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 0x5649811b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x564981189110 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/912795023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 0x56498118e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 -- 172.21.15.5:0/912795023 >> 172.21.15.5:0/912795023 conn(0x5649810eb210 msgr2=0x5649811d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 -- 172.21.15.5:0/912795023 shutdown_connections 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.054+0000 7ff1ab586b80 1 -- 172.21.15.5:0/912795023 wait complete. 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.055+0000 7ff1ab586b80 1 -- start start 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.055+0000 7ff1ab586b80 1 Processor -- start 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56498118b370 0x5649811af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b07a0 0x5649811d4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 0x564981186ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564981079560 con 0x56498118b370 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564981079200 con 0x5649811b7dd0 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564980f606a0 con 0x5649811b07a0 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff1ab586b80 10 client.0 _ll_register_callbacks cb 0x564981082b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56498118b370 0x5649811af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56498118b370 0x5649811af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33248/0 (socket says 172.21.15.5:33248) 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 -- 172.21.15.5:0/1923897408 learned_addr learned my addr 172.21.15.5:0/1923897408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19c967700 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b07a0 0x5649811d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d969700 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 0x564981186ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 -- 172.21.15.5:0/1923897408 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b07a0 msgr2=0x5649811d4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b07a0 0x5649811d4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 -- 172.21.15.5:0/1923897408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 msgr2=0x564981186ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 0x564981186ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 -- 172.21.15.5:0/1923897408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff194021050 con 0x56498118b370 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff19d168700 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56498118b370 0x5649811af430 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff194004600 tx=0x7ff1940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3bf16af3de4e17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff177fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.056+0000 7ff177fff700 1 -- 172.21.15.5:0/1923897408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194003880 con 0x56498118b370 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff177fff700 1 -- 172.21.15.5:0/1923897408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1940039e0 con 0x56498118b370 2024-05-08T23:25:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1923897408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564981036270 con 0x56498118b370 2024-05-08T23:25:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff177fff700 1 -- 172.21.15.5:0/1923897408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff194003ba0 con 0x56498118b370 2024-05-08T23:25:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 10 client.6756 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1923897408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x564980f0ab60 con 0x56498118b370 2024-05-08T23:25:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff177fff700 1 -- 172.21.15.5:0/1923897408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564981036270 con 0x56498118b370 2024-05-08T23:25:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff177fff700 1 -- 172.21.15.5:0/1923897408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7ff194003d00 con 0x56498118b370 2024-05-08T23:25:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 10 client.6756 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 10 client.6756 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 20 client.6756 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5649811872c0 0x5649811e5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 4 client.6756 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1923897408 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56498115c7e0 con 0x5649811872c0 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff19c967700 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5649811872c0 0x5649811e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff19c967700 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5649811872c0 0x5649811e5550 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b387f850f961ffc2 server_cookie=dc3456f2efa2ed9b in_seq=0 out_seq=0 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.057+0000 7ff177fff700 10 client.6756 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff177fff700 1 -- 172.21.15.5:0/1923897408 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56498115c7e0 con 0x5649811872c0 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff177fff700 10 client.6756 handle_command_reply: tid=0 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 client.6756 shutdown 2024-05-08T23:25:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1923897408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56498118b370 msgr2=0x5649811af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56498118b370 0x5649811af430 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff194004600 tx=0x7ff1940047c0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1923897408 shutdown_connections 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5649811872c0 0x5649811e5550 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b7dd0 0x564981186ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56498118b370 0x5649811af430 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1923897408 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b07a0 0x5649811d4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1923897408 >> 172.21.15.5:0/1923897408 conn(0x5649810eb210 msgr2=0x564981187d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff177fff700 0 client.6756 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1923897408 shutdown_connections 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.058+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1923897408 wait complete. 2024-05-08T23:25:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.059+0000 7ff1ab586b80 20 client.6756 trim_cache size 0 max 16384 2024-05-08T23:25:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.074+0000 7ff1ab586b80 1 Processor -- start 2024-05-08T23:25:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff1ab586b80 1 -- start start 2024-05-08T23:25:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56498118b370 0x5649811ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x5649811841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b7dd0 0x5649811af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498118f500 con 0x5649811b07a0 2024-05-08T23:25:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649811800a0 con 0x56498118b370 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649811d3ad0 con 0x5649811b7dd0 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff19c967700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x5649811841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff19c967700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x5649811841e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33258/0 (socket says 172.21.15.5:33258) 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.075+0000 7ff19c967700 1 -- 172.21.15.5:0/710427659 learned_addr learned my addr 172.21.15.5:0/710427659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19d168700 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56498118b370 0x5649811ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19d969700 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b7dd0 0x5649811af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19c967700 1 -- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b7dd0 msgr2=0x5649811af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19c967700 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b7dd0 0x5649811af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19c967700 1 -- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56498118b370 msgr2=0x5649811ebc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19c967700 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56498118b370 0x5649811ebc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19c967700 1 -- 172.21.15.5:0/710427659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649810a3560 con 0x5649811b07a0 2024-05-08T23:25:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19d969700 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b7dd0 0x5649811af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff19c967700 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x5649811841e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff18c058250 tx=0x7ff18c089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ffbff3c928b237 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff1837fe700 1 -- 172.21.15.5:0/710427659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c05c380 con 0x5649811b07a0 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff1837fe700 1 -- 172.21.15.5:0/710427659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c060940 con 0x5649811b07a0 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.076+0000 7ff1837fe700 1 -- 172.21.15.5:0/710427659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0568f0 con 0x5649811b07a0 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 -- 172.21.15.5:0/710427659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 msgr2=0x5649811841e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x5649811841e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff18c058250 tx=0x7ff18c089040 comp rx=0 tx=0).stop 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 -- 172.21.15.5:0/710427659 shutdown_connections 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5649811b7dd0 0x5649811af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56498118b370 0x5649811ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/710427659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b07a0 0x5649811841e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 -- 172.21.15.5:0/710427659 >> 172.21.15.5:0/710427659 conn(0x5649810eb210 msgr2=0x5649811afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 -- 172.21.15.5:0/710427659 shutdown_connections 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 -- 172.21.15.5:0/710427659 wait complete. 2024-05-08T23:25:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 -- start start 2024-05-08T23:25:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.077+0000 7ff1ab586b80 1 Processor -- start 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.078+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 0x56498118df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.078+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b07a0 0x56498123ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.078+0000 7ff1ab586b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b7dd0 0x5649811b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.078+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498118e730 con 0x5649811b7dd0 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.078+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649811b3020 con 0x5649811b07a0 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.078+0000 7ff1ab586b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56498118e4e0 con 0x56498118b370 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff1ab586b80 10 client.0 _ll_register_callbacks cb 0x564981082b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b7dd0 0x5649811b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b7dd0 0x5649811b86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33262/0 (socket says 172.21.15.5:33262) 2024-05-08T23:25:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 -- 172.21.15.5:0/1280938964 learned_addr learned my addr 172.21.15.5:0/1280938964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19c967700 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b07a0 0x56498123ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d168700 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 0x56498118df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 -- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 msgr2=0x56498118df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 0x56498118df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 -- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b07a0 msgr2=0x56498123ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b07a0 0x56498123ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 -- 172.21.15.5:0/1280938964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1980ae710 con 0x5649811b7dd0 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19c967700 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b07a0 0x56498123ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff19d969700 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b7dd0 0x5649811b86d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff190015e70 tx=0x7ff190015ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418259102342386b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff177fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff177fff700 1 -- 172.21.15.5:0/1280938964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1900078d0 con 0x5649811b7dd0 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff177fff700 1 -- 172.21.15.5:0/1280938964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff190007a30 con 0x5649811b7dd0 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1280938964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff19402b070 con 0x5649811b7dd0 2024-05-08T23:25:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.079+0000 7ff177fff700 1 -- 172.21.15.5:0/1280938964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1900020e0 con 0x5649811b7dd0 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff1ab586b80 10 client.6762 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1280938964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x564980f62cd0 con 0x5649811b7dd0 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff177fff700 1 -- 172.21.15.5:0/1280938964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19402b070 con 0x5649811b7dd0 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff177fff700 1 -- 172.21.15.5:0/1280938964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7ff19001e420 con 0x5649811b7dd0 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff1ab586b80 10 client.6762 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff1ab586b80 10 client.6762 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff1ab586b80 20 client.6762 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5649811b8c10 0x56498118b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff1ab586b80 4 client.6762 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1280938964 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649810d5ec0 con 0x5649811b8c10 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff19d168700 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5649811b8c10 0x56498118b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff19d168700 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5649811b8c10 0x56498118b9f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ecad93273f1eee server_cookie=f788c7d90bd19fc5 in_seq=0 out_seq=0 2024-05-08T23:25:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.080+0000 7ff177fff700 10 client.6762 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff177fff700 1 -- 172.21.15.5:0/1280938964 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649810d5ec0 con 0x5649811b8c10 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff177fff700 10 client.6762 handle_command_reply: tid=0 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 client.6762 shutdown 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1280938964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b7dd0 msgr2=0x5649811b86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b7dd0 0x5649811b86d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff190015e70 tx=0x7ff190015ea0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1280938964 shutdown_connections 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649811b7dd0 0x5649811b86d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56498118b370 0x56498118df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5649811b8c10 0x56498118b9f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 --2- 172.21.15.5:0/1280938964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649811b07a0 0x56498123ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff177fff700 0 client.6762 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1280938964 >> 172.21.15.5:0/1280938964 conn(0x5649810eb210 msgr2=0x5649811af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1280938964 shutdown_connections 2024-05-08T23:25:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.092+0000 7ff1ab586b80 1 -- 172.21.15.5:0/1280938964 wait complete. 2024-05-08T23:25:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff1ab586b80 20 client.6762 trim_cache size 0 max 16384 2024-05-08T23:25:11.095 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:25:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:25:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:25:11.095 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:25:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 -- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff17c06ac90 msgr2=0x7ff17c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff17c06ac90 0x7ff17c06d150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff1940090b0 tx=0x7ff194008030 comp rx=0 tx=0).stop 2024-05-08T23:25:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 -- 172.21.15.5:0/606369872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 msgr2=0x7ff19809d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff19809d260 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00d800 tx=0x7ff18c00dbc0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 -- 172.21.15.5:0/606369872 shutdown_connections 2024-05-08T23:25:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff17c06ac90 0x7ff17c06d150 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1980b4d50 0x7ff19809d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff19806aea0 0x7ff198140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 --2- 172.21.15.5:0/606369872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1980b0d80 0x7ff19809d260 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 -- 172.21.15.5:0/606369872 >> 172.21.15.5:0/606369872 conn(0x7ff198099a50 msgr2=0x7ff198146580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 -- 172.21.15.5:0/606369872 shutdown_connections 2024-05-08T23:25:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.093+0000 7ff19f3cc700 1 -- 172.21.15.5:0/606369872 wait complete. 2024-05-08T23:25:11.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 flush journal 2024-05-08T23:25:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.246+0000 7f9092f39700 1 Processor -- start 2024-05-08T23:25:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.246+0000 7f9092f39700 1 -- start start 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f9092f39700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f9092f39700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f908c0aea70 0x7f908c0aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f9092f39700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908c15c920 con 0x7f908c0b5500 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f9092f39700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908c0ad6b0 con 0x7f908c0aea70 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f9092f39700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0b2700 con 0x7f908c06a960 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f90914d6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f908c0aea70 0x7f908c0aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f9090cd5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f908bfff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f908c0b5500 0x7f908c06ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:59380/0 (socket says 172.21.15.5:59380) 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.247+0000 7f908bfff700 1 -- 172.21.15.5:0/126173306 learned_addr learned my addr 172.21.15.5:0/126173306 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f90914d6700 1 -- 172.21.15.5:0/126173306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 msgr2=0x7f908c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f90914d6700 1 --2- 172.21.15.5:0/126173306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f90914d6700 1 -- 172.21.15.5:0/126173306 >> v1:172.21.15.49:6789/0 conn(0x7f908c0b5500 legacy=0x7f908c06ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f90914d6700 1 -- 172.21.15.5:0/126173306 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c0b2910 con 0x7f908c0aea70 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f908b7fe700 1 -- 172.21.15.5:0/126173306 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377762008 0 0) 0x7f908c15c920 con 0x7f908c0b5500 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f9090cd5700 1 --2- 172.21.15.5:0/126173306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f90914d6700 1 --2- 172.21.15.5:0/126173306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 0x7f908c0aef00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9084009d90 tx=0x7f908400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e954d226da12341a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f908b7fe700 1 -- 172.21.15.5:0/126173306 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908400e070 con 0x7f908c0aea70 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f908b7fe700 1 -- 172.21.15.5:0/126173306 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908400bb20 con 0x7f908c0aea70 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f908b7fe700 1 -- 172.21.15.5:0/126173306 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084012430 con 0x7f908c0aea70 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f9092f39700 1 -- 172.21.15.5:0/126173306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 msgr2=0x7f908c0aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.248+0000 7f9092f39700 1 --2- 172.21.15.5:0/126173306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 0x7f908c0aef00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9084009d90 tx=0x7f908400b330 comp rx=0 tx=0).stop 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.249+0000 7f9092f39700 1 -- 172.21.15.5:0/126173306 shutdown_connections 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.249+0000 7f9092f39700 1 --2- 172.21.15.5:0/126173306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 0x7f908c0aef00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.249+0000 7f9092f39700 1 --2- 172.21.15.5:0/126173306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.249+0000 7f9092f39700 1 -- 172.21.15.5:0/126173306 >> 172.21.15.5:0/126173306 conn(0x7f908c0b5a30 msgr2=0x7f908c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.249+0000 7f9092f39700 1 -- 172.21.15.5:0/126173306 shutdown_connections 2024-05-08T23:25:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.249+0000 7f9092f39700 1 -- 172.21.15.5:0/126173306 wait complete. 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.250+0000 7f9092f39700 1 Processor -- start 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.250+0000 7f9092f39700 1 -- start start 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f9092f39700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c09a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f9092f39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 0x7f908c09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f9092f39700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c0b5500 0x7f908c09da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f9092f39700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c09c9a0 con 0x7f908c0b5500 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f9092f39700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c09cc80 con 0x7f908c0aea70 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f9092f39700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c09cf60 con 0x7f908c06a960 2024-05-08T23:25:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f90914d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c0b5500 0x7f908c09da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f90914d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c0b5500 0x7f908c09da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33278/0 (socket says 172.21.15.5:33278) 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f90914d6700 1 -- 172.21.15.5:0/4292627802 learned_addr learned my addr 172.21.15.5:0/4292627802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f908bfff700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 0x7f908c09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f9090cd5700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c09a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f90914d6700 1 -- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 msgr2=0x7f908c09a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f90914d6700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c09a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f90914d6700 1 -- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 msgr2=0x7f908c09c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f90914d6700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 0x7f908c09c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.251+0000 7f90914d6700 1 -- 172.21.15.5:0/4292627802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90800094a0 con 0x7f908c0b5500 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.252+0000 7f90914d6700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c0b5500 0x7f908c09da40 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f908c06dbe0 tx=0x7f9084004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.252+0000 7f90897fa700 1 -- 172.21.15.5:0/4292627802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084012430 con 0x7f908c0b5500 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.252+0000 7f90897fa700 1 -- 172.21.15.5:0/4292627802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9084020070 con 0x7f908c0b5500 2024-05-08T23:25:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.252+0000 7f9092f39700 1 -- 172.21.15.5:0/4292627802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9084009a70 con 0x7f908c0b5500 2024-05-08T23:25:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.252+0000 7f90897fa700 1 -- 172.21.15.5:0/4292627802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084004280 con 0x7f908c0b5500 2024-05-08T23:25:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.252+0000 7f9092f39700 1 -- 172.21.15.5:0/4292627802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c09d520 con 0x7f908c0b5500 2024-05-08T23:25:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.253+0000 7f90897fa700 1 -- 172.21.15.5:0/4292627802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f90840043e0 con 0x7f908c0b5500 2024-05-08T23:25:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.254+0000 7f90897fa700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f907006ac90 0x7f907006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.254+0000 7f90897fa700 1 -- 172.21.15.5:0/4292627802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9084022070 con 0x7f908c0b5500 2024-05-08T23:25:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.254+0000 7f9090cd5700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f907006ac90 0x7f907006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.255+0000 7f9090cd5700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f907006ac90 0x7f907006d150 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9080009d00 tx=0x7f9080008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.256+0000 7f909f0f3b80 1 Processor -- start 2024-05-08T23:25:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.256+0000 7f909f0f3b80 1 -- start start 2024-05-08T23:25:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a68e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a68ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 0x5557a6916250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a6842150 con 0x5557a690e7a0 2024-05-08T23:25:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a683fb10 con 0x5557a68e9370 2024-05-08T23:25:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557a683b400 con 0x5557a6915dd0 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a68e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a68e7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33286/0 (socket says 172.21.15.5:33286) 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 -- 172.21.15.5:0/3691417111 learned_addr learned my addr 172.21.15.5:0/3691417111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f908bfff700 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a68ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f90914d6700 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 0x5557a6916250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 -- 172.21.15.5:0/3691417111 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 msgr2=0x5557a6916250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 0x5557a6916250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 -- 172.21.15.5:0/3691417111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 msgr2=0x5557a68ec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a68ec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 -- 172.21.15.5:0/3691417111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557a66bc3b0 con 0x5557a690e7a0 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.257+0000 7f9090cd5700 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a68e7110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9080009470 tx=0x7f9080014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a35b9ec71ad1695 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f90767fc700 1 -- 172.21.15.5:0/3691417111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080011e00 con 0x5557a690e7a0 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f90767fc700 1 -- 172.21.15.5:0/3691417111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9080015c60 con 0x5557a690e7a0 2024-05-08T23:25:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f90767fc700 1 -- 172.21.15.5:0/3691417111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080021440 con 0x5557a690e7a0 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3691417111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 msgr2=0x5557a68e7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a68e7110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9080009470 tx=0x7f9080014000 comp rx=0 tx=0).stop 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3691417111 shutdown_connections 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 0x5557a6916250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a68e7110 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3691417111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a68ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3691417111 >> 172.21.15.5:0/3691417111 conn(0x5557a6849210 msgr2=0x5557a6930dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3691417111 shutdown_connections 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.258+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3691417111 wait complete. 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.259+0000 7f909f0f3b80 1 -- start start 2024-05-08T23:25:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.259+0000 7f909f0f3b80 1 Processor -- start 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 0x5557a690ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a69195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a6915dd0 0x5557a6933a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a67d7560 con 0x5557a690e7a0 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a67d7200 con 0x5557a6915dd0 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557a66be6a0 con 0x5557a68e9370 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f909f0f3b80 10 client.0 _ll_register_callbacks cb 0x5557a67e0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f908bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a69195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f908bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a69195f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33302/0 (socket says 172.21.15.5:33302) 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f908bfff700 1 -- 172.21.15.5:0/385116633 learned_addr learned my addr 172.21.15.5:0/385116633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f9090cd5700 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 0x5557a690ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.260+0000 7f90914d6700 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a6915dd0 0x5557a6933a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f908bfff700 1 -- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 msgr2=0x5557a690ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f908bfff700 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 0x5557a690ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f908bfff700 1 -- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a6915dd0 msgr2=0x5557a6933a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f908bfff700 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a6915dd0 0x5557a6933a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f908bfff700 1 -- 172.21.15.5:0/385116633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9080011ae0 con 0x5557a690e7a0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f90914d6700 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a6915dd0 0x5557a6933a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f908bfff700 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a69195f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f90780074e0 tx=0x7f90780077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f077a5033c0a04a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f906b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f906b7fe700 1 -- 172.21.15.5:0/385116633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9078001960 con 0x5557a690e7a0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f906b7fe700 1 -- 172.21.15.5:0/385116633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9078001ac0 con 0x5557a690e7a0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 1 -- 172.21.15.5:0/385116633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557a6794270 con 0x5557a690e7a0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f906b7fe700 1 -- 172.21.15.5:0/385116633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9078001cd0 con 0x5557a690e7a0 2024-05-08T23:25:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 10 client.6771 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 1 -- 172.21.15.5:0/385116633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5557a6668b60 con 0x5557a690e7a0 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f906b7fe700 1 -- 172.21.15.5:0/385116633 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5557a6794270 con 0x5557a690e7a0 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f906b7fe700 1 -- 172.21.15.5:0/385116633 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f9078014400 con 0x5557a690e7a0 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 10 client.6771 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 10 client.6771 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.c 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 20 client.6771 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5557a6933420 0x5557a6916f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 4 client.6771 mds_command: new command op to 6516 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.261+0000 7f909f0f3b80 1 -- 172.21.15.5:0/385116633 --> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557a68ba7e0 con 0x5557a6933420 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.262+0000 7f9090cd5700 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5557a6933420 0x5557a6916f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.262+0000 7f9090cd5700 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5557a6933420 0x5557a6916f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc251554dca67e58 server_cookie=526588e04665cd83 in_seq=0 out_seq=0 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.262+0000 7f906b7fe700 10 client.6771 ms_handle_connect on v2:172.21.15.5:6832/2315721152 2024-05-08T23:25:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.262+0000 7f906b7fe700 1 -- 172.21.15.5:0/385116633 <== mds.1 v2:172.21.15.5:6832/2315721152 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557a68ba7e0 con 0x5557a6933420 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.262+0000 7f906b7fe700 10 client.6771 handle_command_reply: tid=0 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.262+0000 7f909f0f3b80 1 client.6771 shutdown 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 -- 172.21.15.5:0/385116633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 msgr2=0x5557a69195f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a69195f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f90780074e0 tx=0x7f90780077f0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 -- 172.21.15.5:0/385116633 shutdown_connections 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a6915dd0 0x5557a6933a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5557a6933420 0x5557a6916f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 0x5557a690ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/385116633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a69195f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f906b7fe700 0 client.6771 ms_handle_reset on v2:172.21.15.5:6832/2315721152 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 -- 172.21.15.5:0/385116633 >> 172.21.15.5:0/385116633 conn(0x5557a6849210 msgr2=0x5557a68eb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 -- 172.21.15.5:0/385116633 shutdown_connections 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 1 -- 172.21.15.5:0/385116633 wait complete. 2024-05-08T23:25:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.263+0000 7f909f0f3b80 20 client.6771 trim_cache size 0 max 16384 2024-05-08T23:25:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.279+0000 7f909f0f3b80 1 Processor -- start 2024-05-08T23:25:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.279+0000 7f909f0f3b80 1 -- start start 2024-05-08T23:25:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.279+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a6934c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.279+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a6933800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 0x5557a6930120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a68e5d50 con 0x5557a690e7a0 2024-05-08T23:25:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a69361b0 con 0x5557a68e9370 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557a6916f70 con 0x5557a6915dd0 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a6933800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f9090cd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a6934c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a6933800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33304/0 (socket says 172.21.15.5:33304) 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 -- 172.21.15.5:0/1584332213 learned_addr learned my addr 172.21.15.5:0/1584332213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f90914d6700 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 0x5557a6930120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 -- 172.21.15.5:0/1584332213 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 msgr2=0x5557a6930120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 0x5557a6930120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 -- 172.21.15.5:0/1584332213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 msgr2=0x5557a6934c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a6934c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 -- 172.21.15.5:0/1584332213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557a6801560 con 0x5557a690e7a0 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f9090cd5700 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a6934c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f908bfff700 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a6933800 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9078000fc0 tx=0x7f907800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46225df7dc0975 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f9076ffd700 1 -- 172.21.15.5:0/1584332213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780014e0 con 0x5557a690e7a0 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f9076ffd700 1 -- 172.21.15.5:0/1584332213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9078029070 con 0x5557a690e7a0 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.280+0000 7f9076ffd700 1 -- 172.21.15.5:0/1584332213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780148c0 con 0x5557a690e7a0 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 -- 172.21.15.5:0/1584332213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 msgr2=0x5557a6933800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a6933800 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9078000fc0 tx=0x7f907800e000 comp rx=0 tx=0).stop 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 -- 172.21.15.5:0/1584332213 shutdown_connections 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a6915dd0 0x5557a6930120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a68e9370 0x5557a6934c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/1584332213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a690e7a0 0x5557a6933800 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 -- 172.21.15.5:0/1584332213 >> 172.21.15.5:0/1584332213 conn(0x5557a6849210 msgr2=0x5557a68e21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 -- 172.21.15.5:0/1584332213 shutdown_connections 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.281+0000 7f909f0f3b80 1 -- 172.21.15.5:0/1584332213 wait complete. 2024-05-08T23:25:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.282+0000 7f909f0f3b80 1 -- start start 2024-05-08T23:25:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.282+0000 7f909f0f3b80 1 Processor -- start 2024-05-08T23:25:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 0x5557a6926db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a690e7a0 0x5557a6930120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f909f0f3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a6915dd0 0x5557a6930660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a6926990 con 0x5557a6915dd0 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a690c740 con 0x5557a690e7a0 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f909f0f3b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557a6916340 con 0x5557a68e9370 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f909f0f3b80 10 client.0 _ll_register_callbacks cb 0x5557a67e0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a6915dd0 0x5557a6930660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a6915dd0 0x5557a6930660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33320/0 (socket says 172.21.15.5:33320) 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 -- 172.21.15.5:0/3772322102 learned_addr learned my addr 172.21.15.5:0/3772322102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f9090cd5700 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 0x5557a6926db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f908bfff700 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a690e7a0 0x5557a6930120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 -- 172.21.15.5:0/3772322102 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 msgr2=0x5557a6926db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 0x5557a6926db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 -- 172.21.15.5:0/3772322102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a690e7a0 msgr2=0x5557a6930120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a690e7a0 0x5557a6930120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 -- 172.21.15.5:0/3772322102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557a66bc3b0 con 0x5557a6915dd0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f90914d6700 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a6915dd0 0x5557a6930660 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9084059ce0 tx=0x7f908405a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4e45c8ba588677 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f906b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f906b7fe700 1 -- 172.21.15.5:0/3772322102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908405a6e0 con 0x5557a6915dd0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f906b7fe700 1 -- 172.21.15.5:0/3772322102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9084059fe0 con 0x5557a6915dd0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.283+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3772322102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9080015c60 con 0x5557a6915dd0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f906b7fe700 1 -- 172.21.15.5:0/3772322102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f908405de10 con 0x5557a6915dd0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f909f0f3b80 10 client.6777 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3772322102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5557a6668b60 con 0x5557a6915dd0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f906b7fe700 1 -- 172.21.15.5:0/3772322102 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9080015c60 con 0x5557a6915dd0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f906b7fe700 1 -- 172.21.15.5:0/3772322102 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f9084061f40 con 0x5557a6915dd0 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f909f0f3b80 10 client.6777 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f909f0f3b80 10 client.6777 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.c 2024-05-08T23:25:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f909f0f3b80 20 client.6777 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5557a6999a30 0x5557a690b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f909f0f3b80 4 client.6777 mds_command: new command op to 6516 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:25:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3772322102 --> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5557a6833ec0 con 0x5557a6999a30 2024-05-08T23:25:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f9090cd5700 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5557a6999a30 0x5557a690b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f906b7fe700 10 client.6777 ms_handle_connect on v2:172.21.15.5:6832/2315721152 2024-05-08T23:25:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.284+0000 7f9090cd5700 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5557a6999a30 0x5557a690b890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43ac1ede238dd22f server_cookie=d835190fab2fdce4 in_seq=0 out_seq=0 2024-05-08T23:25:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f906b7fe700 1 -- 172.21.15.5:0/3772322102 <== mds.1 v2:172.21.15.5:6832/2315721152 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5557a6833ec0 con 0x5557a6999a30 2024-05-08T23:25:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f906b7fe700 10 client.6777 handle_command_reply: tid=0 2024-05-08T23:25:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 client.6777 shutdown 2024-05-08T23:25:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3772322102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a6915dd0 msgr2=0x5557a6930660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a6915dd0 0x5557a6930660 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9084059ce0 tx=0x7f908405a520 comp rx=0 tx=0).stop 2024-05-08T23:25:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3772322102 shutdown_connections 2024-05-08T23:25:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a6915dd0 0x5557a6930660 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5557a6999a30 0x5557a690b890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5557a68e9370 0x5557a6926db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 --2- 172.21.15.5:0/3772322102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5557a690e7a0 0x5557a6930120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f906b7fe700 0 client.6777 ms_handle_reset on v2:172.21.15.5:6832/2315721152 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.292+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3772322102 >> 172.21.15.5:0/3772322102 conn(0x5557a6849210 msgr2=0x5557a6938b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3772322102 shutdown_connections 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f909f0f3b80 1 -- 172.21.15.5:0/3772322102 wait complete. 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f909f0f3b80 20 client.6777 trim_cache size 0 max 16384 2024-05-08T23:25:11.294 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:25:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:25:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:25:11.295 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:25:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 -- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f907006ac90 msgr2=0x7f907006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f907006ac90 0x7f907006d150 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9080009d00 tx=0x7f9080008040 comp rx=0 tx=0).stop 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 -- 172.21.15.5:0/4292627802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c0b5500 msgr2=0x7f908c09da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c0b5500 0x7f908c09da40 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f908c06dbe0 tx=0x7f9084004ad0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 -- 172.21.15.5:0/4292627802 shutdown_connections 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f907006ac90 0x7f907006d150 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c0b5500 0x7f908c09da40 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f908c06a960 0x7f908c09a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 --2- 172.21.15.5:0/4292627802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f908c0aea70 0x7f908c09c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 -- 172.21.15.5:0/4292627802 >> 172.21.15.5:0/4292627802 conn(0x7f908c0b5a30 msgr2=0x7f908c0b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.293+0000 7f9092f39700 1 -- 172.21.15.5:0/4292627802 shutdown_connections 2024-05-08T23:25:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.294+0000 7f9092f39700 1 -- 172.21.15.5:0/4292627802 wait complete. 2024-05-08T23:25:11.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 flush journal 2024-05-08T23:25:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.447+0000 7f4330129700 1 Processor -- start 2024-05-08T23:25:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.447+0000 7f4330129700 1 -- start start 2024-05-08T23:25:11.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.448+0000 7f4330129700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f432806b070 0x7f43280b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.448+0000 7f4330129700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280b16e0 con 0x7f4328162d40 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.448+0000 7f4330129700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43281631b0 con 0x7f432814b930 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.448+0000 7f4330129700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43280b1980 con 0x7f432806b070 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.448+0000 7f432d6c4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f432806b070 0x7f43280b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.448+0000 7f432e6c6700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4328162d40 0x7f43281633f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38548/0 (socket says 172.21.15.5:38548) 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.448+0000 7f432e6c6700 1 -- 172.21.15.5:0/675902965 learned_addr learned my addr 172.21.15.5:0/675902965 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.449+0000 7f432d6c4700 1 -- 172.21.15.5:0/675902965 >> v1:172.21.15.49:6789/0 conn(0x7f432814b930 legacy=0x7f432806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.449+0000 7f432d6c4700 1 -- 172.21.15.5:0/675902965 >> v1:172.21.15.5:6789/0 conn(0x7f4328162d40 legacy=0x7f43281633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.449+0000 7f432cec3700 1 -- 172.21.15.5:0/675902965 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888527197 0 0) 0x7f43280b16e0 con 0x7f4328162d40 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.449+0000 7f432d6c4700 1 -- 172.21.15.5:0/675902965 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280b1c60 con 0x7f432806b070 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.449+0000 7f432d6c4700 1 --2- 172.21.15.5:0/675902965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f432806b070 0x7f43280b32f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4318009870 tx=0x7f4318009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec49776fe55e9ae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.449+0000 7f432cec3700 1 -- 172.21.15.5:0/675902965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318011830 con 0x7f432806b070 2024-05-08T23:25:11.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.449+0000 7f432cec3700 1 -- 172.21.15.5:0/675902965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318011990 con 0x7f432806b070 2024-05-08T23:25:11.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.449+0000 7f432cec3700 1 -- 172.21.15.5:0/675902965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318011d20 con 0x7f432806b070 2024-05-08T23:25:11.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.450+0000 7f4330129700 1 -- 172.21.15.5:0/675902965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f432806b070 msgr2=0x7f43280b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.450+0000 7f4330129700 1 --2- 172.21.15.5:0/675902965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f432806b070 0x7f43280b32f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4318009870 tx=0x7f4318009b80 comp rx=0 tx=0).stop 2024-05-08T23:25:11.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.450+0000 7f4330129700 1 -- 172.21.15.5:0/675902965 shutdown_connections 2024-05-08T23:25:11.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.450+0000 7f4330129700 1 --2- 172.21.15.5:0/675902965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f432806b070 0x7f43280b32f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.450+0000 7f4330129700 1 -- 172.21.15.5:0/675902965 >> 172.21.15.5:0/675902965 conn(0x7f4328099a50 msgr2=0x7f43280b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.450+0000 7f4330129700 1 -- 172.21.15.5:0/675902965 shutdown_connections 2024-05-08T23:25:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.450+0000 7f4330129700 1 -- 172.21.15.5:0/675902965 wait complete. 2024-05-08T23:25:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.451+0000 7f4330129700 1 Processor -- start 2024-05-08T23:25:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.451+0000 7f4330129700 1 -- start start 2024-05-08T23:25:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f4330129700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432806b070 0x7f43281f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f4330129700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f432814b930 0x7f43280cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f4330129700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4328162d40 0x7f43280aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f4330129700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4328143e60 con 0x7f4328162d40 2024-05-08T23:25:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f4330129700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43280cdea0 con 0x7f432806b070 2024-05-08T23:25:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f4330129700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280ce180 con 0x7f432814b930 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432e6c6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4328162d40 0x7f43280aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432e6c6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4328162d40 0x7f43280aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33348/0 (socket says 172.21.15.5:33348) 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432e6c6700 1 -- 172.21.15.5:0/2378920830 learned_addr learned my addr 172.21.15.5:0/2378920830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432dec5700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432806b070 0x7f43281f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432d6c4700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f432814b930 0x7f43280cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432e6c6700 1 -- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f432814b930 msgr2=0x7f43280cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432e6c6700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f432814b930 0x7f43280cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432e6c6700 1 -- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432806b070 msgr2=0x7f43281f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432e6c6700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432806b070 0x7f43281f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432e6c6700 1 -- 172.21.15.5:0/2378920830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180094a0 con 0x7f4328162d40 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.452+0000 7f432dec5700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432806b070 0x7f43281f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.453+0000 7f432e6c6700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4328162d40 0x7f43280aeb20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f432000cba0 tx=0x7f432000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.453+0000 7f431e7fc700 1 -- 172.21.15.5:0/2378920830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432000ab80 con 0x7f4328162d40 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.453+0000 7f431e7fc700 1 -- 172.21.15.5:0/2378920830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432000ad00 con 0x7f4328162d40 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.453+0000 7f4330129700 1 -- 172.21.15.5:0/2378920830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43280ce460 con 0x7f4328162d40 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.453+0000 7f431e7fc700 1 -- 172.21.15.5:0/2378920830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320007310 con 0x7f4328162d40 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.453+0000 7f4330129700 1 -- 172.21.15.5:0/2378920830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43280cea20 con 0x7f4328162d40 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.455+0000 7f431e7fc700 1 -- 172.21.15.5:0/2378920830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4320007470 con 0x7f4328162d40 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.455+0000 7f431e7fc700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f430c06ac90 0x7f430c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.455+0000 7f431e7fc700 1 -- 172.21.15.5:0/2378920830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43200891c0 con 0x7f4328162d40 2024-05-08T23:25:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.456+0000 7f432dec5700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f430c06ac90 0x7f430c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.456+0000 7f432dec5700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f430c06ac90 0x7f430c06d150 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4324000f60 tx=0x7f4324009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f433c2e3b80 1 Processor -- start 2024-05-08T23:25:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f433c2e3b80 1 -- start start 2024-05-08T23:25:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa0a7a0 0x5637fa9e3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 0x5637fa9e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa11dd0 0x5637faa12250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637fa93e150 con 0x5637faa11dd0 2024-05-08T23:25:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637fa93bb10 con 0x5637fa9e5370 2024-05-08T23:25:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637fa937400 con 0x5637faa0a7a0 2024-05-08T23:25:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.458+0000 7f432e6c6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa11dd0 0x5637faa12250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432e6c6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa11dd0 0x5637faa12250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33364/0 (socket says 172.21.15.5:33364) 2024-05-08T23:25:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432e6c6700 1 -- 172.21.15.5:0/457840783 learned_addr learned my addr 172.21.15.5:0/457840783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432d6c4700 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 0x5637fa9e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432dec5700 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa0a7a0 0x5637fa9e3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432e6c6700 1 -- 172.21.15.5:0/457840783 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa0a7a0 msgr2=0x5637fa9e3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432e6c6700 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa0a7a0 0x5637fa9e3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432e6c6700 1 -- 172.21.15.5:0/457840783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 msgr2=0x5637fa9e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432e6c6700 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 0x5637fa9e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432e6c6700 1 -- 172.21.15.5:0/457840783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637fa7b83b0 con 0x5637faa11dd0 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f432e6c6700 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa11dd0 0x5637faa12250 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f432000cfd0 tx=0x7f432000ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d198fc90df7a9c66 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f430b7fe700 1 -- 172.21.15.5:0/457840783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320058050 con 0x5637faa11dd0 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f430b7fe700 1 -- 172.21.15.5:0/457840783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432006be90 con 0x5637faa11dd0 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.459+0000 7f430b7fe700 1 -- 172.21.15.5:0/457840783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432005be80 con 0x5637faa11dd0 2024-05-08T23:25:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 -- 172.21.15.5:0/457840783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa11dd0 msgr2=0x5637faa12250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa11dd0 0x5637faa12250 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f432000cfd0 tx=0x7f432000ae70 comp rx=0 tx=0).stop 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 -- 172.21.15.5:0/457840783 shutdown_connections 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa11dd0 0x5637faa12250 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa0a7a0 0x5637fa9e3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/457840783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 0x5637fa9e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 -- 172.21.15.5:0/457840783 >> 172.21.15.5:0/457840783 conn(0x5637fa945210 msgr2=0x5637faa2cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 -- 172.21.15.5:0/457840783 shutdown_connections 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.460+0000 7f433c2e3b80 1 -- 172.21.15.5:0/457840783 wait complete. 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.461+0000 7f433c2e3b80 1 -- start start 2024-05-08T23:25:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.461+0000 7f433c2e3b80 1 Processor -- start 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 0x5637faa40050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa15680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa0d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637fa8d3560 con 0x5637faa0a7a0 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637fa8d3200 con 0x5637faa11dd0 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637fa7ba6a0 con 0x5637fa9e5370 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f433c2e3b80 10 client.0 _ll_register_callbacks cb 0x5637fa8dcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa15680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa15680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33372/0 (socket says 172.21.15.5:33372) 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 -- 172.21.15.5:0/1710874604 learned_addr learned my addr 172.21.15.5:0/1710874604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432dec5700 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 0x5637faa40050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432e6c6700 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa0d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 -- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 msgr2=0x5637faa40050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 0x5637faa40050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 -- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 msgr2=0x5637faa0d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa0d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 -- 172.21.15.5:0/1710874604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43280cea20 con 0x5637faa0a7a0 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432e6c6700 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa0d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f432d6c4700 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa15680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4318000c00 tx=0x7f4318010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739bbdcb1d257c82 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f4308ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.462+0000 7f4308ff9700 1 -- 172.21.15.5:0/1710874604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318021040 con 0x5637faa0a7a0 2024-05-08T23:25:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f4308ff9700 1 -- 172.21.15.5:0/1710874604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43180036a0 con 0x5637faa0a7a0 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1710874604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637fa890270 con 0x5637faa0a7a0 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f4308ff9700 1 -- 172.21.15.5:0/1710874604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4318003860 con 0x5637faa0a7a0 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 10 client.6792 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1710874604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5637fa764b60 con 0x5637faa0a7a0 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f4308ff9700 1 -- 172.21.15.5:0/1710874604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5637fa890270 con 0x5637faa0a7a0 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f4308ff9700 1 -- 172.21.15.5:0/1710874604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f43180039c0 con 0x5637faa0a7a0 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 10 client.6792 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 10 client.6792 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.d 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 20 client.6792 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x5637faa2e250 0x5637faa2e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 4 client.6792 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1710874604 --> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637fa9b67e0 con 0x5637faa2e250 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.463+0000 7f432dec5700 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x5637faa2e250 0x5637faa2e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f4308ff9700 10 client.6792 ms_handle_connect on v2:172.21.15.49:6834/3197377913 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f432dec5700 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x5637faa2e250 0x5637faa2e630 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f56010c9166568e server_cookie=590122a9cf126509 in_seq=0 out_seq=0 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f4308ff9700 1 -- 172.21.15.5:0/1710874604 <== mds.2 v2:172.21.15.49:6834/3197377913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5637fa9b67e0 con 0x5637faa2e250 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f4308ff9700 10 client.6792 handle_command_reply: tid=0 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f433c2e3b80 1 client.6792 shutdown 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1710874604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 msgr2=0x5637faa15680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa15680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4318000c00 tx=0x7f4318010000 comp rx=0 tx=0).stop 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1710874604 shutdown_connections 2024-05-08T23:25:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x5637faa2e250 0x5637faa2e630 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa0d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 0x5637faa40050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f4308ff9700 0 client.6792 ms_handle_reset on v2:172.21.15.49:6834/3197377913 2024-05-08T23:25:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.464+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1710874604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa15680 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.465+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1710874604 >> 172.21.15.5:0/1710874604 conn(0x5637fa945210 msgr2=0x5637faa0ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.465+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1710874604 shutdown_connections 2024-05-08T23:25:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.465+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1710874604 wait complete. 2024-05-08T23:25:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.465+0000 7f433c2e3b80 20 client.6792 trim_cache size 0 max 16384 2024-05-08T23:25:11.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.480+0000 7f433c2e3b80 1 Processor -- start 2024-05-08T23:25:11.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f433c2e3b80 1 -- start start 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 0x5637faa29eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa0efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa11dd0 0x5637faa09a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637fa9e33b0 con 0x5637faa0a7a0 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637fa9e35f0 con 0x5637fa9e5370 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637faa4ead0 con 0x5637faa11dd0 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f432d6c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa0efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f432d6c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa0efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33382/0 (socket says 172.21.15.5:33382) 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.481+0000 7f432d6c4700 1 -- 172.21.15.5:0/266081638 learned_addr learned my addr 172.21.15.5:0/266081638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f432e6c6700 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa11dd0 0x5637faa09a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f432d6c4700 1 -- 172.21.15.5:0/266081638 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa11dd0 msgr2=0x5637faa09a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f432dec5700 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 0x5637faa29eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f432d6c4700 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa11dd0 0x5637faa09a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f432d6c4700 1 -- 172.21.15.5:0/266081638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 msgr2=0x5637faa29eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f432d6c4700 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 0x5637faa29eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f432d6c4700 1 -- 172.21.15.5:0/266081638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637fa8fd560 con 0x5637faa0a7a0 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f432d6c4700 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa0efd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f43180090b0 tx=0x7f4318009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76e89f0968dc0f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f430bfff700 1 -- 172.21.15.5:0/266081638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318021040 con 0x5637faa0a7a0 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f430bfff700 1 -- 172.21.15.5:0/266081638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43180044c0 con 0x5637faa0a7a0 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.482+0000 7f430bfff700 1 -- 172.21.15.5:0/266081638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431801ada0 con 0x5637faa0a7a0 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 -- 172.21.15.5:0/266081638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 msgr2=0x5637faa0efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa0efd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f43180090b0 tx=0x7f4318009210 comp rx=0 tx=0).stop 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 -- 172.21.15.5:0/266081638 shutdown_connections 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637faa11dd0 0x5637faa09a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637fa9e5370 0x5637faa29eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/266081638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa0efd0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 -- 172.21.15.5:0/266081638 >> 172.21.15.5:0/266081638 conn(0x5637fa945210 msgr2=0x5637fa9de1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 -- 172.21.15.5:0/266081638 shutdown_connections 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 -- 172.21.15.5:0/266081638 wait complete. 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.483+0000 7f433c2e3b80 1 -- start start 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.484+0000 7f433c2e3b80 1 Processor -- start 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 0x5637fa9e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa09a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f433c2e3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa09f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637faa8c980 con 0x5637faa0a7a0 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637faa99850 con 0x5637faa11dd0 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f433c2e3b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637faa95b80 con 0x5637fa9e5370 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f433c2e3b80 10 client.0 _ll_register_callbacks cb 0x5637fa8dcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432d6c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa09a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432d6c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa09a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33392/0 (socket says 172.21.15.5:33392) 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432d6c4700 1 -- 172.21.15.5:0/1168996117 learned_addr learned my addr 172.21.15.5:0/1168996117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432e6c6700 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa09f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432dec5700 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 0x5637fa9e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432d6c4700 1 -- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 msgr2=0x5637fa9e4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432d6c4700 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 0x5637fa9e4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432d6c4700 1 -- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 msgr2=0x5637faa09f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432d6c4700 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa09f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.485+0000 7f432d6c4700 1 -- 172.21.15.5:0/1168996117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637fa764b60 con 0x5637faa0a7a0 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f432d6c4700 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa09a50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4318038040 tx=0x7f43180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c8b2710756c722 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f4308ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f4308ff9700 1 -- 172.21.15.5:0/1168996117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318021050 con 0x5637faa0a7a0 2024-05-08T23:25:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f4308ff9700 1 -- 172.21.15.5:0/1168996117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4318000e40 con 0x5637faa0a7a0 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1168996117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f432006be90 con 0x5637faa0a7a0 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f4308ff9700 1 -- 172.21.15.5:0/1168996117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43180038a0 con 0x5637faa0a7a0 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 10 client.6798 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1168996117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5637fa7bccd0 con 0x5637faa0a7a0 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f4308ff9700 1 -- 172.21.15.5:0/1168996117 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432006be90 con 0x5637faa0a7a0 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f4308ff9700 1 -- 172.21.15.5:0/1168996117 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f4318003a00 con 0x5637faa0a7a0 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 10 client.6798 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 10 client.6798 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.d 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 20 client.6798 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x5637fa9e8320 0x5637faa96010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 4 client.6798 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:25:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1168996117 --> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5637fa92fec0 con 0x5637fa9e8320 2024-05-08T23:25:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.486+0000 7f432dec5700 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x5637fa9e8320 0x5637faa96010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.487+0000 7f4308ff9700 10 client.6798 ms_handle_connect on v2:172.21.15.49:6834/3197377913 2024-05-08T23:25:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.487+0000 7f432dec5700 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x5637fa9e8320 0x5637faa96010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3eb48bc203b56fb2 server_cookie=4176859b12a2398d in_seq=0 out_seq=0 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f4308ff9700 1 -- 172.21.15.5:0/1168996117 <== mds.2 v2:172.21.15.49:6834/3197377913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5637fa92fec0 con 0x5637fa9e8320 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f4308ff9700 10 client.6798 handle_command_reply: tid=0 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 client.6798 shutdown 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1168996117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 msgr2=0x5637faa09a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa09a50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4318038040 tx=0x7f43180047c0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1168996117 shutdown_connections 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x5637fa9e8320 0x5637faa96010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5637faa11dd0 0x5637faa09f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5637fa9e5370 0x5637fa9e4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 --2- 172.21.15.5:0/1168996117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637faa0a7a0 0x5637faa09a50 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f4308ff9700 0 client.6798 ms_handle_reset on v2:172.21.15.49:6834/3197377913 2024-05-08T23:25:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1168996117 >> 172.21.15.5:0/1168996117 conn(0x5637fa945210 msgr2=0x5637faa0f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1168996117 shutdown_connections 2024-05-08T23:25:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 1 -- 172.21.15.5:0/1168996117 wait complete. 2024-05-08T23:25:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.494+0000 7f433c2e3b80 20 client.6798 trim_cache size 0 max 16384 2024-05-08T23:25:11.497 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:25:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:25:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:25:11.498 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:25:11.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 -- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f430c06ac90 msgr2=0x7f430c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f430c06ac90 0x7f430c06d150 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4324000f60 tx=0x7f4324009040 comp rx=0 tx=0).stop 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 -- 172.21.15.5:0/2378920830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4328162d40 msgr2=0x7f43280aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4328162d40 0x7f43280aeb20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f432000cba0 tx=0x7f432000cf60 comp rx=0 tx=0).stop 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 -- 172.21.15.5:0/2378920830 shutdown_connections 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f430c06ac90 0x7f430c06d150 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4328162d40 0x7f43280aeb20 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432806b070 0x7f43281f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 --2- 172.21.15.5:0/2378920830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f432814b930 0x7f43280cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 -- 172.21.15.5:0/2378920830 >> 172.21.15.5:0/2378920830 conn(0x7f4328099a50 msgr2=0x7f43281633d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 -- 172.21.15.5:0/2378920830 shutdown_connections 2024-05-08T23:25:11.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:11.495+0000 7f4330129700 1 -- 172.21.15.5:0/2378920830 wait complete. 2024-05-08T23:25:11.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:25:11.509 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-05-08T23:25:11.537 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/ 2024-05-08T23:25:11.537 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4 2024-05-08T23:25:11.537 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5 2024-05-08T23:25:11.537 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6 2024-05-08T23:25:11.538 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7 2024-05-08T23:25:11.538 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-05-08T23:25:11.539 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:25:11.539 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:25:11.539 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:25:11.539 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:25:11.539 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-05-08T23:25:11.539 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:25:11.665 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-08T23:25:11.856 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:25:11.857 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:25:11.857 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:25:11.857 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:25:11.857 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-05-08T23:25:11.857 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:25:11.982 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-05-08T23:25:12.170 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:25:12.170 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:25:12.170 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:25:12.171 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:25:12.171 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-05-08T23:25:12.171 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:25:12.292 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-05-08T23:25:12.480 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:25:12.480 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:25:12.480 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:25:12.480 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:25:12.480 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-05-08T23:25:12.480 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:25:12.601 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627781 2024-05-08T23:25:12.789 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:25:12.789 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:25:12.789 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:25:12.789 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:25:12.789 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-05-08T23:25:12.789 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:25:12.910 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627782 2024-05-08T23:25:13.100 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:25:13.100 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:25:13.100 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:25:13.100 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:25:13.100 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-05-08T23:25:13.100 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:25:13.221 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627783 2024-05-08T23:25:13.409 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-05-08T23:25:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.504+0000 7f72bdc36080 1 Processor -- start 2024-05-08T23:25:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.504+0000 7f72bdc36080 1 -- start start 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72bdc36080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b426eddd0 0x558b4271a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72bdc36080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b42735f70 0x558b427bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72bdc36080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b427bc710 0x558b426eacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72bdc36080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b426931a0 con 0x558b42735f70 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72bdc36080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4262e6d0 con 0x558b427bc710 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72bdc36080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b4262e370 con 0x558b426eddd0 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b42735f70 0x558b427bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b42735f70 0x558b427bdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33412/0 (socket says 172.21.15.5:33412) 2024-05-08T23:25:13.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 -- 172.21.15.5:0/2998241928 learned_addr learned my addr 172.21.15.5:0/2998241928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b4509700 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b427bc710 0x558b426eacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3d08700 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b426eddd0 0x558b4271a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 -- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b426eddd0 msgr2=0x558b4271a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b426eddd0 0x558b4271a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 -- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b427bc710 msgr2=0x558b426eacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b427bc710 0x558b426eacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 -- 172.21.15.5:0/2998241928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a8005de0 con 0x558b42735f70 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3d08700 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b426eddd0 0x558b4271a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.505+0000 7f72b3507700 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b42735f70 0x558b427bdf80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f72a8015040 tx=0x7f72a800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed224cb7192f9746 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72b2505700 1 -- 172.21.15.5:0/2998241928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8008b60 con 0x558b42735f70 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72b2505700 1 -- 172.21.15.5:0/2998241928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a8008cc0 con 0x558b42735f70 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72b2505700 1 -- 172.21.15.5:0/2998241928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8012630 con 0x558b42735f70 2024-05-08T23:25:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 -- 172.21.15.5:0/2998241928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b42735f70 msgr2=0x558b427bdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b42735f70 0x558b427bdf80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f72a8015040 tx=0x7f72a800d000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 -- 172.21.15.5:0/2998241928 shutdown_connections 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b427bc710 0x558b426eacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b426eddd0 0x558b4271a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 --2- 172.21.15.5:0/2998241928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b42735f70 0x558b427bdf80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 -- 172.21.15.5:0/2998241928 >> 172.21.15.5:0/2998241928 conn(0x558b426ecba0 msgr2=0x558b42718a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 -- 172.21.15.5:0/2998241928 shutdown_connections 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.506+0000 7f72bdc36080 1 -- 172.21.15.5:0/2998241928 wait complete. 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.507+0000 7f72bdc36080 1 Processor -- start 2024-05-08T23:25:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.507+0000 7f72bdc36080 1 -- start start 2024-05-08T23:25:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72bdc36080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b426eddd0 0x558b427b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72bdc36080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b42735f70 0x558b4271beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72bdc36080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b427bc710 0x558b4271c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72bdc36080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b42625680 con 0x558b427bc710 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72bdc36080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b42854080 con 0x558b426eddd0 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72bdc36080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b42854360 con 0x558b42735f70 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b4509700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b427bc710 0x558b4271c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b4509700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b427bc710 0x558b4271c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33424/0 (socket says 172.21.15.5:33424) 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b4509700 1 -- 172.21.15.5:0/4252565321 learned_addr learned my addr 172.21.15.5:0/4252565321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b3507700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b42735f70 0x558b4271beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b3d08700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b426eddd0 0x558b427b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b4509700 1 -- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b42735f70 msgr2=0x558b4271beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b4509700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558b42735f70 0x558b4271beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b4509700 1 -- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b426eddd0 msgr2=0x558b427b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b4509700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558b426eddd0 0x558b427b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.508+0000 7f72b4509700 1 -- 172.21.15.5:0/4252565321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b42657a40 con 0x558b427bc710 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.509+0000 7f72b4509700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b427bc710 0x558b4271c3f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f72ac006590 tx=0x7f72ac00d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.509+0000 7f72b1503700 1 -- 172.21.15.5:0/4252565321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00dd60 con 0x558b427bc710 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.509+0000 7f72b1503700 1 -- 172.21.15.5:0/4252565321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac00dec0 con 0x558b427bc710 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.509+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558b4263d8a0 con 0x558b427bc710 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.509+0000 7f72b1503700 1 -- 172.21.15.5:0/4252565321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0136c0 con 0x558b427bc710 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.509+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b42630ed0 con 0x558b427bc710 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.510+0000 7f72b1503700 1 -- 172.21.15.5:0/4252565321 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f72ac013820 con 0x558b427bc710 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.511+0000 7f72b1503700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72a00695e0 0x7f72a006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.511+0000 7f72b1503700 1 -- 172.21.15.5:0/4252565321 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72ac087da0 con 0x558b427bc710 2024-05-08T23:25:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.511+0000 7f72b3d08700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72a00695e0 0x7f72a006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.511+0000 7f72bdc36080 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x558b426ed1f0 0x558b427b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.511+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 8.12 8:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) v8 -- 0x558b42853c10 con 0x558b426ed1f0 2024-05-08T23:25:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.512+0000 7f72b3507700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x558b426ed1f0 0x558b427b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.512+0000 7f72b3d08700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72a00695e0 0x7f72a006baa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009840 tx=0x7f72a400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.512+0000 7f72b3507700 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x558b426ed1f0 0x558b427b79c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.513+0000 7f72b3507700 1 -- 172.21.15.5:0/4252565321 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v60'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f72a8019040 con 0x558b426ed1f0 2024-05-08T23:25:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x558b426ed1f0 msgr2=0x558b427b79c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x558b426ed1f0 0x558b427b79c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72a00695e0 msgr2=0x7f72a006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72a00695e0 0x7f72a006baa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009840 tx=0x7f72a400c040 comp rx=0 tx=0).stop 2024-05-08T23:25:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b427bc710 msgr2=0x558b4271c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 --2- 172.21.15.5:0/4252565321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b427bc710 0x558b4271c3f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f72ac006590 tx=0x7f72ac00d670 comp rx=0 tx=0).stop 2024-05-08T23:25:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72b4509700 1 -- 172.21.15.5:0/4252565321 reap_dead start 2024-05-08T23:25:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 shutdown_connections 2024-05-08T23:25:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 >> 172.21.15.5:0/4252565321 conn(0x558b426ecba0 msgr2=0x558b42717370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 shutdown_connections 2024-05-08T23:25:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.514+0000 7f72bdc36080 1 -- 172.21.15.5:0/4252565321 wait complete. 2024-05-08T23:25:13.519 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-05-08T23:25:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.571+0000 7f231efdf080 1 Processor -- start 2024-05-08T23:25:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.571+0000 7f231efdf080 1 -- start start 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f231efdf080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f231efdf080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4e14c0 0x55fd7e40b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f231efdf080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dff80 0x55fd7e40bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f231efdf080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7e3b58b0 con 0x55fd7e4dfba0 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f231efdf080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7e3b11a0 con 0x55fd7e4dff80 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f231efdf080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd7e34c6d0 con 0x55fd7e4e14c0 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23150b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23150b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4e1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33454/0 (socket says 172.21.15.5:33454) 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23150b1700 1 -- 172.21.15.5:0/2398883747 learned_addr learned my addr 172.21.15.5:0/2398883747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23148b0700 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4e14c0 0x55fd7e40b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23158b2700 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dff80 0x55fd7e40bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23150b1700 1 -- 172.21.15.5:0/2398883747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4e14c0 msgr2=0x55fd7e40b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23150b1700 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4e14c0 0x55fd7e40b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23150b1700 1 -- 172.21.15.5:0/2398883747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dff80 msgr2=0x55fd7e40bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23150b1700 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dff80 0x55fd7e40bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.572+0000 7f23150b1700 1 -- 172.21.15.5:0/2398883747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd7e34eed0 con 0x55fd7e4dfba0 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f23150b1700 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4e1c80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2304004090 tx=0x7f2300010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1841fcd6c7a1d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f230f7fe700 1 -- 172.21.15.5:0/2398883747 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230000f070 con 0x55fd7e4dfba0 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f230f7fe700 1 -- 172.21.15.5:0/2398883747 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300013790 con 0x55fd7e4dfba0 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f230f7fe700 1 -- 172.21.15.5:0/2398883747 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300013b40 con 0x55fd7e4dfba0 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 -- 172.21.15.5:0/2398883747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 msgr2=0x55fd7e4e1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4e1c80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2304004090 tx=0x7f2300010000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 -- 172.21.15.5:0/2398883747 shutdown_connections 2024-05-08T23:25:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dff80 0x55fd7e40bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4e1c80 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 --2- 172.21.15.5:0/2398883747 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4e14c0 0x55fd7e40b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 -- 172.21.15.5:0/2398883747 >> 172.21.15.5:0/2398883747 conn(0x55fd7e409ac0 msgr2=0x55fd7e4e27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 -- 172.21.15.5:0/2398883747 shutdown_connections 2024-05-08T23:25:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.573+0000 7f231efdf080 1 -- 172.21.15.5:0/2398883747 wait complete. 2024-05-08T23:25:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.574+0000 7f231efdf080 1 Processor -- start 2024-05-08T23:25:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.574+0000 7f231efdf080 1 -- start start 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f231efdf080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4dd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f231efdf080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4dff80 0x55fd7e4e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f231efdf080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dedd0 0x55fd7e4e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f231efdf080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7e34c370 con 0x55fd7e4dfba0 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f231efdf080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7e343680 con 0x55fd7e4dedd0 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f231efdf080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd7e436930 con 0x55fd7e4dff80 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23150b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4dd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23150b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4dd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33466/0 (socket says 172.21.15.5:33466) 2024-05-08T23:25:13.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23150b1700 1 -- 172.21.15.5:0/1495955601 learned_addr learned my addr 172.21.15.5:0/1495955601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23158b2700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dedd0 0x55fd7e4e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23148b0700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4dff80 0x55fd7e4e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23150b1700 1 -- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4dff80 msgr2=0x55fd7e4e09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23150b1700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fd7e4dff80 0x55fd7e4e09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23150b1700 1 -- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dedd0 msgr2=0x55fd7e4e4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23150b1700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dedd0 0x55fd7e4e4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23150b1700 1 -- 172.21.15.5:0/1495955601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2304001e10 con 0x55fd7e4dfba0 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.575+0000 7f23158b2700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd7e4dedd0 0x55fd7e4e4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.576+0000 7f23150b1700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4dd980 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2300009e30 tx=0x7f2300005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.576+0000 7f230e7fc700 1 -- 172.21.15.5:0/1495955601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230000f070 con 0x55fd7e4dfba0 2024-05-08T23:25:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.576+0000 7f230e7fc700 1 -- 172.21.15.5:0/1495955601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300026670 con 0x55fd7e4dfba0 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.576+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fd7e34e930 con 0x55fd7e4dfba0 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.576+0000 7f230e7fc700 1 -- 172.21.15.5:0/1495955601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300026a00 con 0x55fd7e4dfba0 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.576+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd7e435920 con 0x55fd7e4dfba0 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.577+0000 7f230e7fc700 1 -- 172.21.15.5:0/1495955601 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2300005040 con 0x55fd7e4dfba0 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.578+0000 7f230e7fc700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f22f80695e0 0x7f22f806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.578+0000 7f230e7fc700 1 -- 172.21.15.5:0/1495955601 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2300000e40 con 0x55fd7e4dfba0 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.578+0000 7f23148b0700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f22f80695e0 0x7f22f806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.578+0000 7f231efdf080 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55fd7e4c4480 0x55fd7e4c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.578+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 8.12 8:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) v8 -- 0x55fd7e4c4da0 con 0x55fd7e4c4480 2024-05-08T23:25:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.578+0000 7f23158b2700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55fd7e4c4480 0x55fd7e4c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.579+0000 7f23148b0700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f22f80695e0 0x7f22f806baa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55fd7e450e70 tx=0x7f2308002c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.579+0000 7f23158b2700 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55fd7e4c4480 0x55fd7e4c4860 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f23158b2700 1 -- 172.21.15.5:0/1495955601 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v60'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2310006140 con 0x55fd7e4c4480 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55fd7e4c4480 msgr2=0x55fd7e4c4860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55fd7e4c4480 0x55fd7e4c4860 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f22f80695e0 msgr2=0x7f22f806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f22f80695e0 0x7f22f806baa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55fd7e450e70 tx=0x7f2308002c60 comp rx=0 tx=0).stop 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 msgr2=0x55fd7e4dd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 --2- 172.21.15.5:0/1495955601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd7e4dfba0 0x55fd7e4dd980 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2300009e30 tx=0x7f2300005a00 comp rx=0 tx=0).stop 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f23158b2700 1 -- 172.21.15.5:0/1495955601 reap_dead start 2024-05-08T23:25:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 shutdown_connections 2024-05-08T23:25:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 >> 172.21.15.5:0/1495955601 conn(0x55fd7e409ac0 msgr2=0x55fd7e4e27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 shutdown_connections 2024-05-08T23:25:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.580+0000 7f231efdf080 1 -- 172.21.15.5:0/1495955601 wait complete. 2024-05-08T23:25:13.585 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-05-08T23:25:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.637+0000 7f6e87906080 1 Processor -- start 2024-05-08T23:25:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.637+0000 7f6e87906080 1 -- start start 2024-05-08T23:25:13.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.637+0000 7f6e87906080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f450f6f0 0x55b8f44900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.637+0000 7f6e87906080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f4490620 0x55b8f451ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e87906080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f451b340 0x55b8f4513b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e87906080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f43ef1a0 con 0x55b8f451b340 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e87906080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f438a6d0 con 0x55b8f4490620 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e87906080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8f438a370 con 0x55b8f450f6f0 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f451b340 0x55b8f4513b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f451b340 0x55b8f4513b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33476/0 (socket says 172.21.15.5:33476) 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 -- 172.21.15.5:0/3042188634 learned_addr learned my addr 172.21.15.5:0/3042188634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e751d7700 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f4490620 0x55b8f451ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7d9d8700 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f450f6f0 0x55b8f44900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 -- 172.21.15.5:0/3042188634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f450f6f0 msgr2=0x55b8f44900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f450f6f0 0x55b8f44900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 -- 172.21.15.5:0/3042188634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f4490620 msgr2=0x55b8f451ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f4490620 0x55b8f451ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 -- 172.21.15.5:0/3042188634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8f43998a0 con 0x55b8f451b340 2024-05-08T23:25:13.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7e1d9700 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f451b340 0x55b8f4513b10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6e78007010 tx=0x7f6e78010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3934fa23a5517863 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7c9d6700 1 -- 172.21.15.5:0/3042188634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78012450 con 0x55b8f451b340 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7c9d6700 1 -- 172.21.15.5:0/3042188634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e78005a80 con 0x55b8f451b340 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.638+0000 7f6e7c9d6700 1 -- 172.21.15.5:0/3042188634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78012940 con 0x55b8f451b340 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 -- 172.21.15.5:0/3042188634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f451b340 msgr2=0x55b8f4513b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f451b340 0x55b8f4513b10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6e78007010 tx=0x7f6e78010000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 -- 172.21.15.5:0/3042188634 shutdown_connections 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f451b340 0x55b8f4513b10 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f450f6f0 0x55b8f44900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 --2- 172.21.15.5:0/3042188634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f4490620 0x55b8f451ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 -- 172.21.15.5:0/3042188634 >> 172.21.15.5:0/3042188634 conn(0x55b8f4448ba0 msgr2=0x55b8f45192b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 -- 172.21.15.5:0/3042188634 shutdown_connections 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.639+0000 7f6e87906080 1 -- 172.21.15.5:0/3042188634 wait complete. 2024-05-08T23:25:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.640+0000 7f6e87906080 1 Processor -- start 2024-05-08T23:25:13.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.640+0000 7f6e87906080 1 -- start start 2024-05-08T23:25:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.640+0000 7f6e87906080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f4490620 0x55b8f4447860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.640+0000 7f6e87906080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f450f6f0 0x55b8f4511c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.640+0000 7f6e87906080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f451b340 0x55b8f450c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.640+0000 7f6e87906080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f4381680 con 0x55b8f450f6f0 2024-05-08T23:25:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.640+0000 7f6e87906080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f45b46d0 con 0x55b8f451b340 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.640+0000 7f6e87906080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8f450c720 con 0x55b8f4490620 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f450f6f0 0x55b8f4511c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f450f6f0 0x55b8f4511c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33488/0 (socket says 172.21.15.5:33488) 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 -- 172.21.15.5:0/1465444481 learned_addr learned my addr 172.21.15.5:0/1465444481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e7e1d9700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f451b340 0x55b8f450c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e7d9d8700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f4490620 0x55b8f4447860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 -- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f4490620 msgr2=0x55b8f4447860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8f4490620 0x55b8f4447860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 -- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f451b340 msgr2=0x55b8f450c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8f451b340 0x55b8f450c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 -- 172.21.15.5:0/1465444481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7800a100 con 0x55b8f450f6f0 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e751d7700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f450f6f0 0x55b8f4511c10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009f20 tx=0x7f6e6800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e777fe700 1 -- 172.21.15.5:0/1465444481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68014070 con 0x55b8f450f6f0 2024-05-08T23:25:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e777fe700 1 -- 172.21.15.5:0/1465444481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6800e720 con 0x55b8f450f6f0 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b8f438ced0 con 0x55b8f450f6f0 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e777fe700 1 -- 172.21.15.5:0/1465444481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800ead0 con 0x55b8f450f6f0 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.641+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8f438c930 con 0x55b8f450f6f0 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.643+0000 7f6e777fe700 1 -- 172.21.15.5:0/1465444481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6e6800fd50 con 0x55b8f450f6f0 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.643+0000 7f6e777fe700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e600695e0 0x7f6e6006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.644+0000 7f6e777fe700 1 -- 172.21.15.5:0/1465444481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e68004040 con 0x55b8f450f6f0 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.644+0000 7f6e7d9d8700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e600695e0 0x7f6e6006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.644+0000 7f6e87906080 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55b8f45b4b00 0x55b8f45b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.644+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 8.5 8:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) v8 -- 0x55b8f45b5470 con 0x55b8f45b4b00 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.644+0000 7f6e7e1d9700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55b8f45b4b00 0x55b8f45b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.644+0000 7f6e7d9d8700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e600695e0 0x7f6e6006baa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55b8f44905f0 tx=0x7f6e7000c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.644+0000 7f6e7e1d9700 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55b8f45b4b00 0x55b8f45b4f30 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e7e1d9700 1 -- 172.21.15.5:0/1465444481 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v60'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6e780052c0 con 0x55b8f45b4b00 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55b8f45b4b00 msgr2=0x55b8f45b4f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55b8f45b4b00 0x55b8f45b4f30 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e600695e0 msgr2=0x7f6e6006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6e600695e0 0x7f6e6006baa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55b8f44905f0 tx=0x7f6e7000c040 comp rx=0 tx=0).stop 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f450f6f0 msgr2=0x55b8f4511c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 --2- 172.21.15.5:0/1465444481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8f450f6f0 0x55b8f4511c10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009f20 tx=0x7f6e6800f730 comp rx=0 tx=0).stop 2024-05-08T23:25:13.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e7e1d9700 1 -- 172.21.15.5:0/1465444481 reap_dead start 2024-05-08T23:25:13.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 shutdown_connections 2024-05-08T23:25:13.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 >> 172.21.15.5:0/1465444481 conn(0x55b8f4448ba0 msgr2=0x55b8f4446d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 shutdown_connections 2024-05-08T23:25:13.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.646+0000 7f6e87906080 1 -- 172.21.15.5:0/1465444481 wait complete. 2024-05-08T23:25:13.652 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-05-08T23:25:13.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f21a2595080 1 Processor -- start 2024-05-08T23:25:13.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f21a2595080 1 -- start start 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f21a2595080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27a6bfa0 0x557a27a68cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f21a2595080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27ab4dc0 0x557a27b3c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f21a2595080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 0x557a27a96980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f21a2595080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a27a111a0 con 0x557a27ab4dc0 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f21a2595080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a279ac6d0 con 0x557a27b3ee10 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f21a2595080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a279ac370 con 0x557a27a6bfa0 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.702+0000 7f2197e66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27ab4dc0 0x557a27b3c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2197e66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27ab4dc0 0x557a27b3c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33512/0 (socket says 172.21.15.5:33512) 2024-05-08T23:25:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2197e66700 1 -- 172.21.15.5:0/25399749 learned_addr learned my addr 172.21.15.5:0/25399749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2198667700 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27a6bfa0 0x557a27a68cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2198e68700 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 0x557a27a96980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2197e66700 1 -- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27a6bfa0 msgr2=0x557a27a68cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2197e66700 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27a6bfa0 0x557a27a68cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2197e66700 1 -- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 msgr2=0x557a27a96980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2197e66700 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 0x557a27a96980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2197e66700 1 -- 172.21.15.5:0/25399749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219000c3f0 con 0x557a27ab4dc0 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2198e68700 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 0x557a27a96980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2197e66700 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27ab4dc0 0x557a27b3c190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f218c0098b0 tx=0x7f218c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaca6c901448706 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2196e64700 1 -- 172.21.15.5:0/25399749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c00f040 con 0x557a27ab4dc0 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2196e64700 1 -- 172.21.15.5:0/25399749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c0089b0 con 0x557a27ab4dc0 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.703+0000 7f2196e64700 1 -- 172.21.15.5:0/25399749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c008d60 con 0x557a27ab4dc0 2024-05-08T23:25:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 -- 172.21.15.5:0/25399749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27ab4dc0 msgr2=0x557a27b3c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27ab4dc0 0x557a27b3c190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f218c0098b0 tx=0x7f218c005a00 comp rx=0 tx=0).stop 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 -- 172.21.15.5:0/25399749 shutdown_connections 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 0x557a27a96980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27a6bfa0 0x557a27a68cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 --2- 172.21.15.5:0/25399749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27ab4dc0 0x557a27b3c190 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 -- 172.21.15.5:0/25399749 >> 172.21.15.5:0/25399749 conn(0x557a27a6aba0 msgr2=0x557a27ab3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 -- 172.21.15.5:0/25399749 shutdown_connections 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.704+0000 7f21a2595080 1 -- 172.21.15.5:0/25399749 wait complete. 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.705+0000 7f21a2595080 1 Processor -- start 2024-05-08T23:25:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.705+0000 7f21a2595080 1 -- start start 2024-05-08T23:25:13.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f21a2595080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27a6bfa0 0x557a27b2fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f21a2595080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27ab4dc0 0x557a27a9a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f21a2595080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 0x557a27b2adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f21a2595080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a279a3680 con 0x557a27a6bfa0 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f21a2595080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a27b30e40 con 0x557a27b3ee10 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f21a2595080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a27b30230 con 0x557a27ab4dc0 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198667700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27a6bfa0 0x557a27b2fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198667700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27a6bfa0 0x557a27b2fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33522/0 (socket says 172.21.15.5:33522) 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198667700 1 -- 172.21.15.5:0/4184035736 learned_addr learned my addr 172.21.15.5:0/4184035736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2197e66700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27ab4dc0 0x557a27a9a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198667700 1 -- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27ab4dc0 msgr2=0x557a27a9a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198e68700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 0x557a27b2adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198667700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557a27ab4dc0 0x557a27a9a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198667700 1 -- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 msgr2=0x557a27b2adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198667700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a27b3ee10 0x557a27b2adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.706+0000 7f2198667700 1 -- 172.21.15.5:0/4184035736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c009510 con 0x557a27a6bfa0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.707+0000 7f2198667700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27a6bfa0 0x557a27b2fb70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2188002c70 tx=0x7f218800c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.707+0000 7f2195e62700 1 -- 172.21.15.5:0/4184035736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188002b10 con 0x557a27a6bfa0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.707+0000 7f2195e62700 1 -- 172.21.15.5:0/4184035736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2188008030 con 0x557a27a6bfa0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.707+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557a279bb8a0 con 0x557a27a6bfa0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.707+0000 7f2195e62700 1 -- 172.21.15.5:0/4184035736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188002e70 con 0x557a27a6bfa0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.707+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a279aeed0 con 0x557a27a6bfa0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.708+0000 7f2195e62700 1 -- 172.21.15.5:0/4184035736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2188011450 con 0x557a27a6bfa0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.709+0000 7f2195e62700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f21840695e0 0x7f218406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.709+0000 7f2195e62700 1 -- 172.21.15.5:0/4184035736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2188088040 con 0x557a27a6bfa0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.709+0000 7f21a2595080 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x557a27bd4450 0x557a27bd4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.709+0000 7f2197e66700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f21840695e0 0x7f218406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.709+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 8.9 8:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) v8 -- 0x557a27bd4dd0 con 0x557a27bd4450 2024-05-08T23:25:13.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.709+0000 7f2198e68700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x557a27bd4450 0x557a27bd4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.709+0000 7f2197e66700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f21840695e0 0x7f218406baa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f218c009fd0 tx=0x7f218c012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.710+0000 7f2198e68700 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x557a27bd4450 0x557a27bd4890 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.711+0000 7f2198e68700 1 -- 172.21.15.5:0/4184035736 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v60'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f219000e720 con 0x557a27bd4450 2024-05-08T23:25:13.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x557a27bd4450 msgr2=0x557a27bd4890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x557a27bd4450 0x557a27bd4890 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f21840695e0 msgr2=0x7f218406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f21840695e0 0x7f218406baa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f218c009fd0 tx=0x7f218c012000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27a6bfa0 msgr2=0x557a27b2fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 --2- 172.21.15.5:0/4184035736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a27a6bfa0 0x557a27b2fb70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2188002c70 tx=0x7f218800c530 comp rx=0 tx=0).stop 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f2198e68700 1 -- 172.21.15.5:0/4184035736 reap_dead start 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 shutdown_connections 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 >> 172.21.15.5:0/4184035736 conn(0x557a27a6aba0 msgr2=0x557a27b3b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 shutdown_connections 2024-05-08T23:25:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.712+0000 7f21a2595080 1 -- 172.21.15.5:0/4184035736 wait complete. 2024-05-08T23:25:13.716 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-05-08T23:25:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.767+0000 7f99edd2f080 1 Processor -- start 2024-05-08T23:25:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.767+0000 7f99edd2f080 1 -- start start 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99edd2f080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99d0002290 0x563116f13e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99edd2f080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563116f20c60 0x563116f1cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99edd2f080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 0x563116f36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99edd2f080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563116e9d8b0 con 0x7f99d0002290 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99edd2f080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563116e991a0 con 0x563116f1d090 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99edd2f080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563116e346d0 con 0x563116f20c60 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99d0002290 0x563116f13e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99d0002290 0x563116f13e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33542/0 (socket says 172.21.15.5:33542) 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 -- 172.21.15.5:0/2057397687 learned_addr learned my addr 172.21.15.5:0/2057397687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 -- 172.21.15.5:0/2057397687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563116f20c60 msgr2=0x563116f1cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99db600700 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563116f20c60 0x563116f1cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e4602700 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 0x563116f36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563116f20c60 0x563116f1cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 -- 172.21.15.5:0/2057397687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 msgr2=0x563116f36190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 0x563116f36190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 -- 172.21.15.5:0/2057397687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563116e36ed0 con 0x7f99d0002290 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e3e01700 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99d0002290 0x563116f13e30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f99d400ba60 tx=0x7f99d4009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774d6e309bc66e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e2dff700 1 -- 172.21.15.5:0/2057397687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d401f070 con 0x7f99d0002290 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e2dff700 1 -- 172.21.15.5:0/2057397687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d400bcf0 con 0x7f99d0002290 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.768+0000 7f99e2dff700 1 -- 172.21.15.5:0/2057397687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4018620 con 0x7f99d0002290 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 -- 172.21.15.5:0/2057397687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99d0002290 msgr2=0x563116f13e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99d0002290 0x563116f13e30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f99d400ba60 tx=0x7f99d4009000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 -- 172.21.15.5:0/2057397687 shutdown_connections 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 0x563116f36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99d0002290 0x563116f13e30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 --2- 172.21.15.5:0/2057397687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563116f20c60 0x563116f1cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 -- 172.21.15.5:0/2057397687 >> 172.21.15.5:0/2057397687 conn(0x563116ef1ac0 msgr2=0x563116f1d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 -- 172.21.15.5:0/2057397687 shutdown_connections 2024-05-08T23:25:13.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.769+0000 7f99edd2f080 1 -- 172.21.15.5:0/2057397687 wait complete. 2024-05-08T23:25:13.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.770+0000 7f99edd2f080 1 Processor -- start 2024-05-08T23:25:13.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.770+0000 7f99edd2f080 1 -- start start 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99edd2f080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 0x563116fada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99edd2f080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563116f20c60 0x563116f14460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99edd2f080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99d0002290 0x563116facbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99edd2f080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563116e34370 con 0x563116f20c60 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99edd2f080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563116e2b680 con 0x563116f1d090 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99edd2f080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563116fadfb0 con 0x7f99d0002290 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563116f20c60 0x563116f14460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563116f20c60 0x563116f14460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33546/0 (socket says 172.21.15.5:33546) 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 -- 172.21.15.5:0/3789392513 learned_addr learned my addr 172.21.15.5:0/3789392513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99e4602700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99d0002290 0x563116facbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99e3e01700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 0x563116fada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 -- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99d0002290 msgr2=0x563116facbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99d0002290 0x563116facbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 -- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 msgr2=0x563116fada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563116f1d090 0x563116fada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 -- 172.21.15.5:0/3789392513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0001e10 con 0x563116f20c60 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99e4602700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99d0002290 0x563116facbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99db600700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563116f20c60 0x563116f14460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009e20 tx=0x7f99cc0086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99e1dfd700 1 -- 172.21.15.5:0/3789392513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00f070 con 0x563116f20c60 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99e1dfd700 1 -- 172.21.15.5:0/3789392513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc010040 con 0x563116f20c60 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563116e36930 con 0x563116f20c60 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99e1dfd700 1 -- 172.21.15.5:0/3789392513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc016880 con 0x563116f20c60 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.771+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563116f395f0 con 0x563116f20c60 2024-05-08T23:25:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.773+0000 7f99e1dfd700 1 -- 172.21.15.5:0/3789392513 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f99cc0169e0 con 0x563116f20c60 2024-05-08T23:25:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.773+0000 7f99e1dfd700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f99c40695e0 0x7f99c406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.773+0000 7f99e1dfd700 1 -- 172.21.15.5:0/3789392513 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99cc0886f0 con 0x563116f20c60 2024-05-08T23:25:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.773+0000 7f99edd2f080 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x563116f149a0 0x563116fad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.773+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 8.5 8:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) v8 -- 0x563116f16210 con 0x563116f149a0 2024-05-08T23:25:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.774+0000 7f99e3e01700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f99c40695e0 0x7f99c406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.774+0000 7f99e4602700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x563116f149a0 0x563116fad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.774+0000 7f99e3e01700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f99c40695e0 0x7f99c406baa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f99d400bc60 tx=0x7f99d4012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.774+0000 7f99e4602700 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x563116f149a0 0x563116fad0f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99e4602700 1 -- 172.21.15.5:0/3789392513 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v60'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f99dc006050 con 0x563116f149a0 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x563116f149a0 msgr2=0x563116fad0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x563116f149a0 0x563116fad0f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f99c40695e0 msgr2=0x7f99c406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f99c40695e0 0x7f99c406baa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f99d400bc60 tx=0x7f99d4012000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563116f20c60 msgr2=0x563116f14460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 --2- 172.21.15.5:0/3789392513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563116f20c60 0x563116f14460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009e20 tx=0x7f99cc0086e0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99e4602700 1 -- 172.21.15.5:0/3789392513 reap_dead start 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 shutdown_connections 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 >> 172.21.15.5:0/3789392513 conn(0x563116ef1ac0 msgr2=0x563116f387a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 shutdown_connections 2024-05-08T23:25:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.776+0000 7f99edd2f080 1 -- 172.21.15.5:0/3789392513 wait complete. 2024-05-08T23:25:13.780 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-05-08T23:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.830+0000 7f1875467080 1 Processor -- start 2024-05-08T23:25:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.830+0000 7f1875467080 1 -- start start 2024-05-08T23:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.830+0000 7f1875467080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54cedfa0 0x555c54d18af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f1875467080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 0x555c54d1bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f1875467080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54dc0e10 0x555c54d1bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f1875467080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c54c931a0 con 0x555c54dc0e10 2024-05-08T23:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f1875467080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c54c2e6d0 con 0x555c54cedfa0 2024-05-08T23:25:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f1875467080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c54c2e370 con 0x555c54d36dc0 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54dc0e10 0x555c54d1bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54dc0e10 0x555c54d1bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33570/0 (socket says 172.21.15.5:33570) 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 -- 172.21.15.5:0/1355661828 learned_addr learned my addr 172.21.15.5:0/1355661828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186b539700 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54cedfa0 0x555c54d18af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186ad38700 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 0x555c54d1bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 -- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 msgr2=0x555c54d1bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 0x555c54d1bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 -- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54cedfa0 msgr2=0x555c54d18af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54cedfa0 0x555c54d18af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 -- 172.21.15.5:0/1355661828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860005d20 con 0x555c54dc0e10 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186b539700 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54cedfa0 0x555c54d18af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f186bd3a700 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54dc0e10 0x555c54d1bff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f186400c480 tx=0x7f186400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb2a5934a6148d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f1869d36700 1 -- 172.21.15.5:0/1355661828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640115f0 con 0x555c54dc0e10 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f1869d36700 1 -- 172.21.15.5:0/1355661828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864011750 con 0x555c54dc0e10 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.831+0000 7f1869d36700 1 -- 172.21.15.5:0/1355661828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864011b00 con 0x555c54dc0e10 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 -- 172.21.15.5:0/1355661828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54dc0e10 msgr2=0x555c54d1bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54dc0e10 0x555c54d1bff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f186400c480 tx=0x7f186400c790 comp rx=0 tx=0).stop 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 -- 172.21.15.5:0/1355661828 shutdown_connections 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54dc0e10 0x555c54d1bff0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54cedfa0 0x555c54d18af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 --2- 172.21.15.5:0/1355661828 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 0x555c54d1bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 -- 172.21.15.5:0/1355661828 >> 172.21.15.5:0/1355661828 conn(0x555c54cecba0 msgr2=0x555c54dbfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 -- 172.21.15.5:0/1355661828 shutdown_connections 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.832+0000 7f1875467080 1 -- 172.21.15.5:0/1355661828 wait complete. 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.833+0000 7f1875467080 1 Processor -- start 2024-05-08T23:25:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.833+0000 7f1875467080 1 -- start start 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f1875467080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54cedfa0 0x555c54d1bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f1875467080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 0x555c54d1bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f1875467080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54dc0e10 0x555c54dc1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f1875467080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c54c25680 con 0x555c54cedfa0 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f1875467080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c54da1780 con 0x555c54dc0e10 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f1875467080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c54dadf00 con 0x555c54d36dc0 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186b539700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54cedfa0 0x555c54d1bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186b539700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54cedfa0 0x555c54d1bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33580/0 (socket says 172.21.15.5:33580) 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186b539700 1 -- 172.21.15.5:0/1621292617 learned_addr learned my addr 172.21.15.5:0/1621292617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186bd3a700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54dc0e10 0x555c54dc1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186ad38700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 0x555c54d1bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186b539700 1 -- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 msgr2=0x555c54d1bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186b539700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555c54d36dc0 0x555c54d1bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186b539700 1 -- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54dc0e10 msgr2=0x555c54dc1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186b539700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c54dc0e10 0x555c54dc1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.834+0000 7f186b539700 1 -- 172.21.15.5:0/1621292617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186400d040 con 0x555c54cedfa0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.835+0000 7f186b539700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54cedfa0 0x555c54d1bab0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f185c00cef0 tx=0x7f185c00a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.835+0000 7f1868d34700 1 -- 172.21.15.5:0/1621292617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00ac20 con 0x555c54cedfa0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.835+0000 7f1868d34700 1 -- 172.21.15.5:0/1621292617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c00ad80 con 0x555c54cedfa0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.835+0000 7f1868d34700 1 -- 172.21.15.5:0/1621292617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00f600 con 0x555c54cedfa0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.835+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555c54c3d8a0 con 0x555c54cedfa0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.835+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c54c30ed0 con 0x555c54cedfa0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.836+0000 7f1868d34700 1 -- 172.21.15.5:0/1621292617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f185c00d040 con 0x555c54cedfa0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.837+0000 7f1868d34700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1858069610 0x7f185806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.837+0000 7f1868d34700 1 -- 172.21.15.5:0/1621292617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f185c087910 con 0x555c54cedfa0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.837+0000 7f1875467080 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x555c54db7480 0x555c54e643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.837+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- osd_op(unknown.0.0:1 8.1d 8:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) v8 -- 0x555c54dc4af0 con 0x555c54db7480 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.837+0000 7f186ad38700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1858069610 0x7f185806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.837+0000 7f186bd3a700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x555c54db7480 0x555c54e643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.838+0000 7f186bd3a700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x555c54db7480 0x555c54e643a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.838+0000 7f186ad38700 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1858069610 0x7f185806bad0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x555c54e4ee70 tx=0x7f186000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f186bd3a700 1 -- 172.21.15.5:0/1621292617 <== osd.0 v2:172.21.15.5:6824/47134 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v60'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f18640119c0 con 0x555c54db7480 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x555c54db7480 msgr2=0x555c54e643a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x555c54db7480 0x555c54e643a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1858069610 msgr2=0x7f185806bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1858069610 0x7f185806bad0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x555c54e4ee70 tx=0x7f186000f000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54cedfa0 msgr2=0x555c54d1bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 --2- 172.21.15.5:0/1621292617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c54cedfa0 0x555c54d1bab0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f185c00cef0 tx=0x7f185c00a4e0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f186bd3a700 1 -- 172.21.15.5:0/1621292617 reap_dead start 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 shutdown_connections 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 >> 172.21.15.5:0/1621292617 conn(0x555c54cecba0 msgr2=0x555c54dbfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 shutdown_connections 2024-05-08T23:25:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.839+0000 7f1875467080 1 -- 172.21.15.5:0/1621292617 wait complete. 2024-05-08T23:25:13.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /d1/d2/d3 recursive,force 2024-05-08T23:25:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.987+0000 7fd675796700 1 Processor -- start 2024-05-08T23:25:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.987+0000 7fd675796700 1 -- start start 2024-05-08T23:25:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd675796700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd670160f10 0x7fd6700cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd675796700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6700d04e0 0x7fd6700cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd675796700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6700b01c0 con 0x7fd6700d0d80 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd675796700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6700d0960 con 0x7fd6700d04e0 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd675796700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6700d1ba0 con 0x7fd670160f10 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6700d04e0 0x7fd6700cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66e7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd6700d0d80 0x7fd6700b0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38594/0 (socket says 172.21.15.5:38594) 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66e7fc700 1 -- 172.21.15.5:0/2862437780 learned_addr learned my addr 172.21.15.5:0/2862437780 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66effd700 1 --2- 172.21.15.5:0/2862437780 >> v2:172.21.15.49:3300/0 conn(0x7fd670160f10 0x7fd6700cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66f7fe700 1 -- 172.21.15.5:0/2862437780 >> v2:172.21.15.49:3300/0 conn(0x7fd670160f10 msgr2=0x7fd6700cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/2862437780 >> v2:172.21.15.49:3300/0 conn(0x7fd670160f10 0x7fd6700cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66f7fe700 1 -- 172.21.15.5:0/2862437780 >> v1:172.21.15.5:6789/0 conn(0x7fd6700d0d80 legacy=0x7fd6700b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66f7fe700 1 -- 172.21.15.5:0/2862437780 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6700d1e80 con 0x7fd6700d04e0 2024-05-08T23:25:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.988+0000 7fd66effd700 1 --2- 172.21.15.5:0/2862437780 >> v2:172.21.15.49:3300/0 conn(0x7fd670160f10 0x7fd6700cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/2862437780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6700cf210 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd67008d7c0 tx=0x7fd66800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16387ec8553597e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd66dffb700 1 -- 172.21.15.5:0/2862437780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800e070 con 0x7fd6700d04e0 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd66dffb700 1 -- 172.21.15.5:0/2862437780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66800bbd0 con 0x7fd6700d04e0 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd66dffb700 1 -- 172.21.15.5:0/2862437780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668012430 con 0x7fd6700d04e0 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd675796700 1 -- 172.21.15.5:0/2862437780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 msgr2=0x7fd6700cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd675796700 1 --2- 172.21.15.5:0/2862437780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6700cf210 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd67008d7c0 tx=0x7fd66800b330 comp rx=0 tx=0).stop 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd675796700 1 -- 172.21.15.5:0/2862437780 shutdown_connections 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd675796700 1 --2- 172.21.15.5:0/2862437780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6700cf210 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd675796700 1 --2- 172.21.15.5:0/2862437780 >> v2:172.21.15.49:3300/0 conn(0x7fd670160f10 0x7fd6700cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd675796700 1 -- 172.21.15.5:0/2862437780 >> 172.21.15.5:0/2862437780 conn(0x7fd67008afe0 msgr2=0x7fd6700d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd675796700 1 -- 172.21.15.5:0/2862437780 shutdown_connections 2024-05-08T23:25:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.989+0000 7fd675796700 1 -- 172.21.15.5:0/2862437780 wait complete. 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.991+0000 7fd675796700 1 Processor -- start 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.991+0000 7fd675796700 1 -- start start 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd675796700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6701f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd675796700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0d80 0x7fd67014cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd675796700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd670160f10 0x7fd67006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd675796700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6701f7720 con 0x7fd6700d04e0 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd675796700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700c9490 con 0x7fd670160f10 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd675796700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd670152ef0 con 0x7fd6700d0d80 2024-05-08T23:25:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6701f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6701f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33606/0 (socket says 172.21.15.5:33606) 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 -- 172.21.15.5:0/1092908107 learned_addr learned my addr 172.21.15.5:0/1092908107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0d80 0x7fd67014cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd670160f10 0x7fd67006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 -- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0d80 msgr2=0x7fd67014cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0d80 0x7fd67014cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 -- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd670160f10 msgr2=0x7fd67006c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd670160f10 0x7fd67006c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 -- 172.21.15.5:0/1092908107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668009a70 con 0x7fd6700d04e0 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd670160f10 0x7fd67006c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd66effd700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6701f71e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd66000e810 tx=0x7fd66000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd6677fe700 1 -- 172.21.15.5:0/1092908107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000c020 con 0x7fd6700d04e0 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd6677fe700 1 -- 172.21.15.5:0/1092908107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660004250 con 0x7fd6700d04e0 2024-05-08T23:25:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd675796700 1 -- 172.21.15.5:0/1092908107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd670161470 con 0x7fd6700d04e0 2024-05-08T23:25:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.992+0000 7fd6677fe700 1 -- 172.21.15.5:0/1092908107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600036a0 con 0x7fd6700d04e0 2024-05-08T23:25:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.993+0000 7fd675796700 1 -- 172.21.15.5:0/1092908107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6701534b0 con 0x7fd6700d04e0 2024-05-08T23:25:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.994+0000 7fd6677fe700 1 -- 172.21.15.5:0/1092908107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd660003800 con 0x7fd6700d04e0 2024-05-08T23:25:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.995+0000 7fd6677fe700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.995+0000 7fd6677fe700 1 -- 172.21.15.5:0/1092908107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd660088ce0 con 0x7fd6700d04e0 2024-05-08T23:25:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.995+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.996+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd6700b1b90 tx=0x7fd658009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.997+0000 7fd681950b80 1 Processor -- start 2024-05-08T23:25:13.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.998+0000 7fd681950b80 1 -- start start 2024-05-08T23:25:13.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.998+0000 7fd681950b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 0x55797dfa21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.998+0000 7fd681950b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797dfa7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.998+0000 7fd681950b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfd0e60 0x55797dfd12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.998+0000 7fd681950b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797defd1e0 con 0x55797dfa4400 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.998+0000 7fd681950b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797defaba0 con 0x55797dfd0e60 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.998+0000 7fd681950b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55797def6490 con 0x55797dfc9830 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797dfa7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797dfa7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33614/0 (socket says 172.21.15.5:33614) 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 -- 172.21.15.5:0/1284476936 learned_addr learned my addr 172.21.15.5:0/1284476936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66effd700 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 0x55797dfa21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfd0e60 0x55797dfd12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 -- 172.21.15.5:0/1284476936 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 msgr2=0x55797dfa21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 0x55797dfa21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 -- 172.21.15.5:0/1284476936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfd0e60 msgr2=0x55797dfd12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfd0e60 0x55797dfd12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 -- 172.21.15.5:0/1284476936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55797dd77440 con 0x55797dfa4400 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797dfa7130 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd658005be0 tx=0x7fd658011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809164b410918774 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd664ff9700 1 -- 172.21.15.5:0/1284476936 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658007770 con 0x55797dfa4400 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd664ff9700 1 -- 172.21.15.5:0/1284476936 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658021450 con 0x55797dfa4400 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:13.999+0000 7fd664ff9700 1 -- 172.21.15.5:0/1284476936 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580217c0 con 0x55797dfa4400 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 -- 172.21.15.5:0/1284476936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 msgr2=0x55797dfa7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797dfa7130 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd658005be0 tx=0x7fd658011000 comp rx=0 tx=0).stop 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 -- 172.21.15.5:0/1284476936 shutdown_connections 2024-05-08T23:25:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfd0e60 0x55797dfd12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 0x55797dfa21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 --2- 172.21.15.5:0/1284476936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797dfa7130 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 -- 172.21.15.5:0/1284476936 >> 172.21.15.5:0/1284476936 conn(0x55797df042a0 msgr2=0x55797dfebe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 -- 172.21.15.5:0/1284476936 shutdown_connections 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.000+0000 7fd681950b80 1 -- 172.21.15.5:0/1284476936 wait complete. 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.001+0000 7fd681950b80 1 -- start start 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.001+0000 7fd681950b80 1 Processor -- start 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd681950b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 0x55797dffb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd681950b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 0x55797dfcdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd681950b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfd0e60 0x55797dfce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd681950b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797de925f0 con 0x55797dfd0e60 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd681950b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797de92290 con 0x55797dfa4400 2024-05-08T23:25:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd681950b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55797dd79730 con 0x55797dfc9830 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd681950b80 10 client.0 _ll_register_callbacks cb 0x55797de9bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfd0e60 0x55797dfce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfd0e60 0x55797dfce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33618/0 (socket says 172.21.15.5:33618) 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 -- 172.21.15.5:0/1548033204 learned_addr learned my addr 172.21.15.5:0/1548033204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 0x55797dfcdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66effd700 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 0x55797dffb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 -- 172.21.15.5:0/1548033204 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 msgr2=0x55797dfcdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 0x55797dfcdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 -- 172.21.15.5:0/1548033204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 msgr2=0x55797dffb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 0x55797dffb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 -- 172.21.15.5:0/1548033204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65800f040 con 0x55797dfd0e60 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.002+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfd0e60 0x55797dfce220 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd67008bd60 tx=0x7fd668004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df3423e233781f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd64dffb700 1 -- 172.21.15.5:0/1548033204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668020070 con 0x55797dfd0e60 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd64dffb700 1 -- 172.21.15.5:0/1548033204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd668012bd0 con 0x55797dfd0e60 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 1 -- 172.21.15.5:0/1548033204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55797de4f300 con 0x55797dfd0e60 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd64dffb700 1 -- 172.21.15.5:0/1548033204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd668012de0 con 0x55797dfd0e60 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 10 client.6870 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 1 -- 172.21.15.5:0/1548033204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55797dd23bf0 con 0x55797dfd0e60 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd64dffb700 1 -- 172.21.15.5:0/1548033204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55797de4f300 con 0x55797dfd0e60 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd64dffb700 1 -- 172.21.15.5:0/1548033204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fd668004040 con 0x55797dfd0e60 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 10 client.6870 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 10 client.6870 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 20 client.6870 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55797dfed2e0 0x55797dfed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 4 client.6870 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd681950b80 1 -- 172.21.15.5:0/1548033204 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55797df75870 con 0x55797dfed2e0 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.003+0000 7fd66effd700 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55797dfed2e0 0x55797dfed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd66effd700 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55797dfed2e0 0x55797dfed6f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df1cdeb44472412 server_cookie=8452334fd006d6c0 in_seq=0 out_seq=0 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd64dffb700 10 client.6870 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd64dffb700 1 -- 172.21.15.5:0/1548033204 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55797df75870 con 0x55797dfed2e0 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd64dffb700 10 client.6870 handle_command_reply: tid=0 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 client.6870 shutdown 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 -- 172.21.15.5:0/1548033204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfd0e60 msgr2=0x55797dfce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfd0e60 0x55797dfce220 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd67008bd60 tx=0x7fd668004b40 comp rx=0 tx=0).stop 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 -- 172.21.15.5:0/1548033204 shutdown_connections 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfd0e60 0x55797dfce220 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55797dfed2e0 0x55797dfed6f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 0x55797dffb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd64dffb700 0 client.6870 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 --2- 172.21.15.5:0/1548033204 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfc9830 0x55797dfcdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.004+0000 7fd681950b80 1 -- 172.21.15.5:0/1548033204 >> 172.21.15.5:0/1548033204 conn(0x55797df042a0 msgr2=0x55797dfec240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.005+0000 7fd681950b80 1 -- 172.21.15.5:0/1548033204 shutdown_connections 2024-05-08T23:25:14.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.005+0000 7fd681950b80 1 -- 172.21.15.5:0/1548033204 wait complete. 2024-05-08T23:25:14.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.005+0000 7fd681950b80 20 client.6870 trim_cache size 0 max 16384 2024-05-08T23:25:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.023+0000 7fd681950b80 1 Processor -- start 2024-05-08T23:25:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.023+0000 7fd681950b80 1 -- start start 2024-05-08T23:25:14.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd681950b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 0x55797dfd1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd681950b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfc9830 0x55797dfd1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd681950b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797dfec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd681950b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797e00cd50 con 0x55797dfc9830 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd681950b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797dfd2c20 con 0x55797dfa4400 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd681950b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55797dfeead0 con 0x55797dfd0e60 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfc9830 0x55797dfd1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfc9830 0x55797dfd1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33630/0 (socket says 172.21.15.5:33630) 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 -- 172.21.15.5:0/2788625820 learned_addr learned my addr 172.21.15.5:0/2788625820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797dfec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66effd700 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 0x55797dfd1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 -- 172.21.15.5:0/2788625820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 msgr2=0x55797dfec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797dfec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 -- 172.21.15.5:0/2788625820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 msgr2=0x55797dfd1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 0x55797dfd1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 -- 172.21.15.5:0/2788625820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55797debc5f0 con 0x55797dfc9830 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.024+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfc9830 0x55797dfd1780 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd658014000 tx=0x7fd658017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1399f5e573881dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd6657fa700 1 -- 172.21.15.5:0/2788625820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658027040 con 0x55797dfc9830 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd6657fa700 1 -- 172.21.15.5:0/2788625820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658019030 con 0x55797dfc9830 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd6657fa700 1 -- 172.21.15.5:0/2788625820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580215c0 con 0x55797dfc9830 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 -- 172.21.15.5:0/2788625820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfc9830 msgr2=0x55797dfd1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfc9830 0x55797dfd1780 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd658014000 tx=0x7fd658017070 comp rx=0 tx=0).stop 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 -- 172.21.15.5:0/2788625820 shutdown_connections 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797dfec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfa4400 0x55797dfd1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 --2- 172.21.15.5:0/2788625820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfc9830 0x55797dfd1780 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 -- 172.21.15.5:0/2788625820 >> 172.21.15.5:0/2788625820 conn(0x55797df042a0 msgr2=0x55797dfa05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 -- 172.21.15.5:0/2788625820 shutdown_connections 2024-05-08T23:25:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.025+0000 7fd681950b80 1 -- 172.21.15.5:0/2788625820 wait complete. 2024-05-08T23:25:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.026+0000 7fd681950b80 1 -- start start 2024-05-08T23:25:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.026+0000 7fd681950b80 1 Processor -- start 2024-05-08T23:25:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd681950b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797e055250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd681950b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfc9830 0x55797dfccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd681950b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797e004b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd681950b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797dfe8e00 con 0x55797dfa4400 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd681950b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797e055a10 con 0x55797dfc9830 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd681950b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55797e004360 con 0x55797dfd0e60 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd681950b80 10 client.0 _ll_register_callbacks cb 0x55797de9bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797e055250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797e055250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33640/0 (socket says 172.21.15.5:33640) 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 -- 172.21.15.5:0/3631992053 learned_addr learned my addr 172.21.15.5:0/3631992053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfc9830 0x55797dfccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797e004b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 -- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 msgr2=0x55797e004b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797e004b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 -- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfc9830 msgr2=0x55797dfccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfc9830 0x55797dfccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 -- 172.21.15.5:0/3631992053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55797dd77440 con 0x55797dfa4400 2024-05-08T23:25:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfc9830 0x55797dfccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66f7fe700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797e004b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd66effd700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797e055250 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd660057ba0 tx=0x7fd660057cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdca92e585c00c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.027+0000 7fd64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd64dffb700 1 -- 172.21.15.5:0/3631992053 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660054320 con 0x55797dfa4400 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd64dffb700 1 -- 172.21.15.5:0/3631992053 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd660054480 con 0x55797dfa4400 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 1 -- 172.21.15.5:0/3631992053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd658021450 con 0x55797dfa4400 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd64dffb700 1 -- 172.21.15.5:0/3631992053 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd660057e70 con 0x55797dfa4400 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 10 client.6876 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 1 -- 172.21.15.5:0/3631992053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55797dd7bd60 con 0x55797dfa4400 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd64dffb700 1 -- 172.21.15.5:0/3631992053 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658021450 con 0x55797dfa4400 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd64dffb700 1 -- 172.21.15.5:0/3631992053 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fd660057100 con 0x55797dfa4400 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 10 client.6876 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 10 client.6876 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 20 client.6876 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55797dfd3f40 0x55797dfc7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 4 client.6876 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd681950b80 1 -- 172.21.15.5:0/3631992053 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) v1 -- 0x55797deeef50 con 0x55797dfd3f40 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.028+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55797dfd3f40 0x55797dfc7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.029+0000 7fd66e7fc700 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55797dfd3f40 0x55797dfc7590 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85175f026e621249 server_cookie=9879cc0966a03f31 in_seq=0 out_seq=0 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.029+0000 7fd64dffb700 10 client.6876 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.029+0000 7fd64dffb700 1 -- 172.21.15.5:0/3631992053 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55797deeef50 con 0x55797dfd3f40 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.029+0000 7fd64dffb700 10 client.6876 handle_command_reply: tid=0 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.029+0000 7fd681950b80 1 client.6876 shutdown 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 -- 172.21.15.5:0/3631992053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 msgr2=0x55797e055250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797e055250 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd660057ba0 tx=0x7fd660057cc0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.035 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:25:14.030+0000 7f82fcd00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-05-08T23:24:57.097108+0000 1=0+1) n(v3 rc2024-05-08T23:24:57.359102+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={6349=pAsLsXsFsx/pAsLsXsFsx@5},l=6349 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56161caa6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.097108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.359102+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.097108+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-05-08T23:24:57.359102+0000 6=0+6)","error_str":""},"return_code":-61} 2024-05-08T23:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 -- 172.21.15.5:0/3631992053 shutdown_connections 2024-05-08T23:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55797dfd0e60 0x55797e004b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797dfa4400 0x55797e055250 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55797dfd3f40 0x55797dfc7590 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 --2- 172.21.15.5:0/3631992053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55797dfc9830 0x55797dfccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 -- 172.21.15.5:0/3631992053 >> 172.21.15.5:0/3631992053 conn(0x55797df042a0 msgr2=0x55797dfd1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd64dffb700 0 client.6876 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 -- 172.21.15.5:0/3631992053 shutdown_connections 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 1 -- 172.21.15.5:0/3631992053 wait complete. 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.030+0000 7fd681950b80 20 client.6876 trim_cache size 0 max 16384 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:25:14.036 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-08T23:25:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a68f2f52-d2bb-4b0d-aabc-647e8d644493", 2024-05-08T23:25:14.037 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-08T23:25:14.037 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:25:14.037 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:25:14.030+0000 7f81d46bc700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1) n(v2 rc2024-05-08T23:24:57.359102+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6349=pAsLsXsFsx/pAsLsXsFsx@4},l=6349 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55f7be187700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.359102+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-05-08T23:24:57.359102+0000 4=0+4)","error_str":""},"return_code":-61} 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 -- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 msgr2=0x7fd65006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd6700b1b90 tx=0x7fd658009040 comp rx=0 tx=0).stop 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 -- 172.21.15.5:0/1092908107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 msgr2=0x7fd6701f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6701f71e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd66000e810 tx=0x7fd66000ebd0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 -- 172.21.15.5:0/1092908107 shutdown_connections 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd670160f10 0x7fd67006c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700d04e0 0x7fd6701f71e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 --2- 172.21.15.5:0/1092908107 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0d80 0x7fd67014cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 -- 172.21.15.5:0/1092908107 >> 172.21.15.5:0/1092908107 conn(0x7fd67008afe0 msgr2=0x7fd6700b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 -- 172.21.15.5:0/1092908107 shutdown_connections 2024-05-08T23:25:14.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.031+0000 7fd675796700 1 -- 172.21.15.5:0/1092908107 wait complete. 2024-05-08T23:25:14.039 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:25:14.031+0000 7f8302d0c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1) n(v4 rc2024-05-08T23:24:57.359102+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6349=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x56161caa6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.359102+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-05-08T23:24:57.359102+0000 5=0+5)","error_str":""},"return_code":-61} 2024-05-08T23:25:14.039 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:25:14.032+0000 7f81da6c8700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-05-08T23:24:57.099108+0000 1=0+1) n(v1 rc2024-05-08T23:24:57.359102+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6349=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55f7bd226b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.099108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.100108+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.099108+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-05-08T23:24:57.359102+0000 3=0+3)","error_str":""},"return_code":-61} 2024-05-08T23:25:14.040 INFO:tasks.ceph.mds.d.smithi049.stderr:2024-05-08T23:25:14.031+0000 7ffaa691c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-05-08T23:24:57.100108+0000 1=0+1) n(v0 rc2024-05-08T23:24:57.100108+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6349=pAsLsXsFsx/pAsLsXsFsx@4},l=6349 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562427e5eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.100108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.100108+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.100108+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-05-08T23:24:57.100108+0000 2=0+2)","error_str":""},"return_code":-61} 2024-05-08T23:25:14.040 INFO:tasks.ceph.mds.d.smithi049.stderr:2024-05-08T23:25:14.031+0000 7ffaa691c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6349=pAsLsXsFsx/AsLsXsFsx@2},l=6349 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562427e5f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-05-08T23:25:14.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T23:25:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.183+0000 7fba4dc92700 1 Processor -- start 2024-05-08T23:25:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.183+0000 7fba4dc92700 1 -- start start 2024-05-08T23:25:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba4dc92700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba4815db00 0x7fba4806dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba4dc92700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fba480cb2a0 0x7fba480cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba4dc92700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4806fbf0 con 0x7fba480ce120 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba4dc92700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4806edb0 con 0x7fba4815db00 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba4dc92700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4806f090 con 0x7fba480cb2a0 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba477fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba4815db00 0x7fba4806dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba477fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba4815db00 0x7fba4806dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33656/0 (socket says 172.21.15.5:33656) 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba477fe700 1 -- 172.21.15.5:0/2680539294 learned_addr learned my addr 172.21.15.5:0/2680539294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba47fff700 1 --2- 172.21.15.5:0/2680539294 >> v2:172.21.15.49:3300/0 conn(0x7fba480cb2a0 0x7fba480cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba477fe700 1 -- 172.21.15.5:0/2680539294 >> v2:172.21.15.49:3300/0 conn(0x7fba480cb2a0 msgr2=0x7fba480cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba477fe700 1 --2- 172.21.15.5:0/2680539294 >> v2:172.21.15.49:3300/0 conn(0x7fba480cb2a0 0x7fba480cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba477fe700 1 -- 172.21.15.5:0/2680539294 >> v1:172.21.15.49:6789/0 conn(0x7fba480ce120 legacy=0x7fba4806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.184+0000 7fba477fe700 1 -- 172.21.15.5:0/2680539294 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4806f370 con 0x7fba4815db00 2024-05-08T23:25:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba477fe700 1 --2- 172.21.15.5:0/2680539294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba4815db00 0x7fba4806dc80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fba480727d0 tx=0x7fba38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f05e9babe6f05e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba467fc700 1 -- 172.21.15.5:0/2680539294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3800b870 con 0x7fba4815db00 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba467fc700 1 -- 172.21.15.5:0/2680539294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3800b9d0 con 0x7fba4815db00 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba467fc700 1 -- 172.21.15.5:0/2680539294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3800bd60 con 0x7fba4815db00 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba4dc92700 1 -- 172.21.15.5:0/2680539294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba4815db00 msgr2=0x7fba4806dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba4dc92700 1 --2- 172.21.15.5:0/2680539294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba4815db00 0x7fba4806dc80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fba480727d0 tx=0x7fba38009bd0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba4dc92700 1 -- 172.21.15.5:0/2680539294 shutdown_connections 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba4dc92700 1 --2- 172.21.15.5:0/2680539294 >> v2:172.21.15.49:3300/0 conn(0x7fba480cb2a0 0x7fba480cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba4dc92700 1 --2- 172.21.15.5:0/2680539294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba4815db00 0x7fba4806dc80 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba4dc92700 1 -- 172.21.15.5:0/2680539294 >> 172.21.15.5:0/2680539294 conn(0x7fba48099a50 msgr2=0x7fba480d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba4dc92700 1 -- 172.21.15.5:0/2680539294 shutdown_connections 2024-05-08T23:25:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.185+0000 7fba4dc92700 1 -- 172.21.15.5:0/2680539294 wait complete. 2024-05-08T23:25:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.187+0000 7fba4dc92700 1 Processor -- start 2024-05-08T23:25:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.187+0000 7fba4dc92700 1 -- start start 2024-05-08T23:25:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba4dc92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba480cb2a0 0x7fba481f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba4dc92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba480ce120 0x7fba481f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba4dc92700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba4815db00 0x7fba481f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba4dc92700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba480b27a0 con 0x7fba480ce120 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba4dc92700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48156870 con 0x7fba480cb2a0 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba4dc92700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba481562d0 con 0x7fba4815db00 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba480ce120 0x7fba481f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba480ce120 0x7fba481f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33664/0 (socket says 172.21.15.5:33664) 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 -- 172.21.15.5:0/833451838 learned_addr learned my addr 172.21.15.5:0/833451838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba477fe700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba480cb2a0 0x7fba481f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba47fff700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba4815db00 0x7fba481f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 -- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba4815db00 msgr2=0x7fba481f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba4815db00 0x7fba481f6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 -- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba480cb2a0 msgr2=0x7fba481f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba480cb2a0 0x7fba481f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 -- 172.21.15.5:0/833451838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba38009550 con 0x7fba480ce120 2024-05-08T23:25:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba47fff700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba4815db00 0x7fba481f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.188+0000 7fba46ffd700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba480ce120 0x7fba481f6580 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fba3000c510 tx=0x7fba3000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.189+0000 7fba3ffff700 1 -- 172.21.15.5:0/833451838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba300052d0 con 0x7fba480ce120 2024-05-08T23:25:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.189+0000 7fba3ffff700 1 -- 172.21.15.5:0/833451838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba30005430 con 0x7fba480ce120 2024-05-08T23:25:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.189+0000 7fba4dc92700 1 -- 172.21.15.5:0/833451838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba48156590 con 0x7fba480ce120 2024-05-08T23:25:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.189+0000 7fba3ffff700 1 -- 172.21.15.5:0/833451838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba300038c0 con 0x7fba480ce120 2024-05-08T23:25:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.189+0000 7fba4dc92700 1 -- 172.21.15.5:0/833451838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba480b2c30 con 0x7fba480ce120 2024-05-08T23:25:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.190+0000 7fba3ffff700 1 -- 172.21.15.5:0/833451838 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fba30003a20 con 0x7fba480ce120 2024-05-08T23:25:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.191+0000 7fba3ffff700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba2806ac90 0x7fba2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.191+0000 7fba3ffff700 1 -- 172.21.15.5:0/833451838 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba300887f0 con 0x7fba480ce120 2024-05-08T23:25:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.191+0000 7fba477fe700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba2806ac90 0x7fba2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.191+0000 7fba477fe700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba2806ac90 0x7fba2806d150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fba38000c00 tx=0x7fba38008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.193+0000 7fba59e4cb80 1 Processor -- start 2024-05-08T23:25:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.193+0000 7fba59e4cb80 1 -- start start 2024-05-08T23:25:14.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f60191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f5f45150 con 0x5566f5feb720 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f5f42b10 con 0x5566f6018dd0 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566f5f3e400 con 0x5566f6016570 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33676/0 (socket says 172.21.15.5:33676) 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba46ffd700 1 -- 172.21.15.5:0/4288693729 learned_addr learned my addr 172.21.15.5:0/4288693729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba477fe700 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f60191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba47fff700 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba46ffd700 1 -- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 msgr2=0x5566f5feeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba46ffd700 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba46ffd700 1 -- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 msgr2=0x5566f60191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba46ffd700 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f60191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba46ffd700 1 -- 172.21.15.5:0/4288693729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566f5dbf3b0 con 0x5566f5feb720 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.194+0000 7fba477fe700 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f60191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.195+0000 7fba46ffd700 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee970 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fba3005f550 tx=0x7fba300572d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb4f6e844a9c17a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.195+0000 7fba3d7fa700 1 -- 172.21.15.5:0/4288693729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30057690 con 0x5566f5feb720 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.195+0000 7fba3d7fa700 1 -- 172.21.15.5:0/4288693729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba30088050 con 0x5566f5feb720 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.195+0000 7fba3d7fa700 1 -- 172.21.15.5:0/4288693729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3005b4c0 con 0x5566f5feb720 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.195+0000 7fba59e4cb80 1 -- 172.21.15.5:0/4288693729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 msgr2=0x5566f5fee970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.195+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee970 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fba3005f550 tx=0x7fba300572d0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 -- 172.21.15.5:0/4288693729 shutdown_connections 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f60191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/4288693729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee970 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 -- 172.21.15.5:0/4288693729 >> 172.21.15.5:0/4288693729 conn(0x5566f5f4c210 msgr2=0x5566f6033dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 -- 172.21.15.5:0/4288693729 shutdown_connections 2024-05-08T23:25:14.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 -- 172.21.15.5:0/4288693729 wait complete. 2024-05-08T23:25:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 -- start start 2024-05-08T23:25:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.196+0000 7fba59e4cb80 1 Processor -- start 2024-05-08T23:25:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f5feb720 0x5566f6031f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6016570 0x5566f6015cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6018dd0 0x5566f604f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f5eda560 con 0x5566f6016570 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f5eda200 con 0x5566f5feb720 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566f5dc16a0 con 0x5566f6018dd0 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba59e4cb80 10 client.0 _ll_register_callbacks cb 0x5566f5ee3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6016570 0x5566f6015cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6016570 0x5566f6015cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33678/0 (socket says 172.21.15.5:33678) 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba46ffd700 1 -- 172.21.15.5:0/3880885989 learned_addr learned my addr 172.21.15.5:0/3880885989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba477fe700 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f5feb720 0x5566f6031f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.197+0000 7fba47fff700 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6018dd0 0x5566f604f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba46ffd700 1 -- 172.21.15.5:0/3880885989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6018dd0 msgr2=0x5566f604f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba46ffd700 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6018dd0 0x5566f604f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba46ffd700 1 -- 172.21.15.5:0/3880885989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f5feb720 msgr2=0x5566f6031f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba46ffd700 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f5feb720 0x5566f6031f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba46ffd700 1 -- 172.21.15.5:0/3880885989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba480b2c30 con 0x5566f6016570 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba46ffd700 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6016570 0x5566f6015cd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fba30057390 tx=0x7fba30001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e678cf2370eec94e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba267fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba267fc700 1 -- 172.21.15.5:0/3880885989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3005b4c0 con 0x5566f6016570 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba267fc700 1 -- 172.21.15.5:0/3880885989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba30086070 con 0x5566f6016570 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 1 -- 172.21.15.5:0/3880885989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566f5e97270 con 0x5566f6016570 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba267fc700 1 -- 172.21.15.5:0/3880885989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba30076120 con 0x5566f6016570 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 10 client.6888 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 1 -- 172.21.15.5:0/3880885989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5566f5d6bb60 con 0x5566f6016570 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba267fc700 1 -- 172.21.15.5:0/3880885989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5566f5e97270 con 0x5566f6016570 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba267fc700 1 -- 172.21.15.5:0/3880885989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fba30056b80 con 0x5566f6016570 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 10 client.6888 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 10 client.6888 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 20 client.6888 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5566f6029c10 0x5566f602a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 4 client.6888 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.198+0000 7fba59e4cb80 1 -- 172.21.15.5:0/3880885989 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566f5fbd7e0 con 0x5566f6029c10 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.199+0000 7fba477fe700 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5566f6029c10 0x5566f602a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.199+0000 7fba477fe700 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5566f6029c10 0x5566f602a020 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc56e5c588b5ddae server_cookie=84d7d81359f8824c in_seq=0 out_seq=0 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.199+0000 7fba267fc700 10 client.6888 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.199+0000 7fba267fc700 1 -- 172.21.15.5:0/3880885989 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5566f5fbd7e0 con 0x5566f6029c10 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.199+0000 7fba267fc700 10 client.6888 handle_command_reply: tid=0 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 client.6888 shutdown 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 -- 172.21.15.5:0/3880885989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6016570 msgr2=0x5566f6015cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6016570 0x5566f6015cd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fba30057390 tx=0x7fba30001120 comp rx=0 tx=0).stop 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 -- 172.21.15.5:0/3880885989 shutdown_connections 2024-05-08T23:25:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6018dd0 0x5566f604f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5566f6029c10 0x5566f602a020 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f5feb720 0x5566f6031f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/3880885989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6016570 0x5566f6015cd0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba267fc700 0 client.6888 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 -- 172.21.15.5:0/3880885989 >> 172.21.15.5:0/3880885989 conn(0x5566f5f4c210 msgr2=0x5566f6035250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 -- 172.21.15.5:0/3880885989 shutdown_connections 2024-05-08T23:25:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 1 -- 172.21.15.5:0/3880885989 wait complete. 2024-05-08T23:25:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.200+0000 7fba59e4cb80 20 client.6888 trim_cache size 0 max 16384 2024-05-08T23:25:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.218+0000 7fba59e4cb80 1 Processor -- start 2024-05-08T23:25:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.218+0000 7fba59e4cb80 1 -- start start 2024-05-08T23:25:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f6033770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f6034430 con 0x5566f5feb720 2024-05-08T23:25:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f604ba50 con 0x5566f6018dd0 2024-05-08T23:25:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566f601c740 con 0x5566f6016570 2024-05-08T23:25:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33680/0 (socket says 172.21.15.5:33680) 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba477fe700 1 -- 172.21.15.5:0/1737049351 learned_addr learned my addr 172.21.15.5:0/1737049351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba47fff700 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f6033770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba46ffd700 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba477fe700 1 -- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 msgr2=0x5566f5feec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba477fe700 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba477fe700 1 -- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 msgr2=0x5566f6033770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba477fe700 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f6033770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba477fe700 1 -- 172.21.15.5:0/1737049351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566f5f04560 con 0x5566f5feb720 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba47fff700 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f6033770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.219+0000 7fba46ffd700 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba477fe700 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee560 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fba48072c30 tx=0x7fba38011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590eeacbe077a4eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba3dffb700 1 -- 172.21.15.5:0/1737049351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3802c070 con 0x5566f5feb720 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba3dffb700 1 -- 172.21.15.5:0/1737049351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3802d040 con 0x5566f5feb720 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba3dffb700 1 -- 172.21.15.5:0/1737049351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38033830 con 0x5566f5feb720 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba59e4cb80 1 -- 172.21.15.5:0/1737049351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 msgr2=0x5566f5fee560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee560 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fba48072c30 tx=0x7fba38011000 comp rx=0 tx=0).stop 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba59e4cb80 1 -- 172.21.15.5:0/1737049351 shutdown_connections 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6018dd0 0x5566f6033770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f5feb720 0x5566f5fee560 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.220+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/1737049351 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f6016570 0x5566f5feec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.221+0000 7fba59e4cb80 1 -- 172.21.15.5:0/1737049351 >> 172.21.15.5:0/1737049351 conn(0x5566f5f4c210 msgr2=0x5566f5fe51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.221+0000 7fba59e4cb80 1 -- 172.21.15.5:0/1737049351 shutdown_connections 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.221+0000 7fba59e4cb80 1 -- 172.21.15.5:0/1737049351 wait complete. 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.221+0000 7fba59e4cb80 1 -- start start 2024-05-08T23:25:14.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.221+0000 7fba59e4cb80 1 Processor -- start 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f5feb720 0x5566f604aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6016570 0x5566f5fee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba59e4cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6018dd0 0x5566f5feebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f6045d60 con 0x5566f6018dd0 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f60a5260 con 0x5566f6016570 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba59e4cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566f60310d0 con 0x5566f5feb720 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba59e4cb80 10 client.0 _ll_register_callbacks cb 0x5566f5ee3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6018dd0 0x5566f5feebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6018dd0 0x5566f5feebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33690/0 (socket says 172.21.15.5:33690) 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 -- 172.21.15.5:0/2093743178 learned_addr learned my addr 172.21.15.5:0/2093743178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba46ffd700 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6016570 0x5566f5fee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba477fe700 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f5feb720 0x5566f604aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 -- 172.21.15.5:0/2093743178 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f5feb720 msgr2=0x5566f604aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f5feb720 0x5566f604aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 -- 172.21.15.5:0/2093743178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6016570 msgr2=0x5566f5fee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6016570 0x5566f5fee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 -- 172.21.15.5:0/2093743178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4806f370 con 0x5566f6018dd0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba47fff700 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6018dd0 0x5566f5feebc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x5566f6019ec0 tx=0x7fba40015d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ce2dadb6fb037c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.223+0000 7fba267fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba267fc700 1 -- 172.21.15.5:0/2093743178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40003c30 con 0x5566f6018dd0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 1 -- 172.21.15.5:0/2093743178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba30088050 con 0x5566f6018dd0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba267fc700 1 -- 172.21.15.5:0/2093743178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba40003d90 con 0x5566f6018dd0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba267fc700 1 -- 172.21.15.5:0/2093743178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba400136c0 con 0x5566f6018dd0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 10 client.6894 fetch_fsmap learned FSMap version 56 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 1 -- 172.21.15.5:0/2093743178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5566f5dc3cd0 con 0x5566f6018dd0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba267fc700 1 -- 172.21.15.5:0/2093743178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba30088050 con 0x5566f6018dd0 2024-05-08T23:25:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba267fc700 1 -- 172.21.15.5:0/2093743178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fba40013820 con 0x5566f6018dd0 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 10 client.6894 fetch_fsmap finished waiting for FSMap version 56 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 10 client.6894 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 20 client.6894 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5566f5fe82c0 0x5566f60a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 4 client.6894 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba59e4cb80 1 -- 172.21.15.5:0/2093743178 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5566f5f36ec0 con 0x5566f5fe82c0 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.224+0000 7fba477fe700 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5566f5fe82c0 0x5566f60a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba267fc700 10 client.6894 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba477fe700 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5566f5fe82c0 0x5566f60a56b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc997c7c4f29e9d server_cookie=6c9b4b1adc543e74 in_seq=0 out_seq=0 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba267fc700 1 -- 172.21.15.5:0/2093743178 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5566f5f36ec0 con 0x5566f5fe82c0 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba267fc700 10 client.6894 handle_command_reply: tid=0 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 client.6894 shutdown 2024-05-08T23:25:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 -- 172.21.15.5:0/2093743178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6018dd0 msgr2=0x5566f5feebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6018dd0 0x5566f5feebc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x5566f6019ec0 tx=0x7fba40015d80 comp rx=0 tx=0).stop 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 -- 172.21.15.5:0/2093743178 shutdown_connections 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566f6018dd0 0x5566f5feebc0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5566f5feb720 0x5566f604aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5566f5fe82c0 0x5566f60a56b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 --2- 172.21.15.5:0/2093743178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5566f6016570 0x5566f5fee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba267fc700 0 client.6894 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 -- 172.21.15.5:0/2093743178 >> 172.21.15.5:0/2093743178 conn(0x5566f5f4c210 msgr2=0x5566f6033610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 -- 172.21.15.5:0/2093743178 shutdown_connections 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 1 -- 172.21.15.5:0/2093743178 wait complete. 2024-05-08T23:25:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.225+0000 7fba59e4cb80 20 client.6894 trim_cache size 0 max 16384 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "a68f2f52-d2bb-4b0d-aabc-647e8d644493": { 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a68f2f52-d2bb-4b0d-aabc-647e8d644493", 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:25:14.232 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 -- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba2806ac90 msgr2=0x7fba2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba2806ac90 0x7fba2806d150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fba38000c00 tx=0x7fba38008030 comp rx=0 tx=0).stop 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 -- 172.21.15.5:0/833451838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba480ce120 msgr2=0x7fba481f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba480ce120 0x7fba481f6580 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fba3000c510 tx=0x7fba3000c8d0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 -- 172.21.15.5:0/833451838 shutdown_connections 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fba2806ac90 0x7fba2806d150 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fba4815db00 0x7fba481f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba480cb2a0 0x7fba481f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 --2- 172.21.15.5:0/833451838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba480ce120 0x7fba481f6580 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:14.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 -- 172.21.15.5:0/833451838 >> 172.21.15.5:0/833451838 conn(0x7fba48099a50 msgr2=0x7fba480cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:14.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 -- 172.21.15.5:0/833451838 shutdown_connections 2024-05-08T23:25:14.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:14.226+0000 7fba4dc92700 1 -- 172.21.15.5:0/833451838 wait complete. 2024-05-08T23:25:14.239 INFO:tasks.cephfs.filesystem:scrub status for tag:a68f2f52-d2bb-4b0d-aabc-647e8d644493 - {'path': '/d1/d2/d3', 'tag': 'a68f2f52-d2bb-4b0d-aabc-647e8d644493', 'options': 'recursive,force'} 2024-05-08T23:25:17.426 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:25:17.426+0000 7f8a4c50b700 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T23:25:25.104 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:25:25.104+0000 7f8a4c50b700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T23:25:37.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:25:37.982 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:25:44.240 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T23:25:44.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.388+0000 7f5c56d0d700 1 Processor -- start 2024-05-08T23:25:44.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.389+0000 7f5c56d0d700 1 -- start start 2024-05-08T23:25:44.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.389+0000 7f5c56d0d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5c5009a790 0x7f5c50099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.389+0000 7f5c56d0d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5006d010 con 0x7f5c5015da60 2024-05-08T23:25:44.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.389+0000 7f5c56d0d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5015de40 con 0x7f5c50099f90 2024-05-08T23:25:44.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.389+0000 7f5c56d0d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5009a420 con 0x7f5c5009a790 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c54aa9700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5c5015da60 0x7f5c500d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33498/0 (socket says 172.21.15.5:33498) 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c54aa9700 1 -- 172.21.15.5:0/2845634823 learned_addr learned my addr 172.21.15.5:0/2845634823 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2845634823 >> v2:172.21.15.49:3300/0 conn(0x7f5c5009a790 0x7f5c50099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139632973 0 0) 0x7f5c5006d010 con 0x7f5c5015da60 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c380035b0 con 0x7f5c5015da60 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190933050 0 0) 0x7f5c380035b0 con 0x7f5c5015da60 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 >> v2:172.21.15.49:3300/0 conn(0x7f5c5009a790 msgr2=0x7f5c50099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 --2- 172.21.15.5:0/2845634823 >> v2:172.21.15.49:3300/0 conn(0x7f5c5009a790 0x7f5c50099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 >> v1:172.21.15.49:6789/0 conn(0x7f5c50099f90 legacy=0x7f5c500cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c500d0b20 con 0x7f5c5015da60 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f5c40001670 con 0x7f5c5015da60 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 >> v1:172.21.15.5:6789/0 conn(0x7f5c5015da60 legacy=0x7f5c500d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38003a00 0x7f5c38003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c38004850 0x7f5c38004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c38005210 0x7f5c3800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5009a420 con 0x7f5c38005210 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3800c0c0 con 0x7f5c38003a00 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3800c280 con 0x7f5c38004850 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.390+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c38005210 0x7f5c3800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38003a00 0x7f5c38003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c38004850 0x7f5c38004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c552aa700 1 -- 172.21.15.5:0/2845634823 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c38004850 msgr2=0x7f5c38004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c38004850 0x7f5c38004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c552aa700 1 -- 172.21.15.5:0/2845634823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38003a00 msgr2=0x7f5c38003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38003a00 0x7f5c38003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c552aa700 1 -- 172.21.15.5:0/2845634823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380035b0 con 0x7f5c38005210 2024-05-08T23:25:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c552aa700 1 -- 172.21.15.5:0/2845634823 reap_dead start 2024-05-08T23:25:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c38005210 0x7f5c3800ba10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c190 tx=0x7f5c38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb6c19787e7ee27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48001db0 con 0x7f5c38005210 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4800ccf0 con 0x7f5c38005210 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.391+0000 7f5c4f7fe700 1 -- 172.21.15.5:0/2845634823 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48003c40 con 0x7f5c38005210 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.392+0000 7f5c56d0d700 1 -- 172.21.15.5:0/2845634823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c38005210 msgr2=0x7f5c3800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.392+0000 7f5c56d0d700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c38005210 0x7f5c3800ba10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c190 tx=0x7f5c38005880 comp rx=0 tx=0).stop 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.392+0000 7f5c56d0d700 1 -- 172.21.15.5:0/2845634823 shutdown_connections 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.392+0000 7f5c56d0d700 1 --2- 172.21.15.5:0/2845634823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c38005210 0x7f5c3800ba10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.392+0000 7f5c56d0d700 1 -- 172.21.15.5:0/2845634823 >> 172.21.15.5:0/2845634823 conn(0x7f5c500b2f60 msgr2=0x7f5c500b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.392+0000 7f5c56d0d700 1 -- 172.21.15.5:0/2845634823 shutdown_connections 2024-05-08T23:25:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.392+0000 7f5c56d0d700 1 -- 172.21.15.5:0/2845634823 wait complete. 2024-05-08T23:25:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.393+0000 7f5c56d0d700 1 Processor -- start 2024-05-08T23:25:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.393+0000 7f5c56d0d700 1 -- start start 2024-05-08T23:25:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c56d0d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38005210 0x7f5c500c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c56d0d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c5015fc20 0x7f5c501527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c56d0d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c50160050 0x7f5c50152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c56d0d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500b5180 con 0x7f5c50160050 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c56d0d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500b4500 con 0x7f5c38005210 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c56d0d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500b47e0 con 0x7f5c5015fc20 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c552aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c50160050 0x7f5c50152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c552aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c50160050 0x7f5c50152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51424/0 (socket says 172.21.15.5:51424) 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c552aa700 1 -- 172.21.15.5:0/3755281928 learned_addr learned my addr 172.21.15.5:0/3755281928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c5015fc20 0x7f5c501527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.394+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38005210 0x7f5c500c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c552aa700 1 -- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c5015fc20 msgr2=0x7f5c501527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c552aa700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c5015fc20 0x7f5c501527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c552aa700 1 -- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38005210 msgr2=0x7f5c500c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c552aa700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38005210 0x7f5c500c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c552aa700 1 -- 172.21.15.5:0/3755281928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c500b4ac0 con 0x7f5c50160050 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c552aa700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c50160050 0x7f5c50152cf0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005040 tx=0x7f5c480014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c4d7fa700 1 -- 172.21.15.5:0/3755281928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480052d0 con 0x7f5c50160050 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c4d7fa700 1 -- 172.21.15.5:0/3755281928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48005430 con 0x7f5c50160050 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c56d0d700 1 -- 172.21.15.5:0/3755281928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c500b4da0 con 0x7f5c50160050 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c4d7fa700 1 -- 172.21.15.5:0/3755281928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480057a0 con 0x7f5c50160050 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.395+0000 7f5c56d0d700 1 -- 172.21.15.5:0/3755281928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c500c8240 con 0x7f5c50160050 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.397+0000 7f5c4d7fa700 1 -- 172.21.15.5:0/3755281928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5c4801d480 con 0x7f5c50160050 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.397+0000 7f5c4d7fa700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c3c06ac90 0x7f5c3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.397+0000 7f5c4d7fa700 1 -- 172.21.15.5:0/3755281928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c48089c20 con 0x7f5c50160050 2024-05-08T23:25:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.398+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c3c06ac90 0x7f5c3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.398+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c3c06ac90 0x7f5c3c06d150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5c40000c00 tx=0x7f5c40001b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.399+0000 7f5c62ec7b80 1 Processor -- start 2024-05-08T23:25:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c62ec7b80 1 -- start start 2024-05-08T23:25:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b06110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b0b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb17a61150 con 0x55eb17b08370 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb17a5eb10 con 0x55eb17b34dd0 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb17a5a400 con 0x55eb17b2d7a0 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b0b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.400+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b0b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51436/0 (socket says 172.21.15.5:51436) 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c4ffff700 1 -- 172.21.15.5:0/2545048718 learned_addr learned my addr 172.21.15.5:0/2545048718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b06110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c4ffff700 1 -- 172.21.15.5:0/2545048718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 msgr2=0x55eb17b06110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b06110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c4ffff700 1 -- 172.21.15.5:0/2545048718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 msgr2=0x55eb17b35250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c4ffff700 1 -- 172.21.15.5:0/2545048718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb178db3b0 con 0x55eb17b08370 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b0b0a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009c40 tx=0x7f5c44009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87708c56be5b2af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c367fc700 1 -- 172.21.15.5:0/2545048718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44014830 con 0x55eb17b08370 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c367fc700 1 -- 172.21.15.5:0/2545048718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44014990 con 0x55eb17b08370 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.401+0000 7f5c367fc700 1 -- 172.21.15.5:0/2545048718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44014d20 con 0x55eb17b08370 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2545048718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 msgr2=0x55eb17b0b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b0b0a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009c40 tx=0x7f5c44009c70 comp rx=0 tx=0).stop 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2545048718 shutdown_connections 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b06110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2545048718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b0b0a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2545048718 >> 172.21.15.5:0/2545048718 conn(0x55eb17a68210 msgr2=0x55eb17b4fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2545048718 shutdown_connections 2024-05-08T23:25:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.402+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2545048718 wait complete. 2024-05-08T23:25:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.403+0000 7f5c62ec7b80 1 -- start start 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.403+0000 7f5c62ec7b80 1 Processor -- start 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b08370 0x55eb17b5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b2d7a0 0x55eb17b5f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb179f6560 con 0x55eb17b2d7a0 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb179f6200 con 0x55eb17b08370 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb178dd6a0 con 0x55eb17b34dd0 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c62ec7b80 10 client.0 _ll_register_callbacks cb 0x55eb179ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b2d7a0 0x55eb17b5f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b2d7a0 0x55eb17b5f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51452/0 (socket says 172.21.15.5:51452) 2024-05-08T23:25:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c4ffff700 1 -- 172.21.15.5:0/2121653187 learned_addr learned my addr 172.21.15.5:0/2121653187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b08370 0x55eb17b5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c4ffff700 1 -- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 msgr2=0x55eb17b35250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c4ffff700 1 -- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b08370 msgr2=0x55eb17b5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b08370 0x55eb17b5f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c4ffff700 1 -- 172.21.15.5:0/2121653187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c44009870 con 0x55eb17b2d7a0 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.404+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b2d7a0 0x55eb17b5f5f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000c00 tx=0x7f5c44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273629c88d0a58fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2121653187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44014830 con 0x55eb17b2d7a0 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2121653187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c44014e70 con 0x55eb17b2d7a0 2024-05-08T23:25:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2121653187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb179b3270 con 0x55eb17b2d7a0 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2121653187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c440038a0 con 0x55eb17b2d7a0 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 10 client.6906 fetch_fsmap learned FSMap version 59 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2121653187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55eb17887b60 con 0x55eb17b2d7a0 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2121653187 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb179b3270 con 0x55eb17b2d7a0 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2121653187 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f5c44024070 con 0x55eb17b2d7a0 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 10 client.6906 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 10 client.6906 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 20 client.6906 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55eb17b35790 0x55eb17b51250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 4 client.6906 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2121653187 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb17ad97e0 con 0x55eb17b35790 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.405+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55eb17b35790 0x55eb17b51250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.406+0000 7f5c2f7fe700 10 client.6906 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.406+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55eb17b35790 0x55eb17b51250 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8022f8d16f712e64 server_cookie=ba414f00c2512588 in_seq=0 out_seq=0 2024-05-08T23:25:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.406+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2121653187 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb17ad97e0 con 0x55eb17b35790 2024-05-08T23:25:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.406+0000 7f5c2f7fe700 10 client.6906 handle_command_reply: tid=0 2024-05-08T23:25:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.406+0000 7f5c62ec7b80 1 client.6906 shutdown 2024-05-08T23:25:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2121653187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b2d7a0 msgr2=0x55eb17b5f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b2d7a0 0x55eb17b5f5f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000c00 tx=0x7f5c44004750 comp rx=0 tx=0).stop 2024-05-08T23:25:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2121653187 shutdown_connections 2024-05-08T23:25:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 0x55eb17b35250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55eb17b35790 0x55eb17b51250 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b08370 0x55eb17b5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2121653187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b2d7a0 0x55eb17b5f5f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c2f7fe700 0 client.6906 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2121653187 >> 172.21.15.5:0/2121653187 conn(0x55eb17a68210 msgr2=0x55eb17b0a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2121653187 shutdown_connections 2024-05-08T23:25:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2121653187 wait complete. 2024-05-08T23:25:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.407+0000 7f5c62ec7b80 20 client.6906 trim_cache size 0 max 16384 2024-05-08T23:25:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.425+0000 7f5c62ec7b80 1 Processor -- start 2024-05-08T23:25:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.425+0000 7f5c62ec7b80 1 -- start start 2024-05-08T23:25:44.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b37480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b2d7a0 0x55eb17b31490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 0x55eb17b5ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb17b37180 con 0x55eb17b08370 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb17b53840 con 0x55eb17b2d7a0 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb17b36ea0 con 0x55eb17b34dd0 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b37480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b37480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51468/0 (socket says 172.21.15.5:51468) 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 -- 172.21.15.5:0/3987967513 learned_addr learned my addr 172.21.15.5:0/3987967513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 -- 172.21.15.5:0/3987967513 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 msgr2=0x55eb17b5ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/3987967513 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 0x55eb17b5ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 -- 172.21.15.5:0/3987967513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b2d7a0 msgr2=0x55eb17b31490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/3987967513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b2d7a0 0x55eb17b31490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/3987967513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b2d7a0 0x55eb17b31490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 -- 172.21.15.5:0/3987967513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb17a20560 con 0x55eb17b08370 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/3987967513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b37480 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x55eb17b36e70 tx=0x7f5c4000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d240de137867944b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.426+0000 7f5c36ffd700 1 -- 172.21.15.5:0/3987967513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40029070 con 0x55eb17b08370 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c36ffd700 1 -- 172.21.15.5:0/3987967513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4002ab80 con 0x55eb17b08370 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c36ffd700 1 -- 172.21.15.5:0/3987967513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4001a040 con 0x55eb17b08370 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/3987967513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 msgr2=0x55eb17b37480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/3987967513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b37480 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x55eb17b36e70 tx=0x7f5c4000c000 comp rx=0 tx=0).stop 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/3987967513 shutdown_connections 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/3987967513 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b34dd0 0x55eb17b5ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/3987967513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b37480 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/3987967513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b2d7a0 0x55eb17b31490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/3987967513 >> 172.21.15.5:0/3987967513 conn(0x55eb17a68210 msgr2=0x55eb17b71b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/3987967513 shutdown_connections 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.427+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/3987967513 wait complete. 2024-05-08T23:25:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.428+0000 7f5c62ec7b80 1 -- start start 2024-05-08T23:25:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.428+0000 7f5c62ec7b80 1 Processor -- start 2024-05-08T23:25:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b38950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b5ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c62ec7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b5f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb17bb9150 con 0x55eb17b08370 2024-05-08T23:25:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb17b465f0 con 0x55eb17b34dd0 2024-05-08T23:25:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c62ec7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb17b51900 con 0x55eb17b2d7a0 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c62ec7b80 10 client.0 _ll_register_callbacks cb 0x55eb179ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c54aa9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b38950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c54aa9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b38950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51470/0 (socket says 172.21.15.5:51470) 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c54aa9700 1 -- 172.21.15.5:0/2785016555 learned_addr learned my addr 172.21.15.5:0/2785016555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b5ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b5f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c54aa9700 1 -- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 msgr2=0x55eb17b5ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b5ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c54aa9700 1 -- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 msgr2=0x55eb17b5f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b5f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c54aa9700 1 -- 172.21.15.5:0/2785016555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c500d0b20 con 0x55eb17b08370 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c552aa700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b5f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.429+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b5ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c54aa9700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b38950 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5c40007a10 tx=0x7f5c40015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506ad32068ec01fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2785016555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40037070 con 0x55eb17b08370 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2785016555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c4002a680 con 0x55eb17b08370 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2785016555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c44014990 con 0x55eb17b08370 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2785016555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4002a840 con 0x55eb17b08370 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 10 client.6912 fetch_fsmap learned FSMap version 59 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2785016555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x55eb178dfcd0 con 0x55eb17b08370 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2785016555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44014990 con 0x55eb17b08370 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2785016555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f5c4003a020 con 0x55eb17b08370 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 10 client.6912 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:25:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 10 client.6912 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 20 client.6912 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55eb17bb3410 0x55eb17b45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 4 client.6912 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2785016555 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eb17a52ec0 con 0x55eb17bb3410 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.430+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55eb17bb3410 0x55eb17b45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c2f7fe700 10 client.6912 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c4ffff700 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55eb17bb3410 0x55eb17b45760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca8c4d645841f8f server_cookie=abc2975f39b81bd5 in_seq=0 out_seq=0 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c2f7fe700 1 -- 172.21.15.5:0/2785016555 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55eb17a52ec0 con 0x55eb17bb3410 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c2f7fe700 10 client.6912 handle_command_reply: tid=0 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 client.6912 shutdown 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2785016555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 msgr2=0x55eb17b38950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b38950 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5c40007a10 tx=0x7f5c40015000 comp rx=0 tx=0).stop 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2785016555 shutdown_connections 2024-05-08T23:25:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x55eb17bb3410 0x55eb17b45760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb17b34dd0 0x55eb17b5f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb17b08370 0x55eb17b38950 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 --2- 172.21.15.5:0/2785016555 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb17b2d7a0 0x55eb17b5ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c2f7fe700 0 client.6912 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2785016555 >> 172.21.15.5:0/2785016555 conn(0x55eb17a68210 msgr2=0x55eb17b31370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2785016555 shutdown_connections 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.431+0000 7f5c62ec7b80 1 -- 172.21.15.5:0/2785016555 wait complete. 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c62ec7b80 20 client.6912 trim_cache size 0 max 16384 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-08T23:25:44.438 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-08T23:25:44.439 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:25:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 -- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c3c06ac90 msgr2=0x7f5c3c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c3c06ac90 0x7f5c3c06d150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5c40000c00 tx=0x7f5c40001b20 comp rx=0 tx=0).stop 2024-05-08T23:25:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 -- 172.21.15.5:0/3755281928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c50160050 msgr2=0x7f5c50152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c50160050 0x7f5c50152cf0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005040 tx=0x7f5c480014a0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 -- 172.21.15.5:0/3755281928 shutdown_connections 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5c3c06ac90 0x7f5c3c06d150 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c50160050 0x7f5c50152cf0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c38005210 0x7f5c500c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 --2- 172.21.15.5:0/3755281928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c5015fc20 0x7f5c501527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 -- 172.21.15.5:0/3755281928 >> 172.21.15.5:0/3755281928 conn(0x7f5c500b2f60 msgr2=0x7f5c500d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 -- 172.21.15.5:0/3755281928 shutdown_connections 2024-05-08T23:25:44.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.432+0000 7f5c56d0d700 1 -- 172.21.15.5:0/3755281928 wait complete. 2024-05-08T23:25:44.445 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:25:44.446 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 damage ls 2024-05-08T23:25:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.583+0000 7f3a0b9a2700 1 Processor -- start 2024-05-08T23:25:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.584+0000 7f3a0b9a2700 1 -- start start 2024-05-08T23:25:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.584+0000 7f3a0b9a2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.584+0000 7f3a0b9a2700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a040cb160 con 0x7f3a0415db00 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.584+0000 7f3a0b9a2700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0406ad20 con 0x7f3a0406f7c0 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.584+0000 7f3a0b9a2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0406fc50 con 0x7f3a040cc590 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f3a0973e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3a0415db00 0x7f3a040caa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33500/0 (socket says 172.21.15.5:33500) 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f3a0973e700 1 -- 172.21.15.5:0/2541688669 learned_addr learned my addr 172.21.15.5:0/2541688669 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/2541688669 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f39fbfff700 1 -- 172.21.15.5:0/2541688669 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881802036 0 0) 0x7f3a040cb160 con 0x7f3a0415db00 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f39fbfff700 1 -- 172.21.15.5:0/2541688669 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39f00035b0 con 0x7f3a0415db00 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f39fbfff700 1 -- 172.21.15.5:0/2541688669 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735127955 0 0) 0x7f3a0406ad20 con 0x7f3a0406f7c0 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f39fbfff700 1 -- 172.21.15.5:0/2541688669 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a040cb160 con 0x7f3a0406f7c0 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f3a08f3d700 1 -- 172.21.15.5:0/2541688669 >> v1:172.21.15.49:6789/0 conn(0x7f3a0406f7c0 legacy=0x7f3a0406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f3a08f3d700 1 -- 172.21.15.5:0/2541688669 >> v1:172.21.15.5:6789/0 conn(0x7f3a0415db00 legacy=0x7f3a040caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f3a08f3d700 1 -- 172.21.15.5:0/2541688669 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0406b000 con 0x7f3a040cc590 2024-05-08T23:25:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.585+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/2541688669 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406f280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009870 tx=0x7f39f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1187ba9205b4f7d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f39fbfff700 1 -- 172.21.15.5:0/2541688669 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4011830 con 0x7f3a040cc590 2024-05-08T23:25:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f39fbfff700 1 -- 172.21.15.5:0/2541688669 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f4011990 con 0x7f3a040cc590 2024-05-08T23:25:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f39fbfff700 1 -- 172.21.15.5:0/2541688669 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4011d20 con 0x7f3a040cc590 2024-05-08T23:25:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/2541688669 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 msgr2=0x7f3a0406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f3a0b9a2700 1 --2- 172.21.15.5:0/2541688669 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406f280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009870 tx=0x7f39f4009b80 comp rx=0 tx=0).stop 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/2541688669 shutdown_connections 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f3a0b9a2700 1 --2- 172.21.15.5:0/2541688669 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406f280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/2541688669 >> 172.21.15.5:0/2541688669 conn(0x7f3a04099a50 msgr2=0x7f3a040d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/2541688669 shutdown_connections 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.586+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/2541688669 wait complete. 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a0b9a2700 1 Processor -- start 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a0b9a2700 1 -- start start 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a0b9a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a0406f7c0 0x7f3a040c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a0b9a2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a0b9a2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0415db00 0x7f3a0406e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a0b9a2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040c74d0 con 0x7f3a0415db00 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a0b9a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040c8050 con 0x7f3a0406f7c0 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a0b9a2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040c9b60 con 0x7f3a040cc590 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.588+0000 7f3a09f3f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0415db00 0x7f3a0406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a09f3f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0415db00 0x7f3a0406e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51482/0 (socket says 172.21.15.5:51482) 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a09f3f700 1 -- 172.21.15.5:0/317826545 learned_addr learned my addr 172.21.15.5:0/317826545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a0973e700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a0406f7c0 0x7f3a040c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a09f3f700 1 -- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 msgr2=0x7f3a0406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a09f3f700 1 -- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a0406f7c0 msgr2=0x7f3a040c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a0406f7c0 0x7f3a040c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a09f3f700 1 -- 172.21.15.5:0/317826545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40094a0 con 0x7f3a0415db00 2024-05-08T23:25:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0415db00 0x7f3a0406e760 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00cc80 tx=0x7f39fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f39f9ffb700 1 -- 172.21.15.5:0/317826545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc003c00 con 0x7f3a0415db00 2024-05-08T23:25:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f39f9ffb700 1 -- 172.21.15.5:0/317826545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc003d60 con 0x7f3a0415db00 2024-05-08T23:25:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/317826545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a040c7720 con 0x7f3a0415db00 2024-05-08T23:25:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.589+0000 7f39f9ffb700 1 -- 172.21.15.5:0/317826545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00aac0 con 0x7f3a0415db00 2024-05-08T23:25:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.590+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/317826545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a040c7ce0 con 0x7f3a0415db00 2024-05-08T23:25:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.591+0000 7f39f9ffb700 1 -- 172.21.15.5:0/317826545 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f39fc00ac20 con 0x7f3a0415db00 2024-05-08T23:25:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.591+0000 7f39f9ffb700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f39e806ac90 0x7f39e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.591+0000 7f39f9ffb700 1 -- 172.21.15.5:0/317826545 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39fc0892e0 con 0x7f3a0415db00 2024-05-08T23:25:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.592+0000 7f3a0973e700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f39e806ac90 0x7f39e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.592+0000 7f3a0973e700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f39e806ac90 0x7f39e806d150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3a00007fd0 tx=0x7f3a00007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.593+0000 7f3a17b5cb80 1 Processor -- start 2024-05-08T23:25:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a17b5cb80 1 -- start start 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d3b7a0 0x5612b0d14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d16370 0x5612b0d190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d42dd0 0x5612b0d43250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b0c6f150 con 0x5612b0d42dd0 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b0c6cb10 con 0x5612b0d16370 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612b0c68400 con 0x5612b0d3b7a0 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a09f3f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d42dd0 0x5612b0d43250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a09f3f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d42dd0 0x5612b0d43250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51496/0 (socket says 172.21.15.5:51496) 2024-05-08T23:25:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a09f3f700 1 -- 172.21.15.5:0/4074288686 learned_addr learned my addr 172.21.15.5:0/4074288686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a0973e700 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d3b7a0 0x5612b0d14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.594+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d16370 0x5612b0d190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f3a09f3f700 1 -- 172.21.15.5:0/4074288686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d3b7a0 msgr2=0x5612b0d14110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d3b7a0 0x5612b0d14110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f3a09f3f700 1 -- 172.21.15.5:0/4074288686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d16370 msgr2=0x5612b0d190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d16370 0x5612b0d190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f3a09f3f700 1 -- 172.21.15.5:0/4074288686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612b0ae93b0 con 0x5612b0d42dd0 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d42dd0 0x5612b0d43250 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0878c0 tx=0x7f39fc057c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae340398f90678a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f39eeffd700 1 -- 172.21.15.5:0/4074288686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc05c040 con 0x5612b0d42dd0 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f39eeffd700 1 -- 172.21.15.5:0/4074288686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc060600 con 0x5612b0d42dd0 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.595+0000 7f39eeffd700 1 -- 172.21.15.5:0/4074288686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc019de0 con 0x5612b0d42dd0 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/4074288686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d42dd0 msgr2=0x5612b0d43250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d42dd0 0x5612b0d43250 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0878c0 tx=0x7f39fc057c40 comp rx=0 tx=0).stop 2024-05-08T23:25:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/4074288686 shutdown_connections 2024-05-08T23:25:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d42dd0 0x5612b0d43250 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d3b7a0 0x5612b0d14110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/4074288686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d16370 0x5612b0d190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/4074288686 >> 172.21.15.5:0/4074288686 conn(0x5612b0c76210 msgr2=0x5612b0d5ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/4074288686 shutdown_connections 2024-05-08T23:25:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/4074288686 wait complete. 2024-05-08T23:25:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.596+0000 7f3a17b5cb80 1 -- start start 2024-05-08T23:25:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.597+0000 7f3a17b5cb80 1 Processor -- start 2024-05-08T23:25:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d16370 0x5612b0d6cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d3b7a0 0x5612b0d70f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d42dd0 0x5612b0d11ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b0c04560 con 0x5612b0d16370 2024-05-08T23:25:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b0c04200 con 0x5612b0d3b7a0 2024-05-08T23:25:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612b0aeb6a0 con 0x5612b0d42dd0 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a17b5cb80 10 client.0 _ll_register_callbacks cb 0x5612b0c0db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d16370 0x5612b0d6cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d16370 0x5612b0d6cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51508/0 (socket says 172.21.15.5:51508) 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 -- 172.21.15.5:0/2123754003 learned_addr learned my addr 172.21.15.5:0/2123754003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d42dd0 0x5612b0d11ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d3b7a0 0x5612b0d70f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 -- 172.21.15.5:0/2123754003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d42dd0 msgr2=0x5612b0d11ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d42dd0 0x5612b0d11ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 -- 172.21.15.5:0/2123754003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d3b7a0 msgr2=0x5612b0d70f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d3b7a0 0x5612b0d70f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 -- 172.21.15.5:0/2123754003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a040c7ce0 con 0x5612b0d16370 2024-05-08T23:25:44.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a0973e700 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d16370 0x5612b0d6cb70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3a00005fd0 tx=0x7f3a00001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e65d1e4440ed16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f39e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f39e3fff700 1 -- 172.21.15.5:0/2123754003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00026030 con 0x5612b0d16370 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f39e3fff700 1 -- 172.21.15.5:0/2123754003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a00027070 con 0x5612b0d16370 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.598+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2123754003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612b0bc1270 con 0x5612b0d16370 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f39e3fff700 1 -- 172.21.15.5:0/2123754003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a00013700 con 0x5612b0d16370 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a17b5cb80 10 client.6924 fetch_fsmap learned FSMap version 59 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2123754003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x5612b0a95b60 con 0x5612b0d16370 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f39e3fff700 1 -- 172.21.15.5:0/2123754003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a00031020 con 0x5612b0d16370 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f39e3fff700 1 -- 172.21.15.5:0/2123754003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f3a00013860 con 0x5612b0d16370 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a17b5cb80 10 client.6924 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a17b5cb80 10 client.6924 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a17b5cb80 20 client.6924 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:44.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5612b0d44fc0 0x5612b0d453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a17b5cb80 4 client.6924 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2123754003 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612b0ce77e0 con 0x5612b0d44fc0 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5612b0d44fc0 0x5612b0d453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5612b0d44fc0 0x5612b0d453d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4edc489584d2d334 server_cookie=af364095e8be2e46 in_seq=0 out_seq=0 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.599+0000 7f39e3fff700 10 client.6924 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f39e3fff700 1 -- 172.21.15.5:0/2123754003 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612b0ce77e0 con 0x5612b0d44fc0 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f39e3fff700 10 client.6924 handle_command_reply: tid=0 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 client.6924 shutdown 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2123754003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d16370 msgr2=0x5612b0d6cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d16370 0x5612b0d6cb70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3a00005fd0 tx=0x7f3a00001150 comp rx=0 tx=0).stop 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2123754003 shutdown_connections 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d42dd0 0x5612b0d11ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d16370 0x5612b0d6cb70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5612b0d44fc0 0x5612b0d453d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2123754003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d3b7a0 0x5612b0d70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f39e3fff700 0 client.6924 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2123754003 >> 172.21.15.5:0/2123754003 conn(0x5612b0c76210 msgr2=0x5612b0d5e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2123754003 shutdown_connections 2024-05-08T23:25:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.600+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2123754003 wait complete. 2024-05-08T23:25:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.601+0000 7f3a17b5cb80 20 client.6924 trim_cache size 0 max 16384 2024-05-08T23:25:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.615+0000 7f3a17b5cb80 1 Processor -- start 2024-05-08T23:25:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.615+0000 7f3a17b5cb80 1 -- start start 2024-05-08T23:25:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.615+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 0x5612b0d3aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.615+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.615+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 0x5612b0d18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.615+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b0d71430 con 0x5612b0d3b7a0 2024-05-08T23:25:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.615+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b0d0f280 con 0x5612b0d42dd0 2024-05-08T23:25:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.615+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612b0d70220 con 0x5612b0d16370 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d135c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51510/0 (socket says 172.21.15.5:51510) 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 -- 172.21.15.5:0/3851884235 learned_addr learned my addr 172.21.15.5:0/3851884235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 0x5612b0d18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a0973e700 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 0x5612b0d3aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 -- 172.21.15.5:0/3851884235 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 msgr2=0x5612b0d3aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 0x5612b0d3aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 -- 172.21.15.5:0/3851884235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 msgr2=0x5612b0d18ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 0x5612b0d18ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 -- 172.21.15.5:0/3851884235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612b0c2e560 con 0x5612b0d3b7a0 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d135c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f39f40097c0 tx=0x7f39f4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bb2258c8ac8848 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f39ef7fe700 1 -- 172.21.15.5:0/3851884235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4011d00 con 0x5612b0d3b7a0 2024-05-08T23:25:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f39ef7fe700 1 -- 172.21.15.5:0/3851884235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f4024430 con 0x5612b0d3b7a0 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.616+0000 7f39ef7fe700 1 -- 172.21.15.5:0/3851884235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40247a0 con 0x5612b0d3b7a0 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/3851884235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 msgr2=0x5612b0d135c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d135c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f39f40097c0 tx=0x7f39f4008030 comp rx=0 tx=0).stop 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/3851884235 shutdown_connections 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 0x5612b0d18ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 0x5612b0d3aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/3851884235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d135c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/3851884235 >> 172.21.15.5:0/3851884235 conn(0x5612b0c76210 msgr2=0x5612b0d12970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/3851884235 shutdown_connections 2024-05-08T23:25:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.617+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/3851884235 wait complete. 2024-05-08T23:25:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.618+0000 7f3a17b5cb80 1 -- start start 2024-05-08T23:25:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.618+0000 7f3a17b5cb80 1 Processor -- start 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 0x5612b0d6f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a17b5cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 0x5612b0d19000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b0d61520 con 0x5612b0d3b7a0 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b0d6f900 con 0x5612b0d42dd0 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a17b5cb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612b0dc7af0 con 0x5612b0d16370 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a17b5cb80 10 client.0 _ll_register_callbacks cb 0x5612b0c0db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d18ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51522/0 (socket says 172.21.15.5:51522) 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 -- 172.21.15.5:0/2111118504 learned_addr learned my addr 172.21.15.5:0/2111118504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a09f3f700 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 0x5612b0d19000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a0973e700 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 0x5612b0d6f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 -- 172.21.15.5:0/2111118504 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 msgr2=0x5612b0d6f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 0x5612b0d6f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 -- 172.21.15.5:0/2111118504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 msgr2=0x5612b0d19000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 0x5612b0d19000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 -- 172.21.15.5:0/2111118504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0406b000 con 0x5612b0d3b7a0 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f3a08f3d700 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d18ac0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009ed0 tx=0x7f39f400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6447eb5bca3b8a0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.619+0000 7f39e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f39e3fff700 1 -- 172.21.15.5:0/2111118504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40042c0 con 0x5612b0d3b7a0 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f39e3fff700 1 -- 172.21.15.5:0/2111118504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39f401f040 con 0x5612b0d3b7a0 2024-05-08T23:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2111118504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39fc060600 con 0x5612b0d3b7a0 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f39e3fff700 1 -- 172.21.15.5:0/2111118504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f4004420 con 0x5612b0d3b7a0 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 10 client.6930 fetch_fsmap learned FSMap version 59 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2111118504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x5612b0aedcd0 con 0x5612b0d3b7a0 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f39e3fff700 1 -- 172.21.15.5:0/2111118504 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc060600 con 0x5612b0d3b7a0 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f39e3fff700 1 -- 172.21.15.5:0/2111118504 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f39f40036a0 con 0x5612b0d3b7a0 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 10 client.6930 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 10 client.6930 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 20 client.6930 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5612b0dc4490 0x5612b0d5dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 4 client.6930 mds_command: new command op to 6525 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2111118504 --> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612b0c60ec0 con 0x5612b0dc4490 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.620+0000 7f3a0973e700 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5612b0dc4490 0x5612b0d5dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a0973e700 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5612b0dc4490 0x5612b0d5dee0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dddb92f07dacff14 server_cookie=e48dc4759c86a70a in_seq=0 out_seq=0 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f39e3fff700 10 client.6930 ms_handle_connect on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f39e3fff700 1 -- 172.21.15.5:0/2111118504 <== mds.0 v2:172.21.15.5:6834/1169758885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+269 (crc 0 0 0) 0x5612b0c60ec0 con 0x5612b0dc4490 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f39e3fff700 10 client.6930 handle_command_reply: tid=0 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 client.6930 shutdown 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2111118504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 msgr2=0x5612b0d18ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d18ac0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009ed0 tx=0x7f39f400e000 comp rx=0 tx=0).stop 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2111118504 shutdown_connections 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.5:6834/1169758885,v1:172.21.15.5:6835/1169758885] conn(0x5612b0dc4490 0x5612b0d5dee0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612b0d42dd0 0x5612b0d19000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f39e3fff700 0 client.6930 ms_handle_reset on v2:172.21.15.5:6834/1169758885 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5612b0d16370 0x5612b0d6f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 --2- 172.21.15.5:0/2111118504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612b0d3b7a0 0x5612b0d18ac0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2111118504 >> 172.21.15.5:0/2111118504 conn(0x5612b0c76210 msgr2=0x5612b0d13fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2111118504 shutdown_connections 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 1 -- 172.21.15.5:0/2111118504 wait complete. 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.621+0000 7f3a17b5cb80 20 client.6930 trim_cache size 0 max 16384 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 98427953, 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627778, 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3" 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:25:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:25:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-08T23:25:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 102685790, 2024-05-08T23:25:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627779, 2024-05-08T23:25:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3/d4" 2024-05-08T23:25:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:25:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f39e806ac90 msgr2=0x7f39e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f39e806ac90 0x7f39e806d150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3a00007fd0 tx=0x7f3a00007dc0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/317826545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0415db00 msgr2=0x7f3a0406e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0415db00 0x7f3a0406e760 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00cc80 tx=0x7f39fc00cf90 comp rx=0 tx=0).stop 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/317826545 shutdown_connections 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f39e806ac90 0x7f39e806d150 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0415db00 0x7f3a0406e760 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a0406f7c0 0x7f3a040c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 --2- 172.21.15.5:0/317826545 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a040cc590 0x7f3a0406e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/317826545 >> 172.21.15.5:0/317826545 conn(0x7f3a04099a50 msgr2=0x7f3a040b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/317826545 shutdown_connections 2024-05-08T23:25:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.622+0000 7f3a0b9a2700 1 -- 172.21.15.5:0/317826545 wait complete. 2024-05-08T23:25:44.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 damage ls 2024-05-08T23:25:44.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.773+0000 7f82bdfad700 1 Processor -- start 2024-05-08T23:25:44.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.773+0000 7f82bdfad700 1 -- start start 2024-05-08T23:25:44.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.774+0000 7f82bdfad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b815ba10 0x7f82b80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.774+0000 7f82bdfad700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b815f5f0 con 0x7f82b80cfde0 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.774+0000 7f82bdfad700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b815f8d0 con 0x7f82b806b070 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.774+0000 7f82bdfad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b815f3e0 con 0x7f82b815ba10 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.774+0000 7f82b6ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f82b80cfde0 0x7f82b806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33514/0 (socket says 172.21.15.5:33514) 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b6ffd700 1 -- 172.21.15.5:0/1710498598 learned_addr learned my addr 172.21.15.5:0/1710498598 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b815ba10 0x7f82b80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270879207 0 0) 0x7f82b815f5f0 con 0x7f82b80cfde0 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82a00035b0 con 0x7f82b80cfde0 2024-05-08T23:25:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1979917759 0 0) 0x7f82a00035b0 con 0x7f82b80cfde0 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b815ba10 msgr2=0x7f82b80cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b815ba10 0x7f82b80cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 >> v1:172.21.15.49:6789/0 conn(0x7f82b806b070 legacy=0x7f82b80ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b815fbb0 con 0x7f82b80cfde0 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f82ac001670 con 0x7f82b80cfde0 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 >> v1:172.21.15.5:6789/0 conn(0x7f82b80cfde0 legacy=0x7f82b806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a0003a30 0x7f82a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b815ba10 0x7f82a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0004d90 0x7f82a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b815f3e0 con 0x7f82a0003a30 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a000bac0 con 0x7f82a0004d90 2024-05-08T23:25:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a000bc80 con 0x7f82b815ba10 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a0003a30 0x7f82a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b815ba10 0x7f82a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.775+0000 7f82b7fff700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0004d90 0x7f82a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b77fe700 1 -- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b815ba10 msgr2=0x7f82a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b815ba10 0x7f82a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b77fe700 1 -- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0004d90 msgr2=0x7f82a0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0004d90 0x7f82a0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b77fe700 1 -- 172.21.15.5:0/1710498598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00035b0 con 0x7f82a0003a30 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a0003a30 0x7f82a0003e50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009b30 tx=0x7f82a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837e55dd28773b4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800e6c0 con 0x7f82a0003a30 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800e820 con 0x7f82a0003a30 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b67fc700 1 -- 172.21.15.5:0/1710498598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800eb90 con 0x7f82a0003a30 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82bdfad700 1 -- 172.21.15.5:0/1710498598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a0003a30 msgr2=0x7f82a0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82bdfad700 1 --2- 172.21.15.5:0/1710498598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a0003a30 0x7f82a0003e50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009b30 tx=0x7f82a0005880 comp rx=0 tx=0).stop 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.776+0000 7f82b7fff700 1 -- 172.21.15.5:0/1710498598 reap_dead start 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.777+0000 7f82bdfad700 1 -- 172.21.15.5:0/1710498598 shutdown_connections 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.777+0000 7f82bdfad700 1 -- 172.21.15.5:0/1710498598 >> 172.21.15.5:0/1710498598 conn(0x7f82b8099a50 msgr2=0x7f82b80d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.777+0000 7f82bdfad700 1 -- 172.21.15.5:0/1710498598 shutdown_connections 2024-05-08T23:25:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.777+0000 7f82bdfad700 1 -- 172.21.15.5:0/1710498598 wait complete. 2024-05-08T23:25:44.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.778+0000 7f82bdfad700 1 Processor -- start 2024-05-08T23:25:44.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.779+0000 7f82bdfad700 1 -- start start 2024-05-08T23:25:44.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82bdfad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0003a30 0x7f82b806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82bdfad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b80ca330 0x7f82b80b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82bdfad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82b815e9c0 0x7f82b80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82bdfad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80c3610 con 0x7f82b815e9c0 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82bdfad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80c3180 con 0x7f82a0003a30 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82bdfad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80c83d0 con 0x7f82b80ca330 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82b815e9c0 0x7f82b80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82b815e9c0 0x7f82b80b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51544/0 (socket says 172.21.15.5:51544) 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 -- 172.21.15.5:0/3746783383 learned_addr learned my addr 172.21.15.5:0/3746783383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b77fe700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0003a30 0x7f82b806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b80ca330 0x7f82b80b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 -- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b80ca330 msgr2=0x7f82b80b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b80ca330 0x7f82b80b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 -- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0003a30 msgr2=0x7f82b806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0003a30 0x7f82b806f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 -- 172.21.15.5:0/3746783383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b80c7e10 con 0x7f82b815e9c0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f82b7fff700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82b815e9c0 0x7f82b80b2170 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f82b8071a10 tx=0x7f82b000d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.780+0000 7f829ffff700 1 -- 172.21.15.5:0/3746783383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0003c10 con 0x7f82b815e9c0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.781+0000 7f829ffff700 1 -- 172.21.15.5:0/3746783383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b0003d90 con 0x7f82b815e9c0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.781+0000 7f82bdfad700 1 -- 172.21.15.5:0/3746783383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b80c80f0 con 0x7f82b815e9c0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.781+0000 7f829ffff700 1 -- 172.21.15.5:0/3746783383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000d960 con 0x7f82b815e9c0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.781+0000 7f82bdfad700 1 -- 172.21.15.5:0/3746783383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b80c7400 con 0x7f82b815e9c0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.782+0000 7f829ffff700 1 -- 172.21.15.5:0/3746783383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f82b000dac0 con 0x7f82b815e9c0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.782+0000 7f829ffff700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f82a406ac90 0x7f82a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.783+0000 7f829ffff700 1 -- 172.21.15.5:0/3746783383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82b0089420 con 0x7f82b815e9c0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.783+0000 7f82b77fe700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f82a406ac90 0x7f82a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.783+0000 7f82b77fe700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f82a406ac90 0x7f82a406d150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82a80090b0 tx=0x7f82a801f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.785+0000 7f82ca167b80 1 Processor -- start 2024-05-08T23:25:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.785+0000 7f82ca167b80 1 -- start start 2024-05-08T23:25:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x5628155cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 0x5628155f9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82ca167b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562815525150 con 0x5628155cc370 2024-05-08T23:25:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82ca167b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562815522b10 con 0x5628155f17a0 2024-05-08T23:25:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82ca167b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56281551e400 con 0x5628155f8dd0 2024-05-08T23:25:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x5628155cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x5628155cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51552/0 (socket says 172.21.15.5:51552) 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82b6ffd700 1 -- 172.21.15.5:0/138867214 learned_addr learned my addr 172.21.15.5:0/138867214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82b77fe700 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82b7fff700 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 0x5628155f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.786+0000 7f82b6ffd700 1 -- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 msgr2=0x5628155f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 0x5628155f9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f82b6ffd700 1 -- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 msgr2=0x5628155ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f82b6ffd700 1 -- 172.21.15.5:0/138867214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56281539f3b0 con 0x5628155cc370 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f82b77fe700 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x5628155cf0a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f82ac007d10 tx=0x7f82ac004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fc050c536fe4c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f829d7fa700 1 -- 172.21.15.5:0/138867214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac007250 con 0x5628155cc370 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f829d7fa700 1 -- 172.21.15.5:0/138867214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac0073b0 con 0x5628155cc370 2024-05-08T23:25:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.787+0000 7f829d7fa700 1 -- 172.21.15.5:0/138867214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac007740 con 0x5628155cc370 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 -- 172.21.15.5:0/138867214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 msgr2=0x5628155cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x5628155cf0a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f82ac007d10 tx=0x7f82ac004b50 comp rx=0 tx=0).stop 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 -- 172.21.15.5:0/138867214 shutdown_connections 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 0x5628155f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 --2- 172.21.15.5:0/138867214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x5628155cf0a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 -- 172.21.15.5:0/138867214 >> 172.21.15.5:0/138867214 conn(0x56281552c210 msgr2=0x562815613dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 -- 172.21.15.5:0/138867214 shutdown_connections 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 -- 172.21.15.5:0/138867214 wait complete. 2024-05-08T23:25:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.788+0000 7f82ca167b80 1 -- start start 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82ca167b80 1 Processor -- start 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 0x5628155ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f17a0 0x562815626940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f8dd0 0x562815626e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82ca167b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628154ba560 con 0x5628155f17a0 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82ca167b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628154ba200 con 0x5628155f8dd0 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82ca167b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628153a16a0 con 0x5628155cc370 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82ca167b80 10 client.0 _ll_register_callbacks cb 0x5628154c3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.789+0000 7f82b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f17a0 0x562815626940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f17a0 0x562815626940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51564/0 (socket says 172.21.15.5:51564) 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b6ffd700 1 -- 172.21.15.5:0/3147321937 learned_addr learned my addr 172.21.15.5:0/3147321937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b77fe700 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 0x5628155ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b7fff700 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f8dd0 0x562815626e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b6ffd700 1 -- 172.21.15.5:0/3147321937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 msgr2=0x5628155ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 0x5628155ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b6ffd700 1 -- 172.21.15.5:0/3147321937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f8dd0 msgr2=0x562815626e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f8dd0 0x562815626e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b6ffd700 1 -- 172.21.15.5:0/3147321937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b815fbb0 con 0x5628155f17a0 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f17a0 0x562815626940 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f82ac017040 tx=0x7f82ac001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806febf4e61aa176 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82967fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82967fc700 1 -- 172.21.15.5:0/3147321937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac007250 con 0x5628155f17a0 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82967fc700 1 -- 172.21.15.5:0/3147321937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82ac0015f0 con 0x5628155f17a0 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82ca167b80 1 -- 172.21.15.5:0/3147321937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562815477270 con 0x5628155f17a0 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82967fc700 1 -- 172.21.15.5:0/3147321937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82ac015d60 con 0x5628155f17a0 2024-05-08T23:25:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82ca167b80 10 client.6942 fetch_fsmap learned FSMap version 59 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.790+0000 7f82ca167b80 1 -- 172.21.15.5:0/3147321937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x56281534bb60 con 0x5628155f17a0 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82967fc700 1 -- 172.21.15.5:0/3147321937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562815477270 con 0x5628155f17a0 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82967fc700 1 -- 172.21.15.5:0/3147321937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f82ac012050 con 0x5628155f17a0 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82ca167b80 10 client.6942 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82ca167b80 10 client.6942 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.c 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82ca167b80 20 client.6942 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82ca167b80 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5628156273c0 0x562815619470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82ca167b80 4 client.6942 mds_command: new command op to 6516 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82ca167b80 1 -- 172.21.15.5:0/3147321937 --> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56281559d7e0 con 0x5628156273c0 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82b77fe700 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5628156273c0 0x562815619470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82b77fe700 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5628156273c0 0x562815619470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fde1f1d2821274e7 server_cookie=5419ef4694f19abe in_seq=0 out_seq=0 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.791+0000 7f82967fc700 10 client.6942 ms_handle_connect on v2:172.21.15.5:6832/2315721152 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82967fc700 1 -- 172.21.15.5:0/3147321937 <== mds.1 v2:172.21.15.5:6832/2315721152 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56281559d7e0 con 0x5628156273c0 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82967fc700 10 client.6942 handle_command_reply: tid=0 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 client.6942 shutdown 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 -- 172.21.15.5:0/3147321937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f17a0 msgr2=0x562815626940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f17a0 0x562815626940 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f82ac017040 tx=0x7f82ac001a00 comp rx=0 tx=0).stop 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 -- 172.21.15.5:0/3147321937 shutdown_connections 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f8dd0 0x562815626e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x5628156273c0 0x562815619470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 0x5628155ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 --2- 172.21.15.5:0/3147321937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f17a0 0x562815626940 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82967fc700 0 client.6942 ms_handle_reset on v2:172.21.15.5:6832/2315721152 2024-05-08T23:25:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 -- 172.21.15.5:0/3147321937 >> 172.21.15.5:0/3147321937 conn(0x56281552c210 msgr2=0x5628155ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 -- 172.21.15.5:0/3147321937 shutdown_connections 2024-05-08T23:25:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 1 -- 172.21.15.5:0/3147321937 wait complete. 2024-05-08T23:25:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.792+0000 7f82ca167b80 20 client.6942 trim_cache size 0 max 16384 2024-05-08T23:25:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.806+0000 7f82ca167b80 1 Processor -- start 2024-05-08T23:25:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.806+0000 7f82ca167b80 1 -- start start 2024-05-08T23:25:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.807+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 0x5628155ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.807+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.807+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f8dd0 0x5628155c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.807+0000 7f82ca167b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628155cacc0 con 0x5628155f8dd0 2024-05-08T23:25:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.807+0000 7f82ca167b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628155f7d00 con 0x5628155f17a0 2024-05-08T23:25:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.807+0000 7f82ca167b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628155f8820 con 0x5628155cc370 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.807+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f8dd0 0x5628155c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f8dd0 0x5628155c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51576/0 (socket says 172.21.15.5:51576) 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b7fff700 1 -- 172.21.15.5:0/1149996505 learned_addr learned my addr 172.21.15.5:0/1149996505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 0x5628155ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b7fff700 1 -- 172.21.15.5:0/1149996505 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 msgr2=0x5628155ef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b7fff700 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 0x5628155ef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b7fff700 1 -- 172.21.15.5:0/1149996505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 msgr2=0x5628155c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b7fff700 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b7fff700 1 -- 172.21.15.5:0/1149996505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628154e4560 con 0x5628155f8dd0 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f82b7fff700 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f8dd0 0x5628155c8220 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f82b00547c0 tx=0x7f82b0057e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a13ac8d9b195e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f829dffb700 1 -- 172.21.15.5:0/1149996505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b00582c0 con 0x5628155f8dd0 2024-05-08T23:25:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f829dffb700 1 -- 172.21.15.5:0/1149996505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b005c0f0 con 0x5628155f8dd0 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.808+0000 7f829dffb700 1 -- 172.21.15.5:0/1149996505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b00606b0 con 0x5628155f8dd0 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 -- 172.21.15.5:0/1149996505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f8dd0 msgr2=0x5628155c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f8dd0 0x5628155c8220 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f82b00547c0 tx=0x7f82b0057e80 comp rx=0 tx=0).stop 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 -- 172.21.15.5:0/1149996505 shutdown_connections 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155f8dd0 0x5628155c8220 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155cc370 0x5628155ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1149996505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 -- 172.21.15.5:0/1149996505 >> 172.21.15.5:0/1149996505 conn(0x56281552c210 msgr2=0x5628155f0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 -- 172.21.15.5:0/1149996505 shutdown_connections 2024-05-08T23:25:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.809+0000 7f82ca167b80 1 -- 172.21.15.5:0/1149996505 wait complete. 2024-05-08T23:25:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.810+0000 7f82ca167b80 1 -- start start 2024-05-08T23:25:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.810+0000 7f82ca167b80 1 Processor -- start 2024-05-08T23:25:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x562815627820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82ca167b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 0x5628155c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82ca167b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628155f6d30 con 0x5628155cc370 2024-05-08T23:25:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82ca167b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281567f970 con 0x5628155f17a0 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82ca167b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56281567fbc0 con 0x5628155f8dd0 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82ca167b80 10 client.0 _ll_register_callbacks cb 0x5628154c3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82b77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x562815627820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82b77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x562815627820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51590/0 (socket says 172.21.15.5:51590) 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.811+0000 7f82b77fe700 1 -- 172.21.15.5:0/1896741834 learned_addr learned my addr 172.21.15.5:0/1896741834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b7fff700 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 0x5628155c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b77fe700 1 -- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 msgr2=0x5628155c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 0x5628155c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b77fe700 1 -- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 msgr2=0x5628155c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b77fe700 1 -- 172.21.15.5:0/1896741834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b80c7400 con 0x5628155cc370 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82b77fe700 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x562815627820 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f82a8016000 tx=0x7f82a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ad284d93b10eae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82967fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82967fc700 1 -- 172.21.15.5:0/1896741834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8022040 con 0x5628155cc370 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82967fc700 1 -- 172.21.15.5:0/1896741834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82a8004250 con 0x5628155cc370 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82ca167b80 1 -- 172.21.15.5:0/1896741834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82ac0073b0 con 0x5628155cc370 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82967fc700 1 -- 172.21.15.5:0/1896741834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82a8004410 con 0x5628155cc370 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82ca167b80 10 client.6948 fetch_fsmap learned FSMap version 59 2024-05-08T23:25:44.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82ca167b80 1 -- 172.21.15.5:0/1896741834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x5628153a3cd0 con 0x5628155cc370 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.812+0000 7f82967fc700 1 -- 172.21.15.5:0/1896741834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac0073b0 con 0x5628155cc370 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82967fc700 1 -- 172.21.15.5:0/1896741834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f82a8011070 con 0x5628155cc370 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 10 client.6948 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 10 client.6948 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.c 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 20 client.6948 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x562815627d60 0x5628155ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 4 client.6948 mds_command: new command op to 6516 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 -- 172.21.15.5:0/1896741834 --> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562815516ec0 con 0x562815627d60 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x562815627d60 0x5628155ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82b6ffd700 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x562815627d60 0x5628155ce8d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a50e623a1275f77 server_cookie=b3202c8b3e1b07c1 in_seq=0 out_seq=0 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82967fc700 10 client.6948 ms_handle_connect on v2:172.21.15.5:6832/2315721152 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82967fc700 1 -- 172.21.15.5:0/1896741834 <== mds.1 v2:172.21.15.5:6832/2315721152 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x562815516ec0 con 0x562815627d60 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82967fc700 10 client.6948 handle_command_reply: tid=0 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 client.6948 shutdown 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 -- 172.21.15.5:0/1896741834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 msgr2=0x562815627820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x562815627820 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f82a8016000 tx=0x7f82a8004750 comp rx=0 tx=0).stop 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 -- 172.21.15.5:0/1896741834 shutdown_connections 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.5:6832/2315721152,v1:172.21.15.5:6833/2315721152] conn(0x562815627d60 0x5628155ce8d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5628155f8dd0 0x5628155c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628155cc370 0x562815627820 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 --2- 172.21.15.5:0/1896741834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628155f17a0 0x5628155c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82967fc700 0 client.6948 ms_handle_reset on v2:172.21.15.5:6832/2315721152 2024-05-08T23:25:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.813+0000 7f82ca167b80 1 -- 172.21.15.5:0/1896741834 >> 172.21.15.5:0/1896741834 conn(0x56281552c210 msgr2=0x5628155f0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82ca167b80 1 -- 172.21.15.5:0/1896741834 shutdown_connections 2024-05-08T23:25:44.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82ca167b80 1 -- 172.21.15.5:0/1896741834 wait complete. 2024-05-08T23:25:44.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82ca167b80 20 client.6948 trim_cache size 0 max 16384 2024-05-08T23:25:44.822 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:25:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:25:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 1328917570, 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627780, 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3/d4/d5" 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4181180641, 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627781, 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3/d4/d5/d6" 2024-05-08T23:25:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:25:44.824 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 -- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f82a406ac90 msgr2=0x7f82a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f82a406ac90 0x7f82a406d150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82a80090b0 tx=0x7f82a801f040 comp rx=0 tx=0).stop 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 -- 172.21.15.5:0/3746783383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82b815e9c0 msgr2=0x7f82b80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82b815e9c0 0x7f82b80b2170 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f82b8071a10 tx=0x7f82b000d150 comp rx=0 tx=0).stop 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 -- 172.21.15.5:0/3746783383 shutdown_connections 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f82a406ac90 0x7f82a406d150 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82b815e9c0 0x7f82b80b2170 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82a0003a30 0x7f82b806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 --2- 172.21.15.5:0/3746783383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f82b80ca330 0x7f82b80b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 -- 172.21.15.5:0/3746783383 >> 172.21.15.5:0/3746783383 conn(0x7f82b8099a50 msgr2=0x7f82b80b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 -- 172.21.15.5:0/3746783383 shutdown_connections 2024-05-08T23:25:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.814+0000 7f82bdfad700 1 -- 172.21.15.5:0/3746783383 wait complete. 2024-05-08T23:25:44.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 damage ls 2024-05-08T23:25:44.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.965+0000 7fb74c7d7700 1 Processor -- start 2024-05-08T23:25:44.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.965+0000 7fb74c7d7700 1 -- start start 2024-05-08T23:25:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74c7d7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb744141df0 0x7fb7440b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74c7d7700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7440afa40 0x7fb74415fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74c7d7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 0x7fb7440d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74c7d7700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7440d0d10 con 0x7fb744141df0 2024-05-08T23:25:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74c7d7700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb744160360 con 0x7fb7440afa40 2024-05-08T23:25:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74c7d7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440aecf0 con 0x7fb7440d0350 2024-05-08T23:25:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74a573700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb744141df0 0x7fb7440b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74a573700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb744141df0 0x7fb7440b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51602/0 (socket says 172.21.15.5:51602) 2024-05-08T23:25:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb749d72700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7440afa40 0x7fb74415fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74a573700 1 -- 172.21.15.5:0/1716728096 learned_addr learned my addr 172.21.15.5:0/1716728096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.966+0000 7fb74ad74700 1 --2- 172.21.15.5:0/1716728096 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 0x7fb7440d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb74a573700 1 -- 172.21.15.5:0/1716728096 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 msgr2=0x7fb7440d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb74a573700 1 --2- 172.21.15.5:0/1716728096 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 0x7fb7440d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb74a573700 1 -- 172.21.15.5:0/1716728096 >> v2:172.21.15.49:3300/0 conn(0x7fb7440afa40 msgr2=0x7fb74415fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb74a573700 1 --2- 172.21.15.5:0/1716728096 >> v2:172.21.15.49:3300/0 conn(0x7fb7440afa40 0x7fb74415fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb74a573700 1 -- 172.21.15.5:0/1716728096 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440aef60 con 0x7fb744141df0 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb749d72700 1 --2- 172.21.15.5:0/1716728096 >> v2:172.21.15.49:3300/0 conn(0x7fb7440afa40 0x7fb74415fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb74a573700 1 --2- 172.21.15.5:0/1716728096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b4e80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb74406dbe0 tx=0x7fb740009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c30be4f977a51b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb749571700 1 -- 172.21.15.5:0/1716728096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740011830 con 0x7fb744141df0 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb749571700 1 -- 172.21.15.5:0/1716728096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740011990 con 0x7fb744141df0 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.967+0000 7fb749571700 1 -- 172.21.15.5:0/1716728096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740011d20 con 0x7fb744141df0 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 -- 172.21.15.5:0/1716728096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 msgr2=0x7fb7440b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/1716728096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b4e80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb74406dbe0 tx=0x7fb740009bd0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 -- 172.21.15.5:0/1716728096 shutdown_connections 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/1716728096 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 0x7fb7440d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/1716728096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b4e80 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/1716728096 >> v2:172.21.15.49:3300/0 conn(0x7fb7440afa40 0x7fb74415fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 -- 172.21.15.5:0/1716728096 >> 172.21.15.5:0/1716728096 conn(0x7fb7440b5810 msgr2=0x7fb7440b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 -- 172.21.15.5:0/1716728096 shutdown_connections 2024-05-08T23:25:44.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.968+0000 7fb74c7d7700 1 -- 172.21.15.5:0/1716728096 wait complete. 2024-05-08T23:25:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.969+0000 7fb74c7d7700 1 Processor -- start 2024-05-08T23:25:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.969+0000 7fb74c7d7700 1 -- start start 2024-05-08T23:25:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74c7d7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7440afa40 0x7fb7440ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74c7d7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 0x7fb7440cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74c7d7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74c7d7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74415d8e0 con 0x7fb744141df0 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74c7d7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440cc150 con 0x7fb7440afa40 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74c7d7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74415cfb0 con 0x7fb7440d0350 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74ad74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74ad74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51618/0 (socket says 172.21.15.5:51618) 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74ad74700 1 -- 172.21.15.5:0/3410494569 learned_addr learned my addr 172.21.15.5:0/3410494569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74a573700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7440afa40 0x7fb7440ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb749d72700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 0x7fb7440cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74ad74700 1 -- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 msgr2=0x7fb7440cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74ad74700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 0x7fb7440cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74ad74700 1 -- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7440afa40 msgr2=0x7fb7440ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74ad74700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7440afa40 0x7fb7440ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.970+0000 7fb74ad74700 1 -- 172.21.15.5:0/3410494569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340094a0 con 0x7fb744141df0 2024-05-08T23:25:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.971+0000 7fb74ad74700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b29f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00efc0 tx=0x7fb73c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.971+0000 7fb73affd700 1 -- 172.21.15.5:0/3410494569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c00cd40 con 0x7fb744141df0 2024-05-08T23:25:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.971+0000 7fb73affd700 1 -- 172.21.15.5:0/3410494569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c00f040 con 0x7fb744141df0 2024-05-08T23:25:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.971+0000 7fb74c7d7700 1 -- 172.21.15.5:0/3410494569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb740009550 con 0x7fb744141df0 2024-05-08T23:25:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.971+0000 7fb73affd700 1 -- 172.21.15.5:0/3410494569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c0146c0 con 0x7fb744141df0 2024-05-08T23:25:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.971+0000 7fb74c7d7700 1 -- 172.21.15.5:0/3410494569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74415d570 con 0x7fb744141df0 2024-05-08T23:25:44.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.975+0000 7fb758991b80 1 Processor -- start 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.975+0000 7fb73affd700 1 -- 172.21.15.5:0/3410494569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb73c014820 con 0x7fb744141df0 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.976+0000 7fb758991b80 1 -- start start 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.976+0000 7fb73affd700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb72806ac90 0x7fb72806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.976+0000 7fb73affd700 1 -- 172.21.15.5:0/3410494569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb73c089c00 con 0x7fb744141df0 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.976+0000 7fb749d72700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb72806ac90 0x7fb72806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb758991b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d697a0 0x555e69d42110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb758991b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d44370 0x555e69d470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb758991b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d71250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb758991b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69c9d150 con 0x555e69d70dd0 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb758991b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69c9ab10 con 0x555e69d697a0 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb758991b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e69c96400 con 0x555e69d44370 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d71250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d71250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51624/0 (socket says 172.21.15.5:51624) 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb749d72700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d44370 0x555e69d470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74a573700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d697a0 0x555e69d42110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 -- 172.21.15.5:0/1646983171 learned_addr learned my addr 172.21.15.5:0/1646983171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb749d72700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb72806ac90 0x7fb72806d150 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb74415e840 tx=0x7fb734008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 -- 172.21.15.5:0/1646983171 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d44370 msgr2=0x555e69d470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 --2- 172.21.15.5:0/1646983171 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d44370 0x555e69d470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 -- 172.21.15.5:0/1646983171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d697a0 msgr2=0x555e69d42110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 --2- 172.21.15.5:0/1646983171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d697a0 0x555e69d42110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 -- 172.21.15.5:0/1646983171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e69b173b0 con 0x555e69d70dd0 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.977+0000 7fb74ad74700 1 --2- 172.21.15.5:0/1646983171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d71250 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb73c000f80 tx=0x7fb73c00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada54827c5b5d139 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb727fff700 1 -- 172.21.15.5:0/1646983171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c058b30 con 0x555e69d70dd0 2024-05-08T23:25:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb727fff700 1 -- 172.21.15.5:0/1646983171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c05c960 con 0x555e69d70dd0 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb727fff700 1 -- 172.21.15.5:0/1646983171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c060f20 con 0x555e69d70dd0 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 -- 172.21.15.5:0/1646983171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 msgr2=0x555e69d71250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 --2- 172.21.15.5:0/1646983171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d71250 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb73c000f80 tx=0x7fb73c00cea0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 -- 172.21.15.5:0/1646983171 shutdown_connections 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 --2- 172.21.15.5:0/1646983171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d71250 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 --2- 172.21.15.5:0/1646983171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d697a0 0x555e69d42110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 --2- 172.21.15.5:0/1646983171 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d44370 0x555e69d470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 -- 172.21.15.5:0/1646983171 >> 172.21.15.5:0/1646983171 conn(0x555e69ca4210 msgr2=0x555e69d8bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 -- 172.21.15.5:0/1646983171 shutdown_connections 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.978+0000 7fb758991b80 1 -- 172.21.15.5:0/1646983171 wait complete. 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.979+0000 7fb758991b80 1 -- start start 2024-05-08T23:25:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.979+0000 7fb758991b80 1 Processor -- start 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb758991b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69d8c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb758991b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d697a0 0x555e69d9b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb758991b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d70dd0 0x555e69d8d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb758991b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69c32560 con 0x555e69d697a0 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb758991b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69c32200 con 0x555e69d44370 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb758991b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e69b196a0 con 0x555e69d70dd0 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb758991b80 10 client.0 _ll_register_callbacks cb 0x555e69c3bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb749d72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d697a0 0x555e69d9b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb749d72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d697a0 0x555e69d9b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51630/0 (socket says 172.21.15.5:51630) 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb749d72700 1 -- 172.21.15.5:0/1755883964 learned_addr learned my addr 172.21.15.5:0/1755883964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb74a573700 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69d8c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.980+0000 7fb74ad74700 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d70dd0 0x555e69d8d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb749d72700 1 -- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d70dd0 msgr2=0x555e69d8d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb749d72700 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d70dd0 0x555e69d8d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb749d72700 1 -- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 msgr2=0x555e69d8c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb749d72700 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69d8c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb749d72700 1 -- 172.21.15.5:0/1755883964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74415d570 con 0x555e69d697a0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb74ad74700 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d70dd0 0x555e69d8d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb749d72700 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d697a0 0x555e69d9b040 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb7340157f0 tx=0x7fb734004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8979070d2b9a0950 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb7257fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb7257fa700 1 -- 172.21.15.5:0/1755883964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734016970 con 0x555e69d697a0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb7257fa700 1 -- 172.21.15.5:0/1755883964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb734016ad0 con 0x555e69d697a0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb758991b80 1 -- 172.21.15.5:0/1755883964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e69bef270 con 0x555e69d697a0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb7257fa700 1 -- 172.21.15.5:0/1755883964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb734016c90 con 0x555e69d697a0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb758991b80 10 client.6960 fetch_fsmap learned FSMap version 59 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb758991b80 1 -- 172.21.15.5:0/1755883964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x555e69ac3b60 con 0x555e69d697a0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.981+0000 7fb7257fa700 1 -- 172.21.15.5:0/1755883964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e69bef270 con 0x555e69d697a0 2024-05-08T23:25:44.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb7257fa700 1 -- 172.21.15.5:0/1755883964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fb73402b430 con 0x555e69d697a0 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb758991b80 10 client.6960 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb758991b80 10 client.6960 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.d 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb758991b80 20 client.6960 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb758991b80 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x555e69d8e7b0 0x555e69d8ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb758991b80 4 client.6960 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb758991b80 1 -- 172.21.15.5:0/1755883964 --> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e69d157e0 con 0x555e69d8e7b0 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb74a573700 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x555e69d8e7b0 0x555e69d8ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb7257fa700 10 client.6960 ms_handle_connect on v2:172.21.15.49:6834/3197377913 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.982+0000 7fb74a573700 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x555e69d8e7b0 0x555e69d8ebc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc2b6e9b1e2b6040 server_cookie=33ee922317bedfaf in_seq=0 out_seq=0 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb7257fa700 1 -- 172.21.15.5:0/1755883964 <== mds.2 v2:172.21.15.49:6834/3197377913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e69d157e0 con 0x555e69d8e7b0 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb7257fa700 10 client.6960 handle_command_reply: tid=0 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 client.6960 shutdown 2024-05-08T23:25:44.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 -- 172.21.15.5:0/1755883964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d697a0 msgr2=0x555e69d9b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d697a0 0x555e69d9b040 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb7340157f0 tx=0x7fb734004750 comp rx=0 tx=0).stop 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 -- 172.21.15.5:0/1755883964 shutdown_connections 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x555e69d8e7b0 0x555e69d8ebc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d70dd0 0x555e69d8d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69d8c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 --2- 172.21.15.5:0/1755883964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d697a0 0x555e69d9b040 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb7257fa700 0 client.6960 ms_handle_reset on v2:172.21.15.49:6834/3197377913 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 -- 172.21.15.5:0/1755883964 >> 172.21.15.5:0/1755883964 conn(0x555e69ca4210 msgr2=0x555e69d704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 -- 172.21.15.5:0/1755883964 shutdown_connections 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 1 -- 172.21.15.5:0/1755883964 wait complete. 2024-05-08T23:25:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.983+0000 7fb758991b80 20 client.6960 trim_cache size 0 max 16384 2024-05-08T23:25:44.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb758991b80 1 Processor -- start 2024-05-08T23:25:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb758991b80 1 -- start start 2024-05-08T23:25:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb758991b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69dadeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb758991b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d8dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb758991b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d8e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb758991b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69d42e90 con 0x555e69d70dd0 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb758991b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69d8eee0 con 0x555e69d44370 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb758991b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e69d41ad0 con 0x555e69d697a0 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.998+0000 7fb74ad74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d8e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74ad74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d8e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51636/0 (socket says 172.21.15.5:51636) 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74ad74700 1 -- 172.21.15.5:0/835234613 learned_addr learned my addr 172.21.15.5:0/835234613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74a573700 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69dadeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb749d72700 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d8dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74ad74700 1 -- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 msgr2=0x555e69d8dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74ad74700 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d8dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74ad74700 1 -- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 msgr2=0x555e69dadeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74ad74700 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69dadeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74ad74700 1 -- 172.21.15.5:0/835234613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e69c5c560 con 0x555e69d70dd0 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74a573700 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69dadeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb749d72700 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d8dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb74ad74700 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d8e210 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00d010 tx=0x7fb73c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419fe14fc8841964 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb738ff9700 1 -- 172.21.15.5:0/835234613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c058670 con 0x555e69d70dd0 2024-05-08T23:25:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb738ff9700 1 -- 172.21.15.5:0/835234613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c060f20 con 0x555e69d70dd0 2024-05-08T23:25:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:44.999+0000 7fb738ff9700 1 -- 172.21.15.5:0/835234613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c058b30 con 0x555e69d70dd0 2024-05-08T23:25:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 -- 172.21.15.5:0/835234613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 msgr2=0x555e69d8e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d8e210 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00d010 tx=0x7fb73c01f000 comp rx=0 tx=0).stop 2024-05-08T23:25:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 -- 172.21.15.5:0/835234613 shutdown_connections 2024-05-08T23:25:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d70dd0 0x555e69d8e210 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d44370 0x555e69dadeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 --2- 172.21.15.5:0/835234613 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d8dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 -- 172.21.15.5:0/835234613 >> 172.21.15.5:0/835234613 conn(0x555e69ca4210 msgr2=0x555e69d704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 -- 172.21.15.5:0/835234613 shutdown_connections 2024-05-08T23:25:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 -- 172.21.15.5:0/835234613 wait complete. 2024-05-08T23:25:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.000+0000 7fb758991b80 1 -- start start 2024-05-08T23:25:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.001+0000 7fb758991b80 1 Processor -- start 2024-05-08T23:25:45.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb758991b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d44370 0x555e69df4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb758991b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb758991b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d70dd0 0x555e69d9a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb758991b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69d8f4b0 con 0x555e69d44370 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb758991b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69d9b2d0 con 0x555e69d70dd0 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb758991b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e69d82170 con 0x555e69d697a0 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb758991b80 10 client.0 _ll_register_callbacks cb 0x555e69c3bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb74a573700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d44370 0x555e69df4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb74a573700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d44370 0x555e69df4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51650/0 (socket says 172.21.15.5:51650) 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb74a573700 1 -- 172.21.15.5:0/613824653 learned_addr learned my addr 172.21.15.5:0/613824653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb749d72700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.002+0000 7fb74ad74700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d70dd0 0x555e69d9a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb74a573700 1 -- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 msgr2=0x555e69d469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb74a573700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb74a573700 1 -- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d70dd0 msgr2=0x555e69d9a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb74a573700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d70dd0 0x555e69d9a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb74a573700 1 -- 172.21.15.5:0/613824653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e69b173b0 con 0x555e69d44370 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb74ad74700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d70dd0 0x555e69d9a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb749d72700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb74a573700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d44370 0x555e69df4aa0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb7440676a0 tx=0x7fb7400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692089058987202b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb7257fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb7257fa700 1 -- 172.21.15.5:0/613824653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740020040 con 0x555e69d44370 2024-05-08T23:25:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb7257fa700 1 -- 172.21.15.5:0/613824653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb740028600 con 0x555e69d44370 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 1 -- 172.21.15.5:0/613824653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb73c05c960 con 0x555e69d44370 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb7257fa700 1 -- 172.21.15.5:0/613824653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7400287c0 con 0x555e69d44370 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 10 client.6966 fetch_fsmap learned FSMap version 59 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 1 -- 172.21.15.5:0/613824653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x555e69b1bcd0 con 0x555e69d44370 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb7257fa700 1 -- 172.21.15.5:0/613824653 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c05c960 con 0x555e69d44370 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb7257fa700 1 -- 172.21.15.5:0/613824653 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 59) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fb740004040 con 0x555e69d44370 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 10 client.6966 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 10 client.6966 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.d 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 20 client.6966 populate_metadata read hostname 'smithi005' 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x555e69d9ac00 0x555e69d8f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 4 client.6966 mds_command: new command op to 6193 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.003+0000 7fb758991b80 1 -- 172.21.15.5:0/613824653 --> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e69c8eec0 con 0x555e69d9ac00 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.004+0000 7fb749d72700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x555e69d9ac00 0x555e69d8f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.004+0000 7fb749d72700 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x555e69d9ac00 0x555e69d8f680 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d019cf1f5c8fc6c server_cookie=ae63834f54f8638d in_seq=0 out_seq=0 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.004+0000 7fb7257fa700 10 client.6966 ms_handle_connect on v2:172.21.15.49:6834/3197377913 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.004+0000 7fb7257fa700 1 -- 172.21.15.5:0/613824653 <== mds.2 v2:172.21.15.49:6834/3197377913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+296 (crc 0 0 0) 0x555e69c8eec0 con 0x555e69d9ac00 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.004+0000 7fb7257fa700 10 client.6966 handle_command_reply: tid=0 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.004+0000 7fb758991b80 1 client.6966 shutdown 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 -- 172.21.15.5:0/613824653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d44370 msgr2=0x555e69df4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d44370 0x555e69df4aa0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb7440676a0 tx=0x7fb7400047c0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 -- 172.21.15.5:0/613824653 shutdown_connections 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:6834/3197377913,v1:172.21.15.49:6836/3197377913] conn(0x555e69d9ac00 0x555e69d8f680 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e69d70dd0 0x555e69d9a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e69d44370 0x555e69df4aa0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb7257fa700 0 client.6966 ms_handle_reset on v2:172.21.15.49:6834/3197377913 2024-05-08T23:25:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 --2- 172.21.15.5:0/613824653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x555e69d697a0 0x555e69d469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 -- 172.21.15.5:0/613824653 >> 172.21.15.5:0/613824653 conn(0x555e69ca4210 msgr2=0x555e69d704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 -- 172.21.15.5:0/613824653 shutdown_connections 2024-05-08T23:25:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 1 -- 172.21.15.5:0/613824653 wait complete. 2024-05-08T23:25:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb758991b80 20 client.6966 trim_cache size 0 max 16384 2024-05-08T23:25:45.014 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:25:45.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:25:45.014 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-08T23:25:45.014 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 1226779428, 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627782, 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7" 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 1353336576, 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627783, 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7/d8" 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:25:45.015 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:25:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb74c7d7700 1 -- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb72806ac90 msgr2=0x7fb72806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb72806ac90 0x7fb72806d150 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb74415e840 tx=0x7fb734008040 comp rx=0 tx=0).stop 2024-05-08T23:25:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb74c7d7700 1 -- 172.21.15.5:0/3410494569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 msgr2=0x7fb7440b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b29f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00efc0 tx=0x7fb73c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.005+0000 7fb74c7d7700 1 -- 172.21.15.5:0/3410494569 shutdown_connections 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.006+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb72806ac90 0x7fb72806d150 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.006+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb744141df0 0x7fb7440b29f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.006+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7440afa40 0x7fb7440ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.006+0000 7fb74c7d7700 1 --2- 172.21.15.5:0/3410494569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb7440d0350 0x7fb7440cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.006+0000 7fb74c7d7700 1 -- 172.21.15.5:0/3410494569 >> 172.21.15.5:0/3410494569 conn(0x7fb7440b5810 msgr2=0x7fb7440b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.006+0000 7fb74c7d7700 1 -- 172.21.15.5:0/3410494569 shutdown_connections 2024-05-08T23:25:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.006+0000 7fb74c7d7700 1 -- 172.21.15.5:0/3410494569 wait complete. 2024-05-08T23:25:45.019 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:25:45.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:25:45.072 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:25:45.125 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:25:45.126 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:25:45.126 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:45.148 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:25:45.148 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:45.211 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:25:45.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.395+0000 7fe61eb75700 1 Processor -- start 2024-05-08T23:25:45.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.395+0000 7fe61eb75700 1 -- start start 2024-05-08T23:25:45.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe61eb75700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe61815d7a0 0x7fe618161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe61eb75700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6180d3f40 0x7fe6180b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe61eb75700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe61eb75700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe618162450 con 0x7fe6180d3f40 2024-05-08T23:25:45.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe61eb75700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180cee00 con 0x7fe61815d7a0 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe61eb75700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180b27e0 con 0x7fe61815db80 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6180d3f40 0x7fe6180b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6180d3f40 0x7fe6180b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51666/0 (socket says 172.21.15.5:51666) 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 -- 172.21.15.5:0/650369933 learned_addr learned my addr 172.21.15.5:0/650369933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe61c911700 1 --2- 172.21.15.5:0/650369933 >> v2:172.21.15.49:3300/0 conn(0x7fe61815d7a0 0x7fe618161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe61d112700 1 --2- 172.21.15.5:0/650369933 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 -- 172.21.15.5:0/650369933 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 msgr2=0x7fe6180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 --2- 172.21.15.5:0/650369933 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 -- 172.21.15.5:0/650369933 >> v2:172.21.15.49:3300/0 conn(0x7fe61815d7a0 msgr2=0x7fe618161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 --2- 172.21.15.5:0/650369933 >> v2:172.21.15.49:3300/0 conn(0x7fe61815d7a0 0x7fe618161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 -- 172.21.15.5:0/650369933 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6180d4400 con 0x7fe6180d3f40 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe617fff700 1 --2- 172.21.15.5:0/650369933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180b0190 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe6180b61f0 tx=0x7fe60c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28af50c479029355 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe6177fe700 1 -- 172.21.15.5:0/650369933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c005e30 con 0x7fe6180d3f40 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe6177fe700 1 -- 172.21.15.5:0/650369933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c004030 con 0x7fe6180d3f40 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.396+0000 7fe6177fe700 1 -- 172.21.15.5:0/650369933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0043e0 con 0x7fe6180d3f40 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 -- 172.21.15.5:0/650369933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 msgr2=0x7fe6180b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 --2- 172.21.15.5:0/650369933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180b0190 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe6180b61f0 tx=0x7fe60c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 -- 172.21.15.5:0/650369933 shutdown_connections 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 --2- 172.21.15.5:0/650369933 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 --2- 172.21.15.5:0/650369933 >> v2:172.21.15.49:3300/0 conn(0x7fe61815d7a0 0x7fe618161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 --2- 172.21.15.5:0/650369933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180b0190 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 -- 172.21.15.5:0/650369933 >> 172.21.15.5:0/650369933 conn(0x7fe61808afe0 msgr2=0x7fe6180d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 -- 172.21.15.5:0/650369933 shutdown_connections 2024-05-08T23:25:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.397+0000 7fe61eb75700 1 -- 172.21.15.5:0/650369933 wait complete. 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.398+0000 7fe61eb75700 1 Processor -- start 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.398+0000 7fe61eb75700 1 -- start start 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61eb75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61eb75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61815d7a0 0x7fe61815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61eb75700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61eb75700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618157e40 con 0x7fe6180d3f40 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61eb75700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618147ab0 con 0x7fe61815d7a0 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61eb75700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61814a0b0 con 0x7fe61815db80 2024-05-08T23:25:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61c911700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61c911700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51674/0 (socket says 172.21.15.5:51674) 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.399+0000 7fe61c911700 1 -- 172.21.15.5:0/2034865290 learned_addr learned my addr 172.21.15.5:0/2034865290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe617fff700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61815d7a0 0x7fe61815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61d112700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61c911700 1 -- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 msgr2=0x7fe6180ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61c911700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61c911700 1 -- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61815d7a0 msgr2=0x7fe61815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61c911700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61815d7a0 0x7fe61815a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61c911700 1 -- 172.21.15.5:0/2034865290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c009500 con 0x7fe6180d3f40 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe617fff700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61815d7a0 0x7fe61815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61d112700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61c911700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180c6590 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e810 tx=0x7fe60800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe6157fa700 1 -- 172.21.15.5:0/2034865290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608013610 con 0x7fe6180d3f40 2024-05-08T23:25:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe6157fa700 1 -- 172.21.15.5:0/2034865290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe608013770 con 0x7fe6180d3f40 2024-05-08T23:25:45.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61814a390 con 0x7fe6180d3f40 2024-05-08T23:25:45.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe6157fa700 1 -- 172.21.15.5:0/2034865290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608013b20 con 0x7fe6180d3f40 2024-05-08T23:25:45.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.400+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61814a950 con 0x7fe6180d3f40 2024-05-08T23:25:45.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.401+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe618068000 con 0x7fe6180d3f40 2024-05-08T23:25:45.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.402+0000 7fe6157fa700 1 -- 172.21.15.5:0/2034865290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe608013c80 con 0x7fe6180d3f40 2024-05-08T23:25:45.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.402+0000 7fe6157fa700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe60406ac90 0x7fe60406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.403+0000 7fe6157fa700 1 -- 172.21.15.5:0/2034865290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe608089180 con 0x7fe6180d3f40 2024-05-08T23:25:45.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.403+0000 7fe617fff700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe60406ac90 0x7fe60406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.403+0000 7fe617fff700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe60406ac90 0x7fe60406d150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6180b61f0 tx=0x7fe60c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.408+0000 7fe6157fa700 1 -- 172.21.15.5:0/2034865290 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe608054240 con 0x7fe6180d3f40 2024-05-08T23:25:45.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.631+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6180d0e30 con 0x7fe6180d3f40 2024-05-08T23:25:45.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.633+0000 7fe6157fa700 1 -- 172.21.15.5:0/2034865290 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16001 (secure 0 0 0) 0x7fe608057860 con 0x7fe6180d3f40 2024-05-08T23:25:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:45.633 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:46.285230+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:24:43.276676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T23:24:44.254746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:25:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe60406ac90 msgr2=0x7fe60406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe60406ac90 0x7fe60406d150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6180b61f0 tx=0x7fe60c008070 comp rx=0 tx=0).stop 2024-05-08T23:25:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 msgr2=0x7fe6180c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180c6590 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e810 tx=0x7fe60800ebd0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 shutdown_connections 2024-05-08T23:25:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe60406ac90 0x7fe60406d150 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe61815db80 0x7fe6180ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6180d3f40 0x7fe6180c6590 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 --2- 172.21.15.5:0/2034865290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe61815d7a0 0x7fe61815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 >> 172.21.15.5:0/2034865290 conn(0x7fe61808afe0 msgr2=0x7fe61806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 shutdown_connections 2024-05-08T23:25:45.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.635+0000 7fe61eb75700 1 -- 172.21.15.5:0/2034865290 wait complete. 2024-05-08T23:25:45.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:24:43.276676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:24:44.254746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:25:45.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:25:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.788+0000 7f36b57c6700 1 Processor -- start 2024-05-08T23:25:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.788+0000 7f36b57c6700 1 -- start start 2024-05-08T23:25:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36b57c6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f36b00b5990 0x7f36b00ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36b57c6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36b006e750 0x7f36b0161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36b57c6700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b00ae9f0 con 0x7f36b01612a0 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36b57c6700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b00adce0 con 0x7f36b006e750 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36b57c6700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b0161c70 con 0x7f36b00b5990 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36af7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36b006e750 0x7f36b0161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36ae7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f36b01612a0 0x7f36b015dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33522/0 (socket says 172.21.15.5:33522) 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36ae7fc700 1 -- 172.21.15.5:0/3366222060 learned_addr learned my addr 172.21.15.5:0/3366222060 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36aeffd700 1 --2- 172.21.15.5:0/3366222060 >> v2:172.21.15.49:3300/0 conn(0x7f36b00b5990 0x7f36b00ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36af7fe700 1 -- 172.21.15.5:0/3366222060 >> v2:172.21.15.49:3300/0 conn(0x7f36b00b5990 msgr2=0x7f36b00ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36af7fe700 1 --2- 172.21.15.5:0/3366222060 >> v2:172.21.15.49:3300/0 conn(0x7f36b00b5990 0x7f36b00ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36af7fe700 1 -- 172.21.15.5:0/3366222060 >> v1:172.21.15.5:6789/0 conn(0x7f36b01612a0 legacy=0x7f36b015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36af7fe700 1 -- 172.21.15.5:0/3366222060 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b00aecd0 con 0x7f36b006e750 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36adffb700 1 -- 172.21.15.5:0/3366222060 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367638938 0 0) 0x7f36b00ae9f0 con 0x7f36b01612a0 2024-05-08T23:25:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.789+0000 7f36aeffd700 1 --2- 172.21.15.5:0/3366222060 >> v2:172.21.15.49:3300/0 conn(0x7f36b00b5990 0x7f36b00ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36af7fe700 1 --2- 172.21.15.5:0/3366222060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b006e750 0x7f36b0161730 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f36b00cfaa0 tx=0x7f36a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda54bbba78c8c30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36adffb700 1 -- 172.21.15.5:0/3366222060 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a800e070 con 0x7f36b006e750 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36adffb700 1 -- 172.21.15.5:0/3366222060 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a800bbd0 con 0x7f36b006e750 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36adffb700 1 -- 172.21.15.5:0/3366222060 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8012430 con 0x7f36b006e750 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36b57c6700 1 -- 172.21.15.5:0/3366222060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b006e750 msgr2=0x7f36b0161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36b57c6700 1 --2- 172.21.15.5:0/3366222060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b006e750 0x7f36b0161730 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f36b00cfaa0 tx=0x7f36a800b330 comp rx=0 tx=0).stop 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36b57c6700 1 -- 172.21.15.5:0/3366222060 shutdown_connections 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36b57c6700 1 --2- 172.21.15.5:0/3366222060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b006e750 0x7f36b0161730 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36b57c6700 1 --2- 172.21.15.5:0/3366222060 >> v2:172.21.15.49:3300/0 conn(0x7f36b00b5990 0x7f36b00ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36b57c6700 1 -- 172.21.15.5:0/3366222060 >> 172.21.15.5:0/3366222060 conn(0x7f36b006a960 msgr2=0x7f36b006ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36b57c6700 1 -- 172.21.15.5:0/3366222060 shutdown_connections 2024-05-08T23:25:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.790+0000 7f36b57c6700 1 -- 172.21.15.5:0/3366222060 wait complete. 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.792+0000 7f36b57c6700 1 Processor -- start 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.792+0000 7f36b57c6700 1 -- start start 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36b57c6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b006e750 0x7f36b01490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36b57c6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b5990 0x7f36b0149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36b57c6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36b01612a0 0x7f36b01f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36b57c6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b015c9e0 con 0x7f36b00b5990 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36b57c6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b015ede0 con 0x7f36b006e750 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36b57c6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b015cc10 con 0x7f36b01612a0 2024-05-08T23:25:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b5990 0x7f36b0149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b5990 0x7f36b0149600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51696/0 (socket says 172.21.15.5:51696) 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 -- 172.21.15.5:0/4063573018 learned_addr learned my addr 172.21.15.5:0/4063573018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36aeffd700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b006e750 0x7f36b01490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36af7fe700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36b01612a0 0x7f36b01f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 -- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36b01612a0 msgr2=0x7f36b01f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36b01612a0 0x7f36b01f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 -- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b006e750 msgr2=0x7f36b01490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b006e750 0x7f36b01490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 -- 172.21.15.5:0/4063573018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a8009a70 con 0x7f36b00b5990 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36aeffd700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b006e750 0x7f36b01490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.793+0000 7f36ae7fc700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b5990 0x7f36b0149600 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c550 tx=0x7f36a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.794+0000 7f36977fe700 1 -- 172.21.15.5:0/4063573018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4004c80 con 0x7f36b00b5990 2024-05-08T23:25:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.794+0000 7f36977fe700 1 -- 172.21.15.5:0/4063573018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a4004e00 con 0x7f36b00b5990 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.794+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b015cef0 con 0x7f36b00b5990 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.794+0000 7f36977fe700 1 -- 172.21.15.5:0/4063573018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4007310 con 0x7f36b00b5990 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.794+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b015d4b0 con 0x7f36b00b5990 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.795+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b015f000 con 0x7f36b00b5990 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.795+0000 7f36977fe700 1 -- 172.21.15.5:0/4063573018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f36a4007470 con 0x7f36b00b5990 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.796+0000 7f36977fe700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f369c06ac90 0x7f369c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.796+0000 7f36977fe700 1 -- 172.21.15.5:0/4063573018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36a4088f70 con 0x7f36b00b5990 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.796+0000 7f36aeffd700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f369c06ac90 0x7f369c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.796+0000 7f36aeffd700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f369c06ac90 0x7f369c06d150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009a00 tx=0x7f36a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:45.801+0000 7f36977fe700 1 -- 172.21.15.5:0/4063573018 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f36a40bc900 con 0x7f36b00b5990 2024-05-08T23:25:46.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.016+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36b0074bc0 con 0x7f36b00b5990 2024-05-08T23:25:46.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.016+0000 7f36977fe700 1 -- 172.21.15.5:0/4063573018 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36b015f000 con 0x7f36b00b5990 2024-05-08T23:25:46.017 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f369c06ac90 msgr2=0x7f369c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f369c06ac90 0x7f369c06d150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009a00 tx=0x7f36a0008040 comp rx=0 tx=0).stop 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b5990 msgr2=0x7f36b0149600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b5990 0x7f36b0149600 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c550 tx=0x7f36a400c910 comp rx=0 tx=0).stop 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 shutdown_connections 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f369c06ac90 0x7f369c06d150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36b01612a0 0x7f36b01f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b006e750 0x7f36b01490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 --2- 172.21.15.5:0/4063573018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b5990 0x7f36b0149600 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 >> 172.21.15.5:0/4063573018 conn(0x7f36b006a960 msgr2=0x7f36b01544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 shutdown_connections 2024-05-08T23:25:46.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.018+0000 7f36b57c6700 1 -- 172.21.15.5:0/4063573018 wait complete. 2024-05-08T23:25:46.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:25:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.171+0000 7f36534bc700 1 Processor -- start 2024-05-08T23:25:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.171+0000 7f36534bc700 1 -- start start 2024-05-08T23:25:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f364c154df0 0x7f364c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f364c0d0890 0x7f364c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c15add0 con 0x7f364c0b43b0 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c15abe0 con 0x7f364c0d0890 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c0d0d20 con 0x7f364c154df0 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3650a57700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f364c0d0890 0x7f364c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3651a59700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f364c0b43b0 0x7f364c0ada60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33538/0 (socket says 172.21.15.5:33538) 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3651a59700 1 -- 172.21.15.5:0/3538244019 learned_addr learned my addr 172.21.15.5:0/3538244019 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3651258700 1 --2- 172.21.15.5:0/3538244019 >> v2:172.21.15.49:3300/0 conn(0x7f364c154df0 0x7f364c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3650a57700 1 -- 172.21.15.5:0/3538244019 >> v2:172.21.15.49:3300/0 conn(0x7f364c154df0 msgr2=0x7f364c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3650a57700 1 --2- 172.21.15.5:0/3538244019 >> v2:172.21.15.49:3300/0 conn(0x7f364c154df0 0x7f364c0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3650a57700 1 -- 172.21.15.5:0/3538244019 >> v1:172.21.15.5:6789/0 conn(0x7f364c0b43b0 legacy=0x7f364c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3650a57700 1 -- 172.21.15.5:0/3538244019 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c15b070 con 0x7f364c0d0890 2024-05-08T23:25:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3650a57700 1 --2- 172.21.15.5:0/3538244019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0d0890 0x7f364c0b3e70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f364c0675f0 tx=0x7f363c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3804402a5e11088b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3643fff700 1 -- 172.21.15.5:0/3538244019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011830 con 0x7f364c0d0890 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3643fff700 1 -- 172.21.15.5:0/3538244019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c011990 con 0x7f364c0d0890 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f3643fff700 1 -- 172.21.15.5:0/3538244019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011d20 con 0x7f364c0d0890 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 -- 172.21.15.5:0/3538244019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0d0890 msgr2=0x7f364c0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 --2- 172.21.15.5:0/3538244019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0d0890 0x7f364c0b3e70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f364c0675f0 tx=0x7f363c009b20 comp rx=0 tx=0).stop 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 -- 172.21.15.5:0/3538244019 shutdown_connections 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 --2- 172.21.15.5:0/3538244019 >> v2:172.21.15.49:3300/0 conn(0x7f364c154df0 0x7f364c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 --2- 172.21.15.5:0/3538244019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0d0890 0x7f364c0b3e70 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 -- 172.21.15.5:0/3538244019 >> 172.21.15.5:0/3538244019 conn(0x7f364c099a50 msgr2=0x7f364c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 -- 172.21.15.5:0/3538244019 shutdown_connections 2024-05-08T23:25:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.173+0000 7f36534bc700 1 -- 172.21.15.5:0/3538244019 wait complete. 2024-05-08T23:25:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.175+0000 7f36534bc700 1 Processor -- start 2024-05-08T23:25:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.175+0000 7f36534bc700 1 -- start start 2024-05-08T23:25:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.175+0000 7f36534bc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0b43b0 0x7f364c144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f36534bc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0d0890 0x7f364c09d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f36534bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c154df0 0x7f364c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f36534bc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0cd660 con 0x7f364c0b43b0 2024-05-08T23:25:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f36534bc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c1452f0 con 0x7f364c154df0 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f36534bc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c09dd60 con 0x7f364c0d0890 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0b43b0 0x7f364c144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0b43b0 0x7f364c144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51718/0 (socket says 172.21.15.5:51718) 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 -- 172.21.15.5:0/3060359718 learned_addr learned my addr 172.21.15.5:0/3060359718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3650a57700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0d0890 0x7f364c09d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651a59700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c154df0 0x7f364c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 -- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0d0890 msgr2=0x7f364c09d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0d0890 0x7f364c09d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 -- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c154df0 msgr2=0x7f364c0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c154df0 0x7f364c0cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 -- 172.21.15.5:0/3060359718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0094a0 con 0x7f364c0b43b0 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3651258700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0b43b0 0x7f364c144db0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f364800d800 tx=0x7f364800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3641ffb700 1 -- 172.21.15.5:0/3060359718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36480125f0 con 0x7f364c0b43b0 2024-05-08T23:25:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3641ffb700 1 -- 172.21.15.5:0/3060359718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648012750 con 0x7f364c0b43b0 2024-05-08T23:25:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c09e040 con 0x7f364c0b43b0 2024-05-08T23:25:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f3641ffb700 1 -- 172.21.15.5:0/3060359718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648012b00 con 0x7f364c0b43b0 2024-05-08T23:25:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.177+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c0cd2f0 con 0x7f364c0b43b0 2024-05-08T23:25:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.178+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c074bc0 con 0x7f364c0b43b0 2024-05-08T23:25:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.178+0000 7f3641ffb700 1 -- 172.21.15.5:0/3060359718 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3648012c60 con 0x7f364c0b43b0 2024-05-08T23:25:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.179+0000 7f3641ffb700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363006ac90 0x7f363006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.179+0000 7f3641ffb700 1 -- 172.21.15.5:0/3060359718 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3648089420 con 0x7f364c0b43b0 2024-05-08T23:25:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.179+0000 7f3650a57700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363006ac90 0x7f363006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.179+0000 7f3650a57700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363006ac90 0x7f363006d150 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f363c0090b0 tx=0x7f363c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.184+0000 7f3641ffb700 1 -- 172.21.15.5:0/3060359718 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3648054580 con 0x7f364c0b43b0 2024-05-08T23:25:46.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.402+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f364c0b09e0 con 0x7f364c0b43b0 2024-05-08T23:25:46.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.402+0000 7f3641ffb700 1 -- 172.21.15.5:0/3060359718 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3648057ba0 con 0x7f364c0b43b0 2024-05-08T23:25:46.403 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:25:46.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363006ac90 msgr2=0x7f363006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363006ac90 0x7f363006d150 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f363c0090b0 tx=0x7f363c008030 comp rx=0 tx=0).stop 2024-05-08T23:25:46.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0b43b0 msgr2=0x7f364c144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0b43b0 0x7f364c144db0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f364800d800 tx=0x7f364800db10 comp rx=0 tx=0).stop 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 shutdown_connections 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f363006ac90 0x7f363006d150 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c154df0 0x7f364c0cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364c0b43b0 0x7f364c144db0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 --2- 172.21.15.5:0/3060359718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f364c0d0890 0x7f364c09d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 >> 172.21.15.5:0/3060359718 conn(0x7f364c099a50 msgr2=0x7f364c06b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 shutdown_connections 2024-05-08T23:25:46.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.405+0000 7f36534bc700 1 -- 172.21.15.5:0/3060359718 wait complete. 2024-05-08T23:25:46.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:25:46.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.558+0000 7f14f6fd4700 1 Processor -- start 2024-05-08T23:25:46.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.558+0000 7f14f6fd4700 1 -- start start 2024-05-08T23:25:46.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.559+0000 7f14f6fd4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14f00d0f10 0x7f14f015e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.559+0000 7f14f6fd4700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f14f00af5a0 0x7f14f015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.559+0000 7f14f6fd4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f00cf970 con 0x7f14f00cbf50 2024-05-08T23:25:46.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.559+0000 7f14f6fd4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f00cfc50 con 0x7f14f00d0f10 2024-05-08T23:25:46.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.559+0000 7f14f6fd4700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f00cff30 con 0x7f14f00af5a0 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.559+0000 7f14effff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14f00d0f10 0x7f14f015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.559+0000 7f14f4d70700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f14f00cbf50 0x7f14f0161d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33542/0 (socket says 172.21.15.5:33542) 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.559+0000 7f14f4d70700 1 -- 172.21.15.5:0/103751699 learned_addr learned my addr 172.21.15.5:0/103751699 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14f5571700 1 --2- 172.21.15.5:0/103751699 >> v2:172.21.15.49:3300/0 conn(0x7f14f00af5a0 0x7f14f015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14effff700 1 -- 172.21.15.5:0/103751699 >> v2:172.21.15.49:3300/0 conn(0x7f14f00af5a0 msgr2=0x7f14f015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14effff700 1 --2- 172.21.15.5:0/103751699 >> v2:172.21.15.49:3300/0 conn(0x7f14f00af5a0 0x7f14f015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14effff700 1 -- 172.21.15.5:0/103751699 >> v1:172.21.15.5:6789/0 conn(0x7f14f00cbf50 legacy=0x7f14f0161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14effff700 1 -- 172.21.15.5:0/103751699 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f01617e0 con 0x7f14f00d0f10 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14effff700 1 --2- 172.21.15.5:0/103751699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f015e050 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f14f006f140 tx=0x7f14d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3913d3fda6ee89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14ef7fe700 1 -- 172.21.15.5:0/103751699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d8011830 con 0x7f14f00d0f10 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14ef7fe700 1 -- 172.21.15.5:0/103751699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d8011990 con 0x7f14f00d0f10 2024-05-08T23:25:46.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.560+0000 7f14ef7fe700 1 -- 172.21.15.5:0/103751699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d8011d20 con 0x7f14f00d0f10 2024-05-08T23:25:46.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.561+0000 7f14f6fd4700 1 -- 172.21.15.5:0/103751699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 msgr2=0x7f14f015e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.561+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/103751699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f015e050 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f14f006f140 tx=0x7f14d8009b20 comp rx=0 tx=0).stop 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.561+0000 7f14f6fd4700 1 -- 172.21.15.5:0/103751699 shutdown_connections 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.561+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/103751699 >> v2:172.21.15.49:3300/0 conn(0x7f14f00af5a0 0x7f14f015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.561+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/103751699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f015e050 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.561+0000 7f14f6fd4700 1 -- 172.21.15.5:0/103751699 >> 172.21.15.5:0/103751699 conn(0x7f14f0099a50 msgr2=0x7f14f00d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.561+0000 7f14f6fd4700 1 -- 172.21.15.5:0/103751699 shutdown_connections 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.561+0000 7f14f6fd4700 1 -- 172.21.15.5:0/103751699 wait complete. 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.563+0000 7f14f6fd4700 1 Processor -- start 2024-05-08T23:25:46.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.564+0000 7f14f6fd4700 1 -- start start 2024-05-08T23:25:46.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f6fd4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14f00af5a0 0x7f14f00b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f6fd4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14f00cbf50 0x7f14f01527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f6fd4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f00b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f6fd4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b2ef0 con 0x7f14f00d0f10 2024-05-08T23:25:46.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f6fd4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b4d80 con 0x7f14f00cbf50 2024-05-08T23:25:46.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f6fd4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b3140 con 0x7f14f00af5a0 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f00b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f00b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51740/0 (socket says 172.21.15.5:51740) 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 -- 172.21.15.5:0/2567470870 learned_addr learned my addr 172.21.15.5:0/2567470870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14effff700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14f00cbf50 0x7f14f01527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f4d70700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14f00af5a0 0x7f14f00b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 -- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14f00af5a0 msgr2=0x7f14f00b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14f00af5a0 0x7f14f00b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 -- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14f00cbf50 msgr2=0x7f14f01527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14f00cbf50 0x7f14f01527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 -- 172.21.15.5:0/2567470870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80094a0 con 0x7f14f00d0f10 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f5571700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f00b43a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f14e8007e40 tx=0x7f14e800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14ed7fa700 1 -- 172.21.15.5:0/2567470870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e800ed30 con 0x7f14f00d0f10 2024-05-08T23:25:46.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14ed7fa700 1 -- 172.21.15.5:0/2567470870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e8014420 con 0x7f14f00d0f10 2024-05-08T23:25:46.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f00b3420 con 0x7f14f00d0f10 2024-05-08T23:25:46.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14ed7fa700 1 -- 172.21.15.5:0/2567470870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e80147d0 con 0x7f14f00d0f10 2024-05-08T23:25:46.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.565+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f0152d30 con 0x7f14f00d0f10 2024-05-08T23:25:46.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.566+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f00b3940 con 0x7f14f00d0f10 2024-05-08T23:25:46.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.566+0000 7f14ed7fa700 1 -- 172.21.15.5:0/2567470870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f14e8014930 con 0x7f14f00d0f10 2024-05-08T23:25:46.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.567+0000 7f14ed7fa700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f14d006ac90 0x7f14d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.567+0000 7f14ed7fa700 1 -- 172.21.15.5:0/2567470870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14e8089980 con 0x7f14f00d0f10 2024-05-08T23:25:46.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.567+0000 7f14f4d70700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f14d006ac90 0x7f14d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.567+0000 7f14f4d70700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f14d006ac90 0x7f14d006d150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005910 tx=0x7f14e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.572+0000 7f14ed7fa700 1 -- 172.21.15.5:0/2567470870 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f14f00b3940 con 0x7f14f00d0f10 2024-05-08T23:25:46.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.788+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14f00b3940 con 0x7f14f00d0f10 2024-05-08T23:25:46.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.788+0000 7f14ed7fa700 1 -- 172.21.15.5:0/2567470870 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14f00b3940 con 0x7f14f00d0f10 2024-05-08T23:25:46.788 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f14d006ac90 msgr2=0x7f14d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f14d006ac90 0x7f14d006d150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005910 tx=0x7f14e0009040 comp rx=0 tx=0).stop 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 msgr2=0x7f14f00b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f00b43a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f14e8007e40 tx=0x7f14e800e3c0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 shutdown_connections 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f14d006ac90 0x7f14d006d150 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f00d0f10 0x7f14f00b43a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14f00af5a0 0x7f14f00b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 --2- 172.21.15.5:0/2567470870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14f00cbf50 0x7f14f01527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 >> 172.21.15.5:0/2567470870 conn(0x7f14f0099a50 msgr2=0x7f14f006acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.790+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 shutdown_connections 2024-05-08T23:25:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.791+0000 7f14f6fd4700 1 -- 172.21.15.5:0/2567470870 wait complete. 2024-05-08T23:25:46.804 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:25:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.943+0000 7f904de5e700 1 Processor -- start 2024-05-08T23:25:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.943+0000 7f904de5e700 1 -- start start 2024-05-08T23:25:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.944+0000 7f904de5e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f904815d7a0 0x7f9048161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.944+0000 7f904de5e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9048162450 con 0x7f904815db80 2024-05-08T23:25:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.944+0000 7f904de5e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90480cee00 con 0x7f90480d3f40 2024-05-08T23:25:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.944+0000 7f904de5e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90480b27e0 con 0x7f904815d7a0 2024-05-08T23:25:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.944+0000 7f9047fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f904815db80 0x7f90480cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33558/0 (socket says 172.21.15.5:33558) 2024-05-08T23:25:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.944+0000 7f9047fff700 1 -- 172.21.15.5:0/3331917769 learned_addr learned my addr 172.21.15.5:0/3331917769 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.944+0000 7f90477fe700 1 --2- 172.21.15.5:0/3331917769 >> v2:172.21.15.49:3300/0 conn(0x7f904815d7a0 0x7f9048161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707900210 0 0) 0x7f9040001490 con 0x7f904815db80 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90300035b0 con 0x7f904815db80 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1135367769 0 0) 0x7f90300035b0 con 0x7f904815db80 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 >> v2:172.21.15.49:3300/0 conn(0x7f904815d7a0 msgr2=0x7f9048161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 --2- 172.21.15.5:0/3331917769 >> v2:172.21.15.49:3300/0 conn(0x7f904815d7a0 0x7f9048161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 >> v1:172.21.15.49:6789/0 conn(0x7f90480d3f40 legacy=0x7f90480b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480d4400 con 0x7f904815db80 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f9040001b80 con 0x7f904815db80 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 >> v1:172.21.15.5:6789/0 conn(0x7f904815db80 legacy=0x7f90480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9030003a00 0x7f9030003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f90300048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90300050d0 0x7f903000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480b27e0 con 0x7f9030004400 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903000c030 con 0x7f9030003a00 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903000c1f0 con 0x7f90300050d0 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f9046ffd700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f90300048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f90477fe700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9030003a00 0x7f9030003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f9047fff700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90300050d0 0x7f903000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f9046ffd700 1 -- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90300050d0 msgr2=0x7f903000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f9046ffd700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90300050d0 0x7f903000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f9046ffd700 1 -- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9030003a00 msgr2=0x7f9030003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f9046ffd700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9030003a00 0x7f9030003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f9046ffd700 1 -- 172.21.15.5:0/3331917769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300035b0 con 0x7f9030004400 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.945+0000 7f9047fff700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90300050d0 0x7f903000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f9047fff700 1 -- 172.21.15.5:0/3331917769 reap_dead start 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f9046ffd700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f90300048d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f903c001930 tx=0x7f9030005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55b8d1f45b0b0d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c0013a0 con 0x7f9030004400 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c0052d0 con 0x7f9030004400 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f90467fc700 1 -- 172.21.15.5:0/3331917769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c0036d0 con 0x7f9030004400 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f904de5e700 1 -- 172.21.15.5:0/3331917769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 msgr2=0x7f90300048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f904de5e700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f90300048d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f903c001930 tx=0x7f9030005880 comp rx=0 tx=0).stop 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f904de5e700 1 -- 172.21.15.5:0/3331917769 shutdown_connections 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f904de5e700 1 --2- 172.21.15.5:0/3331917769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f90300048d0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.946+0000 7f904de5e700 1 -- 172.21.15.5:0/3331917769 >> 172.21.15.5:0/3331917769 conn(0x7f904808afe0 msgr2=0x7f90480d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.947+0000 7f904de5e700 1 -- 172.21.15.5:0/3331917769 shutdown_connections 2024-05-08T23:25:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.947+0000 7f904de5e700 1 -- 172.21.15.5:0/3331917769 wait complete. 2024-05-08T23:25:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.948+0000 7f904de5e700 1 Processor -- start 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.948+0000 7f904de5e700 1 -- start start 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f904de5e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f904814ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f904de5e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90480cc8d0 0x7f90480ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f904de5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90480ca760 0x7f90480cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f904de5e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480cc150 con 0x7f9030004400 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f904de5e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480cd290 con 0x7f90480ca760 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f904de5e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904814cf50 con 0x7f90480cc8d0 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f904814ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f904814ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51754/0 (socket says 172.21.15.5:51754) 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 -- 172.21.15.5:0/1022946313 learned_addr learned my addr 172.21.15.5:0/1022946313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 -- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90480cc8d0 msgr2=0x7f90480ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90480cc8d0 0x7f90480ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 -- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90480ca760 msgr2=0x7f90480cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90480ca760 0x7f90480cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 -- 172.21.15.5:0/1022946313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480cb4d0 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.949+0000 7f90477fe700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f904814ca10 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9038011aa0 tx=0x7f9038011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.950+0000 7f902ffff700 1 -- 172.21.15.5:0/1022946313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903800f960 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.950+0000 7f902ffff700 1 -- 172.21.15.5:0/1022946313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903800fac0 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.950+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90480cb7b0 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.950+0000 7f902ffff700 1 -- 172.21.15.5:0/1022946313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903800fe70 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.950+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90480cbd70 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.951+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90480cc290 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.951+0000 7f902ffff700 1 -- 172.21.15.5:0/1022946313 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9038014440 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.952+0000 7f902ffff700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f903406acc0 0x7f903406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.952+0000 7f902ffff700 1 -- 172.21.15.5:0/1022946313 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90380899c0 con 0x7f9030004400 2024-05-08T23:25:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.952+0000 7f9046ffd700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f903406acc0 0x7f903406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.953+0000 7f9046ffd700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f903406acc0 0x7f903406d180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90480d3ba0 tx=0x7f903c00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:46.956+0000 7f902ffff700 1 -- 172.21.15.5:0/1022946313 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f90480cc290 con 0x7f9030004400 2024-05-08T23:25:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.192+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9048068000 con 0x7f9030004400 2024-05-08T23:25:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.192+0000 7f902ffff700 1 -- 172.21.15.5:0/1022946313 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9038054c30 con 0x7f9030004400 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f903406acc0 msgr2=0x7f903406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f903406acc0 0x7f903406d180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90480d3ba0 tx=0x7f903c00a000 comp rx=0 tx=0).stop 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 msgr2=0x7f904814ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f904814ca10 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9038011aa0 tx=0x7f9038011e60 comp rx=0 tx=0).stop 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 shutdown_connections 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f903406acc0 0x7f903406d180 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90480ca760 0x7f90480cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9030004400 0x7f904814ca10 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 --2- 172.21.15.5:0/1022946313 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90480cc8d0 0x7f90480ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.194+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 >> 172.21.15.5:0/1022946313 conn(0x7f904808afe0 msgr2=0x7f90480b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.195+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 shutdown_connections 2024-05-08T23:25:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.195+0000 7f904de5e700 1 -- 172.21.15.5:0/1022946313 wait complete. 2024-05-08T23:25:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:25:47.207 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:25:47.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:25:47.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T23:25:47.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:25:47.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:25:47.254 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:25:47.274 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:25:47.274 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:47.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:25:47.296 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:25:47.296 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:47.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:25:47.335 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:25:47.335 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:47.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:25:47.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:25:47.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.522+0000 7f7f47950700 1 Processor -- start 2024-05-08T23:25:47.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.522+0000 7f7f47950700 1 -- start start 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f47950700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f400cf6c0 0x7f7f400cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f47950700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7f400b16e0 0x7f7f400c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f47950700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f400cffe0 con 0x7f7f400b11b0 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f47950700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f400c5400 con 0x7f7f400cf6c0 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f47950700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f400c56e0 con 0x7f7f400b16e0 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f456ec700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f400cf6c0 0x7f7f400cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f456ec700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f400cf6c0 0x7f7f400cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49758/0 (socket says 172.21.15.5:49758) 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f456ec700 1 -- 172.21.15.5:0/350022868 learned_addr learned my addr 172.21.15.5:0/350022868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f45eed700 1 --2- 172.21.15.5:0/350022868 >> v2:172.21.15.49:3300/0 conn(0x7f7f400b16e0 0x7f7f400c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f456ec700 1 -- 172.21.15.5:0/350022868 >> v2:172.21.15.49:3300/0 conn(0x7f7f400b16e0 msgr2=0x7f7f400c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f456ec700 1 --2- 172.21.15.5:0/350022868 >> v2:172.21.15.49:3300/0 conn(0x7f7f400b16e0 0x7f7f400c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f456ec700 1 -- 172.21.15.5:0/350022868 >> v1:172.21.15.49:6789/0 conn(0x7f7f400b11b0 legacy=0x7f7f400caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f456ec700 1 -- 172.21.15.5:0/350022868 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f400c59c0 con 0x7f7f400cf6c0 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.523+0000 7f7f45eed700 1 --2- 172.21.15.5:0/350022868 >> v2:172.21.15.49:3300/0 conn(0x7f7f400b16e0 0x7f7f400c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f456ec700 1 --2- 172.21.15.5:0/350022868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400cf6c0 0x7f7f400cfaa0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7f40071ba0 tx=0x7f7f3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b62b7f40ee4a65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f37fff700 1 -- 172.21.15.5:0/350022868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00b870 con 0x7f7f400cf6c0 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f37fff700 1 -- 172.21.15.5:0/350022868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c00b9d0 con 0x7f7f400cf6c0 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f37fff700 1 -- 172.21.15.5:0/350022868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00bd60 con 0x7f7f400cf6c0 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f47950700 1 -- 172.21.15.5:0/350022868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400cf6c0 msgr2=0x7f7f400cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f47950700 1 --2- 172.21.15.5:0/350022868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400cf6c0 0x7f7f400cfaa0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7f40071ba0 tx=0x7f7f3c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f47950700 1 -- 172.21.15.5:0/350022868 shutdown_connections 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f47950700 1 --2- 172.21.15.5:0/350022868 >> v2:172.21.15.49:3300/0 conn(0x7f7f400b16e0 0x7f7f400c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f47950700 1 --2- 172.21.15.5:0/350022868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400cf6c0 0x7f7f400cfaa0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.524+0000 7f7f47950700 1 -- 172.21.15.5:0/350022868 >> 172.21.15.5:0/350022868 conn(0x7f7f40099a50 msgr2=0x7f7f400b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.525+0000 7f7f47950700 1 -- 172.21.15.5:0/350022868 shutdown_connections 2024-05-08T23:25:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.525+0000 7f7f47950700 1 -- 172.21.15.5:0/350022868 wait complete. 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.526+0000 7f7f47950700 1 Processor -- start 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.526+0000 7f7f47950700 1 -- start start 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f47950700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f400b11b0 0x7f7f400c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f47950700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400b16e0 0x7f7f400cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f47950700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f400cf6c0 0x7f7f40160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f47950700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400ca040 con 0x7f7f400b16e0 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f47950700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400c94c0 con 0x7f7f400b11b0 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f47950700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400c97a0 con 0x7f7f400cf6c0 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400b16e0 0x7f7f400cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400b16e0 0x7f7f400cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49762/0 (socket says 172.21.15.5:49762) 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 -- 172.21.15.5:0/419112069 learned_addr learned my addr 172.21.15.5:0/419112069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f456ec700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f400b11b0 0x7f7f400c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f45eed700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f400cf6c0 0x7f7f40160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 -- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f400cf6c0 msgr2=0x7f7f40160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f400cf6c0 0x7f7f40160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 -- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f400b11b0 msgr2=0x7f7f400c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f400b11b0 0x7f7f400c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 -- 172.21.15.5:0/419112069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c009550 con 0x7f7f400b16e0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f44eeb700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400b16e0 0x7f7f400cd580 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c450 tx=0x7f7f3000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f35ffb700 1 -- 172.21.15.5:0/419112069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f300052d0 con 0x7f7f400b16e0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f35ffb700 1 -- 172.21.15.5:0/419112069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f30005430 con 0x7f7f400b16e0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f400c9a80 con 0x7f7f400b16e0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.527+0000 7f7f35ffb700 1 -- 172.21.15.5:0/419112069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f300038c0 con 0x7f7f400b16e0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.528+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f400cdac0 con 0x7f7f400b16e0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.529+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f40145cd0 con 0x7f7f400b16e0 2024-05-08T23:25:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.529+0000 7f7f35ffb700 1 -- 172.21.15.5:0/419112069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7f30003a20 con 0x7f7f400b16e0 2024-05-08T23:25:47.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.529+0000 7f7f35ffb700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f2406ac90 0x7f7f2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.530+0000 7f7f35ffb700 1 -- 172.21.15.5:0/419112069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f30088690 con 0x7f7f400b16e0 2024-05-08T23:25:47.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.530+0000 7f7f456ec700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f2406ac90 0x7f7f2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.530+0000 7f7f456ec700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f2406ac90 0x7f7f2406d150 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000c00 tx=0x7f7f3c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.534+0000 7f7f35ffb700 1 -- 172.21.15.5:0/419112069 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7f40145cd0 con 0x7f7f400b16e0 2024-05-08T23:25:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.803+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f40145cd0 con 0x7f7f400b16e0 2024-05-08T23:25:47.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.803+0000 7f7f35ffb700 1 -- 172.21.15.5:0/419112069 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4765 (secure 0 0 0) 0x7f7f40145cd0 con 0x7f7f400b16e0 2024-05-08T23:25:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:47.803 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:25:18.479282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6525,"mds_1":6516,"mds_2":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":2,"incarnation":53,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6516":{"gid":6516,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f2406ac90 msgr2=0x7f7f2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f2406ac90 0x7f7f2406d150 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000c00 tx=0x7f7f3c008030 comp rx=0 tx=0).stop 2024-05-08T23:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400b16e0 msgr2=0x7f7f400cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400b16e0 0x7f7f400cd580 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c450 tx=0x7f7f3000c760 comp rx=0 tx=0).stop 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 shutdown_connections 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7f2406ac90 0x7f7f2406d150 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7f400cf6c0 0x7f7f40160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f400b11b0 0x7f7f400c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 --2- 172.21.15.5:0/419112069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f400b16e0 0x7f7f400cd580 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 >> 172.21.15.5:0/419112069 conn(0x7f7f40099a50 msgr2=0x7f7f400d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 shutdown_connections 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.806+0000 7f7f47950700 1 -- 172.21.15.5:0/419112069 wait complete. 2024-05-08T23:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 59 2024-05-08T23:25:47.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:25:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.956+0000 7f5f1b845700 1 Processor -- start 2024-05-08T23:25:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.957+0000 7f5f1b845700 1 -- start start 2024-05-08T23:25:47.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.957+0000 7f5f1b845700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5f1406b140 0x7f5f140ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.957+0000 7f5f1b845700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f1415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f1b845700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f140cdbb0 con 0x7f5f1415c1c0 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f1b845700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f140cd310 con 0x7f5f1406b140 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f1b845700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140cd5f0 con 0x7f5f1406a960 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f19de2700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5f1415c1c0 0x7f5f140cf760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51702/0 (socket says 172.21.15.5:51702) 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f19de2700 1 -- 172.21.15.5:0/3443850635 learned_addr learned my addr 172.21.15.5:0/3443850635 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f195e1700 1 --2- 172.21.15.5:0/3443850635 >> v2:172.21.15.49:3300/0 conn(0x7f5f1406b140 0x7f5f140ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f18de0700 1 --2- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f1415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f07fff700 1 -- 172.21.15.5:0/3443850635 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550799386 0 0) 0x7f5f140cdbb0 con 0x7f5f1415c1c0 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f07fff700 1 -- 172.21.15.5:0/3443850635 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f000035b0 con 0x7f5f1415c1c0 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f07fff700 1 -- 172.21.15.5:0/3443850635 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2001929737 0 0) 0x7f5f000035b0 con 0x7f5f1415c1c0 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f195e1700 1 -- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 msgr2=0x7f5f1415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f195e1700 1 --2- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f1415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f195e1700 1 -- 172.21.15.5:0/3443850635 >> v1:172.21.15.5:6789/0 conn(0x7f5f1415c1c0 legacy=0x7f5f140cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f195e1700 1 -- 172.21.15.5:0/3443850635 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f140cd8d0 con 0x7f5f1406b140 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.958+0000 7f5f18de0700 1 --2- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f1415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f195e1700 1 --2- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1406b140 0x7f5f140ccd30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000f9b0 tx=0x7f5f1000fcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e4113a30c053cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f07fff700 1 -- 172.21.15.5:0/3443850635 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0019f0 con 0x7f5f1406b140 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f07fff700 1 -- 172.21.15.5:0/3443850635 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f10009a00 con 0x7f5f1406b140 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f07fff700 1 -- 172.21.15.5:0/3443850635 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10009d90 con 0x7f5f1406b140 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f1b845700 1 -- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1406b140 msgr2=0x7f5f140ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f1b845700 1 --2- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1406b140 0x7f5f140ccd30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000f9b0 tx=0x7f5f1000fcc0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f1b845700 1 -- 172.21.15.5:0/3443850635 shutdown_connections 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f1b845700 1 --2- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1406b140 0x7f5f140ccd30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f1b845700 1 --2- 172.21.15.5:0/3443850635 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f1415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.959+0000 7f5f1b845700 1 -- 172.21.15.5:0/3443850635 >> 172.21.15.5:0/3443850635 conn(0x7f5f140d0320 msgr2=0x7f5f140d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.960+0000 7f5f1b845700 1 -- 172.21.15.5:0/3443850635 shutdown_connections 2024-05-08T23:25:47.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.960+0000 7f5f1b845700 1 -- 172.21.15.5:0/3443850635 wait complete. 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.961+0000 7f5f1b845700 1 Processor -- start 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.961+0000 7f5f1b845700 1 -- start start 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f1b845700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f140c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f1b845700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1406b140 0x7f5f141554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f1b845700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1415c1c0 0x7f5f140c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f1b845700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140c6260 con 0x7f5f1406b140 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f1b845700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140c55e0 con 0x7f5f1415c1c0 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f1b845700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140c58c0 con 0x7f5f1406a960 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1406b140 0x7f5f141554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1406b140 0x7f5f141554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49778/0 (socket says 172.21.15.5:49778) 2024-05-08T23:25:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 -- 172.21.15.5:0/776331492 learned_addr learned my addr 172.21.15.5:0/776331492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f19de2700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1415c1c0 0x7f5f140c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f195e1700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f140c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 -- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 msgr2=0x7f5f140c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f140c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 -- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1415c1c0 msgr2=0x7f5f140c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1415c1c0 0x7f5f140c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 -- 172.21.15.5:0/776331492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f080094a0 con 0x7f5f1406b140 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f18de0700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1406b140 0x7f5f141554c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800ebe0 tx=0x7f5f0800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f05ffb700 1 -- 172.21.15.5:0/776331492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08009760 con 0x7f5f1406b140 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f05ffb700 1 -- 172.21.15.5:0/776331492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0800c850 con 0x7f5f1406b140 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1000f610 con 0x7f5f1406b140 2024-05-08T23:25:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.962+0000 7f5f05ffb700 1 -- 172.21.15.5:0/776331492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0800cbe0 con 0x7f5f1406b140 2024-05-08T23:25:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.963+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f140c3cc0 con 0x7f5f1406b140 2024-05-08T23:25:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.964+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f140c64d0 con 0x7f5f1406b140 2024-05-08T23:25:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.964+0000 7f5f05ffb700 1 -- 172.21.15.5:0/776331492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f0800cd40 con 0x7f5f1406b140 2024-05-08T23:25:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.965+0000 7f5f05ffb700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ef806ac90 0x7f5ef806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.965+0000 7f5f05ffb700 1 -- 172.21.15.5:0/776331492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f08089290 con 0x7f5f1406b140 2024-05-08T23:25:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.965+0000 7f5f195e1700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ef806ac90 0x7f5ef806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.965+0000 7f5f195e1700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ef806ac90 0x7f5ef806d150 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5f100090b0 tx=0x7f5f10013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:47.969+0000 7f5f05ffb700 1 -- 172.21.15.5:0/776331492 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f08054500 con 0x7f5f1406b140 2024-05-08T23:25:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.187+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f14068000 con 0x7f5f1406b140 2024-05-08T23:25:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.188+0000 7f5f05ffb700 1 -- 172.21.15.5:0/776331492 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16001 (secure 0 0 0) 0x7f5f08057b20 con 0x7f5f1406b140 2024-05-08T23:25:48.188 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:48.188 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:46.285230+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:24:43.276676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T23:24:44.254746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:25:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ef806ac90 msgr2=0x7f5ef806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ef806ac90 0x7f5ef806d150 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5f100090b0 tx=0x7f5f10013000 comp rx=0 tx=0).stop 2024-05-08T23:25:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1406b140 msgr2=0x7f5f141554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1406b140 0x7f5f141554c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800ebe0 tx=0x7f5f0800eef0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 shutdown_connections 2024-05-08T23:25:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ef806ac90 0x7f5ef806d150 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f1415c1c0 0x7f5f140c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f1406a960 0x7f5f140c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 --2- 172.21.15.5:0/776331492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1406b140 0x7f5f141554c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 >> 172.21.15.5:0/776331492 conn(0x7f5f140d0320 msgr2=0x7f5f140af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 shutdown_connections 2024-05-08T23:25:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.191+0000 7f5f1b845700 1 -- 172.21.15.5:0/776331492 wait complete. 2024-05-08T23:25:48.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:24:43.276676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:24:44.254746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:25:48.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:25:48.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.345+0000 7f62cf7bb700 1 Processor -- start 2024-05-08T23:25:48.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.345+0000 7f62cf7bb700 1 -- start start 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.346+0000 7f62cf7bb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62c806f670 0x7f62c80cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.346+0000 7f62cf7bb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62c806fa50 0x7f62c80b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.346+0000 7f62cf7bb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c80cda40 con 0x7f62c80b4370 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.346+0000 7f62cf7bb700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c8162790 con 0x7f62c806f670 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.346+0000 7f62cf7bb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c8162a70 con 0x7f62c806fa50 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.346+0000 7f62cd557700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62c806f670 0x7f62c80cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.346+0000 7f62cdd58700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f62c80b4370 0x7f62c806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51714/0 (socket says 172.21.15.5:51714) 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.346+0000 7f62cdd58700 1 -- 172.21.15.5:0/3161625551 learned_addr learned my addr 172.21.15.5:0/3161625551 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62ccd56700 1 --2- 172.21.15.5:0/3161625551 >> v2:172.21.15.49:3300/0 conn(0x7f62c806fa50 0x7f62c80b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62cd557700 1 -- 172.21.15.5:0/3161625551 >> v2:172.21.15.49:3300/0 conn(0x7f62c806fa50 msgr2=0x7f62c80b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62cd557700 1 --2- 172.21.15.5:0/3161625551 >> v2:172.21.15.49:3300/0 conn(0x7f62c806fa50 0x7f62c80b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62cd557700 1 -- 172.21.15.5:0/3161625551 >> v1:172.21.15.5:6789/0 conn(0x7f62c80b4370 legacy=0x7f62c806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62cd557700 1 -- 172.21.15.5:0/3161625551 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c8162d50 con 0x7f62c806f670 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62bffff700 1 -- 172.21.15.5:0/3161625551 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288978555 0 0) 0x7f62c80cda40 con 0x7f62c80b4370 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62cd557700 1 --2- 172.21.15.5:0/3161625551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c80cf500 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f62c806ea50 tx=0x7f62c4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba95e880a55204df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62bffff700 1 -- 172.21.15.5:0/3161625551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4011830 con 0x7f62c806f670 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62bffff700 1 -- 172.21.15.5:0/3161625551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c4011990 con 0x7f62c806f670 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.347+0000 7f62bffff700 1 -- 172.21.15.5:0/3161625551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4011d20 con 0x7f62c806f670 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.348+0000 7f62cf7bb700 1 -- 172.21.15.5:0/3161625551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 msgr2=0x7f62c80cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.348+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/3161625551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c80cf500 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f62c806ea50 tx=0x7f62c4009b20 comp rx=0 tx=0).stop 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.348+0000 7f62cf7bb700 1 -- 172.21.15.5:0/3161625551 shutdown_connections 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.348+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/3161625551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c80cf500 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.348+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/3161625551 >> v2:172.21.15.49:3300/0 conn(0x7f62c806fa50 0x7f62c80b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.348+0000 7f62cf7bb700 1 -- 172.21.15.5:0/3161625551 >> 172.21.15.5:0/3161625551 conn(0x7f62c80713c0 msgr2=0x7f62c80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:48.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.348+0000 7f62cf7bb700 1 -- 172.21.15.5:0/3161625551 shutdown_connections 2024-05-08T23:25:48.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.348+0000 7f62cf7bb700 1 -- 172.21.15.5:0/3161625551 wait complete. 2024-05-08T23:25:48.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.349+0000 7f62cf7bb700 1 Processor -- start 2024-05-08T23:25:48.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.349+0000 7f62cf7bb700 1 -- start start 2024-05-08T23:25:48.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cf7bb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c815d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cf7bb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62c806fa50 0x7f62c8150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cf7bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62c80b4370 0x7f62c815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cf7bb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c815cd80 con 0x7f62c806f670 2024-05-08T23:25:48.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cf7bb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c8151490 con 0x7f62c80b4370 2024-05-08T23:25:48.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cf7bb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c815e620 con 0x7f62c806fa50 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cd557700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c815d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cd557700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c815d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49794/0 (socket says 172.21.15.5:49794) 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.350+0000 7f62cd557700 1 -- 172.21.15.5:0/2219895306 learned_addr learned my addr 172.21.15.5:0/2219895306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62ccd56700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62c806fa50 0x7f62c8150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cdd58700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62c80b4370 0x7f62c815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cd557700 1 -- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62c806fa50 msgr2=0x7f62c8150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cd557700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62c806fa50 0x7f62c8150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cd557700 1 -- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62c80b4370 msgr2=0x7f62c815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cd557700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62c80b4370 0x7f62c815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cd557700 1 -- 172.21.15.5:0/2219895306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c40094a0 con 0x7f62c806f670 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62ccd56700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62c806fa50 0x7f62c8150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cdd58700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62c80b4370 0x7f62c815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cd557700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c815d1a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f62c40090b0 tx=0x7f62c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62bdffb700 1 -- 172.21.15.5:0/2219895306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c40038a0 con 0x7f62c806f670 2024-05-08T23:25:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62bdffb700 1 -- 172.21.15.5:0/2219895306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c401f070 con 0x7f62c806f670 2024-05-08T23:25:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c815c6c0 con 0x7f62c806f670 2024-05-08T23:25:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62bdffb700 1 -- 172.21.15.5:0/2219895306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4003c20 con 0x7f62c806f670 2024-05-08T23:25:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.351+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c8161260 con 0x7f62c806f670 2024-05-08T23:25:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.352+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c815cff0 con 0x7f62c806f670 2024-05-08T23:25:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.353+0000 7f62bdffb700 1 -- 172.21.15.5:0/2219895306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f62c401a430 con 0x7f62c806f670 2024-05-08T23:25:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.353+0000 7f62bdffb700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62ac06ac90 0x7f62ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.353+0000 7f62bdffb700 1 -- 172.21.15.5:0/2219895306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62c4021070 con 0x7f62c806f670 2024-05-08T23:25:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.353+0000 7f62ccd56700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62ac06ac90 0x7f62ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.354+0000 7f62ccd56700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62ac06ac90 0x7f62ac06d150 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f62c8162070 tx=0x7f62b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:48.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.358+0000 7f62bdffb700 1 -- 172.21.15.5:0/2219895306 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f62c4055f40 con 0x7f62c806f670 2024-05-08T23:25:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.575+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62c8074bc0 con 0x7f62c806f670 2024-05-08T23:25:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.576+0000 7f62bdffb700 1 -- 172.21.15.5:0/2219895306 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62c402b020 con 0x7f62c806f670 2024-05-08T23:25:48.575 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62ac06ac90 msgr2=0x7f62ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62ac06ac90 0x7f62ac06d150 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f62c8162070 tx=0x7f62b8008040 comp rx=0 tx=0).stop 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 msgr2=0x7f62c815d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c815d1a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f62c40090b0 tx=0x7f62c4004750 comp rx=0 tx=0).stop 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 shutdown_connections 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62ac06ac90 0x7f62ac06d150 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62c80b4370 0x7f62c815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c806f670 0x7f62c815d1a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 --2- 172.21.15.5:0/2219895306 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62c806fa50 0x7f62c8150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 >> 172.21.15.5:0/2219895306 conn(0x7f62c80713c0 msgr2=0x7f62c815b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 shutdown_connections 2024-05-08T23:25:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.578+0000 7f62cf7bb700 1 -- 172.21.15.5:0/2219895306 wait complete. 2024-05-08T23:25:48.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:25:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.731+0000 7f0a03d26700 1 Processor -- start 2024-05-08T23:25:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.732+0000 7f0a03d26700 1 -- start start 2024-05-08T23:25:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.732+0000 7f0a03d26700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc06bd60 0x7f09fc0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.732+0000 7f0a03d26700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09fc0cdc90 0x7f09fc0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a03d26700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc0b1200 con 0x7f09fc0d13a0 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a03d26700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc0cfcc0 con 0x7f09fc0cdc90 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a03d26700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0ce5c0 con 0x7f09fc06bd60 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a012c1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc06bd60 0x7f09fc0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a022c3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09fc0cdc90 0x7f09fc0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a012c1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc06bd60 0x7f09fc0cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:59764/0 (socket says 172.21.15.5:59764) 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a012c1700 1 -- 172.21.15.5:0/2904417905 learned_addr learned my addr 172.21.15.5:0/2904417905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a01ac2700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f09fc0d13a0 0x7f09fc0d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:56350/0 (socket says 172.21.15.5:56350) 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a012c1700 1 -- 172.21.15.5:0/2904417905 >> v2:172.21.15.49:3300/0 conn(0x7f09fc0cdc90 msgr2=0x7f09fc0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a012c1700 1 --2- 172.21.15.5:0/2904417905 >> v2:172.21.15.49:3300/0 conn(0x7f09fc0cdc90 0x7f09fc0b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a012c1700 1 -- 172.21.15.5:0/2904417905 >> v1:172.21.15.49:6789/0 conn(0x7f09fc0d13a0 legacy=0x7f09fc0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a012c1700 1 -- 172.21.15.5:0/2904417905 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0ce8a0 con 0x7f09fc06bd60 2024-05-08T23:25:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.733+0000 7f0a012c1700 1 --2- 172.21.15.5:0/2904417905 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc06bd60 0x7f09fc0cf780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098f0 tx=0x7f09ec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76101c5c5f2d4996 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a00ac0700 1 -- 172.21.15.5:0/2904417905 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec011830 con 0x7f09fc06bd60 2024-05-08T23:25:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a00ac0700 1 -- 172.21.15.5:0/2904417905 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec011990 con 0x7f09fc06bd60 2024-05-08T23:25:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a00ac0700 1 -- 172.21.15.5:0/2904417905 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec011d20 con 0x7f09fc06bd60 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a03d26700 1 -- 172.21.15.5:0/2904417905 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc06bd60 msgr2=0x7f09fc0cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a03d26700 1 --2- 172.21.15.5:0/2904417905 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc06bd60 0x7f09fc0cf780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098f0 tx=0x7f09ec009c00 comp rx=0 tx=0).stop 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a03d26700 1 -- 172.21.15.5:0/2904417905 shutdown_connections 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a03d26700 1 --2- 172.21.15.5:0/2904417905 >> v2:172.21.15.49:3300/0 conn(0x7f09fc0cdc90 0x7f09fc0b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a03d26700 1 --2- 172.21.15.5:0/2904417905 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc06bd60 0x7f09fc0cf780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a03d26700 1 -- 172.21.15.5:0/2904417905 >> 172.21.15.5:0/2904417905 conn(0x7f09fc08afe0 msgr2=0x7f09fc08b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a03d26700 1 -- 172.21.15.5:0/2904417905 shutdown_connections 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.734+0000 7f0a03d26700 1 -- 172.21.15.5:0/2904417905 wait complete. 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.736+0000 7f0a03d26700 1 Processor -- start 2024-05-08T23:25:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.736+0000 7f0a03d26700 1 -- start start 2024-05-08T23:25:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.736+0000 7f0a03d26700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09fc06bd60 0x7f09fc0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.736+0000 7f0a03d26700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc0cdc90 0x7f09fc14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.736+0000 7f0a03d26700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09fc0d13a0 0x7f09fc15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.736+0000 7f0a03d26700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0cd7a0 con 0x7f09fc0d13a0 2024-05-08T23:25:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.736+0000 7f0a03d26700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0ccf40 con 0x7f09fc06bd60 2024-05-08T23:25:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.736+0000 7f0a03d26700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0b5410 con 0x7f09fc0cdc90 2024-05-08T23:25:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09fc0d13a0 0x7f09fc15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09fc0d13a0 0x7f09fc15d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49804/0 (socket says 172.21.15.5:49804) 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 -- 172.21.15.5:0/1172057040 learned_addr learned my addr 172.21.15.5:0/1172057040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a012c1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc0cdc90 0x7f09fc14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a01ac2700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09fc06bd60 0x7f09fc0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 -- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc0cdc90 msgr2=0x7f09fc14f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc0cdc90 0x7f09fc14f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 -- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09fc06bd60 msgr2=0x7f09fc0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09fc06bd60 0x7f09fc0cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 -- 172.21.15.5:0/1172057040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009550 con 0x7f09fc0d13a0 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a012c1700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc0cdc90 0x7f09fc14f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a022c3700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09fc0d13a0 0x7f09fc15d8c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009dd0 tx=0x7f09f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f09f27fc700 1 -- 172.21.15.5:0/1172057040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4009c40 con 0x7f09fc0d13a0 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f09f27fc700 1 -- 172.21.15.5:0/1172057040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f400f040 con 0x7f09fc0d13a0 2024-05-08T23:25:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f09f27fc700 1 -- 172.21.15.5:0/1172057040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400ceb0 con 0x7f09fc0d13a0 2024-05-08T23:25:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc0ae570 con 0x7f09fc0d13a0 2024-05-08T23:25:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.737+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc0aeb30 con 0x7f09fc0d13a0 2024-05-08T23:25:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.739+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc0b55d0 con 0x7f09fc0d13a0 2024-05-08T23:25:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.739+0000 7f09f27fc700 1 -- 172.21.15.5:0/1172057040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f09f4014480 con 0x7f09fc0d13a0 2024-05-08T23:25:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.739+0000 7f09f27fc700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09e006ac90 0x7f09e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.739+0000 7f09f27fc700 1 -- 172.21.15.5:0/1172057040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09f4089c10 con 0x7f09fc0d13a0 2024-05-08T23:25:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.740+0000 7f0a01ac2700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09e006ac90 0x7f09e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.740+0000 7f0a01ac2700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09e006ac90 0x7f09e006d150 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000f60 tx=0x7f09f8007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.744+0000 7f09f27fc700 1 -- 172.21.15.5:0/1172057040 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f09f4054cd0 con 0x7f09fc0d13a0 2024-05-08T23:25:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.965+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09fc074bc0 con 0x7f09fc0d13a0 2024-05-08T23:25:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.965+0000 7f09f27fc700 1 -- 172.21.15.5:0/1172057040 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09f40582f0 con 0x7f09fc0d13a0 2024-05-08T23:25:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:25:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09e006ac90 msgr2=0x7f09e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09e006ac90 0x7f09e006d150 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000f60 tx=0x7f09f8007ed0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09fc0d13a0 msgr2=0x7f09fc15d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09fc0d13a0 0x7f09fc15d8c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009dd0 tx=0x7f09f400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 shutdown_connections 2024-05-08T23:25:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09e006ac90 0x7f09e006d150 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09fc0d13a0 0x7f09fc15d8c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09fc06bd60 0x7f09fc0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 --2- 172.21.15.5:0/1172057040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09fc0cdc90 0x7f09fc14f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 >> 172.21.15.5:0/1172057040 conn(0x7f09fc08afe0 msgr2=0x7f09fc0cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 shutdown_connections 2024-05-08T23:25:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:48.968+0000 7f0a03d26700 1 -- 172.21.15.5:0/1172057040 wait complete. 2024-05-08T23:25:48.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:25:49.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.119+0000 7fa18832d700 1 Processor -- start 2024-05-08T23:25:49.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.119+0000 7fa18832d700 1 -- start start 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa18832d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa18015db00 0x7fa18006dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa18832d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18006fbf0 con 0x7fa1800cb2a0 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa18832d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18006edb0 con 0x7fa1800ce120 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa18832d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18006f090 con 0x7fa18015db00 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa1868ca700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa1800cb2a0 0x7fa1800cb6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51722/0 (socket says 172.21.15.5:51722) 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa1868ca700 1 -- 172.21.15.5:0/4135033489 learned_addr learned my addr 172.21.15.5:0/4135033489 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa1860c9700 1 --2- 172.21.15.5:0/4135033489 >> v2:172.21.15.49:3300/0 conn(0x7fa18015db00 0x7fa18006dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa1850c7700 1 -- 172.21.15.5:0/4135033489 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604342451 0 0) 0x7fa18006fbf0 con 0x7fa1800cb2a0 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.120+0000 7fa1850c7700 1 -- 172.21.15.5:0/4135033489 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa16c0057c0 con 0x7fa1800cb2a0 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1850c7700 1 -- 172.21.15.5:0/4135033489 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191621047 0 0) 0x7fa18006edb0 con 0x7fa1800ce120 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1850c7700 1 -- 172.21.15.5:0/4135033489 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa18006fbf0 con 0x7fa1800ce120 2024-05-08T23:25:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1860c9700 1 -- 172.21.15.5:0/4135033489 >> v1:172.21.15.49:6789/0 conn(0x7fa1800ce120 legacy=0x7fa18006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1860c9700 1 -- 172.21.15.5:0/4135033489 >> v1:172.21.15.5:6789/0 conn(0x7fa1800cb2a0 legacy=0x7fa1800cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1860c9700 1 -- 172.21.15.5:0/4135033489 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18006f370 con 0x7fa18015db00 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1860c9700 1 --2- 172.21.15.5:0/4135033489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015db00 0x7fa18006dc80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009840 tx=0x7fa17c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765cf5b9c89ef76c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1850c7700 1 -- 172.21.15.5:0/4135033489 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c011830 con 0x7fa18015db00 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1850c7700 1 -- 172.21.15.5:0/4135033489 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c011990 con 0x7fa18015db00 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.121+0000 7fa1850c7700 1 -- 172.21.15.5:0/4135033489 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c011d20 con 0x7fa18015db00 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.122+0000 7fa18832d700 1 -- 172.21.15.5:0/4135033489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015db00 msgr2=0x7fa18006dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.122+0000 7fa18832d700 1 --2- 172.21.15.5:0/4135033489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015db00 0x7fa18006dc80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009840 tx=0x7fa17c009b50 comp rx=0 tx=0).stop 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.122+0000 7fa18832d700 1 -- 172.21.15.5:0/4135033489 shutdown_connections 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.122+0000 7fa18832d700 1 --2- 172.21.15.5:0/4135033489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa18015db00 0x7fa18006dc80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.122+0000 7fa18832d700 1 -- 172.21.15.5:0/4135033489 >> 172.21.15.5:0/4135033489 conn(0x7fa180099a50 msgr2=0x7fa1800d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.122+0000 7fa18832d700 1 -- 172.21.15.5:0/4135033489 shutdown_connections 2024-05-08T23:25:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.122+0000 7fa18832d700 1 -- 172.21.15.5:0/4135033489 wait complete. 2024-05-08T23:25:49.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.123+0000 7fa18832d700 1 Processor -- start 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.123+0000 7fa18832d700 1 -- start start 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa18832d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1800cb2a0 0x7fa1801509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa18832d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1800ce120 0x7fa1801514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa18832d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa18015db00 0x7fa1800adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa18832d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa180156b60 con 0x7fa18015db00 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa18832d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa180155ee0 con 0x7fa1800cb2a0 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa18832d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801561c0 con 0x7fa1800ce120 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa18015db00 0x7fa1800adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa18015db00 0x7fa1800adac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49820/0 (socket says 172.21.15.5:49820) 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 -- 172.21.15.5:0/1499490779 learned_addr learned my addr 172.21.15.5:0/1499490779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1860c9700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1800cb2a0 0x7fa1801509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1858c8700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1800ce120 0x7fa1801514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 -- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1800ce120 msgr2=0x7fa1801514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1800ce120 0x7fa1801514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 -- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1800cb2a0 msgr2=0x7fa1801509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1800cb2a0 0x7fa1801509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 -- 172.21.15.5:0/1499490779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0094a0 con 0x7fa18015db00 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1858c8700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1800ce120 0x7fa1801514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.124+0000 7fa1868ca700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa18015db00 0x7fa1800adac0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa17800cba0 tx=0x7fa17800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.125+0000 7fa176ffd700 1 -- 172.21.15.5:0/1499490779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800aa70 con 0x7fa18015db00 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.125+0000 7fa176ffd700 1 -- 172.21.15.5:0/1499490779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17800abd0 con 0x7fa18015db00 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.125+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1801564a0 con 0x7fa18015db00 2024-05-08T23:25:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.125+0000 7fa176ffd700 1 -- 172.21.15.5:0/1499490779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178012400 con 0x7fa18015db00 2024-05-08T23:25:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.125+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa180155a50 con 0x7fa18015db00 2024-05-08T23:25:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.126+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180155c70 con 0x7fa18015db00 2024-05-08T23:25:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.127+0000 7fa176ffd700 1 -- 172.21.15.5:0/1499490779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa17801b3b0 con 0x7fa18015db00 2024-05-08T23:25:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.128+0000 7fa176ffd700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa16406ac90 0x7fa16406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.128+0000 7fa176ffd700 1 -- 172.21.15.5:0/1499490779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1780894f0 con 0x7fa18015db00 2024-05-08T23:25:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.128+0000 7fa1860c9700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa16406ac90 0x7fa16406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.129+0000 7fa1860c9700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa16406ac90 0x7fa16406d150 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009fd0 tx=0x7fa17c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.132+0000 7fa176ffd700 1 -- 172.21.15.5:0/1499490779 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa178058160 con 0x7fa18015db00 2024-05-08T23:25:49.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.349+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa180156ca0 con 0x7fa18015db00 2024-05-08T23:25:49.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.350+0000 7fa176ffd700 1 -- 172.21.15.5:0/1499490779 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa178057cf0 con 0x7fa18015db00 2024-05-08T23:25:49.350 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:25:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa16406ac90 msgr2=0x7fa16406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa16406ac90 0x7fa16406d150 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009fd0 tx=0x7fa17c01c040 comp rx=0 tx=0).stop 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa18015db00 msgr2=0x7fa1800adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa18015db00 0x7fa1800adac0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa17800cba0 tx=0x7fa17800ceb0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 shutdown_connections 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa16406ac90 0x7fa16406d150 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa18015db00 0x7fa1800adac0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1800cb2a0 0x7fa1801509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 --2- 172.21.15.5:0/1499490779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1800ce120 0x7fa1801514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 >> 172.21.15.5:0/1499490779 conn(0x7fa180099a50 msgr2=0x7fa1800cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 shutdown_connections 2024-05-08T23:25:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.352+0000 7fa18832d700 1 -- 172.21.15.5:0/1499490779 wait complete. 2024-05-08T23:25:49.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:25:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.501+0000 7f19679b2700 1 Processor -- start 2024-05-08T23:25:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.501+0000 7f19679b2700 1 -- start start 2024-05-08T23:25:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.502+0000 7f19679b2700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19600cf840 0x7f19600af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.502+0000 7f19679b2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f1960161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.502+0000 7f19679b2700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19600cd980 con 0x7f196006a960 2024-05-08T23:25:49.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.502+0000 7f19679b2700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19600ccdb0 con 0x7f19600cf840 2024-05-08T23:25:49.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.502+0000 7f19679b2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19600cd050 con 0x7f19600d0eb0 2024-05-08T23:25:49.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f1964f4d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f196006a960 0x7f196015d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51732/0 (socket says 172.21.15.5:51732) 2024-05-08T23:25:49.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f1964f4d700 1 -- 172.21.15.5:0/1745193849 learned_addr learned my addr 172.21.15.5:0/1745193849 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:49.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f196574e700 1 --2- 172.21.15.5:0/1745193849 >> v2:172.21.15.49:3300/0 conn(0x7f19600cf840 0x7f19600af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f1965f4f700 1 --2- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f1960161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f1957fff700 1 -- 172.21.15.5:0/1745193849 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102790887 0 0) 0x7f19600cd980 con 0x7f196006a960 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f1957fff700 1 -- 172.21.15.5:0/1745193849 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f194c0035b0 con 0x7f196006a960 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f196574e700 1 -- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 msgr2=0x7f1960161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f196574e700 1 --2- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f1960161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f1957fff700 1 -- 172.21.15.5:0/1745193849 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2047677384 0 0) 0x7f194c0035b0 con 0x7f196006a960 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f196574e700 1 -- 172.21.15.5:0/1745193849 >> v1:172.21.15.5:6789/0 conn(0x7f196006a960 legacy=0x7f196015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f196574e700 1 -- 172.21.15.5:0/1745193849 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19600cd330 con 0x7f19600cf840 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f1965f4f700 1 --2- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f1960161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.503+0000 7f196574e700 1 --2- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19600cf840 0x7f19600af440 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f195c009840 tx=0x7f195c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458af01333046c6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f1957fff700 1 -- 172.21.15.5:0/1745193849 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950001670 con 0x7f19600cf840 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f1957fff700 1 -- 172.21.15.5:0/1745193849 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c004030 con 0x7f19600cf840 2024-05-08T23:25:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f1957fff700 1 -- 172.21.15.5:0/1745193849 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0043a0 con 0x7f19600cf840 2024-05-08T23:25:49.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f19679b2700 1 -- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19600cf840 msgr2=0x7f19600af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f19679b2700 1 --2- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19600cf840 0x7f19600af440 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f195c009840 tx=0x7f195c009b50 comp rx=0 tx=0).stop 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f19679b2700 1 -- 172.21.15.5:0/1745193849 shutdown_connections 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f19679b2700 1 --2- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f1960161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f19679b2700 1 --2- 172.21.15.5:0/1745193849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19600cf840 0x7f19600af440 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f19679b2700 1 -- 172.21.15.5:0/1745193849 >> 172.21.15.5:0/1745193849 conn(0x7f19600d0320 msgr2=0x7f19600d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f19679b2700 1 -- 172.21.15.5:0/1745193849 shutdown_connections 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.504+0000 7f19679b2700 1 -- 172.21.15.5:0/1745193849 wait complete. 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f19679b2700 1 Processor -- start 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f19679b2700 1 -- start start 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f19679b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f196006a960 0x7f1960156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f19679b2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19600cf840 0x7f19600b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f19679b2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f196015e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f19679b2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196015f0a0 con 0x7f19600cf840 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f19679b2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196015eae0 con 0x7f196006a960 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f19679b2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196015edc0 con 0x7f19600d0eb0 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.506+0000 7f1964f4d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19600cf840 0x7f19600b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1964f4d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19600cf840 0x7f19600b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49836/0 (socket says 172.21.15.5:49836) 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1964f4d700 1 -- 172.21.15.5:0/2646462937 learned_addr learned my addr 172.21.15.5:0/2646462937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1965f4f700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f196015e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f196574e700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f196006a960 0x7f1960156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1964f4d700 1 -- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 msgr2=0x7f196015e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1964f4d700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f196015e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1964f4d700 1 -- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f196006a960 msgr2=0x7f1960156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1964f4d700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f196006a960 0x7f1960156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1964f4d700 1 -- 172.21.15.5:0/2646462937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009a70 con 0x7f19600cf840 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1964f4d700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19600cf840 0x7f19600b4000 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f195000c520 tx=0x7f195000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1955ffb700 1 -- 172.21.15.5:0/2646462937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950004c80 con 0x7f19600cf840 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1955ffb700 1 -- 172.21.15.5:0/2646462937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1950004de0 con 0x7f19600cf840 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c0094a0 con 0x7f19600cf840 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.507+0000 7f1955ffb700 1 -- 172.21.15.5:0/2646462937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950007310 con 0x7f19600cf840 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.508+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19600b3ae0 con 0x7f19600cf840 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.508+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19600b3d00 con 0x7f19600cf840 2024-05-08T23:25:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.509+0000 7f1955ffb700 1 -- 172.21.15.5:0/2646462937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1950007470 con 0x7f19600cf840 2024-05-08T23:25:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.509+0000 7f1955ffb700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f194406ac90 0x7f194406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.509+0000 7f1955ffb700 1 -- 172.21.15.5:0/2646462937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1950088f20 con 0x7f19600cf840 2024-05-08T23:25:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.510+0000 7f196574e700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f194406ac90 0x7f194406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.510+0000 7f196574e700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f194406ac90 0x7f194406d150 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f195c0090b0 tx=0x7f195c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.514+0000 7f1955ffb700 1 -- 172.21.15.5:0/2646462937 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1950054190 con 0x7f19600cf840 2024-05-08T23:25:49.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.780+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1960068000 con 0x7f19600cf840 2024-05-08T23:25:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.781+0000 7f1955ffb700 1 -- 172.21.15.5:0/2646462937 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4765 (secure 0 0 0) 0x7f1950058e60 con 0x7f19600cf840 2024-05-08T23:25:49.781 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:49.781 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:25:18.479282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6525,"mds_1":6516,"mds_2":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":2,"incarnation":53,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6516":{"gid":6516,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f194406ac90 msgr2=0x7f194406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f194406ac90 0x7f194406d150 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f195c0090b0 tx=0x7f195c008030 comp rx=0 tx=0).stop 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19600cf840 msgr2=0x7f19600b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19600cf840 0x7f19600b4000 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f195000c520 tx=0x7f195000c830 comp rx=0 tx=0).stop 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 shutdown_connections 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f194406ac90 0x7f194406d150 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19600d0eb0 0x7f196015e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f196006a960 0x7f1960156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.783+0000 7f19679b2700 1 --2- 172.21.15.5:0/2646462937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19600cf840 0x7f19600b4000 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.784+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 >> 172.21.15.5:0/2646462937 conn(0x7f19600d0320 msgr2=0x7f19600ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.784+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 shutdown_connections 2024-05-08T23:25:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.784+0000 7f19679b2700 1 -- 172.21.15.5:0/2646462937 wait complete. 2024-05-08T23:25:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 59 2024-05-08T23:25:49.797 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:25:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.934+0000 7f70a9646700 1 Processor -- start 2024-05-08T23:25:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.934+0000 7f70a9646700 1 -- start start 2024-05-08T23:25:49.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a9646700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70a415d780 0x7f70a4161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a9646700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f70a406c450 0x7f70a41619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a9646700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a406c9c0 con 0x7f70a415db60 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a9646700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a406cca0 con 0x7f70a415d780 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a9646700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a406cf80 con 0x7f70a406c450 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a2ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70a415d780 0x7f70a4161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a2ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70a415d780 0x7f70a4161470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49844/0 (socket says 172.21.15.5:49844) 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a2ffd700 1 -- 172.21.15.5:0/1461749092 learned_addr learned my addr 172.21.15.5:0/1461749092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a37fe700 1 --2- 172.21.15.5:0/1461749092 >> v2:172.21.15.49:3300/0 conn(0x7f70a406c450 0x7f70a41619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a2ffd700 1 -- 172.21.15.5:0/1461749092 >> v2:172.21.15.49:3300/0 conn(0x7f70a406c450 msgr2=0x7f70a41619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a2ffd700 1 --2- 172.21.15.5:0/1461749092 >> v2:172.21.15.49:3300/0 conn(0x7f70a406c450 0x7f70a41619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a2ffd700 1 -- 172.21.15.5:0/1461749092 >> v1:172.21.15.49:6789/0 conn(0x7f70a415db60 legacy=0x7f70a40ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.935+0000 7f70a2ffd700 1 -- 172.21.15.5:0/1461749092 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a406d260 con 0x7f70a415d780 2024-05-08T23:25:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a2ffd700 1 --2- 172.21.15.5:0/1461749092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a4161470 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f70a41613b0 tx=0x7f7094009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a075bd9e22135b74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a1ffb700 1 -- 172.21.15.5:0/1461749092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094004230 con 0x7f70a415d780 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a1ffb700 1 -- 172.21.15.5:0/1461749092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7094004390 con 0x7f70a415d780 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a1ffb700 1 -- 172.21.15.5:0/1461749092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70940036a0 con 0x7f70a415d780 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a9646700 1 -- 172.21.15.5:0/1461749092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 msgr2=0x7f70a4161470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a9646700 1 --2- 172.21.15.5:0/1461749092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a4161470 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f70a41613b0 tx=0x7f7094009bd0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a9646700 1 -- 172.21.15.5:0/1461749092 shutdown_connections 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a9646700 1 --2- 172.21.15.5:0/1461749092 >> v2:172.21.15.49:3300/0 conn(0x7f70a406c450 0x7f70a41619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a9646700 1 --2- 172.21.15.5:0/1461749092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a4161470 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a9646700 1 -- 172.21.15.5:0/1461749092 >> 172.21.15.5:0/1461749092 conn(0x7f70a408afe0 msgr2=0x7f70a40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a9646700 1 -- 172.21.15.5:0/1461749092 shutdown_connections 2024-05-08T23:25:49.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.936+0000 7f70a9646700 1 -- 172.21.15.5:0/1461749092 wait complete. 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.938+0000 7f70a9646700 1 Processor -- start 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.938+0000 7f70a9646700 1 -- start start 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a9646700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a406c450 0x7f70a4151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a9646700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a9646700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a415db60 0x7f70a40c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a9646700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4152050 con 0x7f70a415d780 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a9646700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40b8730 con 0x7f70a406c450 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a9646700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40b9100 con 0x7f70a415db60 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a40b8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49852/0 (socket says 172.21.15.5:49852) 2024-05-08T23:25:49.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 -- 172.21.15.5:0/204709608 learned_addr learned my addr 172.21.15.5:0/204709608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a2ffd700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a406c450 0x7f70a4151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a37fe700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a415db60 0x7f70a40c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 -- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a415db60 msgr2=0x7f70a40c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a415db60 0x7f70a40c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 -- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a406c450 msgr2=0x7f70a4151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a406c450 0x7f70a4151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 -- 172.21.15.5:0/204709608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009550 con 0x7f70a415d780 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a2ffd700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a406c450 0x7f70a4151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.939+0000 7f70a27fc700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a40b8bc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f709800b450 tx=0x7f709800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.940+0000 7f708b7fe700 1 -- 172.21.15.5:0/204709608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098011830 con 0x7f70a415d780 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.940+0000 7f708b7fe700 1 -- 172.21.15.5:0/204709608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7098011990 con 0x7f70a415d780 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.940+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a40b7a40 con 0x7f70a415d780 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.940+0000 7f708b7fe700 1 -- 172.21.15.5:0/204709608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098011d40 con 0x7f70a415d780 2024-05-08T23:25:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.940+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a40b83c0 con 0x7f70a415d780 2024-05-08T23:25:49.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.941+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a40678f0 con 0x7f70a415d780 2024-05-08T23:25:49.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.941+0000 7f708b7fe700 1 -- 172.21.15.5:0/204709608 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f709801a400 con 0x7f70a415d780 2024-05-08T23:25:49.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.942+0000 7f708b7fe700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f709006ac90 0x7f709006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:49.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.942+0000 7f708b7fe700 1 -- 172.21.15.5:0/204709608 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7098089070 con 0x7f70a415d780 2024-05-08T23:25:49.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.942+0000 7f70a2ffd700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f709006ac90 0x7f709006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:49.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.943+0000 7f70a2ffd700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f709006ac90 0x7f709006d150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7094009c10 tx=0x7f7094008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:49.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:49.946+0000 7f708b7fe700 1 -- 172.21.15.5:0/204709608 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f70980541c0 con 0x7f70a415d780 2024-05-08T23:25:50.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.167+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70a406da60 con 0x7f70a415d780 2024-05-08T23:25:50.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.168+0000 7f708b7fe700 1 -- 172.21.15.5:0/204709608 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16001 (secure 0 0 0) 0x7f7098015090 con 0x7f70a415d780 2024-05-08T23:25:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:50.169 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:46.285230+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:24:43.276676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T23:24:44.254746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:25:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f709006ac90 msgr2=0x7f709006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f709006ac90 0x7f709006d150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7094009c10 tx=0x7f7094008030 comp rx=0 tx=0).stop 2024-05-08T23:25:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 msgr2=0x7f70a40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a40b8bc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f709800b450 tx=0x7f709800b810 comp rx=0 tx=0).stop 2024-05-08T23:25:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 shutdown_connections 2024-05-08T23:25:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f709006ac90 0x7f709006d150 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f70a415db60 0x7f70a40c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70a406c450 0x7f70a4151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 --2- 172.21.15.5:0/204709608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70a415d780 0x7f70a40b8bc0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 >> 172.21.15.5:0/204709608 conn(0x7f70a408afe0 msgr2=0x7f70a40cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 shutdown_connections 2024-05-08T23:25:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.171+0000 7f70a9646700 1 -- 172.21.15.5:0/204709608 wait complete. 2024-05-08T23:25:50.185 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:25:50.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:25:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.321+0000 7f6bf7bd2700 1 Processor -- start 2024-05-08T23:25:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.321+0000 7f6bf7bd2700 1 -- start start 2024-05-08T23:25:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.322+0000 7f6bf7bd2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015e200 0x7f6bf015e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.322+0000 7f6bf7bd2700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf00ce740 con 0x7f6bf015a630 2024-05-08T23:25:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.322+0000 7f6bf7bd2700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf015eb90 con 0x7f6bf00cdef0 2024-05-08T23:25:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.322+0000 7f6bf7bd2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf00cb670 con 0x7f6bf015e200 2024-05-08T23:25:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.322+0000 7f6bf596e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6bf015a630 0x7f6bf00d0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51744/0 (socket says 172.21.15.5:51744) 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.322+0000 7f6bf596e700 1 -- 172.21.15.5:0/1453387383 learned_addr learned my addr 172.21.15.5:0/1453387383 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.322+0000 7f6bf616f700 1 --2- 172.21.15.5:0/1453387383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015e200 0x7f6bf015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf496c700 1 -- 172.21.15.5:0/1453387383 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253926807 0 0) 0x7f6bec001110 con 0x7f6bf015a630 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf496c700 1 -- 172.21.15.5:0/1453387383 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bdc0035b0 con 0x7f6bf015a630 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf496c700 1 -- 172.21.15.5:0/1453387383 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576256398 0 0) 0x7f6be0001110 con 0x7f6bf00cdef0 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf496c700 1 -- 172.21.15.5:0/1453387383 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bec001110 con 0x7f6bf00cdef0 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf616f700 1 -- 172.21.15.5:0/1453387383 >> v1:172.21.15.49:6789/0 conn(0x7f6bf00cdef0 legacy=0x7f6bf00cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf616f700 1 -- 172.21.15.5:0/1453387383 >> v1:172.21.15.5:6789/0 conn(0x7f6bf015a630 legacy=0x7f6bf00d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf616f700 1 -- 172.21.15.5:0/1453387383 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf00cadc0 con 0x7f6bf015e200 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf616f700 1 --2- 172.21.15.5:0/1453387383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015e200 0x7f6bf015e650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009dc0 tx=0x7f6be800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb7a577bcc248b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf496c700 1 -- 172.21.15.5:0/1453387383 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be800e070 con 0x7f6bf015e200 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf496c700 1 -- 172.21.15.5:0/1453387383 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be800bb20 con 0x7f6bf015e200 2024-05-08T23:25:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.323+0000 7f6bf496c700 1 -- 172.21.15.5:0/1453387383 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be800be90 con 0x7f6bf015e200 2024-05-08T23:25:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.324+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/1453387383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015e200 msgr2=0x7f6bf015e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.324+0000 7f6bf7bd2700 1 --2- 172.21.15.5:0/1453387383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015e200 0x7f6bf015e650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009dc0 tx=0x7f6be800b330 comp rx=0 tx=0).stop 2024-05-08T23:25:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.324+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/1453387383 shutdown_connections 2024-05-08T23:25:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.324+0000 7f6bf7bd2700 1 --2- 172.21.15.5:0/1453387383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015e200 0x7f6bf015e650 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.324+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/1453387383 >> 172.21.15.5:0/1453387383 conn(0x7f6bf0099a50 msgr2=0x7f6bf00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.324+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/1453387383 shutdown_connections 2024-05-08T23:25:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.324+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/1453387383 wait complete. 2024-05-08T23:25:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf7bd2700 1 Processor -- start 2024-05-08T23:25:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf7bd2700 1 -- start start 2024-05-08T23:25:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf7bd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf00cdef0 0x7f6bf014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf7bd2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015a630 0x7f6bf00c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf7bd2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf015e200 0x7f6bf00c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf7bd2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf014dfe0 con 0x7f6bf00cdef0 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf7bd2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf014b090 con 0x7f6bf015e200 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf7bd2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf014b370 con 0x7f6bf015a630 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.326+0000 7f6bf596e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf00cdef0 0x7f6bf014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf596e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf00cdef0 0x7f6bf014daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49862/0 (socket says 172.21.15.5:49862) 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf596e700 1 -- 172.21.15.5:0/2684049488 learned_addr learned my addr 172.21.15.5:0/2684049488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf616f700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf015e200 0x7f6bf00c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf516d700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015a630 0x7f6bf00c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf596e700 1 -- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015a630 msgr2=0x7f6bf00c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf596e700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015a630 0x7f6bf00c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf596e700 1 -- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf015e200 msgr2=0x7f6bf00c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf596e700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf015e200 0x7f6bf00c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf596e700 1 -- 172.21.15.5:0/2684049488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8009a70 con 0x7f6bf00cdef0 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf596e700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf00cdef0 0x7f6bf014daa0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00c700 tx=0x7f6bec00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6be67fc700 1 -- 172.21.15.5:0/2684049488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec005d70 con 0x7f6bf00cdef0 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6be67fc700 1 -- 172.21.15.5:0/2684049488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec0070f0 con 0x7f6bf00cdef0 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf014b650 con 0x7f6bf00cdef0 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6be67fc700 1 -- 172.21.15.5:0/2684049488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0074a0 con 0x7f6bf00cdef0 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.327+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf014bc10 con 0x7f6bf00cdef0 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.328+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf014be30 con 0x7f6bf00cdef0 2024-05-08T23:25:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.329+0000 7f6be67fc700 1 -- 172.21.15.5:0/2684049488 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6bec007600 con 0x7f6bf00cdef0 2024-05-08T23:25:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.329+0000 7f6be67fc700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bd406ac90 0x7f6bd406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.329+0000 7f6be67fc700 1 -- 172.21.15.5:0/2684049488 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bec088880 con 0x7f6bf00cdef0 2024-05-08T23:25:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.329+0000 7f6bf516d700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bd406ac90 0x7f6bd406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.330+0000 7f6bf516d700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bd406ac90 0x7f6bd406d150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6bf015de00 tx=0x7f6be0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.334+0000 7f6be67fc700 1 -- 172.21.15.5:0/2684049488 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6bec053a60 con 0x7f6bf00cdef0 2024-05-08T23:25:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.598+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bf00ce740 con 0x7f6bf00cdef0 2024-05-08T23:25:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.598+0000 7f6be67fc700 1 -- 172.21.15.5:0/2684049488 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4765 (secure 0 0 0) 0x7f6bf00ce740 con 0x7f6bf00cdef0 2024-05-08T23:25:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:50.599 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:25:18.479282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6525,"mds_1":6516,"mds_2":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":2,"incarnation":53,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6516":{"gid":6516,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:25:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bd406ac90 msgr2=0x7f6bd406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bd406ac90 0x7f6bd406d150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6bf015de00 tx=0x7f6be0009040 comp rx=0 tx=0).stop 2024-05-08T23:25:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf00cdef0 msgr2=0x7f6bf014daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf00cdef0 0x7f6bf014daa0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00c700 tx=0x7f6bec00cac0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 shutdown_connections 2024-05-08T23:25:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bd406ac90 0x7f6bd406d150 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf015e200 0x7f6bf00c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf00cdef0 0x7f6bf014daa0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 --2- 172.21.15.5:0/2684049488 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bf015a630 0x7f6bf00c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 >> 172.21.15.5:0/2684049488 conn(0x7f6bf0099a50 msgr2=0x7f6bf006f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 shutdown_connections 2024-05-08T23:25:50.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.601+0000 7f6bf7bd2700 1 -- 172.21.15.5:0/2684049488 wait complete. 2024-05-08T23:25:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 59 2024-05-08T23:25:50.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:25:50.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.751+0000 7fb9b192a700 1 Processor -- start 2024-05-08T23:25:50.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.751+0000 7fb9b192a700 1 -- start start 2024-05-08T23:25:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.751+0000 7fb9b192a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.751+0000 7fb9b192a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac06c400 con 0x7fb9ac0b2700 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9b192a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac0d0cd0 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9b192a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0d09f0 con 0x7fb9ac15db10 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9aa7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb9ac0b2700 0x7fb9ac06c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51760/0 (socket says 172.21.15.5:51760) 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9aa7fc700 1 -- 172.21.15.5:0/483837799 learned_addr learned my addr 172.21.15.5:0/483837799 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9aaffd700 1 --2- 172.21.15.5:0/483837799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9a9ffb700 1 -- 172.21.15.5:0/483837799 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137006492 0 0) 0x7fb9ac06c400 con 0x7fb9ac0b2700 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9a9ffb700 1 -- 172.21.15.5:0/483837799 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9940035b0 con 0x7fb9ac0b2700 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9a9ffb700 1 -- 172.21.15.5:0/483837799 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487259690 0 0) 0x7fb9ac0d0cd0 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9a9ffb700 1 -- 172.21.15.5:0/483837799 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ac06c400 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9aaffd700 1 -- 172.21.15.5:0/483837799 >> v1:172.21.15.49:6789/0 conn(0x7fb9ac0cbb00 legacy=0x7fb9ac1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9aaffd700 1 -- 172.21.15.5:0/483837799 >> v1:172.21.15.5:6789/0 conn(0x7fb9ac0b2700 legacy=0x7fb9ac06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9aaffd700 1 -- 172.21.15.5:0/483837799 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac06bdb0 con 0x7fb9ac15db10 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.752+0000 7fb9aaffd700 1 --2- 172.21.15.5:0/483837799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac0d0410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009df0 tx=0x7fb99c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=588a13903ff0cce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9a9ffb700 1 -- 172.21.15.5:0/483837799 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00e070 con 0x7fb9ac15db10 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9a9ffb700 1 -- 172.21.15.5:0/483837799 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c00bae0 con 0x7fb9ac15db10 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9a9ffb700 1 -- 172.21.15.5:0/483837799 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00be90 con 0x7fb9ac15db10 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9b192a700 1 -- 172.21.15.5:0/483837799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 msgr2=0x7fb9ac0d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9b192a700 1 --2- 172.21.15.5:0/483837799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac0d0410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009df0 tx=0x7fb99c00b330 comp rx=0 tx=0).stop 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9b192a700 1 -- 172.21.15.5:0/483837799 shutdown_connections 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9b192a700 1 --2- 172.21.15.5:0/483837799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac0d0410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9b192a700 1 -- 172.21.15.5:0/483837799 >> 172.21.15.5:0/483837799 conn(0x7fb9ac0b2fa0 msgr2=0x7fb9ac0b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9b192a700 1 -- 172.21.15.5:0/483837799 shutdown_connections 2024-05-08T23:25:50.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.753+0000 7fb9b192a700 1 -- 172.21.15.5:0/483837799 wait complete. 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.755+0000 7fb9b192a700 1 Processor -- start 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.755+0000 7fb9b192a700 1 -- start start 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9b192a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ac0b2700 0x7fb9ac0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9b192a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9ac0cbb00 0x7fb9ac0703b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9b192a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac1606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9b192a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac15ea30 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9b192a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0cb480 con 0x7fb9ac0b2700 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9b192a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0cab40 con 0x7fb9ac15db10 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9ac0cbb00 0x7fb9ac0703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9ac0cbb00 0x7fb9ac0703b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49872/0 (socket says 172.21.15.5:49872) 2024-05-08T23:25:50.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 -- 172.21.15.5:0/912801284 learned_addr learned my addr 172.21.15.5:0/912801284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aaffd700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ac0b2700 0x7fb9ac0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9ab7fe700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac1606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 -- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 msgr2=0x7fb9ac1606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac1606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 -- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ac0b2700 msgr2=0x7fb9ac0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ac0b2700 0x7fb9ac0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 -- 172.21.15.5:0/912801284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c009a70 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aaffd700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ac0b2700 0x7fb9ac0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb9aa7fc700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9ac0cbb00 0x7fb9ac0703b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c700 tx=0x7fb9a000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb993fff700 1 -- 172.21.15.5:0/912801284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0005d70 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.756+0000 7fb993fff700 1 -- 172.21.15.5:0/912801284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a00070d0 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.757+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac0b4690 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.757+0000 7fb993fff700 1 -- 172.21.15.5:0/912801284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0007460 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.757+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac0b4c50 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.758+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac15eb70 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.758+0000 7fb993fff700 1 -- 172.21.15.5:0/912801284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb9a00075c0 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.758+0000 7fb993fff700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb99806ac90 0x7fb99806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:50.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.759+0000 7fb993fff700 1 -- 172.21.15.5:0/912801284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9a00888c0 con 0x7fb9ac0cbb00 2024-05-08T23:25:50.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.759+0000 7fb9aaffd700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb99806ac90 0x7fb99806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:50.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.759+0000 7fb9aaffd700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb99806ac90 0x7fb99806d150 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb99c014040 tx=0x7fb99c01d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:50.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:50.763+0000 7fb993fff700 1 -- 172.21.15.5:0/912801284 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb9ac15eb70 con 0x7fb9ac0cbb00 2024-05-08T23:25:51.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.029+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9ac15eb70 con 0x7fb9ac0cbb00 2024-05-08T23:25:51.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.030+0000 7fb993fff700 1 -- 172.21.15.5:0/912801284 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4765 (secure 0 0 0) 0x7fb9ac15eb70 con 0x7fb9ac0cbb00 2024-05-08T23:25:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:51.030 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:24:45.283486+0000","modified":"2024-05-08T23:25:18.479282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6525,"mds_1":6516,"mds_2":6193},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6193":{"gid":6193,"name":"d","rank":2,"incarnation":53,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6836/3197377913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3197377913},{"type":"v1","addr":"172.21.15.49:6836","nonce":3197377913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6516":{"gid":6516,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6833/2315721152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2315721152},{"type":"v1","addr":"172.21.15.5:6833","nonce":2315721152}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6525":{"gid":6525,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1169758885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1169758885},{"type":"v1","addr":"172.21.15.5:6835","nonce":1169758885}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T23:25:51.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb99806ac90 msgr2=0x7fb99806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb99806ac90 0x7fb99806d150 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb99c014040 tx=0x7fb99c01d040 comp rx=0 tx=0).stop 2024-05-08T23:25:51.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9ac0cbb00 msgr2=0x7fb9ac0703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9ac0cbb00 0x7fb9ac0703b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c700 tx=0x7fb9a000cac0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 shutdown_connections 2024-05-08T23:25:51.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb99806ac90 0x7fb99806d150 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9ac15db10 0x7fb9ac1606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ac0b2700 0x7fb9ac0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 --2- 172.21.15.5:0/912801284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9ac0cbb00 0x7fb9ac0703b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 >> 172.21.15.5:0/912801284 conn(0x7fb9ac0b2fa0 msgr2=0x7fb9ac15b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:51.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 shutdown_connections 2024-05-08T23:25:51.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.032+0000 7fb9b192a700 1 -- 172.21.15.5:0/912801284 wait complete. 2024-05-08T23:25:51.034 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 59 2024-05-08T23:25:51.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:25:51.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.181+0000 7fab3e7fd700 1 Processor -- start 2024-05-08T23:25:51.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.181+0000 7fab3e7fd700 1 -- start start 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3e7fd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fab380af030 0x7fab3815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3e7fd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab380b3990 0x7fab3806ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3e7fd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab3806e300 0x7fab38161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3e7fd700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3806e790 con 0x7fab3806e300 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3e7fd700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3815ede0 con 0x7fab380af030 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3e7fd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab380b4a80 con 0x7fab380b3990 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3cd9a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab3806e300 0x7fab38161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3cd9a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab3806e300 0x7fab38161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49888/0 (socket says 172.21.15.5:49888) 2024-05-08T23:25:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab3cd9a700 1 -- 172.21.15.5:0/590233785 learned_addr learned my addr 172.21.15.5:0/590233785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab37fff700 1 --2- 172.21.15.5:0/590233785 >> v2:172.21.15.49:3300/0 conn(0x7fab380af030 0x7fab3815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.182+0000 7fab377fe700 1 --2- 172.21.15.5:0/590233785 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab380b3990 0x7fab3806ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab3cd9a700 1 -- 172.21.15.5:0/590233785 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab380b3990 msgr2=0x7fab3806ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab3cd9a700 1 --2- 172.21.15.5:0/590233785 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab380b3990 0x7fab3806ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab3cd9a700 1 -- 172.21.15.5:0/590233785 >> v2:172.21.15.49:3300/0 conn(0x7fab380af030 msgr2=0x7fab3815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab3cd9a700 1 --2- 172.21.15.5:0/590233785 >> v2:172.21.15.49:3300/0 conn(0x7fab380af030 0x7fab3815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab3cd9a700 1 -- 172.21.15.5:0/590233785 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3806d800 con 0x7fab3806e300 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab3cd9a700 1 --2- 172.21.15.5:0/590233785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3806e300 0x7fab38161410 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fab3806feb0 tx=0x7fab3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c8e6b0ba20b12b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab36ffd700 1 -- 172.21.15.5:0/590233785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000e070 con 0x7fab3806e300 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab36ffd700 1 -- 172.21.15.5:0/590233785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000b910 con 0x7fab3806e300 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.183+0000 7fab36ffd700 1 -- 172.21.15.5:0/590233785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000bcc0 con 0x7fab3806e300 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 -- 172.21.15.5:0/590233785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3806e300 msgr2=0x7fab38161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/590233785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3806e300 0x7fab38161410 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fab3806feb0 tx=0x7fab3000b330 comp rx=0 tx=0).stop 2024-05-08T23:25:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 -- 172.21.15.5:0/590233785 shutdown_connections 2024-05-08T23:25:51.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/590233785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3806e300 0x7fab38161410 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/590233785 >> v2:172.21.15.49:3300/0 conn(0x7fab380af030 0x7fab3815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/590233785 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab380b3990 0x7fab3806ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 -- 172.21.15.5:0/590233785 >> 172.21.15.5:0/590233785 conn(0x7fab380cfce0 msgr2=0x7fab380d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:51.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 -- 172.21.15.5:0/590233785 shutdown_connections 2024-05-08T23:25:51.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.184+0000 7fab3e7fd700 1 -- 172.21.15.5:0/590233785 wait complete. 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.186+0000 7fab3e7fd700 1 Processor -- start 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.186+0000 7fab3e7fd700 1 -- start start 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab3e7fd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3806e300 0x7fab381f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab3e7fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab380af030 0x7fab3806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab3e7fd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab380b3990 0x7fab380c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab3e7fd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab381f2f70 con 0x7fab380af030 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab3e7fd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab381f7460 con 0x7fab380b3990 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab3e7fd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab381f31f0 con 0x7fab3806e300 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab380af030 0x7fab3806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab380af030 0x7fab3806bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49896/0 (socket says 172.21.15.5:49896) 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 -- 172.21.15.5:0/2720589291 learned_addr learned my addr 172.21.15.5:0/2720589291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab37fff700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3806e300 0x7fab381f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab3cd9a700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab380b3990 0x7fab380c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 -- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3806e300 msgr2=0x7fab381f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3806e300 0x7fab381f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 -- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab380b3990 msgr2=0x7fab380c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab380b3990 0x7fab380c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 -- 172.21.15.5:0/2720589291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009b30 con 0x7fab380af030 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab3cd9a700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab380b3990 0x7fab380c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.187+0000 7fab377fe700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab380af030 0x7fab3806bd10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00e870 tx=0x7fab2c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.188+0000 7fab34ff9700 1 -- 172.21.15.5:0/2720589291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c009590 con 0x7fab380af030 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.188+0000 7fab34ff9700 1 -- 172.21.15.5:0/2720589291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c00ee70 con 0x7fab380af030 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.188+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab381f34d0 con 0x7fab380af030 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.188+0000 7fab34ff9700 1 -- 172.21.15.5:0/2720589291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00ca50 con 0x7fab380af030 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.188+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab381f3a90 con 0x7fab380af030 2024-05-08T23:25:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.189+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3806f860 con 0x7fab380af030 2024-05-08T23:25:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.189+0000 7fab34ff9700 1 -- 172.21.15.5:0/2720589291 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fab2c00cbb0 con 0x7fab380af030 2024-05-08T23:25:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.190+0000 7fab34ff9700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab2406ac90 0x7fab2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.190+0000 7fab34ff9700 1 -- 172.21.15.5:0/2720589291 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab2c089330 con 0x7fab380af030 2024-05-08T23:25:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.190+0000 7fab37fff700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab2406ac90 0x7fab2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.190+0000 7fab37fff700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab2406ac90 0x7fab2406d150 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab381f64c0 tx=0x7fab28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:51.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.194+0000 7fab34ff9700 1 -- 172.21.15.5:0/2720589291 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fab2c054510 con 0x7fab380af030 2024-05-08T23:25:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.416+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab38068000 con 0x7fab380af030 2024-05-08T23:25:51.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.417+0000 7fab34ff9700 1 -- 172.21.15.5:0/2720589291 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16001 (secure 0 0 0) 0x7fab2c00ce60 con 0x7fab380af030 2024-05-08T23:25:51.417 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:51.418 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:24:46.285230+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:24:43.276676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T23:24:44.254746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:25:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab2406ac90 msgr2=0x7fab2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab2406ac90 0x7fab2406d150 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab381f64c0 tx=0x7fab28008040 comp rx=0 tx=0).stop 2024-05-08T23:25:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab380af030 msgr2=0x7fab3806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab380af030 0x7fab3806bd10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00e870 tx=0x7fab2c00ec30 comp rx=0 tx=0).stop 2024-05-08T23:25:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 shutdown_connections 2024-05-08T23:25:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab2406ac90 0x7fab2406d150 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab380b3990 0x7fab380c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3806e300 0x7fab381f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 --2- 172.21.15.5:0/2720589291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab380af030 0x7fab3806bd10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 >> 172.21.15.5:0/2720589291 conn(0x7fab380cfce0 msgr2=0x7fab380ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 shutdown_connections 2024-05-08T23:25:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.420+0000 7fab3e7fd700 1 -- 172.21.15.5:0/2720589291 wait complete. 2024-05-08T23:25:51.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:25:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.571+0000 7f6b7eb28700 1 Processor -- start 2024-05-08T23:25:51.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.571+0000 7f6b7eb28700 1 -- start start 2024-05-08T23:25:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.572+0000 7f6b7eb28700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cc060 0x7f6b780cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.572+0000 7f6b7eb28700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6b780cfde0 0x7f6b7806a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.572+0000 7f6b7eb28700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780cd7c0 con 0x7f6b7806aea0 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.572+0000 7f6b7eb28700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7806b330 con 0x7f6b780cfde0 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.572+0000 7f6b7eb28700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780ccc80 con 0x7f6b780cc060 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7c8c4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cc060 0x7f6b780cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b77fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6b780cfde0 0x7f6b7806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7d0c5700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6b7806aea0 0x7f6b781616a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:56402/0 (socket says 172.21.15.5:56402) 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7c8c4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cc060 0x7f6b780cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:59870/0 (socket says 172.21.15.5:59870) 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7d0c5700 1 -- 172.21.15.5:0/1706299384 learned_addr learned my addr 172.21.15.5:0/1706299384 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b777fe700 1 -- 172.21.15.5:0/1706299384 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429306808 0 0) 0x7f6b780cd7c0 con 0x7f6b7806aea0 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b777fe700 1 -- 172.21.15.5:0/1706299384 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b600035b0 con 0x7f6b7806aea0 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7c8c4700 1 -- 172.21.15.5:0/1706299384 >> v2:172.21.15.49:3300/0 conn(0x7f6b780cfde0 msgr2=0x7f6b7806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7c8c4700 1 --2- 172.21.15.5:0/1706299384 >> v2:172.21.15.49:3300/0 conn(0x7f6b780cfde0 0x7f6b7806a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7c8c4700 1 -- 172.21.15.5:0/1706299384 >> v1:172.21.15.49:6789/0 conn(0x7f6b7806aea0 legacy=0x7f6b781616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7c8c4700 1 -- 172.21.15.5:0/1706299384 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780cce90 con 0x7f6b780cc060 2024-05-08T23:25:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b77fff700 1 --2- 172.21.15.5:0/1706299384 >> v2:172.21.15.49:3300/0 conn(0x7f6b780cfde0 0x7f6b7806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b7c8c4700 1 --2- 172.21.15.5:0/1706299384 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cc060 0x7f6b780cf8a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009840 tx=0x7f6b68009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb0bb08a0cba72f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.573+0000 7f6b777fe700 1 -- 172.21.15.5:0/1706299384 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68011830 con 0x7f6b780cc060 2024-05-08T23:25:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b777fe700 1 -- 172.21.15.5:0/1706299384 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b68011990 con 0x7f6b780cc060 2024-05-08T23:25:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b777fe700 1 -- 172.21.15.5:0/1706299384 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68011d20 con 0x7f6b780cc060 2024-05-08T23:25:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1706299384 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cc060 msgr2=0x7f6b780cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1706299384 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cc060 0x7f6b780cf8a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009840 tx=0x7f6b68009b50 comp rx=0 tx=0).stop 2024-05-08T23:25:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1706299384 shutdown_connections 2024-05-08T23:25:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1706299384 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cc060 0x7f6b780cf8a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1706299384 >> v2:172.21.15.49:3300/0 conn(0x7f6b780cfde0 0x7f6b7806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1706299384 >> 172.21.15.5:0/1706299384 conn(0x7f6b78099a50 msgr2=0x7f6b780d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1706299384 shutdown_connections 2024-05-08T23:25:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.574+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1706299384 wait complete. 2024-05-08T23:25:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.576+0000 7f6b7eb28700 1 Processor -- start 2024-05-08T23:25:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.576+0000 7f6b7eb28700 1 -- start start 2024-05-08T23:25:51.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7eb28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b7806aea0 0x7f6b7815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7eb28700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b780cc060 0x7f6b780b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7eb28700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cfde0 0x7f6b780b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7eb28700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780b2c30 con 0x7f6b7806aea0 2024-05-08T23:25:51.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7eb28700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7815b270 con 0x7f6b780cc060 2024-05-08T23:25:51.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7eb28700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b781f30c0 con 0x7f6b780cfde0 2024-05-08T23:25:51.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7c8c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b7806aea0 0x7f6b7815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7c8c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b7806aea0 0x7f6b7815ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49910/0 (socket says 172.21.15.5:49910) 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7c8c4700 1 -- 172.21.15.5:0/1115122702 learned_addr learned my addr 172.21.15.5:0/1115122702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b7d0c5700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cfde0 0x7f6b780b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.577+0000 7f6b77fff700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b780cc060 0x7f6b780b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b7c8c4700 1 -- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cfde0 msgr2=0x7f6b780b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b7c8c4700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cfde0 0x7f6b780b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b7c8c4700 1 -- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b780cc060 msgr2=0x7f6b780b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b7c8c4700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b780cc060 0x7f6b780b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b7c8c4700 1 -- 172.21.15.5:0/1115122702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0094a0 con 0x7f6b7806aea0 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b7c8c4700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b7806aea0 0x7f6b7815ad30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6b68000c00 tx=0x7f6b68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b757fa700 1 -- 172.21.15.5:0/1115122702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68004590 con 0x7f6b7806aea0 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b757fa700 1 -- 172.21.15.5:0/1115122702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6801f070 con 0x7f6b7806aea0 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b680094a0 con 0x7f6b7806aea0 2024-05-08T23:25:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b757fa700 1 -- 172.21.15.5:0/1115122702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680038e0 con 0x7f6b7806aea0 2024-05-08T23:25:51.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.578+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7815f0d0 con 0x7f6b7806aea0 2024-05-08T23:25:51.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.579+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b78068000 con 0x7f6b7806aea0 2024-05-08T23:25:51.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.580+0000 7f6b757fa700 1 -- 172.21.15.5:0/1115122702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b68003a40 con 0x7f6b7806aea0 2024-05-08T23:25:51.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.580+0000 7f6b757fa700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b6406ac90 0x7f6b6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:51.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.580+0000 7f6b757fa700 1 -- 172.21.15.5:0/1115122702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b68021070 con 0x7f6b7806aea0 2024-05-08T23:25:51.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.580+0000 7f6b77fff700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b6406ac90 0x7f6b6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:51.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.581+0000 7f6b77fff700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b6406ac90 0x7f6b6406d150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b780ae550 tx=0x7f6b6c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.585+0000 7f6b757fa700 1 -- 172.21.15.5:0/1115122702 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6b68055d00 con 0x7f6b7806aea0 2024-05-08T23:25:51.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.850+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6b780aeea0 con 0x7f6b7806aea0 2024-05-08T23:25:51.851 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:25:51.851+0000 7f8a49d06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:25:51.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.858+0000 7f6b757fa700 1 -- 172.21.15.5:0/1115122702 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6b6802b090 con 0x7f6b7806aea0 2024-05-08T23:25:51.863 INFO:tasks.ceph.mds.d.smithi049.stderr: -1354> 2024-05-08T23:25:14.031+0000 7ffaa691c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-05-08T23:24:57.100108+0000 1=0+1) n(v0 rc2024-05-08T23:24:57.100108+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6349=pAsLsXsFsx/pAsLsXsFsx@4},l=6349 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562427e5eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.100108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.100108+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.100108+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-05-08T23:24:57.100108+0000 2=0+2)","error_str":""},"return_code":-61} 2024-05-08T23:25:51.863 INFO:tasks.ceph.mds.d.smithi049.stderr: -1333> 2024-05-08T23:25:14.031+0000 7ffaa691c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6349=pAsLsXsFsx/AsLsXsFsx@2},l=6349 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562427e5f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-05-08T23:25:51.865 INFO:tasks.ceph.mds.c.smithi005.stderr: -1464> 2024-05-08T23:25:14.030+0000 7f81d46bc700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1) n(v2 rc2024-05-08T23:24:57.359102+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6349=pAsLsXsFsx/pAsLsXsFsx@4},l=6349 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55f7be187700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.359102+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-05-08T23:24:57.359102+0000 4=0+4)","error_str":""},"return_code":-61} 2024-05-08T23:25:51.865 INFO:tasks.ceph.mds.c.smithi005.stderr: -1344> 2024-05-08T23:25:14.032+0000 7f81da6c8700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-05-08T23:24:57.099108+0000 1=0+1) n(v1 rc2024-05-08T23:24:57.359102+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6349=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55f7bd226b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.099108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.100108+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.099108+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-05-08T23:24:57.359102+0000 3=0+3)","error_str":""},"return_code":-61} 2024-05-08T23:25:51.870 INFO:tasks.ceph.mds.a.smithi005.stderr: -3401> 2024-05-08T23:25:14.030+0000 7f82fcd00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-05-08T23:24:57.097108+0000 1=0+1) n(v3 rc2024-05-08T23:24:57.359102+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={6349=pAsLsXsFsx/pAsLsXsFsx@5},l=6349 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56161caa6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.097108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.359102+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.097108+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-05-08T23:24:57.359102+0000 6=0+6)","error_str":""},"return_code":-61} 2024-05-08T23:25:51.870 INFO:tasks.ceph.mds.a.smithi005.stderr: -3298> 2024-05-08T23:25:14.031+0000 7f8302d0c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1) n(v4 rc2024-05-08T23:24:57.359102+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6349=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x56161caa6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:24:57.359102+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-05-08T23:24:57.098108+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-05-08T23:24:57.359102+0000 5=0+5)","error_str":""},"return_code":-61} 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b6406ac90 msgr2=0x7f6b6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b6406ac90 0x7f6b6406d150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b780ae550 tx=0x7f6b6c009210 comp rx=0 tx=0).stop 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b7806aea0 msgr2=0x7f6b7815ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b7806aea0 0x7f6b7815ad30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6b68000c00 tx=0x7f6b68004750 comp rx=0 tx=0).stop 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 shutdown_connections 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b6406ac90 0x7f6b6406d150 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b780cfde0 0x7f6b780b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b7806aea0 0x7f6b7815ad30 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 --2- 172.21.15.5:0/1115122702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b780cc060 0x7f6b780b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.870+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 >> 172.21.15.5:0/1115122702 conn(0x7f6b78099a50 msgr2=0x7f6b7806f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.871+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 shutdown_connections 2024-05-08T23:25:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:51.871+0000 7f6b7eb28700 1 -- 172.21.15.5:0/1115122702 wait complete. 2024-05-08T23:25:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:25:51.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:25:51.892 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:25:51.896 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:25:51.904 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:25:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.026+0000 7feda2f5c700 1 Processor -- start 2024-05-08T23:25:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.026+0000 7feda2f5c700 1 -- start start 2024-05-08T23:25:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda2f5c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fed9c0cdc90 0x7fed9c0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda2f5c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9c0b1200 con 0x7fed9c0d13a0 2024-05-08T23:25:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda2f5c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9c0ce5c0 con 0x7fed9c06bd60 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda2f5c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9c0ce8a0 con 0x7fed9c0cdc90 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda14f9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fed9c0cdc90 0x7fed9c0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda0cf8700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fed9c0d13a0 0x7fed9c0d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51786/0 (socket says 172.21.15.5:51786) 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda0cf8700 1 -- 172.21.15.5:0/81684308 learned_addr learned my addr 172.21.15.5:0/81684308 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda14f9700 1 -- 172.21.15.5:0/81684308 >> v1:172.21.15.49:6789/0 conn(0x7fed9c06bd60 legacy=0x7fed9c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda14f9700 1 -- 172.21.15.5:0/81684308 >> v1:172.21.15.5:6789/0 conn(0x7fed9c0d13a0 legacy=0x7fed9c0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7feda14f9700 1 -- 172.21.15.5:0/81684308 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c0ceb80 con 0x7fed9c0cdc90 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.027+0000 7fed9b7fe700 1 -- 172.21.15.5:0/81684308 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58151749 0 0) 0x7fed9c0b1200 con 0x7fed9c0d13a0 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7feda14f9700 1 --2- 172.21.15.5:0/81684308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c0cdc90 0x7fed9c0b19c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fed94009d90 tx=0x7fed9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d83016211963b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7fed9b7fe700 1 -- 172.21.15.5:0/81684308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9400e070 con 0x7fed9c0cdc90 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7fed9b7fe700 1 -- 172.21.15.5:0/81684308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9400bbd0 con 0x7fed9c0cdc90 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7fed9b7fe700 1 -- 172.21.15.5:0/81684308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94012430 con 0x7fed9c0cdc90 2024-05-08T23:25:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7feda2f5c700 1 -- 172.21.15.5:0/81684308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c0cdc90 msgr2=0x7fed9c0b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7feda2f5c700 1 --2- 172.21.15.5:0/81684308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c0cdc90 0x7fed9c0b19c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fed94009d90 tx=0x7fed9400b330 comp rx=0 tx=0).stop 2024-05-08T23:25:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7feda2f5c700 1 -- 172.21.15.5:0/81684308 shutdown_connections 2024-05-08T23:25:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7feda2f5c700 1 --2- 172.21.15.5:0/81684308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c0cdc90 0x7fed9c0b19c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7feda2f5c700 1 -- 172.21.15.5:0/81684308 >> 172.21.15.5:0/81684308 conn(0x7fed9c08afe0 msgr2=0x7fed9c08b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7feda2f5c700 1 -- 172.21.15.5:0/81684308 shutdown_connections 2024-05-08T23:25:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.028+0000 7feda2f5c700 1 -- 172.21.15.5:0/81684308 wait complete. 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.030+0000 7feda2f5c700 1 Processor -- start 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.030+0000 7feda2f5c700 1 -- start start 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda2f5c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c06bd60 0x7fed9c145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda2f5c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cdc90 0x7fed9c1499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda2f5c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fed9c0d13a0 0x7fed9c147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda2f5c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c1474e0 con 0x7fed9c06bd60 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda2f5c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c1461f0 con 0x7fed9c0cdc90 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda2f5c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c1468d0 con 0x7fed9c0d13a0 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c06bd60 0x7fed9c145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c06bd60 0x7fed9c145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49956/0 (socket says 172.21.15.5:49956) 2024-05-08T23:25:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 -- 172.21.15.5:0/2285288604 learned_addr learned my addr 172.21.15.5:0/2285288604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda14f9700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fed9c0d13a0 0x7fed9c147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7fed9bfff700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cdc90 0x7fed9c1499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 -- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fed9c0d13a0 msgr2=0x7fed9c147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fed9c0d13a0 0x7fed9c147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 -- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cdc90 msgr2=0x7fed9c1499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cdc90 0x7fed9c1499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 -- 172.21.15.5:0/2285288604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94009a70 con 0x7fed9c06bd60 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7fed9bfff700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cdc90 0x7fed9c1499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.031+0000 7feda0cf8700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c06bd60 0x7fed9c145cb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c520 tx=0x7fed9000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.032+0000 7fed997fa700 1 -- 172.21.15.5:0/2285288604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90004c80 con 0x7fed9c06bd60 2024-05-08T23:25:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.032+0000 7fed997fa700 1 -- 172.21.15.5:0/2285288604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90004de0 con 0x7fed9c06bd60 2024-05-08T23:25:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.032+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c146bb0 con 0x7fed9c06bd60 2024-05-08T23:25:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.032+0000 7fed997fa700 1 -- 172.21.15.5:0/2285288604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90007310 con 0x7fed9c06bd60 2024-05-08T23:25:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.032+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c147170 con 0x7fed9c06bd60 2024-05-08T23:25:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.033+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9c15f4e0 con 0x7fed9c06bd60 2024-05-08T23:25:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.033+0000 7fed997fa700 1 -- 172.21.15.5:0/2285288604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fed90007470 con 0x7fed9c06bd60 2024-05-08T23:25:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.034+0000 7fed997fa700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fed8006ac90 0x7fed8006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.034+0000 7fed997fa700 1 -- 172.21.15.5:0/2285288604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fed900896f0 con 0x7fed9c06bd60 2024-05-08T23:25:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.034+0000 7fed9bfff700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fed8006ac90 0x7fed8006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.035+0000 7fed9bfff700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fed8006ac90 0x7fed8006d150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0cdb80 tx=0x7fed88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.038+0000 7fed997fa700 1 -- 172.21.15.5:0/2285288604 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fed900547d0 con 0x7fed9c06bd60 2024-05-08T23:25:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.311+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fed9c074bc0 con 0x7fed9c06bd60 2024-05-08T23:25:52.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.860+0000 7fed997fa700 1 -- 172.21.15.5:0/2285288604 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed90057df0 con 0x7fed9c06bd60 2024-05-08T23:25:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fed8006ac90 msgr2=0x7fed8006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fed8006ac90 0x7fed8006d150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0cdb80 tx=0x7fed88009040 comp rx=0 tx=0).stop 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c06bd60 msgr2=0x7fed9c145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c06bd60 0x7fed9c145cb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c520 tx=0x7fed9000c830 comp rx=0 tx=0).stop 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 shutdown_connections 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fed8006ac90 0x7fed8006d150 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fed9c0d13a0 0x7fed9c147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed9c06bd60 0x7fed9c145cb0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 --2- 172.21.15.5:0/2285288604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cdc90 0x7fed9c1499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 >> 172.21.15.5:0/2285288604 conn(0x7fed9c08afe0 msgr2=0x7fed9c06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 shutdown_connections 2024-05-08T23:25:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:52.863+0000 7feda2f5c700 1 -- 172.21.15.5:0/2285288604 wait complete. 2024-05-08T23:25:52.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:25:53.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.067+0000 7fe9df319700 1 Processor -- start 2024-05-08T23:25:53.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.067+0000 7fe9df319700 1 -- start start 2024-05-08T23:25:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.068+0000 7fe9df319700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9d806b070 0x7fe9d80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.068+0000 7fe9df319700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d80b16e0 con 0x7fe9d8162d40 2024-05-08T23:25:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.068+0000 7fe9df319700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d81631b0 con 0x7fe9d814b930 2024-05-08T23:25:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.068+0000 7fe9df319700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d80b1980 con 0x7fe9d806b070 2024-05-08T23:25:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.068+0000 7fe9dc8b4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9d806b070 0x7fe9d80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9dd8b6700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe9d8162d40 0x7fe9d81633f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51800/0 (socket says 172.21.15.5:51800) 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9dd8b6700 1 -- 172.21.15.5:0/3351023452 learned_addr learned my addr 172.21.15.5:0/3351023452 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9dc8b4700 1 -- 172.21.15.5:0/3351023452 >> v1:172.21.15.49:6789/0 conn(0x7fe9d814b930 legacy=0x7fe9d806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9dc8b4700 1 -- 172.21.15.5:0/3351023452 >> v1:172.21.15.5:6789/0 conn(0x7fe9d8162d40 legacy=0x7fe9d81633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9dc8b4700 1 -- 172.21.15.5:0/3351023452 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80b1c60 con 0x7fe9d806b070 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9cffff700 1 -- 172.21.15.5:0/3351023452 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988924102 0 0) 0x7fe9d80b16e0 con 0x7fe9d8162d40 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9dc8b4700 1 --2- 172.21.15.5:0/3351023452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d80b32f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009870 tx=0x7fe9c8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d640a787046dee48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9cffff700 1 -- 172.21.15.5:0/3351023452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8011830 con 0x7fe9d806b070 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9cffff700 1 -- 172.21.15.5:0/3351023452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c8011990 con 0x7fe9d806b070 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.069+0000 7fe9cffff700 1 -- 172.21.15.5:0/3351023452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8011d20 con 0x7fe9d806b070 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.070+0000 7fe9df319700 1 -- 172.21.15.5:0/3351023452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 msgr2=0x7fe9d80b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.070+0000 7fe9df319700 1 --2- 172.21.15.5:0/3351023452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d80b32f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009870 tx=0x7fe9c8009b80 comp rx=0 tx=0).stop 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.070+0000 7fe9df319700 1 -- 172.21.15.5:0/3351023452 shutdown_connections 2024-05-08T23:25:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.070+0000 7fe9df319700 1 --2- 172.21.15.5:0/3351023452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d80b32f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.070+0000 7fe9df319700 1 -- 172.21.15.5:0/3351023452 >> 172.21.15.5:0/3351023452 conn(0x7fe9d8099a50 msgr2=0x7fe9d80b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.070+0000 7fe9df319700 1 -- 172.21.15.5:0/3351023452 shutdown_connections 2024-05-08T23:25:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.070+0000 7fe9df319700 1 -- 172.21.15.5:0/3351023452 wait complete. 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.072+0000 7fe9df319700 1 Processor -- start 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.072+0000 7fe9df319700 1 -- start start 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9df319700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d81f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9df319700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d814b930 0x7fe9d80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9df319700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d8162d40 0x7fe9d80aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9df319700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8143e60 con 0x7fe9d806b070 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9df319700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80cdea0 con 0x7fe9d814b930 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9df319700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80ce180 con 0x7fe9d8162d40 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd0b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d81f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd0b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d81f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49986/0 (socket says 172.21.15.5:49986) 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd0b5700 1 -- 172.21.15.5:0/2237435965 learned_addr learned my addr 172.21.15.5:0/2237435965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dc8b4700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d814b930 0x7fe9d80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd0b5700 1 -- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d8162d40 msgr2=0x7fe9d80aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd8b6700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d8162d40 0x7fe9d80aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd0b5700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d8162d40 0x7fe9d80aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd0b5700 1 -- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d814b930 msgr2=0x7fe9d80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd0b5700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d814b930 0x7fe9d80cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd0b5700 1 -- 172.21.15.5:0/2237435965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80094a0 con 0x7fe9d806b070 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dc8b4700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d814b930 0x7fe9d80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.073+0000 7fe9dd8b6700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d8162d40 0x7fe9d80aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.074+0000 7fe9dd0b5700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d81f7350 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400c4e0 tx=0x7fe9d400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.074+0000 7fe9cdffb700 1 -- 172.21.15.5:0/2237435965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40052d0 con 0x7fe9d806b070 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.074+0000 7fe9cdffb700 1 -- 172.21.15.5:0/2237435965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4005430 con 0x7fe9d806b070 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.074+0000 7fe9cdffb700 1 -- 172.21.15.5:0/2237435965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40038c0 con 0x7fe9d806b070 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.074+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d80ce460 con 0x7fe9d806b070 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.074+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d80cea20 con 0x7fe9d806b070 2024-05-08T23:25:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.075+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d8074bc0 con 0x7fe9d806b070 2024-05-08T23:25:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.075+0000 7fe9cdffb700 1 -- 172.21.15.5:0/2237435965 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe9d4003a20 con 0x7fe9d806b070 2024-05-08T23:25:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.076+0000 7fe9cdffb700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9bc06ac90 0x7fe9bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.076+0000 7fe9cdffb700 1 -- 172.21.15.5:0/2237435965 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe9d4088860 con 0x7fe9d806b070 2024-05-08T23:25:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.076+0000 7fe9dc8b4700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9bc06ac90 0x7fe9bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.076+0000 7fe9dc8b4700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9bc06ac90 0x7fe9bc06d150 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80b47e0 tx=0x7fe9c8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:53.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.081+0000 7fe9cdffb700 1 -- 172.21.15.5:0/2237435965 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe9d40539d0 con 0x7fe9d806b070 2024-05-08T23:25:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.296+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9d806f7e0 con 0x7fe9d806b070 2024-05-08T23:25:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.864+0000 7fe9cdffb700 1 -- 172.21.15.5:0/2237435965 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe9d40170d0 con 0x7fe9d806b070 2024-05-08T23:25:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9bc06ac90 msgr2=0x7fe9bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9bc06ac90 0x7fe9bc06d150 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80b47e0 tx=0x7fe9c8008030 comp rx=0 tx=0).stop 2024-05-08T23:25:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 msgr2=0x7fe9d81f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d81f7350 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400c4e0 tx=0x7fe9d400c8a0 comp rx=0 tx=0).stop 2024-05-08T23:25:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 shutdown_connections 2024-05-08T23:25:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9bc06ac90 0x7fe9bc06d150 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d8162d40 0x7fe9d80aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d806b070 0x7fe9d81f7350 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 --2- 172.21.15.5:0/2237435965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d814b930 0x7fe9d80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 >> 172.21.15.5:0/2237435965 conn(0x7fe9d8099a50 msgr2=0x7fe9d81633d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 shutdown_connections 2024-05-08T23:25:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:53.868+0000 7fe9df319700 1 -- 172.21.15.5:0/2237435965 wait complete. 2024-05-08T23:25:53.870 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:25:53.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:25:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.031+0000 7fe7f2348700 1 Processor -- start 2024-05-08T23:25:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.031+0000 7fe7f2348700 1 -- start start 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f2348700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f2348700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe7ec06b070 0x7fe7ec1454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f2348700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec145040 con 0x7fe7ec0b4d50 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f2348700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b38c0 con 0x7fe7ec06b070 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f2348700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b16e0 con 0x7fe7ec0b0d80 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f08e5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe7ec06b070 0x7fe7ec1454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f08e5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe7ec06b070 0x7fe7ec1454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50000/0 (socket says 172.21.15.5:50000) 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f08e5700 1 -- 172.21.15.5:0/1166876090 learned_addr learned my addr 172.21.15.5:0/1166876090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7ebfff700 1 --2- 172.21.15.5:0/1166876090 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:54.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f08e5700 1 -- 172.21.15.5:0/1166876090 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 msgr2=0x7fe7ec0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f08e5700 1 --2- 172.21.15.5:0/1166876090 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f08e5700 1 -- 172.21.15.5:0/1166876090 >> v1:172.21.15.49:6789/0 conn(0x7fe7ec0b4d50 legacy=0x7fe7ec06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f08e5700 1 -- 172.21.15.5:0/1166876090 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0b19c0 con 0x7fe7ec06b070 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7ebfff700 1 --2- 172.21.15.5:0/1166876090 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.032+0000 7fe7f08e5700 1 --2- 172.21.15.5:0/1166876090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec1454d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009e50 tx=0x7fe7e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cd74c34e252cb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7eaffd700 1 -- 172.21.15.5:0/1166876090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400e070 con 0x7fe7ec06b070 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7eaffd700 1 -- 172.21.15.5:0/1166876090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e4003a20 con 0x7fe7ec06b070 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7eaffd700 1 -- 172.21.15.5:0/1166876090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4003dd0 con 0x7fe7ec06b070 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7f2348700 1 -- 172.21.15.5:0/1166876090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 msgr2=0x7fe7ec1454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7f2348700 1 --2- 172.21.15.5:0/1166876090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec1454d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009e50 tx=0x7fe7e400b330 comp rx=0 tx=0).stop 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7f2348700 1 -- 172.21.15.5:0/1166876090 shutdown_connections 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7f2348700 1 --2- 172.21.15.5:0/1166876090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec1454d0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7f2348700 1 --2- 172.21.15.5:0/1166876090 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7f2348700 1 -- 172.21.15.5:0/1166876090 >> 172.21.15.5:0/1166876090 conn(0x7fe7ec099a50 msgr2=0x7fe7ec0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:54.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7f2348700 1 -- 172.21.15.5:0/1166876090 shutdown_connections 2024-05-08T23:25:54.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.033+0000 7fe7f2348700 1 -- 172.21.15.5:0/1166876090 wait complete. 2024-05-08T23:25:54.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.035+0000 7fe7f2348700 1 Processor -- start 2024-05-08T23:25:54.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.035+0000 7fe7f2348700 1 -- start start 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.035+0000 7fe7f2348700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.035+0000 7fe7f2348700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec09dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7f2348700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7ec0b4d50 0x7fe7ec0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7f2348700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1f3db0 con 0x7fe7ec06b070 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7f2348700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec136ac0 con 0x7fe7ec0b4d50 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7f2348700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1f3130 con 0x7fe7ec0b0d80 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec0cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50002/0 (socket says 172.21.15.5:50002) 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 -- 172.21.15.5:0/3081467123 learned_addr learned my addr 172.21.15.5:0/3081467123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7eb7fe700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec09dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7f08e5700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7ec0b4d50 0x7fe7ec0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 -- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 msgr2=0x7fe7ec09dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:54.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec09dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 -- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7ec0b4d50 msgr2=0x7fe7ec0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7ec0b4d50 0x7fe7ec0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 -- 172.21.15.5:0/3081467123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4009b30 con 0x7fe7ec06b070 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7ebfff700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec0cf670 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c750 tx=0x7fe7dc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7e8ff9700 1 -- 172.21.15.5:0/3081467123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc011800 con 0x7fe7ec06b070 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7e8ff9700 1 -- 172.21.15.5:0/3081467123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc011960 con 0x7fe7ec06b070 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec1f3410 con 0x7fe7ec06b070 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7e8ff9700 1 -- 172.21.15.5:0/3081467123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc011cd0 con 0x7fe7ec06b070 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.036+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec1f39d0 con 0x7fe7ec06b070 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.037+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec074bc0 con 0x7fe7ec06b070 2024-05-08T23:25:54.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.038+0000 7fe7e8ff9700 1 -- 172.21.15.5:0/3081467123 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe7dc021350 con 0x7fe7ec06b070 2024-05-08T23:25:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.038+0000 7fe7e8ff9700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7d806ac90 0x7fe7d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.038+0000 7fe7e8ff9700 1 -- 172.21.15.5:0/3081467123 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5529+0+0 (secure 0 0 0) 0x7fe7dc0892a0 con 0x7fe7ec06b070 2024-05-08T23:25:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.039+0000 7fe7eb7fe700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7d806ac90 0x7fe7d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.039+0000 7fe7eb7fe700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7d806ac90 0x7fe7d806d150 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec09b4a0 tx=0x7fe7e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:54.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.043+0000 7fe7e8ff9700 1 -- 172.21.15.5:0/3081467123 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe7dc054520 con 0x7fe7ec06b070 2024-05-08T23:25:54.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.260+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7ec0b33a0 con 0x7fe7ec06b070 2024-05-08T23:25:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.869+0000 7fe7e8ff9700 1 -- 172.21.15.5:0/3081467123 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe7dc057b40 con 0x7fe7ec06b070 2024-05-08T23:25:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7d806ac90 msgr2=0x7fe7d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7d806ac90 0x7fe7d806d150 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec09b4a0 tx=0x7fe7e0008040 comp rx=0 tx=0).stop 2024-05-08T23:25:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 msgr2=0x7fe7ec0cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec0cf670 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c750 tx=0x7fe7dc00cb10 comp rx=0 tx=0).stop 2024-05-08T23:25:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 shutdown_connections 2024-05-08T23:25:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7d806ac90 0x7fe7d806d150 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7ec0b4d50 0x7fe7ec0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec06b070 0x7fe7ec0cf670 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 --2- 172.21.15.5:0/3081467123 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7ec0b0d80 0x7fe7ec09dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 >> 172.21.15.5:0/3081467123 conn(0x7fe7ec099a50 msgr2=0x7fe7ec0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 shutdown_connections 2024-05-08T23:25:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:54.875+0000 7fe7f2348700 1 -- 172.21.15.5:0/3081467123 wait complete. 2024-05-08T23:25:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-08T23:25:54.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T23:25:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.035+0000 7fa95080a700 1 Processor -- start 2024-05-08T23:25:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.036+0000 7fa95080a700 1 -- start start 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa95080a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c0b0d80 0x7fa94c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa95080a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa94c0b4d50 0x7fa94c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa95080a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c145040 con 0x7fa94c06aea0 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa95080a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c06b330 con 0x7fa94c0b4d50 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa95080a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0b38c0 con 0x7fa94c0b0d80 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa94ad9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa94c06aea0 0x7fa94c0d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51816/0 (socket says 172.21.15.5:51816) 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa94ad9d700 1 -- 172.21.15.5:0/2732064919 learned_addr learned my addr 172.21.15.5:0/2732064919 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa949d9b700 1 --2- 172.21.15.5:0/2732064919 >> v2:172.21.15.49:3300/0 conn(0x7fa94c0b4d50 0x7fa94c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa94a59c700 1 --2- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c0b0d80 0x7fa94c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa94959a700 1 -- 172.21.15.5:0/2732064919 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168875511 0 0) 0x7fa94c145040 con 0x7fa94c06aea0 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa94959a700 1 -- 172.21.15.5:0/2732064919 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9340035b0 con 0x7fa94c06aea0 2024-05-08T23:25:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa949d9b700 1 -- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c0b0d80 msgr2=0x7fa94c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa949d9b700 1 --2- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c0b0d80 0x7fa94c0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa949d9b700 1 -- 172.21.15.5:0/2732064919 >> v1:172.21.15.5:6789/0 conn(0x7fa94c06aea0 legacy=0x7fa94c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa94959a700 1 -- 172.21.15.5:0/2732064919 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481617795 0 0) 0x7fa9340035b0 con 0x7fa94c06aea0 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa949d9b700 1 -- 172.21.15.5:0/2732064919 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0b16e0 con 0x7fa94c0b4d50 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.037+0000 7fa94a59c700 1 --2- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c0b0d80 0x7fa94c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.038+0000 7fa949d9b700 1 --2- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b4d50 0x7fa94c06a960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa940009840 tx=0x7fa940009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cb48e4c7104932 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.038+0000 7fa94959a700 1 -- 172.21.15.5:0/2732064919 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940011830 con 0x7fa94c0b4d50 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.038+0000 7fa94959a700 1 -- 172.21.15.5:0/2732064919 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa940011990 con 0x7fa94c0b4d50 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.038+0000 7fa94959a700 1 -- 172.21.15.5:0/2732064919 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940011d20 con 0x7fa94c0b4d50 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.038+0000 7fa95080a700 1 -- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b4d50 msgr2=0x7fa94c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.038+0000 7fa95080a700 1 --2- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b4d50 0x7fa94c06a960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa940009840 tx=0x7fa940009b50 comp rx=0 tx=0).stop 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.039+0000 7fa95080a700 1 -- 172.21.15.5:0/2732064919 shutdown_connections 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.039+0000 7fa95080a700 1 --2- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c0b0d80 0x7fa94c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.039+0000 7fa95080a700 1 --2- 172.21.15.5:0/2732064919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b4d50 0x7fa94c06a960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.039+0000 7fa95080a700 1 -- 172.21.15.5:0/2732064919 >> 172.21.15.5:0/2732064919 conn(0x7fa94c099a50 msgr2=0x7fa94c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.039+0000 7fa95080a700 1 -- 172.21.15.5:0/2732064919 shutdown_connections 2024-05-08T23:25:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.039+0000 7fa95080a700 1 -- 172.21.15.5:0/2732064919 wait complete. 2024-05-08T23:25:55.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.040+0000 7fa95080a700 1 Processor -- start 2024-05-08T23:25:55.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.040+0000 7fa95080a700 1 -- start start 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa95080a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c06aea0 0x7fa94c1f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa95080a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b0d80 0x7fa94c1f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa95080a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c0b4d50 0x7fa94c133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa95080a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c1f75e0 con 0x7fa94c0b4d50 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa95080a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c139d80 con 0x7fa94c0b0d80 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa95080a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c09b480 con 0x7fa94c06aea0 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c0b4d50 0x7fa94c133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c0b4d50 0x7fa94c133a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50006/0 (socket says 172.21.15.5:50006) 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94ad9d700 1 -- 172.21.15.5:0/1718562114 learned_addr learned my addr 172.21.15.5:0/1718562114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa949d9b700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b0d80 0x7fa94c1f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94ad9d700 1 -- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c06aea0 msgr2=0x7fa94c1f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94a59c700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c06aea0 0x7fa94c1f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94ad9d700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c06aea0 0x7fa94c1f2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94ad9d700 1 -- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b0d80 msgr2=0x7fa94c1f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94ad9d700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b0d80 0x7fa94c1f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.041+0000 7fa94ad9d700 1 -- 172.21.15.5:0/1718562114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0094a0 con 0x7fa94c0b4d50 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.042+0000 7fa949d9b700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b0d80 0x7fa94c1f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.042+0000 7fa94ad9d700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c0b4d50 0x7fa94c133a00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa94400cc10 tx=0x7fa9440021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.042+0000 7fa93affd700 1 -- 172.21.15.5:0/1718562114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400ab80 con 0x7fa94c0b4d50 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.042+0000 7fa93affd700 1 -- 172.21.15.5:0/1718562114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94400ace0 con 0x7fa94c0b4d50 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.042+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9400094a0 con 0x7fa94c0b4d50 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.042+0000 7fa93affd700 1 -- 172.21.15.5:0/1718562114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944012620 con 0x7fa94c0b4d50 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.042+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c0ae590 con 0x7fa94c0b4d50 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.043+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c074bc0 con 0x7fa94c0b4d50 2024-05-08T23:25:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.044+0000 7fa93affd700 1 -- 172.21.15.5:0/1718562114 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa944012780 con 0x7fa94c0b4d50 2024-05-08T23:25:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.044+0000 7fa93affd700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa92c06ac90 0x7fa92c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.044+0000 7fa93affd700 1 -- 172.21.15.5:0/1718562114 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa944089590 con 0x7fa94c0b4d50 2024-05-08T23:25:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.044+0000 7fa94a59c700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa92c06ac90 0x7fa92c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.045+0000 7fa94a59c700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa92c06ac90 0x7fa92c06d150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009c20 tx=0x7fa93c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.049+0000 7fa93affd700 1 -- 172.21.15.5:0/1718562114 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa944054a70 con 0x7fa94c0b4d50 2024-05-08T23:25:55.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.263+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa94c09b7c0 con 0x7fa94c0b4d50 2024-05-08T23:25:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.267+0000 7fa93affd700 1 -- 172.21.15.5:0/1718562114 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa944058090 con 0x7fa94c0b4d50 2024-05-08T23:25:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa92c06ac90 msgr2=0x7fa92c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa92c06ac90 0x7fa92c06d150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009c20 tx=0x7fa93c008040 comp rx=0 tx=0).stop 2024-05-08T23:25:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c0b4d50 msgr2=0x7fa94c133a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c0b4d50 0x7fa94c133a00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa94400cc10 tx=0x7fa9440021c0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 shutdown_connections 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa92c06ac90 0x7fa92c06d150 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c0b4d50 0x7fa94c133a00 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa94c06aea0 0x7fa94c1f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 --2- 172.21.15.5:0/1718562114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa94c0b0d80 0x7fa94c1f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 >> 172.21.15.5:0/1718562114 conn(0x7fa94c099a50 msgr2=0x7fa94c146d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 shutdown_connections 2024-05-08T23:25:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.270+0000 7fa95080a700 1 -- 172.21.15.5:0/1718562114 wait complete. 2024-05-08T23:25:55.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-05-08T23:25:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.420+0000 7f066fd8b700 1 Processor -- start 2024-05-08T23:25:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.420+0000 7f066fd8b700 1 -- start start 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066fd8b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0668161b60 0x7f066806f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066fd8b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0668161f40 0x7f06680c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066fd8b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06680c3830 0x7f06681659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066fd8b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06680c3c80 con 0x7f0668161b60 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066fd8b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0668166760 con 0x7f0668161f40 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066fd8b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066806e360 con 0x7f06680c3830 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066db27700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0668161b60 0x7f066806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066db27700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0668161b60 0x7f066806f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50022/0 (socket says 172.21.15.5:50022) 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066db27700 1 -- 172.21.15.5:0/3075105150 learned_addr learned my addr 172.21.15.5:0/3075105150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066d326700 1 --2- 172.21.15.5:0/3075105150 >> v2:172.21.15.49:3300/0 conn(0x7f0668161f40 0x7f06680c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.421+0000 7f066e328700 1 --2- 172.21.15.5:0/3075105150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06680c3830 0x7f06681659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066db27700 1 -- 172.21.15.5:0/3075105150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06680c3830 msgr2=0x7f06681659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066db27700 1 --2- 172.21.15.5:0/3075105150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06680c3830 0x7f06681659a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066db27700 1 -- 172.21.15.5:0/3075105150 >> v2:172.21.15.49:3300/0 conn(0x7f0668161f40 msgr2=0x7f06680c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066db27700 1 --2- 172.21.15.5:0/3075105150 >> v2:172.21.15.49:3300/0 conn(0x7f0668161f40 0x7f06680c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066db27700 1 -- 172.21.15.5:0/3075105150 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066806e640 con 0x7f0668161b60 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066db27700 1 --2- 172.21.15.5:0/3075105150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668161b60 0x7f066806f120 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f066806b9f0 tx=0x7f0664009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdd753280fd6864 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066cb25700 1 -- 172.21.15.5:0/3075105150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066400b870 con 0x7f0668161b60 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066cb25700 1 -- 172.21.15.5:0/3075105150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066400b9d0 con 0x7f0668161b60 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.422+0000 7f066cb25700 1 -- 172.21.15.5:0/3075105150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066400bd60 con 0x7f0668161b60 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 -- 172.21.15.5:0/3075105150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668161b60 msgr2=0x7f066806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3075105150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668161b60 0x7f066806f120 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f066806b9f0 tx=0x7f0664009bd0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 -- 172.21.15.5:0/3075105150 shutdown_connections 2024-05-08T23:25:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3075105150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06680c3830 0x7f06681659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3075105150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668161b60 0x7f066806f120 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3075105150 >> v2:172.21.15.49:3300/0 conn(0x7f0668161f40 0x7f06680c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 -- 172.21.15.5:0/3075105150 >> 172.21.15.5:0/3075105150 conn(0x7f06680d0320 msgr2=0x7f06680d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 -- 172.21.15.5:0/3075105150 shutdown_connections 2024-05-08T23:25:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.423+0000 7f066fd8b700 1 -- 172.21.15.5:0/3075105150 wait complete. 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.424+0000 7f066fd8b700 1 Processor -- start 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.424+0000 7f066fd8b700 1 -- start start 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066fd8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06680c3830 0x7f06680c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066fd8b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0668161b60 0x7f06680c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066fd8b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0668161f40 0x7f06680b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066fd8b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680c63c0 con 0x7f06680c3830 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066fd8b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680c57b0 con 0x7f0668161b60 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066fd8b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06680c5a90 con 0x7f0668161f40 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066db27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06680c3830 0x7f06680c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066db27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06680c3830 0x7f06680c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50038/0 (socket says 172.21.15.5:50038) 2024-05-08T23:25:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066db27700 1 -- 172.21.15.5:0/3397516960 learned_addr learned my addr 172.21.15.5:0/3397516960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066d326700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0668161b60 0x7f06680c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066e328700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0668161f40 0x7f06680b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066db27700 1 -- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0668161f40 msgr2=0x7f06680b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066db27700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0668161f40 0x7f06680b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066db27700 1 -- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0668161b60 msgr2=0x7f06680c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066db27700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0668161b60 0x7f06680c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066db27700 1 -- 172.21.15.5:0/3397516960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664009550 con 0x7f06680c3830 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066e328700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0668161f40 0x7f06680b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.425+0000 7f066d326700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0668161b60 0x7f06680c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.426+0000 7f066db27700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06680c3830 0x7f06680c6850 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0664000c00 tx=0x7f0664004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.426+0000 7f065a7fc700 1 -- 172.21.15.5:0/3397516960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664004520 con 0x7f06680c3830 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.426+0000 7f065a7fc700 1 -- 172.21.15.5:0/3397516960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06640036a0 con 0x7f06680c3830 2024-05-08T23:25:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.426+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06680c5d70 con 0x7f06680c3830 2024-05-08T23:25:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.426+0000 7f065a7fc700 1 -- 172.21.15.5:0/3397516960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664003a50 con 0x7f06680c3830 2024-05-08T23:25:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.426+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06680c9fd0 con 0x7f06680c3830 2024-05-08T23:25:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.427+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06680ca160 con 0x7f06680c3830 2024-05-08T23:25:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.427+0000 7f065a7fc700 1 -- 172.21.15.5:0/3397516960 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0664003bb0 con 0x7f06680c3830 2024-05-08T23:25:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.428+0000 7f065a7fc700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f064c06ac90 0x7f064c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.428+0000 7f065a7fc700 1 -- 172.21.15.5:0/3397516960 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f066408a680 con 0x7f06680c3830 2024-05-08T23:25:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.428+0000 7f066d326700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f064c06ac90 0x7f064c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.428+0000 7f066d326700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f064c06ac90 0x7f064c06d150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f066809e440 tx=0x7f065c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.433+0000 7f065a7fc700 1 -- 172.21.15.5:0/3397516960 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0664055b40 con 0x7f06680c3830 2024-05-08T23:25:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.763+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) v1 -- 0x7f0668068000 con 0x7f06680c3830 2024-05-08T23:25:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.871+0000 7f065a7fc700 1 -- 172.21.15.5:0/3397516960 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v252) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0664003e60 con 0x7f06680c3830 2024-05-08T23:25:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f064c06ac90 msgr2=0x7f064c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f064c06ac90 0x7f064c06d150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f066809e440 tx=0x7f065c008040 comp rx=0 tx=0).stop 2024-05-08T23:25:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06680c3830 msgr2=0x7f06680c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06680c3830 0x7f06680c6850 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0664000c00 tx=0x7f0664004750 comp rx=0 tx=0).stop 2024-05-08T23:25:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 shutdown_connections 2024-05-08T23:25:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f064c06ac90 0x7f064c06d150 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0668161f40 0x7f06680b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06680c3830 0x7f06680c6850 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 --2- 172.21.15.5:0/3397516960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0668161b60 0x7f06680c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 >> 172.21.15.5:0/3397516960 conn(0x7f06680d0320 msgr2=0x7f0668160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 shutdown_connections 2024-05-08T23:25:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:55.874+0000 7f066fd8b700 1 -- 172.21.15.5:0/3397516960 wait complete. 2024-05-08T23:25:55.888 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-05-08T23:25:55.888 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-05-08T23:25:55.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:25:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.027+0000 7fb69877b700 1 Processor -- start 2024-05-08T23:25:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.027+0000 7fb69877b700 1 -- start start 2024-05-08T23:25:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.028+0000 7fb69877b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb6900b2480 0x7fb69009a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.028+0000 7fb69877b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6900cbbc0 0x7fb6900cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.028+0000 7fb69877b700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900d0840 con 0x7fb690161030 2024-05-08T23:25:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.028+0000 7fb69877b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69006ccc0 con 0x7fb6900cbbc0 2024-05-08T23:25:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.028+0000 7fb69877b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900d0660 con 0x7fb6900b2480 2024-05-08T23:25:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.028+0000 7fb696d18700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6900cbbc0 0x7fb6900cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.028+0000 7fb696d18700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6900cbbc0 0x7fb6900cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50046/0 (socket says 172.21.15.5:50046) 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.028+0000 7fb696d18700 1 -- 172.21.15.5:0/3231739433 learned_addr learned my addr 172.21.15.5:0/3231739433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb695d16700 1 --2- 172.21.15.5:0/3231739433 >> v2:172.21.15.49:3300/0 conn(0x7fb6900b2480 0x7fb69009a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb696d18700 1 -- 172.21.15.5:0/3231739433 >> v2:172.21.15.49:3300/0 conn(0x7fb6900b2480 msgr2=0x7fb69009a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb696d18700 1 --2- 172.21.15.5:0/3231739433 >> v2:172.21.15.49:3300/0 conn(0x7fb6900b2480 0x7fb69009a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb696d18700 1 -- 172.21.15.5:0/3231739433 >> v1:172.21.15.49:6789/0 conn(0x7fb690161030 legacy=0x7fb690099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb696d18700 1 -- 172.21.15.5:0/3231739433 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69006c600 con 0x7fb6900cbbc0 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb696d18700 1 --2- 172.21.15.5:0/3231739433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900cbbc0 0x7fb6900cc010 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb6900b2430 tx=0x7fb68800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c087afa518bd6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb695515700 1 -- 172.21.15.5:0/3231739433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800e070 con 0x7fb6900cbbc0 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb695515700 1 -- 172.21.15.5:0/3231739433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688003a20 con 0x7fb6900cbbc0 2024-05-08T23:25:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.029+0000 7fb695515700 1 -- 172.21.15.5:0/3231739433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688003dd0 con 0x7fb6900cbbc0 2024-05-08T23:25:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.030+0000 7fb69877b700 1 -- 172.21.15.5:0/3231739433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900cbbc0 msgr2=0x7fb6900cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.030+0000 7fb69877b700 1 --2- 172.21.15.5:0/3231739433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900cbbc0 0x7fb6900cc010 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb6900b2430 tx=0x7fb68800b330 comp rx=0 tx=0).stop 2024-05-08T23:25:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.030+0000 7fb69877b700 1 -- 172.21.15.5:0/3231739433 shutdown_connections 2024-05-08T23:25:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.030+0000 7fb69877b700 1 --2- 172.21.15.5:0/3231739433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900cbbc0 0x7fb6900cc010 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.030+0000 7fb69877b700 1 --2- 172.21.15.5:0/3231739433 >> v2:172.21.15.49:3300/0 conn(0x7fb6900b2480 0x7fb69009a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.030+0000 7fb69877b700 1 -- 172.21.15.5:0/3231739433 >> 172.21.15.5:0/3231739433 conn(0x7fb6900b2f20 msgr2=0x7fb6900b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.030+0000 7fb69877b700 1 -- 172.21.15.5:0/3231739433 shutdown_connections 2024-05-08T23:25:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.030+0000 7fb69877b700 1 -- 172.21.15.5:0/3231739433 wait complete. 2024-05-08T23:25:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.031+0000 7fb69877b700 1 Processor -- start 2024-05-08T23:25:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.032+0000 7fb69877b700 1 -- start start 2024-05-08T23:25:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.032+0000 7fb69877b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900b2480 0x7fb6900c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb69877b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6900cbbc0 0x7fb6900c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb69877b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb690161030 0x7fb6900c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb69877b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900c95c0 con 0x7fb6900b2480 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb69877b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900c7e30 con 0x7fb690161030 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb69877b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900c9f20 con 0x7fb6900cbbc0 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900b2480 0x7fb6900c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900b2480 0x7fb6900c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50054/0 (socket says 172.21.15.5:50054) 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 -- 172.21.15.5:0/4270758851 learned_addr learned my addr 172.21.15.5:0/4270758851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb695d16700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6900cbbc0 0x7fb6900c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696d18700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb690161030 0x7fb6900c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 -- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6900cbbc0 msgr2=0x7fb6900c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6900cbbc0 0x7fb6900c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 -- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb690161030 msgr2=0x7fb6900c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb690161030 0x7fb6900c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 -- 172.21.15.5:0/4270758851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688009b30 con 0x7fb6900b2480 2024-05-08T23:25:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb696517700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900b2480 0x7fb6900c78f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c4d0 tx=0x7fb68c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb686ffd700 1 -- 172.21.15.5:0/4270758851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c011790 con 0x7fb6900b2480 2024-05-08T23:25:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb686ffd700 1 -- 172.21.15.5:0/4270758851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c0118f0 con 0x7fb6900b2480 2024-05-08T23:25:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6900c8940 con 0x7fb6900b2480 2024-05-08T23:25:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.033+0000 7fb686ffd700 1 -- 172.21.15.5:0/4270758851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c011ca0 con 0x7fb6900b2480 2024-05-08T23:25:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.034+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6900c8f00 con 0x7fb6900b2480 2024-05-08T23:25:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.034+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6900c9830 con 0x7fb6900b2480 2024-05-08T23:25:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.035+0000 7fb686ffd700 1 -- 172.21.15.5:0/4270758851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb68c01c460 con 0x7fb6900b2480 2024-05-08T23:25:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.035+0000 7fb686ffd700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb67406ac90 0x7fb67406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.036+0000 7fb686ffd700 1 -- 172.21.15.5:0/4270758851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb68c088dc0 con 0x7fb6900b2480 2024-05-08T23:25:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.036+0000 7fb695d16700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb67406ac90 0x7fb67406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.036+0000 7fb695d16700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb67406ac90 0x7fb67406d150 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb69009f430 tx=0x7fb680008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.040+0000 7fb686ffd700 1 -- 172.21.15.5:0/4270758851 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb68c0541f0 con 0x7fb6900b2480 2024-05-08T23:25:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.257+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb690068000 con 0x7fb6900b2480 2024-05-08T23:25:56.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.258+0000 7fb686ffd700 1 -- 172.21.15.5:0/4270758851 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+12678 (secure 0 0 0) 0x7fb68c057810 con 0x7fb6900b2480 2024-05-08T23:25:56.258 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:56.259 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":63,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:25:54.864131+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6833/2315721152":"2024-05-09T23:25:51.852427+0000","172.21.15.5:6835/1169758885":"2024-05-09T23:25:51.852432+0000","172.21.15.5:6834/1169758885":"2024-05-09T23:25:51.852432+0000","172.21.15.49:6836/3197377913":"2024-05-09T23:25:51.852410+0000","172.21.15.5:6832/2315721152":"2024-05-09T23:25:51.852427+0000","172.21.15.49:6834/3197377913":"2024-05-09T23:25:51.852410+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb67406ac90 msgr2=0x7fb67406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb67406ac90 0x7fb67406d150 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb69009f430 tx=0x7fb680008040 comp rx=0 tx=0).stop 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900b2480 msgr2=0x7fb6900c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900b2480 0x7fb6900c78f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c4d0 tx=0x7fb68c00c7e0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 shutdown_connections 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb67406ac90 0x7fb67406d150 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb690161030 0x7fb6900c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6900b2480 0x7fb6900c78f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 --2- 172.21.15.5:0/4270758851 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb6900cbbc0 0x7fb6900c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 >> 172.21.15.5:0/4270758851 conn(0x7fb6900b2f20 msgr2=0x7fb69015f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.260+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 shutdown_connections 2024-05-08T23:25:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.261+0000 7fb69877b700 1 -- 172.21.15.5:0/4270758851 wait complete. 2024-05-08T23:25:56.275 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:25:56.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:25:56.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.414+0000 7f03deffa700 1 Processor -- start 2024-05-08T23:25:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.414+0000 7f03deffa700 1 -- start start 2024-05-08T23:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.415+0000 7f03deffa700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03d8161ff0 0x7f03d806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.415+0000 7f03deffa700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d806c2e0 con 0x7f03d815d900 2024-05-08T23:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.415+0000 7f03deffa700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d815dce0 con 0x7f03d806beb0 2024-05-08T23:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.415+0000 7f03deffa700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d80ce5b0 con 0x7f03d8161ff0 2024-05-08T23:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.415+0000 7f03d7fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03d8161ff0 0x7f03d806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.415+0000 7f03dcd96700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f03d815d900 0x7f03d80cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51830/0 (socket says 172.21.15.5:51830) 2024-05-08T23:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.415+0000 7f03dcd96700 1 -- 172.21.15.5:0/910055093 learned_addr learned my addr 172.21.15.5:0/910055093 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.416+0000 7f03d7fff700 1 -- 172.21.15.5:0/910055093 >> v1:172.21.15.49:6789/0 conn(0x7f03d806beb0 legacy=0x7f03d80ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.416+0000 7f03d7fff700 1 -- 172.21.15.5:0/910055093 >> v1:172.21.15.5:6789/0 conn(0x7f03d815d900 legacy=0x7f03d80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.416+0000 7f03d7fff700 1 -- 172.21.15.5:0/910055093 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d8161510 con 0x7f03d8161ff0 2024-05-08T23:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.416+0000 7f03d77fe700 1 -- 172.21.15.5:0/910055093 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410829613 0 0) 0x7f03d806c2e0 con 0x7f03d815d900 2024-05-08T23:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.416+0000 7f03d7fff700 1 --2- 172.21.15.5:0/910055093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d806b970 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009840 tx=0x7f03c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dc8d9b5c38eb67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.416+0000 7f03d77fe700 1 -- 172.21.15.5:0/910055093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4011830 con 0x7f03d8161ff0 2024-05-08T23:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.416+0000 7f03d77fe700 1 -- 172.21.15.5:0/910055093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4011990 con 0x7f03d8161ff0 2024-05-08T23:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.416+0000 7f03d77fe700 1 -- 172.21.15.5:0/910055093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4011d20 con 0x7f03d8161ff0 2024-05-08T23:25:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.417+0000 7f03deffa700 1 -- 172.21.15.5:0/910055093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 msgr2=0x7f03d806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.417+0000 7f03deffa700 1 --2- 172.21.15.5:0/910055093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d806b970 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009840 tx=0x7f03c4009b50 comp rx=0 tx=0).stop 2024-05-08T23:25:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.417+0000 7f03deffa700 1 -- 172.21.15.5:0/910055093 shutdown_connections 2024-05-08T23:25:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.417+0000 7f03deffa700 1 --2- 172.21.15.5:0/910055093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d806b970 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.417+0000 7f03deffa700 1 -- 172.21.15.5:0/910055093 >> 172.21.15.5:0/910055093 conn(0x7f03d80713c0 msgr2=0x7f03d80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.417+0000 7f03deffa700 1 -- 172.21.15.5:0/910055093 shutdown_connections 2024-05-08T23:25:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.417+0000 7f03deffa700 1 -- 172.21.15.5:0/910055093 wait complete. 2024-05-08T23:25:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.418+0000 7f03deffa700 1 Processor -- start 2024-05-08T23:25:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.418+0000 7f03deffa700 1 -- start start 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03deffa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d806beb0 0x7f03d8148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03deffa700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d900 0x7f03d814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03deffa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d8154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03deffa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d815fab0 con 0x7f03d8161ff0 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03deffa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d814efa0 con 0x7f03d806beb0 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03deffa700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d815f110 con 0x7f03d815d900 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03dd597700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d8154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03dd597700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d8154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50072/0 (socket says 172.21.15.5:50072) 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.419+0000 7f03dd597700 1 -- 172.21.15.5:0/489334208 learned_addr learned my addr 172.21.15.5:0/489334208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03dcd96700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d806beb0 0x7f03d8148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03d7fff700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d900 0x7f03d814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03dd597700 1 -- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d900 msgr2=0x7f03d814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03dd597700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d900 0x7f03d814e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03dd597700 1 -- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d806beb0 msgr2=0x7f03d8148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03dd597700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d806beb0 0x7f03d8148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03dd597700 1 -- 172.21.15.5:0/489334208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c40094a0 con 0x7f03d8161ff0 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03dd597700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d8154de0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f03d000cb40 tx=0x7f03d000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03d57fa700 1 -- 172.21.15.5:0/489334208 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0003c00 con 0x7f03d8161ff0 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03d57fa700 1 -- 172.21.15.5:0/489334208 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0003d60 con 0x7f03d8161ff0 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d815f3f0 con 0x7f03d8161ff0 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03d57fa700 1 -- 172.21.15.5:0/489334208 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d000ab90 con 0x7f03d8161ff0 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.420+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d8149380 con 0x7f03d8161ff0 2024-05-08T23:25:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.421+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d815fbf0 con 0x7f03d8161ff0 2024-05-08T23:25:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.422+0000 7f03d57fa700 1 -- 172.21.15.5:0/489334208 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f03d000acf0 con 0x7f03d8161ff0 2024-05-08T23:25:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.422+0000 7f03d57fa700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.422+0000 7f03d57fa700 1 -- 172.21.15.5:0/489334208 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f03d0089140 con 0x7f03d8161ff0 2024-05-08T23:25:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.422+0000 7f03dcd96700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.423+0000 7f03dcd96700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f03cc007fd0 tx=0x7f03cc007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.427+0000 7f03d57fa700 1 -- 172.21.15.5:0/489334208 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f03d815fbf0 con 0x7f03d8161ff0 2024-05-08T23:25:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.641+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03d815fbf0 con 0x7f03d8161ff0 2024-05-08T23:25:56.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.642+0000 7f03d57fa700 1 -- 172.21.15.5:0/489334208 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03d815fbf0 con 0x7f03d8161ff0 2024-05-08T23:25:56.642 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:25:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 msgr2=0x7f03bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f03cc007fd0 tx=0x7f03cc007c20 comp rx=0 tx=0).stop 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 msgr2=0x7f03d8154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d8154de0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f03d000cb40 tx=0x7f03d000cf00 comp rx=0 tx=0).stop 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 shutdown_connections 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d8161ff0 0x7f03d8154de0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d806beb0 0x7f03d8148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 --2- 172.21.15.5:0/489334208 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d900 0x7f03d814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 >> 172.21.15.5:0/489334208 conn(0x7f03d80713c0 msgr2=0x7f03d80cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 shutdown_connections 2024-05-08T23:25:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.644+0000 7f03deffa700 1 -- 172.21.15.5:0/489334208 wait complete. 2024-05-08T23:25:56.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-05-08T23:25:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.794+0000 7f6633f63700 1 Processor -- start 2024-05-08T23:25:56.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.795+0000 7f6633f63700 1 -- start start 2024-05-08T23:25:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.795+0000 7f6633f63700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f662c15d7a0 0x7f662c161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.795+0000 7f6633f63700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f662c15db80 0x7f662c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.795+0000 7f6633f63700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662c162450 con 0x7f662c0d3f40 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.795+0000 7f6633f63700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662c0cee00 con 0x7f662c15d7a0 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.795+0000 7f6633f63700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662c0b27e0 con 0x7f662c15db80 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6631cff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f662c15d7a0 0x7f662c161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6631cff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f662c15d7a0 0x7f662c161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50076/0 (socket says 172.21.15.5:50076) 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6631cff700 1 -- 172.21.15.5:0/1790121869 learned_addr learned my addr 172.21.15.5:0/1790121869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6632500700 1 --2- 172.21.15.5:0/1790121869 >> v2:172.21.15.49:3300/0 conn(0x7f662c15db80 0x7f662c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6631cff700 1 -- 172.21.15.5:0/1790121869 >> v2:172.21.15.49:3300/0 conn(0x7f662c15db80 msgr2=0x7f662c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6631cff700 1 --2- 172.21.15.5:0/1790121869 >> v2:172.21.15.49:3300/0 conn(0x7f662c15db80 0x7f662c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6631cff700 1 -- 172.21.15.5:0/1790121869 >> v1:172.21.15.49:6789/0 conn(0x7f662c0d3f40 legacy=0x7f662c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6631cff700 1 -- 172.21.15.5:0/1790121869 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c0d4400 con 0x7f662c15d7a0 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6631cff700 1 --2- 172.21.15.5:0/1790121869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c15d7a0 0x7f662c161f10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f662c08b9e0 tx=0x7f6628009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1861a76884e5385e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6630cfd700 1 -- 172.21.15.5:0/1790121869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628011830 con 0x7f662c15d7a0 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6630cfd700 1 -- 172.21.15.5:0/1790121869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6628011990 con 0x7f662c15d7a0 2024-05-08T23:25:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.796+0000 7f6630cfd700 1 -- 172.21.15.5:0/1790121869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628011d20 con 0x7f662c15d7a0 2024-05-08T23:25:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.797+0000 7f6633f63700 1 -- 172.21.15.5:0/1790121869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c15d7a0 msgr2=0x7f662c161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.797+0000 7f6633f63700 1 --2- 172.21.15.5:0/1790121869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c15d7a0 0x7f662c161f10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f662c08b9e0 tx=0x7f6628009bd0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.797+0000 7f6633f63700 1 -- 172.21.15.5:0/1790121869 shutdown_connections 2024-05-08T23:25:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.797+0000 7f6633f63700 1 --2- 172.21.15.5:0/1790121869 >> v2:172.21.15.49:3300/0 conn(0x7f662c15db80 0x7f662c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.797+0000 7f6633f63700 1 --2- 172.21.15.5:0/1790121869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c15d7a0 0x7f662c161f10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.797+0000 7f6633f63700 1 -- 172.21.15.5:0/1790121869 >> 172.21.15.5:0/1790121869 conn(0x7f662c08afe0 msgr2=0x7f662c0d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.797+0000 7f6633f63700 1 -- 172.21.15.5:0/1790121869 shutdown_connections 2024-05-08T23:25:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.797+0000 7f6633f63700 1 -- 172.21.15.5:0/1790121869 wait complete. 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.798+0000 7f6633f63700 1 Processor -- start 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6633f63700 1 -- start start 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6633f63700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c0d3f40 0x7f662c154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6633f63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f662c15d7a0 0x7f662c0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6633f63700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f662c15db80 0x7f662c0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6633f63700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c15e980 con 0x7f662c0d3f40 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6633f63700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0c7c90 con 0x7f662c15d7a0 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6633f63700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c15dfe0 con 0x7f662c15db80 2024-05-08T23:25:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6631cff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c0d3f40 0x7f662c154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6631cff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c0d3f40 0x7f662c154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50092/0 (socket says 172.21.15.5:50092) 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.799+0000 7f6631cff700 1 -- 172.21.15.5:0/33598573 learned_addr learned my addr 172.21.15.5:0/33598573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f66314fe700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f662c15d7a0 0x7f662c0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6632500700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f662c15db80 0x7f662c0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6631cff700 1 -- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f662c15db80 msgr2=0x7f662c0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6631cff700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f662c15db80 0x7f662c0b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6631cff700 1 -- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f662c15d7a0 msgr2=0x7f662c0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6631cff700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f662c15d7a0 0x7f662c0b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6631cff700 1 -- 172.21.15.5:0/33598573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628009550 con 0x7f662c0d3f40 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6631cff700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c0d3f40 0x7f662c154bd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6628000c00 tx=0x7f6628004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f66227fc700 1 -- 172.21.15.5:0/33598573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628004570 con 0x7f662c0d3f40 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f66227fc700 1 -- 172.21.15.5:0/33598573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662801f070 con 0x7f662c0d3f40 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662c15e2c0 con 0x7f662c0d3f40 2024-05-08T23:25:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f66227fc700 1 -- 172.21.15.5:0/33598573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280038e0 con 0x7f662c0d3f40 2024-05-08T23:25:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.800+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c155660 con 0x7f662c0d3f40 2024-05-08T23:25:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.801+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662c068000 con 0x7f662c0d3f40 2024-05-08T23:25:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.802+0000 7f66227fc700 1 -- 172.21.15.5:0/33598573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6628021030 con 0x7f662c0d3f40 2024-05-08T23:25:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.802+0000 7f66227fc700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f661006ac90 0x7f661006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.802+0000 7f66227fc700 1 -- 172.21.15.5:0/33598573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f662808a530 con 0x7f662c0d3f40 2024-05-08T23:25:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.802+0000 7f66314fe700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f661006ac90 0x7f661006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.803+0000 7f66314fe700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f661006ac90 0x7f661006d150 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f662c158340 tx=0x7f661c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:56.807+0000 7f66227fc700 1 -- 172.21.15.5:0/33598573 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6628055b10 con 0x7f662c0d3f40 2024-05-08T23:25:57.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.122+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) v1 -- 0x7f662c0d1600 con 0x7f662c0d3f40 2024-05-08T23:25:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.877+0000 7f66227fc700 1 -- 172.21.15.5:0/33598573 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v254) v1 ==== 144+0+0 (secure 0 0 0) 0x7f66280150d0 con 0x7f662c0d3f40 2024-05-08T23:25:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f661006ac90 msgr2=0x7f661006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f661006ac90 0x7f661006d150 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f662c158340 tx=0x7f661c009040 comp rx=0 tx=0).stop 2024-05-08T23:25:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c0d3f40 msgr2=0x7f662c154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c0d3f40 0x7f662c154bd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6628000c00 tx=0x7f6628004750 comp rx=0 tx=0).stop 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 shutdown_connections 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f661006ac90 0x7f661006d150 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f662c15db80 0x7f662c0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f662c0d3f40 0x7f662c154bd0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 --2- 172.21.15.5:0/33598573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f662c15d7a0 0x7f662c0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 >> 172.21.15.5:0/33598573 conn(0x7f662c08afe0 msgr2=0x7f662c0d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 shutdown_connections 2024-05-08T23:25:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:57.880+0000 7f6633f63700 1 -- 172.21.15.5:0/33598573 wait complete. 2024-05-08T23:25:57.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T23:25:58.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.038+0000 7feb3088e700 1 Processor -- start 2024-05-08T23:25:58.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.039+0000 7feb3088e700 1 -- start start 2024-05-08T23:25:58.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.039+0000 7feb3088e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0aea70 0x7feb2c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.039+0000 7feb3088e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb2c0d0a60 0x7feb2c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.039+0000 7feb3088e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c15add0 con 0x7feb2c154df0 2024-05-08T23:25:58.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.039+0000 7feb3088e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c15abe0 con 0x7feb2c0d0a60 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.039+0000 7feb3088e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c15b070 con 0x7feb2c0aea70 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.039+0000 7feb2a59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7feb2c154df0 0x7feb2c0d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35076/0 (socket says 172.21.15.5:35076) 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb2a59c700 1 -- 172.21.15.5:0/1032153321 learned_addr learned my addr 172.21.15.5:0/1032153321 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb2ad9d700 1 --2- 172.21.15.5:0/1032153321 >> v2:172.21.15.5:3300/0 conn(0x7feb2c0d0a60 0x7feb2c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb21d9b700 1 --2- 172.21.15.5:0/1032153321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0aea70 0x7feb2c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb29d9b700 1 -- 172.21.15.5:0/1032153321 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025342122 0 0) 0x7feb2c15add0 con 0x7feb2c154df0 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb29d9b700 1 -- 172.21.15.5:0/1032153321 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb180035b0 con 0x7feb2c154df0 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb2ad9d700 1 -- 172.21.15.5:0/1032153321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0aea70 msgr2=0x7feb2c0b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb2ad9d700 1 --2- 172.21.15.5:0/1032153321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0aea70 0x7feb2c0b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb2ad9d700 1 -- 172.21.15.5:0/1032153321 >> v1:172.21.15.5:6789/0 conn(0x7feb2c154df0 legacy=0x7feb2c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb2ad9d700 1 -- 172.21.15.5:0/1032153321 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c15b350 con 0x7feb2c0d0a60 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb2ad9d700 1 --2- 172.21.15.5:0/1032153321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0d0a60 0x7feb2c0b43b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7feb24009d90 tx=0x7feb2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abdf79a509eb3c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:58.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb29d9b700 1 -- 172.21.15.5:0/1032153321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2400e070 con 0x7feb2c0d0a60 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb29d9b700 1 -- 172.21.15.5:0/1032153321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2400bb20 con 0x7feb2c0d0a60 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.040+0000 7feb29d9b700 1 -- 172.21.15.5:0/1032153321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24012430 con 0x7feb2c0d0a60 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.041+0000 7feb3088e700 1 -- 172.21.15.5:0/1032153321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0d0a60 msgr2=0x7feb2c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.041+0000 7feb3088e700 1 --2- 172.21.15.5:0/1032153321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0d0a60 0x7feb2c0b43b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7feb24009d90 tx=0x7feb2400b330 comp rx=0 tx=0).stop 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.041+0000 7feb3088e700 1 -- 172.21.15.5:0/1032153321 shutdown_connections 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.041+0000 7feb3088e700 1 --2- 172.21.15.5:0/1032153321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0d0a60 0x7feb2c0b43b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.041+0000 7feb3088e700 1 --2- 172.21.15.5:0/1032153321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0aea70 0x7feb2c0b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.041+0000 7feb3088e700 1 -- 172.21.15.5:0/1032153321 >> 172.21.15.5:0/1032153321 conn(0x7feb2c099a50 msgr2=0x7feb2c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.041+0000 7feb3088e700 1 -- 172.21.15.5:0/1032153321 shutdown_connections 2024-05-08T23:25:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.041+0000 7feb3088e700 1 -- 172.21.15.5:0/1032153321 wait complete. 2024-05-08T23:25:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.043+0000 7feb3088e700 1 Processor -- start 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.043+0000 7feb3088e700 1 -- start start 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.043+0000 7feb3088e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0aea70 0x7feb2c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.043+0000 7feb3088e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0d0a60 0x7feb2c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb3088e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb2c154df0 0x7feb2c06aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb3088e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c15c8e0 con 0x7feb2c0aea70 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb3088e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c15d790 con 0x7feb2c154df0 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb3088e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c15da70 con 0x7feb2c0d0a60 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0aea70 0x7feb2c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0aea70 0x7feb2c15d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40670/0 (socket says 172.21.15.5:40670) 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 -- 172.21.15.5:0/2053476670 learned_addr learned my addr 172.21.15.5:0/2053476670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2ad9d700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb2c154df0 0x7feb2c06aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb21d9b700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0d0a60 0x7feb2c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 -- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0d0a60 msgr2=0x7feb2c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0d0a60 0x7feb2c06a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 -- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb2c154df0 msgr2=0x7feb2c06aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb2c154df0 0x7feb2c06aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 -- 172.21.15.5:0/2053476670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24009a70 con 0x7feb2c0aea70 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb2a59c700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0aea70 0x7feb2c15d050 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7feb1c001eb0 tx=0x7feb1c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb237fe700 1 -- 172.21.15.5:0/2053476670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c005d70 con 0x7feb2c0aea70 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb237fe700 1 -- 172.21.15.5:0/2053476670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c0070d0 con 0x7feb2c0aea70 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c15dd50 con 0x7feb2c0aea70 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.044+0000 7feb237fe700 1 -- 172.21.15.5:0/2053476670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c007460 con 0x7feb2c0aea70 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.045+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c0cfd30 con 0x7feb2c0aea70 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.045+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c0cfef0 con 0x7feb2c0aea70 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.046+0000 7feb237fe700 1 -- 172.21.15.5:0/2053476670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feb1c0075c0 con 0x7feb2c0aea70 2024-05-08T23:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.046+0000 7feb237fe700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb0c06ac90 0x7feb0c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.047+0000 7feb237fe700 1 -- 172.21.15.5:0/2053476670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5115+0+0 (secure 0 0 0) 0x7feb1c0884a0 con 0x7feb2c0aea70 2024-05-08T23:25:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.047+0000 7feb21d9b700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb0c06ac90 0x7feb0c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.047+0000 7feb21d9b700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb0c06ac90 0x7feb0c06d150 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0d0780 tx=0x7feb14009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.051+0000 7feb237fe700 1 -- 172.21.15.5:0/2053476670 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feb1c0538d0 con 0x7feb2c0aea70 2024-05-08T23:25:58.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.268+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7feb2c074bc0 con 0x7feb2c0aea70 2024-05-08T23:25:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.272+0000 7feb237fe700 1 -- 172.21.15.5:0/2053476670 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7feb1c056ef0 con 0x7feb2c0aea70 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb0c06ac90 msgr2=0x7feb0c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb0c06ac90 0x7feb0c06d150 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0d0780 tx=0x7feb14009210 comp rx=0 tx=0).stop 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0aea70 msgr2=0x7feb2c15d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0aea70 0x7feb2c15d050 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7feb1c001eb0 tx=0x7feb1c00c7b0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 shutdown_connections 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb0c06ac90 0x7feb0c06d150 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb2c154df0 0x7feb2c06aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb2c0aea70 0x7feb2c15d050 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 --2- 172.21.15.5:0/2053476670 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb2c0d0a60 0x7feb2c06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 >> 172.21.15.5:0/2053476670 conn(0x7feb2c099a50 msgr2=0x7feb2c06f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 shutdown_connections 2024-05-08T23:25:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.274+0000 7feb3088e700 1 -- 172.21.15.5:0/2053476670 wait complete. 2024-05-08T23:25:58.289 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:25:58.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:25:58.337 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:25:58.394 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:25:58.395 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:25:58.395 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:58.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:25:58.416 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:25:58.416 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:58.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:25:58.437 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:25:58.437 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:58.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:25:58.703 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:25:58.704 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:25:58.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:25:58.726 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T23:25:58.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:25:58.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.872+0000 7f6688f55700 1 Processor -- start 2024-05-08T23:25:58.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.872+0000 7f6688f55700 1 -- start start 2024-05-08T23:25:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.873+0000 7f6688f55700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f66840d3b30 0x7f66840cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.873+0000 7f6688f55700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f668415e6c0 0x7f6684161850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.873+0000 7f6688f55700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66840d0f80 con 0x7f668415d2d0 2024-05-08T23:25:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.873+0000 7f6688f55700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840ce290 con 0x7f668415e6c0 2024-05-08T23:25:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.873+0000 7f6688f55700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840cedd0 con 0x7f66840d3b30 2024-05-08T23:25:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.873+0000 7f6682d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f668415e6c0 0x7f6684161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.873+0000 7f6682d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f668415e6c0 0x7f6684161850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40684/0 (socket says 172.21.15.5:40684) 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.873+0000 7f6682d9d700 1 -- 172.21.15.5:0/802365932 learned_addr learned my addr 172.21.15.5:0/802365932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f6681d9b700 1 --2- 172.21.15.5:0/802365932 >> v2:172.21.15.49:3300/0 conn(0x7f66840d3b30 0x7f66840cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f6682d9d700 1 -- 172.21.15.5:0/802365932 >> v2:172.21.15.49:3300/0 conn(0x7f66840d3b30 msgr2=0x7f66840cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f6682d9d700 1 --2- 172.21.15.5:0/802365932 >> v2:172.21.15.49:3300/0 conn(0x7f66840d3b30 0x7f66840cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f6682d9d700 1 -- 172.21.15.5:0/802365932 >> v1:172.21.15.49:6789/0 conn(0x7f668415d2d0 legacy=0x7f66840d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f6682d9d700 1 -- 172.21.15.5:0/802365932 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840ce4a0 con 0x7f668415e6c0 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f6681d9b700 1 --2- 172.21.15.5:0/802365932 >> v2:172.21.15.49:3300/0 conn(0x7f66840d3b30 0x7f66840cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f6682d9d700 1 --2- 172.21.15.5:0/802365932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f6684161850 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f668409d0e0 tx=0x7f667c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25efb4fca0e830b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f668159a700 1 -- 172.21.15.5:0/802365932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00e070 con 0x7f668415e6c0 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f668159a700 1 -- 172.21.15.5:0/802365932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c003a20 con 0x7f668415e6c0 2024-05-08T23:25:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.874+0000 7f668159a700 1 -- 172.21.15.5:0/802365932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c003dd0 con 0x7f668415e6c0 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.875+0000 7f6688f55700 1 -- 172.21.15.5:0/802365932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 msgr2=0x7f6684161850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.875+0000 7f6688f55700 1 --2- 172.21.15.5:0/802365932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f6684161850 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f668409d0e0 tx=0x7f667c00b330 comp rx=0 tx=0).stop 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.875+0000 7f6688f55700 1 -- 172.21.15.5:0/802365932 shutdown_connections 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.875+0000 7f6688f55700 1 --2- 172.21.15.5:0/802365932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f6684161850 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.875+0000 7f6688f55700 1 --2- 172.21.15.5:0/802365932 >> v2:172.21.15.49:3300/0 conn(0x7f66840d3b30 0x7f66840cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.875+0000 7f6688f55700 1 -- 172.21.15.5:0/802365932 >> 172.21.15.5:0/802365932 conn(0x7f668409c9b0 msgr2=0x7f668408afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.875+0000 7f6688f55700 1 -- 172.21.15.5:0/802365932 shutdown_connections 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.875+0000 7f6688f55700 1 -- 172.21.15.5:0/802365932 wait complete. 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.876+0000 7f6688f55700 1 Processor -- start 2024-05-08T23:25:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.877+0000 7f6688f55700 1 -- start start 2024-05-08T23:25:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.877+0000 7f6688f55700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66840d3b30 0x7f66840cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.877+0000 7f6688f55700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f668415d2d0 0x7f66840b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.877+0000 7f6688f55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f66840d0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.877+0000 7f6688f55700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840b5590 con 0x7f668415e6c0 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.877+0000 7f6688f55700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840b9530 con 0x7f66840d3b30 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.877+0000 7f6688f55700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66840b89b0 con 0x7f668415d2d0 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.877+0000 7f6682d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f66840d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6682d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f66840d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40694/0 (socket says 172.21.15.5:40694) 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6682d9d700 1 -- 172.21.15.5:0/3890322965 learned_addr learned my addr 172.21.15.5:0/3890322965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f668259c700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66840d3b30 0x7f66840cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6681d9b700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f668415d2d0 0x7f66840b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6682d9d700 1 -- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f668415d2d0 msgr2=0x7f66840b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6682d9d700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f668415d2d0 0x7f66840b80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6682d9d700 1 -- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66840d3b30 msgr2=0x7f66840cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6682d9d700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66840d3b30 0x7f66840cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6682d9d700 1 -- 172.21.15.5:0/3890322965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c009b30 con 0x7f668415e6c0 2024-05-08T23:25:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6682d9d700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f66840d0090 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f667c015040 tx=0x7f667c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6672ffd700 1 -- 172.21.15.5:0/3890322965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0094f0 con 0x7f668415e6c0 2024-05-08T23:25:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6672ffd700 1 -- 172.21.15.5:0/3890322965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c004240 con 0x7f668415e6c0 2024-05-08T23:25:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6672ffd700 1 -- 172.21.15.5:0/3890322965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0045f0 con 0x7f668415e6c0 2024-05-08T23:25:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668415ecd0 con 0x7f668415e6c0 2024-05-08T23:25:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.878+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66840b8f70 con 0x7f668415e6c0 2024-05-08T23:25:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.879+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66840ca7e0 con 0x7f668415e6c0 2024-05-08T23:25:58.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.880+0000 7f6672ffd700 1 -- 172.21.15.5:0/3890322965 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f667c019070 con 0x7f668415e6c0 2024-05-08T23:25:58.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.880+0000 7f6672ffd700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f666406ac90 0x7f666406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:58.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.880+0000 7f6672ffd700 1 -- 172.21.15.5:0/3890322965 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f667c089fa0 con 0x7f668415e6c0 2024-05-08T23:25:58.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.881+0000 7f668259c700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f666406ac90 0x7f666406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:58.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.881+0000 7f668259c700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f666406ac90 0x7f666406d150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6674007fd0 tx=0x7f6674007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:58.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:58.885+0000 7f6672ffd700 1 -- 172.21.15.5:0/3890322965 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f667c058e50 con 0x7f668415e6c0 2024-05-08T23:25:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.153+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66840678f0 con 0x7f668415e6c0 2024-05-08T23:25:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.154+0000 7f6672ffd700 1 -- 172.21.15.5:0/3890322965 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f667c020090 con 0x7f668415e6c0 2024-05-08T23:25:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:25:59.154 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2413116038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":2413116038},{"type":"v1","addr":"172.21.15.49:6837","nonce":2413116038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6709,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3719081079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3719081079},{"type":"v1","addr":"172.21.15.49:6836","nonce":3719081079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/139281319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":139281319},{"type":"v1","addr":"172.21.15.5:6833","nonce":139281319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7092,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1531159308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1531159308},{"type":"v1","addr":"172.21.15.5:6835","nonce":1531159308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[]} 2024-05-08T23:25:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f666406ac90 msgr2=0x7f666406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f666406ac90 0x7f666406d150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6674007fd0 tx=0x7f6674007ec0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 msgr2=0x7f66840d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f66840d0090 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f667c015040 tx=0x7f667c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 shutdown_connections 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f666406ac90 0x7f666406d150 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f668415e6c0 0x7f66840d0090 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66840d3b30 0x7f66840cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 --2- 172.21.15.5:0/3890322965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f668415d2d0 0x7f66840b80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 >> 172.21.15.5:0/3890322965 conn(0x7f668409c9b0 msgr2=0x7f66840d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 shutdown_connections 2024-05-08T23:25:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.156+0000 7f6688f55700 1 -- 172.21.15.5:0/3890322965 wait complete. 2024-05-08T23:25:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 61 2024-05-08T23:25:59.170 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:25:59.171 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T23:25:59.171 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:25:59.171 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:25:59.171 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:25:59.171 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T23:25:59.171 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:25:59.171 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:25:59.172 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:25:59.172 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T23:25:59.172 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:25:59.172 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:25:59.172 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:25:59.172 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T23:25:59.172 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:25:59.172 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:25:59.181 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:25:59.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:25:59.182 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:25:59.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:25:59.184 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:25:59.184 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:25:59.185 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:25:59.185 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:25:59.224 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:25:59.225 DEBUG:teuthology.parallel:result is None 2024-05-08T23:25:59.225 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:25:59.226 DEBUG:teuthology.parallel:result is None 2024-05-08T23:25:59.228 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:25:59.228 DEBUG:teuthology.parallel:result is None 2024-05-08T23:25:59.229 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:25:59.229 DEBUG:teuthology.parallel:result is None 2024-05-08T23:25:59.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T23:25:59.316 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:25:59.316 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:25:59.319 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:25:59.319 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:25:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.378+0000 7f3d7e819700 1 Processor -- start 2024-05-08T23:25:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.378+0000 7f3d7e819700 1 -- start start 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d7e819700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d78151530 0x7f3d780b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d7e819700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3d7806b970 0x7f3d7806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d7e819700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d7815e070 0x7f3d7815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d7e819700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7815bb50 con 0x7f3d7806b970 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d7e819700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d780ce760 con 0x7f3d7815e070 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d7e819700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7806c330 con 0x7f3d78151530 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d777fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3d7806b970 0x7f3d7806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d777fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3d7806b970 0x7f3d7806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40730/0 (socket says 172.21.15.5:40730) 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d777fe700 1 -- 172.21.15.5:0/2770742962 learned_addr learned my addr 172.21.15.5:0/2770742962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d77fff700 1 --2- 172.21.15.5:0/2770742962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d78151530 0x7f3d780b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.379+0000 7f3d7cdb6700 1 --2- 172.21.15.5:0/2770742962 >> v2:172.21.15.49:3300/0 conn(0x7f3d7815e070 0x7f3d7815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d777fe700 1 -- 172.21.15.5:0/2770742962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d78151530 msgr2=0x7f3d780b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d777fe700 1 --2- 172.21.15.5:0/2770742962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d78151530 0x7f3d780b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d777fe700 1 -- 172.21.15.5:0/2770742962 >> v2:172.21.15.49:3300/0 conn(0x7f3d7815e070 msgr2=0x7f3d7815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d777fe700 1 --2- 172.21.15.5:0/2770742962 >> v2:172.21.15.49:3300/0 conn(0x7f3d7815e070 0x7f3d7815e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d777fe700 1 -- 172.21.15.5:0/2770742962 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d780cf580 con 0x7f3d7806b970 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d777fe700 1 --2- 172.21.15.5:0/2770742962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d7806bdf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3d780712e0 tx=0x7f3d60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a4812077a45438 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d76ffd700 1 -- 172.21.15.5:0/2770742962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60011830 con 0x7f3d7806b970 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d76ffd700 1 -- 172.21.15.5:0/2770742962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d60011990 con 0x7f3d7806b970 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.380+0000 7f3d76ffd700 1 -- 172.21.15.5:0/2770742962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60011d20 con 0x7f3d7806b970 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 -- 172.21.15.5:0/2770742962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 msgr2=0x7f3d7806bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 --2- 172.21.15.5:0/2770742962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d7806bdf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3d780712e0 tx=0x7f3d60009bd0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 -- 172.21.15.5:0/2770742962 shutdown_connections 2024-05-08T23:25:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 --2- 172.21.15.5:0/2770742962 >> v2:172.21.15.49:3300/0 conn(0x7f3d7815e070 0x7f3d7815e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 --2- 172.21.15.5:0/2770742962 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d78151530 0x7f3d780b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 --2- 172.21.15.5:0/2770742962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d7806bdf0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 -- 172.21.15.5:0/2770742962 >> 172.21.15.5:0/2770742962 conn(0x7f3d78071350 msgr2=0x7f3d78071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 -- 172.21.15.5:0/2770742962 shutdown_connections 2024-05-08T23:25:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.381+0000 7f3d7e819700 1 -- 172.21.15.5:0/2770742962 wait complete. 2024-05-08T23:25:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.382+0000 7f3d7e819700 1 Processor -- start 2024-05-08T23:25:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.383+0000 7f3d7e819700 1 -- start start 2024-05-08T23:25:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d7e819700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d78144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d7e819700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d78151530 0x7f3d78147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d7e819700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d7815e070 0x7f3d7814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d7e819700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780c9210 con 0x7f3d7806b970 2024-05-08T23:25:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d7e819700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780c94f0 con 0x7f3d78151530 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d7e819700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780c97d0 con 0x7f3d7815e070 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d78144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d78144b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40744/0 (socket says 172.21.15.5:40744) 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 -- 172.21.15.5:0/4269194866 learned_addr learned my addr 172.21.15.5:0/4269194866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d777fe700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d78151530 0x7f3d78147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d7cdb6700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d7815e070 0x7f3d7814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 -- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d7815e070 msgr2=0x7f3d7814a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d7815e070 0x7f3d7814a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 -- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d78151530 msgr2=0x7f3d78147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d78151530 0x7f3d78147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 -- 172.21.15.5:0/4269194866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d60009550 con 0x7f3d7806b970 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.384+0000 7f3d77fff700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d78144b40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800e9b0 tx=0x7f3d6800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.385+0000 7f3d74ff9700 1 -- 172.21.15.5:0/4269194866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68009710 con 0x7f3d7806b970 2024-05-08T23:25:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.385+0000 7f3d74ff9700 1 -- 172.21.15.5:0/4269194866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6800ee50 con 0x7f3d7806b970 2024-05-08T23:25:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.385+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d780c9ab0 con 0x7f3d7806b970 2024-05-08T23:25:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.385+0000 7f3d74ff9700 1 -- 172.21.15.5:0/4269194866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800ca50 con 0x7f3d7806b970 2024-05-08T23:25:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.385+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d78147cc0 con 0x7f3d7806b970 2024-05-08T23:25:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.386+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d78066fb0 con 0x7f3d7806b970 2024-05-08T23:25:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.386+0000 7f3d74ff9700 1 -- 172.21.15.5:0/4269194866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3d6800cbb0 con 0x7f3d7806b970 2024-05-08T23:25:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.387+0000 7f3d74ff9700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3d5806ac90 0x7f3d5806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.387+0000 7f3d74ff9700 1 -- 172.21.15.5:0/4269194866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f3d68089120 con 0x7f3d7806b970 2024-05-08T23:25:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.387+0000 7f3d777fe700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3d5806ac90 0x7f3d5806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.387+0000 7f3d777fe700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3d5806ac90 0x7f3d5806d150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009c10 tx=0x7f3d60008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.391+0000 7f3d74ff9700 1 -- 172.21.15.5:0/4269194866 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3d68054550 con 0x7f3d7806b970 2024-05-08T23:25:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:25:59.626+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3d78068000 con 0x7f3d7806b970 2024-05-08T23:26:00.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.354+0000 7f3d74ff9700 1 -- 172.21.15.5:0/4269194866 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3d6800ce60 con 0x7f3d7806b970 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3d5806ac90 msgr2=0x7f3d5806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3d5806ac90 0x7f3d5806d150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009c10 tx=0x7f3d60008030 comp rx=0 tx=0).stop 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 msgr2=0x7f3d78144b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d78144b40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800e9b0 tx=0x7f3d6800ecc0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 shutdown_connections 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3d5806ac90 0x7f3d5806d150 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d7815e070 0x7f3d7814a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d7806b970 0x7f3d78144b40 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 --2- 172.21.15.5:0/4269194866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d78151530 0x7f3d78147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 >> 172.21.15.5:0/4269194866 conn(0x7f3d78071350 msgr2=0x7f3d7814b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 shutdown_connections 2024-05-08T23:26:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.357+0000 7f3d7e819700 1 -- 172.21.15.5:0/4269194866 wait complete. 2024-05-08T23:26:00.359 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-08T23:26:00.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T23:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.514+0000 7ff9ea546700 1 Processor -- start 2024-05-08T23:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.514+0000 7ff9ea546700 1 -- start start 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9ea546700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff9e415ef60 0x7ff9e40b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9ea546700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e40cfb70 0x7ff9e40b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9ea546700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e409ad40 con 0x7ff9e40cf000 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9ea546700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e40cd400 con 0x7ff9e415ef60 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9ea546700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e40ccb60 con 0x7ff9e40cfb70 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9e3fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff9e415ef60 0x7ff9e40b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9e37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff9e40cf000 0x7ff9e40cf460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35118/0 (socket says 172.21.15.5:35118) 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9e3fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff9e415ef60 0x7ff9e40b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40750/0 (socket says 172.21.15.5:40750) 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9e37fe700 1 -- 172.21.15.5:0/1840418301 learned_addr learned my addr 172.21.15.5:0/1840418301 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9e8ae3700 1 --2- 172.21.15.5:0/1840418301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e40cfb70 0x7ff9e40b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.515+0000 7ff9e3fff700 1 -- 172.21.15.5:0/1840418301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e40cfb70 msgr2=0x7ff9e40b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e3fff700 1 --2- 172.21.15.5:0/1840418301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e40cfb70 0x7ff9e40b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e3fff700 1 -- 172.21.15.5:0/1840418301 >> v1:172.21.15.5:6789/0 conn(0x7ff9e40cf000 legacy=0x7ff9e40cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e3fff700 1 -- 172.21.15.5:0/1840418301 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40cce40 con 0x7ff9e415ef60 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e2ffd700 1 -- 172.21.15.5:0/1840418301 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440403293 0 0) 0x7ff9e409ad40 con 0x7ff9e40cf000 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e8ae3700 1 --2- 172.21.15.5:0/1840418301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e40cfb70 0x7ff9e40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e3fff700 1 --2- 172.21.15.5:0/1840418301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e415ef60 0x7ff9e40b2340 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009840 tx=0x7ff9d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95f9b3762140610 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e2ffd700 1 -- 172.21.15.5:0/1840418301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4011830 con 0x7ff9e415ef60 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e2ffd700 1 -- 172.21.15.5:0/1840418301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d4011990 con 0x7ff9e415ef60 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9e2ffd700 1 -- 172.21.15.5:0/1840418301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4011d20 con 0x7ff9e415ef60 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9ea546700 1 -- 172.21.15.5:0/1840418301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e415ef60 msgr2=0x7ff9e40b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.516+0000 7ff9ea546700 1 --2- 172.21.15.5:0/1840418301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e415ef60 0x7ff9e40b2340 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009840 tx=0x7ff9d4009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.517+0000 7ff9ea546700 1 -- 172.21.15.5:0/1840418301 shutdown_connections 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.517+0000 7ff9ea546700 1 --2- 172.21.15.5:0/1840418301 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e40cfb70 0x7ff9e40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.517+0000 7ff9ea546700 1 --2- 172.21.15.5:0/1840418301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e415ef60 0x7ff9e40b2340 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.517+0000 7ff9ea546700 1 -- 172.21.15.5:0/1840418301 >> 172.21.15.5:0/1840418301 conn(0x7ff9e40b2f20 msgr2=0x7ff9e40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:00.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.517+0000 7ff9ea546700 1 -- 172.21.15.5:0/1840418301 shutdown_connections 2024-05-08T23:26:00.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.517+0000 7ff9ea546700 1 -- 172.21.15.5:0/1840418301 wait complete. 2024-05-08T23:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.518+0000 7ff9ea546700 1 Processor -- start 2024-05-08T23:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.518+0000 7ff9ea546700 1 -- start start 2024-05-08T23:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9ea546700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e40cf000 0x7ff9e40c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9ea546700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e40cfb70 0x7ff9e409d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9ea546700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e415ef60 0x7ff9e409ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9ea546700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e406d4d0 con 0x7ff9e40cfb70 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9ea546700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e409cbd0 con 0x7ff9e40cf000 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9ea546700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e409ceb0 con 0x7ff9e415ef60 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e40cfb70 0x7ff9e409d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e40cfb70 0x7ff9e409d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40754/0 (socket says 172.21.15.5:40754) 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e37fe700 1 -- 172.21.15.5:0/2172186989 learned_addr learned my addr 172.21.15.5:0/2172186989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e3fff700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e40cf000 0x7ff9e40c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e8ae3700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e415ef60 0x7ff9e409ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e37fe700 1 -- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e415ef60 msgr2=0x7ff9e409ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e37fe700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e415ef60 0x7ff9e409ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e37fe700 1 -- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e40cf000 msgr2=0x7ff9e40c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e37fe700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e40cf000 0x7ff9e40c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e37fe700 1 -- 172.21.15.5:0/2172186989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40094a0 con 0x7ff9e40cfb70 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.519+0000 7ff9e3fff700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e40cf000 0x7ff9e40c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.520+0000 7ff9e37fe700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e40cfb70 0x7ff9e409d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80011e0 tx=0x7ff9d800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.520+0000 7ff9e0ff9700 1 -- 172.21.15.5:0/2172186989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8004c80 con 0x7ff9e40cfb70 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.520+0000 7ff9e0ff9700 1 -- 172.21.15.5:0/2172186989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d8004de0 con 0x7ff9e40cfb70 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.520+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e409d190 con 0x7ff9e40cfb70 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.520+0000 7ff9e0ff9700 1 -- 172.21.15.5:0/2172186989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8007310 con 0x7ff9e40cfb70 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.520+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e40cdc00 con 0x7ff9e40cfb70 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.521+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e4068000 con 0x7ff9e40cfb70 2024-05-08T23:26:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.521+0000 7ff9e0ff9700 1 -- 172.21.15.5:0/2172186989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff9d8007470 con 0x7ff9e40cfb70 2024-05-08T23:26:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.522+0000 7ff9e0ff9700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff9d006ac90 0x7ff9d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.522+0000 7ff9e0ff9700 1 -- 172.21.15.5:0/2172186989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9d8088cf0 con 0x7ff9e40cfb70 2024-05-08T23:26:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.522+0000 7ff9e3fff700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff9d006ac90 0x7ff9d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.523+0000 7ff9e3fff700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff9d006ac90 0x7ff9d006d150 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40090b0 tx=0x7ff9d401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:00.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.527+0000 7ff9e0ff9700 1 -- 172.21.15.5:0/2172186989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff9d8054210 con 0x7ff9e40cfb70 2024-05-08T23:26:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.835+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff9e40cf7e0 con 0x7ff9e40cfb70 2024-05-08T23:26:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.836+0000 7ff9e0ff9700 1 -- 172.21.15.5:0/2172186989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff9d8057830 con 0x7ff9e40cfb70 2024-05-08T23:26:00.836 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:00.836 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:00.836 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-08T23:26:00.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.836 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-08T23:26:00.836 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm4+KLORAAJGma8zcoGLift7K5wyvWlA==", 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm5iMNDBAANT+UrtTUBpYyQ9dzJyS01Q==", 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:00.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmRixkBhAAk9s24ERPJ/6nCJY3g7QppA==", 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmPmurExAAxIzDzbLZ3s35XVJrvYiVCA==", 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCrCDxmI4/bNBAAzNXcuLQCS9gaAJ0r4iYKlA==", 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCuCDxmMS6HNhAAcTP6nStED06bdM4MtbEPjQ==", 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCxCDxmDGv5NRAAljFQZXLGWwt+UXZ+HCdx0Q==", 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC0CDxmL+5LNRAAVPySIUtICBwIokJL9fHFtw==", 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:00.841 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC5CDxm/F15BhAAEx9mKA9izSmb+xyxjEi9FA==", 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:00.842 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:00.843 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.843 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-08T23:26:00.843 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC8CDxm0zAYBhAAjHT+TZFqk5MWvd0P5LmwfA==", 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC/CDxmPcbiBRAAOmWNgaQGi6YGKTp8SJekUg==", 2024-05-08T23:26:00.844 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDCCDxm9FsDBRAA/lfmrpmL4UO3s/tJJ4o6yQ==", 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm9tLAGBAAK8Lq89gJ902XqzXYAbYfkg==", 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.846 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmCStoIBAAhraObJ5LS9LtF97+hIcTFQ==", 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm2NzHHBAAUbeD58MntIoiwT8LekjIlw==", 2024-05-08T23:26:00.847 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmyvveIxAATEn7nOPHqtqSbdfQid25HA==", 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:26:00.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm3ZIxDxAAM8M5YUfK9G2PkTzU4wY/+w==", 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxmntlgNRAA7JOIS+0NL1KmSSZhZjXlVQ==", 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm1dPQMBAAhu0CK6DcucQteuTo7qN2IQ==", 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:26:00.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:26:00.851 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:00.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:00.851 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-08T23:26:00.851 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:00.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.838+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff9d006ac90 msgr2=0x7ff9d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.838+0000 7ff9ea546700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff9d006ac90 0x7ff9d006d150 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40090b0 tx=0x7ff9d401c040 comp rx=0 tx=0).stop 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.838+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e40cfb70 msgr2=0x7ff9e409d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.838+0000 7ff9ea546700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e40cfb70 0x7ff9e409d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80011e0 tx=0x7ff9d800ca30 comp rx=0 tx=0).stop 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.839+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 shutdown_connections 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.839+0000 7ff9ea546700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff9d006ac90 0x7ff9d006d150 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.839+0000 7ff9ea546700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff9e415ef60 0x7ff9e409ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.839+0000 7ff9ea546700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e40cf000 0x7ff9e40c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.839+0000 7ff9ea546700 1 --2- 172.21.15.5:0/2172186989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e40cfb70 0x7ff9e409d990 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.839+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 >> 172.21.15.5:0/2172186989 conn(0x7ff9e40b2f20 msgr2=0x7ff9e40b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.839+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 shutdown_connections 2024-05-08T23:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.839+0000 7ff9ea546700 1 -- 172.21.15.5:0/2172186989 wait complete. 2024-05-08T23:26:00.854 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:00.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.990+0000 7fc52f5bd700 1 Processor -- start 2024-05-08T23:26:00.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.990+0000 7fc52f5bd700 1 -- start start 2024-05-08T23:26:00.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.991+0000 7fc52f5bd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc5280ccc40 0x7fc5281665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.991+0000 7fc52f5bd700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280cd3b0 con 0x7fc52815f230 2024-05-08T23:26:00.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.991+0000 7fc52f5bd700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280af300 con 0x7fc5280aef20 2024-05-08T23:26:00.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.991+0000 7fc52f5bd700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc528141df0 con 0x7fc5280ccc40 2024-05-08T23:26:00.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.991+0000 7fc52cb58700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc52815f230 0x7fc5280cc530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35122/0 (socket says 172.21.15.5:35122) 2024-05-08T23:26:00.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.991+0000 7fc52cb58700 1 -- 172.21.15.5:0/939356923 learned_addr learned my addr 172.21.15.5:0/939356923 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.991+0000 7fc52db5a700 1 --2- 172.21.15.5:0/939356923 >> v2:172.21.15.49:3300/0 conn(0x7fc5280ccc40 0x7fc5281665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.991+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383909882 0 0) 0x7fc5280cd3b0 con 0x7fc52815f230 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5140035b0 con 0x7fc52815f230 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198047594 0 0) 0x7fc5280af300 con 0x7fc5280aef20 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5280cd3b0 con 0x7fc5280aef20 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2047198000 0 0) 0x7fc5140035b0 con 0x7fc52815f230 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 >> v2:172.21.15.49:3300/0 conn(0x7fc5280ccc40 msgr2=0x7fc5281665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 --2- 172.21.15.5:0/939356923 >> v2:172.21.15.49:3300/0 conn(0x7fc5280ccc40 0x7fc5281665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 >> v1:172.21.15.49:6789/0 conn(0x7fc5280aef20 legacy=0x7fc52815c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52815f6f0 con 0x7fc52815f230 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fc51c001670 con 0x7fc52815f230 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 >> v1:172.21.15.5:6789/0 conn(0x7fc52815f230 legacy=0x7fc5280cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc514003b90 0x7fc514003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514004850 0x7fc514004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc514005210 0x7fc51400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc528141df0 con 0x7fc514003b90 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51400c140 con 0x7fc514005210 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51400c300 con 0x7fc514004850 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52d359700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc514003b90 0x7fc514003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52db5a700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc514005210 0x7fc51400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52cb58700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514004850 0x7fc514004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52d359700 1 -- 172.21.15.5:0/939356923 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514004850 msgr2=0x7fc514004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52d359700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514004850 0x7fc514004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52d359700 1 -- 172.21.15.5:0/939356923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc514005210 msgr2=0x7fc51400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52d359700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc514005210 0x7fc51400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52db5a700 1 -- 172.21.15.5:0/939356923 reap_dead start 2024-05-08T23:26:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.992+0000 7fc52d359700 1 -- 172.21.15.5:0/939356923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140035b0 con 0x7fc514003b90 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc52d359700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc514003b90 0x7fc514003f90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc524001940 tx=0x7fc514005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7befabd37a119a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524001430 con 0x7fc514003b90 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524001d70 con 0x7fc514003b90 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc51bfff700 1 -- 172.21.15.5:0/939356923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524005530 con 0x7fc514003b90 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc52f5bd700 1 -- 172.21.15.5:0/939356923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc514003b90 msgr2=0x7fc514003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc52f5bd700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc514003b90 0x7fc514003f90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc524001940 tx=0x7fc514005880 comp rx=0 tx=0).stop 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc52f5bd700 1 -- 172.21.15.5:0/939356923 shutdown_connections 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc52f5bd700 1 --2- 172.21.15.5:0/939356923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc514003b90 0x7fc514003f90 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.993+0000 7fc52f5bd700 1 -- 172.21.15.5:0/939356923 >> 172.21.15.5:0/939356923 conn(0x7fc5280cfce0 msgr2=0x7fc5280d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.994+0000 7fc52f5bd700 1 -- 172.21.15.5:0/939356923 shutdown_connections 2024-05-08T23:26:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.994+0000 7fc52f5bd700 1 -- 172.21.15.5:0/939356923 wait complete. 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.995+0000 7fc52f5bd700 1 Processor -- start 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.995+0000 7fc52f5bd700 1 -- start start 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52f5bd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514003b90 0x7fc528138bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52f5bd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5280b0e30 0x7fc5281539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52f5bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc528153ee0 0x7fc528154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52f5bd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281534c0 con 0x7fc5280b0e30 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52f5bd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc528139500 con 0x7fc528153ee0 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52f5bd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281397e0 con 0x7fc514003b90 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5280b0e30 0x7fc5281539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5280b0e30 0x7fc5281539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40774/0 (socket says 172.21.15.5:40774) 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 -- 172.21.15.5:0/2513227527 learned_addr learned my addr 172.21.15.5:0/2513227527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52d359700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514003b90 0x7fc528138bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52db5a700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc528153ee0 0x7fc528154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 -- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514003b90 msgr2=0x7fc528138bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514003b90 0x7fc528138bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 -- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc528153ee0 msgr2=0x7fc528154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc528153ee0 0x7fc528154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 -- 172.21.15.5:0/2513227527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc528139ac0 con 0x7fc5280b0e30 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52cb58700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5280b0e30 0x7fc5281539a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc51c007310 tx=0x7fc51c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc519ffb700 1 -- 172.21.15.5:0/2513227527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00f660 con 0x7fc5280b0e30 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc519ffb700 1 -- 172.21.15.5:0/2513227527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00f800 con 0x7fc5280b0e30 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc528139da0 con 0x7fc5280b0e30 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.996+0000 7fc519ffb700 1 -- 172.21.15.5:0/2513227527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00fbb0 con 0x7fc5280b0e30 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.997+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5281392d0 con 0x7fc5280b0e30 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.998+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528068000 con 0x7fc5280b0e30 2024-05-08T23:26:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.998+0000 7fc519ffb700 1 -- 172.21.15.5:0/2513227527 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc51c00fd10 con 0x7fc5280b0e30 2024-05-08T23:26:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.999+0000 7fc519ffb700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc50c06ac90 0x7fc50c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.999+0000 7fc519ffb700 1 -- 172.21.15.5:0/2513227527 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc51c088ed0 con 0x7fc5280b0e30 2024-05-08T23:26:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:00.999+0000 7fc52d359700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc50c06ac90 0x7fc50c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.000+0000 7fc52d359700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc50c06ac90 0x7fc50c06d150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc524005040 tx=0x7fc52400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.003+0000 7fc519ffb700 1 -- 172.21.15.5:0/2513227527 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc51c0543f0 con 0x7fc5280b0e30 2024-05-08T23:26:01.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.221+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5280afe40 con 0x7fc5280b0e30 2024-05-08T23:26:01.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.222+0000 7fc519ffb700 1 -- 172.21.15.5:0/2513227527 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+12293 (secure 0 0 0) 0x7fc51c057a10 con 0x7fc5280b0e30 2024-05-08T23:26:01.222 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:01.223 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":68,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:00.349013+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc50c06ac90 msgr2=0x7fc50c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc50c06ac90 0x7fc50c06d150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc524005040 tx=0x7fc52400a000 comp rx=0 tx=0).stop 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5280b0e30 msgr2=0x7fc5281539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5280b0e30 0x7fc5281539a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc51c007310 tx=0x7fc51c007620 comp rx=0 tx=0).stop 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 shutdown_connections 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc50c06ac90 0x7fc50c06d150 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc528153ee0 0x7fc528154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc514003b90 0x7fc528138bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 --2- 172.21.15.5:0/2513227527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5280b0e30 0x7fc5281539a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 >> 172.21.15.5:0/2513227527 conn(0x7fc5280cfce0 msgr2=0x7fc5280ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 shutdown_connections 2024-05-08T23:26:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.225+0000 7fc52f5bd700 1 -- 172.21.15.5:0/2513227527 wait complete. 2024-05-08T23:26:01.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:26:01.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:26:01.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.379+0000 7fb9deb2c700 1 Processor -- start 2024-05-08T23:26:01.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.379+0000 7fb9deb2c700 1 -- start start 2024-05-08T23:26:01.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9deb2c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb9d806f8c0 0x7fb9d806a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9deb2c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80b3870 con 0x7fb9d80af990 2024-05-08T23:26:01.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9deb2c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80b4ee0 con 0x7fb9d815d880 2024-05-08T23:26:01.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9deb2c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d80b32f0 con 0x7fb9d806f8c0 2024-05-08T23:26:01.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9d7fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb9d80af990 0x7fb9d806e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35138/0 (socket says 172.21.15.5:35138) 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9d7fff700 1 -- 172.21.15.5:0/1054577910 learned_addr learned my addr 172.21.15.5:0/1054577910 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9dd0c9700 1 --2- 172.21.15.5:0/1054577910 >> v2:172.21.15.49:3300/0 conn(0x7fb9d806f8c0 0x7fb9d806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9d77fe700 1 -- 172.21.15.5:0/1054577910 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041469989 0 0) 0x7fb9d80b3870 con 0x7fb9d80af990 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9d77fe700 1 -- 172.21.15.5:0/1054577910 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c40035b0 con 0x7fb9d80af990 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9d77fe700 1 -- 172.21.15.5:0/1054577910 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 107164529 0 0) 0x7fb9c40035b0 con 0x7fb9d80af990 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9dd0c9700 1 -- 172.21.15.5:0/1054577910 >> v1:172.21.15.49:6789/0 conn(0x7fb9d815d880 legacy=0x7fb9d80ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9dd0c9700 1 -- 172.21.15.5:0/1054577910 >> v1:172.21.15.5:6789/0 conn(0x7fb9d80af990 legacy=0x7fb9d806e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.380+0000 7fb9dd0c9700 1 -- 172.21.15.5:0/1054577910 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80b5180 con 0x7fb9d806f8c0 2024-05-08T23:26:01.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9dd0c9700 1 --2- 172.21.15.5:0/1054577910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 0x7fb9d806a960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009d90 tx=0x7fb9d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ead0ef82caf8c638 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9d77fe700 1 -- 172.21.15.5:0/1054577910 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000e070 con 0x7fb9d806f8c0 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9d77fe700 1 -- 172.21.15.5:0/1054577910 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0003a20 con 0x7fb9d806f8c0 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9d77fe700 1 -- 172.21.15.5:0/1054577910 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0001670 con 0x7fb9d806f8c0 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9deb2c700 1 -- 172.21.15.5:0/1054577910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 msgr2=0x7fb9d806a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9deb2c700 1 --2- 172.21.15.5:0/1054577910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 0x7fb9d806a960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009d90 tx=0x7fb9d000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9deb2c700 1 -- 172.21.15.5:0/1054577910 shutdown_connections 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9deb2c700 1 --2- 172.21.15.5:0/1054577910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 0x7fb9d806a960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9deb2c700 1 -- 172.21.15.5:0/1054577910 >> 172.21.15.5:0/1054577910 conn(0x7fb9d80b5810 msgr2=0x7fb9d80b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9deb2c700 1 -- 172.21.15.5:0/1054577910 shutdown_connections 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.381+0000 7fb9deb2c700 1 -- 172.21.15.5:0/1054577910 wait complete. 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.383+0000 7fb9deb2c700 1 Processor -- start 2024-05-08T23:26:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.383+0000 7fb9deb2c700 1 -- start start 2024-05-08T23:26:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9deb2c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 0x7fb9d81f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9deb2c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9d80af990 0x7fb9d81539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9deb2c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9d815d880 0x7fb9d8143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9deb2c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81f7ca0 con 0x7fb9d815d880 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9deb2c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8153ef0 con 0x7fb9d806f8c0 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9deb2c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81541c0 con 0x7fb9d80af990 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dd0c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9d815d880 0x7fb9d8143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dd0c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9d815d880 0x7fb9d8143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40778/0 (socket says 172.21.15.5:40778) 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dd0c9700 1 -- 172.21.15.5:0/2573756556 learned_addr learned my addr 172.21.15.5:0/2573756556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dc8c8700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 0x7fb9d81f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9d7fff700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9d80af990 0x7fb9d81539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dd0c9700 1 -- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9d80af990 msgr2=0x7fb9d81539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dd0c9700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9d80af990 0x7fb9d81539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dd0c9700 1 -- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 msgr2=0x7fb9d81f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dd0c9700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 0x7fb9d81f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9dd0c9700 1 -- 172.21.15.5:0/2573756556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d0009a70 con 0x7fb9d815d880 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.384+0000 7fb9d7fff700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9d80af990 0x7fb9d81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.385+0000 7fb9dd0c9700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9d815d880 0x7fb9d8143790 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000b370 tx=0x7fb9d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.385+0000 7fb9d57fa700 1 -- 172.21.15.5:0/2573756556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0009450 con 0x7fb9d815d880 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.385+0000 7fb9d57fa700 1 -- 172.21.15.5:0/2573756556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d00045e0 con 0x7fb9d815d880 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.385+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d81544a0 con 0x7fb9d815d880 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.385+0000 7fb9d57fa700 1 -- 172.21.15.5:0/2573756556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0012450 con 0x7fb9d815d880 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.385+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d8154a60 con 0x7fb9d815d880 2024-05-08T23:26:01.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.386+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d80b6190 con 0x7fb9d815d880 2024-05-08T23:26:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.386+0000 7fb9d57fa700 1 -- 172.21.15.5:0/2573756556 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb9d00125b0 con 0x7fb9d815d880 2024-05-08T23:26:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.387+0000 7fb9d57fa700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb9b806ac90 0x7fb9b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.387+0000 7fb9d57fa700 1 -- 172.21.15.5:0/2573756556 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb9d008a890 con 0x7fb9d815d880 2024-05-08T23:26:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.387+0000 7fb9dc8c8700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb9b806ac90 0x7fb9b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.388+0000 7fb9dc8c8700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb9b806ac90 0x7fb9b806d150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8156500 tx=0x7fb9c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.392+0000 7fb9d57fa700 1 -- 172.21.15.5:0/2573756556 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb9d0055dd0 con 0x7fb9d815d880 2024-05-08T23:26:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.611+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9d8074bc0 con 0x7fb9d815d880 2024-05-08T23:26:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.611+0000 7fb9d57fa700 1 -- 172.21.15.5:0/2573756556 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9d00593f0 con 0x7fb9d815d880 2024-05-08T23:26:01.612 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:26:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb9b806ac90 msgr2=0x7fb9b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb9b806ac90 0x7fb9b806d150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8156500 tx=0x7fb9c8009040 comp rx=0 tx=0).stop 2024-05-08T23:26:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9d815d880 msgr2=0x7fb9d8143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9d815d880 0x7fb9d8143790 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000b370 tx=0x7fb9d0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 shutdown_connections 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb9b806ac90 0x7fb9b806d150 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9d815d880 0x7fb9d8143790 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9d806f8c0 0x7fb9d81f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 --2- 172.21.15.5:0/2573756556 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb9d80af990 0x7fb9d81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 >> 172.21.15.5:0/2573756556 conn(0x7fb9d80b5810 msgr2=0x7fb9d815f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 shutdown_connections 2024-05-08T23:26:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.614+0000 7fb9deb2c700 1 -- 172.21.15.5:0/2573756556 wait complete. 2024-05-08T23:26:01.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:26:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.764+0000 7fc135e56700 1 Processor -- start 2024-05-08T23:26:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.764+0000 7fc135e56700 1 -- start start 2024-05-08T23:26:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc135e56700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc1300d3f40 0x7fc130161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc135e56700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc13015db80 0x7fc1300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc135e56700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc130162450 con 0x7fc13015d7a0 2024-05-08T23:26:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc135e56700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1300cee00 con 0x7fc13015db80 2024-05-08T23:26:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc135e56700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1300b27e0 con 0x7fc1300d3f40 2024-05-08T23:26:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc12ffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc13015db80 0x7fc1300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc12effd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc1300d3f40 0x7fc130161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc12ffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc13015db80 0x7fc1300cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40792/0 (socket says 172.21.15.5:40792) 2024-05-08T23:26:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc12ffff700 1 -- 172.21.15.5:0/2431527368 learned_addr learned my addr 172.21.15.5:0/2431527368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc12ffff700 1 -- 172.21.15.5:0/2431527368 >> v2:172.21.15.49:3300/0 conn(0x7fc1300d3f40 msgr2=0x7fc130161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc12ffff700 1 --2- 172.21.15.5:0/2431527368 >> v2:172.21.15.49:3300/0 conn(0x7fc1300d3f40 0x7fc130161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc12ffff700 1 -- 172.21.15.5:0/2431527368 >> v1:172.21.15.49:6789/0 conn(0x7fc13015d7a0 legacy=0x7fc1300b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.765+0000 7fc12ffff700 1 -- 172.21.15.5:0/2431527368 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1300d4400 con 0x7fc13015db80 2024-05-08T23:26:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc12ffff700 1 --2- 172.21.15.5:0/2431527368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300cf290 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc1300d0de0 tx=0x7fc12800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df900f5a57833578 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc12e7fc700 1 -- 172.21.15.5:0/2431527368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12800e070 con 0x7fc13015db80 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc12e7fc700 1 -- 172.21.15.5:0/2431527368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12800b910 con 0x7fc13015db80 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc12e7fc700 1 -- 172.21.15.5:0/2431527368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12800bcc0 con 0x7fc13015db80 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc135e56700 1 -- 172.21.15.5:0/2431527368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 msgr2=0x7fc1300cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc135e56700 1 --2- 172.21.15.5:0/2431527368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300cf290 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc1300d0de0 tx=0x7fc12800b330 comp rx=0 tx=0).stop 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc135e56700 1 -- 172.21.15.5:0/2431527368 shutdown_connections 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc135e56700 1 --2- 172.21.15.5:0/2431527368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300cf290 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc135e56700 1 --2- 172.21.15.5:0/2431527368 >> v2:172.21.15.49:3300/0 conn(0x7fc1300d3f40 0x7fc130161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.766+0000 7fc135e56700 1 -- 172.21.15.5:0/2431527368 >> 172.21.15.5:0/2431527368 conn(0x7fc13008afe0 msgr2=0x7fc1300d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.767+0000 7fc135e56700 1 -- 172.21.15.5:0/2431527368 shutdown_connections 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.767+0000 7fc135e56700 1 -- 172.21.15.5:0/2431527368 wait complete. 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.768+0000 7fc135e56700 1 Processor -- start 2024-05-08T23:26:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.768+0000 7fc135e56700 1 -- start start 2024-05-08T23:26:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc135e56700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1300d3f40 0x7fc1300c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc135e56700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc13015d7a0 0x7fc130154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc135e56700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc135e56700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc130159e70 con 0x7fc13015db80 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc135e56700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1301591f0 con 0x7fc1300d3f40 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc135e56700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1301594d0 con 0x7fc13015d7a0 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40808/0 (socket says 172.21.15.5:40808) 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12ffff700 1 -- 172.21.15.5:0/4172864052 learned_addr learned my addr 172.21.15.5:0/4172864052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12ffff700 1 -- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc13015d7a0 msgr2=0x7fc130154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12effd700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc13015d7a0 0x7fc130154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12f7fe700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1300d3f40 0x7fc1300c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12ffff700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc13015d7a0 0x7fc130154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12ffff700 1 -- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1300d3f40 msgr2=0x7fc1300c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12ffff700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1300d3f40 0x7fc1300c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12ffff700 1 -- 172.21.15.5:0/4172864052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc128009b30 con 0x7fc13015db80 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12f7fe700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1300d3f40 0x7fc1300c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.769+0000 7fc12effd700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc13015d7a0 0x7fc130154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.770+0000 7fc12ffff700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300c7320 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc13008ba90 tx=0x7fc128004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.770+0000 7fc117fff700 1 -- 172.21.15.5:0/4172864052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280099b0 con 0x7fc13015db80 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.770+0000 7fc117fff700 1 -- 172.21.15.5:0/4172864052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280092e0 con 0x7fc13015db80 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.770+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1301597b0 con 0x7fc13015db80 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.770+0000 7fc117fff700 1 -- 172.21.15.5:0/4172864052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12801c640 con 0x7fc13015db80 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.770+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1300c7860 con 0x7fc13015db80 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.771+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc130159fb0 con 0x7fc13015db80 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.771+0000 7fc117fff700 1 -- 172.21.15.5:0/4172864052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc12801c7a0 con 0x7fc13015db80 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.772+0000 7fc117fff700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc11c06ac90 0x7fc11c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.772+0000 7fc117fff700 1 -- 172.21.15.5:0/4172864052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc12808aca0 con 0x7fc13015db80 2024-05-08T23:26:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.772+0000 7fc12f7fe700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc11c06ac90 0x7fc11c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.772+0000 7fc12f7fe700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc11c06ac90 0x7fc11c06d150 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc120000f60 tx=0x7fc120009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.777+0000 7fc117fff700 1 -- 172.21.15.5:0/4172864052 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc130159fb0 con 0x7fc13015db80 2024-05-08T23:26:01.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.995+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc130159fb0 con 0x7fc13015db80 2024-05-08T23:26:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.996+0000 7fc117fff700 1 -- 172.21.15.5:0/4172864052 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc130159fb0 con 0x7fc13015db80 2024-05-08T23:26:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:01.996 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:26:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-08T23:26:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-08T23:26:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:01.997 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.998+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc11c06ac90 msgr2=0x7fc11c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.998+0000 7fc135e56700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc11c06ac90 0x7fc11c06d150 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc120000f60 tx=0x7fc120009040 comp rx=0 tx=0).stop 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.998+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 msgr2=0x7fc1300c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.998+0000 7fc135e56700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300c7320 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc13008ba90 tx=0x7fc128004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.998+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 shutdown_connections 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.998+0000 7fc135e56700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc11c06ac90 0x7fc11c06d150 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.999+0000 7fc135e56700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc13015db80 0x7fc1300c7320 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.999+0000 7fc135e56700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1300d3f40 0x7fc1300c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.999+0000 7fc135e56700 1 --2- 172.21.15.5:0/4172864052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc13015d7a0 0x7fc130154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.999+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 >> 172.21.15.5:0/4172864052 conn(0x7fc13008afe0 msgr2=0x7fc13015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.999+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 shutdown_connections 2024-05-08T23:26:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:01.999+0000 7fc135e56700 1 -- 172.21.15.5:0/4172864052 wait complete. 2024-05-08T23:26:02.013 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:26:02.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:26:02.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.154+0000 7fa9179e7700 1 Processor -- start 2024-05-08T23:26:02.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.154+0000 7fa9179e7700 1 -- start start 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa9179e7700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9100d1910 0x7fa9100cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa9179e7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9100cf560 con 0x7fa91015b8c0 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa9179e7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9100ccff0 con 0x7fa91015b4e0 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa9179e7700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9100cefe0 con 0x7fa9100d1910 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa914f82700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa91015b8c0 0x7fa9100d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35148/0 (socket says 172.21.15.5:35148) 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa914f82700 1 -- 172.21.15.5:0/2043677830 learned_addr learned my addr 172.21.15.5:0/2043677830 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa915f84700 1 --2- 172.21.15.5:0/2043677830 >> v2:172.21.15.49:3300/0 conn(0x7fa9100d1910 0x7fa9100cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225387659 0 0) 0x7fa9100cf560 con 0x7fa91015b8c0 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8fc0035b0 con 0x7fa91015b8c0 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.155+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525651247 0 0) 0x7fa8fc0035b0 con 0x7fa91015b8c0 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 >> v2:172.21.15.49:3300/0 conn(0x7fa9100d1910 msgr2=0x7fa9100cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 --2- 172.21.15.5:0/2043677830 >> v2:172.21.15.49:3300/0 conn(0x7fa9100d1910 0x7fa9100cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 >> v1:172.21.15.49:6789/0 conn(0x7fa91015b4e0 legacy=0x7fa9100b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100cd2d0 con 0x7fa91015b8c0 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fa90c001670 con 0x7fa91015b8c0 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 >> v1:172.21.15.5:6789/0 conn(0x7fa91015b8c0 legacy=0x7fa9100d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc003a50 0x7fa8fc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8fc004850 0x7fa8fc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc005210 0x7fa8fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100cefe0 con 0x7fa8fc005210 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc00c110 con 0x7fa8fc004850 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc00c2d0 con 0x7fa8fc003a50 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915f84700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc005210 0x7fa8fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa914f82700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8fc004850 0x7fa8fc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915783700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc003a50 0x7fa8fc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915f84700 1 -- 172.21.15.5:0/2043677830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc003a50 msgr2=0x7fa8fc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915f84700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc003a50 0x7fa8fc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915f84700 1 -- 172.21.15.5:0/2043677830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8fc004850 msgr2=0x7fa8fc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915f84700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8fc004850 0x7fa8fc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915f84700 1 -- 172.21.15.5:0/2043677830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc0035b0 con 0x7fa8fc005210 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915f84700 1 -- 172.21.15.5:0/2043677830 reap_dead start 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa915f84700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc005210 0x7fa8fc00ba10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa908009fc0 tx=0x7fa8fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b198846661dd554 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908003a20 con 0x7fa8fc005210 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800f040 con 0x7fa8fc005210 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.156+0000 7fa903fff700 1 -- 172.21.15.5:0/2043677830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908003de0 con 0x7fa8fc005210 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.157+0000 7fa9179e7700 1 -- 172.21.15.5:0/2043677830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc005210 msgr2=0x7fa8fc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.157+0000 7fa9179e7700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc005210 0x7fa8fc00ba10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa908009fc0 tx=0x7fa8fc005880 comp rx=0 tx=0).stop 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.157+0000 7fa9179e7700 1 -- 172.21.15.5:0/2043677830 shutdown_connections 2024-05-08T23:26:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.157+0000 7fa9179e7700 1 --2- 172.21.15.5:0/2043677830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc005210 0x7fa8fc00ba10 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.157+0000 7fa9179e7700 1 -- 172.21.15.5:0/2043677830 >> 172.21.15.5:0/2043677830 conn(0x7fa91008afe0 msgr2=0x7fa91008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.157+0000 7fa9179e7700 1 -- 172.21.15.5:0/2043677830 shutdown_connections 2024-05-08T23:26:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.157+0000 7fa9179e7700 1 -- 172.21.15.5:0/2043677830 wait complete. 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.158+0000 7fa9179e7700 1 Processor -- start 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa9179e7700 1 -- start start 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa9179e7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc005210 0x7fa9101f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa9179e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91015e9d0 0x7fa9100c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa9179e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9100b6620 0x7fa9100b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa9179e7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c5b40 con 0x7fa91015e9d0 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa9179e7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100b6ff0 con 0x7fa9100b6620 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa9179e7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91014c2c0 con 0x7fa8fc005210 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa914f82700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91015e9d0 0x7fa9100c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa914f82700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91015e9d0 0x7fa9100c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40832/0 (socket says 172.21.15.5:40832) 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.159+0000 7fa914f82700 1 -- 172.21.15.5:0/4123654264 learned_addr learned my addr 172.21.15.5:0/4123654264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa915783700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc005210 0x7fa9101f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa915f84700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9100b6620 0x7fa9100b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa914f82700 1 -- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc005210 msgr2=0x7fa9101f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa914f82700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc005210 0x7fa9101f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa914f82700 1 -- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9100b6620 msgr2=0x7fa9100b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa914f82700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9100b6620 0x7fa9100b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa914f82700 1 -- 172.21.15.5:0/4123654264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100c4c70 con 0x7fa91015e9d0 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa915f84700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9100b6620 0x7fa9100b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa915783700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc005210 0x7fa9101f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa914f82700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91015e9d0 0x7fa9100c62f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa90c001d60 tx=0x7fa90c007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa901ffb700 1 -- 172.21.15.5:0/4123654264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00f5e0 con 0x7fa91015e9d0 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa901ffb700 1 -- 172.21.15.5:0/4123654264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00f780 con 0x7fa91015e9d0 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9101f4750 con 0x7fa91015e9d0 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa901ffb700 1 -- 172.21.15.5:0/4123654264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00fb30 con 0x7fa91015e9d0 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.160+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9100c4f70 con 0x7fa91015e9d0 2024-05-08T23:26:02.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.162+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9100c5c80 con 0x7fa91015e9d0 2024-05-08T23:26:02.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.162+0000 7fa901ffb700 1 -- 172.21.15.5:0/4123654264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa90c00fc90 con 0x7fa91015e9d0 2024-05-08T23:26:02.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.162+0000 7fa901ffb700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa8f406ac90 0x7fa8f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:02.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.163+0000 7fa901ffb700 1 -- 172.21.15.5:0/4123654264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa90c088f30 con 0x7fa91015e9d0 2024-05-08T23:26:02.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.163+0000 7fa915783700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa8f406ac90 0x7fa8f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:02.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.163+0000 7fa915783700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa8f406ac90 0x7fa8f406d150 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa904000c00 tx=0x7fa904009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:02.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.167+0000 7fa901ffb700 1 -- 172.21.15.5:0/4123654264 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa9100c5c80 con 0x7fa91015e9d0 2024-05-08T23:26:02.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.385+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa9100c5c80 con 0x7fa91015e9d0 2024-05-08T23:26:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.894+0000 7fa901ffb700 1 -- 172.21.15.5:0/4123654264 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa9100c5c80 con 0x7fa91015e9d0 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa8f406ac90 msgr2=0x7fa8f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa8f406ac90 0x7fa8f406d150 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa904000c00 tx=0x7fa904009040 comp rx=0 tx=0).stop 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91015e9d0 msgr2=0x7fa9100c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91015e9d0 0x7fa9100c62f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa90c001d60 tx=0x7fa90c007570 comp rx=0 tx=0).stop 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 shutdown_connections 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa8f406ac90 0x7fa8f406d150 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9100b6620 0x7fa9100b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa8fc005210 0x7fa9101f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 --2- 172.21.15.5:0/4123654264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91015e9d0 0x7fa9100c62f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:02.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 >> 172.21.15.5:0/4123654264 conn(0x7fa91008afe0 msgr2=0x7fa91015a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:02.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 shutdown_connections 2024-05-08T23:26:02.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:02.899+0000 7fa9179e7700 1 -- 172.21.15.5:0/4123654264 wait complete. 2024-05-08T23:26:02.900 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-08T23:26:02.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:26:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.064+0000 7fe8c94ac700 1 Processor -- start 2024-05-08T23:26:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.064+0000 7fe8c94ac700 1 -- start start 2024-05-08T23:26:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.064+0000 7fe8c94ac700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.064+0000 7fe8c94ac700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8c40727b0 0x7fe8c40c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.064+0000 7fe8c94ac700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c40cd7c0 con 0x7fe8c4072260 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.064+0000 7fe8c94ac700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c40cd1b0 con 0x7fe8c40727b0 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.064+0000 7fe8c94ac700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c415aba0 con 0x7fe8c415a7c0 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c2ffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c27fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8c40727b0 0x7fe8c40c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c2ffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:36890/0 (socket says 172.21.15.5:36890) 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c2ffd700 1 -- 172.21.15.5:0/3279810889 learned_addr learned my addr 172.21.15.5:0/3279810889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c2ffd700 1 -- 172.21.15.5:0/3279810889 >> v2:172.21.15.49:3300/0 conn(0x7fe8c40727b0 msgr2=0x7fe8c40c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c2ffd700 1 --2- 172.21.15.5:0/3279810889 >> v2:172.21.15.49:3300/0 conn(0x7fe8c40727b0 0x7fe8c40c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c2ffd700 1 -- 172.21.15.5:0/3279810889 >> v1:172.21.15.49:6789/0 conn(0x7fe8c4072260 legacy=0x7fe8c40b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c2ffd700 1 -- 172.21.15.5:0/3279810889 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40b3e70 con 0x7fe8c415a7c0 2024-05-08T23:26:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.065+0000 7fe8c27fc700 1 --2- 172.21.15.5:0/3279810889 >> v2:172.21.15.49:3300/0 conn(0x7fe8c40727b0 0x7fe8c40c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c2ffd700 1 --2- 172.21.15.5:0/3279810889 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40098f0 tx=0x7fe8b4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8b97df23136138 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c1ffb700 1 -- 172.21.15.5:0/3279810889 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4011830 con 0x7fe8c415a7c0 2024-05-08T23:26:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c1ffb700 1 -- 172.21.15.5:0/3279810889 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b4011990 con 0x7fe8c415a7c0 2024-05-08T23:26:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c1ffb700 1 -- 172.21.15.5:0/3279810889 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4011d20 con 0x7fe8c415a7c0 2024-05-08T23:26:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c94ac700 1 -- 172.21.15.5:0/3279810889 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c415a7c0 msgr2=0x7fe8c40cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/3279810889 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40098f0 tx=0x7fe8b4009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c94ac700 1 -- 172.21.15.5:0/3279810889 shutdown_connections 2024-05-08T23:26:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/3279810889 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/3279810889 >> v2:172.21.15.49:3300/0 conn(0x7fe8c40727b0 0x7fe8c40c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c94ac700 1 -- 172.21.15.5:0/3279810889 >> 172.21.15.5:0/3279810889 conn(0x7fe8c406a960 msgr2=0x7fe8c406ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:03.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c94ac700 1 -- 172.21.15.5:0/3279810889 shutdown_connections 2024-05-08T23:26:03.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.066+0000 7fe8c94ac700 1 -- 172.21.15.5:0/3279810889 wait complete. 2024-05-08T23:26:03.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.068+0000 7fe8c94ac700 1 Processor -- start 2024-05-08T23:26:03.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.068+0000 7fe8c94ac700 1 -- start start 2024-05-08T23:26:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c94ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4072260 0x7fe8c4157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c94ac700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c40727b0 0x7fe8c4154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c94ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c94ac700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c4157ed0 con 0x7fe8c4072260 2024-05-08T23:26:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c94ac700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c4155380 con 0x7fe8c415a7c0 2024-05-08T23:26:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c94ac700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c415b820 con 0x7fe8c40727b0 2024-05-08T23:26:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4072260 0x7fe8c4157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4072260 0x7fe8c4157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40842/0 (socket says 172.21.15.5:40842) 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 -- 172.21.15.5:0/975729597 learned_addr learned my addr 172.21.15.5:0/975729597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c27fc700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c40727b0 0x7fe8c4154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c37fe700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 -- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c40727b0 msgr2=0x7fe8c4154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c40727b0 0x7fe8c4154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 -- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c415a7c0 msgr2=0x7fe8c40cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 -- 172.21.15.5:0/975729597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80094a0 con 0x7fe8c4072260 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.069+0000 7fe8c2ffd700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4072260 0x7fe8c4157990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4014040 tx=0x7fe8b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.070+0000 7fe8ab7fe700 1 -- 172.21.15.5:0/975729597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4004300 con 0x7fe8c4072260 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.070+0000 7fe8ab7fe700 1 -- 172.21.15.5:0/975729597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b4004480 con 0x7fe8c4072260 2024-05-08T23:26:03.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.070+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b4009550 con 0x7fe8c4072260 2024-05-08T23:26:03.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.070+0000 7fe8ab7fe700 1 -- 172.21.15.5:0/975729597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40045f0 con 0x7fe8c4072260 2024-05-08T23:26:03.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.070+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c40ad210 con 0x7fe8c4072260 2024-05-08T23:26:03.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.071+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c4074bc0 con 0x7fe8c4072260 2024-05-08T23:26:03.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.071+0000 7fe8ab7fe700 1 -- 172.21.15.5:0/975729597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe8b4023070 con 0x7fe8c4072260 2024-05-08T23:26:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.072+0000 7fe8ab7fe700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe8b006ac90 0x7fe8b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.072+0000 7fe8ab7fe700 1 -- 172.21.15.5:0/975729597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe8b408be70 con 0x7fe8c4072260 2024-05-08T23:26:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.072+0000 7fe8c27fc700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe8b006ac90 0x7fe8b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.073+0000 7fe8c27fc700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe8b006ac90 0x7fe8b006d150 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4157940 tx=0x7fe8b8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.077+0000 7fe8ab7fe700 1 -- 172.21.15.5:0/975729597 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe8b40572c0 con 0x7fe8c4072260 2024-05-08T23:26:03.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.296+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe8c40ced00 con 0x7fe8c4072260 2024-05-08T23:26:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.897+0000 7fe8ab7fe700 1 -- 172.21.15.5:0/975729597 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe8b40150d0 con 0x7fe8c4072260 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.903+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe8b006ac90 msgr2=0x7fe8b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.903+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe8b006ac90 0x7fe8b006d150 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4157940 tx=0x7fe8b8009210 comp rx=0 tx=0).stop 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.903+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4072260 msgr2=0x7fe8c4157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.903+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4072260 0x7fe8c4157990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4014040 tx=0x7fe8b4004750 comp rx=0 tx=0).stop 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.904+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 shutdown_connections 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.904+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe8b006ac90 0x7fe8b006d150 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4157940 tx=0x7fe8b8009210 comp rx=0 tx=0).stop 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.904+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c415a7c0 0x7fe8c40cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.904+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4072260 0x7fe8c4157990 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.904+0000 7fe8c94ac700 1 --2- 172.21.15.5:0/975729597 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8c40727b0 0x7fe8c4154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.904+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 >> 172.21.15.5:0/975729597 conn(0x7fe8c406a960 msgr2=0x7fe8c406ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.904+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 shutdown_connections 2024-05-08T23:26:03.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:03.904+0000 7fe8c94ac700 1 -- 172.21.15.5:0/975729597 wait complete. 2024-05-08T23:26:03.915 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-08T23:26:03.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:26:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.071+0000 7f13e7999700 1 Processor -- start 2024-05-08T23:26:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.072+0000 7f13e7999700 1 -- start start 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e7999700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f13e015b8c0 0x7f13e00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e7999700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e00cf560 con 0x7f13e00d1740 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e7999700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e00ccff0 con 0x7f13e015b4e0 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e7999700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e00cefe0 con 0x7f13e015b8c0 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e5f36700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f13e00d1740 0x7f13e00cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35164/0 (socket says 172.21.15.5:35164) 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e5f36700 1 -- 172.21.15.5:0/752375374 learned_addr learned my addr 172.21.15.5:0/752375374 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e4f34700 1 --2- 172.21.15.5:0/752375374 >> v2:172.21.15.49:3300/0 conn(0x7f13e015b8c0 0x7f13e00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13d7fff700 1 -- 172.21.15.5:0/752375374 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593090831 0 0) 0x7f13e00cf560 con 0x7f13e00d1740 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13d7fff700 1 -- 172.21.15.5:0/752375374 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13cc0035b0 con 0x7f13e00d1740 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e4f34700 1 -- 172.21.15.5:0/752375374 >> v1:172.21.15.49:6789/0 conn(0x7f13e015b4e0 legacy=0x7f13e00b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13d7fff700 1 -- 172.21.15.5:0/752375374 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751994398 0 0) 0x7f13e00ccff0 con 0x7f13e015b4e0 2024-05-08T23:26:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e4f34700 1 -- 172.21.15.5:0/752375374 >> v1:172.21.15.5:6789/0 conn(0x7f13e00d1740 legacy=0x7f13e00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.073+0000 7f13e4f34700 1 -- 172.21.15.5:0/752375374 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e00cd2d0 con 0x7f13e015b8c0 2024-05-08T23:26:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13e4f34700 1 --2- 172.21.15.5:0/752375374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 0x7f13e00d1200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009840 tx=0x7f13d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cea8ee142ebc588 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13d7fff700 1 -- 172.21.15.5:0/752375374 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0011830 con 0x7f13e015b8c0 2024-05-08T23:26:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13d7fff700 1 -- 172.21.15.5:0/752375374 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d0011990 con 0x7f13e015b8c0 2024-05-08T23:26:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13d7fff700 1 -- 172.21.15.5:0/752375374 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0011d20 con 0x7f13e015b8c0 2024-05-08T23:26:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13e7999700 1 -- 172.21.15.5:0/752375374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 msgr2=0x7f13e00d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13e7999700 1 --2- 172.21.15.5:0/752375374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 0x7f13e00d1200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009840 tx=0x7f13d0009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13e7999700 1 -- 172.21.15.5:0/752375374 shutdown_connections 2024-05-08T23:26:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13e7999700 1 --2- 172.21.15.5:0/752375374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 0x7f13e00d1200 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.074+0000 7f13e7999700 1 -- 172.21.15.5:0/752375374 >> 172.21.15.5:0/752375374 conn(0x7f13e008afe0 msgr2=0x7f13e008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.075+0000 7f13e7999700 1 -- 172.21.15.5:0/752375374 shutdown_connections 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.075+0000 7f13e7999700 1 -- 172.21.15.5:0/752375374 wait complete. 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.076+0000 7f13e7999700 1 Processor -- start 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.076+0000 7f13e7999700 1 -- start start 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e7999700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13e00d1740 0x7f13e00ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e7999700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13e015b4e0 0x7f13e015d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e7999700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 0x7f13e014b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e7999700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e00b5b20 con 0x7f13e00d1740 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e7999700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e006bb80 con 0x7f13e015b8c0 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e7999700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e015dcc0 con 0x7f13e015b4e0 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13e00d1740 0x7f13e00ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13e00d1740 0x7f13e00ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40846/0 (socket says 172.21.15.5:40846) 2024-05-08T23:26:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 -- 172.21.15.5:0/2849335863 learned_addr learned my addr 172.21.15.5:0/2849335863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e4f34700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13e015b4e0 0x7f13e015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5f36700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 0x7f13e014b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 -- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13e015b4e0 msgr2=0x7f13e015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13e015b4e0 0x7f13e015d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 -- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 msgr2=0x7f13e014b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 0x7f13e014b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 -- 172.21.15.5:0/2849335863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d00094a0 con 0x7f13e00d1740 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e5735700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13e00d1740 0x7f13e00ca030 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0011e0 tx=0x7f13dc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13d5ffb700 1 -- 172.21.15.5:0/2849335863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0054e0 con 0x7f13e00d1740 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13d5ffb700 1 -- 172.21.15.5:0/2849335863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc003680 con 0x7f13e00d1740 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.077+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e00ca6c0 con 0x7f13e00d1740 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.078+0000 7f13d5ffb700 1 -- 172.21.15.5:0/2849335863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc003a30 con 0x7f13e00d1740 2024-05-08T23:26:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.078+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e00b57b0 con 0x7f13e00d1740 2024-05-08T23:26:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.079+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e00b5610 con 0x7f13e00d1740 2024-05-08T23:26:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.079+0000 7f13d5ffb700 1 -- 172.21.15.5:0/2849335863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f13dc003b90 con 0x7f13e00d1740 2024-05-08T23:26:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.080+0000 7f13d5ffb700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f13c406ac90 0x7f13c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.080+0000 7f13d5ffb700 1 -- 172.21.15.5:0/2849335863 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f13dc0887a0 con 0x7f13e00d1740 2024-05-08T23:26:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.080+0000 7f13e4f34700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f13c406ac90 0x7f13c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.081+0000 7f13e4f34700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f13c406ac90 0x7f13c406d150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009fd0 tx=0x7f13d001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.085+0000 7f13d5ffb700 1 -- 172.21.15.5:0/2849335863 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f13e00b5610 con 0x7f13e00d1740 2024-05-08T23:26:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.357+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f13e00b5610 con 0x7f13e00d1740 2024-05-08T23:26:04.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.363+0000 7f13d5ffb700 1 -- 172.21.15.5:0/2849335863 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7f13e00b5610 con 0x7f13e00d1740 2024-05-08T23:26:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.368+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f13c406ac90 msgr2=0x7f13c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.368+0000 7f13e7999700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f13c406ac90 0x7f13c406d150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009fd0 tx=0x7f13d001c040 comp rx=0 tx=0).stop 2024-05-08T23:26:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.368+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13e00d1740 msgr2=0x7f13e00ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.368+0000 7f13e7999700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13e00d1740 0x7f13e00ca030 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0011e0 tx=0x7f13dc00cae0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.369+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 shutdown_connections 2024-05-08T23:26:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.370+0000 7f13e7999700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f13c406ac90 0x7f13c406d150 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.370+0000 7f13e7999700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e015b8c0 0x7f13e014b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.370+0000 7f13e7999700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13e00d1740 0x7f13e00ca030 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.370+0000 7f13e7999700 1 --2- 172.21.15.5:0/2849335863 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13e015b4e0 0x7f13e015d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.370+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 >> 172.21.15.5:0/2849335863 conn(0x7f13e008afe0 msgr2=0x7f13e00aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.370+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 shutdown_connections 2024-05-08T23:26:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.370+0000 7f13e7999700 1 -- 172.21.15.5:0/2849335863 wait complete. 2024-05-08T23:26:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 10 and data pool 11 2024-05-08T23:26:04.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.531+0000 7f20dbaaf700 1 Processor -- start 2024-05-08T23:26:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.531+0000 7f20dbaaf700 1 -- start start 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20dbaaf700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d40af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20dbaaf700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f20d406a960 0x7f20d415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20dbaaf700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40cd980 con 0x7f20d40d0eb0 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20dbaaf700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d40ccdb0 con 0x7f20d406a960 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20dbaaf700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40cd050 con 0x7f20d40cf840 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20d904a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f20d406a960 0x7f20d415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20da04c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f20d40d0eb0 0x7f20d406dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35172/0 (socket says 172.21.15.5:35172) 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20da04c700 1 -- 172.21.15.5:0/1478073423 learned_addr learned my addr 172.21.15.5:0/1478073423 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20d984b700 1 --2- 172.21.15.5:0/1478073423 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d40af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20d8849700 1 -- 172.21.15.5:0/1478073423 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432272434 0 0) 0x7f20d40cd980 con 0x7f20d40d0eb0 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20d8849700 1 -- 172.21.15.5:0/1478073423 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c00035b0 con 0x7f20d40d0eb0 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20d904a700 1 -- 172.21.15.5:0/1478073423 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 msgr2=0x7f20d40af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20d904a700 1 --2- 172.21.15.5:0/1478073423 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d40af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20d904a700 1 -- 172.21.15.5:0/1478073423 >> v1:172.21.15.5:6789/0 conn(0x7f20d40d0eb0 legacy=0x7f20d406dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.532+0000 7f20d904a700 1 -- 172.21.15.5:0/1478073423 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d40cd330 con 0x7f20d406a960 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20d904a700 1 --2- 172.21.15.5:0/1478073423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d415d4f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71843ddf7c194e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20d8849700 1 -- 172.21.15.5:0/1478073423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4011830 con 0x7f20d406a960 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20d8849700 1 -- 172.21.15.5:0/1478073423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c4011990 con 0x7f20d406a960 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20d8849700 1 -- 172.21.15.5:0/1478073423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4011d20 con 0x7f20d406a960 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1478073423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 msgr2=0x7f20d415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1478073423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d415d4f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c4009b80 comp rx=0 tx=0).stop 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1478073423 shutdown_connections 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1478073423 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d40af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1478073423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d415d4f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.533+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1478073423 >> 172.21.15.5:0/1478073423 conn(0x7f20d40d0320 msgr2=0x7f20d40d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.534+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1478073423 shutdown_connections 2024-05-08T23:26:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.534+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1478073423 wait complete. 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.535+0000 7f20dbaaf700 1 Processor -- start 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.535+0000 7f20dbaaf700 1 -- start start 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20dbaaf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d40b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20dbaaf700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d41f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20dbaaf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40d0eb0 0x7f20d41f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20dbaaf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40b2350 con 0x7f20d406a960 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20dbaaf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40b44a0 con 0x7f20d40d0eb0 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20dbaaf700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40b3890 con 0x7f20d40cf840 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d40b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d40b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40850/0 (socket says 172.21.15.5:40850) 2024-05-08T23:26:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 -- 172.21.15.5:0/1556052441 learned_addr learned my addr 172.21.15.5:0/1556052441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20da04c700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40d0eb0 0x7f20d41f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d904a700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d41f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 -- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 msgr2=0x7f20d41f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d41f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 -- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40d0eb0 msgr2=0x7f20d41f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40d0eb0 0x7f20d41f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 -- 172.21.15.5:0/1556052441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40094a0 con 0x7f20d406a960 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20da04c700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40d0eb0 0x7f20d41f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d904a700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d41f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.536+0000 7f20d984b700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d40b2f20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f20d000e810 tx=0x7f20d000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.537+0000 7f20ca7fc700 1 -- 172.21.15.5:0/1556052441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0013610 con 0x7f20d406a960 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.537+0000 7f20ca7fc700 1 -- 172.21.15.5:0/1556052441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d0013770 con 0x7f20d406a960 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.537+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d40b3b70 con 0x7f20d406a960 2024-05-08T23:26:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.537+0000 7f20ca7fc700 1 -- 172.21.15.5:0/1556052441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0013b20 con 0x7f20d406a960 2024-05-08T23:26:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.537+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d40b4130 con 0x7f20d406a960 2024-05-08T23:26:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.538+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d406c760 con 0x7f20d406a960 2024-05-08T23:26:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.538+0000 7f20ca7fc700 1 -- 172.21.15.5:0/1556052441 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f20d0013c80 con 0x7f20d406a960 2024-05-08T23:26:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.539+0000 7f20ca7fc700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f20b806acc0 0x7f20b806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.539+0000 7f20ca7fc700 1 -- 172.21.15.5:0/1556052441 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20d0088fc0 con 0x7f20d406a960 2024-05-08T23:26:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.539+0000 7f20d904a700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f20b806acc0 0x7f20b806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.539+0000 7f20d904a700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f20b806acc0 0x7f20b806d180 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.544+0000 7f20ca7fc700 1 -- 172.21.15.5:0/1556052441 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f20d00541b0 con 0x7f20d406a960 2024-05-08T23:26:04.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.761+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20d4068000 con 0x7f20d406a960 2024-05-08T23:26:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.762+0000 7f20ca7fc700 1 -- 172.21.15.5:0/1556052441 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16017 (secure 0 0 0) 0x7f20d00577d0 con 0x7f20d406a960 2024-05-08T23:26:04.762 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:04.762 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:04.359316+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:02.418282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:03.329387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f20b806acc0 msgr2=0x7f20b806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f20b806acc0 0x7f20b806d180 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c4008030 comp rx=0 tx=0).stop 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 msgr2=0x7f20d40b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d40b2f20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f20d000e810 tx=0x7f20d000ebd0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 shutdown_connections 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f20b806acc0 0x7f20b806d180 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40d0eb0 0x7f20d41f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d406a960 0x7f20d40b2f20 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.764+0000 7f20dbaaf700 1 --2- 172.21.15.5:0/1556052441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20d40cf840 0x7f20d41f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.765+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 >> 172.21.15.5:0/1556052441 conn(0x7f20d40d0320 msgr2=0x7f20d40aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.765+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 shutdown_connections 2024-05-08T23:26:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.765+0000 7f20dbaaf700 1 -- 172.21.15.5:0/1556052441 wait complete. 2024-05-08T23:26:04.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:04.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.916+0000 7f207e4cb700 1 Processor -- start 2024-05-08T23:26:04.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.916+0000 7f207e4cb700 1 -- start start 2024-05-08T23:26:04.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f207e4cb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f2078099f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f207e4cb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207809aca0 con 0x7f2078099a50 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f207e4cb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20780cd9e0 con 0x7f207815b810 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f207e4cb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207809a440 con 0x7f20780cd4e0 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f20777fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2078099a50 0x7f20780ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35178/0 (socket says 172.21.15.5:35178) 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f20777fe700 1 -- 172.21.15.5:0/4111452800 learned_addr learned my addr 172.21.15.5:0/4111452800 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f207ca68700 1 --2- 172.21.15.5:0/4111452800 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f2078099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f2076ffd700 1 -- 172.21.15.5:0/4111452800 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046422920 0 0) 0x7f207809aca0 con 0x7f2078099a50 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.917+0000 7f2076ffd700 1 -- 172.21.15.5:0/4111452800 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20600035b0 con 0x7f2078099a50 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f2076ffd700 1 -- 172.21.15.5:0/4111452800 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495423389 0 0) 0x7f20780cd9e0 con 0x7f207815b810 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f2076ffd700 1 -- 172.21.15.5:0/4111452800 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f207809aca0 con 0x7f207815b810 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f207ca68700 1 -- 172.21.15.5:0/4111452800 >> v1:172.21.15.49:6789/0 conn(0x7f207815b810 legacy=0x7f20780cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f207ca68700 1 -- 172.21.15.5:0/4111452800 >> v1:172.21.15.5:6789/0 conn(0x7f2078099a50 legacy=0x7f20780ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f207ca68700 1 -- 172.21.15.5:0/4111452800 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780cf220 con 0x7f20780cd4e0 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f207ca68700 1 --2- 172.21.15.5:0/4111452800 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f2078099f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2070009dc0 tx=0x7f207000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=505ca354225c8b4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f2076ffd700 1 -- 172.21.15.5:0/4111452800 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000e070 con 0x7f20780cd4e0 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f2076ffd700 1 -- 172.21.15.5:0/4111452800 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207000bb20 con 0x7f20780cd4e0 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.918+0000 7f2076ffd700 1 -- 172.21.15.5:0/4111452800 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070012430 con 0x7f20780cd4e0 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.919+0000 7f207e4cb700 1 -- 172.21.15.5:0/4111452800 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 msgr2=0x7f2078099f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.919+0000 7f207e4cb700 1 --2- 172.21.15.5:0/4111452800 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f2078099f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2070009dc0 tx=0x7f207000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.919+0000 7f207e4cb700 1 -- 172.21.15.5:0/4111452800 shutdown_connections 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.919+0000 7f207e4cb700 1 --2- 172.21.15.5:0/4111452800 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f2078099f00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.919+0000 7f207e4cb700 1 -- 172.21.15.5:0/4111452800 >> 172.21.15.5:0/4111452800 conn(0x7f20780b3010 msgr2=0x7f20780b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.919+0000 7f207e4cb700 1 -- 172.21.15.5:0/4111452800 shutdown_connections 2024-05-08T23:26:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.919+0000 7f207e4cb700 1 -- 172.21.15.5:0/4111452800 wait complete. 2024-05-08T23:26:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.920+0000 7f207e4cb700 1 Processor -- start 2024-05-08T23:26:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.920+0000 7f207e4cb700 1 -- start start 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207e4cb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2078099a50 0x7f2078158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207e4cb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f207806c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207e4cb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207815b810 0x7f20780b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207e4cb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078159390 con 0x7f207815b810 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207e4cb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207806d0f0 con 0x7f2078099a50 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207e4cb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207806d340 con 0x7f20780cd4e0 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207ca68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207815b810 0x7f20780b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207ca68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207815b810 0x7f20780b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40864/0 (socket says 172.21.15.5:40864) 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.921+0000 7f207ca68700 1 -- 172.21.15.5:0/2839001228 learned_addr learned my addr 172.21.15.5:0/2839001228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f20777fe700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f207806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f2077fff700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2078099a50 0x7f2078158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f207ca68700 1 -- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 msgr2=0x7f207806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f207ca68700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f207806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f207ca68700 1 -- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2078099a50 msgr2=0x7f2078158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f207ca68700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2078099a50 0x7f2078158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f207ca68700 1 -- 172.21.15.5:0/2839001228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070009a70 con 0x7f207815b810 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f207ca68700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207815b810 0x7f20780b4c20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f207000b370 tx=0x7f2070004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f2074ff9700 1 -- 172.21.15.5:0/2839001228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207001b810 con 0x7f207815b810 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.922+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207806d620 con 0x7f207815b810 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.923+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20780b4420 con 0x7f207815b810 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.923+0000 7f2074ff9700 1 -- 172.21.15.5:0/2839001228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2070020070 con 0x7f207815b810 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.923+0000 7f2074ff9700 1 -- 172.21.15.5:0/2839001228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207001bb70 con 0x7f207815b810 2024-05-08T23:26:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.923+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078068000 con 0x7f207815b810 2024-05-08T23:26:04.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.924+0000 7f2074ff9700 1 -- 172.21.15.5:0/2839001228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f207001bcd0 con 0x7f207815b810 2024-05-08T23:26:04.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.924+0000 7f2074ff9700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f206406ac90 0x7f206406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:04.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.924+0000 7f2074ff9700 1 -- 172.21.15.5:0/2839001228 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2070022070 con 0x7f207815b810 2024-05-08T23:26:04.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.925+0000 7f2077fff700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f206406ac90 0x7f206406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:04.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.925+0000 7f2077fff700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f206406ac90 0x7f206406d150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2068001220 tx=0x7f2068009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:04.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:04.929+0000 7f2074ff9700 1 -- 172.21.15.5:0/2839001228 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2070056670 con 0x7f207815b810 2024-05-08T23:26:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.149+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20780b3c70 con 0x7f207815b810 2024-05-08T23:26:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.150+0000 7f2074ff9700 1 -- 172.21.15.5:0/2839001228 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16017 (secure 0 0 0) 0x7f2070059c90 con 0x7f207815b810 2024-05-08T23:26:05.150 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:05.150 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:04.359316+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:02.418282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:03.329387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f206406ac90 msgr2=0x7f206406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f206406ac90 0x7f206406d150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2068001220 tx=0x7f2068009040 comp rx=0 tx=0).stop 2024-05-08T23:26:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207815b810 msgr2=0x7f20780b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207815b810 0x7f20780b4c20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f207000b370 tx=0x7f2070004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 shutdown_connections 2024-05-08T23:26:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f206406ac90 0x7f206406d150 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207815b810 0x7f20780b4c20 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2078099a50 0x7f2078158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 --2- 172.21.15.5:0/2839001228 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f20780cd4e0 0x7f207806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 >> 172.21.15.5:0/2839001228 conn(0x7f20780b3010 msgr2=0x7f20780cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 shutdown_connections 2024-05-08T23:26:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.153+0000 7f207e4cb700 1 -- 172.21.15.5:0/2839001228 wait complete. 2024-05-08T23:26:05.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:26:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.307+0000 7f9c3d421700 1 Processor -- start 2024-05-08T23:26:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.307+0000 7f9c3d421700 1 -- start start 2024-05-08T23:26:05.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c3d421700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c3815dc50 0x7f9c380d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:05.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c3d421700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9c380d41b0 0x7f9c380ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:05.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c3d421700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3815e140 con 0x7f9c380cff90 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c3d421700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3815eae0 con 0x7f9c3815dc50 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c3d421700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3815e420 con 0x7f9c380d41b0 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c367fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c3815dc50 0x7f9c380d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c367fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c3815dc50 0x7f9c380d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40874/0 (socket says 172.21.15.5:40874) 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c367fc700 1 -- 172.21.15.5:0/1196970863 learned_addr learned my addr 172.21.15.5:0/1196970863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c377fe700 1 --2- 172.21.15.5:0/1196970863 >> v2:172.21.15.49:3300/0 conn(0x7f9c380d41b0 0x7f9c380ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c367fc700 1 -- 172.21.15.5:0/1196970863 >> v2:172.21.15.49:3300/0 conn(0x7f9c380d41b0 msgr2=0x7f9c380ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c367fc700 1 --2- 172.21.15.5:0/1196970863 >> v2:172.21.15.49:3300/0 conn(0x7f9c380d41b0 0x7f9c380ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c367fc700 1 -- 172.21.15.5:0/1196970863 >> v1:172.21.15.49:6789/0 conn(0x7f9c380cff90 legacy=0x7f9c380d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c367fc700 1 -- 172.21.15.5:0/1196970863 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3815e700 con 0x7f9c3815dc50 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.308+0000 7f9c367fc700 1 --2- 172.21.15.5:0/1196970863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3815dc50 0x7f9c380d11e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0098f0 tx=0x7f9c2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d37dc3867d8ee3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c35ffb700 1 -- 172.21.15.5:0/1196970863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c004230 con 0x7f9c3815dc50 2024-05-08T23:26:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c35ffb700 1 -- 172.21.15.5:0/1196970863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c004390 con 0x7f9c3815dc50 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c35ffb700 1 -- 172.21.15.5:0/1196970863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0036a0 con 0x7f9c3815dc50 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c3d421700 1 -- 172.21.15.5:0/1196970863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3815dc50 msgr2=0x7f9c380d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c3d421700 1 --2- 172.21.15.5:0/1196970863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3815dc50 0x7f9c380d11e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0098f0 tx=0x7f9c2c009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c3d421700 1 -- 172.21.15.5:0/1196970863 shutdown_connections 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c3d421700 1 --2- 172.21.15.5:0/1196970863 >> v2:172.21.15.49:3300/0 conn(0x7f9c380d41b0 0x7f9c380ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c3d421700 1 --2- 172.21.15.5:0/1196970863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3815dc50 0x7f9c380d11e0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c3d421700 1 -- 172.21.15.5:0/1196970863 >> 172.21.15.5:0/1196970863 conn(0x7f9c3808afe0 msgr2=0x7f9c380d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c3d421700 1 -- 172.21.15.5:0/1196970863 shutdown_connections 2024-05-08T23:26:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.309+0000 7f9c3d421700 1 -- 172.21.15.5:0/1196970863 wait complete. 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.311+0000 7f9c3d421700 1 Processor -- start 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.311+0000 7f9c3d421700 1 -- start start 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c3d421700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c380cff90 0x7f9c380c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c3d421700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c380d41b0 0x7f9c381e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c3d421700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c3815dc50 0x7f9c380c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c3d421700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c381e85e0 con 0x7f9c380d41b0 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c3d421700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c381e7730 con 0x7f9c380cff90 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c3d421700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c381e7960 con 0x7f9c3815dc50 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c380d41b0 0x7f9c381e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c380d41b0 0x7f9c381e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40884/0 (socket says 172.21.15.5:40884) 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 -- 172.21.15.5:0/2327810724 learned_addr learned my addr 172.21.15.5:0/2327810724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c377fe700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c3815dc50 0x7f9c380c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c36ffd700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c380cff90 0x7f9c380c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 -- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c3815dc50 msgr2=0x7f9c380c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c3815dc50 0x7f9c380c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 -- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c380cff90 msgr2=0x7f9c380c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c380cff90 0x7f9c380c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 -- 172.21.15.5:0/2327810724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c009550 con 0x7f9c380d41b0 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c367fc700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c380d41b0 0x7f9c381e71f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009130 tx=0x7f9c2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c1f7fe700 1 -- 172.21.15.5:0/2327810724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c003d60 con 0x7f9c380d41b0 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c1f7fe700 1 -- 172.21.15.5:0/2327810724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c011630 con 0x7f9c380d41b0 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c381e7c40 con 0x7f9c380d41b0 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.312+0000 7f9c1f7fe700 1 -- 172.21.15.5:0/2327810724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0119e0 con 0x7f9c380d41b0 2024-05-08T23:26:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.313+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c381e8200 con 0x7f9c380d41b0 2024-05-08T23:26:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.313+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c381e8720 con 0x7f9c380d41b0 2024-05-08T23:26:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.314+0000 7f9c1f7fe700 1 -- 172.21.15.5:0/2327810724 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9c2c011b40 con 0x7f9c380d41b0 2024-05-08T23:26:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.314+0000 7f9c1f7fe700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9c2406ac90 0x7f9c2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.315+0000 7f9c1f7fe700 1 -- 172.21.15.5:0/2327810724 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c2c08abd0 con 0x7f9c380d41b0 2024-05-08T23:26:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.315+0000 7f9c36ffd700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9c2406ac90 0x7f9c2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.315+0000 7f9c36ffd700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9c2406ac90 0x7f9c2406d150 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9c28007fd0 tx=0x7f9c28007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.319+0000 7f9c1f7fe700 1 -- 172.21.15.5:0/2327810724 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9c381e8720 con 0x7f9c380d41b0 2024-05-08T23:26:05.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:05.591+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9c381e8720 con 0x7f9c380d41b0 2024-05-08T23:26:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.366+0000 7f9c1f7fe700 1 -- 172.21.15.5:0/2327810724 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9c381e8720 con 0x7f9c380d41b0 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9c2406ac90 msgr2=0x7f9c2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9c2406ac90 0x7f9c2406d150 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9c28007fd0 tx=0x7f9c28007ec0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c380d41b0 msgr2=0x7f9c381e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c380d41b0 0x7f9c381e71f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009130 tx=0x7f9c2c004750 comp rx=0 tx=0).stop 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 shutdown_connections 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9c2406ac90 0x7f9c2406d150 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c3815dc50 0x7f9c380c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c380cff90 0x7f9c380c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 --2- 172.21.15.5:0/2327810724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c380d41b0 0x7f9c381e71f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.369+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 >> 172.21.15.5:0/2327810724 conn(0x7f9c3808afe0 msgr2=0x7f9c380b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.370+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 shutdown_connections 2024-05-08T23:26:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.370+0000 7f9c3d421700 1 -- 172.21.15.5:0/2327810724 wait complete. 2024-05-08T23:26:06.385 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:26:06.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:26:06.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.529+0000 7fbb6ebed700 1 Processor -- start 2024-05-08T23:26:06.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.529+0000 7fbb6ebed700 1 -- start start 2024-05-08T23:26:06.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.530+0000 7fbb6ebed700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb680af990 0x7fbb6806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:06.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.530+0000 7fbb6ebed700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb6806e7e0 0x7fbb6806ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:06.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.530+0000 7fbb6ebed700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb680b3870 con 0x7fbb6815d880 2024-05-08T23:26:06.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.530+0000 7fbb6ebed700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6806fe00 con 0x7fbb680af990 2024-05-08T23:26:06.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb6ebed700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb680b32f0 con 0x7fbb6806e7e0 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb67fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb680af990 0x7fbb6806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb6c989700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbb6815d880 0x7fbb680ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35182/0 (socket says 172.21.15.5:35182) 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb67fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb680af990 0x7fbb6806f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40890/0 (socket says 172.21.15.5:40890) 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb6c989700 1 -- 172.21.15.5:0/2243304206 learned_addr learned my addr 172.21.15.5:0/2243304206 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb6d18a700 1 --2- 172.21.15.5:0/2243304206 >> v2:172.21.15.49:3300/0 conn(0x7fbb6806e7e0 0x7fbb6806ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb67fff700 1 -- 172.21.15.5:0/2243304206 >> v2:172.21.15.49:3300/0 conn(0x7fbb6806e7e0 msgr2=0x7fbb6806ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb67fff700 1 --2- 172.21.15.5:0/2243304206 >> v2:172.21.15.49:3300/0 conn(0x7fbb6806e7e0 0x7fbb6806ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb67fff700 1 -- 172.21.15.5:0/2243304206 >> v1:172.21.15.5:6789/0 conn(0x7fbb6815d880 legacy=0x7fbb680ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb67fff700 1 -- 172.21.15.5:0/2243304206 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680b4ee0 con 0x7fbb680af990 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb67fff700 1 --2- 172.21.15.5:0/2243304206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb680af990 0x7fbb6806f8c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fbb680af0e0 tx=0x7fbb50009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d95b545290b63c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb677fe700 1 -- 172.21.15.5:0/2243304206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011830 con 0x7fbb680af990 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb677fe700 1 -- 172.21.15.5:0/2243304206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb50011990 con 0x7fbb680af990 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.531+0000 7fbb677fe700 1 -- 172.21.15.5:0/2243304206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011d20 con 0x7fbb680af990 2024-05-08T23:26:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.532+0000 7fbb6ebed700 1 -- 172.21.15.5:0/2243304206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb680af990 msgr2=0x7fbb6806f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.532+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/2243304206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb680af990 0x7fbb6806f8c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fbb680af0e0 tx=0x7fbb50009b20 comp rx=0 tx=0).stop 2024-05-08T23:26:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.532+0000 7fbb6ebed700 1 -- 172.21.15.5:0/2243304206 shutdown_connections 2024-05-08T23:26:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.532+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/2243304206 >> v2:172.21.15.49:3300/0 conn(0x7fbb6806e7e0 0x7fbb6806ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.532+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/2243304206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb680af990 0x7fbb6806f8c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.532+0000 7fbb6ebed700 1 -- 172.21.15.5:0/2243304206 >> 172.21.15.5:0/2243304206 conn(0x7fbb680b5810 msgr2=0x7fbb680b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.532+0000 7fbb6ebed700 1 -- 172.21.15.5:0/2243304206 shutdown_connections 2024-05-08T23:26:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.532+0000 7fbb6ebed700 1 -- 172.21.15.5:0/2243304206 wait complete. 2024-05-08T23:26:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.533+0000 7fbb6ebed700 1 Processor -- start 2024-05-08T23:26:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.533+0000 7fbb6ebed700 1 -- start start 2024-05-08T23:26:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.534+0000 7fbb6ebed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb6806e7e0 0x7fbb681f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.534+0000 7fbb6ebed700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb680af990 0x7fbb681f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.534+0000 7fbb6ebed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6815d880 0x7fbb6809d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.534+0000 7fbb6ebed700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6814c970 con 0x7fbb6806e7e0 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.534+0000 7fbb6ebed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6814cc20 con 0x7fbb6815d880 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.534+0000 7fbb6ebed700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680cd360 con 0x7fbb680af990 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb6806e7e0 0x7fbb681f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb6806e7e0 0x7fbb681f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40904/0 (socket says 172.21.15.5:40904) 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 -- 172.21.15.5:0/95444153 learned_addr learned my addr 172.21.15.5:0/95444153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb67fff700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb680af990 0x7fbb681f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6d18a700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6815d880 0x7fbb6809d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 -- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb680af990 msgr2=0x7fbb681f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb680af990 0x7fbb681f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 -- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6815d880 msgr2=0x7fbb6809d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6815d880 0x7fbb6809d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 -- 172.21.15.5:0/95444153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500094a0 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6c989700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb6806e7e0 0x7fbb681f0450 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c4b0 tx=0x7fbb5800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb657fa700 1 -- 172.21.15.5:0/95444153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58005d70 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb657fa700 1 -- 172.21.15.5:0/95444153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580070d0 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb681f3e60 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb657fa700 1 -- 172.21.15.5:0/95444153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58007440 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.535+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6814f450 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.536+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb68074bc0 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.537+0000 7fbb657fa700 1 -- 172.21.15.5:0/95444153 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbb580075a0 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.537+0000 7fbb657fa700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb4806ac90 0x7fbb4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.537+0000 7fbb657fa700 1 -- 172.21.15.5:0/95444153 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb58088660 con 0x7fbb6806e7e0 2024-05-08T23:26:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.538+0000 7fbb67fff700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb4806ac90 0x7fbb4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.538+0000 7fbb67fff700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb4806ac90 0x7fbb4806d150 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbb500090b0 tx=0x7fbb50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.542+0000 7fbb657fa700 1 -- 172.21.15.5:0/95444153 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbb580538f0 con 0x7fbb6806e7e0 2024-05-08T23:26:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:06.813+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbb6814cd60 con 0x7fbb6806e7e0 2024-05-08T23:26:07.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.368+0000 7fbb657fa700 1 -- 172.21.15.5:0/95444153 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbb58056f10 con 0x7fbb6806e7e0 2024-05-08T23:26:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.371+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb4806ac90 msgr2=0x7fbb4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.371+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb4806ac90 0x7fbb4806d150 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbb500090b0 tx=0x7fbb50008030 comp rx=0 tx=0).stop 2024-05-08T23:26:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.371+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb6806e7e0 msgr2=0x7fbb681f0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.371+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb6806e7e0 0x7fbb681f0450 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c4b0 tx=0x7fbb5800c870 comp rx=0 tx=0).stop 2024-05-08T23:26:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.372+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 shutdown_connections 2024-05-08T23:26:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.372+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbb4806ac90 0x7fbb4806d150 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.372+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6815d880 0x7fbb6809d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.372+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb6806e7e0 0x7fbb681f0450 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.372+0000 7fbb6ebed700 1 --2- 172.21.15.5:0/95444153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbb680af990 0x7fbb681f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.372+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 >> 172.21.15.5:0/95444153 conn(0x7fbb680b5810 msgr2=0x7fbb6815fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:07.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.372+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 shutdown_connections 2024-05-08T23:26:07.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.372+0000 7fbb6ebed700 1 -- 172.21.15.5:0/95444153 wait complete. 2024-05-08T23:26:07.387 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:26:07.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:07.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.528+0000 7f0e92927700 1 Processor -- start 2024-05-08T23:26:07.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.529+0000 7f0e92927700 1 -- start start 2024-05-08T23:26:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.529+0000 7f0e92927700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e8c0cd4e0 0x7f0e8c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.529+0000 7f0e92927700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cecf0 0x7f0e8c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.529+0000 7f0e92927700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c15c280 con 0x7f0e8c06f130 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.529+0000 7f0e92927700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c06e780 con 0x7f0e8c0cd4e0 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.529+0000 7f0e92927700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c06ea60 con 0x7f0e8c0cecf0 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e90ec4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cecf0 0x7f0e8c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e8bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e8c0cd4e0 0x7f0e8c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e8b7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0e8c06f130 0x7f0e8c0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:35084/0 (socket says 172.21.15.5:35084) 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/1292293879 learned_addr learned my addr 172.21.15.5:0/1292293879 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e8affd700 1 -- 172.21.15.5:0/1292293879 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129456794 0 0) 0x7f0e8c15c280 con 0x7f0e8c06f130 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e8affd700 1 -- 172.21.15.5:0/1292293879 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e740057c0 con 0x7f0e8c06f130 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e90ec4700 1 -- 172.21.15.5:0/1292293879 >> v2:172.21.15.49:3300/0 conn(0x7f0e8c0cd4e0 msgr2=0x7f0e8c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e90ec4700 1 --2- 172.21.15.5:0/1292293879 >> v2:172.21.15.49:3300/0 conn(0x7f0e8c0cd4e0 0x7f0e8c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e90ec4700 1 -- 172.21.15.5:0/1292293879 >> v1:172.21.15.49:6789/0 conn(0x7f0e8c06f130 legacy=0x7f0e8c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e90ec4700 1 -- 172.21.15.5:0/1292293879 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c06ed40 con 0x7f0e8c0cecf0 2024-05-08T23:26:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.530+0000 7f0e90ec4700 1 --2- 172.21.15.5:0/1292293879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cecf0 0x7f0e8c15c710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009d90 tx=0x7f0e8400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d2c9d2fa38128f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e8affd700 1 -- 172.21.15.5:0/1292293879 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400e070 con 0x7f0e8c0cecf0 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e8affd700 1 -- 172.21.15.5:0/1292293879 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8400bb20 con 0x7f0e8c0cecf0 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e8affd700 1 -- 172.21.15.5:0/1292293879 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84012430 con 0x7f0e8c0cecf0 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e92927700 1 -- 172.21.15.5:0/1292293879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cecf0 msgr2=0x7f0e8c15c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e92927700 1 --2- 172.21.15.5:0/1292293879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cecf0 0x7f0e8c15c710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009d90 tx=0x7f0e8400b330 comp rx=0 tx=0).stop 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e92927700 1 -- 172.21.15.5:0/1292293879 shutdown_connections 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e92927700 1 --2- 172.21.15.5:0/1292293879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cecf0 0x7f0e8c15c710 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e92927700 1 --2- 172.21.15.5:0/1292293879 >> v2:172.21.15.49:3300/0 conn(0x7f0e8c0cd4e0 0x7f0e8c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e92927700 1 -- 172.21.15.5:0/1292293879 >> 172.21.15.5:0/1292293879 conn(0x7f0e8c0d0320 msgr2=0x7f0e8c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e92927700 1 -- 172.21.15.5:0/1292293879 shutdown_connections 2024-05-08T23:26:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.531+0000 7f0e92927700 1 -- 172.21.15.5:0/1292293879 wait complete. 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.532+0000 7f0e92927700 1 Processor -- start 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.533+0000 7f0e92927700 1 -- start start 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.533+0000 7f0e92927700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c06f130 0x7f0e8c1ef150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.533+0000 7f0e92927700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cd4e0 0x7f0e8c14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.533+0000 7f0e92927700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cecf0 0x7f0e8c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.533+0000 7f0e92927700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c14ed70 con 0x7f0e8c06f130 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.533+0000 7f0e92927700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c1ef780 con 0x7f0e8c0cecf0 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.533+0000 7f0e92927700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c1efa60 con 0x7f0e8c0cd4e0 2024-05-08T23:26:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c06f130 0x7f0e8c1ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c06f130 0x7f0e8c1ef150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44740/0 (socket says 172.21.15.5:44740) 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 -- 172.21.15.5:0/377205291 learned_addr learned my addr 172.21.15.5:0/377205291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cd4e0 0x7f0e8c14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e90ec4700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cecf0 0x7f0e8c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 -- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cd4e0 msgr2=0x7f0e8c14e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cd4e0 0x7f0e8c14e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 -- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cecf0 msgr2=0x7f0e8c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cecf0 0x7f0e8c15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 -- 172.21.15.5:0/377205291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84009a70 con 0x7f0e8c06f130 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e90ec4700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cecf0 0x7f0e8c15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e8bfff700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c06f130 0x7f0e8c1ef150 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00e9b0 tx=0x7f0e7c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e88ff9700 1 -- 172.21.15.5:0/377205291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c009710 con 0x7f0e8c06f130 2024-05-08T23:26:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e88ff9700 1 -- 172.21.15.5:0/377205291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c00c850 con 0x7f0e8c06f130 2024-05-08T23:26:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c1efd40 con 0x7f0e8c06f130 2024-05-08T23:26:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e88ff9700 1 -- 172.21.15.5:0/377205291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00cbc0 con 0x7f0e8c06f130 2024-05-08T23:26:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.534+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c0b1670 con 0x7f0e8c06f130 2024-05-08T23:26:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.535+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c068000 con 0x7f0e8c06f130 2024-05-08T23:26:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.536+0000 7f0e88ff9700 1 -- 172.21.15.5:0/377205291 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e7c00cd20 con 0x7f0e8c06f130 2024-05-08T23:26:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.536+0000 7f0e88ff9700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e7806ac90 0x7f0e7806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.536+0000 7f0e88ff9700 1 -- 172.21.15.5:0/377205291 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e7c089490 con 0x7f0e8c06f130 2024-05-08T23:26:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.537+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e7806ac90 0x7f0e7806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.537+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e7806ac90 0x7f0e7806d150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0cd240 tx=0x7f0e80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:07.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.541+0000 7f0e88ff9700 1 -- 172.21.15.5:0/377205291 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0e7c054670 con 0x7f0e8c06f130 2024-05-08T23:26:07.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.809+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e8c06d440 con 0x7f0e8c06f130 2024-05-08T23:26:07.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.810+0000 7f0e88ff9700 1 -- 172.21.15.5:0/377205291 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4727 (secure 0 0 0) 0x7f0e7c057c90 con 0x7f0e8c06f130 2024-05-08T23:26:07.810 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:07.810 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:07.366240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e7806ac90 msgr2=0x7f0e7806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e7806ac90 0x7f0e7806d150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0cd240 tx=0x7f0e80009040 comp rx=0 tx=0).stop 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c06f130 msgr2=0x7f0e8c1ef150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c06f130 0x7f0e8c1ef150 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00e9b0 tx=0x7f0e7c00ed70 comp rx=0 tx=0).stop 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 shutdown_connections 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e7806ac90 0x7f0e7806d150 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e8c0cecf0 0x7f0e8c15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c06f130 0x7f0e8c1ef150 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.812+0000 7f0e92927700 1 --2- 172.21.15.5:0/377205291 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e8c0cd4e0 0x7f0e8c14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.813+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 >> 172.21.15.5:0/377205291 conn(0x7f0e8c0d0320 msgr2=0x7f0e8c0aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:07.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.813+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 shutdown_connections 2024-05-08T23:26:07.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.813+0000 7f0e92927700 1 -- 172.21.15.5:0/377205291 wait complete. 2024-05-08T23:26:07.814 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 66 2024-05-08T23:26:07.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:07.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.963+0000 7f879bfd8700 1 Processor -- start 2024-05-08T23:26:07.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.963+0000 7f879bfd8700 1 -- start start 2024-05-08T23:26:07.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879bfd8700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8794161c40 0x7f8794162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879bfd8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8794166320 0x7f87940c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879bfd8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f87940c3910 0x7f879406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879bfd8700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87940c3d80 con 0x7f87940c3910 2024-05-08T23:26:07.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879bfd8700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87940afb80 con 0x7f8794161c40 2024-05-08T23:26:07.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879bfd8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879406e0d0 con 0x7f8794166320 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879a575700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f87940c3910 0x7f879406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879a575700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f87940c3910 0x7f879406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44746/0 (socket says 172.21.15.5:44746) 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.964+0000 7f879a575700 1 -- 172.21.15.5:0/3600359717 learned_addr learned my addr 172.21.15.5:0/3600359717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f8799573700 1 --2- 172.21.15.5:0/3600359717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8794166320 0x7f87940c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f8799d74700 1 --2- 172.21.15.5:0/3600359717 >> v2:172.21.15.49:3300/0 conn(0x7f8794161c40 0x7f8794162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f879a575700 1 -- 172.21.15.5:0/3600359717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8794166320 msgr2=0x7f87940c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f879a575700 1 --2- 172.21.15.5:0/3600359717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8794166320 0x7f87940c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f879a575700 1 -- 172.21.15.5:0/3600359717 >> v2:172.21.15.49:3300/0 conn(0x7f8794161c40 msgr2=0x7f8794162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f879a575700 1 --2- 172.21.15.5:0/3600359717 >> v2:172.21.15.49:3300/0 conn(0x7f8794161c40 0x7f8794162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f879a575700 1 -- 172.21.15.5:0/3600359717 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794162650 con 0x7f87940c3910 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f8799573700 1 --2- 172.21.15.5:0/3600359717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8794166320 0x7f87940c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f8799d74700 1 --2- 172.21.15.5:0/3600359717 >> v2:172.21.15.49:3300/0 conn(0x7f8794161c40 0x7f8794162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f879a575700 1 --2- 172.21.15.5:0/3600359717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87940c3910 0x7f879406a960 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8794070650 tx=0x7f878c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e30ff774ce66cda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f8798d72700 1 -- 172.21.15.5:0/3600359717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c00e070 con 0x7f87940c3910 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f8798d72700 1 -- 172.21.15.5:0/3600359717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c00b9c0 con 0x7f87940c3910 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.965+0000 7f8798d72700 1 -- 172.21.15.5:0/3600359717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c00bd70 con 0x7f87940c3910 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 -- 172.21.15.5:0/3600359717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87940c3910 msgr2=0x7f879406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 --2- 172.21.15.5:0/3600359717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87940c3910 0x7f879406a960 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8794070650 tx=0x7f878c00b330 comp rx=0 tx=0).stop 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 -- 172.21.15.5:0/3600359717 shutdown_connections 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 --2- 172.21.15.5:0/3600359717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87940c3910 0x7f879406a960 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 --2- 172.21.15.5:0/3600359717 >> v2:172.21.15.49:3300/0 conn(0x7f8794161c40 0x7f8794162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 --2- 172.21.15.5:0/3600359717 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8794166320 0x7f87940c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 -- 172.21.15.5:0/3600359717 >> 172.21.15.5:0/3600359717 conn(0x7f8794099a50 msgr2=0x7f87940d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 -- 172.21.15.5:0/3600359717 shutdown_connections 2024-05-08T23:26:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.966+0000 7f879bfd8700 1 -- 172.21.15.5:0/3600359717 wait complete. 2024-05-08T23:26:07.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.967+0000 7f879bfd8700 1 Processor -- start 2024-05-08T23:26:07.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.967+0000 7f879bfd8700 1 -- start start 2024-05-08T23:26:07.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879bfd8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87940c3910 0x7f87940b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879bfd8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8794161c40 0x7f87940ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879bfd8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8794166320 0x7f87940b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879bfd8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87940b4550 con 0x7f8794166320 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879bfd8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87940b4830 con 0x7f8794161c40 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879bfd8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87940b4b10 con 0x7f87940c3910 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879a575700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8794166320 0x7f87940b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879a575700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8794166320 0x7f87940b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44758/0 (socket says 172.21.15.5:44758) 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879a575700 1 -- 172.21.15.5:0/1390976312 learned_addr learned my addr 172.21.15.5:0/1390976312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f8799d74700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87940c3910 0x7f87940b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f8799573700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8794161c40 0x7f87940ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879a575700 1 -- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87940c3910 msgr2=0x7f87940b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879a575700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87940c3910 0x7f87940b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879a575700 1 -- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8794161c40 msgr2=0x7f87940ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879a575700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8794161c40 0x7f87940ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f879a575700 1 -- 172.21.15.5:0/1390976312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c009b30 con 0x7f8794166320 2024-05-08T23:26:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.968+0000 7f8799573700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8794161c40 0x7f87940ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.969+0000 7f879a575700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8794166320 0x7f87940b1000 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b370 tx=0x7f878c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.969+0000 7f878a7fc700 1 -- 172.21.15.5:0/1390976312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0099b0 con 0x7f8794166320 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.969+0000 7f878a7fc700 1 -- 172.21.15.5:0/1390976312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c0092e0 con 0x7f8794166320 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.969+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87940b4df0 con 0x7f8794166320 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.969+0000 7f878a7fc700 1 -- 172.21.15.5:0/1390976312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c01b630 con 0x7f8794166320 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.969+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87940b53b0 con 0x7f8794166320 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.970+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8794068000 con 0x7f8794166320 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.970+0000 7f878a7fc700 1 -- 172.21.15.5:0/1390976312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f878c01b790 con 0x7f8794166320 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.971+0000 7f878a7fc700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f877806ac90 0x7f877806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.971+0000 7f878a7fc700 1 -- 172.21.15.5:0/1390976312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f878c08a9a0 con 0x7f8794166320 2024-05-08T23:26:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.971+0000 7f8799d74700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f877806ac90 0x7f877806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:07.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.972+0000 7f8799d74700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f877806ac90 0x7f877806d150 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8790006fd0 tx=0x7f8790008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:07.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:07.976+0000 7f878a7fc700 1 -- 172.21.15.5:0/1390976312 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f878c055cd0 con 0x7f8794166320 2024-05-08T23:26:08.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:26:08.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:26:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.196+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f879406d530 con 0x7f8794166320 2024-05-08T23:26:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.197+0000 7f878a7fc700 1 -- 172.21.15.5:0/1390976312 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16004 (secure 0 0 0) 0x7f878c082e60 con 0x7f8794166320 2024-05-08T23:26:08.198 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:08.198 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:05.360948+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:02.418282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:03.329387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f877806ac90 msgr2=0x7f877806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f877806ac90 0x7f877806d150 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8790006fd0 tx=0x7f8790008040 comp rx=0 tx=0).stop 2024-05-08T23:26:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8794166320 msgr2=0x7f87940b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8794166320 0x7f87940b1000 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b370 tx=0x7f878c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 shutdown_connections 2024-05-08T23:26:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f877806ac90 0x7f877806d150 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8794166320 0x7f87940b1000 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87940c3910 0x7f87940b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 --2- 172.21.15.5:0/1390976312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8794161c40 0x7f87940ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 >> 172.21.15.5:0/1390976312 conn(0x7f8794099a50 msgr2=0x7f87940cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 shutdown_connections 2024-05-08T23:26:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.200+0000 7f879bfd8700 1 -- 172.21.15.5:0/1390976312 wait complete. 2024-05-08T23:26:08.214 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:26:08.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:26:08.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.351+0000 7f53631e7700 1 Processor -- start 2024-05-08T23:26:08.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.351+0000 7f53631e7700 1 -- start start 2024-05-08T23:26:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.352+0000 7f53631e7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f535c0cd4e0 0x7f535c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.352+0000 7f53631e7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c15c280 con 0x7f535c0cecf0 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.352+0000 7f53631e7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c06e780 con 0x7f535c06f130 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.352+0000 7f53631e7700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c06ea60 con 0x7f535c0cd4e0 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.352+0000 7f5360f83700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f535c0cd4e0 0x7f535c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.352+0000 7f5361784700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f535c0cecf0 0x7f535c15c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38892/0 (socket says 172.21.15.5:38892) 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.352+0000 7f5361784700 1 -- 172.21.15.5:0/3608255888 learned_addr learned my addr 172.21.15.5:0/3608255888 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.352+0000 7f5360f83700 1 -- 172.21.15.5:0/3608255888 >> v1:172.21.15.49:6789/0 conn(0x7f535c06f130 legacy=0x7f535c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f5360f83700 1 -- 172.21.15.5:0/3608255888 >> v1:172.21.15.5:6789/0 conn(0x7f535c0cecf0 legacy=0x7f535c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f535b7fe700 1 -- 172.21.15.5:0/3608255888 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120756584 0 0) 0x7f535c15c280 con 0x7f535c0cecf0 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f5360f83700 1 -- 172.21.15.5:0/3608255888 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c06ed40 con 0x7f535c0cd4e0 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f5360f83700 1 --2- 172.21.15.5:0/3608255888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c0cd4e0 0x7f535c06f910 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f534c009840 tx=0x7f534c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a474d83f55fdea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f535b7fe700 1 -- 172.21.15.5:0/3608255888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c004230 con 0x7f535c0cd4e0 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f535b7fe700 1 -- 172.21.15.5:0/3608255888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c004390 con 0x7f535c0cd4e0 2024-05-08T23:26:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f535b7fe700 1 -- 172.21.15.5:0/3608255888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0036a0 con 0x7f535c0cd4e0 2024-05-08T23:26:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f53631e7700 1 -- 172.21.15.5:0/3608255888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c0cd4e0 msgr2=0x7f535c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.353+0000 7f53631e7700 1 --2- 172.21.15.5:0/3608255888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c0cd4e0 0x7f535c06f910 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f534c009840 tx=0x7f534c009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.354+0000 7f53631e7700 1 -- 172.21.15.5:0/3608255888 shutdown_connections 2024-05-08T23:26:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.354+0000 7f53631e7700 1 --2- 172.21.15.5:0/3608255888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c0cd4e0 0x7f535c06f910 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.354+0000 7f53631e7700 1 -- 172.21.15.5:0/3608255888 >> 172.21.15.5:0/3608255888 conn(0x7f535c0d0320 msgr2=0x7f535c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.354+0000 7f53631e7700 1 -- 172.21.15.5:0/3608255888 shutdown_connections 2024-05-08T23:26:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.354+0000 7f53631e7700 1 -- 172.21.15.5:0/3608255888 wait complete. 2024-05-08T23:26:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.355+0000 7f53631e7700 1 Processor -- start 2024-05-08T23:26:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.355+0000 7f53631e7700 1 -- start start 2024-05-08T23:26:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.356+0000 7f53631e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c06f130 0x7f535c1eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53631e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535c0cd4e0 0x7f535c0c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53631e7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f535c0cecf0 0x7f535c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53631e7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0b3c00 con 0x7f535c06f130 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53631e7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c156fb0 con 0x7f535c0cd4e0 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53631e7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0c54c0 con 0x7f535c0cecf0 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c06f130 0x7f535c1eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c06f130 0x7f535c1eeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44774/0 (socket says 172.21.15.5:44774) 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 -- 172.21.15.5:0/2702205529 learned_addr learned my addr 172.21.15.5:0/2702205529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5361784700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f535c0cecf0 0x7f535c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f535bfff700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535c0cd4e0 0x7f535c0c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 -- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f535c0cecf0 msgr2=0x7f535c0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f535c0cecf0 0x7f535c0b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 -- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535c0cd4e0 msgr2=0x7f535c0c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535c0cd4e0 0x7f535c0c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 -- 172.21.15.5:0/2702205529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c0094a0 con 0x7f535c06f130 2024-05-08T23:26:08.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f5360f83700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c06f130 0x7f535c1eeb70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f535c070560 tx=0x7f534c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:08.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53597fa700 1 -- 172.21.15.5:0/2702205529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c003e20 con 0x7f535c06f130 2024-05-08T23:26:08.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53597fa700 1 -- 172.21.15.5:0/2702205529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c01e070 con 0x7f535c06f130 2024-05-08T23:26:08.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c0c57a0 con 0x7f535c06f130 2024-05-08T23:26:08.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.357+0000 7f53597fa700 1 -- 172.21.15.5:0/2702205529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c011850 con 0x7f535c06f130 2024-05-08T23:26:08.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.358+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c0c5d60 con 0x7f535c06f130 2024-05-08T23:26:08.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.359+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c068000 con 0x7f535c06f130 2024-05-08T23:26:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.359+0000 7f53597fa700 1 -- 172.21.15.5:0/2702205529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f534c0119b0 con 0x7f535c06f130 2024-05-08T23:26:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.360+0000 7f53597fa700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f533c06ac90 0x7f533c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.360+0000 7f53597fa700 1 -- 172.21.15.5:0/2702205529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f534c08a570 con 0x7f535c06f130 2024-05-08T23:26:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.360+0000 7f535bfff700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f533c06ac90 0x7f533c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.361+0000 7f535bfff700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f533c06ac90 0x7f533c06d150 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f535c0ccfc0 tx=0x7f5344007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.364+0000 7f53597fa700 1 -- 172.21.15.5:0/2702205529 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f534c055760 con 0x7f535c06f130 2024-05-08T23:26:08.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.578+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f535c0d0e40 con 0x7f533c06ac90 2024-05-08T23:26:08.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.588+0000 7f53597fa700 1 -- 172.21.15.5:0/2702205529 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287789 (secure 0 0 0) 0x7f5344003a90 con 0x7f533c06ac90 2024-05-08T23:26:08.592 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f533c06ac90 msgr2=0x7f533c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f533c06ac90 0x7f533c06d150 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f535c0ccfc0 tx=0x7f5344007ec0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c06f130 msgr2=0x7f535c1eeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c06f130 0x7f535c1eeb70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f535c070560 tx=0x7f534c004750 comp rx=0 tx=0).stop 2024-05-08T23:26:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 shutdown_connections 2024-05-08T23:26:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f533c06ac90 0x7f533c06d150 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f535c0cecf0 0x7f535c0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f535c06f130 0x7f535c1eeb70 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 --2- 172.21.15.5:0/2702205529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535c0cd4e0 0x7f535c0c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 >> 172.21.15.5:0/2702205529 conn(0x7f535c0d0320 msgr2=0x7f535c15c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:08.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 shutdown_connections 2024-05-08T23:26:08.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.591+0000 7f53631e7700 1 -- 172.21.15.5:0/2702205529 wait complete. 2024-05-08T23:26:08.596 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:26:08.605 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-05-08T23:26:07.367069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":185,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":232368,"kb_used_data":7996,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754742352,"statfs":{"total":773094113280,"available":772856168448,"internally_reserved":0,"allocated":8187904,"data_stored":4142023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":229625397},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.475465"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.385435+0000","last_change":"2024-05-08T23:26:04.382812+0000","last_active":"2024-05-08T23:26:04.385435+0000","last_peered":"2024-05-08T23:26:04.385435+0000","last_clean":"2024-05-08T23:26:04.385435+0000","last_became_active":"2024-05-08T23:26:04.382518+0000","last_became_peered":"2024-05-08T23:26:04.382518+0000","last_unstale":"2024-05-08T23:26:04.385435+0000","last_undegraded":"2024-05-08T23:26:04.385435+0000","last_fullsized":"2024-05-08T23:26:04.385435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:16.459465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106783+0000","last_change":"2024-05-08T23:26:04.386202+0000","last_active":"2024-05-08T23:26:06.106783+0000","last_peered":"2024-05-08T23:26:06.106783+0000","last_clean":"2024-05-08T23:26:06.106783+0000","last_became_active":"2024-05-08T23:26:04.385897+0000","last_became_peered":"2024-05-08T23:26:04.385897+0000","last_unstale":"2024-05-08T23:26:06.106783+0000","last_undegraded":"2024-05-08T23:26:06.106783+0000","last_fullsized":"2024-05-08T23:26:06.106783+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:15.891844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365071+0000","last_change":"2024-05-08T23:26:04.376144+0000","last_active":"2024-05-08T23:26:05.365071+0000","last_peered":"2024-05-08T23:26:05.365071+0000","last_clean":"2024-05-08T23:26:05.365071+0000","last_became_active":"2024-05-08T23:26:04.375870+0000","last_became_peered":"2024-05-08T23:26:04.375870+0000","last_unstale":"2024-05-08T23:26:05.365071+0000","last_undegraded":"2024-05-08T23:26:05.365071+0000","last_fullsized":"2024-05-08T23:26:05.365071+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:53.570674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365640+0000","last_change":"2024-05-08T23:26:04.391205+0000","last_active":"2024-05-08T23:26:05.365640+0000","last_peered":"2024-05-08T23:26:05.365640+0000","last_clean":"2024-05-08T23:26:05.365640+0000","last_became_active":"2024-05-08T23:26:04.390895+0000","last_became_peered":"2024-05-08T23:26:04.390895+0000","last_unstale":"2024-05-08T23:26:05.365640+0000","last_undegraded":"2024-05-08T23:26:05.365640+0000","last_fullsized":"2024-05-08T23:26:05.365640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:00.837462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365140+0000","last_change":"2024-05-08T23:26:04.380499+0000","last_active":"2024-05-08T23:26:05.365140+0000","last_peered":"2024-05-08T23:26:05.365140+0000","last_clean":"2024-05-08T23:26:05.365140+0000","last_became_active":"2024-05-08T23:26:04.380163+0000","last_became_peered":"2024-05-08T23:26:04.380163+0000","last_unstale":"2024-05-08T23:26:05.365140+0000","last_undegraded":"2024-05-08T23:26:05.365140+0000","last_fullsized":"2024-05-08T23:26:05.365140+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:39.534202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.392388+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.392064+0000","last_became_peered":"2024-05-08T23:26:04.392064+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:55.698089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.387389+0000","last_change":"2024-05-08T23:26:04.377831+0000","last_active":"2024-05-08T23:26:04.387389+0000","last_peered":"2024-05-08T23:26:04.387389+0000","last_clean":"2024-05-08T23:26:04.387389+0000","last_became_active":"2024-05-08T23:26:04.377518+0000","last_became_peered":"2024-05-08T23:26:04.377518+0000","last_unstale":"2024-05-08T23:26:04.387389+0000","last_undegraded":"2024-05-08T23:26:04.387389+0000","last_fullsized":"2024-05-08T23:26:04.387389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106353+0000","last_change":"2024-05-08T23:26:04.388002+0000","last_active":"2024-05-08T23:26:06.106353+0000","last_peered":"2024-05-08T23:26:06.106353+0000","last_clean":"2024-05-08T23:26:06.106353+0000","last_became_active":"2024-05-08T23:26:04.386006+0000","last_became_peered":"2024-05-08T23:26:04.386006+0000","last_unstale":"2024-05-08T23:26:06.106353+0000","last_undegraded":"2024-05-08T23:26:06.106353+0000","last_fullsized":"2024-05-08T23:26:06.106353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:34:35.709822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365561+0000","last_change":"2024-05-08T23:26:04.389861+0000","last_active":"2024-05-08T23:26:05.365561+0000","last_peered":"2024-05-08T23:26:05.365561+0000","last_clean":"2024-05-08T23:26:05.365561+0000","last_became_active":"2024-05-08T23:26:04.389582+0000","last_became_peered":"2024-05-08T23:26:04.389582+0000","last_unstale":"2024-05-08T23:26:05.365561+0000","last_undegraded":"2024-05-08T23:26:05.365561+0000","last_fullsized":"2024-05-08T23:26:05.365561+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:12.594073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391883+0000","last_change":"2024-05-08T23:26:04.390913+0000","last_active":"2024-05-08T23:26:04.391883+0000","last_peered":"2024-05-08T23:26:04.391883+0000","last_clean":"2024-05-08T23:26:04.391883+0000","last_became_active":"2024-05-08T23:26:04.390654+0000","last_became_peered":"2024-05-08T23:26:04.390654+0000","last_unstale":"2024-05-08T23:26:04.391883+0000","last_undegraded":"2024-05-08T23:26:04.391883+0000","last_fullsized":"2024-05-08T23:26:04.391883+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365672+0000","last_change":"2024-05-08T23:26:04.382718+0000","last_active":"2024-05-08T23:26:05.365672+0000","last_peered":"2024-05-08T23:26:05.365672+0000","last_clean":"2024-05-08T23:26:05.365672+0000","last_became_active":"2024-05-08T23:26:04.381951+0000","last_became_peered":"2024-05-08T23:26:04.381951+0000","last_unstale":"2024-05-08T23:26:05.365672+0000","last_undegraded":"2024-05-08T23:26:05.365672+0000","last_fullsized":"2024-05-08T23:26:05.365672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:18.398011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528368+0000","last_change":"2024-05-08T23:26:04.393225+0000","last_active":"2024-05-08T23:26:05.528368+0000","last_peered":"2024-05-08T23:26:05.528368+0000","last_clean":"2024-05-08T23:26:05.528368+0000","last_became_active":"2024-05-08T23:26:04.392933+0000","last_became_peered":"2024-05-08T23:26:04.392933+0000","last_unstale":"2024-05-08T23:26:05.528368+0000","last_undegraded":"2024-05-08T23:26:05.528368+0000","last_fullsized":"2024-05-08T23:26:05.528368+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366036+0000","last_change":"2024-05-08T23:26:04.380528+0000","last_active":"2024-05-08T23:26:05.366036+0000","last_peered":"2024-05-08T23:26:05.366036+0000","last_clean":"2024-05-08T23:26:05.366036+0000","last_became_active":"2024-05-08T23:26:04.380129+0000","last_became_peered":"2024-05-08T23:26:04.380129+0000","last_unstale":"2024-05-08T23:26:05.366036+0000","last_undegraded":"2024-05-08T23:26:05.366036+0000","last_fullsized":"2024-05-08T23:26:05.366036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.038766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528231+0000","last_change":"2024-05-08T23:26:04.382434+0000","last_active":"2024-05-08T23:26:05.528231+0000","last_peered":"2024-05-08T23:26:05.528231+0000","last_clean":"2024-05-08T23:26:05.528231+0000","last_became_active":"2024-05-08T23:26:04.382015+0000","last_became_peered":"2024-05-08T23:26:04.382015+0000","last_unstale":"2024-05-08T23:26:05.528231+0000","last_undegraded":"2024-05-08T23:26:05.528231+0000","last_fullsized":"2024-05-08T23:26:05.528231+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365565+0000","last_change":"2024-05-08T23:26:04.386467+0000","last_active":"2024-05-08T23:26:05.365565+0000","last_peered":"2024-05-08T23:26:05.365565+0000","last_clean":"2024-05-08T23:26:05.365565+0000","last_became_active":"2024-05-08T23:26:04.386005+0000","last_became_peered":"2024-05-08T23:26:04.386005+0000","last_unstale":"2024-05-08T23:26:05.365565+0000","last_undegraded":"2024-05-08T23:26:05.365565+0000","last_fullsized":"2024-05-08T23:26:05.365565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:08.141995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365004+0000","last_change":"2024-05-08T23:26:04.384544+0000","last_active":"2024-05-08T23:26:05.365004+0000","last_peered":"2024-05-08T23:26:05.365004+0000","last_clean":"2024-05-08T23:26:05.365004+0000","last_became_active":"2024-05-08T23:26:04.384240+0000","last_became_peered":"2024-05-08T23:26:04.384240+0000","last_unstale":"2024-05-08T23:26:05.365004+0000","last_undegraded":"2024-05-08T23:26:05.365004+0000","last_fullsized":"2024-05-08T23:26:05.365004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:44.177271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106649+0000","last_change":"2024-05-08T23:26:04.387389+0000","last_active":"2024-05-08T23:26:06.106649+0000","last_peered":"2024-05-08T23:26:06.106649+0000","last_clean":"2024-05-08T23:26:06.106649+0000","last_became_active":"2024-05-08T23:26:04.383192+0000","last_became_peered":"2024-05-08T23:26:04.383192+0000","last_unstale":"2024-05-08T23:26:06.106649+0000","last_undegraded":"2024-05-08T23:26:06.106649+0000","last_fullsized":"2024-05-08T23:26:06.106649+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528319+0000","last_change":"2024-05-08T23:26:04.374428+0000","last_active":"2024-05-08T23:26:05.528319+0000","last_peered":"2024-05-08T23:26:05.528319+0000","last_clean":"2024-05-08T23:26:05.528319+0000","last_became_active":"2024-05-08T23:26:04.374165+0000","last_became_peered":"2024-05-08T23:26:04.374165+0000","last_unstale":"2024-05-08T23:26:05.528319+0000","last_undegraded":"2024-05-08T23:26:05.528319+0000","last_fullsized":"2024-05-08T23:26:05.528319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382847+0000","last_change":"2024-05-08T23:26:04.380605+0000","last_active":"2024-05-08T23:26:04.382847+0000","last_peered":"2024-05-08T23:26:04.382847+0000","last_clean":"2024-05-08T23:26:04.382847+0000","last_became_active":"2024-05-08T23:26:04.380307+0000","last_became_peered":"2024-05-08T23:26:04.380307+0000","last_unstale":"2024-05-08T23:26:04.382847+0000","last_undegraded":"2024-05-08T23:26:04.382847+0000","last_fullsized":"2024-05-08T23:26:04.382847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:27.367580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.395921+0000","last_change":"2024-05-08T23:26:04.395150+0000","last_active":"2024-05-08T23:26:04.395921+0000","last_peered":"2024-05-08T23:26:04.395921+0000","last_clean":"2024-05-08T23:26:04.395921+0000","last_became_active":"2024-05-08T23:26:04.394845+0000","last_became_peered":"2024-05-08T23:26:04.394845+0000","last_unstale":"2024-05-08T23:26:04.395921+0000","last_undegraded":"2024-05-08T23:26:04.395921+0000","last_fullsized":"2024-05-08T23:26:04.395921+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.390865+0000","last_change":"2024-05-08T23:26:04.387174+0000","last_active":"2024-05-08T23:26:04.390865+0000","last_peered":"2024-05-08T23:26:04.390865+0000","last_clean":"2024-05-08T23:26:04.390865+0000","last_became_active":"2024-05-08T23:26:04.386653+0000","last_became_peered":"2024-05-08T23:26:04.386653+0000","last_unstale":"2024-05-08T23:26:04.390865+0000","last_undegraded":"2024-05-08T23:26:04.390865+0000","last_fullsized":"2024-05-08T23:26:04.390865+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:52.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365110+0000","last_change":"2024-05-08T23:26:04.374586+0000","last_active":"2024-05-08T23:26:05.365110+0000","last_peered":"2024-05-08T23:26:05.365110+0000","last_clean":"2024-05-08T23:26:05.365110+0000","last_became_active":"2024-05-08T23:26:04.374382+0000","last_became_peered":"2024-05-08T23:26:04.374382+0000","last_unstale":"2024-05-08T23:26:05.365110+0000","last_undegraded":"2024-05-08T23:26:05.365110+0000","last_fullsized":"2024-05-08T23:26:05.365110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:41.989450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364727+0000","last_change":"2024-05-08T23:26:04.376502+0000","last_active":"2024-05-08T23:26:05.364727+0000","last_peered":"2024-05-08T23:26:05.364727+0000","last_clean":"2024-05-08T23:26:05.364727+0000","last_became_active":"2024-05-08T23:26:04.376226+0000","last_became_peered":"2024-05-08T23:26:04.376226+0000","last_unstale":"2024-05-08T23:26:05.364727+0000","last_undegraded":"2024-05-08T23:26:05.364727+0000","last_fullsized":"2024-05-08T23:26:05.364727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:36.452021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.393993+0000","last_change":"2024-05-08T23:26:04.393135+0000","last_active":"2024-05-08T23:26:04.393993+0000","last_peered":"2024-05-08T23:26:04.393993+0000","last_clean":"2024-05-08T23:26:04.393993+0000","last_became_active":"2024-05-08T23:26:04.392824+0000","last_became_peered":"2024-05-08T23:26:04.392824+0000","last_unstale":"2024-05-08T23:26:04.393993+0000","last_undegraded":"2024-05-08T23:26:04.393993+0000","last_fullsized":"2024-05-08T23:26:04.393993+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:47.989361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391112+0000","last_change":"2024-05-08T23:26:04.387882+0000","last_active":"2024-05-08T23:26:04.391112+0000","last_peered":"2024-05-08T23:26:04.391112+0000","last_clean":"2024-05-08T23:26:04.391112+0000","last_became_active":"2024-05-08T23:26:04.387600+0000","last_became_peered":"2024-05-08T23:26:04.387600+0000","last_unstale":"2024-05-08T23:26:04.391112+0000","last_undegraded":"2024-05-08T23:26:04.391112+0000","last_fullsized":"2024-05-08T23:26:04.391112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:29.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106297+0000","last_change":"2024-05-08T23:26:04.390849+0000","last_active":"2024-05-08T23:26:06.106297+0000","last_peered":"2024-05-08T23:26:06.106297+0000","last_clean":"2024-05-08T23:26:06.106297+0000","last_became_active":"2024-05-08T23:26:04.390577+0000","last_became_peered":"2024-05-08T23:26:04.390577+0000","last_unstale":"2024-05-08T23:26:06.106297+0000","last_undegraded":"2024-05-08T23:26:06.106297+0000","last_fullsized":"2024-05-08T23:26:06.106297+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:00.857944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389116+0000","last_change":"2024-05-08T23:26:04.387838+0000","last_active":"2024-05-08T23:26:04.389116+0000","last_peered":"2024-05-08T23:26:04.389116+0000","last_clean":"2024-05-08T23:26:04.389116+0000","last_became_active":"2024-05-08T23:26:04.387594+0000","last_became_peered":"2024-05-08T23:26:04.387594+0000","last_unstale":"2024-05-08T23:26:04.389116+0000","last_undegraded":"2024-05-08T23:26:04.389116+0000","last_fullsized":"2024-05-08T23:26:04.389116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389890+0000","last_change":"2024-05-08T23:26:03.918800+0000","last_active":"2024-05-08T23:26:04.389890+0000","last_peered":"2024-05-08T23:26:04.389890+0000","last_clean":"2024-05-08T23:26:04.389890+0000","last_became_active":"2024-05-08T23:26:03.918509+0000","last_became_peered":"2024-05-08T23:26:03.918509+0000","last_unstale":"2024-05-08T23:26:04.389890+0000","last_undegraded":"2024-05-08T23:26:04.389890+0000","last_fullsized":"2024-05-08T23:26:04.389890+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:46.488568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365685+0000","last_change":"2024-05-08T23:26:04.386283+0000","last_active":"2024-05-08T23:26:05.365685+0000","last_peered":"2024-05-08T23:26:05.365685+0000","last_clean":"2024-05-08T23:26:05.365685+0000","last_became_active":"2024-05-08T23:26:04.385690+0000","last_became_peered":"2024-05-08T23:26:04.385690+0000","last_unstale":"2024-05-08T23:26:05.365685+0000","last_undegraded":"2024-05-08T23:26:05.365685+0000","last_fullsized":"2024-05-08T23:26:05.365685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:12.667029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528769+0000","last_change":"2024-05-08T23:26:03.912463+0000","last_active":"2024-05-08T23:26:05.528769+0000","last_peered":"2024-05-08T23:26:05.528769+0000","last_clean":"2024-05-08T23:26:05.528769+0000","last_became_active":"2024-05-08T23:26:03.912178+0000","last_became_peered":"2024-05-08T23:26:03.912178+0000","last_unstale":"2024-05-08T23:26:05.528769+0000","last_undegraded":"2024-05-08T23:26:05.528769+0000","last_fullsized":"2024-05-08T23:26:05.528769+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:02.228610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.385614+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.385325+0000","last_became_peered":"2024-05-08T23:26:04.385325+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.979369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382824+0000","last_change":"2024-05-08T23:26:04.380473+0000","last_active":"2024-05-08T23:26:04.382824+0000","last_peered":"2024-05-08T23:26:04.382824+0000","last_clean":"2024-05-08T23:26:04.382824+0000","last_became_active":"2024-05-08T23:26:04.380170+0000","last_became_peered":"2024-05-08T23:26:04.380170+0000","last_unstale":"2024-05-08T23:26:04.382824+0000","last_undegraded":"2024-05-08T23:26:04.382824+0000","last_fullsized":"2024-05-08T23:26:04.382824+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:30.884597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106822+0000","last_change":"2024-05-08T23:26:03.918662+0000","last_active":"2024-05-08T23:26:06.106822+0000","last_peered":"2024-05-08T23:26:06.106822+0000","last_clean":"2024-05-08T23:26:06.106822+0000","last_became_active":"2024-05-08T23:26:03.918367+0000","last_became_peered":"2024-05-08T23:26:03.918367+0000","last_unstale":"2024-05-08T23:26:06.106822+0000","last_undegraded":"2024-05-08T23:26:06.106822+0000","last_fullsized":"2024-05-08T23:26:06.106822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.392305+0000","last_change":"2024-05-08T23:26:04.391595+0000","last_active":"2024-05-08T23:26:04.392305+0000","last_peered":"2024-05-08T23:26:04.392305+0000","last_clean":"2024-05-08T23:26:04.392305+0000","last_became_active":"2024-05-08T23:26:04.391291+0000","last_became_peered":"2024-05-08T23:26:04.391291+0000","last_unstale":"2024-05-08T23:26:04.392305+0000","last_undegraded":"2024-05-08T23:26:04.392305+0000","last_fullsized":"2024-05-08T23:26:04.392305+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:17.695615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364906+0000","last_change":"2024-05-08T23:26:03.916776+0000","last_active":"2024-05-08T23:26:05.364906+0000","last_peered":"2024-05-08T23:26:05.364906+0000","last_clean":"2024-05-08T23:26:05.364906+0000","last_became_active":"2024-05-08T23:26:03.916402+0000","last_became_peered":"2024-05-08T23:26:03.916402+0000","last_unstale":"2024-05-08T23:26:05.364906+0000","last_undegraded":"2024-05-08T23:26:05.364906+0000","last_fullsized":"2024-05-08T23:26:05.364906+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:04.288000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106261+0000","last_change":"2024-05-08T23:26:03.915034+0000","last_active":"2024-05-08T23:26:06.106261+0000","last_peered":"2024-05-08T23:26:06.106261+0000","last_clean":"2024-05-08T23:26:06.106261+0000","last_became_active":"2024-05-08T23:26:03.914666+0000","last_became_peered":"2024-05-08T23:26:03.914666+0000","last_unstale":"2024-05-08T23:26:06.106261+0000","last_undegraded":"2024-05-08T23:26:06.106261+0000","last_fullsized":"2024-05-08T23:26:06.106261+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:43.175140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365034+0000","last_change":"2024-05-08T23:26:04.381034+0000","last_active":"2024-05-08T23:26:05.365034+0000","last_peered":"2024-05-08T23:26:05.365034+0000","last_clean":"2024-05-08T23:26:05.365034+0000","last_became_active":"2024-05-08T23:26:04.380769+0000","last_became_peered":"2024-05-08T23:26:04.380769+0000","last_unstale":"2024-05-08T23:26:05.365034+0000","last_undegraded":"2024-05-08T23:26:05.365034+0000","last_fullsized":"2024-05-08T23:26:05.365034+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:35.530781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365301+0000","last_change":"2024-05-08T23:26:03.917031+0000","last_active":"2024-05-08T23:26:05.365301+0000","last_peered":"2024-05-08T23:26:05.365301+0000","last_clean":"2024-05-08T23:26:05.365301+0000","last_became_active":"2024-05-08T23:26:03.916394+0000","last_became_peered":"2024-05-08T23:26:03.916394+0000","last_unstale":"2024-05-08T23:26:05.365301+0000","last_undegraded":"2024-05-08T23:26:05.365301+0000","last_fullsized":"2024-05-08T23:26:05.365301+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:38.532072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528242+0000","last_change":"2024-05-08T23:26:04.374371+0000","last_active":"2024-05-08T23:26:05.528242+0000","last_peered":"2024-05-08T23:26:05.528242+0000","last_clean":"2024-05-08T23:26:05.528242+0000","last_became_active":"2024-05-08T23:26:04.374097+0000","last_became_peered":"2024-05-08T23:26:04.374097+0000","last_unstale":"2024-05-08T23:26:05.528242+0000","last_undegraded":"2024-05-08T23:26:05.528242+0000","last_fullsized":"2024-05-08T23:26:05.528242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365139+0000","last_change":"2024-05-08T23:26:03.911594+0000","last_active":"2024-05-08T23:26:05.365139+0000","last_peered":"2024-05-08T23:26:05.365139+0000","last_clean":"2024-05-08T23:26:05.365139+0000","last_became_active":"2024-05-08T23:26:03.911273+0000","last_became_peered":"2024-05-08T23:26:03.911273+0000","last_unstale":"2024-05-08T23:26:05.365139+0000","last_undegraded":"2024-05-08T23:26:05.365139+0000","last_fullsized":"2024-05-08T23:26:05.365139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:30.134959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.373540+0000","last_change":"2024-05-08T23:26:04.372794+0000","last_active":"2024-05-08T23:26:04.373540+0000","last_peered":"2024-05-08T23:26:04.373540+0000","last_clean":"2024-05-08T23:26:04.373540+0000","last_became_active":"2024-05-08T23:26:04.372536+0000","last_became_peered":"2024-05-08T23:26:04.372536+0000","last_unstale":"2024-05-08T23:26:04.373540+0000","last_undegraded":"2024-05-08T23:26:04.373540+0000","last_fullsized":"2024-05-08T23:26:04.373540+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:06.816807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391925+0000","last_change":"2024-05-08T23:26:04.388857+0000","last_active":"2024-05-08T23:26:04.391925+0000","last_peered":"2024-05-08T23:26:04.391925+0000","last_clean":"2024-05-08T23:26:04.391925+0000","last_became_active":"2024-05-08T23:26:04.388581+0000","last_became_peered":"2024-05-08T23:26:04.388581+0000","last_unstale":"2024-05-08T23:26:04.391925+0000","last_undegraded":"2024-05-08T23:26:04.391925+0000","last_fullsized":"2024-05-08T23:26:04.391925+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106865+0000","last_change":"2024-05-08T23:26:03.907421+0000","last_active":"2024-05-08T23:26:06.106865+0000","last_peered":"2024-05-08T23:26:06.106865+0000","last_clean":"2024-05-08T23:26:06.106865+0000","last_became_active":"2024-05-08T23:26:03.907143+0000","last_became_peered":"2024-05-08T23:26:03.907143+0000","last_unstale":"2024-05-08T23:26:06.106865+0000","last_undegraded":"2024-05-08T23:26:06.106865+0000","last_fullsized":"2024-05-08T23:26:06.106865+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:07.208552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106627+0000","last_change":"2024-05-08T23:26:04.386708+0000","last_active":"2024-05-08T23:26:06.106627+0000","last_peered":"2024-05-08T23:26:06.106627+0000","last_clean":"2024-05-08T23:26:06.106627+0000","last_became_active":"2024-05-08T23:26:04.385984+0000","last_became_peered":"2024-05-08T23:26:04.385984+0000","last_unstale":"2024-05-08T23:26:06.106627+0000","last_undegraded":"2024-05-08T23:26:06.106627+0000","last_fullsized":"2024-05-08T23:26:06.106627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:47.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365174+0000","last_change":"2024-05-08T23:26:03.913646+0000","last_active":"2024-05-08T23:26:05.365174+0000","last_peered":"2024-05-08T23:26:05.365174+0000","last_clean":"2024-05-08T23:26:05.365174+0000","last_became_active":"2024-05-08T23:26:03.913208+0000","last_became_peered":"2024-05-08T23:26:03.913208+0000","last_unstale":"2024-05-08T23:26:05.365174+0000","last_undegraded":"2024-05-08T23:26:05.365174+0000","last_fullsized":"2024-05-08T23:26:05.365174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:02.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365662+0000","last_change":"2024-05-08T23:26:04.375029+0000","last_active":"2024-05-08T23:26:05.365662+0000","last_peered":"2024-05-08T23:26:05.365662+0000","last_clean":"2024-05-08T23:26:05.365662+0000","last_became_active":"2024-05-08T23:26:04.374797+0000","last_became_peered":"2024-05-08T23:26:04.374797+0000","last_unstale":"2024-05-08T23:26:05.365662+0000","last_undegraded":"2024-05-08T23:26:05.365662+0000","last_fullsized":"2024-05-08T23:26:05.365662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:18.355499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365807+0000","last_change":"2024-05-08T23:26:03.913239+0000","last_active":"2024-05-08T23:26:05.365807+0000","last_peered":"2024-05-08T23:26:05.365807+0000","last_clean":"2024-05-08T23:26:05.365807+0000","last_became_active":"2024-05-08T23:26:03.912892+0000","last_became_peered":"2024-05-08T23:26:03.912892+0000","last_unstale":"2024-05-08T23:26:05.365807+0000","last_undegraded":"2024-05-08T23:26:05.365807+0000","last_fullsized":"2024-05-08T23:26:05.365807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:28.036636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.385254+0000","last_change":"2024-05-08T23:26:04.381564+0000","last_active":"2024-05-08T23:26:04.385254+0000","last_peered":"2024-05-08T23:26:04.385254+0000","last_clean":"2024-05-08T23:26:04.385254+0000","last_became_active":"2024-05-08T23:26:04.381293+0000","last_became_peered":"2024-05-08T23:26:04.381293+0000","last_unstale":"2024-05-08T23:26:04.385254+0000","last_undegraded":"2024-05-08T23:26:04.385254+0000","last_fullsized":"2024-05-08T23:26:04.385254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365371+0000","last_change":"2024-05-08T23:26:03.917272+0000","last_active":"2024-05-08T23:26:05.365371+0000","last_peered":"2024-05-08T23:26:05.365371+0000","last_clean":"2024-05-08T23:26:05.365371+0000","last_became_active":"2024-05-08T23:26:03.916960+0000","last_became_peered":"2024-05-08T23:26:03.916960+0000","last_unstale":"2024-05-08T23:26:05.365371+0000","last_undegraded":"2024-05-08T23:26:05.365371+0000","last_fullsized":"2024-05-08T23:26:05.365371+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.380000+0000","last_change":"2024-05-08T23:26:04.375997+0000","last_active":"2024-05-08T23:26:04.380000+0000","last_peered":"2024-05-08T23:26:04.380000+0000","last_clean":"2024-05-08T23:26:04.380000+0000","last_became_active":"2024-05-08T23:26:04.375654+0000","last_became_peered":"2024-05-08T23:26:04.375654+0000","last_unstale":"2024-05-08T23:26:04.380000+0000","last_undegraded":"2024-05-08T23:26:04.380000+0000","last_fullsized":"2024-05-08T23:26:04.380000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:03.230741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106390+0000","last_change":"2024-05-08T23:26:03.911788+0000","last_active":"2024-05-08T23:26:06.106390+0000","last_peered":"2024-05-08T23:26:06.106390+0000","last_clean":"2024-05-08T23:26:06.106390+0000","last_became_active":"2024-05-08T23:26:03.911527+0000","last_became_peered":"2024-05-08T23:26:03.911527+0000","last_unstale":"2024-05-08T23:26:06.106390+0000","last_undegraded":"2024-05-08T23:26:06.106390+0000","last_fullsized":"2024-05-08T23:26:06.106390+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:44.784641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528178+0000","last_change":"2024-05-08T23:26:04.394848+0000","last_active":"2024-05-08T23:26:05.528178+0000","last_peered":"2024-05-08T23:26:05.528178+0000","last_clean":"2024-05-08T23:26:05.528178+0000","last_became_active":"2024-05-08T23:26:04.394520+0000","last_became_peered":"2024-05-08T23:26:04.394520+0000","last_unstale":"2024-05-08T23:26:05.528178+0000","last_undegraded":"2024-05-08T23:26:05.528178+0000","last_fullsized":"2024-05-08T23:26:05.528178+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367381+0000","last_change":"2024-05-08T23:26:03.904993+0000","last_active":"2024-05-08T23:26:04.367381+0000","last_peered":"2024-05-08T23:26:04.367381+0000","last_clean":"2024-05-08T23:26:04.367381+0000","last_became_active":"2024-05-08T23:26:03.904689+0000","last_became_peered":"2024-05-08T23:26:03.904689+0000","last_unstale":"2024-05-08T23:26:04.367381+0000","last_undegraded":"2024-05-08T23:26:04.367381+0000","last_fullsized":"2024-05-08T23:26:04.367381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:54.695959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365441+0000","last_change":"2024-05-08T23:26:04.386132+0000","last_active":"2024-05-08T23:26:05.365441+0000","last_peered":"2024-05-08T23:26:05.365441+0000","last_clean":"2024-05-08T23:26:05.365441+0000","last_became_active":"2024-05-08T23:26:04.385842+0000","last_became_peered":"2024-05-08T23:26:04.385842+0000","last_unstale":"2024-05-08T23:26:05.365441+0000","last_undegraded":"2024-05-08T23:26:05.365441+0000","last_fullsized":"2024-05-08T23:26:05.365441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:42.318612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106703+0000","last_change":"2024-05-08T23:26:03.917170+0000","last_active":"2024-05-08T23:26:06.106703+0000","last_peered":"2024-05-08T23:26:06.106703+0000","last_clean":"2024-05-08T23:26:06.106703+0000","last_became_active":"2024-05-08T23:26:03.916888+0000","last_became_peered":"2024-05-08T23:26:03.916888+0000","last_unstale":"2024-05-08T23:26:06.106703+0000","last_undegraded":"2024-05-08T23:26:06.106703+0000","last_fullsized":"2024-05-08T23:26:06.106703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:59.835332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.388091+0000","last_change":"2024-05-08T23:26:04.385163+0000","last_active":"2024-05-08T23:26:04.388091+0000","last_peered":"2024-05-08T23:26:04.388091+0000","last_clean":"2024-05-08T23:26:04.388091+0000","last_became_active":"2024-05-08T23:26:04.384430+0000","last_became_peered":"2024-05-08T23:26:04.384430+0000","last_unstale":"2024-05-08T23:26:04.388091+0000","last_undegraded":"2024-05-08T23:26:04.388091+0000","last_fullsized":"2024-05-08T23:26:04.388091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:13.482666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365760+0000","last_change":"2024-05-08T23:26:04.382870+0000","last_active":"2024-05-08T23:26:05.365760+0000","last_peered":"2024-05-08T23:26:05.365760+0000","last_clean":"2024-05-08T23:26:05.365760+0000","last_became_active":"2024-05-08T23:26:04.382320+0000","last_became_peered":"2024-05-08T23:26:04.382320+0000","last_unstale":"2024-05-08T23:26:05.365760+0000","last_undegraded":"2024-05-08T23:26:05.365760+0000","last_fullsized":"2024-05-08T23:26:05.365760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:17.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106864+0000","last_change":"2024-05-08T23:26:03.910712+0000","last_active":"2024-05-08T23:26:06.106864+0000","last_peered":"2024-05-08T23:26:06.106864+0000","last_clean":"2024-05-08T23:26:06.106864+0000","last_became_active":"2024-05-08T23:26:03.910424+0000","last_became_peered":"2024-05-08T23:26:03.910424+0000","last_unstale":"2024-05-08T23:26:06.106864+0000","last_undegraded":"2024-05-08T23:26:06.106864+0000","last_fullsized":"2024-05-08T23:26:06.106864+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:16.285704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365799+0000","last_change":"2024-05-08T23:26:04.378589+0000","last_active":"2024-05-08T23:26:05.365799+0000","last_peered":"2024-05-08T23:26:05.365799+0000","last_clean":"2024-05-08T23:26:05.365799+0000","last_became_active":"2024-05-08T23:26:04.378300+0000","last_became_peered":"2024-05-08T23:26:04.378300+0000","last_unstale":"2024-05-08T23:26:05.365799+0000","last_undegraded":"2024-05-08T23:26:05.365799+0000","last_fullsized":"2024-05-08T23:26:05.365799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.786772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.368459+0000","last_change":"2024-05-08T23:26:03.916915+0000","last_active":"2024-05-08T23:26:04.368459+0000","last_peered":"2024-05-08T23:26:04.368459+0000","last_clean":"2024-05-08T23:26:04.368459+0000","last_became_active":"2024-05-08T23:26:03.916632+0000","last_became_peered":"2024-05-08T23:26:03.916632+0000","last_unstale":"2024-05-08T23:26:04.368459+0000","last_undegraded":"2024-05-08T23:26:04.368459+0000","last_fullsized":"2024-05-08T23:26:04.368459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:22.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364817+0000","last_change":"2024-05-08T23:26:04.375139+0000","last_active":"2024-05-08T23:26:05.364817+0000","last_peered":"2024-05-08T23:26:05.364817+0000","last_clean":"2024-05-08T23:26:05.364817+0000","last_became_active":"2024-05-08T23:26:04.374874+0000","last_became_peered":"2024-05-08T23:26:04.374874+0000","last_unstale":"2024-05-08T23:26:05.364817+0000","last_undegraded":"2024-05-08T23:26:05.364817+0000","last_fullsized":"2024-05-08T23:26:05.364817+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:42.364785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106533+0000","last_change":"2024-05-08T23:26:03.915036+0000","last_active":"2024-05-08T23:26:06.106533+0000","last_peered":"2024-05-08T23:26:06.106533+0000","last_clean":"2024-05-08T23:26:06.106533+0000","last_became_active":"2024-05-08T23:26:03.914616+0000","last_became_peered":"2024-05-08T23:26:03.914616+0000","last_unstale":"2024-05-08T23:26:06.106533+0000","last_undegraded":"2024-05-08T23:26:06.106533+0000","last_fullsized":"2024-05-08T23:26:06.106533+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.591942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364460+0000","last_change":"2024-05-08T23:26:04.384503+0000","last_active":"2024-05-08T23:26:05.364460+0000","last_peered":"2024-05-08T23:26:05.364460+0000","last_clean":"2024-05-08T23:26:05.364460+0000","last_became_active":"2024-05-08T23:26:04.383639+0000","last_became_peered":"2024-05-08T23:26:04.383639+0000","last_unstale":"2024-05-08T23:26:05.364460+0000","last_undegraded":"2024-05-08T23:26:05.364460+0000","last_fullsized":"2024-05-08T23:26:05.364460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:13.674487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367498+0000","last_change":"2024-05-08T23:26:03.912730+0000","last_active":"2024-05-08T23:26:04.367498+0000","last_peered":"2024-05-08T23:26:04.367498+0000","last_clean":"2024-05-08T23:26:04.367498+0000","last_became_active":"2024-05-08T23:26:03.912415+0000","last_became_peered":"2024-05-08T23:26:03.912415+0000","last_unstale":"2024-05-08T23:26:04.367498+0000","last_undegraded":"2024-05-08T23:26:04.367498+0000","last_fullsized":"2024-05-08T23:26:04.367498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:35.080457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364494+0000","last_change":"2024-05-08T23:26:04.383931+0000","last_active":"2024-05-08T23:26:05.364494+0000","last_peered":"2024-05-08T23:26:05.364494+0000","last_clean":"2024-05-08T23:26:05.364494+0000","last_became_active":"2024-05-08T23:26:04.383598+0000","last_became_peered":"2024-05-08T23:26:04.383598+0000","last_unstale":"2024-05-08T23:26:05.364494+0000","last_undegraded":"2024-05-08T23:26:05.364494+0000","last_fullsized":"2024-05-08T23:26:05.364494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:27.479478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528163+0000","last_change":"2024-05-08T23:26:03.908029+0000","last_active":"2024-05-08T23:26:05.528163+0000","last_peered":"2024-05-08T23:26:05.528163+0000","last_clean":"2024-05-08T23:26:05.528163+0000","last_became_active":"2024-05-08T23:26:03.907475+0000","last_became_peered":"2024-05-08T23:26:03.907475+0000","last_unstale":"2024-05-08T23:26:05.528163+0000","last_undegraded":"2024-05-08T23:26:05.528163+0000","last_fullsized":"2024-05-08T23:26:05.528163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:26.905129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528734+0000","last_change":"2024-05-08T23:26:04.383939+0000","last_active":"2024-05-08T23:26:05.528734+0000","last_peered":"2024-05-08T23:26:05.528734+0000","last_clean":"2024-05-08T23:26:05.528734+0000","last_became_active":"2024-05-08T23:26:04.383318+0000","last_became_peered":"2024-05-08T23:26:04.383318+0000","last_unstale":"2024-05-08T23:26:05.528734+0000","last_undegraded":"2024-05-08T23:26:05.528734+0000","last_fullsized":"2024-05-08T23:26:05.528734+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106789+0000","last_change":"2024-05-08T23:26:03.917152+0000","last_active":"2024-05-08T23:26:06.106789+0000","last_peered":"2024-05-08T23:26:06.106789+0000","last_clean":"2024-05-08T23:26:06.106789+0000","last_became_active":"2024-05-08T23:26:03.916848+0000","last_became_peered":"2024-05-08T23:26:03.916848+0000","last_unstale":"2024-05-08T23:26:06.106789+0000","last_undegraded":"2024-05-08T23:26:06.106789+0000","last_fullsized":"2024-05-08T23:26:06.106789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:17.395880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.386965+0000","last_change":"2024-05-08T23:26:04.377822+0000","last_active":"2024-05-08T23:26:04.386965+0000","last_peered":"2024-05-08T23:26:04.386965+0000","last_clean":"2024-05-08T23:26:04.386965+0000","last_became_active":"2024-05-08T23:26:04.377510+0000","last_became_peered":"2024-05-08T23:26:04.377510+0000","last_unstale":"2024-05-08T23:26:04.386965+0000","last_undegraded":"2024-05-08T23:26:04.386965+0000","last_fullsized":"2024-05-08T23:26:04.386965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:09.803388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367888+0000","last_change":"2024-05-08T23:26:03.904714+0000","last_active":"2024-05-08T23:26:04.367888+0000","last_peered":"2024-05-08T23:26:04.367888+0000","last_clean":"2024-05-08T23:26:04.367888+0000","last_became_active":"2024-05-08T23:26:03.904414+0000","last_became_peered":"2024-05-08T23:26:03.904414+0000","last_unstale":"2024-05-08T23:26:04.367888+0000","last_undegraded":"2024-05-08T23:26:04.367888+0000","last_fullsized":"2024-05-08T23:26:04.367888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:14.889713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391857+0000","last_change":"2024-05-08T23:26:04.388899+0000","last_active":"2024-05-08T23:26:04.391857+0000","last_peered":"2024-05-08T23:26:04.391857+0000","last_clean":"2024-05-08T23:26:04.391857+0000","last_became_active":"2024-05-08T23:26:04.388628+0000","last_became_peered":"2024-05-08T23:26:04.388628+0000","last_unstale":"2024-05-08T23:26:04.391857+0000","last_undegraded":"2024-05-08T23:26:04.391857+0000","last_fullsized":"2024-05-08T23:26:04.391857+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365794+0000","last_change":"2024-05-08T23:26:03.910274+0000","last_active":"2024-05-08T23:26:05.365794+0000","last_peered":"2024-05-08T23:26:05.365794+0000","last_clean":"2024-05-08T23:26:05.365794+0000","last_became_active":"2024-05-08T23:26:03.909977+0000","last_became_peered":"2024-05-08T23:26:03.909977+0000","last_unstale":"2024-05-08T23:26:05.365794+0000","last_undegraded":"2024-05-08T23:26:05.365794+0000","last_fullsized":"2024-05-08T23:26:05.365794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:09.127009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382218+0000","last_change":"2024-05-08T23:26:04.377703+0000","last_active":"2024-05-08T23:26:04.382218+0000","last_peered":"2024-05-08T23:26:04.382218+0000","last_clean":"2024-05-08T23:26:04.382218+0000","last_became_active":"2024-05-08T23:26:04.377184+0000","last_became_peered":"2024-05-08T23:26:04.377184+0000","last_unstale":"2024-05-08T23:26:04.382218+0000","last_undegraded":"2024-05-08T23:26:04.382218+0000","last_fullsized":"2024-05-08T23:26:04.382218+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528645+0000","last_change":"2024-05-08T23:26:03.910256+0000","last_active":"2024-05-08T23:26:05.528645+0000","last_peered":"2024-05-08T23:26:05.528645+0000","last_clean":"2024-05-08T23:26:05.528645+0000","last_became_active":"2024-05-08T23:26:03.909930+0000","last_became_peered":"2024-05-08T23:26:03.909930+0000","last_unstale":"2024-05-08T23:26:05.528645+0000","last_undegraded":"2024-05-08T23:26:05.528645+0000","last_fullsized":"2024-05-08T23:26:05.528645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:46.987230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366137+0000","last_change":"2024-05-08T23:26:04.390986+0000","last_active":"2024-05-08T23:26:05.366137+0000","last_peered":"2024-05-08T23:26:05.366137+0000","last_clean":"2024-05-08T23:26:05.366137+0000","last_became_active":"2024-05-08T23:26:04.390656+0000","last_became_peered":"2024-05-08T23:26:04.390656+0000","last_unstale":"2024-05-08T23:26:05.366137+0000","last_undegraded":"2024-05-08T23:26:05.366137+0000","last_fullsized":"2024-05-08T23:26:05.366137+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:26.611874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365238+0000","last_change":"2024-05-08T23:26:03.902695+0000","last_active":"2024-05-08T23:26:05.365238+0000","last_peered":"2024-05-08T23:26:05.365238+0000","last_clean":"2024-05-08T23:26:05.365238+0000","last_became_active":"2024-05-08T23:26:03.902382+0000","last_became_peered":"2024-05-08T23:26:03.902382+0000","last_unstale":"2024-05-08T23:26:05.365238+0000","last_undegraded":"2024-05-08T23:26:05.365238+0000","last_fullsized":"2024-05-08T23:26:05.365238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:08.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106322+0000","last_change":"2024-05-08T23:26:04.387935+0000","last_active":"2024-05-08T23:26:06.106322+0000","last_peered":"2024-05-08T23:26:06.106322+0000","last_clean":"2024-05-08T23:26:06.106322+0000","last_became_active":"2024-05-08T23:26:04.387678+0000","last_became_peered":"2024-05-08T23:26:04.387678+0000","last_unstale":"2024-05-08T23:26:06.106322+0000","last_undegraded":"2024-05-08T23:26:06.106322+0000","last_fullsized":"2024-05-08T23:26:06.106322+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:23.335791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364796+0000","last_change":"2024-05-08T23:26:03.916887+0000","last_active":"2024-05-08T23:26:05.364796+0000","last_peered":"2024-05-08T23:26:05.364796+0000","last_clean":"2024-05-08T23:26:05.364796+0000","last_became_active":"2024-05-08T23:26:03.916407+0000","last_became_peered":"2024-05-08T23:26:03.916407+0000","last_unstale":"2024-05-08T23:26:05.364796+0000","last_undegraded":"2024-05-08T23:26:05.364796+0000","last_fullsized":"2024-05-08T23:26:05.364796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:34.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365058+0000","last_change":"2024-05-08T23:26:04.389795+0000","last_active":"2024-05-08T23:26:05.365058+0000","last_peered":"2024-05-08T23:26:05.365058+0000","last_clean":"2024-05-08T23:26:05.365058+0000","last_became_active":"2024-05-08T23:26:04.389433+0000","last_became_peered":"2024-05-08T23:26:04.389433+0000","last_unstale":"2024-05-08T23:26:05.365058+0000","last_undegraded":"2024-05-08T23:26:05.365058+0000","last_fullsized":"2024-05-08T23:26:05.365058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:23.681716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106146+0000","last_change":"2024-05-08T23:26:04.390648+0000","last_active":"2024-05-08T23:26:06.106146+0000","last_peered":"2024-05-08T23:26:06.106146+0000","last_clean":"2024-05-08T23:26:06.106146+0000","last_became_active":"2024-05-08T23:26:04.390358+0000","last_became_peered":"2024-05-08T23:26:04.390358+0000","last_unstale":"2024-05-08T23:26:06.106146+0000","last_undegraded":"2024-05-08T23:26:06.106146+0000","last_fullsized":"2024-05-08T23:26:06.106146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:19.130856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106499+0000","last_change":"2024-05-08T23:26:03.908056+0000","last_active":"2024-05-08T23:26:06.106499+0000","last_peered":"2024-05-08T23:26:06.106499+0000","last_clean":"2024-05-08T23:26:06.106499+0000","last_became_active":"2024-05-08T23:26:03.907603+0000","last_became_peered":"2024-05-08T23:26:03.907603+0000","last_unstale":"2024-05-08T23:26:06.106499+0000","last_undegraded":"2024-05-08T23:26:06.106499+0000","last_fullsized":"2024-05-08T23:26:06.106499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:17.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364615+0000","last_change":"2024-05-08T23:26:04.384097+0000","last_active":"2024-05-08T23:26:05.364615+0000","last_peered":"2024-05-08T23:26:05.364615+0000","last_clean":"2024-05-08T23:26:05.364615+0000","last_became_active":"2024-05-08T23:26:04.383812+0000","last_became_peered":"2024-05-08T23:26:04.383812+0000","last_unstale":"2024-05-08T23:26:05.364615+0000","last_undegraded":"2024-05-08T23:26:05.364615+0000","last_fullsized":"2024-05-08T23:26:05.364615+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:08.068641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106387+0000","last_change":"2024-05-08T23:26:03.918580+0000","last_active":"2024-05-08T23:26:06.106387+0000","last_peered":"2024-05-08T23:26:06.106387+0000","last_clean":"2024-05-08T23:26:06.106387+0000","last_became_active":"2024-05-08T23:26:03.918308+0000","last_became_peered":"2024-05-08T23:26:03.918308+0000","last_unstale":"2024-05-08T23:26:06.106387+0000","last_undegraded":"2024-05-08T23:26:06.106387+0000","last_fullsized":"2024-05-08T23:26:06.106387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:22.679586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-05-08T23:26:03.912381+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:26:03.912381+0000","last_peered":"2024-05-08T23:26:03.912381+0000","last_clean":"2024-05-08T23:26:03.912381+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:26:03.912381+0000","last_undegraded":"2024-05-08T23:26:03.912381+0000","last_fullsized":"2024-05-08T23:26:03.912381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1466368,"data_stored":958984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85999999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.495},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67400000000000004,"5min":1.038,"15min":1.038},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76500000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76600000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69899999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73899999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82499999999999996,"5min":0.87,"15min":0.87},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.95799999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996}]},{"osd":2,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":1.115,"15min":1.115},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.436}]},{"osd":5,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76500000000000001,"5min":1.052,"15min":1.052},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84399999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":376480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.371,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56100000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.42299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":1.01,"15min":1.01},"last":0.432},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":1.028,"15min":1.028},"last":0.34200000000000003}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":1.105,"15min":1.105},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.54000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1470464,"data_stored":964668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48399999999999999,"15min":0.72599999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.52900000000000003,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":0.71399999999999997},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60799999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.51600000000000001,"15min":0.77800000000000002},"min":{"1min":0.312,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52900000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54800000000000004,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58099999999999996,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65400000000000003,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69099999999999995,"15min":0.93500000000000005},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.70399999999999996,"15min":0.90800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.91300000000000003},"min":{"1min":0.50700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75700000000000001,"15min":1.0069999999999999},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.73999999999999999}]},{"osd":6,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73499999999999999,"15min":0.97499999999999998},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81799999999999995,"5min":7.734,"15min":7.734},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.77800000000000002}]},{"osd":7,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.96399999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.92200000000000004},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.83299999999999996,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":375864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61799999999999999,"5min":0.621,"15min":0.621},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57399999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.441}]},{"osd":1,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.439,"15min":0.40999999999999998},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.64100000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42699999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79900000000000004,"5min":1.024,"15min":1.024},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81799999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.50700000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.159,"15min":1.159},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91900000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.622}]},{"osd":7,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.70999999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89800000000000002,"5min":1.194,"15min":1.194},"last":0.60199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39500000000000002,"15min":0.41699999999999998},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.58199999999999996,"5min":1.46,"15min":1.46},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.41099999999999998,"15min":0.42099999999999999},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52100000000000002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40100000000000002,"15min":0.40999999999999998},"min":{"1min":0.28999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45800000000000002,"15min":0.46999999999999997},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61199999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.66200000000000003,"15min":0.879},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.56399999999999995,"5min":11.964,"15min":11.964},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.503,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67300000000000004,"5min":1.593,"15min":1.593},"last":0.439}]},{"osd":4,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93200000000000005,"5min":1.375,"15min":1.375},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.624}]},{"osd":5,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.536,"15min":1.536},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.68600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.874,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":1.204,"15min":1.204},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.50700000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.878,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":1.339,"15min":1.339},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73199999999999998,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92800000000000005,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.72399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":349133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59899999999999998,"15min":0.65200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72899999999999998,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.63800000000000001,"15min":0.73499999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.78700000000000003,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.61499999999999999,"15min":0.67600000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.753,"5min":2.145,"15min":2.145},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.64100000000000001,"15min":0.69699999999999995},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.63600000000000001,"15min":0.69599999999999995},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.70299999999999996,"15min":0.80300000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.70999999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.88700000000000001,"15min":1.1719999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77500000000000002,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.5}]},{"osd":5,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55300000000000005,"15min":0.59099999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72899999999999998,"5min":0.877,"15min":0.877},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77800000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.38600000000000001}]},{"osd":6,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.63},"min":{"1min":0.34699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.66200000000000003,"15min":0.69099999999999995},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82599999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60199999999999998,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65400000000000003,"15min":0.70399999999999996},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82199999999999995,"5min":1.24,"15min":1.24},"last":0.45600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":368090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85199999999999998,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.433},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":0.89400000000000002},"min":{"1min":0.35199999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":8.798,"15min":8.798},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.77500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":4.71,"15min":4.71},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70699999999999996,"15min":0.96799999999999997},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0009999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.79100000000000004},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.82199999999999995,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.93000000000000005},"min":{"1min":0.46999999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78500000000000003,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73899999999999999,"15min":1.0109999999999999},"min":{"1min":0.51100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0489999999999999,"5min":10.427,"15min":10.427},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58599999999999997,"5min":0.874,"15min":0.874},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.502}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85299999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.01},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:26:08.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:26:08.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.784+0000 7f65e81e2700 1 Processor -- start 2024-05-08T23:26:08.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.784+0000 7f65e81e2700 1 -- start start 2024-05-08T23:26:08.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.785+0000 7f65e81e2700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f65e015db00 0x7f65e006dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.785+0000 7f65e81e2700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f65e00cb2a0 0x7f65e00cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.785+0000 7f65e81e2700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e006fbf0 con 0x7f65e00ce120 2024-05-08T23:26:08.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.785+0000 7f65e81e2700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e006edb0 con 0x7f65e015db00 2024-05-08T23:26:08.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.785+0000 7f65e81e2700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e006f090 con 0x7f65e00cb2a0 2024-05-08T23:26:08.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.785+0000 7f65e5f7e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f65e015db00 0x7f65e006dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.785+0000 7f65e577d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f65e00ce120 0x7f65e006e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38900/0 (socket says 172.21.15.5:38900) 2024-05-08T23:26:08.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.785+0000 7f65e577d700 1 -- 172.21.15.5:0/3860593297 learned_addr learned my addr 172.21.15.5:0/3860593297 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e677f700 1 --2- 172.21.15.5:0/3860593297 >> v2:172.21.15.49:3300/0 conn(0x7f65e00cb2a0 0x7f65e00cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e5f7e700 1 -- 172.21.15.5:0/3860593297 >> v2:172.21.15.49:3300/0 conn(0x7f65e00cb2a0 msgr2=0x7f65e00cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e5f7e700 1 --2- 172.21.15.5:0/3860593297 >> v2:172.21.15.49:3300/0 conn(0x7f65e00cb2a0 0x7f65e00cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e5f7e700 1 -- 172.21.15.5:0/3860593297 >> v1:172.21.15.5:6789/0 conn(0x7f65e00ce120 legacy=0x7f65e006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e5f7e700 1 -- 172.21.15.5:0/3860593297 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e006f370 con 0x7f65e015db00 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e4f7c700 1 -- 172.21.15.5:0/3860593297 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845923822 0 0) 0x7f65e006fbf0 con 0x7f65e00ce120 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e677f700 1 --2- 172.21.15.5:0/3860593297 >> v2:172.21.15.49:3300/0 conn(0x7f65e00cb2a0 0x7f65e00cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e5f7e700 1 --2- 172.21.15.5:0/3860593297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e006dc80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f65e00727d0 tx=0x7f65dc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd267f03602250b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e4f7c700 1 -- 172.21.15.5:0/3860593297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc011830 con 0x7f65e015db00 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e4f7c700 1 -- 172.21.15.5:0/3860593297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc011990 con 0x7f65e015db00 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.786+0000 7f65e4f7c700 1 -- 172.21.15.5:0/3860593297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc011d20 con 0x7f65e015db00 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.787+0000 7f65e81e2700 1 -- 172.21.15.5:0/3860593297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 msgr2=0x7f65e006dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.787+0000 7f65e81e2700 1 --2- 172.21.15.5:0/3860593297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e006dc80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f65e00727d0 tx=0x7f65dc009b20 comp rx=0 tx=0).stop 2024-05-08T23:26:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.787+0000 7f65e81e2700 1 -- 172.21.15.5:0/3860593297 shutdown_connections 2024-05-08T23:26:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.787+0000 7f65e81e2700 1 --2- 172.21.15.5:0/3860593297 >> v2:172.21.15.49:3300/0 conn(0x7f65e00cb2a0 0x7f65e00cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.787+0000 7f65e81e2700 1 --2- 172.21.15.5:0/3860593297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e006dc80 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.787+0000 7f65e81e2700 1 -- 172.21.15.5:0/3860593297 >> 172.21.15.5:0/3860593297 conn(0x7f65e0099a50 msgr2=0x7f65e00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.787+0000 7f65e81e2700 1 -- 172.21.15.5:0/3860593297 shutdown_connections 2024-05-08T23:26:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.787+0000 7f65e81e2700 1 -- 172.21.15.5:0/3860593297 wait complete. 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.788+0000 7f65e81e2700 1 Processor -- start 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.788+0000 7f65e81e2700 1 -- start start 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e81e2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f65e00cb2a0 0x7f65e0152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e81e2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65e00ce120 0x7f65e01541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e81e2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e00b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e81e2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00c8980 con 0x7f65e015db00 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e81e2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0153020 con 0x7f65e00ce120 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e81e2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00c8650 con 0x7f65e00cb2a0 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e677f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e00b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e677f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e00b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44798/0 (socket says 172.21.15.5:44798) 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e677f700 1 -- 172.21.15.5:0/2048668165 learned_addr learned my addr 172.21.15.5:0/2048668165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e577d700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65e00ce120 0x7f65e01541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.789+0000 7f65e5f7e700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f65e00cb2a0 0x7f65e0152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e677f700 1 -- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f65e00cb2a0 msgr2=0x7f65e0152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e677f700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f65e00cb2a0 0x7f65e0152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e677f700 1 -- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65e00ce120 msgr2=0x7f65e01541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e677f700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65e00ce120 0x7f65e01541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e677f700 1 -- 172.21.15.5:0/2048668165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0094a0 con 0x7f65e015db00 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e5f7e700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f65e00cb2a0 0x7f65e0152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e677f700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e00b0420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f65d8007df0 tx=0x7f65d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65d67fc700 1 -- 172.21.15.5:0/2048668165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d800ddd0 con 0x7f65e015db00 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65d67fc700 1 -- 172.21.15.5:0/2048668165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d800f040 con 0x7f65e015db00 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e0155270 con 0x7f65e015db00 2024-05-08T23:26:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65d67fc700 1 -- 172.21.15.5:0/2048668165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8003c60 con 0x7f65e015db00 2024-05-08T23:26:08.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.790+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e0155830 con 0x7f65e015db00 2024-05-08T23:26:08.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.791+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e00c8b30 con 0x7f65e015db00 2024-05-08T23:26:08.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.791+0000 7f65d67fc700 1 -- 172.21.15.5:0/2048668165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f65d8004030 con 0x7f65e015db00 2024-05-08T23:26:08.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.792+0000 7f65d67fc700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65c406ac90 0x7f65c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:08.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.792+0000 7f65d67fc700 1 -- 172.21.15.5:0/2048668165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65d80894f0 con 0x7f65e015db00 2024-05-08T23:26:08.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.792+0000 7f65e5f7e700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65c406ac90 0x7f65c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:08.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.793+0000 7f65e5f7e700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65c406ac90 0x7f65c406d150 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0090b0 tx=0x7f65dc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:08.797+0000 7f65d67fc700 1 -- 172.21.15.5:0/2048668165 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f65e00c8b30 con 0x7f65e015db00 2024-05-08T23:26:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.011+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65e015dee0 con 0x7f65c406ac90 2024-05-08T23:26:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.019+0000 7f65d67fc700 1 -- 172.21.15.5:0/2048668165 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287789 (secure 0 0 0) 0x7f65dc011b00 con 0x7f65c406ac90 2024-05-08T23:26:09.023 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65c406ac90 msgr2=0x7f65c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65c406ac90 0x7f65c406d150 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0090b0 tx=0x7f65dc008030 comp rx=0 tx=0).stop 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 msgr2=0x7f65e00b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e00b0420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f65d8007df0 tx=0x7f65d800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 shutdown_connections 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f65c406ac90 0x7f65c406d150 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65e015db00 0x7f65e00b0420 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f65e00cb2a0 0x7f65e0152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 --2- 172.21.15.5:0/2048668165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65e00ce120 0x7f65e01541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 >> 172.21.15.5:0/2048668165 conn(0x7f65e0099a50 msgr2=0x7f65e00c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 shutdown_connections 2024-05-08T23:26:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.023+0000 7f65e81e2700 1 -- 172.21.15.5:0/2048668165 wait complete. 2024-05-08T23:26:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:26:09.037 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-05-08T23:26:07.367069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":185,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":232368,"kb_used_data":7996,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754742352,"statfs":{"total":773094113280,"available":772856168448,"internally_reserved":0,"allocated":8187904,"data_stored":4142023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":229625397},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.475465"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.385435+0000","last_change":"2024-05-08T23:26:04.382812+0000","last_active":"2024-05-08T23:26:04.385435+0000","last_peered":"2024-05-08T23:26:04.385435+0000","last_clean":"2024-05-08T23:26:04.385435+0000","last_became_active":"2024-05-08T23:26:04.382518+0000","last_became_peered":"2024-05-08T23:26:04.382518+0000","last_unstale":"2024-05-08T23:26:04.385435+0000","last_undegraded":"2024-05-08T23:26:04.385435+0000","last_fullsized":"2024-05-08T23:26:04.385435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:16.459465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106783+0000","last_change":"2024-05-08T23:26:04.386202+0000","last_active":"2024-05-08T23:26:06.106783+0000","last_peered":"2024-05-08T23:26:06.106783+0000","last_clean":"2024-05-08T23:26:06.106783+0000","last_became_active":"2024-05-08T23:26:04.385897+0000","last_became_peered":"2024-05-08T23:26:04.385897+0000","last_unstale":"2024-05-08T23:26:06.106783+0000","last_undegraded":"2024-05-08T23:26:06.106783+0000","last_fullsized":"2024-05-08T23:26:06.106783+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:15.891844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365071+0000","last_change":"2024-05-08T23:26:04.376144+0000","last_active":"2024-05-08T23:26:05.365071+0000","last_peered":"2024-05-08T23:26:05.365071+0000","last_clean":"2024-05-08T23:26:05.365071+0000","last_became_active":"2024-05-08T23:26:04.375870+0000","last_became_peered":"2024-05-08T23:26:04.375870+0000","last_unstale":"2024-05-08T23:26:05.365071+0000","last_undegraded":"2024-05-08T23:26:05.365071+0000","last_fullsized":"2024-05-08T23:26:05.365071+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:53.570674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365640+0000","last_change":"2024-05-08T23:26:04.391205+0000","last_active":"2024-05-08T23:26:05.365640+0000","last_peered":"2024-05-08T23:26:05.365640+0000","last_clean":"2024-05-08T23:26:05.365640+0000","last_became_active":"2024-05-08T23:26:04.390895+0000","last_became_peered":"2024-05-08T23:26:04.390895+0000","last_unstale":"2024-05-08T23:26:05.365640+0000","last_undegraded":"2024-05-08T23:26:05.365640+0000","last_fullsized":"2024-05-08T23:26:05.365640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:00.837462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365140+0000","last_change":"2024-05-08T23:26:04.380499+0000","last_active":"2024-05-08T23:26:05.365140+0000","last_peered":"2024-05-08T23:26:05.365140+0000","last_clean":"2024-05-08T23:26:05.365140+0000","last_became_active":"2024-05-08T23:26:04.380163+0000","last_became_peered":"2024-05-08T23:26:04.380163+0000","last_unstale":"2024-05-08T23:26:05.365140+0000","last_undegraded":"2024-05-08T23:26:05.365140+0000","last_fullsized":"2024-05-08T23:26:05.365140+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:39.534202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.392388+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.392064+0000","last_became_peered":"2024-05-08T23:26:04.392064+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:55.698089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.387389+0000","last_change":"2024-05-08T23:26:04.377831+0000","last_active":"2024-05-08T23:26:04.387389+0000","last_peered":"2024-05-08T23:26:04.387389+0000","last_clean":"2024-05-08T23:26:04.387389+0000","last_became_active":"2024-05-08T23:26:04.377518+0000","last_became_peered":"2024-05-08T23:26:04.377518+0000","last_unstale":"2024-05-08T23:26:04.387389+0000","last_undegraded":"2024-05-08T23:26:04.387389+0000","last_fullsized":"2024-05-08T23:26:04.387389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106353+0000","last_change":"2024-05-08T23:26:04.388002+0000","last_active":"2024-05-08T23:26:06.106353+0000","last_peered":"2024-05-08T23:26:06.106353+0000","last_clean":"2024-05-08T23:26:06.106353+0000","last_became_active":"2024-05-08T23:26:04.386006+0000","last_became_peered":"2024-05-08T23:26:04.386006+0000","last_unstale":"2024-05-08T23:26:06.106353+0000","last_undegraded":"2024-05-08T23:26:06.106353+0000","last_fullsized":"2024-05-08T23:26:06.106353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:34:35.709822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365561+0000","last_change":"2024-05-08T23:26:04.389861+0000","last_active":"2024-05-08T23:26:05.365561+0000","last_peered":"2024-05-08T23:26:05.365561+0000","last_clean":"2024-05-08T23:26:05.365561+0000","last_became_active":"2024-05-08T23:26:04.389582+0000","last_became_peered":"2024-05-08T23:26:04.389582+0000","last_unstale":"2024-05-08T23:26:05.365561+0000","last_undegraded":"2024-05-08T23:26:05.365561+0000","last_fullsized":"2024-05-08T23:26:05.365561+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:12.594073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391883+0000","last_change":"2024-05-08T23:26:04.390913+0000","last_active":"2024-05-08T23:26:04.391883+0000","last_peered":"2024-05-08T23:26:04.391883+0000","last_clean":"2024-05-08T23:26:04.391883+0000","last_became_active":"2024-05-08T23:26:04.390654+0000","last_became_peered":"2024-05-08T23:26:04.390654+0000","last_unstale":"2024-05-08T23:26:04.391883+0000","last_undegraded":"2024-05-08T23:26:04.391883+0000","last_fullsized":"2024-05-08T23:26:04.391883+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365672+0000","last_change":"2024-05-08T23:26:04.382718+0000","last_active":"2024-05-08T23:26:05.365672+0000","last_peered":"2024-05-08T23:26:05.365672+0000","last_clean":"2024-05-08T23:26:05.365672+0000","last_became_active":"2024-05-08T23:26:04.381951+0000","last_became_peered":"2024-05-08T23:26:04.381951+0000","last_unstale":"2024-05-08T23:26:05.365672+0000","last_undegraded":"2024-05-08T23:26:05.365672+0000","last_fullsized":"2024-05-08T23:26:05.365672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:18.398011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528368+0000","last_change":"2024-05-08T23:26:04.393225+0000","last_active":"2024-05-08T23:26:05.528368+0000","last_peered":"2024-05-08T23:26:05.528368+0000","last_clean":"2024-05-08T23:26:05.528368+0000","last_became_active":"2024-05-08T23:26:04.392933+0000","last_became_peered":"2024-05-08T23:26:04.392933+0000","last_unstale":"2024-05-08T23:26:05.528368+0000","last_undegraded":"2024-05-08T23:26:05.528368+0000","last_fullsized":"2024-05-08T23:26:05.528368+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366036+0000","last_change":"2024-05-08T23:26:04.380528+0000","last_active":"2024-05-08T23:26:05.366036+0000","last_peered":"2024-05-08T23:26:05.366036+0000","last_clean":"2024-05-08T23:26:05.366036+0000","last_became_active":"2024-05-08T23:26:04.380129+0000","last_became_peered":"2024-05-08T23:26:04.380129+0000","last_unstale":"2024-05-08T23:26:05.366036+0000","last_undegraded":"2024-05-08T23:26:05.366036+0000","last_fullsized":"2024-05-08T23:26:05.366036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.038766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528231+0000","last_change":"2024-05-08T23:26:04.382434+0000","last_active":"2024-05-08T23:26:05.528231+0000","last_peered":"2024-05-08T23:26:05.528231+0000","last_clean":"2024-05-08T23:26:05.528231+0000","last_became_active":"2024-05-08T23:26:04.382015+0000","last_became_peered":"2024-05-08T23:26:04.382015+0000","last_unstale":"2024-05-08T23:26:05.528231+0000","last_undegraded":"2024-05-08T23:26:05.528231+0000","last_fullsized":"2024-05-08T23:26:05.528231+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365565+0000","last_change":"2024-05-08T23:26:04.386467+0000","last_active":"2024-05-08T23:26:05.365565+0000","last_peered":"2024-05-08T23:26:05.365565+0000","last_clean":"2024-05-08T23:26:05.365565+0000","last_became_active":"2024-05-08T23:26:04.386005+0000","last_became_peered":"2024-05-08T23:26:04.386005+0000","last_unstale":"2024-05-08T23:26:05.365565+0000","last_undegraded":"2024-05-08T23:26:05.365565+0000","last_fullsized":"2024-05-08T23:26:05.365565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:08.141995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365004+0000","last_change":"2024-05-08T23:26:04.384544+0000","last_active":"2024-05-08T23:26:05.365004+0000","last_peered":"2024-05-08T23:26:05.365004+0000","last_clean":"2024-05-08T23:26:05.365004+0000","last_became_active":"2024-05-08T23:26:04.384240+0000","last_became_peered":"2024-05-08T23:26:04.384240+0000","last_unstale":"2024-05-08T23:26:05.365004+0000","last_undegraded":"2024-05-08T23:26:05.365004+0000","last_fullsized":"2024-05-08T23:26:05.365004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:44.177271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106649+0000","last_change":"2024-05-08T23:26:04.387389+0000","last_active":"2024-05-08T23:26:06.106649+0000","last_peered":"2024-05-08T23:26:06.106649+0000","last_clean":"2024-05-08T23:26:06.106649+0000","last_became_active":"2024-05-08T23:26:04.383192+0000","last_became_peered":"2024-05-08T23:26:04.383192+0000","last_unstale":"2024-05-08T23:26:06.106649+0000","last_undegraded":"2024-05-08T23:26:06.106649+0000","last_fullsized":"2024-05-08T23:26:06.106649+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528319+0000","last_change":"2024-05-08T23:26:04.374428+0000","last_active":"2024-05-08T23:26:05.528319+0000","last_peered":"2024-05-08T23:26:05.528319+0000","last_clean":"2024-05-08T23:26:05.528319+0000","last_became_active":"2024-05-08T23:26:04.374165+0000","last_became_peered":"2024-05-08T23:26:04.374165+0000","last_unstale":"2024-05-08T23:26:05.528319+0000","last_undegraded":"2024-05-08T23:26:05.528319+0000","last_fullsized":"2024-05-08T23:26:05.528319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382847+0000","last_change":"2024-05-08T23:26:04.380605+0000","last_active":"2024-05-08T23:26:04.382847+0000","last_peered":"2024-05-08T23:26:04.382847+0000","last_clean":"2024-05-08T23:26:04.382847+0000","last_became_active":"2024-05-08T23:26:04.380307+0000","last_became_peered":"2024-05-08T23:26:04.380307+0000","last_unstale":"2024-05-08T23:26:04.382847+0000","last_undegraded":"2024-05-08T23:26:04.382847+0000","last_fullsized":"2024-05-08T23:26:04.382847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:27.367580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.395921+0000","last_change":"2024-05-08T23:26:04.395150+0000","last_active":"2024-05-08T23:26:04.395921+0000","last_peered":"2024-05-08T23:26:04.395921+0000","last_clean":"2024-05-08T23:26:04.395921+0000","last_became_active":"2024-05-08T23:26:04.394845+0000","last_became_peered":"2024-05-08T23:26:04.394845+0000","last_unstale":"2024-05-08T23:26:04.395921+0000","last_undegraded":"2024-05-08T23:26:04.395921+0000","last_fullsized":"2024-05-08T23:26:04.395921+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.390865+0000","last_change":"2024-05-08T23:26:04.387174+0000","last_active":"2024-05-08T23:26:04.390865+0000","last_peered":"2024-05-08T23:26:04.390865+0000","last_clean":"2024-05-08T23:26:04.390865+0000","last_became_active":"2024-05-08T23:26:04.386653+0000","last_became_peered":"2024-05-08T23:26:04.386653+0000","last_unstale":"2024-05-08T23:26:04.390865+0000","last_undegraded":"2024-05-08T23:26:04.390865+0000","last_fullsized":"2024-05-08T23:26:04.390865+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:52.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365110+0000","last_change":"2024-05-08T23:26:04.374586+0000","last_active":"2024-05-08T23:26:05.365110+0000","last_peered":"2024-05-08T23:26:05.365110+0000","last_clean":"2024-05-08T23:26:05.365110+0000","last_became_active":"2024-05-08T23:26:04.374382+0000","last_became_peered":"2024-05-08T23:26:04.374382+0000","last_unstale":"2024-05-08T23:26:05.365110+0000","last_undegraded":"2024-05-08T23:26:05.365110+0000","last_fullsized":"2024-05-08T23:26:05.365110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:41.989450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364727+0000","last_change":"2024-05-08T23:26:04.376502+0000","last_active":"2024-05-08T23:26:05.364727+0000","last_peered":"2024-05-08T23:26:05.364727+0000","last_clean":"2024-05-08T23:26:05.364727+0000","last_became_active":"2024-05-08T23:26:04.376226+0000","last_became_peered":"2024-05-08T23:26:04.376226+0000","last_unstale":"2024-05-08T23:26:05.364727+0000","last_undegraded":"2024-05-08T23:26:05.364727+0000","last_fullsized":"2024-05-08T23:26:05.364727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:36.452021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.393993+0000","last_change":"2024-05-08T23:26:04.393135+0000","last_active":"2024-05-08T23:26:04.393993+0000","last_peered":"2024-05-08T23:26:04.393993+0000","last_clean":"2024-05-08T23:26:04.393993+0000","last_became_active":"2024-05-08T23:26:04.392824+0000","last_became_peered":"2024-05-08T23:26:04.392824+0000","last_unstale":"2024-05-08T23:26:04.393993+0000","last_undegraded":"2024-05-08T23:26:04.393993+0000","last_fullsized":"2024-05-08T23:26:04.393993+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:47.989361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391112+0000","last_change":"2024-05-08T23:26:04.387882+0000","last_active":"2024-05-08T23:26:04.391112+0000","last_peered":"2024-05-08T23:26:04.391112+0000","last_clean":"2024-05-08T23:26:04.391112+0000","last_became_active":"2024-05-08T23:26:04.387600+0000","last_became_peered":"2024-05-08T23:26:04.387600+0000","last_unstale":"2024-05-08T23:26:04.391112+0000","last_undegraded":"2024-05-08T23:26:04.391112+0000","last_fullsized":"2024-05-08T23:26:04.391112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:29.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106297+0000","last_change":"2024-05-08T23:26:04.390849+0000","last_active":"2024-05-08T23:26:06.106297+0000","last_peered":"2024-05-08T23:26:06.106297+0000","last_clean":"2024-05-08T23:26:06.106297+0000","last_became_active":"2024-05-08T23:26:04.390577+0000","last_became_peered":"2024-05-08T23:26:04.390577+0000","last_unstale":"2024-05-08T23:26:06.106297+0000","last_undegraded":"2024-05-08T23:26:06.106297+0000","last_fullsized":"2024-05-08T23:26:06.106297+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:00.857944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389116+0000","last_change":"2024-05-08T23:26:04.387838+0000","last_active":"2024-05-08T23:26:04.389116+0000","last_peered":"2024-05-08T23:26:04.389116+0000","last_clean":"2024-05-08T23:26:04.389116+0000","last_became_active":"2024-05-08T23:26:04.387594+0000","last_became_peered":"2024-05-08T23:26:04.387594+0000","last_unstale":"2024-05-08T23:26:04.389116+0000","last_undegraded":"2024-05-08T23:26:04.389116+0000","last_fullsized":"2024-05-08T23:26:04.389116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389890+0000","last_change":"2024-05-08T23:26:03.918800+0000","last_active":"2024-05-08T23:26:04.389890+0000","last_peered":"2024-05-08T23:26:04.389890+0000","last_clean":"2024-05-08T23:26:04.389890+0000","last_became_active":"2024-05-08T23:26:03.918509+0000","last_became_peered":"2024-05-08T23:26:03.918509+0000","last_unstale":"2024-05-08T23:26:04.389890+0000","last_undegraded":"2024-05-08T23:26:04.389890+0000","last_fullsized":"2024-05-08T23:26:04.389890+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:46.488568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365685+0000","last_change":"2024-05-08T23:26:04.386283+0000","last_active":"2024-05-08T23:26:05.365685+0000","last_peered":"2024-05-08T23:26:05.365685+0000","last_clean":"2024-05-08T23:26:05.365685+0000","last_became_active":"2024-05-08T23:26:04.385690+0000","last_became_peered":"2024-05-08T23:26:04.385690+0000","last_unstale":"2024-05-08T23:26:05.365685+0000","last_undegraded":"2024-05-08T23:26:05.365685+0000","last_fullsized":"2024-05-08T23:26:05.365685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:12.667029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528769+0000","last_change":"2024-05-08T23:26:03.912463+0000","last_active":"2024-05-08T23:26:05.528769+0000","last_peered":"2024-05-08T23:26:05.528769+0000","last_clean":"2024-05-08T23:26:05.528769+0000","last_became_active":"2024-05-08T23:26:03.912178+0000","last_became_peered":"2024-05-08T23:26:03.912178+0000","last_unstale":"2024-05-08T23:26:05.528769+0000","last_undegraded":"2024-05-08T23:26:05.528769+0000","last_fullsized":"2024-05-08T23:26:05.528769+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:02.228610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.385614+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.385325+0000","last_became_peered":"2024-05-08T23:26:04.385325+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.979369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382824+0000","last_change":"2024-05-08T23:26:04.380473+0000","last_active":"2024-05-08T23:26:04.382824+0000","last_peered":"2024-05-08T23:26:04.382824+0000","last_clean":"2024-05-08T23:26:04.382824+0000","last_became_active":"2024-05-08T23:26:04.380170+0000","last_became_peered":"2024-05-08T23:26:04.380170+0000","last_unstale":"2024-05-08T23:26:04.382824+0000","last_undegraded":"2024-05-08T23:26:04.382824+0000","last_fullsized":"2024-05-08T23:26:04.382824+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:30.884597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106822+0000","last_change":"2024-05-08T23:26:03.918662+0000","last_active":"2024-05-08T23:26:06.106822+0000","last_peered":"2024-05-08T23:26:06.106822+0000","last_clean":"2024-05-08T23:26:06.106822+0000","last_became_active":"2024-05-08T23:26:03.918367+0000","last_became_peered":"2024-05-08T23:26:03.918367+0000","last_unstale":"2024-05-08T23:26:06.106822+0000","last_undegraded":"2024-05-08T23:26:06.106822+0000","last_fullsized":"2024-05-08T23:26:06.106822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.392305+0000","last_change":"2024-05-08T23:26:04.391595+0000","last_active":"2024-05-08T23:26:04.392305+0000","last_peered":"2024-05-08T23:26:04.392305+0000","last_clean":"2024-05-08T23:26:04.392305+0000","last_became_active":"2024-05-08T23:26:04.391291+0000","last_became_peered":"2024-05-08T23:26:04.391291+0000","last_unstale":"2024-05-08T23:26:04.392305+0000","last_undegraded":"2024-05-08T23:26:04.392305+0000","last_fullsized":"2024-05-08T23:26:04.392305+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:17.695615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364906+0000","last_change":"2024-05-08T23:26:03.916776+0000","last_active":"2024-05-08T23:26:05.364906+0000","last_peered":"2024-05-08T23:26:05.364906+0000","last_clean":"2024-05-08T23:26:05.364906+0000","last_became_active":"2024-05-08T23:26:03.916402+0000","last_became_peered":"2024-05-08T23:26:03.916402+0000","last_unstale":"2024-05-08T23:26:05.364906+0000","last_undegraded":"2024-05-08T23:26:05.364906+0000","last_fullsized":"2024-05-08T23:26:05.364906+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:04.288000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106261+0000","last_change":"2024-05-08T23:26:03.915034+0000","last_active":"2024-05-08T23:26:06.106261+0000","last_peered":"2024-05-08T23:26:06.106261+0000","last_clean":"2024-05-08T23:26:06.106261+0000","last_became_active":"2024-05-08T23:26:03.914666+0000","last_became_peered":"2024-05-08T23:26:03.914666+0000","last_unstale":"2024-05-08T23:26:06.106261+0000","last_undegraded":"2024-05-08T23:26:06.106261+0000","last_fullsized":"2024-05-08T23:26:06.106261+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:43.175140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365034+0000","last_change":"2024-05-08T23:26:04.381034+0000","last_active":"2024-05-08T23:26:05.365034+0000","last_peered":"2024-05-08T23:26:05.365034+0000","last_clean":"2024-05-08T23:26:05.365034+0000","last_became_active":"2024-05-08T23:26:04.380769+0000","last_became_peered":"2024-05-08T23:26:04.380769+0000","last_unstale":"2024-05-08T23:26:05.365034+0000","last_undegraded":"2024-05-08T23:26:05.365034+0000","last_fullsized":"2024-05-08T23:26:05.365034+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:35.530781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365301+0000","last_change":"2024-05-08T23:26:03.917031+0000","last_active":"2024-05-08T23:26:05.365301+0000","last_peered":"2024-05-08T23:26:05.365301+0000","last_clean":"2024-05-08T23:26:05.365301+0000","last_became_active":"2024-05-08T23:26:03.916394+0000","last_became_peered":"2024-05-08T23:26:03.916394+0000","last_unstale":"2024-05-08T23:26:05.365301+0000","last_undegraded":"2024-05-08T23:26:05.365301+0000","last_fullsized":"2024-05-08T23:26:05.365301+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:38.532072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528242+0000","last_change":"2024-05-08T23:26:04.374371+0000","last_active":"2024-05-08T23:26:05.528242+0000","last_peered":"2024-05-08T23:26:05.528242+0000","last_clean":"2024-05-08T23:26:05.528242+0000","last_became_active":"2024-05-08T23:26:04.374097+0000","last_became_peered":"2024-05-08T23:26:04.374097+0000","last_unstale":"2024-05-08T23:26:05.528242+0000","last_undegraded":"2024-05-08T23:26:05.528242+0000","last_fullsized":"2024-05-08T23:26:05.528242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365139+0000","last_change":"2024-05-08T23:26:03.911594+0000","last_active":"2024-05-08T23:26:05.365139+0000","last_peered":"2024-05-08T23:26:05.365139+0000","last_clean":"2024-05-08T23:26:05.365139+0000","last_became_active":"2024-05-08T23:26:03.911273+0000","last_became_peered":"2024-05-08T23:26:03.911273+0000","last_unstale":"2024-05-08T23:26:05.365139+0000","last_undegraded":"2024-05-08T23:26:05.365139+0000","last_fullsized":"2024-05-08T23:26:05.365139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:30.134959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.373540+0000","last_change":"2024-05-08T23:26:04.372794+0000","last_active":"2024-05-08T23:26:04.373540+0000","last_peered":"2024-05-08T23:26:04.373540+0000","last_clean":"2024-05-08T23:26:04.373540+0000","last_became_active":"2024-05-08T23:26:04.372536+0000","last_became_peered":"2024-05-08T23:26:04.372536+0000","last_unstale":"2024-05-08T23:26:04.373540+0000","last_undegraded":"2024-05-08T23:26:04.373540+0000","last_fullsized":"2024-05-08T23:26:04.373540+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:06.816807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391925+0000","last_change":"2024-05-08T23:26:04.388857+0000","last_active":"2024-05-08T23:26:04.391925+0000","last_peered":"2024-05-08T23:26:04.391925+0000","last_clean":"2024-05-08T23:26:04.391925+0000","last_became_active":"2024-05-08T23:26:04.388581+0000","last_became_peered":"2024-05-08T23:26:04.388581+0000","last_unstale":"2024-05-08T23:26:04.391925+0000","last_undegraded":"2024-05-08T23:26:04.391925+0000","last_fullsized":"2024-05-08T23:26:04.391925+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106865+0000","last_change":"2024-05-08T23:26:03.907421+0000","last_active":"2024-05-08T23:26:06.106865+0000","last_peered":"2024-05-08T23:26:06.106865+0000","last_clean":"2024-05-08T23:26:06.106865+0000","last_became_active":"2024-05-08T23:26:03.907143+0000","last_became_peered":"2024-05-08T23:26:03.907143+0000","last_unstale":"2024-05-08T23:26:06.106865+0000","last_undegraded":"2024-05-08T23:26:06.106865+0000","last_fullsized":"2024-05-08T23:26:06.106865+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:07.208552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106627+0000","last_change":"2024-05-08T23:26:04.386708+0000","last_active":"2024-05-08T23:26:06.106627+0000","last_peered":"2024-05-08T23:26:06.106627+0000","last_clean":"2024-05-08T23:26:06.106627+0000","last_became_active":"2024-05-08T23:26:04.385984+0000","last_became_peered":"2024-05-08T23:26:04.385984+0000","last_unstale":"2024-05-08T23:26:06.106627+0000","last_undegraded":"2024-05-08T23:26:06.106627+0000","last_fullsized":"2024-05-08T23:26:06.106627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:47.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365174+0000","last_change":"2024-05-08T23:26:03.913646+0000","last_active":"2024-05-08T23:26:05.365174+0000","last_peered":"2024-05-08T23:26:05.365174+0000","last_clean":"2024-05-08T23:26:05.365174+0000","last_became_active":"2024-05-08T23:26:03.913208+0000","last_became_peered":"2024-05-08T23:26:03.913208+0000","last_unstale":"2024-05-08T23:26:05.365174+0000","last_undegraded":"2024-05-08T23:26:05.365174+0000","last_fullsized":"2024-05-08T23:26:05.365174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:02.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365662+0000","last_change":"2024-05-08T23:26:04.375029+0000","last_active":"2024-05-08T23:26:05.365662+0000","last_peered":"2024-05-08T23:26:05.365662+0000","last_clean":"2024-05-08T23:26:05.365662+0000","last_became_active":"2024-05-08T23:26:04.374797+0000","last_became_peered":"2024-05-08T23:26:04.374797+0000","last_unstale":"2024-05-08T23:26:05.365662+0000","last_undegraded":"2024-05-08T23:26:05.365662+0000","last_fullsized":"2024-05-08T23:26:05.365662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:18.355499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365807+0000","last_change":"2024-05-08T23:26:03.913239+0000","last_active":"2024-05-08T23:26:05.365807+0000","last_peered":"2024-05-08T23:26:05.365807+0000","last_clean":"2024-05-08T23:26:05.365807+0000","last_became_active":"2024-05-08T23:26:03.912892+0000","last_became_peered":"2024-05-08T23:26:03.912892+0000","last_unstale":"2024-05-08T23:26:05.365807+0000","last_undegraded":"2024-05-08T23:26:05.365807+0000","last_fullsized":"2024-05-08T23:26:05.365807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:28.036636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.385254+0000","last_change":"2024-05-08T23:26:04.381564+0000","last_active":"2024-05-08T23:26:04.385254+0000","last_peered":"2024-05-08T23:26:04.385254+0000","last_clean":"2024-05-08T23:26:04.385254+0000","last_became_active":"2024-05-08T23:26:04.381293+0000","last_became_peered":"2024-05-08T23:26:04.381293+0000","last_unstale":"2024-05-08T23:26:04.385254+0000","last_undegraded":"2024-05-08T23:26:04.385254+0000","last_fullsized":"2024-05-08T23:26:04.385254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365371+0000","last_change":"2024-05-08T23:26:03.917272+0000","last_active":"2024-05-08T23:26:05.365371+0000","last_peered":"2024-05-08T23:26:05.365371+0000","last_clean":"2024-05-08T23:26:05.365371+0000","last_became_active":"2024-05-08T23:26:03.916960+0000","last_became_peered":"2024-05-08T23:26:03.916960+0000","last_unstale":"2024-05-08T23:26:05.365371+0000","last_undegraded":"2024-05-08T23:26:05.365371+0000","last_fullsized":"2024-05-08T23:26:05.365371+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.380000+0000","last_change":"2024-05-08T23:26:04.375997+0000","last_active":"2024-05-08T23:26:04.380000+0000","last_peered":"2024-05-08T23:26:04.380000+0000","last_clean":"2024-05-08T23:26:04.380000+0000","last_became_active":"2024-05-08T23:26:04.375654+0000","last_became_peered":"2024-05-08T23:26:04.375654+0000","last_unstale":"2024-05-08T23:26:04.380000+0000","last_undegraded":"2024-05-08T23:26:04.380000+0000","last_fullsized":"2024-05-08T23:26:04.380000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:03.230741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106390+0000","last_change":"2024-05-08T23:26:03.911788+0000","last_active":"2024-05-08T23:26:06.106390+0000","last_peered":"2024-05-08T23:26:06.106390+0000","last_clean":"2024-05-08T23:26:06.106390+0000","last_became_active":"2024-05-08T23:26:03.911527+0000","last_became_peered":"2024-05-08T23:26:03.911527+0000","last_unstale":"2024-05-08T23:26:06.106390+0000","last_undegraded":"2024-05-08T23:26:06.106390+0000","last_fullsized":"2024-05-08T23:26:06.106390+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:44.784641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528178+0000","last_change":"2024-05-08T23:26:04.394848+0000","last_active":"2024-05-08T23:26:05.528178+0000","last_peered":"2024-05-08T23:26:05.528178+0000","last_clean":"2024-05-08T23:26:05.528178+0000","last_became_active":"2024-05-08T23:26:04.394520+0000","last_became_peered":"2024-05-08T23:26:04.394520+0000","last_unstale":"2024-05-08T23:26:05.528178+0000","last_undegraded":"2024-05-08T23:26:05.528178+0000","last_fullsized":"2024-05-08T23:26:05.528178+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367381+0000","last_change":"2024-05-08T23:26:03.904993+0000","last_active":"2024-05-08T23:26:04.367381+0000","last_peered":"2024-05-08T23:26:04.367381+0000","last_clean":"2024-05-08T23:26:04.367381+0000","last_became_active":"2024-05-08T23:26:03.904689+0000","last_became_peered":"2024-05-08T23:26:03.904689+0000","last_unstale":"2024-05-08T23:26:04.367381+0000","last_undegraded":"2024-05-08T23:26:04.367381+0000","last_fullsized":"2024-05-08T23:26:04.367381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:54.695959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365441+0000","last_change":"2024-05-08T23:26:04.386132+0000","last_active":"2024-05-08T23:26:05.365441+0000","last_peered":"2024-05-08T23:26:05.365441+0000","last_clean":"2024-05-08T23:26:05.365441+0000","last_became_active":"2024-05-08T23:26:04.385842+0000","last_became_peered":"2024-05-08T23:26:04.385842+0000","last_unstale":"2024-05-08T23:26:05.365441+0000","last_undegraded":"2024-05-08T23:26:05.365441+0000","last_fullsized":"2024-05-08T23:26:05.365441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:42.318612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:03.890624+0000","last_change":"2024-05-08T23:26:03.890624+0000","last_active":"2024-05-08T23:26:03.890624+0000","last_peered":"2024-05-08T23:26:03.890624+0000","last_clean":"2024-05-08T23:26:03.890624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:03.890624+0000","last_undegraded":"2024-05-08T23:26:03.890624+0000","last_fullsized":"2024-05-08T23:26:03.890624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106703+0000","last_change":"2024-05-08T23:26:03.917170+0000","last_active":"2024-05-08T23:26:06.106703+0000","last_peered":"2024-05-08T23:26:06.106703+0000","last_clean":"2024-05-08T23:26:06.106703+0000","last_became_active":"2024-05-08T23:26:03.916888+0000","last_became_peered":"2024-05-08T23:26:03.916888+0000","last_unstale":"2024-05-08T23:26:06.106703+0000","last_undegraded":"2024-05-08T23:26:06.106703+0000","last_fullsized":"2024-05-08T23:26:06.106703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:59.835332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.388091+0000","last_change":"2024-05-08T23:26:04.385163+0000","last_active":"2024-05-08T23:26:04.388091+0000","last_peered":"2024-05-08T23:26:04.388091+0000","last_clean":"2024-05-08T23:26:04.388091+0000","last_became_active":"2024-05-08T23:26:04.384430+0000","last_became_peered":"2024-05-08T23:26:04.384430+0000","last_unstale":"2024-05-08T23:26:04.388091+0000","last_undegraded":"2024-05-08T23:26:04.388091+0000","last_fullsized":"2024-05-08T23:26:04.388091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:13.482666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365760+0000","last_change":"2024-05-08T23:26:04.382870+0000","last_active":"2024-05-08T23:26:05.365760+0000","last_peered":"2024-05-08T23:26:05.365760+0000","last_clean":"2024-05-08T23:26:05.365760+0000","last_became_active":"2024-05-08T23:26:04.382320+0000","last_became_peered":"2024-05-08T23:26:04.382320+0000","last_unstale":"2024-05-08T23:26:05.365760+0000","last_undegraded":"2024-05-08T23:26:05.365760+0000","last_fullsized":"2024-05-08T23:26:05.365760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:17.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106864+0000","last_change":"2024-05-08T23:26:03.910712+0000","last_active":"2024-05-08T23:26:06.106864+0000","last_peered":"2024-05-08T23:26:06.106864+0000","last_clean":"2024-05-08T23:26:06.106864+0000","last_became_active":"2024-05-08T23:26:03.910424+0000","last_became_peered":"2024-05-08T23:26:03.910424+0000","last_unstale":"2024-05-08T23:26:06.106864+0000","last_undegraded":"2024-05-08T23:26:06.106864+0000","last_fullsized":"2024-05-08T23:26:06.106864+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:16.285704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365799+0000","last_change":"2024-05-08T23:26:04.378589+0000","last_active":"2024-05-08T23:26:05.365799+0000","last_peered":"2024-05-08T23:26:05.365799+0000","last_clean":"2024-05-08T23:26:05.365799+0000","last_became_active":"2024-05-08T23:26:04.378300+0000","last_became_peered":"2024-05-08T23:26:04.378300+0000","last_unstale":"2024-05-08T23:26:05.365799+0000","last_undegraded":"2024-05-08T23:26:05.365799+0000","last_fullsized":"2024-05-08T23:26:05.365799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.786772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.368459+0000","last_change":"2024-05-08T23:26:03.916915+0000","last_active":"2024-05-08T23:26:04.368459+0000","last_peered":"2024-05-08T23:26:04.368459+0000","last_clean":"2024-05-08T23:26:04.368459+0000","last_became_active":"2024-05-08T23:26:03.916632+0000","last_became_peered":"2024-05-08T23:26:03.916632+0000","last_unstale":"2024-05-08T23:26:04.368459+0000","last_undegraded":"2024-05-08T23:26:04.368459+0000","last_fullsized":"2024-05-08T23:26:04.368459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:22.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364817+0000","last_change":"2024-05-08T23:26:04.375139+0000","last_active":"2024-05-08T23:26:05.364817+0000","last_peered":"2024-05-08T23:26:05.364817+0000","last_clean":"2024-05-08T23:26:05.364817+0000","last_became_active":"2024-05-08T23:26:04.374874+0000","last_became_peered":"2024-05-08T23:26:04.374874+0000","last_unstale":"2024-05-08T23:26:05.364817+0000","last_undegraded":"2024-05-08T23:26:05.364817+0000","last_fullsized":"2024-05-08T23:26:05.364817+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:42.364785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106533+0000","last_change":"2024-05-08T23:26:03.915036+0000","last_active":"2024-05-08T23:26:06.106533+0000","last_peered":"2024-05-08T23:26:06.106533+0000","last_clean":"2024-05-08T23:26:06.106533+0000","last_became_active":"2024-05-08T23:26:03.914616+0000","last_became_peered":"2024-05-08T23:26:03.914616+0000","last_unstale":"2024-05-08T23:26:06.106533+0000","last_undegraded":"2024-05-08T23:26:06.106533+0000","last_fullsized":"2024-05-08T23:26:06.106533+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.591942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364460+0000","last_change":"2024-05-08T23:26:04.384503+0000","last_active":"2024-05-08T23:26:05.364460+0000","last_peered":"2024-05-08T23:26:05.364460+0000","last_clean":"2024-05-08T23:26:05.364460+0000","last_became_active":"2024-05-08T23:26:04.383639+0000","last_became_peered":"2024-05-08T23:26:04.383639+0000","last_unstale":"2024-05-08T23:26:05.364460+0000","last_undegraded":"2024-05-08T23:26:05.364460+0000","last_fullsized":"2024-05-08T23:26:05.364460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:13.674487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367498+0000","last_change":"2024-05-08T23:26:03.912730+0000","last_active":"2024-05-08T23:26:04.367498+0000","last_peered":"2024-05-08T23:26:04.367498+0000","last_clean":"2024-05-08T23:26:04.367498+0000","last_became_active":"2024-05-08T23:26:03.912415+0000","last_became_peered":"2024-05-08T23:26:03.912415+0000","last_unstale":"2024-05-08T23:26:04.367498+0000","last_undegraded":"2024-05-08T23:26:04.367498+0000","last_fullsized":"2024-05-08T23:26:04.367498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:35.080457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364494+0000","last_change":"2024-05-08T23:26:04.383931+0000","last_active":"2024-05-08T23:26:05.364494+0000","last_peered":"2024-05-08T23:26:05.364494+0000","last_clean":"2024-05-08T23:26:05.364494+0000","last_became_active":"2024-05-08T23:26:04.383598+0000","last_became_peered":"2024-05-08T23:26:04.383598+0000","last_unstale":"2024-05-08T23:26:05.364494+0000","last_undegraded":"2024-05-08T23:26:05.364494+0000","last_fullsized":"2024-05-08T23:26:05.364494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:27.479478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528163+0000","last_change":"2024-05-08T23:26:03.908029+0000","last_active":"2024-05-08T23:26:05.528163+0000","last_peered":"2024-05-08T23:26:05.528163+0000","last_clean":"2024-05-08T23:26:05.528163+0000","last_became_active":"2024-05-08T23:26:03.907475+0000","last_became_peered":"2024-05-08T23:26:03.907475+0000","last_unstale":"2024-05-08T23:26:05.528163+0000","last_undegraded":"2024-05-08T23:26:05.528163+0000","last_fullsized":"2024-05-08T23:26:05.528163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:26.905129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528734+0000","last_change":"2024-05-08T23:26:04.383939+0000","last_active":"2024-05-08T23:26:05.528734+0000","last_peered":"2024-05-08T23:26:05.528734+0000","last_clean":"2024-05-08T23:26:05.528734+0000","last_became_active":"2024-05-08T23:26:04.383318+0000","last_became_peered":"2024-05-08T23:26:04.383318+0000","last_unstale":"2024-05-08T23:26:05.528734+0000","last_undegraded":"2024-05-08T23:26:05.528734+0000","last_fullsized":"2024-05-08T23:26:05.528734+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106789+0000","last_change":"2024-05-08T23:26:03.917152+0000","last_active":"2024-05-08T23:26:06.106789+0000","last_peered":"2024-05-08T23:26:06.106789+0000","last_clean":"2024-05-08T23:26:06.106789+0000","last_became_active":"2024-05-08T23:26:03.916848+0000","last_became_peered":"2024-05-08T23:26:03.916848+0000","last_unstale":"2024-05-08T23:26:06.106789+0000","last_undegraded":"2024-05-08T23:26:06.106789+0000","last_fullsized":"2024-05-08T23:26:06.106789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:17.395880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.386965+0000","last_change":"2024-05-08T23:26:04.377822+0000","last_active":"2024-05-08T23:26:04.386965+0000","last_peered":"2024-05-08T23:26:04.386965+0000","last_clean":"2024-05-08T23:26:04.386965+0000","last_became_active":"2024-05-08T23:26:04.377510+0000","last_became_peered":"2024-05-08T23:26:04.377510+0000","last_unstale":"2024-05-08T23:26:04.386965+0000","last_undegraded":"2024-05-08T23:26:04.386965+0000","last_fullsized":"2024-05-08T23:26:04.386965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:09.803388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367888+0000","last_change":"2024-05-08T23:26:03.904714+0000","last_active":"2024-05-08T23:26:04.367888+0000","last_peered":"2024-05-08T23:26:04.367888+0000","last_clean":"2024-05-08T23:26:04.367888+0000","last_became_active":"2024-05-08T23:26:03.904414+0000","last_became_peered":"2024-05-08T23:26:03.904414+0000","last_unstale":"2024-05-08T23:26:04.367888+0000","last_undegraded":"2024-05-08T23:26:04.367888+0000","last_fullsized":"2024-05-08T23:26:04.367888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:14.889713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391857+0000","last_change":"2024-05-08T23:26:04.388899+0000","last_active":"2024-05-08T23:26:04.391857+0000","last_peered":"2024-05-08T23:26:04.391857+0000","last_clean":"2024-05-08T23:26:04.391857+0000","last_became_active":"2024-05-08T23:26:04.388628+0000","last_became_peered":"2024-05-08T23:26:04.388628+0000","last_unstale":"2024-05-08T23:26:04.391857+0000","last_undegraded":"2024-05-08T23:26:04.391857+0000","last_fullsized":"2024-05-08T23:26:04.391857+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365794+0000","last_change":"2024-05-08T23:26:03.910274+0000","last_active":"2024-05-08T23:26:05.365794+0000","last_peered":"2024-05-08T23:26:05.365794+0000","last_clean":"2024-05-08T23:26:05.365794+0000","last_became_active":"2024-05-08T23:26:03.909977+0000","last_became_peered":"2024-05-08T23:26:03.909977+0000","last_unstale":"2024-05-08T23:26:05.365794+0000","last_undegraded":"2024-05-08T23:26:05.365794+0000","last_fullsized":"2024-05-08T23:26:05.365794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:09.127009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382218+0000","last_change":"2024-05-08T23:26:04.377703+0000","last_active":"2024-05-08T23:26:04.382218+0000","last_peered":"2024-05-08T23:26:04.382218+0000","last_clean":"2024-05-08T23:26:04.382218+0000","last_became_active":"2024-05-08T23:26:04.377184+0000","last_became_peered":"2024-05-08T23:26:04.377184+0000","last_unstale":"2024-05-08T23:26:04.382218+0000","last_undegraded":"2024-05-08T23:26:04.382218+0000","last_fullsized":"2024-05-08T23:26:04.382218+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528645+0000","last_change":"2024-05-08T23:26:03.910256+0000","last_active":"2024-05-08T23:26:05.528645+0000","last_peered":"2024-05-08T23:26:05.528645+0000","last_clean":"2024-05-08T23:26:05.528645+0000","last_became_active":"2024-05-08T23:26:03.909930+0000","last_became_peered":"2024-05-08T23:26:03.909930+0000","last_unstale":"2024-05-08T23:26:05.528645+0000","last_undegraded":"2024-05-08T23:26:05.528645+0000","last_fullsized":"2024-05-08T23:26:05.528645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:46.987230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366137+0000","last_change":"2024-05-08T23:26:04.390986+0000","last_active":"2024-05-08T23:26:05.366137+0000","last_peered":"2024-05-08T23:26:05.366137+0000","last_clean":"2024-05-08T23:26:05.366137+0000","last_became_active":"2024-05-08T23:26:04.390656+0000","last_became_peered":"2024-05-08T23:26:04.390656+0000","last_unstale":"2024-05-08T23:26:05.366137+0000","last_undegraded":"2024-05-08T23:26:05.366137+0000","last_fullsized":"2024-05-08T23:26:05.366137+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:26.611874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365238+0000","last_change":"2024-05-08T23:26:03.902695+0000","last_active":"2024-05-08T23:26:05.365238+0000","last_peered":"2024-05-08T23:26:05.365238+0000","last_clean":"2024-05-08T23:26:05.365238+0000","last_became_active":"2024-05-08T23:26:03.902382+0000","last_became_peered":"2024-05-08T23:26:03.902382+0000","last_unstale":"2024-05-08T23:26:05.365238+0000","last_undegraded":"2024-05-08T23:26:05.365238+0000","last_fullsized":"2024-05-08T23:26:05.365238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:08.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106322+0000","last_change":"2024-05-08T23:26:04.387935+0000","last_active":"2024-05-08T23:26:06.106322+0000","last_peered":"2024-05-08T23:26:06.106322+0000","last_clean":"2024-05-08T23:26:06.106322+0000","last_became_active":"2024-05-08T23:26:04.387678+0000","last_became_peered":"2024-05-08T23:26:04.387678+0000","last_unstale":"2024-05-08T23:26:06.106322+0000","last_undegraded":"2024-05-08T23:26:06.106322+0000","last_fullsized":"2024-05-08T23:26:06.106322+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:23.335791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364796+0000","last_change":"2024-05-08T23:26:03.916887+0000","last_active":"2024-05-08T23:26:05.364796+0000","last_peered":"2024-05-08T23:26:05.364796+0000","last_clean":"2024-05-08T23:26:05.364796+0000","last_became_active":"2024-05-08T23:26:03.916407+0000","last_became_peered":"2024-05-08T23:26:03.916407+0000","last_unstale":"2024-05-08T23:26:05.364796+0000","last_undegraded":"2024-05-08T23:26:05.364796+0000","last_fullsized":"2024-05-08T23:26:05.364796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:34.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365058+0000","last_change":"2024-05-08T23:26:04.389795+0000","last_active":"2024-05-08T23:26:05.365058+0000","last_peered":"2024-05-08T23:26:05.365058+0000","last_clean":"2024-05-08T23:26:05.365058+0000","last_became_active":"2024-05-08T23:26:04.389433+0000","last_became_peered":"2024-05-08T23:26:04.389433+0000","last_unstale":"2024-05-08T23:26:05.365058+0000","last_undegraded":"2024-05-08T23:26:05.365058+0000","last_fullsized":"2024-05-08T23:26:05.365058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:23.681716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:26:02.888493+0000","last_change":"2024-05-08T23:26:02.888493+0000","last_active":"2024-05-08T23:26:02.888493+0000","last_peered":"2024-05-08T23:26:02.888493+0000","last_clean":"2024-05-08T23:26:02.888493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:26:02.888493+0000","last_undegraded":"2024-05-08T23:26:02.888493+0000","last_fullsized":"2024-05-08T23:26:02.888493+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106146+0000","last_change":"2024-05-08T23:26:04.390648+0000","last_active":"2024-05-08T23:26:06.106146+0000","last_peered":"2024-05-08T23:26:06.106146+0000","last_clean":"2024-05-08T23:26:06.106146+0000","last_became_active":"2024-05-08T23:26:04.390358+0000","last_became_peered":"2024-05-08T23:26:04.390358+0000","last_unstale":"2024-05-08T23:26:06.106146+0000","last_undegraded":"2024-05-08T23:26:06.106146+0000","last_fullsized":"2024-05-08T23:26:06.106146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:19.130856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106499+0000","last_change":"2024-05-08T23:26:03.908056+0000","last_active":"2024-05-08T23:26:06.106499+0000","last_peered":"2024-05-08T23:26:06.106499+0000","last_clean":"2024-05-08T23:26:06.106499+0000","last_became_active":"2024-05-08T23:26:03.907603+0000","last_became_peered":"2024-05-08T23:26:03.907603+0000","last_unstale":"2024-05-08T23:26:06.106499+0000","last_undegraded":"2024-05-08T23:26:06.106499+0000","last_fullsized":"2024-05-08T23:26:06.106499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:17.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364615+0000","last_change":"2024-05-08T23:26:04.384097+0000","last_active":"2024-05-08T23:26:05.364615+0000","last_peered":"2024-05-08T23:26:05.364615+0000","last_clean":"2024-05-08T23:26:05.364615+0000","last_became_active":"2024-05-08T23:26:04.383812+0000","last_became_peered":"2024-05-08T23:26:04.383812+0000","last_unstale":"2024-05-08T23:26:05.364615+0000","last_undegraded":"2024-05-08T23:26:05.364615+0000","last_fullsized":"2024-05-08T23:26:05.364615+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:08.068641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106387+0000","last_change":"2024-05-08T23:26:03.918580+0000","last_active":"2024-05-08T23:26:06.106387+0000","last_peered":"2024-05-08T23:26:06.106387+0000","last_clean":"2024-05-08T23:26:06.106387+0000","last_became_active":"2024-05-08T23:26:03.918308+0000","last_became_peered":"2024-05-08T23:26:03.918308+0000","last_unstale":"2024-05-08T23:26:06.106387+0000","last_undegraded":"2024-05-08T23:26:06.106387+0000","last_fullsized":"2024-05-08T23:26:06.106387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:22.679586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-05-08T23:26:03.912381+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:26:03.912381+0000","last_peered":"2024-05-08T23:26:03.912381+0000","last_clean":"2024-05-08T23:26:03.912381+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:26:03.912381+0000","last_undegraded":"2024-05-08T23:26:03.912381+0000","last_fullsized":"2024-05-08T23:26:03.912381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1466368,"data_stored":958984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85999999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.495},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67400000000000004,"5min":1.038,"15min":1.038},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76500000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76600000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69899999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73899999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82499999999999996,"5min":0.87,"15min":0.87},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.95799999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996}]},{"osd":2,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":1.115,"15min":1.115},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.436}]},{"osd":5,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76500000000000001,"5min":1.052,"15min":1.052},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84399999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":376480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.371,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56100000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.42299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":1.01,"15min":1.01},"last":0.432},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":1.028,"15min":1.028},"last":0.34200000000000003}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":1.105,"15min":1.105},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.54000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1470464,"data_stored":964668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48399999999999999,"15min":0.72599999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.52900000000000003,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":0.71399999999999997},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60799999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.51600000000000001,"15min":0.77800000000000002},"min":{"1min":0.312,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52900000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54800000000000004,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58099999999999996,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65400000000000003,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69099999999999995,"15min":0.93500000000000005},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.70399999999999996,"15min":0.90800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.91300000000000003},"min":{"1min":0.50700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75700000000000001,"15min":1.0069999999999999},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.73999999999999999}]},{"osd":6,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73499999999999999,"15min":0.97499999999999998},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81799999999999995,"5min":7.734,"15min":7.734},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.77800000000000002}]},{"osd":7,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.96399999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.92200000000000004},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.83299999999999996,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":375864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61799999999999999,"5min":0.621,"15min":0.621},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57399999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.441}]},{"osd":1,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.439,"15min":0.40999999999999998},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.64100000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42699999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79900000000000004,"5min":1.024,"15min":1.024},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81799999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.50700000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.159,"15min":1.159},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91900000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.622}]},{"osd":7,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.70999999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89800000000000002,"5min":1.194,"15min":1.194},"last":0.60199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39500000000000002,"15min":0.41699999999999998},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.58199999999999996,"5min":1.46,"15min":1.46},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.41099999999999998,"15min":0.42099999999999999},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52100000000000002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40100000000000002,"15min":0.40999999999999998},"min":{"1min":0.28999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45800000000000002,"15min":0.46999999999999997},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61199999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.66200000000000003,"15min":0.879},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.56399999999999995,"5min":11.964,"15min":11.964},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.503,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67300000000000004,"5min":1.593,"15min":1.593},"last":0.439}]},{"osd":4,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93200000000000005,"5min":1.375,"15min":1.375},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.624}]},{"osd":5,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.536,"15min":1.536},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.68600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.874,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":1.204,"15min":1.204},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.50700000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.878,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":1.339,"15min":1.339},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73199999999999998,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92800000000000005,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.72399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":349133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59899999999999998,"15min":0.65200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72899999999999998,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.63800000000000001,"15min":0.73499999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.78700000000000003,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.61499999999999999,"15min":0.67600000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.753,"5min":2.145,"15min":2.145},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.64100000000000001,"15min":0.69699999999999995},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.63600000000000001,"15min":0.69599999999999995},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.70299999999999996,"15min":0.80300000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.70999999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.88700000000000001,"15min":1.1719999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77500000000000002,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.5}]},{"osd":5,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55300000000000005,"15min":0.59099999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72899999999999998,"5min":0.877,"15min":0.877},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77800000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.38600000000000001}]},{"osd":6,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.63},"min":{"1min":0.34699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.66200000000000003,"15min":0.69099999999999995},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82599999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed May 8 23:25:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60199999999999998,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65400000000000003,"15min":0.70399999999999996},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82199999999999995,"5min":1.24,"15min":1.24},"last":0.45600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":368090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85199999999999998,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.433},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":0.89400000000000002},"min":{"1min":0.35199999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":8.798,"15min":8.798},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.77500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":4.71,"15min":4.71},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70699999999999996,"15min":0.96799999999999997},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0009999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.79100000000000004},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.82199999999999995,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.93000000000000005},"min":{"1min":0.46999999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78500000000000003,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73899999999999999,"15min":1.0109999999999999},"min":{"1min":0.51100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0489999999999999,"5min":10.427,"15min":10.427},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58599999999999997,"5min":0.874,"15min":0.874},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.502}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85299999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.01},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:26:09.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:26:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.217+0000 7f031faef700 1 Processor -- start 2024-05-08T23:26:09.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.217+0000 7f031faef700 1 -- start start 2024-05-08T23:26:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031faef700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0318161030 0x7f0318099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031faef700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f03180b2480 0x7f0318099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031faef700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f031809a4d0 0x7f03180cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031faef700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031806ccc0 con 0x7f031809a4d0 2024-05-08T23:26:09.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031faef700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03180d0840 con 0x7f03180b2480 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031faef700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031806c600 con 0x7f0318161030 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f031809a4d0 0x7f03180cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f031809a4d0 0x7f03180cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44810/0 (socket says 172.21.15.5:44810) 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 -- 172.21.15.5:0/1705405975 learned_addr learned my addr 172.21.15.5:0/1705405975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031d08a700 1 --2- 172.21.15.5:0/1705405975 >> v2:172.21.15.49:3300/0 conn(0x7f03180b2480 0x7f0318099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031d88b700 1 --2- 172.21.15.5:0/1705405975 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0318161030 0x7f0318099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 -- 172.21.15.5:0/1705405975 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0318161030 msgr2=0x7f0318099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 --2- 172.21.15.5:0/1705405975 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0318161030 0x7f0318099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 -- 172.21.15.5:0/1705405975 >> v2:172.21.15.49:3300/0 conn(0x7f03180b2480 msgr2=0x7f0318099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 --2- 172.21.15.5:0/1705405975 >> v2:172.21.15.49:3300/0 conn(0x7f03180b2480 0x7f0318099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 -- 172.21.15.5:0/1705405975 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031806c8e0 con 0x7f031809a4d0 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.218+0000 7f031e08c700 1 --2- 172.21.15.5:0/1705405975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180cf650 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f03180676a0 tx=0x7f031000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9653eb7be5c4eb96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:09.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031c889700 1 -- 172.21.15.5:0/1705405975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031000e070 con 0x7f031809a4d0 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031c889700 1 -- 172.21.15.5:0/1705405975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031000b9c0 con 0x7f031809a4d0 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031c889700 1 -- 172.21.15.5:0/1705405975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031000bd70 con 0x7f031809a4d0 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 -- 172.21.15.5:0/1705405975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 msgr2=0x7f03180cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 --2- 172.21.15.5:0/1705405975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180cf650 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f03180676a0 tx=0x7f031000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 -- 172.21.15.5:0/1705405975 shutdown_connections 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 --2- 172.21.15.5:0/1705405975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180cf650 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 --2- 172.21.15.5:0/1705405975 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0318161030 0x7f0318099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 --2- 172.21.15.5:0/1705405975 >> v2:172.21.15.49:3300/0 conn(0x7f03180b2480 0x7f0318099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 -- 172.21.15.5:0/1705405975 >> 172.21.15.5:0/1705405975 conn(0x7f03180b2f20 msgr2=0x7f03180b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 -- 172.21.15.5:0/1705405975 shutdown_connections 2024-05-08T23:26:09.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.219+0000 7f031faef700 1 -- 172.21.15.5:0/1705405975 wait complete. 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.221+0000 7f031faef700 1 Processor -- start 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.221+0000 7f031faef700 1 -- start start 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.221+0000 7f031faef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.221+0000 7f031faef700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03180b2480 0x7f0318155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.221+0000 7f031faef700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318161030 0x7f03180b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.221+0000 7f031faef700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318155560 con 0x7f031809a4d0 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.221+0000 7f031faef700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03180c64c0 con 0x7f0318161030 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.221+0000 7f031faef700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03180c67a0 con 0x7f03180b2480 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44812/0 (socket says 172.21.15.5:44812) 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 -- 172.21.15.5:0/2032365246 learned_addr learned my addr 172.21.15.5:0/2032365246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d08a700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03180b2480 0x7f0318155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031e08c700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318161030 0x7f03180b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 -- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03180b2480 msgr2=0x7f0318155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03180b2480 0x7f0318155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 -- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318161030 msgr2=0x7f03180b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318161030 0x7f03180b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 -- 172.21.15.5:0/2032365246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0310009b30 con 0x7f031809a4d0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031d88b700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180c7560 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f031400d800 tx=0x7f031400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f030e7fc700 1 -- 172.21.15.5:0/2032365246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140125f0 con 0x7f031809a4d0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f030e7fc700 1 -- 172.21.15.5:0/2032365246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314012750 con 0x7f031809a4d0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f030e7fc700 1 -- 172.21.15.5:0/2032365246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314012b00 con 0x7f031809a4d0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03180c6a80 con 0x7f031809a4d0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.222+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03180c7040 con 0x7f031809a4d0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.223+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0318068000 con 0x7f031809a4d0 2024-05-08T23:26:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.224+0000 7f030e7fc700 1 -- 172.21.15.5:0/2032365246 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0314012c60 con 0x7f031809a4d0 2024-05-08T23:26:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.224+0000 7f030e7fc700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02fc06acc0 0x7f02fc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.225+0000 7f030e7fc700 1 -- 172.21.15.5:0/2032365246 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03140892a0 con 0x7f031809a4d0 2024-05-08T23:26:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.225+0000 7f031d08a700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02fc06acc0 0x7f02fc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.225+0000 7f031d08a700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02fc06acc0 0x7f02fc06d180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0308009950 tx=0x7f0308008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:09.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.229+0000 7f030e7fc700 1 -- 172.21.15.5:0/2032365246 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0314054510 con 0x7f031809a4d0 2024-05-08T23:26:09.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.444+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03180d0f50 con 0x7f02fc06acc0 2024-05-08T23:26:09.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.453+0000 7f030e7fc700 1 -- 172.21.15.5:0/2032365246 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289729 (secure 0 0 0) 0x7f03180d0f50 con 0x7f02fc06acc0 2024-05-08T23:26:09.457 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02fc06acc0 msgr2=0x7f02fc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02fc06acc0 0x7f02fc06d180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0308009950 tx=0x7f0308008040 comp rx=0 tx=0).stop 2024-05-08T23:26:09.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 msgr2=0x7f03180c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180c7560 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f031400d800 tx=0x7f031400db10 comp rx=0 tx=0).stop 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 shutdown_connections 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02fc06acc0 0x7f02fc06d180 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318161030 0x7f03180b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031809a4d0 0x7f03180c7560 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 --2- 172.21.15.5:0/2032365246 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03180b2480 0x7f0318155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 >> 172.21.15.5:0/2032365246 conn(0x7f03180b2f20 msgr2=0x7f031806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 shutdown_connections 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.456+0000 7f031faef700 1 -- 172.21.15.5:0/2032365246 wait complete. 2024-05-08T23:26:09.460 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:26:09.471 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-05-08T23:26:09.367539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236508,"kb_used_data":8036,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":754738212,"statfs":{"total":773094113280,"available":772851929088,"internally_reserved":0,"allocated":8228864,"data_stored":4160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":233819701},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001099"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.385435+0000","last_change":"2024-05-08T23:26:04.382812+0000","last_active":"2024-05-08T23:26:04.385435+0000","last_peered":"2024-05-08T23:26:04.385435+0000","last_clean":"2024-05-08T23:26:04.385435+0000","last_became_active":"2024-05-08T23:26:04.382518+0000","last_became_peered":"2024-05-08T23:26:04.382518+0000","last_unstale":"2024-05-08T23:26:04.385435+0000","last_undegraded":"2024-05-08T23:26:04.385435+0000","last_fullsized":"2024-05-08T23:26:04.385435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:16.459465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106783+0000","last_change":"2024-05-08T23:26:04.386202+0000","last_active":"2024-05-08T23:26:06.106783+0000","last_peered":"2024-05-08T23:26:06.106783+0000","last_clean":"2024-05-08T23:26:06.106783+0000","last_became_active":"2024-05-08T23:26:04.385897+0000","last_became_peered":"2024-05-08T23:26:04.385897+0000","last_unstale":"2024-05-08T23:26:06.106783+0000","last_undegraded":"2024-05-08T23:26:06.106783+0000","last_fullsized":"2024-05-08T23:26:06.106783+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:15.891844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365071+0000","last_change":"2024-05-08T23:26:04.376144+0000","last_active":"2024-05-08T23:26:05.365071+0000","last_peered":"2024-05-08T23:26:05.365071+0000","last_clean":"2024-05-08T23:26:05.365071+0000","last_became_active":"2024-05-08T23:26:04.375870+0000","last_became_peered":"2024-05-08T23:26:04.375870+0000","last_unstale":"2024-05-08T23:26:05.365071+0000","last_undegraded":"2024-05-08T23:26:05.365071+0000","last_fullsized":"2024-05-08T23:26:05.365071+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:53.570674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365640+0000","last_change":"2024-05-08T23:26:04.391205+0000","last_active":"2024-05-08T23:26:05.365640+0000","last_peered":"2024-05-08T23:26:05.365640+0000","last_clean":"2024-05-08T23:26:05.365640+0000","last_became_active":"2024-05-08T23:26:04.390895+0000","last_became_peered":"2024-05-08T23:26:04.390895+0000","last_unstale":"2024-05-08T23:26:05.365640+0000","last_undegraded":"2024-05-08T23:26:05.365640+0000","last_fullsized":"2024-05-08T23:26:05.365640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:00.837462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365140+0000","last_change":"2024-05-08T23:26:04.380499+0000","last_active":"2024-05-08T23:26:05.365140+0000","last_peered":"2024-05-08T23:26:05.365140+0000","last_clean":"2024-05-08T23:26:05.365140+0000","last_became_active":"2024-05-08T23:26:04.380163+0000","last_became_peered":"2024-05-08T23:26:04.380163+0000","last_unstale":"2024-05-08T23:26:05.365140+0000","last_undegraded":"2024-05-08T23:26:05.365140+0000","last_fullsized":"2024-05-08T23:26:05.365140+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:39.534202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.392388+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.392064+0000","last_became_peered":"2024-05-08T23:26:04.392064+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:55.698089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.387389+0000","last_change":"2024-05-08T23:26:04.377831+0000","last_active":"2024-05-08T23:26:04.387389+0000","last_peered":"2024-05-08T23:26:04.387389+0000","last_clean":"2024-05-08T23:26:04.387389+0000","last_became_active":"2024-05-08T23:26:04.377518+0000","last_became_peered":"2024-05-08T23:26:04.377518+0000","last_unstale":"2024-05-08T23:26:04.387389+0000","last_undegraded":"2024-05-08T23:26:04.387389+0000","last_fullsized":"2024-05-08T23:26:04.387389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106353+0000","last_change":"2024-05-08T23:26:04.388002+0000","last_active":"2024-05-08T23:26:06.106353+0000","last_peered":"2024-05-08T23:26:06.106353+0000","last_clean":"2024-05-08T23:26:06.106353+0000","last_became_active":"2024-05-08T23:26:04.386006+0000","last_became_peered":"2024-05-08T23:26:04.386006+0000","last_unstale":"2024-05-08T23:26:06.106353+0000","last_undegraded":"2024-05-08T23:26:06.106353+0000","last_fullsized":"2024-05-08T23:26:06.106353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:34:35.709822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365561+0000","last_change":"2024-05-08T23:26:04.389861+0000","last_active":"2024-05-08T23:26:05.365561+0000","last_peered":"2024-05-08T23:26:05.365561+0000","last_clean":"2024-05-08T23:26:05.365561+0000","last_became_active":"2024-05-08T23:26:04.389582+0000","last_became_peered":"2024-05-08T23:26:04.389582+0000","last_unstale":"2024-05-08T23:26:05.365561+0000","last_undegraded":"2024-05-08T23:26:05.365561+0000","last_fullsized":"2024-05-08T23:26:05.365561+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:12.594073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389862+0000","last_change":"2024-05-08T23:26:04.384004+0000","last_active":"2024-05-08T23:26:04.389862+0000","last_peered":"2024-05-08T23:26:04.389862+0000","last_clean":"2024-05-08T23:26:04.389862+0000","last_became_active":"2024-05-08T23:26:04.383584+0000","last_became_peered":"2024-05-08T23:26:04.383584+0000","last_unstale":"2024-05-08T23:26:04.389862+0000","last_undegraded":"2024-05-08T23:26:04.389862+0000","last_fullsized":"2024-05-08T23:26:04.389862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:52.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391883+0000","last_change":"2024-05-08T23:26:04.390913+0000","last_active":"2024-05-08T23:26:04.391883+0000","last_peered":"2024-05-08T23:26:04.391883+0000","last_clean":"2024-05-08T23:26:04.391883+0000","last_became_active":"2024-05-08T23:26:04.390654+0000","last_became_peered":"2024-05-08T23:26:04.390654+0000","last_unstale":"2024-05-08T23:26:04.391883+0000","last_undegraded":"2024-05-08T23:26:04.391883+0000","last_fullsized":"2024-05-08T23:26:04.391883+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365672+0000","last_change":"2024-05-08T23:26:04.382718+0000","last_active":"2024-05-08T23:26:05.365672+0000","last_peered":"2024-05-08T23:26:05.365672+0000","last_clean":"2024-05-08T23:26:05.365672+0000","last_became_active":"2024-05-08T23:26:04.381951+0000","last_became_peered":"2024-05-08T23:26:04.381951+0000","last_unstale":"2024-05-08T23:26:05.365672+0000","last_undegraded":"2024-05-08T23:26:05.365672+0000","last_fullsized":"2024-05-08T23:26:05.365672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:18.398011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528368+0000","last_change":"2024-05-08T23:26:04.393225+0000","last_active":"2024-05-08T23:26:05.528368+0000","last_peered":"2024-05-08T23:26:05.528368+0000","last_clean":"2024-05-08T23:26:05.528368+0000","last_became_active":"2024-05-08T23:26:04.392933+0000","last_became_peered":"2024-05-08T23:26:04.392933+0000","last_unstale":"2024-05-08T23:26:05.528368+0000","last_undegraded":"2024-05-08T23:26:05.528368+0000","last_fullsized":"2024-05-08T23:26:05.528368+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366036+0000","last_change":"2024-05-08T23:26:04.380528+0000","last_active":"2024-05-08T23:26:05.366036+0000","last_peered":"2024-05-08T23:26:05.366036+0000","last_clean":"2024-05-08T23:26:05.366036+0000","last_became_active":"2024-05-08T23:26:04.380129+0000","last_became_peered":"2024-05-08T23:26:04.380129+0000","last_unstale":"2024-05-08T23:26:05.366036+0000","last_undegraded":"2024-05-08T23:26:05.366036+0000","last_fullsized":"2024-05-08T23:26:05.366036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.038766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528231+0000","last_change":"2024-05-08T23:26:04.382434+0000","last_active":"2024-05-08T23:26:05.528231+0000","last_peered":"2024-05-08T23:26:05.528231+0000","last_clean":"2024-05-08T23:26:05.528231+0000","last_became_active":"2024-05-08T23:26:04.382015+0000","last_became_peered":"2024-05-08T23:26:04.382015+0000","last_unstale":"2024-05-08T23:26:05.528231+0000","last_undegraded":"2024-05-08T23:26:05.528231+0000","last_fullsized":"2024-05-08T23:26:05.528231+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.395912+0000","last_change":"2024-05-08T23:26:04.395148+0000","last_active":"2024-05-08T23:26:04.395912+0000","last_peered":"2024-05-08T23:26:04.395912+0000","last_clean":"2024-05-08T23:26:04.395912+0000","last_became_active":"2024-05-08T23:26:04.394850+0000","last_became_peered":"2024-05-08T23:26:04.394850+0000","last_unstale":"2024-05-08T23:26:04.395912+0000","last_undegraded":"2024-05-08T23:26:04.395912+0000","last_fullsized":"2024-05-08T23:26:04.395912+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:39.285912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365565+0000","last_change":"2024-05-08T23:26:04.386467+0000","last_active":"2024-05-08T23:26:05.365565+0000","last_peered":"2024-05-08T23:26:05.365565+0000","last_clean":"2024-05-08T23:26:05.365565+0000","last_became_active":"2024-05-08T23:26:04.386005+0000","last_became_peered":"2024-05-08T23:26:04.386005+0000","last_unstale":"2024-05-08T23:26:05.365565+0000","last_undegraded":"2024-05-08T23:26:05.365565+0000","last_fullsized":"2024-05-08T23:26:05.365565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:08.141995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365004+0000","last_change":"2024-05-08T23:26:04.384544+0000","last_active":"2024-05-08T23:26:05.365004+0000","last_peered":"2024-05-08T23:26:05.365004+0000","last_clean":"2024-05-08T23:26:05.365004+0000","last_became_active":"2024-05-08T23:26:04.384240+0000","last_became_peered":"2024-05-08T23:26:04.384240+0000","last_unstale":"2024-05-08T23:26:05.365004+0000","last_undegraded":"2024-05-08T23:26:05.365004+0000","last_fullsized":"2024-05-08T23:26:05.365004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:44.177271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106649+0000","last_change":"2024-05-08T23:26:04.387389+0000","last_active":"2024-05-08T23:26:06.106649+0000","last_peered":"2024-05-08T23:26:06.106649+0000","last_clean":"2024-05-08T23:26:06.106649+0000","last_became_active":"2024-05-08T23:26:04.383192+0000","last_became_peered":"2024-05-08T23:26:04.383192+0000","last_unstale":"2024-05-08T23:26:06.106649+0000","last_undegraded":"2024-05-08T23:26:06.106649+0000","last_fullsized":"2024-05-08T23:26:06.106649+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528319+0000","last_change":"2024-05-08T23:26:04.374428+0000","last_active":"2024-05-08T23:26:05.528319+0000","last_peered":"2024-05-08T23:26:05.528319+0000","last_clean":"2024-05-08T23:26:05.528319+0000","last_became_active":"2024-05-08T23:26:04.374165+0000","last_became_peered":"2024-05-08T23:26:04.374165+0000","last_unstale":"2024-05-08T23:26:05.528319+0000","last_undegraded":"2024-05-08T23:26:05.528319+0000","last_fullsized":"2024-05-08T23:26:05.528319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.381684+0000","last_change":"2024-05-08T23:26:04.378977+0000","last_active":"2024-05-08T23:26:04.381684+0000","last_peered":"2024-05-08T23:26:04.381684+0000","last_clean":"2024-05-08T23:26:04.381684+0000","last_became_active":"2024-05-08T23:26:04.378470+0000","last_became_peered":"2024-05-08T23:26:04.378470+0000","last_unstale":"2024-05-08T23:26:04.381684+0000","last_undegraded":"2024-05-08T23:26:04.381684+0000","last_fullsized":"2024-05-08T23:26:04.381684+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.979369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389858+0000","last_change":"2024-05-08T23:26:04.383843+0000","last_active":"2024-05-08T23:26:04.389858+0000","last_peered":"2024-05-08T23:26:04.389858+0000","last_clean":"2024-05-08T23:26:04.389858+0000","last_became_active":"2024-05-08T23:26:04.383503+0000","last_became_peered":"2024-05-08T23:26:04.383503+0000","last_unstale":"2024-05-08T23:26:04.389858+0000","last_undegraded":"2024-05-08T23:26:04.389858+0000","last_fullsized":"2024-05-08T23:26:04.389858+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:17.695615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382847+0000","last_change":"2024-05-08T23:26:04.380605+0000","last_active":"2024-05-08T23:26:04.382847+0000","last_peered":"2024-05-08T23:26:04.382847+0000","last_clean":"2024-05-08T23:26:04.382847+0000","last_became_active":"2024-05-08T23:26:04.380307+0000","last_became_peered":"2024-05-08T23:26:04.380307+0000","last_unstale":"2024-05-08T23:26:04.382847+0000","last_undegraded":"2024-05-08T23:26:04.382847+0000","last_fullsized":"2024-05-08T23:26:04.382847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:27.367580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.395921+0000","last_change":"2024-05-08T23:26:04.395150+0000","last_active":"2024-05-08T23:26:04.395921+0000","last_peered":"2024-05-08T23:26:04.395921+0000","last_clean":"2024-05-08T23:26:04.395921+0000","last_became_active":"2024-05-08T23:26:04.394845+0000","last_became_peered":"2024-05-08T23:26:04.394845+0000","last_unstale":"2024-05-08T23:26:04.395921+0000","last_undegraded":"2024-05-08T23:26:04.395921+0000","last_fullsized":"2024-05-08T23:26:04.395921+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389884+0000","last_change":"2024-05-08T23:26:04.383933+0000","last_active":"2024-05-08T23:26:04.389884+0000","last_peered":"2024-05-08T23:26:04.389884+0000","last_clean":"2024-05-08T23:26:04.389884+0000","last_became_active":"2024-05-08T23:26:04.383614+0000","last_became_peered":"2024-05-08T23:26:04.383614+0000","last_unstale":"2024-05-08T23:26:04.389884+0000","last_undegraded":"2024-05-08T23:26:04.389884+0000","last_fullsized":"2024-05-08T23:26:04.389884+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:47.989361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.390865+0000","last_change":"2024-05-08T23:26:04.387174+0000","last_active":"2024-05-08T23:26:04.390865+0000","last_peered":"2024-05-08T23:26:04.390865+0000","last_clean":"2024-05-08T23:26:04.390865+0000","last_became_active":"2024-05-08T23:26:04.386653+0000","last_became_peered":"2024-05-08T23:26:04.386653+0000","last_unstale":"2024-05-08T23:26:04.390865+0000","last_undegraded":"2024-05-08T23:26:04.390865+0000","last_fullsized":"2024-05-08T23:26:04.390865+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:52.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365110+0000","last_change":"2024-05-08T23:26:04.374586+0000","last_active":"2024-05-08T23:26:05.365110+0000","last_peered":"2024-05-08T23:26:05.365110+0000","last_clean":"2024-05-08T23:26:05.365110+0000","last_became_active":"2024-05-08T23:26:04.374382+0000","last_became_peered":"2024-05-08T23:26:04.374382+0000","last_unstale":"2024-05-08T23:26:05.365110+0000","last_undegraded":"2024-05-08T23:26:05.365110+0000","last_fullsized":"2024-05-08T23:26:05.365110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:41.989450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389885+0000","last_change":"2024-05-08T23:26:04.384048+0000","last_active":"2024-05-08T23:26:04.389885+0000","last_peered":"2024-05-08T23:26:04.389885+0000","last_clean":"2024-05-08T23:26:04.389885+0000","last_became_active":"2024-05-08T23:26:04.383520+0000","last_became_peered":"2024-05-08T23:26:04.383520+0000","last_unstale":"2024-05-08T23:26:04.389885+0000","last_undegraded":"2024-05-08T23:26:04.389885+0000","last_fullsized":"2024-05-08T23:26:04.389885+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:09.803388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364727+0000","last_change":"2024-05-08T23:26:04.376502+0000","last_active":"2024-05-08T23:26:05.364727+0000","last_peered":"2024-05-08T23:26:05.364727+0000","last_clean":"2024-05-08T23:26:05.364727+0000","last_became_active":"2024-05-08T23:26:04.376226+0000","last_became_peered":"2024-05-08T23:26:04.376226+0000","last_unstale":"2024-05-08T23:26:05.364727+0000","last_undegraded":"2024-05-08T23:26:05.364727+0000","last_fullsized":"2024-05-08T23:26:05.364727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:36.452021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.393993+0000","last_change":"2024-05-08T23:26:04.393135+0000","last_active":"2024-05-08T23:26:04.393993+0000","last_peered":"2024-05-08T23:26:04.393993+0000","last_clean":"2024-05-08T23:26:04.393993+0000","last_became_active":"2024-05-08T23:26:04.392824+0000","last_became_peered":"2024-05-08T23:26:04.392824+0000","last_unstale":"2024-05-08T23:26:04.393993+0000","last_undegraded":"2024-05-08T23:26:04.393993+0000","last_fullsized":"2024-05-08T23:26:04.393993+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:47.989361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391112+0000","last_change":"2024-05-08T23:26:04.387882+0000","last_active":"2024-05-08T23:26:04.391112+0000","last_peered":"2024-05-08T23:26:04.391112+0000","last_clean":"2024-05-08T23:26:04.391112+0000","last_became_active":"2024-05-08T23:26:04.387600+0000","last_became_peered":"2024-05-08T23:26:04.387600+0000","last_unstale":"2024-05-08T23:26:04.391112+0000","last_undegraded":"2024-05-08T23:26:04.391112+0000","last_fullsized":"2024-05-08T23:26:04.391112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:29.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106297+0000","last_change":"2024-05-08T23:26:04.390849+0000","last_active":"2024-05-08T23:26:06.106297+0000","last_peered":"2024-05-08T23:26:06.106297+0000","last_clean":"2024-05-08T23:26:06.106297+0000","last_became_active":"2024-05-08T23:26:04.390577+0000","last_became_peered":"2024-05-08T23:26:04.390577+0000","last_unstale":"2024-05-08T23:26:06.106297+0000","last_undegraded":"2024-05-08T23:26:06.106297+0000","last_fullsized":"2024-05-08T23:26:06.106297+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:00.857944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389116+0000","last_change":"2024-05-08T23:26:04.387838+0000","last_active":"2024-05-08T23:26:04.389116+0000","last_peered":"2024-05-08T23:26:04.389116+0000","last_clean":"2024-05-08T23:26:04.389116+0000","last_became_active":"2024-05-08T23:26:04.387594+0000","last_became_peered":"2024-05-08T23:26:04.387594+0000","last_unstale":"2024-05-08T23:26:04.389116+0000","last_undegraded":"2024-05-08T23:26:04.389116+0000","last_fullsized":"2024-05-08T23:26:04.389116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389890+0000","last_change":"2024-05-08T23:26:03.918800+0000","last_active":"2024-05-08T23:26:04.389890+0000","last_peered":"2024-05-08T23:26:04.389890+0000","last_clean":"2024-05-08T23:26:04.389890+0000","last_became_active":"2024-05-08T23:26:03.918509+0000","last_became_peered":"2024-05-08T23:26:03.918509+0000","last_unstale":"2024-05-08T23:26:04.389890+0000","last_undegraded":"2024-05-08T23:26:04.389890+0000","last_fullsized":"2024-05-08T23:26:04.389890+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:46.488568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365685+0000","last_change":"2024-05-08T23:26:04.386283+0000","last_active":"2024-05-08T23:26:05.365685+0000","last_peered":"2024-05-08T23:26:05.365685+0000","last_clean":"2024-05-08T23:26:05.365685+0000","last_became_active":"2024-05-08T23:26:04.385690+0000","last_became_peered":"2024-05-08T23:26:04.385690+0000","last_unstale":"2024-05-08T23:26:05.365685+0000","last_undegraded":"2024-05-08T23:26:05.365685+0000","last_fullsized":"2024-05-08T23:26:05.365685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:12.667029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528769+0000","last_change":"2024-05-08T23:26:03.912463+0000","last_active":"2024-05-08T23:26:05.528769+0000","last_peered":"2024-05-08T23:26:05.528769+0000","last_clean":"2024-05-08T23:26:05.528769+0000","last_became_active":"2024-05-08T23:26:03.912178+0000","last_became_peered":"2024-05-08T23:26:03.912178+0000","last_unstale":"2024-05-08T23:26:05.528769+0000","last_undegraded":"2024-05-08T23:26:05.528769+0000","last_fullsized":"2024-05-08T23:26:05.528769+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:02.228610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.385614+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.385325+0000","last_became_peered":"2024-05-08T23:26:04.385325+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.979369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.394595+0000","last_change":"2024-05-08T23:26:03.911580+0000","last_active":"2024-05-08T23:26:04.394595+0000","last_peered":"2024-05-08T23:26:04.394595+0000","last_clean":"2024-05-08T23:26:04.394595+0000","last_became_active":"2024-05-08T23:26:03.911071+0000","last_became_peered":"2024-05-08T23:26:03.911071+0000","last_unstale":"2024-05-08T23:26:04.394595+0000","last_undegraded":"2024-05-08T23:26:04.394595+0000","last_fullsized":"2024-05-08T23:26:04.394595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:42:39.409566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382824+0000","last_change":"2024-05-08T23:26:04.380473+0000","last_active":"2024-05-08T23:26:04.382824+0000","last_peered":"2024-05-08T23:26:04.382824+0000","last_clean":"2024-05-08T23:26:04.382824+0000","last_became_active":"2024-05-08T23:26:04.380170+0000","last_became_peered":"2024-05-08T23:26:04.380170+0000","last_unstale":"2024-05-08T23:26:04.382824+0000","last_undegraded":"2024-05-08T23:26:04.382824+0000","last_fullsized":"2024-05-08T23:26:04.382824+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:30.884597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106822+0000","last_change":"2024-05-08T23:26:03.918662+0000","last_active":"2024-05-08T23:26:06.106822+0000","last_peered":"2024-05-08T23:26:06.106822+0000","last_clean":"2024-05-08T23:26:06.106822+0000","last_became_active":"2024-05-08T23:26:03.918367+0000","last_became_peered":"2024-05-08T23:26:03.918367+0000","last_unstale":"2024-05-08T23:26:06.106822+0000","last_undegraded":"2024-05-08T23:26:06.106822+0000","last_fullsized":"2024-05-08T23:26:06.106822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.392305+0000","last_change":"2024-05-08T23:26:04.391595+0000","last_active":"2024-05-08T23:26:04.392305+0000","last_peered":"2024-05-08T23:26:04.392305+0000","last_clean":"2024-05-08T23:26:04.392305+0000","last_became_active":"2024-05-08T23:26:04.391291+0000","last_became_peered":"2024-05-08T23:26:04.391291+0000","last_unstale":"2024-05-08T23:26:04.392305+0000","last_undegraded":"2024-05-08T23:26:04.392305+0000","last_fullsized":"2024-05-08T23:26:04.392305+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:17.695615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364906+0000","last_change":"2024-05-08T23:26:03.916776+0000","last_active":"2024-05-08T23:26:05.364906+0000","last_peered":"2024-05-08T23:26:05.364906+0000","last_clean":"2024-05-08T23:26:05.364906+0000","last_became_active":"2024-05-08T23:26:03.916402+0000","last_became_peered":"2024-05-08T23:26:03.916402+0000","last_unstale":"2024-05-08T23:26:05.364906+0000","last_undegraded":"2024-05-08T23:26:05.364906+0000","last_fullsized":"2024-05-08T23:26:05.364906+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:04.288000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.384180+0000","last_change":"2024-05-08T23:26:04.381800+0000","last_active":"2024-05-08T23:26:04.384180+0000","last_peered":"2024-05-08T23:26:04.384180+0000","last_clean":"2024-05-08T23:26:04.384180+0000","last_became_active":"2024-05-08T23:26:04.381476+0000","last_became_peered":"2024-05-08T23:26:04.381476+0000","last_unstale":"2024-05-08T23:26:04.384180+0000","last_undegraded":"2024-05-08T23:26:04.384180+0000","last_fullsized":"2024-05-08T23:26:04.384180+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.907260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106261+0000","last_change":"2024-05-08T23:26:03.915034+0000","last_active":"2024-05-08T23:26:06.106261+0000","last_peered":"2024-05-08T23:26:06.106261+0000","last_clean":"2024-05-08T23:26:06.106261+0000","last_became_active":"2024-05-08T23:26:03.914666+0000","last_became_peered":"2024-05-08T23:26:03.914666+0000","last_unstale":"2024-05-08T23:26:06.106261+0000","last_undegraded":"2024-05-08T23:26:06.106261+0000","last_fullsized":"2024-05-08T23:26:06.106261+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:43.175140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365034+0000","last_change":"2024-05-08T23:26:04.381034+0000","last_active":"2024-05-08T23:26:05.365034+0000","last_peered":"2024-05-08T23:26:05.365034+0000","last_clean":"2024-05-08T23:26:05.365034+0000","last_became_active":"2024-05-08T23:26:04.380769+0000","last_became_peered":"2024-05-08T23:26:04.380769+0000","last_unstale":"2024-05-08T23:26:05.365034+0000","last_undegraded":"2024-05-08T23:26:05.365034+0000","last_fullsized":"2024-05-08T23:26:05.365034+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:35.530781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365301+0000","last_change":"2024-05-08T23:26:03.917031+0000","last_active":"2024-05-08T23:26:05.365301+0000","last_peered":"2024-05-08T23:26:05.365301+0000","last_clean":"2024-05-08T23:26:05.365301+0000","last_became_active":"2024-05-08T23:26:03.916394+0000","last_became_peered":"2024-05-08T23:26:03.916394+0000","last_unstale":"2024-05-08T23:26:05.365301+0000","last_undegraded":"2024-05-08T23:26:05.365301+0000","last_fullsized":"2024-05-08T23:26:05.365301+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:38.532072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528242+0000","last_change":"2024-05-08T23:26:04.374371+0000","last_active":"2024-05-08T23:26:05.528242+0000","last_peered":"2024-05-08T23:26:05.528242+0000","last_clean":"2024-05-08T23:26:05.528242+0000","last_became_active":"2024-05-08T23:26:04.374097+0000","last_became_peered":"2024-05-08T23:26:04.374097+0000","last_unstale":"2024-05-08T23:26:05.528242+0000","last_undegraded":"2024-05-08T23:26:05.528242+0000","last_fullsized":"2024-05-08T23:26:05.528242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365139+0000","last_change":"2024-05-08T23:26:03.911594+0000","last_active":"2024-05-08T23:26:05.365139+0000","last_peered":"2024-05-08T23:26:05.365139+0000","last_clean":"2024-05-08T23:26:05.365139+0000","last_became_active":"2024-05-08T23:26:03.911273+0000","last_became_peered":"2024-05-08T23:26:03.911273+0000","last_unstale":"2024-05-08T23:26:05.365139+0000","last_undegraded":"2024-05-08T23:26:05.365139+0000","last_fullsized":"2024-05-08T23:26:05.365139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:30.134959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.373540+0000","last_change":"2024-05-08T23:26:04.372794+0000","last_active":"2024-05-08T23:26:04.373540+0000","last_peered":"2024-05-08T23:26:04.373540+0000","last_clean":"2024-05-08T23:26:04.373540+0000","last_became_active":"2024-05-08T23:26:04.372536+0000","last_became_peered":"2024-05-08T23:26:04.372536+0000","last_unstale":"2024-05-08T23:26:04.373540+0000","last_undegraded":"2024-05-08T23:26:04.373540+0000","last_fullsized":"2024-05-08T23:26:04.373540+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:06.816807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.372316+0000","last_change":"2024-05-08T23:26:03.917379+0000","last_active":"2024-05-08T23:26:04.372316+0000","last_peered":"2024-05-08T23:26:04.372316+0000","last_clean":"2024-05-08T23:26:04.372316+0000","last_became_active":"2024-05-08T23:26:03.917063+0000","last_became_peered":"2024-05-08T23:26:03.917063+0000","last_unstale":"2024-05-08T23:26:04.372316+0000","last_undegraded":"2024-05-08T23:26:04.372316+0000","last_fullsized":"2024-05-08T23:26:04.372316+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:43.443817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391925+0000","last_change":"2024-05-08T23:26:04.388857+0000","last_active":"2024-05-08T23:26:04.391925+0000","last_peered":"2024-05-08T23:26:04.391925+0000","last_clean":"2024-05-08T23:26:04.391925+0000","last_became_active":"2024-05-08T23:26:04.388581+0000","last_became_peered":"2024-05-08T23:26:04.388581+0000","last_unstale":"2024-05-08T23:26:04.391925+0000","last_undegraded":"2024-05-08T23:26:04.391925+0000","last_fullsized":"2024-05-08T23:26:04.391925+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106865+0000","last_change":"2024-05-08T23:26:03.907421+0000","last_active":"2024-05-08T23:26:06.106865+0000","last_peered":"2024-05-08T23:26:06.106865+0000","last_clean":"2024-05-08T23:26:06.106865+0000","last_became_active":"2024-05-08T23:26:03.907143+0000","last_became_peered":"2024-05-08T23:26:03.907143+0000","last_unstale":"2024-05-08T23:26:06.106865+0000","last_undegraded":"2024-05-08T23:26:06.106865+0000","last_fullsized":"2024-05-08T23:26:06.106865+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:07.208552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106627+0000","last_change":"2024-05-08T23:26:04.386708+0000","last_active":"2024-05-08T23:26:06.106627+0000","last_peered":"2024-05-08T23:26:06.106627+0000","last_clean":"2024-05-08T23:26:06.106627+0000","last_became_active":"2024-05-08T23:26:04.385984+0000","last_became_peered":"2024-05-08T23:26:04.385984+0000","last_unstale":"2024-05-08T23:26:06.106627+0000","last_undegraded":"2024-05-08T23:26:06.106627+0000","last_fullsized":"2024-05-08T23:26:06.106627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:47.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365174+0000","last_change":"2024-05-08T23:26:03.913646+0000","last_active":"2024-05-08T23:26:05.365174+0000","last_peered":"2024-05-08T23:26:05.365174+0000","last_clean":"2024-05-08T23:26:05.365174+0000","last_became_active":"2024-05-08T23:26:03.913208+0000","last_became_peered":"2024-05-08T23:26:03.913208+0000","last_unstale":"2024-05-08T23:26:05.365174+0000","last_undegraded":"2024-05-08T23:26:05.365174+0000","last_fullsized":"2024-05-08T23:26:05.365174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:02.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365662+0000","last_change":"2024-05-08T23:26:04.375029+0000","last_active":"2024-05-08T23:26:05.365662+0000","last_peered":"2024-05-08T23:26:05.365662+0000","last_clean":"2024-05-08T23:26:05.365662+0000","last_became_active":"2024-05-08T23:26:04.374797+0000","last_became_peered":"2024-05-08T23:26:04.374797+0000","last_unstale":"2024-05-08T23:26:05.365662+0000","last_undegraded":"2024-05-08T23:26:05.365662+0000","last_fullsized":"2024-05-08T23:26:05.365662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:18.355499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365807+0000","last_change":"2024-05-08T23:26:03.913239+0000","last_active":"2024-05-08T23:26:05.365807+0000","last_peered":"2024-05-08T23:26:05.365807+0000","last_clean":"2024-05-08T23:26:05.365807+0000","last_became_active":"2024-05-08T23:26:03.912892+0000","last_became_peered":"2024-05-08T23:26:03.912892+0000","last_unstale":"2024-05-08T23:26:05.365807+0000","last_undegraded":"2024-05-08T23:26:05.365807+0000","last_fullsized":"2024-05-08T23:26:05.365807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:28.036636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.385254+0000","last_change":"2024-05-08T23:26:04.381564+0000","last_active":"2024-05-08T23:26:04.385254+0000","last_peered":"2024-05-08T23:26:04.385254+0000","last_clean":"2024-05-08T23:26:04.385254+0000","last_became_active":"2024-05-08T23:26:04.381293+0000","last_became_peered":"2024-05-08T23:26:04.381293+0000","last_unstale":"2024-05-08T23:26:04.385254+0000","last_undegraded":"2024-05-08T23:26:04.385254+0000","last_fullsized":"2024-05-08T23:26:04.385254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365371+0000","last_change":"2024-05-08T23:26:03.917272+0000","last_active":"2024-05-08T23:26:05.365371+0000","last_peered":"2024-05-08T23:26:05.365371+0000","last_clean":"2024-05-08T23:26:05.365371+0000","last_became_active":"2024-05-08T23:26:03.916960+0000","last_became_peered":"2024-05-08T23:26:03.916960+0000","last_unstale":"2024-05-08T23:26:05.365371+0000","last_undegraded":"2024-05-08T23:26:05.365371+0000","last_fullsized":"2024-05-08T23:26:05.365371+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.380000+0000","last_change":"2024-05-08T23:26:04.375997+0000","last_active":"2024-05-08T23:26:04.380000+0000","last_peered":"2024-05-08T23:26:04.380000+0000","last_clean":"2024-05-08T23:26:04.380000+0000","last_became_active":"2024-05-08T23:26:04.375654+0000","last_became_peered":"2024-05-08T23:26:04.375654+0000","last_unstale":"2024-05-08T23:26:04.380000+0000","last_undegraded":"2024-05-08T23:26:04.380000+0000","last_fullsized":"2024-05-08T23:26:04.380000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:03.230741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106390+0000","last_change":"2024-05-08T23:26:03.911788+0000","last_active":"2024-05-08T23:26:06.106390+0000","last_peered":"2024-05-08T23:26:06.106390+0000","last_clean":"2024-05-08T23:26:06.106390+0000","last_became_active":"2024-05-08T23:26:03.911527+0000","last_became_peered":"2024-05-08T23:26:03.911527+0000","last_unstale":"2024-05-08T23:26:06.106390+0000","last_undegraded":"2024-05-08T23:26:06.106390+0000","last_fullsized":"2024-05-08T23:26:06.106390+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:44.784641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528178+0000","last_change":"2024-05-08T23:26:04.394848+0000","last_active":"2024-05-08T23:26:05.528178+0000","last_peered":"2024-05-08T23:26:05.528178+0000","last_clean":"2024-05-08T23:26:05.528178+0000","last_became_active":"2024-05-08T23:26:04.394520+0000","last_became_peered":"2024-05-08T23:26:04.394520+0000","last_unstale":"2024-05-08T23:26:05.528178+0000","last_undegraded":"2024-05-08T23:26:05.528178+0000","last_fullsized":"2024-05-08T23:26:05.528178+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367381+0000","last_change":"2024-05-08T23:26:03.904993+0000","last_active":"2024-05-08T23:26:04.367381+0000","last_peered":"2024-05-08T23:26:04.367381+0000","last_clean":"2024-05-08T23:26:04.367381+0000","last_became_active":"2024-05-08T23:26:03.904689+0000","last_became_peered":"2024-05-08T23:26:03.904689+0000","last_unstale":"2024-05-08T23:26:04.367381+0000","last_undegraded":"2024-05-08T23:26:04.367381+0000","last_fullsized":"2024-05-08T23:26:04.367381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:54.695959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365441+0000","last_change":"2024-05-08T23:26:04.386132+0000","last_active":"2024-05-08T23:26:05.365441+0000","last_peered":"2024-05-08T23:26:05.365441+0000","last_clean":"2024-05-08T23:26:05.365441+0000","last_became_active":"2024-05-08T23:26:04.385842+0000","last_became_peered":"2024-05-08T23:26:04.385842+0000","last_unstale":"2024-05-08T23:26:05.365441+0000","last_undegraded":"2024-05-08T23:26:05.365441+0000","last_fullsized":"2024-05-08T23:26:05.365441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:42.318612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.388729+0000","last_change":"2024-05-08T23:26:03.912054+0000","last_active":"2024-05-08T23:26:04.388729+0000","last_peered":"2024-05-08T23:26:04.388729+0000","last_clean":"2024-05-08T23:26:04.388729+0000","last_became_active":"2024-05-08T23:26:03.911775+0000","last_became_peered":"2024-05-08T23:26:03.911775+0000","last_unstale":"2024-05-08T23:26:04.388729+0000","last_undegraded":"2024-05-08T23:26:04.388729+0000","last_fullsized":"2024-05-08T23:26:04.388729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:25.875643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382844+0000","last_change":"2024-05-08T23:26:04.380260+0000","last_active":"2024-05-08T23:26:04.382844+0000","last_peered":"2024-05-08T23:26:04.382844+0000","last_clean":"2024-05-08T23:26:04.382844+0000","last_became_active":"2024-05-08T23:26:04.379931+0000","last_became_peered":"2024-05-08T23:26:04.379931+0000","last_unstale":"2024-05-08T23:26:04.382844+0000","last_undegraded":"2024-05-08T23:26:04.382844+0000","last_fullsized":"2024-05-08T23:26:04.382844+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:03.230741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106703+0000","last_change":"2024-05-08T23:26:03.917170+0000","last_active":"2024-05-08T23:26:06.106703+0000","last_peered":"2024-05-08T23:26:06.106703+0000","last_clean":"2024-05-08T23:26:06.106703+0000","last_became_active":"2024-05-08T23:26:03.916888+0000","last_became_peered":"2024-05-08T23:26:03.916888+0000","last_unstale":"2024-05-08T23:26:06.106703+0000","last_undegraded":"2024-05-08T23:26:06.106703+0000","last_fullsized":"2024-05-08T23:26:06.106703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:59.835332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.388091+0000","last_change":"2024-05-08T23:26:04.385163+0000","last_active":"2024-05-08T23:26:04.388091+0000","last_peered":"2024-05-08T23:26:04.388091+0000","last_clean":"2024-05-08T23:26:04.388091+0000","last_became_active":"2024-05-08T23:26:04.384430+0000","last_became_peered":"2024-05-08T23:26:04.384430+0000","last_unstale":"2024-05-08T23:26:04.388091+0000","last_undegraded":"2024-05-08T23:26:04.388091+0000","last_fullsized":"2024-05-08T23:26:04.388091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:13.482666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.394524+0000","last_change":"2024-05-08T23:26:03.909206+0000","last_active":"2024-05-08T23:26:04.394524+0000","last_peered":"2024-05-08T23:26:04.394524+0000","last_clean":"2024-05-08T23:26:04.394524+0000","last_became_active":"2024-05-08T23:26:03.908682+0000","last_became_peered":"2024-05-08T23:26:03.908682+0000","last_unstale":"2024-05-08T23:26:04.394524+0000","last_undegraded":"2024-05-08T23:26:04.394524+0000","last_fullsized":"2024-05-08T23:26:04.394524+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:48.022908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365760+0000","last_change":"2024-05-08T23:26:04.382870+0000","last_active":"2024-05-08T23:26:05.365760+0000","last_peered":"2024-05-08T23:26:05.365760+0000","last_clean":"2024-05-08T23:26:05.365760+0000","last_became_active":"2024-05-08T23:26:04.382320+0000","last_became_peered":"2024-05-08T23:26:04.382320+0000","last_unstale":"2024-05-08T23:26:05.365760+0000","last_undegraded":"2024-05-08T23:26:05.365760+0000","last_fullsized":"2024-05-08T23:26:05.365760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:17.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106864+0000","last_change":"2024-05-08T23:26:03.910712+0000","last_active":"2024-05-08T23:26:06.106864+0000","last_peered":"2024-05-08T23:26:06.106864+0000","last_clean":"2024-05-08T23:26:06.106864+0000","last_became_active":"2024-05-08T23:26:03.910424+0000","last_became_peered":"2024-05-08T23:26:03.910424+0000","last_unstale":"2024-05-08T23:26:06.106864+0000","last_undegraded":"2024-05-08T23:26:06.106864+0000","last_fullsized":"2024-05-08T23:26:06.106864+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:16.285704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365799+0000","last_change":"2024-05-08T23:26:04.378589+0000","last_active":"2024-05-08T23:26:05.365799+0000","last_peered":"2024-05-08T23:26:05.365799+0000","last_clean":"2024-05-08T23:26:05.365799+0000","last_became_active":"2024-05-08T23:26:04.378300+0000","last_became_peered":"2024-05-08T23:26:04.378300+0000","last_unstale":"2024-05-08T23:26:05.365799+0000","last_undegraded":"2024-05-08T23:26:05.365799+0000","last_fullsized":"2024-05-08T23:26:05.365799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.786772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.368459+0000","last_change":"2024-05-08T23:26:03.916915+0000","last_active":"2024-05-08T23:26:04.368459+0000","last_peered":"2024-05-08T23:26:04.368459+0000","last_clean":"2024-05-08T23:26:04.368459+0000","last_became_active":"2024-05-08T23:26:03.916632+0000","last_became_peered":"2024-05-08T23:26:03.916632+0000","last_unstale":"2024-05-08T23:26:04.368459+0000","last_undegraded":"2024-05-08T23:26:04.368459+0000","last_fullsized":"2024-05-08T23:26:04.368459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:22.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364817+0000","last_change":"2024-05-08T23:26:04.375139+0000","last_active":"2024-05-08T23:26:05.364817+0000","last_peered":"2024-05-08T23:26:05.364817+0000","last_clean":"2024-05-08T23:26:05.364817+0000","last_became_active":"2024-05-08T23:26:04.374874+0000","last_became_peered":"2024-05-08T23:26:04.374874+0000","last_unstale":"2024-05-08T23:26:05.364817+0000","last_undegraded":"2024-05-08T23:26:05.364817+0000","last_fullsized":"2024-05-08T23:26:05.364817+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:42.364785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106533+0000","last_change":"2024-05-08T23:26:03.915036+0000","last_active":"2024-05-08T23:26:06.106533+0000","last_peered":"2024-05-08T23:26:06.106533+0000","last_clean":"2024-05-08T23:26:06.106533+0000","last_became_active":"2024-05-08T23:26:03.914616+0000","last_became_peered":"2024-05-08T23:26:03.914616+0000","last_unstale":"2024-05-08T23:26:06.106533+0000","last_undegraded":"2024-05-08T23:26:06.106533+0000","last_fullsized":"2024-05-08T23:26:06.106533+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.591942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364460+0000","last_change":"2024-05-08T23:26:04.384503+0000","last_active":"2024-05-08T23:26:05.364460+0000","last_peered":"2024-05-08T23:26:05.364460+0000","last_clean":"2024-05-08T23:26:05.364460+0000","last_became_active":"2024-05-08T23:26:04.383639+0000","last_became_peered":"2024-05-08T23:26:04.383639+0000","last_unstale":"2024-05-08T23:26:05.364460+0000","last_undegraded":"2024-05-08T23:26:05.364460+0000","last_fullsized":"2024-05-08T23:26:05.364460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:13.674487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367498+0000","last_change":"2024-05-08T23:26:03.912730+0000","last_active":"2024-05-08T23:26:04.367498+0000","last_peered":"2024-05-08T23:26:04.367498+0000","last_clean":"2024-05-08T23:26:04.367498+0000","last_became_active":"2024-05-08T23:26:03.912415+0000","last_became_peered":"2024-05-08T23:26:03.912415+0000","last_unstale":"2024-05-08T23:26:04.367498+0000","last_undegraded":"2024-05-08T23:26:04.367498+0000","last_fullsized":"2024-05-08T23:26:04.367498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:35.080457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364494+0000","last_change":"2024-05-08T23:26:04.383931+0000","last_active":"2024-05-08T23:26:05.364494+0000","last_peered":"2024-05-08T23:26:05.364494+0000","last_clean":"2024-05-08T23:26:05.364494+0000","last_became_active":"2024-05-08T23:26:04.383598+0000","last_became_peered":"2024-05-08T23:26:04.383598+0000","last_unstale":"2024-05-08T23:26:05.364494+0000","last_undegraded":"2024-05-08T23:26:05.364494+0000","last_fullsized":"2024-05-08T23:26:05.364494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:27.479478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528163+0000","last_change":"2024-05-08T23:26:03.908029+0000","last_active":"2024-05-08T23:26:05.528163+0000","last_peered":"2024-05-08T23:26:05.528163+0000","last_clean":"2024-05-08T23:26:05.528163+0000","last_became_active":"2024-05-08T23:26:03.907475+0000","last_became_peered":"2024-05-08T23:26:03.907475+0000","last_unstale":"2024-05-08T23:26:05.528163+0000","last_undegraded":"2024-05-08T23:26:05.528163+0000","last_fullsized":"2024-05-08T23:26:05.528163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:26.905129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528734+0000","last_change":"2024-05-08T23:26:04.383939+0000","last_active":"2024-05-08T23:26:05.528734+0000","last_peered":"2024-05-08T23:26:05.528734+0000","last_clean":"2024-05-08T23:26:05.528734+0000","last_became_active":"2024-05-08T23:26:04.383318+0000","last_became_peered":"2024-05-08T23:26:04.383318+0000","last_unstale":"2024-05-08T23:26:05.528734+0000","last_undegraded":"2024-05-08T23:26:05.528734+0000","last_fullsized":"2024-05-08T23:26:05.528734+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106789+0000","last_change":"2024-05-08T23:26:03.917152+0000","last_active":"2024-05-08T23:26:06.106789+0000","last_peered":"2024-05-08T23:26:06.106789+0000","last_clean":"2024-05-08T23:26:06.106789+0000","last_became_active":"2024-05-08T23:26:03.916848+0000","last_became_peered":"2024-05-08T23:26:03.916848+0000","last_unstale":"2024-05-08T23:26:06.106789+0000","last_undegraded":"2024-05-08T23:26:06.106789+0000","last_fullsized":"2024-05-08T23:26:06.106789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:17.395880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.386965+0000","last_change":"2024-05-08T23:26:04.377822+0000","last_active":"2024-05-08T23:26:04.386965+0000","last_peered":"2024-05-08T23:26:04.386965+0000","last_clean":"2024-05-08T23:26:04.386965+0000","last_became_active":"2024-05-08T23:26:04.377510+0000","last_became_peered":"2024-05-08T23:26:04.377510+0000","last_unstale":"2024-05-08T23:26:04.386965+0000","last_undegraded":"2024-05-08T23:26:04.386965+0000","last_fullsized":"2024-05-08T23:26:04.386965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:09.803388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367888+0000","last_change":"2024-05-08T23:26:03.904714+0000","last_active":"2024-05-08T23:26:04.367888+0000","last_peered":"2024-05-08T23:26:04.367888+0000","last_clean":"2024-05-08T23:26:04.367888+0000","last_became_active":"2024-05-08T23:26:03.904414+0000","last_became_peered":"2024-05-08T23:26:03.904414+0000","last_unstale":"2024-05-08T23:26:04.367888+0000","last_undegraded":"2024-05-08T23:26:04.367888+0000","last_fullsized":"2024-05-08T23:26:04.367888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:14.889713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391857+0000","last_change":"2024-05-08T23:26:04.388899+0000","last_active":"2024-05-08T23:26:04.391857+0000","last_peered":"2024-05-08T23:26:04.391857+0000","last_clean":"2024-05-08T23:26:04.391857+0000","last_became_active":"2024-05-08T23:26:04.388628+0000","last_became_peered":"2024-05-08T23:26:04.388628+0000","last_unstale":"2024-05-08T23:26:04.391857+0000","last_undegraded":"2024-05-08T23:26:04.391857+0000","last_fullsized":"2024-05-08T23:26:04.391857+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365794+0000","last_change":"2024-05-08T23:26:03.910274+0000","last_active":"2024-05-08T23:26:05.365794+0000","last_peered":"2024-05-08T23:26:05.365794+0000","last_clean":"2024-05-08T23:26:05.365794+0000","last_became_active":"2024-05-08T23:26:03.909977+0000","last_became_peered":"2024-05-08T23:26:03.909977+0000","last_unstale":"2024-05-08T23:26:05.365794+0000","last_undegraded":"2024-05-08T23:26:05.365794+0000","last_fullsized":"2024-05-08T23:26:05.365794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:09.127009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382218+0000","last_change":"2024-05-08T23:26:04.377703+0000","last_active":"2024-05-08T23:26:04.382218+0000","last_peered":"2024-05-08T23:26:04.382218+0000","last_clean":"2024-05-08T23:26:04.382218+0000","last_became_active":"2024-05-08T23:26:04.377184+0000","last_became_peered":"2024-05-08T23:26:04.377184+0000","last_unstale":"2024-05-08T23:26:04.382218+0000","last_undegraded":"2024-05-08T23:26:04.382218+0000","last_fullsized":"2024-05-08T23:26:04.382218+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528645+0000","last_change":"2024-05-08T23:26:03.910256+0000","last_active":"2024-05-08T23:26:05.528645+0000","last_peered":"2024-05-08T23:26:05.528645+0000","last_clean":"2024-05-08T23:26:05.528645+0000","last_became_active":"2024-05-08T23:26:03.909930+0000","last_became_peered":"2024-05-08T23:26:03.909930+0000","last_unstale":"2024-05-08T23:26:05.528645+0000","last_undegraded":"2024-05-08T23:26:05.528645+0000","last_fullsized":"2024-05-08T23:26:05.528645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:46.987230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366137+0000","last_change":"2024-05-08T23:26:04.390986+0000","last_active":"2024-05-08T23:26:05.366137+0000","last_peered":"2024-05-08T23:26:05.366137+0000","last_clean":"2024-05-08T23:26:05.366137+0000","last_became_active":"2024-05-08T23:26:04.390656+0000","last_became_peered":"2024-05-08T23:26:04.390656+0000","last_unstale":"2024-05-08T23:26:05.366137+0000","last_undegraded":"2024-05-08T23:26:05.366137+0000","last_fullsized":"2024-05-08T23:26:05.366137+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:26.611874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365238+0000","last_change":"2024-05-08T23:26:03.902695+0000","last_active":"2024-05-08T23:26:05.365238+0000","last_peered":"2024-05-08T23:26:05.365238+0000","last_clean":"2024-05-08T23:26:05.365238+0000","last_became_active":"2024-05-08T23:26:03.902382+0000","last_became_peered":"2024-05-08T23:26:03.902382+0000","last_unstale":"2024-05-08T23:26:05.365238+0000","last_undegraded":"2024-05-08T23:26:05.365238+0000","last_fullsized":"2024-05-08T23:26:05.365238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:08.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106322+0000","last_change":"2024-05-08T23:26:04.387935+0000","last_active":"2024-05-08T23:26:06.106322+0000","last_peered":"2024-05-08T23:26:06.106322+0000","last_clean":"2024-05-08T23:26:06.106322+0000","last_became_active":"2024-05-08T23:26:04.387678+0000","last_became_peered":"2024-05-08T23:26:04.387678+0000","last_unstale":"2024-05-08T23:26:06.106322+0000","last_undegraded":"2024-05-08T23:26:06.106322+0000","last_fullsized":"2024-05-08T23:26:06.106322+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:23.335791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364796+0000","last_change":"2024-05-08T23:26:03.916887+0000","last_active":"2024-05-08T23:26:05.364796+0000","last_peered":"2024-05-08T23:26:05.364796+0000","last_clean":"2024-05-08T23:26:05.364796+0000","last_became_active":"2024-05-08T23:26:03.916407+0000","last_became_peered":"2024-05-08T23:26:03.916407+0000","last_unstale":"2024-05-08T23:26:05.364796+0000","last_undegraded":"2024-05-08T23:26:05.364796+0000","last_fullsized":"2024-05-08T23:26:05.364796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:34.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365058+0000","last_change":"2024-05-08T23:26:04.389795+0000","last_active":"2024-05-08T23:26:05.365058+0000","last_peered":"2024-05-08T23:26:05.365058+0000","last_clean":"2024-05-08T23:26:05.365058+0000","last_became_active":"2024-05-08T23:26:04.389433+0000","last_became_peered":"2024-05-08T23:26:04.389433+0000","last_unstale":"2024-05-08T23:26:05.365058+0000","last_undegraded":"2024-05-08T23:26:05.365058+0000","last_fullsized":"2024-05-08T23:26:05.365058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:23.681716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.372078+0000","last_change":"2024-05-08T23:26:03.914133+0000","last_active":"2024-05-08T23:26:04.372078+0000","last_peered":"2024-05-08T23:26:04.372078+0000","last_clean":"2024-05-08T23:26:04.372078+0000","last_became_active":"2024-05-08T23:26:03.913863+0000","last_became_peered":"2024-05-08T23:26:03.913863+0000","last_unstale":"2024-05-08T23:26:04.372078+0000","last_undegraded":"2024-05-08T23:26:04.372078+0000","last_fullsized":"2024-05-08T23:26:04.372078+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:27.115050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106146+0000","last_change":"2024-05-08T23:26:04.390648+0000","last_active":"2024-05-08T23:26:06.106146+0000","last_peered":"2024-05-08T23:26:06.106146+0000","last_clean":"2024-05-08T23:26:06.106146+0000","last_became_active":"2024-05-08T23:26:04.390358+0000","last_became_peered":"2024-05-08T23:26:04.390358+0000","last_unstale":"2024-05-08T23:26:06.106146+0000","last_undegraded":"2024-05-08T23:26:06.106146+0000","last_fullsized":"2024-05-08T23:26:06.106146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:19.130856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106499+0000","last_change":"2024-05-08T23:26:03.908056+0000","last_active":"2024-05-08T23:26:06.106499+0000","last_peered":"2024-05-08T23:26:06.106499+0000","last_clean":"2024-05-08T23:26:06.106499+0000","last_became_active":"2024-05-08T23:26:03.907603+0000","last_became_peered":"2024-05-08T23:26:03.907603+0000","last_unstale":"2024-05-08T23:26:06.106499+0000","last_undegraded":"2024-05-08T23:26:06.106499+0000","last_fullsized":"2024-05-08T23:26:06.106499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:17.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364615+0000","last_change":"2024-05-08T23:26:04.384097+0000","last_active":"2024-05-08T23:26:05.364615+0000","last_peered":"2024-05-08T23:26:05.364615+0000","last_clean":"2024-05-08T23:26:05.364615+0000","last_became_active":"2024-05-08T23:26:04.383812+0000","last_became_peered":"2024-05-08T23:26:04.383812+0000","last_unstale":"2024-05-08T23:26:05.364615+0000","last_undegraded":"2024-05-08T23:26:05.364615+0000","last_fullsized":"2024-05-08T23:26:05.364615+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:08.068641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106387+0000","last_change":"2024-05-08T23:26:03.918580+0000","last_active":"2024-05-08T23:26:06.106387+0000","last_peered":"2024-05-08T23:26:06.106387+0000","last_clean":"2024-05-08T23:26:06.106387+0000","last_became_active":"2024-05-08T23:26:03.918308+0000","last_became_peered":"2024-05-08T23:26:03.918308+0000","last_unstale":"2024-05-08T23:26:06.106387+0000","last_undegraded":"2024-05-08T23:26:06.106387+0000","last_fullsized":"2024-05-08T23:26:06.106387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:22.679586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-05-08T23:26:03.912381+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:26:03.912381+0000","last_peered":"2024-05-08T23:26:03.912381+0000","last_clean":"2024-05-08T23:26:03.912381+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:26:03.912381+0000","last_undegraded":"2024-05-08T23:26:03.912381+0000","last_fullsized":"2024-05-08T23:26:03.912381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1466368,"data_stored":958984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85999999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.495},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67400000000000004,"5min":1.038,"15min":1.038},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76500000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76600000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69899999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73899999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82499999999999996,"5min":0.87,"15min":0.87},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.95799999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996}]},{"osd":2,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":1.115,"15min":1.115},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.436}]},{"osd":5,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76500000000000001,"5min":1.052,"15min":1.052},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84399999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":376480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.371,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56100000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.42299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":1.01,"15min":1.01},"last":0.432},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":1.028,"15min":1.028},"last":0.34200000000000003}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":1.105,"15min":1.105},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.54000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1470464,"data_stored":964668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48399999999999999,"15min":0.72599999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.52900000000000003,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":0.71399999999999997},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60799999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.51600000000000001,"15min":0.77800000000000002},"min":{"1min":0.312,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52900000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54800000000000004,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58099999999999996,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65400000000000003,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69099999999999995,"15min":0.93500000000000005},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.70399999999999996,"15min":0.90800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.91300000000000003},"min":{"1min":0.50700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75700000000000001,"15min":1.0069999999999999},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.73999999999999999}]},{"osd":6,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73499999999999999,"15min":0.97499999999999998},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81799999999999995,"5min":7.734,"15min":7.734},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.77800000000000002}]},{"osd":7,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.96399999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.92200000000000004},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.83299999999999996,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":375864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61799999999999999,"5min":0.621,"15min":0.621},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57399999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.441}]},{"osd":1,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.439,"15min":0.40999999999999998},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.64100000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42699999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79900000000000004,"5min":1.024,"15min":1.024},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81799999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.50700000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.159,"15min":1.159},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91900000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.622}]},{"osd":7,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.70999999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89800000000000002,"5min":1.194,"15min":1.194},"last":0.60199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39500000000000002,"15min":0.41699999999999998},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.58199999999999996,"5min":1.46,"15min":1.46},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.41099999999999998,"15min":0.42099999999999999},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52100000000000002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40100000000000002,"15min":0.40999999999999998},"min":{"1min":0.28999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45800000000000002,"15min":0.46999999999999997},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61199999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.66200000000000003,"15min":0.879},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.56399999999999995,"5min":11.964,"15min":11.964},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.503,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67300000000000004,"5min":1.593,"15min":1.593},"last":0.439}]},{"osd":4,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93200000000000005,"5min":1.375,"15min":1.375},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.624}]},{"osd":5,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.536,"15min":1.536},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.68600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.874,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":1.204,"15min":1.204},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.50700000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.878,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":1.339,"15min":1.339},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73199999999999998,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92800000000000005,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.72399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":868352,"data_stored":368070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.64200000000000002},"min":{"1min":0.33700000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73399999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59499999999999997,"15min":0.72099999999999997},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.746,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.748,"5min":2.145,"15min":2.145},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60199999999999998,"15min":0.68400000000000005},"min":{"1min":0.36899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.623}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68500000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64700000000000002,"15min":0.78500000000000003},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81699999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.74299999999999999,"15min":1.1240000000000001},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.877,"15min":0.877},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64100000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63400000000000001,"15min":0.68200000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73899999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77400000000000002,"5min":1.24,"15min":1.24},"last":0.68799999999999994}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":368090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85199999999999998,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.433},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":0.89400000000000002},"min":{"1min":0.35199999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":8.798,"15min":8.798},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.77500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":4.71,"15min":4.71},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70699999999999996,"15min":0.96799999999999997},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0009999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.79100000000000004},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.82199999999999995,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.93000000000000005},"min":{"1min":0.46999999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78500000000000003,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73899999999999999,"15min":1.0109999999999999},"min":{"1min":0.51100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0489999999999999,"5min":10.427,"15min":10.427},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58599999999999997,"5min":0.874,"15min":0.874},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.502}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85299999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.01},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:26:09.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:26:09.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.648+0000 7f1db6546700 1 Processor -- start 2024-05-08T23:26:09.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.648+0000 7f1db6546700 1 -- start start 2024-05-08T23:26:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1db6546700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db00b2340 0x7f1db00cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1db6546700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1db00b4620 0x7f1db00b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1db6546700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db009ad40 con 0x7f1db015ef60 2024-05-08T23:26:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1db6546700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db00cd400 con 0x7f1db00b4620 2024-05-08T23:26:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1db6546700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00ccb60 con 0x7f1db00b2340 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1daf7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db00b2340 0x7f1db00cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1db4ae3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1db00b4620 0x7f1db00b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1db4ae3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1db00b4620 0x7f1db00b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40276/0 (socket says 172.21.15.5:40276) 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1db4ae3700 1 -- 172.21.15.5:0/3217460046 learned_addr learned my addr 172.21.15.5:0/3217460046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.649+0000 7f1daf7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db00b2340 0x7f1db00cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:47656/0 (socket says 172.21.15.5:47656) 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1db4ae3700 1 -- 172.21.15.5:0/3217460046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db00b2340 msgr2=0x7f1db00cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1db4ae3700 1 --2- 172.21.15.5:0/3217460046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db00b2340 0x7f1db00cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1db4ae3700 1 -- 172.21.15.5:0/3217460046 >> v1:172.21.15.49:6789/0 conn(0x7f1db015ef60 legacy=0x7f1db00cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1db4ae3700 1 -- 172.21.15.5:0/3217460046 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00cce40 con 0x7f1db00b4620 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1daeffd700 1 -- 172.21.15.5:0/3217460046 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159727574 0 0) 0x7f1db009ad40 con 0x7f1db015ef60 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1daf7fe700 1 --2- 172.21.15.5:0/3217460046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db00b2340 0x7f1db00cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1db4ae3700 1 --2- 172.21.15.5:0/3217460046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b4620 0x7f1db00b4ab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009e50 tx=0x7f1da800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8480ed9c46de184 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1daeffd700 1 -- 172.21.15.5:0/3217460046 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da800e070 con 0x7f1db00b4620 2024-05-08T23:26:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1daeffd700 1 -- 172.21.15.5:0/3217460046 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da800b910 con 0x7f1db00b4620 2024-05-08T23:26:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.650+0000 7f1daeffd700 1 -- 172.21.15.5:0/3217460046 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da800bcc0 con 0x7f1db00b4620 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.651+0000 7f1db6546700 1 -- 172.21.15.5:0/3217460046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b4620 msgr2=0x7f1db00b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.651+0000 7f1db6546700 1 --2- 172.21.15.5:0/3217460046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b4620 0x7f1db00b4ab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009e50 tx=0x7f1da800b330 comp rx=0 tx=0).stop 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.651+0000 7f1db6546700 1 -- 172.21.15.5:0/3217460046 shutdown_connections 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.651+0000 7f1db6546700 1 --2- 172.21.15.5:0/3217460046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b4620 0x7f1db00b4ab0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.651+0000 7f1db6546700 1 --2- 172.21.15.5:0/3217460046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db00b2340 0x7f1db00cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.651+0000 7f1db6546700 1 -- 172.21.15.5:0/3217460046 >> 172.21.15.5:0/3217460046 conn(0x7f1db00b2f20 msgr2=0x7f1db00b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.651+0000 7f1db6546700 1 -- 172.21.15.5:0/3217460046 shutdown_connections 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.651+0000 7f1db6546700 1 -- 172.21.15.5:0/3217460046 wait complete. 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.652+0000 7f1db6546700 1 Processor -- start 2024-05-08T23:26:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.653+0000 7f1db6546700 1 -- start start 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.653+0000 7f1db6546700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b2340 0x7f1db014f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.653+0000 7f1db6546700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1db00b4620 0x7f1db00c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.653+0000 7f1db6546700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db015ef60 0x7f1db00c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.653+0000 7f1db6546700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db014fe80 con 0x7f1db00b4620 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.653+0000 7f1db6546700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db01f1570 con 0x7f1db00b2340 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.653+0000 7f1db6546700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00c51d0 con 0x7f1db015ef60 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.653+0000 7f1daf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1db00b4620 0x7f1db00c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1db00b4620 0x7f1db00c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44826/0 (socket says 172.21.15.5:44826) 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daf7fe700 1 -- 172.21.15.5:0/1422995372 learned_addr learned my addr 172.21.15.5:0/1422995372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1db4ae3700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db015ef60 0x7f1db00c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daffff700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b2340 0x7f1db014f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daf7fe700 1 -- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db015ef60 msgr2=0x7f1db00c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daf7fe700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db015ef60 0x7f1db00c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daf7fe700 1 -- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b2340 msgr2=0x7f1db014f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daf7fe700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b2340 0x7f1db014f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daf7fe700 1 -- 172.21.15.5:0/1422995372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da40094a0 con 0x7f1db00b4620 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1daf7fe700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1db00b4620 0x7f1db00c5dd0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1da400ebe0 tx=0x7f1da400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1dacff9700 1 -- 172.21.15.5:0/1422995372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4009760 con 0x7f1db00b4620 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1dacff9700 1 -- 172.21.15.5:0/1422995372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da400c850 con 0x7f1db00b4620 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da8009b30 con 0x7f1db00b4620 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1dacff9700 1 -- 172.21.15.5:0/1422995372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da400cbe0 con 0x7f1db00b4620 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.654+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db00c6c90 con 0x7f1db00b4620 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.655+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db01f1710 con 0x7f1db00b4620 2024-05-08T23:26:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.656+0000 7f1dacff9700 1 -- 172.21.15.5:0/1422995372 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1da400cd40 con 0x7f1db00b4620 2024-05-08T23:26:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.656+0000 7f1dacff9700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d9c06acc0 0x7f1d9c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.656+0000 7f1dacff9700 1 -- 172.21.15.5:0/1422995372 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1da40892d0 con 0x7f1db00b4620 2024-05-08T23:26:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.657+0000 7f1daffff700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d9c06acc0 0x7f1d9c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.657+0000 7f1daffff700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d9c06acc0 0x7f1d9c06d180 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000f60 tx=0x7f1da0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.661+0000 7f1dacff9700 1 -- 172.21.15.5:0/1422995372 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1db01f1710 con 0x7f1db00b4620 2024-05-08T23:26:09.887 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:26:09.887+0000 7f42240c8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:26:09.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.971+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1db01f1710 con 0x7f1db00b4620 2024-05-08T23:26:09.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.972+0000 7f1dacff9700 1 -- 172.21.15.5:0/1422995372 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1432 (secure 0 0 0) 0x7f1db01f1710 con 0x7f1db00b4620 2024-05-08T23:26:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:09.972 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":324,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":72,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":242184192,"bytes_avail":772851929088,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":66,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":7179}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-05-08T23:25:41.360892+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d9c06acc0 msgr2=0x7f1d9c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d9c06acc0 0x7f1d9c06d180 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000f60 tx=0x7f1da0009040 comp rx=0 tx=0).stop 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1db00b4620 msgr2=0x7f1db00c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1db00b4620 0x7f1db00c5dd0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1da400ebe0 tx=0x7f1da400eef0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 shutdown_connections 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d9c06acc0 0x7f1d9c06d180 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1db015ef60 0x7f1db00c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1db00b2340 0x7f1db014f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 --2- 172.21.15.5:0/1422995372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1db00b4620 0x7f1db00c5dd0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:09.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.974+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 >> 172.21.15.5:0/1422995372 conn(0x7f1db00b2f20 msgr2=0x7f1db00b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.975+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 shutdown_connections 2024-05-08T23:26:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:09.975+0000 7f1db6546700 1 -- 172.21.15.5:0/1422995372 wait complete. 2024-05-08T23:26:09.988 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T23:26:09.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:26:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.124+0000 7f4f4ee0d700 1 Processor -- start 2024-05-08T23:26:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.124+0000 7f4f4ee0d700 1 -- start start 2024-05-08T23:26:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.125+0000 7f4f4ee0d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f480cd4e0 0x7f4f4806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4ee0d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4815c280 con 0x7f4f480cecf0 2024-05-08T23:26:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4ee0d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4806e780 con 0x7f4f4806f130 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4ee0d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4806ea60 con 0x7f4f480cd4e0 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4cba9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f480cd4e0 0x7f4f4806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4cba9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f480cd4e0 0x7f4f4806f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44838/0 (socket says 172.21.15.5:44838) 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4d3aa700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4f480cecf0 0x7f4f4815c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38912/0 (socket says 172.21.15.5:38912) 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4cba9700 1 -- 172.21.15.5:0/3267724680 learned_addr learned my addr 172.21.15.5:0/3267724680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4cba9700 1 -- 172.21.15.5:0/3267724680 >> v1:172.21.15.49:6789/0 conn(0x7f4f4806f130 legacy=0x7f4f480ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4cba9700 1 -- 172.21.15.5:0/3267724680 >> v1:172.21.15.5:6789/0 conn(0x7f4f480cecf0 legacy=0x7f4f4815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4cba9700 1 -- 172.21.15.5:0/3267724680 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4806ed40 con 0x7f4f480cd4e0 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f477fe700 1 -- 172.21.15.5:0/3267724680 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824214118 0 0) 0x7f4f4815c280 con 0x7f4f480cecf0 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f4cba9700 1 --2- 172.21.15.5:0/3267724680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cd4e0 0x7f4f4806f910 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4f380098f0 tx=0x7f4f38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99babce4fee275d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f477fe700 1 -- 172.21.15.5:0/3267724680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38004230 con 0x7f4f480cd4e0 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f477fe700 1 -- 172.21.15.5:0/3267724680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f38004390 con 0x7f4f480cd4e0 2024-05-08T23:26:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.126+0000 7f4f477fe700 1 -- 172.21.15.5:0/3267724680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f380036a0 con 0x7f4f480cd4e0 2024-05-08T23:26:10.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.127+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/3267724680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cd4e0 msgr2=0x7f4f4806f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.127+0000 7f4f4ee0d700 1 --2- 172.21.15.5:0/3267724680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cd4e0 0x7f4f4806f910 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4f380098f0 tx=0x7f4f38009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.127+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/3267724680 shutdown_connections 2024-05-08T23:26:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.127+0000 7f4f4ee0d700 1 --2- 172.21.15.5:0/3267724680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cd4e0 0x7f4f4806f910 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.127+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/3267724680 >> 172.21.15.5:0/3267724680 conn(0x7f4f480d0320 msgr2=0x7f4f480d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.127+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/3267724680 shutdown_connections 2024-05-08T23:26:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.127+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/3267724680 wait complete. 2024-05-08T23:26:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.128+0000 7f4f4ee0d700 1 Processor -- start 2024-05-08T23:26:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.128+0000 7f4f4ee0d700 1 -- start start 2024-05-08T23:26:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.129+0000 7f4f4ee0d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f4806f130 0x7f4f48158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.129+0000 7f4f4ee0d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f480cd4e0 0x7f4f480b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.129+0000 7f4f4ee0d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cecf0 0x7f4f480b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.129+0000 7f4f4ee0d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480b2fe0 con 0x7f4f480cecf0 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.129+0000 7f4f4ee0d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f481591a0 con 0x7f4f480cd4e0 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.129+0000 7f4f4ee0d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480b08a0 con 0x7f4f4806f130 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cecf0 0x7f4f480b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cecf0 0x7f4f480b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44852/0 (socket says 172.21.15.5:44852) 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 -- 172.21.15.5:0/2453349567 learned_addr learned my addr 172.21.15.5:0/2453349567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f47fff700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f480cd4e0 0x7f4f480b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4cba9700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f4806f130 0x7f4f48158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 -- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f4806f130 msgr2=0x7f4f48158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f4806f130 0x7f4f48158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 -- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f480cd4e0 msgr2=0x7f4f480b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f480cd4e0 0x7f4f480b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 -- 172.21.15.5:0/2453349567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38009550 con 0x7f4f480cecf0 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4d3aa700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cecf0 0x7f4f480b19a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000cb60 tx=0x7f4f4000ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f457fa700 1 -- 172.21.15.5:0/2453349567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000a980 con 0x7f4f480cecf0 2024-05-08T23:26:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f457fa700 1 -- 172.21.15.5:0/2453349567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4000aae0 con 0x7f4f480cecf0 2024-05-08T23:26:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f480b0b80 con 0x7f4f480cecf0 2024-05-08T23:26:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f457fa700 1 -- 172.21.15.5:0/2453349567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000ae90 con 0x7f4f480cecf0 2024-05-08T23:26:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.130+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f480b1140 con 0x7f4f480cecf0 2024-05-08T23:26:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.131+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f48068000 con 0x7f4f480cecf0 2024-05-08T23:26:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.132+0000 7f4f457fa700 1 -- 172.21.15.5:0/2453349567 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4f400070d0 con 0x7f4f480cecf0 2024-05-08T23:26:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.132+0000 7f4f457fa700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f2806ac90 0x7f4f2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.132+0000 7f4f457fa700 1 -- 172.21.15.5:0/2453349567 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f40088d10 con 0x7f4f480cecf0 2024-05-08T23:26:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.133+0000 7f4f4cba9700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f2806ac90 0x7f4f2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.133+0000 7f4f4cba9700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f2806ac90 0x7f4f2806d150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009130 tx=0x7f4f38008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.137+0000 7f4f457fa700 1 -- 172.21.15.5:0/2453349567 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4f40053ef0 con 0x7f4f480cecf0 2024-05-08T23:26:10.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.350+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f4806e540 con 0x7f4f2806ac90 2024-05-08T23:26:10.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.359+0000 7f4f457fa700 1 -- 172.21.15.5:0/2453349567 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289729 (secure 0 0 0) 0x7f4f4806e540 con 0x7f4f2806ac90 2024-05-08T23:26:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:10.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f2806ac90 msgr2=0x7f4f2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:10.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f2806ac90 0x7f4f2806d150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009130 tx=0x7f4f38008030 comp rx=0 tx=0).stop 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cecf0 msgr2=0x7f4f480b19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cecf0 0x7f4f480b19a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000cb60 tx=0x7f4f4000ce70 comp rx=0 tx=0).stop 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 shutdown_connections 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f2806ac90 0x7f4f2806d150 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f480cecf0 0x7f4f480b19a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f4806f130 0x7f4f48158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 --2- 172.21.15.5:0/2453349567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f480cd4e0 0x7f4f480b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 >> 172.21.15.5:0/2453349567 conn(0x7f4f480d0320 msgr2=0x7f4f4815b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 shutdown_connections 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:10.363+0000 7f4f4ee0d700 1 -- 172.21.15.5:0/2453349567 wait complete. 2024-05-08T23:26:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:26:10.377 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-05-08T23:26:09.367539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236508,"kb_used_data":8036,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":754738212,"statfs":{"total":773094113280,"available":772851929088,"internally_reserved":0,"allocated":8228864,"data_stored":4160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":233819701},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001099"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.385435+0000","last_change":"2024-05-08T23:26:04.382812+0000","last_active":"2024-05-08T23:26:04.385435+0000","last_peered":"2024-05-08T23:26:04.385435+0000","last_clean":"2024-05-08T23:26:04.385435+0000","last_became_active":"2024-05-08T23:26:04.382518+0000","last_became_peered":"2024-05-08T23:26:04.382518+0000","last_unstale":"2024-05-08T23:26:04.385435+0000","last_undegraded":"2024-05-08T23:26:04.385435+0000","last_fullsized":"2024-05-08T23:26:04.385435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:16.459465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106783+0000","last_change":"2024-05-08T23:26:04.386202+0000","last_active":"2024-05-08T23:26:06.106783+0000","last_peered":"2024-05-08T23:26:06.106783+0000","last_clean":"2024-05-08T23:26:06.106783+0000","last_became_active":"2024-05-08T23:26:04.385897+0000","last_became_peered":"2024-05-08T23:26:04.385897+0000","last_unstale":"2024-05-08T23:26:06.106783+0000","last_undegraded":"2024-05-08T23:26:06.106783+0000","last_fullsized":"2024-05-08T23:26:06.106783+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:15.891844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365071+0000","last_change":"2024-05-08T23:26:04.376144+0000","last_active":"2024-05-08T23:26:05.365071+0000","last_peered":"2024-05-08T23:26:05.365071+0000","last_clean":"2024-05-08T23:26:05.365071+0000","last_became_active":"2024-05-08T23:26:04.375870+0000","last_became_peered":"2024-05-08T23:26:04.375870+0000","last_unstale":"2024-05-08T23:26:05.365071+0000","last_undegraded":"2024-05-08T23:26:05.365071+0000","last_fullsized":"2024-05-08T23:26:05.365071+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:53.570674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365640+0000","last_change":"2024-05-08T23:26:04.391205+0000","last_active":"2024-05-08T23:26:05.365640+0000","last_peered":"2024-05-08T23:26:05.365640+0000","last_clean":"2024-05-08T23:26:05.365640+0000","last_became_active":"2024-05-08T23:26:04.390895+0000","last_became_peered":"2024-05-08T23:26:04.390895+0000","last_unstale":"2024-05-08T23:26:05.365640+0000","last_undegraded":"2024-05-08T23:26:05.365640+0000","last_fullsized":"2024-05-08T23:26:05.365640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:00.837462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365140+0000","last_change":"2024-05-08T23:26:04.380499+0000","last_active":"2024-05-08T23:26:05.365140+0000","last_peered":"2024-05-08T23:26:05.365140+0000","last_clean":"2024-05-08T23:26:05.365140+0000","last_became_active":"2024-05-08T23:26:04.380163+0000","last_became_peered":"2024-05-08T23:26:04.380163+0000","last_unstale":"2024-05-08T23:26:05.365140+0000","last_undegraded":"2024-05-08T23:26:05.365140+0000","last_fullsized":"2024-05-08T23:26:05.365140+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:39.534202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.392388+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.392064+0000","last_became_peered":"2024-05-08T23:26:04.392064+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:55.698089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.387389+0000","last_change":"2024-05-08T23:26:04.377831+0000","last_active":"2024-05-08T23:26:04.387389+0000","last_peered":"2024-05-08T23:26:04.387389+0000","last_clean":"2024-05-08T23:26:04.387389+0000","last_became_active":"2024-05-08T23:26:04.377518+0000","last_became_peered":"2024-05-08T23:26:04.377518+0000","last_unstale":"2024-05-08T23:26:04.387389+0000","last_undegraded":"2024-05-08T23:26:04.387389+0000","last_fullsized":"2024-05-08T23:26:04.387389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106353+0000","last_change":"2024-05-08T23:26:04.388002+0000","last_active":"2024-05-08T23:26:06.106353+0000","last_peered":"2024-05-08T23:26:06.106353+0000","last_clean":"2024-05-08T23:26:06.106353+0000","last_became_active":"2024-05-08T23:26:04.386006+0000","last_became_peered":"2024-05-08T23:26:04.386006+0000","last_unstale":"2024-05-08T23:26:06.106353+0000","last_undegraded":"2024-05-08T23:26:06.106353+0000","last_fullsized":"2024-05-08T23:26:06.106353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:34:35.709822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365561+0000","last_change":"2024-05-08T23:26:04.389861+0000","last_active":"2024-05-08T23:26:05.365561+0000","last_peered":"2024-05-08T23:26:05.365561+0000","last_clean":"2024-05-08T23:26:05.365561+0000","last_became_active":"2024-05-08T23:26:04.389582+0000","last_became_peered":"2024-05-08T23:26:04.389582+0000","last_unstale":"2024-05-08T23:26:05.365561+0000","last_undegraded":"2024-05-08T23:26:05.365561+0000","last_fullsized":"2024-05-08T23:26:05.365561+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:12.594073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389862+0000","last_change":"2024-05-08T23:26:04.384004+0000","last_active":"2024-05-08T23:26:04.389862+0000","last_peered":"2024-05-08T23:26:04.389862+0000","last_clean":"2024-05-08T23:26:04.389862+0000","last_became_active":"2024-05-08T23:26:04.383584+0000","last_became_peered":"2024-05-08T23:26:04.383584+0000","last_unstale":"2024-05-08T23:26:04.389862+0000","last_undegraded":"2024-05-08T23:26:04.389862+0000","last_fullsized":"2024-05-08T23:26:04.389862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:52.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391883+0000","last_change":"2024-05-08T23:26:04.390913+0000","last_active":"2024-05-08T23:26:04.391883+0000","last_peered":"2024-05-08T23:26:04.391883+0000","last_clean":"2024-05-08T23:26:04.391883+0000","last_became_active":"2024-05-08T23:26:04.390654+0000","last_became_peered":"2024-05-08T23:26:04.390654+0000","last_unstale":"2024-05-08T23:26:04.391883+0000","last_undegraded":"2024-05-08T23:26:04.391883+0000","last_fullsized":"2024-05-08T23:26:04.391883+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365672+0000","last_change":"2024-05-08T23:26:04.382718+0000","last_active":"2024-05-08T23:26:05.365672+0000","last_peered":"2024-05-08T23:26:05.365672+0000","last_clean":"2024-05-08T23:26:05.365672+0000","last_became_active":"2024-05-08T23:26:04.381951+0000","last_became_peered":"2024-05-08T23:26:04.381951+0000","last_unstale":"2024-05-08T23:26:05.365672+0000","last_undegraded":"2024-05-08T23:26:05.365672+0000","last_fullsized":"2024-05-08T23:26:05.365672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:18.398011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528368+0000","last_change":"2024-05-08T23:26:04.393225+0000","last_active":"2024-05-08T23:26:05.528368+0000","last_peered":"2024-05-08T23:26:05.528368+0000","last_clean":"2024-05-08T23:26:05.528368+0000","last_became_active":"2024-05-08T23:26:04.392933+0000","last_became_peered":"2024-05-08T23:26:04.392933+0000","last_unstale":"2024-05-08T23:26:05.528368+0000","last_undegraded":"2024-05-08T23:26:05.528368+0000","last_fullsized":"2024-05-08T23:26:05.528368+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366036+0000","last_change":"2024-05-08T23:26:04.380528+0000","last_active":"2024-05-08T23:26:05.366036+0000","last_peered":"2024-05-08T23:26:05.366036+0000","last_clean":"2024-05-08T23:26:05.366036+0000","last_became_active":"2024-05-08T23:26:04.380129+0000","last_became_peered":"2024-05-08T23:26:04.380129+0000","last_unstale":"2024-05-08T23:26:05.366036+0000","last_undegraded":"2024-05-08T23:26:05.366036+0000","last_fullsized":"2024-05-08T23:26:05.366036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.038766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528231+0000","last_change":"2024-05-08T23:26:04.382434+0000","last_active":"2024-05-08T23:26:05.528231+0000","last_peered":"2024-05-08T23:26:05.528231+0000","last_clean":"2024-05-08T23:26:05.528231+0000","last_became_active":"2024-05-08T23:26:04.382015+0000","last_became_peered":"2024-05-08T23:26:04.382015+0000","last_unstale":"2024-05-08T23:26:05.528231+0000","last_undegraded":"2024-05-08T23:26:05.528231+0000","last_fullsized":"2024-05-08T23:26:05.528231+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.395912+0000","last_change":"2024-05-08T23:26:04.395148+0000","last_active":"2024-05-08T23:26:04.395912+0000","last_peered":"2024-05-08T23:26:04.395912+0000","last_clean":"2024-05-08T23:26:04.395912+0000","last_became_active":"2024-05-08T23:26:04.394850+0000","last_became_peered":"2024-05-08T23:26:04.394850+0000","last_unstale":"2024-05-08T23:26:04.395912+0000","last_undegraded":"2024-05-08T23:26:04.395912+0000","last_fullsized":"2024-05-08T23:26:04.395912+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:39.285912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365565+0000","last_change":"2024-05-08T23:26:04.386467+0000","last_active":"2024-05-08T23:26:05.365565+0000","last_peered":"2024-05-08T23:26:05.365565+0000","last_clean":"2024-05-08T23:26:05.365565+0000","last_became_active":"2024-05-08T23:26:04.386005+0000","last_became_peered":"2024-05-08T23:26:04.386005+0000","last_unstale":"2024-05-08T23:26:05.365565+0000","last_undegraded":"2024-05-08T23:26:05.365565+0000","last_fullsized":"2024-05-08T23:26:05.365565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:08.141995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365004+0000","last_change":"2024-05-08T23:26:04.384544+0000","last_active":"2024-05-08T23:26:05.365004+0000","last_peered":"2024-05-08T23:26:05.365004+0000","last_clean":"2024-05-08T23:26:05.365004+0000","last_became_active":"2024-05-08T23:26:04.384240+0000","last_became_peered":"2024-05-08T23:26:04.384240+0000","last_unstale":"2024-05-08T23:26:05.365004+0000","last_undegraded":"2024-05-08T23:26:05.365004+0000","last_fullsized":"2024-05-08T23:26:05.365004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:44.177271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106649+0000","last_change":"2024-05-08T23:26:04.387389+0000","last_active":"2024-05-08T23:26:06.106649+0000","last_peered":"2024-05-08T23:26:06.106649+0000","last_clean":"2024-05-08T23:26:06.106649+0000","last_became_active":"2024-05-08T23:26:04.383192+0000","last_became_peered":"2024-05-08T23:26:04.383192+0000","last_unstale":"2024-05-08T23:26:06.106649+0000","last_undegraded":"2024-05-08T23:26:06.106649+0000","last_fullsized":"2024-05-08T23:26:06.106649+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528319+0000","last_change":"2024-05-08T23:26:04.374428+0000","last_active":"2024-05-08T23:26:05.528319+0000","last_peered":"2024-05-08T23:26:05.528319+0000","last_clean":"2024-05-08T23:26:05.528319+0000","last_became_active":"2024-05-08T23:26:04.374165+0000","last_became_peered":"2024-05-08T23:26:04.374165+0000","last_unstale":"2024-05-08T23:26:05.528319+0000","last_undegraded":"2024-05-08T23:26:05.528319+0000","last_fullsized":"2024-05-08T23:26:05.528319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.381684+0000","last_change":"2024-05-08T23:26:04.378977+0000","last_active":"2024-05-08T23:26:04.381684+0000","last_peered":"2024-05-08T23:26:04.381684+0000","last_clean":"2024-05-08T23:26:04.381684+0000","last_became_active":"2024-05-08T23:26:04.378470+0000","last_became_peered":"2024-05-08T23:26:04.378470+0000","last_unstale":"2024-05-08T23:26:04.381684+0000","last_undegraded":"2024-05-08T23:26:04.381684+0000","last_fullsized":"2024-05-08T23:26:04.381684+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.979369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389858+0000","last_change":"2024-05-08T23:26:04.383843+0000","last_active":"2024-05-08T23:26:04.389858+0000","last_peered":"2024-05-08T23:26:04.389858+0000","last_clean":"2024-05-08T23:26:04.389858+0000","last_became_active":"2024-05-08T23:26:04.383503+0000","last_became_peered":"2024-05-08T23:26:04.383503+0000","last_unstale":"2024-05-08T23:26:04.389858+0000","last_undegraded":"2024-05-08T23:26:04.389858+0000","last_fullsized":"2024-05-08T23:26:04.389858+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:17.695615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382847+0000","last_change":"2024-05-08T23:26:04.380605+0000","last_active":"2024-05-08T23:26:04.382847+0000","last_peered":"2024-05-08T23:26:04.382847+0000","last_clean":"2024-05-08T23:26:04.382847+0000","last_became_active":"2024-05-08T23:26:04.380307+0000","last_became_peered":"2024-05-08T23:26:04.380307+0000","last_unstale":"2024-05-08T23:26:04.382847+0000","last_undegraded":"2024-05-08T23:26:04.382847+0000","last_fullsized":"2024-05-08T23:26:04.382847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:27.367580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.395921+0000","last_change":"2024-05-08T23:26:04.395150+0000","last_active":"2024-05-08T23:26:04.395921+0000","last_peered":"2024-05-08T23:26:04.395921+0000","last_clean":"2024-05-08T23:26:04.395921+0000","last_became_active":"2024-05-08T23:26:04.394845+0000","last_became_peered":"2024-05-08T23:26:04.394845+0000","last_unstale":"2024-05-08T23:26:04.395921+0000","last_undegraded":"2024-05-08T23:26:04.395921+0000","last_fullsized":"2024-05-08T23:26:04.395921+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389884+0000","last_change":"2024-05-08T23:26:04.383933+0000","last_active":"2024-05-08T23:26:04.389884+0000","last_peered":"2024-05-08T23:26:04.389884+0000","last_clean":"2024-05-08T23:26:04.389884+0000","last_became_active":"2024-05-08T23:26:04.383614+0000","last_became_peered":"2024-05-08T23:26:04.383614+0000","last_unstale":"2024-05-08T23:26:04.389884+0000","last_undegraded":"2024-05-08T23:26:04.389884+0000","last_fullsized":"2024-05-08T23:26:04.389884+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:47.989361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.390865+0000","last_change":"2024-05-08T23:26:04.387174+0000","last_active":"2024-05-08T23:26:04.390865+0000","last_peered":"2024-05-08T23:26:04.390865+0000","last_clean":"2024-05-08T23:26:04.390865+0000","last_became_active":"2024-05-08T23:26:04.386653+0000","last_became_peered":"2024-05-08T23:26:04.386653+0000","last_unstale":"2024-05-08T23:26:04.390865+0000","last_undegraded":"2024-05-08T23:26:04.390865+0000","last_fullsized":"2024-05-08T23:26:04.390865+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:52.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365110+0000","last_change":"2024-05-08T23:26:04.374586+0000","last_active":"2024-05-08T23:26:05.365110+0000","last_peered":"2024-05-08T23:26:05.365110+0000","last_clean":"2024-05-08T23:26:05.365110+0000","last_became_active":"2024-05-08T23:26:04.374382+0000","last_became_peered":"2024-05-08T23:26:04.374382+0000","last_unstale":"2024-05-08T23:26:05.365110+0000","last_undegraded":"2024-05-08T23:26:05.365110+0000","last_fullsized":"2024-05-08T23:26:05.365110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:41.989450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389885+0000","last_change":"2024-05-08T23:26:04.384048+0000","last_active":"2024-05-08T23:26:04.389885+0000","last_peered":"2024-05-08T23:26:04.389885+0000","last_clean":"2024-05-08T23:26:04.389885+0000","last_became_active":"2024-05-08T23:26:04.383520+0000","last_became_peered":"2024-05-08T23:26:04.383520+0000","last_unstale":"2024-05-08T23:26:04.389885+0000","last_undegraded":"2024-05-08T23:26:04.389885+0000","last_fullsized":"2024-05-08T23:26:04.389885+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:09.803388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364727+0000","last_change":"2024-05-08T23:26:04.376502+0000","last_active":"2024-05-08T23:26:05.364727+0000","last_peered":"2024-05-08T23:26:05.364727+0000","last_clean":"2024-05-08T23:26:05.364727+0000","last_became_active":"2024-05-08T23:26:04.376226+0000","last_became_peered":"2024-05-08T23:26:04.376226+0000","last_unstale":"2024-05-08T23:26:05.364727+0000","last_undegraded":"2024-05-08T23:26:05.364727+0000","last_fullsized":"2024-05-08T23:26:05.364727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:36.452021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.393993+0000","last_change":"2024-05-08T23:26:04.393135+0000","last_active":"2024-05-08T23:26:04.393993+0000","last_peered":"2024-05-08T23:26:04.393993+0000","last_clean":"2024-05-08T23:26:04.393993+0000","last_became_active":"2024-05-08T23:26:04.392824+0000","last_became_peered":"2024-05-08T23:26:04.392824+0000","last_unstale":"2024-05-08T23:26:04.393993+0000","last_undegraded":"2024-05-08T23:26:04.393993+0000","last_fullsized":"2024-05-08T23:26:04.393993+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:47.989361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391112+0000","last_change":"2024-05-08T23:26:04.387882+0000","last_active":"2024-05-08T23:26:04.391112+0000","last_peered":"2024-05-08T23:26:04.391112+0000","last_clean":"2024-05-08T23:26:04.391112+0000","last_became_active":"2024-05-08T23:26:04.387600+0000","last_became_peered":"2024-05-08T23:26:04.387600+0000","last_unstale":"2024-05-08T23:26:04.391112+0000","last_undegraded":"2024-05-08T23:26:04.391112+0000","last_fullsized":"2024-05-08T23:26:04.391112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:29.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106297+0000","last_change":"2024-05-08T23:26:04.390849+0000","last_active":"2024-05-08T23:26:06.106297+0000","last_peered":"2024-05-08T23:26:06.106297+0000","last_clean":"2024-05-08T23:26:06.106297+0000","last_became_active":"2024-05-08T23:26:04.390577+0000","last_became_peered":"2024-05-08T23:26:04.390577+0000","last_unstale":"2024-05-08T23:26:06.106297+0000","last_undegraded":"2024-05-08T23:26:06.106297+0000","last_fullsized":"2024-05-08T23:26:06.106297+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:00.857944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389116+0000","last_change":"2024-05-08T23:26:04.387838+0000","last_active":"2024-05-08T23:26:04.389116+0000","last_peered":"2024-05-08T23:26:04.389116+0000","last_clean":"2024-05-08T23:26:04.389116+0000","last_became_active":"2024-05-08T23:26:04.387594+0000","last_became_peered":"2024-05-08T23:26:04.387594+0000","last_unstale":"2024-05-08T23:26:04.389116+0000","last_undegraded":"2024-05-08T23:26:04.389116+0000","last_fullsized":"2024-05-08T23:26:04.389116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.389890+0000","last_change":"2024-05-08T23:26:03.918800+0000","last_active":"2024-05-08T23:26:04.389890+0000","last_peered":"2024-05-08T23:26:04.389890+0000","last_clean":"2024-05-08T23:26:04.389890+0000","last_became_active":"2024-05-08T23:26:03.918509+0000","last_became_peered":"2024-05-08T23:26:03.918509+0000","last_unstale":"2024-05-08T23:26:04.389890+0000","last_undegraded":"2024-05-08T23:26:04.389890+0000","last_fullsized":"2024-05-08T23:26:04.389890+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:46.488568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365685+0000","last_change":"2024-05-08T23:26:04.386283+0000","last_active":"2024-05-08T23:26:05.365685+0000","last_peered":"2024-05-08T23:26:05.365685+0000","last_clean":"2024-05-08T23:26:05.365685+0000","last_became_active":"2024-05-08T23:26:04.385690+0000","last_became_peered":"2024-05-08T23:26:04.385690+0000","last_unstale":"2024-05-08T23:26:05.365685+0000","last_undegraded":"2024-05-08T23:26:05.365685+0000","last_fullsized":"2024-05-08T23:26:05.365685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:12.667029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528769+0000","last_change":"2024-05-08T23:26:03.912463+0000","last_active":"2024-05-08T23:26:05.528769+0000","last_peered":"2024-05-08T23:26:05.528769+0000","last_clean":"2024-05-08T23:26:05.528769+0000","last_became_active":"2024-05-08T23:26:03.912178+0000","last_became_peered":"2024-05-08T23:26:03.912178+0000","last_unstale":"2024-05-08T23:26:05.528769+0000","last_undegraded":"2024-05-08T23:26:05.528769+0000","last_fullsized":"2024-05-08T23:26:05.528769+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:02.228610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.385614+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.385325+0000","last_became_peered":"2024-05-08T23:26:04.385325+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.979369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.394595+0000","last_change":"2024-05-08T23:26:03.911580+0000","last_active":"2024-05-08T23:26:04.394595+0000","last_peered":"2024-05-08T23:26:04.394595+0000","last_clean":"2024-05-08T23:26:04.394595+0000","last_became_active":"2024-05-08T23:26:03.911071+0000","last_became_peered":"2024-05-08T23:26:03.911071+0000","last_unstale":"2024-05-08T23:26:04.394595+0000","last_undegraded":"2024-05-08T23:26:04.394595+0000","last_fullsized":"2024-05-08T23:26:04.394595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:42:39.409566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382824+0000","last_change":"2024-05-08T23:26:04.380473+0000","last_active":"2024-05-08T23:26:04.382824+0000","last_peered":"2024-05-08T23:26:04.382824+0000","last_clean":"2024-05-08T23:26:04.382824+0000","last_became_active":"2024-05-08T23:26:04.380170+0000","last_became_peered":"2024-05-08T23:26:04.380170+0000","last_unstale":"2024-05-08T23:26:04.382824+0000","last_undegraded":"2024-05-08T23:26:04.382824+0000","last_fullsized":"2024-05-08T23:26:04.382824+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:30.884597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106822+0000","last_change":"2024-05-08T23:26:03.918662+0000","last_active":"2024-05-08T23:26:06.106822+0000","last_peered":"2024-05-08T23:26:06.106822+0000","last_clean":"2024-05-08T23:26:06.106822+0000","last_became_active":"2024-05-08T23:26:03.918367+0000","last_became_peered":"2024-05-08T23:26:03.918367+0000","last_unstale":"2024-05-08T23:26:06.106822+0000","last_undegraded":"2024-05-08T23:26:06.106822+0000","last_fullsized":"2024-05-08T23:26:06.106822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.392305+0000","last_change":"2024-05-08T23:26:04.391595+0000","last_active":"2024-05-08T23:26:04.392305+0000","last_peered":"2024-05-08T23:26:04.392305+0000","last_clean":"2024-05-08T23:26:04.392305+0000","last_became_active":"2024-05-08T23:26:04.391291+0000","last_became_peered":"2024-05-08T23:26:04.391291+0000","last_unstale":"2024-05-08T23:26:04.392305+0000","last_undegraded":"2024-05-08T23:26:04.392305+0000","last_fullsized":"2024-05-08T23:26:04.392305+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:17.695615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364906+0000","last_change":"2024-05-08T23:26:03.916776+0000","last_active":"2024-05-08T23:26:05.364906+0000","last_peered":"2024-05-08T23:26:05.364906+0000","last_clean":"2024-05-08T23:26:05.364906+0000","last_became_active":"2024-05-08T23:26:03.916402+0000","last_became_peered":"2024-05-08T23:26:03.916402+0000","last_unstale":"2024-05-08T23:26:05.364906+0000","last_undegraded":"2024-05-08T23:26:05.364906+0000","last_fullsized":"2024-05-08T23:26:05.364906+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:04.288000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.384180+0000","last_change":"2024-05-08T23:26:04.381800+0000","last_active":"2024-05-08T23:26:04.384180+0000","last_peered":"2024-05-08T23:26:04.384180+0000","last_clean":"2024-05-08T23:26:04.384180+0000","last_became_active":"2024-05-08T23:26:04.381476+0000","last_became_peered":"2024-05-08T23:26:04.381476+0000","last_unstale":"2024-05-08T23:26:04.384180+0000","last_undegraded":"2024-05-08T23:26:04.384180+0000","last_fullsized":"2024-05-08T23:26:04.384180+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.907260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106261+0000","last_change":"2024-05-08T23:26:03.915034+0000","last_active":"2024-05-08T23:26:06.106261+0000","last_peered":"2024-05-08T23:26:06.106261+0000","last_clean":"2024-05-08T23:26:06.106261+0000","last_became_active":"2024-05-08T23:26:03.914666+0000","last_became_peered":"2024-05-08T23:26:03.914666+0000","last_unstale":"2024-05-08T23:26:06.106261+0000","last_undegraded":"2024-05-08T23:26:06.106261+0000","last_fullsized":"2024-05-08T23:26:06.106261+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:43.175140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365034+0000","last_change":"2024-05-08T23:26:04.381034+0000","last_active":"2024-05-08T23:26:05.365034+0000","last_peered":"2024-05-08T23:26:05.365034+0000","last_clean":"2024-05-08T23:26:05.365034+0000","last_became_active":"2024-05-08T23:26:04.380769+0000","last_became_peered":"2024-05-08T23:26:04.380769+0000","last_unstale":"2024-05-08T23:26:05.365034+0000","last_undegraded":"2024-05-08T23:26:05.365034+0000","last_fullsized":"2024-05-08T23:26:05.365034+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:35.530781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365301+0000","last_change":"2024-05-08T23:26:03.917031+0000","last_active":"2024-05-08T23:26:05.365301+0000","last_peered":"2024-05-08T23:26:05.365301+0000","last_clean":"2024-05-08T23:26:05.365301+0000","last_became_active":"2024-05-08T23:26:03.916394+0000","last_became_peered":"2024-05-08T23:26:03.916394+0000","last_unstale":"2024-05-08T23:26:05.365301+0000","last_undegraded":"2024-05-08T23:26:05.365301+0000","last_fullsized":"2024-05-08T23:26:05.365301+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:38.532072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528242+0000","last_change":"2024-05-08T23:26:04.374371+0000","last_active":"2024-05-08T23:26:05.528242+0000","last_peered":"2024-05-08T23:26:05.528242+0000","last_clean":"2024-05-08T23:26:05.528242+0000","last_became_active":"2024-05-08T23:26:04.374097+0000","last_became_peered":"2024-05-08T23:26:04.374097+0000","last_unstale":"2024-05-08T23:26:05.528242+0000","last_undegraded":"2024-05-08T23:26:05.528242+0000","last_fullsized":"2024-05-08T23:26:05.528242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365139+0000","last_change":"2024-05-08T23:26:03.911594+0000","last_active":"2024-05-08T23:26:05.365139+0000","last_peered":"2024-05-08T23:26:05.365139+0000","last_clean":"2024-05-08T23:26:05.365139+0000","last_became_active":"2024-05-08T23:26:03.911273+0000","last_became_peered":"2024-05-08T23:26:03.911273+0000","last_unstale":"2024-05-08T23:26:05.365139+0000","last_undegraded":"2024-05-08T23:26:05.365139+0000","last_fullsized":"2024-05-08T23:26:05.365139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:30.134959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.373540+0000","last_change":"2024-05-08T23:26:04.372794+0000","last_active":"2024-05-08T23:26:04.373540+0000","last_peered":"2024-05-08T23:26:04.373540+0000","last_clean":"2024-05-08T23:26:04.373540+0000","last_became_active":"2024-05-08T23:26:04.372536+0000","last_became_peered":"2024-05-08T23:26:04.372536+0000","last_unstale":"2024-05-08T23:26:04.373540+0000","last_undegraded":"2024-05-08T23:26:04.373540+0000","last_fullsized":"2024-05-08T23:26:04.373540+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:06.816807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.372316+0000","last_change":"2024-05-08T23:26:03.917379+0000","last_active":"2024-05-08T23:26:04.372316+0000","last_peered":"2024-05-08T23:26:04.372316+0000","last_clean":"2024-05-08T23:26:04.372316+0000","last_became_active":"2024-05-08T23:26:03.917063+0000","last_became_peered":"2024-05-08T23:26:03.917063+0000","last_unstale":"2024-05-08T23:26:04.372316+0000","last_undegraded":"2024-05-08T23:26:04.372316+0000","last_fullsized":"2024-05-08T23:26:04.372316+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:43.443817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391925+0000","last_change":"2024-05-08T23:26:04.388857+0000","last_active":"2024-05-08T23:26:04.391925+0000","last_peered":"2024-05-08T23:26:04.391925+0000","last_clean":"2024-05-08T23:26:04.391925+0000","last_became_active":"2024-05-08T23:26:04.388581+0000","last_became_peered":"2024-05-08T23:26:04.388581+0000","last_unstale":"2024-05-08T23:26:04.391925+0000","last_undegraded":"2024-05-08T23:26:04.391925+0000","last_fullsized":"2024-05-08T23:26:04.391925+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106865+0000","last_change":"2024-05-08T23:26:03.907421+0000","last_active":"2024-05-08T23:26:06.106865+0000","last_peered":"2024-05-08T23:26:06.106865+0000","last_clean":"2024-05-08T23:26:06.106865+0000","last_became_active":"2024-05-08T23:26:03.907143+0000","last_became_peered":"2024-05-08T23:26:03.907143+0000","last_unstale":"2024-05-08T23:26:06.106865+0000","last_undegraded":"2024-05-08T23:26:06.106865+0000","last_fullsized":"2024-05-08T23:26:06.106865+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:07.208552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106627+0000","last_change":"2024-05-08T23:26:04.386708+0000","last_active":"2024-05-08T23:26:06.106627+0000","last_peered":"2024-05-08T23:26:06.106627+0000","last_clean":"2024-05-08T23:26:06.106627+0000","last_became_active":"2024-05-08T23:26:04.385984+0000","last_became_peered":"2024-05-08T23:26:04.385984+0000","last_unstale":"2024-05-08T23:26:06.106627+0000","last_undegraded":"2024-05-08T23:26:06.106627+0000","last_fullsized":"2024-05-08T23:26:06.106627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:47.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365174+0000","last_change":"2024-05-08T23:26:03.913646+0000","last_active":"2024-05-08T23:26:05.365174+0000","last_peered":"2024-05-08T23:26:05.365174+0000","last_clean":"2024-05-08T23:26:05.365174+0000","last_became_active":"2024-05-08T23:26:03.913208+0000","last_became_peered":"2024-05-08T23:26:03.913208+0000","last_unstale":"2024-05-08T23:26:05.365174+0000","last_undegraded":"2024-05-08T23:26:05.365174+0000","last_fullsized":"2024-05-08T23:26:05.365174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:02.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365662+0000","last_change":"2024-05-08T23:26:04.375029+0000","last_active":"2024-05-08T23:26:05.365662+0000","last_peered":"2024-05-08T23:26:05.365662+0000","last_clean":"2024-05-08T23:26:05.365662+0000","last_became_active":"2024-05-08T23:26:04.374797+0000","last_became_peered":"2024-05-08T23:26:04.374797+0000","last_unstale":"2024-05-08T23:26:05.365662+0000","last_undegraded":"2024-05-08T23:26:05.365662+0000","last_fullsized":"2024-05-08T23:26:05.365662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:18.355499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365807+0000","last_change":"2024-05-08T23:26:03.913239+0000","last_active":"2024-05-08T23:26:05.365807+0000","last_peered":"2024-05-08T23:26:05.365807+0000","last_clean":"2024-05-08T23:26:05.365807+0000","last_became_active":"2024-05-08T23:26:03.912892+0000","last_became_peered":"2024-05-08T23:26:03.912892+0000","last_unstale":"2024-05-08T23:26:05.365807+0000","last_undegraded":"2024-05-08T23:26:05.365807+0000","last_fullsized":"2024-05-08T23:26:05.365807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:28.036636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.385254+0000","last_change":"2024-05-08T23:26:04.381564+0000","last_active":"2024-05-08T23:26:04.385254+0000","last_peered":"2024-05-08T23:26:04.385254+0000","last_clean":"2024-05-08T23:26:04.385254+0000","last_became_active":"2024-05-08T23:26:04.381293+0000","last_became_peered":"2024-05-08T23:26:04.381293+0000","last_unstale":"2024-05-08T23:26:04.385254+0000","last_undegraded":"2024-05-08T23:26:04.385254+0000","last_fullsized":"2024-05-08T23:26:04.385254+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365371+0000","last_change":"2024-05-08T23:26:03.917272+0000","last_active":"2024-05-08T23:26:05.365371+0000","last_peered":"2024-05-08T23:26:05.365371+0000","last_clean":"2024-05-08T23:26:05.365371+0000","last_became_active":"2024-05-08T23:26:03.916960+0000","last_became_peered":"2024-05-08T23:26:03.916960+0000","last_unstale":"2024-05-08T23:26:05.365371+0000","last_undegraded":"2024-05-08T23:26:05.365371+0000","last_fullsized":"2024-05-08T23:26:05.365371+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.380000+0000","last_change":"2024-05-08T23:26:04.375997+0000","last_active":"2024-05-08T23:26:04.380000+0000","last_peered":"2024-05-08T23:26:04.380000+0000","last_clean":"2024-05-08T23:26:04.380000+0000","last_became_active":"2024-05-08T23:26:04.375654+0000","last_became_peered":"2024-05-08T23:26:04.375654+0000","last_unstale":"2024-05-08T23:26:04.380000+0000","last_undegraded":"2024-05-08T23:26:04.380000+0000","last_fullsized":"2024-05-08T23:26:04.380000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:03.230741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106390+0000","last_change":"2024-05-08T23:26:03.911788+0000","last_active":"2024-05-08T23:26:06.106390+0000","last_peered":"2024-05-08T23:26:06.106390+0000","last_clean":"2024-05-08T23:26:06.106390+0000","last_became_active":"2024-05-08T23:26:03.911527+0000","last_became_peered":"2024-05-08T23:26:03.911527+0000","last_unstale":"2024-05-08T23:26:06.106390+0000","last_undegraded":"2024-05-08T23:26:06.106390+0000","last_fullsized":"2024-05-08T23:26:06.106390+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:44.784641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528178+0000","last_change":"2024-05-08T23:26:04.394848+0000","last_active":"2024-05-08T23:26:05.528178+0000","last_peered":"2024-05-08T23:26:05.528178+0000","last_clean":"2024-05-08T23:26:05.528178+0000","last_became_active":"2024-05-08T23:26:04.394520+0000","last_became_peered":"2024-05-08T23:26:04.394520+0000","last_unstale":"2024-05-08T23:26:05.528178+0000","last_undegraded":"2024-05-08T23:26:05.528178+0000","last_fullsized":"2024-05-08T23:26:05.528178+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367381+0000","last_change":"2024-05-08T23:26:03.904993+0000","last_active":"2024-05-08T23:26:04.367381+0000","last_peered":"2024-05-08T23:26:04.367381+0000","last_clean":"2024-05-08T23:26:04.367381+0000","last_became_active":"2024-05-08T23:26:03.904689+0000","last_became_peered":"2024-05-08T23:26:03.904689+0000","last_unstale":"2024-05-08T23:26:04.367381+0000","last_undegraded":"2024-05-08T23:26:04.367381+0000","last_fullsized":"2024-05-08T23:26:04.367381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:54.695959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365441+0000","last_change":"2024-05-08T23:26:04.386132+0000","last_active":"2024-05-08T23:26:05.365441+0000","last_peered":"2024-05-08T23:26:05.365441+0000","last_clean":"2024-05-08T23:26:05.365441+0000","last_became_active":"2024-05-08T23:26:04.385842+0000","last_became_peered":"2024-05-08T23:26:04.385842+0000","last_unstale":"2024-05-08T23:26:05.365441+0000","last_undegraded":"2024-05-08T23:26:05.365441+0000","last_fullsized":"2024-05-08T23:26:05.365441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:42.318612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.388729+0000","last_change":"2024-05-08T23:26:03.912054+0000","last_active":"2024-05-08T23:26:04.388729+0000","last_peered":"2024-05-08T23:26:04.388729+0000","last_clean":"2024-05-08T23:26:04.388729+0000","last_became_active":"2024-05-08T23:26:03.911775+0000","last_became_peered":"2024-05-08T23:26:03.911775+0000","last_unstale":"2024-05-08T23:26:04.388729+0000","last_undegraded":"2024-05-08T23:26:04.388729+0000","last_fullsized":"2024-05-08T23:26:04.388729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:25.875643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382844+0000","last_change":"2024-05-08T23:26:04.380260+0000","last_active":"2024-05-08T23:26:04.382844+0000","last_peered":"2024-05-08T23:26:04.382844+0000","last_clean":"2024-05-08T23:26:04.382844+0000","last_became_active":"2024-05-08T23:26:04.379931+0000","last_became_peered":"2024-05-08T23:26:04.379931+0000","last_unstale":"2024-05-08T23:26:04.382844+0000","last_undegraded":"2024-05-08T23:26:04.382844+0000","last_fullsized":"2024-05-08T23:26:04.382844+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:03.230741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106703+0000","last_change":"2024-05-08T23:26:03.917170+0000","last_active":"2024-05-08T23:26:06.106703+0000","last_peered":"2024-05-08T23:26:06.106703+0000","last_clean":"2024-05-08T23:26:06.106703+0000","last_became_active":"2024-05-08T23:26:03.916888+0000","last_became_peered":"2024-05-08T23:26:03.916888+0000","last_unstale":"2024-05-08T23:26:06.106703+0000","last_undegraded":"2024-05-08T23:26:06.106703+0000","last_fullsized":"2024-05-08T23:26:06.106703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:59.835332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.388091+0000","last_change":"2024-05-08T23:26:04.385163+0000","last_active":"2024-05-08T23:26:04.388091+0000","last_peered":"2024-05-08T23:26:04.388091+0000","last_clean":"2024-05-08T23:26:04.388091+0000","last_became_active":"2024-05-08T23:26:04.384430+0000","last_became_peered":"2024-05-08T23:26:04.384430+0000","last_unstale":"2024-05-08T23:26:04.388091+0000","last_undegraded":"2024-05-08T23:26:04.388091+0000","last_fullsized":"2024-05-08T23:26:04.388091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:13.482666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.394524+0000","last_change":"2024-05-08T23:26:03.909206+0000","last_active":"2024-05-08T23:26:04.394524+0000","last_peered":"2024-05-08T23:26:04.394524+0000","last_clean":"2024-05-08T23:26:04.394524+0000","last_became_active":"2024-05-08T23:26:03.908682+0000","last_became_peered":"2024-05-08T23:26:03.908682+0000","last_unstale":"2024-05-08T23:26:04.394524+0000","last_undegraded":"2024-05-08T23:26:04.394524+0000","last_fullsized":"2024-05-08T23:26:04.394524+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:48.022908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365760+0000","last_change":"2024-05-08T23:26:04.382870+0000","last_active":"2024-05-08T23:26:05.365760+0000","last_peered":"2024-05-08T23:26:05.365760+0000","last_clean":"2024-05-08T23:26:05.365760+0000","last_became_active":"2024-05-08T23:26:04.382320+0000","last_became_peered":"2024-05-08T23:26:04.382320+0000","last_unstale":"2024-05-08T23:26:05.365760+0000","last_undegraded":"2024-05-08T23:26:05.365760+0000","last_fullsized":"2024-05-08T23:26:05.365760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:17.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106864+0000","last_change":"2024-05-08T23:26:03.910712+0000","last_active":"2024-05-08T23:26:06.106864+0000","last_peered":"2024-05-08T23:26:06.106864+0000","last_clean":"2024-05-08T23:26:06.106864+0000","last_became_active":"2024-05-08T23:26:03.910424+0000","last_became_peered":"2024-05-08T23:26:03.910424+0000","last_unstale":"2024-05-08T23:26:06.106864+0000","last_undegraded":"2024-05-08T23:26:06.106864+0000","last_fullsized":"2024-05-08T23:26:06.106864+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:16.285704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365799+0000","last_change":"2024-05-08T23:26:04.378589+0000","last_active":"2024-05-08T23:26:05.365799+0000","last_peered":"2024-05-08T23:26:05.365799+0000","last_clean":"2024-05-08T23:26:05.365799+0000","last_became_active":"2024-05-08T23:26:04.378300+0000","last_became_peered":"2024-05-08T23:26:04.378300+0000","last_unstale":"2024-05-08T23:26:05.365799+0000","last_undegraded":"2024-05-08T23:26:05.365799+0000","last_fullsized":"2024-05-08T23:26:05.365799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.786772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.368459+0000","last_change":"2024-05-08T23:26:03.916915+0000","last_active":"2024-05-08T23:26:04.368459+0000","last_peered":"2024-05-08T23:26:04.368459+0000","last_clean":"2024-05-08T23:26:04.368459+0000","last_became_active":"2024-05-08T23:26:03.916632+0000","last_became_peered":"2024-05-08T23:26:03.916632+0000","last_unstale":"2024-05-08T23:26:04.368459+0000","last_undegraded":"2024-05-08T23:26:04.368459+0000","last_fullsized":"2024-05-08T23:26:04.368459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:22.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364817+0000","last_change":"2024-05-08T23:26:04.375139+0000","last_active":"2024-05-08T23:26:05.364817+0000","last_peered":"2024-05-08T23:26:05.364817+0000","last_clean":"2024-05-08T23:26:05.364817+0000","last_became_active":"2024-05-08T23:26:04.374874+0000","last_became_peered":"2024-05-08T23:26:04.374874+0000","last_unstale":"2024-05-08T23:26:05.364817+0000","last_undegraded":"2024-05-08T23:26:05.364817+0000","last_fullsized":"2024-05-08T23:26:05.364817+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:42.364785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106533+0000","last_change":"2024-05-08T23:26:03.915036+0000","last_active":"2024-05-08T23:26:06.106533+0000","last_peered":"2024-05-08T23:26:06.106533+0000","last_clean":"2024-05-08T23:26:06.106533+0000","last_became_active":"2024-05-08T23:26:03.914616+0000","last_became_peered":"2024-05-08T23:26:03.914616+0000","last_unstale":"2024-05-08T23:26:06.106533+0000","last_undegraded":"2024-05-08T23:26:06.106533+0000","last_fullsized":"2024-05-08T23:26:06.106533+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.591942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364460+0000","last_change":"2024-05-08T23:26:04.384503+0000","last_active":"2024-05-08T23:26:05.364460+0000","last_peered":"2024-05-08T23:26:05.364460+0000","last_clean":"2024-05-08T23:26:05.364460+0000","last_became_active":"2024-05-08T23:26:04.383639+0000","last_became_peered":"2024-05-08T23:26:04.383639+0000","last_unstale":"2024-05-08T23:26:05.364460+0000","last_undegraded":"2024-05-08T23:26:05.364460+0000","last_fullsized":"2024-05-08T23:26:05.364460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:13.674487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367498+0000","last_change":"2024-05-08T23:26:03.912730+0000","last_active":"2024-05-08T23:26:04.367498+0000","last_peered":"2024-05-08T23:26:04.367498+0000","last_clean":"2024-05-08T23:26:04.367498+0000","last_became_active":"2024-05-08T23:26:03.912415+0000","last_became_peered":"2024-05-08T23:26:03.912415+0000","last_unstale":"2024-05-08T23:26:04.367498+0000","last_undegraded":"2024-05-08T23:26:04.367498+0000","last_fullsized":"2024-05-08T23:26:04.367498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:35.080457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364494+0000","last_change":"2024-05-08T23:26:04.383931+0000","last_active":"2024-05-08T23:26:05.364494+0000","last_peered":"2024-05-08T23:26:05.364494+0000","last_clean":"2024-05-08T23:26:05.364494+0000","last_became_active":"2024-05-08T23:26:04.383598+0000","last_became_peered":"2024-05-08T23:26:04.383598+0000","last_unstale":"2024-05-08T23:26:05.364494+0000","last_undegraded":"2024-05-08T23:26:05.364494+0000","last_fullsized":"2024-05-08T23:26:05.364494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:27.479478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528163+0000","last_change":"2024-05-08T23:26:03.908029+0000","last_active":"2024-05-08T23:26:05.528163+0000","last_peered":"2024-05-08T23:26:05.528163+0000","last_clean":"2024-05-08T23:26:05.528163+0000","last_became_active":"2024-05-08T23:26:03.907475+0000","last_became_peered":"2024-05-08T23:26:03.907475+0000","last_unstale":"2024-05-08T23:26:05.528163+0000","last_undegraded":"2024-05-08T23:26:05.528163+0000","last_fullsized":"2024-05-08T23:26:05.528163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:26.905129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528734+0000","last_change":"2024-05-08T23:26:04.383939+0000","last_active":"2024-05-08T23:26:05.528734+0000","last_peered":"2024-05-08T23:26:05.528734+0000","last_clean":"2024-05-08T23:26:05.528734+0000","last_became_active":"2024-05-08T23:26:04.383318+0000","last_became_peered":"2024-05-08T23:26:04.383318+0000","last_unstale":"2024-05-08T23:26:05.528734+0000","last_undegraded":"2024-05-08T23:26:05.528734+0000","last_fullsized":"2024-05-08T23:26:05.528734+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106789+0000","last_change":"2024-05-08T23:26:03.917152+0000","last_active":"2024-05-08T23:26:06.106789+0000","last_peered":"2024-05-08T23:26:06.106789+0000","last_clean":"2024-05-08T23:26:06.106789+0000","last_became_active":"2024-05-08T23:26:03.916848+0000","last_became_peered":"2024-05-08T23:26:03.916848+0000","last_unstale":"2024-05-08T23:26:06.106789+0000","last_undegraded":"2024-05-08T23:26:06.106789+0000","last_fullsized":"2024-05-08T23:26:06.106789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:17.395880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.386965+0000","last_change":"2024-05-08T23:26:04.377822+0000","last_active":"2024-05-08T23:26:04.386965+0000","last_peered":"2024-05-08T23:26:04.386965+0000","last_clean":"2024-05-08T23:26:04.386965+0000","last_became_active":"2024-05-08T23:26:04.377510+0000","last_became_peered":"2024-05-08T23:26:04.377510+0000","last_unstale":"2024-05-08T23:26:04.386965+0000","last_undegraded":"2024-05-08T23:26:04.386965+0000","last_fullsized":"2024-05-08T23:26:04.386965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:09.803388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.367888+0000","last_change":"2024-05-08T23:26:03.904714+0000","last_active":"2024-05-08T23:26:04.367888+0000","last_peered":"2024-05-08T23:26:04.367888+0000","last_clean":"2024-05-08T23:26:04.367888+0000","last_became_active":"2024-05-08T23:26:03.904414+0000","last_became_peered":"2024-05-08T23:26:03.904414+0000","last_unstale":"2024-05-08T23:26:04.367888+0000","last_undegraded":"2024-05-08T23:26:04.367888+0000","last_fullsized":"2024-05-08T23:26:04.367888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:14.889713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.391857+0000","last_change":"2024-05-08T23:26:04.388899+0000","last_active":"2024-05-08T23:26:04.391857+0000","last_peered":"2024-05-08T23:26:04.391857+0000","last_clean":"2024-05-08T23:26:04.391857+0000","last_became_active":"2024-05-08T23:26:04.388628+0000","last_became_peered":"2024-05-08T23:26:04.388628+0000","last_unstale":"2024-05-08T23:26:04.391857+0000","last_undegraded":"2024-05-08T23:26:04.391857+0000","last_fullsized":"2024-05-08T23:26:04.391857+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365794+0000","last_change":"2024-05-08T23:26:03.910274+0000","last_active":"2024-05-08T23:26:05.365794+0000","last_peered":"2024-05-08T23:26:05.365794+0000","last_clean":"2024-05-08T23:26:05.365794+0000","last_became_active":"2024-05-08T23:26:03.909977+0000","last_became_peered":"2024-05-08T23:26:03.909977+0000","last_unstale":"2024-05-08T23:26:05.365794+0000","last_undegraded":"2024-05-08T23:26:05.365794+0000","last_fullsized":"2024-05-08T23:26:05.365794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:09.127009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.382218+0000","last_change":"2024-05-08T23:26:04.377703+0000","last_active":"2024-05-08T23:26:04.382218+0000","last_peered":"2024-05-08T23:26:04.382218+0000","last_clean":"2024-05-08T23:26:04.382218+0000","last_became_active":"2024-05-08T23:26:04.377184+0000","last_became_peered":"2024-05-08T23:26:04.377184+0000","last_unstale":"2024-05-08T23:26:04.382218+0000","last_undegraded":"2024-05-08T23:26:04.382218+0000","last_fullsized":"2024-05-08T23:26:04.382218+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528645+0000","last_change":"2024-05-08T23:26:03.910256+0000","last_active":"2024-05-08T23:26:05.528645+0000","last_peered":"2024-05-08T23:26:05.528645+0000","last_clean":"2024-05-08T23:26:05.528645+0000","last_became_active":"2024-05-08T23:26:03.909930+0000","last_became_peered":"2024-05-08T23:26:03.909930+0000","last_unstale":"2024-05-08T23:26:05.528645+0000","last_undegraded":"2024-05-08T23:26:05.528645+0000","last_fullsized":"2024-05-08T23:26:05.528645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:46.987230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366137+0000","last_change":"2024-05-08T23:26:04.390986+0000","last_active":"2024-05-08T23:26:05.366137+0000","last_peered":"2024-05-08T23:26:05.366137+0000","last_clean":"2024-05-08T23:26:05.366137+0000","last_became_active":"2024-05-08T23:26:04.390656+0000","last_became_peered":"2024-05-08T23:26:04.390656+0000","last_unstale":"2024-05-08T23:26:05.366137+0000","last_undegraded":"2024-05-08T23:26:05.366137+0000","last_fullsized":"2024-05-08T23:26:05.366137+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:26.611874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365238+0000","last_change":"2024-05-08T23:26:03.902695+0000","last_active":"2024-05-08T23:26:05.365238+0000","last_peered":"2024-05-08T23:26:05.365238+0000","last_clean":"2024-05-08T23:26:05.365238+0000","last_became_active":"2024-05-08T23:26:03.902382+0000","last_became_peered":"2024-05-08T23:26:03.902382+0000","last_unstale":"2024-05-08T23:26:05.365238+0000","last_undegraded":"2024-05-08T23:26:05.365238+0000","last_fullsized":"2024-05-08T23:26:05.365238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:08.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106322+0000","last_change":"2024-05-08T23:26:04.387935+0000","last_active":"2024-05-08T23:26:06.106322+0000","last_peered":"2024-05-08T23:26:06.106322+0000","last_clean":"2024-05-08T23:26:06.106322+0000","last_became_active":"2024-05-08T23:26:04.387678+0000","last_became_peered":"2024-05-08T23:26:04.387678+0000","last_unstale":"2024-05-08T23:26:06.106322+0000","last_undegraded":"2024-05-08T23:26:06.106322+0000","last_fullsized":"2024-05-08T23:26:06.106322+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:23.335791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364796+0000","last_change":"2024-05-08T23:26:03.916887+0000","last_active":"2024-05-08T23:26:05.364796+0000","last_peered":"2024-05-08T23:26:05.364796+0000","last_clean":"2024-05-08T23:26:05.364796+0000","last_became_active":"2024-05-08T23:26:03.916407+0000","last_became_peered":"2024-05-08T23:26:03.916407+0000","last_unstale":"2024-05-08T23:26:05.364796+0000","last_undegraded":"2024-05-08T23:26:05.364796+0000","last_fullsized":"2024-05-08T23:26:05.364796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:34.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365058+0000","last_change":"2024-05-08T23:26:04.389795+0000","last_active":"2024-05-08T23:26:05.365058+0000","last_peered":"2024-05-08T23:26:05.365058+0000","last_clean":"2024-05-08T23:26:05.365058+0000","last_became_active":"2024-05-08T23:26:04.389433+0000","last_became_peered":"2024-05-08T23:26:04.389433+0000","last_unstale":"2024-05-08T23:26:05.365058+0000","last_undegraded":"2024-05-08T23:26:05.365058+0000","last_fullsized":"2024-05-08T23:26:05.365058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:23.681716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.372078+0000","last_change":"2024-05-08T23:26:03.914133+0000","last_active":"2024-05-08T23:26:04.372078+0000","last_peered":"2024-05-08T23:26:04.372078+0000","last_clean":"2024-05-08T23:26:04.372078+0000","last_became_active":"2024-05-08T23:26:03.913863+0000","last_became_peered":"2024-05-08T23:26:03.913863+0000","last_unstale":"2024-05-08T23:26:04.372078+0000","last_undegraded":"2024-05-08T23:26:04.372078+0000","last_fullsized":"2024-05-08T23:26:04.372078+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:27.115050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106146+0000","last_change":"2024-05-08T23:26:04.390648+0000","last_active":"2024-05-08T23:26:06.106146+0000","last_peered":"2024-05-08T23:26:06.106146+0000","last_clean":"2024-05-08T23:26:06.106146+0000","last_became_active":"2024-05-08T23:26:04.390358+0000","last_became_peered":"2024-05-08T23:26:04.390358+0000","last_unstale":"2024-05-08T23:26:06.106146+0000","last_undegraded":"2024-05-08T23:26:06.106146+0000","last_fullsized":"2024-05-08T23:26:06.106146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:19.130856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106499+0000","last_change":"2024-05-08T23:26:03.908056+0000","last_active":"2024-05-08T23:26:06.106499+0000","last_peered":"2024-05-08T23:26:06.106499+0000","last_clean":"2024-05-08T23:26:06.106499+0000","last_became_active":"2024-05-08T23:26:03.907603+0000","last_became_peered":"2024-05-08T23:26:03.907603+0000","last_unstale":"2024-05-08T23:26:06.106499+0000","last_undegraded":"2024-05-08T23:26:06.106499+0000","last_fullsized":"2024-05-08T23:26:06.106499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:17.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364615+0000","last_change":"2024-05-08T23:26:04.384097+0000","last_active":"2024-05-08T23:26:05.364615+0000","last_peered":"2024-05-08T23:26:05.364615+0000","last_clean":"2024-05-08T23:26:05.364615+0000","last_became_active":"2024-05-08T23:26:04.383812+0000","last_became_peered":"2024-05-08T23:26:04.383812+0000","last_unstale":"2024-05-08T23:26:05.364615+0000","last_undegraded":"2024-05-08T23:26:05.364615+0000","last_fullsized":"2024-05-08T23:26:05.364615+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:08.068641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106387+0000","last_change":"2024-05-08T23:26:03.918580+0000","last_active":"2024-05-08T23:26:06.106387+0000","last_peered":"2024-05-08T23:26:06.106387+0000","last_clean":"2024-05-08T23:26:06.106387+0000","last_became_active":"2024-05-08T23:26:03.918308+0000","last_became_peered":"2024-05-08T23:26:03.918308+0000","last_unstale":"2024-05-08T23:26:06.106387+0000","last_undegraded":"2024-05-08T23:26:06.106387+0000","last_fullsized":"2024-05-08T23:26:06.106387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:22.679586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-05-08T23:26:03.912381+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:26:03.912381+0000","last_peered":"2024-05-08T23:26:03.912381+0000","last_clean":"2024-05-08T23:26:03.912381+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:26:03.912381+0000","last_undegraded":"2024-05-08T23:26:03.912381+0000","last_fullsized":"2024-05-08T23:26:03.912381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1466368,"data_stored":958984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85999999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.495},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67400000000000004,"5min":1.038,"15min":1.038},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76500000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76600000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69899999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73899999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82499999999999996,"5min":0.87,"15min":0.87},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed May 8 23:25:06 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.95799999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996}]},{"osd":2,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":1.115,"15min":1.115},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.436}]},{"osd":5,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76500000000000001,"5min":1.052,"15min":1.052},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84399999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":376480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.371,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56100000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.42299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":1.01,"15min":1.01},"last":0.432},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":1.028,"15min":1.028},"last":0.34200000000000003}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":1.105,"15min":1.105},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.54000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1470464,"data_stored":964668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48399999999999999,"15min":0.72599999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.52900000000000003,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":0.71399999999999997},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60799999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.51600000000000001,"15min":0.77800000000000002},"min":{"1min":0.312,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52900000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54800000000000004,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58099999999999996,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65400000000000003,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69099999999999995,"15min":0.93500000000000005},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.70399999999999996,"15min":0.90800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.91300000000000003},"min":{"1min":0.50700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75700000000000001,"15min":1.0069999999999999},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.73999999999999999}]},{"osd":6,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73499999999999999,"15min":0.97499999999999998},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81799999999999995,"5min":7.734,"15min":7.734},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.77800000000000002}]},{"osd":7,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.96399999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.92200000000000004},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.83299999999999996,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":375864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61799999999999999,"5min":0.621,"15min":0.621},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57399999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.441}]},{"osd":1,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.439,"15min":0.40999999999999998},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.33000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.64100000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42699999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79900000000000004,"5min":1.024,"15min":1.024},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81799999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.50700000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.159,"15min":1.159},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91900000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.622}]},{"osd":7,"last update":"Wed May 8 23:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.70999999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89800000000000002,"5min":1.194,"15min":1.194},"last":0.60199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39500000000000002,"15min":0.41699999999999998},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.58199999999999996,"5min":1.46,"15min":1.46},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.41099999999999998,"15min":0.42099999999999999},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52100000000000002,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40100000000000002,"15min":0.40999999999999998},"min":{"1min":0.28999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45800000000000002,"15min":0.46999999999999997},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61199999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.66200000000000003,"15min":0.879},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.56399999999999995,"5min":11.964,"15min":11.964},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.503,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67300000000000004,"5min":1.593,"15min":1.593},"last":0.439}]},{"osd":4,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93200000000000005,"5min":1.375,"15min":1.375},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.624}]},{"osd":5,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.536,"15min":1.536},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.68600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.874,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":1.204,"15min":1.204},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.50700000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.878,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Wed May 8 23:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":1.339,"15min":1.339},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73199999999999998,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92800000000000005,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.72399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":868352,"data_stored":368070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.64200000000000002},"min":{"1min":0.33700000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73399999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59499999999999997,"15min":0.72099999999999997},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.746,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.748,"5min":2.145,"15min":2.145},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60199999999999998,"15min":0.68400000000000005},"min":{"1min":0.36899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.623}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68500000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64700000000000002,"15min":0.78500000000000003},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81699999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.74299999999999999,"15min":1.1240000000000001},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.877,"15min":0.877},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64100000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63400000000000001,"15min":0.68200000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73899999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77400000000000002,"5min":1.24,"15min":1.24},"last":0.68799999999999994}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":368090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85199999999999998,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.433},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":0.89400000000000002},"min":{"1min":0.35199999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":8.798,"15min":8.798},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.77500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":4.71,"15min":4.71},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70699999999999996,"15min":0.96799999999999997},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0009999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.79100000000000004},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.82199999999999995,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.93000000000000005},"min":{"1min":0.46999999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78500000000000003,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73899999999999999,"15min":1.0109999999999999},"min":{"1min":0.51100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0489999999999999,"5min":10.427,"15min":10.427},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58599999999999997,"5min":0.874,"15min":0.874},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.502}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85299999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.01},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:26:13.383 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:26:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.527+0000 7f760209e700 1 Processor -- start 2024-05-08T23:26:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.527+0000 7f760209e700 1 -- start start 2024-05-08T23:26:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f760209e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75fc0b5ba0 0x7f75fc0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f760209e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc06b070 0x7f75fc15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f760209e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75fc0b3170 con 0x7f75fc0cfe10 2024-05-08T23:26:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f760209e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75fc0cc5e0 con 0x7f75fc0b5ba0 2024-05-08T23:26:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f760209e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0cc000 con 0x7f75fc06b070 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75fb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75fc0b5ba0 0x7f75fc0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75faffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f75fc0cfe10 0x7f75fc06a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38918/0 (socket says 172.21.15.5:38918) 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75faffd700 1 -- 172.21.15.5:0/4169274735 learned_addr learned my addr 172.21.15.5:0/4169274735 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75fbfff700 1 --2- 172.21.15.5:0/4169274735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc06b070 0x7f75fc15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75fb7fe700 1 -- 172.21.15.5:0/4169274735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc06b070 msgr2=0x7f75fc15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75fb7fe700 1 --2- 172.21.15.5:0/4169274735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc06b070 0x7f75fc15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75fb7fe700 1 -- 172.21.15.5:0/4169274735 >> v1:172.21.15.5:6789/0 conn(0x7f75fc0cfe10 legacy=0x7f75fc06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75fb7fe700 1 -- 172.21.15.5:0/4169274735 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc0cb760 con 0x7f75fc0b5ba0 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75fa7fc700 1 -- 172.21.15.5:0/4169274735 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955539756 0 0) 0x7f75fc0b3170 con 0x7f75fc0cfe10 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.528+0000 7f75fb7fe700 1 --2- 172.21.15.5:0/4169274735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc0b5ba0 0x7f75fc0cf8d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009840 tx=0x7f75ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ac9131a030acc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f75fa7fc700 1 -- 172.21.15.5:0/4169274735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec011830 con 0x7f75fc0b5ba0 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f75fa7fc700 1 -- 172.21.15.5:0/4169274735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec011990 con 0x7f75fc0b5ba0 2024-05-08T23:26:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f75fa7fc700 1 -- 172.21.15.5:0/4169274735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec011d20 con 0x7f75fc0b5ba0 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f760209e700 1 -- 172.21.15.5:0/4169274735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc0b5ba0 msgr2=0x7f75fc0cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f760209e700 1 --2- 172.21.15.5:0/4169274735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc0b5ba0 0x7f75fc0cf8d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009840 tx=0x7f75ec009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f760209e700 1 -- 172.21.15.5:0/4169274735 shutdown_connections 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f760209e700 1 --2- 172.21.15.5:0/4169274735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc06b070 0x7f75fc15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f760209e700 1 --2- 172.21.15.5:0/4169274735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc0b5ba0 0x7f75fc0cf8d0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f760209e700 1 -- 172.21.15.5:0/4169274735 >> 172.21.15.5:0/4169274735 conn(0x7f75fc099a50 msgr2=0x7f75fc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f760209e700 1 -- 172.21.15.5:0/4169274735 shutdown_connections 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.529+0000 7f760209e700 1 -- 172.21.15.5:0/4169274735 wait complete. 2024-05-08T23:26:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.531+0000 7f760209e700 1 Processor -- start 2024-05-08T23:26:13.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.531+0000 7f760209e700 1 -- start start 2024-05-08T23:26:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f760209e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc06b070 0x7f75fc15d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f760209e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75fc0b5ba0 0x7f75fc0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f760209e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc0cfe10 0x7f75fc0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f760209e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0c7f60 con 0x7f75fc06b070 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f760209e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0c8240 con 0x7f75fc0b5ba0 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f760209e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc09cc30 con 0x7f75fc0cfe10 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f75fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc06b070 0x7f75fc15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f75fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc06b070 0x7f75fc15d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44862/0 (socket says 172.21.15.5:44862) 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.532+0000 7f75fb7fe700 1 -- 172.21.15.5:0/2464512872 learned_addr learned my addr 172.21.15.5:0/2464512872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75fbfff700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc0cfe10 0x7f75fc0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75faffd700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75fc0b5ba0 0x7f75fc0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75fb7fe700 1 -- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc0cfe10 msgr2=0x7f75fc0c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75fb7fe700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc0cfe10 0x7f75fc0c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75fb7fe700 1 -- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75fc0b5ba0 msgr2=0x7f75fc0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75fb7fe700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75fc0b5ba0 0x7f75fc0c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75fb7fe700 1 -- 172.21.15.5:0/2464512872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec0094a0 con 0x7f75fc06b070 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75fbfff700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc0cfe10 0x7f75fc0c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75fb7fe700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc06b070 0x7f75fc15d870 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009fd0 tx=0x7f75ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75e3fff700 1 -- 172.21.15.5:0/2464512872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0038a0 con 0x7f75fc06b070 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75e3fff700 1 -- 172.21.15.5:0/2464512872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec01f070 con 0x7f75fc06b070 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f75e3fff700 1 -- 172.21.15.5:0/2464512872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec003c40 con 0x7f75fc06b070 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75fc09cf10 con 0x7f75fc06b070 2024-05-08T23:26:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.533+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc09c930 con 0x7f75fc06b070 2024-05-08T23:26:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.534+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75fc068000 con 0x7f75fc06b070 2024-05-08T23:26:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.535+0000 7f75e3fff700 1 -- 172.21.15.5:0/2464512872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f75ec026480 con 0x7f75fc06b070 2024-05-08T23:26:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.535+0000 7f75e3fff700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f75e806ac90 0x7f75e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.535+0000 7f75e3fff700 1 -- 172.21.15.5:0/2464512872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75ec021070 con 0x7f75fc06b070 2024-05-08T23:26:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.535+0000 7f75faffd700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f75e806ac90 0x7f75e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.536+0000 7f75faffd700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f75e806ac90 0x7f75e806d150 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f75f0000f60 tx=0x7f75f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.540+0000 7f75e3fff700 1 -- 172.21.15.5:0/2464512872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f75ec055f30 con 0x7f75fc06b070 2024-05-08T23:26:13.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.752+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75fc09cac0 con 0x7f75e806ac90 2024-05-08T23:26:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.762+0000 7f75e3fff700 1 -- 172.21.15.5:0/2464512872 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289628 (secure 0 0 0) 0x7f75fc09cac0 con 0x7f75e806ac90 2024-05-08T23:26:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f75e806ac90 msgr2=0x7f75e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f75e806ac90 0x7f75e806d150 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f75f0000f60 tx=0x7f75f0009040 comp rx=0 tx=0).stop 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc06b070 msgr2=0x7f75fc15d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc06b070 0x7f75fc15d870 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009fd0 tx=0x7f75ec004750 comp rx=0 tx=0).stop 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 shutdown_connections 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f75e806ac90 0x7f75e806d150 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f75fc0cfe10 0x7f75fc0c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc06b070 0x7f75fc15d870 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 --2- 172.21.15.5:0/2464512872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75fc0b5ba0 0x7f75fc0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 >> 172.21.15.5:0/2464512872 conn(0x7f75fc099a50 msgr2=0x7f75fc0d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 shutdown_connections 2024-05-08T23:26:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.765+0000 7f760209e700 1 -- 172.21.15.5:0/2464512872 wait complete. 2024-05-08T23:26:13.769 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:26:13.779 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-05-08T23:26:13.368504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236520,"kb_used_data":8072,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":754738200,"statfs":{"total":773094113280,"available":772851916800,"internally_reserved":0,"allocated":8265728,"data_stored":4179860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":233819701},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.006177"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347398+0000","last_change":"2024-05-08T23:26:04.382812+0000","last_active":"2024-05-08T23:26:07.347398+0000","last_peered":"2024-05-08T23:26:07.347398+0000","last_clean":"2024-05-08T23:26:07.347398+0000","last_became_active":"2024-05-08T23:26:04.382518+0000","last_became_peered":"2024-05-08T23:26:04.382518+0000","last_unstale":"2024-05-08T23:26:07.347398+0000","last_undegraded":"2024-05-08T23:26:07.347398+0000","last_fullsized":"2024-05-08T23:26:07.347398+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:16.459465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106783+0000","last_change":"2024-05-08T23:26:04.386202+0000","last_active":"2024-05-08T23:26:06.106783+0000","last_peered":"2024-05-08T23:26:06.106783+0000","last_clean":"2024-05-08T23:26:06.106783+0000","last_became_active":"2024-05-08T23:26:04.385897+0000","last_became_peered":"2024-05-08T23:26:04.385897+0000","last_unstale":"2024-05-08T23:26:06.106783+0000","last_undegraded":"2024-05-08T23:26:06.106783+0000","last_fullsized":"2024-05-08T23:26:06.106783+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:15.891844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365071+0000","last_change":"2024-05-08T23:26:04.376144+0000","last_active":"2024-05-08T23:26:05.365071+0000","last_peered":"2024-05-08T23:26:05.365071+0000","last_clean":"2024-05-08T23:26:05.365071+0000","last_became_active":"2024-05-08T23:26:04.375870+0000","last_became_peered":"2024-05-08T23:26:04.375870+0000","last_unstale":"2024-05-08T23:26:05.365071+0000","last_undegraded":"2024-05-08T23:26:05.365071+0000","last_fullsized":"2024-05-08T23:26:05.365071+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:53.570674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365640+0000","last_change":"2024-05-08T23:26:04.391205+0000","last_active":"2024-05-08T23:26:05.365640+0000","last_peered":"2024-05-08T23:26:05.365640+0000","last_clean":"2024-05-08T23:26:05.365640+0000","last_became_active":"2024-05-08T23:26:04.390895+0000","last_became_peered":"2024-05-08T23:26:04.390895+0000","last_unstale":"2024-05-08T23:26:05.365640+0000","last_undegraded":"2024-05-08T23:26:05.365640+0000","last_fullsized":"2024-05-08T23:26:05.365640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:00.837462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365140+0000","last_change":"2024-05-08T23:26:04.380499+0000","last_active":"2024-05-08T23:26:05.365140+0000","last_peered":"2024-05-08T23:26:05.365140+0000","last_clean":"2024-05-08T23:26:05.365140+0000","last_became_active":"2024-05-08T23:26:04.380163+0000","last_became_peered":"2024-05-08T23:26:04.380163+0000","last_unstale":"2024-05-08T23:26:05.365140+0000","last_undegraded":"2024-05-08T23:26:05.365140+0000","last_fullsized":"2024-05-08T23:26:05.365140+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:39.534202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.392388+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.392064+0000","last_became_peered":"2024-05-08T23:26:04.392064+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:55.698089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.342834+0000","last_change":"2024-05-08T23:26:04.377831+0000","last_active":"2024-05-08T23:26:07.342834+0000","last_peered":"2024-05-08T23:26:07.342834+0000","last_clean":"2024-05-08T23:26:07.342834+0000","last_became_active":"2024-05-08T23:26:04.377518+0000","last_became_peered":"2024-05-08T23:26:04.377518+0000","last_unstale":"2024-05-08T23:26:07.342834+0000","last_undegraded":"2024-05-08T23:26:07.342834+0000","last_fullsized":"2024-05-08T23:26:07.342834+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106353+0000","last_change":"2024-05-08T23:26:04.388002+0000","last_active":"2024-05-08T23:26:06.106353+0000","last_peered":"2024-05-08T23:26:06.106353+0000","last_clean":"2024-05-08T23:26:06.106353+0000","last_became_active":"2024-05-08T23:26:04.386006+0000","last_became_peered":"2024-05-08T23:26:04.386006+0000","last_unstale":"2024-05-08T23:26:06.106353+0000","last_undegraded":"2024-05-08T23:26:06.106353+0000","last_fullsized":"2024-05-08T23:26:06.106353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:34:35.709822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365561+0000","last_change":"2024-05-08T23:26:04.389861+0000","last_active":"2024-05-08T23:26:05.365561+0000","last_peered":"2024-05-08T23:26:05.365561+0000","last_clean":"2024-05-08T23:26:05.365561+0000","last_became_active":"2024-05-08T23:26:04.389582+0000","last_became_peered":"2024-05-08T23:26:04.389582+0000","last_unstale":"2024-05-08T23:26:05.365561+0000","last_undegraded":"2024-05-08T23:26:05.365561+0000","last_fullsized":"2024-05-08T23:26:05.365561+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:12.594073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368663+0000","last_change":"2024-05-08T23:26:04.384004+0000","last_active":"2024-05-08T23:26:07.368663+0000","last_peered":"2024-05-08T23:26:07.368663+0000","last_clean":"2024-05-08T23:26:07.368663+0000","last_became_active":"2024-05-08T23:26:04.383584+0000","last_became_peered":"2024-05-08T23:26:04.383584+0000","last_unstale":"2024-05-08T23:26:07.368663+0000","last_undegraded":"2024-05-08T23:26:07.368663+0000","last_fullsized":"2024-05-08T23:26:07.368663+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:52.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347353+0000","last_change":"2024-05-08T23:26:04.390913+0000","last_active":"2024-05-08T23:26:07.347353+0000","last_peered":"2024-05-08T23:26:07.347353+0000","last_clean":"2024-05-08T23:26:07.347353+0000","last_became_active":"2024-05-08T23:26:04.390654+0000","last_became_peered":"2024-05-08T23:26:04.390654+0000","last_unstale":"2024-05-08T23:26:07.347353+0000","last_undegraded":"2024-05-08T23:26:07.347353+0000","last_fullsized":"2024-05-08T23:26:07.347353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365672+0000","last_change":"2024-05-08T23:26:04.382718+0000","last_active":"2024-05-08T23:26:05.365672+0000","last_peered":"2024-05-08T23:26:05.365672+0000","last_clean":"2024-05-08T23:26:05.365672+0000","last_became_active":"2024-05-08T23:26:04.381951+0000","last_became_peered":"2024-05-08T23:26:04.381951+0000","last_unstale":"2024-05-08T23:26:05.365672+0000","last_undegraded":"2024-05-08T23:26:05.365672+0000","last_fullsized":"2024-05-08T23:26:05.365672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:18.398011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528368+0000","last_change":"2024-05-08T23:26:04.393225+0000","last_active":"2024-05-08T23:26:05.528368+0000","last_peered":"2024-05-08T23:26:05.528368+0000","last_clean":"2024-05-08T23:26:05.528368+0000","last_became_active":"2024-05-08T23:26:04.392933+0000","last_became_peered":"2024-05-08T23:26:04.392933+0000","last_unstale":"2024-05-08T23:26:05.528368+0000","last_undegraded":"2024-05-08T23:26:05.528368+0000","last_fullsized":"2024-05-08T23:26:05.528368+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366036+0000","last_change":"2024-05-08T23:26:04.380528+0000","last_active":"2024-05-08T23:26:05.366036+0000","last_peered":"2024-05-08T23:26:05.366036+0000","last_clean":"2024-05-08T23:26:05.366036+0000","last_became_active":"2024-05-08T23:26:04.380129+0000","last_became_peered":"2024-05-08T23:26:04.380129+0000","last_unstale":"2024-05-08T23:26:05.366036+0000","last_undegraded":"2024-05-08T23:26:05.366036+0000","last_fullsized":"2024-05-08T23:26:05.366036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.038766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528231+0000","last_change":"2024-05-08T23:26:04.382434+0000","last_active":"2024-05-08T23:26:05.528231+0000","last_peered":"2024-05-08T23:26:05.528231+0000","last_clean":"2024-05-08T23:26:05.528231+0000","last_became_active":"2024-05-08T23:26:04.382015+0000","last_became_peered":"2024-05-08T23:26:04.382015+0000","last_unstale":"2024-05-08T23:26:05.528231+0000","last_undegraded":"2024-05-08T23:26:05.528231+0000","last_fullsized":"2024-05-08T23:26:05.528231+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:28.039644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368520+0000","last_change":"2024-05-08T23:26:04.395148+0000","last_active":"2024-05-08T23:26:07.368520+0000","last_peered":"2024-05-08T23:26:07.368520+0000","last_clean":"2024-05-08T23:26:07.368520+0000","last_became_active":"2024-05-08T23:26:04.394850+0000","last_became_peered":"2024-05-08T23:26:04.394850+0000","last_unstale":"2024-05-08T23:26:07.368520+0000","last_undegraded":"2024-05-08T23:26:07.368520+0000","last_fullsized":"2024-05-08T23:26:07.368520+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:39.285912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365565+0000","last_change":"2024-05-08T23:26:04.386467+0000","last_active":"2024-05-08T23:26:05.365565+0000","last_peered":"2024-05-08T23:26:05.365565+0000","last_clean":"2024-05-08T23:26:05.365565+0000","last_became_active":"2024-05-08T23:26:04.386005+0000","last_became_peered":"2024-05-08T23:26:04.386005+0000","last_unstale":"2024-05-08T23:26:05.365565+0000","last_undegraded":"2024-05-08T23:26:05.365565+0000","last_fullsized":"2024-05-08T23:26:05.365565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:08.141995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365004+0000","last_change":"2024-05-08T23:26:04.384544+0000","last_active":"2024-05-08T23:26:05.365004+0000","last_peered":"2024-05-08T23:26:05.365004+0000","last_clean":"2024-05-08T23:26:05.365004+0000","last_became_active":"2024-05-08T23:26:04.384240+0000","last_became_peered":"2024-05-08T23:26:04.384240+0000","last_unstale":"2024-05-08T23:26:05.365004+0000","last_undegraded":"2024-05-08T23:26:05.365004+0000","last_fullsized":"2024-05-08T23:26:05.365004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:44.177271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106649+0000","last_change":"2024-05-08T23:26:04.387389+0000","last_active":"2024-05-08T23:26:06.106649+0000","last_peered":"2024-05-08T23:26:06.106649+0000","last_clean":"2024-05-08T23:26:06.106649+0000","last_became_active":"2024-05-08T23:26:04.383192+0000","last_became_peered":"2024-05-08T23:26:04.383192+0000","last_unstale":"2024-05-08T23:26:06.106649+0000","last_undegraded":"2024-05-08T23:26:06.106649+0000","last_fullsized":"2024-05-08T23:26:06.106649+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528319+0000","last_change":"2024-05-08T23:26:04.374428+0000","last_active":"2024-05-08T23:26:05.528319+0000","last_peered":"2024-05-08T23:26:05.528319+0000","last_clean":"2024-05-08T23:26:05.528319+0000","last_became_active":"2024-05-08T23:26:04.374165+0000","last_became_peered":"2024-05-08T23:26:04.374165+0000","last_unstale":"2024-05-08T23:26:05.528319+0000","last_undegraded":"2024-05-08T23:26:05.528319+0000","last_fullsized":"2024-05-08T23:26:05.528319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368601+0000","last_change":"2024-05-08T23:26:04.378977+0000","last_active":"2024-05-08T23:26:07.368601+0000","last_peered":"2024-05-08T23:26:07.368601+0000","last_clean":"2024-05-08T23:26:07.368601+0000","last_became_active":"2024-05-08T23:26:04.378470+0000","last_became_peered":"2024-05-08T23:26:04.378470+0000","last_unstale":"2024-05-08T23:26:07.368601+0000","last_undegraded":"2024-05-08T23:26:07.368601+0000","last_fullsized":"2024-05-08T23:26:07.368601+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.979369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368589+0000","last_change":"2024-05-08T23:26:04.383843+0000","last_active":"2024-05-08T23:26:07.368589+0000","last_peered":"2024-05-08T23:26:07.368589+0000","last_clean":"2024-05-08T23:26:07.368589+0000","last_became_active":"2024-05-08T23:26:04.383503+0000","last_became_peered":"2024-05-08T23:26:04.383503+0000","last_unstale":"2024-05-08T23:26:07.368589+0000","last_undegraded":"2024-05-08T23:26:07.368589+0000","last_fullsized":"2024-05-08T23:26:07.368589+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:17.695615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347276+0000","last_change":"2024-05-08T23:26:04.380605+0000","last_active":"2024-05-08T23:26:07.347276+0000","last_peered":"2024-05-08T23:26:07.347276+0000","last_clean":"2024-05-08T23:26:07.347276+0000","last_became_active":"2024-05-08T23:26:04.380307+0000","last_became_peered":"2024-05-08T23:26:04.380307+0000","last_unstale":"2024-05-08T23:26:07.347276+0000","last_undegraded":"2024-05-08T23:26:07.347276+0000","last_fullsized":"2024-05-08T23:26:07.347276+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:27.367580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347249+0000","last_change":"2024-05-08T23:26:04.395150+0000","last_active":"2024-05-08T23:26:07.347249+0000","last_peered":"2024-05-08T23:26:07.347249+0000","last_clean":"2024-05-08T23:26:07.347249+0000","last_became_active":"2024-05-08T23:26:04.394845+0000","last_became_peered":"2024-05-08T23:26:04.394845+0000","last_unstale":"2024-05-08T23:26:07.347249+0000","last_undegraded":"2024-05-08T23:26:07.347249+0000","last_fullsized":"2024-05-08T23:26:07.347249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368695+0000","last_change":"2024-05-08T23:26:04.383933+0000","last_active":"2024-05-08T23:26:07.368695+0000","last_peered":"2024-05-08T23:26:07.368695+0000","last_clean":"2024-05-08T23:26:07.368695+0000","last_became_active":"2024-05-08T23:26:04.383614+0000","last_became_peered":"2024-05-08T23:26:04.383614+0000","last_unstale":"2024-05-08T23:26:07.368695+0000","last_undegraded":"2024-05-08T23:26:07.368695+0000","last_fullsized":"2024-05-08T23:26:07.368695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:47.989361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.342988+0000","last_change":"2024-05-08T23:26:04.387174+0000","last_active":"2024-05-08T23:26:07.342988+0000","last_peered":"2024-05-08T23:26:07.342988+0000","last_clean":"2024-05-08T23:26:07.342988+0000","last_became_active":"2024-05-08T23:26:04.386653+0000","last_became_peered":"2024-05-08T23:26:04.386653+0000","last_unstale":"2024-05-08T23:26:07.342988+0000","last_undegraded":"2024-05-08T23:26:07.342988+0000","last_fullsized":"2024-05-08T23:26:07.342988+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:52.309419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365110+0000","last_change":"2024-05-08T23:26:04.374586+0000","last_active":"2024-05-08T23:26:05.365110+0000","last_peered":"2024-05-08T23:26:05.365110+0000","last_clean":"2024-05-08T23:26:05.365110+0000","last_became_active":"2024-05-08T23:26:04.374382+0000","last_became_peered":"2024-05-08T23:26:04.374382+0000","last_unstale":"2024-05-08T23:26:05.365110+0000","last_undegraded":"2024-05-08T23:26:05.365110+0000","last_fullsized":"2024-05-08T23:26:05.365110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:41.989450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368639+0000","last_change":"2024-05-08T23:26:04.384048+0000","last_active":"2024-05-08T23:26:07.368639+0000","last_peered":"2024-05-08T23:26:07.368639+0000","last_clean":"2024-05-08T23:26:07.368639+0000","last_became_active":"2024-05-08T23:26:04.383520+0000","last_became_peered":"2024-05-08T23:26:04.383520+0000","last_unstale":"2024-05-08T23:26:07.368639+0000","last_undegraded":"2024-05-08T23:26:07.368639+0000","last_fullsized":"2024-05-08T23:26:07.368639+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:09.803388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364727+0000","last_change":"2024-05-08T23:26:04.376502+0000","last_active":"2024-05-08T23:26:05.364727+0000","last_peered":"2024-05-08T23:26:05.364727+0000","last_clean":"2024-05-08T23:26:05.364727+0000","last_became_active":"2024-05-08T23:26:04.376226+0000","last_became_peered":"2024-05-08T23:26:04.376226+0000","last_unstale":"2024-05-08T23:26:05.364727+0000","last_undegraded":"2024-05-08T23:26:05.364727+0000","last_fullsized":"2024-05-08T23:26:05.364727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:36.452021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.342928+0000","last_change":"2024-05-08T23:26:04.393135+0000","last_active":"2024-05-08T23:26:07.342928+0000","last_peered":"2024-05-08T23:26:07.342928+0000","last_clean":"2024-05-08T23:26:07.342928+0000","last_became_active":"2024-05-08T23:26:04.392824+0000","last_became_peered":"2024-05-08T23:26:04.392824+0000","last_unstale":"2024-05-08T23:26:07.342928+0000","last_undegraded":"2024-05-08T23:26:07.342928+0000","last_fullsized":"2024-05-08T23:26:07.342928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:47.989361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.342860+0000","last_change":"2024-05-08T23:26:04.387882+0000","last_active":"2024-05-08T23:26:07.342860+0000","last_peered":"2024-05-08T23:26:07.342860+0000","last_clean":"2024-05-08T23:26:07.342860+0000","last_became_active":"2024-05-08T23:26:04.387600+0000","last_became_peered":"2024-05-08T23:26:04.387600+0000","last_unstale":"2024-05-08T23:26:07.342860+0000","last_undegraded":"2024-05-08T23:26:07.342860+0000","last_fullsized":"2024-05-08T23:26:07.342860+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:29.594480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106297+0000","last_change":"2024-05-08T23:26:04.390849+0000","last_active":"2024-05-08T23:26:06.106297+0000","last_peered":"2024-05-08T23:26:06.106297+0000","last_clean":"2024-05-08T23:26:06.106297+0000","last_became_active":"2024-05-08T23:26:04.390577+0000","last_became_peered":"2024-05-08T23:26:04.390577+0000","last_unstale":"2024-05-08T23:26:06.106297+0000","last_undegraded":"2024-05-08T23:26:06.106297+0000","last_fullsized":"2024-05-08T23:26:06.106297+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:00.857944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347387+0000","last_change":"2024-05-08T23:26:04.387838+0000","last_active":"2024-05-08T23:26:07.347387+0000","last_peered":"2024-05-08T23:26:07.347387+0000","last_clean":"2024-05-08T23:26:07.347387+0000","last_became_active":"2024-05-08T23:26:04.387594+0000","last_became_peered":"2024-05-08T23:26:04.387594+0000","last_unstale":"2024-05-08T23:26:07.347387+0000","last_undegraded":"2024-05-08T23:26:07.347387+0000","last_fullsized":"2024-05-08T23:26:07.347387+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:07.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347429+0000","last_change":"2024-05-08T23:26:03.918800+0000","last_active":"2024-05-08T23:26:07.347429+0000","last_peered":"2024-05-08T23:26:07.347429+0000","last_clean":"2024-05-08T23:26:07.347429+0000","last_became_active":"2024-05-08T23:26:03.918509+0000","last_became_peered":"2024-05-08T23:26:03.918509+0000","last_unstale":"2024-05-08T23:26:07.347429+0000","last_undegraded":"2024-05-08T23:26:07.347429+0000","last_fullsized":"2024-05-08T23:26:07.347429+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:46.488568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365685+0000","last_change":"2024-05-08T23:26:04.386283+0000","last_active":"2024-05-08T23:26:05.365685+0000","last_peered":"2024-05-08T23:26:05.365685+0000","last_clean":"2024-05-08T23:26:05.365685+0000","last_became_active":"2024-05-08T23:26:04.385690+0000","last_became_peered":"2024-05-08T23:26:04.385690+0000","last_unstale":"2024-05-08T23:26:05.365685+0000","last_undegraded":"2024-05-08T23:26:05.365685+0000","last_fullsized":"2024-05-08T23:26:05.365685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:12.667029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528769+0000","last_change":"2024-05-08T23:26:03.912463+0000","last_active":"2024-05-08T23:26:05.528769+0000","last_peered":"2024-05-08T23:26:05.528769+0000","last_clean":"2024-05-08T23:26:05.528769+0000","last_became_active":"2024-05-08T23:26:03.912178+0000","last_became_peered":"2024-05-08T23:26:03.912178+0000","last_unstale":"2024-05-08T23:26:05.528769+0000","last_undegraded":"2024-05-08T23:26:05.528769+0000","last_fullsized":"2024-05-08T23:26:05.528769+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:02.228610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106437+0000","last_change":"2024-05-08T23:26:04.385614+0000","last_active":"2024-05-08T23:26:06.106437+0000","last_peered":"2024-05-08T23:26:06.106437+0000","last_clean":"2024-05-08T23:26:06.106437+0000","last_became_active":"2024-05-08T23:26:04.385325+0000","last_became_peered":"2024-05-08T23:26:04.385325+0000","last_unstale":"2024-05-08T23:26:06.106437+0000","last_undegraded":"2024-05-08T23:26:06.106437+0000","last_fullsized":"2024-05-08T23:26:06.106437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.979369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368651+0000","last_change":"2024-05-08T23:26:03.911580+0000","last_active":"2024-05-08T23:26:07.368651+0000","last_peered":"2024-05-08T23:26:07.368651+0000","last_clean":"2024-05-08T23:26:07.368651+0000","last_became_active":"2024-05-08T23:26:03.911071+0000","last_became_peered":"2024-05-08T23:26:03.911071+0000","last_unstale":"2024-05-08T23:26:07.368651+0000","last_undegraded":"2024-05-08T23:26:07.368651+0000","last_fullsized":"2024-05-08T23:26:07.368651+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:42:39.409566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347366+0000","last_change":"2024-05-08T23:26:04.380473+0000","last_active":"2024-05-08T23:26:07.347366+0000","last_peered":"2024-05-08T23:26:07.347366+0000","last_clean":"2024-05-08T23:26:07.347366+0000","last_became_active":"2024-05-08T23:26:04.380170+0000","last_became_peered":"2024-05-08T23:26:04.380170+0000","last_unstale":"2024-05-08T23:26:07.347366+0000","last_undegraded":"2024-05-08T23:26:07.347366+0000","last_fullsized":"2024-05-08T23:26:07.347366+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:30.884597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106822+0000","last_change":"2024-05-08T23:26:03.918662+0000","last_active":"2024-05-08T23:26:06.106822+0000","last_peered":"2024-05-08T23:26:06.106822+0000","last_clean":"2024-05-08T23:26:06.106822+0000","last_became_active":"2024-05-08T23:26:03.918367+0000","last_became_peered":"2024-05-08T23:26:03.918367+0000","last_unstale":"2024-05-08T23:26:06.106822+0000","last_undegraded":"2024-05-08T23:26:06.106822+0000","last_fullsized":"2024-05-08T23:26:06.106822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.342960+0000","last_change":"2024-05-08T23:26:04.391595+0000","last_active":"2024-05-08T23:26:07.342960+0000","last_peered":"2024-05-08T23:26:07.342960+0000","last_clean":"2024-05-08T23:26:07.342960+0000","last_became_active":"2024-05-08T23:26:04.391291+0000","last_became_peered":"2024-05-08T23:26:04.391291+0000","last_unstale":"2024-05-08T23:26:07.342960+0000","last_undegraded":"2024-05-08T23:26:07.342960+0000","last_fullsized":"2024-05-08T23:26:07.342960+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:17.695615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364906+0000","last_change":"2024-05-08T23:26:03.916776+0000","last_active":"2024-05-08T23:26:05.364906+0000","last_peered":"2024-05-08T23:26:05.364906+0000","last_clean":"2024-05-08T23:26:05.364906+0000","last_became_active":"2024-05-08T23:26:03.916402+0000","last_became_peered":"2024-05-08T23:26:03.916402+0000","last_unstale":"2024-05-08T23:26:05.364906+0000","last_undegraded":"2024-05-08T23:26:05.364906+0000","last_fullsized":"2024-05-08T23:26:05.364906+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:04.288000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368623+0000","last_change":"2024-05-08T23:26:04.381800+0000","last_active":"2024-05-08T23:26:07.368623+0000","last_peered":"2024-05-08T23:26:07.368623+0000","last_clean":"2024-05-08T23:26:07.368623+0000","last_became_active":"2024-05-08T23:26:04.381476+0000","last_became_peered":"2024-05-08T23:26:04.381476+0000","last_unstale":"2024-05-08T23:26:07.368623+0000","last_undegraded":"2024-05-08T23:26:07.368623+0000","last_fullsized":"2024-05-08T23:26:07.368623+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.907260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106261+0000","last_change":"2024-05-08T23:26:03.915034+0000","last_active":"2024-05-08T23:26:06.106261+0000","last_peered":"2024-05-08T23:26:06.106261+0000","last_clean":"2024-05-08T23:26:06.106261+0000","last_became_active":"2024-05-08T23:26:03.914666+0000","last_became_peered":"2024-05-08T23:26:03.914666+0000","last_unstale":"2024-05-08T23:26:06.106261+0000","last_undegraded":"2024-05-08T23:26:06.106261+0000","last_fullsized":"2024-05-08T23:26:06.106261+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:43.175140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365034+0000","last_change":"2024-05-08T23:26:04.381034+0000","last_active":"2024-05-08T23:26:05.365034+0000","last_peered":"2024-05-08T23:26:05.365034+0000","last_clean":"2024-05-08T23:26:05.365034+0000","last_became_active":"2024-05-08T23:26:04.380769+0000","last_became_peered":"2024-05-08T23:26:04.380769+0000","last_unstale":"2024-05-08T23:26:05.365034+0000","last_undegraded":"2024-05-08T23:26:05.365034+0000","last_fullsized":"2024-05-08T23:26:05.365034+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:35.530781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365301+0000","last_change":"2024-05-08T23:26:03.917031+0000","last_active":"2024-05-08T23:26:05.365301+0000","last_peered":"2024-05-08T23:26:05.365301+0000","last_clean":"2024-05-08T23:26:05.365301+0000","last_became_active":"2024-05-08T23:26:03.916394+0000","last_became_peered":"2024-05-08T23:26:03.916394+0000","last_unstale":"2024-05-08T23:26:05.365301+0000","last_undegraded":"2024-05-08T23:26:05.365301+0000","last_fullsized":"2024-05-08T23:26:05.365301+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:38.532072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528242+0000","last_change":"2024-05-08T23:26:04.374371+0000","last_active":"2024-05-08T23:26:05.528242+0000","last_peered":"2024-05-08T23:26:05.528242+0000","last_clean":"2024-05-08T23:26:05.528242+0000","last_became_active":"2024-05-08T23:26:04.374097+0000","last_became_peered":"2024-05-08T23:26:04.374097+0000","last_unstale":"2024-05-08T23:26:05.528242+0000","last_undegraded":"2024-05-08T23:26:05.528242+0000","last_fullsized":"2024-05-08T23:26:05.528242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365139+0000","last_change":"2024-05-08T23:26:03.911594+0000","last_active":"2024-05-08T23:26:05.365139+0000","last_peered":"2024-05-08T23:26:05.365139+0000","last_clean":"2024-05-08T23:26:05.365139+0000","last_became_active":"2024-05-08T23:26:03.911273+0000","last_became_peered":"2024-05-08T23:26:03.911273+0000","last_unstale":"2024-05-08T23:26:05.365139+0000","last_undegraded":"2024-05-08T23:26:05.365139+0000","last_fullsized":"2024-05-08T23:26:05.365139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:30.134959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.343002+0000","last_change":"2024-05-08T23:26:04.372794+0000","last_active":"2024-05-08T23:26:07.343002+0000","last_peered":"2024-05-08T23:26:07.343002+0000","last_clean":"2024-05-08T23:26:07.343002+0000","last_became_active":"2024-05-08T23:26:04.372536+0000","last_became_peered":"2024-05-08T23:26:04.372536+0000","last_unstale":"2024-05-08T23:26:07.343002+0000","last_undegraded":"2024-05-08T23:26:07.343002+0000","last_fullsized":"2024-05-08T23:26:07.343002+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:06.816807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368707+0000","last_change":"2024-05-08T23:26:03.917379+0000","last_active":"2024-05-08T23:26:07.368707+0000","last_peered":"2024-05-08T23:26:07.368707+0000","last_clean":"2024-05-08T23:26:07.368707+0000","last_became_active":"2024-05-08T23:26:03.917063+0000","last_became_peered":"2024-05-08T23:26:03.917063+0000","last_unstale":"2024-05-08T23:26:07.368707+0000","last_undegraded":"2024-05-08T23:26:07.368707+0000","last_fullsized":"2024-05-08T23:26:07.368707+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:43.443817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347408+0000","last_change":"2024-05-08T23:26:04.388857+0000","last_active":"2024-05-08T23:26:07.347408+0000","last_peered":"2024-05-08T23:26:07.347408+0000","last_clean":"2024-05-08T23:26:07.347408+0000","last_became_active":"2024-05-08T23:26:04.388581+0000","last_became_peered":"2024-05-08T23:26:04.388581+0000","last_unstale":"2024-05-08T23:26:07.347408+0000","last_undegraded":"2024-05-08T23:26:07.347408+0000","last_fullsized":"2024-05-08T23:26:07.347408+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:52.232929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106865+0000","last_change":"2024-05-08T23:26:03.907421+0000","last_active":"2024-05-08T23:26:06.106865+0000","last_peered":"2024-05-08T23:26:06.106865+0000","last_clean":"2024-05-08T23:26:06.106865+0000","last_became_active":"2024-05-08T23:26:03.907143+0000","last_became_peered":"2024-05-08T23:26:03.907143+0000","last_unstale":"2024-05-08T23:26:06.106865+0000","last_undegraded":"2024-05-08T23:26:06.106865+0000","last_fullsized":"2024-05-08T23:26:06.106865+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:07.208552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106627+0000","last_change":"2024-05-08T23:26:04.386708+0000","last_active":"2024-05-08T23:26:06.106627+0000","last_peered":"2024-05-08T23:26:06.106627+0000","last_clean":"2024-05-08T23:26:06.106627+0000","last_became_active":"2024-05-08T23:26:04.385984+0000","last_became_peered":"2024-05-08T23:26:04.385984+0000","last_unstale":"2024-05-08T23:26:06.106627+0000","last_undegraded":"2024-05-08T23:26:06.106627+0000","last_fullsized":"2024-05-08T23:26:06.106627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:47.490698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365174+0000","last_change":"2024-05-08T23:26:03.913646+0000","last_active":"2024-05-08T23:26:05.365174+0000","last_peered":"2024-05-08T23:26:05.365174+0000","last_clean":"2024-05-08T23:26:05.365174+0000","last_became_active":"2024-05-08T23:26:03.913208+0000","last_became_peered":"2024-05-08T23:26:03.913208+0000","last_unstale":"2024-05-08T23:26:05.365174+0000","last_undegraded":"2024-05-08T23:26:05.365174+0000","last_fullsized":"2024-05-08T23:26:05.365174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:02.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365662+0000","last_change":"2024-05-08T23:26:04.375029+0000","last_active":"2024-05-08T23:26:05.365662+0000","last_peered":"2024-05-08T23:26:05.365662+0000","last_clean":"2024-05-08T23:26:05.365662+0000","last_became_active":"2024-05-08T23:26:04.374797+0000","last_became_peered":"2024-05-08T23:26:04.374797+0000","last_unstale":"2024-05-08T23:26:05.365662+0000","last_undegraded":"2024-05-08T23:26:05.365662+0000","last_fullsized":"2024-05-08T23:26:05.365662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:18.355499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365807+0000","last_change":"2024-05-08T23:26:03.913239+0000","last_active":"2024-05-08T23:26:05.365807+0000","last_peered":"2024-05-08T23:26:05.365807+0000","last_clean":"2024-05-08T23:26:05.365807+0000","last_became_active":"2024-05-08T23:26:03.912892+0000","last_became_peered":"2024-05-08T23:26:03.912892+0000","last_unstale":"2024-05-08T23:26:05.365807+0000","last_undegraded":"2024-05-08T23:26:05.365807+0000","last_fullsized":"2024-05-08T23:26:05.365807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:28.036636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347288+0000","last_change":"2024-05-08T23:26:04.381564+0000","last_active":"2024-05-08T23:26:07.347288+0000","last_peered":"2024-05-08T23:26:07.347288+0000","last_clean":"2024-05-08T23:26:07.347288+0000","last_became_active":"2024-05-08T23:26:04.381293+0000","last_became_peered":"2024-05-08T23:26:04.381293+0000","last_unstale":"2024-05-08T23:26:07.347288+0000","last_undegraded":"2024-05-08T23:26:07.347288+0000","last_fullsized":"2024-05-08T23:26:07.347288+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:36.961593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365371+0000","last_change":"2024-05-08T23:26:03.917272+0000","last_active":"2024-05-08T23:26:05.365371+0000","last_peered":"2024-05-08T23:26:05.365371+0000","last_clean":"2024-05-08T23:26:05.365371+0000","last_became_active":"2024-05-08T23:26:03.916960+0000","last_became_peered":"2024-05-08T23:26:03.916960+0000","last_unstale":"2024-05-08T23:26:05.365371+0000","last_undegraded":"2024-05-08T23:26:05.365371+0000","last_fullsized":"2024-05-08T23:26:05.365371+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:41.316481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.342975+0000","last_change":"2024-05-08T23:26:04.375997+0000","last_active":"2024-05-08T23:26:07.342975+0000","last_peered":"2024-05-08T23:26:07.342975+0000","last_clean":"2024-05-08T23:26:07.342975+0000","last_became_active":"2024-05-08T23:26:04.375654+0000","last_became_peered":"2024-05-08T23:26:04.375654+0000","last_unstale":"2024-05-08T23:26:07.342975+0000","last_undegraded":"2024-05-08T23:26:07.342975+0000","last_fullsized":"2024-05-08T23:26:07.342975+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:03.230741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106390+0000","last_change":"2024-05-08T23:26:03.911788+0000","last_active":"2024-05-08T23:26:06.106390+0000","last_peered":"2024-05-08T23:26:06.106390+0000","last_clean":"2024-05-08T23:26:06.106390+0000","last_became_active":"2024-05-08T23:26:03.911527+0000","last_became_peered":"2024-05-08T23:26:03.911527+0000","last_unstale":"2024-05-08T23:26:06.106390+0000","last_undegraded":"2024-05-08T23:26:06.106390+0000","last_fullsized":"2024-05-08T23:26:06.106390+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:44.784641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528178+0000","last_change":"2024-05-08T23:26:04.394848+0000","last_active":"2024-05-08T23:26:05.528178+0000","last_peered":"2024-05-08T23:26:05.528178+0000","last_clean":"2024-05-08T23:26:05.528178+0000","last_became_active":"2024-05-08T23:26:04.394520+0000","last_became_peered":"2024-05-08T23:26:04.394520+0000","last_unstale":"2024-05-08T23:26:05.528178+0000","last_undegraded":"2024-05-08T23:26:05.528178+0000","last_fullsized":"2024-05-08T23:26:05.528178+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:56.406270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.342884+0000","last_change":"2024-05-08T23:26:03.904993+0000","last_active":"2024-05-08T23:26:07.342884+0000","last_peered":"2024-05-08T23:26:07.342884+0000","last_clean":"2024-05-08T23:26:07.342884+0000","last_became_active":"2024-05-08T23:26:03.904689+0000","last_became_peered":"2024-05-08T23:26:03.904689+0000","last_unstale":"2024-05-08T23:26:07.342884+0000","last_undegraded":"2024-05-08T23:26:07.342884+0000","last_fullsized":"2024-05-08T23:26:07.342884+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:54.695959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365441+0000","last_change":"2024-05-08T23:26:04.386132+0000","last_active":"2024-05-08T23:26:05.365441+0000","last_peered":"2024-05-08T23:26:05.365441+0000","last_clean":"2024-05-08T23:26:05.365441+0000","last_became_active":"2024-05-08T23:26:04.385842+0000","last_became_peered":"2024-05-08T23:26:04.385842+0000","last_unstale":"2024-05-08T23:26:05.365441+0000","last_undegraded":"2024-05-08T23:26:05.365441+0000","last_fullsized":"2024-05-08T23:26:05.365441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:42.318612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368554+0000","last_change":"2024-05-08T23:26:03.912054+0000","last_active":"2024-05-08T23:26:07.368554+0000","last_peered":"2024-05-08T23:26:07.368554+0000","last_clean":"2024-05-08T23:26:07.368554+0000","last_became_active":"2024-05-08T23:26:03.911775+0000","last_became_peered":"2024-05-08T23:26:03.911775+0000","last_unstale":"2024-05-08T23:26:07.368554+0000","last_undegraded":"2024-05-08T23:26:07.368554+0000","last_fullsized":"2024-05-08T23:26:07.368554+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:25.875643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368673+0000","last_change":"2024-05-08T23:26:04.380260+0000","last_active":"2024-05-08T23:26:07.368673+0000","last_peered":"2024-05-08T23:26:07.368673+0000","last_clean":"2024-05-08T23:26:07.368673+0000","last_became_active":"2024-05-08T23:26:04.379931+0000","last_became_peered":"2024-05-08T23:26:04.379931+0000","last_unstale":"2024-05-08T23:26:07.368673+0000","last_undegraded":"2024-05-08T23:26:07.368673+0000","last_fullsized":"2024-05-08T23:26:07.368673+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:03.230741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106703+0000","last_change":"2024-05-08T23:26:03.917170+0000","last_active":"2024-05-08T23:26:06.106703+0000","last_peered":"2024-05-08T23:26:06.106703+0000","last_clean":"2024-05-08T23:26:06.106703+0000","last_became_active":"2024-05-08T23:26:03.916888+0000","last_became_peered":"2024-05-08T23:26:03.916888+0000","last_unstale":"2024-05-08T23:26:06.106703+0000","last_undegraded":"2024-05-08T23:26:06.106703+0000","last_fullsized":"2024-05-08T23:26:06.106703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:59.835332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347418+0000","last_change":"2024-05-08T23:26:04.385163+0000","last_active":"2024-05-08T23:26:07.347418+0000","last_peered":"2024-05-08T23:26:07.347418+0000","last_clean":"2024-05-08T23:26:07.347418+0000","last_became_active":"2024-05-08T23:26:04.384430+0000","last_became_peered":"2024-05-08T23:26:04.384430+0000","last_unstale":"2024-05-08T23:26:07.347418+0000","last_undegraded":"2024-05-08T23:26:07.347418+0000","last_fullsized":"2024-05-08T23:26:07.347418+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:13.482666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368684+0000","last_change":"2024-05-08T23:26:03.909206+0000","last_active":"2024-05-08T23:26:07.368684+0000","last_peered":"2024-05-08T23:26:07.368684+0000","last_clean":"2024-05-08T23:26:07.368684+0000","last_became_active":"2024-05-08T23:26:03.908682+0000","last_became_peered":"2024-05-08T23:26:03.908682+0000","last_unstale":"2024-05-08T23:26:07.368684+0000","last_undegraded":"2024-05-08T23:26:07.368684+0000","last_fullsized":"2024-05-08T23:26:07.368684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:48.022908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365760+0000","last_change":"2024-05-08T23:26:04.382870+0000","last_active":"2024-05-08T23:26:05.365760+0000","last_peered":"2024-05-08T23:26:05.365760+0000","last_clean":"2024-05-08T23:26:05.365760+0000","last_became_active":"2024-05-08T23:26:04.382320+0000","last_became_peered":"2024-05-08T23:26:04.382320+0000","last_unstale":"2024-05-08T23:26:05.365760+0000","last_undegraded":"2024-05-08T23:26:05.365760+0000","last_fullsized":"2024-05-08T23:26:05.365760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:17.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106864+0000","last_change":"2024-05-08T23:26:03.910712+0000","last_active":"2024-05-08T23:26:06.106864+0000","last_peered":"2024-05-08T23:26:06.106864+0000","last_clean":"2024-05-08T23:26:06.106864+0000","last_became_active":"2024-05-08T23:26:03.910424+0000","last_became_peered":"2024-05-08T23:26:03.910424+0000","last_unstale":"2024-05-08T23:26:06.106864+0000","last_undegraded":"2024-05-08T23:26:06.106864+0000","last_fullsized":"2024-05-08T23:26:06.106864+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:16.285704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365799+0000","last_change":"2024-05-08T23:26:04.378589+0000","last_active":"2024-05-08T23:26:05.365799+0000","last_peered":"2024-05-08T23:26:05.365799+0000","last_clean":"2024-05-08T23:26:05.365799+0000","last_became_active":"2024-05-08T23:26:04.378300+0000","last_became_peered":"2024-05-08T23:26:04.378300+0000","last_unstale":"2024-05-08T23:26:05.365799+0000","last_undegraded":"2024-05-08T23:26:05.365799+0000","last_fullsized":"2024-05-08T23:26:05.365799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.786772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347307+0000","last_change":"2024-05-08T23:26:03.916915+0000","last_active":"2024-05-08T23:26:07.347307+0000","last_peered":"2024-05-08T23:26:07.347307+0000","last_clean":"2024-05-08T23:26:07.347307+0000","last_became_active":"2024-05-08T23:26:03.916632+0000","last_became_peered":"2024-05-08T23:26:03.916632+0000","last_unstale":"2024-05-08T23:26:07.347307+0000","last_undegraded":"2024-05-08T23:26:07.347307+0000","last_fullsized":"2024-05-08T23:26:07.347307+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:22.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364817+0000","last_change":"2024-05-08T23:26:04.375139+0000","last_active":"2024-05-08T23:26:05.364817+0000","last_peered":"2024-05-08T23:26:05.364817+0000","last_clean":"2024-05-08T23:26:05.364817+0000","last_became_active":"2024-05-08T23:26:04.374874+0000","last_became_peered":"2024-05-08T23:26:04.374874+0000","last_unstale":"2024-05-08T23:26:05.364817+0000","last_undegraded":"2024-05-08T23:26:05.364817+0000","last_fullsized":"2024-05-08T23:26:05.364817+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:42.364785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106533+0000","last_change":"2024-05-08T23:26:03.915036+0000","last_active":"2024-05-08T23:26:06.106533+0000","last_peered":"2024-05-08T23:26:06.106533+0000","last_clean":"2024-05-08T23:26:06.106533+0000","last_became_active":"2024-05-08T23:26:03.914616+0000","last_became_peered":"2024-05-08T23:26:03.914616+0000","last_unstale":"2024-05-08T23:26:06.106533+0000","last_undegraded":"2024-05-08T23:26:06.106533+0000","last_fullsized":"2024-05-08T23:26:06.106533+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.591942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364460+0000","last_change":"2024-05-08T23:26:04.384503+0000","last_active":"2024-05-08T23:26:05.364460+0000","last_peered":"2024-05-08T23:26:05.364460+0000","last_clean":"2024-05-08T23:26:05.364460+0000","last_became_active":"2024-05-08T23:26:04.383639+0000","last_became_peered":"2024-05-08T23:26:04.383639+0000","last_unstale":"2024-05-08T23:26:05.364460+0000","last_undegraded":"2024-05-08T23:26:05.364460+0000","last_fullsized":"2024-05-08T23:26:05.364460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:13.674487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.342906+0000","last_change":"2024-05-08T23:26:03.912730+0000","last_active":"2024-05-08T23:26:07.342906+0000","last_peered":"2024-05-08T23:26:07.342906+0000","last_clean":"2024-05-08T23:26:07.342906+0000","last_became_active":"2024-05-08T23:26:03.912415+0000","last_became_peered":"2024-05-08T23:26:03.912415+0000","last_unstale":"2024-05-08T23:26:07.342906+0000","last_undegraded":"2024-05-08T23:26:07.342906+0000","last_fullsized":"2024-05-08T23:26:07.342906+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:35.080457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364494+0000","last_change":"2024-05-08T23:26:04.383931+0000","last_active":"2024-05-08T23:26:05.364494+0000","last_peered":"2024-05-08T23:26:05.364494+0000","last_clean":"2024-05-08T23:26:05.364494+0000","last_became_active":"2024-05-08T23:26:04.383598+0000","last_became_peered":"2024-05-08T23:26:04.383598+0000","last_unstale":"2024-05-08T23:26:05.364494+0000","last_undegraded":"2024-05-08T23:26:05.364494+0000","last_fullsized":"2024-05-08T23:26:05.364494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:27.479478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528163+0000","last_change":"2024-05-08T23:26:03.908029+0000","last_active":"2024-05-08T23:26:05.528163+0000","last_peered":"2024-05-08T23:26:05.528163+0000","last_clean":"2024-05-08T23:26:05.528163+0000","last_became_active":"2024-05-08T23:26:03.907475+0000","last_became_peered":"2024-05-08T23:26:03.907475+0000","last_unstale":"2024-05-08T23:26:05.528163+0000","last_undegraded":"2024-05-08T23:26:05.528163+0000","last_fullsized":"2024-05-08T23:26:05.528163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:26.905129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528734+0000","last_change":"2024-05-08T23:26:04.383939+0000","last_active":"2024-05-08T23:26:05.528734+0000","last_peered":"2024-05-08T23:26:05.528734+0000","last_clean":"2024-05-08T23:26:05.528734+0000","last_became_active":"2024-05-08T23:26:04.383318+0000","last_became_peered":"2024-05-08T23:26:04.383318+0000","last_unstale":"2024-05-08T23:26:05.528734+0000","last_undegraded":"2024-05-08T23:26:05.528734+0000","last_fullsized":"2024-05-08T23:26:05.528734+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106789+0000","last_change":"2024-05-08T23:26:03.917152+0000","last_active":"2024-05-08T23:26:06.106789+0000","last_peered":"2024-05-08T23:26:06.106789+0000","last_clean":"2024-05-08T23:26:06.106789+0000","last_became_active":"2024-05-08T23:26:03.916848+0000","last_became_peered":"2024-05-08T23:26:03.916848+0000","last_unstale":"2024-05-08T23:26:06.106789+0000","last_undegraded":"2024-05-08T23:26:06.106789+0000","last_fullsized":"2024-05-08T23:26:06.106789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:05:17.395880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.343026+0000","last_change":"2024-05-08T23:26:04.377822+0000","last_active":"2024-05-08T23:26:07.343026+0000","last_peered":"2024-05-08T23:26:07.343026+0000","last_clean":"2024-05-08T23:26:07.343026+0000","last_became_active":"2024-05-08T23:26:04.377510+0000","last_became_peered":"2024-05-08T23:26:04.377510+0000","last_unstale":"2024-05-08T23:26:07.343026+0000","last_undegraded":"2024-05-08T23:26:07.343026+0000","last_fullsized":"2024-05-08T23:26:07.343026+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:09.803388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"72'1","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.601968+0000","last_change":"2024-05-08T23:26:03.904714+0000","last_active":"2024-05-08T23:26:07.601968+0000","last_peered":"2024-05-08T23:26:07.601968+0000","last_clean":"2024-05-08T23:26:07.601968+0000","last_became_active":"2024-05-08T23:26:03.904414+0000","last_became_peered":"2024-05-08T23:26:03.904414+0000","last_unstale":"2024-05-08T23:26:07.601968+0000","last_undegraded":"2024-05-08T23:26:07.601968+0000","last_fullsized":"2024-05-08T23:26:07.601968+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:14.889713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347330+0000","last_change":"2024-05-08T23:26:04.388899+0000","last_active":"2024-05-08T23:26:07.347330+0000","last_peered":"2024-05-08T23:26:07.347330+0000","last_clean":"2024-05-08T23:26:07.347330+0000","last_became_active":"2024-05-08T23:26:04.388628+0000","last_became_peered":"2024-05-08T23:26:04.388628+0000","last_unstale":"2024-05-08T23:26:07.347330+0000","last_undegraded":"2024-05-08T23:26:07.347330+0000","last_fullsized":"2024-05-08T23:26:07.347330+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:31.092826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365794+0000","last_change":"2024-05-08T23:26:03.910274+0000","last_active":"2024-05-08T23:26:05.365794+0000","last_peered":"2024-05-08T23:26:05.365794+0000","last_clean":"2024-05-08T23:26:05.365794+0000","last_became_active":"2024-05-08T23:26:03.909977+0000","last_became_peered":"2024-05-08T23:26:03.909977+0000","last_unstale":"2024-05-08T23:26:05.365794+0000","last_undegraded":"2024-05-08T23:26:05.365794+0000","last_fullsized":"2024-05-08T23:26:05.365794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:09.127009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.347377+0000","last_change":"2024-05-08T23:26:04.377703+0000","last_active":"2024-05-08T23:26:07.347377+0000","last_peered":"2024-05-08T23:26:07.347377+0000","last_clean":"2024-05-08T23:26:07.347377+0000","last_became_active":"2024-05-08T23:26:04.377184+0000","last_became_peered":"2024-05-08T23:26:04.377184+0000","last_unstale":"2024-05-08T23:26:07.347377+0000","last_undegraded":"2024-05-08T23:26:07.347377+0000","last_fullsized":"2024-05-08T23:26:07.347377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:09.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.528645+0000","last_change":"2024-05-08T23:26:03.910256+0000","last_active":"2024-05-08T23:26:05.528645+0000","last_peered":"2024-05-08T23:26:05.528645+0000","last_clean":"2024-05-08T23:26:05.528645+0000","last_became_active":"2024-05-08T23:26:03.909930+0000","last_became_peered":"2024-05-08T23:26:03.909930+0000","last_unstale":"2024-05-08T23:26:05.528645+0000","last_undegraded":"2024-05-08T23:26:05.528645+0000","last_fullsized":"2024-05-08T23:26:05.528645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:46.987230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.366137+0000","last_change":"2024-05-08T23:26:04.390986+0000","last_active":"2024-05-08T23:26:05.366137+0000","last_peered":"2024-05-08T23:26:05.366137+0000","last_clean":"2024-05-08T23:26:05.366137+0000","last_became_active":"2024-05-08T23:26:04.390656+0000","last_became_peered":"2024-05-08T23:26:04.390656+0000","last_unstale":"2024-05-08T23:26:05.366137+0000","last_undegraded":"2024-05-08T23:26:05.366137+0000","last_fullsized":"2024-05-08T23:26:05.366137+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:26.611874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365238+0000","last_change":"2024-05-08T23:26:03.902695+0000","last_active":"2024-05-08T23:26:05.365238+0000","last_peered":"2024-05-08T23:26:05.365238+0000","last_clean":"2024-05-08T23:26:05.365238+0000","last_became_active":"2024-05-08T23:26:03.902382+0000","last_became_peered":"2024-05-08T23:26:03.902382+0000","last_unstale":"2024-05-08T23:26:05.365238+0000","last_undegraded":"2024-05-08T23:26:05.365238+0000","last_fullsized":"2024-05-08T23:26:05.365238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:08.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106322+0000","last_change":"2024-05-08T23:26:04.387935+0000","last_active":"2024-05-08T23:26:06.106322+0000","last_peered":"2024-05-08T23:26:06.106322+0000","last_clean":"2024-05-08T23:26:06.106322+0000","last_became_active":"2024-05-08T23:26:04.387678+0000","last_became_peered":"2024-05-08T23:26:04.387678+0000","last_unstale":"2024-05-08T23:26:06.106322+0000","last_undegraded":"2024-05-08T23:26:06.106322+0000","last_fullsized":"2024-05-08T23:26:06.106322+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:23.335791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364796+0000","last_change":"2024-05-08T23:26:03.916887+0000","last_active":"2024-05-08T23:26:05.364796+0000","last_peered":"2024-05-08T23:26:05.364796+0000","last_clean":"2024-05-08T23:26:05.364796+0000","last_became_active":"2024-05-08T23:26:03.916407+0000","last_became_peered":"2024-05-08T23:26:03.916407+0000","last_unstale":"2024-05-08T23:26:05.364796+0000","last_undegraded":"2024-05-08T23:26:05.364796+0000","last_fullsized":"2024-05-08T23:26:05.364796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:34.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.365058+0000","last_change":"2024-05-08T23:26:04.389795+0000","last_active":"2024-05-08T23:26:05.365058+0000","last_peered":"2024-05-08T23:26:05.365058+0000","last_clean":"2024-05-08T23:26:05.365058+0000","last_became_active":"2024-05-08T23:26:04.389433+0000","last_became_peered":"2024-05-08T23:26:04.389433+0000","last_unstale":"2024-05-08T23:26:05.365058+0000","last_undegraded":"2024-05-08T23:26:05.365058+0000","last_fullsized":"2024-05-08T23:26:05.365058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:23.681716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:07.368578+0000","last_change":"2024-05-08T23:26:03.914133+0000","last_active":"2024-05-08T23:26:07.368578+0000","last_peered":"2024-05-08T23:26:07.368578+0000","last_clean":"2024-05-08T23:26:07.368578+0000","last_became_active":"2024-05-08T23:26:03.913863+0000","last_became_peered":"2024-05-08T23:26:03.913863+0000","last_unstale":"2024-05-08T23:26:07.368578+0000","last_undegraded":"2024-05-08T23:26:07.368578+0000","last_fullsized":"2024-05-08T23:26:07.368578+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:27.115050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106146+0000","last_change":"2024-05-08T23:26:04.390648+0000","last_active":"2024-05-08T23:26:06.106146+0000","last_peered":"2024-05-08T23:26:06.106146+0000","last_clean":"2024-05-08T23:26:06.106146+0000","last_became_active":"2024-05-08T23:26:04.390358+0000","last_became_peered":"2024-05-08T23:26:04.390358+0000","last_unstale":"2024-05-08T23:26:06.106146+0000","last_undegraded":"2024-05-08T23:26:06.106146+0000","last_fullsized":"2024-05-08T23:26:06.106146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:19.130856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106499+0000","last_change":"2024-05-08T23:26:03.908056+0000","last_active":"2024-05-08T23:26:06.106499+0000","last_peered":"2024-05-08T23:26:06.106499+0000","last_clean":"2024-05-08T23:26:06.106499+0000","last_became_active":"2024-05-08T23:26:03.907603+0000","last_became_peered":"2024-05-08T23:26:03.907603+0000","last_unstale":"2024-05-08T23:26:06.106499+0000","last_undegraded":"2024-05-08T23:26:06.106499+0000","last_fullsized":"2024-05-08T23:26:06.106499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:17.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:05.364615+0000","last_change":"2024-05-08T23:26:04.384097+0000","last_active":"2024-05-08T23:26:05.364615+0000","last_peered":"2024-05-08T23:26:05.364615+0000","last_clean":"2024-05-08T23:26:05.364615+0000","last_became_active":"2024-05-08T23:26:04.383812+0000","last_became_peered":"2024-05-08T23:26:04.383812+0000","last_unstale":"2024-05-08T23:26:05.364615+0000","last_undegraded":"2024-05-08T23:26:05.364615+0000","last_fullsized":"2024-05-08T23:26:05.364615+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:03.890624+0000","last_clean_scrub_stamp":"2024-05-08T23:26:03.890624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:08.068641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T23:26:06.106387+0000","last_change":"2024-05-08T23:26:03.918580+0000","last_active":"2024-05-08T23:26:06.106387+0000","last_peered":"2024-05-08T23:26:06.106387+0000","last_clean":"2024-05-08T23:26:06.106387+0000","last_became_active":"2024-05-08T23:26:03.918308+0000","last_became_peered":"2024-05-08T23:26:03.918308+0000","last_unstale":"2024-05-08T23:26:06.106387+0000","last_undegraded":"2024-05-08T23:26:06.106387+0000","last_fullsized":"2024-05-08T23:26:06.106387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:02.888493+0000","last_clean_scrub_stamp":"2024-05-08T23:26:02.888493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:22.679586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-08T23:26:04.369609+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:26:04.369609+0000","last_peered":"2024-05-08T23:26:04.369609+0000","last_clean":"2024-05-08T23:26:04.369609+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:26:04.369609+0000","last_undegraded":"2024-05-08T23:26:04.369609+0000","last_fullsized":"2024-05-08T23:26:04.369609+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1478656,"data_stored":965284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.626}]},{"osd":1,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75700000000000001,"5min":1.038,"15min":1.038},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.376,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58599999999999997}]},{"osd":2,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.58599999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60899999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.40699999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.29499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.432},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68000000000000005,"5min":0.87,"15min":0.87},"last":0.505}]},{"osd":5,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73499999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.754,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77000000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.41499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":1.115,"15min":1.115},"last":0.621}]},{"osd":3,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41699999999999998}]},{"osd":5,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76500000000000001,"5min":1.052,"15min":1.052},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38100000000000001}]},{"osd":7,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84399999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.40699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":376480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.371,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56100000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.42299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36599999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":1.01,"15min":1.01},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":1.028,"15min":1.028},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":1.105,"15min":1.105},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.54000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.749},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.60699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1470464,"data_stored":964668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48399999999999999,"15min":0.72599999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.52900000000000003,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.252},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":0.71399999999999997},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60799999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.51600000000000001,"15min":0.77800000000000002},"min":{"1min":0.312,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.374},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52900000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54800000000000004,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58099999999999996,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65400000000000003,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.33300000000000002}]},{"osd":4,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69099999999999995,"15min":0.93500000000000005},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.70399999999999996,"15min":0.90800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.91300000000000003},"min":{"1min":0.50700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75700000000000001,"15min":1.0069999999999999},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73499999999999999,"15min":0.97499999999999998},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81799999999999995,"5min":7.734,"15min":7.734},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.96399999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.92200000000000004},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.83299999999999996,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.626}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":375864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.50700000000000001,"5min":0.621,"15min":0.621},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65800000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.317}]},{"osd":1,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44900000000000001,"15min":0.41299999999999998},"min":{"1min":0.33300000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.58499999999999996,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.435,"15min":0.42399999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.621,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56200000000000006,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.49099999999999999,"15min":0.46500000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.43099999999999999}]},{"osd":4,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64500000000000002,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90300000000000002,"5min":1.024,"15min":1.024},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.628,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91600000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83199999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.51100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.89200000000000002,"5min":1.159,"15min":1.159},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69099999999999995,"15min":0.67200000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90000000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.754},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92500000000000004,"5min":1.194,"15min":1.194},"last":0.70099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":374412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.41599999999999998},"min":{"1min":0.308,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":1.46,"15min":1.46},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40400000000000003,"15min":0.41899999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.496,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.45900000000000002}]},{"osd":1,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.55800000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.32900000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60899999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.55400000000000005,"15min":0.84299999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":11.964,"15min":11.964},"last":0.497},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73899999999999999,"5min":1.593,"15min":1.593},"last":0.48799999999999999}]},{"osd":4,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.629,"15min":0.65900000000000003},"min":{"1min":0.48399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78900000000000003,"5min":1.375,"15min":1.375},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85099999999999998,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.747}]},{"osd":5,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.50700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.94799999999999995,"5min":1.536,"15min":1.536},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.68400000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76900000000000002,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65400000000000003,"15min":0.68000000000000005},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89900000000000002,"5min":1.204,"15min":1.204},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69399999999999995,"15min":0.72899999999999998},"min":{"1min":0.57399999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90700000000000003,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.90700000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":0.77000000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86099999999999999,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.86099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":880640,"data_stored":374370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.64200000000000002},"min":{"1min":0.33700000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73399999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59499999999999997,"15min":0.72099999999999997},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.746,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.748,"5min":2.145,"15min":2.145},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60199999999999998,"15min":0.68400000000000005},"min":{"1min":0.36899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68500000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64700000000000002,"15min":0.78500000000000003},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.52600000000000002}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81699999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.74299999999999999,"15min":1.1240000000000001},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.877,"15min":0.877},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64100000000000001}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.504},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63400000000000001,"15min":0.68200000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73899999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77400000000000002,"5min":1.24,"15min":1.24},"last":0.63200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":374390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85199999999999998,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.502},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":0.89400000000000002},"min":{"1min":0.35199999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":8.798,"15min":8.798},"last":0.61299999999999999}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.77500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":4.71,"15min":4.71},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70699999999999996,"15min":0.96799999999999997},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0009999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.79100000000000004},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.82199999999999995,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.93000000000000005},"min":{"1min":0.46999999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78500000000000003,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73899999999999999,"15min":1.0109999999999999},"min":{"1min":0.51100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0489999999999999,"5min":10.427,"15min":10.427},"last":0.56000000000000005}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58599999999999997,"5min":0.874,"15min":0.874},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85299999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.01},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:26:13.784 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:26:13.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T23:26:13.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.965+0000 7f18ce17d700 1 Processor -- start 2024-05-08T23:26:13.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.965+0000 7f18ce17d700 1 -- start start 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.966+0000 7f18ce17d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c80cbf50 0x7f18c8161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.966+0000 7f18ce17d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f18c815e760 0x7f18c806a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.966+0000 7f18ce17d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c80cf970 con 0x7f18c8162240 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.966+0000 7f18ce17d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c80cfc50 con 0x7f18c815e760 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.966+0000 7f18ce17d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80cff30 con 0x7f18c80cbf50 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.966+0000 7f18c7fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f18c815e760 0x7f18c806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.966+0000 7f18c6ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f18c8162240 0x7f18c815e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38922/0 (socket says 172.21.15.5:38922) 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.966+0000 7f18c6ffd700 1 -- 172.21.15.5:0/1167293707 learned_addr learned my addr 172.21.15.5:0/1167293707 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c77fe700 1 --2- 172.21.15.5:0/1167293707 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c80cbf50 0x7f18c8161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c7fff700 1 -- 172.21.15.5:0/1167293707 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c80cbf50 msgr2=0x7f18c8161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c7fff700 1 --2- 172.21.15.5:0/1167293707 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c80cbf50 0x7f18c8161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c7fff700 1 -- 172.21.15.5:0/1167293707 >> v1:172.21.15.5:6789/0 conn(0x7f18c8162240 legacy=0x7f18c815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c7fff700 1 -- 172.21.15.5:0/1167293707 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c81617e0 con 0x7f18c815e760 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c67fc700 1 -- 172.21.15.5:0/1167293707 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402985985 0 0) 0x7f18c80cf970 con 0x7f18c8162240 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c77fe700 1 --2- 172.21.15.5:0/1167293707 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c80cbf50 0x7f18c8161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c7fff700 1 --2- 172.21.15.5:0/1167293707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c815e760 0x7f18c806a960 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009d90 tx=0x7f18c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f4c62da42b4d52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c67fc700 1 -- 172.21.15.5:0/1167293707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000e070 con 0x7f18c815e760 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c67fc700 1 -- 172.21.15.5:0/1167293707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c000bbd0 con 0x7f18c815e760 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.967+0000 7f18c67fc700 1 -- 172.21.15.5:0/1167293707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0012430 con 0x7f18c815e760 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.968+0000 7f18ce17d700 1 -- 172.21.15.5:0/1167293707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c815e760 msgr2=0x7f18c806a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.968+0000 7f18ce17d700 1 --2- 172.21.15.5:0/1167293707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c815e760 0x7f18c806a960 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009d90 tx=0x7f18c000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.968+0000 7f18ce17d700 1 -- 172.21.15.5:0/1167293707 shutdown_connections 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.968+0000 7f18ce17d700 1 --2- 172.21.15.5:0/1167293707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c815e760 0x7f18c806a960 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.968+0000 7f18ce17d700 1 --2- 172.21.15.5:0/1167293707 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c80cbf50 0x7f18c8161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.968+0000 7f18ce17d700 1 -- 172.21.15.5:0/1167293707 >> 172.21.15.5:0/1167293707 conn(0x7f18c8099a50 msgr2=0x7f18c80d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:13.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.968+0000 7f18ce17d700 1 -- 172.21.15.5:0/1167293707 shutdown_connections 2024-05-08T23:26:13.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.968+0000 7f18ce17d700 1 -- 172.21.15.5:0/1167293707 wait complete. 2024-05-08T23:26:13.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.969+0000 7f18ce17d700 1 Processor -- start 2024-05-08T23:26:13.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.970+0000 7f18ce17d700 1 -- start start 2024-05-08T23:26:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18ce17d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c80cbf50 0x7f18c80adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18ce17d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c8162240 0x7f18c8152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18ce17d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c8152aa0 0x7f18c80b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18ce17d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8152f20 con 0x7f18c80cbf50 2024-05-08T23:26:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18ce17d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81f6970 con 0x7f18c8162240 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18ce17d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81f6c50 con 0x7f18c8152aa0 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c80cbf50 0x7f18c80adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c80cbf50 0x7f18c80adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44880/0 (socket says 172.21.15.5:44880) 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 -- 172.21.15.5:0/3532647167 learned_addr learned my addr 172.21.15.5:0/3532647167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c6ffd700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c8162240 0x7f18c8152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c7fff700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c8152aa0 0x7f18c80b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 -- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c8152aa0 msgr2=0x7f18c80b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c8152aa0 0x7f18c80b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 -- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c8162240 msgr2=0x7f18c8152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c8162240 0x7f18c8152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 -- 172.21.15.5:0/3532647167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009a70 con 0x7f18c80cbf50 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.971+0000 7f18c77fe700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c80cbf50 0x7f18c80adeb0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c750 tx=0x7f18b800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.972+0000 7f18affff700 1 -- 172.21.15.5:0/3532647167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8011800 con 0x7f18c80cbf50 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.972+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c81f6f30 con 0x7f18c80cbf50 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.972+0000 7f18affff700 1 -- 172.21.15.5:0/3532647167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8011960 con 0x7f18c80cbf50 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.972+0000 7f18affff700 1 -- 172.21.15.5:0/3532647167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8011cd0 con 0x7f18c80cbf50 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.972+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c81f74f0 con 0x7f18c80cbf50 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.973+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c8074bc0 con 0x7f18c80cbf50 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.974+0000 7f18affff700 1 -- 172.21.15.5:0/3532647167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f18b8023480 con 0x7f18c80cbf50 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.974+0000 7f18affff700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f18b406ac90 0x7f18b406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.974+0000 7f18affff700 1 -- 172.21.15.5:0/3532647167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18b80896b0 con 0x7f18c80cbf50 2024-05-08T23:26:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.974+0000 7f18c6ffd700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f18b406ac90 0x7f18b406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.975+0000 7f18c6ffd700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f18b406ac90 0x7f18b406d150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f18c80b4350 tx=0x7f18bc007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:13.979+0000 7f18affff700 1 -- 172.21.15.5:0/3532647167 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f18b80547f0 con 0x7f18c80cbf50 2024-05-08T23:26:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.284+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f18c80b62b0 con 0x7f18c80cbf50 2024-05-08T23:26:14.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.288+0000 7f18affff700 1 -- 172.21.15.5:0/3532647167 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f18b8057e10 con 0x7f18c80cbf50 2024-05-08T23:26:14.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f18b406ac90 msgr2=0x7f18b406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f18b406ac90 0x7f18b406d150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f18c80b4350 tx=0x7f18bc007e70 comp rx=0 tx=0).stop 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c80cbf50 msgr2=0x7f18c80adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c80cbf50 0x7f18c80adeb0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c750 tx=0x7f18b800ca60 comp rx=0 tx=0).stop 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 shutdown_connections 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f18b406ac90 0x7f18b406d150 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f18c8152aa0 0x7f18c80b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c80cbf50 0x7f18c80adeb0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 --2- 172.21.15.5:0/3532647167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c8162240 0x7f18c8152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 >> 172.21.15.5:0/3532647167 conn(0x7f18c8099a50 msgr2=0x7f18c80af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 shutdown_connections 2024-05-08T23:26:14.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.290+0000 7f18ce17d700 1 -- 172.21.15.5:0/3532647167 wait complete. 2024-05-08T23:26:14.292 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-08T23:26:14.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:14.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.440+0000 7f8dfb42c700 1 Processor -- start 2024-05-08T23:26:14.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.441+0000 7f8dfb42c700 1 -- start start 2024-05-08T23:26:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.441+0000 7f8dfb42c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8df415d8e0 0x7f8df40d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.441+0000 7f8dfb42c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df40d3ce0 0x7f8df41677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8dfb42c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40ce8a0 con 0x7f8df4161440 2024-05-08T23:26:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8dfb42c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df4161c90 con 0x7f8df415d8e0 2024-05-08T23:26:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8dfb42c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4161f70 con 0x7f8df40d3ce0 2024-05-08T23:26:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8df89c7700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8df4161440 0x7f8df40d35d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38934/0 (socket says 172.21.15.5:38934) 2024-05-08T23:26:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8df89c7700 1 -- 172.21.15.5:0/3660467409 learned_addr learned my addr 172.21.15.5:0/3660467409 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8df91c8700 1 --2- 172.21.15.5:0/3660467409 >> v2:172.21.15.49:3300/0 conn(0x7f8df415d8e0 0x7f8df40d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8df99c9700 1 --2- 172.21.15.5:0/3660467409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df40d3ce0 0x7f8df41677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8debfff700 1 -- 172.21.15.5:0/3660467409 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587913279 0 0) 0x7f8df40ce8a0 con 0x7f8df4161440 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8debfff700 1 -- 172.21.15.5:0/3660467409 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de00035b0 con 0x7f8df4161440 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8df91c8700 1 -- 172.21.15.5:0/3660467409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df40d3ce0 msgr2=0x7f8df41677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8df91c8700 1 --2- 172.21.15.5:0/3660467409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df40d3ce0 0x7f8df41677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8df91c8700 1 -- 172.21.15.5:0/3660467409 >> v1:172.21.15.5:6789/0 conn(0x7f8df4161440 legacy=0x7f8df40d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.442+0000 7f8df91c8700 1 -- 172.21.15.5:0/3660467409 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df4162250 con 0x7f8df415d8e0 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8df91c8700 1 --2- 172.21.15.5:0/3660467409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 0x7f8df40d2450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009840 tx=0x7f8df0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace03698b4c45bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8debfff700 1 -- 172.21.15.5:0/3660467409 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0011830 con 0x7f8df415d8e0 2024-05-08T23:26:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8debfff700 1 -- 172.21.15.5:0/3660467409 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df0011990 con 0x7f8df415d8e0 2024-05-08T23:26:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8debfff700 1 -- 172.21.15.5:0/3660467409 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0011d20 con 0x7f8df415d8e0 2024-05-08T23:26:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8dfb42c700 1 -- 172.21.15.5:0/3660467409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 msgr2=0x7f8df40d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/3660467409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 0x7f8df40d2450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009840 tx=0x7f8df0009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8dfb42c700 1 -- 172.21.15.5:0/3660467409 shutdown_connections 2024-05-08T23:26:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/3660467409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df40d3ce0 0x7f8df41677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/3660467409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 0x7f8df40d2450 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.443+0000 7f8dfb42c700 1 -- 172.21.15.5:0/3660467409 >> 172.21.15.5:0/3660467409 conn(0x7f8df409c950 msgr2=0x7f8df408afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.444+0000 7f8dfb42c700 1 -- 172.21.15.5:0/3660467409 shutdown_connections 2024-05-08T23:26:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.444+0000 7f8dfb42c700 1 -- 172.21.15.5:0/3660467409 wait complete. 2024-05-08T23:26:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.445+0000 7f8dfb42c700 1 Processor -- start 2024-05-08T23:26:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.445+0000 7f8dfb42c700 1 -- start start 2024-05-08T23:26:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8dfb42c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8df40d3ce0 0x7f8df40b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8dfb42c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 0x7f8df41f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8dfb42c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df4161440 0x7f8df40cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8dfb42c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40cfc80 con 0x7f8df40d3ce0 2024-05-08T23:26:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8dfb42c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df415cd50 con 0x7f8df415d8e0 2024-05-08T23:26:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8dfb42c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40ccfc0 con 0x7f8df4161440 2024-05-08T23:26:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8df91c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8df40d3ce0 0x7f8df40b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8df91c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8df40d3ce0 0x7f8df40b8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44888/0 (socket says 172.21.15.5:44888) 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.446+0000 7f8df91c8700 1 -- 172.21.15.5:0/1969287385 learned_addr learned my addr 172.21.15.5:0/1969287385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df89c7700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 0x7f8df41f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df99c9700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df4161440 0x7f8df40cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df91c8700 1 -- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df4161440 msgr2=0x7f8df40cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df91c8700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df4161440 0x7f8df40cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df91c8700 1 -- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 msgr2=0x7f8df41f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df91c8700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 0x7f8df41f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df91c8700 1 -- 172.21.15.5:0/1969287385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df00094a0 con 0x7f8df40d3ce0 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df89c7700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 0x7f8df41f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8df91c8700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8df40d3ce0 0x7f8df40b8be0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8df00090b0 tx=0x7f8df0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8de9ffb700 1 -- 172.21.15.5:0/1969287385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00036a0 con 0x7f8df40d3ce0 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8de9ffb700 1 -- 172.21.15.5:0/1969287385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df001f070 con 0x7f8df40d3ce0 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df40cd2a0 con 0x7f8df40d3ce0 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8de9ffb700 1 -- 172.21.15.5:0/1969287385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0003a40 con 0x7f8df40d3ce0 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.447+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df41600f0 con 0x7f8df40d3ce0 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.448+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df4068000 con 0x7f8df40d3ce0 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.449+0000 7f8de9ffb700 1 -- 172.21.15.5:0/1969287385 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8df0003ba0 con 0x7f8df40d3ce0 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.449+0000 7f8de9ffb700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dd806ac90 0x7f8dd806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.449+0000 7f8de9ffb700 1 -- 172.21.15.5:0/1969287385 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8df0021070 con 0x7f8df40d3ce0 2024-05-08T23:26:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.450+0000 7f8df89c7700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dd806ac90 0x7f8dd806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.450+0000 7f8df89c7700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dd806ac90 0x7f8dd806d150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8df40d2ee0 tx=0x7f8de4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.454+0000 7f8de9ffb700 1 -- 172.21.15.5:0/1969287385 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8df0055dc0 con 0x7f8df40d3ce0 2024-05-08T23:26:14.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.724+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8df40ca690 con 0x7f8df40d3ce0 2024-05-08T23:26:14.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.725+0000 7f8de9ffb700 1 -- 172.21.15.5:0/1969287385 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4727 (secure 0 0 0) 0x7f8df0015020 con 0x7f8df40d3ce0 2024-05-08T23:26:14.725 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:14.725 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:07.366240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dd806ac90 msgr2=0x7f8dd806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dd806ac90 0x7f8dd806d150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8df40d2ee0 tx=0x7f8de4009040 comp rx=0 tx=0).stop 2024-05-08T23:26:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8df40d3ce0 msgr2=0x7f8df40b8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8df40d3ce0 0x7f8df40b8be0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8df00090b0 tx=0x7f8df0004750 comp rx=0 tx=0).stop 2024-05-08T23:26:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 shutdown_connections 2024-05-08T23:26:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dd806ac90 0x7f8dd806d150 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8df4161440 0x7f8df40cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8df40d3ce0 0x7f8df40b8be0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 --2- 172.21.15.5:0/1969287385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8df415d8e0 0x7f8df41f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 >> 172.21.15.5:0/1969287385 conn(0x7f8df409c950 msgr2=0x7f8df40d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 shutdown_connections 2024-05-08T23:26:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:14.727+0000 7f8dfb42c700 1 -- 172.21.15.5:0/1969287385 wait complete. 2024-05-08T23:26:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 66 2024-05-08T23:26:15.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:26:04.359632+0000', 'modified': '2024-05-08T23:26:07.366240+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7179': {'gid': 7179, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6834/690633393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 690633393}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 690633393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:26:15.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:26:15.144 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:26:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.290+0000 7fee6c425700 1 Processor -- start 2024-05-08T23:26:15.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.290+0000 7fee6c425700 1 -- start start 2024-05-08T23:26:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.291+0000 7fee6c425700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 0x7fee640b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.291+0000 7fee6c425700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fee640b2f60 0x7fee640d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.291+0000 7fee6c425700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee640ce3f0 con 0x7fee640d2100 2024-05-08T23:26:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.291+0000 7fee6c425700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee640cdc50 con 0x7fee640b2f60 2024-05-08T23:26:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.291+0000 7fee6c425700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640d2590 con 0x7fee64160f00 2024-05-08T23:26:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee6a9c2700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fee640d2100 0x7fee64161610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38938/0 (socket says 172.21.15.5:38938) 2024-05-08T23:26:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee6a9c2700 1 -- 172.21.15.5:0/223645900 learned_addr learned my addr 172.21.15.5:0/223645900 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 0x7fee640b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee699c0700 1 --2- 172.21.15.5:0/223645900 >> v2:172.21.15.49:3300/0 conn(0x7fee640b2f60 0x7fee640d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074894665 0 0) 0x7fee640ce3f0 con 0x7fee640d2100 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee500035b0 con 0x7fee640d2100 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3685086191 0 0) 0x7fee500035b0 con 0x7fee640d2100 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 msgr2=0x7fee640b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 0x7fee640b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 >> v2:172.21.15.49:3300/0 conn(0x7fee640b2f60 msgr2=0x7fee640d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 --2- 172.21.15.5:0/223645900 >> v2:172.21.15.49:3300/0 conn(0x7fee640b2f60 0x7fee640d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640b42c0 con 0x7fee640d2100 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee699c0700 1 --2- 172.21.15.5:0/223645900 >> v2:172.21.15.49:3300/0 conn(0x7fee640b2f60 0x7fee640d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fee5c0019f0 con 0x7fee640d2100 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 >> v1:172.21.15.5:6789/0 conn(0x7fee640d2100 legacy=0x7fee64161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee50003a00 0x7fee50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 0x7fee50004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee50004d90 0x7fee50005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640d2590 con 0x7fee50003a00 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5000ba10 con 0x7fee50004d90 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5000bb50 con 0x7fee64160f00 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.292+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee50003a00 0x7fee50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee699c0700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 0x7fee50004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee6a9c2700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee50004d90 0x7fee50005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee6a1c1700 1 -- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 msgr2=0x7fee50004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 0x7fee50004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee6a1c1700 1 -- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee50004d90 msgr2=0x7fee50005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee50004d90 0x7fee50005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee6a1c1700 1 -- 172.21.15.5:0/223645900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500035b0 con 0x7fee50003a00 2024-05-08T23:26:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee699c0700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee64160f00 0x7fee50004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee50003a00 0x7fee50003e20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fee60009b40 tx=0x7fee50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4128a20ca0e3a90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60005a20 con 0x7fee50003a00 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee60005b80 con 0x7fee50003a00 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.293+0000 7fee691bf700 1 -- 172.21.15.5:0/223645900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000e6c0 con 0x7fee50003a00 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.294+0000 7fee6c425700 1 -- 172.21.15.5:0/223645900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee50003a00 msgr2=0x7fee50003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.294+0000 7fee6c425700 1 --2- 172.21.15.5:0/223645900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee50003a00 0x7fee50003e20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fee60009b40 tx=0x7fee50005880 comp rx=0 tx=0).stop 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.294+0000 7fee6a9c2700 1 -- 172.21.15.5:0/223645900 reap_dead start 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.294+0000 7fee6c425700 1 -- 172.21.15.5:0/223645900 shutdown_connections 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.294+0000 7fee6c425700 1 -- 172.21.15.5:0/223645900 >> 172.21.15.5:0/223645900 conn(0x7fee6408afe0 msgr2=0x7fee640d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.294+0000 7fee6c425700 1 -- 172.21.15.5:0/223645900 shutdown_connections 2024-05-08T23:26:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.294+0000 7fee6c425700 1 -- 172.21.15.5:0/223645900 wait complete. 2024-05-08T23:26:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.295+0000 7fee6c425700 1 Processor -- start 2024-05-08T23:26:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.295+0000 7fee6c425700 1 -- start start 2024-05-08T23:26:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee6c425700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee50003a00 0x7fee64149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee6c425700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee641f74c0 0x7fee640cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee6c425700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee640cd360 0x7fee640c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee6c425700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c7f50 con 0x7fee641f74c0 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee6c425700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee64149c80 con 0x7fee640cd360 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee6c425700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640b8a90 con 0x7fee50003a00 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee699c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee641f74c0 0x7fee640cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee699c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee641f74c0 0x7fee640cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44906/0 (socket says 172.21.15.5:44906) 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee699c0700 1 -- 172.21.15.5:0/4279974324 learned_addr learned my addr 172.21.15.5:0/4279974324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee50003a00 0x7fee64149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee6a9c2700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee640cd360 0x7fee640c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee699c0700 1 -- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee50003a00 msgr2=0x7fee64149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee699c0700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee50003a00 0x7fee64149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee699c0700 1 -- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee640cd360 msgr2=0x7fee640c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee699c0700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee640cd360 0x7fee640c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.296+0000 7fee699c0700 1 -- 172.21.15.5:0/4279974324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640b8d70 con 0x7fee641f74c0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.297+0000 7fee699c0700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee641f74c0 0x7fee640cce20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fee54011940 tx=0x7fee54011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.297+0000 7fee5affd700 1 -- 172.21.15.5:0/4279974324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5400f8f0 con 0x7fee641f74c0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.297+0000 7fee5affd700 1 -- 172.21.15.5:0/4279974324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5400fa50 con 0x7fee641f74c0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.297+0000 7fee5affd700 1 -- 172.21.15.5:0/4279974324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5400fdc0 con 0x7fee641f74c0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.297+0000 7fee6c425700 1 -- 172.21.15.5:0/4279974324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee640b9050 con 0x7fee641f74c0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.297+0000 7fee6c425700 1 -- 172.21.15.5:0/4279974324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee640c8920 con 0x7fee641f74c0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.299+0000 7fee5affd700 1 -- 172.21.15.5:0/4279974324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fee54015460 con 0x7fee641f74c0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.299+0000 7fee5affd700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee4806ac90 0x7fee4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.299+0000 7fee5affd700 1 -- 172.21.15.5:0/4279974324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee54089b80 con 0x7fee641f74c0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.300+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee4806ac90 0x7fee4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.300+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee4806ac90 0x7fee4806d150 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee60000c00 tx=0x7fee60008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.301+0000 7fee785dfb80 1 Processor -- start 2024-05-08T23:26:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.301+0000 7fee785dfb80 1 -- start start 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 0x56019e9110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e916050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e940200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee785dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e86c100 con 0x56019e913320 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee785dfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e869ac0 con 0x56019e93fd80 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee785dfb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019e8653b0 con 0x56019e938750 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee699c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e916050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee699c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e916050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44912/0 (socket says 172.21.15.5:44912) 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee699c0700 1 -- 172.21.15.5:0/2347028067 learned_addr learned my addr 172.21.15.5:0/2347028067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 0x56019e9110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.302+0000 7fee6a9c2700 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e940200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee699c0700 1 -- 172.21.15.5:0/2347028067 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 msgr2=0x56019e9110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee699c0700 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 0x56019e9110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee699c0700 1 -- 172.21.15.5:0/2347028067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 msgr2=0x56019e940200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee699c0700 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e940200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee699c0700 1 -- 172.21.15.5:0/2347028067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019e6e6360 con 0x56019e913320 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee699c0700 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e916050 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fee540608e0 tx=0x7fee54058630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b06ebcb5aaaaf4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee47fff700 1 -- 172.21.15.5:0/2347028067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5405c850 con 0x56019e913320 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee47fff700 1 -- 172.21.15.5:0/2347028067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee54060e10 con 0x56019e913320 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.303+0000 7fee47fff700 1 -- 172.21.15.5:0/2347028067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54055130 con 0x56019e913320 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 -- 172.21.15.5:0/2347028067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 msgr2=0x56019e916050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e916050 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fee540608e0 tx=0x7fee54058630 comp rx=0 tx=0).stop 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 -- 172.21.15.5:0/2347028067 shutdown_connections 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e940200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 0x56019e9110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 --2- 172.21.15.5:0/2347028067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e916050 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 -- 172.21.15.5:0/2347028067 >> 172.21.15.5:0/2347028067 conn(0x56019e8731c0 msgr2=0x56019e95ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 -- 172.21.15.5:0/2347028067 shutdown_connections 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.304+0000 7fee785dfb80 1 -- 172.21.15.5:0/2347028067 wait complete. 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.305+0000 7fee785dfb80 1 -- start start 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.305+0000 7fee785dfb80 1 Processor -- start 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e913320 0x56019e937570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e940f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e93fd80 0x56019e95b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee785dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e801510 con 0x56019e938750 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee785dfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e8011b0 con 0x56019e913320 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee785dfb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019e6e8650 con 0x56019e93fd80 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee785dfb80 10 client.0 _ll_register_callbacks cb 0x56019e80aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee699c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e940f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee699c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e940f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44916/0 (socket says 172.21.15.5:44916) 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.306+0000 7fee699c0700 1 -- 172.21.15.5:0/1024529615 learned_addr learned my addr 172.21.15.5:0/1024529615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e913320 0x56019e937570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee6a9c2700 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e93fd80 0x56019e95b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee699c0700 1 -- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e93fd80 msgr2=0x56019e95b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee699c0700 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e93fd80 0x56019e95b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee699c0700 1 -- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e913320 msgr2=0x56019e937570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee699c0700 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e913320 0x56019e937570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee699c0700 1 -- 172.21.15.5:0/1024529615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee640c8920 con 0x56019e938750 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e913320 0x56019e937570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee699c0700 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e940f90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fee540046e0 tx=0x7fee54000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f49b074e0b0676 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee457fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee457fa700 1 -- 172.21.15.5:0/1024529615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5401b500 con 0x56019e938750 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee457fa700 1 -- 172.21.15.5:0/1024529615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee5405c850 con 0x56019e938750 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee785dfb80 1 -- 172.21.15.5:0/1024529615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56019e7be220 con 0x56019e938750 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee457fa700 1 -- 172.21.15.5:0/1024529615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee54077300 con 0x56019e938750 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee785dfb80 10 client.7347 fetch_fsmap learned FSMap version 66 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee785dfb80 1 -- 172.21.15.5:0/1024529615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x56019e692b10 con 0x56019e938750 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.307+0000 7fee457fa700 1 -- 172.21.15.5:0/1024529615 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5407b020 con 0x56019e938750 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee457fa700 1 -- 172.21.15.5:0/1024529615 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fee54058c80 con 0x56019e938750 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee785dfb80 10 client.7347 fetch_fsmap finished waiting for FSMap version 66 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee785dfb80 10 client.7347 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee785dfb80 20 client.7347 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x56019e96a0c0 0x56019e96a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee785dfb80 4 client.7347 mds_command: new command op to 7179 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee785dfb80 1 -- 172.21.15.5:0/1024529615 --> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56019e8e4790 con 0x56019e96a0c0 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x56019e96a0c0 0x56019e96a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee457fa700 10 client.7347 ms_handle_connect on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.308+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x56019e96a0c0 0x56019e96a4a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a11f73bfabd9bb server_cookie=ac8a183a8a53ae95 in_seq=0 out_seq=0 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee457fa700 1 -- 172.21.15.5:0/1024529615 <== mds.0 v2:172.21.15.5:6832/690633393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56019e8e4790 con 0x56019e96a0c0 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee457fa700 10 client.7347 handle_command_reply: tid=0 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 client.7347 shutdown 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 -- 172.21.15.5:0/1024529615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 msgr2=0x56019e940f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e940f90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fee540046e0 tx=0x7fee54000bc0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 -- 172.21.15.5:0/1024529615 shutdown_connections 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x56019e96a0c0 0x56019e96a4a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e93fd80 0x56019e95b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e913320 0x56019e937570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee457fa700 0 client.7347 ms_handle_reset on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1024529615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e940f90 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 -- 172.21.15.5:0/1024529615 >> 172.21.15.5:0/1024529615 conn(0x56019e8731c0 msgr2=0x56019e90f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 -- 172.21.15.5:0/1024529615 shutdown_connections 2024-05-08T23:26:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 1 -- 172.21.15.5:0/1024529615 wait complete. 2024-05-08T23:26:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.309+0000 7fee785dfb80 20 client.7347 trim_cache size 0 max 16384 2024-05-08T23:26:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.329+0000 7fee785dfb80 1 Processor -- start 2024-05-08T23:26:15.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.329+0000 7fee785dfb80 1 -- start start 2024-05-08T23:26:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.329+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e937ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.329+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 0x56019e97c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.329+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e969480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.329+0000 7fee785dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e916ca0 con 0x56019e913320 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.329+0000 7fee785dfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e8f20a0 con 0x56019e93fd80 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.329+0000 7fee785dfb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019e910190 con 0x56019e938750 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e937ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e937ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44930/0 (socket says 172.21.15.5:44930) 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 -- 172.21.15.5:0/658840295 learned_addr learned my addr 172.21.15.5:0/658840295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 -- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 msgr2=0x56019e97c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee699c0700 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 0x56019e97c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a9c2700 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e969480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 0x56019e97c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 -- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 msgr2=0x56019e969480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e969480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 -- 172.21.15.5:0/658840295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019e82b510 con 0x56019e913320 2024-05-08T23:26:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a9c2700 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e969480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e937ff0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fee500046e0 tx=0x7fee60014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d31ef565de9c849 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee58da4700 1 -- 172.21.15.5:0/658840295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000e6c0 con 0x56019e913320 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee58da4700 1 -- 172.21.15.5:0/658840295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee60021040 con 0x56019e913320 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.330+0000 7fee58da4700 1 -- 172.21.15.5:0/658840295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000ea20 con 0x56019e913320 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 -- 172.21.15.5:0/658840295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 msgr2=0x56019e937ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e937ff0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fee500046e0 tx=0x7fee60014000 comp rx=0 tx=0).stop 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 -- 172.21.15.5:0/658840295 shutdown_connections 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e969480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e913320 0x56019e937ff0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 --2- 172.21.15.5:0/658840295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e938750 0x56019e97c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 -- 172.21.15.5:0/658840295 >> 172.21.15.5:0/658840295 conn(0x56019e8731c0 msgr2=0x56019e90c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 -- 172.21.15.5:0/658840295 shutdown_connections 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.331+0000 7fee785dfb80 1 -- 172.21.15.5:0/658840295 wait complete. 2024-05-08T23:26:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.332+0000 7fee785dfb80 1 -- start start 2024-05-08T23:26:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.332+0000 7fee785dfb80 1 Processor -- start 2024-05-08T23:26:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e913320 0x56019e9c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e90ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee785dfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e90f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee785dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e90fc20 con 0x56019e938750 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee785dfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e9c6440 con 0x56019e93fd80 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee785dfb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019e9735e0 con 0x56019e913320 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee785dfb80 10 client.0 _ll_register_callbacks cb 0x56019e80aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e90ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e90ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44942/0 (socket says 172.21.15.5:44942) 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 -- 172.21.15.5:0/1193306319 learned_addr learned my addr 172.21.15.5:0/1193306319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e913320 0x56019e9c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee6a9c2700 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e90f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 -- 172.21.15.5:0/1193306319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e913320 msgr2=0x56019e9c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e913320 0x56019e9c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 -- 172.21.15.5:0/1193306319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 msgr2=0x56019e90f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e90f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 -- 172.21.15.5:0/1193306319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6002f040 con 0x56019e938750 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee699c0700 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e90ec90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fee54058580 tx=0x7fee5400ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba62606837c82415 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee457fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee457fa700 1 -- 172.21.15.5:0/1193306319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5401b500 con 0x56019e938750 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee457fa700 1 -- 172.21.15.5:0/1193306319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee5407c080 con 0x56019e938750 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.333+0000 7fee785dfb80 1 -- 172.21.15.5:0/1193306319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee54060e10 con 0x56019e938750 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee457fa700 1 -- 172.21.15.5:0/1193306319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee54060980 con 0x56019e938750 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee785dfb80 10 client.7353 fetch_fsmap learned FSMap version 66 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee785dfb80 1 -- 172.21.15.5:0/1193306319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x56019e6eac80 con 0x56019e938750 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee457fa700 1 -- 172.21.15.5:0/1193306319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee54060e10 con 0x56019e938750 2024-05-08T23:26:15.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee457fa700 1 -- 172.21.15.5:0/1193306319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fee54057f10 con 0x56019e938750 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee785dfb80 10 client.7353 fetch_fsmap finished waiting for FSMap version 66 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee785dfb80 10 client.7353 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee785dfb80 20 client.7353 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x56019e913700 0x56019e913ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee785dfb80 4 client.7353 mds_command: new command op to 7179 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee785dfb80 1 -- 172.21.15.5:0/1193306319 --> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56019e85de70 con 0x56019e913700 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x56019e913700 0x56019e913ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee457fa700 10 client.7353 ms_handle_connect on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.334+0000 7fee6a1c1700 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x56019e913700 0x56019e913ae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bebb1123dbdb313d server_cookie=af57de53a62dbfaf in_seq=0 out_seq=0 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee457fa700 1 -- 172.21.15.5:0/1193306319 <== mds.0 v2:172.21.15.5:6832/690633393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56019e85de70 con 0x56019e913700 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee457fa700 10 client.7353 handle_command_reply: tid=0 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 client.7353 shutdown 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 -- 172.21.15.5:0/1193306319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 msgr2=0x56019e90ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e90ec90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fee54058580 tx=0x7fee5400ffa0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 -- 172.21.15.5:0/1193306319 shutdown_connections 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x56019e913700 0x56019e913ae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56019e93fd80 0x56019e90f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee457fa700 0 client.7353 ms_handle_reset on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56019e913320 0x56019e9c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 --2- 172.21.15.5:0/1193306319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019e938750 0x56019e90ec90 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 -- 172.21.15.5:0/1193306319 >> 172.21.15.5:0/1193306319 conn(0x56019e8731c0 msgr2=0x56019e95b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 -- 172.21.15.5:0/1193306319 shutdown_connections 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 1 -- 172.21.15.5:0/1193306319 wait complete. 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.335+0000 7fee785dfb80 20 client.7353 trim_cache size 0 max 16384 2024-05-08T23:26:15.341 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7179, 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 10.974670511999999, 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 66, 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 72, 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 72, 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 16.016547752000001 2024-05-08T23:26:15.342 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 -- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee4806ac90 msgr2=0x7fee4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee4806ac90 0x7fee4806d150 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee60000c00 tx=0x7fee60008030 comp rx=0 tx=0).stop 2024-05-08T23:26:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 -- 172.21.15.5:0/4279974324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee641f74c0 msgr2=0x7fee640cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee641f74c0 0x7fee640cce20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fee54011940 tx=0x7fee54011c50 comp rx=0 tx=0).stop 2024-05-08T23:26:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 -- 172.21.15.5:0/4279974324 shutdown_connections 2024-05-08T23:26:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fee4806ac90 0x7fee4806d150 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee640cd360 0x7fee640c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fee50003a00 0x7fee64149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 --2- 172.21.15.5:0/4279974324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee641f74c0 0x7fee640cce20 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 -- 172.21.15.5:0/4279974324 >> 172.21.15.5:0/4279974324 conn(0x7fee6408afe0 msgr2=0x7fee640d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 -- 172.21.15.5:0/4279974324 shutdown_connections 2024-05-08T23:26:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.336+0000 7fee6c425700 1 -- 172.21.15.5:0/4279974324 wait complete. 2024-05-08T23:26:15.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:26:15.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.486+0000 7fc49e645700 1 Processor -- start 2024-05-08T23:26:15.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.486+0000 7fc49e645700 1 -- start start 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49e645700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc4980cf840 0x7fc4980af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49e645700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980d0eb0 0x7fc498161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49e645700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4980cd980 con 0x7fc49806a960 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49e645700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4980ccdb0 con 0x7fc4980cf840 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49e645700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980cd050 con 0x7fc4980d0eb0 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc4977fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc49806a960 0x7fc49815d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38940/0 (socket says 172.21.15.5:38940) 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc4977fe700 1 -- 172.21.15.5:0/4015573534 learned_addr learned my addr 172.21.15.5:0/4015573534 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49cbe2700 1 --2- 172.21.15.5:0/4015573534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980d0eb0 0x7fc498161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc497fff700 1 --2- 172.21.15.5:0/4015573534 >> v2:172.21.15.49:3300/0 conn(0x7fc4980cf840 0x7fc4980af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc496ffd700 1 -- 172.21.15.5:0/4015573534 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694516155 0 0) 0x7fc4980cd980 con 0x7fc49806a960 2024-05-08T23:26:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc496ffd700 1 -- 172.21.15.5:0/4015573534 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4800035b0 con 0x7fc49806a960 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc496ffd700 1 -- 172.21.15.5:0/4015573534 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3278602438 0 0) 0x7fc4800035b0 con 0x7fc49806a960 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49cbe2700 1 -- 172.21.15.5:0/4015573534 >> v2:172.21.15.49:3300/0 conn(0x7fc4980cf840 msgr2=0x7fc4980af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49cbe2700 1 --2- 172.21.15.5:0/4015573534 >> v2:172.21.15.49:3300/0 conn(0x7fc4980cf840 0x7fc4980af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49cbe2700 1 -- 172.21.15.5:0/4015573534 >> v1:172.21.15.5:6789/0 conn(0x7fc49806a960 legacy=0x7fc49815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.487+0000 7fc49cbe2700 1 -- 172.21.15.5:0/4015573534 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980cd330 con 0x7fc4980d0eb0 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.488+0000 7fc497fff700 1 --2- 172.21.15.5:0/4015573534 >> v2:172.21.15.49:3300/0 conn(0x7fc4980cf840 0x7fc4980af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.488+0000 7fc49cbe2700 1 --2- 172.21.15.5:0/4015573534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980d0eb0 0x7fc498161620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc490009d90 tx=0x7fc49000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63b879003979dec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.488+0000 7fc496ffd700 1 -- 172.21.15.5:0/4015573534 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49000e070 con 0x7fc4980d0eb0 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.488+0000 7fc496ffd700 1 -- 172.21.15.5:0/4015573534 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc490003a20 con 0x7fc4980d0eb0 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.488+0000 7fc496ffd700 1 -- 172.21.15.5:0/4015573534 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c001670 con 0x7fc4980d0eb0 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.489+0000 7fc49e645700 1 -- 172.21.15.5:0/4015573534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980d0eb0 msgr2=0x7fc498161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.489+0000 7fc49e645700 1 --2- 172.21.15.5:0/4015573534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980d0eb0 0x7fc498161620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc490009d90 tx=0x7fc49000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.489+0000 7fc49e645700 1 -- 172.21.15.5:0/4015573534 shutdown_connections 2024-05-08T23:26:15.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.489+0000 7fc49e645700 1 --2- 172.21.15.5:0/4015573534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980d0eb0 0x7fc498161620 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.489+0000 7fc49e645700 1 --2- 172.21.15.5:0/4015573534 >> v2:172.21.15.49:3300/0 conn(0x7fc4980cf840 0x7fc4980af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.489+0000 7fc49e645700 1 -- 172.21.15.5:0/4015573534 >> 172.21.15.5:0/4015573534 conn(0x7fc4980d0320 msgr2=0x7fc4980d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:15.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.489+0000 7fc49e645700 1 -- 172.21.15.5:0/4015573534 shutdown_connections 2024-05-08T23:26:15.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.489+0000 7fc49e645700 1 -- 172.21.15.5:0/4015573534 wait complete. 2024-05-08T23:26:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.490+0000 7fc49e645700 1 Processor -- start 2024-05-08T23:26:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.490+0000 7fc49e645700 1 -- start start 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49e645700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc49806a960 0x7fc498160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49e645700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980cf840 0x7fc49815e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49e645700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4980d0eb0 0x7fc498150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49e645700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49815f350 con 0x7fc4980d0eb0 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49e645700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980c5140 con 0x7fc49806a960 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49e645700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980c5420 con 0x7fc4980cf840 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49cbe2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4980d0eb0 0x7fc498150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49cbe2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4980d0eb0 0x7fc498150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44954/0 (socket says 172.21.15.5:44954) 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49cbe2700 1 -- 172.21.15.5:0/1299723058 learned_addr learned my addr 172.21.15.5:0/1299723058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc497fff700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc49806a960 0x7fc498160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc4977fe700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980cf840 0x7fc49815e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49cbe2700 1 -- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980cf840 msgr2=0x7fc49815e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49cbe2700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980cf840 0x7fc49815e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49cbe2700 1 -- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc49806a960 msgr2=0x7fc498160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.491+0000 7fc49cbe2700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc49806a960 0x7fc498160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.492+0000 7fc49cbe2700 1 -- 172.21.15.5:0/1299723058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4880094a0 con 0x7fc4980d0eb0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.492+0000 7fc4977fe700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980cf840 0x7fc49815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.492+0000 7fc49cbe2700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4980d0eb0 0x7fc498150400 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc490004000 tx=0x7fc490004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.492+0000 7fc494ff9700 1 -- 172.21.15.5:0/1299723058 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900045f0 con 0x7fc4980d0eb0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.492+0000 7fc494ff9700 1 -- 172.21.15.5:0/1299723058 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc490004750 con 0x7fc4980d0eb0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.492+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc490009a70 con 0x7fc4980d0eb0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.492+0000 7fc494ff9700 1 -- 172.21.15.5:0/1299723058 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490009450 con 0x7fc4980d0eb0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.492+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4981f3710 con 0x7fc4980d0eb0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.493+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4980b2c80 con 0x7fc4980d0eb0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.493+0000 7fc494ff9700 1 -- 172.21.15.5:0/1299723058 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc490012450 con 0x7fc4980d0eb0 2024-05-08T23:26:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.494+0000 7fc494ff9700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc48406ac90 0x7fc48406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.494+0000 7fc494ff9700 1 -- 172.21.15.5:0/1299723058 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc49008abb0 con 0x7fc4980d0eb0 2024-05-08T23:26:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.494+0000 7fc497fff700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc48406ac90 0x7fc48406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.495+0000 7fc497fff700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc48406ac90 0x7fc48406d150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc4980af8b0 tx=0x7fc488008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:15.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.499+0000 7fc494ff9700 1 -- 172.21.15.5:0/1299723058 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc4900598b0 con 0x7fc4980d0eb0 2024-05-08T23:26:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.770+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc498068000 con 0x7fc4980d0eb0 2024-05-08T23:26:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.771+0000 7fc494ff9700 1 -- 172.21.15.5:0/1299723058 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc490029060 con 0x7fc4980d0eb0 2024-05-08T23:26:15.771 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc48406ac90 msgr2=0x7fc48406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc48406ac90 0x7fc48406d150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc4980af8b0 tx=0x7fc488008040 comp rx=0 tx=0).stop 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4980d0eb0 msgr2=0x7fc498150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4980d0eb0 0x7fc498150400 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc490004000 tx=0x7fc490004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 shutdown_connections 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc48406ac90 0x7fc48406d150 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4980d0eb0 0x7fc498150400 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc49806a960 0x7fc498160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 --2- 172.21.15.5:0/1299723058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4980cf840 0x7fc49815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 >> 172.21.15.5:0/1299723058 conn(0x7fc4980d0320 msgr2=0x7fc49806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 shutdown_connections 2024-05-08T23:26:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:15.773+0000 7fc49e645700 1 -- 172.21.15.5:0/1299723058 wait complete. 2024-05-08T23:26:15.787 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T23:26:15.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:26:15.787 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-08T23:26:15.787 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:26:15.787 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:26:15.788 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T23:26:15.788 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T23:26:15.788 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:26:15.788 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:26:15.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:15.788 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-08T23:26:15.810 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:26:15.810 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:26:15.811 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41941sec preferred_lft 41941sec 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::1c2e:b3ff:fe6b:56f/64 scope link 2024-05-08T23:26:15.812 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::84a4:2aff:fe7c:ce1a/64 scope link 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run.smithi005.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether e2:3e:2c:7a:5d:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::e03e:2cff:fe7a:5db1/64 scope link 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:26:15.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:15.813 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:26:15.881 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-08T23:26:15.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:15.883 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-08T23:26:15.947 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-08T23:26:15.947 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:26:15.947 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:26:15.947 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:26:15.947 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:26:15.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:15.948 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:26:15.948 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:26:15.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:26:15.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:26:15.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:26:15.948 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:26:16.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:16.073 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:26:16.093 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:26:16.093 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:26:16.094 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:26:16.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:16.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-08T23:26:16.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:26:16.169 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:26:16.231 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:26:16.231 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:26:16.231 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:26:16.293 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T23:26:16.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:16.293 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:26:16.391 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:26:16.391 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:26:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:26:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:26:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:26:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:26:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:26:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:26:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:26:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:26:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:26:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:26:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:26:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:26:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:26:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:26:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:26:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:26:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:26:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:26:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:26:16.435 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:26:16.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:16.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:26:16.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:16.467 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:26:16.563 INFO:teuthology.orchestra.run.smithi005.stdout:client6931 2024-05-08T23:26:16.563 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:26:16.564 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:16.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T23:26:16.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.772+0000 7f491d6ca700 1 Processor -- start 2024-05-08T23:26:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.772+0000 7f491d6ca700 1 -- start start 2024-05-08T23:26:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.773+0000 7f491d6ca700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f491806ef30 0x7f491806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.773+0000 7f491d6ca700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49180cef60 con 0x7f491815e310 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.773+0000 7f491d6ca700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491815eee0 con 0x7f491815dcc0 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.773+0000 7f491d6ca700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180b2de0 con 0x7f491806ef30 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f49177fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f491815e310 0x7f49180adb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38954/0 (socket says 172.21.15.5:38954) 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f49177fe700 1 -- 172.21.15.5:0/69054802 learned_addr learned my addr 172.21.15.5:0/69054802 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f49167fc700 1 --2- 172.21.15.5:0/69054802 >> v2:172.21.15.49:3300/0 conn(0x7f491806ef30 0x7f491806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f4915ffb700 1 -- 172.21.15.5:0/69054802 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970976969 0 0) 0x7f49180cef60 con 0x7f491815e310 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f4915ffb700 1 -- 172.21.15.5:0/69054802 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49040035b0 con 0x7f491815e310 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f4915ffb700 1 -- 172.21.15.5:0/69054802 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760111753 0 0) 0x7f491815eee0 con 0x7f491815dcc0 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f4915ffb700 1 -- 172.21.15.5:0/69054802 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49180cef60 con 0x7f491815dcc0 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f49167fc700 1 -- 172.21.15.5:0/69054802 >> v1:172.21.15.49:6789/0 conn(0x7f491815dcc0 legacy=0x7f49180b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f49167fc700 1 -- 172.21.15.5:0/69054802 >> v1:172.21.15.5:6789/0 conn(0x7f491815e310 legacy=0x7f49180adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f4915ffb700 1 -- 172.21.15.5:0/69054802 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3363197821 0 0) 0x7f49040035b0 con 0x7f491815e310 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f49167fc700 1 -- 172.21.15.5:0/69054802 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180ce980 con 0x7f491806ef30 2024-05-08T23:26:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.774+0000 7f49167fc700 1 --2- 172.21.15.5:0/69054802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 0x7f491806fa10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4900009840 tx=0x7f4900009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6968e7bd26ea6800 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f4915ffb700 1 -- 172.21.15.5:0/69054802 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900011830 con 0x7f491806ef30 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f4915ffb700 1 -- 172.21.15.5:0/69054802 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900011990 con 0x7f491806ef30 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f4915ffb700 1 -- 172.21.15.5:0/69054802 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900011d20 con 0x7f491806ef30 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f491d6ca700 1 -- 172.21.15.5:0/69054802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 msgr2=0x7f491806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f491d6ca700 1 --2- 172.21.15.5:0/69054802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 0x7f491806fa10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4900009840 tx=0x7f4900009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f491d6ca700 1 -- 172.21.15.5:0/69054802 shutdown_connections 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f491d6ca700 1 --2- 172.21.15.5:0/69054802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 0x7f491806fa10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f491d6ca700 1 -- 172.21.15.5:0/69054802 >> 172.21.15.5:0/69054802 conn(0x7f4918099a50 msgr2=0x7f49180b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f491d6ca700 1 -- 172.21.15.5:0/69054802 shutdown_connections 2024-05-08T23:26:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.775+0000 7f491d6ca700 1 -- 172.21.15.5:0/69054802 wait complete. 2024-05-08T23:26:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.777+0000 7f491d6ca700 1 Processor -- start 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.777+0000 7f491d6ca700 1 -- start start 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f491d6ca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 0x7f49180b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f491d6ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491815dcc0 0x7f49180adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f491d6ca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f491815e310 0x7f49180c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f491d6ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180c77b0 con 0x7f491815dcc0 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f491d6ca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180c7a90 con 0x7f491806ef30 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f491d6ca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49180c7d70 con 0x7f491815e310 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491815dcc0 0x7f49180adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491815dcc0 0x7f49180adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44956/0 (socket says 172.21.15.5:44956) 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 -- 172.21.15.5:0/2965014128 learned_addr learned my addr 172.21.15.5:0/2965014128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f4916ffd700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 0x7f49180b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49177fe700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f491815e310 0x7f49180c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 -- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f491815e310 msgr2=0x7f49180c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f491815e310 0x7f49180c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 -- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 msgr2=0x7f49180b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 0x7f49180b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 -- 172.21.15.5:0/2965014128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000094a0 con 0x7f491815dcc0 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f4916ffd700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 0x7f49180b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49177fe700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f491815e310 0x7f49180c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.778+0000 7f49167fc700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491815dcc0 0x7f49180adc70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f49000090b0 tx=0x7f4900004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.779+0000 7f490f7fe700 1 -- 172.21.15.5:0/2965014128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490001a860 con 0x7f491815dcc0 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.779+0000 7f490f7fe700 1 -- 172.21.15.5:0/2965014128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490001a9c0 con 0x7f491815dcc0 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.779+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49180c8050 con 0x7f491815dcc0 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.779+0000 7f490f7fe700 1 -- 172.21.15.5:0/2965014128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490001ad30 con 0x7f491815dcc0 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.779+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49180c73d0 con 0x7f491815dcc0 2024-05-08T23:26:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.780+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4918074bc0 con 0x7f491815dcc0 2024-05-08T23:26:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.780+0000 7f490f7fe700 1 -- 172.21.15.5:0/2965014128 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4900022070 con 0x7f491815dcc0 2024-05-08T23:26:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.781+0000 7f490f7fe700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f48f806ac90 0x7f48f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.781+0000 7f490f7fe700 1 -- 172.21.15.5:0/2965014128 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4900089fe0 con 0x7f491815dcc0 2024-05-08T23:26:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.781+0000 7f4916ffd700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f48f806ac90 0x7f48f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.782+0000 7f4916ffd700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f48f806ac90 0x7f48f806d150 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f49100011e0 tx=0x7f4910009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:16.785+0000 7f490f7fe700 1 -- 172.21.15.5:0/2965014128 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4900055360 con 0x7f491815dcc0 2024-05-08T23:26:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.052+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f49180cd980 con 0x7f491815dcc0 2024-05-08T23:26:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.398+0000 7f490f7fe700 1 -- 172.21.15.5:0/2965014128 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v67) v1 ==== 105+0+0 (secure 0 0 0) 0x7f49000150b0 con 0x7f491815dcc0 2024-05-08T23:26:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f48f806ac90 msgr2=0x7f48f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f48f806ac90 0x7f48f806d150 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f49100011e0 tx=0x7f4910009040 comp rx=0 tx=0).stop 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491815dcc0 msgr2=0x7f49180adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491815dcc0 0x7f49180adc70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f49000090b0 tx=0x7f4900004750 comp rx=0 tx=0).stop 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 shutdown_connections 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f48f806ac90 0x7f48f806d150 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f491815e310 0x7f49180c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f491806ef30 0x7f49180b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 --2- 172.21.15.5:0/2965014128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491815dcc0 0x7f49180adc70 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 >> 172.21.15.5:0/2965014128 conn(0x7f4918099a50 msgr2=0x7f49180cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 shutdown_connections 2024-05-08T23:26:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.402+0000 7f491d6ca700 1 -- 172.21.15.5:0/2965014128 wait complete. 2024-05-08T23:26:17.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:17.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.561+0000 7fae43458700 1 Processor -- start 2024-05-08T23:26:17.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.561+0000 7fae43458700 1 -- start start 2024-05-08T23:26:17.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.562+0000 7fae43458700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15d7a0 0x7fae3c161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:17.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.562+0000 7fae43458700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae3c15db80 0x7fae3c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:17.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.562+0000 7fae43458700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c162450 con 0x7fae3c0d3f40 2024-05-08T23:26:17.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.562+0000 7fae43458700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c0cee00 con 0x7fae3c15db80 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.562+0000 7fae43458700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0b27e0 con 0x7fae3c15d7a0 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.562+0000 7fae419f5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae3c15db80 0x7fae3c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae419f5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae3c15db80 0x7fae3c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50460/0 (socket says 172.21.15.5:50460) 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae419f5700 1 -- 172.21.15.5:0/3998757140 learned_addr learned my addr 172.21.15.5:0/3998757140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae411f4700 1 --2- 172.21.15.5:0/3998757140 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15d7a0 0x7fae3c161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae419f5700 1 -- 172.21.15.5:0/3998757140 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15d7a0 msgr2=0x7fae3c161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae419f5700 1 --2- 172.21.15.5:0/3998757140 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15d7a0 0x7fae3c161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae419f5700 1 -- 172.21.15.5:0/3998757140 >> v1:172.21.15.49:6789/0 conn(0x7fae3c0d3f40 legacy=0x7fae3c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae419f5700 1 -- 172.21.15.5:0/3998757140 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0d4400 con 0x7fae3c15db80 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae419f5700 1 --2- 172.21.15.5:0/3998757140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15db80 0x7fae3c0cf290 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fae34009e50 tx=0x7fae3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd81fea30ce4b977 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae33fff700 1 -- 172.21.15.5:0/3998757140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400e070 con 0x7fae3c15db80 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae33fff700 1 -- 172.21.15.5:0/3998757140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34003a20 con 0x7fae3c15db80 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.563+0000 7fae33fff700 1 -- 172.21.15.5:0/3998757140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34003dd0 con 0x7fae3c15db80 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.564+0000 7fae43458700 1 -- 172.21.15.5:0/3998757140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15db80 msgr2=0x7fae3c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.564+0000 7fae43458700 1 --2- 172.21.15.5:0/3998757140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15db80 0x7fae3c0cf290 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fae34009e50 tx=0x7fae3400b330 comp rx=0 tx=0).stop 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.564+0000 7fae43458700 1 -- 172.21.15.5:0/3998757140 shutdown_connections 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.564+0000 7fae43458700 1 --2- 172.21.15.5:0/3998757140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15db80 0x7fae3c0cf290 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.564+0000 7fae43458700 1 --2- 172.21.15.5:0/3998757140 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15d7a0 0x7fae3c161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.564+0000 7fae43458700 1 -- 172.21.15.5:0/3998757140 >> 172.21.15.5:0/3998757140 conn(0x7fae3c08afe0 msgr2=0x7fae3c0d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.564+0000 7fae43458700 1 -- 172.21.15.5:0/3998757140 shutdown_connections 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.564+0000 7fae43458700 1 -- 172.21.15.5:0/3998757140 wait complete. 2024-05-08T23:26:17.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.565+0000 7fae43458700 1 Processor -- start 2024-05-08T23:26:17.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.565+0000 7fae43458700 1 -- start start 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae43458700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3c0d3f40 0x7fae3c154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae43458700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15d7a0 0x7fae3c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae43458700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15db80 0x7fae3c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae43458700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0ccb80 con 0x7fae3c15d7a0 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae43458700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0cb3b0 con 0x7fae3c0d3f40 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae43458700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0ccdb0 con 0x7fae3c15db80 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae409f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15d7a0 0x7fae3c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae409f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15d7a0 0x7fae3c0cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50474/0 (socket says 172.21.15.5:50474) 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae409f3700 1 -- 172.21.15.5:0/1502266145 learned_addr learned my addr 172.21.15.5:0/1502266145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae419f5700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15db80 0x7fae3c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.566+0000 7fae411f4700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3c0d3f40 0x7fae3c154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae409f3700 1 -- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15db80 msgr2=0x7fae3c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae409f3700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15db80 0x7fae3c0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae409f3700 1 -- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3c0d3f40 msgr2=0x7fae3c154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae409f3700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3c0d3f40 0x7fae3c154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae409f3700 1 -- 172.21.15.5:0/1502266145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009b30 con 0x7fae3c15d7a0 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae409f3700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15d7a0 0x7fae3c0cae70 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00b450 tx=0x7fae2c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae31ffb700 1 -- 172.21.15.5:0/1502266145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00be60 con 0x7fae3c15d7a0 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae31ffb700 1 -- 172.21.15.5:0/1502266145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c004020 con 0x7fae3c15d7a0 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c0cd090 con 0x7fae3c15d7a0 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae31ffb700 1 -- 172.21.15.5:0/1502266145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0043d0 con 0x7fae3c15d7a0 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.567+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c0cd650 con 0x7fae3c15d7a0 2024-05-08T23:26:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.568+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c0cd840 con 0x7fae3c15d7a0 2024-05-08T23:26:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.569+0000 7fae31ffb700 1 -- 172.21.15.5:0/1502266145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fae2c0036d0 con 0x7fae3c15d7a0 2024-05-08T23:26:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.569+0000 7fae31ffb700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae2006ac90 0x7fae2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.569+0000 7fae31ffb700 1 -- 172.21.15.5:0/1502266145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae2c088ad0 con 0x7fae3c15d7a0 2024-05-08T23:26:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.570+0000 7fae411f4700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae2006ac90 0x7fae2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.570+0000 7fae411f4700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae2006ac90 0x7fae2006d150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fae38006fd0 tx=0x7fae38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:17.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.574+0000 7fae31ffb700 1 -- 172.21.15.5:0/1502266145 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fae2c053c10 con 0x7fae3c15d7a0 2024-05-08T23:26:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.852+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae3c068000 con 0x7fae3c15d7a0 2024-05-08T23:26:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.853+0000 7fae31ffb700 1 -- 172.21.15.5:0/1502266145 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fae2c015070 con 0x7fae3c15d7a0 2024-05-08T23:26:17.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:17.853 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":68,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:17.399416+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7179,"mds_1":6790},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6790":{"gid":6790,"name":"b","rank":1,"incarnation":68,"state":"up:creating","state_seq":1,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.855+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae2006ac90 msgr2=0x7fae2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.855+0000 7fae43458700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae2006ac90 0x7fae2006d150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fae38006fd0 tx=0x7fae38008040 comp rx=0 tx=0).stop 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15d7a0 msgr2=0x7fae3c0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15d7a0 0x7fae3c0cae70 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00b450 tx=0x7fae2c00b760 comp rx=0 tx=0).stop 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 shutdown_connections 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fae2006ac90 0x7fae2006d150 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fae3c15db80 0x7fae3c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae3c0d3f40 0x7fae3c154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 --2- 172.21.15.5:0/1502266145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c15d7a0 0x7fae3c0cae70 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 >> 172.21.15.5:0/1502266145 conn(0x7fae3c08afe0 msgr2=0x7fae3c15cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 shutdown_connections 2024-05-08T23:26:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:17.856+0000 7fae43458700 1 -- 172.21.15.5:0/1502266145 wait complete. 2024-05-08T23:26:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 68 2024-05-08T23:26:18.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:26:04.359632+0000', 'modified': '2024-05-08T23:26:17.399416+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7179, 'mds_1': 6790}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6790': {'gid': 6790, 'name': 'b', 'rank': 1, 'incarnation': 68, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.49:6837/3300231911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6835', 'nonce': 3300231911}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 3300231911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7179': {'gid': 7179, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6834/690633393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 690633393}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 690633393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:26:18.272 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6790:up:creating 2024-05-08T23:26:19.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.425+0000 7fa1e6047700 1 Processor -- start 2024-05-08T23:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.425+0000 7fa1e6047700 1 -- start start 2024-05-08T23:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.426+0000 7fa1e6047700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1e006b140 0x7fa1e00ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.426+0000 7fa1e6047700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa1e00cf760 0x7fa1e00cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.426+0000 7fa1e6047700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e00cdbb0 con 0x7fa1e006a960 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.426+0000 7fa1e6047700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e00cd310 con 0x7fa1e006b140 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.426+0000 7fa1e6047700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e00cd5f0 con 0x7fa1e00cf760 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1deffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa1e006a960 0x7fa1e015bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47286/0 (socket says 172.21.15.5:47286) 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1deffd700 1 -- 172.21.15.5:0/904584474 learned_addr learned my addr 172.21.15.5:0/904584474 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1df7fe700 1 --2- 172.21.15.5:0/904584474 >> v2:172.21.15.5:3300/0 conn(0x7fa1e006b140 0x7fa1e00ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1dffff700 1 --2- 172.21.15.5:0/904584474 >> v2:172.21.15.49:3300/0 conn(0x7fa1e00cf760 0x7fa1e00cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1df7fe700 1 -- 172.21.15.5:0/904584474 >> v2:172.21.15.49:3300/0 conn(0x7fa1e00cf760 msgr2=0x7fa1e00cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1df7fe700 1 --2- 172.21.15.5:0/904584474 >> v2:172.21.15.49:3300/0 conn(0x7fa1e00cf760 0x7fa1e00cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1df7fe700 1 -- 172.21.15.5:0/904584474 >> v1:172.21.15.5:6789/0 conn(0x7fa1e006a960 legacy=0x7fa1e015bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1df7fe700 1 -- 172.21.15.5:0/904584474 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00cd8d0 con 0x7fa1e006b140 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1de7fc700 1 -- 172.21.15.5:0/904584474 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060726369 0 0) 0x7fa1e00cdbb0 con 0x7fa1e006a960 2024-05-08T23:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1df7fe700 1 --2- 172.21.15.5:0/904584474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e00ccd30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00cab50 tx=0x7fa1d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bdf19689882bd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1de7fc700 1 -- 172.21.15.5:0/904584474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0011830 con 0x7fa1e006b140 2024-05-08T23:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1de7fc700 1 -- 172.21.15.5:0/904584474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d0011990 con 0x7fa1e006b140 2024-05-08T23:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.427+0000 7fa1de7fc700 1 -- 172.21.15.5:0/904584474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0011d20 con 0x7fa1e006b140 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.428+0000 7fa1e6047700 1 -- 172.21.15.5:0/904584474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 msgr2=0x7fa1e00ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.428+0000 7fa1e6047700 1 --2- 172.21.15.5:0/904584474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e00ccd30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00cab50 tx=0x7fa1d0009b20 comp rx=0 tx=0).stop 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.428+0000 7fa1e6047700 1 -- 172.21.15.5:0/904584474 shutdown_connections 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.428+0000 7fa1e6047700 1 --2- 172.21.15.5:0/904584474 >> v2:172.21.15.49:3300/0 conn(0x7fa1e00cf760 0x7fa1e00cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.428+0000 7fa1e6047700 1 --2- 172.21.15.5:0/904584474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e00ccd30 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.428+0000 7fa1e6047700 1 -- 172.21.15.5:0/904584474 >> 172.21.15.5:0/904584474 conn(0x7fa1e00d0320 msgr2=0x7fa1e00d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.428+0000 7fa1e6047700 1 -- 172.21.15.5:0/904584474 shutdown_connections 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.428+0000 7fa1e6047700 1 -- 172.21.15.5:0/904584474 wait complete. 2024-05-08T23:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.430+0000 7fa1e6047700 1 Processor -- start 2024-05-08T23:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.430+0000 7fa1e6047700 1 -- start start 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1e6047700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1e006a960 0x7fa1e00c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1e6047700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e0149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1e6047700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1e00cf760 0x7fa1e00c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1e6047700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00c4430 con 0x7fa1e006b140 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1e6047700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00c4710 con 0x7fa1e006a960 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1e6047700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00c49f0 con 0x7fa1e00cf760 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e0149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e0149f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50486/0 (socket says 172.21.15.5:50486) 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1deffd700 1 -- 172.21.15.5:0/1051299668 learned_addr learned my addr 172.21.15.5:0/1051299668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1df7fe700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1e006a960 0x7fa1e00c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.431+0000 7fa1dffff700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1e00cf760 0x7fa1e00c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1deffd700 1 -- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1e00cf760 msgr2=0x7fa1e00c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1deffd700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1e00cf760 0x7fa1e00c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1deffd700 1 -- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1e006a960 msgr2=0x7fa1e00c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1deffd700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1e006a960 0x7fa1e00c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1deffd700 1 -- 172.21.15.5:0/1051299668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d00094a0 con 0x7fa1e006b140 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1dffff700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1e00cf760 0x7fa1e00c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1deffd700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e0149f70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c520 tx=0x7fa1d400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1c7fff700 1 -- 172.21.15.5:0/1051299668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4004c80 con 0x7fa1e006b140 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1c7fff700 1 -- 172.21.15.5:0/1051299668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d4004de0 con 0x7fa1e006b140 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e00c3cc0 con 0x7fa1e006b140 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1c7fff700 1 -- 172.21.15.5:0/1051299668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40072d0 con 0x7fa1e006b140 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.432+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e00c4fb0 con 0x7fa1e006b140 2024-05-08T23:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.433+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e00c51d0 con 0x7fa1e006b140 2024-05-08T23:26:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.434+0000 7fa1c7fff700 1 -- 172.21.15.5:0/1051299668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa1d4007430 con 0x7fa1e006b140 2024-05-08T23:26:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.434+0000 7fa1c7fff700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1cc06ac90 0x7fa1cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.434+0000 7fa1c7fff700 1 -- 172.21.15.5:0/1051299668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1d4088ef0 con 0x7fa1e006b140 2024-05-08T23:26:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.434+0000 7fa1df7fe700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1cc06ac90 0x7fa1cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.435+0000 7fa1df7fe700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1cc06ac90 0x7fa1cc06d150 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00090b0 tx=0x7fa1d0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.439+0000 7fa1c7fff700 1 -- 172.21.15.5:0/1051299668 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa1d4054160 con 0x7fa1e006b140 2024-05-08T23:26:19.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.708+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1e0068000 con 0x7fa1e006b140 2024-05-08T23:26:19.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.709+0000 7fa1c7fff700 1 -- 172.21.15.5:0/1051299668 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4755 (secure 0 0 0) 0x7fa1d4057780 con 0x7fa1e006b140 2024-05-08T23:26:19.709 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:19.709 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:19.407425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7179,"mds_1":6790,"mds_2":6787},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6787":{"gid":6787,"name":"d","rank":2,"incarnation":70,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6790":{"gid":6790,"name":"b","rank":1,"incarnation":68,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1cc06ac90 msgr2=0x7fa1cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1cc06ac90 0x7fa1cc06d150 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00090b0 tx=0x7fa1d0008030 comp rx=0 tx=0).stop 2024-05-08T23:26:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 msgr2=0x7fa1e0149f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e0149f70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c520 tx=0x7fa1d400c830 comp rx=0 tx=0).stop 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 shutdown_connections 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1cc06ac90 0x7fa1cc06d150 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1e00cf760 0x7fa1e00c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1e006a960 0x7fa1e00c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 --2- 172.21.15.5:0/1051299668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e006b140 0x7fa1e0149f70 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 >> 172.21.15.5:0/1051299668 conn(0x7fa1e00d0320 msgr2=0x7fa1e00ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 shutdown_connections 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:19.711+0000 7fa1e6047700 1 -- 172.21.15.5:0/1051299668 wait complete. 2024-05-08T23:26:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 71 2024-05-08T23:26:19.888 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:26:19.889+0000 7f42240c8700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:26:19.888 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:26:19.889+0000 7f42240c8700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T23:26:20.128 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:26:04.359632+0000', 'modified': '2024-05-08T23:26:19.407425+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7179, 'mds_1': 6790, 'mds_2': 6787}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6787': {'gid': 6787, 'name': 'd', 'rank': 2, 'incarnation': 70, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6836/2469967007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 2469967007}, {'type': 'v1', 'addr': '172.21.15.49:6836', 'nonce': 2469967007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_6790': {'gid': 6790, 'name': 'b', 'rank': 1, 'incarnation': 68, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6837/3300231911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6835', 'nonce': 3300231911}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 3300231911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7179': {'gid': 7179, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6834/690633393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 690633393}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 690633393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:26:20.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:26:20.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T23:26:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.275+0000 7f9339b03700 1 Processor -- start 2024-05-08T23:26:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.275+0000 7f9339b03700 1 -- start start 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f9339b03700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f933415bbd0 0x7f933415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f9339b03700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93340ccde0 0x7f933415f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f9339b03700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340cfdc0 con 0x7f93340cc980 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f9339b03700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93340cefa0 con 0x7f933415bbd0 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f9339b03700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340cd270 con 0x7f93340ccde0 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f93337fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f933415bbd0 0x7f933415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f93337fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f933415bbd0 0x7f933415bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50488/0 (socket says 172.21.15.5:50488) 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f93337fe700 1 -- 172.21.15.5:0/328465820 learned_addr learned my addr 172.21.15.5:0/328465820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.276+0000 7f9333fff700 1 --2- 172.21.15.5:0/328465820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93340ccde0 0x7f933415f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.277+0000 7f93337fe700 1 -- 172.21.15.5:0/328465820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93340ccde0 msgr2=0x7f933415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.277+0000 7f93337fe700 1 --2- 172.21.15.5:0/328465820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93340ccde0 0x7f933415f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.277+0000 7f93337fe700 1 -- 172.21.15.5:0/328465820 >> v1:172.21.15.49:6789/0 conn(0x7f93340cc980 legacy=0x7f933415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.277+0000 7f93337fe700 1 -- 172.21.15.5:0/328465820 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933415c4f0 con 0x7f933415bbd0 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.277+0000 7f93337fe700 1 --2- 172.21.15.5:0/328465820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933415bbd0 0x7f933415bfb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f93240098f0 tx=0x7f9324009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3962bf061ea6d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.277+0000 7f93327fc700 1 -- 172.21.15.5:0/328465820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324011830 con 0x7f933415bbd0 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.277+0000 7f93327fc700 1 -- 172.21.15.5:0/328465820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9324011990 con 0x7f933415bbd0 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.277+0000 7f93327fc700 1 -- 172.21.15.5:0/328465820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324011d20 con 0x7f933415bbd0 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.278+0000 7f9339b03700 1 -- 172.21.15.5:0/328465820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933415bbd0 msgr2=0x7f933415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.278+0000 7f9339b03700 1 --2- 172.21.15.5:0/328465820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933415bbd0 0x7f933415bfb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f93240098f0 tx=0x7f9324009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.278+0000 7f9339b03700 1 -- 172.21.15.5:0/328465820 shutdown_connections 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.278+0000 7f9339b03700 1 --2- 172.21.15.5:0/328465820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f93340ccde0 0x7f933415f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.278+0000 7f9339b03700 1 --2- 172.21.15.5:0/328465820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933415bbd0 0x7f933415bfb0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.278+0000 7f9339b03700 1 -- 172.21.15.5:0/328465820 >> 172.21.15.5:0/328465820 conn(0x7f93340b2ec0 msgr2=0x7f93340b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.278+0000 7f9339b03700 1 -- 172.21.15.5:0/328465820 shutdown_connections 2024-05-08T23:26:20.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.278+0000 7f9339b03700 1 -- 172.21.15.5:0/328465820 wait complete. 2024-05-08T23:26:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.279+0000 7f9339b03700 1 Processor -- start 2024-05-08T23:26:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.279+0000 7f9339b03700 1 -- start start 2024-05-08T23:26:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f9339b03700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93340cc980 0x7f93340b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f9339b03700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93340ccde0 0x7f93340b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f9339b03700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f933415bbd0 0x7f93341573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f9339b03700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340ae660 con 0x7f93340cc980 2024-05-08T23:26:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f9339b03700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b4ac0 con 0x7f93340ccde0 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f9339b03700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b4280 con 0x7f933415bbd0 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f93337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93340cc980 0x7f93340b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f93337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93340cc980 0x7f93340b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50498/0 (socket says 172.21.15.5:50498) 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f93337fe700 1 -- 172.21.15.5:0/4285411998 learned_addr learned my addr 172.21.15.5:0/4285411998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f93337fe700 1 -- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f933415bbd0 msgr2=0x7f93341573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f93337fe700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f933415bbd0 0x7f93341573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f93337fe700 1 -- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93340ccde0 msgr2=0x7f93340b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f9332ffd700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93340ccde0 0x7f93340b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f93337fe700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93340ccde0 0x7f93340b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.280+0000 7f93337fe700 1 -- 172.21.15.5:0/4285411998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324009550 con 0x7f93340cc980 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.281+0000 7f93337fe700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93340cc980 0x7f93340b3d40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9324000c00 tx=0x7f9324004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.281+0000 7f932bfff700 1 -- 172.21.15.5:0/4285411998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240036a0 con 0x7f93340cc980 2024-05-08T23:26:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.281+0000 7f932bfff700 1 -- 172.21.15.5:0/4285411998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932401e070 con 0x7f93340cc980 2024-05-08T23:26:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.281+0000 7f9339b03700 1 -- 172.21.15.5:0/4285411998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93340b4500 con 0x7f93340cc980 2024-05-08T23:26:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.281+0000 7f932bfff700 1 -- 172.21.15.5:0/4285411998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324003a40 con 0x7f93340cc980 2024-05-08T23:26:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.281+0000 7f9339b03700 1 -- 172.21.15.5:0/4285411998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340ae280 con 0x7f93340cc980 2024-05-08T23:26:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.282+0000 7f932bfff700 1 -- 172.21.15.5:0/4285411998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9324003ba0 con 0x7f93340cc980 2024-05-08T23:26:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.283+0000 7f932bfff700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f931406ac90 0x7f931406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.283+0000 7f932bfff700 1 -- 172.21.15.5:0/4285411998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f932408a200 con 0x7f93340cc980 2024-05-08T23:26:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.283+0000 7f9332ffd700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f931406ac90 0x7f931406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.284+0000 7f9332ffd700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f931406ac90 0x7f931406d150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f93340b6d10 tx=0x7f931c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.285+0000 7f9345cbdb80 1 Processor -- start 2024-05-08T23:26:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.285+0000 7f9345cbdb80 1 -- start start 2024-05-08T23:26:20.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.286+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c94a750 0x56464c9230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.286+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c928050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.286+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c951d80 0x56464c952200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.286+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c87e100 con 0x56464c94a750 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.286+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c87bac0 con 0x56464c951d80 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.286+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56464c8773b0 con 0x56464c925320 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.286+0000 7f93337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c94a750 0x56464c9230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.286+0000 7f93337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c94a750 0x56464c9230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50512/0 (socket says 172.21.15.5:50512) 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93337fe700 1 -- 172.21.15.5:0/2315722189 learned_addr learned my addr 172.21.15.5:0/2315722189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f9332ffd700 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c928050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f9333fff700 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c951d80 0x56464c952200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93337fe700 1 -- 172.21.15.5:0/2315722189 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 msgr2=0x56464c928050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93337fe700 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c928050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93337fe700 1 -- 172.21.15.5:0/2315722189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c951d80 msgr2=0x56464c952200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93337fe700 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c951d80 0x56464c952200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93337fe700 1 -- 172.21.15.5:0/2315722189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464c6f8360 con 0x56464c94a750 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93337fe700 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c94a750 0x56464c9230c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9324009520 tx=0x7f9324058a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1018ae53c857b272 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93297fa700 1 -- 172.21.15.5:0/2315722189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324059090 con 0x56464c94a750 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93297fa700 1 -- 172.21.15.5:0/2315722189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932405c070 con 0x56464c94a750 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.287+0000 7f93297fa700 1 -- 172.21.15.5:0/2315722189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932405cec0 con 0x56464c94a750 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2315722189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c94a750 msgr2=0x56464c9230c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c94a750 0x56464c9230c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9324009520 tx=0x7f9324058a70 comp rx=0 tx=0).stop 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2315722189 shutdown_connections 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c951d80 0x56464c952200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c94a750 0x56464c9230c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2315722189 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c928050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2315722189 >> 172.21.15.5:0/2315722189 conn(0x56464c8851c0 msgr2=0x56464c96cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2315722189 shutdown_connections 2024-05-08T23:26:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.288+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2315722189 wait complete. 2024-05-08T23:26:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.289+0000 7f9345cbdb80 1 -- start start 2024-05-08T23:26:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.289+0000 7f9345cbdb80 1 Processor -- start 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c944dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c949370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 0x56464c96e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c813510 con 0x56464c925320 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c8131b0 con 0x56464c94a750 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56464c6fa650 con 0x56464c951d80 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9345cbdb80 10 client.0 _ll_register_callbacks cb 0x56464c81cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c944dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c944dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50522/0 (socket says 172.21.15.5:50522) 2024-05-08T23:26:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 -- 172.21.15.5:0/2624916485 learned_addr learned my addr 172.21.15.5:0/2624916485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9332ffd700 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c949370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f9333fff700 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 0x56464c96e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 -- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 msgr2=0x56464c96e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 0x56464c96e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 -- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 msgr2=0x56464c949370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c949370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 -- 172.21.15.5:0/2624916485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340ae280 con 0x56464c925320 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93337fe700 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c944dd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9324058d90 tx=0x7f9324012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3d69ca94e6fb92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93127fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.290+0000 7f93127fc700 1 -- 172.21.15.5:0/2624916485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324059090 con 0x56464c925320 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f93127fc700 1 -- 172.21.15.5:0/2624916485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f932405cec0 con 0x56464c925320 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2624916485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56464c7d0220 con 0x56464c925320 2024-05-08T23:26:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f93127fc700 1 -- 172.21.15.5:0/2624916485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9324088300 con 0x56464c925320 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 10 client.7392 fetch_fsmap learned FSMap version 71 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2624916485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x56464c6a4b10 con 0x56464c925320 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f93127fc700 1 -- 172.21.15.5:0/2624916485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56464c7d0220 con 0x56464c925320 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f93127fc700 1 -- 172.21.15.5:0/2624916485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 71) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f93240557a0 con 0x56464c925320 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 10 client.7392 fetch_fsmap finished waiting for FSMap version 71 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 10 client.7392 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 20 client.7392 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x56464c980290 0x56464c97f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 4 client.7392 mds_command: new command op to 6787 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2624916485 --> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56464c8f6790 con 0x56464c980290 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.291+0000 7f9332ffd700 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x56464c980290 0x56464c97f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.292+0000 7f93127fc700 10 client.7392 ms_handle_connect on v2:172.21.15.49:6834/2469967007 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.292+0000 7f9332ffd700 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x56464c980290 0x56464c97f590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2a8bbf77cf9c716 server_cookie=422fc59f13f0c773 in_seq=0 out_seq=0 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.292+0000 7f93127fc700 1 -- 172.21.15.5:0/2624916485 <== mds.2 v2:172.21.15.49:6834/2469967007 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56464c8f6790 con 0x56464c980290 2024-05-08T23:26:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.292+0000 7f93127fc700 10 client.7392 handle_command_reply: tid=0 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.292+0000 7f9345cbdb80 1 client.7392 shutdown 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2624916485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 msgr2=0x56464c944dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c944dd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9324058d90 tx=0x7f9324012f90 comp rx=0 tx=0).stop 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2624916485 shutdown_connections 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x56464c980290 0x56464c97f590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 0x56464c96e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c944dd0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f93127fc700 0 client.7392 ms_handle_reset on v2:172.21.15.49:6834/2469967007 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/2624916485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c949370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2624916485 >> 172.21.15.5:0/2624916485 conn(0x56464c8851c0 msgr2=0x56464c952200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2624916485 shutdown_connections 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 1 -- 172.21.15.5:0/2624916485 wait complete. 2024-05-08T23:26:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.293+0000 7f9345cbdb80 20 client.7392 trim_cache size 0 max 16384 2024-05-08T23:26:20.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.312+0000 7f9345cbdb80 1 Processor -- start 2024-05-08T23:26:20.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.312+0000 7f9345cbdb80 1 -- start start 2024-05-08T23:26:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c970770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c96d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 0x56464c94ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c963c70 con 0x56464c925320 2024-05-08T23:26:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c96dd60 con 0x56464c94a750 2024-05-08T23:26:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56464c96c7d0 con 0x56464c951d80 2024-05-08T23:26:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f93337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c970770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f93337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c970770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50534/0 (socket says 172.21.15.5:50534) 2024-05-08T23:26:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f93337fe700 1 -- 172.21.15.5:0/1776883558 learned_addr learned my addr 172.21.15.5:0/1776883558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f93337fe700 1 -- 172.21.15.5:0/1776883558 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 msgr2=0x56464c94ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f93337fe700 1 --2- 172.21.15.5:0/1776883558 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 0x56464c94ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f93337fe700 1 -- 172.21.15.5:0/1776883558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 msgr2=0x56464c96d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f93337fe700 1 --2- 172.21.15.5:0/1776883558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c96d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.313+0000 7f93337fe700 1 -- 172.21.15.5:0/1776883558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464c83d510 con 0x56464c925320 2024-05-08T23:26:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f93337fe700 1 --2- 172.21.15.5:0/1776883558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c970770 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f932400b2f0 tx=0x7f9324084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fdbaa7fddd38e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9329ffb700 1 -- 172.21.15.5:0/1776883558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324059810 con 0x56464c925320 2024-05-08T23:26:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9329ffb700 1 -- 172.21.15.5:0/1776883558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932405cec0 con 0x56464c925320 2024-05-08T23:26:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9329ffb700 1 -- 172.21.15.5:0/1776883558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324059090 con 0x56464c925320 2024-05-08T23:26:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1776883558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 msgr2=0x56464c970770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1776883558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c970770 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f932400b2f0 tx=0x7f9324084040 comp rx=0 tx=0).stop 2024-05-08T23:26:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1776883558 shutdown_connections 2024-05-08T23:26:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1776883558 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c951d80 0x56464c94ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1776883558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c925320 0x56464c970770 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1776883558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c96d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.314+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1776883558 >> 172.21.15.5:0/1776883558 conn(0x56464c8851c0 msgr2=0x56464c9242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.315+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1776883558 shutdown_connections 2024-05-08T23:26:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.315+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1776883558 wait complete. 2024-05-08T23:26:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.315+0000 7f9345cbdb80 1 -- start start 2024-05-08T23:26:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.315+0000 7f9345cbdb80 1 Processor -- start 2024-05-08T23:26:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c986f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c94ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9345cbdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c951d80 0x56464c94b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c96f920 con 0x56464c951d80 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c9d8a60 con 0x56464c94a750 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9345cbdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56464c963580 con 0x56464c925320 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9345cbdb80 10 client.0 _ll_register_callbacks cb 0x56464c81cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9333fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c951d80 0x56464c94b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9333fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c951d80 0x56464c94b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50536/0 (socket says 172.21.15.5:50536) 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9333fff700 1 -- 172.21.15.5:0/1170806186 learned_addr learned my addr 172.21.15.5:0/1170806186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f9332ffd700 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c94ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.316+0000 7f93337fe700 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c986f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9333fff700 1 -- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 msgr2=0x56464c986f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9333fff700 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c986f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9333fff700 1 -- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 msgr2=0x56464c94ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9333fff700 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c94ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9333fff700 1 -- 172.21.15.5:0/1170806186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932407d070 con 0x56464c951d80 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f93337fe700 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c986f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9333fff700 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c951d80 0x56464c94b070 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f932c014dd0 tx=0x7f932c014e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd691ff10bbfa662 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f93127fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f93127fc700 1 -- 172.21.15.5:0/1170806186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c00e040 con 0x56464c951d80 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f93127fc700 1 -- 172.21.15.5:0/1170806186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f932c01a070 con 0x56464c951d80 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1170806186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f932405c070 con 0x56464c951d80 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f93127fc700 1 -- 172.21.15.5:0/1170806186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f932c003e30 con 0x56464c951d80 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 10 client.7398 fetch_fsmap learned FSMap version 71 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1170806186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x56464c6fcc80 con 0x56464c951d80 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f93127fc700 1 -- 172.21.15.5:0/1170806186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932405c070 con 0x56464c951d80 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f93127fc700 1 -- 172.21.15.5:0/1170806186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 71) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f932c0127d0 con 0x56464c951d80 2024-05-08T23:26:20.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 10 client.7398 fetch_fsmap finished waiting for FSMap version 71 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 10 client.7398 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 20 client.7398 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x56464c9d5670 0x56464c9d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 4 client.7398 mds_command: new command op to 6787 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.317+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1170806186 --> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56464c86fe70 con 0x56464c9d5670 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.318+0000 7f93337fe700 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x56464c9d5670 0x56464c9d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.318+0000 7f93337fe700 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x56464c9d5670 0x56464c9d5a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe634ea359f63cd6 server_cookie=cb607f58ab127a2f in_seq=0 out_seq=0 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.318+0000 7f93127fc700 10 client.7398 ms_handle_connect on v2:172.21.15.49:6834/2469967007 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.318+0000 7f93127fc700 1 -- 172.21.15.5:0/1170806186 <== mds.2 v2:172.21.15.49:6834/2469967007 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56464c86fe70 con 0x56464c9d5670 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.318+0000 7f93127fc700 10 client.7398 handle_command_reply: tid=0 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.318+0000 7f9345cbdb80 1 client.7398 shutdown 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.318+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1170806186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c951d80 msgr2=0x56464c94b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.318+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c951d80 0x56464c94b070 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f932c014dd0 tx=0x7f932c014e00 comp rx=0 tx=0).stop 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1170806186 shutdown_connections 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x56464c9d5670 0x56464c9d5a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56464c951d80 0x56464c94b070 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56464c925320 0x56464c986f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 1 --2- 172.21.15.5:0/1170806186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56464c94a750 0x56464c94ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f93127fc700 0 client.7398 ms_handle_reset on v2:172.21.15.49:6834/2469967007 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1170806186 >> 172.21.15.5:0/1170806186 conn(0x56464c8851c0 msgr2=0x56464c96cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1170806186 shutdown_connections 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 1 -- 172.21.15.5:0/1170806186 wait complete. 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9345cbdb80 20 client.7398 trim_cache size 0 max 16384 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6787, 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 1.9127630280000001, 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 71, 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 72, 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 72, 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 20.997259504999999 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 -- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f931406ac90 msgr2=0x7f931406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f931406ac90 0x7f931406d150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f93340b6d10 tx=0x7f931c009040 comp rx=0 tx=0).stop 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 -- 172.21.15.5:0/4285411998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93340cc980 msgr2=0x7f93340b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93340cc980 0x7f93340b3d40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9324000c00 tx=0x7f9324004750 comp rx=0 tx=0).stop 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 -- 172.21.15.5:0/4285411998 shutdown_connections 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f931406ac90 0x7f931406d150 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f933415bbd0 0x7f93341573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93340cc980 0x7f93340b3d40 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 --2- 172.21.15.5:0/4285411998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93340ccde0 0x7f93340b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 -- 172.21.15.5:0/4285411998 >> 172.21.15.5:0/4285411998 conn(0x7f93340b2ec0 msgr2=0x7f93340b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.319+0000 7f9339b03700 1 -- 172.21.15.5:0/4285411998 shutdown_connections 2024-05-08T23:26:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.320+0000 7f9339b03700 1 -- 172.21.15.5:0/4285411998 wait complete. 2024-05-08T23:26:20.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:26:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.475+0000 7fe301797700 1 Processor -- start 2024-05-08T23:26:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.475+0000 7fe301797700 1 -- start start 2024-05-08T23:26:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe301797700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2fc0b47f0 0x7fe2fc0d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe301797700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2fc0d2100 0x7fe2fc0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe301797700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc0ce3f0 con 0x7fe2fc160f00 2024-05-08T23:26:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe301797700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc0cdc50 con 0x7fe2fc0d2100 2024-05-08T23:26:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe301797700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc0d2570 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2fc0d2100 0x7fe2fc0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2fc0d2100 0x7fe2fc0ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50550/0 (socket says 172.21.15.5:50550) 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fb7fe700 1 -- 172.21.15.5:0/2937026624 learned_addr learned my addr 172.21.15.5:0/2937026624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/2937026624 >> v2:172.21.15.49:3300/0 conn(0x7fe2fc0b47f0 0x7fe2fc0d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fb7fe700 1 -- 172.21.15.5:0/2937026624 >> v2:172.21.15.49:3300/0 conn(0x7fe2fc0b47f0 msgr2=0x7fe2fc0d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/2937026624 >> v2:172.21.15.49:3300/0 conn(0x7fe2fc0b47f0 0x7fe2fc0d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fb7fe700 1 -- 172.21.15.5:0/2937026624 >> v1:172.21.15.49:6789/0 conn(0x7fe2fc160f00 legacy=0x7fe2fc0b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fb7fe700 1 -- 172.21.15.5:0/2937026624 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc0b42c0 con 0x7fe2fc0d2100 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/2937026624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0d2100 0x7fe2fc0ce860 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc08d8c0 tx=0x7fe2f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40535d56c0dd8715 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.476+0000 7fe2f9ffb700 1 -- 172.21.15.5:0/2937026624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400e070 con 0x7fe2fc0d2100 2024-05-08T23:26:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe2f9ffb700 1 -- 172.21.15.5:0/2937026624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f400b8d0 con 0x7fe2fc0d2100 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe2f9ffb700 1 -- 172.21.15.5:0/2937026624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400bc80 con 0x7fe2fc0d2100 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe301797700 1 -- 172.21.15.5:0/2937026624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0d2100 msgr2=0x7fe2fc0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe301797700 1 --2- 172.21.15.5:0/2937026624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0d2100 0x7fe2fc0ce860 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc08d8c0 tx=0x7fe2f400b330 comp rx=0 tx=0).stop 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe301797700 1 -- 172.21.15.5:0/2937026624 shutdown_connections 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe301797700 1 --2- 172.21.15.5:0/2937026624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0d2100 0x7fe2fc0ce860 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe301797700 1 --2- 172.21.15.5:0/2937026624 >> v2:172.21.15.49:3300/0 conn(0x7fe2fc0b47f0 0x7fe2fc0d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe301797700 1 -- 172.21.15.5:0/2937026624 >> 172.21.15.5:0/2937026624 conn(0x7fe2fc08afe0 msgr2=0x7fe2fc0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe301797700 1 -- 172.21.15.5:0/2937026624 shutdown_connections 2024-05-08T23:26:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.477+0000 7fe301797700 1 -- 172.21.15.5:0/2937026624 wait complete. 2024-05-08T23:26:20.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.479+0000 7fe301797700 1 Processor -- start 2024-05-08T23:26:20.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.479+0000 7fe301797700 1 -- start start 2024-05-08T23:26:20.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.479+0000 7fe301797700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0b47f0 0x7fe2fc0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.479+0000 7fe301797700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2fc0d2100 0x7fe2fc1f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.479+0000 7fe301797700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe2fc160f00 0x7fe2fc1f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.479+0000 7fe301797700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc0cea90 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.479+0000 7fe301797700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc0ce760 con 0x7fe2fc0d2100 2024-05-08T23:26:20.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.479+0000 7fe301797700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc0ccfd0 con 0x7fe2fc160f00 2024-05-08T23:26:20.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0b47f0 0x7fe2fc0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0b47f0 0x7fe2fc0ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50554/0 (socket says 172.21.15.5:50554) 2024-05-08T23:26:20.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2fa7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2fc0d2100 0x7fe2fc1f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2fb7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe2fc160f00 0x7fe2fc1f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 -- 172.21.15.5:0/453660583 learned_addr learned my addr 172.21.15.5:0/453660583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 -- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe2fc160f00 msgr2=0x7fe2fc1f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe2fc160f00 0x7fe2fc1f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 -- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2fc0d2100 msgr2=0x7fe2fc1f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2fc0d2100 0x7fe2fc1f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 -- 172.21.15.5:0/453660583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f4009b30 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2fc0d2100 0x7fe2fc1f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2faffd700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0b47f0 0x7fe2fc0ca6d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00c480 tx=0x7fe2ec00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2e37fe700 1 -- 172.21.15.5:0/453660583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0052d0 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2e37fe700 1 -- 172.21.15.5:0/453660583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec00ce90 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe301797700 1 -- 172.21.15.5:0/453660583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc0cd2b0 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.480+0000 7fe2e37fe700 1 -- 172.21.15.5:0/453660583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0068b0 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.481+0000 7fe301797700 1 -- 172.21.15.5:0/453660583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc0cac10 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.482+0000 7fe2e37fe700 1 -- 172.21.15.5:0/453660583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe2ec0036d0 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.483+0000 7fe2e37fe700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe2e806ac90 0x7fe2e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.483+0000 7fe2e37fe700 1 -- 172.21.15.5:0/453660583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2ec0884c0 con 0x7fe2fc0b47f0 2024-05-08T23:26:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.483+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe2e806ac90 0x7fe2e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.483+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe2e806ac90 0x7fe2e806d150 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0c70a0 tx=0x7fe2f0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.485+0000 7fe30d951b80 1 Processor -- start 2024-05-08T23:26:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.485+0000 7fe30d951b80 1 -- start start 2024-05-08T23:26:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb670ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb6711d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe30d951b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb67049100 con 0x55eb670f0320 2024-05-08T23:26:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe30d951b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb67046ac0 con 0x55eb6711cd80 2024-05-08T23:26:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe30d951b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb670423b0 con 0x55eb67115750 2024-05-08T23:26:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe2fa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe2fa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670f3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50568/0 (socket says 172.21.15.5:50568) 2024-05-08T23:26:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe2fa7fc700 1 -- 172.21.15.5:0/3034614793 learned_addr learned my addr 172.21.15.5:0/3034614793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb6711d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.486+0000 7fe2faffd700 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb670ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2fa7fc700 1 -- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 msgr2=0x55eb670ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb670ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2fa7fc700 1 -- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 msgr2=0x55eb6711d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb6711d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2fa7fc700 1 -- 172.21.15.5:0/3034614793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb66ec3360 con 0x55eb670f0320 2024-05-08T23:26:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb6711d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670f3050 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0004080 tx=0x7fe2f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff3ad888e3db771 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2e0ff9700 1 -- 172.21.15.5:0/3034614793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0003b30 con 0x55eb670f0320 2024-05-08T23:26:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2e0ff9700 1 -- 172.21.15.5:0/3034614793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f001b070 con 0x55eb670f0320 2024-05-08T23:26:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.487+0000 7fe2e0ff9700 1 -- 172.21.15.5:0/3034614793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0022440 con 0x55eb670f0320 2024-05-08T23:26:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 -- 172.21.15.5:0/3034614793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 msgr2=0x55eb670f3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670f3050 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0004080 tx=0x7fe2f0014000 comp rx=0 tx=0).stop 2024-05-08T23:26:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 -- 172.21.15.5:0/3034614793 shutdown_connections 2024-05-08T23:26:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb6711d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb670ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3034614793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670f3050 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 -- 172.21.15.5:0/3034614793 >> 172.21.15.5:0/3034614793 conn(0x55eb670501c0 msgr2=0x55eb67137d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 -- 172.21.15.5:0/3034614793 shutdown_connections 2024-05-08T23:26:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.488+0000 7fe30d951b80 1 -- 172.21.15.5:0/3034614793 wait complete. 2024-05-08T23:26:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.489+0000 7fe30d951b80 1 -- start start 2024-05-08T23:26:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.489+0000 7fe30d951b80 1 Processor -- start 2024-05-08T23:26:20.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.489+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb670f0320 0x55eb6711df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb67115750 0x55eb671470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb6711cd80 0x55eb671475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe30d951b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb66fde510 con 0x55eb67115750 2024-05-08T23:26:20.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe30d951b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb66fde1b0 con 0x55eb670f0320 2024-05-08T23:26:20.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe30d951b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb66ec5650 con 0x55eb6711cd80 2024-05-08T23:26:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe30d951b80 10 client.0 _ll_register_callbacks cb 0x55eb66fe7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb67115750 0x55eb671470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb67115750 0x55eb671470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50576/0 (socket says 172.21.15.5:50576) 2024-05-08T23:26:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 -- 172.21.15.5:0/3889473599 learned_addr learned my addr 172.21.15.5:0/3889473599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb6711cd80 0x55eb671475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2faffd700 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb670f0320 0x55eb6711df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 -- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb6711cd80 msgr2=0x55eb671475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb6711cd80 0x55eb671475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 -- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb670f0320 msgr2=0x55eb6711df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb670f0320 0x55eb6711df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 -- 172.21.15.5:0/3889473599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f0011d50 con 0x55eb67115750 2024-05-08T23:26:20.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb67115750 0x55eb671470a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0006c60 tx=0x7fe2f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddf89bf861d3d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:20.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/3889473599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0022a70 con 0x55eb67115750 2024-05-08T23:26:20.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/3889473599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2f0022bd0 con 0x55eb67115750 2024-05-08T23:26:20.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.490+0000 7fe30d951b80 1 -- 172.21.15.5:0/3889473599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb66f9b220 con 0x55eb67115750 2024-05-08T23:26:20.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/3889473599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2f0022d90 con 0x55eb67115750 2024-05-08T23:26:20.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe30d951b80 10 client.7410 fetch_fsmap learned FSMap version 71 2024-05-08T23:26:20.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe30d951b80 1 -- 172.21.15.5:0/3889473599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55eb66e6fb10 con 0x55eb67115750 2024-05-08T23:26:20.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/3889473599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb66f9b220 con 0x55eb67115750 2024-05-08T23:26:20.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/3889473599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 71) v1 ==== 3138+0+0 (secure 0 0 0) 0x7fe2f0035470 con 0x55eb67115750 2024-05-08T23:26:20.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe30d951b80 10 client.7410 fetch_fsmap finished waiting for FSMap version 71 2024-05-08T23:26:20.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe30d951b80 10 client.7410 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:20.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe30d951b80 20 client.7410 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:20.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55eb6714ad10 0x55eb6714b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe30d951b80 4 client.7410 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:20.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe30d951b80 1 -- 172.21.15.5:0/3889473599 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb670c1790 con 0x55eb6714ad10 2024-05-08T23:26:20.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.491+0000 7fe2faffd700 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55eb6714ad10 0x55eb6714b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.492+0000 7fe2faffd700 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55eb6714ad10 0x55eb6714b0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20a913e5f60ed5a3 server_cookie=435119e475b2b533 in_seq=0 out_seq=0 2024-05-08T23:26:20.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.492+0000 7fe2d9ffb700 10 client.7410 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:20.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.492+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/3889473599 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb670c1790 con 0x55eb6714ad10 2024-05-08T23:26:20.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.492+0000 7fe2d9ffb700 10 client.7410 handle_command_reply: tid=0 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.492+0000 7fe30d951b80 1 client.7410 shutdown 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 -- 172.21.15.5:0/3889473599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb67115750 msgr2=0x55eb671470a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb67115750 0x55eb671470a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0006c60 tx=0x7fe2f00047c0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 -- 172.21.15.5:0/3889473599 shutdown_connections 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55eb6714ad10 0x55eb6714b0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb6711cd80 0x55eb671475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb670f0320 0x55eb6711df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 --2- 172.21.15.5:0/3889473599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb67115750 0x55eb671470a0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe2d9ffb700 0 client.7410 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 -- 172.21.15.5:0/3889473599 >> 172.21.15.5:0/3889473599 conn(0x55eb670501c0 msgr2=0x55eb670f2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 -- 172.21.15.5:0/3889473599 shutdown_connections 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 1 -- 172.21.15.5:0/3889473599 wait complete. 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.493+0000 7fe30d951b80 20 client.7410 trim_cache size 0 max 16384 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.513+0000 7fe30d951b80 1 Processor -- start 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.513+0000 7fe30d951b80 1 -- start start 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.513+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.513+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb67158960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.513+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb67119e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.513+0000 7fe30d951b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb670eeaa0 con 0x55eb670f0320 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.513+0000 7fe30d951b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb670ed090 con 0x55eb6711cd80 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.513+0000 7fe30d951b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb67137f20 con 0x55eb67115750 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670ec2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50582/0 (socket says 172.21.15.5:50582) 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 -- 172.21.15.5:0/4072977386 learned_addr learned my addr 172.21.15.5:0/4072977386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb67158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb67119e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 -- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 msgr2=0x55eb67158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb67158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 -- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 msgr2=0x55eb67119e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb67119e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 -- 172.21.15.5:0/4072977386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb67008510 con 0x55eb670f0320 2024-05-08T23:26:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2fa7fc700 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb67158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb67119e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2faffd700 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670ec2d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0572f0 tx=0x7fe2ec0868c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7397912af91ab63e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2e17fa700 1 -- 172.21.15.5:0/4072977386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec05f750 con 0x55eb670f0320 2024-05-08T23:26:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2e17fa700 1 -- 172.21.15.5:0/4072977386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec0554a0 con 0x55eb670f0320 2024-05-08T23:26:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.514+0000 7fe2e17fa700 1 -- 172.21.15.5:0/4072977386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0575c0 con 0x55eb670f0320 2024-05-08T23:26:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 -- 172.21.15.5:0/4072977386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 msgr2=0x55eb670ec2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670ec2d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0572f0 tx=0x7fe2ec0868c0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 -- 172.21.15.5:0/4072977386 shutdown_connections 2024-05-08T23:26:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb6711cd80 0x55eb67119e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb670f0320 0x55eb670ec2d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 --2- 172.21.15.5:0/4072977386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb67115750 0x55eb67158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 -- 172.21.15.5:0/4072977386 >> 172.21.15.5:0/4072977386 conn(0x55eb670501c0 msgr2=0x55eb670e9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 -- 172.21.15.5:0/4072977386 shutdown_connections 2024-05-08T23:26:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 -- 172.21.15.5:0/4072977386 wait complete. 2024-05-08T23:26:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.515+0000 7fe30d951b80 1 -- start start 2024-05-08T23:26:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.516+0000 7fe30d951b80 1 Processor -- start 2024-05-08T23:26:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.516+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb670f0320 0x55eb671a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.516+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb67115750 0x55eb67119e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.516+0000 7fe30d951b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb6711cd80 0x55eb6711a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.516+0000 7fe30d951b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb671a2450 con 0x55eb6711cd80 2024-05-08T23:26:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.516+0000 7fe30d951b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb671496c0 con 0x55eb67115750 2024-05-08T23:26:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.516+0000 7fe30d951b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb67146e80 con 0x55eb670f0320 2024-05-08T23:26:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.516+0000 7fe30d951b80 10 client.0 _ll_register_callbacks cb 0x55eb66fe7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb6711cd80 0x55eb6711a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fa7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb67115750 0x55eb67119e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb6711cd80 0x55eb6711a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50592/0 (socket says 172.21.15.5:50592) 2024-05-08T23:26:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb670f0320 0x55eb671a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 -- 172.21.15.5:0/1567567883 learned_addr learned my addr 172.21.15.5:0/1567567883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 -- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb670f0320 msgr2=0x55eb671a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb670f0320 0x55eb671a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 -- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb67115750 msgr2=0x55eb67119e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb67115750 0x55eb67119e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 -- 172.21.15.5:0/1567567883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc0cac10 con 0x55eb6711cd80 2024-05-08T23:26:20.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2fb7fe700 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb6711cd80 0x55eb6711a340 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc06b790 tx=0x7fe2f4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60026d5ffead26d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:20.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/1567567883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4004450 con 0x55eb6711cd80 2024-05-08T23:26:20.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/1567567883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2f40045b0 con 0x55eb6711cd80 2024-05-08T23:26:20.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe30d951b80 1 -- 172.21.15.5:0/1567567883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2f001b070 con 0x55eb6711cd80 2024-05-08T23:26:20.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/1567567883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2f40092e0 con 0x55eb6711cd80 2024-05-08T23:26:20.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe30d951b80 10 client.7416 fetch_fsmap learned FSMap version 71 2024-05-08T23:26:20.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.517+0000 7fe30d951b80 1 -- 172.21.15.5:0/1567567883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55eb66ec7c80 con 0x55eb6711cd80 2024-05-08T23:26:20.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/1567567883 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f001b070 con 0x55eb6711cd80 2024-05-08T23:26:20.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/1567567883 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 71) v1 ==== 3138+0+0 (secure 0 0 0) 0x7fe2f400b050 con 0x55eb6711cd80 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe30d951b80 10 client.7416 fetch_fsmap finished waiting for FSMap version 71 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe30d951b80 10 client.7416 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe30d951b80 20 client.7416 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe30d951b80 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55eb6712dd30 0x55eb6712e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe30d951b80 4 client.7416 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe30d951b80 1 -- 172.21.15.5:0/1567567883 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eb6703ae70 con 0x55eb6712dd30 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe2faffd700 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55eb6712dd30 0x55eb6712e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe2d9ffb700 10 client.7416 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.518+0000 7fe2faffd700 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55eb6712dd30 0x55eb6712e110 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da6a82f4a3d0aa5a server_cookie=73c8bea631980ea4 in_seq=0 out_seq=0 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe2d9ffb700 1 -- 172.21.15.5:0/1567567883 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55eb6703ae70 con 0x55eb6712dd30 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe2d9ffb700 10 client.7416 handle_command_reply: tid=0 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 client.7416 shutdown 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 -- 172.21.15.5:0/1567567883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb6711cd80 msgr2=0x55eb6711a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb6711cd80 0x55eb6711a340 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc06b790 tx=0x7fe2f4004b40 comp rx=0 tx=0).stop 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 -- 172.21.15.5:0/1567567883 shutdown_connections 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55eb6712dd30 0x55eb6712e110 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb6711cd80 0x55eb6711a340 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eb670f0320 0x55eb671a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 --2- 172.21.15.5:0/1567567883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb67115750 0x55eb67119e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe2d9ffb700 0 client.7416 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 -- 172.21.15.5:0/1567567883 >> 172.21.15.5:0/1567567883 conn(0x55eb670501c0 msgr2=0x55eb67115b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 -- 172.21.15.5:0/1567567883 shutdown_connections 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 1 -- 172.21.15.5:0/1567567883 wait complete. 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe30d951b80 20 client.7416 trim_cache size 0 max 16384 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6790, 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:26:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 3.1176860190000002, 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 71, 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 72, 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 72, 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 21.197303642000001 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe301797700 1 -- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe2e806ac90 msgr2=0x7fe2e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe301797700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe2e806ac90 0x7fe2e806d150 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0c70a0 tx=0x7fe2f0009210 comp rx=0 tx=0).stop 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe301797700 1 -- 172.21.15.5:0/453660583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0b47f0 msgr2=0x7fe2fc0ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.519+0000 7fe301797700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0b47f0 0x7fe2fc0ca6d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00c480 tx=0x7fe2ec00c790 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.520+0000 7fe301797700 1 -- 172.21.15.5:0/453660583 shutdown_connections 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.520+0000 7fe301797700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe2e806ac90 0x7fe2e806d150 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.520+0000 7fe301797700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe2fc160f00 0x7fe2fc1f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.520+0000 7fe301797700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2fc0b47f0 0x7fe2fc0ca6d0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.520+0000 7fe301797700 1 --2- 172.21.15.5:0/453660583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2fc0d2100 0x7fe2fc1f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.520+0000 7fe301797700 1 -- 172.21.15.5:0/453660583 >> 172.21.15.5:0/453660583 conn(0x7fe2fc08afe0 msgr2=0x7fe2fc06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.520+0000 7fe301797700 1 -- 172.21.15.5:0/453660583 shutdown_connections 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.520+0000 7fe301797700 1 -- 172.21.15.5:0/453660583 wait complete. 2024-05-08T23:26:20.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:26:20.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.691+0000 7f53a81ab700 1 Processor -- start 2024-05-08T23:26:20.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.691+0000 7f53a81ab700 1 -- start start 2024-05-08T23:26:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.692+0000 7f53a81ab700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00cb8c0 0x7f53a00b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.692+0000 7f53a81ab700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53a00716b0 0x7f53a006d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.692+0000 7f53a81ab700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a00b4b20 con 0x7f53a0157420 2024-05-08T23:26:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.692+0000 7f53a81ab700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a015acf0 con 0x7f53a00716b0 2024-05-08T23:26:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.692+0000 7f53a81ab700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a0157800 con 0x7f53a00cb8c0 2024-05-08T23:26:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.692+0000 7f53a5f47700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f53a0157420 0x7f53a0070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47296/0 (socket says 172.21.15.5:47296) 2024-05-08T23:26:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.692+0000 7f53a5f47700 1 -- 172.21.15.5:0/859179818 learned_addr learned my addr 172.21.15.5:0/859179818 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a5746700 1 --2- 172.21.15.5:0/859179818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00cb8c0 0x7f53a00b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a6748700 1 --2- 172.21.15.5:0/859179818 >> v2:172.21.15.49:3300/0 conn(0x7f53a00716b0 0x7f53a006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a4f45700 1 -- 172.21.15.5:0/859179818 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954322764 0 0) 0x7f539c001110 con 0x7f53a0157420 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a4f45700 1 -- 172.21.15.5:0/859179818 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f538c0035b0 con 0x7f53a0157420 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a5746700 1 -- 172.21.15.5:0/859179818 >> v2:172.21.15.49:3300/0 conn(0x7f53a00716b0 msgr2=0x7f53a006d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a5746700 1 --2- 172.21.15.5:0/859179818 >> v2:172.21.15.49:3300/0 conn(0x7f53a00716b0 0x7f53a006d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a5746700 1 -- 172.21.15.5:0/859179818 >> v1:172.21.15.5:6789/0 conn(0x7f53a0157420 legacy=0x7f53a0070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a5746700 1 -- 172.21.15.5:0/859179818 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a0159db0 con 0x7f53a00cb8c0 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a5746700 1 --2- 172.21.15.5:0/859179818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00cb8c0 0x7f53a00b45e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5394009840 tx=0x7f5394009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf783c2f1fca599 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a4f45700 1 -- 172.21.15.5:0/859179818 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400c880 con 0x7f53a00cb8c0 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a4f45700 1 -- 172.21.15.5:0/859179818 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400c9e0 con 0x7f53a00cb8c0 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.693+0000 7f53a4f45700 1 -- 172.21.15.5:0/859179818 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400cd70 con 0x7f53a00cb8c0 2024-05-08T23:26:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.694+0000 7f53a81ab700 1 -- 172.21.15.5:0/859179818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00cb8c0 msgr2=0x7f53a00b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.694+0000 7f53a81ab700 1 --2- 172.21.15.5:0/859179818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00cb8c0 0x7f53a00b45e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5394009840 tx=0x7f5394009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.694+0000 7f53a81ab700 1 -- 172.21.15.5:0/859179818 shutdown_connections 2024-05-08T23:26:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.694+0000 7f53a81ab700 1 --2- 172.21.15.5:0/859179818 >> v2:172.21.15.49:3300/0 conn(0x7f53a00716b0 0x7f53a006d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.694+0000 7f53a81ab700 1 --2- 172.21.15.5:0/859179818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00cb8c0 0x7f53a00b45e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.694+0000 7f53a81ab700 1 -- 172.21.15.5:0/859179818 >> 172.21.15.5:0/859179818 conn(0x7f53a006ad80 msgr2=0x7f53a006b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.694+0000 7f53a81ab700 1 -- 172.21.15.5:0/859179818 shutdown_connections 2024-05-08T23:26:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.694+0000 7f53a81ab700 1 -- 172.21.15.5:0/859179818 wait complete. 2024-05-08T23:26:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.695+0000 7f53a81ab700 1 Processor -- start 2024-05-08T23:26:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.695+0000 7f53a81ab700 1 -- start start 2024-05-08T23:26:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a81ab700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00716b0 0x7f53a0140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a81ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a00cb8c0 0x7f53a0141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a81ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a0157420 0x7f53a00af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a81ab700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a014eb00 con 0x7f53a0157420 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a81ab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00b1f40 con 0x7f53a00cb8c0 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a81ab700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a014ed50 con 0x7f53a00716b0 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a6748700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a0157420 0x7f53a00af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a6748700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a0157420 0x7f53a00af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50602/0 (socket says 172.21.15.5:50602) 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.696+0000 7f53a6748700 1 -- 172.21.15.5:0/1866764965 learned_addr learned my addr 172.21.15.5:0/1866764965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a5746700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a00cb8c0 0x7f53a0141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a5f47700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00716b0 0x7f53a0140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a6748700 1 -- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00716b0 msgr2=0x7f53a0140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a6748700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00716b0 0x7f53a0140ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a6748700 1 -- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a00cb8c0 msgr2=0x7f53a0141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a6748700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a00cb8c0 0x7f53a0141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a6748700 1 -- 172.21.15.5:0/1866764965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940094a0 con 0x7f53a0157420 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a6748700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a0157420 0x7f53a00af800 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f539800efc0 tx=0x7f539800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53927fc700 1 -- 172.21.15.5:0/1866764965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398009ca0 con 0x7f53a0157420 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53927fc700 1 -- 172.21.15.5:0/1866764965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539800f040 con 0x7f53a0157420 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a81ab700 1 -- 172.21.15.5:0/1866764965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a014f030 con 0x7f53a0157420 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53927fc700 1 -- 172.21.15.5:0/1866764965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398014480 con 0x7f53a0157420 2024-05-08T23:26:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.697+0000 7f53a81ab700 1 -- 172.21.15.5:0/1866764965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a01418a0 con 0x7f53a0157420 2024-05-08T23:26:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.699+0000 7f53927fc700 1 -- 172.21.15.5:0/1866764965 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f53980145e0 con 0x7f53a0157420 2024-05-08T23:26:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.700+0000 7f53927fc700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f538406ac90 0x7f538406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.700+0000 7f53927fc700 1 -- 172.21.15.5:0/1866764965 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f539808a020 con 0x7f53a0157420 2024-05-08T23:26:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.700+0000 7f53a5f47700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f538406ac90 0x7f538406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.700+0000 7f53a5f47700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f538406ac90 0x7f538406d150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f539c001270 tx=0x7f539c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.702+0000 7f53b4365b80 1 Processor -- start 2024-05-08T23:26:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.702+0000 7f53b4365b80 1 -- start start 2024-05-08T23:26:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.702+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0db750 0x55f72e0b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.702+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 0x55f72e0e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53b4365b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72e00f100 con 0x55f72e0b6320 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53b4365b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72e00cac0 con 0x55f72e0e2d80 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53b4365b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f72e0083b0 con 0x55f72e0db750 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50616/0 (socket says 172.21.15.5:50616) 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 -- 172.21.15.5:0/395973265 learned_addr learned my addr 172.21.15.5:0/395973265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5f47700 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0db750 0x55f72e0b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a6748700 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 0x55f72e0e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 -- 172.21.15.5:0/395973265 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0db750 msgr2=0x55f72e0b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0db750 0x55f72e0b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 -- 172.21.15.5:0/395973265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 msgr2=0x55f72e0e3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 0x55f72e0e3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 -- 172.21.15.5:0/395973265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f72de89360 con 0x55f72e0b6320 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53a5746700 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0b9050 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5394009b90 tx=0x7f5394011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286594cea8835bed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53837fe700 1 -- 172.21.15.5:0/395973265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539401e070 con 0x55f72e0b6320 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53837fe700 1 -- 172.21.15.5:0/395973265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400c680 con 0x55f72e0b6320 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.703+0000 7f53837fe700 1 -- 172.21.15.5:0/395973265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400cd50 con 0x55f72e0b6320 2024-05-08T23:26:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 -- 172.21.15.5:0/395973265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 msgr2=0x55f72e0b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0b9050 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5394009b90 tx=0x7f5394011000 comp rx=0 tx=0).stop 2024-05-08T23:26:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 -- 172.21.15.5:0/395973265 shutdown_connections 2024-05-08T23:26:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 0x55f72e0e3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0db750 0x55f72e0b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 --2- 172.21.15.5:0/395973265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0b9050 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 -- 172.21.15.5:0/395973265 >> 172.21.15.5:0/395973265 conn(0x55f72e0161c0 msgr2=0x55f72e0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 -- 172.21.15.5:0/395973265 shutdown_connections 2024-05-08T23:26:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.704+0000 7f53b4365b80 1 -- 172.21.15.5:0/395973265 wait complete. 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.705+0000 7f53b4365b80 1 -- start start 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.705+0000 7f53b4365b80 1 Processor -- start 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e10da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0db750 0x55f72e101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 0x55f72e1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53b4365b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72dfa4510 con 0x55f72e0db750 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53b4365b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72dfa41b0 con 0x55f72e0e2d80 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53b4365b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f72de8b650 con 0x55f72e0b6320 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53b4365b80 10 client.0 _ll_register_callbacks cb 0x55f72dfadae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5746700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0db750 0x55f72e101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5746700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0db750 0x55f72e101eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50628/0 (socket says 172.21.15.5:50628) 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5746700 1 -- 172.21.15.5:0/4181821419 learned_addr learned my addr 172.21.15.5:0/4181821419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5f47700 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e10da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a6748700 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 0x55f72e1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5746700 1 -- 172.21.15.5:0/4181821419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 msgr2=0x55f72e10da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5746700 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e10da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5746700 1 -- 172.21.15.5:0/4181821419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 msgr2=0x55f72e1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5746700 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 0x55f72e1007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.706+0000 7f53a5746700 1 -- 172.21.15.5:0/4181821419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a0159db0 con 0x55f72e0db750 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53a5746700 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0db750 0x55f72e101eb0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5394009840 tx=0x7f53940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ca57764b3ffa58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f5380ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f5380ff9700 1 -- 172.21.15.5:0/4181821419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539401e040 con 0x55f72e0db750 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f5380ff9700 1 -- 172.21.15.5:0/4181821419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f539400c7f0 con 0x55f72e0db750 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 1 -- 172.21.15.5:0/4181821419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f72df61220 con 0x55f72e0db750 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f5380ff9700 1 -- 172.21.15.5:0/4181821419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5394006ab0 con 0x55f72e0db750 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 10 client.7428 fetch_fsmap learned FSMap version 71 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 1 -- 172.21.15.5:0/4181821419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55f72de35b10 con 0x55f72e0db750 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f5380ff9700 1 -- 172.21.15.5:0/4181821419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f72df61220 con 0x55f72e0db750 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f5380ff9700 1 -- 172.21.15.5:0/4181821419 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 71) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f5394016020 con 0x55f72e0db750 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 10 client.7428 fetch_fsmap finished waiting for FSMap version 71 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 10 client.7428 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 20 client.7428 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55f72e0ff200 0x55f72e0ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 4 client.7428 mds_command: new command op to 7179 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53b4365b80 1 -- 172.21.15.5:0/4181821419 --> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f72e087790 con 0x55f72e0ff200 2024-05-08T23:26:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.707+0000 7f53a5f47700 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55f72e0ff200 0x55f72e0ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.708+0000 7f5380ff9700 10 client.7428 ms_handle_connect on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.708+0000 7f53a5f47700 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55f72e0ff200 0x55f72e0ff5e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d790f6557e61c9 server_cookie=13039ccd5848d10f in_seq=0 out_seq=0 2024-05-08T23:26:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.708+0000 7f5380ff9700 1 -- 172.21.15.5:0/4181821419 <== mds.0 v2:172.21.15.5:6832/690633393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f72e087790 con 0x55f72e0ff200 2024-05-08T23:26:20.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.708+0000 7f5380ff9700 10 client.7428 handle_command_reply: tid=0 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.708+0000 7f53b4365b80 1 client.7428 shutdown 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 -- 172.21.15.5:0/4181821419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0db750 msgr2=0x55f72e101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0db750 0x55f72e101eb0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5394009840 tx=0x7f53940047c0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 -- 172.21.15.5:0/4181821419 shutdown_connections 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55f72e0ff200 0x55f72e0ff5e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0e2d80 0x55f72e1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e10da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f5380ff9700 0 client.7428 ms_handle_reset on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 --2- 172.21.15.5:0/4181821419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0db750 0x55f72e101eb0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 -- 172.21.15.5:0/4181821419 >> 172.21.15.5:0/4181821419 conn(0x55f72e0161c0 msgr2=0x55f72e0e3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 -- 172.21.15.5:0/4181821419 shutdown_connections 2024-05-08T23:26:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 1 -- 172.21.15.5:0/4181821419 wait complete. 2024-05-08T23:26:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.709+0000 7f53b4365b80 20 client.7428 trim_cache size 0 max 16384 2024-05-08T23:26:20.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.728+0000 7f53b4365b80 1 Processor -- start 2024-05-08T23:26:20.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.728+0000 7f53b4365b80 1 -- start start 2024-05-08T23:26:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 0x55f72e0ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53b4365b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72e0e3160 con 0x55f72e0b6320 2024-05-08T23:26:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53b4365b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72e0e07e0 con 0x55f72e0db750 2024-05-08T23:26:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53b4365b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f72e095010 con 0x55f72e0e2d80 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53a5f47700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53a5f47700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0e5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50634/0 (socket says 172.21.15.5:50634) 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.729+0000 7f53a5f47700 1 -- 172.21.15.5:0/1284887104 learned_addr learned my addr 172.21.15.5:0/1284887104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f53a5f47700 1 -- 172.21.15.5:0/1284887104 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0e2d80 msgr2=0x55f72e0ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f53a5f47700 1 --2- 172.21.15.5:0/1284887104 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f53a5f47700 1 -- 172.21.15.5:0/1284887104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 msgr2=0x55f72e0ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f53a5f47700 1 --2- 172.21.15.5:0/1284887104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 0x55f72e0ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f53a5f47700 1 -- 172.21.15.5:0/1284887104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f72dfce510 con 0x55f72e0b6320 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f53a5f47700 1 --2- 172.21.15.5:0/1284887104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0e5ab0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f539c000f60 tx=0x7f539c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4835dc65ef5b3827 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f5383fff700 1 -- 172.21.15.5:0/1284887104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c007560 con 0x55f72e0b6320 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f5383fff700 1 -- 172.21.15.5:0/1284887104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c0076c0 con 0x55f72e0b6320 2024-05-08T23:26:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.730+0000 7f5383fff700 1 -- 172.21.15.5:0/1284887104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c0262b0 con 0x55f72e0b6320 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 -- 172.21.15.5:0/1284887104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 msgr2=0x55f72e0e5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 --2- 172.21.15.5:0/1284887104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0e5ab0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f539c000f60 tx=0x7f539c012000 comp rx=0 tx=0).stop 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 -- 172.21.15.5:0/1284887104 shutdown_connections 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 --2- 172.21.15.5:0/1284887104 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 --2- 172.21.15.5:0/1284887104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0b6320 0x55f72e0e5ab0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 --2- 172.21.15.5:0/1284887104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 0x55f72e0ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 -- 172.21.15.5:0/1284887104 >> 172.21.15.5:0/1284887104 conn(0x55f72e0161c0 msgr2=0x55f72e0af190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 -- 172.21.15.5:0/1284887104 shutdown_connections 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 -- 172.21.15.5:0/1284887104 wait complete. 2024-05-08T23:26:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 -- start start 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.731+0000 7f53b4365b80 1 Processor -- start 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e0fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 0x55f72e0ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53b4365b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53b4365b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72e16c6d0 con 0x55f72e0e2d80 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53b4365b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72e0b35c0 con 0x55f72e0db750 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53b4365b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f72e0b38a0 con 0x55f72e0b6320 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53b4365b80 10 client.0 _ll_register_callbacks cb 0x55f72dfadae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53a6748700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.732+0000 7f53a6748700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50642/0 (socket says 172.21.15.5:50642) 2024-05-08T23:26:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a6748700 1 -- 172.21.15.5:0/3534994195 learned_addr learned my addr 172.21.15.5:0/3534994195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a5746700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 0x55f72e0ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a5f47700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e0fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a6748700 1 -- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 msgr2=0x55f72e0fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a6748700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e0fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a6748700 1 -- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 msgr2=0x55f72e0ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a6748700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 0x55f72e0ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a6748700 1 -- 172.21.15.5:0/3534994195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c00f040 con 0x55f72e0e2d80 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a5f47700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e0fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a5746700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 0x55f72e0ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53a6748700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f539800ef90 tx=0x7f5398058f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569641386ad0233f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f5380ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f5380ff9700 1 -- 172.21.15.5:0/3534994195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53980583c0 con 0x55f72e0e2d80 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f5380ff9700 1 -- 172.21.15.5:0/3534994195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5398058520 con 0x55f72e0e2d80 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53b4365b80 1 -- 172.21.15.5:0/3534994195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f539400c680 con 0x55f72e0e2d80 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f5380ff9700 1 -- 172.21.15.5:0/3534994195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5398088170 con 0x55f72e0e2d80 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53b4365b80 10 client.7434 fetch_fsmap learned FSMap version 71 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.733+0000 7f53b4365b80 1 -- 172.21.15.5:0/3534994195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55f72de8dc80 con 0x55f72e0e2d80 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f5380ff9700 1 -- 172.21.15.5:0/3534994195 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400c680 con 0x55f72e0e2d80 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f5380ff9700 1 -- 172.21.15.5:0/3534994195 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 71) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f5398060e30 con 0x55f72e0e2d80 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53b4365b80 10 client.7434 fetch_fsmap finished waiting for FSMap version 71 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53b4365b80 10 client.7434 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53b4365b80 20 client.7434 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53b4365b80 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55f72e0b6c50 0x55f72e0b7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53b4365b80 4 client.7434 mds_command: new command op to 7179 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53b4365b80 1 -- 172.21.15.5:0/3534994195 --> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f72e000e70 con 0x55f72e0b6c50 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53a5f47700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55f72e0b6c50 0x55f72e0b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f5380ff9700 10 client.7434 ms_handle_connect on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53a5f47700 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55f72e0b6c50 0x55f72e0b7030 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a248ee073ae516ad server_cookie=f10a510874768f57 in_seq=0 out_seq=0 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f5380ff9700 1 -- 172.21.15.5:0/3534994195 <== mds.0 v2:172.21.15.5:6832/690633393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f72e000e70 con 0x55f72e0b6c50 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f5380ff9700 10 client.7434 handle_command_reply: tid=0 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.734+0000 7f53b4365b80 1 client.7434 shutdown 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 -- 172.21.15.5:0/3534994195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0e2d80 msgr2=0x55f72e0ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f539800ef90 tx=0x7f5398058f80 comp rx=0 tx=0).stop 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 -- 172.21.15.5:0/3534994195 shutdown_connections 2024-05-08T23:26:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55f72e0b6c50 0x55f72e0b7030 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72e0e2d80 0x55f72e0ffb20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f72e0b6320 0x55f72e0fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f5380ff9700 0 client.7434 ms_handle_reset on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 --2- 172.21.15.5:0/3534994195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f72e0db750 0x55f72e0ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 -- 172.21.15.5:0/3534994195 >> 172.21.15.5:0/3534994195 conn(0x55f72e0161c0 msgr2=0x55f72e0b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 -- 172.21.15.5:0/3534994195 shutdown_connections 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 1 -- 172.21.15.5:0/3534994195 wait complete. 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53b4365b80 20 client.7434 trim_cache size 0 max 16384 2024-05-08T23:26:20.742 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7179, 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 16.374501325000001, 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 71, 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 72, 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 72, 2024-05-08T23:26:20.743 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 21.416378624 2024-05-08T23:26:20.744 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53a81ab700 1 -- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f538406ac90 msgr2=0x7f538406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53a81ab700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f538406ac90 0x7f538406d150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f539c001270 tx=0x7f539c009040 comp rx=0 tx=0).stop 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53a81ab700 1 -- 172.21.15.5:0/1866764965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a0157420 msgr2=0x7f53a00af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.735+0000 7f53a81ab700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a0157420 0x7f53a00af800 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f539800efc0 tx=0x7f539800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.736+0000 7f53a81ab700 1 -- 172.21.15.5:0/1866764965 shutdown_connections 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.736+0000 7f53a81ab700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f538406ac90 0x7f538406d150 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.736+0000 7f53a81ab700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a0157420 0x7f53a00af800 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.736+0000 7f53a81ab700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f53a00716b0 0x7f53a0140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.736+0000 7f53a81ab700 1 --2- 172.21.15.5:0/1866764965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a00cb8c0 0x7f53a0141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.736+0000 7f53a81ab700 1 -- 172.21.15.5:0/1866764965 >> 172.21.15.5:0/1866764965 conn(0x7f53a006ad80 msgr2=0x7f53a014d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.736+0000 7f53a81ab700 1 -- 172.21.15.5:0/1866764965 shutdown_connections 2024-05-08T23:26:20.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.736+0000 7f53a81ab700 1 -- 172.21.15.5:0/1866764965 wait complete. 2024-05-08T23:26:20.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.885+0000 7f72f9757700 1 Processor -- start 2024-05-08T23:26:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.885+0000 7f72f9757700 1 -- start start 2024-05-08T23:26:20.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.886+0000 7f72f9757700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72f406b140 0x7f72f40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.886+0000 7f72f9757700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72f40cf760 0x7f72f40cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.886+0000 7f72f9757700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f40cdbb0 con 0x7f72f406a960 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.886+0000 7f72f9757700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f40cd310 con 0x7f72f406b140 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.886+0000 7f72f9757700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f40cd5f0 con 0x7f72f40cf760 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.886+0000 7f72f2ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72f406b140 0x7f72f40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.886+0000 7f72f2ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72f406b140 0x7f72f40ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50658/0 (socket says 172.21.15.5:50658) 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.886+0000 7f72f2ffd700 1 -- 172.21.15.5:0/2732952323 learned_addr learned my addr 172.21.15.5:0/2732952323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f37fe700 1 --2- 172.21.15.5:0/2732952323 >> v2:172.21.15.49:3300/0 conn(0x7f72f40cf760 0x7f72f40cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f2ffd700 1 -- 172.21.15.5:0/2732952323 >> v2:172.21.15.49:3300/0 conn(0x7f72f40cf760 msgr2=0x7f72f40cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f2ffd700 1 --2- 172.21.15.5:0/2732952323 >> v2:172.21.15.49:3300/0 conn(0x7f72f40cf760 0x7f72f40cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f2ffd700 1 -- 172.21.15.5:0/2732952323 >> v1:172.21.15.49:6789/0 conn(0x7f72f406a960 legacy=0x7f72f415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f2ffd700 1 -- 172.21.15.5:0/2732952323 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40cd8d0 con 0x7f72f406b140 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f2ffd700 1 --2- 172.21.15.5:0/2732952323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406b140 0x7f72f40ccd30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f72f40cab50 tx=0x7f72e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa4ac1de9ad7cc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f1ffb700 1 -- 172.21.15.5:0/2732952323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400b870 con 0x7f72f406b140 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f1ffb700 1 -- 172.21.15.5:0/2732952323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e400b9d0 con 0x7f72f406b140 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.887+0000 7f72f1ffb700 1 -- 172.21.15.5:0/2732952323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400bd60 con 0x7f72f406b140 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.888+0000 7f72f9757700 1 -- 172.21.15.5:0/2732952323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406b140 msgr2=0x7f72f40ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.888+0000 7f72f9757700 1 --2- 172.21.15.5:0/2732952323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406b140 0x7f72f40ccd30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f72f40cab50 tx=0x7f72e4009bd0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.888+0000 7f72f9757700 1 -- 172.21.15.5:0/2732952323 shutdown_connections 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.888+0000 7f72f9757700 1 --2- 172.21.15.5:0/2732952323 >> v2:172.21.15.49:3300/0 conn(0x7f72f40cf760 0x7f72f40cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.888+0000 7f72f9757700 1 --2- 172.21.15.5:0/2732952323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406b140 0x7f72f40ccd30 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.888+0000 7f72f9757700 1 -- 172.21.15.5:0/2732952323 >> 172.21.15.5:0/2732952323 conn(0x7f72f40d0320 msgr2=0x7f72f40d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.888+0000 7f72f9757700 1 -- 172.21.15.5:0/2732952323 shutdown_connections 2024-05-08T23:26:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.888+0000 7f72f9757700 1 -- 172.21.15.5:0/2732952323 wait complete. 2024-05-08T23:26:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.890+0000 7f72f9757700 1 Processor -- start 2024-05-08T23:26:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.890+0000 7f72f9757700 1 -- start start 2024-05-08T23:26:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f9757700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406a960 0x7f72f41f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f9757700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72f406b140 0x7f72f414e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f9757700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f40cf760 0x7f72f41f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f9757700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f41f1270 con 0x7f72f406a960 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f9757700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40c8e50 con 0x7f72f40cf760 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f9757700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40c90a0 con 0x7f72f406b140 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406a960 0x7f72f41f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406a960 0x7f72f41f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50662/0 (socket says 172.21.15.5:50662) 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 -- 172.21.15.5:0/3814377649 learned_addr learned my addr 172.21.15.5:0/3814377649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 -- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72f406b140 msgr2=0x7f72f414e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72f406b140 0x7f72f414e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 -- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f40cf760 msgr2=0x7f72f41f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f40cf760 0x7f72f41f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 -- 172.21.15.5:0/3814377649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4009550 con 0x7f72f406a960 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.891+0000 7f72f2ffd700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406a960 0x7f72f41f0d30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000c00 tx=0x7f72e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.892+0000 7f72db7fe700 1 -- 172.21.15.5:0/3814377649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4004540 con 0x7f72f406a960 2024-05-08T23:26:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.892+0000 7f72db7fe700 1 -- 172.21.15.5:0/3814377649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e40036a0 con 0x7f72f406a960 2024-05-08T23:26:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.892+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f40c3cc0 con 0x7f72f406a960 2024-05-08T23:26:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.892+0000 7f72db7fe700 1 -- 172.21.15.5:0/3814377649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4003a30 con 0x7f72f406a960 2024-05-08T23:26:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.892+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f40c9660 con 0x7f72f406a960 2024-05-08T23:26:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.893+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f4068000 con 0x7f72f406a960 2024-05-08T23:26:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.893+0000 7f72db7fe700 1 -- 172.21.15.5:0/3814377649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f72e4021070 con 0x7f72f406a960 2024-05-08T23:26:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.894+0000 7f72db7fe700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72e006ac90 0x7f72e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.894+0000 7f72db7fe700 1 -- 172.21.15.5:0/3814377649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72e408aa60 con 0x7f72f406a960 2024-05-08T23:26:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.894+0000 7f72f27fc700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72e006ac90 0x7f72e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.895+0000 7f72f27fc700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72e006ac90 0x7f72e006d150 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f72e8000f60 tx=0x7f72e8007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:20.899+0000 7f72db7fe700 1 -- 172.21.15.5:0/3814377649 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f72e4055cd0 con 0x7f72f406a960 2024-05-08T23:26:21.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.166+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72f40b62a0 con 0x7f72f406a960 2024-05-08T23:26:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.167+0000 7f72db7fe700 1 -- 172.21.15.5:0/3814377649 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4755 (secure 0 0 0) 0x7f72e4015020 con 0x7f72f406a960 2024-05-08T23:26:21.167 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:21.167 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:19.407425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7179,"mds_1":6790,"mds_2":6787},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6787":{"gid":6787,"name":"d","rank":2,"incarnation":70,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6790":{"gid":6790,"name":"b","rank":1,"incarnation":68,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72e006ac90 msgr2=0x7f72e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72e006ac90 0x7f72e006d150 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f72e8000f60 tx=0x7f72e8007a40 comp rx=0 tx=0).stop 2024-05-08T23:26:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406a960 msgr2=0x7f72f41f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406a960 0x7f72f41f0d30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000c00 tx=0x7f72e4004750 comp rx=0 tx=0).stop 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 shutdown_connections 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72e006ac90 0x7f72e006d150 secure :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f72e8000f60 tx=0x7f72e8007a40 comp rx=0 tx=0).stop 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72f40cf760 0x7f72f41f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f406a960 0x7f72f41f0d30 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 --2- 172.21.15.5:0/3814377649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72f406b140 0x7f72f414e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 >> 172.21.15.5:0/3814377649 conn(0x7f72f40d0320 msgr2=0x7f72f4155670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 shutdown_connections 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:21.169+0000 7f72f9757700 1 -- 172.21.15.5:0/3814377649 wait complete. 2024-05-08T23:26:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 71 2024-05-08T23:26:21.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:21.183 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:26:21.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:21.211 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:26:21.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:21.277 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-05-08T23:26:21.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:21.341 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-05-08T23:26:21.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:21.403 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-05-08T23:26:21.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:26:21.673 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:26:21.674 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:26:21.674 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:26:23.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:26:23.838 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:26:23.839 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:26:23.839 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:26:25.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:26:26.004 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:26:26.005 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:26:26.005 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:26:28.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:26:28.171 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:26:28.171 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:26:28.171 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:26:30.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:26:30.337 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-05-08T23:26:30.337 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:26:30.337 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:26:30.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-08T23:26:30.496 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:26:30.497 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:26:30.497 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:26:32.498 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-08T23:26:32.655 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:26:32.655 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:26:32.655 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:26:34.656 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-08T23:26:34.814 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-08T23:26:34.814 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:26:34.815 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:26:34.815 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-08T23:26:34.959 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:26:34.959 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:26:34.960 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:26:34.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-05-08T23:26:35.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.109+0000 7fc3b50ef700 1 Processor -- start 2024-05-08T23:26:35.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.109+0000 7fc3b50ef700 1 -- start start 2024-05-08T23:26:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.109+0000 7fc3b50ef700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b00d1910 0x7fc3b00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3b50ef700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b00cf560 con 0x7fc3b015b4e0 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3b50ef700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b00ccff0 con 0x7fc3b015b8c0 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3b50ef700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00cefe0 con 0x7fc3b00d1910 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3aed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc3b015b4e0 0x7fc3b00b1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53332/0 (socket says 172.21.15.5:53332) 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3aed9d700 1 -- 172.21.15.5:0/1667589459 learned_addr learned my addr 172.21.15.5:0/1667589459 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3af59e700 1 --2- 172.21.15.5:0/1667589459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b00d1910 0x7fc3b00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3add9b700 1 -- 172.21.15.5:0/1667589459 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266749523 0 0) 0x7fc3b00cf560 con 0x7fc3b015b4e0 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3add9b700 1 -- 172.21.15.5:0/1667589459 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3980035b0 con 0x7fc3b015b4e0 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3add9b700 1 -- 172.21.15.5:0/1667589459 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302593996 0 0) 0x7fc3b00ccff0 con 0x7fc3b015b8c0 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3add9b700 1 -- 172.21.15.5:0/1667589459 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3b00cf560 con 0x7fc3b015b8c0 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3af59e700 1 -- 172.21.15.5:0/1667589459 >> v1:172.21.15.49:6789/0 conn(0x7fc3b015b8c0 legacy=0x7fc3b00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3af59e700 1 -- 172.21.15.5:0/1667589459 >> v1:172.21.15.5:6789/0 conn(0x7fc3b015b4e0 legacy=0x7fc3b00b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3af59e700 1 -- 172.21.15.5:0/1667589459 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b00cd2d0 con 0x7fc3b00d1910 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.110+0000 7fc3af59e700 1 --2- 172.21.15.5:0/1667589459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b00d1910 0x7fc3b00cc2c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009dc0 tx=0x7fc3a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=543decee0a25a3ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3add9b700 1 -- 172.21.15.5:0/1667589459 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800e070 con 0x7fc3b00d1910 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3add9b700 1 -- 172.21.15.5:0/1667589459 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a800bb20 con 0x7fc3b00d1910 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3add9b700 1 -- 172.21.15.5:0/1667589459 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8012430 con 0x7fc3b00d1910 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3b50ef700 1 -- 172.21.15.5:0/1667589459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b00d1910 msgr2=0x7fc3b00cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3b50ef700 1 --2- 172.21.15.5:0/1667589459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b00d1910 0x7fc3b00cc2c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009dc0 tx=0x7fc3a800b330 comp rx=0 tx=0).stop 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3b50ef700 1 -- 172.21.15.5:0/1667589459 shutdown_connections 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3b50ef700 1 --2- 172.21.15.5:0/1667589459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b00d1910 0x7fc3b00cc2c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3b50ef700 1 -- 172.21.15.5:0/1667589459 >> 172.21.15.5:0/1667589459 conn(0x7fc3b008afe0 msgr2=0x7fc3b008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3b50ef700 1 -- 172.21.15.5:0/1667589459 shutdown_connections 2024-05-08T23:26:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.111+0000 7fc3b50ef700 1 -- 172.21.15.5:0/1667589459 wait complete. 2024-05-08T23:26:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.113+0000 7fc3b50ef700 1 Processor -- start 2024-05-08T23:26:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.113+0000 7fc3b50ef700 1 -- start start 2024-05-08T23:26:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3b50ef700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00d1910 0x7fc3b015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3b50ef700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b015b4e0 0x7fc3b00c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3b50ef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b015b8c0 0x7fc3b00af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3b50ef700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00c5e90 con 0x7fc3b015b8c0 2024-05-08T23:26:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3b50ef700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00c6800 con 0x7fc3b00d1910 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3b50ef700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b015c280 con 0x7fc3b015b4e0 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b015b8c0 0x7fc3b00af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b015b8c0 0x7fc3b00af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45968/0 (socket says 172.21.15.5:45968) 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 -- 172.21.15.5:0/4177984224 learned_addr learned my addr 172.21.15.5:0/4177984224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b015b4e0 0x7fc3b00c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00d1910 0x7fc3b015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 -- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b015b4e0 msgr2=0x7fc3b00c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b015b4e0 0x7fc3b00c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 -- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00d1910 msgr2=0x7fc3b015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00d1910 0x7fc3b015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 -- 172.21.15.5:0/4177984224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8009a70 con 0x7fc3b015b8c0 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00d1910 0x7fc3b015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b015b4e0 0x7fc3b00c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.114+0000 7fc3af59e700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b015b8c0 0x7fc3b00af7f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00c4f10 tx=0x7fc3a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.115+0000 7fc3977fe700 1 -- 172.21.15.5:0/4177984224 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8012430 con 0x7fc3b015b8c0 2024-05-08T23:26:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.115+0000 7fc3977fe700 1 -- 172.21.15.5:0/4177984224 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8020070 con 0x7fc3b015b8c0 2024-05-08T23:26:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.115+0000 7fc3b50ef700 1 -- 172.21.15.5:0/4177984224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b015c560 con 0x7fc3b015b8c0 2024-05-08T23:26:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.115+0000 7fc3977fe700 1 -- 172.21.15.5:0/4177984224 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8004280 con 0x7fc3b015b8c0 2024-05-08T23:26:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.115+0000 7fc3b50ef700 1 -- 172.21.15.5:0/4177984224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b015cb20 con 0x7fc3b015b8c0 2024-05-08T23:26:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.116+0000 7fc3977fe700 1 -- 172.21.15.5:0/4177984224 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc3a80043e0 con 0x7fc3b015b8c0 2024-05-08T23:26:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.117+0000 7fc3977fe700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc39c06ac90 0x7fc39c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.117+0000 7fc3977fe700 1 -- 172.21.15.5:0/4177984224 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3a8022070 con 0x7fc3b015b8c0 2024-05-08T23:26:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.117+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc39c06ac90 0x7fc39c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.117+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc39c06ac90 0x7fc39c06d150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0000f60 tx=0x7fc3a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.119+0000 7fc3c12a9b80 1 Processor -- start 2024-05-08T23:26:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.119+0000 7fc3c12a9b80 1 -- start start 2024-05-08T23:26:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9b630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b8d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9ab9150 con 0x55dfc9b60370 2024-05-08T23:26:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9ab6b10 con 0x55dfc9b857a0 2024-05-08T23:26:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9ab2400 con 0x55dfc9b8cdd0 2024-05-08T23:26:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9b630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9b630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45974/0 (socket says 172.21.15.5:45974) 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3ae59c700 1 -- 172.21.15.5:0/2534125211 learned_addr learned my addr 172.21.15.5:0/2534125211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3af59e700 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b8d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3ae59c700 1 -- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 msgr2=0x55dfc9b8d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b8d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3ae59c700 1 -- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 msgr2=0x55dfc9b5e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3ae59c700 1 -- 172.21.15.5:0/2534125211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfc99333b0 con 0x55dfc9b60370 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.120+0000 7fc3af59e700 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b8d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.121+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9b630a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40016c0 tx=0x7fc3a40019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4a9bf4ef60db23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.121+0000 7fc394ff9700 1 -- 172.21.15.5:0/2534125211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40052e0 con 0x55dfc9b60370 2024-05-08T23:26:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.121+0000 7fc394ff9700 1 -- 172.21.15.5:0/2534125211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a4005440 con 0x55dfc9b60370 2024-05-08T23:26:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.121+0000 7fc394ff9700 1 -- 172.21.15.5:0/2534125211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40038a0 con 0x55dfc9b60370 2024-05-08T23:26:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.121+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2534125211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 msgr2=0x55dfc9b630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.121+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9b630a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40016c0 tx=0x7fc3a40019d0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2534125211 shutdown_connections 2024-05-08T23:26:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b8d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2534125211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9b630a0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2534125211 >> 172.21.15.5:0/2534125211 conn(0x55dfc9ac0210 msgr2=0x55dfc9ba7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2534125211 shutdown_connections 2024-05-08T23:26:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2534125211 wait complete. 2024-05-08T23:26:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 -- start start 2024-05-08T23:26:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.122+0000 7fc3c12a9b80 1 Processor -- start 2024-05-08T23:26:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9bab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b857a0 0x55dfc9ba7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b8cdd0 0x55dfc9b82e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9a4e560 con 0x55dfc9b60370 2024-05-08T23:26:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9a4e200 con 0x55dfc9b8cdd0 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfc99356a0 con 0x55dfc9b857a0 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3c12a9b80 10 client.0 _ll_register_callbacks cb 0x55dfc9a57b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3aed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9bab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3aed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9bab9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45982/0 (socket says 172.21.15.5:45982) 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3aed9d700 1 -- 172.21.15.5:0/2638297514 learned_addr learned my addr 172.21.15.5:0/2638297514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3af59e700 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b8cdd0 0x55dfc9b82e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.123+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b857a0 0x55dfc9ba7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3aed9d700 1 -- 172.21.15.5:0/2638297514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b857a0 msgr2=0x55dfc9ba7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b857a0 0x55dfc9ba7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3aed9d700 1 -- 172.21.15.5:0/2638297514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b8cdd0 msgr2=0x55dfc9b82e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b8cdd0 0x55dfc9b82e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3aed9d700 1 -- 172.21.15.5:0/2638297514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a400f040 con 0x55dfc9b60370 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9bab9a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0007100 tx=0x7fc3a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834d33f4f5e9b60c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc38dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc38dffb700 1 -- 172.21.15.5:0/2638297514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000e830 con 0x55dfc9b60370 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc38dffb700 1 -- 172.21.15.5:0/2638297514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3a0022a90 con 0x55dfc9b60370 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2638297514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfc9a0b270 con 0x55dfc9b60370 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc38dffb700 1 -- 172.21.15.5:0/2638297514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3a0022c50 con 0x55dfc9b60370 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3c12a9b80 10 client.7452 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2638297514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55dfc98dfb60 con 0x55dfc9b60370 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.124+0000 7fc38dffb700 1 -- 172.21.15.5:0/2638297514 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dfc9a0b270 con 0x55dfc9b60370 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc38dffb700 1 -- 172.21.15.5:0/2638297514 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fc3a000e990 con 0x55dfc9b60370 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc3c12a9b80 10 client.7452 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc3c12a9b80 10 client.7452 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc3c12a9b80 20 client.7452 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55dfc9ba5850 0x55dfc9ba5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc3c12a9b80 4 client.7452 mds_command: new command op to 7179 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2638297514 --> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfc9b317e0 con 0x55dfc9ba5850 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55dfc9ba5850 0x55dfc9ba5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55dfc9ba5850 0x55dfc9ba5c60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3898dba494c2cc3 server_cookie=2757bbe74484316d in_seq=0 out_seq=0 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.125+0000 7fc38dffb700 10 client.7452 ms_handle_connect on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc38dffb700 1 -- 172.21.15.5:0/2638297514 <== mds.0 v2:172.21.15.5:6832/690633393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfc9b317e0 con 0x55dfc9ba5850 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc38dffb700 10 client.7452 handle_command_reply: tid=0 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 client.7452 shutdown 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2638297514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 msgr2=0x55dfc9bab9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9bab9a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0007100 tx=0x7fc3a0000ed0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2638297514 shutdown_connections 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55dfc9ba5850 0x55dfc9ba5c60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b8cdd0 0x55dfc9b82e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9bab9a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2638297514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b857a0 0x55dfc9ba7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc38dffb700 0 client.7452 ms_handle_reset on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2638297514 >> 172.21.15.5:0/2638297514 conn(0x55dfc9ac0210 msgr2=0x55dfc9b622e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2638297514 shutdown_connections 2024-05-08T23:26:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2638297514 wait complete. 2024-05-08T23:26:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.126+0000 7fc3c12a9b80 20 client.7452 trim_cache size 0 max 16384 2024-05-08T23:26:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.142+0000 7fc3c12a9b80 1 Processor -- start 2024-05-08T23:26:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.142+0000 7fc3c12a9b80 1 -- start start 2024-05-08T23:26:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b60370 0x55dfc9bc9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9ba5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9ba7ca0 con 0x55dfc9b857a0 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9b595f0 con 0x55dfc9b60370 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9b8c250 con 0x55dfc9b8cdd0 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3aed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b60370 0x55dfc9bc9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45986/0 (socket says 172.21.15.5:45986) 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 -- 172.21.15.5:0/1815717225 learned_addr learned my addr 172.21.15.5:0/1815717225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3af59e700 1 --2- 172.21.15.5:0/1815717225 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9ba5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 -- 172.21.15.5:0/1815717225 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 msgr2=0x55dfc9ba5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/1815717225 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9ba5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 -- 172.21.15.5:0/1815717225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b60370 msgr2=0x55dfc9bc9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/1815717225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b60370 0x55dfc9bc9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 -- 172.21.15.5:0/1815717225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfc9a78560 con 0x55dfc9b857a0 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.143+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/1815717225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5f3b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4007d10 tx=0x7fc3a400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f1aa21c1ed5ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3957fa700 1 -- 172.21.15.5:0/1815717225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40052e0 con 0x55dfc9b857a0 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3957fa700 1 -- 172.21.15.5:0/1815717225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a4025040 con 0x55dfc9b857a0 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3957fa700 1 -- 172.21.15.5:0/1815717225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4025b00 con 0x55dfc9b857a0 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/1815717225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b857a0 msgr2=0x55dfc9b5f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/1815717225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5f3b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4007d10 tx=0x7fc3a400a000 comp rx=0 tx=0).stop 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/1815717225 shutdown_connections 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/1815717225 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9ba5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/1815717225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b60370 0x55dfc9bc9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/1815717225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b857a0 0x55dfc9b5f3b0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/1815717225 >> 172.21.15.5:0/1815717225 conn(0x55dfc9ac0210 msgr2=0x55dfc9bafab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/1815717225 shutdown_connections 2024-05-08T23:26:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.144+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/1815717225 wait complete. 2024-05-08T23:26:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.145+0000 7fc3c12a9b80 1 -- start start 2024-05-08T23:26:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.145+0000 7fc3c12a9b80 1 Processor -- start 2024-05-08T23:26:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9c0ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9c14020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3c12a9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b83910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9c147e0 con 0x55dfc9b60370 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9b5fa00 con 0x55dfc9b857a0 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3c12a9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfc9b616a0 con 0x55dfc9b8cdd0 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3c12a9b80 10 client.0 _ll_register_callbacks cb 0x55dfc9a57b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9c0ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9c0ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45994/0 (socket says 172.21.15.5:45994) 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 -- 172.21.15.5:0/2791698682 learned_addr learned my addr 172.21.15.5:0/2791698682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9c14020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3af59e700 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b83910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 -- 172.21.15.5:0/2791698682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 msgr2=0x55dfc9b83910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b83910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 -- 172.21.15.5:0/2791698682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 msgr2=0x55dfc9c14020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9c14020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 -- 172.21.15.5:0/2791698682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfc99333b0 con 0x55dfc9b60370 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc3aed9d700 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9c0ab10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0001eb0 tx=0x7fc3a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d20d7dada0dccc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.146+0000 7fc38dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc38dffb700 1 -- 172.21.15.5:0/2791698682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000e440 con 0x55dfc9b60370 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc38dffb700 1 -- 172.21.15.5:0/2791698682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3a000ea40 con 0x55dfc9b60370 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2791698682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3a4005440 con 0x55dfc9b60370 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc38dffb700 1 -- 172.21.15.5:0/2791698682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3a00076f0 con 0x55dfc9b60370 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 10 client.7458 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2791698682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55dfc9937cd0 con 0x55dfc9b60370 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc38dffb700 1 -- 172.21.15.5:0/2791698682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a4005440 con 0x55dfc9b60370 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc38dffb700 1 -- 172.21.15.5:0/2791698682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fc3a0004870 con 0x55dfc9b60370 2024-05-08T23:26:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 10 client.7458 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 10 client.7458 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-08T23:26:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 20 client.7458 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55dfc9baef10 0x55dfc9baf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 4 client.7458 mds_command: new command op to 7179 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:26:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2791698682 --> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dfc9aaaec0 con 0x55dfc9baef10 2024-05-08T23:26:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.147+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55dfc9baef10 0x55dfc9baf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.148+0000 7fc38dffb700 10 client.7458 ms_handle_connect on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.148+0000 7fc3ae59c700 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55dfc9baef10 0x55dfc9baf320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784416919b3cbe2a server_cookie=3c306923abcc9172 in_seq=0 out_seq=0 2024-05-08T23:26:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.159+0000 7fc38dffb700 1 -- 172.21.15.5:0/2791698682 <== mds.0 v2:172.21.15.5:6832/690633393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dfc9aaaec0 con 0x55dfc9baef10 2024-05-08T23:26:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.159+0000 7fc38dffb700 10 client.7458 handle_command_reply: tid=0 2024-05-08T23:26:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.159+0000 7fc3c12a9b80 1 client.7458 shutdown 2024-05-08T23:26:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2791698682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 msgr2=0x55dfc9c0ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9c0ab10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0001eb0 tx=0x7fc3a0013000 comp rx=0 tx=0).stop 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2791698682 shutdown_connections 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.5:6832/690633393,v1:172.21.15.5:6834/690633393] conn(0x55dfc9baef10 0x55dfc9baf320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55dfc9b8cdd0 0x55dfc9b83910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc38dffb700 0 client.7458 ms_handle_reset on v2:172.21.15.5:6832/690633393 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfc9b60370 0x55dfc9c0ab10 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 --2- 172.21.15.5:0/2791698682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dfc9b857a0 0x55dfc9c14020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2791698682 >> 172.21.15.5:0/2791698682 conn(0x55dfc9ac0210 msgr2=0x55dfc9b62c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2791698682 shutdown_connections 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 1 -- 172.21.15.5:0/2791698682 wait complete. 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3c12a9b80 20 client.7458 trim_cache size 0 max 16384 2024-05-08T23:26:35.160 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:26:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:26:35.161 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3b50ef700 1 -- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc39c06ac90 msgr2=0x7fc39c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.160+0000 7fc3b50ef700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc39c06ac90 0x7fc39c06d150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0000f60 tx=0x7fc3a0009040 comp rx=0 tx=0).stop 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 -- 172.21.15.5:0/4177984224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b015b8c0 msgr2=0x7fc3b00af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b015b8c0 0x7fc3b00af7f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00c4f10 tx=0x7fc3a8004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 -- 172.21.15.5:0/4177984224 shutdown_connections 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc39c06ac90 0x7fc39c06d150 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b015b8c0 0x7fc3b00af7f0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00d1910 0x7fc3b015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 --2- 172.21.15.5:0/4177984224 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3b015b4e0 0x7fc3b00c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 -- 172.21.15.5:0/4177984224 >> 172.21.15.5:0/4177984224 conn(0x7fc3b008afe0 msgr2=0x7fc3b00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 -- 172.21.15.5:0/4177984224 shutdown_connections 2024-05-08T23:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.161+0000 7fc3b50ef700 1 -- 172.21.15.5:0/4177984224 wait complete. 2024-05-08T23:26:35.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:1 flush journal 2024-05-08T23:26:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.312+0000 7f13544a5700 1 Processor -- start 2024-05-08T23:26:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.313+0000 7f13544a5700 1 -- start start 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f13544a5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b0d80 0x7f134c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f13544a5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f134c0b4d50 0x7f134c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f13544a5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134c145040 con 0x7f134c06aea0 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f13544a5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134c06b330 con 0x7f134c0b4d50 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f13544a5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c0b38c0 con 0x7f134c0b0d80 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f1352a42700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f134c06aea0 0x7f134c0d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53340/0 (socket says 172.21.15.5:53340) 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f1352a42700 1 -- 172.21.15.5:0/3823014660 learned_addr learned my addr 172.21.15.5:0/3823014660 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f1352241700 1 --2- 172.21.15.5:0/3823014660 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b0d80 0x7f134c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f1351a40700 1 --2- 172.21.15.5:0/3823014660 >> v2:172.21.15.49:3300/0 conn(0x7f134c0b4d50 0x7f134c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f135123f700 1 -- 172.21.15.5:0/3823014660 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896671144 0 0) 0x7f134c145040 con 0x7f134c06aea0 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f135123f700 1 -- 172.21.15.5:0/3823014660 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13380035b0 con 0x7f134c06aea0 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f1351a40700 1 -- 172.21.15.5:0/3823014660 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b0d80 msgr2=0x7f134c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f1351a40700 1 --2- 172.21.15.5:0/3823014660 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b0d80 0x7f134c0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f1351a40700 1 -- 172.21.15.5:0/3823014660 >> v1:172.21.15.5:6789/0 conn(0x7f134c06aea0 legacy=0x7f134c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.314+0000 7f1351a40700 1 -- 172.21.15.5:0/3823014660 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c0b16e0 con 0x7f134c0b4d50 2024-05-08T23:26:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f1351a40700 1 --2- 172.21.15.5:0/3823014660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b4d50 0x7f134c06a960 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f133c009840 tx=0x7f133c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d848a159af9eec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f135123f700 1 -- 172.21.15.5:0/3823014660 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c011830 con 0x7f134c0b4d50 2024-05-08T23:26:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f135123f700 1 -- 172.21.15.5:0/3823014660 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c011990 con 0x7f134c0b4d50 2024-05-08T23:26:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f135123f700 1 -- 172.21.15.5:0/3823014660 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c011d20 con 0x7f134c0b4d50 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f13544a5700 1 -- 172.21.15.5:0/3823014660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b4d50 msgr2=0x7f134c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f13544a5700 1 --2- 172.21.15.5:0/3823014660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b4d50 0x7f134c06a960 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f133c009840 tx=0x7f133c009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f13544a5700 1 -- 172.21.15.5:0/3823014660 shutdown_connections 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f13544a5700 1 --2- 172.21.15.5:0/3823014660 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b0d80 0x7f134c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f13544a5700 1 --2- 172.21.15.5:0/3823014660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b4d50 0x7f134c06a960 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f13544a5700 1 -- 172.21.15.5:0/3823014660 >> 172.21.15.5:0/3823014660 conn(0x7f134c099a50 msgr2=0x7f134c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f13544a5700 1 -- 172.21.15.5:0/3823014660 shutdown_connections 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.315+0000 7f13544a5700 1 -- 172.21.15.5:0/3823014660 wait complete. 2024-05-08T23:26:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.317+0000 7f13544a5700 1 Processor -- start 2024-05-08T23:26:35.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.317+0000 7f13544a5700 1 -- start start 2024-05-08T23:26:35.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f13544a5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f134c06aea0 0x7f134c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f13544a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b0d80 0x7f134c09af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f13544a5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b4d50 0x7f134c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f13544a5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c0c9bb0 con 0x7f134c06aea0 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f13544a5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c09b480 con 0x7f134c0b0d80 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f13544a5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c133e60 con 0x7f134c0b4d50 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f134c06aea0 0x7f134c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f134c06aea0 0x7f134c0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46000/0 (socket says 172.21.15.5:46000) 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 -- 172.21.15.5:0/4087121210 learned_addr learned my addr 172.21.15.5:0/4087121210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1351a40700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b0d80 0x7f134c09af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352a42700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b4d50 0x7f134c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 -- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b4d50 msgr2=0x7f134c0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b4d50 0x7f134c0b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 -- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b0d80 msgr2=0x7f134c09af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b0d80 0x7f134c09af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 -- 172.21.15.5:0/4087121210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0094a0 con 0x7f134c06aea0 2024-05-08T23:26:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1352241700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f134c06aea0 0x7f134c0c9670 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f134800e9b0 tx=0x7f134800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.318+0000 7f1342ffd700 1 -- 172.21.15.5:0/4087121210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348009710 con 0x7f134c06aea0 2024-05-08T23:26:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.319+0000 7f1342ffd700 1 -- 172.21.15.5:0/4087121210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134800ee50 con 0x7f134c06aea0 2024-05-08T23:26:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.319+0000 7f13544a5700 1 -- 172.21.15.5:0/4087121210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134c134140 con 0x7f134c06aea0 2024-05-08T23:26:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.319+0000 7f1342ffd700 1 -- 172.21.15.5:0/4087121210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134800ca50 con 0x7f134c06aea0 2024-05-08T23:26:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.319+0000 7f13544a5700 1 -- 172.21.15.5:0/4087121210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c134700 con 0x7f134c06aea0 2024-05-08T23:26:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.320+0000 7f1342ffd700 1 -- 172.21.15.5:0/4087121210 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f134800cbb0 con 0x7f134c06aea0 2024-05-08T23:26:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.321+0000 7f1342ffd700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f133006ac90 0x7f133006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.321+0000 7f1342ffd700 1 -- 172.21.15.5:0/4087121210 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1348089370 con 0x7f134c06aea0 2024-05-08T23:26:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.321+0000 7f1351a40700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f133006ac90 0x7f133006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.322+0000 7f1351a40700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f133006ac90 0x7f133006d150 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f133c000c00 tx=0x7f133c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.323+0000 7f136065fb80 1 Processor -- start 2024-05-08T23:26:35.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.323+0000 7f136065fb80 1 -- start start 2024-05-08T23:26:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f136065fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dc87a0 0x5577d1da1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f136065fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1da60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f136065fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1dd0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f136065fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d1cfc150 con 0x5577d1da3370 2024-05-08T23:26:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f136065fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d1cf9b10 con 0x5577d1dcfdd0 2024-05-08T23:26:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f136065fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577d1cf5400 con 0x5577d1dc87a0 2024-05-08T23:26:35.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1351a40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1da60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1351a40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1da60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46016/0 (socket says 172.21.15.5:46016) 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1352a42700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1dd0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1352241700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dc87a0 0x5577d1da1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1351a40700 1 -- 172.21.15.5:0/1687034989 learned_addr learned my addr 172.21.15.5:0/1687034989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1352a42700 1 -- 172.21.15.5:0/1687034989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dc87a0 msgr2=0x5577d1da1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1352a42700 1 --2- 172.21.15.5:0/1687034989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dc87a0 0x5577d1da1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1352a42700 1 -- 172.21.15.5:0/1687034989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 msgr2=0x5577d1da60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1352a42700 1 --2- 172.21.15.5:0/1687034989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1da60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.324+0000 7f1352a42700 1 -- 172.21.15.5:0/1687034989 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d1b763b0 con 0x5577d1dcfdd0 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.325+0000 7f1352a42700 1 --2- 172.21.15.5:0/1687034989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1dd0250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1344002150 tx=0x7f1344007320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e6558ba05c65fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.325+0000 7f132ffff700 1 -- 172.21.15.5:0/1687034989 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344007b50 con 0x5577d1dcfdd0 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.325+0000 7f132ffff700 1 -- 172.21.15.5:0/1687034989 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13440133f0 con 0x5577d1dcfdd0 2024-05-08T23:26:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.325+0000 7f132ffff700 1 -- 172.21.15.5:0/1687034989 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344013780 con 0x5577d1dcfdd0 2024-05-08T23:26:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 -- 172.21.15.5:0/1687034989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 msgr2=0x5577d1dd0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 --2- 172.21.15.5:0/1687034989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1dd0250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1344002150 tx=0x7f1344007320 comp rx=0 tx=0).stop 2024-05-08T23:26:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 -- 172.21.15.5:0/1687034989 shutdown_connections 2024-05-08T23:26:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 --2- 172.21.15.5:0/1687034989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1dd0250 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 --2- 172.21.15.5:0/1687034989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dc87a0 0x5577d1da1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 --2- 172.21.15.5:0/1687034989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1da60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 -- 172.21.15.5:0/1687034989 >> 172.21.15.5:0/1687034989 conn(0x5577d1d03210 msgr2=0x5577d1deadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 -- 172.21.15.5:0/1687034989 shutdown_connections 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.326+0000 7f136065fb80 1 -- 172.21.15.5:0/1687034989 wait complete. 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.327+0000 7f136065fb80 1 -- start start 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.327+0000 7f136065fb80 1 Processor -- start 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f136065fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1da3370 0x5577d1dee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f136065fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1dc7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f136065fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 0x5577d1dd2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f136065fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d1c91560 con 0x5577d1dc87a0 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f136065fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d1c91200 con 0x5577d1da3370 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f136065fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577d1b786a0 con 0x5577d1dcfdd0 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f136065fb80 10 client.0 _ll_register_callbacks cb 0x5577d1c9ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1dc7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1dc7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46028/0 (socket says 172.21.15.5:46028) 2024-05-08T23:26:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 -- 172.21.15.5:0/2423225028 learned_addr learned my addr 172.21.15.5:0/2423225028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1352241700 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1da3370 0x5577d1dee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1352a42700 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 0x5577d1dd2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 -- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 msgr2=0x5577d1dd2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 0x5577d1dd2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 -- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1da3370 msgr2=0x5577d1dee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1da3370 0x5577d1dee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 -- 172.21.15.5:0/2423225028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13380035b0 con 0x5577d1dc87a0 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1352241700 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1da3370 0x5577d1dee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f1351a40700 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1dc7640 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f133c0042c0 tx=0x7f133c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e774a3879781763 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.328+0000 7f132d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f132d7fa700 1 -- 172.21.15.5:0/2423225028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c003e40 con 0x5577d1dc87a0 2024-05-08T23:26:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f132d7fa700 1 -- 172.21.15.5:0/2423225028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f133c01f070 con 0x5577d1dc87a0 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 1 -- 172.21.15.5:0/2423225028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577d1c4e270 con 0x5577d1dc87a0 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f132d7fa700 1 -- 172.21.15.5:0/2423225028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f133c0045d0 con 0x5577d1dc87a0 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 10 client.7470 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 1 -- 172.21.15.5:0/2423225028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5577d1b22b60 con 0x5577d1dc87a0 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f132d7fa700 1 -- 172.21.15.5:0/2423225028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d1c4e270 con 0x5577d1dc87a0 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f132d7fa700 1 -- 172.21.15.5:0/2423225028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f133c01a420 con 0x5577d1dc87a0 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 10 client.7470 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 10 client.7470 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.b 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 20 client.7470 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x5577d1dc99d0 0x5577d1dd01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 4 client.7470 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.329+0000 7f136065fb80 1 -- 172.21.15.5:0/2423225028 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577d1d747e0 con 0x5577d1dc99d0 2024-05-08T23:26:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.330+0000 7f1352241700 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x5577d1dc99d0 0x5577d1dd01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.330+0000 7f132d7fa700 10 client.7470 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.330+0000 7f1352241700 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x5577d1dc99d0 0x5577d1dd01b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53bf1b2780246c6b server_cookie=9400001fc812cc9b in_seq=0 out_seq=0 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f132d7fa700 1 -- 172.21.15.5:0/2423225028 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577d1d747e0 con 0x5577d1dc99d0 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f132d7fa700 10 client.7470 handle_command_reply: tid=0 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 client.7470 shutdown 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 -- 172.21.15.5:0/2423225028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 msgr2=0x5577d1dc7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1dc7640 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f133c0042c0 tx=0x7f133c00e000 comp rx=0 tx=0).stop 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 -- 172.21.15.5:0/2423225028 shutdown_connections 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x5577d1dc99d0 0x5577d1dd01b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 0x5577d1dd2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1da3370 0x5577d1dee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f132d7fa700 0 client.7470 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 --2- 172.21.15.5:0/2423225028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1dc7640 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 -- 172.21.15.5:0/2423225028 >> 172.21.15.5:0/2423225028 conn(0x5577d1d03210 msgr2=0x5577d1da5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 -- 172.21.15.5:0/2423225028 shutdown_connections 2024-05-08T23:26:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 1 -- 172.21.15.5:0/2423225028 wait complete. 2024-05-08T23:26:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.331+0000 7f136065fb80 20 client.7470 trim_cache size 0 max 16384 2024-05-08T23:26:35.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.347+0000 7f136065fb80 1 Processor -- start 2024-05-08T23:26:35.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.347+0000 7f136065fb80 1 -- start start 2024-05-08T23:26:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f136065fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1dc5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f136065fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dc87a0 0x5577d1dd01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f136065fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 0x5577d1dd06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f136065fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d1e0cce0 con 0x5577d1da3370 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f136065fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d1da0890 con 0x5577d1dc87a0 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f136065fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577d1da4b70 con 0x5577d1dcfdd0 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352241700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1dc5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352241700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1dc5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46044/0 (socket says 172.21.15.5:46044) 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352241700 1 -- 172.21.15.5:0/4221404276 learned_addr learned my addr 172.21.15.5:0/4221404276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1351a40700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dc87a0 0x5577d1dd01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352a42700 1 --2- 172.21.15.5:0/4221404276 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 0x5577d1dd06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352241700 1 -- 172.21.15.5:0/4221404276 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 msgr2=0x5577d1dd06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352241700 1 --2- 172.21.15.5:0/4221404276 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 0x5577d1dd06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352241700 1 -- 172.21.15.5:0/4221404276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dc87a0 msgr2=0x5577d1dd01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352241700 1 --2- 172.21.15.5:0/4221404276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dc87a0 0x5577d1dd01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.348+0000 7f1352241700 1 -- 172.21.15.5:0/4221404276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d1cbb560 con 0x5577d1da3370 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f1352241700 1 --2- 172.21.15.5:0/4221404276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1dc5000 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1348054670 tx=0x7f134805e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6b9161a07e94ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f1340da4700 1 -- 172.21.15.5:0/4221404276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134805bfa0 con 0x5577d1da3370 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f1340da4700 1 -- 172.21.15.5:0/4221404276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134806f070 con 0x5577d1da3370 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f1340da4700 1 -- 172.21.15.5:0/4221404276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348060560 con 0x5577d1da3370 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 -- 172.21.15.5:0/4221404276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 msgr2=0x5577d1dc5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 --2- 172.21.15.5:0/4221404276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1dc5000 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1348054670 tx=0x7f134805e040 comp rx=0 tx=0).stop 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 -- 172.21.15.5:0/4221404276 shutdown_connections 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 --2- 172.21.15.5:0/4221404276 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1dcfdd0 0x5577d1dd06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 --2- 172.21.15.5:0/4221404276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1da3370 0x5577d1dc5000 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 --2- 172.21.15.5:0/4221404276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dc87a0 0x5577d1dd01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 -- 172.21.15.5:0/4221404276 >> 172.21.15.5:0/4221404276 conn(0x5577d1d03210 msgr2=0x5577d1dee040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 -- 172.21.15.5:0/4221404276 shutdown_connections 2024-05-08T23:26:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.349+0000 7f136065fb80 1 -- 172.21.15.5:0/4221404276 wait complete. 2024-05-08T23:26:35.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.350+0000 7f136065fb80 1 -- start start 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.350+0000 7f136065fb80 1 Processor -- start 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f136065fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1da3370 0x5577d1da3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f136065fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1d9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f136065fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1d9fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f136065fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d1da3ee0 con 0x5577d1dc87a0 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f136065fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d1da3c90 con 0x5577d1dcfdd0 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f136065fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577d1e57620 con 0x5577d1da3370 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f136065fb80 10 client.0 _ll_register_callbacks cb 0x7f133c011bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f1351a40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1d9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f1351a40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1d9f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46048/0 (socket says 172.21.15.5:46048) 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f1351a40700 1 -- 172.21.15.5:0/1703456443 learned_addr learned my addr 172.21.15.5:0/1703456443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f1352a42700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1d9fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.351+0000 7f1352241700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1da3370 0x5577d1da3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f1351a40700 1 -- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1da3370 msgr2=0x5577d1da3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f1351a40700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1da3370 0x5577d1da3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f1351a40700 1 -- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 msgr2=0x5577d1d9fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f1351a40700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1d9fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f1351a40700 1 -- 172.21.15.5:0/1703456443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c134700 con 0x5577d1dc87a0 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f1352241700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1da3370 0x5577d1da3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f1352a42700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1d9fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f1351a40700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1d9f760 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f133c011b80 tx=0x7f133c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daeda6a6e4ff1f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f132d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f132d7fa700 1 -- 172.21.15.5:0/1703456443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c01adc0 con 0x5577d1dc87a0 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f132d7fa700 1 -- 172.21.15.5:0/1703456443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f133c01f040 con 0x5577d1dc87a0 2024-05-08T23:26:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f136065fb80 1 -- 172.21.15.5:0/1703456443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13440133f0 con 0x5577d1dc87a0 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f132d7fa700 1 -- 172.21.15.5:0/1703456443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f133c0117a0 con 0x5577d1dc87a0 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f136065fb80 10 client.7476 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f136065fb80 1 -- 172.21.15.5:0/1703456443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5577d1b7acd0 con 0x5577d1dc87a0 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f132d7fa700 1 -- 172.21.15.5:0/1703456443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13440133f0 con 0x5577d1dc87a0 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f132d7fa700 1 -- 172.21.15.5:0/1703456443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f133c02f410 con 0x5577d1dc87a0 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f136065fb80 10 client.7476 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.352+0000 7f136065fb80 10 client.7476 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.b 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.353+0000 7f136065fb80 20 client.7476 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.353+0000 7f136065fb80 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x5577d1dc6ae0 0x5577d1e567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.353+0000 7f136065fb80 4 client.7476 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.353+0000 7f136065fb80 1 -- 172.21.15.5:0/1703456443 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5577d1cedec0 con 0x5577d1dc6ae0 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.353+0000 7f1352241700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x5577d1dc6ae0 0x5577d1e567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.353+0000 7f1352241700 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x5577d1dc6ae0 0x5577d1e567d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e730772aa106121 server_cookie=ebbe6ec197bccc62 in_seq=0 out_seq=0 2024-05-08T23:26:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.353+0000 7f132d7fa700 10 client.7476 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.360+0000 7f132d7fa700 1 -- 172.21.15.5:0/1703456443 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5577d1cedec0 con 0x5577d1dc6ae0 2024-05-08T23:26:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.360+0000 7f132d7fa700 10 client.7476 handle_command_reply: tid=0 2024-05-08T23:26:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.360+0000 7f136065fb80 1 client.7476 shutdown 2024-05-08T23:26:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 -- 172.21.15.5:0/1703456443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 msgr2=0x5577d1d9f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1d9f760 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f133c011b80 tx=0x7f133c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 -- 172.21.15.5:0/1703456443 shutdown_connections 2024-05-08T23:26:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x5577d1dc6ae0 0x5577d1e567d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5577d1dcfdd0 0x5577d1d9fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577d1da3370 0x5577d1da3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 --2- 172.21.15.5:0/1703456443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577d1dc87a0 0x5577d1d9f760 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f132d7fa700 0 client.7476 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 -- 172.21.15.5:0/1703456443 >> 172.21.15.5:0/1703456443 conn(0x5577d1d03210 msgr2=0x5577d1dc5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 -- 172.21.15.5:0/1703456443 shutdown_connections 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 1 -- 172.21.15.5:0/1703456443 wait complete. 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.361+0000 7f136065fb80 20 client.7476 trim_cache size 0 max 16384 2024-05-08T23:26:35.361 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:35.362 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:26:35.362 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:26:35.362 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 -- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f133006ac90 msgr2=0x7f133006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f133006ac90 0x7f133006d150 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f133c000c00 tx=0x7f133c01c040 comp rx=0 tx=0).stop 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 -- 172.21.15.5:0/4087121210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f134c06aea0 msgr2=0x7f134c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f134c06aea0 0x7f134c0c9670 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f134800e9b0 tx=0x7f134800ecc0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 -- 172.21.15.5:0/4087121210 shutdown_connections 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f133006ac90 0x7f133006d150 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f134c0b4d50 0x7f134c0b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f134c06aea0 0x7f134c0c9670 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 --2- 172.21.15.5:0/4087121210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f134c0b0d80 0x7f134c09af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 -- 172.21.15.5:0/4087121210 >> 172.21.15.5:0/4087121210 conn(0x7f134c099a50 msgr2=0x7f134c146090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 -- 172.21.15.5:0/4087121210 shutdown_connections 2024-05-08T23:26:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.362+0000 7f13544a5700 1 -- 172.21.15.5:0/4087121210 wait complete. 2024-05-08T23:26:35.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:2 flush journal 2024-05-08T23:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.514+0000 7f601610f700 1 Processor -- start 2024-05-08T23:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.514+0000 7f601610f700 1 -- start start 2024-05-08T23:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f601610f700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f601015b4e0 0x7f60100b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f601610f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60100d1910 0x7f60100cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f601610f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60100cf560 con 0x7f601015b8c0 2024-05-08T23:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f601610f700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60100cefe0 con 0x7f601015b4e0 2024-05-08T23:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f601610f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60100ccff0 con 0x7f60100d1910 2024-05-08T23:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f600effd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f601015b8c0 0x7f60100d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53352/0 (socket says 172.21.15.5:53352) 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f600effd700 1 -- 172.21.15.5:0/991171561 learned_addr learned my addr 172.21.15.5:0/991171561 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f600f7fe700 1 --2- 172.21.15.5:0/991171561 >> v2:172.21.15.49:3300/0 conn(0x7f601015b4e0 0x7f60100b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f600ffff700 1 --2- 172.21.15.5:0/991171561 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60100d1910 0x7f60100cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f600e7fc700 1 -- 172.21.15.5:0/991171561 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480218817 0 0) 0x7f60100cf560 con 0x7f601015b8c0 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.515+0000 7f600e7fc700 1 -- 172.21.15.5:0/991171561 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff80035b0 con 0x7f601015b8c0 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600ffff700 1 -- 172.21.15.5:0/991171561 >> v2:172.21.15.49:3300/0 conn(0x7f601015b4e0 msgr2=0x7f60100b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600ffff700 1 --2- 172.21.15.5:0/991171561 >> v2:172.21.15.49:3300/0 conn(0x7f601015b4e0 0x7f60100b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600ffff700 1 -- 172.21.15.5:0/991171561 >> v1:172.21.15.5:6789/0 conn(0x7f601015b8c0 legacy=0x7f60100d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600e7fc700 1 -- 172.21.15.5:0/991171561 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 906963790 0 0) 0x7f5ff80035b0 con 0x7f601015b8c0 2024-05-08T23:26:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600ffff700 1 -- 172.21.15.5:0/991171561 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60100cd2d0 con 0x7f60100d1910 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600ffff700 1 --2- 172.21.15.5:0/991171561 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60100d1910 0x7f60100cc2c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6008009d90 tx=0x7f600800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269bd5c2e76f3e45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600e7fc700 1 -- 172.21.15.5:0/991171561 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600800e070 con 0x7f60100d1910 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600e7fc700 1 -- 172.21.15.5:0/991171561 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6008003a20 con 0x7f60100d1910 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.516+0000 7f600e7fc700 1 -- 172.21.15.5:0/991171561 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004001670 con 0x7f60100d1910 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.517+0000 7f601610f700 1 -- 172.21.15.5:0/991171561 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60100d1910 msgr2=0x7f60100cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.517+0000 7f601610f700 1 --2- 172.21.15.5:0/991171561 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60100d1910 0x7f60100cc2c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6008009d90 tx=0x7f600800b330 comp rx=0 tx=0).stop 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.517+0000 7f601610f700 1 -- 172.21.15.5:0/991171561 shutdown_connections 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.517+0000 7f601610f700 1 --2- 172.21.15.5:0/991171561 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f60100d1910 0x7f60100cc2c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.517+0000 7f601610f700 1 --2- 172.21.15.5:0/991171561 >> v2:172.21.15.49:3300/0 conn(0x7f601015b4e0 0x7f60100b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.517+0000 7f601610f700 1 -- 172.21.15.5:0/991171561 >> 172.21.15.5:0/991171561 conn(0x7f601008afe0 msgr2=0x7f601008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.517+0000 7f601610f700 1 -- 172.21.15.5:0/991171561 shutdown_connections 2024-05-08T23:26:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.517+0000 7f601610f700 1 -- 172.21.15.5:0/991171561 wait complete. 2024-05-08T23:26:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.518+0000 7f601610f700 1 Processor -- start 2024-05-08T23:26:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.518+0000 7f601610f700 1 -- start start 2024-05-08T23:26:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f601610f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60100d1910 0x7f6010145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f601610f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601015b4e0 0x7f6010145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f601610f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f601015b8c0 0x7f60100ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f601610f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601015c6d0 con 0x7f601015b4e0 2024-05-08T23:26:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f601610f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6010146350 con 0x7f60100d1910 2024-05-08T23:26:35.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f601610f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60100c4cb0 con 0x7f601015b8c0 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f600effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601015b4e0 0x7f6010145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f600effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601015b4e0 0x7f6010145990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46058/0 (socket says 172.21.15.5:46058) 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f600effd700 1 -- 172.21.15.5:0/2933203244 learned_addr learned my addr 172.21.15.5:0/2933203244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f600ffff700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f601015b8c0 0x7f60100ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.519+0000 7f600f7fe700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60100d1910 0x7f6010145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f600effd700 1 -- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f601015b8c0 msgr2=0x7f60100ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f600effd700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f601015b8c0 0x7f60100ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f600effd700 1 -- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60100d1910 msgr2=0x7f6010145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f600effd700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60100d1910 0x7f6010145450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f600effd700 1 -- 172.21.15.5:0/2933203244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008009a70 con 0x7f601015b4e0 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f600ffff700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f601015b8c0 0x7f60100ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f600f7fe700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60100d1910 0x7f6010145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f600effd700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601015b4e0 0x7f6010145990 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f600400c780 tx=0x7f600400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f5ff7fff700 1 -- 172.21.15.5:0/2933203244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004004c80 con 0x7f601015b4e0 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f5ff7fff700 1 -- 172.21.15.5:0/2933203244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004004e00 con 0x7f601015b4e0 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f601610f700 1 -- 172.21.15.5:0/2933203244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601015c050 con 0x7f601015b4e0 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f5ff7fff700 1 -- 172.21.15.5:0/2933203244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004007310 con 0x7f601015b4e0 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.520+0000 7f601610f700 1 -- 172.21.15.5:0/2933203244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60100c8b10 con 0x7f601015b4e0 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.522+0000 7f5ff7fff700 1 -- 172.21.15.5:0/2933203244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6004007470 con 0x7f601015b4e0 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.522+0000 7f5ff7fff700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ffc06ac90 0x7f5ffc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.522+0000 7f5ff7fff700 1 -- 172.21.15.5:0/2933203244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6004088ee0 con 0x7f601015b4e0 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.522+0000 7f600f7fe700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ffc06ac90 0x7f5ffc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.523+0000 7f600f7fe700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ffc06ac90 0x7f5ffc06d150 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6000009fd0 tx=0x7f6000009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.524+0000 7f60222c9b80 1 Processor -- start 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.524+0000 7f60222c9b80 1 -- start start 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20ce37a0 0x562e20cbc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20cbe370 0x562e20cc10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20ceb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f60222c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e20c17150 con 0x562e20ce37a0 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f60222c9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e20c14b10 con 0x562e20cbe370 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f60222c9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e20c10400 con 0x562e20ceadd0 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f600f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20ce37a0 0x562e20cbc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f600f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20ce37a0 0x562e20cbc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46060/0 (socket says 172.21.15.5:46060) 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f600f7fe700 1 -- 172.21.15.5:0/1841384381 learned_addr learned my addr 172.21.15.5:0/1841384381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f600effd700 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20cbe370 0x562e20cc10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.525+0000 7f600ffff700 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20ceb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f600f7fe700 1 -- 172.21.15.5:0/1841384381 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 msgr2=0x562e20ceb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f600f7fe700 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20ceb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f600f7fe700 1 -- 172.21.15.5:0/1841384381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20cbe370 msgr2=0x562e20cc10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f600f7fe700 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20cbe370 0x562e20cc10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f600f7fe700 1 -- 172.21.15.5:0/1841384381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e20a913b0 con 0x562e20ce37a0 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f600f7fe700 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20ce37a0 0x562e20cbc110 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6000004160 tx=0x7f6000014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5c3aec98d1c74f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f5ff57fa700 1 -- 172.21.15.5:0/1841384381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000003b30 con 0x562e20ce37a0 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f5ff57fa700 1 -- 172.21.15.5:0/1841384381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600001b070 con 0x562e20ce37a0 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.526+0000 7f5ff57fa700 1 -- 172.21.15.5:0/1841384381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000003e90 con 0x562e20ce37a0 2024-05-08T23:26:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 -- 172.21.15.5:0/1841384381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20ce37a0 msgr2=0x562e20cbc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20ce37a0 0x562e20cbc110 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6000004160 tx=0x7f6000014000 comp rx=0 tx=0).stop 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 -- 172.21.15.5:0/1841384381 shutdown_connections 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20ceb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20ce37a0 0x562e20cbc110 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 --2- 172.21.15.5:0/1841384381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20cbe370 0x562e20cc10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 -- 172.21.15.5:0/1841384381 >> 172.21.15.5:0/1841384381 conn(0x562e20c1e210 msgr2=0x562e20d05dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 -- 172.21.15.5:0/1841384381 shutdown_connections 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.527+0000 7f60222c9b80 1 -- 172.21.15.5:0/1841384381 wait complete. 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.528+0000 7f60222c9b80 1 -- start start 2024-05-08T23:26:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.528+0000 7f60222c9b80 1 Processor -- start 2024-05-08T23:26:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 0x562e20ce0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20d09580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f60222c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e20bac560 con 0x562e20cbe370 2024-05-08T23:26:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f60222c9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e20bac200 con 0x562e20ce37a0 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f60222c9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e20a936a0 con 0x562e20ceadd0 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f60222c9b80 10 client.0 _ll_register_callbacks cb 0x562e20bb5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46064/0 (socket says 172.21.15.5:46064) 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 -- 172.21.15.5:0/4153992731 learned_addr learned my addr 172.21.15.5:0/4153992731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600ffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20d09580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600effd700 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 0x562e20ce0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 -- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 msgr2=0x562e20d09580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20d09580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 -- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 msgr2=0x562e20ce0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 0x562e20ce0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 -- 172.21.15.5:0/4153992731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6000011e00 con 0x562e20cbe370 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f600f7fe700 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cee350 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f60000119e0 tx=0x7f60000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b515984669c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f5fee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4153992731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000022a70 con 0x562e20cbe370 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4153992731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6000022bd0 con 0x562e20cbe370 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.529+0000 7f60222c9b80 1 -- 172.21.15.5:0/4153992731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e20b69270 con 0x562e20cbe370 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4153992731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6000003e80 con 0x562e20cbe370 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f60222c9b80 10 client.7488 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f60222c9b80 1 -- 172.21.15.5:0/4153992731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x562e20a3db60 con 0x562e20cbe370 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4153992731 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e20b69270 con 0x562e20cbe370 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4153992731 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f6000014440 con 0x562e20cbe370 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f60222c9b80 10 client.7488 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f60222c9b80 10 client.7488 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.d 2024-05-08T23:26:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f60222c9b80 20 client.7488 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x562e20d09ac0 0x562e20d07250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f60222c9b80 4 client.7488 mds_command: new command op to 6787 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f60222c9b80 1 -- 172.21.15.5:0/4153992731 --> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e20c8f7e0 con 0x562e20d09ac0 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.530+0000 7f600effd700 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x562e20d09ac0 0x562e20d07250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f600effd700 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x562e20d09ac0 0x562e20d07250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83152119b40defdf server_cookie=3a821600fadd9e9a in_seq=0 out_seq=0 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f5fee7fc700 10 client.7488 ms_handle_connect on v2:172.21.15.49:6834/2469967007 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4153992731 <== mds.2 v2:172.21.15.49:6834/2469967007 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e20c8f7e0 con 0x562e20d09ac0 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f5fee7fc700 10 client.7488 handle_command_reply: tid=0 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 client.7488 shutdown 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 -- 172.21.15.5:0/4153992731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 msgr2=0x562e20cee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cee350 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f60000119e0 tx=0x7f60000047c0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 -- 172.21.15.5:0/4153992731 shutdown_connections 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x562e20d09ac0 0x562e20d07250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20d09580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cee350 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f5fee7fc700 0 client.7488 ms_handle_reset on v2:172.21.15.49:6834/2469967007 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4153992731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 0x562e20ce0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 -- 172.21.15.5:0/4153992731 >> 172.21.15.5:0/4153992731 conn(0x562e20c1e210 msgr2=0x562e20cc0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 -- 172.21.15.5:0/4153992731 shutdown_connections 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.531+0000 7f60222c9b80 1 -- 172.21.15.5:0/4153992731 wait complete. 2024-05-08T23:26:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.532+0000 7f60222c9b80 20 client.7488 trim_cache size 0 max 16384 2024-05-08T23:26:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.547+0000 7f60222c9b80 1 Processor -- start 2024-05-08T23:26:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.547+0000 7f60222c9b80 1 -- start start 2024-05-08T23:26:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cc1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 0x562e20d27d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20cbff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f60222c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e20cbb240 con 0x562e20cbe370 2024-05-08T23:26:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f60222c9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e20cbcdb0 con 0x562e20ce37a0 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f60222c9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e20ceeab0 con 0x562e20ceadd0 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f600f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cc1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f600f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cc1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46072/0 (socket says 172.21.15.5:46072) 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f600f7fe700 1 -- 172.21.15.5:0/3417553689 learned_addr learned my addr 172.21.15.5:0/3417553689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f600effd700 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 0x562e20d27d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.548+0000 7f600ffff700 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20cbff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f600f7fe700 1 -- 172.21.15.5:0/3417553689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 msgr2=0x562e20cbff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f600f7fe700 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20cbff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f600f7fe700 1 -- 172.21.15.5:0/3417553689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 msgr2=0x562e20d27d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f600f7fe700 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 0x562e20d27d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f600f7fe700 1 -- 172.21.15.5:0/3417553689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e20bd6560 con 0x562e20cbe370 2024-05-08T23:26:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f600f7fe700 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cc1250 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6000006c50 tx=0x7f600002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707b393891e82f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f5ff5ffb700 1 -- 172.21.15.5:0/3417553689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000003e40 con 0x562e20cbe370 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f5ff5ffb700 1 -- 172.21.15.5:0/3417553689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6000030070 con 0x562e20cbe370 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.549+0000 7f5ff5ffb700 1 -- 172.21.15.5:0/3417553689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000014d90 con 0x562e20cbe370 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 -- 172.21.15.5:0/3417553689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 msgr2=0x562e20cc1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cc1250 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6000006c50 tx=0x7f600002e070 comp rx=0 tx=0).stop 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 -- 172.21.15.5:0/3417553689 shutdown_connections 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ceadd0 0x562e20cbff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20cc1250 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 --2- 172.21.15.5:0/3417553689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ce37a0 0x562e20d27d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 -- 172.21.15.5:0/3417553689 >> 172.21.15.5:0/3417553689 conn(0x562e20c1e210 msgr2=0x562e20ce0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 -- 172.21.15.5:0/3417553689 shutdown_connections 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 -- 172.21.15.5:0/3417553689 wait complete. 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 -- start start 2024-05-08T23:26:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.550+0000 7f60222c9b80 1 Processor -- start 2024-05-08T23:26:35.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20d6e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ce37a0 0x562e20ce0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f60222c9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ceadd0 0x562e20cbfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f60222c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e20d05840 con 0x562e20cbe370 2024-05-08T23:26:35.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f60222c9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e20cc2210 con 0x562e20ceadd0 2024-05-08T23:26:35.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f60222c9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e20cc24d0 con 0x562e20ce37a0 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f60222c9b80 10 client.0 _ll_register_callbacks cb 0x562e20bb5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f600f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20d6e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f600f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20d6e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46080/0 (socket says 172.21.15.5:46080) 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f600f7fe700 1 -- 172.21.15.5:0/4260630848 learned_addr learned my addr 172.21.15.5:0/4260630848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f600ffff700 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ceadd0 0x562e20cbfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.551+0000 7f600effd700 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ce37a0 0x562e20ce0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f600f7fe700 1 -- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ce37a0 msgr2=0x562e20ce0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f600f7fe700 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ce37a0 0x562e20ce0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f600f7fe700 1 -- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ceadd0 msgr2=0x562e20cbfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f600f7fe700 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ceadd0 0x562e20cbfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f600f7fe700 1 -- 172.21.15.5:0/4260630848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e20a3db60 con 0x562e20cbe370 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f600f7fe700 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20d6e130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f600000b010 tx=0x7f6000018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7303eea1c2419f0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f5fee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:35.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4260630848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000003e40 con 0x562e20cbe370 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4260630848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6000014950 con 0x562e20cbe370 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 1 -- 172.21.15.5:0/4260630848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f600001b070 con 0x562e20cbe370 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4260630848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f600003f470 con 0x562e20cbe370 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 10 client.7494 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 1 -- 172.21.15.5:0/4260630848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x562e20a95cd0 con 0x562e20cbe370 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4260630848 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600001b070 con 0x562e20cbe370 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4260630848 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f600003f5d0 con 0x562e20cbe370 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 10 client.7494 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 10 client.7494 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.d 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 20 client.7494 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x562e20d05140 0x562e20cc0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 4 client.7494 mds_command: new command op to 6787 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.552+0000 7f60222c9b80 1 -- 172.21.15.5:0/4260630848 --> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e20c08ec0 con 0x562e20d05140 2024-05-08T23:26:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.553+0000 7f600effd700 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x562e20d05140 0x562e20cc0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.553+0000 7f5fee7fc700 10 client.7494 ms_handle_connect on v2:172.21.15.49:6834/2469967007 2024-05-08T23:26:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.553+0000 7f600effd700 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x562e20d05140 0x562e20cc0430 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3687475a7c9eaae4 server_cookie=a6b1852c0678bae5 in_seq=0 out_seq=0 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f5fee7fc700 1 -- 172.21.15.5:0/4260630848 <== mds.2 v2:172.21.15.49:6834/2469967007 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e20c08ec0 con 0x562e20d05140 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f5fee7fc700 10 client.7494 handle_command_reply: tid=0 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 client.7494 shutdown 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 -- 172.21.15.5:0/4260630848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 msgr2=0x562e20d6e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20d6e130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f600000b010 tx=0x7f6000018000 comp rx=0 tx=0).stop 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 -- 172.21.15.5:0/4260630848 shutdown_connections 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:6834/2469967007,v1:172.21.15.49:6836/2469967007] conn(0x562e20d05140 0x562e20cc0430 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e20ceadd0 0x562e20cbfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e20cbe370 0x562e20d6e130 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 --2- 172.21.15.5:0/4260630848 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562e20ce37a0 0x562e20ce0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f5fee7fc700 0 client.7494 ms_handle_reset on v2:172.21.15.49:6834/2469967007 2024-05-08T23:26:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 -- 172.21.15.5:0/4260630848 >> 172.21.15.5:0/4260630848 conn(0x562e20c1e210 msgr2=0x562e20d27d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 -- 172.21.15.5:0/4260630848 shutdown_connections 2024-05-08T23:26:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.559+0000 7f60222c9b80 1 -- 172.21.15.5:0/4260630848 wait complete. 2024-05-08T23:26:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f60222c9b80 20 client.7494 trim_cache size 0 max 16384 2024-05-08T23:26:35.560 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:26:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:26:35.560 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 -- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ffc06ac90 msgr2=0x7f5ffc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ffc06ac90 0x7f5ffc06d150 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6000009fd0 tx=0x7f6000009210 comp rx=0 tx=0).stop 2024-05-08T23:26:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 -- 172.21.15.5:0/2933203244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601015b4e0 msgr2=0x7f6010145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601015b4e0 0x7f6010145990 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f600400c780 tx=0x7f600400ca90 comp rx=0 tx=0).stop 2024-05-08T23:26:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 -- 172.21.15.5:0/2933203244 shutdown_connections 2024-05-08T23:26:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ffc06ac90 0x7f5ffc06d150 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f601015b8c0 0x7f60100ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60100d1910 0x7f6010145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 --2- 172.21.15.5:0/2933203244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601015b4e0 0x7f6010145990 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 -- 172.21.15.5:0/2933203244 >> 172.21.15.5:0/2933203244 conn(0x7f601008afe0 msgr2=0x7f60100aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 -- 172.21.15.5:0/2933203244 shutdown_connections 2024-05-08T23:26:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.560+0000 7f601610f700 1 -- 172.21.15.5:0/2933203244 wait complete. 2024-05-08T23:26:35.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.709+0000 7f72f281e700 1 Processor -- start 2024-05-08T23:26:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.709+0000 7f72f281e700 1 -- start start 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72f281e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72ec0b51d0 0x7f72ec0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72f281e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72ec0b3030 0x7f72ec06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72f281e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec141ac0 con 0x7f72ec1416e0 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72f281e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ec0b34a0 con 0x7f72ec0b51d0 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72f281e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ec0b5680 con 0x7f72ec0b3030 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72eb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72ec0b51d0 0x7f72ec0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72ebfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f72ec1416e0 0x7f72ec0b4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53366/0 (socket says 172.21.15.5:53366) 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72eb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72ec0b51d0 0x7f72ec0b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46092/0 (socket says 172.21.15.5:46092) 2024-05-08T23:26:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72ebfff700 1 -- 172.21.15.5:0/664430891 learned_addr learned my addr 172.21.15.5:0/664430891 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.710+0000 7f72f0dbb700 1 --2- 172.21.15.5:0/664430891 >> v2:172.21.15.49:3300/0 conn(0x7f72ec0b3030 0x7f72ec06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eb7fe700 1 -- 172.21.15.5:0/664430891 >> v2:172.21.15.49:3300/0 conn(0x7f72ec0b3030 msgr2=0x7f72ec06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eb7fe700 1 --2- 172.21.15.5:0/664430891 >> v2:172.21.15.49:3300/0 conn(0x7f72ec0b3030 0x7f72ec06b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eb7fe700 1 -- 172.21.15.5:0/664430891 >> v1:172.21.15.5:6789/0 conn(0x7f72ec1416e0 legacy=0x7f72ec0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eb7fe700 1 -- 172.21.15.5:0/664430891 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec0b5960 con 0x7f72ec0b51d0 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eaffd700 1 -- 172.21.15.5:0/664430891 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180183991 0 0) 0x7f72ec141ac0 con 0x7f72ec1416e0 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eb7fe700 1 --2- 172.21.15.5:0/664430891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b51d0 0x7f72ec0b2af0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f72ec156120 tx=0x7f72e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d23a1c568169c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eaffd700 1 -- 172.21.15.5:0/664430891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0011830 con 0x7f72ec0b51d0 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eaffd700 1 -- 172.21.15.5:0/664430891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e0011990 con 0x7f72ec0b51d0 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.711+0000 7f72eaffd700 1 -- 172.21.15.5:0/664430891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0011d20 con 0x7f72ec0b51d0 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.712+0000 7f72f281e700 1 -- 172.21.15.5:0/664430891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b51d0 msgr2=0x7f72ec0b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.712+0000 7f72f281e700 1 --2- 172.21.15.5:0/664430891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b51d0 0x7f72ec0b2af0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f72ec156120 tx=0x7f72e0009b20 comp rx=0 tx=0).stop 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.712+0000 7f72f281e700 1 -- 172.21.15.5:0/664430891 shutdown_connections 2024-05-08T23:26:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.712+0000 7f72f281e700 1 --2- 172.21.15.5:0/664430891 >> v2:172.21.15.49:3300/0 conn(0x7f72ec0b3030 0x7f72ec06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.712+0000 7f72f281e700 1 --2- 172.21.15.5:0/664430891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b51d0 0x7f72ec0b2af0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.712+0000 7f72f281e700 1 -- 172.21.15.5:0/664430891 >> 172.21.15.5:0/664430891 conn(0x7f72ec0cfa80 msgr2=0x7f72ec0cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.712+0000 7f72f281e700 1 -- 172.21.15.5:0/664430891 shutdown_connections 2024-05-08T23:26:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.712+0000 7f72f281e700 1 -- 172.21.15.5:0/664430891 wait complete. 2024-05-08T23:26:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.713+0000 7f72f281e700 1 Processor -- start 2024-05-08T23:26:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.713+0000 7f72f281e700 1 -- start start 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72f281e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b3030 0x7f72ec154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72f281e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72ec0b51d0 0x7f72ec156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72f281e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ec1416e0 0x7f72ec1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72f281e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1f0d50 con 0x7f72ec0b3030 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72f281e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec15fd60 con 0x7f72ec1416e0 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72f281e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec156ce0 con 0x7f72ec0b51d0 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b3030 0x7f72ec154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b3030 0x7f72ec154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46102/0 (socket says 172.21.15.5:46102) 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72ebfff700 1 -- 172.21.15.5:0/4207301350 learned_addr learned my addr 172.21.15.5:0/4207301350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72f0dbb700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ec1416e0 0x7f72ec1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72eb7fe700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72ec0b51d0 0x7f72ec156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72ebfff700 1 -- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72ec0b51d0 msgr2=0x7f72ec156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.714+0000 7f72ebfff700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72ec0b51d0 0x7f72ec156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72ebfff700 1 -- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ec1416e0 msgr2=0x7f72ec1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72ebfff700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ec1416e0 0x7f72ec1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72ebfff700 1 -- 172.21.15.5:0/4207301350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e00094a0 con 0x7f72ec0b3030 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72ebfff700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b3030 0x7f72ec154c90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c550 tx=0x7f72dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72e8ff9700 1 -- 172.21.15.5:0/4207301350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc004c80 con 0x7f72ec0b3030 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72e8ff9700 1 -- 172.21.15.5:0/4207301350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc004e00 con 0x7f72ec0b3030 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec156fc0 con 0x7f72ec0b3030 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72e8ff9700 1 -- 172.21.15.5:0/4207301350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc007310 con 0x7f72ec0b3030 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.715+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec157580 con 0x7f72ec0b3030 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.716+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec068000 con 0x7f72ec0b3030 2024-05-08T23:26:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.717+0000 7f72e8ff9700 1 -- 172.21.15.5:0/4207301350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f72dc007470 con 0x7f72ec0b3030 2024-05-08T23:26:35.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.717+0000 7f72e8ff9700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72d806ac90 0x7f72d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:35.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.717+0000 7f72e8ff9700 1 -- 172.21.15.5:0/4207301350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72dc088ea0 con 0x7f72ec0b3030 2024-05-08T23:26:35.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.717+0000 7f72eb7fe700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72d806ac90 0x7f72d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:35.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.718+0000 7f72eb7fe700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72d806ac90 0x7f72d806d150 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0b55b0 tx=0x7f72e0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.722+0000 7f72e8ff9700 1 -- 172.21.15.5:0/4207301350 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f72dc054080 con 0x7f72ec0b3030 2024-05-08T23:26:35.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.987+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72ec0b2c70 con 0x7f72ec0b3030 2024-05-08T23:26:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.988+0000 7f72e8ff9700 1 -- 172.21.15.5:0/4207301350 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4759 (secure 0 0 0) 0x7f72dc0576a0 con 0x7f72ec0b3030 2024-05-08T23:26:35.988 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:35.988 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:34.436951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7179,"mds_1":6790,"mds_2":6787},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6787":{"gid":6787,"name":"d","rank":2,"incarnation":70,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6790":{"gid":6790,"name":"b","rank":1,"incarnation":68,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72d806ac90 msgr2=0x7f72d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72d806ac90 0x7f72d806d150 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0b55b0 tx=0x7f72e0008030 comp rx=0 tx=0).stop 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b3030 msgr2=0x7f72ec154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b3030 0x7f72ec154c90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c550 tx=0x7f72dc00c860 comp rx=0 tx=0).stop 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 shutdown_connections 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f72d806ac90 0x7f72d806d150 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72ec1416e0 0x7f72ec1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72ec0b3030 0x7f72ec154c90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 --2- 172.21.15.5:0/4207301350 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f72ec0b51d0 0x7f72ec156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 >> 172.21.15.5:0/4207301350 conn(0x7f72ec0cfa80 msgr2=0x7f72ec0b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 shutdown_connections 2024-05-08T23:26:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:35.990+0000 7f72f281e700 1 -- 172.21.15.5:0/4207301350 wait complete. 2024-05-08T23:26:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 73 2024-05-08T23:26:36.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub start /d1/d2/d3 2024-05-08T23:26:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.143+0000 7f727334c700 1 Processor -- start 2024-05-08T23:26:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.143+0000 7f727334c700 1 -- start start 2024-05-08T23:26:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.144+0000 7f727334c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f726c15b960 0x7f726c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.144+0000 7f727334c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f726c15f980 0x7f726c0cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.144+0000 7f727334c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726c0d04a0 con 0x7f726c15b580 2024-05-08T23:26:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.144+0000 7f727334c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c0cbfa0 con 0x7f726c15b960 2024-05-08T23:26:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.144+0000 7f727334c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c0cc220 con 0x7f726c15f980 2024-05-08T23:26:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.144+0000 7f72708e7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f726c15b960 0x7f726c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f72708e7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f726c15b960 0x7f726c0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46112/0 (socket says 172.21.15.5:46112) 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f72708e7700 1 -- 172.21.15.5:0/509614907 learned_addr learned my addr 172.21.15.5:0/509614907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f72718e9700 1 --2- 172.21.15.5:0/509614907 >> v2:172.21.15.49:3300/0 conn(0x7f726c15f980 0x7f726c0cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f72708e7700 1 -- 172.21.15.5:0/509614907 >> v2:172.21.15.49:3300/0 conn(0x7f726c15f980 msgr2=0x7f726c0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f72708e7700 1 --2- 172.21.15.5:0/509614907 >> v2:172.21.15.49:3300/0 conn(0x7f726c15f980 0x7f726c0cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f72708e7700 1 -- 172.21.15.5:0/509614907 >> v1:172.21.15.49:6789/0 conn(0x7f726c15b580 legacy=0x7f726c15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f72708e7700 1 -- 172.21.15.5:0/509614907 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0cc500 con 0x7f726c15b960 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f72708e7700 1 --2- 172.21.15.5:0/509614907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f726c15b960 0x7f726c0cfa20 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f726c0723c0 tx=0x7f725c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aac1c7bc8540cdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f7263fff700 1 -- 172.21.15.5:0/509614907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c011830 con 0x7f726c15b960 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f7263fff700 1 -- 172.21.15.5:0/509614907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c011990 con 0x7f726c15b960 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.145+0000 7f7263fff700 1 -- 172.21.15.5:0/509614907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c011d20 con 0x7f726c15b960 2024-05-08T23:26:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.146+0000 7f727334c700 1 -- 172.21.15.5:0/509614907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f726c15b960 msgr2=0x7f726c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.146+0000 7f727334c700 1 --2- 172.21.15.5:0/509614907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f726c15b960 0x7f726c0cfa20 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f726c0723c0 tx=0x7f725c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.146+0000 7f727334c700 1 -- 172.21.15.5:0/509614907 shutdown_connections 2024-05-08T23:26:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.146+0000 7f727334c700 1 --2- 172.21.15.5:0/509614907 >> v2:172.21.15.49:3300/0 conn(0x7f726c15f980 0x7f726c0cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.146+0000 7f727334c700 1 --2- 172.21.15.5:0/509614907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f726c15b960 0x7f726c0cfa20 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.146+0000 7f727334c700 1 -- 172.21.15.5:0/509614907 >> 172.21.15.5:0/509614907 conn(0x7f726c0b2f80 msgr2=0x7f726c0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.146+0000 7f727334c700 1 -- 172.21.15.5:0/509614907 shutdown_connections 2024-05-08T23:26:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.146+0000 7f727334c700 1 -- 172.21.15.5:0/509614907 wait complete. 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.147+0000 7f727334c700 1 Processor -- start 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.147+0000 7f727334c700 1 -- start start 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f727334c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f726c15b580 0x7f726c0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f727334c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f726c15b960 0x7f726c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f727334c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f726c15f980 0x7f726c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f727334c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0b0ae0 con 0x7f726c15b960 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f727334c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c15c280 con 0x7f726c15b580 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f727334c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0b0520 con 0x7f726c15f980 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f72708e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f726c15b960 0x7f726c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f72708e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f726c15b960 0x7f726c06c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46128/0 (socket says 172.21.15.5:46128) 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f72708e7700 1 -- 172.21.15.5:0/3147126478 learned_addr learned my addr 172.21.15.5:0/3147126478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f72718e9700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f726c15f980 0x7f726c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.148+0000 7f72710e8700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f726c15b580 0x7f726c0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f72708e7700 1 -- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f726c15f980 msgr2=0x7f726c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f72708e7700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f726c15f980 0x7f726c06cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f72708e7700 1 -- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f726c15b580 msgr2=0x7f726c0afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f72708e7700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f726c15b580 0x7f726c0afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f72708e7700 1 -- 172.21.15.5:0/3147126478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009550 con 0x7f726c15b960 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f72710e8700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f726c15b580 0x7f726c0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f72718e9700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f726c15f980 0x7f726c06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f72708e7700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f726c15b960 0x7f726c06c5d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f725c000c00 tx=0x7f725c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f7261ffb700 1 -- 172.21.15.5:0/3147126478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0045b0 con 0x7f726c15b960 2024-05-08T23:26:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f7261ffb700 1 -- 172.21.15.5:0/3147126478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c01f070 con 0x7f726c15b960 2024-05-08T23:26:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f727334c700 1 -- 172.21.15.5:0/3147126478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c0b0800 con 0x7f726c15b960 2024-05-08T23:26:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f7261ffb700 1 -- 172.21.15.5:0/3147126478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0038c0 con 0x7f726c15b960 2024-05-08T23:26:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.149+0000 7f727334c700 1 -- 172.21.15.5:0/3147126478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c1f3b60 con 0x7f726c15b960 2024-05-08T23:26:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.151+0000 7f7261ffb700 1 -- 172.21.15.5:0/3147126478 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f725c021030 con 0x7f726c15b960 2024-05-08T23:26:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.151+0000 7f7261ffb700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f725006ac90 0x7f725006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.152+0000 7f7261ffb700 1 -- 172.21.15.5:0/3147126478 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f725c08a7c0 con 0x7f726c15b960 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.152+0000 7f72710e8700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f725006ac90 0x7f725006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.152+0000 7f72710e8700 1 --2- 172.21.15.5:0/3147126478 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f725006ac90 0x7f725006d150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7268000f60 tx=0x7f7268009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.154+0000 7f727f506b80 1 Processor -- start 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.154+0000 7f727f506b80 1 -- start start 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.154+0000 7f727f506b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 0x55aac349f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.154+0000 7f727f506b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34a1380 0x55aac34a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.154+0000 7f727f506b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.154+0000 7f727f506b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac33fa160 con 0x55aac34cdde0 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.154+0000 7f727f506b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac33f7b20 con 0x55aac34a1380 2024-05-08T23:26:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.154+0000 7f727f506b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac33f3410 con 0x55aac34c67b0 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46144/0 (socket says 172.21.15.5:46144) 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 -- 172.21.15.5:0/3279765443 learned_addr learned my addr 172.21.15.5:0/3279765443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72710e8700 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 0x55aac349f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72708e7700 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34a1380 0x55aac34a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 -- 172.21.15.5:0/3279765443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 msgr2=0x55aac349f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 0x55aac349f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 -- 172.21.15.5:0/3279765443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34a1380 msgr2=0x55aac34a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34a1380 0x55aac34a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 -- 172.21.15.5:0/3279765443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aac32743c0 con 0x55aac34cdde0 2024-05-08T23:26:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f72718e9700 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34ce260 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f72640074f0 tx=0x7f7264009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39c1f7e8f423f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f7256ffd700 1 -- 172.21.15.5:0/3279765443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264015670 con 0x55aac34cdde0 2024-05-08T23:26:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.155+0000 7f7256ffd700 1 -- 172.21.15.5:0/3279765443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726400f050 con 0x55aac34cdde0 2024-05-08T23:26:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f7256ffd700 1 -- 172.21.15.5:0/3279765443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72640159d0 con 0x55aac34cdde0 2024-05-08T23:26:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 -- 172.21.15.5:0/3279765443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 msgr2=0x55aac34ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34ce260 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f72640074f0 tx=0x7f7264009750 comp rx=0 tx=0).stop 2024-05-08T23:26:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 -- 172.21.15.5:0/3279765443 shutdown_connections 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34ce260 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 0x55aac349f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 --2- 172.21.15.5:0/3279765443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34a1380 0x55aac34a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 -- 172.21.15.5:0/3279765443 >> 172.21.15.5:0/3279765443 conn(0x55aac3401220 msgr2=0x55aac34e8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 -- 172.21.15.5:0/3279765443 shutdown_connections 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.156+0000 7f727f506b80 1 -- 172.21.15.5:0/3279765443 wait complete. 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.157+0000 7f727f506b80 1 -- start start 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.157+0000 7f727f506b80 1 Processor -- start 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f727f506b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac34cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f727f506b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34c67b0 0x55aac34d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f727f506b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f727f506b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac338f570 con 0x55aac34cdde0 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f727f506b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac338f210 con 0x55aac34c67b0 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f727f506b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac32766b0 con 0x55aac34a1380 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f727f506b80 10 client.0 _ll_register_callbacks cb 0x55aac3398b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f72718e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f72718e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34d0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46158/0 (socket says 172.21.15.5:46158) 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f72718e9700 1 -- 172.21.15.5:0/4183450735 learned_addr learned my addr 172.21.15.5:0/4183450735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f72710e8700 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac34cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.158+0000 7f72708e7700 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34c67b0 0x55aac34d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f72718e9700 1 -- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 msgr2=0x55aac34cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f72718e9700 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac34cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f72718e9700 1 -- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34c67b0 msgr2=0x55aac34d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f72718e9700 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34c67b0 0x55aac34d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f72718e9700 1 -- 172.21.15.5:0/4183450735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7264009e30 con 0x55aac34cdde0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f72718e9700 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34d0ce0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7264007ae0 tx=0x7f7264004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4beef6e73de757 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f724bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f724bfff700 1 -- 172.21.15.5:0/4183450735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264015e50 con 0x55aac34cdde0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f724bfff700 1 -- 172.21.15.5:0/4183450735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7264009410 con 0x55aac34cdde0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f727f506b80 1 -- 172.21.15.5:0/4183450735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aac334c280 con 0x55aac34cdde0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f724bfff700 1 -- 172.21.15.5:0/4183450735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f726401f400 con 0x55aac34cdde0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f727f506b80 10 client.7515 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f727f506b80 1 -- 172.21.15.5:0/4183450735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55aac3220b70 con 0x55aac34cdde0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f724bfff700 1 -- 172.21.15.5:0/4183450735 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aac334c280 con 0x55aac34cdde0 2024-05-08T23:26:36.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f724bfff700 1 -- 172.21.15.5:0/4183450735 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f726401f560 con 0x55aac34cdde0 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f727f506b80 10 client.7515 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.159+0000 7f727f506b80 10 client.7515 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.160+0000 7f727f506b80 20 client.7515 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.160+0000 7f727f506b80 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55aac34c5170 0x55aac34c5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.160+0000 7f727f506b80 4 client.7515 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.160+0000 7f727f506b80 1 -- 172.21.15.5:0/4183450735 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aac34727f0 con 0x55aac34c5170 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.160+0000 7f72710e8700 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55aac34c5170 0x55aac34c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.160+0000 7f72710e8700 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55aac34c5170 0x55aac34c5550 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=945aad8bc9073230 server_cookie=e5943dc1b8779c0f in_seq=0 out_seq=0 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.160+0000 7f724bfff700 10 client.7515 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f724bfff700 1 -- 172.21.15.5:0/4183450735 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aac34727f0 con 0x55aac34c5170 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f724bfff700 10 client.7515 handle_command_reply: tid=0 2024-05-08T23:26:36.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 client.7515 shutdown 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 -- 172.21.15.5:0/4183450735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 msgr2=0x55aac34d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34d0ce0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7264007ae0 tx=0x7f7264004b40 comp rx=0 tx=0).stop 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 -- 172.21.15.5:0/4183450735 shutdown_connections 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55aac34c5170 0x55aac34c5550 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34cdde0 0x55aac34d0ce0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac34cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f724bfff700 0 client.7515 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 --2- 172.21.15.5:0/4183450735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34c67b0 0x55aac34d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 -- 172.21.15.5:0/4183450735 >> 172.21.15.5:0/4183450735 conn(0x55aac3401220 msgr2=0x55aac34a3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 -- 172.21.15.5:0/4183450735 shutdown_connections 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 1 -- 172.21.15.5:0/4183450735 wait complete. 2024-05-08T23:26:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.161+0000 7f727f506b80 20 client.7515 trim_cache size 0 max 16384 2024-05-08T23:26:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.180+0000 7f727f506b80 1 Processor -- start 2024-05-08T23:26:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.180+0000 7f727f506b80 1 -- start start 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f727f506b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34a1380 0x55aac34ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f727f506b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 0x55aac34c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f727f506b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f727f506b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac34a56b0 con 0x55aac34a1380 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f727f506b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac34a2bc0 con 0x55aac34cdde0 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f727f506b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac3480070 con 0x55aac34c67b0 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34a1380 0x55aac34ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34a1380 0x55aac34ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46174/0 (socket says 172.21.15.5:46174) 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 -- 172.21.15.5:0/991881888 learned_addr learned my addr 172.21.15.5:0/991881888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72718e9700 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72708e7700 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 0x55aac34c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 -- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 msgr2=0x55aac34c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 0x55aac34c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 -- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 msgr2=0x55aac34c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 -- 172.21.15.5:0/991881888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aac33b9570 con 0x55aac34a1380 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72718e9700 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.181+0000 7f72710e8700 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34a1380 0x55aac34ce9c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x55aac34ecfe0 tx=0x7f7268013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427c92e3772e2b0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f72577fe700 1 -- 172.21.15.5:0/991881888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268006c40 con 0x55aac34a1380 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f72577fe700 1 -- 172.21.15.5:0/991881888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7268006da0 con 0x55aac34a1380 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f72577fe700 1 -- 172.21.15.5:0/991881888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268007040 con 0x55aac34a1380 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 -- 172.21.15.5:0/991881888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34a1380 msgr2=0x55aac34ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34a1380 0x55aac34ce9c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x55aac34ecfe0 tx=0x7f7268013000 comp rx=0 tx=0).stop 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 -- 172.21.15.5:0/991881888 shutdown_connections 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34a1380 0x55aac34ce9c0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 --2- 172.21.15.5:0/991881888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34c67b0 0x55aac34c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 -- 172.21.15.5:0/991881888 >> 172.21.15.5:0/991881888 conn(0x55aac3401220 msgr2=0x55aac34d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 -- 172.21.15.5:0/991881888 shutdown_connections 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.182+0000 7f727f506b80 1 -- 172.21.15.5:0/991881888 wait complete. 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.183+0000 7f727f506b80 1 -- start start 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.183+0000 7f727f506b80 1 Processor -- start 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f727f506b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac350ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f727f506b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34c67b0 0x55aac3551dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f727f506b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f727f506b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac34ea8c0 con 0x55aac34c67b0 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f727f506b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac34d15a0 con 0x55aac34cdde0 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f727f506b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac34cbdc0 con 0x55aac34a1380 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f727f506b80 10 client.0 _ll_register_callbacks cb 0x55aac3398b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72708e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34c67b0 0x55aac3551dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72708e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34c67b0 0x55aac3551dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46178/0 (socket says 172.21.15.5:46178) 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72708e7700 1 -- 172.21.15.5:0/488720097 learned_addr learned my addr 172.21.15.5:0/488720097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72718e9700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72708e7700 1 -- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 msgr2=0x55aac350ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72710e8700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac350ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72708e7700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac350ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72708e7700 1 -- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 msgr2=0x55aac34c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72708e7700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72708e7700 1 -- 172.21.15.5:0/488720097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c0d04a0 con 0x55aac34c67b0 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72718e9700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.184+0000 7f72710e8700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac350ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f72708e7700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34c67b0 0x55aac3551dd0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f725c061510 tx=0x7f725c059e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdb40a05f73802e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f724bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f724bfff700 1 -- 172.21.15.5:0/488720097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c05d2b0 con 0x55aac34c67b0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f724bfff700 1 -- 172.21.15.5:0/488720097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f725c082be0 con 0x55aac34c67b0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 1 -- 172.21.15.5:0/488720097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f726400f050 con 0x55aac34c67b0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f724bfff700 1 -- 172.21.15.5:0/488720097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f725c082da0 con 0x55aac34c67b0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 10 client.7521 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 1 -- 172.21.15.5:0/488720097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55aac3278ce0 con 0x55aac34c67b0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f724bfff700 1 -- 172.21.15.5:0/488720097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726400f050 con 0x55aac34c67b0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f724bfff700 1 -- 172.21.15.5:0/488720097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f725c073060 con 0x55aac34c67b0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 10 client.7521 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 10 client.7521 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 20 client.7521 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55aac34a32a0 0x55aac34a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 4 client.7521 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3"}] 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.185+0000 7f727f506b80 1 -- 172.21.15.5:0/488720097 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3"}) v1 -- 0x55aac33ebed0 con 0x55aac34a32a0 2024-05-08T23:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f72710e8700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55aac34a32a0 0x55aac34a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f72710e8700 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55aac34a32a0 0x55aac34a3680 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b53378913796159e server_cookie=eae3aff2e8ddf851 in_seq=0 out_seq=0 2024-05-08T23:26:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f724bfff700 10 client.7521 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f724bfff700 1 -- 172.21.15.5:0/488720097 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x55aac33ebed0 con 0x55aac34a32a0 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f724bfff700 10 client.7521 handle_command_reply: tid=0 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 client.7521 shutdown 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 -- 172.21.15.5:0/488720097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34c67b0 msgr2=0x55aac3551dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34c67b0 0x55aac3551dd0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f725c061510 tx=0x7f725c059e50 comp rx=0 tx=0).stop 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 -- 172.21.15.5:0/488720097 shutdown_connections 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55aac34a32a0 0x55aac34a3680 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aac34cdde0 0x55aac34c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aac34a1380 0x55aac350ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 --2- 172.21.15.5:0/488720097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aac34c67b0 0x55aac3551dd0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f724bfff700 0 client.7521 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 -- 172.21.15.5:0/488720097 >> 172.21.15.5:0/488720097 conn(0x55aac3401220 msgr2=0x55aac34fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 -- 172.21.15.5:0/488720097 shutdown_connections 2024-05-08T23:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.186+0000 7f727f506b80 1 -- 172.21.15.5:0/488720097 wait complete. 2024-05-08T23:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.187+0000 7f727f506b80 20 client.7521 trim_cache size 0 max 16384 2024-05-08T23:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:Error EXDEV: Not rank 0 2024-05-08T23:26:36.195 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-05-08T23:26:36.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub abort 2024-05-08T23:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.335+0000 7f5f72f6c700 1 Processor -- start 2024-05-08T23:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.335+0000 7f5f72f6c700 1 -- start start 2024-05-08T23:26:36.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f72f6c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5f6c0cfd10 0x7f5f6c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f72f6c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f72f6c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6c0cc980 con 0x7f5f6c15bb20 2024-05-08T23:26:36.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f72f6c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c0d00f0 con 0x7f5f6c0cfd10 2024-05-08T23:26:36.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f72f6c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c06e420 con 0x7f5f6c06edf0 2024-05-08T23:26:36.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f6bfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5f6c15bb20 0x7f5f6c06e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53374/0 (socket says 172.21.15.5:53374) 2024-05-08T23:26:36.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f6bfff700 1 -- 172.21.15.5:0/3167555268 learned_addr learned my addr 172.21.15.5:0/3167555268 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:36.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f70d08700 1 --2- 172.21.15.5:0/3167555268 >> v2:172.21.15.49:3300/0 conn(0x7f5f6c0cfd10 0x7f5f6c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.336+0000 7f5f71509700 1 --2- 172.21.15.5:0/3167555268 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f6b7fe700 1 -- 172.21.15.5:0/3167555268 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213046761 0 0) 0x7f5f6c0cc980 con 0x7f5f6c15bb20 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f6b7fe700 1 -- 172.21.15.5:0/3167555268 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f580035b0 con 0x7f5f6c15bb20 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f71509700 1 -- 172.21.15.5:0/3167555268 >> v2:172.21.15.49:3300/0 conn(0x7f5f6c0cfd10 msgr2=0x7f5f6c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f71509700 1 --2- 172.21.15.5:0/3167555268 >> v2:172.21.15.49:3300/0 conn(0x7f5f6c0cfd10 0x7f5f6c06dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f71509700 1 -- 172.21.15.5:0/3167555268 >> v1:172.21.15.5:6789/0 conn(0x7f5f6c15bb20 legacy=0x7f5f6c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f6b7fe700 1 -- 172.21.15.5:0/3167555268 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530407794 0 0) 0x7f5f580035b0 con 0x7f5f6c15bb20 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f71509700 1 -- 172.21.15.5:0/3167555268 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c0ccbd0 con 0x7f5f6c06edf0 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f70d08700 1 --2- 172.21.15.5:0/3167555268 >> v2:172.21.15.49:3300/0 conn(0x7f5f6c0cfd10 0x7f5f6c06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f71509700 1 --2- 172.21.15.5:0/3167555268 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c159f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009d90 tx=0x7f5f6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b1eb7f72f0a62f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f6b7fe700 1 -- 172.21.15.5:0/3167555268 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400e070 con 0x7f5f6c06edf0 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f6b7fe700 1 -- 172.21.15.5:0/3167555268 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f64003a20 con 0x7f5f6c06edf0 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.337+0000 7f5f6b7fe700 1 -- 172.21.15.5:0/3167555268 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54001670 con 0x7f5f6c06edf0 2024-05-08T23:26:36.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.338+0000 7f5f72f6c700 1 -- 172.21.15.5:0/3167555268 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 msgr2=0x7f5f6c159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.338+0000 7f5f72f6c700 1 --2- 172.21.15.5:0/3167555268 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c159f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009d90 tx=0x7f5f6400b330 comp rx=0 tx=0).stop 2024-05-08T23:26:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.338+0000 7f5f72f6c700 1 -- 172.21.15.5:0/3167555268 shutdown_connections 2024-05-08T23:26:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.338+0000 7f5f72f6c700 1 --2- 172.21.15.5:0/3167555268 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c159f80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.338+0000 7f5f72f6c700 1 --2- 172.21.15.5:0/3167555268 >> v2:172.21.15.49:3300/0 conn(0x7f5f6c0cfd10 0x7f5f6c06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.338+0000 7f5f72f6c700 1 -- 172.21.15.5:0/3167555268 >> 172.21.15.5:0/3167555268 conn(0x7f5f6c099a50 msgr2=0x7f5f6c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.338+0000 7f5f72f6c700 1 -- 172.21.15.5:0/3167555268 shutdown_connections 2024-05-08T23:26:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.338+0000 7f5f72f6c700 1 -- 172.21.15.5:0/3167555268 wait complete. 2024-05-08T23:26:36.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.339+0000 7f5f72f6c700 1 Processor -- start 2024-05-08T23:26:36.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.339+0000 7f5f72f6c700 1 -- start start 2024-05-08T23:26:36.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f72f6c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f72f6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f6c0cfd10 0x7f5f6c0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f72f6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f6c15bb20 0x7f5f6c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f72f6c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0b5bd0 con 0x7f5f6c15bb20 2024-05-08T23:26:36.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f72f6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0c90d0 con 0x7f5f6c0cfd10 2024-05-08T23:26:36.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f72f6c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0b4150 con 0x7f5f6c06edf0 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f71509700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f6c15bb20 0x7f5f6c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f71509700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f6c15bb20 0x7f5f6c15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46182/0 (socket says 172.21.15.5:46182) 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.340+0000 7f5f71509700 1 -- 172.21.15.5:0/1199321445 learned_addr learned my addr 172.21.15.5:0/1199321445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f6c0cfd10 0x7f5f6c0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f71509700 1 -- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 msgr2=0x7f5f6c0b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f71509700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c0b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f71509700 1 -- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f6c0cfd10 msgr2=0x7f5f6c0b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f71509700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f6c0cfd10 0x7f5f6c0b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f71509700 1 -- 172.21.15.5:0/1199321445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0094a0 con 0x7f5f6c15bb20 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f6c0cfd10 0x7f5f6c0b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5f6c06edf0 0x7f5f6c0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f71509700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f6c15bb20 0x7f5f6c15df70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5f64004000 tx=0x7f5f64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f697fa700 1 -- 172.21.15.5:0/1199321445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f640046b0 con 0x7f5f6c15bb20 2024-05-08T23:26:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f697fa700 1 -- 172.21.15.5:0/1199321445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f64004850 con 0x7f5f6c15bb20 2024-05-08T23:26:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f72f6c700 1 -- 172.21.15.5:0/1199321445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f64009a70 con 0x7f5f6c15bb20 2024-05-08T23:26:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f697fa700 1 -- 172.21.15.5:0/1199321445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6401c680 con 0x7f5f6c15bb20 2024-05-08T23:26:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.341+0000 7f5f72f6c700 1 -- 172.21.15.5:0/1199321445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c0c9310 con 0x7f5f6c15bb20 2024-05-08T23:26:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.343+0000 7f5f697fa700 1 -- 172.21.15.5:0/1199321445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f6401c7e0 con 0x7f5f6c15bb20 2024-05-08T23:26:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.344+0000 7f5f697fa700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4c06ac90 0x7f5f4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.344+0000 7f5f697fa700 1 -- 172.21.15.5:0/1199321445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f6408ade0 con 0x7f5f6c15bb20 2024-05-08T23:26:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.344+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4c06ac90 0x7f5f4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.344+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1199321445 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5f4c06ac90 0x7f5f4c06d150 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009cf0 tx=0x7f5f5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.345+0000 7f5f7f126b80 1 Processor -- start 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f7f126b80 1 -- start start 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a03c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a058230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a011180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc49f68130 con 0x55fc4a00f350 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc49f65af0 con 0x55fc4a03bfa0 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc49f613e0 con 0x55fc4a0393b0 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f6bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a058230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f6bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a058230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46194/0 (socket says 172.21.15.5:46194) 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f6bfff700 1 -- 172.21.15.5:0/1755839263 learned_addr learned my addr 172.21.15.5:0/1755839263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.346+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a03c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f71509700 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a011180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f6bfff700 1 -- 172.21.15.5:0/1755839263 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 msgr2=0x55fc4a011180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a011180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f6bfff700 1 -- 172.21.15.5:0/1755839263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 msgr2=0x55fc4a03c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a03c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f6bfff700 1 -- 172.21.15.5:0/1755839263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc49de2390 con 0x55fc4a00f350 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a058230 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5f540079b0 tx=0x7f5f54007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed757ee3c3ee824 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f627fc700 1 -- 172.21.15.5:0/1755839263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54004c80 con 0x55fc4a00f350 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f627fc700 1 -- 172.21.15.5:0/1755839263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f54004de0 con 0x55fc4a00f350 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.347+0000 7f5f627fc700 1 -- 172.21.15.5:0/1755839263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f540072d0 con 0x55fc4a00f350 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1755839263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 msgr2=0x55fc4a058230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a058230 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5f540079b0 tx=0x7f5f54007cc0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1755839263 shutdown_connections 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a011180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a03c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1755839263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a058230 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1755839263 >> 172.21.15.5:0/1755839263 conn(0x55fc49f6f1f0 msgr2=0x55fc4a056db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1755839263 shutdown_connections 2024-05-08T23:26:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1755839263 wait complete. 2024-05-08T23:26:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.348+0000 7f5f7f126b80 1 -- start start 2024-05-08T23:26:36.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.349+0000 7f5f7f126b80 1 Processor -- start 2024-05-08T23:26:36.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a032c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a0393b0 0x55fc4a03ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a03bfa0 0x55fc4a035530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc49efd540 con 0x55fc4a00f350 2024-05-08T23:26:36.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc49efd1e0 con 0x55fc4a0393b0 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc49de4680 con 0x55fc4a03bfa0 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f7f126b80 10 client.0 _ll_register_callbacks cb 0x55fc49f06b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a032c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a032c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46202/0 (socket says 172.21.15.5:46202) 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 -- 172.21.15.5:0/3142477619 learned_addr learned my addr 172.21.15.5:0/3142477619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f71509700 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a03bfa0 0x55fc4a035530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a0393b0 0x55fc4a03ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 -- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a03bfa0 msgr2=0x55fc4a035530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a03bfa0 0x55fc4a035530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 -- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a0393b0 msgr2=0x55fc4a03ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a0393b0 0x55fc4a03ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 -- 172.21.15.5:0/3142477619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f54001520 con 0x55fc4a00f350 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f70d08700 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a032c10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c015a20 tx=0x7f5f5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152df54fa3d0eedb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.350+0000 7f5f4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/3142477619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c004220 con 0x55fc4a00f350 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/3142477619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f5c004380 con 0x55fc4a00f350 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 1 -- 172.21.15.5:0/3142477619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc49eba250 con 0x55fc4a00f350 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/3142477619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f5c004580 con 0x55fc4a00f350 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 10 client.7533 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 1 -- 172.21.15.5:0/3142477619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55fc49d8eb40 con 0x55fc4a00f350 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/3142477619 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc49eba250 con 0x55fc4a00f350 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/3142477619 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f5f5c016ac0 con 0x55fc4a00f350 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 10 client.7533 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 10 client.7533 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 20 client.7533 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55fc4a03b060 0x55fc4a03a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 4 client.7533 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:36.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f7f126b80 1 -- 172.21.15.5:0/3142477619 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc49fe07c0 con 0x55fc4a03b060 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.351+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55fc4a03b060 0x55fc4a03a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.352+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55fc4a03b060 0x55fc4a03a030 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6f9d616bcc87087 server_cookie=781eb113ed7e2f3d in_seq=0 out_seq=0 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.352+0000 7f5f4b7fe700 10 client.7533 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.352+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/3142477619 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc49fe07c0 con 0x55fc4a03b060 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.352+0000 7f5f4b7fe700 10 client.7533 handle_command_reply: tid=0 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.352+0000 7f5f7f126b80 1 client.7533 shutdown 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 -- 172.21.15.5:0/3142477619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 msgr2=0x55fc4a032c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a032c10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c015a20 tx=0x7f5f5c004750 comp rx=0 tx=0).stop 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 -- 172.21.15.5:0/3142477619 shutdown_connections 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55fc4a03b060 0x55fc4a03a030 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a03bfa0 0x55fc4a035530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a032c10 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f4b7fe700 0 client.7533 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/3142477619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a0393b0 0x55fc4a03ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 -- 172.21.15.5:0/3142477619 >> 172.21.15.5:0/3142477619 conn(0x55fc49f6f1f0 msgr2=0x55fc4a00f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 -- 172.21.15.5:0/3142477619 shutdown_connections 2024-05-08T23:26:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 1 -- 172.21.15.5:0/3142477619 wait complete. 2024-05-08T23:26:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.353+0000 7f5f7f126b80 20 client.7533 trim_cache size 0 max 16384 2024-05-08T23:26:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.372+0000 7f5f7f126b80 1 Processor -- start 2024-05-08T23:26:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.372+0000 7f5f7f126b80 1 -- start start 2024-05-08T23:26:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a00f350 0x55fc4a038b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a059620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a03bfa0 0x55fc4a0572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a00fbd0 con 0x55fc4a03bfa0 2024-05-08T23:26:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a00f8d0 con 0x55fc4a00f350 2024-05-08T23:26:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a00b090 con 0x55fc4a0393b0 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a03bfa0 0x55fc4a0572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a03bfa0 0x55fc4a0572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46216/0 (socket says 172.21.15.5:46216) 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 -- 172.21.15.5:0/1371526467 learned_addr learned my addr 172.21.15.5:0/1371526467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a00f350 0x55fc4a038b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a059620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 -- 172.21.15.5:0/1371526467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 msgr2=0x55fc4a059620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a059620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 -- 172.21.15.5:0/1371526467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a00f350 msgr2=0x55fc4a038b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a00f350 0x55fc4a038b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 -- 172.21.15.5:0/1371526467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc49f27540 con 0x55fc4a03bfa0 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.373+0000 7f5f71509700 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a03bfa0 0x55fc4a0572f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5f64089300 tx=0x7f5f64056060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ed8770c4f9b7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f62ffd700 1 -- 172.21.15.5:0/1371526467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64062060 con 0x55fc4a03bfa0 2024-05-08T23:26:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f62ffd700 1 -- 172.21.15.5:0/1371526467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f64058010 con 0x55fc4a03bfa0 2024-05-08T23:26:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f62ffd700 1 -- 172.21.15.5:0/1371526467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64019330 con 0x55fc4a03bfa0 2024-05-08T23:26:36.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1371526467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a03bfa0 msgr2=0x55fc4a0572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a03bfa0 0x55fc4a0572f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5f64089300 tx=0x7f5f64056060 comp rx=0 tx=0).stop 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1371526467 shutdown_connections 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a03bfa0 0x55fc4a0572f0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a00f350 0x55fc4a038b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1371526467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a059620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1371526467 >> 172.21.15.5:0/1371526467 conn(0x55fc49f6f1f0 msgr2=0x55fc4a00e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1371526467 shutdown_connections 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.374+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1371526467 wait complete. 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.375+0000 7f5f7f126b80 1 -- start start 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.375+0000 7f5f7f126b80 1 Processor -- start 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a058a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f7f126b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a04cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a04d600 con 0x55fc4a00f350 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a0348c0 con 0x55fc4a03bfa0 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f7f126b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a0585a0 con 0x55fc4a0393b0 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f7f126b80 10 client.0 _ll_register_callbacks cb 0x55fc49f06b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f70d08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f70d08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a06a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46220/0 (socket says 172.21.15.5:46220) 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f70d08700 1 -- 172.21.15.5:0/1567093214 learned_addr learned my addr 172.21.15.5:0/1567093214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f71509700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a04cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.376+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a058a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f70d08700 1 -- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 msgr2=0x55fc4a058a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a058a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f70d08700 1 -- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 msgr2=0x55fc4a04cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a04cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f70d08700 1 -- 172.21.15.5:0/1567093214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c0c9310 con 0x55fc4a00f350 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f71509700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a04cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a058a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f70d08700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a06a270 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009f70 tx=0x7f5f5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46401d2f1d7c626 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/1567093214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c039040 con 0x55fc4a00f350 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/1567093214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f5c005c20 con 0x55fc4a00f350 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1567093214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f54004de0 con 0x55fc4a00f350 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/1567093214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f5c005de0 con 0x55fc4a00f350 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f7f126b80 10 client.7539 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1567093214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55fc49de6cb0 con 0x55fc4a00f350 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.377+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/1567093214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f54004de0 con 0x55fc4a00f350 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/1567093214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f5f5c006240 con 0x55fc4a00f350 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f7f126b80 10 client.7539 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f7f126b80 10 client.7539 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f7f126b80 20 client.7539 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55fc4a032ce0 0x55fc4a0330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f7f126b80 4 client.7539 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1567093214 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55fc49f59ea0 con 0x55fc4a032ce0 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55fc4a032ce0 0x55fc4a0330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f6bfff700 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55fc4a032ce0 0x55fc4a0330c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1434b662d091b7ff server_cookie=12d81cf9894c619a in_seq=0 out_seq=0 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f4b7fe700 10 client.7539 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f4b7fe700 1 -- 172.21.15.5:0/1567093214 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x55fc49f59ea0 con 0x55fc4a032ce0 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f4b7fe700 10 client.7539 handle_command_reply: tid=0 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.378+0000 7f5f7f126b80 1 client.7539 shutdown 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1567093214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 msgr2=0x55fc4a06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a06a270 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009f70 tx=0x7f5f5c017000 comp rx=0 tx=0).stop 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1567093214 shutdown_connections 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55fc4a032ce0 0x55fc4a0330c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc4a03bfa0 0x55fc4a04cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc4a00f350 0x55fc4a06a270 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f4b7fe700 0 client.7539 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 --2- 172.21.15.5:0/1567093214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55fc4a0393b0 0x55fc4a058a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1567093214 >> 172.21.15.5:0/1567093214 conn(0x55fc49f6f1f0 msgr2=0x55fc4a03fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1567093214 shutdown_connections 2024-05-08T23:26:36.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 1 -- 172.21.15.5:0/1567093214 wait complete. 2024-05-08T23:26:36.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.379+0000 7f5f7f126b80 20 client.7539 trim_cache size 0 max 16384 2024-05-08T23:26:36.385 INFO:teuthology.orchestra.run.smithi005.stderr:Error EXDEV: Not rank 0 2024-05-08T23:26:36.385 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-05-08T23:26:36.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub pause 2024-05-08T23:26:36.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.527+0000 7f86a7f8c700 1 Processor -- start 2024-05-08T23:26:36.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.527+0000 7f86a7f8c700 1 -- start start 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.528+0000 7f86a7f8c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f86a00c4410 0x7f86a00c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.528+0000 7f86a7f8c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00d0630 0x7f86a006bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.528+0000 7f86a7f8c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a01668e0 con 0x7f86a006bb50 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.528+0000 7f86a7f8c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a006c4f0 con 0x7f86a00c4410 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.528+0000 7f86a7f8c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00c4d30 con 0x7f86a00d0630 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a5527700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f86a006bb50 0x7f86a0099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:56134/0 (socket says 172.21.15.5:56134) 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a5d28700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f86a00c4410 0x7f86a00c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a5527700 1 -- 172.21.15.5:0/4224374489 learned_addr learned my addr 172.21.15.5:0/4224374489 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a6529700 1 --2- 172.21.15.5:0/4224374489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00d0630 0x7f86a006bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a4d26700 1 -- 172.21.15.5:0/4224374489 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691849171 0 0) 0x7f86a01668e0 con 0x7f86a006bb50 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a4d26700 1 -- 172.21.15.5:0/4224374489 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f868c0035b0 con 0x7f86a006bb50 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a6529700 1 -- 172.21.15.5:0/4224374489 >> v2:172.21.15.49:3300/0 conn(0x7f86a00c4410 msgr2=0x7f86a00c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a6529700 1 --2- 172.21.15.5:0/4224374489 >> v2:172.21.15.49:3300/0 conn(0x7f86a00c4410 0x7f86a00c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a6529700 1 -- 172.21.15.5:0/4224374489 >> v1:172.21.15.49:6789/0 conn(0x7f86a006bb50 legacy=0x7f86a0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a6529700 1 -- 172.21.15.5:0/4224374489 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a01659f0 con 0x7f86a00d0630 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a5d28700 1 --2- 172.21.15.5:0/4224374489 >> v2:172.21.15.49:3300/0 conn(0x7f86a00c4410 0x7f86a00c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.529+0000 7f86a6529700 1 --2- 172.21.15.5:0/4224374489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00d0630 0x7f86a006bfb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8698009d90 tx=0x7f869800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d54c03678dba5db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a4d26700 1 -- 172.21.15.5:0/4224374489 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800e070 con 0x7f86a00d0630 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a4d26700 1 -- 172.21.15.5:0/4224374489 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869800bb20 con 0x7f86a00d0630 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a4d26700 1 -- 172.21.15.5:0/4224374489 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698012430 con 0x7f86a00d0630 2024-05-08T23:26:36.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a7f8c700 1 -- 172.21.15.5:0/4224374489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00d0630 msgr2=0x7f86a006bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a7f8c700 1 --2- 172.21.15.5:0/4224374489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00d0630 0x7f86a006bfb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8698009d90 tx=0x7f869800b330 comp rx=0 tx=0).stop 2024-05-08T23:26:36.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a7f8c700 1 -- 172.21.15.5:0/4224374489 shutdown_connections 2024-05-08T23:26:36.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a7f8c700 1 --2- 172.21.15.5:0/4224374489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00d0630 0x7f86a006bfb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a7f8c700 1 --2- 172.21.15.5:0/4224374489 >> v2:172.21.15.49:3300/0 conn(0x7f86a00c4410 0x7f86a00c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a7f8c700 1 -- 172.21.15.5:0/4224374489 >> 172.21.15.5:0/4224374489 conn(0x7f86a00b2f30 msgr2=0x7f86a00b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a7f8c700 1 -- 172.21.15.5:0/4224374489 shutdown_connections 2024-05-08T23:26:36.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.530+0000 7f86a7f8c700 1 -- 172.21.15.5:0/4224374489 wait complete. 2024-05-08T23:26:36.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.531+0000 7f86a7f8c700 1 Processor -- start 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a7f8c700 1 -- start start 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a7f8c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a006bb50 0x7f86a0162a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a7f8c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00c4410 0x7f86a00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a7f8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86a00d0630 0x7f86a00cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a7f8c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a01652d0 con 0x7f86a00d0630 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a7f8c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00b4b00 con 0x7f86a006bb50 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a7f8c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00b4de0 con 0x7f86a00c4410 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a6529700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86a00d0630 0x7f86a00cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a6529700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86a00d0630 0x7f86a00cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46224/0 (socket says 172.21.15.5:46224) 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.532+0000 7f86a6529700 1 -- 172.21.15.5:0/1072118359 learned_addr learned my addr 172.21.15.5:0/1072118359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a5d28700 1 --2- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a006bb50 0x7f86a0162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a5527700 1 --2- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00c4410 0x7f86a00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a6529700 1 -- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00c4410 msgr2=0x7f86a00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a6529700 1 --2- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86a00c4410 0x7f86a00cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a6529700 1 -- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a006bb50 msgr2=0x7f86a0162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a6529700 1 --2- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86a006bb50 0x7f86a0162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a6529700 1 -- 172.21.15.5:0/1072118359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0094a0 con 0x7f86a00d0630 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a6529700 1 --2- 172.21.15.5:0/1072118359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86a00d0630 0x7f86a00cce40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8698014040 tx=0x7f8698004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86967fc700 1 -- 172.21.15.5:0/1072118359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980129f0 con 0x7f86a00d0630 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86967fc700 1 -- 172.21.15.5:0/1072118359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698020070 con 0x7f86a00d0630 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86967fc700 1 -- 172.21.15.5:0/1072118359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698012db0 con 0x7f86a00d0630 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a7f8c700 1 -- 172.21.15.5:0/1072118359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8698009a70 con 0x7f86a00d0630 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.533+0000 7f86a7f8c700 1 -- 172.21.15.5:0/1072118359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a00b53a0 con 0x7f86a00d0630 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.535+0000 7f86967fc700 1 -- 172.21.15.5:0/1072118359 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8698004040 con 0x7f86a00d0630 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.536+0000 7f86967fc700 1 --2- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f868406ac90 0x7f868406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.536+0000 7f86967fc700 1 -- 172.21.15.5:0/1072118359 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8698022070 con 0x7f86a00d0630 2024-05-08T23:26:36.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.536+0000 7f86a5d28700 1 --2- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f868406ac90 0x7f868406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.537+0000 7f86a5d28700 1 --2- 172.21.15.5:0/1072118359 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f868406ac90 0x7f868406d150 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f869c009d70 tx=0x7f869c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.538+0000 7f86b4146b80 1 Processor -- start 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.538+0000 7f86b4146b80 1 -- start start 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.538+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb2780 0x563a8ac8b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 0x563a8ac90080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8acba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86b4146b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8abe6130 con 0x563a8acb9db0 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86b4146b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8abe3af0 con 0x563a8acb2780 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86b4146b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a8abdf3e0 con 0x563a8ac8d350 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8acba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8acba230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46228/0 (socket says 172.21.15.5:46228) 2024-05-08T23:26:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 -- 172.21.15.5:0/1909523599 learned_addr learned my addr 172.21.15.5:0/1909523599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a5d28700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb2780 0x563a8ac8b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a5527700 1 --2- 172.21.15.5:0/1909523599 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 0x563a8ac90080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 -- 172.21.15.5:0/1909523599 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 msgr2=0x563a8ac90080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 --2- 172.21.15.5:0/1909523599 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 0x563a8ac90080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 -- 172.21.15.5:0/1909523599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb2780 msgr2=0x563a8ac8b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 --2- 172.21.15.5:0/1909523599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb2780 0x563a8ac8b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 -- 172.21.15.5:0/1909523599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a8aa60390 con 0x563a8acb9db0 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86a6529700 1 --2- 172.21.15.5:0/1909523599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8acba230 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8698012fd0 tx=0x7f869800cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b67a03ed2f6e9c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86837fe700 1 -- 172.21.15.5:0/1909523599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698059d60 con 0x563a8acb9db0 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86837fe700 1 -- 172.21.15.5:0/1909523599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869805db90 con 0x563a8acb9db0 2024-05-08T23:26:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.539+0000 7f86837fe700 1 -- 172.21.15.5:0/1909523599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698062150 con 0x563a8acb9db0 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 -- 172.21.15.5:0/1909523599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 msgr2=0x563a8acba230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1909523599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8acba230 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8698012fd0 tx=0x7f869800cfd0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 -- 172.21.15.5:0/1909523599 shutdown_connections 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1909523599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8acba230 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1909523599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb2780 0x563a8ac8b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1909523599 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 0x563a8ac90080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 -- 172.21.15.5:0/1909523599 >> 172.21.15.5:0/1909523599 conn(0x563a8abed1f0 msgr2=0x563a8acd4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 -- 172.21.15.5:0/1909523599 shutdown_connections 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.540+0000 7f86b4146b80 1 -- 172.21.15.5:0/1909523599 wait complete. 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.541+0000 7f86b4146b80 1 -- start start 2024-05-08T23:26:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.541+0000 7f86b4146b80 1 Processor -- start 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 0x563a8acd4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8acb8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb9db0 0x563a8acb8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86b4146b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8ab7b540 con 0x563a8acb2780 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86b4146b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8ab7b1e0 con 0x563a8acb9db0 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86b4146b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a8aa62680 con 0x563a8ac8d350 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86b4146b80 10 client.0 _ll_register_callbacks cb 0x563a8ab84b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5527700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8acb8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5527700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8acb8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46230/0 (socket says 172.21.15.5:46230) 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5527700 1 -- 172.21.15.5:0/3710384667 learned_addr learned my addr 172.21.15.5:0/3710384667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5d28700 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 0x563a8acd4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a6529700 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb9db0 0x563a8acb8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5527700 1 -- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 msgr2=0x563a8acd4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5527700 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 0x563a8acd4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5527700 1 -- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb9db0 msgr2=0x563a8acb8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5527700 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb9db0 0x563a8acb8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.542+0000 7f86a5527700 1 -- 172.21.15.5:0/3710384667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a00b53a0 con 0x563a8acb2780 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86a5527700 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8acb8330 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f86a015fd10 tx=0x7f8690001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1187bead88027d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f8680ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f8680ff9700 1 -- 172.21.15.5:0/3710384667 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690015440 con 0x563a8acb2780 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f8680ff9700 1 -- 172.21.15.5:0/3710384667 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86900155a0 con 0x563a8acb2780 2024-05-08T23:26:36.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 1 -- 172.21.15.5:0/3710384667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a8ab38250 con 0x563a8acb2780 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f8680ff9700 1 -- 172.21.15.5:0/3710384667 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8690015760 con 0x563a8acb2780 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 10 client.7548 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 1 -- 172.21.15.5:0/3710384667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x563a8aa0cb40 con 0x563a8acb2780 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f8680ff9700 1 -- 172.21.15.5:0/3710384667 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a8ab38250 con 0x563a8acb2780 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f8680ff9700 1 -- 172.21.15.5:0/3710384667 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f8690012050 con 0x563a8acb2780 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 10 client.7548 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 10 client.7548 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 20 client.7548 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x563a8ace7fa0 0x563a8acd70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 4 client.7548 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.543+0000 7f86b4146b80 1 -- 172.21.15.5:0/3710384667 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a8ac5e7c0 con 0x563a8ace7fa0 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.544+0000 7f86a5d28700 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x563a8ace7fa0 0x563a8acd70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.544+0000 7f8680ff9700 10 client.7548 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.544+0000 7f86a5d28700 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x563a8ace7fa0 0x563a8acd70e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f14829f9ac71aca server_cookie=719ba80dfdff7ad3 in_seq=0 out_seq=0 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f8680ff9700 1 -- 172.21.15.5:0/3710384667 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a8ac5e7c0 con 0x563a8ace7fa0 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f8680ff9700 10 client.7548 handle_command_reply: tid=0 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 client.7548 shutdown 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 -- 172.21.15.5:0/3710384667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 msgr2=0x563a8acb8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8acb8330 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f86a015fd10 tx=0x7f8690001650 comp rx=0 tx=0).stop 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 -- 172.21.15.5:0/3710384667 shutdown_connections 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x563a8ace7fa0 0x563a8acd70e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8acb9db0 0x563a8acb8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8ac8d350 0x563a8acd4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f8680ff9700 0 client.7548 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 --2- 172.21.15.5:0/3710384667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8acb8330 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 -- 172.21.15.5:0/3710384667 >> 172.21.15.5:0/3710384667 conn(0x563a8abed1f0 msgr2=0x563a8acba1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 -- 172.21.15.5:0/3710384667 shutdown_connections 2024-05-08T23:26:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 1 -- 172.21.15.5:0/3710384667 wait complete. 2024-05-08T23:26:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.545+0000 7f86b4146b80 20 client.7548 trim_cache size 0 max 16384 2024-05-08T23:26:36.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.563+0000 7f86b4146b80 1 Processor -- start 2024-05-08T23:26:36.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86b4146b80 1 -- start start 2024-05-08T23:26:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ac8b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8ac8fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb9db0 0x563a8ac902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86b4146b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8acf7210 con 0x563a8acb2780 2024-05-08T23:26:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86b4146b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8acd8400 con 0x563a8ac8d350 2024-05-08T23:26:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86b4146b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a8ac8f720 con 0x563a8acb9db0 2024-05-08T23:26:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86a5527700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8ac8fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86a5527700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8ac8fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46236/0 (socket says 172.21.15.5:46236) 2024-05-08T23:26:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.564+0000 7f86a5527700 1 -- 172.21.15.5:0/424281702 learned_addr learned my addr 172.21.15.5:0/424281702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a6529700 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb9db0 0x563a8ac902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a5d28700 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ac8b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a5527700 1 -- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb9db0 msgr2=0x563a8ac902e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a5527700 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb9db0 0x563a8ac902e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a5527700 1 -- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 msgr2=0x563a8ac8b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a5527700 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ac8b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a5527700 1 -- 172.21.15.5:0/424281702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a8aba5540 con 0x563a8acb2780 2024-05-08T23:26:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a5d28700 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ac8b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f86a5527700 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8ac8fda0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8690001300 tx=0x7f869000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f9db45296f015f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f8683fff700 1 -- 172.21.15.5:0/424281702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690015c50 con 0x563a8acb2780 2024-05-08T23:26:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f8683fff700 1 -- 172.21.15.5:0/424281702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869002b070 con 0x563a8acb2780 2024-05-08T23:26:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.565+0000 7f8683fff700 1 -- 172.21.15.5:0/424281702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690015440 con 0x563a8acb2780 2024-05-08T23:26:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 -- 172.21.15.5:0/424281702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 msgr2=0x563a8ac8fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8ac8fda0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8690001300 tx=0x7f869000f000 comp rx=0 tx=0).stop 2024-05-08T23:26:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 -- 172.21.15.5:0/424281702 shutdown_connections 2024-05-08T23:26:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb9db0 0x563a8ac902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ac8b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 --2- 172.21.15.5:0/424281702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb2780 0x563a8ac8fda0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 -- 172.21.15.5:0/424281702 >> 172.21.15.5:0/424281702 conn(0x563a8abed1f0 msgr2=0x563a8ac861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 -- 172.21.15.5:0/424281702 shutdown_connections 2024-05-08T23:26:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 -- 172.21.15.5:0/424281702 wait complete. 2024-05-08T23:26:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 -- start start 2024-05-08T23:26:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.566+0000 7f86b4146b80 1 Processor -- start 2024-05-08T23:26:36.576 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-05-08T23:26:36.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.567+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ad26c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.567+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb2780 0x563a8acca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.567+0000 7f86b4146b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8accac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.567+0000 7f86b4146b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8ad3df10 con 0x563a8acb9db0 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.567+0000 7f86b4146b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8ac902b0 con 0x563a8ac8d350 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.567+0000 7f86b4146b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a8ad3dac0 con 0x563a8acb2780 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.567+0000 7f86b4146b80 10 client.0 _ll_register_callbacks cb 0x563a8ab84b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8accac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8accac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46252/0 (socket says 172.21.15.5:46252) 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 -- 172.21.15.5:0/1256522435 learned_addr learned my addr 172.21.15.5:0/1256522435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a5d28700 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ad26c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a5527700 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb2780 0x563a8acca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 -- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb2780 msgr2=0x563a8acca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb2780 0x563a8acca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 -- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 msgr2=0x563a8ad26c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ad26c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 -- 172.21.15.5:0/1256522435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a8aa0cb40 con 0x563a8acb9db0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86a6529700 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8accac80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8698059cf0 tx=0x7f869801e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825bead1b7bd6651 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f8680ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f8680ff9700 1 -- 172.21.15.5:0/1256522435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698062150 con 0x563a8acb9db0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f8680ff9700 1 -- 172.21.15.5:0/1256522435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8698076070 con 0x563a8acb9db0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86b4146b80 1 -- 172.21.15.5:0/1256522435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f869805db90 con 0x563a8acb9db0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f8680ff9700 1 -- 172.21.15.5:0/1256522435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8698059d60 con 0x563a8acb9db0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86b4146b80 10 client.7554 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f86b4146b80 1 -- 172.21.15.5:0/1256522435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x563a8aa64cb0 con 0x563a8acb9db0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.568+0000 7f8680ff9700 1 -- 172.21.15.5:0/1256522435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869805db90 con 0x563a8acb9db0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f8680ff9700 1 -- 172.21.15.5:0/1256522435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f8698061cc0 con 0x563a8acb9db0 2024-05-08T23:26:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 10 client.7554 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 10 client.7554 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 20 client.7554 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x563a8acb0180 0x563a8acb0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 4 client.7554 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 1 -- 172.21.15.5:0/1256522435 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x563a8abd7ea0 con 0x563a8acb0180 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86a5d28700 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x563a8acb0180 0x563a8acb0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f8680ff9700 10 client.7554 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86a5d28700 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x563a8acb0180 0x563a8acb0560 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f6271f372c5f353 server_cookie=b7fcffd3a60e394 in_seq=0 out_seq=0 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f8680ff9700 1 -- 172.21.15.5:0/1256522435 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x563a8abd7ea0 con 0x563a8acb0180 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f8680ff9700 10 client.7554 handle_command_reply: tid=0 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 1 client.7554 shutdown 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 1 -- 172.21.15.5:0/1256522435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 msgr2=0x563a8accac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.569+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8accac80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8698059cf0 tx=0x7f869801e070 comp rx=0 tx=0).stop 2024-05-08T23:26:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 1 -- 172.21.15.5:0/1256522435 shutdown_connections 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x563a8acb0180 0x563a8acb0560 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a8acb9db0 0x563a8accac80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563a8ac8d350 0x563a8ad26c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f8680ff9700 0 client.7554 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 1 --2- 172.21.15.5:0/1256522435 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563a8acb2780 0x563a8acca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 1 -- 172.21.15.5:0/1256522435 >> 172.21.15.5:0/1256522435 conn(0x563a8abed1f0 msgr2=0x563a8acb90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 1 -- 172.21.15.5:0/1256522435 shutdown_connections 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 1 -- 172.21.15.5:0/1256522435 wait complete. 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.570+0000 7f86b4146b80 20 client.7554 trim_cache size 0 max 16384 2024-05-08T23:26:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:Error EXDEV: Not rank 0 2024-05-08T23:26:36.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub resume 2024-05-08T23:26:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.719+0000 7f87e137c700 1 Processor -- start 2024-05-08T23:26:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.719+0000 7f87e137c700 1 -- start start 2024-05-08T23:26:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87e137c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87dc15dae0 0x7f87dc1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87e137c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f87dc161a30 0x7f87dc0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87e137c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc161e80 con 0x7f87dc15d700 2024-05-08T23:26:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87e137c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0b49b0 con 0x7f87dc161a30 2024-05-08T23:26:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87e137c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0b3da0 con 0x7f87dc15dae0 2024-05-08T23:26:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87db7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f87dc161a30 0x7f87dc0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87daffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f87dc15d700 0x7f87dc0b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53380/0 (socket says 172.21.15.5:53380) 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87daffd700 1 -- 172.21.15.5:0/1734511437 learned_addr learned my addr 172.21.15.5:0/1734511437 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87da7fc700 1 --2- 172.21.15.5:0/1734511437 >> v2:172.21.15.49:3300/0 conn(0x7f87dc15dae0 0x7f87dc1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87db7fe700 1 -- 172.21.15.5:0/1734511437 >> v2:172.21.15.49:3300/0 conn(0x7f87dc15dae0 msgr2=0x7f87dc1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87db7fe700 1 --2- 172.21.15.5:0/1734511437 >> v2:172.21.15.49:3300/0 conn(0x7f87dc15dae0 0x7f87dc1614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87db7fe700 1 -- 172.21.15.5:0/1734511437 >> v1:172.21.15.5:6789/0 conn(0x7f87dc15d700 legacy=0x7f87dc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87db7fe700 1 -- 172.21.15.5:0/1734511437 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0b4080 con 0x7f87dc161a30 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.720+0000 7f87d9ffb700 1 -- 172.21.15.5:0/1734511437 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508623335 0 0) 0x7f87cc001290 con 0x7f87dc15d700 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87db7fe700 1 --2- 172.21.15.5:0/1734511437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87dc161a30 0x7f87dc0b0140 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0ce190 tx=0x7f87d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db8ae3eb88d0bc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87d9ffb700 1 -- 172.21.15.5:0/1734511437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400e070 con 0x7f87dc161a30 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87d9ffb700 1 -- 172.21.15.5:0/1734511437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d400bbd0 con 0x7f87dc161a30 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87d9ffb700 1 -- 172.21.15.5:0/1734511437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4012430 con 0x7f87dc161a30 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87e137c700 1 -- 172.21.15.5:0/1734511437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87dc161a30 msgr2=0x7f87dc0b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87e137c700 1 --2- 172.21.15.5:0/1734511437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87dc161a30 0x7f87dc0b0140 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0ce190 tx=0x7f87d400b330 comp rx=0 tx=0).stop 2024-05-08T23:26:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87e137c700 1 -- 172.21.15.5:0/1734511437 shutdown_connections 2024-05-08T23:26:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87e137c700 1 --2- 172.21.15.5:0/1734511437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87dc161a30 0x7f87dc0b0140 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87e137c700 1 --2- 172.21.15.5:0/1734511437 >> v2:172.21.15.49:3300/0 conn(0x7f87dc15dae0 0x7f87dc1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87e137c700 1 -- 172.21.15.5:0/1734511437 >> 172.21.15.5:0/1734511437 conn(0x7f87dc08afe0 msgr2=0x7f87dc0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87e137c700 1 -- 172.21.15.5:0/1734511437 shutdown_connections 2024-05-08T23:26:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.721+0000 7f87e137c700 1 -- 172.21.15.5:0/1734511437 wait complete. 2024-05-08T23:26:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.723+0000 7f87e137c700 1 Processor -- start 2024-05-08T23:26:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.723+0000 7f87e137c700 1 -- start start 2024-05-08T23:26:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.724+0000 7f87e137c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87dc15d700 0x7f87dc151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.724+0000 7f87e137c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87dc15dae0 0x7f87dc0b8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.724+0000 7f87e137c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87dc161a30 0x7f87dc0b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.724+0000 7f87e137c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0b8ea0 con 0x7f87dc161a30 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.724+0000 7f87e137c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0b1c40 con 0x7f87dc15dae0 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.724+0000 7f87e137c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1523d0 con 0x7f87dc15d700 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.724+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87dc161a30 0x7f87dc0b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87dc161a30 0x7f87dc0b8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46266/0 (socket says 172.21.15.5:46266) 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87db7fe700 1 -- 172.21.15.5:0/2907045120 learned_addr learned my addr 172.21.15.5:0/2907045120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87da7fc700 1 --2- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87dc15dae0 0x7f87dc0b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87daffd700 1 --2- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87dc15d700 0x7f87dc151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87db7fe700 1 -- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87dc15d700 msgr2=0x7f87dc151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87db7fe700 1 --2- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f87dc15d700 0x7f87dc151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87db7fe700 1 -- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87dc15dae0 msgr2=0x7f87dc0b8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87db7fe700 1 --2- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87dc15dae0 0x7f87dc0b8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87db7fe700 1 -- 172.21.15.5:0/2907045120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4009a70 con 0x7f87dc161a30 2024-05-08T23:26:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87db7fe700 1 --2- 172.21.15.5:0/2907045120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87dc161a30 0x7f87dc0b8960 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f87dc08ba20 tx=0x7f87d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87c37fe700 1 -- 172.21.15.5:0/2907045120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4012430 con 0x7f87dc161a30 2024-05-08T23:26:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87c37fe700 1 -- 172.21.15.5:0/2907045120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d4020070 con 0x7f87dc161a30 2024-05-08T23:26:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87e137c700 1 -- 172.21.15.5:0/2907045120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc0b1ee0 con 0x7f87dc161a30 2024-05-08T23:26:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87c37fe700 1 -- 172.21.15.5:0/2907045120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4004280 con 0x7f87dc161a30 2024-05-08T23:26:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.725+0000 7f87e137c700 1 -- 172.21.15.5:0/2907045120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0b24a0 con 0x7f87dc161a30 2024-05-08T23:26:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.727+0000 7f87c37fe700 1 -- 172.21.15.5:0/2907045120 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f87d40043e0 con 0x7f87dc161a30 2024-05-08T23:26:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.727+0000 7f87c37fe700 1 --2- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87c806ac90 0x7f87c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.727+0000 7f87c37fe700 1 -- 172.21.15.5:0/2907045120 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87d4022070 con 0x7f87dc161a30 2024-05-08T23:26:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.728+0000 7f87daffd700 1 --2- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87c806ac90 0x7f87c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.728+0000 7f87daffd700 1 --2- 172.21.15.5:0/2907045120 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f87c806ac90 0x7f87c806d150 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f87cc007fd0 tx=0x7f87cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.730+0000 7f87ed536b80 1 Processor -- start 2024-05-08T23:26:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.730+0000 7f87ed536b80 1 -- start start 2024-05-08T23:26:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 0x55efb1a4b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a4d360 0x55efb1a50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87ed536b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb19a6140 con 0x55efb1a4d360 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87ed536b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb19a3b00 con 0x55efb1a79dc0 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87ed536b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efb199f3f0 con 0x55efb1a72790 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a4d360 0x55efb1a50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a4d360 0x55efb1a50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46278/0 (socket says 172.21.15.5:46278) 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 -- 172.21.15.5:0/1772409420 learned_addr learned my addr 172.21.15.5:0/1772409420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87daffd700 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 0x55efb1a4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87db7fe700 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 -- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 msgr2=0x55efb1a4b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 0x55efb1a4b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 -- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a79dc0 msgr2=0x55efb1a7a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 -- 172.21.15.5:0/1772409420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efb18203a0 con 0x55efb1a4d360 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87db7fe700 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87da7fc700 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a4d360 0x55efb1a50090 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009b30 tx=0x7f87d0009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fa6f6f756b9529 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.731+0000 7f87c0ff9700 1 -- 172.21.15.5:0/1772409420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0010af0 con 0x55efb1a4d360 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87c0ff9700 1 -- 172.21.15.5:0/1772409420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d000c050 con 0x55efb1a4d360 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87c0ff9700 1 -- 172.21.15.5:0/1772409420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0010e90 con 0x55efb1a4d360 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 -- 172.21.15.5:0/1772409420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a4d360 msgr2=0x55efb1a50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a4d360 0x55efb1a50090 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009b30 tx=0x7f87d0009e40 comp rx=0 tx=0).stop 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 -- 172.21.15.5:0/1772409420 shutdown_connections 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 0x55efb1a4b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1772409420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a4d360 0x55efb1a50090 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 -- 172.21.15.5:0/1772409420 >> 172.21.15.5:0/1772409420 conn(0x55efb19ad200 msgr2=0x55efb1a94dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 -- 172.21.15.5:0/1772409420 shutdown_connections 2024-05-08T23:26:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.732+0000 7f87ed536b80 1 -- 172.21.15.5:0/1772409420 wait complete. 2024-05-08T23:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.733+0000 7f87ed536b80 1 -- start start 2024-05-08T23:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.733+0000 7f87ed536b80 1 Processor -- start 2024-05-08T23:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a4d360 0x55efb1aa3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 0x55efb1a96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a96780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87ed536b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb193b550 con 0x55efb1a79dc0 2024-05-08T23:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87ed536b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb193b1f0 con 0x55efb1a4d360 2024-05-08T23:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87ed536b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efb1822690 con 0x55efb1a72790 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87ed536b80 10 client.0 _ll_register_callbacks cb 0x55efb1944b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a96780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a96780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46280/0 (socket says 172.21.15.5:46280) 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87da7fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 0x55efb1a96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87db7fe700 1 -- 172.21.15.5:0/4142566351 learned_addr learned my addr 172.21.15.5:0/4142566351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.734+0000 7f87daffd700 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a4d360 0x55efb1aa3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87db7fe700 1 -- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 msgr2=0x55efb1a96240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87db7fe700 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 0x55efb1a96240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87db7fe700 1 -- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a4d360 msgr2=0x55efb1aa3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87db7fe700 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a4d360 0x55efb1aa3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87db7fe700 1 -- 172.21.15.5:0/4142566351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d0009760 con 0x55efb1a79dc0 2024-05-08T23:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87db7fe700 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a96780 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f87d4059cc0 tx=0x7f87d405a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a8b5a67fbc87da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87b9ffb700 1 -- 172.21.15.5:0/4142566351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4059fc0 con 0x55efb1a79dc0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87b9ffb700 1 -- 172.21.15.5:0/4142566351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87d405ddf0 con 0x55efb1a79dc0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87ed536b80 1 -- 172.21.15.5:0/4142566351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efb18f8260 con 0x55efb1a79dc0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87b9ffb700 1 -- 172.21.15.5:0/4142566351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d40623b0 con 0x55efb1a79dc0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87ed536b80 10 client.7569 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87ed536b80 1 -- 172.21.15.5:0/4142566351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55efb17ccb50 con 0x55efb1a79dc0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87b9ffb700 1 -- 172.21.15.5:0/4142566351 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55efb18f8260 con 0x55efb1a79dc0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87b9ffb700 1 -- 172.21.15.5:0/4142566351 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f87d4089170 con 0x55efb1a79dc0 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87ed536b80 10 client.7569 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87ed536b80 10 client.7569 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87ed536b80 20 client.7569 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.735+0000 7f87ed536b80 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55efb1a7d4b0 0x55efb1ab1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.736+0000 7f87ed536b80 4 client.7569 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.736+0000 7f87ed536b80 1 -- 172.21.15.5:0/4142566351 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efb1a1e7d0 con 0x55efb1a7d4b0 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.736+0000 7f87daffd700 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55efb1a7d4b0 0x55efb1ab1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.736+0000 7f87b9ffb700 10 client.7569 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.736+0000 7f87daffd700 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55efb1a7d4b0 0x55efb1ab1920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c79efd528f99ce99 server_cookie=a03bb31edded13dd in_seq=0 out_seq=0 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87b9ffb700 1 -- 172.21.15.5:0/4142566351 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55efb1a1e7d0 con 0x55efb1a7d4b0 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87b9ffb700 10 client.7569 handle_command_reply: tid=0 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 client.7569 shutdown 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 -- 172.21.15.5:0/4142566351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 msgr2=0x55efb1a96780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a96780 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f87d4059cc0 tx=0x7f87d405a500 comp rx=0 tx=0).stop 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 -- 172.21.15.5:0/4142566351 shutdown_connections 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55efb1a7d4b0 0x55efb1ab1920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a96780 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a4d360 0x55efb1aa3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87b9ffb700 0 client.7569 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 --2- 172.21.15.5:0/4142566351 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a72790 0x55efb1a96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 -- 172.21.15.5:0/4142566351 >> 172.21.15.5:0/4142566351 conn(0x55efb19ad200 msgr2=0x55efb1a48cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 -- 172.21.15.5:0/4142566351 shutdown_connections 2024-05-08T23:26:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 1 -- 172.21.15.5:0/4142566351 wait complete. 2024-05-08T23:26:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.737+0000 7f87ed536b80 20 client.7569 trim_cache size 0 max 16384 2024-05-08T23:26:36.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.755+0000 7f87ed536b80 1 Processor -- start 2024-05-08T23:26:36.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.755+0000 7f87ed536b80 1 -- start start 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1a764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a4a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87ed536b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb1a76260 con 0x55efb1a79dc0 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87ed536b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb1a7a770 con 0x55efb1a72790 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87ed536b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efb1a75ac0 con 0x55efb1a4d360 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46294/0 (socket says 172.21.15.5:46294) 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 -- 172.21.15.5:0/2779690474 learned_addr learned my addr 172.21.15.5:0/2779690474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87da7fc700 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a4a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87daffd700 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1a764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 -- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 msgr2=0x55efb1a764e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1a764e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 -- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 msgr2=0x55efb1a4a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a4a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 -- 172.21.15.5:0/2779690474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efb1965550 con 0x55efb1a79dc0 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87da7fc700 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a4a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87daffd700 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1a764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.756+0000 7f87db7fe700 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a230 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f87d400b730 tx=0x7f87d405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4895f0e4359d9ef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87c17fa700 1 -- 172.21.15.5:0/2779690474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4055db0 con 0x55efb1a79dc0 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87c17fa700 1 -- 172.21.15.5:0/2779690474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d405ddf0 con 0x55efb1a79dc0 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87c17fa700 1 -- 172.21.15.5:0/2779690474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d40623b0 con 0x55efb1a79dc0 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 -- 172.21.15.5:0/2779690474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 msgr2=0x55efb1a7a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a230 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f87d400b730 tx=0x7f87d405b000 comp rx=0 tx=0).stop 2024-05-08T23:26:36.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 -- 172.21.15.5:0/2779690474 shutdown_connections 2024-05-08T23:26:36.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a230 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1a764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 --2- 172.21.15.5:0/2779690474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a4a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 -- 172.21.15.5:0/2779690474 >> 172.21.15.5:0/2779690474 conn(0x55efb19ad200 msgr2=0x55efb1a461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 -- 172.21.15.5:0/2779690474 shutdown_connections 2024-05-08T23:26:36.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.757+0000 7f87ed536b80 1 -- 172.21.15.5:0/2779690474 wait complete. 2024-05-08T23:26:36.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.758+0000 7f87ed536b80 1 -- start start 2024-05-08T23:26:36.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.758+0000 7f87ed536b80 1 Processor -- start 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1afd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a7a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87ed536b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87ed536b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb1afdaf0 con 0x55efb1a79dc0 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87ed536b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb1a4a5b0 con 0x55efb1a72790 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87ed536b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efb1afd690 con 0x55efb1a4d360 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87ed536b80 10 client.0 _ll_register_callbacks cb 0x55efb1944b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46308/0 (socket says 172.21.15.5:46308) 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 -- 172.21.15.5:0/1927529985 learned_addr learned my addr 172.21.15.5:0/1927529985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:36.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87daffd700 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1afd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87da7fc700 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a7a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 -- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 msgr2=0x55efb1afd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1afd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 -- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 msgr2=0x55efb1a7a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a7a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 -- 172.21.15.5:0/1927529985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efb17ccb50 con 0x55efb1a79dc0 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87da7fc700 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a7a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87db7fe700 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a770 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f87d4059cc0 tx=0x7f87d4089520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8190bf7605d30a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87b9ffb700 1 -- 172.21.15.5:0/1927529985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4055db0 con 0x55efb1a79dc0 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87b9ffb700 1 -- 172.21.15.5:0/1927529985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87d4059fc0 con 0x55efb1a79dc0 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.759+0000 7f87ed536b80 1 -- 172.21.15.5:0/1927529985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87d000c050 con 0x55efb1a79dc0 2024-05-08T23:26:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87b9ffb700 1 -- 172.21.15.5:0/1927529985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d40623b0 con 0x55efb1a79dc0 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87ed536b80 10 client.7575 fetch_fsmap learned FSMap version 73 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87ed536b80 1 -- 172.21.15.5:0/1927529985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55efb1824cc0 con 0x55efb1a79dc0 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87b9ffb700 1 -- 172.21.15.5:0/1927529985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d000c050 con 0x55efb1a79dc0 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87b9ffb700 1 -- 172.21.15.5:0/1927529985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f87d4089170 con 0x55efb1a79dc0 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87ed536b80 10 client.7575 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87ed536b80 10 client.7575 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87ed536b80 20 client.7575 populate_metadata read hostname 'smithi005' 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55efb1b00da0 0x55efb1b01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87ed536b80 4 client.7575 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87ed536b80 1 -- 172.21.15.5:0/1927529985 --> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55efb1997eb0 con 0x55efb1b00da0 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.760+0000 7f87daffd700 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55efb1b00da0 0x55efb1b01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87b9ffb700 10 client.7575 ms_handle_connect on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87daffd700 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55efb1b00da0 0x55efb1b01180 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5b4e767b8b835b4 server_cookie=a5b23a4e195d7bb3 in_seq=0 out_seq=0 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87b9ffb700 1 -- 172.21.15.5:0/1927529985 <== mds.1 v2:172.21.15.49:6835/3300231911 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x55efb1997eb0 con 0x55efb1b00da0 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87b9ffb700 10 client.7575 handle_command_reply: tid=0 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 client.7575 shutdown 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 -- 172.21.15.5:0/1927529985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 msgr2=0x55efb1a7a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a770 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f87d4059cc0 tx=0x7f87d4089520 comp rx=0 tx=0).stop 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 -- 172.21.15.5:0/1927529985 shutdown_connections 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:6835/3300231911,v1:172.21.15.49:6837/3300231911] conn(0x55efb1b00da0 0x55efb1b01180 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb1a79dc0 0x55efb1a7a770 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55efb1a4d360 0x55efb1afd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87b9ffb700 0 client.7575 ms_handle_reset on v2:172.21.15.49:6835/3300231911 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 --2- 172.21.15.5:0/1927529985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb1a72790 0x55efb1a7a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 -- 172.21.15.5:0/1927529985 >> 172.21.15.5:0/1927529985 conn(0x55efb19ad200 msgr2=0x55efb1a76260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 -- 172.21.15.5:0/1927529985 shutdown_connections 2024-05-08T23:26:36.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 1 -- 172.21.15.5:0/1927529985 wait complete. 2024-05-08T23:26:36.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:36.761+0000 7f87ed536b80 20 client.7575 trim_cache size 0 max 16384 2024-05-08T23:26:36.768 INFO:teuthology.orchestra.run.smithi005.stderr:Error EXDEV: Not rank 0 2024-05-08T23:26:36.768 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-05-08T23:26:36.769 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:26:36.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:26:36.814 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:26:36.871 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:26:36.872 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:26:36.872 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:36.893 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:26:36.893 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:36.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.141+0000 7f26da363700 1 Processor -- start 2024-05-08T23:26:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.141+0000 7f26da363700 1 -- start start 2024-05-08T23:26:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.141+0000 7f26da363700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26d40ccbd0 0x7f26d4168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.141+0000 7f26da363700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d409a870 con 0x7f26d4161030 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.141+0000 7f26da363700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d40d0840 con 0x7f26d40b2480 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.141+0000 7f26da363700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d406ccc0 con 0x7f26d40ccbd0 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d3fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f26d4161030 0x7f26d4099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53388/0 (socket says 172.21.15.5:53388) 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d3fff700 1 -- 172.21.15.5:0/1033986261 learned_addr learned my addr 172.21.15.5:0/1033986261 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d8900700 1 --2- 172.21.15.5:0/1033986261 >> v2:172.21.15.49:3300/0 conn(0x7f26d40ccbd0 0x7f26d4168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d2ffd700 1 -- 172.21.15.5:0/1033986261 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143393694 0 0) 0x7f26d409a870 con 0x7f26d4161030 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d2ffd700 1 -- 172.21.15.5:0/1033986261 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26bc0035b0 con 0x7f26d4161030 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d2ffd700 1 -- 172.21.15.5:0/1033986261 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672210208 0 0) 0x7f26d40d0840 con 0x7f26d40b2480 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d2ffd700 1 -- 172.21.15.5:0/1033986261 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d409a870 con 0x7f26d40b2480 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d8900700 1 -- 172.21.15.5:0/1033986261 >> v1:172.21.15.49:6789/0 conn(0x7f26d40b2480 legacy=0x7f26d409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d8900700 1 -- 172.21.15.5:0/1033986261 >> v1:172.21.15.5:6789/0 conn(0x7f26d4161030 legacy=0x7f26d4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d8900700 1 -- 172.21.15.5:0/1033986261 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d406c600 con 0x7f26d40ccbd0 2024-05-08T23:26:37.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.142+0000 7f26d8900700 1 --2- 172.21.15.5:0/1033986261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40ccbd0 0x7f26d4168580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009d90 tx=0x7f26cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4aa05d71a29440 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26d2ffd700 1 -- 172.21.15.5:0/1033986261 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00e070 con 0x7f26d40ccbd0 2024-05-08T23:26:37.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26d2ffd700 1 -- 172.21.15.5:0/1033986261 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc00bb20 con 0x7f26d40ccbd0 2024-05-08T23:26:37.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26d2ffd700 1 -- 172.21.15.5:0/1033986261 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc012430 con 0x7f26d40ccbd0 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26da363700 1 -- 172.21.15.5:0/1033986261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40ccbd0 msgr2=0x7f26d4168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26da363700 1 --2- 172.21.15.5:0/1033986261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40ccbd0 0x7f26d4168580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009d90 tx=0x7f26cc00b330 comp rx=0 tx=0).stop 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26da363700 1 -- 172.21.15.5:0/1033986261 shutdown_connections 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26da363700 1 --2- 172.21.15.5:0/1033986261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40ccbd0 0x7f26d4168580 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26da363700 1 -- 172.21.15.5:0/1033986261 >> 172.21.15.5:0/1033986261 conn(0x7f26d40b2f20 msgr2=0x7f26d40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26da363700 1 -- 172.21.15.5:0/1033986261 shutdown_connections 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.143+0000 7f26da363700 1 -- 172.21.15.5:0/1033986261 wait complete. 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.145+0000 7f26da363700 1 Processor -- start 2024-05-08T23:26:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.145+0000 7f26da363700 1 -- start start 2024-05-08T23:26:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26da363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40b2480 0x7f26d414a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26da363700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26d40ccbd0 0x7f26d4150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26da363700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26d4161030 0x7f26d4145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26da363700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40d0660 con 0x7f26d40ccbd0 2024-05-08T23:26:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26da363700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d4157bf0 con 0x7f26d40b2480 2024-05-08T23:26:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26da363700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d4157ed0 con 0x7f26d4161030 2024-05-08T23:26:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26d40ccbd0 0x7f26d4150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26d40ccbd0 0x7f26d4150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46320/0 (socket says 172.21.15.5:46320) 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 -- 172.21.15.5:0/2710856521 learned_addr learned my addr 172.21.15.5:0/2710856521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d3fff700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40b2480 0x7f26d414a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d8900700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26d4161030 0x7f26d4145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 -- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26d4161030 msgr2=0x7f26d4145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26d4161030 0x7f26d4145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 -- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40b2480 msgr2=0x7f26d414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40b2480 0x7f26d414a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 -- 172.21.15.5:0/2710856521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc009a70 con 0x7f26d40ccbd0 2024-05-08T23:26:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.146+0000 7f26d37fe700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26d40ccbd0 0x7f26d4150640 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c500 tx=0x7f26c800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.147+0000 7f26d0ff9700 1 -- 172.21.15.5:0/2710856521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80052d0 con 0x7f26d40ccbd0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.147+0000 7f26d0ff9700 1 -- 172.21.15.5:0/2710856521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c8005450 con 0x7f26d40ccbd0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.147+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d41581b0 con 0x7f26d40ccbd0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.147+0000 7f26d0ff9700 1 -- 172.21.15.5:0/2710856521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80038c0 con 0x7f26d40ccbd0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.147+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d4158770 con 0x7f26d40ccbd0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.148+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d4158990 con 0x7f26d40ccbd0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.148+0000 7f26d0ff9700 1 -- 172.21.15.5:0/2710856521 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f26c8003a20 con 0x7f26d40ccbd0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.149+0000 7f26d0ff9700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26c006ac90 0x7f26c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.149+0000 7f26d0ff9700 1 -- 172.21.15.5:0/2710856521 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26c8088600 con 0x7f26d40ccbd0 2024-05-08T23:26:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.149+0000 7f26d3fff700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26c006ac90 0x7f26c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.149+0000 7f26d3fff700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26c006ac90 0x7f26c006d150 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f26c4007fd0 tx=0x7f26c4007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.153+0000 7f26d0ff9700 1 -- 172.21.15.5:0/2710856521 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f26c8053870 con 0x7f26d40ccbd0 2024-05-08T23:26:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.373+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26d4068000 con 0x7f26d40ccbd0 2024-05-08T23:26:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.374+0000 7f26d0ff9700 1 -- 172.21.15.5:0/2710856521 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16004 (secure 0 0 0) 0x7f26c8056e90 con 0x7f26d40ccbd0 2024-05-08T23:26:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:37.374 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:05.360948+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:02.418282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:03.329387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.376+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26c006ac90 msgr2=0x7f26c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.376+0000 7f26da363700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26c006ac90 0x7f26c006d150 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f26c4007fd0 tx=0x7f26c4007dc0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.376+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26d40ccbd0 msgr2=0x7f26d4150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.376+0000 7f26da363700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26d40ccbd0 0x7f26d4150640 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c500 tx=0x7f26c800c810 comp rx=0 tx=0).stop 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.377+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 shutdown_connections 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.377+0000 7f26da363700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26c006ac90 0x7f26c006d150 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.377+0000 7f26da363700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26d4161030 0x7f26d4145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.377+0000 7f26da363700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d40b2480 0x7f26d414a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.377+0000 7f26da363700 1 --2- 172.21.15.5:0/2710856521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26d40ccbd0 0x7f26d4150640 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.377+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 >> 172.21.15.5:0/2710856521 conn(0x7f26d40b2f20 msgr2=0x7f26d406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.377+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 shutdown_connections 2024-05-08T23:26:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.377+0000 7f26da363700 1 -- 172.21.15.5:0/2710856521 wait complete. 2024-05-08T23:26:37.390 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:26:02.418282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:26:03.329387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:26:37.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:26:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.531+0000 7fcc0debf700 1 Processor -- start 2024-05-08T23:26:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.531+0000 7fcc0debf700 1 -- start start 2024-05-08T23:26:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc0debf700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcc0815b4e0 0x7fcc080b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc0debf700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc0815b8c0 0x7fcc080d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc0debf700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcc080d1740 0x7fcc080cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc0debf700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc080cefe0 con 0x7fcc080d1740 2024-05-08T23:26:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc0debf700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc080cf560 con 0x7fcc0815b4e0 2024-05-08T23:26:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc0debf700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc080d1bb0 con 0x7fcc0815b8c0 2024-05-08T23:26:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc07fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcc080d1740 0x7fcc080cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc07fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcc080d1740 0x7fcc080cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49022/0 (socket says 172.21.15.5:49022) 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc07fff700 1 -- 172.21.15.5:0/233112995 learned_addr learned my addr 172.21.15.5:0/233112995 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc077fe700 1 --2- 172.21.15.5:0/233112995 >> v2:172.21.15.49:3300/0 conn(0x7fcc0815b4e0 0x7fcc080b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.532+0000 7fcc06ffd700 1 --2- 172.21.15.5:0/233112995 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc0815b8c0 0x7fcc080d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc07fff700 1 -- 172.21.15.5:0/233112995 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc0815b8c0 msgr2=0x7fcc080d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc07fff700 1 --2- 172.21.15.5:0/233112995 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc0815b8c0 0x7fcc080d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc07fff700 1 -- 172.21.15.5:0/233112995 >> v2:172.21.15.49:3300/0 conn(0x7fcc0815b4e0 msgr2=0x7fcc080b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc07fff700 1 --2- 172.21.15.5:0/233112995 >> v2:172.21.15.49:3300/0 conn(0x7fcc0815b4e0 0x7fcc080b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc07fff700 1 -- 172.21.15.5:0/233112995 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc080ccff0 con 0x7fcc080d1740 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc07fff700 1 --2- 172.21.15.5:0/233112995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc080d1740 0x7fcc080cc2c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fcc080b0f90 tx=0x7fcc0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f44e1dc497d80f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc067fc700 1 -- 172.21.15.5:0/233112995 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000e070 con 0x7fcc080d1740 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc067fc700 1 -- 172.21.15.5:0/233112995 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000b9c0 con 0x7fcc080d1740 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc067fc700 1 -- 172.21.15.5:0/233112995 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000bd70 con 0x7fcc080d1740 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc0debf700 1 -- 172.21.15.5:0/233112995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc080d1740 msgr2=0x7fcc080cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc0debf700 1 --2- 172.21.15.5:0/233112995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc080d1740 0x7fcc080cc2c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fcc080b0f90 tx=0x7fcc0000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc0debf700 1 -- 172.21.15.5:0/233112995 shutdown_connections 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc0debf700 1 --2- 172.21.15.5:0/233112995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc080d1740 0x7fcc080cc2c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc0debf700 1 --2- 172.21.15.5:0/233112995 >> v2:172.21.15.49:3300/0 conn(0x7fcc0815b4e0 0x7fcc080b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc0debf700 1 --2- 172.21.15.5:0/233112995 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc0815b8c0 0x7fcc080d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc0debf700 1 -- 172.21.15.5:0/233112995 >> 172.21.15.5:0/233112995 conn(0x7fcc0808afe0 msgr2=0x7fcc0808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.533+0000 7fcc0debf700 1 -- 172.21.15.5:0/233112995 shutdown_connections 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.534+0000 7fcc0debf700 1 -- 172.21.15.5:0/233112995 wait complete. 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.535+0000 7fcc0debf700 1 Processor -- start 2024-05-08T23:26:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.535+0000 7fcc0debf700 1 -- start start 2024-05-08T23:26:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc0debf700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc080d1740 0x7fcc080b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc0debf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc0815b4e0 0x7fcc081f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc0debf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0815b8c0 0x7fcc0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc0debf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc080b0610 con 0x7fcc0815b8c0 2024-05-08T23:26:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc0debf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081f38e0 con 0x7fcc0815b4e0 2024-05-08T23:26:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc0debf700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081f4390 con 0x7fcc080d1740 2024-05-08T23:26:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0815b8c0 0x7fcc0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0815b8c0 0x7fcc0806d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49034/0 (socket says 172.21.15.5:49034) 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 -- 172.21.15.5:0/1475730649 learned_addr learned my addr 172.21.15.5:0/1475730649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc077fe700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc080d1740 0x7fcc080b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc06ffd700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc0815b4e0 0x7fcc081f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 -- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc080d1740 msgr2=0x7fcc080b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc080d1740 0x7fcc080b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 -- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc0815b4e0 msgr2=0x7fcc081f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc0815b4e0 0x7fcc081f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 -- 172.21.15.5:0/1475730649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc00009b30 con 0x7fcc0815b8c0 2024-05-08T23:26:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.536+0000 7fcc07fff700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0815b8c0 0x7fcc0806d3c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fcc080cef90 tx=0x7fcc00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.537+0000 7fcbfffff700 1 -- 172.21.15.5:0/1475730649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000099b0 con 0x7fcc0815b8c0 2024-05-08T23:26:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.537+0000 7fcbfffff700 1 -- 172.21.15.5:0/1475730649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00004250 con 0x7fcc0815b8c0 2024-05-08T23:26:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.537+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0806bc30 con 0x7fcc0815b8c0 2024-05-08T23:26:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.537+0000 7fcbfffff700 1 -- 172.21.15.5:0/1475730649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00004600 con 0x7fcc0815b8c0 2024-05-08T23:26:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.537+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc081f3dd0 con 0x7fcc0815b8c0 2024-05-08T23:26:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.538+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0806bdc0 con 0x7fcc0815b8c0 2024-05-08T23:26:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.538+0000 7fcbfffff700 1 -- 172.21.15.5:0/1475730649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcc00004760 con 0x7fcc0815b8c0 2024-05-08T23:26:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.539+0000 7fcbfffff700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbe806ac90 0x7fcbe806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.539+0000 7fcbfffff700 1 -- 172.21.15.5:0/1475730649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc0008a770 con 0x7fcc0815b8c0 2024-05-08T23:26:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.539+0000 7fcc077fe700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbe806ac90 0x7fcbe806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.540+0000 7fcc077fe700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbe806ac90 0x7fcbe806d150 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8006b20 tx=0x7fcbf8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.543+0000 7fcbfffff700 1 -- 172.21.15.5:0/1475730649 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcc000559f0 con 0x7fcc0815b8c0 2024-05-08T23:26:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.759+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc08074bc0 con 0x7fcc0815b8c0 2024-05-08T23:26:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.759+0000 7fcbfffff700 1 -- 172.21.15.5:0/1475730649 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc000200a0 con 0x7fcc0815b8c0 2024-05-08T23:26:37.760 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:26:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbe806ac90 msgr2=0x7fcbe806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbe806ac90 0x7fcbe806d150 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8006b20 tx=0x7fcbf8008040 comp rx=0 tx=0).stop 2024-05-08T23:26:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0815b8c0 msgr2=0x7fcc0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0815b8c0 0x7fcc0806d3c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fcc080cef90 tx=0x7fcc00004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 shutdown_connections 2024-05-08T23:26:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcbe806ac90 0x7fcbe806d150 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0815b8c0 0x7fcc0806d3c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcc080d1740 0x7fcc080b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 --2- 172.21.15.5:0/1475730649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc0815b4e0 0x7fcc081f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 >> 172.21.15.5:0/1475730649 conn(0x7fcc0808afe0 msgr2=0x7fcc080b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 shutdown_connections 2024-05-08T23:26:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.762+0000 7fcc0debf700 1 -- 172.21.15.5:0/1475730649 wait complete. 2024-05-08T23:26:37.776 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:26:37.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.915+0000 7f6b6c71c700 1 Processor -- start 2024-05-08T23:26:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.915+0000 7f6b6c71c700 1 -- start start 2024-05-08T23:26:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6c71c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6b640cff30 0x7f6b640b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6c71c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b640d04b0 con 0x7f6b6409a160 2024-05-08T23:26:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6c71c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b640b0910 con 0x7f6b6415b710 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6c71c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b640b0bf0 con 0x7f6b640cff30 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b69cb7700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6b6409a160 0x7f6b640b16d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47174/0 (socket says 172.21.15.5:47174) 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b69cb7700 1 -- 172.21.15.5:0/3941937974 learned_addr learned my addr 172.21.15.5:0/3941937974 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6acb9700 1 --2- 172.21.15.5:0/3941937974 >> v2:172.21.15.49:3300/0 conn(0x7f6b640cff30 0x7f6b640b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b694b6700 1 -- 172.21.15.5:0/3941937974 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610858931 0 0) 0x7f6b640d04b0 con 0x7f6b6409a160 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b694b6700 1 -- 172.21.15.5:0/3941937974 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b500035b0 con 0x7f6b6409a160 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b694b6700 1 -- 172.21.15.5:0/3941937974 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219784995 0 0) 0x7f6b640b0910 con 0x7f6b6415b710 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b694b6700 1 -- 172.21.15.5:0/3941937974 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b640d04b0 con 0x7f6b6415b710 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6acb9700 1 -- 172.21.15.5:0/3941937974 >> v1:172.21.15.49:6789/0 conn(0x7f6b6415b710 legacy=0x7f6b64099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6acb9700 1 -- 172.21.15.5:0/3941937974 >> v1:172.21.15.5:6789/0 conn(0x7f6b6409a160 legacy=0x7f6b640b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6acb9700 1 -- 172.21.15.5:0/3941937974 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b640b0ed0 con 0x7f6b640cff30 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.917+0000 7f6b6acb9700 1 --2- 172.21.15.5:0/3941937974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 0x7f6b640b1de0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009d90 tx=0x7f6b5c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a050083d4b83c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b694b6700 1 -- 172.21.15.5:0/3941937974 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00e070 con 0x7f6b640cff30 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b694b6700 1 -- 172.21.15.5:0/3941937974 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c00bb20 con 0x7f6b640cff30 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b694b6700 1 -- 172.21.15.5:0/3941937974 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c012430 con 0x7f6b640cff30 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3941937974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 msgr2=0x7f6b640b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b6c71c700 1 --2- 172.21.15.5:0/3941937974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 0x7f6b640b1de0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009d90 tx=0x7f6b5c00b330 comp rx=0 tx=0).stop 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3941937974 shutdown_connections 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b6c71c700 1 --2- 172.21.15.5:0/3941937974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 0x7f6b640b1de0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3941937974 >> 172.21.15.5:0/3941937974 conn(0x7f6b640b2f60 msgr2=0x7f6b640b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3941937974 shutdown_connections 2024-05-08T23:26:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.918+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3941937974 wait complete. 2024-05-08T23:26:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.920+0000 7f6b6c71c700 1 Processor -- start 2024-05-08T23:26:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.920+0000 7f6b6c71c700 1 -- start start 2024-05-08T23:26:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.920+0000 7f6b6c71c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b6409a160 0x7f6b640ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.920+0000 7f6b6c71c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 0x7f6b64141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.920+0000 7f6b6c71c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b6415b710 0x7f6b641418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.920+0000 7f6b6c71c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640c4f40 con 0x7f6b6415b710 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.920+0000 7f6b6c71c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640caf30 con 0x7f6b640cff30 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.920+0000 7f6b6c71c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640c9fa0 con 0x7f6b6409a160 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b6415b710 0x7f6b641418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b6415b710 0x7f6b641418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49042/0 (socket says 172.21.15.5:49042) 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 -- 172.21.15.5:0/3993893590 learned_addr learned my addr 172.21.15.5:0/3993893590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b69cb7700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 0x7f6b64141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6a4b8700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b6409a160 0x7f6b640ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 -- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b6409a160 msgr2=0x7f6b640ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b6409a160 0x7f6b640ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 -- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 msgr2=0x7f6b64141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 0x7f6b64141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 -- 172.21.15.5:0/3993893590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c009a70 con 0x7f6b6415b710 2024-05-08T23:26:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6acb9700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b6415b710 0x7f6b641418b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b680 tx=0x7f6b5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b5affd700 1 -- 172.21.15.5:0/3993893590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c01c810 con 0x7f6b6415b710 2024-05-08T23:26:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b5affd700 1 -- 172.21.15.5:0/3993893590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c020070 con 0x7f6b6415b710 2024-05-08T23:26:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b640c5150 con 0x7f6b6415b710 2024-05-08T23:26:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b5affd700 1 -- 172.21.15.5:0/3993893590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c01cbb0 con 0x7f6b6415b710 2024-05-08T23:26:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.921+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b640ca1f0 con 0x7f6b6415b710 2024-05-08T23:26:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.922+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b64074bc0 con 0x7f6b6415b710 2024-05-08T23:26:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.923+0000 7f6b5affd700 1 -- 172.21.15.5:0/3993893590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6b5c01cd10 con 0x7f6b6415b710 2024-05-08T23:26:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.923+0000 7f6b5affd700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b4806ac90 0x7f6b4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.923+0000 7f6b5affd700 1 -- 172.21.15.5:0/3993893590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b5c022070 con 0x7f6b6415b710 2024-05-08T23:26:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.924+0000 7f6b6a4b8700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b4806ac90 0x7f6b4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.924+0000 7f6b6a4b8700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b4806ac90 0x7f6b4806d150 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6b60007fd0 tx=0x7f6b60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:37.928+0000 7f6b5affd700 1 -- 172.21.15.5:0/3993893590 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6b5c056560 con 0x7f6b6415b710 2024-05-08T23:26:38.134 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:26:38.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:26:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.144+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b640cebb0 con 0x7f6b6415b710 2024-05-08T23:26:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.145+0000 7f6b5affd700 1 -- 172.21.15.5:0/3993893590 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b5c059b80 con 0x7f6b6415b710 2024-05-08T23:26:38.145 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:26:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b4806ac90 msgr2=0x7f6b4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b4806ac90 0x7f6b4806d150 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6b60007fd0 tx=0x7f6b60009040 comp rx=0 tx=0).stop 2024-05-08T23:26:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b6415b710 msgr2=0x7f6b641418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b6415b710 0x7f6b641418b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b680 tx=0x7f6b5c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 shutdown_connections 2024-05-08T23:26:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b4806ac90 0x7f6b4806d150 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b6415b710 0x7f6b641418b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6b6409a160 0x7f6b640ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 --2- 172.21.15.5:0/3993893590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b640cff30 0x7f6b64141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 >> 172.21.15.5:0/3993893590 conn(0x7f6b640b2f60 msgr2=0x7f6b640b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 shutdown_connections 2024-05-08T23:26:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.147+0000 7f6b6c71c700 1 -- 172.21.15.5:0/3993893590 wait complete. 2024-05-08T23:26:38.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:26:38.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.385+0000 7fda73349700 1 Processor -- start 2024-05-08T23:26:38.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.386+0000 7fda73349700 1 -- start start 2024-05-08T23:26:38.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda73349700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c06f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda73349700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fda6c06e7e0 0x7fda6c06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda73349700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c0b3870 con 0x7fda6c15d880 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda73349700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6c06fe00 con 0x7fda6c06e7e0 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda73349700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c06f130 con 0x7fda6c0af990 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda718e6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fda6c06e7e0 0x7fda6c06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda718e6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fda6c06e7e0 0x7fda6c06ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49046/0 (socket says 172.21.15.5:49046) 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda718e6700 1 -- 172.21.15.5:0/250804434 learned_addr learned my addr 172.21.15.5:0/250804434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda708e4700 1 --2- 172.21.15.5:0/250804434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c06f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda718e6700 1 -- 172.21.15.5:0/250804434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 msgr2=0x7fda6c06f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda718e6700 1 --2- 172.21.15.5:0/250804434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c06f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda718e6700 1 -- 172.21.15.5:0/250804434 >> v1:172.21.15.49:6789/0 conn(0x7fda6c15d880 legacy=0x7fda6c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda718e6700 1 -- 172.21.15.5:0/250804434 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0b4ee0 con 0x7fda6c06e7e0 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda708e4700 1 --2- 172.21.15.5:0/250804434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c06f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda718e6700 1 --2- 172.21.15.5:0/250804434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c06e7e0 0x7fda6c06ebf0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fda64009e50 tx=0x7fda6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293183ee3ed39a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda63fff700 1 -- 172.21.15.5:0/250804434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400e070 con 0x7fda6c06e7e0 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda63fff700 1 -- 172.21.15.5:0/250804434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6400b9c0 con 0x7fda6c06e7e0 2024-05-08T23:26:38.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.387+0000 7fda63fff700 1 -- 172.21.15.5:0/250804434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400bd70 con 0x7fda6c06e7e0 2024-05-08T23:26:38.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.388+0000 7fda73349700 1 -- 172.21.15.5:0/250804434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c06e7e0 msgr2=0x7fda6c06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.388+0000 7fda73349700 1 --2- 172.21.15.5:0/250804434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c06e7e0 0x7fda6c06ebf0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fda64009e50 tx=0x7fda6400b330 comp rx=0 tx=0).stop 2024-05-08T23:26:38.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.388+0000 7fda73349700 1 -- 172.21.15.5:0/250804434 shutdown_connections 2024-05-08T23:26:38.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.388+0000 7fda73349700 1 --2- 172.21.15.5:0/250804434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c06e7e0 0x7fda6c06ebf0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.388+0000 7fda73349700 1 --2- 172.21.15.5:0/250804434 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c06f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.388+0000 7fda73349700 1 -- 172.21.15.5:0/250804434 >> 172.21.15.5:0/250804434 conn(0x7fda6c0b5810 msgr2=0x7fda6c0b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:38.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.388+0000 7fda73349700 1 -- 172.21.15.5:0/250804434 shutdown_connections 2024-05-08T23:26:38.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.388+0000 7fda73349700 1 -- 172.21.15.5:0/250804434 wait complete. 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.389+0000 7fda73349700 1 Processor -- start 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda73349700 1 -- start start 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda73349700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c06e7e0 0x7fda6c0c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda73349700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c1f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda73349700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15d880 0x7fda6c1f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda73349700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c14d0b0 con 0x7fda6c15d880 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda73349700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c14c4a0 con 0x7fda6c06e7e0 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda73349700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c14c780 con 0x7fda6c0af990 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda718e6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15d880 0x7fda6c1f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda718e6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15d880 0x7fda6c1f2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49060/0 (socket says 172.21.15.5:49060) 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.390+0000 7fda718e6700 1 -- 172.21.15.5:0/1298067383 learned_addr learned my addr 172.21.15.5:0/1298067383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda708e4700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c1f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda710e5700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c06e7e0 0x7fda6c0c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda718e6700 1 -- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 msgr2=0x7fda6c1f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda718e6700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c1f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda718e6700 1 -- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c06e7e0 msgr2=0x7fda6c0c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda718e6700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c06e7e0 0x7fda6c0c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda718e6700 1 -- 172.21.15.5:0/1298067383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64009b30 con 0x7fda6c15d880 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda718e6700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15d880 0x7fda6c1f2410 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0676a0 tx=0x7fda64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda61ffb700 1 -- 172.21.15.5:0/1298067383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64004960 con 0x7fda6c15d880 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda61ffb700 1 -- 172.21.15.5:0/1298067383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda640092e0 con 0x7fda6c15d880 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c14ca60 con 0x7fda6c15d880 2024-05-08T23:26:38.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda61ffb700 1 -- 172.21.15.5:0/1298067383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6401c660 con 0x7fda6c15d880 2024-05-08T23:26:38.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.391+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c0c9d90 con 0x7fda6c15d880 2024-05-08T23:26:38.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.392+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c14d260 con 0x7fda6c15d880 2024-05-08T23:26:38.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.393+0000 7fda61ffb700 1 -- 172.21.15.5:0/1298067383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fda6401c7c0 con 0x7fda6c15d880 2024-05-08T23:26:38.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.393+0000 7fda61ffb700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5006ac90 0x7fda5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.393+0000 7fda61ffb700 1 -- 172.21.15.5:0/1298067383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda6408b080 con 0x7fda6c15d880 2024-05-08T23:26:38.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.394+0000 7fda710e5700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5006ac90 0x7fda5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.394+0000 7fda710e5700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5006ac90 0x7fda5006d150 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fda68000f60 tx=0x7fda68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:38.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.398+0000 7fda61ffb700 1 -- 172.21.15.5:0/1298067383 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fda6c14d260 con 0x7fda6c15d880 2024-05-08T23:26:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.620+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda6c0b34a0 con 0x7fda6c15d880 2024-05-08T23:26:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.620+0000 7fda61ffb700 1 -- 172.21.15.5:0/1298067383 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda64056250 con 0x7fda6c15d880 2024-05-08T23:26:38.620 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:26:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.622+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5006ac90 msgr2=0x7fda5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.622+0000 7fda73349700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5006ac90 0x7fda5006d150 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fda68000f60 tx=0x7fda68009040 comp rx=0 tx=0).stop 2024-05-08T23:26:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.622+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15d880 msgr2=0x7fda6c1f2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.622+0000 7fda73349700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15d880 0x7fda6c1f2410 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0676a0 tx=0x7fda64004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.623+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 shutdown_connections 2024-05-08T23:26:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.623+0000 7fda73349700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda5006ac90 0x7fda5006d150 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.623+0000 7fda73349700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c15d880 0x7fda6c1f2410 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.623+0000 7fda73349700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda6c06e7e0 0x7fda6c0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.623+0000 7fda73349700 1 --2- 172.21.15.5:0/1298067383 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda6c0af990 0x7fda6c1f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.623+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 >> 172.21.15.5:0/1298067383 conn(0x7fda6c0b5810 msgr2=0x7fda6c15fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.623+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 shutdown_connections 2024-05-08T23:26:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.623+0000 7fda73349700 1 -- 172.21.15.5:0/1298067383 wait complete. 2024-05-08T23:26:38.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:26:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.774+0000 7f1228d14700 1 Processor -- start 2024-05-08T23:26:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.774+0000 7f1228d14700 1 -- start start 2024-05-08T23:26:38.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1228d14700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240cf840 0x7f12240af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1228d14700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12240d0eb0 0x7f1224161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1228d14700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12240cd980 con 0x7f122406a960 2024-05-08T23:26:38.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1228d14700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12240ccdb0 con 0x7f12240d0eb0 2024-05-08T23:26:38.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1228d14700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12240cd050 con 0x7f12240cf840 2024-05-08T23:26:38.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1222d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12240d0eb0 0x7f1224161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1222d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12240d0eb0 0x7f1224161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49066/0 (socket says 172.21.15.5:49066) 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1222d9d700 1 -- 172.21.15.5:0/3174423734 learned_addr learned my addr 172.21.15.5:0/3174423734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f122259c700 1 --2- 172.21.15.5:0/3174423734 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240cf840 0x7f12240af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1222d9d700 1 -- 172.21.15.5:0/3174423734 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240cf840 msgr2=0x7f12240af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1222d9d700 1 --2- 172.21.15.5:0/3174423734 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240cf840 0x7f12240af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1222d9d700 1 -- 172.21.15.5:0/3174423734 >> v1:172.21.15.49:6789/0 conn(0x7f122406a960 legacy=0x7f122415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.775+0000 7f1222d9d700 1 -- 172.21.15.5:0/3174423734 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240cd330 con 0x7f12240d0eb0 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f1222d9d700 1 --2- 172.21.15.5:0/3174423734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12240d0eb0 0x7f1224161620 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f121c009e50 tx=0x7f121c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e42f68d43877729 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f122159a700 1 -- 172.21.15.5:0/3174423734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00e070 con 0x7f12240d0eb0 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f122159a700 1 -- 172.21.15.5:0/3174423734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c00b910 con 0x7f12240d0eb0 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f122159a700 1 -- 172.21.15.5:0/3174423734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00bcc0 con 0x7f12240d0eb0 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f1228d14700 1 -- 172.21.15.5:0/3174423734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12240d0eb0 msgr2=0x7f1224161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f1228d14700 1 --2- 172.21.15.5:0/3174423734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12240d0eb0 0x7f1224161620 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f121c009e50 tx=0x7f121c00b330 comp rx=0 tx=0).stop 2024-05-08T23:26:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f1228d14700 1 -- 172.21.15.5:0/3174423734 shutdown_connections 2024-05-08T23:26:38.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f1228d14700 1 --2- 172.21.15.5:0/3174423734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12240d0eb0 0x7f1224161620 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f1228d14700 1 --2- 172.21.15.5:0/3174423734 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240cf840 0x7f12240af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.776+0000 7f1228d14700 1 -- 172.21.15.5:0/3174423734 >> 172.21.15.5:0/3174423734 conn(0x7f12240d0320 msgr2=0x7f12240d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:38.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.777+0000 7f1228d14700 1 -- 172.21.15.5:0/3174423734 shutdown_connections 2024-05-08T23:26:38.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.777+0000 7f1228d14700 1 -- 172.21.15.5:0/3174423734 wait complete. 2024-05-08T23:26:38.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.778+0000 7f1228d14700 1 Processor -- start 2024-05-08T23:26:38.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.778+0000 7f1228d14700 1 -- start start 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f1228d14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122406a960 0x7f12240b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f1228d14700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12240cf840 0x7f1224157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f1228d14700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240d0eb0 0x7f12240c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f1228d14700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12240b2a90 con 0x7f122406a960 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f1228d14700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241f6fb0 con 0x7f12240cf840 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f1228d14700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12240b3680 con 0x7f12240d0eb0 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f122259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122406a960 0x7f12240b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f122259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122406a960 0x7f12240b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49072/0 (socket says 172.21.15.5:49072) 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f122259c700 1 -- 172.21.15.5:0/2087168 learned_addr learned my addr 172.21.15.5:0/2087168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f1221d9b700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12240cf840 0x7f1224157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f1222d9d700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240d0eb0 0x7f12240c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f122259c700 1 -- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240d0eb0 msgr2=0x7f12240c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.779+0000 7f122259c700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240d0eb0 0x7f12240c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f122259c700 1 -- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12240cf840 msgr2=0x7f1224157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f122259c700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12240cf840 0x7f1224157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f122259c700 1 -- 172.21.15.5:0/2087168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c009b30 con 0x7f122406a960 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f122259c700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122406a960 0x7f12240b2550 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f121400e9e0 tx=0x7f121400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f1212ffd700 1 -- 172.21.15.5:0/2087168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214009710 con 0x7f122406a960 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f1212ffd700 1 -- 172.21.15.5:0/2087168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121400ee80 con 0x7f122406a960 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12241f74e0 con 0x7f122406a960 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f1212ffd700 1 -- 172.21.15.5:0/2087168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121400ca50 con 0x7f122406a960 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.780+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12240b3030 con 0x7f122406a960 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.781+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1224068000 con 0x7f122406a960 2024-05-08T23:26:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.782+0000 7f1212ffd700 1 -- 172.21.15.5:0/2087168 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f121400cbb0 con 0x7f122406a960 2024-05-08T23:26:38.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.782+0000 7f1212ffd700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f120406ac90 0x7f120406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:38.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.782+0000 7f1212ffd700 1 -- 172.21.15.5:0/2087168 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1214089250 con 0x7f122406a960 2024-05-08T23:26:38.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.783+0000 7f1221d9b700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f120406ac90 0x7f120406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:38.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.783+0000 7f1221d9b700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f120406ac90 0x7f120406d150 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f12240aea20 tx=0x7f1218009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:38.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:38.787+0000 7f1212ffd700 1 -- 172.21.15.5:0/2087168 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f12140544c0 con 0x7f122406a960 2024-05-08T23:26:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.018+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f12240b62a0 con 0x7f122406a960 2024-05-08T23:26:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.019+0000 7f1212ffd700 1 -- 172.21.15.5:0/2087168 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1214057ae0 con 0x7f122406a960 2024-05-08T23:26:39.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f120406ac90 msgr2=0x7f120406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f120406ac90 0x7f120406d150 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f12240aea20 tx=0x7f1218009040 comp rx=0 tx=0).stop 2024-05-08T23:26:39.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122406a960 msgr2=0x7f12240b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122406a960 0x7f12240b2550 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f121400e9e0 tx=0x7f121400ecf0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 shutdown_connections 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f120406ac90 0x7f120406d150 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12240d0eb0 0x7f12240c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122406a960 0x7f12240b2550 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 --2- 172.21.15.5:0/2087168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12240cf840 0x7f1224157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 >> 172.21.15.5:0/2087168 conn(0x7f12240d0320 msgr2=0x7f12240737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 shutdown_connections 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.021+0000 7f1228d14700 1 -- 172.21.15.5:0/2087168 wait complete. 2024-05-08T23:26:39.022 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:26:39.034 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:26:39.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:39.034 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T23:26:39.080 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:26:39.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:26:39.080 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:26:39.100 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:26:39.100 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:39.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:26:39.122 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:26:39.122 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:39.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:26:39.161 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:26:39.161 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:39.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:26:39.184 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:39.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.346+0000 7f02c50ec700 1 Processor -- start 2024-05-08T23:26:39.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.347+0000 7f02c50ec700 1 -- start start 2024-05-08T23:26:39.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.347+0000 7f02c50ec700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02c015da10 0x7f02c00d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.347+0000 7f02c50ec700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c006c3d0 0x7f02c006c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.347+0000 7f02c50ec700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c00cf990 con 0x7f02c006bd80 2024-05-08T23:26:39.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.347+0000 7f02c50ec700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c009ab40 con 0x7f02c015da10 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.347+0000 7f02c50ec700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c006cda0 con 0x7f02c006c3d0 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bf59e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c006c3d0 0x7f02c006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bed9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02c015da10 0x7f02c00d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bed9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02c015da10 0x7f02c00d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36418/0 (socket says 172.21.15.5:36418) 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bed9d700 1 -- 172.21.15.5:0/2010282985 learned_addr learned my addr 172.21.15.5:0/2010282985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bf59e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c006c3d0 0x7f02c006c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:47588/0 (socket says 172.21.15.5:47588) 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bed9d700 1 -- 172.21.15.5:0/2010282985 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c006c3d0 msgr2=0x7f02c006c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bed9d700 1 --2- 172.21.15.5:0/2010282985 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c006c3d0 0x7f02c006c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bed9d700 1 -- 172.21.15.5:0/2010282985 >> v1:172.21.15.49:6789/0 conn(0x7f02c006bd80 legacy=0x7f02c009a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bed9d700 1 -- 172.21.15.5:0/2010282985 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c00d04b0 con 0x7f02c015da10 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bdd9b700 1 -- 172.21.15.5:0/2010282985 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543226858 0 0) 0x7f02c00cf990 con 0x7f02c006bd80 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.348+0000 7f02bed9d700 1 --2- 172.21.15.5:0/2010282985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c015da10 0x7f02c00d0940 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009920 tx=0x7f02b0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3531e826ea39ca23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02bdd9b700 1 -- 172.21.15.5:0/2010282985 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011830 con 0x7f02c015da10 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02bdd9b700 1 -- 172.21.15.5:0/2010282985 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0011990 con 0x7f02c015da10 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02bdd9b700 1 -- 172.21.15.5:0/2010282985 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011d20 con 0x7f02c015da10 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02c50ec700 1 -- 172.21.15.5:0/2010282985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c015da10 msgr2=0x7f02c00d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02c50ec700 1 --2- 172.21.15.5:0/2010282985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c015da10 0x7f02c00d0940 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009920 tx=0x7f02b0009c30 comp rx=0 tx=0).stop 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02c50ec700 1 -- 172.21.15.5:0/2010282985 shutdown_connections 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02c50ec700 1 --2- 172.21.15.5:0/2010282985 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c006c3d0 0x7f02c006c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02c50ec700 1 --2- 172.21.15.5:0/2010282985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c015da10 0x7f02c00d0940 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02c50ec700 1 -- 172.21.15.5:0/2010282985 >> 172.21.15.5:0/2010282985 conn(0x7f02c00b3010 msgr2=0x7f02c00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02c50ec700 1 -- 172.21.15.5:0/2010282985 shutdown_connections 2024-05-08T23:26:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.349+0000 7f02c50ec700 1 -- 172.21.15.5:0/2010282985 wait complete. 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.351+0000 7f02c50ec700 1 Processor -- start 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.351+0000 7f02c50ec700 1 -- start start 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02c50ec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c006bd80 0x7f02c01f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02c50ec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c006c3d0 0x7f02c015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02c50ec700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c015da10 0x7f02c00cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02c50ec700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00b6370 con 0x7f02c006c3d0 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02c50ec700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c01f3f30 con 0x7f02c006bd80 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02c50ec700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00b5760 con 0x7f02c015da10 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c006c3d0 0x7f02c015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c006c3d0 0x7f02c015b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49076/0 (socket says 172.21.15.5:49076) 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 -- 172.21.15.5:0/585102554 learned_addr learned my addr 172.21.15.5:0/585102554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02bed9d700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c006bd80 0x7f02c01f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02bf59e700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c015da10 0x7f02c00cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 -- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c015da10 msgr2=0x7f02c00cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c015da10 0x7f02c00cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 -- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c006bd80 msgr2=0x7f02c01f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c006bd80 0x7f02c01f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 -- 172.21.15.5:0/585102554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0009550 con 0x7f02c006c3d0 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.352+0000 7f02be59c700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c006c3d0 0x7f02c015b830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c450 tx=0x7f02b400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.353+0000 7f02a77fe700 1 -- 172.21.15.5:0/585102554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40054e0 con 0x7f02c006c3d0 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.353+0000 7f02a77fe700 1 -- 172.21.15.5:0/585102554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b400ce60 con 0x7f02c006c3d0 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.353+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c00b5a40 con 0x7f02c006c3d0 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.353+0000 7f02a77fe700 1 -- 172.21.15.5:0/585102554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40038a0 con 0x7f02c006c3d0 2024-05-08T23:26:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.353+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c00b6000 con 0x7f02c006c3d0 2024-05-08T23:26:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.354+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c00b64e0 con 0x7f02c006c3d0 2024-05-08T23:26:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.354+0000 7f02a77fe700 1 -- 172.21.15.5:0/585102554 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f02b4003a00 con 0x7f02c006c3d0 2024-05-08T23:26:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.355+0000 7f02a77fe700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02ac06ac90 0x7f02ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.355+0000 7f02a77fe700 1 -- 172.21.15.5:0/585102554 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02b40886b0 con 0x7f02c006c3d0 2024-05-08T23:26:39.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.355+0000 7f02bed9d700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02ac06ac90 0x7f02ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.356+0000 7f02bed9d700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02ac06ac90 0x7f02ac06d150 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f02b0000c00 tx=0x7f02b0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:39.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.360+0000 7f02a77fe700 1 -- 172.21.15.5:0/585102554 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f02b40538a0 con 0x7f02c006c3d0 2024-05-08T23:26:39.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.626+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02c0068000 con 0x7f02c006c3d0 2024-05-08T23:26:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.627+0000 7f02a77fe700 1 -- 172.21.15.5:0/585102554 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4761 (secure 0 0 0) 0x7f02b4003cb0 con 0x7f02c006c3d0 2024-05-08T23:26:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:39.627 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:39.452410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7179,"mds_1":6790,"mds_2":6787},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6787":{"gid":6787,"name":"d","rank":2,"incarnation":70,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6790":{"gid":6790,"name":"b","rank":1,"incarnation":68,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02ac06ac90 msgr2=0x7f02ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02ac06ac90 0x7f02ac06d150 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f02b0000c00 tx=0x7f02b0008030 comp rx=0 tx=0).stop 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c006c3d0 msgr2=0x7f02c015b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c006c3d0 0x7f02c015b830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c450 tx=0x7f02b400c760 comp rx=0 tx=0).stop 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 shutdown_connections 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f02ac06ac90 0x7f02ac06d150 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f02c015da10 0x7f02c00cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c006bd80 0x7f02c01f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 --2- 172.21.15.5:0/585102554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c006c3d0 0x7f02c015b830 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.629+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 >> 172.21.15.5:0/585102554 conn(0x7f02c00b3010 msgr2=0x7f02c0099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.630+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 shutdown_connections 2024-05-08T23:26:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.630+0000 7f02c50ec700 1 -- 172.21.15.5:0/585102554 wait complete. 2024-05-08T23:26:39.631 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 74 2024-05-08T23:26:39.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.780+0000 7fa280e4e700 1 Processor -- start 2024-05-08T23:26:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.780+0000 7fa280e4e700 1 -- start start 2024-05-08T23:26:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa280e4e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa280e4e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa27c0cecf0 0x7fa27c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa280e4e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c15c280 con 0x7fa27c06f130 2024-05-08T23:26:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa280e4e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c06e780 con 0x7fa27c0cecf0 2024-05-08T23:26:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa280e4e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c06ea60 con 0x7fa27c0cd4e0 2024-05-08T23:26:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa27ad9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa27c0cecf0 0x7fa27c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa27ad9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa27c0cecf0 0x7fa27c15c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49082/0 (socket says 172.21.15.5:49082) 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa27ad9d700 1 -- 172.21.15.5:0/4149305386 learned_addr learned my addr 172.21.15.5:0/4149305386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa27a59c700 1 --2- 172.21.15.5:0/4149305386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa27ad9d700 1 -- 172.21.15.5:0/4149305386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 msgr2=0x7fa27c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.781+0000 7fa27ad9d700 1 --2- 172.21.15.5:0/4149305386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa27ad9d700 1 -- 172.21.15.5:0/4149305386 >> v1:172.21.15.49:6789/0 conn(0x7fa27c06f130 legacy=0x7fa27c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa27ad9d700 1 -- 172.21.15.5:0/4149305386 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c06ed40 con 0x7fa27c0cecf0 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa27a59c700 1 --2- 172.21.15.5:0/4149305386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa27ad9d700 1 --2- 172.21.15.5:0/4149305386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c0cecf0 0x7fa27c15c710 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa274009e50 tx=0x7fa27400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879182e6260ddf16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa27959a700 1 -- 172.21.15.5:0/4149305386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400e070 con 0x7fa27c0cecf0 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa27959a700 1 -- 172.21.15.5:0/4149305386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27400b9c0 con 0x7fa27c0cecf0 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa27959a700 1 -- 172.21.15.5:0/4149305386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400bd70 con 0x7fa27c0cecf0 2024-05-08T23:26:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa280e4e700 1 -- 172.21.15.5:0/4149305386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c0cecf0 msgr2=0x7fa27c15c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.782+0000 7fa280e4e700 1 --2- 172.21.15.5:0/4149305386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c0cecf0 0x7fa27c15c710 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa274009e50 tx=0x7fa27400b330 comp rx=0 tx=0).stop 2024-05-08T23:26:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.783+0000 7fa280e4e700 1 -- 172.21.15.5:0/4149305386 shutdown_connections 2024-05-08T23:26:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.783+0000 7fa280e4e700 1 --2- 172.21.15.5:0/4149305386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c0cecf0 0x7fa27c15c710 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.783+0000 7fa280e4e700 1 --2- 172.21.15.5:0/4149305386 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.783+0000 7fa280e4e700 1 -- 172.21.15.5:0/4149305386 >> 172.21.15.5:0/4149305386 conn(0x7fa27c0d0320 msgr2=0x7fa27c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.783+0000 7fa280e4e700 1 -- 172.21.15.5:0/4149305386 shutdown_connections 2024-05-08T23:26:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.783+0000 7fa280e4e700 1 -- 172.21.15.5:0/4149305386 wait complete. 2024-05-08T23:26:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.784+0000 7fa280e4e700 1 Processor -- start 2024-05-08T23:26:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.784+0000 7fa280e4e700 1 -- start start 2024-05-08T23:26:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa280e4e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c06f130 0x7fa27c159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa280e4e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c0b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa280e4e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa27c0cecf0 0x7fa27c0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa280e4e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b5ce0 con 0x7fa27c06f130 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa280e4e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b1870 con 0x7fa27c0cecf0 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa280e4e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b1b50 con 0x7fa27c0cd4e0 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa27a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c06f130 0x7fa27c159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa27a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c06f130 0x7fa27c159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49088/0 (socket says 172.21.15.5:49088) 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa27a59c700 1 -- 172.21.15.5:0/2114050681 learned_addr learned my addr 172.21.15.5:0/2114050681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa27ad9d700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa27c0cecf0 0x7fa27c0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.785+0000 7fa279d9b700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c0b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa27a59c700 1 -- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 msgr2=0x7fa27c0b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa27a59c700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c0b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa27a59c700 1 -- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa27c0cecf0 msgr2=0x7fa27c0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa27a59c700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa27c0cecf0 0x7fa27c0b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa27a59c700 1 -- 172.21.15.5:0/2114050681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274009b30 con 0x7fa27c06f130 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa279d9b700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa27a59c700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c06f130 0x7fa27c159400 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00e810 tx=0x7fa26c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa272ffd700 1 -- 172.21.15.5:0/2114050681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c004230 con 0x7fa27c06f130 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa272ffd700 1 -- 172.21.15.5:0/2114050681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c0043b0 con 0x7fa27c06f130 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c0b1e30 con 0x7fa27c06f130 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa272ffd700 1 -- 172.21.15.5:0/2114050681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0038e0 con 0x7fa27c06f130 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.786+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c159940 con 0x7fa27c06f130 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.787+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c0b21a0 con 0x7fa27c06f130 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.788+0000 7fa272ffd700 1 -- 172.21.15.5:0/2114050681 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa26c003a40 con 0x7fa27c06f130 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.788+0000 7fa272ffd700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa25c06ac90 0x7fa25c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.788+0000 7fa272ffd700 1 -- 172.21.15.5:0/2114050681 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa26c088cc0 con 0x7fa27c06f130 2024-05-08T23:26:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.788+0000 7fa279d9b700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa25c06ac90 0x7fa25c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.789+0000 7fa279d9b700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa25c06ac90 0x7fa25c06d150 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0ca560 tx=0x7fa264009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:39.793+0000 7fa272ffd700 1 -- 172.21.15.5:0/2114050681 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa26c053f30 con 0x7fa27c06f130 2024-05-08T23:26:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.011+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa27c068000 con 0x7fa27c06f130 2024-05-08T23:26:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.012+0000 7fa272ffd700 1 -- 172.21.15.5:0/2114050681 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16004 (secure 0 0 0) 0x7fa26c0094c0 con 0x7fa27c06f130 2024-05-08T23:26:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:40.014 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:05.360948+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:02.418282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:03.329387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa25c06ac90 msgr2=0x7fa25c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa25c06ac90 0x7fa25c06d150 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0ca560 tx=0x7fa264009040 comp rx=0 tx=0).stop 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c06f130 msgr2=0x7fa27c159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c06f130 0x7fa27c159400 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00e810 tx=0x7fa26c00eb20 comp rx=0 tx=0).stop 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 shutdown_connections 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa25c06ac90 0x7fa25c06d150 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa27c0cecf0 0x7fa27c0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa27c06f130 0x7fa27c159400 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 --2- 172.21.15.5:0/2114050681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa27c0cd4e0 0x7fa27c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 >> 172.21.15.5:0/2114050681 conn(0x7fa27c0d0320 msgr2=0x7fa27c0ad1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 shutdown_connections 2024-05-08T23:26:40.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.015+0000 7fa280e4e700 1 -- 172.21.15.5:0/2114050681 wait complete. 2024-05-08T23:26:40.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:26:02.418282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:26:03.329387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:26:40.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:26:40.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.170+0000 7f645d1c3700 1 Processor -- start 2024-05-08T23:26:40.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.170+0000 7f645d1c3700 1 -- start start 2024-05-08T23:26:40.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f645d1c3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6458145040 0x7f64580b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f645d1c3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f645815fbb0 0x7f64580d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f645d1c3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64580af6e0 con 0x7f64580d0890 2024-05-08T23:26:40.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f645d1c3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64580d0d20 con 0x7f645815fbb0 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f645d1c3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64580b6460 con 0x7f6458145040 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f645759e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f64580d0890 0x7f645806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47178/0 (socket says 172.21.15.5:47178) 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f645759e700 1 -- 172.21.15.5:0/2762325705 learned_addr learned my addr 172.21.15.5:0/2762325705 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f645659c700 1 --2- 172.21.15.5:0/2762325705 >> v2:172.21.15.49:3300/0 conn(0x7f645815fbb0 0x7f64580d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6456d9d700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6458145040 0x7f64580b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053064703 0 0) 0x7f64580af6e0 con 0x7f64580d0890 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64400035b0 con 0x7f64580d0890 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 914751604 0 0) 0x7f64400035b0 con 0x7f64580d0890 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6458145040 msgr2=0x7f64580b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6458145040 0x7f64580b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 >> v2:172.21.15.49:3300/0 conn(0x7f645815fbb0 msgr2=0x7f64580d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 --2- 172.21.15.5:0/2762325705 >> v2:172.21.15.49:3300/0 conn(0x7f645815fbb0 0x7f64580d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580aeb50 con 0x7f64580d0890 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.171+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f64500019f0 con 0x7f64580d0890 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 >> v1:172.21.15.5:6789/0 conn(0x7f64580d0890 legacy=0x7f645806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6440003a30 0x7f6440003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6440004850 0x7f6440004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6458145040 0x7f6440005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64580d0d20 con 0x7f6440003a30 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64580b6460 con 0x7f6440004850 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644000ba10 con 0x7f6458145040 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6456d9d700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6440003a30 0x7f6440003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f645659c700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6440004850 0x7f6440004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f645759e700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6458145040 0x7f6440005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6456d9d700 1 -- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6458145040 msgr2=0x7f6440005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:40.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6456d9d700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6458145040 0x7f6440005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6456d9d700 1 -- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6440004850 msgr2=0x7f6440004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6456d9d700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6440004850 0x7f6440004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6456d9d700 1 -- 172.21.15.5:0/2762325705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64400035b0 con 0x7f6440003a30 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f645659c700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6440004850 0x7f6440004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6456d9d700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6440003a30 0x7f6440003e50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6448009b60 tx=0x7f6440005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e5d0dea6518602 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644800e6c0 con 0x7f6440003a30 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644800e820 con 0x7f6440003a30 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.172+0000 7f6455d9b700 1 -- 172.21.15.5:0/2762325705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644800eb90 con 0x7f6440003a30 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.173+0000 7f645d1c3700 1 -- 172.21.15.5:0/2762325705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6440003a30 msgr2=0x7f6440003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.173+0000 7f645d1c3700 1 --2- 172.21.15.5:0/2762325705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6440003a30 0x7f6440003e50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6448009b60 tx=0x7f6440005880 comp rx=0 tx=0).stop 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.173+0000 7f645759e700 1 -- 172.21.15.5:0/2762325705 reap_dead start 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.173+0000 7f645d1c3700 1 -- 172.21.15.5:0/2762325705 shutdown_connections 2024-05-08T23:26:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.173+0000 7f645d1c3700 1 -- 172.21.15.5:0/2762325705 >> 172.21.15.5:0/2762325705 conn(0x7f6458099a50 msgr2=0x7f64580b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.173+0000 7f645d1c3700 1 -- 172.21.15.5:0/2762325705 shutdown_connections 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.173+0000 7f645d1c3700 1 -- 172.21.15.5:0/2762325705 wait complete. 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.175+0000 7f645d1c3700 1 Processor -- start 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.175+0000 7f645d1c3700 1 -- start start 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.175+0000 7f645d1c3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f645813eec0 0x7f645815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.175+0000 7f645d1c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645815af90 0x7f64580ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.175+0000 7f645d1c3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64580caea0 0x7f645815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.175+0000 7f645d1c3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458140dc0 con 0x7f64580caea0 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.175+0000 7f645d1c3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64580cb350 con 0x7f645815af90 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.175+0000 7f645d1c3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645815e490 con 0x7f645813eec0 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64580caea0 0x7f645815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64580caea0 0x7f645815bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49114/0 (socket says 172.21.15.5:49114) 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 -- 172.21.15.5:0/144592966 learned_addr learned my addr 172.21.15.5:0/144592966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645659c700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645815af90 0x7f64580ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f6456d9d700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f645813eec0 0x7f645815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 -- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f645813eec0 msgr2=0x7f645815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f645813eec0 0x7f645815df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 -- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645815af90 msgr2=0x7f64580ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645815af90 0x7f64580ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 -- 172.21.15.5:0/144592966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6458140140 con 0x7f64580caea0 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f6456d9d700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f645813eec0 0x7f645815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645759e700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64580caea0 0x7f645815bae0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f6458067590 tx=0x7f6450007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f643f7fe700 1 -- 172.21.15.5:0/144592966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450017070 con 0x7f64580caea0 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f643f7fe700 1 -- 172.21.15.5:0/144592966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645000ec90 con 0x7f64580caea0 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6458140420 con 0x7f64580caea0 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f643f7fe700 1 -- 172.21.15.5:0/144592966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645000ee20 con 0x7f64580caea0 2024-05-08T23:26:40.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.176+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64581409e0 con 0x7f64580caea0 2024-05-08T23:26:40.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.177+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6458140f00 con 0x7f64580caea0 2024-05-08T23:26:40.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.178+0000 7f643f7fe700 1 -- 172.21.15.5:0/144592966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6450013420 con 0x7f64580caea0 2024-05-08T23:26:40.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.178+0000 7f643f7fe700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f644406ac90 0x7f644406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.179+0000 7f643f7fe700 1 -- 172.21.15.5:0/144592966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6450088f10 con 0x7f64580caea0 2024-05-08T23:26:40.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.179+0000 7f6456d9d700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f644406ac90 0x7f644406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.179+0000 7f6456d9d700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f644406ac90 0x7f644406d150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f64480090b0 tx=0x7f644801e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.183+0000 7f643f7fe700 1 -- 172.21.15.5:0/144592966 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6458140f00 con 0x7f64580caea0 2024-05-08T23:26:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.398+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6458140f00 con 0x7f64580caea0 2024-05-08T23:26:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.398+0000 7f643f7fe700 1 -- 172.21.15.5:0/144592966 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6458140f00 con 0x7f64580caea0 2024-05-08T23:26:40.398 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:26:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.400+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f644406ac90 msgr2=0x7f644406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.400+0000 7f645d1c3700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f644406ac90 0x7f644406d150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f64480090b0 tx=0x7f644801e040 comp rx=0 tx=0).stop 2024-05-08T23:26:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.400+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64580caea0 msgr2=0x7f645815bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.400+0000 7f645d1c3700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64580caea0 0x7f645815bae0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f6458067590 tx=0x7f6450007c90 comp rx=0 tx=0).stop 2024-05-08T23:26:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.401+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 shutdown_connections 2024-05-08T23:26:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.401+0000 7f645d1c3700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f644406ac90 0x7f644406d150 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.401+0000 7f645d1c3700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64580caea0 0x7f645815bae0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.401+0000 7f645d1c3700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f645813eec0 0x7f645815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.401+0000 7f645d1c3700 1 --2- 172.21.15.5:0/144592966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f645815af90 0x7f64580ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.401+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 >> 172.21.15.5:0/144592966 conn(0x7f6458099a50 msgr2=0x7f64580b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:40.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.401+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 shutdown_connections 2024-05-08T23:26:40.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.401+0000 7f645d1c3700 1 -- 172.21.15.5:0/144592966 wait complete. 2024-05-08T23:26:40.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:26:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.555+0000 7f4a2b11e700 1 Processor -- start 2024-05-08T23:26:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.555+0000 7f4a2b11e700 1 -- start start 2024-05-08T23:26:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a2b11e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 0x7f4a2406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a2b11e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4a2415e310 0x7f4a2415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a2b11e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240cef60 con 0x7f4a2415dcc0 2024-05-08T23:26:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a2b11e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240b2de0 con 0x7f4a2415e310 2024-05-08T23:26:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a2b11e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2415ece0 con 0x7f4a2406ef30 2024-05-08T23:26:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a296bb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4a2415e310 0x7f4a2415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a296bb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4a2415e310 0x7f4a2415e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49122/0 (socket says 172.21.15.5:49122) 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a296bb700 1 -- 172.21.15.5:0/301740451 learned_addr learned my addr 172.21.15.5:0/301740451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.556+0000 7f4a23fff700 1 --2- 172.21.15.5:0/301740451 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 0x7f4a2406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.557+0000 7f4a296bb700 1 -- 172.21.15.5:0/301740451 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 msgr2=0x7f4a2406fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.557+0000 7f4a296bb700 1 --2- 172.21.15.5:0/301740451 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 0x7f4a2406fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.557+0000 7f4a296bb700 1 -- 172.21.15.5:0/301740451 >> v1:172.21.15.49:6789/0 conn(0x7f4a2415dcc0 legacy=0x7f4a240b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.557+0000 7f4a296bb700 1 -- 172.21.15.5:0/301740451 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2415eee0 con 0x7f4a2415e310 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.557+0000 7f4a296bb700 1 --2- 172.21.15.5:0/301740451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415e310 0x7f4a2415e7a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00d150 tx=0x7f4a1c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59430be49c79015b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.557+0000 7f4a237fe700 1 -- 172.21.15.5:0/301740451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c010070 con 0x7f4a2415e310 2024-05-08T23:26:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.557+0000 7f4a237fe700 1 -- 172.21.15.5:0/301740451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c00dd10 con 0x7f4a2415e310 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.557+0000 7f4a237fe700 1 -- 172.21.15.5:0/301740451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00b310 con 0x7f4a2415e310 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.558+0000 7f4a2b11e700 1 -- 172.21.15.5:0/301740451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415e310 msgr2=0x7f4a2415e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.558+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/301740451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415e310 0x7f4a2415e7a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00d150 tx=0x7f4a1c00d460 comp rx=0 tx=0).stop 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.558+0000 7f4a2b11e700 1 -- 172.21.15.5:0/301740451 shutdown_connections 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.558+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/301740451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415e310 0x7f4a2415e7a0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.558+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/301740451 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 0x7f4a2406fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.558+0000 7f4a2b11e700 1 -- 172.21.15.5:0/301740451 >> 172.21.15.5:0/301740451 conn(0x7f4a24099a50 msgr2=0x7f4a240b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.558+0000 7f4a2b11e700 1 -- 172.21.15.5:0/301740451 shutdown_connections 2024-05-08T23:26:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.558+0000 7f4a2b11e700 1 -- 172.21.15.5:0/301740451 wait complete. 2024-05-08T23:26:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.559+0000 7f4a2b11e700 1 Processor -- start 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.559+0000 7f4a2b11e700 1 -- start start 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a2b11e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 0x7f4a240b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a2b11e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415dcc0 0x7f4a240c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a2b11e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a2415e310 0x7f4a240b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a2b11e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a241f4610 con 0x7f4a2415dcc0 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a2b11e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a241562d0 con 0x7f4a2415e310 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a2b11e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240c3310 con 0x7f4a2406ef30 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a23fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415dcc0 0x7f4a240c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a23fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415dcc0 0x7f4a240c2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49136/0 (socket says 172.21.15.5:49136) 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a23fff700 1 -- 172.21.15.5:0/399258000 learned_addr learned my addr 172.21.15.5:0/399258000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a28eba700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 0x7f4a240b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.560+0000 7f4a296bb700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a2415e310 0x7f4a240b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a23fff700 1 -- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 msgr2=0x7f4a240b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a23fff700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 0x7f4a240b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a23fff700 1 -- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a2415e310 msgr2=0x7f4a240b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a23fff700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a2415e310 0x7f4a240b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a23fff700 1 -- 172.21.15.5:0/399258000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c00bd40 con 0x7f4a2415dcc0 2024-05-08T23:26:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a296bb700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a2415e310 0x7f4a240b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a23fff700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415dcc0 0x7f4a240c2dd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e870 tx=0x7f4a1000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a217fa700 1 -- 172.21.15.5:0/399258000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10009590 con 0x7f4a2415dcc0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a217fa700 1 -- 172.21.15.5:0/399258000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10013680 con 0x7f4a2415dcc0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a241f3a00 con 0x7f4a2415dcc0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a217fa700 1 -- 172.21.15.5:0/399258000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a100139f0 con 0x7f4a2415dcc0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.561+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a241f3fc0 con 0x7f4a2415dcc0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.562+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2415d820 con 0x7f4a2415dcc0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.563+0000 7f4a217fa700 1 -- 172.21.15.5:0/399258000 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a10013b50 con 0x7f4a2415dcc0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.563+0000 7f4a217fa700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a0806ac90 0x7f4a0806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.563+0000 7f4a217fa700 1 -- 172.21.15.5:0/399258000 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a10089220 con 0x7f4a2415dcc0 2024-05-08T23:26:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.564+0000 7f4a28eba700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a0806ac90 0x7f4a0806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.564+0000 7f4a28eba700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a0806ac90 0x7f4a0806d150 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4a240cf920 tx=0x7f4a18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.568+0000 7f4a217fa700 1 -- 172.21.15.5:0/399258000 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4a10054400 con 0x7f4a2415dcc0 2024-05-08T23:26:40.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.785+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a240af1b0 con 0x7f4a2415dcc0 2024-05-08T23:26:40.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.785+0000 7f4a217fa700 1 -- 172.21.15.5:0/399258000 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a10057a20 con 0x7f4a2415dcc0 2024-05-08T23:26:40.786 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:26:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a0806ac90 msgr2=0x7f4a0806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a0806ac90 0x7f4a0806d150 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4a240cf920 tx=0x7f4a18009040 comp rx=0 tx=0).stop 2024-05-08T23:26:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415dcc0 msgr2=0x7f4a240c2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415dcc0 0x7f4a240c2dd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e870 tx=0x7f4a1000eb80 comp rx=0 tx=0).stop 2024-05-08T23:26:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 shutdown_connections 2024-05-08T23:26:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a0806ac90 0x7f4a0806d150 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a2415e310 0x7f4a240b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a2406ef30 0x7f4a240b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 --2- 172.21.15.5:0/399258000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a2415dcc0 0x7f4a240c2dd0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 >> 172.21.15.5:0/399258000 conn(0x7f4a24099a50 msgr2=0x7f4a240afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 shutdown_connections 2024-05-08T23:26:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.788+0000 7f4a2b11e700 1 -- 172.21.15.5:0/399258000 wait complete. 2024-05-08T23:26:40.802 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:26:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.940+0000 7f330598c700 1 Processor -- start 2024-05-08T23:26:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.940+0000 7f330598c700 1 -- start start 2024-05-08T23:26:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.941+0000 7f330598c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f330006c080 0x7f330015bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.941+0000 7f330598c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b48c0 0x7f330015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.941+0000 7f330598c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330015f3c0 con 0x7f33000b44e0 2024-05-08T23:26:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.941+0000 7f330598c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33000a84b0 con 0x7f330006c080 2024-05-08T23:26:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.941+0000 7f330598c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3300160160 con 0x7f33000b48c0 2024-05-08T23:26:40.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32fe7fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f330006c080 0x7f330015bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32ff7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b48c0 0x7f330015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32feffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f33000b44e0 0x7f330006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:42518/0 (socket says 172.21.15.5:42518) 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32ff7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b48c0 0x7f330015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:47670/0 (socket says 172.21.15.5:47670) 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32fe7fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f330006c080 0x7f330015bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36482/0 (socket says 172.21.15.5:36482) 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32feffd700 1 -- 172.21.15.5:0/1855205679 learned_addr learned my addr 172.21.15.5:0/1855205679 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32fe7fc700 1 -- 172.21.15.5:0/1855205679 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b48c0 msgr2=0x7f330015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32fe7fc700 1 --2- 172.21.15.5:0/1855205679 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b48c0 0x7f330015c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32fe7fc700 1 -- 172.21.15.5:0/1855205679 >> v1:172.21.15.49:6789/0 conn(0x7f33000b44e0 legacy=0x7f330006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32fe7fc700 1 -- 172.21.15.5:0/1855205679 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330015f680 con 0x7f330006c080 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32fdffb700 1 -- 172.21.15.5:0/1855205679 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700025161 0 0) 0x7f330015f3c0 con 0x7f33000b44e0 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32ff7fe700 1 --2- 172.21.15.5:0/1855205679 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b48c0 0x7f330015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.942+0000 7f32fe7fc700 1 --2- 172.21.15.5:0/1855205679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f330006c080 0x7f330015bf60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009840 tx=0x7f32f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78a9e7edd3258f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f32fdffb700 1 -- 172.21.15.5:0/1855205679 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011830 con 0x7f330006c080 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f32fdffb700 1 -- 172.21.15.5:0/1855205679 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f0011990 con 0x7f330006c080 2024-05-08T23:26:40.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f32fdffb700 1 -- 172.21.15.5:0/1855205679 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011d20 con 0x7f330006c080 2024-05-08T23:26:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f330598c700 1 -- 172.21.15.5:0/1855205679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f330006c080 msgr2=0x7f330015bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f330598c700 1 --2- 172.21.15.5:0/1855205679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f330006c080 0x7f330015bf60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009840 tx=0x7f32f0009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f330598c700 1 -- 172.21.15.5:0/1855205679 shutdown_connections 2024-05-08T23:26:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f330598c700 1 --2- 172.21.15.5:0/1855205679 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b48c0 0x7f330015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f330598c700 1 --2- 172.21.15.5:0/1855205679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f330006c080 0x7f330015bf60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f330598c700 1 -- 172.21.15.5:0/1855205679 >> 172.21.15.5:0/1855205679 conn(0x7f33000713c0 msgr2=0x7f33000717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f330598c700 1 -- 172.21.15.5:0/1855205679 shutdown_connections 2024-05-08T23:26:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.943+0000 7f330598c700 1 -- 172.21.15.5:0/1855205679 wait complete. 2024-05-08T23:26:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.945+0000 7f330598c700 1 Processor -- start 2024-05-08T23:26:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.945+0000 7f330598c700 1 -- start start 2024-05-08T23:26:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f330598c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330006c080 0x7f330014ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f330598c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b44e0 0x7f330014b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f330598c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33000b48c0 0x7f330014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f330598c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330014e720 con 0x7f330006c080 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f330598c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3300152ec0 con 0x7f33000b48c0 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f330598c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330014de30 con 0x7f33000b44e0 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330006c080 0x7f330014ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330006c080 0x7f330014ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49140/0 (socket says 172.21.15.5:49140) 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 -- 172.21.15.5:0/2138853346 learned_addr learned my addr 172.21.15.5:0/2138853346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32fe7fc700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b44e0 0x7f330014b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32ff7fe700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33000b48c0 0x7f330014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 -- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b44e0 msgr2=0x7f330014b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b44e0 0x7f330014b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 -- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33000b48c0 msgr2=0x7f330014f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33000b48c0 0x7f330014f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 -- 172.21.15.5:0/2138853346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f8009a70 con 0x7f330006c080 2024-05-08T23:26:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32ff7fe700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33000b48c0 0x7f330014f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32feffd700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330006c080 0x7f330014ebb0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f32e80011e0 tx=0x7f32e800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.946+0000 7f32f7fff700 1 -- 172.21.15.5:0/2138853346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8004c80 con 0x7f330006c080 2024-05-08T23:26:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.947+0000 7f32f7fff700 1 -- 172.21.15.5:0/2138853346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8004e00 con 0x7f330006c080 2024-05-08T23:26:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.947+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f00094a0 con 0x7f330006c080 2024-05-08T23:26:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.947+0000 7f32f7fff700 1 -- 172.21.15.5:0/2138853346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8007310 con 0x7f330006c080 2024-05-08T23:26:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.947+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330014e3b0 con 0x7f330006c080 2024-05-08T23:26:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.948+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330014ea00 con 0x7f330006c080 2024-05-08T23:26:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.948+0000 7f32f7fff700 1 -- 172.21.15.5:0/2138853346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f32e8007470 con 0x7f330006c080 2024-05-08T23:26:40.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.949+0000 7f32f7fff700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f32e006ac90 0x7f32e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:40.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.949+0000 7f32f7fff700 1 -- 172.21.15.5:0/2138853346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32e8089120 con 0x7f330006c080 2024-05-08T23:26:40.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.949+0000 7f32fe7fc700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f32e006ac90 0x7f32e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:40.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.949+0000 7f32fe7fc700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f32e006ac90 0x7f32e006d150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f32f00090b0 tx=0x7f32f001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:40.953+0000 7f32f7fff700 1 -- 172.21.15.5:0/2138853346 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f32e8054300 con 0x7f330006c080 2024-05-08T23:26:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.169+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3300074bc0 con 0x7f330006c080 2024-05-08T23:26:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.170+0000 7f32f7fff700 1 -- 172.21.15.5:0/2138853346 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32e8057920 con 0x7f330006c080 2024-05-08T23:26:41.170 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:26:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f32e006ac90 msgr2=0x7f32e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f32e006ac90 0x7f32e006d150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f32f00090b0 tx=0x7f32f001c040 comp rx=0 tx=0).stop 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330006c080 msgr2=0x7f330014ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330006c080 0x7f330014ebb0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f32e80011e0 tx=0x7f32e800cae0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 shutdown_connections 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f32e006ac90 0x7f32e006d150 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33000b48c0 0x7f330014f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330006c080 0x7f330014ebb0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 --2- 172.21.15.5:0/2138853346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f33000b44e0 0x7f330014b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 >> 172.21.15.5:0/2138853346 conn(0x7f33000713c0 msgr2=0x7f330015cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 shutdown_connections 2024-05-08T23:26:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.172+0000 7f330598c700 1 -- 172.21.15.5:0/2138853346 wait complete. 2024-05-08T23:26:41.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:41.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.322+0000 7f6d9d305700 1 Processor -- start 2024-05-08T23:26:41.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.322+0000 7f6d9d305700 1 -- start start 2024-05-08T23:26:41.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.323+0000 7f6d9d305700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d98161030 0x7f6d98099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.323+0000 7f6d9d305700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 0x7f6d98099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.323+0000 7f6d9d305700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d980d0840 con 0x7f6d9809a4d0 2024-05-08T23:26:41.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.323+0000 7f6d9d305700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9806ccc0 con 0x7f6d98161030 2024-05-08T23:26:41.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.323+0000 7f6d9d305700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9806c600 con 0x7f6d980b2480 2024-05-08T23:26:41.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.323+0000 7f6d96ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d98161030 0x7f6d98099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d96ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d98161030 0x7f6d98099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49152/0 (socket says 172.21.15.5:49152) 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d96ffd700 1 -- 172.21.15.5:0/4028498678 learned_addr learned my addr 172.21.15.5:0/4028498678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d967fc700 1 --2- 172.21.15.5:0/4028498678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 0x7f6d98099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d96ffd700 1 -- 172.21.15.5:0/4028498678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 msgr2=0x7f6d98099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d96ffd700 1 --2- 172.21.15.5:0/4028498678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 0x7f6d98099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d96ffd700 1 -- 172.21.15.5:0/4028498678 >> v1:172.21.15.49:6789/0 conn(0x7f6d9809a4d0 legacy=0x7f6d980cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d96ffd700 1 -- 172.21.15.5:0/4028498678 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9806c8e0 con 0x7f6d98161030 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d95ffb700 1 -- 172.21.15.5:0/4028498678 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104372296 0 0) 0x7f6d980d0840 con 0x7f6d9809a4d0 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d96ffd700 1 --2- 172.21.15.5:0/4028498678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d98161030 0x7f6d98099a50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6d880098f0 tx=0x7f6d88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cee8ed7100f0721 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d95ffb700 1 -- 172.21.15.5:0/4028498678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88011830 con 0x7f6d98161030 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d95ffb700 1 -- 172.21.15.5:0/4028498678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d88011990 con 0x7f6d98161030 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.324+0000 7f6d95ffb700 1 -- 172.21.15.5:0/4028498678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88011d20 con 0x7f6d98161030 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.325+0000 7f6d9d305700 1 -- 172.21.15.5:0/4028498678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d98161030 msgr2=0x7f6d98099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.325+0000 7f6d9d305700 1 --2- 172.21.15.5:0/4028498678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d98161030 0x7f6d98099a50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6d880098f0 tx=0x7f6d88009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.325+0000 7f6d9d305700 1 -- 172.21.15.5:0/4028498678 shutdown_connections 2024-05-08T23:26:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.325+0000 7f6d9d305700 1 --2- 172.21.15.5:0/4028498678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d98161030 0x7f6d98099a50 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.325+0000 7f6d9d305700 1 --2- 172.21.15.5:0/4028498678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 0x7f6d98099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.325+0000 7f6d9d305700 1 -- 172.21.15.5:0/4028498678 >> 172.21.15.5:0/4028498678 conn(0x7f6d980b2f20 msgr2=0x7f6d980b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.325+0000 7f6d9d305700 1 -- 172.21.15.5:0/4028498678 shutdown_connections 2024-05-08T23:26:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.325+0000 7f6d9d305700 1 -- 172.21.15.5:0/4028498678 wait complete. 2024-05-08T23:26:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.326+0000 7f6d9d305700 1 Processor -- start 2024-05-08T23:26:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d9d305700 1 -- start start 2024-05-08T23:26:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d9d305700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9809a4d0 0x7f6d980c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d9d305700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 0x7f6d980c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d9d305700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d98161030 0x7f6d980c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d9d305700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981f3660 con 0x7f6d9809a4d0 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d9d305700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d98154c20 con 0x7f6d98161030 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d9d305700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980c63a0 con 0x7f6d980b2480 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9809a4d0 0x7f6d980c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9809a4d0 0x7f6d980c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49156/0 (socket says 172.21.15.5:49156) 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.327+0000 7f6d96ffd700 1 -- 172.21.15.5:0/2302875680 learned_addr learned my addr 172.21.15.5:0/2302875680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d967fc700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 0x7f6d980c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d977fe700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d98161030 0x7f6d980c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d96ffd700 1 -- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 msgr2=0x7f6d980c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d96ffd700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 0x7f6d980c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d96ffd700 1 -- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d98161030 msgr2=0x7f6d980c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d96ffd700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d98161030 0x7f6d980c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d96ffd700 1 -- 172.21.15.5:0/2302875680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88009550 con 0x7f6d9809a4d0 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d96ffd700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9809a4d0 0x7f6d980c7440 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6d98072530 tx=0x7f6d88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d7f7fe700 1 -- 172.21.15.5:0/2302875680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88004300 con 0x7f6d9809a4d0 2024-05-08T23:26:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d7f7fe700 1 -- 172.21.15.5:0/2302875680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8801f070 con 0x7f6d9809a4d0 2024-05-08T23:26:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d980c6680 con 0x7f6d9809a4d0 2024-05-08T23:26:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d7f7fe700 1 -- 172.21.15.5:0/2302875680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d880036a0 con 0x7f6d9809a4d0 2024-05-08T23:26:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.328+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d980c6c40 con 0x7f6d9809a4d0 2024-05-08T23:26:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.329+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d98068000 con 0x7f6d9809a4d0 2024-05-08T23:26:41.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.330+0000 7f6d7f7fe700 1 -- 172.21.15.5:0/2302875680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d88021030 con 0x7f6d9809a4d0 2024-05-08T23:26:41.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.330+0000 7f6d7f7fe700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d8406ac90 0x7f6d8406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.330+0000 7f6d7f7fe700 1 -- 172.21.15.5:0/2302875680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d8808a670 con 0x7f6d9809a4d0 2024-05-08T23:26:41.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.330+0000 7f6d967fc700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d8406ac90 0x7f6d8406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.331+0000 7f6d967fc700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d8406ac90 0x7f6d8406d150 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6d98155180 tx=0x7f6d8c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.335+0000 7f6d7f7fe700 1 -- 172.21.15.5:0/2302875680 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6d88055860 con 0x7f6d9809a4d0 2024-05-08T23:26:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.604+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d980d0f50 con 0x7f6d9809a4d0 2024-05-08T23:26:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.604+0000 7f6d7f7fe700 1 -- 172.21.15.5:0/2302875680 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4761 (secure 0 0 0) 0x7f6d88058e80 con 0x7f6d9809a4d0 2024-05-08T23:26:41.605 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:41.605 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:39.452410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7179,"mds_1":6790,"mds_2":6787},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6787":{"gid":6787,"name":"d","rank":2,"incarnation":70,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6790":{"gid":6790,"name":"b","rank":1,"incarnation":68,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d8406ac90 msgr2=0x7f6d8406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d8406ac90 0x7f6d8406d150 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6d98155180 tx=0x7f6d8c009210 comp rx=0 tx=0).stop 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9809a4d0 msgr2=0x7f6d980c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9809a4d0 0x7f6d980c7440 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6d98072530 tx=0x7f6d88004750 comp rx=0 tx=0).stop 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 shutdown_connections 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d8406ac90 0x7f6d8406d150 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d98161030 0x7f6d980c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9809a4d0 0x7f6d980c7440 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 --2- 172.21.15.5:0/2302875680 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d980b2480 0x7f6d980c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 >> 172.21.15.5:0/2302875680 conn(0x7f6d980b2f20 msgr2=0x7f6d980d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 shutdown_connections 2024-05-08T23:26:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.607+0000 7f6d9d305700 1 -- 172.21.15.5:0/2302875680 wait complete. 2024-05-08T23:26:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 74 2024-05-08T23:26:41.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:41.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.757+0000 7fc10fb0d700 1 Processor -- start 2024-05-08T23:26:41.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.757+0000 7fc10fb0d700 1 -- start start 2024-05-08T23:26:41.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.758+0000 7fc10fb0d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc10806ef30 0x7fc10806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.758+0000 7fc10fb0d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1080cef60 con 0x7fc10815dcc0 2024-05-08T23:26:41.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.758+0000 7fc10fb0d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10815eee0 con 0x7fc10815e310 2024-05-08T23:26:41.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.758+0000 7fc10fb0d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1080b2de0 con 0x7fc10806ef30 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.758+0000 7fc10d8a9700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc10815dcc0 0x7fc1080b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47188/0 (socket says 172.21.15.5:47188) 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.758+0000 7fc10d8a9700 1 -- 172.21.15.5:0/3751785484 learned_addr learned my addr 172.21.15.5:0/3751785484 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.758+0000 7fc10d0a8700 1 --2- 172.21.15.5:0/3751785484 >> v2:172.21.15.49:3300/0 conn(0x7fc10806ef30 0x7fc10806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10c8a7700 1 -- 172.21.15.5:0/3751785484 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533661982 0 0) 0x7fc1080cef60 con 0x7fc10815dcc0 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10c8a7700 1 -- 172.21.15.5:0/3751785484 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f40035b0 con 0x7fc10815dcc0 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10d0a8700 1 -- 172.21.15.5:0/3751785484 >> v1:172.21.15.49:6789/0 conn(0x7fc10815e310 legacy=0x7fc1080adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10d0a8700 1 -- 172.21.15.5:0/3751785484 >> v1:172.21.15.5:6789/0 conn(0x7fc10815dcc0 legacy=0x7fc1080b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10d0a8700 1 -- 172.21.15.5:0/3751785484 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1080ce980 con 0x7fc10806ef30 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10d0a8700 1 --2- 172.21.15.5:0/3751785484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10806ef30 0x7fc10806fa10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009840 tx=0x7fc0f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6c9d53c1e83689 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10c8a7700 1 -- 172.21.15.5:0/3751785484 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8011830 con 0x7fc10806ef30 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10c8a7700 1 -- 172.21.15.5:0/3751785484 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f8011990 con 0x7fc10806ef30 2024-05-08T23:26:41.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.759+0000 7fc10c8a7700 1 -- 172.21.15.5:0/3751785484 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8011d20 con 0x7fc10806ef30 2024-05-08T23:26:41.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.760+0000 7fc10fb0d700 1 -- 172.21.15.5:0/3751785484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10806ef30 msgr2=0x7fc10806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.760+0000 7fc10fb0d700 1 --2- 172.21.15.5:0/3751785484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10806ef30 0x7fc10806fa10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009840 tx=0x7fc0f8009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.760+0000 7fc10fb0d700 1 -- 172.21.15.5:0/3751785484 shutdown_connections 2024-05-08T23:26:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.760+0000 7fc10fb0d700 1 --2- 172.21.15.5:0/3751785484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10806ef30 0x7fc10806fa10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.760+0000 7fc10fb0d700 1 -- 172.21.15.5:0/3751785484 >> 172.21.15.5:0/3751785484 conn(0x7fc108099a50 msgr2=0x7fc1080b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.760+0000 7fc10fb0d700 1 -- 172.21.15.5:0/3751785484 shutdown_connections 2024-05-08T23:26:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.760+0000 7fc10fb0d700 1 -- 172.21.15.5:0/3751785484 wait complete. 2024-05-08T23:26:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.761+0000 7fc10fb0d700 1 Processor -- start 2024-05-08T23:26:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.761+0000 7fc10fb0d700 1 -- start start 2024-05-08T23:26:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10fb0d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10806ef30 0x7fc1080c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10fb0d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc10815dcc0 0x7fc10814db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10fb0d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10815e310 0x7fc1080c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10fb0d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10814e700 con 0x7fc10806ef30 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10fb0d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1080c6dc0 con 0x7fc10815e310 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10fb0d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1080c70a0 con 0x7fc10815dcc0 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10d8a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10806ef30 0x7fc1080c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10d8a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10806ef30 0x7fc1080c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49172/0 (socket says 172.21.15.5:49172) 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10d8a9700 1 -- 172.21.15.5:0/1465119972 learned_addr learned my addr 172.21.15.5:0/1465119972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.762+0000 7fc10e0aa700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10815e310 0x7fc1080c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10d0a8700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc10815dcc0 0x7fc10814db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10d8a9700 1 -- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc10815dcc0 msgr2=0x7fc10814db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10d8a9700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc10815dcc0 0x7fc10814db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10d8a9700 1 -- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10815e310 msgr2=0x7fc1080c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10d8a9700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10815e310 0x7fc1080c3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10d8a9700 1 -- 172.21.15.5:0/1465119972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f80094a0 con 0x7fc10806ef30 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10d0a8700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc10815dcc0 0x7fc10814db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10d8a9700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10806ef30 0x7fc1080c7f50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc10400c700 tx=0x7fc10400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc0fe7fc700 1 -- 172.21.15.5:0/1465119972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104005d70 con 0x7fc10806ef30 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc0fe7fc700 1 -- 172.21.15.5:0/1465119972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1040070d0 con 0x7fc10806ef30 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc0fe7fc700 1 -- 172.21.15.5:0/1465119972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104007460 con 0x7fc10806ef30 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1080c7470 con 0x7fc10806ef30 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.763+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1080c7a30 con 0x7fc10806ef30 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.764+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1080cf900 con 0x7fc10806ef30 2024-05-08T23:26:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.765+0000 7fc0fe7fc700 1 -- 172.21.15.5:0/1465119972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc1040075c0 con 0x7fc10806ef30 2024-05-08T23:26:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.765+0000 7fc0fe7fc700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc0ec06ac90 0x7fc0ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.765+0000 7fc0fe7fc700 1 -- 172.21.15.5:0/1465119972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1040888e0 con 0x7fc10806ef30 2024-05-08T23:26:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.765+0000 7fc10d0a8700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc0ec06ac90 0x7fc0ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.766+0000 7fc10d0a8700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc0ec06ac90 0x7fc0ec06d150 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc1080c8520 tx=0x7fc0f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.770+0000 7fc0fe7fc700 1 -- 172.21.15.5:0/1465119972 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc104053ad0 con 0x7fc10806ef30 2024-05-08T23:26:41.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.989+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc108068000 con 0x7fc10806ef30 2024-05-08T23:26:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.991+0000 7fc0fe7fc700 1 -- 172.21.15.5:0/1465119972 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16004 (secure 0 0 0) 0x7fc1040570f0 con 0x7fc10806ef30 2024-05-08T23:26:41.991 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:41.991 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:05.360948+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:02.418282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:03.329387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc0ec06ac90 msgr2=0x7fc0ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc0ec06ac90 0x7fc0ec06d150 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc1080c8520 tx=0x7fc0f8008030 comp rx=0 tx=0).stop 2024-05-08T23:26:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10806ef30 msgr2=0x7fc1080c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10806ef30 0x7fc1080c7f50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc10400c700 tx=0x7fc10400cac0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 shutdown_connections 2024-05-08T23:26:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc0ec06ac90 0x7fc0ec06d150 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc10815e310 0x7fc1080c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10806ef30 0x7fc1080c7f50 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 --2- 172.21.15.5:0/1465119972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc10815dcc0 0x7fc10814db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 >> 172.21.15.5:0/1465119972 conn(0x7fc108099a50 msgr2=0x7fc1080b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 shutdown_connections 2024-05-08T23:26:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:41.993+0000 7fc10fb0d700 1 -- 172.21.15.5:0/1465119972 wait complete. 2024-05-08T23:26:42.007 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:26:42.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.144+0000 7f0bb7d24700 1 Processor -- start 2024-05-08T23:26:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.144+0000 7f0bb7d24700 1 -- start start 2024-05-08T23:26:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb7d24700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0bb00cd4e0 0x7f0bb006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb7d24700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0bb006f130 0x7f0bb00ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb7d24700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb00ceb20 0x7f0bb015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb7d24700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb00cefb0 con 0x7f0bb006f130 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb7d24700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb015c280 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb7d24700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb006e780 con 0x7f0bb00ceb20 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb52bf700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0bb006f130 0x7f0bb00ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb52bf700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0bb006f130 0x7f0bb00ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49176/0 (socket says 172.21.15.5:49176) 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb52bf700 1 -- 172.21.15.5:0/3937728072 learned_addr learned my addr 172.21.15.5:0/3937728072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb5ac0700 1 --2- 172.21.15.5:0/3937728072 >> v2:172.21.15.49:3300/0 conn(0x7f0bb00cd4e0 0x7f0bb006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb62c1700 1 --2- 172.21.15.5:0/3937728072 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb00ceb20 0x7f0bb015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb52bf700 1 -- 172.21.15.5:0/3937728072 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb00ceb20 msgr2=0x7f0bb015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb52bf700 1 --2- 172.21.15.5:0/3937728072 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb00ceb20 0x7f0bb015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb52bf700 1 -- 172.21.15.5:0/3937728072 >> v2:172.21.15.49:3300/0 conn(0x7f0bb00cd4e0 msgr2=0x7f0bb006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb52bf700 1 --2- 172.21.15.5:0/3937728072 >> v2:172.21.15.49:3300/0 conn(0x7f0bb00cd4e0 0x7f0bb006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.145+0000 7f0bb52bf700 1 -- 172.21.15.5:0/3937728072 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb006ea60 con 0x7f0bb006f130 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb52bf700 1 --2- 172.21.15.5:0/3937728072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb006f130 0x7f0bb00ce5e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0072ef0 tx=0x7f0ba000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc2e3da60f270a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb4abe700 1 -- 172.21.15.5:0/3937728072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000dd30 con 0x7f0bb006f130 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb4abe700 1 -- 172.21.15.5:0/3937728072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba000de90 con 0x7f0bb006f130 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb4abe700 1 -- 172.21.15.5:0/3937728072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0010690 con 0x7f0bb006f130 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb7d24700 1 -- 172.21.15.5:0/3937728072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb006f130 msgr2=0x7f0bb00ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb7d24700 1 --2- 172.21.15.5:0/3937728072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb006f130 0x7f0bb00ce5e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0072ef0 tx=0x7f0ba000be70 comp rx=0 tx=0).stop 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb7d24700 1 -- 172.21.15.5:0/3937728072 shutdown_connections 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb7d24700 1 --2- 172.21.15.5:0/3937728072 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb00ceb20 0x7f0bb015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb7d24700 1 --2- 172.21.15.5:0/3937728072 >> v2:172.21.15.49:3300/0 conn(0x7f0bb00cd4e0 0x7f0bb006f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb7d24700 1 --2- 172.21.15.5:0/3937728072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb006f130 0x7f0bb00ce5e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.146+0000 7f0bb7d24700 1 -- 172.21.15.5:0/3937728072 >> 172.21.15.5:0/3937728072 conn(0x7f0bb00d0320 msgr2=0x7f0bb00d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.147+0000 7f0bb7d24700 1 -- 172.21.15.5:0/3937728072 shutdown_connections 2024-05-08T23:26:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.147+0000 7f0bb7d24700 1 -- 172.21.15.5:0/3937728072 wait complete. 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.148+0000 7f0bb7d24700 1 Processor -- start 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.148+0000 7f0bb7d24700 1 -- start start 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb7d24700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb006f130 0x7f0bb01eead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb7d24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb00cd4e0 0x7f0bb00c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb7d24700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bb00ceb20 0x7f0bb015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb7d24700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00b3f10 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb7d24700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00c8340 con 0x7f0bb00ceb20 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb7d24700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00b3300 con 0x7f0bb006f130 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb00cd4e0 0x7f0bb00c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb00cd4e0 0x7f0bb00c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49178/0 (socket says 172.21.15.5:49178) 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 -- 172.21.15.5:0/533281295 learned_addr learned my addr 172.21.15.5:0/533281295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb5ac0700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb006f130 0x7f0bb01eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb62c1700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bb00ceb20 0x7f0bb015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 -- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb006f130 msgr2=0x7f0bb01eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb006f130 0x7f0bb01eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 -- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bb00ceb20 msgr2=0x7f0bb015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bb00ceb20 0x7f0bb015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 -- 172.21.15.5:0/533281295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba000b7f0 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb5ac0700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb006f130 0x7f0bb01eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb62c1700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bb00ceb20 0x7f0bb015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0bb52bf700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb00cd4e0 0x7f0bb00c7e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000beb0 tx=0x7f0ba0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0ba67fc700 1 -- 172.21.15.5:0/533281295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba001d070 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.149+0000 7f0ba67fc700 1 -- 172.21.15.5:0/533281295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba001e040 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.150+0000 7f0ba67fc700 1 -- 172.21.15.5:0/533281295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0010470 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.150+0000 7f0bb7d24700 1 -- 172.21.15.5:0/533281295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb00b35e0 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.150+0000 7f0bb7d24700 1 -- 172.21.15.5:0/533281295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb00b3ba0 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.151+0000 7f0b93fff700 1 -- 172.21.15.5:0/533281295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c0052f0 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.151+0000 7f0ba67fc700 1 -- 172.21.15.5:0/533281295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0ba0023460 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.152+0000 7f0ba67fc700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac90 0x7f0b9406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.152+0000 7f0ba67fc700 1 -- 172.21.15.5:0/533281295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ba008cd90 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.152+0000 7f0bb5ac0700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac90 0x7f0b9406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.152+0000 7f0bb5ac0700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac90 0x7f0b9406d150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00cfdf0 tx=0x7f0bac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:42.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.156+0000 7f0ba67fc700 1 -- 172.21.15.5:0/533281295 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ba0058000 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.421+0000 7f0b93fff700 1 -- 172.21.15.5:0/533281295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b8c0055a0 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.422+0000 7f0ba67fc700 1 -- 172.21.15.5:0/533281295 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4761 (secure 0 0 0) 0x7f0ba005b620 con 0x7f0bb00cd4e0 2024-05-08T23:26:42.423 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:42.423 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:39.452410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7179,"mds_1":6790,"mds_2":6787},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6787":{"gid":6787,"name":"d","rank":2,"incarnation":70,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6790":{"gid":6790,"name":"b","rank":1,"incarnation":68,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 -- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac90 msgr2=0x7f0b9406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac90 0x7f0b9406d150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00cfdf0 tx=0x7f0bac008040 comp rx=0 tx=0).stop 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 -- 172.21.15.5:0/533281295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb00cd4e0 msgr2=0x7f0bb00c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb00cd4e0 0x7f0bb00c7e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000beb0 tx=0x7f0ba0004750 comp rx=0 tx=0).stop 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 -- 172.21.15.5:0/533281295 shutdown_connections 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b9406ac90 0x7f0b9406d150 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bb00ceb20 0x7f0bb015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0bb006f130 0x7f0bb01eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 --2- 172.21.15.5:0/533281295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb00cd4e0 0x7f0bb00c7e00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.424+0000 7f0b93fff700 1 -- 172.21.15.5:0/533281295 >> 172.21.15.5:0/533281295 conn(0x7f0bb00d0320 msgr2=0x7f0bb01589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.425+0000 7f0b93fff700 1 -- 172.21.15.5:0/533281295 shutdown_connections 2024-05-08T23:26:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.425+0000 7f0b93fff700 1 -- 172.21.15.5:0/533281295 wait complete. 2024-05-08T23:26:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 74 2024-05-08T23:26:42.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.575+0000 7fb0df36c700 1 Processor -- start 2024-05-08T23:26:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.576+0000 7fb0df36c700 1 -- start start 2024-05-08T23:26:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.576+0000 7fb0df36c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb0d815ef60 0x7fb0d80b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.576+0000 7fb0df36c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d80cf000 0x7fb0d80cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0df36c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d80cfe30 con 0x7fb0d80cf9a0 2024-05-08T23:26:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0df36c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d80cd400 con 0x7fb0d815ef60 2024-05-08T23:26:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0df36c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d809ad40 con 0x7fb0d80cf000 2024-05-08T23:26:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dc907700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d80cf000 0x7fb0d80cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dd108700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb0d815ef60 0x7fb0d80b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dd909700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb0d80cf9a0 0x7fb0d8099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:42532/0 (socket says 172.21.15.5:42532) 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dd909700 1 -- 172.21.15.5:0/3987742711 learned_addr learned my addr 172.21.15.5:0/3987742711 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dc907700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d80cf000 0x7fb0d80cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:47742/0 (socket says 172.21.15.5:47742) 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dc907700 1 -- 172.21.15.5:0/3987742711 >> v2:172.21.15.49:3300/0 conn(0x7fb0d815ef60 msgr2=0x7fb0d80b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dc907700 1 --2- 172.21.15.5:0/3987742711 >> v2:172.21.15.49:3300/0 conn(0x7fb0d815ef60 0x7fb0d80b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dc907700 1 -- 172.21.15.5:0/3987742711 >> v1:172.21.15.49:6789/0 conn(0x7fb0d80cf9a0 legacy=0x7fb0d8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dc907700 1 -- 172.21.15.5:0/3987742711 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d80ccb60 con 0x7fb0d80cf000 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0cffff700 1 -- 172.21.15.5:0/3987742711 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529909855 0 0) 0x7fb0d80cfe30 con 0x7fb0d80cf9a0 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.577+0000 7fb0dd108700 1 --2- 172.21.15.5:0/3987742711 >> v2:172.21.15.49:3300/0 conn(0x7fb0d815ef60 0x7fb0d80b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0dc907700 1 --2- 172.21.15.5:0/3987742711 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d80cf000 0x7fb0d80cf460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009840 tx=0x7fb0c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a200557f01754 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0cffff700 1 -- 172.21.15.5:0/3987742711 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8011830 con 0x7fb0d80cf000 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0cffff700 1 -- 172.21.15.5:0/3987742711 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c8011990 con 0x7fb0d80cf000 2024-05-08T23:26:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0cffff700 1 -- 172.21.15.5:0/3987742711 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8011d20 con 0x7fb0d80cf000 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0df36c700 1 -- 172.21.15.5:0/3987742711 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d80cf000 msgr2=0x7fb0d80cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0df36c700 1 --2- 172.21.15.5:0/3987742711 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d80cf000 0x7fb0d80cf460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009840 tx=0x7fb0c8009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0df36c700 1 -- 172.21.15.5:0/3987742711 shutdown_connections 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0df36c700 1 --2- 172.21.15.5:0/3987742711 >> v2:172.21.15.49:3300/0 conn(0x7fb0d815ef60 0x7fb0d80b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0df36c700 1 --2- 172.21.15.5:0/3987742711 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d80cf000 0x7fb0d80cf460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.578+0000 7fb0df36c700 1 -- 172.21.15.5:0/3987742711 >> 172.21.15.5:0/3987742711 conn(0x7fb0d80b2f20 msgr2=0x7fb0d80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.579+0000 7fb0df36c700 1 -- 172.21.15.5:0/3987742711 shutdown_connections 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.579+0000 7fb0df36c700 1 -- 172.21.15.5:0/3987742711 wait complete. 2024-05-08T23:26:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.580+0000 7fb0df36c700 1 Processor -- start 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.580+0000 7fb0df36c700 1 -- start start 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0df36c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d80cf000 0x7fb0d80caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0df36c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d80cf9a0 0x7fb0d81f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0df36c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d815ef60 0x7fb0d80c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0df36c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d806bcd0 con 0x7fb0d80cf000 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0df36c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d806bfb0 con 0x7fb0d80cf9a0 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0df36c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d806c290 con 0x7fb0d815ef60 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d80cf000 0x7fb0d80caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d80cf000 0x7fb0d80caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49182/0 (socket says 172.21.15.5:49182) 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 -- 172.21.15.5:0/551425404 learned_addr learned my addr 172.21.15.5:0/551425404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dc907700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d80cf9a0 0x7fb0d81f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd909700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d815ef60 0x7fb0d80c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 -- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d815ef60 msgr2=0x7fb0d80c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d815ef60 0x7fb0d80c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 -- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d80cf9a0 msgr2=0x7fb0d81f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d80cf9a0 0x7fb0d81f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 -- 172.21.15.5:0/551425404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d40094a0 con 0x7fb0d80cf000 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.581+0000 7fb0dd108700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d80cf000 0x7fb0d80caf70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400ebe0 tx=0x7fb0d4009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.582+0000 7fb0cdffb700 1 -- 172.21.15.5:0/551425404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400c8b0 con 0x7fb0d80cf000 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.582+0000 7fb0cdffb700 1 -- 172.21.15.5:0/551425404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d400ca10 con 0x7fb0d80cf000 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.582+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c80094a0 con 0x7fb0d80cf000 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.582+0000 7fb0cdffb700 1 -- 172.21.15.5:0/551425404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400cda0 con 0x7fb0d80cf000 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.582+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d806d3f0 con 0x7fb0d80cf000 2024-05-08T23:26:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.583+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d8068000 con 0x7fb0d80cf000 2024-05-08T23:26:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.583+0000 7fb0cdffb700 1 -- 172.21.15.5:0/551425404 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb0d4013470 con 0x7fb0d80cf000 2024-05-08T23:26:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.584+0000 7fb0cdffb700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0bc06ac90 0x7fb0bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.584+0000 7fb0cdffb700 1 -- 172.21.15.5:0/551425404 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0d4089500 con 0x7fb0d80cf000 2024-05-08T23:26:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.584+0000 7fb0dc907700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0bc06ac90 0x7fb0bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.585+0000 7fb0dc907700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0bc06ac90 0x7fb0bc06d150 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009fd0 tx=0x7fb0c801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:42.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.589+0000 7fb0cdffb700 1 -- 172.21.15.5:0/551425404 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb0d40546e0 con 0x7fb0d80cf000 2024-05-08T23:26:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.857+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0d80cf460 con 0x7fb0d80cf000 2024-05-08T23:26:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.858+0000 7fb0cdffb700 1 -- 172.21.15.5:0/551425404 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4761 (secure 0 0 0) 0x7fb0d4057d00 con 0x7fb0d80cf000 2024-05-08T23:26:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:42.858 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:04.359632+0000","modified":"2024-05-08T23:26:39.452410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7179,"mds_1":6790,"mds_2":6787},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6787":{"gid":6787,"name":"d","rank":2,"incarnation":70,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6836/2469967007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2469967007},{"type":"v1","addr":"172.21.15.49:6836","nonce":2469967007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6790":{"gid":6790,"name":"b","rank":1,"incarnation":68,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/3300231911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3300231911},{"type":"v1","addr":"172.21.15.49:6837","nonce":3300231911}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7179":{"gid":7179,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/690633393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":690633393},{"type":"v1","addr":"172.21.15.5:6834","nonce":690633393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T23:26:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0bc06ac90 msgr2=0x7fb0bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0bc06ac90 0x7fb0bc06d150 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009fd0 tx=0x7fb0c801c040 comp rx=0 tx=0).stop 2024-05-08T23:26:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d80cf000 msgr2=0x7fb0d80caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d80cf000 0x7fb0d80caf70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400ebe0 tx=0x7fb0d4009e20 comp rx=0 tx=0).stop 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 shutdown_connections 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0bc06ac90 0x7fb0bc06d150 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0d815ef60 0x7fb0d80c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d80cf000 0x7fb0d80caf70 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 --2- 172.21.15.5:0/551425404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0d80cf9a0 0x7fb0d81f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 >> 172.21.15.5:0/551425404 conn(0x7fb0d80b2f20 msgr2=0x7fb0d80b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 shutdown_connections 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:42.860+0000 7fb0df36c700 1 -- 172.21.15.5:0/551425404 wait complete. 2024-05-08T23:26:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 74 2024-05-08T23:26:42.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.009+0000 7f499bf17700 1 Processor -- start 2024-05-08T23:26:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.010+0000 7f499bf17700 1 -- start start 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.010+0000 7f499bf17700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f499415db00 0x7f49940cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.010+0000 7f499bf17700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49940caa50 0x7f499406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.010+0000 7f499bf17700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499406ad20 con 0x7f49940caeb0 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.010+0000 7f499bf17700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499415dee0 con 0x7f499415db00 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.010+0000 7f499bf17700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49940ccad0 con 0x7f49940caa50 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4999cb3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f499415db00 0x7f49940cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f499a4b4700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f49940caeb0 0x7f499406e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47204/0 (socket says 172.21.15.5:47204) 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f499a4b4700 1 -- 172.21.15.5:0/2546038852 learned_addr learned my addr 172.21.15.5:0/2546038852 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f49994b2700 1 --2- 172.21.15.5:0/2546038852 >> v2:172.21.15.49:3300/0 conn(0x7f49940caa50 0x7f499406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4999cb3700 1 -- 172.21.15.5:0/2546038852 >> v2:172.21.15.49:3300/0 conn(0x7f49940caa50 msgr2=0x7f499406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4999cb3700 1 --2- 172.21.15.5:0/2546038852 >> v2:172.21.15.49:3300/0 conn(0x7f49940caa50 0x7f499406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4999cb3700 1 -- 172.21.15.5:0/2546038852 >> v1:172.21.15.5:6789/0 conn(0x7f49940caeb0 legacy=0x7f499406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4999cb3700 1 -- 172.21.15.5:0/2546038852 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499406b000 con 0x7f499415db00 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f49994b2700 1 --2- 172.21.15.5:0/2546038852 >> v2:172.21.15.49:3300/0 conn(0x7f49940caa50 0x7f499406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4999cb3700 1 --2- 172.21.15.5:0/2546038852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499415db00 0x7f49940cc590 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f49940cdef0 tx=0x7f4990009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c9c995ce7d453 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4998cb1700 1 -- 172.21.15.5:0/2546038852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990011830 con 0x7f499415db00 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4998cb1700 1 -- 172.21.15.5:0/2546038852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4990011990 con 0x7f499415db00 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.011+0000 7f4998cb1700 1 -- 172.21.15.5:0/2546038852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990011d20 con 0x7f499415db00 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.012+0000 7f499bf17700 1 -- 172.21.15.5:0/2546038852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499415db00 msgr2=0x7f49940cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.012+0000 7f499bf17700 1 --2- 172.21.15.5:0/2546038852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499415db00 0x7f49940cc590 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f49940cdef0 tx=0x7f4990009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.012+0000 7f499bf17700 1 -- 172.21.15.5:0/2546038852 shutdown_connections 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.012+0000 7f499bf17700 1 --2- 172.21.15.5:0/2546038852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499415db00 0x7f49940cc590 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.012+0000 7f499bf17700 1 --2- 172.21.15.5:0/2546038852 >> v2:172.21.15.49:3300/0 conn(0x7f49940caa50 0x7f499406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.012+0000 7f499bf17700 1 -- 172.21.15.5:0/2546038852 >> 172.21.15.5:0/2546038852 conn(0x7f4994099a50 msgr2=0x7f49940d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.012+0000 7f499bf17700 1 -- 172.21.15.5:0/2546038852 shutdown_connections 2024-05-08T23:26:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.012+0000 7f499bf17700 1 -- 172.21.15.5:0/2546038852 wait complete. 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.014+0000 7f499bf17700 1 Processor -- start 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.014+0000 7f499bf17700 1 -- start start 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f499bf17700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940caa50 0x7f49940b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f499bf17700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49940caeb0 0x7f49940c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f499bf17700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499415db00 0x7f49941f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f499bf17700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941f3e90 con 0x7f49940caa50 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f499bf17700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940c5ab0 con 0x7f499415db00 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f499bf17700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49941f3370 con 0x7f49940caeb0 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f4999cb3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940caa50 0x7f49940b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f4999cb3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940caa50 0x7f49940b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49200/0 (socket says 172.21.15.5:49200) 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f4999cb3700 1 -- 172.21.15.5:0/3859662188 learned_addr learned my addr 172.21.15.5:0/3859662188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f499a4b4700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499415db00 0x7f49941f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f49994b2700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49940caeb0 0x7f49940c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f4999cb3700 1 -- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49940caeb0 msgr2=0x7f49940c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f4999cb3700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49940caeb0 0x7f49940c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f4999cb3700 1 -- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499415db00 msgr2=0x7f49941f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f4999cb3700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499415db00 0x7f49941f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.015+0000 7f4999cb3700 1 -- 172.21.15.5:0/3859662188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900094a0 con 0x7f49940caa50 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.016+0000 7f4999cb3700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940caa50 0x7f49940b5e70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4990009b90 tx=0x7f4990004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.016+0000 7f498a7fc700 1 -- 172.21.15.5:0/3859662188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900036a0 con 0x7f49940caa50 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.016+0000 7f498a7fc700 1 -- 172.21.15.5:0/3859662188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499001e070 con 0x7f49940caa50 2024-05-08T23:26:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.016+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49941f35f0 con 0x7f49940caa50 2024-05-08T23:26:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.016+0000 7f498a7fc700 1 -- 172.21.15.5:0/3859662188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990003a40 con 0x7f49940caa50 2024-05-08T23:26:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.016+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49941f3bb0 con 0x7f49940caa50 2024-05-08T23:26:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.017+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4994068000 con 0x7f49940caa50 2024-05-08T23:26:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.017+0000 7f498a7fc700 1 -- 172.21.15.5:0/3859662188 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4990003ba0 con 0x7f49940caa50 2024-05-08T23:26:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.018+0000 7f498a7fc700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f497806ac90 0x7f497806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.018+0000 7f498a7fc700 1 -- 172.21.15.5:0/3859662188 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f499008a420 con 0x7f49940caa50 2024-05-08T23:26:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.018+0000 7f49994b2700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f497806ac90 0x7f497806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.019+0000 7f49994b2700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f497806ac90 0x7f497806d150 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f499415d170 tx=0x7f4984008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.023+0000 7f498a7fc700 1 -- 172.21.15.5:0/3859662188 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4990055560 con 0x7f49940caa50 2024-05-08T23:26:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.239+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49940add30 con 0x7f49940caa50 2024-05-08T23:26:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.240+0000 7f498a7fc700 1 -- 172.21.15.5:0/3859662188 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16004 (secure 0 0 0) 0x7f4990058b80 con 0x7f49940caa50 2024-05-08T23:26:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:43.241 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:05.360948+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:02.418282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:03.329387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.242+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f497806ac90 msgr2=0x7f497806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.242+0000 7f499bf17700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f497806ac90 0x7f497806d150 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f499415d170 tx=0x7f4984008040 comp rx=0 tx=0).stop 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.242+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940caa50 msgr2=0x7f49940b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.242+0000 7f499bf17700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940caa50 0x7f49940b5e70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4990009b90 tx=0x7f4990004750 comp rx=0 tx=0).stop 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.243+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 shutdown_connections 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.243+0000 7f499bf17700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f497806ac90 0x7f497806d150 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.243+0000 7f499bf17700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f499415db00 0x7f49941f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.243+0000 7f499bf17700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940caa50 0x7f49940b5e70 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.243+0000 7f499bf17700 1 --2- 172.21.15.5:0/3859662188 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49940caeb0 0x7f49940c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.243+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 >> 172.21.15.5:0/3859662188 conn(0x7f4994099a50 msgr2=0x7f49940aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.243+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 shutdown_connections 2024-05-08T23:26:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.243+0000 7f499bf17700 1 -- 172.21.15.5:0/3859662188 wait complete. 2024-05-08T23:26:43.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:26:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.395+0000 7fc2b4ef3700 1 Processor -- start 2024-05-08T23:26:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.395+0000 7fc2b4ef3700 1 -- start start 2024-05-08T23:26:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2b4ef3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2b015b900 0x7fc2b00cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2b4ef3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 0x7fc2b0099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2b4ef3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2b009a410 0x7fc2b00cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2b4ef3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b00b1410 con 0x7fc2b009a410 2024-05-08T23:26:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2b4ef3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b009a8a0 con 0x7fc2b015b900 2024-05-08T23:26:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2b4ef3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00b0e30 con 0x7fc2b0099a50 2024-05-08T23:26:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2aed9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2b009a410 0x7fc2b00cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2aed9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2b009a410 0x7fc2b00cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49206/0 (socket says 172.21.15.5:49206) 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2aed9d700 1 -- 172.21.15.5:0/2398269548 learned_addr learned my addr 172.21.15.5:0/2398269548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/2398269548 >> v2:172.21.15.49:3300/0 conn(0x7fc2b015b900 0x7fc2b00cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2add9b700 1 --2- 172.21.15.5:0/2398269548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 0x7fc2b0099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2aed9d700 1 -- 172.21.15.5:0/2398269548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 msgr2=0x7fc2b0099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2aed9d700 1 --2- 172.21.15.5:0/2398269548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 0x7fc2b0099ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2aed9d700 1 -- 172.21.15.5:0/2398269548 >> v2:172.21.15.49:3300/0 conn(0x7fc2b015b900 msgr2=0x7fc2b00cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2aed9d700 1 --2- 172.21.15.5:0/2398269548 >> v2:172.21.15.49:3300/0 conn(0x7fc2b015b900 0x7fc2b00cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.396+0000 7fc2aed9d700 1 -- 172.21.15.5:0/2398269548 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b00cffd0 con 0x7fc2b009a410 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2aed9d700 1 --2- 172.21.15.5:0/2398269548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00cbf40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0072130 tx=0x7fc2a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9147234d69819c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2ad59a700 1 -- 172.21.15.5:0/2398269548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a800e070 con 0x7fc2b009a410 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2ad59a700 1 -- 172.21.15.5:0/2398269548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a800b9c0 con 0x7fc2b009a410 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2ad59a700 1 -- 172.21.15.5:0/2398269548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a800bd70 con 0x7fc2b009a410 2024-05-08T23:26:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2398269548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 msgr2=0x7fc2b00cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2398269548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00cbf40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0072130 tx=0x7fc2a800b330 comp rx=0 tx=0).stop 2024-05-08T23:26:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2398269548 shutdown_connections 2024-05-08T23:26:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2398269548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00cbf40 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2398269548 >> v2:172.21.15.49:3300/0 conn(0x7fc2b015b900 0x7fc2b00cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2398269548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 0x7fc2b0099ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2398269548 >> 172.21.15.5:0/2398269548 conn(0x7fc2b00b2f20 msgr2=0x7fc2b00b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.397+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2398269548 shutdown_connections 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.398+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2398269548 wait complete. 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.399+0000 7fc2b4ef3700 1 Processor -- start 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.399+0000 7fc2b4ef3700 1 -- start start 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2b4ef3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 0x7fc2b00b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2b4ef3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2b4ef3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015b900 0x7fc2b00aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2b4ef3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b006ca80 con 0x7fc2b009a410 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2b4ef3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00b5460 con 0x7fc2b015b900 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2b4ef3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00b4160 con 0x7fc2b0099a50 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49220/0 (socket says 172.21.15.5:49220) 2024-05-08T23:26:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 -- 172.21.15.5:0/2037107704 learned_addr learned my addr 172.21.15.5:0/2037107704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 0x7fc2b00b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2aed9d700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015b900 0x7fc2b00aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 -- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 msgr2=0x7fc2b00b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 0x7fc2b00b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 -- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015b900 msgr2=0x7fc2b00aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015b900 0x7fc2b00aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 -- 172.21.15.5:0/2037107704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8009b30 con 0x7fc2b009a410 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.400+0000 7fc2add9b700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00ae950 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000d800 tx=0x7fc2a000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.401+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2037107704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00125f0 con 0x7fc2b009a410 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.401+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2037107704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0012750 con 0x7fc2b009a410 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.401+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b00b4440 con 0x7fc2b009a410 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.401+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2037107704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0012ae0 con 0x7fc2b009a410 2024-05-08T23:26:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.401+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b00b4a00 con 0x7fc2b009a410 2024-05-08T23:26:43.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.402+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b00b4c20 con 0x7fc2b009a410 2024-05-08T23:26:43.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.402+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2037107704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc2a0012c40 con 0x7fc2b009a410 2024-05-08T23:26:43.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.403+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.403+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2037107704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2a0089250 con 0x7fc2b009a410 2024-05-08T23:26:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.403+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.403+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc298006b20 tx=0x7fc298008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.407+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2037107704 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc2a0054570 con 0x7fc2b009a410 2024-05-08T23:26:43.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.678+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc2b0068000 con 0x7fc2b009a410 2024-05-08T23:26:43.678 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:26:43.679+0000 7f8a49d06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:26:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.685+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2037107704 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc2a0057b90 con 0x7fc2b009a410 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 msgr2=0x7fc29006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc298006b20 tx=0x7fc298008040 comp rx=0 tx=0).stop 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 msgr2=0x7fc2b00ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00ae950 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000d800 tx=0x7fc2a000db10 comp rx=0 tx=0).stop 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 shutdown_connections 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015b900 0x7fc2b00aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b0099a50 0x7fc2b00b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 --2- 172.21.15.5:0/2037107704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b009a410 0x7fc2b00ae950 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 >> 172.21.15.5:0/2037107704 conn(0x7fc2b00b2f20 msgr2=0x7fc2b00b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 shutdown_connections 2024-05-08T23:26:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.688+0000 7fc2b4ef3700 1 -- 172.21.15.5:0/2037107704 wait complete. 2024-05-08T23:26:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:26:43.692 INFO:tasks.ceph.mds.c.smithi005.stderr: -7171> 2024-05-08T23:26:19.889+0000 7f42240c8700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:26:43.692 INFO:tasks.ceph.mds.c.smithi005.stderr: -7169> 2024-05-08T23:26:19.889+0000 7f42240c8700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T23:26:43.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:26:43.718 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:26:43.718 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:26:43.727 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:26:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.843+0000 7fe0faa38700 1 Processor -- start 2024-05-08T23:26:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.843+0000 7fe0faa38700 1 -- start start 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0faa38700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0f40b0d80 0x7fe0f40b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0faa38700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe0f40b4d50 0x7fe0f406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0faa38700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f406aea0 0x7fe0f40d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0faa38700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f406b330 con 0x7fe0f40b0d80 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0faa38700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f4145040 con 0x7fe0f40b4d50 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0faa38700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f40b38c0 con 0x7fe0f406aea0 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f3fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0f40b0d80 0x7fe0f40b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f3fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0f40b0d80 0x7fe0f40b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49256/0 (socket says 172.21.15.5:49256) 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f3fff700 1 -- 172.21.15.5:0/815746179 learned_addr learned my addr 172.21.15.5:0/815746179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f37fe700 1 --2- 172.21.15.5:0/815746179 >> v2:172.21.15.49:3300/0 conn(0x7fe0f40b4d50 0x7fe0f406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f8fd5700 1 --2- 172.21.15.5:0/815746179 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f406aea0 0x7fe0f40d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f3fff700 1 -- 172.21.15.5:0/815746179 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f406aea0 msgr2=0x7fe0f40d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f3fff700 1 --2- 172.21.15.5:0/815746179 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f406aea0 0x7fe0f40d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f3fff700 1 -- 172.21.15.5:0/815746179 >> v2:172.21.15.49:3300/0 conn(0x7fe0f40b4d50 msgr2=0x7fe0f406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f3fff700 1 --2- 172.21.15.5:0/815746179 >> v2:172.21.15.49:3300/0 conn(0x7fe0f40b4d50 0x7fe0f406a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f3fff700 1 -- 172.21.15.5:0/815746179 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f40b16e0 con 0x7fe0f40b0d80 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.844+0000 7fe0f37fe700 1 --2- 172.21.15.5:0/815746179 >> v2:172.21.15.49:3300/0 conn(0x7fe0f40b4d50 0x7fe0f406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0f3fff700 1 --2- 172.21.15.5:0/815746179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f40b0d80 0x7fe0f40b3380 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40715b0 tx=0x7fe0e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33044658ef3c985b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0f2ffd700 1 -- 172.21.15.5:0/815746179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400b870 con 0x7fe0f40b0d80 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0f2ffd700 1 -- 172.21.15.5:0/815746179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e400b9d0 con 0x7fe0f40b0d80 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0f2ffd700 1 -- 172.21.15.5:0/815746179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400bd60 con 0x7fe0f40b0d80 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 -- 172.21.15.5:0/815746179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f40b0d80 msgr2=0x7fe0f40b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 --2- 172.21.15.5:0/815746179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f40b0d80 0x7fe0f40b3380 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40715b0 tx=0x7fe0e4009bd0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 -- 172.21.15.5:0/815746179 shutdown_connections 2024-05-08T23:26:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 --2- 172.21.15.5:0/815746179 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f406aea0 0x7fe0f40d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 --2- 172.21.15.5:0/815746179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f40b0d80 0x7fe0f40b3380 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 --2- 172.21.15.5:0/815746179 >> v2:172.21.15.49:3300/0 conn(0x7fe0f40b4d50 0x7fe0f406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 -- 172.21.15.5:0/815746179 >> 172.21.15.5:0/815746179 conn(0x7fe0f4099a50 msgr2=0x7fe0f40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 -- 172.21.15.5:0/815746179 shutdown_connections 2024-05-08T23:26:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.845+0000 7fe0faa38700 1 -- 172.21.15.5:0/815746179 wait complete. 2024-05-08T23:26:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.847+0000 7fe0faa38700 1 Processor -- start 2024-05-08T23:26:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.847+0000 7fe0faa38700 1 -- start start 2024-05-08T23:26:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0faa38700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f406aea0 0x7fe0f409e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0faa38700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f40b0d80 0x7fe0f40ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0faa38700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0f40b4d50 0x7fe0f41f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0faa38700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f413e9c0 con 0x7fe0f406aea0 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0faa38700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f41f73f0 con 0x7fe0f40b4d50 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0faa38700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f413ec30 con 0x7fe0f40b0d80 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f406aea0 0x7fe0f409e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f406aea0 0x7fe0f409e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49272/0 (socket says 172.21.15.5:49272) 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 -- 172.21.15.5:0/3938888174 learned_addr learned my addr 172.21.15.5:0/3938888174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f8fd5700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0f40b4d50 0x7fe0f41f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f37fe700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f40b0d80 0x7fe0f40ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 -- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f40b0d80 msgr2=0x7fe0f40ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f40b0d80 0x7fe0f40ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 -- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0f40b4d50 msgr2=0x7fe0f41f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0f40b4d50 0x7fe0f41f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:43.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 -- 172.21.15.5:0/3938888174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4009550 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f37fe700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f40b0d80 0x7fe0f40ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f8fd5700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0f40b4d50 0x7fe0f41f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.848+0000 7fe0f3fff700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f406aea0 0x7fe0f409e040 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4000c00 tx=0x7fe0e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.849+0000 7fe0f0ff9700 1 -- 172.21.15.5:0/3938888174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4004540 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.849+0000 7fe0f0ff9700 1 -- 172.21.15.5:0/3938888174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e40036a0 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.849+0000 7fe0f0ff9700 1 -- 172.21.15.5:0/3938888174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4003a50 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.849+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f413ef10 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.849+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f413f4d0 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.850+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f4074bc0 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.850+0000 7fe0f0ff9700 1 -- 172.21.15.5:0/3938888174 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe0e4021070 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.851+0000 7fe0f0ff9700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0e006ac90 0x7fe0e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.851+0000 7fe0f0ff9700 1 -- 172.21.15.5:0/3938888174 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe0e408aab0 con 0x7fe0f406aea0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.851+0000 7fe0f37fe700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0e006ac90 0x7fe0e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.851+0000 7fe0f37fe700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0e006ac90 0x7fe0e006d150 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe0f409e580 tx=0x7fe0e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:43.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:43.856+0000 7fe0f0ff9700 1 -- 172.21.15.5:0/3938888174 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe0e4055d40 con 0x7fe0f406aea0 2024-05-08T23:26:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.132+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe0f4144b70 con 0x7fe0f406aea0 2024-05-08T23:26:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.687+0000 7fe0f0ff9700 1 -- 172.21.15.5:0/3938888174 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0e40150a0 con 0x7fe0f406aea0 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0e006ac90 msgr2=0x7fe0e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0e006ac90 0x7fe0e006d150 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe0f409e580 tx=0x7fe0e8008040 comp rx=0 tx=0).stop 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f406aea0 msgr2=0x7fe0f409e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f406aea0 0x7fe0f409e040 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4000c00 tx=0x7fe0e4004750 comp rx=0 tx=0).stop 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 shutdown_connections 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0e006ac90 0x7fe0e006d150 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0f40b4d50 0x7fe0f41f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0f406aea0 0x7fe0f409e040 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 --2- 172.21.15.5:0/3938888174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0f40b0d80 0x7fe0f40ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 >> 172.21.15.5:0/3938888174 conn(0x7fe0f4099a50 msgr2=0x7fe0f406eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.690+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 shutdown_connections 2024-05-08T23:26:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.691+0000 7fe0faa38700 1 -- 172.21.15.5:0/3938888174 wait complete. 2024-05-08T23:26:44.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:26:44.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.895+0000 7f0aadbf5700 1 Processor -- start 2024-05-08T23:26:44.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.895+0000 7f0aadbf5700 1 -- start start 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aadbf5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0aa80b4d50 0x7f0aa806a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aadbf5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa8145040 con 0x7f0aa80b0d80 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aadbf5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa80b16e0 con 0x7f0aa806aea0 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aadbf5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa806b330 con 0x7f0aa80b4d50 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aa6ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0aa80b4d50 0x7f0aa806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aa77fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0aa80b0d80 0x7f0aa80b3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47220/0 (socket says 172.21.15.5:47220) 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aa77fe700 1 -- 172.21.15.5:0/1560224724 learned_addr learned my addr 172.21.15.5:0/1560224724 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aa6ffd700 1 -- 172.21.15.5:0/1560224724 >> v1:172.21.15.49:6789/0 conn(0x7f0aa806aea0 legacy=0x7f0aa81454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aa6ffd700 1 -- 172.21.15.5:0/1560224724 >> v1:172.21.15.5:6789/0 conn(0x7f0aa80b0d80 legacy=0x7f0aa80b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.896+0000 7f0aa6ffd700 1 -- 172.21.15.5:0/1560224724 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80b19c0 con 0x7f0aa80b4d50 2024-05-08T23:26:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aa6ffd700 1 --2- 172.21.15.5:0/1560224724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa806a960 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009840 tx=0x7f0a9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd700235b3ba7c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:44.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aa67fc700 1 -- 172.21.15.5:0/1560224724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c004230 con 0x7f0aa80b4d50 2024-05-08T23:26:44.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aa67fc700 1 -- 172.21.15.5:0/1560224724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c004390 con 0x7f0aa80b4d50 2024-05-08T23:26:44.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aa67fc700 1 -- 172.21.15.5:0/1560224724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0036a0 con 0x7f0aa80b4d50 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aadbf5700 1 -- 172.21.15.5:0/1560224724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 msgr2=0x7f0aa806a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aadbf5700 1 --2- 172.21.15.5:0/1560224724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa806a960 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009840 tx=0x7f0a9c009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aadbf5700 1 -- 172.21.15.5:0/1560224724 shutdown_connections 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aadbf5700 1 --2- 172.21.15.5:0/1560224724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa806a960 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aadbf5700 1 -- 172.21.15.5:0/1560224724 >> 172.21.15.5:0/1560224724 conn(0x7f0aa8099a50 msgr2=0x7f0aa80b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aadbf5700 1 -- 172.21.15.5:0/1560224724 shutdown_connections 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.897+0000 7f0aadbf5700 1 -- 172.21.15.5:0/1560224724 wait complete. 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.899+0000 7f0aadbf5700 1 Processor -- start 2024-05-08T23:26:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.899+0000 7f0aadbf5700 1 -- start start 2024-05-08T23:26:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aadbf5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aa806aea0 0x7f0aa815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aadbf5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0aa80b0d80 0x7f0aa806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aadbf5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa8142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aadbf5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa815d0e0 con 0x7f0aa80b4d50 2024-05-08T23:26:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aadbf5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa813fcb0 con 0x7f0aa806aea0 2024-05-08T23:26:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aadbf5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa813ff90 con 0x7f0aa80b0d80 2024-05-08T23:26:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa8142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa8142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49290/0 (socket says 172.21.15.5:49290) 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 -- 172.21.15.5:0/3690716025 learned_addr learned my addr 172.21.15.5:0/3690716025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa77fe700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aa806aea0 0x7f0aa815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa6ffd700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0aa80b0d80 0x7f0aa806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 -- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0aa80b0d80 msgr2=0x7f0aa806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0aa80b0d80 0x7f0aa806ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 -- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aa806aea0 msgr2=0x7f0aa815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aa806aea0 0x7f0aa815cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 -- 172.21.15.5:0/3690716025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0094a0 con 0x7f0aa80b4d50 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa77fe700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aa806aea0 0x7f0aa815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.900+0000 7f0aa7fff700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa8142ef0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000c7a0 tx=0x7f0aa000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.901+0000 7f0a8ffff700 1 -- 172.21.15.5:0/3690716025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0003e10 con 0x7f0aa80b4d50 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.901+0000 7f0a8ffff700 1 -- 172.21.15.5:0/3690716025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0005070 con 0x7f0aa80b4d50 2024-05-08T23:26:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.901+0000 7f0a8ffff700 1 -- 172.21.15.5:0/3690716025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0005420 con 0x7f0aa80b4d50 2024-05-08T23:26:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.901+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa8140270 con 0x7f0aa80b4d50 2024-05-08T23:26:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.901+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa8140830 con 0x7f0aa80b4d50 2024-05-08T23:26:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.902+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa8074bc0 con 0x7f0aa80b4d50 2024-05-08T23:26:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.902+0000 7f0a8ffff700 1 -- 172.21.15.5:0/3690716025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0aa0005580 con 0x7f0aa80b4d50 2024-05-08T23:26:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.903+0000 7f0a8ffff700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0a9406ac90 0x7f0a9406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:44.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.903+0000 7f0a8ffff700 1 -- 172.21.15.5:0/3690716025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0aa00885b0 con 0x7f0aa80b4d50 2024-05-08T23:26:44.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.903+0000 7f0aa77fe700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0a9406ac90 0x7f0a9406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:44.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.903+0000 7f0aa77fe700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0a9406ac90 0x7f0a9406d150 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0a98000f60 tx=0x7f0a98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:44.907+0000 7f0a8ffff700 1 -- 172.21.15.5:0/3690716025 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0aa0053720 con 0x7f0aa80b4d50 2024-05-08T23:26:45.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.125+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0aa8160d20 con 0x7f0aa80b4d50 2024-05-08T23:26:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.691+0000 7f0a8ffff700 1 -- 172.21.15.5:0/3690716025 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0aa0056d40 con 0x7f0aa80b4d50 2024-05-08T23:26:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0a9406ac90 msgr2=0x7f0a9406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0a9406ac90 0x7f0a9406d150 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0a98000f60 tx=0x7f0a98009040 comp rx=0 tx=0).stop 2024-05-08T23:26:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 msgr2=0x7f0aa8142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa8142ef0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000c7a0 tx=0x7f0aa000cab0 comp rx=0 tx=0).stop 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 shutdown_connections 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0a9406ac90 0x7f0a9406d150 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa80b4d50 0x7f0aa8142ef0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aa806aea0 0x7f0aa815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 --2- 172.21.15.5:0/3690716025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0aa80b0d80 0x7f0aa806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 >> 172.21.15.5:0/3690716025 conn(0x7f0aa8099a50 msgr2=0x7f0aa80cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 shutdown_connections 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.696+0000 7f0aadbf5700 1 -- 172.21.15.5:0/3690716025 wait complete. 2024-05-08T23:26:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:26:45.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:26:45.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.862+0000 7f6bbdaf5700 1 Processor -- start 2024-05-08T23:26:45.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.862+0000 7f6bbdaf5700 1 -- start start 2024-05-08T23:26:45.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bbdaf5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bb8160f20 0x7f6bb815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:45.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bbdaf5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb806faa0 con 0x7f6bb80b32f0 2024-05-08T23:26:45.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bbdaf5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb806dcd0 con 0x7f6bb806a960 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bbdaf5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb80b3770 con 0x7f6bb8160f20 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bb6ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6bb80b32f0 0x7f6bb815e7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47224/0 (socket says 172.21.15.5:47224) 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bb6ffd700 1 -- 172.21.15.5:0/521066543 learned_addr learned my addr 172.21.15.5:0/521066543 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bb77fe700 1 --2- 172.21.15.5:0/521066543 >> v2:172.21.15.49:3300/0 conn(0x7f6bb8160f20 0x7f6bb815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bb67fc700 1 -- 172.21.15.5:0/521066543 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377129538 0 0) 0x7f6bb806faa0 con 0x7f6bb80b32f0 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.863+0000 7f6bb67fc700 1 -- 172.21.15.5:0/521066543 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba00035b0 con 0x7f6bb80b32f0 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.864+0000 7f6bb77fe700 1 -- 172.21.15.5:0/521066543 >> v1:172.21.15.49:6789/0 conn(0x7f6bb806a960 legacy=0x7f6bb806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.864+0000 7f6bb77fe700 1 -- 172.21.15.5:0/521066543 >> v1:172.21.15.5:6789/0 conn(0x7f6bb80b32f0 legacy=0x7f6bb815e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.864+0000 7f6bb67fc700 1 -- 172.21.15.5:0/521066543 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341361488 0 0) 0x7f6bb806dcd0 con 0x7f6bb806a960 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.864+0000 7f6bb77fe700 1 -- 172.21.15.5:0/521066543 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb80b51a0 con 0x7f6bb8160f20 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.864+0000 7f6bb77fe700 1 --2- 172.21.15.5:0/521066543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb8160f20 0x7f6bb815ef80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009840 tx=0x7f6ba8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0435bb4ab949c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:45.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.864+0000 7f6bb67fc700 1 -- 172.21.15.5:0/521066543 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8011830 con 0x7f6bb8160f20 2024-05-08T23:26:45.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.864+0000 7f6bb67fc700 1 -- 172.21.15.5:0/521066543 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba8011990 con 0x7f6bb8160f20 2024-05-08T23:26:45.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.864+0000 7f6bb67fc700 1 -- 172.21.15.5:0/521066543 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8011d00 con 0x7f6bb8160f20 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.865+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/521066543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb8160f20 msgr2=0x7f6bb815ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.865+0000 7f6bbdaf5700 1 --2- 172.21.15.5:0/521066543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb8160f20 0x7f6bb815ef80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009840 tx=0x7f6ba8009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.865+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/521066543 shutdown_connections 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.865+0000 7f6bbdaf5700 1 --2- 172.21.15.5:0/521066543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb8160f20 0x7f6bb815ef80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.865+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/521066543 >> 172.21.15.5:0/521066543 conn(0x7f6bb80b5810 msgr2=0x7f6bb80b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.865+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/521066543 shutdown_connections 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.865+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/521066543 wait complete. 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.866+0000 7f6bbdaf5700 1 Processor -- start 2024-05-08T23:26:45.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.867+0000 7f6bbdaf5700 1 -- start start 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bbdaf5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb806a960 0x7f6bb80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bbdaf5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb80b32f0 0x7f6bb80aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bbdaf5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bb8160f20 0x7f6bb815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bbdaf5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb80af110 con 0x7f6bb806a960 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bbdaf5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb80c9ce0 con 0x7f6bb80b32f0 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bbdaf5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb80c9fc0 con 0x7f6bb8160f20 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb806a960 0x7f6bb80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb806a960 0x7f6bb80c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49302/0 (socket says 172.21.15.5:49302) 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 -- 172.21.15.5:0/757979737 learned_addr learned my addr 172.21.15.5:0/757979737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb6ffd700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb80b32f0 0x7f6bb80aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb7fff700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bb8160f20 0x7f6bb815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:45.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 -- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bb8160f20 msgr2=0x7f6bb815d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bb8160f20 0x7f6bb815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 -- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb80b32f0 msgr2=0x7f6bb80aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb80b32f0 0x7f6bb80aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 -- 172.21.15.5:0/757979737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80094a0 con 0x7f6bb806a960 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb6ffd700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb80b32f0 0x7f6bb80aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6bb77fe700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb806a960 0x7f6bb80c9670 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009b90 tx=0x7f6ba8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.868+0000 7f6b9ffff700 1 -- 172.21.15.5:0/757979737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80038a0 con 0x7f6bb806a960 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.869+0000 7f6b9ffff700 1 -- 172.21.15.5:0/757979737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba801e070 con 0x7f6bb806a960 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.869+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb80ca2a0 con 0x7f6bb806a960 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.869+0000 7f6b9ffff700 1 -- 172.21.15.5:0/757979737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8003c60 con 0x7f6bb806a960 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.869+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb80ca860 con 0x7f6bb806a960 2024-05-08T23:26:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.870+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb80caa50 con 0x7f6bb806a960 2024-05-08T23:26:45.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.870+0000 7f6b9ffff700 1 -- 172.21.15.5:0/757979737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6ba802d450 con 0x7f6bb806a960 2024-05-08T23:26:45.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.871+0000 7f6b9ffff700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ba406ac90 0x7f6ba406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:45.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.871+0000 7f6b9ffff700 1 -- 172.21.15.5:0/757979737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5529+0+0 (secure 0 0 0) 0x7f6ba808a2d0 con 0x7f6bb806a960 2024-05-08T23:26:45.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.871+0000 7f6bb6ffd700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ba406ac90 0x7f6ba406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:45.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.872+0000 7f6bb6ffd700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ba406ac90 0x7f6ba406d150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80cb430 tx=0x7f6bac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:45.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:45.876+0000 7f6b9ffff700 1 -- 172.21.15.5:0/757979737 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6ba8055550 con 0x7f6bb806a960 2024-05-08T23:26:46.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.091+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6bb8074bc0 con 0x7f6bb806a960 2024-05-08T23:26:46.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.696+0000 7f6b9ffff700 1 -- 172.21.15.5:0/757979737 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6ba8058b70 con 0x7f6bb806a960 2024-05-08T23:26:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ba406ac90 msgr2=0x7f6ba406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ba406ac90 0x7f6ba406d150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80cb430 tx=0x7f6bac009040 comp rx=0 tx=0).stop 2024-05-08T23:26:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb806a960 msgr2=0x7f6bb80c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb806a960 0x7f6bb80c9670 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009b90 tx=0x7f6ba8004750 comp rx=0 tx=0).stop 2024-05-08T23:26:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 shutdown_connections 2024-05-08T23:26:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ba406ac90 0x7f6ba406d150 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bb8160f20 0x7f6bb815d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb806a960 0x7f6bb80c9670 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 --2- 172.21.15.5:0/757979737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bb80b32f0 0x7f6bb80aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.703+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 >> 172.21.15.5:0/757979737 conn(0x7f6bb80b5810 msgr2=0x7f6bb80b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:46.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.704+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 shutdown_connections 2024-05-08T23:26:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.705+0000 7f6bbdaf5700 1 -- 172.21.15.5:0/757979737 wait complete. 2024-05-08T23:26:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-08T23:26:46.727 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T23:26:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.870+0000 7fcfc68a5700 1 Processor -- start 2024-05-08T23:26:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.870+0000 7fcfc68a5700 1 -- start start 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfc68a5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcfc0145040 0x7fcfc00b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfc68a5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc015fbb0 0x7fcfc00d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfc68a5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcfc00d0890 0x7fcfc006a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfc68a5700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc00d0d20 con 0x7fcfc0145040 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfc68a5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc00b6460 con 0x7fcfc00d0890 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfc68a5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00af6e0 con 0x7fcfc015fbb0 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcfc0145040 0x7fcfc00b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcfc0145040 0x7fcfc00b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49316/0 (socket says 172.21.15.5:49316) 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbffff700 1 -- 172.21.15.5:0/2002701945 learned_addr learned my addr 172.21.15.5:0/2002701945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbffff700 1 -- 172.21.15.5:0/2002701945 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc015fbb0 msgr2=0x7fcfc00d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbf7fe700 1 --2- 172.21.15.5:0/2002701945 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc015fbb0 0x7fcfc00d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbffff700 1 --2- 172.21.15.5:0/2002701945 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc015fbb0 0x7fcfc00d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfc4e42700 1 --2- 172.21.15.5:0/2002701945 >> v2:172.21.15.49:3300/0 conn(0x7fcfc00d0890 0x7fcfc006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbffff700 1 -- 172.21.15.5:0/2002701945 >> v2:172.21.15.49:3300/0 conn(0x7fcfc00d0890 msgr2=0x7fcfc006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbffff700 1 --2- 172.21.15.5:0/2002701945 >> v2:172.21.15.49:3300/0 conn(0x7fcfc00d0890 0x7fcfc006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfbffff700 1 -- 172.21.15.5:0/2002701945 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00aeb50 con 0x7fcfc0145040 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.871+0000 7fcfc4e42700 1 --2- 172.21.15.5:0/2002701945 >> v2:172.21.15.49:3300/0 conn(0x7fcfc00d0890 0x7fcfc006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfbffff700 1 --2- 172.21.15.5:0/2002701945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0145040 0x7fcfc00b5f20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcfc015f100 tx=0x7fcfb0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e77e0df66d592e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfbeffd700 1 -- 172.21.15.5:0/2002701945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000b860 con 0x7fcfc0145040 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfbeffd700 1 -- 172.21.15.5:0/2002701945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb000b9c0 con 0x7fcfc0145040 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfbeffd700 1 -- 172.21.15.5:0/2002701945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000bd50 con 0x7fcfc0145040 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 -- 172.21.15.5:0/2002701945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0145040 msgr2=0x7fcfc00b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/2002701945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0145040 0x7fcfc00b5f20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcfc015f100 tx=0x7fcfb0009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 -- 172.21.15.5:0/2002701945 shutdown_connections 2024-05-08T23:26:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/2002701945 >> v2:172.21.15.49:3300/0 conn(0x7fcfc00d0890 0x7fcfc006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/2002701945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0145040 0x7fcfc00b5f20 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/2002701945 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc015fbb0 0x7fcfc00d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 -- 172.21.15.5:0/2002701945 >> 172.21.15.5:0/2002701945 conn(0x7fcfc0099a50 msgr2=0x7fcfc00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:46.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 -- 172.21.15.5:0/2002701945 shutdown_connections 2024-05-08T23:26:46.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.872+0000 7fcfc68a5700 1 -- 172.21.15.5:0/2002701945 wait complete. 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.874+0000 7fcfc68a5700 1 Processor -- start 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.874+0000 7fcfc68a5700 1 -- start start 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfc68a5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc00d0890 0x7fcfc009a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfc68a5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0145040 0x7fcfc009cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfc68a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc015fbb0 0x7fcfc015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfc68a5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc009d820 con 0x7fcfc00d0890 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfc68a5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc009db00 con 0x7fcfc015fbb0 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfc68a5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc009dde0 con 0x7fcfc0145040 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc00d0890 0x7fcfc009a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc00d0890 0x7fcfc009a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49326/0 (socket says 172.21.15.5:49326) 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 -- 172.21.15.5:0/4034504981 learned_addr learned my addr 172.21.15.5:0/4034504981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:46.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbf7fe700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0145040 0x7fcfc009cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfc4e42700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc015fbb0 0x7fcfc015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 -- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0145040 msgr2=0x7fcfc009cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0145040 0x7fcfc009cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 -- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc015fbb0 msgr2=0x7fcfc015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc015fbb0 0x7fcfc015aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 -- 172.21.15.5:0/4034504981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0009550 con 0x7fcfc00d0890 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfc4e42700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc015fbb0 0x7fcfc015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbf7fe700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0145040 0x7fcfc009cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.875+0000 7fcfbffff700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc00d0890 0x7fcfc009a650 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0000c00 tx=0x7fcfb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.876+0000 7fcfbcff9700 1 -- 172.21.15.5:0/4034504981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00036a0 con 0x7fcfc00d0890 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.876+0000 7fcfbcff9700 1 -- 172.21.15.5:0/4034504981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb001f070 con 0x7fcfc00d0890 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.876+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc009e0c0 con 0x7fcfc00d0890 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.876+0000 7fcfbcff9700 1 -- 172.21.15.5:0/4034504981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0003a60 con 0x7fcfc00d0890 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.876+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc009ab90 con 0x7fcfc00d0890 2024-05-08T23:26:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.877+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc01328d0 con 0x7fcfc00d0890 2024-05-08T23:26:46.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.877+0000 7fcfbcff9700 1 -- 172.21.15.5:0/4034504981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcfb0003bc0 con 0x7fcfc00d0890 2024-05-08T23:26:46.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.877+0000 7fcfbcff9700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcfac06ac90 0x7fcfac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:46.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.878+0000 7fcfbcff9700 1 -- 172.21.15.5:0/4034504981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcfb0021070 con 0x7fcfc00d0890 2024-05-08T23:26:46.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.878+0000 7fcfbf7fe700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcfac06ac90 0x7fcfac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:46.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.878+0000 7fcfbf7fe700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcfac06ac90 0x7fcfac06d150 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00d06c0 tx=0x7fcfb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:46.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:46.882+0000 7fcfbcff9700 1 -- 172.21.15.5:0/4034504981 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcfb0055ef0 con 0x7fcfc00d0890 2024-05-08T23:26:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.102+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcfc0074bc0 con 0x7fcfc00d0890 2024-05-08T23:26:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.106+0000 7fcfbcff9700 1 -- 172.21.15.5:0/4034504981 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcfb0059510 con 0x7fcfc00d0890 2024-05-08T23:26:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcfac06ac90 msgr2=0x7fcfac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcfac06ac90 0x7fcfac06d150 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00d06c0 tx=0x7fcfb4008040 comp rx=0 tx=0).stop 2024-05-08T23:26:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc00d0890 msgr2=0x7fcfc009a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc00d0890 0x7fcfc009a650 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0000c00 tx=0x7fcfb0004750 comp rx=0 tx=0).stop 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 shutdown_connections 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcfac06ac90 0x7fcfac06d150 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc015fbb0 0x7fcfc015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc00d0890 0x7fcfc009a650 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 --2- 172.21.15.5:0/4034504981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0145040 0x7fcfc009cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 >> 172.21.15.5:0/4034504981 conn(0x7fcfc0099a50 msgr2=0x7fcfc0142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 shutdown_connections 2024-05-08T23:26:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.109+0000 7fcfc68a5700 1 -- 172.21.15.5:0/4034504981 wait complete. 2024-05-08T23:26:47.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-05-08T23:26:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.260+0000 7f6aff141700 1 Processor -- start 2024-05-08T23:26:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.260+0000 7f6aff141700 1 -- start start 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.260+0000 7f6aff141700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af80d3f40 0x7f6af8161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6aff141700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af8162450 con 0x7f6af815d7a0 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6aff141700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af80cee00 con 0x7f6af815db80 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6aff141700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80b27e0 con 0x7f6af80d3f40 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6afcedd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6af815d7a0 0x7f6af80b0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47238/0 (socket says 172.21.15.5:47238) 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6afcedd700 1 -- 172.21.15.5:0/1196112234 learned_addr learned my addr 172.21.15.5:0/1196112234 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af7fff700 1 --2- 172.21.15.5:0/1196112234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af80d3f40 0x7f6af8161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af77fe700 1 -- 172.21.15.5:0/1196112234 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084423524 0 0) 0x7f6ae8001110 con 0x7f6af815d7a0 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af77fe700 1 -- 172.21.15.5:0/1196112234 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae00035b0 con 0x7f6af815d7a0 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af77fe700 1 -- 172.21.15.5:0/1196112234 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189951093 0 0) 0x7f6af80cee00 con 0x7f6af815db80 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af77fe700 1 -- 172.21.15.5:0/1196112234 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae8001110 con 0x7f6af815db80 2024-05-08T23:26:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af77fe700 1 -- 172.21.15.5:0/1196112234 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2954294448 0 0) 0x7f6ae00035b0 con 0x7f6af815d7a0 2024-05-08T23:26:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af7fff700 1 -- 172.21.15.5:0/1196112234 >> v1:172.21.15.49:6789/0 conn(0x7f6af815db80 legacy=0x7f6af80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af7fff700 1 -- 172.21.15.5:0/1196112234 >> v1:172.21.15.5:6789/0 conn(0x7f6af815d7a0 legacy=0x7f6af80b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.261+0000 7f6af7fff700 1 -- 172.21.15.5:0/1196112234 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80d4400 con 0x7f6af80d3f40 2024-05-08T23:26:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6af7fff700 1 --2- 172.21.15.5:0/1196112234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af80d3f40 0x7f6af8161f10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009850 tx=0x7f6aec009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f6d615f8e72a7a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6af77fe700 1 -- 172.21.15.5:0/1196112234 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8001800 con 0x7f6af80d3f40 2024-05-08T23:26:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6af77fe700 1 -- 172.21.15.5:0/1196112234 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec005c30 con 0x7f6af80d3f40 2024-05-08T23:26:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6af77fe700 1 -- 172.21.15.5:0/1196112234 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec004030 con 0x7f6af80d3f40 2024-05-08T23:26:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6aff141700 1 -- 172.21.15.5:0/1196112234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af80d3f40 msgr2=0x7f6af8161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6aff141700 1 --2- 172.21.15.5:0/1196112234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af80d3f40 0x7f6af8161f10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009850 tx=0x7f6aec009b60 comp rx=0 tx=0).stop 2024-05-08T23:26:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6aff141700 1 -- 172.21.15.5:0/1196112234 shutdown_connections 2024-05-08T23:26:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6aff141700 1 --2- 172.21.15.5:0/1196112234 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af80d3f40 0x7f6af8161f10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6aff141700 1 -- 172.21.15.5:0/1196112234 >> 172.21.15.5:0/1196112234 conn(0x7f6af808afe0 msgr2=0x7f6af80d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6aff141700 1 -- 172.21.15.5:0/1196112234 shutdown_connections 2024-05-08T23:26:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.262+0000 7f6aff141700 1 -- 172.21.15.5:0/1196112234 wait complete. 2024-05-08T23:26:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.264+0000 7f6aff141700 1 Processor -- start 2024-05-08T23:26:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.264+0000 7f6aff141700 1 -- start start 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6aff141700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af80d3f40 0x7f6af814c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6aff141700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af815d7a0 0x7f6af814bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6aff141700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af815db80 0x7f6af80c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6aff141700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af814c010 con 0x7f6af815d7a0 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6aff141700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80cb4f0 con 0x7f6af80d3f40 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6aff141700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80cb790 con 0x7f6af815db80 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af815d7a0 0x7f6af814bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af815d7a0 0x7f6af814bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49334/0 (socket says 172.21.15.5:49334) 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 -- 172.21.15.5:0/1747423439 learned_addr learned my addr 172.21.15.5:0/1747423439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6afcedd700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af80d3f40 0x7f6af814c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6afd6de700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af815db80 0x7f6af80c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 -- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af815db80 msgr2=0x7f6af80c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af815db80 0x7f6af80c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 -- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af80d3f40 msgr2=0x7f6af814c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af80d3f40 0x7f6af814c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 -- 172.21.15.5:0/1747423439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec009430 con 0x7f6af815d7a0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af7fff700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af815d7a0 0x7f6af814bad0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6af808ba90 tx=0x7f6aec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.265+0000 7f6af57fa700 1 -- 172.21.15.5:0/1747423439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec003c30 con 0x7f6af815d7a0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.266+0000 7f6af57fa700 1 -- 172.21.15.5:0/1747423439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec003d90 con 0x7f6af815d7a0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.266+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af80cba70 con 0x7f6af815d7a0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.266+0000 7f6af57fa700 1 -- 172.21.15.5:0/1747423439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec011660 con 0x7f6af815d7a0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.266+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af80c8a60 con 0x7f6af815d7a0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.267+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af8068000 con 0x7f6af815d7a0 2024-05-08T23:26:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.267+0000 7f6af57fa700 1 -- 172.21.15.5:0/1747423439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6aec0117c0 con 0x7f6af815d7a0 2024-05-08T23:26:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.267+0000 7f6af57fa700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ae406ac90 0x7f6ae406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.268+0000 7f6af57fa700 1 -- 172.21.15.5:0/1747423439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6aec08a7b0 con 0x7f6af815d7a0 2024-05-08T23:26:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.268+0000 7f6afcedd700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ae406ac90 0x7f6ae406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.268+0000 7f6afcedd700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ae406ac90 0x7f6ae406d150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005af0 tx=0x7f6ae8001c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.272+0000 7f6af57fa700 1 -- 172.21.15.5:0/1747423439 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6aec055cf0 con 0x7f6af815d7a0 2024-05-08T23:26:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.578+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) v1 -- 0x7f6af80d0e30 con 0x7f6af815d7a0 2024-05-08T23:26:47.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.703+0000 7f6af57fa700 1 -- 172.21.15.5:0/1747423439 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v296) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6aec02c0d0 con 0x7f6af815d7a0 2024-05-08T23:26:47.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ae406ac90 msgr2=0x7f6ae406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ae406ac90 0x7f6ae406d150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005af0 tx=0x7f6ae8001c90 comp rx=0 tx=0).stop 2024-05-08T23:26:47.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af815d7a0 msgr2=0x7f6af814bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af815d7a0 0x7f6af814bad0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6af808ba90 tx=0x7f6aec004750 comp rx=0 tx=0).stop 2024-05-08T23:26:47.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 shutdown_connections 2024-05-08T23:26:47.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6ae406ac90 0x7f6ae406d150 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6af815db80 0x7f6af80c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6af80d3f40 0x7f6af814c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 --2- 172.21.15.5:0/1747423439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af815d7a0 0x7f6af814bad0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 >> 172.21.15.5:0/1747423439 conn(0x7f6af808afe0 msgr2=0x7f6af80b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 shutdown_connections 2024-05-08T23:26:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.706+0000 7f6aff141700 1 -- 172.21.15.5:0/1747423439 wait complete. 2024-05-08T23:26:47.721 INFO:tasks.cephfs_test_runner:test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-05-08T23:26:47.721 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-05-08T23:26:47.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.860+0000 7f3eea2e4700 1 Processor -- start 2024-05-08T23:26:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.860+0000 7f3eea2e4700 1 -- start start 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.861+0000 7f3eea2e4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee415ef60 0x7f3ee40b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.861+0000 7f3eea2e4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ee40cf000 0x7f3ee40cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.861+0000 7f3eea2e4700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee40cfe50 con 0x7f3ee40cf9c0 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.861+0000 7f3eea2e4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee40cd400 con 0x7f3ee40cf000 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.861+0000 7f3eea2e4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee409ad40 con 0x7f3ee415ef60 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.861+0000 7f3ee37fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ee40cf000 0x7f3ee40cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee37fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ee40cf000 0x7f3ee40cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33668/0 (socket says 172.21.15.5:33668) 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee37fe700 1 -- 172.21.15.5:0/3213625170 learned_addr learned my addr 172.21.15.5:0/3213625170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee3fff700 1 --2- 172.21.15.5:0/3213625170 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee415ef60 0x7f3ee40b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee37fe700 1 -- 172.21.15.5:0/3213625170 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee415ef60 msgr2=0x7f3ee40b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee37fe700 1 --2- 172.21.15.5:0/3213625170 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee415ef60 0x7f3ee40b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee37fe700 1 -- 172.21.15.5:0/3213625170 >> v1:172.21.15.49:6789/0 conn(0x7f3ee40cf9c0 legacy=0x7f3ee4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee37fe700 1 -- 172.21.15.5:0/3213625170 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee40ccb60 con 0x7f3ee40cf000 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/3213625170 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201060964 0 0) 0x7f3ee40cfe50 con 0x7f3ee40cf9c0 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee37fe700 1 --2- 172.21.15.5:0/3213625170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee40cf000 0x7f3ee40cf480 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0098f0 tx=0x7f3ecc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1765d50de34243 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/3213625170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc011830 con 0x7f3ee40cf000 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/3213625170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc011990 con 0x7f3ee40cf000 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.862+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/3213625170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc011d20 con 0x7f3ee40cf000 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.863+0000 7f3eea2e4700 1 -- 172.21.15.5:0/3213625170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee40cf000 msgr2=0x7f3ee40cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.863+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/3213625170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee40cf000 0x7f3ee40cf480 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0098f0 tx=0x7f3ecc009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.863+0000 7f3eea2e4700 1 -- 172.21.15.5:0/3213625170 shutdown_connections 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.863+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/3213625170 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee415ef60 0x7f3ee40b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.863+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/3213625170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee40cf000 0x7f3ee40cf480 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.863+0000 7f3eea2e4700 1 -- 172.21.15.5:0/3213625170 >> 172.21.15.5:0/3213625170 conn(0x7f3ee40b2f20 msgr2=0x7f3ee40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.863+0000 7f3eea2e4700 1 -- 172.21.15.5:0/3213625170 shutdown_connections 2024-05-08T23:26:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.863+0000 7f3eea2e4700 1 -- 172.21.15.5:0/3213625170 wait complete. 2024-05-08T23:26:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.865+0000 7f3eea2e4700 1 Processor -- start 2024-05-08T23:26:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.865+0000 7f3eea2e4700 1 -- start start 2024-05-08T23:26:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3eea2e4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee40cf000 0x7f3ee40c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3eea2e4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ee40cf9c0 0x7f3ee4158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3eea2e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee415ef60 0x7f3ee41f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3eea2e4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee40c87b0 con 0x7f3ee415ef60 2024-05-08T23:26:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3eea2e4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee40c8ed0 con 0x7f3ee40cf9c0 2024-05-08T23:26:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3eea2e4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41f6670 con 0x7f3ee40cf000 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee415ef60 0x7f3ee41f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee415ef60 0x7f3ee41f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33676/0 (socket says 172.21.15.5:33676) 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 -- 172.21.15.5:0/887864065 learned_addr learned my addr 172.21.15.5:0/887864065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee3fff700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee40cf000 0x7f3ee40c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee37fe700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ee40cf9c0 0x7f3ee4158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 -- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee40cf000 msgr2=0x7f3ee40c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee40cf000 0x7f3ee40c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 -- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ee40cf9c0 msgr2=0x7f3ee4158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ee40cf9c0 0x7f3ee4158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 -- 172.21.15.5:0/887864065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc009550 con 0x7f3ee415ef60 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.866+0000 7f3ee8881700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee415ef60 0x7f3ee41f6de0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00c7a0 tx=0x7f3edc00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.867+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/887864065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc003e30 con 0x7f3ee415ef60 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.867+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/887864065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc005070 con 0x7f3ee415ef60 2024-05-08T23:26:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.867+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee40c9100 con 0x7f3ee415ef60 2024-05-08T23:26:47.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.867+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/887864065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc005420 con 0x7f3ee415ef60 2024-05-08T23:26:47.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.867+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee40c96c0 con 0x7f3ee415ef60 2024-05-08T23:26:47.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.868+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee4068000 con 0x7f3ee415ef60 2024-05-08T23:26:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.868+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/887864065 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3edc005580 con 0x7f3ee415ef60 2024-05-08T23:26:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.869+0000 7f3ee0ff9700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3ec406acc0 0x7f3ec406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.869+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/887864065 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f3edc088700 con 0x7f3ee415ef60 2024-05-08T23:26:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.869+0000 7f3ee3fff700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3ec406acc0 0x7f3ec406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.870+0000 7f3ee3fff700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3ec406acc0 0x7f3ec406d180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009ad0 tx=0x7f3ed4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:47.874+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/887864065 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3edc053bc0 con 0x7f3ee415ef60 2024-05-08T23:26:48.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.096+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ee415d160 con 0x7f3ee415ef60 2024-05-08T23:26:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.097+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/887864065 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+12679 (secure 0 0 0) 0x7f3edc017020 con 0x7f3ee415ef60 2024-05-08T23:26:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:48.097 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:46.691090+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6837/3300231911":"2024-05-09T23:26:43.679648+0000","172.21.15.5:6834/690633393":"2024-05-09T23:26:43.679653+0000","172.21.15.49:6835/3300231911":"2024-05-09T23:26:43.679648+0000","172.21.15.5:6832/690633393":"2024-05-09T23:26:43.679653+0000","172.21.15.49:6836/2469967007":"2024-05-09T23:26:43.679632+0000","172.21.15.49:6834/2469967007":"2024-05-09T23:26:43.679632+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.099+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3ec406acc0 msgr2=0x7f3ec406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.099+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3ec406acc0 0x7f3ec406d180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009ad0 tx=0x7f3ed4008040 comp rx=0 tx=0).stop 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.099+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee415ef60 msgr2=0x7f3ee41f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.099+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee415ef60 0x7f3ee41f6de0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00c7a0 tx=0x7f3edc00cb60 comp rx=0 tx=0).stop 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.100+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 shutdown_connections 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.100+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3ec406acc0 0x7f3ec406d180 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.100+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ee415ef60 0x7f3ee41f6de0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.100+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3ee40cf000 0x7f3ee40c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.100+0000 7f3eea2e4700 1 --2- 172.21.15.5:0/887864065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ee40cf9c0 0x7f3ee4158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.100+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 >> 172.21.15.5:0/887864065 conn(0x7f3ee40b2f20 msgr2=0x7f3ee40b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.100+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 shutdown_connections 2024-05-08T23:26:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.100+0000 7f3eea2e4700 1 -- 172.21.15.5:0/887864065 wait complete. 2024-05-08T23:26:48.114 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:26:48.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:26:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.253+0000 7fb260ef2700 1 Processor -- start 2024-05-08T23:26:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.253+0000 7fb260ef2700 1 -- start start 2024-05-08T23:26:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.254+0000 7fb260ef2700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb25c15bb20 0x7fb25c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.254+0000 7fb260ef2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c06ec20 0x7fb25c159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.254+0000 7fb260ef2700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25c0cc980 con 0x7fb25c0cfd10 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.254+0000 7fb260ef2700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c06f0b0 con 0x7fb25c15bb20 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.254+0000 7fb260ef2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0d00f0 con 0x7fb25c06ec20 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb259d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb25c15bb20 0x7fb25c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25ad9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c06ec20 0x7fb25c159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25a59c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb25c0cfd10 0x7fb25c06dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:53178/0 (socket says 172.21.15.5:53178) 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25ad9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c06ec20 0x7fb25c159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:36178/0 (socket says 172.21.15.5:36178) 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25a59c700 1 -- 172.21.15.5:0/1853624939 learned_addr learned my addr 172.21.15.5:0/1853624939 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25ad9d700 1 -- 172.21.15.5:0/1853624939 >> v2:172.21.15.49:3300/0 conn(0x7fb25c15bb20 msgr2=0x7fb25c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25ad9d700 1 --2- 172.21.15.5:0/1853624939 >> v2:172.21.15.49:3300/0 conn(0x7fb25c15bb20 0x7fb25c06e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25ad9d700 1 -- 172.21.15.5:0/1853624939 >> v1:172.21.15.49:6789/0 conn(0x7fb25c0cfd10 legacy=0x7fb25c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25ad9d700 1 -- 172.21.15.5:0/1853624939 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c0ccbd0 con 0x7fb25c06ec20 2024-05-08T23:26:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb259d9b700 1 --2- 172.21.15.5:0/1853624939 >> v2:172.21.15.49:3300/0 conn(0x7fb25c15bb20 0x7fb25c06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.255+0000 7fb25ad9d700 1 --2- 172.21.15.5:0/1853624939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c06ec20 0x7fb25c159f80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb254009d90 tx=0x7fb25400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ceff9b70a4d9a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb25959a700 1 -- 172.21.15.5:0/1853624939 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400e070 con 0x7fb25c06ec20 2024-05-08T23:26:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb25959a700 1 -- 172.21.15.5:0/1853624939 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25400bb20 con 0x7fb25c06ec20 2024-05-08T23:26:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb25959a700 1 -- 172.21.15.5:0/1853624939 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254012430 con 0x7fb25c06ec20 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb260ef2700 1 -- 172.21.15.5:0/1853624939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c06ec20 msgr2=0x7fb25c159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb260ef2700 1 --2- 172.21.15.5:0/1853624939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c06ec20 0x7fb25c159f80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb254009d90 tx=0x7fb25400b330 comp rx=0 tx=0).stop 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb260ef2700 1 -- 172.21.15.5:0/1853624939 shutdown_connections 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb260ef2700 1 --2- 172.21.15.5:0/1853624939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c06ec20 0x7fb25c159f80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb260ef2700 1 --2- 172.21.15.5:0/1853624939 >> v2:172.21.15.49:3300/0 conn(0x7fb25c15bb20 0x7fb25c06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb260ef2700 1 -- 172.21.15.5:0/1853624939 >> 172.21.15.5:0/1853624939 conn(0x7fb25c099a50 msgr2=0x7fb25c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb260ef2700 1 -- 172.21.15.5:0/1853624939 shutdown_connections 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.256+0000 7fb260ef2700 1 -- 172.21.15.5:0/1853624939 wait complete. 2024-05-08T23:26:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.258+0000 7fb260ef2700 1 Processor -- start 2024-05-08T23:26:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.258+0000 7fb260ef2700 1 -- start start 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb260ef2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb25c06ec20 0x7fb25c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb260ef2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb25c0cfd10 0x7fb25c15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb260ef2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c15bb20 0x7fb25c159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb260ef2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b15b0 con 0x7fb25c06ec20 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb260ef2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b2a70 con 0x7fb25c0cfd10 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb260ef2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b0f60 con 0x7fb25c15bb20 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb25c06ec20 0x7fb25c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb25c06ec20 0x7fb25c15a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33680/0 (socket says 172.21.15.5:33680) 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 -- 172.21.15.5:0/3615008061 learned_addr learned my addr 172.21.15.5:0/3615008061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb259d9b700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb25c0cfd10 0x7fb25c15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25ad9d700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c15bb20 0x7fb25c159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 -- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c15bb20 msgr2=0x7fb25c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c15bb20 0x7fb25c159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 -- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb25c0cfd10 msgr2=0x7fb25c15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb25c0cfd10 0x7fb25c15bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 -- 172.21.15.5:0/3615008061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2440094a0 con 0x7fb25c06ec20 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb25a59c700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb25c06ec20 0x7fb25c15a9b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00c520 tx=0x7fb24c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.259+0000 7fb252ffd700 1 -- 172.21.15.5:0/3615008061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c005d70 con 0x7fb25c06ec20 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.260+0000 7fb252ffd700 1 -- 172.21.15.5:0/3615008061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c0070d0 con 0x7fb25c06ec20 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.260+0000 7fb252ffd700 1 -- 172.21.15.5:0/3615008061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c007440 con 0x7fb25c06ec20 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.260+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb254009a70 con 0x7fb25c06ec20 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.260+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c15c490 con 0x7fb25c06ec20 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.261+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c0b2d50 con 0x7fb25c06ec20 2024-05-08T23:26:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.261+0000 7fb252ffd700 1 -- 172.21.15.5:0/3615008061 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb24c0075a0 con 0x7fb25c06ec20 2024-05-08T23:26:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.261+0000 7fb252ffd700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb23c06ac90 0x7fb23c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.262+0000 7fb252ffd700 1 -- 172.21.15.5:0/3615008061 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb24c0883f0 con 0x7fb25c06ec20 2024-05-08T23:26:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.262+0000 7fb259d9b700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb23c06ac90 0x7fb23c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.262+0000 7fb259d9b700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb23c06ac90 0x7fb23c06d150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb25c15bf20 tx=0x7fb244008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.266+0000 7fb252ffd700 1 -- 172.21.15.5:0/3615008061 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb25c0b2d50 con 0x7fb25c06ec20 2024-05-08T23:26:48.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.480+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb25c0b2d50 con 0x7fb25c06ec20 2024-05-08T23:26:48.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.481+0000 7fb252ffd700 1 -- 172.21.15.5:0/3615008061 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb25c0b2d50 con 0x7fb25c06ec20 2024-05-08T23:26:48.481 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb23c06ac90 msgr2=0x7fb23c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb23c06ac90 0x7fb23c06d150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb25c15bf20 tx=0x7fb244008040 comp rx=0 tx=0).stop 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb25c06ec20 msgr2=0x7fb25c15a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb25c06ec20 0x7fb25c15a9b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00c520 tx=0x7fb24c00c8e0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 shutdown_connections 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb23c06ac90 0x7fb23c06d150 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb25c15bb20 0x7fb25c159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb25c06ec20 0x7fb25c15a9b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 --2- 172.21.15.5:0/3615008061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb25c0cfd10 0x7fb25c15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 >> 172.21.15.5:0/3615008061 conn(0x7fb25c099a50 msgr2=0x7fb25c0cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 shutdown_connections 2024-05-08T23:26:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.483+0000 7fb260ef2700 1 -- 172.21.15.5:0/3615008061 wait complete. 2024-05-08T23:26:48.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-05-08T23:26:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.634+0000 7f712bb6a700 1 Processor -- start 2024-05-08T23:26:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.634+0000 7f712bb6a700 1 -- start start 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.635+0000 7f712bb6a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7124161f80 0x7f712406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.635+0000 7f712bb6a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f71240ccbc0 0x7f71240ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.635+0000 7f712bb6a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71240d0060 con 0x7f71240cec00 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.635+0000 7f712bb6a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7124162360 con 0x7f71240ccbc0 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.635+0000 7f712bb6a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712406ea70 con 0x7f7124161f80 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7129906700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7124161f80 0x7f712406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7129105700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f71240ccbc0 0x7f71240ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f712a107700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f71240cec00 0x7f71240d04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:53192/0 (socket says 172.21.15.5:53192) 2024-05-08T23:26:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f712a107700 1 -- 172.21.15.5:0/579513842 learned_addr learned my addr 172.21.15.5:0/579513842 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7128904700 1 -- 172.21.15.5:0/579513842 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570399709 0 0) 0x7f711c001490 con 0x7f71240cec00 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7128904700 1 -- 172.21.15.5:0/579513842 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71100035b0 con 0x7f71240cec00 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7129906700 1 -- 172.21.15.5:0/579513842 >> v2:172.21.15.49:3300/0 conn(0x7f71240ccbc0 msgr2=0x7f71240ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7129906700 1 --2- 172.21.15.5:0/579513842 >> v2:172.21.15.49:3300/0 conn(0x7f71240ccbc0 0x7f71240ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7129906700 1 -- 172.21.15.5:0/579513842 >> v1:172.21.15.49:6789/0 conn(0x7f71240cec00 legacy=0x7f71240d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7129906700 1 -- 172.21.15.5:0/579513842 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71240cd130 con 0x7f7124161f80 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7129105700 1 --2- 172.21.15.5:0/579513842 >> v2:172.21.15.49:3300/0 conn(0x7f71240ccbc0 0x7f71240ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7129906700 1 --2- 172.21.15.5:0/579513842 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7124161f80 0x7f712406e530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7120009840 tx=0x7f7120009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a4fdfb5fa933670 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7128904700 1 -- 172.21.15.5:0/579513842 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120011830 con 0x7f7124161f80 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7128904700 1 -- 172.21.15.5:0/579513842 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120011990 con 0x7f7124161f80 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.636+0000 7f7128904700 1 -- 172.21.15.5:0/579513842 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120011d20 con 0x7f7124161f80 2024-05-08T23:26:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.637+0000 7f712bb6a700 1 -- 172.21.15.5:0/579513842 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7124161f80 msgr2=0x7f712406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.637+0000 7f712bb6a700 1 --2- 172.21.15.5:0/579513842 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7124161f80 0x7f712406e530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7120009840 tx=0x7f7120009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.637+0000 7f712bb6a700 1 -- 172.21.15.5:0/579513842 shutdown_connections 2024-05-08T23:26:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.637+0000 7f712bb6a700 1 --2- 172.21.15.5:0/579513842 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7124161f80 0x7f712406e530 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.637+0000 7f712bb6a700 1 --2- 172.21.15.5:0/579513842 >> v2:172.21.15.49:3300/0 conn(0x7f71240ccbc0 0x7f71240ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.637+0000 7f712bb6a700 1 -- 172.21.15.5:0/579513842 >> 172.21.15.5:0/579513842 conn(0x7f71240b5810 msgr2=0x7f71240b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.637+0000 7f712bb6a700 1 -- 172.21.15.5:0/579513842 shutdown_connections 2024-05-08T23:26:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.637+0000 7f712bb6a700 1 -- 172.21.15.5:0/579513842 wait complete. 2024-05-08T23:26:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.638+0000 7f712bb6a700 1 Processor -- start 2024-05-08T23:26:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.638+0000 7f712bb6a700 1 -- start start 2024-05-08T23:26:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f712bb6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71240ccbc0 0x7f71241fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f712bb6a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f71240cec00 0x7f71240aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f712bb6a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7124161f80 0x7f7124154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f712bb6a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71241fc4f0 con 0x7f71240ccbc0 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f712bb6a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7124164340 con 0x7f7124161f80 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f712bb6a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7124164620 con 0x7f71240cec00 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f7129906700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71240ccbc0 0x7f71241fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f7129906700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71240ccbc0 0x7f71241fbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33694/0 (socket says 172.21.15.5:33694) 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.639+0000 7f7129906700 1 -- 172.21.15.5:0/3795161197 learned_addr learned my addr 172.21.15.5:0/3795161197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f712a107700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7124161f80 0x7f7124154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f7129105700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f71240cec00 0x7f71240aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f7129906700 1 -- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f71240cec00 msgr2=0x7f71240aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f7129906700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f71240cec00 0x7f71240aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f7129906700 1 -- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7124161f80 msgr2=0x7f7124154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f7129906700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7124161f80 0x7f7124154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f7129906700 1 -- 172.21.15.5:0/3795161197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140094a0 con 0x7f71240ccbc0 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f712a107700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7124161f80 0x7f7124154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f7129906700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71240ccbc0 0x7f71241fbfb0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7120009b90 tx=0x7f7120004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f711a7fc700 1 -- 172.21.15.5:0/3795161197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120004590 con 0x7f71240ccbc0 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f711a7fc700 1 -- 172.21.15.5:0/3795161197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71200036a0 con 0x7f71240ccbc0 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71200094a0 con 0x7f71240ccbc0 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f711a7fc700 1 -- 172.21.15.5:0/3795161197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120003a30 con 0x7f71240ccbc0 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.640+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7124164be0 con 0x7f71240ccbc0 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.641+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7124068000 con 0x7f71240ccbc0 2024-05-08T23:26:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.642+0000 7f711a7fc700 1 -- 172.21.15.5:0/3795161197 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7120003b90 con 0x7f71240ccbc0 2024-05-08T23:26:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.642+0000 7f711a7fc700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f710806ac90 0x7f710806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.642+0000 7f711a7fc700 1 -- 172.21.15.5:0/3795161197 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f712008a460 con 0x7f71240ccbc0 2024-05-08T23:26:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.642+0000 7f7129105700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f710806ac90 0x7f710806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.643+0000 7f7129105700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f710806ac90 0x7f710806d150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7124165630 tx=0x7f7114008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:48.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.647+0000 7f711a7fc700 1 -- 172.21.15.5:0/3795161197 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f71200558a0 con 0x7f71240ccbc0 2024-05-08T23:26:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:48.956+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) v1 -- 0x7f71240b65d0 con 0x7f71240ccbc0 2024-05-08T23:26:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.709+0000 7f711a7fc700 1 -- 172.21.15.5:0/3795161197 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v298) v1 ==== 157+0+0 (secure 0 0 0) 0x7f7120058ec0 con 0x7f71240ccbc0 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f710806ac90 msgr2=0x7f710806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f710806ac90 0x7f710806d150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7124165630 tx=0x7f7114008040 comp rx=0 tx=0).stop 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71240ccbc0 msgr2=0x7f71241fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71240ccbc0 0x7f71241fbfb0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7120009b90 tx=0x7f7120004750 comp rx=0 tx=0).stop 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 shutdown_connections 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f710806ac90 0x7f710806d150 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7124161f80 0x7f7124154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71240ccbc0 0x7f71241fbfb0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 --2- 172.21.15.5:0/3795161197 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f71240cec00 0x7f71240aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 >> 172.21.15.5:0/3795161197 conn(0x7f71240b5810 msgr2=0x7f71241610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 shutdown_connections 2024-05-08T23:26:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.712+0000 7f712bb6a700 1 -- 172.21.15.5:0/3795161197 wait complete. 2024-05-08T23:26:49.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T23:26:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.872+0000 7fa2eec48700 1 Processor -- start 2024-05-08T23:26:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.872+0000 7fa2eec48700 1 -- start start 2024-05-08T23:26:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.873+0000 7fa2eec48700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa2e806bd60 0x7fa2e80cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.873+0000 7fa2eec48700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80cdc90 0x7fa2e80b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.873+0000 7fa2eec48700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e80b1200 con 0x7fa2e80d13a0 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.873+0000 7fa2eec48700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e80cfcc0 con 0x7fa2e806bd60 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.873+0000 7fa2eec48700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80ce5c0 con 0x7fa2e80cdc90 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2e7fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa2e806bd60 0x7fa2e80cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2ed1e5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80cdc90 0x7fa2e80b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2ec9e4700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa2e80d13a0 0x7fa2e80d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:53204/0 (socket says 172.21.15.5:53204) 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2e7fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa2e806bd60 0x7fa2e80cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40606/0 (socket says 172.21.15.5:40606) 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2ec9e4700 1 -- 172.21.15.5:0/899363876 learned_addr learned my addr 172.21.15.5:0/899363876 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2ed1e5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80cdc90 0x7fa2e80b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:36204/0 (socket says 172.21.15.5:36204) 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2e77fe700 1 -- 172.21.15.5:0/899363876 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315203962 0 0) 0x7fa2e80b1200 con 0x7fa2e80d13a0 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2e77fe700 1 -- 172.21.15.5:0/899363876 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d00035b0 con 0x7fa2e80d13a0 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2e7fff700 1 -- 172.21.15.5:0/899363876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80cdc90 msgr2=0x7fa2e80b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2e7fff700 1 --2- 172.21.15.5:0/899363876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80cdc90 0x7fa2e80b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2e7fff700 1 -- 172.21.15.5:0/899363876 >> v1:172.21.15.49:6789/0 conn(0x7fa2e80d13a0 legacy=0x7fa2e80d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2e7fff700 1 -- 172.21.15.5:0/899363876 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80ce8a0 con 0x7fa2e806bd60 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.874+0000 7fa2ed1e5700 1 --2- 172.21.15.5:0/899363876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80cdc90 0x7fa2e80b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2e7fff700 1 --2- 172.21.15.5:0/899363876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e806bd60 0x7fa2e80cf780 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009840 tx=0x7fa2dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e972c595b64c5f4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2e77fe700 1 -- 172.21.15.5:0/899363876 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc011830 con 0x7fa2e806bd60 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2e77fe700 1 -- 172.21.15.5:0/899363876 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc011990 con 0x7fa2e806bd60 2024-05-08T23:26:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2e77fe700 1 -- 172.21.15.5:0/899363876 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc011d20 con 0x7fa2e806bd60 2024-05-08T23:26:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2eec48700 1 -- 172.21.15.5:0/899363876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e806bd60 msgr2=0x7fa2e80cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2eec48700 1 --2- 172.21.15.5:0/899363876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e806bd60 0x7fa2e80cf780 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009840 tx=0x7fa2dc009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2eec48700 1 -- 172.21.15.5:0/899363876 shutdown_connections 2024-05-08T23:26:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2eec48700 1 --2- 172.21.15.5:0/899363876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80cdc90 0x7fa2e80b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2eec48700 1 --2- 172.21.15.5:0/899363876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e806bd60 0x7fa2e80cf780 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2eec48700 1 -- 172.21.15.5:0/899363876 >> 172.21.15.5:0/899363876 conn(0x7fa2e808afe0 msgr2=0x7fa2e808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2eec48700 1 -- 172.21.15.5:0/899363876 shutdown_connections 2024-05-08T23:26:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.875+0000 7fa2eec48700 1 -- 172.21.15.5:0/899363876 wait complete. 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.877+0000 7fa2eec48700 1 Processor -- start 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.877+0000 7fa2eec48700 1 -- start start 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.878+0000 7fa2eec48700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e806bd60 0x7fa2e81f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.878+0000 7fa2eec48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e80cdc90 0x7fa2e81f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.878+0000 7fa2eec48700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80d13a0 0x7fa2e81f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.878+0000 7fa2eec48700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e815f2d0 con 0x7fa2e806bd60 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.878+0000 7fa2eec48700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e815f5b0 con 0x7fa2e80cdc90 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.878+0000 7fa2eec48700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e815f890 con 0x7fa2e80d13a0 2024-05-08T23:26:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e806bd60 0x7fa2e81f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e806bd60 0x7fa2e81f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33702/0 (socket says 172.21.15.5:33702) 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 -- 172.21.15.5:0/3687399940 learned_addr learned my addr 172.21.15.5:0/3687399940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ed1e5700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80d13a0 0x7fa2e81f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2e7fff700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e80cdc90 0x7fa2e81f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 -- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80d13a0 msgr2=0x7fa2e81f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80d13a0 0x7fa2e81f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 -- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e80cdc90 msgr2=0x7fa2e81f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e80cdc90 0x7fa2e81f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 -- 172.21.15.5:0/3687399940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0009a70 con 0x7fa2e806bd60 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2ec9e4700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e806bd60 0x7fa2e81f6bc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c700 tx=0x7fa2d800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2e57fa700 1 -- 172.21.15.5:0/3687399940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8005d70 con 0x7fa2e806bd60 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2e57fa700 1 -- 172.21.15.5:0/3687399940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d80070d0 con 0x7fa2e806bd60 2024-05-08T23:26:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc0094a0 con 0x7fa2e806bd60 2024-05-08T23:26:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2e57fa700 1 -- 172.21.15.5:0/3687399940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8007460 con 0x7fa2e806bd60 2024-05-08T23:26:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.879+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e80c8860 con 0x7fa2e806bd60 2024-05-08T23:26:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.880+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e8074bc0 con 0x7fa2e806bd60 2024-05-08T23:26:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.881+0000 7fa2e57fa700 1 -- 172.21.15.5:0/3687399940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa2d80075c0 con 0x7fa2e806bd60 2024-05-08T23:26:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.881+0000 7fa2e57fa700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2d406ac90 0x7fa2d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.881+0000 7fa2e57fa700 1 -- 172.21.15.5:0/3687399940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa2d8088700 con 0x7fa2e806bd60 2024-05-08T23:26:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.882+0000 7fa2e7fff700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2d406ac90 0x7fa2d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.882+0000 7fa2e7fff700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2d406ac90 0x7fa2d406d150 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc000c00 tx=0x7fa2dc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:49.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:49.886+0000 7fa2e57fa700 1 -- 172.21.15.5:0/3687399940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa2d8053b40 con 0x7fa2e806bd60 2024-05-08T23:26:50.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.105+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa2e80cb460 con 0x7fa2e806bd60 2024-05-08T23:26:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.109+0000 7fa2e57fa700 1 -- 172.21.15.5:0/3687399940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa2d8057160 con 0x7fa2e806bd60 2024-05-08T23:26:50.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2d406ac90 msgr2=0x7fa2d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2d406ac90 0x7fa2d406d150 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc000c00 tx=0x7fa2dc01c040 comp rx=0 tx=0).stop 2024-05-08T23:26:50.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e806bd60 msgr2=0x7fa2e81f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e806bd60 0x7fa2e81f6bc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c700 tx=0x7fa2d800cac0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 shutdown_connections 2024-05-08T23:26:50.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2d406ac90 0x7fa2d406d150 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e80d13a0 0x7fa2e81f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e806bd60 0x7fa2e81f6bc0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 --2- 172.21.15.5:0/3687399940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e80cdc90 0x7fa2e81f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 >> 172.21.15.5:0/3687399940 conn(0x7fa2e808afe0 msgr2=0x7fa2e80d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 shutdown_connections 2024-05-08T23:26:50.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.112+0000 7fa2eec48700 1 -- 172.21.15.5:0/3687399940 wait complete. 2024-05-08T23:26:50.126 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:26:50.126 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:26:50.174 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:26:50.225 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:26:50.226 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:26:50.226 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:50.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:26:50.247 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:26:50.247 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:50.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:26:50.268 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:26:50.268 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:50.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:26:50.310 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:26:50.310 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:26:50.330 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:26:50.330 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T23:26:50.330 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.494+0000 7f4f35095700 1 Processor -- start 2024-05-08T23:26:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.495+0000 7f4f35095700 1 -- start start 2024-05-08T23:26:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.495+0000 7f4f35095700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.495+0000 7f4f35095700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f300d0eb0 0x7f4f3015da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.495+0000 7f4f35095700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f300cd980 con 0x7f4f300cf840 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.495+0000 7f4f35095700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f300ccdb0 con 0x7f4f300d0eb0 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.495+0000 7f4f35095700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f300cd050 con 0x7f4f3006a960 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2f59e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f300d0eb0 0x7f4f3015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2f59e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f300d0eb0 0x7f4f3015da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33708/0 (socket says 172.21.15.5:33708) 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2f59e700 1 -- 172.21.15.5:0/2854100194 learned_addr learned my addr 172.21.15.5:0/2854100194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2e59c700 1 --2- 172.21.15.5:0/2854100194 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2f59e700 1 -- 172.21.15.5:0/2854100194 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 msgr2=0x7f4f3015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2f59e700 1 --2- 172.21.15.5:0/2854100194 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3015d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2f59e700 1 -- 172.21.15.5:0/2854100194 >> v1:172.21.15.49:6789/0 conn(0x7f4f300cf840 legacy=0x7f4f300af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2f59e700 1 -- 172.21.15.5:0/2854100194 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300cd330 con 0x7f4f300d0eb0 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2e59c700 1 --2- 172.21.15.5:0/2854100194 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2f59e700 1 --2- 172.21.15.5:0/2854100194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300d0eb0 0x7f4f3015da30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009e50 tx=0x7f4f2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bc277dee9c533a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2dd9b700 1 -- 172.21.15.5:0/2854100194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800e070 con 0x7f4f300d0eb0 2024-05-08T23:26:50.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2dd9b700 1 -- 172.21.15.5:0/2854100194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f28003a20 con 0x7f4f300d0eb0 2024-05-08T23:26:50.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.496+0000 7f4f2dd9b700 1 -- 172.21.15.5:0/2854100194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28003dd0 con 0x7f4f300d0eb0 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.497+0000 7f4f35095700 1 -- 172.21.15.5:0/2854100194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300d0eb0 msgr2=0x7f4f3015da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.497+0000 7f4f35095700 1 --2- 172.21.15.5:0/2854100194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300d0eb0 0x7f4f3015da30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009e50 tx=0x7f4f2800b330 comp rx=0 tx=0).stop 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.497+0000 7f4f35095700 1 -- 172.21.15.5:0/2854100194 shutdown_connections 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.497+0000 7f4f35095700 1 --2- 172.21.15.5:0/2854100194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300d0eb0 0x7f4f3015da30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.497+0000 7f4f35095700 1 --2- 172.21.15.5:0/2854100194 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.497+0000 7f4f35095700 1 -- 172.21.15.5:0/2854100194 >> 172.21.15.5:0/2854100194 conn(0x7f4f300d0320 msgr2=0x7f4f300d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.497+0000 7f4f35095700 1 -- 172.21.15.5:0/2854100194 shutdown_connections 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.497+0000 7f4f35095700 1 -- 172.21.15.5:0/2854100194 wait complete. 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.499+0000 7f4f35095700 1 Processor -- start 2024-05-08T23:26:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.499+0000 7f4f35095700 1 -- start start 2024-05-08T23:26:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f35095700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3014d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f35095700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300cf840 0x7f4f30159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f35095700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f300d0eb0 0x7f4f300c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f35095700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3014dd20 con 0x7f4f300cf840 2024-05-08T23:26:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f35095700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f301581c0 con 0x7f4f300d0eb0 2024-05-08T23:26:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f35095700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f301584a0 con 0x7f4f3006a960 2024-05-08T23:26:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300cf840 0x7f4f30159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300cf840 0x7f4f30159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33710/0 (socket says 172.21.15.5:33710) 2024-05-08T23:26:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 -- 172.21.15.5:0/1233926251 learned_addr learned my addr 172.21.15.5:0/1233926251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:50.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2f59e700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f300d0eb0 0x7f4f300c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2ed9d700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3014d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 -- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 msgr2=0x7f4f3014d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3014d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 -- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f300d0eb0 msgr2=0x7f4f300c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f300d0eb0 0x7f4f300c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 -- 172.21.15.5:0/1233926251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f28009b30 con 0x7f4f300cf840 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f2e59c700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300cf840 0x7f4f30159260 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c740 tx=0x7f4f2400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.500+0000 7f4f177fe700 1 -- 172.21.15.5:0/1233926251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24004220 con 0x7f4f300cf840 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.501+0000 7f4f177fe700 1 -- 172.21.15.5:0/1233926251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f24004380 con 0x7f4f300cf840 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.501+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f30158780 con 0x7f4f300cf840 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.501+0000 7f4f177fe700 1 -- 172.21.15.5:0/1233926251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f240036c0 con 0x7f4f300cf840 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.501+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f30158d40 con 0x7f4f300cf840 2024-05-08T23:26:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.502+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f30068000 con 0x7f4f300cf840 2024-05-08T23:26:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.502+0000 7f4f177fe700 1 -- 172.21.15.5:0/1233926251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4f24003820 con 0x7f4f300cf840 2024-05-08T23:26:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.503+0000 7f4f177fe700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.503+0000 7f4f177fe700 1 -- 172.21.15.5:0/1233926251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4f240892e0 con 0x7f4f300cf840 2024-05-08T23:26:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.503+0000 7f4f2ed9d700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.503+0000 7f4f2ed9d700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4f20007fd0 tx=0x7f4f20007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.507+0000 7f4f177fe700 1 -- 172.21.15.5:0/1233926251 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4f24054710 con 0x7f4f300cf840 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.773+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f300b62a0 con 0x7f4f300cf840 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.773+0000 7f4f177fe700 1 -- 172.21.15.5:0/1233926251 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f4f24057d30 con 0x7f4f300cf840 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2569383883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2569383883},{"type":"v1","addr":"172.21.15.5:6835","nonce":2569383883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6422,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2160645223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2160645223},{"type":"v1","addr":"172.21.15.49:6837","nonce":2160645223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3883131153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":3883131153},{"type":"v1","addr":"172.21.15.49:6836","nonce":3883131153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7683,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1880144805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1880144805},{"type":"v1","addr":"172.21.15.5:6834","nonce":1880144805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[]} 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 msgr2=0x7f4f1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4f20007fd0 tx=0x7f4f20007ec0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300cf840 msgr2=0x7f4f30159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300cf840 0x7f4f30159260 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c740 tx=0x7f4f2400ca50 comp rx=0 tx=0).stop 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 shutdown_connections 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f300d0eb0 0x7f4f300c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f3006a960 0x7f4f3014d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 --2- 172.21.15.5:0/1233926251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300cf840 0x7f4f30159260 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 >> 172.21.15.5:0/1233926251 conn(0x7f4f300d0320 msgr2=0x7f4f3006f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 shutdown_connections 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:50.776+0000 7f4f35095700 1 -- 172.21.15.5:0/1233926251 wait complete. 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 76 2024-05-08T23:26:50.790 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:26:50.790 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T23:26:50.790 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:26:50.790 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:26:50.791 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:26:50.791 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T23:26:50.791 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:26:50.791 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:26:50.791 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:26:50.791 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T23:26:50.791 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:26:50.791 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:26:50.792 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:26:50.792 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T23:26:50.792 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:26:50.792 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:26:50.806 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:26:50.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:26:50.807 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:26:50.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:26:50.808 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:26:50.809 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:26:50.810 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:26:50.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:26:50.846 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:26:50.846 DEBUG:teuthology.parallel:result is None 2024-05-08T23:26:50.847 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:26:50.847 DEBUG:teuthology.parallel:result is None 2024-05-08T23:26:50.851 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:26:50.851 DEBUG:teuthology.parallel:result is None 2024-05-08T23:26:50.851 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:26:50.852 DEBUG:teuthology.parallel:result is None 2024-05-08T23:26:50.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T23:26:50.938 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:26:50.939 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:26:50.942 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:26:50.943 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:26:51.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.000+0000 7f4447d8b700 1 Processor -- start 2024-05-08T23:26:51.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.001+0000 7f4447d8b700 1 -- start start 2024-05-08T23:26:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.001+0000 7f4447d8b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4440155da0 0x7f444006e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.001+0000 7f4447d8b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444006d4f0 0x7f444015f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.001+0000 7f4447d8b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f444015f660 0x7f444015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.001+0000 7f4447d8b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444006dcb0 con 0x7f444015f660 2024-05-08T23:26:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.001+0000 7f4447d8b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444006e740 con 0x7f4440155da0 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.001+0000 7f4447d8b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444015fae0 con 0x7f444006d4f0 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f444015f660 0x7f444015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f444015f660 0x7f444015bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33758/0 (socket says 172.21.15.5:33758) 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 -- 172.21.15.5:0/2303777997 learned_addr learned my addr 172.21.15.5:0/2303777997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4445326700 1 --2- 172.21.15.5:0/2303777997 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444006d4f0 0x7f444015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4445b27700 1 --2- 172.21.15.5:0/2303777997 >> v2:172.21.15.49:3300/0 conn(0x7f4440155da0 0x7f444006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 -- 172.21.15.5:0/2303777997 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444006d4f0 msgr2=0x7f444015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 --2- 172.21.15.5:0/2303777997 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444006d4f0 0x7f444015f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 -- 172.21.15.5:0/2303777997 >> v2:172.21.15.49:3300/0 conn(0x7f4440155da0 msgr2=0x7f444006e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 --2- 172.21.15.5:0/2303777997 >> v2:172.21.15.49:3300/0 conn(0x7f4440155da0 0x7f444006e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 -- 172.21.15.5:0/2303777997 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444006be40 con 0x7f444015f660 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4446328700 1 --2- 172.21.15.5:0/2303777997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444015f660 0x7f444015bb50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4440145310 tx=0x7f443800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4330a54cd0a73921 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4444b25700 1 -- 172.21.15.5:0/2303777997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443800e070 con 0x7f444015f660 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4444b25700 1 -- 172.21.15.5:0/2303777997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443800b910 con 0x7f444015f660 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.002+0000 7f4444b25700 1 -- 172.21.15.5:0/2303777997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443800bcc0 con 0x7f444015f660 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 -- 172.21.15.5:0/2303777997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444015f660 msgr2=0x7f444015bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 --2- 172.21.15.5:0/2303777997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444015f660 0x7f444015bb50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4440145310 tx=0x7f443800b330 comp rx=0 tx=0).stop 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 -- 172.21.15.5:0/2303777997 shutdown_connections 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 --2- 172.21.15.5:0/2303777997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444015f660 0x7f444015bb50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 --2- 172.21.15.5:0/2303777997 >> v2:172.21.15.49:3300/0 conn(0x7f4440155da0 0x7f444006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 --2- 172.21.15.5:0/2303777997 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444006d4f0 0x7f444015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 -- 172.21.15.5:0/2303777997 >> 172.21.15.5:0/2303777997 conn(0x7f44400cc580 msgr2=0x7f44400cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 -- 172.21.15.5:0/2303777997 shutdown_connections 2024-05-08T23:26:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.003+0000 7f4447d8b700 1 -- 172.21.15.5:0/2303777997 wait complete. 2024-05-08T23:26:51.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.004+0000 7f4447d8b700 1 Processor -- start 2024-05-08T23:26:51.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4447d8b700 1 -- start start 2024-05-08T23:26:51.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4447d8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444006d4f0 0x7f444015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4447d8b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4440155da0 0x7f444013cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4447d8b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444015f660 0x7f44401434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4447d8b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4440157270 con 0x7f444006d4f0 2024-05-08T23:26:51.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4447d8b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44400b3460 con 0x7f4440155da0 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4447d8b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44400b3740 con 0x7f444015f660 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4445b27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444006d4f0 0x7f444015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4445b27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444006d4f0 0x7f444015a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33764/0 (socket says 172.21.15.5:33764) 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.005+0000 7f4445b27700 1 -- 172.21.15.5:0/333678928 learned_addr learned my addr 172.21.15.5:0/333678928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4446328700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444015f660 0x7f44401434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4445326700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4440155da0 0x7f444013cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4445b27700 1 -- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444015f660 msgr2=0x7f44401434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4445b27700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444015f660 0x7f44401434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4445b27700 1 -- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4440155da0 msgr2=0x7f444013cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4445b27700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4440155da0 0x7f444013cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4445b27700 1 -- 172.21.15.5:0/333678928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4438009b30 con 0x7f444006d4f0 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4446328700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444015f660 0x7f44401434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4445326700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4440155da0 0x7f444013cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4445b27700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444006d4f0 0x7f444015a8d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f443c00e9b0 tx=0x7f443c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f44367fc700 1 -- 172.21.15.5:0/333678928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c009710 con 0x7f444006d4f0 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f44367fc700 1 -- 172.21.15.5:0/333678928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c00c850 con 0x7f444006d4f0 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44400b3a20 con 0x7f444006d4f0 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f44367fc700 1 -- 172.21.15.5:0/333678928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c00cbc0 con 0x7f444006d4f0 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.006+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444015ae10 con 0x7f444006d4f0 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.007+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44401573e0 con 0x7f444006d4f0 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.008+0000 7f44367fc700 1 -- 172.21.15.5:0/333678928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f443c00cd20 con 0x7f444006d4f0 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.008+0000 7f44367fc700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f442406ac90 0x7f442406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.008+0000 7f44367fc700 1 -- 172.21.15.5:0/333678928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f443c0891e0 con 0x7f444006d4f0 2024-05-08T23:26:51.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.009+0000 7f4445326700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f442406ac90 0x7f442406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.009+0000 7f4445326700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f442406ac90 0x7f442406d150 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f444015b190 tx=0x7f4430008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.013+0000 7f44367fc700 1 -- 172.21.15.5:0/333678928 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f44401573e0 con 0x7f444006d4f0 2024-05-08T23:26:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.253+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f44401573b0 con 0x7f444006d4f0 2024-05-08T23:26:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.974+0000 7f44367fc700 1 -- 172.21.15.5:0/333678928 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f443c054610 con 0x7f444006d4f0 2024-05-08T23:26:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f442406ac90 msgr2=0x7f442406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f442406ac90 0x7f442406d150 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f444015b190 tx=0x7f4430008040 comp rx=0 tx=0).stop 2024-05-08T23:26:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444006d4f0 msgr2=0x7f444015a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444006d4f0 0x7f444015a8d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f443c00e9b0 tx=0x7f443c00ed70 comp rx=0 tx=0).stop 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 shutdown_connections 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f442406ac90 0x7f442406d150 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f444015f660 0x7f44401434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444006d4f0 0x7f444015a8d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 --2- 172.21.15.5:0/333678928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4440155da0 0x7f444013cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 >> 172.21.15.5:0/333678928 conn(0x7f44400cc580 msgr2=0x7f4440144b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 shutdown_connections 2024-05-08T23:26:51.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:51.977+0000 7f4447d8b700 1 -- 172.21.15.5:0/333678928 wait complete. 2024-05-08T23:26:51.979 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-08T23:26:51.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T23:26:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.134+0000 7f9e4640b700 1 Processor -- start 2024-05-08T23:26:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.134+0000 7f9e4640b700 1 -- start start 2024-05-08T23:26:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e4640b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e4015ef60 0x7f9e400b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e4640b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9e400cfb70 0x7f9e400b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e4640b700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4009ad40 con 0x7f9e400cf000 2024-05-08T23:26:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e4640b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e400cd400 con 0x7f9e4015ef60 2024-05-08T23:26:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e4640b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e400ccb60 con 0x7f9e400cfb70 2024-05-08T23:26:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e3ffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e4015ef60 0x7f9e400b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e3ffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e4015ef60 0x7f9e400b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33778/0 (socket says 172.21.15.5:33778) 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e3ffff700 1 -- 172.21.15.5:0/4256076554 learned_addr learned my addr 172.21.15.5:0/4256076554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e449a8700 1 --2- 172.21.15.5:0/4256076554 >> v2:172.21.15.49:3300/0 conn(0x7f9e400cfb70 0x7f9e400b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e3ffff700 1 -- 172.21.15.5:0/4256076554 >> v2:172.21.15.49:3300/0 conn(0x7f9e400cfb70 msgr2=0x7f9e400b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e3ffff700 1 --2- 172.21.15.5:0/4256076554 >> v2:172.21.15.49:3300/0 conn(0x7f9e400cfb70 0x7f9e400b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e3ffff700 1 -- 172.21.15.5:0/4256076554 >> v1:172.21.15.49:6789/0 conn(0x7f9e400cf000 legacy=0x7f9e400cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.135+0000 7f9e3ffff700 1 -- 172.21.15.5:0/4256076554 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e400cce40 con 0x7f9e4015ef60 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e3ffff700 1 --2- 172.21.15.5:0/4256076554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e4015ef60 0x7f9e400b2340 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f9e40072340 tx=0x7f9e30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff65481da7320ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e3effd700 1 -- 172.21.15.5:0/4256076554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000b870 con 0x7f9e4015ef60 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e3effd700 1 -- 172.21.15.5:0/4256076554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3000b9d0 con 0x7f9e4015ef60 2024-05-08T23:26:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e3effd700 1 -- 172.21.15.5:0/4256076554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000bd60 con 0x7f9e4015ef60 2024-05-08T23:26:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e4640b700 1 -- 172.21.15.5:0/4256076554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e4015ef60 msgr2=0x7f9e400b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e4640b700 1 --2- 172.21.15.5:0/4256076554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e4015ef60 0x7f9e400b2340 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f9e40072340 tx=0x7f9e30009bd0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e4640b700 1 -- 172.21.15.5:0/4256076554 shutdown_connections 2024-05-08T23:26:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e4640b700 1 --2- 172.21.15.5:0/4256076554 >> v2:172.21.15.49:3300/0 conn(0x7f9e400cfb70 0x7f9e400b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e4640b700 1 --2- 172.21.15.5:0/4256076554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e4015ef60 0x7f9e400b2340 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.136+0000 7f9e4640b700 1 -- 172.21.15.5:0/4256076554 >> 172.21.15.5:0/4256076554 conn(0x7f9e400b2f20 msgr2=0x7f9e400b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.137+0000 7f9e4640b700 1 -- 172.21.15.5:0/4256076554 shutdown_connections 2024-05-08T23:26:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.137+0000 7f9e4640b700 1 -- 172.21.15.5:0/4256076554 wait complete. 2024-05-08T23:26:52.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.138+0000 7f9e4640b700 1 Processor -- start 2024-05-08T23:26:52.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.138+0000 7f9e4640b700 1 -- start start 2024-05-08T23:26:52.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e4640b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e400cf000 0x7f9e400cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e4640b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e400cfb70 0x7f9e401f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e4640b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4015ef60 0x7f9e4009ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e4640b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401f16b0 con 0x7f9e400cfb70 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e4640b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4009d7b0 con 0x7f9e4015ef60 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e4640b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401f0d50 con 0x7f9e400cf000 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e400cfb70 0x7f9e401f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e400cfb70 0x7f9e401f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33794/0 (socket says 172.21.15.5:33794) 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 -- 172.21.15.5:0/1824234636 learned_addr learned my addr 172.21.15.5:0/1824234636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3ffff700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e400cf000 0x7f9e400cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e449a8700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4015ef60 0x7f9e4009ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 -- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e400cf000 msgr2=0x7f9e400cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e400cf000 0x7f9e400cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 -- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4015ef60 msgr2=0x7f9e4009ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4015ef60 0x7f9e4009ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 -- 172.21.15.5:0/1824234636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009550 con 0x7f9e400cfb70 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.139+0000 7f9e3f7fe700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e400cfb70 0x7f9e401f1ad0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800c510 tx=0x7f9e2800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.140+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1824234636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280052d0 con 0x7f9e400cfb70 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.140+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1824234636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e28005430 con 0x7f9e400cfb70 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.140+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e401f0ff0 con 0x7f9e400cfb70 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.140+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1824234636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280038c0 con 0x7f9e400cfb70 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.140+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4009f200 con 0x7f9e400cfb70 2024-05-08T23:26:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.141+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e40068000 con 0x7f9e400cfb70 2024-05-08T23:26:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.141+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1824234636 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9e28003a20 con 0x7f9e400cfb70 2024-05-08T23:26:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.142+0000 7f9e3cff9700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e2006ac90 0x7f9e2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.142+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1824234636 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9e280883e0 con 0x7f9e400cfb70 2024-05-08T23:26:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.142+0000 7f9e3ffff700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e2006ac90 0x7f9e2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.142+0000 7f9e3ffff700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e2006ac90 0x7f9e2006d150 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9e30014040 tx=0x7f9e30008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.147+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1824234636 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9e280539a0 con 0x7f9e400cfb70 2024-05-08T23:26:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.457+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9e400cea20 con 0x7f9e400cfb70 2024-05-08T23:26:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.457+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1824234636 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9e28056fc0 con 0x7f9e400cfb70 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm4+KLORAAJGma8zcoGLift7K5wyvWlA==", 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:52.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm5iMNDBAANT+UrtTUBpYyQ9dzJyS01Q==", 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmRixkBhAAk9s24ERPJ/6nCJY3g7QppA==", 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmPmurExAAxIzDzbLZ3s35XVJrvYiVCA==", 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCrCDxmI4/bNBAAzNXcuLQCS9gaAJ0r4iYKlA==", 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCuCDxmMS6HNhAAcTP6nStED06bdM4MtbEPjQ==", 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCxCDxmDGv5NRAAljFQZXLGWwt+UXZ+HCdx0Q==", 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC0CDxmL+5LNRAAVPySIUtICBwIokJL9fHFtw==", 2024-05-08T23:26:52.464 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC5CDxm/F15BhAAEx9mKA9izSmb+xyxjEi9FA==", 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:52.465 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC8CDxm0zAYBhAAjHT+TZFqk5MWvd0P5LmwfA==", 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC/CDxmPcbiBRAAOmWNgaQGi6YGKTp8SJekUg==", 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-08T23:26:52.467 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDCCDxm9FsDBRAA/lfmrpmL4UO3s/tJJ4o6yQ==", 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm9tLAGBAAK8Lq89gJ902XqzXYAbYfkg==", 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:52.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmCStoIBAAhraObJ5LS9LtF97+hIcTFQ==", 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:26:52.469 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm2NzHHBAAUbeD58MntIoiwT8LekjIlw==", 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmyvveIxAATEn7nOPHqtqSbdfQid25HA==", 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm3ZIxDxAAM8M5YUfK9G2PkTzU4wY/+w==", 2024-05-08T23:26:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:26:52.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:26:52.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:26:52.472 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxmntlgNRAA7JOIS+0NL1KmSSZhZjXlVQ==", 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-08T23:26:52.473 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm1dPQMBAAhu0CK6DcucQteuTo7qN2IQ==", 2024-05-08T23:26:52.474 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:26:52.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:26:52.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:26:52.474 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:26:52.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:52.474 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-08T23:26:52.474 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e2006ac90 msgr2=0x7f9e2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e2006ac90 0x7f9e2006d150 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9e30014040 tx=0x7f9e30008030 comp rx=0 tx=0).stop 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e400cfb70 msgr2=0x7f9e401f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e400cfb70 0x7f9e401f1ad0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800c510 tx=0x7f9e2800c8d0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 shutdown_connections 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9e2006ac90 0x7f9e2006d150 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e4015ef60 0x7f9e4009ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e400cf000 0x7f9e400cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 --2- 172.21.15.5:0/1824234636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e400cfb70 0x7f9e401f1ad0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 >> 172.21.15.5:0/1824234636 conn(0x7f9e400b2f20 msgr2=0x7f9e400b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 shutdown_connections 2024-05-08T23:26:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.460+0000 7f9e4640b700 1 -- 172.21.15.5:0/1824234636 wait complete. 2024-05-08T23:26:52.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:52.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.614+0000 7fcbaed9e700 1 Processor -- start 2024-05-08T23:26:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.614+0000 7fcbaed9e700 1 -- start start 2024-05-08T23:26:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.615+0000 7fcbaed9e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba80ccb30 0x7fcba80ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.615+0000 7fcbaed9e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80cad70 con 0x7fcba806e620 2024-05-08T23:26:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.615+0000 7fcbaed9e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80cfdd0 con 0x7fcba80ce160 2024-05-08T23:26:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.615+0000 7fcbaed9e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80cd450 con 0x7fcba80ccb30 2024-05-08T23:26:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.615+0000 7fcbad59b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcba806e620 0x7fcba80cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60464/0 (socket says 172.21.15.5:60464) 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.615+0000 7fcbad59b700 1 -- 172.21.15.5:0/1238138289 learned_addr learned my addr 172.21.15.5:0/1238138289 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbadd9c700 1 --2- 172.21.15.5:0/1238138289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba80ccb30 0x7fcba80ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbacd9a700 1 -- 172.21.15.5:0/1238138289 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871080852 0 0) 0x7fcba80cad70 con 0x7fcba806e620 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbacd9a700 1 -- 172.21.15.5:0/1238138289 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb940035b0 con 0x7fcba806e620 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbacd9a700 1 -- 172.21.15.5:0/1238138289 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481744034 0 0) 0x7fcba80cfdd0 con 0x7fcba80ce160 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbacd9a700 1 -- 172.21.15.5:0/1238138289 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcba80cad70 con 0x7fcba80ce160 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbadd9c700 1 -- 172.21.15.5:0/1238138289 >> v1:172.21.15.49:6789/0 conn(0x7fcba80ce160 legacy=0x7fcba80cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbadd9c700 1 -- 172.21.15.5:0/1238138289 >> v1:172.21.15.5:6789/0 conn(0x7fcba806e620 legacy=0x7fcba80cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbadd9c700 1 -- 172.21.15.5:0/1238138289 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba806ee50 con 0x7fcba80ccb30 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.616+0000 7fcbadd9c700 1 --2- 172.21.15.5:0/1238138289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba80ccb30 0x7fcba80ccf10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009870 tx=0x7fcba4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ccec5a642e9c53e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbacd9a700 1 -- 172.21.15.5:0/1238138289 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4011830 con 0x7fcba80ccb30 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbacd9a700 1 -- 172.21.15.5:0/1238138289 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4011990 con 0x7fcba80ccb30 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbacd9a700 1 -- 172.21.15.5:0/1238138289 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4011d20 con 0x7fcba80ccb30 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbaed9e700 1 -- 172.21.15.5:0/1238138289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba80ccb30 msgr2=0x7fcba80ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbaed9e700 1 --2- 172.21.15.5:0/1238138289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba80ccb30 0x7fcba80ccf10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009870 tx=0x7fcba4009b80 comp rx=0 tx=0).stop 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbaed9e700 1 -- 172.21.15.5:0/1238138289 shutdown_connections 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbaed9e700 1 --2- 172.21.15.5:0/1238138289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba80ccb30 0x7fcba80ccf10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbaed9e700 1 -- 172.21.15.5:0/1238138289 >> 172.21.15.5:0/1238138289 conn(0x7fcba8099a50 msgr2=0x7fcba80b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbaed9e700 1 -- 172.21.15.5:0/1238138289 shutdown_connections 2024-05-08T23:26:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.617+0000 7fcbaed9e700 1 -- 172.21.15.5:0/1238138289 wait complete. 2024-05-08T23:26:52.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.619+0000 7fcbaed9e700 1 Processor -- start 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.619+0000 7fcbaed9e700 1 -- start start 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbaed9e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba806e620 0x7fcba80b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbaed9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80ccb30 0x7fcba80b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbaed9e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcba80ce160 0x7fcba81cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbaed9e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80c4550 con 0x7fcba80ccb30 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbaed9e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80b5bd0 con 0x7fcba80ce160 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbaed9e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba81cc200 con 0x7fcba806e620 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80ccb30 0x7fcba80b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80ccb30 0x7fcba80b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33798/0 (socket says 172.21.15.5:33798) 2024-05-08T23:26:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 -- 172.21.15.5:0/2728681896 learned_addr learned my addr 172.21.15.5:0/2728681896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbadd9c700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba806e620 0x7fcba80b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbae59d700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcba80ce160 0x7fcba81cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 -- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba806e620 msgr2=0x7fcba80b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba806e620 0x7fcba80b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 -- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcba80ce160 msgr2=0x7fcba81cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcba80ce160 0x7fcba81cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 -- 172.21.15.5:0/2728681896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba40094a0 con 0x7fcba80ccb30 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.620+0000 7fcbad59b700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80ccb30 0x7fcba80b24a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005910 tx=0x7fcb9800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.621+0000 7fcb9e7fc700 1 -- 172.21.15.5:0/2728681896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98005d70 con 0x7fcba80ccb30 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.621+0000 7fcb9e7fc700 1 -- 172.21.15.5:0/2728681896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9800ce20 con 0x7fcba80ccb30 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.621+0000 7fcb9e7fc700 1 -- 172.21.15.5:0/2728681896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb980072f0 con 0x7fcba80ccb30 2024-05-08T23:26:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.621+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba81c8670 con 0x7fcba80ccb30 2024-05-08T23:26:52.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.621+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba81c8020 con 0x7fcba80ccb30 2024-05-08T23:26:52.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.622+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba81c7e80 con 0x7fcba80ccb30 2024-05-08T23:26:52.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.622+0000 7fcb9e7fc700 1 -- 172.21.15.5:0/2728681896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcb98007450 con 0x7fcba80ccb30 2024-05-08T23:26:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.623+0000 7fcb9e7fc700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb8c06ac90 0x7fcb8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.623+0000 7fcb9e7fc700 1 -- 172.21.15.5:0/2728681896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcb98088320 con 0x7fcba80ccb30 2024-05-08T23:26:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.623+0000 7fcbadd9c700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb8c06ac90 0x7fcb8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.624+0000 7fcbadd9c700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb8c06ac90 0x7fcb8c06d150 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcba40097c0 tx=0x7fcba401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.628+0000 7fcb9e7fc700 1 -- 172.21.15.5:0/2728681896 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcba81c7e80 con 0x7fcba80ccb30 2024-05-08T23:26:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.846+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcba81c7e80 con 0x7fcba80ccb30 2024-05-08T23:26:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.847+0000 7fcb9e7fc700 1 -- 172.21.15.5:0/2728681896 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+12294 (secure 0 0 0) 0x7fcba81c7e80 con 0x7fcba80ccb30 2024-05-08T23:26:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:52.847 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:51.968634+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb8c06ac90 msgr2=0x7fcb8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb8c06ac90 0x7fcb8c06d150 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcba40097c0 tx=0x7fcba401c040 comp rx=0 tx=0).stop 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80ccb30 msgr2=0x7fcba80b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80ccb30 0x7fcba80b24a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005910 tx=0x7fcb9800c7b0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 shutdown_connections 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb8c06ac90 0x7fcb8c06d150 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcba80ce160 0x7fcba81cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcba806e620 0x7fcba80b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 --2- 172.21.15.5:0/2728681896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80ccb30 0x7fcba80b24a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 >> 172.21.15.5:0/2728681896 conn(0x7fcba8099a50 msgr2=0x7fcba80ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 shutdown_connections 2024-05-08T23:26:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:52.849+0000 7fcbaed9e700 1 -- 172.21.15.5:0/2728681896 wait complete. 2024-05-08T23:26:52.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:26:52.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:26:53.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.003+0000 7f138f279700 1 Processor -- start 2024-05-08T23:26:53.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.004+0000 7f138f279700 1 -- start start 2024-05-08T23:26:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.004+0000 7f138f279700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f138815b4e0 0x7f13880b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.004+0000 7f138f279700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f13880d1910 0x7f13880cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.004+0000 7f138f279700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880cf560 con 0x7f138815b8c0 2024-05-08T23:26:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138f279700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880cefe0 con 0x7f13880d1910 2024-05-08T23:26:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138f279700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880ccff0 con 0x7f138815b4e0 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138d816700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f13880d1910 0x7f13880cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138c814700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f138815b8c0 0x7f13880d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60478/0 (socket says 172.21.15.5:60478) 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138c814700 1 -- 172.21.15.5:0/1478609975 learned_addr learned my addr 172.21.15.5:0/1478609975 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138d015700 1 --2- 172.21.15.5:0/1478609975 >> v2:172.21.15.49:3300/0 conn(0x7f138815b4e0 0x7f13880b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138d816700 1 -- 172.21.15.5:0/1478609975 >> v2:172.21.15.49:3300/0 conn(0x7f138815b4e0 msgr2=0x7f13880b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138d816700 1 --2- 172.21.15.5:0/1478609975 >> v2:172.21.15.49:3300/0 conn(0x7f138815b4e0 0x7f13880b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138d816700 1 -- 172.21.15.5:0/1478609975 >> v1:172.21.15.5:6789/0 conn(0x7f138815b8c0 legacy=0x7f13880d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138d816700 1 -- 172.21.15.5:0/1478609975 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880cd2d0 con 0x7f13880d1910 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f137ffff700 1 -- 172.21.15.5:0/1478609975 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026992024 0 0) 0x7f13880cf560 con 0x7f138815b8c0 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138d015700 1 --2- 172.21.15.5:0/1478609975 >> v2:172.21.15.49:3300/0 conn(0x7f138815b4e0 0x7f13880b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f138d816700 1 --2- 172.21.15.5:0/1478609975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13880d1910 0x7f13880cc2c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f13880b0f90 tx=0x7f138000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d670c6d4b25be506 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f137ffff700 1 -- 172.21.15.5:0/1478609975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000e070 con 0x7f13880d1910 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f137ffff700 1 -- 172.21.15.5:0/1478609975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138000bbd0 con 0x7f13880d1910 2024-05-08T23:26:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.005+0000 7f137ffff700 1 -- 172.21.15.5:0/1478609975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380012430 con 0x7f13880d1910 2024-05-08T23:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.006+0000 7f138f279700 1 -- 172.21.15.5:0/1478609975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13880d1910 msgr2=0x7f13880cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.006+0000 7f138f279700 1 --2- 172.21.15.5:0/1478609975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13880d1910 0x7f13880cc2c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f13880b0f90 tx=0x7f138000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.006+0000 7f138f279700 1 -- 172.21.15.5:0/1478609975 shutdown_connections 2024-05-08T23:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.006+0000 7f138f279700 1 --2- 172.21.15.5:0/1478609975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13880d1910 0x7f13880cc2c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.006+0000 7f138f279700 1 --2- 172.21.15.5:0/1478609975 >> v2:172.21.15.49:3300/0 conn(0x7f138815b4e0 0x7f13880b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.006+0000 7f138f279700 1 -- 172.21.15.5:0/1478609975 >> 172.21.15.5:0/1478609975 conn(0x7f138808afe0 msgr2=0x7f138808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.006+0000 7f138f279700 1 -- 172.21.15.5:0/1478609975 shutdown_connections 2024-05-08T23:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.006+0000 7f138f279700 1 -- 172.21.15.5:0/1478609975 wait complete. 2024-05-08T23:26:53.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.007+0000 7f138f279700 1 Processor -- start 2024-05-08T23:26:53.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138f279700 1 -- start start 2024-05-08T23:26:53.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138f279700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13880d1910 0x7f13880c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138f279700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f138815b4e0 0x7f13880c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138f279700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138815b8c0 0x7f13880c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138f279700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13881f5710 con 0x7f138815b8c0 2024-05-08T23:26:53.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138f279700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1388150aa0 con 0x7f13880d1910 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138f279700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1388150170 con 0x7f138815b4e0 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138d816700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138815b8c0 0x7f13880c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.008+0000 7f138d816700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138815b8c0 0x7f13880c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33808/0 (socket says 172.21.15.5:33808) 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138d816700 1 -- 172.21.15.5:0/4256721718 learned_addr learned my addr 172.21.15.5:0/4256721718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138d015700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13880d1910 0x7f13880c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138d816700 1 -- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f138815b4e0 msgr2=0x7f13880c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138d816700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f138815b4e0 0x7f13880c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138d816700 1 -- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13880d1910 msgr2=0x7f13880c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138d816700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13880d1910 0x7f13880c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138d816700 1 -- 172.21.15.5:0/4256721718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1380009a70 con 0x7f138815b8c0 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138d816700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138815b8c0 0x7f13880c6df0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f138000b730 tx=0x7f1380004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f137dffb700 1 -- 172.21.15.5:0/4256721718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380012430 con 0x7f138815b8c0 2024-05-08T23:26:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f137dffb700 1 -- 172.21.15.5:0/4256721718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380020070 con 0x7f138815b8c0 2024-05-08T23:26:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1388150450 con 0x7f138815b8c0 2024-05-08T23:26:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f137dffb700 1 -- 172.21.15.5:0/4256721718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380004280 con 0x7f138815b8c0 2024-05-08T23:26:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.009+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13880c9330 con 0x7f138815b8c0 2024-05-08T23:26:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.010+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13880c95b0 con 0x7f138815b8c0 2024-05-08T23:26:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.011+0000 7f137dffb700 1 -- 172.21.15.5:0/4256721718 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f13800043e0 con 0x7f138815b8c0 2024-05-08T23:26:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.011+0000 7f137dffb700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f136c06ac90 0x7f136c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.011+0000 7f137dffb700 1 -- 172.21.15.5:0/4256721718 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1380022070 con 0x7f138815b8c0 2024-05-08T23:26:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.012+0000 7f138d015700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f136c06ac90 0x7f136c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.012+0000 7f138d015700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f136c06ac90 0x7f136c06d150 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1384006fd0 tx=0x7f1384008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.016+0000 7f137dffb700 1 -- 172.21.15.5:0/4256721718 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1380056770 con 0x7f138815b8c0 2024-05-08T23:26:53.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.234+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13880cd5b0 con 0x7f138815b8c0 2024-05-08T23:26:53.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.234+0000 7f137dffb700 1 -- 172.21.15.5:0/4256721718 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13880cd5b0 con 0x7f138815b8c0 2024-05-08T23:26:53.235 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:26:53.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f136c06ac90 msgr2=0x7f136c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f136c06ac90 0x7f136c06d150 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1384006fd0 tx=0x7f1384008040 comp rx=0 tx=0).stop 2024-05-08T23:26:53.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138815b8c0 msgr2=0x7f13880c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138815b8c0 0x7f13880c6df0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f138000b730 tx=0x7f1380004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 shutdown_connections 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f136c06ac90 0x7f136c06d150 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138815b8c0 0x7f13880c6df0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13880d1910 0x7f13880c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 --2- 172.21.15.5:0/4256721718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f138815b4e0 0x7f13880c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 >> 172.21.15.5:0/4256721718 conn(0x7f138808afe0 msgr2=0x7f13880cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 shutdown_connections 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.237+0000 7f138f279700 1 -- 172.21.15.5:0/4256721718 wait complete. 2024-05-08T23:26:53.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:26:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.387+0000 7fc8e89f7700 1 Processor -- start 2024-05-08T23:26:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.387+0000 7fc8e89f7700 1 -- start start 2024-05-08T23:26:53.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e89f7700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8e415da10 0x7fc8e40d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e89f7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406c3d0 0x7fc8e406c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e89f7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40cf990 con 0x7fc8e406bd80 2024-05-08T23:26:53.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e89f7700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e409ab40 con 0x7fc8e415da10 2024-05-08T23:26:53.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e89f7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e406cda0 con 0x7fc8e406c3d0 2024-05-08T23:26:53.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e2d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406c3d0 0x7fc8e406c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e259c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8e415da10 0x7fc8e40d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e2d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406c3d0 0x7fc8e406c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:36316/0 (socket says 172.21.15.5:36316) 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e259c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8e415da10 0x7fc8e40d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40742/0 (socket says 172.21.15.5:40742) 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e2d9d700 1 -- 172.21.15.5:0/2236087115 learned_addr learned my addr 172.21.15.5:0/2236087115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.388+0000 7fc8e1d9b700 1 --1- 172.21.15.5:0/2236087115 >> v1:172.21.15.49:6789/0 conn(0x7fc8e406bd80 0x7fc8e409a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:53238/0 (socket says 172.21.15.5:53238) 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e2d9d700 1 -- 172.21.15.5:0/2236087115 >> v2:172.21.15.49:3300/0 conn(0x7fc8e415da10 msgr2=0x7fc8e40d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e2d9d700 1 --2- 172.21.15.5:0/2236087115 >> v2:172.21.15.49:3300/0 conn(0x7fc8e415da10 0x7fc8e40d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e2d9d700 1 -- 172.21.15.5:0/2236087115 >> v1:172.21.15.49:6789/0 conn(0x7fc8e406bd80 legacy=0x7fc8e409a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e2d9d700 1 -- 172.21.15.5:0/2236087115 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40d04b0 con 0x7fc8e406c3d0 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e259c700 1 --2- 172.21.15.5:0/2236087115 >> v2:172.21.15.49:3300/0 conn(0x7fc8e415da10 0x7fc8e40d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e2d9d700 1 --2- 172.21.15.5:0/2236087115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406c3d0 0x7fc8e406c860 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009e50 tx=0x7fc8dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73e837d055b192e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e159a700 1 -- 172.21.15.5:0/2236087115 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00e070 con 0x7fc8e406c3d0 2024-05-08T23:26:53.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e159a700 1 -- 172.21.15.5:0/2236087115 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc00b910 con 0x7fc8e406c3d0 2024-05-08T23:26:53.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.389+0000 7fc8e159a700 1 -- 172.21.15.5:0/2236087115 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00bcc0 con 0x7fc8e406c3d0 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.390+0000 7fc8e89f7700 1 -- 172.21.15.5:0/2236087115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406c3d0 msgr2=0x7fc8e406c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.390+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/2236087115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406c3d0 0x7fc8e406c860 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009e50 tx=0x7fc8dc00b330 comp rx=0 tx=0).stop 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.390+0000 7fc8e89f7700 1 -- 172.21.15.5:0/2236087115 shutdown_connections 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.390+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/2236087115 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406c3d0 0x7fc8e406c860 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.390+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/2236087115 >> v2:172.21.15.49:3300/0 conn(0x7fc8e415da10 0x7fc8e40d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.390+0000 7fc8e89f7700 1 -- 172.21.15.5:0/2236087115 >> 172.21.15.5:0/2236087115 conn(0x7fc8e40b3010 msgr2=0x7fc8e40b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.390+0000 7fc8e89f7700 1 -- 172.21.15.5:0/2236087115 shutdown_connections 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.390+0000 7fc8e89f7700 1 -- 172.21.15.5:0/2236087115 wait complete. 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.391+0000 7fc8e89f7700 1 Processor -- start 2024-05-08T23:26:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.391+0000 7fc8e89f7700 1 -- start start 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e89f7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406bd80 0x7fc8e4159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e89f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8e406c3d0 0x7fc8e40c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e89f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e415da10 0x7fc8e40c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e89f7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40c59d0 con 0x7fc8e415da10 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e89f7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40c61e0 con 0x7fc8e406c3d0 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e89f7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40c5c20 con 0x7fc8e406bd80 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e415da10 0x7fc8e40c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e415da10 0x7fc8e40c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33816/0 (socket says 172.21.15.5:33816) 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.392+0000 7fc8e2d9d700 1 -- 172.21.15.5:0/3595056802 learned_addr learned my addr 172.21.15.5:0/3595056802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e259c700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406bd80 0x7fc8e4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e1d9b700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8e406c3d0 0x7fc8e40c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e2d9d700 1 -- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406bd80 msgr2=0x7fc8e4159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e2d9d700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406bd80 0x7fc8e4159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e2d9d700 1 -- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8e406c3d0 msgr2=0x7fc8e40c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e2d9d700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8e406c3d0 0x7fc8e40c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e2d9d700 1 -- 172.21.15.5:0/3595056802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40094a0 con 0x7fc8e415da10 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e2d9d700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e415da10 0x7fc8e40c7640 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc004270 tx=0x7fc8dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8d2ffd700 1 -- 172.21.15.5:0/3595056802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0048e0 con 0x7fc8e415da10 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8d2ffd700 1 -- 172.21.15.5:0/3595056802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0099b0 con 0x7fc8e415da10 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc009b30 con 0x7fc8e415da10 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8d2ffd700 1 -- 172.21.15.5:0/3595056802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc01c680 con 0x7fc8e415da10 2024-05-08T23:26:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.393+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e40b42e0 con 0x7fc8e415da10 2024-05-08T23:26:53.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.394+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e4068000 con 0x7fc8e415da10 2024-05-08T23:26:53.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.395+0000 7fc8d2ffd700 1 -- 172.21.15.5:0/3595056802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc8dc01c7e0 con 0x7fc8e415da10 2024-05-08T23:26:53.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.395+0000 7fc8d2ffd700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc8c406ac90 0x7fc8c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.395+0000 7fc8d2ffd700 1 -- 172.21.15.5:0/3595056802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc8dc08ac00 con 0x7fc8e415da10 2024-05-08T23:26:53.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.395+0000 7fc8e259c700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc8c406ac90 0x7fc8c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.396+0000 7fc8e259c700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc8c406ac90 0x7fc8c406d150 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009d70 tx=0x7fc8d4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.400+0000 7fc8d2ffd700 1 -- 172.21.15.5:0/3595056802 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc8dc091080 con 0x7fc8e415da10 2024-05-08T23:26:53.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.617+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc8e40b3c70 con 0x7fc8e415da10 2024-05-08T23:26:53.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.617+0000 7fc8d2ffd700 1 -- 172.21.15.5:0/3595056802 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc8dc056140 con 0x7fc8e415da10 2024-05-08T23:26:53.618 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:53.618 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:26:53.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:26:53.618 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-08T23:26:53.618 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-08T23:26:53.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:26:53.618 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:26:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc8c406ac90 msgr2=0x7fc8c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc8c406ac90 0x7fc8c406d150 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009d70 tx=0x7fc8d4009210 comp rx=0 tx=0).stop 2024-05-08T23:26:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e415da10 msgr2=0x7fc8e40c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e415da10 0x7fc8e40c7640 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc004270 tx=0x7fc8dc004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 shutdown_connections 2024-05-08T23:26:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc8c406ac90 0x7fc8c406d150 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e415da10 0x7fc8e40c7640 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8e406bd80 0x7fc8e4159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 --2- 172.21.15.5:0/3595056802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8e406c3d0 0x7fc8e40c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 >> 172.21.15.5:0/3595056802 conn(0x7fc8e40b3010 msgr2=0x7fc8e40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 shutdown_connections 2024-05-08T23:26:53.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.620+0000 7fc8e89f7700 1 -- 172.21.15.5:0/3595056802 wait complete. 2024-05-08T23:26:53.634 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:26:53.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:26:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.774+0000 7f44200cf700 1 Processor -- start 2024-05-08T23:26:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.775+0000 7f44200cf700 1 -- start start 2024-05-08T23:26:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.775+0000 7f44200cf700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f44180d3d20 0x7f44180b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.775+0000 7f44200cf700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180b3fc0 0x7f44180b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.775+0000 7f44200cf700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44180cf610 con 0x7f44180cff30 2024-05-08T23:26:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.775+0000 7f44200cf700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44180d1fc0 con 0x7f44180d3d20 2024-05-08T23:26:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.775+0000 7f44200cf700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44180d22a0 con 0x7f44180b3fc0 2024-05-08T23:26:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441d66a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f44180d3d20 0x7f44180b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441d66a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f44180d3d20 0x7f44180b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33818/0 (socket says 172.21.15.5:33818) 2024-05-08T23:26:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441d66a700 1 -- 172.21.15.5:0/2489087879 learned_addr learned my addr 172.21.15.5:0/2489087879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441e66c700 1 --2- 172.21.15.5:0/2489087879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180b3fc0 0x7f44180b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441d66a700 1 -- 172.21.15.5:0/2489087879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180b3fc0 msgr2=0x7f44180b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441d66a700 1 --2- 172.21.15.5:0/2489087879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180b3fc0 0x7f44180b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441d66a700 1 -- 172.21.15.5:0/2489087879 >> v1:172.21.15.49:6789/0 conn(0x7f44180cff30 legacy=0x7f44180d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441d66a700 1 -- 172.21.15.5:0/2489087879 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180d2580 con 0x7f44180d3d20 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441e66c700 1 --2- 172.21.15.5:0/2489087879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180b3fc0 0x7f44180b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441d66a700 1 --2- 172.21.15.5:0/2489087879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180d3d20 0x7f44180b3a80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f44080098f0 tx=0x7f4408009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6aeb25393c2fecd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441ce69700 1 -- 172.21.15.5:0/2489087879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440800b8a0 con 0x7f44180d3d20 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441ce69700 1 -- 172.21.15.5:0/2489087879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440800ba00 con 0x7f44180d3d20 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.776+0000 7f441ce69700 1 -- 172.21.15.5:0/2489087879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440800bd90 con 0x7f44180d3d20 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.777+0000 7f44200cf700 1 -- 172.21.15.5:0/2489087879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180d3d20 msgr2=0x7f44180b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.777+0000 7f44200cf700 1 --2- 172.21.15.5:0/2489087879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180d3d20 0x7f44180b3a80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f44080098f0 tx=0x7f4408009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.777+0000 7f44200cf700 1 -- 172.21.15.5:0/2489087879 shutdown_connections 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.777+0000 7f44200cf700 1 --2- 172.21.15.5:0/2489087879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180b3fc0 0x7f44180b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.777+0000 7f44200cf700 1 --2- 172.21.15.5:0/2489087879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180d3d20 0x7f44180b3a80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.777+0000 7f44200cf700 1 -- 172.21.15.5:0/2489087879 >> 172.21.15.5:0/2489087879 conn(0x7f441808afe0 msgr2=0x7f44180d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.777+0000 7f44200cf700 1 -- 172.21.15.5:0/2489087879 shutdown_connections 2024-05-08T23:26:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.777+0000 7f44200cf700 1 -- 172.21.15.5:0/2489087879 wait complete. 2024-05-08T23:26:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.779+0000 7f44200cf700 1 Processor -- start 2024-05-08T23:26:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.779+0000 7f44200cf700 1 -- start start 2024-05-08T23:26:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f44200cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44180b3fc0 0x7f44180cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f44200cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180cff30 0x7f441815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f44200cf700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180d3d20 0x7f44180c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f44200cf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44180b80b0 con 0x7f44180cff30 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f44200cf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44180b8390 con 0x7f44180b3fc0 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f44200cf700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44180b8670 con 0x7f44180d3d20 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441d66a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180cff30 0x7f441815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441d66a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180cff30 0x7f441815db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33834/0 (socket says 172.21.15.5:33834) 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441d66a700 1 -- 172.21.15.5:0/2587873212 learned_addr learned my addr 172.21.15.5:0/2587873212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441de6b700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44180b3fc0 0x7f44180cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441d66a700 1 -- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180d3d20 msgr2=0x7f44180c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441e66c700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180d3d20 0x7f44180c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441d66a700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180d3d20 0x7f44180c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441d66a700 1 -- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44180b3fc0 msgr2=0x7f44180cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441d66a700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44180b3fc0 0x7f44180cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441d66a700 1 -- 172.21.15.5:0/2587873212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408009550 con 0x7f44180cff30 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.780+0000 7f441de6b700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44180b3fc0 0x7f44180cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.781+0000 7f441d66a700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180cff30 0x7f441815db30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f4408000c00 tx=0x7f4408004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.781+0000 7f440e7fc700 1 -- 172.21.15.5:0/2587873212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408004480 con 0x7f44180cff30 2024-05-08T23:26:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.781+0000 7f440e7fc700 1 -- 172.21.15.5:0/2587873212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440801f070 con 0x7f44180cff30 2024-05-08T23:26:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.781+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44180b8950 con 0x7f44180cff30 2024-05-08T23:26:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.781+0000 7f440e7fc700 1 -- 172.21.15.5:0/2587873212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44080038e0 con 0x7f44180cff30 2024-05-08T23:26:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.781+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44180cdb30 con 0x7f44180cff30 2024-05-08T23:26:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.782+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4418074bc0 con 0x7f44180cff30 2024-05-08T23:26:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.782+0000 7f440e7fc700 1 -- 172.21.15.5:0/2587873212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4408021030 con 0x7f44180cff30 2024-05-08T23:26:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.783+0000 7f440e7fc700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f43fc06acc0 0x7f43fc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.783+0000 7f440e7fc700 1 -- 172.21.15.5:0/2587873212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f440808a310 con 0x7f44180cff30 2024-05-08T23:26:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.783+0000 7f441de6b700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f43fc06acc0 0x7f43fc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.784+0000 7f441de6b700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f43fc06acc0 0x7f43fc06d180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4414000f60 tx=0x7f4414009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:53.788+0000 7f440e7fc700 1 -- 172.21.15.5:0/2587873212 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f44080558d0 con 0x7f44180cff30 2024-05-08T23:26:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.004+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f44180d4520 con 0x7f44180cff30 2024-05-08T23:26:54.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.727+0000 7f440e7fc700 1 -- 172.21.15.5:0/2587873212 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4408058ef0 con 0x7f44180cff30 2024-05-08T23:26:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f43fc06acc0 msgr2=0x7f43fc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f43fc06acc0 0x7f43fc06d180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4414000f60 tx=0x7f4414009040 comp rx=0 tx=0).stop 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180cff30 msgr2=0x7f441815db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180cff30 0x7f441815db30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f4408000c00 tx=0x7f4408004750 comp rx=0 tx=0).stop 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 shutdown_connections 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f43fc06acc0 0x7f43fc06d180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44180d3d20 0x7f44180c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44180b3fc0 0x7f44180cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 --2- 172.21.15.5:0/2587873212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44180cff30 0x7f441815db30 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 >> 172.21.15.5:0/2587873212 conn(0x7f441808afe0 msgr2=0x7f44180b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 shutdown_connections 2024-05-08T23:26:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.733+0000 7f44200cf700 1 -- 172.21.15.5:0/2587873212 wait complete. 2024-05-08T23:26:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-08T23:26:54.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:26:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.898+0000 7f7c2af90700 1 Processor -- start 2024-05-08T23:26:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.898+0000 7f7c2af90700 1 -- start start 2024-05-08T23:26:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c2af90700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7c240cbf50 0x7f7c24161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c2af90700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c2415e760 0x7f7c2406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c2af90700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c240cf970 con 0x7f7c24162240 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c2af90700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c240cfc50 con 0x7f7c240cbf50 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c2af90700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240cff30 con 0x7f7c2415e760 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c28d2c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7c240cbf50 0x7f7c24161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c23fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c24162240 0x7f7c2415e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60480/0 (socket says 172.21.15.5:60480) 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c23fff700 1 -- 172.21.15.5:0/898931052 learned_addr learned my addr 172.21.15.5:0/898931052 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c28d2c700 1 --2- 172.21.15.5:0/898931052 >> v2:172.21.15.5:3300/0 conn(0x7f7c240cbf50 0x7f7c24161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33836/0 (socket says 172.21.15.5:33836) 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c2952d700 1 --2- 172.21.15.5:0/898931052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c2415e760 0x7f7c2406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c28d2c700 1 -- 172.21.15.5:0/898931052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c2415e760 msgr2=0x7f7c2406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c28d2c700 1 --2- 172.21.15.5:0/898931052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c2415e760 0x7f7c2406a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c28d2c700 1 -- 172.21.15.5:0/898931052 >> v1:172.21.15.5:6789/0 conn(0x7f7c24162240 legacy=0x7f7c2415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c28d2c700 1 -- 172.21.15.5:0/898931052 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c241617e0 con 0x7f7c240cbf50 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c237fe700 1 -- 172.21.15.5:0/898931052 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861784881 0 0) 0x7f7c240cf970 con 0x7f7c24162240 2024-05-08T23:26:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.899+0000 7f7c28d2c700 1 --2- 172.21.15.5:0/898931052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240cbf50 0x7f7c24161d00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009840 tx=0x7f7c14009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9667a2f8825cab27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c237fe700 1 -- 172.21.15.5:0/898931052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14011830 con 0x7f7c240cbf50 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c237fe700 1 -- 172.21.15.5:0/898931052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c14011990 con 0x7f7c240cbf50 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c237fe700 1 -- 172.21.15.5:0/898931052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14011d20 con 0x7f7c240cbf50 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c2af90700 1 -- 172.21.15.5:0/898931052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240cbf50 msgr2=0x7f7c24161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c2af90700 1 --2- 172.21.15.5:0/898931052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240cbf50 0x7f7c24161d00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009840 tx=0x7f7c14009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c2af90700 1 -- 172.21.15.5:0/898931052 shutdown_connections 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c2af90700 1 --2- 172.21.15.5:0/898931052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c2415e760 0x7f7c2406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c2af90700 1 --2- 172.21.15.5:0/898931052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240cbf50 0x7f7c24161d00 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c2af90700 1 -- 172.21.15.5:0/898931052 >> 172.21.15.5:0/898931052 conn(0x7f7c24099a50 msgr2=0x7f7c240d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c2af90700 1 -- 172.21.15.5:0/898931052 shutdown_connections 2024-05-08T23:26:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.900+0000 7f7c2af90700 1 -- 172.21.15.5:0/898931052 wait complete. 2024-05-08T23:26:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.901+0000 7f7c2af90700 1 Processor -- start 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.902+0000 7f7c2af90700 1 -- start start 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.902+0000 7f7c2af90700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c240cbf50 0x7f7c2415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.902+0000 7f7c2af90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c24162240 0x7f7c24158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2af90700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240b2020 0x7f7c2409ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2af90700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2415c340 con 0x7f7c240b2020 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2af90700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240b54e0 con 0x7f7c24162240 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2af90700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240b57c0 con 0x7f7c240cbf50 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240b2020 0x7f7c2409ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240b2020 0x7f7c2409ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33838/0 (socket says 172.21.15.5:33838) 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 -- 172.21.15.5:0/2584476015 learned_addr learned my addr 172.21.15.5:0/2584476015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c28d2c700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c240cbf50 0x7f7c2415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c23fff700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c24162240 0x7f7c24158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 -- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c240cbf50 msgr2=0x7f7c2415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c240cbf50 0x7f7c2415bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 -- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c24162240 msgr2=0x7f7c24158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c24162240 0x7f7c24158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 -- 172.21.15.5:0/2584476015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c140094a0 con 0x7f7c240b2020 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2952d700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240b2020 0x7f7c2409ae00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00ef90 tx=0x7f7c1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c217fa700 1 -- 172.21.15.5:0/2584476015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00cc90 con 0x7f7c240b2020 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c217fa700 1 -- 172.21.15.5:0/2584476015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c00f040 con 0x7f7c240b2020 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c217fa700 1 -- 172.21.15.5:0/2584476015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c014480 con 0x7f7c240b2020 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.903+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c240b1820 con 0x7f7c240b2020 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.904+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c24159370 con 0x7f7c240b2020 2024-05-08T23:26:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.904+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c240afb90 con 0x7f7c240b2020 2024-05-08T23:26:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.905+0000 7f7c217fa700 1 -- 172.21.15.5:0/2584476015 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7c1c0145e0 con 0x7f7c240b2020 2024-05-08T23:26:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.906+0000 7f7c217fa700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c1006ac90 0x7f7c1006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.906+0000 7f7c217fa700 1 -- 172.21.15.5:0/2584476015 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7c1c089d10 con 0x7f7c240b2020 2024-05-08T23:26:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.906+0000 7f7c28d2c700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c1006ac90 0x7f7c1006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.906+0000 7f7c28d2c700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c1006ac90 0x7f7c1006d150 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7c140090b0 tx=0x7f7c14008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:54.911+0000 7f7c217fa700 1 -- 172.21.15.5:0/2584476015 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7c1c08f020 con 0x7f7c240b2020 2024-05-08T23:26:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.129+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7c24074bc0 con 0x7f7c240b2020 2024-05-08T23:26:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.733+0000 7f7c217fa700 1 -- 172.21.15.5:0/2584476015 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7c1c018020 con 0x7f7c240b2020 2024-05-08T23:26:55.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c1006ac90 msgr2=0x7f7c1006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c1006ac90 0x7f7c1006d150 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7c140090b0 tx=0x7f7c14008030 comp rx=0 tx=0).stop 2024-05-08T23:26:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240b2020 msgr2=0x7f7c2409ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240b2020 0x7f7c2409ae00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00ef90 tx=0x7f7c1c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 shutdown_connections 2024-05-08T23:26:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c1006ac90 0x7f7c1006d150 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c240b2020 0x7f7c2409ae00 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c240cbf50 0x7f7c2415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 --2- 172.21.15.5:0/2584476015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c24162240 0x7f7c24158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 >> 172.21.15.5:0/2584476015 conn(0x7f7c24099a50 msgr2=0x7f7c240ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 shutdown_connections 2024-05-08T23:26:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.745+0000 7f7c2af90700 1 -- 172.21.15.5:0/2584476015 wait complete. 2024-05-08T23:26:55.748 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-08T23:26:55.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:26:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.909+0000 7f4f09b49700 1 Processor -- start 2024-05-08T23:26:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.909+0000 7f4f09b49700 1 -- start start 2024-05-08T23:26:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.910+0000 7f4f09b49700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0415b930 0x7f4f0409a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.910+0000 7f4f09b49700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4f0409a160 0x7f4f040b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.910+0000 7f4f09b49700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f040cd440 con 0x7f4f0415b550 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.910+0000 7f4f09b49700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f040ccba0 con 0x7f4f0409a160 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.910+0000 7f4f09b49700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040cce80 con 0x7f4f0415b930 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.910+0000 7f4f037fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4f0415b550 0x7f4f04099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60490/0 (socket says 172.21.15.5:60490) 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.910+0000 7f4f037fe700 1 -- 172.21.15.5:0/2783834033 learned_addr learned my addr 172.21.15.5:0/2783834033 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2783834033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0415b930 0x7f4f0409a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f03fff700 1 --2- 172.21.15.5:0/2783834033 >> v2:172.21.15.49:3300/0 conn(0x7f4f0409a160 0x7f4f040b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f027fc700 1 -- 172.21.15.5:0/2783834033 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556259442 0 0) 0x7f4f040cd440 con 0x7f4f0415b550 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f027fc700 1 -- 172.21.15.5:0/2783834033 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eec0035b0 con 0x7f4f0415b550 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f02ffd700 1 -- 172.21.15.5:0/2783834033 >> v2:172.21.15.49:3300/0 conn(0x7f4f0409a160 msgr2=0x7f4f040b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2783834033 >> v2:172.21.15.49:3300/0 conn(0x7f4f0409a160 0x7f4f040b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f02ffd700 1 -- 172.21.15.5:0/2783834033 >> v1:172.21.15.5:6789/0 conn(0x7f4f0415b550 legacy=0x7f4f04099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f02ffd700 1 -- 172.21.15.5:0/2783834033 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f040cd160 con 0x7f4f0415b930 2024-05-08T23:26:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f03fff700 1 --2- 172.21.15.5:0/2783834033 >> v2:172.21.15.49:3300/0 conn(0x7f4f0409a160 0x7f4f040b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.911+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2783834033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0415b930 0x7f4f0409a8b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009840 tx=0x7f4ef8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a2a7227e8b12cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f027fc700 1 -- 172.21.15.5:0/2783834033 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8011830 con 0x7f4f0415b930 2024-05-08T23:26:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f027fc700 1 -- 172.21.15.5:0/2783834033 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef8011990 con 0x7f4f0415b930 2024-05-08T23:26:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f027fc700 1 -- 172.21.15.5:0/2783834033 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8011d00 con 0x7f4f0415b930 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f09b49700 1 -- 172.21.15.5:0/2783834033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0415b930 msgr2=0x7f4f0409a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f09b49700 1 --2- 172.21.15.5:0/2783834033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0415b930 0x7f4f0409a8b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009840 tx=0x7f4ef8009b50 comp rx=0 tx=0).stop 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f09b49700 1 -- 172.21.15.5:0/2783834033 shutdown_connections 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f09b49700 1 --2- 172.21.15.5:0/2783834033 >> v2:172.21.15.49:3300/0 conn(0x7f4f0409a160 0x7f4f040b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f09b49700 1 --2- 172.21.15.5:0/2783834033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0415b930 0x7f4f0409a8b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f09b49700 1 -- 172.21.15.5:0/2783834033 >> 172.21.15.5:0/2783834033 conn(0x7f4f040b2ec0 msgr2=0x7f4f040b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f09b49700 1 -- 172.21.15.5:0/2783834033 shutdown_connections 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.912+0000 7f4f09b49700 1 -- 172.21.15.5:0/2783834033 wait complete. 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.913+0000 7f4f09b49700 1 Processor -- start 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.914+0000 7f4f09b49700 1 -- start start 2024-05-08T23:26:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.914+0000 7f4f09b49700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0409a160 0x7f4f041f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.914+0000 7f4f09b49700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f0415b550 0x7f4f040c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.914+0000 7f4f09b49700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f0415b930 0x7f4f040c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.914+0000 7f4f09b49700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040cab50 con 0x7f4f0415b550 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.914+0000 7f4f09b49700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040c9ed0 con 0x7f4f0415b930 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.914+0000 7f4f09b49700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040ca1b0 con 0x7f4f0409a160 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f0415b550 0x7f4f040c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f0415b550 0x7f4f040c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33854/0 (socket says 172.21.15.5:33854) 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 -- 172.21.15.5:0/1239282352 learned_addr learned my addr 172.21.15.5:0/1239282352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f037fe700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0409a160 0x7f4f041f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f03fff700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f0415b930 0x7f4f040c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 -- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0409a160 msgr2=0x7f4f041f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0409a160 0x7f4f041f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 -- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f0415b930 msgr2=0x7f4f040c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f0415b930 0x7f4f040c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 -- 172.21.15.5:0/1239282352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc009a70 con 0x7f4f0415b550 2024-05-08T23:26:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f0415b550 0x7f4f040c94f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009fd0 tx=0x7f4ef8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4eebfff700 1 -- 172.21.15.5:0/1239282352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80036a0 con 0x7f4f0415b550 2024-05-08T23:26:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4eebfff700 1 -- 172.21.15.5:0/1239282352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef801f070 con 0x7f4f0415b550 2024-05-08T23:26:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef80094a0 con 0x7f4f0415b550 2024-05-08T23:26:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4eebfff700 1 -- 172.21.15.5:0/1239282352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8003a20 con 0x7f4f0415b550 2024-05-08T23:26:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.915+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f040ca770 con 0x7f4f0415b550 2024-05-08T23:26:55.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.917+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f040cac90 con 0x7f4f0415b550 2024-05-08T23:26:55.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.917+0000 7f4eebfff700 1 -- 172.21.15.5:0/1239282352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4ef8003b80 con 0x7f4f0415b550 2024-05-08T23:26:55.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.918+0000 7f4eebfff700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ef006ac90 0x7f4ef006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:55.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.918+0000 7f4eebfff700 1 -- 172.21.15.5:0/1239282352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4ef8021070 con 0x7f4f0415b550 2024-05-08T23:26:55.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.918+0000 7f4f037fe700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ef006ac90 0x7f4ef006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:55.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.918+0000 7f4f037fe700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ef006ac90 0x7f4ef006d150 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4005910 tx=0x7f4ef4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:55.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:55.923+0000 7f4eebfff700 1 -- 172.21.15.5:0/1239282352 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4f040cac90 con 0x7f4f0415b550 2024-05-08T23:26:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.190+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4f040cac90 con 0x7f4f0415b550 2024-05-08T23:26:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.198+0000 7f4eebfff700 1 -- 172.21.15.5:0/1239282352 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4f040cac90 con 0x7f4f0415b550 2024-05-08T23:26:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.204+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ef006ac90 msgr2=0x7f4ef006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.204+0000 7f4f09b49700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ef006ac90 0x7f4ef006d150 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4005910 tx=0x7f4ef4009040 comp rx=0 tx=0).stop 2024-05-08T23:26:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.204+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f0415b550 msgr2=0x7f4f040c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.204+0000 7f4f09b49700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f0415b550 0x7f4f040c94f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009fd0 tx=0x7f4ef8004750 comp rx=0 tx=0).stop 2024-05-08T23:26:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.205+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 shutdown_connections 2024-05-08T23:26:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.205+0000 7f4f09b49700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ef006ac90 0x7f4ef006d150 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.205+0000 7f4f09b49700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f0415b930 0x7f4f040c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.205+0000 7f4f09b49700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f0409a160 0x7f4f041f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.205+0000 7f4f09b49700 1 --2- 172.21.15.5:0/1239282352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f0415b550 0x7f4f040c94f0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.205+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 >> 172.21.15.5:0/1239282352 conn(0x7f4f040b2ec0 msgr2=0x7f4f040af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.208+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 shutdown_connections 2024-05-08T23:26:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.208+0000 7f4f09b49700 1 -- 172.21.15.5:0/1239282352 wait complete. 2024-05-08T23:26:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 12 and data pool 13 2024-05-08T23:26:56.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.370+0000 7f6f70423700 1 Processor -- start 2024-05-08T23:26:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.370+0000 7f6f70423700 1 -- start start 2024-05-08T23:26:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.371+0000 7f6f70423700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6f680b2480 0x7f6f6809a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.371+0000 7f6f70423700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6f680cbbc0 0x7f6f680cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.371+0000 7f6f70423700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f680d0840 con 0x7f6f68161030 2024-05-08T23:26:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.371+0000 7f6f70423700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6806ccc0 con 0x7f6f680cbbc0 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.371+0000 7f6f70423700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f680d0660 con 0x7f6f680b2480 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6e9c0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6f680cbbc0 0x7f6f680cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6e1bf700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6f68161030 0x7f6f68099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60506/0 (socket says 172.21.15.5:60506) 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6e1bf700 1 -- 172.21.15.5:0/1508361367 learned_addr learned my addr 172.21.15.5:0/1508361367 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6d9be700 1 --2- 172.21.15.5:0/1508361367 >> v2:172.21.15.49:3300/0 conn(0x7f6f680b2480 0x7f6f6809a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6e9c0700 1 -- 172.21.15.5:0/1508361367 >> v2:172.21.15.49:3300/0 conn(0x7f6f680b2480 msgr2=0x7f6f6809a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6e9c0700 1 --2- 172.21.15.5:0/1508361367 >> v2:172.21.15.49:3300/0 conn(0x7f6f680b2480 0x7f6f6809a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6e9c0700 1 -- 172.21.15.5:0/1508361367 >> v1:172.21.15.5:6789/0 conn(0x7f6f68161030 legacy=0x7f6f68099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6e9c0700 1 -- 172.21.15.5:0/1508361367 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6806c600 con 0x7f6f680cbbc0 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6e9c0700 1 --2- 172.21.15.5:0/1508361367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f680cbbc0 0x7f6f680cc010 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6f680b2430 tx=0x7f6f6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa0084fe885b5fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6d1bd700 1 -- 172.21.15.5:0/1508361367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000e070 con 0x7f6f680cbbc0 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6d1bd700 1 -- 172.21.15.5:0/1508361367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6000bbd0 con 0x7f6f680cbbc0 2024-05-08T23:26:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.372+0000 7f6f6d1bd700 1 -- 172.21.15.5:0/1508361367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60012430 con 0x7f6f680cbbc0 2024-05-08T23:26:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.373+0000 7f6f70423700 1 -- 172.21.15.5:0/1508361367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f680cbbc0 msgr2=0x7f6f680cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.373+0000 7f6f70423700 1 --2- 172.21.15.5:0/1508361367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f680cbbc0 0x7f6f680cc010 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6f680b2430 tx=0x7f6f6000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.373+0000 7f6f70423700 1 -- 172.21.15.5:0/1508361367 shutdown_connections 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.373+0000 7f6f70423700 1 --2- 172.21.15.5:0/1508361367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f680cbbc0 0x7f6f680cc010 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.373+0000 7f6f70423700 1 --2- 172.21.15.5:0/1508361367 >> v2:172.21.15.49:3300/0 conn(0x7f6f680b2480 0x7f6f6809a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.373+0000 7f6f70423700 1 -- 172.21.15.5:0/1508361367 >> 172.21.15.5:0/1508361367 conn(0x7f6f680b2f20 msgr2=0x7f6f680b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.373+0000 7f6f70423700 1 -- 172.21.15.5:0/1508361367 shutdown_connections 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.373+0000 7f6f70423700 1 -- 172.21.15.5:0/1508361367 wait complete. 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.374+0000 7f6f70423700 1 Processor -- start 2024-05-08T23:26:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.374+0000 7f6f70423700 1 -- start start 2024-05-08T23:26:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f70423700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f680b2480 0x7f6f6815e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f70423700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f680cbbc0 0x7f6f681f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f70423700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f68161030 0x7f6f6814c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f70423700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681f7540 con 0x7f6f68161030 2024-05-08T23:26:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f70423700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681f7820 con 0x7f6f680b2480 2024-05-08T23:26:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f70423700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c6180 con 0x7f6f680cbbc0 2024-05-08T23:26:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f6e9c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f68161030 0x7f6f6814c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f6e9c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f68161030 0x7f6f6814c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33872/0 (socket says 172.21.15.5:33872) 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f6e9c0700 1 -- 172.21.15.5:0/830898394 learned_addr learned my addr 172.21.15.5:0/830898394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f6d9be700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f680cbbc0 0x7f6f681f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.375+0000 7f6f6e1bf700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f680b2480 0x7f6f6815e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f6e9c0700 1 -- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f680cbbc0 msgr2=0x7f6f681f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f6e9c0700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f680cbbc0 0x7f6f681f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f6e9c0700 1 -- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f680b2480 msgr2=0x7f6f6815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f6e9c0700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f680b2480 0x7f6f6815e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f6e9c0700 1 -- 172.21.15.5:0/830898394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f60009a70 con 0x7f6f68161030 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f6e9c0700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f68161030 0x7f6f6814c450 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6f60014040 tx=0x7f6f60004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f5effd700 1 -- 172.21.15.5:0/830898394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60012430 con 0x7f6f68161030 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f5effd700 1 -- 172.21.15.5:0/830898394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f60020070 con 0x7f6f68161030 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f5effd700 1 -- 172.21.15.5:0/830898394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60004280 con 0x7f6f68161030 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f680c6460 con 0x7f6f68161030 2024-05-08T23:26:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.376+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6815ee30 con 0x7f6f68161030 2024-05-08T23:26:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.377+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f680c5fc0 con 0x7f6f68161030 2024-05-08T23:26:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.378+0000 7f6f5effd700 1 -- 172.21.15.5:0/830898394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6f600043e0 con 0x7f6f68161030 2024-05-08T23:26:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.378+0000 7f6f5effd700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f4c06ac90 0x7f6f4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.378+0000 7f6f5effd700 1 -- 172.21.15.5:0/830898394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f60022070 con 0x7f6f68161030 2024-05-08T23:26:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.379+0000 7f6f6e1bf700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f4c06ac90 0x7f6f4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.379+0000 7f6f6e1bf700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f4c06ac90 0x7f6f4c06d150 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6f64005910 tx=0x7f6f64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.383+0000 7f6f5effd700 1 -- 172.21.15.5:0/830898394 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6f600562e0 con 0x7f6f68161030 2024-05-08T23:26:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.603+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f68068000 con 0x7f6f68161030 2024-05-08T23:26:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.604+0000 7f6f5effd700 1 -- 172.21.15.5:0/830898394 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16009 (secure 0 0 0) 0x7f6f60059900 con 0x7f6f68161030 2024-05-08T23:26:56.604 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:56.604 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:56.191619+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:54.037450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:55.162938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f4c06ac90 msgr2=0x7f6f4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f4c06ac90 0x7f6f4c06d150 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6f64005910 tx=0x7f6f64009040 comp rx=0 tx=0).stop 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f68161030 msgr2=0x7f6f6814c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f68161030 0x7f6f6814c450 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6f60014040 tx=0x7f6f60004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 shutdown_connections 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6f4c06ac90 0x7f6f4c06d150 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f68161030 0x7f6f6814c450 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f680b2480 0x7f6f6815e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 --2- 172.21.15.5:0/830898394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6f680cbbc0 0x7f6f681f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 >> 172.21.15.5:0/830898394 conn(0x7f6f680b2f20 msgr2=0x7f6f6815fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.606+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 shutdown_connections 2024-05-08T23:26:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.607+0000 7f6f70423700 1 -- 172.21.15.5:0/830898394 wait complete. 2024-05-08T23:26:56.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.757+0000 7f90b06eb700 1 Processor -- start 2024-05-08T23:26:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.757+0000 7f90b06eb700 1 -- start start 2024-05-08T23:26:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90b06eb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90a806ae10 0x7f90a815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90b06eb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a80b3170 con 0x7f90a80b5ba0 2024-05-08T23:26:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90b06eb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a80cc5e0 con 0x7f90a806a960 2024-05-08T23:26:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90b06eb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a80cc000 con 0x7f90a806ae10 2024-05-08T23:26:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90ae487700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f90a80b5ba0 0x7f90a80cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60512/0 (socket says 172.21.15.5:60512) 2024-05-08T23:26:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90ae487700 1 -- 172.21.15.5:0/1960302851 learned_addr learned my addr 172.21.15.5:0/1960302851 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90aec88700 1 --2- 172.21.15.5:0/1960302851 >> v2:172.21.15.49:3300/0 conn(0x7f90a806ae10 0x7f90a815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90ad485700 1 -- 172.21.15.5:0/1960302851 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268975509 0 0) 0x7f90a80b3170 con 0x7f90a80b5ba0 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.758+0000 7f90ad485700 1 -- 172.21.15.5:0/1960302851 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90940035b0 con 0x7f90a80b5ba0 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90ad485700 1 -- 172.21.15.5:0/1960302851 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922587368 0 0) 0x7f90a80cc5e0 con 0x7f90a806a960 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90ad485700 1 -- 172.21.15.5:0/1960302851 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a80b3170 con 0x7f90a806a960 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90ad485700 1 -- 172.21.15.5:0/1960302851 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134504638 0 0) 0x7f90940035b0 con 0x7f90a80b5ba0 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90aec88700 1 -- 172.21.15.5:0/1960302851 >> v1:172.21.15.49:6789/0 conn(0x7f90a806a960 legacy=0x7f90a815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90aec88700 1 -- 172.21.15.5:0/1960302851 >> v1:172.21.15.5:6789/0 conn(0x7f90a80b5ba0 legacy=0x7f90a80cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90aec88700 1 -- 172.21.15.5:0/1960302851 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a806b260 con 0x7f90a806ae10 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90aec88700 1 --2- 172.21.15.5:0/1960302851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806ae10 0x7f90a815d6b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009d90 tx=0x7f90a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd469cb518b1e2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90ad485700 1 -- 172.21.15.5:0/1960302851 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000e070 con 0x7f90a806ae10 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90ad485700 1 -- 172.21.15.5:0/1960302851 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0003a20 con 0x7f90a806ae10 2024-05-08T23:26:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.759+0000 7f90ad485700 1 -- 172.21.15.5:0/1960302851 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4001670 con 0x7f90a806ae10 2024-05-08T23:26:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.760+0000 7f90b06eb700 1 -- 172.21.15.5:0/1960302851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806ae10 msgr2=0x7f90a815d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.760+0000 7f90b06eb700 1 --2- 172.21.15.5:0/1960302851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806ae10 0x7f90a815d6b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009d90 tx=0x7f90a000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.760+0000 7f90b06eb700 1 -- 172.21.15.5:0/1960302851 shutdown_connections 2024-05-08T23:26:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.760+0000 7f90b06eb700 1 --2- 172.21.15.5:0/1960302851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806ae10 0x7f90a815d6b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.760+0000 7f90b06eb700 1 -- 172.21.15.5:0/1960302851 >> 172.21.15.5:0/1960302851 conn(0x7f90a8099a50 msgr2=0x7f90a80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.760+0000 7f90b06eb700 1 -- 172.21.15.5:0/1960302851 shutdown_connections 2024-05-08T23:26:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.760+0000 7f90b06eb700 1 -- 172.21.15.5:0/1960302851 wait complete. 2024-05-08T23:26:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.762+0000 7f90b06eb700 1 Processor -- start 2024-05-08T23:26:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.762+0000 7f90b06eb700 1 -- start start 2024-05-08T23:26:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90b06eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806a960 0x7f90a81f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90b06eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90a806ae10 0x7f90a815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90b06eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a80b5ba0 0x7f90a815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90b06eb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81f3600 con 0x7f90a80b5ba0 2024-05-08T23:26:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90b06eb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81f41d0 con 0x7f90a806a960 2024-05-08T23:26:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90b06eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c5ab0 con 0x7f90a806ae10 2024-05-08T23:26:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a80b5ba0 0x7f90a815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a80b5ba0 0x7f90a815cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33878/0 (socket says 172.21.15.5:33878) 2024-05-08T23:26:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 -- 172.21.15.5:0/1613238471 learned_addr learned my addr 172.21.15.5:0/1613238471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90ae487700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806a960 0x7f90a81f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90adc86700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90a806ae10 0x7f90a815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 -- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90a806ae10 msgr2=0x7f90a815c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90a806ae10 0x7f90a815c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 -- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806a960 msgr2=0x7f90a81f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806a960 0x7f90a81f3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 -- 172.21.15.5:0/1613238471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0009a70 con 0x7f90a80b5ba0 2024-05-08T23:26:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.763+0000 7f90aec88700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a80b5ba0 0x7f90a815cb40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f90a000b370 tx=0x7f90a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.764+0000 7f909affd700 1 -- 172.21.15.5:0/1613238471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0004440 con 0x7f90a80b5ba0 2024-05-08T23:26:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.764+0000 7f909affd700 1 -- 172.21.15.5:0/1613238471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a00045a0 con 0x7f90a80b5ba0 2024-05-08T23:26:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.764+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a80c5d90 con 0x7f90a80b5ba0 2024-05-08T23:26:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.764+0000 7f909affd700 1 -- 172.21.15.5:0/1613238471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0004950 con 0x7f90a80b5ba0 2024-05-08T23:26:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.764+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a80c6350 con 0x7f90a80b5ba0 2024-05-08T23:26:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.765+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a8068000 con 0x7f90a80b5ba0 2024-05-08T23:26:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.765+0000 7f909affd700 1 -- 172.21.15.5:0/1613238471 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f90a001d470 con 0x7f90a80b5ba0 2024-05-08T23:26:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.766+0000 7f909affd700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f908c06ac90 0x7f908c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.766+0000 7f909affd700 1 -- 172.21.15.5:0/1613238471 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90a008af50 con 0x7f90a80b5ba0 2024-05-08T23:26:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.766+0000 7f90ae487700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f908c06ac90 0x7f908c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.766+0000 7f90ae487700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f908c06ac90 0x7f908c06d150 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f90a4000fd0 tx=0x7f90a4000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.770+0000 7f909affd700 1 -- 172.21.15.5:0/1613238471 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f90a0059bd0 con 0x7f90a80b5ba0 2024-05-08T23:26:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.985+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90a8161c60 con 0x7f90a80b5ba0 2024-05-08T23:26:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.986+0000 7f909affd700 1 -- 172.21.15.5:0/1613238471 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16009 (secure 0 0 0) 0x7f90a0059760 con 0x7f90a80b5ba0 2024-05-08T23:26:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:56.987 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:56.191619+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:54.037450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:55.162938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:26:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f908c06ac90 msgr2=0x7f908c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f908c06ac90 0x7f908c06d150 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f90a4000fd0 tx=0x7f90a4000ed0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a80b5ba0 msgr2=0x7f90a815cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a80b5ba0 0x7f90a815cb40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f90a000b370 tx=0x7f90a0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 shutdown_connections 2024-05-08T23:26:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f908c06ac90 0x7f908c06d150 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a80b5ba0 0x7f90a815cb40 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90a806a960 0x7f90a81f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 --2- 172.21.15.5:0/1613238471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f90a806ae10 0x7f90a815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 >> 172.21.15.5:0/1613238471 conn(0x7f90a8099a50 msgr2=0x7f90a806e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 shutdown_connections 2024-05-08T23:26:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:56.989+0000 7f90b06eb700 1 -- 172.21.15.5:0/1613238471 wait complete. 2024-05-08T23:26:57.003 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:26:57.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.143+0000 7fc97a8ca700 1 Processor -- start 2024-05-08T23:26:57.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.143+0000 7fc97a8ca700 1 -- start start 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc97a8ca700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc97415a7c0 0x7fc97415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc97a8ca700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc9740af1d0 0x7fc9740af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc97a8ca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc9740b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc97a8ca700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9740aecf0 con 0x7fc97415a7c0 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc97a8ca700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9740afb70 con 0x7fc9740af1d0 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc97a8ca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740b46d0 con 0x7fc9740b3d00 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc973fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc97415a7c0 0x7fc97415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc973fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc97415a7c0 0x7fc97415aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33894/0 (socket says 172.21.15.5:33894) 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc973fff700 1 -- 172.21.15.5:0/361771280 learned_addr learned my addr 172.21.15.5:0/361771280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc9737fe700 1 --2- 172.21.15.5:0/361771280 >> v2:172.21.15.49:3300/0 conn(0x7fc9740af1d0 0x7fc9740af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.144+0000 7fc978e67700 1 --2- 172.21.15.5:0/361771280 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc9740b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc973fff700 1 -- 172.21.15.5:0/361771280 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 msgr2=0x7fc9740b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc973fff700 1 --2- 172.21.15.5:0/361771280 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc9740b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc973fff700 1 -- 172.21.15.5:0/361771280 >> v2:172.21.15.49:3300/0 conn(0x7fc9740af1d0 msgr2=0x7fc9740af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc973fff700 1 --2- 172.21.15.5:0/361771280 >> v2:172.21.15.49:3300/0 conn(0x7fc9740af1d0 0x7fc9740af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc973fff700 1 -- 172.21.15.5:0/361771280 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740b4950 con 0x7fc97415a7c0 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc978e67700 1 --2- 172.21.15.5:0/361771280 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc9740b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc9737fe700 1 --2- 172.21.15.5:0/361771280 >> v2:172.21.15.49:3300/0 conn(0x7fc9740af1d0 0x7fc9740af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc973fff700 1 --2- 172.21.15.5:0/361771280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415aba0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc9740718f0 tx=0x7fc964009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cef73ca45792be7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc972ffd700 1 -- 172.21.15.5:0/361771280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400b860 con 0x7fc97415a7c0 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc972ffd700 1 -- 172.21.15.5:0/361771280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96400b9c0 con 0x7fc97415a7c0 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.145+0000 7fc972ffd700 1 -- 172.21.15.5:0/361771280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400bd50 con 0x7fc97415a7c0 2024-05-08T23:26:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 -- 172.21.15.5:0/361771280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 msgr2=0x7fc97415aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/361771280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415aba0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc9740718f0 tx=0x7fc964009c00 comp rx=0 tx=0).stop 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 -- 172.21.15.5:0/361771280 shutdown_connections 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/361771280 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc9740b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/361771280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415aba0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/361771280 >> v2:172.21.15.49:3300/0 conn(0x7fc9740af1d0 0x7fc9740af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 -- 172.21.15.5:0/361771280 >> 172.21.15.5:0/361771280 conn(0x7fc9740b5810 msgr2=0x7fc9740b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 -- 172.21.15.5:0/361771280 shutdown_connections 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.146+0000 7fc97a8ca700 1 -- 172.21.15.5:0/361771280 wait complete. 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.147+0000 7fc97a8ca700 1 Processor -- start 2024-05-08T23:26:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.147+0000 7fc97a8ca700 1 -- start start 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc97a8ca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9740af1d0 0x7fc9741439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc97a8ca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc97415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc97a8ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc97a8ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc974145d50 con 0x7fc97415a7c0 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc97a8ca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740b21d0 con 0x7fc9740af1d0 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc97a8ca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc974146590 con 0x7fc9740b3d00 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc978e67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc978e67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33906/0 (socket says 172.21.15.5:33906) 2024-05-08T23:26:57.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc978e67700 1 -- 172.21.15.5:0/1733626977 learned_addr learned my addr 172.21.15.5:0/1733626977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc9737fe700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc97415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc973fff700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9740af1d0 0x7fc9741439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc978e67700 1 -- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 msgr2=0x7fc97415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc978e67700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc97415cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc978e67700 1 -- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9740af1d0 msgr2=0x7fc9741439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc978e67700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9740af1d0 0x7fc9741439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.148+0000 7fc978e67700 1 -- 172.21.15.5:0/1733626977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964009550 con 0x7fc97415a7c0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.149+0000 7fc978e67700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415d1a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007ef0 tx=0x7fc96c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.149+0000 7fc970ff9700 1 -- 172.21.15.5:0/1733626977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00de60 con 0x7fc97415a7c0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.149+0000 7fc970ff9700 1 -- 172.21.15.5:0/1733626977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00f040 con 0x7fc97415a7c0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.149+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc974143020 con 0x7fc97415a7c0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.149+0000 7fc970ff9700 1 -- 172.21.15.5:0/1733626977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c003c60 con 0x7fc97415a7c0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.149+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc974145f40 con 0x7fc97415a7c0 2024-05-08T23:26:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.150+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc974074bc0 con 0x7fc97415a7c0 2024-05-08T23:26:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.150+0000 7fc970ff9700 1 -- 172.21.15.5:0/1733626977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc96c004030 con 0x7fc97415a7c0 2024-05-08T23:26:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.151+0000 7fc970ff9700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc96006ac90 0x7fc96006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.151+0000 7fc970ff9700 1 -- 172.21.15.5:0/1733626977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc96c089540 con 0x7fc97415a7c0 2024-05-08T23:26:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.151+0000 7fc973fff700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc96006ac90 0x7fc96006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.152+0000 7fc973fff700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc96006ac90 0x7fc96006d150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc964014040 tx=0x7fc964008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:57.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.156+0000 7fc970ff9700 1 -- 172.21.15.5:0/1733626977 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc96c054710 con 0x7fc97415a7c0 2024-05-08T23:26:57.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:57.426+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc974152080 con 0x7fc97415a7c0 2024-05-08T23:26:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.208+0000 7fc970ff9700 1 -- 172.21.15.5:0/1733626977 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc96c057d30 con 0x7fc97415a7c0 2024-05-08T23:26:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc96006ac90 msgr2=0x7fc96006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc96006ac90 0x7fc96006d150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc964014040 tx=0x7fc964008030 comp rx=0 tx=0).stop 2024-05-08T23:26:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 msgr2=0x7fc97415d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415d1a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007ef0 tx=0x7fc96c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 shutdown_connections 2024-05-08T23:26:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc96006ac90 0x7fc96006d150 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc97415a7c0 0x7fc97415d1a0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9740af1d0 0x7fc9741439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 --2- 172.21.15.5:0/1733626977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc9740b3d00 0x7fc97415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 >> 172.21.15.5:0/1733626977 conn(0x7fc9740b5810 msgr2=0x7fc974159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 shutdown_connections 2024-05-08T23:26:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.211+0000 7fc97a8ca700 1 -- 172.21.15.5:0/1733626977 wait complete. 2024-05-08T23:26:58.226 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:26:58.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:26:58.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.370+0000 7f5d56430700 1 Processor -- start 2024-05-08T23:26:58.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.370+0000 7f5d56430700 1 -- start start 2024-05-08T23:26:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d56430700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d500cbf50 0x7f5d50161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d56430700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d50162240 0x7f5d5015e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d56430700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d500d0f10 0x7f5d5015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d56430700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d500cf970 con 0x7f5d500cbf50 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d56430700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d500cfc50 con 0x7f5d500d0f10 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d56430700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500cff30 con 0x7f5d50162240 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4ffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d500cbf50 0x7f5d50161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4ffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d500cbf50 0x7f5d50161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37064/0 (socket says 172.21.15.5:37064) 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4ffff700 1 -- 172.21.15.5:0/1983766314 learned_addr learned my addr 172.21.15.5:0/1983766314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4f7fe700 1 --2- 172.21.15.5:0/1983766314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d50162240 0x7f5d5015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d549cd700 1 --2- 172.21.15.5:0/1983766314 >> v2:172.21.15.49:3300/0 conn(0x7f5d500d0f10 0x7f5d5015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4ffff700 1 -- 172.21.15.5:0/1983766314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d50162240 msgr2=0x7f5d5015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4ffff700 1 --2- 172.21.15.5:0/1983766314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d50162240 0x7f5d5015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4ffff700 1 -- 172.21.15.5:0/1983766314 >> v2:172.21.15.49:3300/0 conn(0x7f5d500d0f10 msgr2=0x7f5d5015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4ffff700 1 --2- 172.21.15.5:0/1983766314 >> v2:172.21.15.49:3300/0 conn(0x7f5d500d0f10 0x7f5d5015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d4ffff700 1 -- 172.21.15.5:0/1983766314 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d501617e0 con 0x7f5d500cbf50 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.371+0000 7f5d549cd700 1 --2- 172.21.15.5:0/1983766314 >> v2:172.21.15.49:3300/0 conn(0x7f5d500d0f10 0x7f5d5015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d4ffff700 1 --2- 172.21.15.5:0/1983766314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d500cbf50 0x7f5d50161d00 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5d50072c50 tx=0x7f5d40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867e5b2d8930f927 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d4effd700 1 -- 172.21.15.5:0/1983766314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40011830 con 0x7f5d500cbf50 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d4effd700 1 -- 172.21.15.5:0/1983766314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40011990 con 0x7f5d500cbf50 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d4effd700 1 -- 172.21.15.5:0/1983766314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40011d20 con 0x7f5d500cbf50 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d56430700 1 -- 172.21.15.5:0/1983766314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d500cbf50 msgr2=0x7f5d50161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d56430700 1 --2- 172.21.15.5:0/1983766314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d500cbf50 0x7f5d50161d00 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5d50072c50 tx=0x7f5d40009bd0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d56430700 1 -- 172.21.15.5:0/1983766314 shutdown_connections 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d56430700 1 --2- 172.21.15.5:0/1983766314 >> v2:172.21.15.49:3300/0 conn(0x7f5d500d0f10 0x7f5d5015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d56430700 1 --2- 172.21.15.5:0/1983766314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d500cbf50 0x7f5d50161d00 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d56430700 1 --2- 172.21.15.5:0/1983766314 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d50162240 0x7f5d5015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d56430700 1 -- 172.21.15.5:0/1983766314 >> 172.21.15.5:0/1983766314 conn(0x7f5d50099a50 msgr2=0x7f5d500d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.372+0000 7f5d56430700 1 -- 172.21.15.5:0/1983766314 shutdown_connections 2024-05-08T23:26:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.373+0000 7f5d56430700 1 -- 172.21.15.5:0/1983766314 wait complete. 2024-05-08T23:26:58.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.374+0000 7f5d56430700 1 Processor -- start 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.375+0000 7f5d56430700 1 -- start start 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.375+0000 7f5d56430700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d500cbf50 0x7f5d500b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.375+0000 7f5d56430700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d500d0f10 0x7f5d5014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.375+0000 7f5d56430700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d50162240 0x7f5d500b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.375+0000 7f5d56430700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500b2c90 con 0x7f5d50162240 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.375+0000 7f5d56430700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500b0d00 con 0x7f5d500cbf50 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.375+0000 7f5d56430700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5014c630 con 0x7f5d500d0f10 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d50162240 0x7f5d500b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d50162240 0x7f5d500b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37068/0 (socket says 172.21.15.5:37068) 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 -- 172.21.15.5:0/3254605158 learned_addr learned my addr 172.21.15.5:0/3254605158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d4f7fe700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d500d0f10 0x7f5d5014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d4ffff700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d500cbf50 0x7f5d500b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 -- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d500d0f10 msgr2=0x7f5d5014c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d500d0f10 0x7f5d5014c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 -- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d500cbf50 msgr2=0x7f5d500b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d500cbf50 0x7f5d500b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 -- 172.21.15.5:0/3254605158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d40009550 con 0x7f5d50162240 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d549cd700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d50162240 0x7f5d500b50c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009fa0 tx=0x7f5d4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d4cff9700 1 -- 172.21.15.5:0/3254605158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4800ddb0 con 0x7f5d50162240 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d4cff9700 1 -- 172.21.15.5:0/3254605158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4800f040 con 0x7f5d50162240 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5006e030 con 0x7f5d50162240 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d4cff9700 1 -- 172.21.15.5:0/3254605158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48003c60 con 0x7f5d50162240 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.376+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d500b1c70 con 0x7f5d50162240 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.377+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d50074bc0 con 0x7f5d50162240 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.378+0000 7f5d4cff9700 1 -- 172.21.15.5:0/3254605158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5d48004030 con 0x7f5d50162240 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.378+0000 7f5d4cff9700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5d3006ac90 0x7f5d3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.379+0000 7f5d4cff9700 1 -- 172.21.15.5:0/3254605158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d480893b0 con 0x7f5d50162240 2024-05-08T23:26:58.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.379+0000 7f5d4ffff700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5d3006ac90 0x7f5d3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:58.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.379+0000 7f5d4ffff700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5d3006ac90 0x7f5d3006d150 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d40014040 tx=0x7f5d40008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:58.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.383+0000 7f5d4cff9700 1 -- 172.21.15.5:0/3254605158 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5d48054590 con 0x7f5d50162240 2024-05-08T23:26:58.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:58.653+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5d500b62b0 con 0x7f5d50162240 2024-05-08T23:26:59.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.210+0000 7f5d4cff9700 1 -- 172.21.15.5:0/3254605158 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5d48057bb0 con 0x7f5d50162240 2024-05-08T23:26:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5d3006ac90 msgr2=0x7f5d3006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5d3006ac90 0x7f5d3006d150 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d40014040 tx=0x7f5d40008030 comp rx=0 tx=0).stop 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d50162240 msgr2=0x7f5d500b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d50162240 0x7f5d500b50c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009fa0 tx=0x7f5d4800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 shutdown_connections 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5d3006ac90 0x7f5d3006d150 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d50162240 0x7f5d500b50c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d500cbf50 0x7f5d500b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 --2- 172.21.15.5:0/3254605158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5d500d0f10 0x7f5d5014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 >> 172.21.15.5:0/3254605158 conn(0x7f5d50099a50 msgr2=0x7f5d500ad0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 shutdown_connections 2024-05-08T23:26:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.213+0000 7f5d56430700 1 -- 172.21.15.5:0/3254605158 wait complete. 2024-05-08T23:26:59.229 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:26:59.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:59.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.369+0000 7fe3dec1c700 1 Processor -- start 2024-05-08T23:26:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.369+0000 7fe3dec1c700 1 -- start start 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.370+0000 7fe3dec1c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3d80cd4e0 0x7fe3d8099f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.370+0000 7fe3dec1c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d809aca0 con 0x7fe3d815b810 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.370+0000 7fe3dec1c700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d80cd9e0 con 0x7fe3d8099a50 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.370+0000 7fe3dec1c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d809a440 con 0x7fe3d80cd4e0 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.370+0000 7fe3dc9b8700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe3d815b810 0x7fe3d80cffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47102/0 (socket says 172.21.15.5:47102) 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.370+0000 7fe3dc9b8700 1 -- 172.21.15.5:0/800120180 learned_addr learned my addr 172.21.15.5:0/800120180 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.370+0000 7fe3dd1b9700 1 --2- 172.21.15.5:0/800120180 >> v2:172.21.15.5:3300/0 conn(0x7fe3d80cd4e0 0x7fe3d8099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3d77fe700 1 -- 172.21.15.5:0/800120180 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580857571 0 0) 0x7fe3d809aca0 con 0x7fe3d815b810 2024-05-08T23:26:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3d77fe700 1 -- 172.21.15.5:0/800120180 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3c00035b0 con 0x7fe3d815b810 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3dd1b9700 1 -- 172.21.15.5:0/800120180 >> v1:172.21.15.49:6789/0 conn(0x7fe3d8099a50 legacy=0x7fe3d80ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3dd1b9700 1 -- 172.21.15.5:0/800120180 >> v1:172.21.15.5:6789/0 conn(0x7fe3d815b810 legacy=0x7fe3d80cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3dd1b9700 1 -- 172.21.15.5:0/800120180 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80cf220 con 0x7fe3d80cd4e0 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3dd1b9700 1 --2- 172.21.15.5:0/800120180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d80cd4e0 0x7fe3d8099f00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009d90 tx=0x7fe3d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d4b9fcc2acfdbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3d77fe700 1 -- 172.21.15.5:0/800120180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000e070 con 0x7fe3d80cd4e0 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3d77fe700 1 -- 172.21.15.5:0/800120180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d000bb20 con 0x7fe3d80cd4e0 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.371+0000 7fe3d77fe700 1 -- 172.21.15.5:0/800120180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000be90 con 0x7fe3d80cd4e0 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.372+0000 7fe3dec1c700 1 -- 172.21.15.5:0/800120180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d80cd4e0 msgr2=0x7fe3d8099f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.372+0000 7fe3dec1c700 1 --2- 172.21.15.5:0/800120180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d80cd4e0 0x7fe3d8099f00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009d90 tx=0x7fe3d000b330 comp rx=0 tx=0).stop 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.372+0000 7fe3dec1c700 1 -- 172.21.15.5:0/800120180 shutdown_connections 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.372+0000 7fe3dec1c700 1 --2- 172.21.15.5:0/800120180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d80cd4e0 0x7fe3d8099f00 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.372+0000 7fe3dec1c700 1 -- 172.21.15.5:0/800120180 >> 172.21.15.5:0/800120180 conn(0x7fe3d80b3010 msgr2=0x7fe3d80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.372+0000 7fe3dec1c700 1 -- 172.21.15.5:0/800120180 shutdown_connections 2024-05-08T23:26:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.372+0000 7fe3dec1c700 1 -- 172.21.15.5:0/800120180 wait complete. 2024-05-08T23:26:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.374+0000 7fe3dec1c700 1 Processor -- start 2024-05-08T23:26:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.374+0000 7fe3dec1c700 1 -- start start 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dec1c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8099a50 0x7fe3d814ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dec1c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d80cd4e0 0x7fe3d80c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dec1c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3d815b810 0x7fe3d80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dec1c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80c5590 con 0x7fe3d8099a50 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dec1c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80c5860 con 0x7fe3d80cd4e0 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dec1c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80c5b40 con 0x7fe3d815b810 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dc9b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8099a50 0x7fe3d814ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dc9b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8099a50 0x7fe3d814ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37094/0 (socket says 172.21.15.5:37094) 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dc9b8700 1 -- 172.21.15.5:0/1391661018 learned_addr learned my addr 172.21.15.5:0/1391661018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3d7fff700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d80cd4e0 0x7fe3d80c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dd1b9700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3d815b810 0x7fe3d80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dc9b8700 1 -- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3d815b810 msgr2=0x7fe3d80c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dc9b8700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3d815b810 0x7fe3d80c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dc9b8700 1 -- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d80cd4e0 msgr2=0x7fe3d80c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dc9b8700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d80cd4e0 0x7fe3d80c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.375+0000 7fe3dc9b8700 1 -- 172.21.15.5:0/1391661018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d0009a70 con 0x7fe3d8099a50 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.376+0000 7fe3dc9b8700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8099a50 0x7fe3d814ca70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800c700 tx=0x7fe3c800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.376+0000 7fe3d57fa700 1 -- 172.21.15.5:0/1391661018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8005d70 con 0x7fe3d8099a50 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.376+0000 7fe3d57fa700 1 -- 172.21.15.5:0/1391661018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c80070d0 con 0x7fe3d8099a50 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.376+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d80c5e20 con 0x7fe3d8099a50 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.376+0000 7fe3d57fa700 1 -- 172.21.15.5:0/1391661018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8007460 con 0x7fe3d8099a50 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.376+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d80c63e0 con 0x7fe3d8099a50 2024-05-08T23:26:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.377+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d8068000 con 0x7fe3d8099a50 2024-05-08T23:26:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.377+0000 7fe3d57fa700 1 -- 172.21.15.5:0/1391661018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe3c80075c0 con 0x7fe3d8099a50 2024-05-08T23:26:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.378+0000 7fe3d57fa700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3c406ac90 0x7fe3c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.378+0000 7fe3d57fa700 1 -- 172.21.15.5:0/1391661018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3c8088950 con 0x7fe3d8099a50 2024-05-08T23:26:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.378+0000 7fe3d7fff700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3c406ac90 0x7fe3c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.378+0000 7fe3d7fff700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3c406ac90 0x7fe3c406d150 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80c85a0 tx=0x7fe3cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.383+0000 7fe3d57fa700 1 -- 172.21.15.5:0/1391661018 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe3c8053b40 con 0x7fe3d8099a50 2024-05-08T23:26:59.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.650+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3d80b3c70 con 0x7fe3d8099a50 2024-05-08T23:26:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.651+0000 7fe3d57fa700 1 -- 172.21.15.5:0/1391661018 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4730 (secure 0 0 0) 0x7fe3c8057160 con 0x7fe3d8099a50 2024-05-08T23:26:59.651 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:26:59.651 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7336,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7755,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:26:59.207563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:26:59.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3c406ac90 msgr2=0x7fe3c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3c406ac90 0x7fe3c406d150 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80c85a0 tx=0x7fe3cc009040 comp rx=0 tx=0).stop 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8099a50 msgr2=0x7fe3d814ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8099a50 0x7fe3d814ca70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800c700 tx=0x7fe3c800cac0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 shutdown_connections 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe3c406ac90 0x7fe3c406d150 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3d815b810 0x7fe3d80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8099a50 0x7fe3d814ca70 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 --2- 172.21.15.5:0/1391661018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d80cd4e0 0x7fe3d80c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 >> 172.21.15.5:0/1391661018 conn(0x7fe3d80b3010 msgr2=0x7fe3d80cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 shutdown_connections 2024-05-08T23:26:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.653+0000 7fe3dec1c700 1 -- 172.21.15.5:0/1391661018 wait complete. 2024-05-08T23:26:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 82 2024-05-08T23:26:59.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:26:59.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.803+0000 7f3a60d55700 1 Processor -- start 2024-05-08T23:26:59.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.803+0000 7f3a60d55700 1 -- start start 2024-05-08T23:26:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.804+0000 7f3a60d55700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.804+0000 7f3a60d55700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c0d0840 con 0x7f3a5c0b2480 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.804+0000 7f3a60d55700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c06c600 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.804+0000 7f3a60d55700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c06ccc0 con 0x7f3a5c161030 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.804+0000 7f3a59d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3a5c0b2480 0x7f3a5c099f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47106/0 (socket says 172.21.15.5:47106) 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.804+0000 7f3a59d9b700 1 -- 172.21.15.5:0/3286830957 learned_addr learned my addr 172.21.15.5:0/3286830957 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5a59c700 1 --2- 172.21.15.5:0/3286830957 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5959a700 1 -- 172.21.15.5:0/3286830957 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218121898 0 0) 0x7f3a5c0d0840 con 0x7f3a5c0b2480 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5959a700 1 -- 172.21.15.5:0/3286830957 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a440035b0 con 0x7f3a5c0b2480 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5959a700 1 -- 172.21.15.5:0/3286830957 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730121423 0 0) 0x7f3a5c06c600 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5959a700 1 -- 172.21.15.5:0/3286830957 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a5c0d0840 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5959a700 1 -- 172.21.15.5:0/3286830957 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91474615 0 0) 0x7f3a440035b0 con 0x7f3a5c0b2480 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5a59c700 1 -- 172.21.15.5:0/3286830957 >> v1:172.21.15.49:6789/0 conn(0x7f3a5c0cbbc0 legacy=0x7f3a5c0cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5a59c700 1 -- 172.21.15.5:0/3286830957 >> v1:172.21.15.5:6789/0 conn(0x7f3a5c0b2480 legacy=0x7f3a5c099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5a59c700 1 -- 172.21.15.5:0/3286830957 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c06c8e0 con 0x7f3a5c161030 2024-05-08T23:26:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.805+0000 7f3a5a59c700 1 --2- 172.21.15.5:0/3286830957 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c099a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009870 tx=0x7f3a4c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee8ca122eb56666 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a5959a700 1 -- 172.21.15.5:0/3286830957 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50001670 con 0x7f3a5c161030 2024-05-08T23:26:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a5959a700 1 -- 172.21.15.5:0/3286830957 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c004030 con 0x7f3a5c161030 2024-05-08T23:26:59.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a5959a700 1 -- 172.21.15.5:0/3286830957 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c0043c0 con 0x7f3a5c161030 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a60d55700 1 -- 172.21.15.5:0/3286830957 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 msgr2=0x7f3a5c099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a60d55700 1 --2- 172.21.15.5:0/3286830957 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c099a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009870 tx=0x7f3a4c009b80 comp rx=0 tx=0).stop 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a60d55700 1 -- 172.21.15.5:0/3286830957 shutdown_connections 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a60d55700 1 --2- 172.21.15.5:0/3286830957 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c099a50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a60d55700 1 -- 172.21.15.5:0/3286830957 >> 172.21.15.5:0/3286830957 conn(0x7f3a5c0b2f20 msgr2=0x7f3a5c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a60d55700 1 -- 172.21.15.5:0/3286830957 shutdown_connections 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.806+0000 7f3a60d55700 1 -- 172.21.15.5:0/3286830957 wait complete. 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.808+0000 7f3a60d55700 1 Processor -- start 2024-05-08T23:26:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.808+0000 7f3a60d55700 1 -- start start 2024-05-08T23:26:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a60d55700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a5c0b2480 0x7f3a5c14a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a60d55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5c0cbbc0 0x7f3a5c1475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a60d55700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a60d55700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0d0660 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a60d55700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c14a590 con 0x7f3a5c0b2480 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a60d55700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0c9cc0 con 0x7f3a5c161030 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5c0cbbc0 0x7f3a5c1475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5c0cbbc0 0x7f3a5c1475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37102/0 (socket says 172.21.15.5:37102) 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 -- 172.21.15.5:0/4290196534 learned_addr learned my addr 172.21.15.5:0/4290196534 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:26:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a5a59c700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a5c0b2480 0x7f3a5c14a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a5ad9d700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 -- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 msgr2=0x7f3a5c0c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c0c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 -- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a5c0b2480 msgr2=0x7f3a5c14a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a5c0b2480 0x7f3a5c14a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 -- 172.21.15.5:0/4290196534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0094a0 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a59d9b700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5c0cbbc0 0x7f3a5c1475b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000c750 tx=0x7f3a5000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a4affd700 1 -- 172.21.15.5:0/4290196534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50004c80 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a4affd700 1 -- 172.21.15.5:0/4290196534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50004e00 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c0c9fa0 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.809+0000 7f3a4affd700 1 -- 172.21.15.5:0/4290196534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50007310 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.810+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c0ca560 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.810+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c068000 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.811+0000 7f3a4affd700 1 -- 172.21.15.5:0/4290196534 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a50007470 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.811+0000 7f3a4affd700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a3c06ac90 0x7f3a3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.811+0000 7f3a4affd700 1 -- 172.21.15.5:0/4290196534 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a500891b0 con 0x7f3a5c0cbbc0 2024-05-08T23:26:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.812+0000 7f3a5a59c700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a3c06ac90 0x7f3a3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.812+0000 7f3a5a59c700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a3c06ac90 0x7f3a3c06d150 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0097c0 tx=0x7f3a4c013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:26:59.816+0000 7f3a4affd700 1 -- 172.21.15.5:0/4290196534 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3a50054300 con 0x7f3a5c0cbbc0 2024-05-08T23:27:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.035+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a5c06c5f0 con 0x7f3a5c0cbbc0 2024-05-08T23:27:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.036+0000 7f3a4affd700 1 -- 172.21.15.5:0/4290196534 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15996 (secure 0 0 0) 0x7f3a50057920 con 0x7f3a5c0cbbc0 2024-05-08T23:27:00.036 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:00.036 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:57.193438+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:54.037450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:55.162938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a3c06ac90 msgr2=0x7f3a3c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a3c06ac90 0x7f3a3c06d150 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0097c0 tx=0x7f3a4c013040 comp rx=0 tx=0).stop 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5c0cbbc0 msgr2=0x7f3a5c1475b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5c0cbbc0 0x7f3a5c1475b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000c750 tx=0x7f3a5000cb10 comp rx=0 tx=0).stop 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 shutdown_connections 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a3c06ac90 0x7f3a3c06d150 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a5c161030 0x7f3a5c0c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a5c0b2480 0x7f3a5c14a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 --2- 172.21.15.5:0/4290196534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5c0cbbc0 0x7f3a5c1475b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 >> 172.21.15.5:0/4290196534 conn(0x7f3a5c0b2f20 msgr2=0x7f3a5c06ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.038+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 shutdown_connections 2024-05-08T23:27:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.039+0000 7f3a60d55700 1 -- 172.21.15.5:0/4290196534 wait complete. 2024-05-08T23:27:00.053 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:27:00.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:27:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.189+0000 7efe99195700 1 Processor -- start 2024-05-08T23:27:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.189+0000 7efe99195700 1 -- start start 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe99195700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940b29e0 0x7efe940cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe99195700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efe940c6740 0x7efe940c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe99195700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9415ba40 con 0x7efe940d0f50 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe99195700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe940c7110 con 0x7efe940c6740 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe99195700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940ca960 con 0x7efe940b29e0 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe92d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940b29e0 0x7efe940cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe9359e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efe940c6740 0x7efe940c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe92d9d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940b29e0 0x7efe940cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:54960/0 (socket says 172.21.15.5:54960) 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe92d9d700 1 -- 172.21.15.5:0/2282036977 learned_addr learned my addr 172.21.15.5:0/2282036977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe91d9b700 1 -- 172.21.15.5:0/2282036977 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977691855 0 0) 0x7efe9415ba40 con 0x7efe940d0f50 2024-05-08T23:27:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe91d9b700 1 -- 172.21.15.5:0/2282036977 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe7c0035b0 con 0x7efe940d0f50 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe9359e700 1 -- 172.21.15.5:0/2282036977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940b29e0 msgr2=0x7efe940cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe9359e700 1 --2- 172.21.15.5:0/2282036977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940b29e0 0x7efe940cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe9359e700 1 -- 172.21.15.5:0/2282036977 >> v1:172.21.15.49:6789/0 conn(0x7efe940d0f50 legacy=0x7efe9415fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe9359e700 1 -- 172.21.15.5:0/2282036977 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940cabb0 con 0x7efe940c6740 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.190+0000 7efe92d9d700 1 --2- 172.21.15.5:0/2282036977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940b29e0 0x7efe940cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe9359e700 1 --2- 172.21.15.5:0/2282036977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940c6740 0x7efe940c6bd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009d90 tx=0x7efe8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a00598831eef01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe91d9b700 1 -- 172.21.15.5:0/2282036977 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c00e070 con 0x7efe940c6740 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe91d9b700 1 -- 172.21.15.5:0/2282036977 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c00bb20 con 0x7efe940c6740 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe91d9b700 1 -- 172.21.15.5:0/2282036977 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c012430 con 0x7efe940c6740 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe99195700 1 -- 172.21.15.5:0/2282036977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940c6740 msgr2=0x7efe940c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe99195700 1 --2- 172.21.15.5:0/2282036977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940c6740 0x7efe940c6bd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009d90 tx=0x7efe8c00b330 comp rx=0 tx=0).stop 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe99195700 1 -- 172.21.15.5:0/2282036977 shutdown_connections 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe99195700 1 --2- 172.21.15.5:0/2282036977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940c6740 0x7efe940c6bd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe99195700 1 --2- 172.21.15.5:0/2282036977 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940b29e0 0x7efe940cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe99195700 1 -- 172.21.15.5:0/2282036977 >> 172.21.15.5:0/2282036977 conn(0x7efe940d03a0 msgr2=0x7efe940d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:00.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe99195700 1 -- 172.21.15.5:0/2282036977 shutdown_connections 2024-05-08T23:27:00.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.191+0000 7efe99195700 1 -- 172.21.15.5:0/2282036977 wait complete. 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.193+0000 7efe99195700 1 Processor -- start 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.193+0000 7efe99195700 1 -- start start 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe99195700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940b29e0 0x7efe94154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe99195700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940c6740 0x7efe94155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe99195700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe940d0f50 0x7efe941f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe99195700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9406dcb0 con 0x7efe940d0f50 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe99195700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe94157dc0 con 0x7efe940b29e0 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe99195700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe941554d0 con 0x7efe940c6740 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe9359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe940d0f50 0x7efe941f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe9359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe940d0f50 0x7efe941f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37104/0 (socket says 172.21.15.5:37104) 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe9359e700 1 -- 172.21.15.5:0/3113779636 learned_addr learned my addr 172.21.15.5:0/3113779636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe9259c700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940c6740 0x7efe94155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe92d9d700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940b29e0 0x7efe94154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.194+0000 7efe9359e700 1 -- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940c6740 msgr2=0x7efe94155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe9359e700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940c6740 0x7efe94155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe9359e700 1 -- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940b29e0 msgr2=0x7efe94154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe9359e700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940b29e0 0x7efe94154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe9359e700 1 -- 172.21.15.5:0/3113779636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84009550 con 0x7efe940d0f50 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe9359e700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe940d0f50 0x7efe941f4570 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7efe8c014040 tx=0x7efe8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe7b7fe700 1 -- 172.21.15.5:0/3113779636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0129f0 con 0x7efe940d0f50 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe7b7fe700 1 -- 172.21.15.5:0/3113779636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c020070 con 0x7efe940d0f50 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c009a70 con 0x7efe940d0f50 2024-05-08T23:27:00.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe7b7fe700 1 -- 172.21.15.5:0/3113779636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c012db0 con 0x7efe940d0f50 2024-05-08T23:27:00.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.195+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe94158340 con 0x7efe940d0f50 2024-05-08T23:27:00.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.196+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe94068000 con 0x7efe940d0f50 2024-05-08T23:27:00.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.197+0000 7efe7b7fe700 1 -- 172.21.15.5:0/3113779636 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efe8c004040 con 0x7efe940d0f50 2024-05-08T23:27:00.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.197+0000 7efe7b7fe700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efe8006ac90 0x7efe8006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.197+0000 7efe7b7fe700 1 -- 172.21.15.5:0/3113779636 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe8c022070 con 0x7efe940d0f50 2024-05-08T23:27:00.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.197+0000 7efe92d9d700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efe8006ac90 0x7efe8006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.198+0000 7efe92d9d700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efe8006ac90 0x7efe8006d150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe84006d30 tx=0x7efe840092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:00.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.202+0000 7efe7b7fe700 1 -- 172.21.15.5:0/3113779636 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efe8c056250 con 0x7efe940d0f50 2024-05-08T23:27:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.418+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe940cb450 con 0x7efe8006ac90 2024-05-08T23:27:00.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.428+0000 7efe7b7fe700 1 -- 172.21.15.5:0/3113779636 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289754 (secure 0 0 0) 0x7efe840099e0 con 0x7efe8006ac90 2024-05-08T23:27:00.432 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:00.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efe8006ac90 msgr2=0x7efe8006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efe8006ac90 0x7efe8006d150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe84006d30 tx=0x7efe840092c0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe940d0f50 msgr2=0x7efe941f4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe940d0f50 0x7efe941f4570 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7efe8c014040 tx=0x7efe8c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 shutdown_connections 2024-05-08T23:27:00.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efe8006ac90 0x7efe8006d150 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe940d0f50 0x7efe941f4570 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe940b29e0 0x7efe94154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 --2- 172.21.15.5:0/3113779636 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efe940c6740 0x7efe94155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 >> 172.21.15.5:0/3113779636 conn(0x7efe940d03a0 msgr2=0x7efe940cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:00.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 shutdown_connections 2024-05-08T23:27:00.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.431+0000 7efe99195700 1 -- 172.21.15.5:0/3113779636 wait complete. 2024-05-08T23:27:00.435 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:27:00.445 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-05-08T23:26:59.379841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245832,"kb_used_data":9192,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754728888,"statfs":{"total":773094113280,"available":772842381312,"internally_reserved":0,"allocated":9412608,"data_stored":4689708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":242208309},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.652872"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199432+0000","last_change":"2024-05-08T23:26:56.214420+0000","last_active":"2024-05-08T23:26:57.199432+0000","last_peered":"2024-05-08T23:26:57.199432+0000","last_clean":"2024-05-08T23:26:57.199432+0000","last_became_active":"2024-05-08T23:26:56.213393+0000","last_became_peered":"2024-05-08T23:26:56.213393+0000","last_unstale":"2024-05-08T23:26:57.199432+0000","last_undegraded":"2024-05-08T23:26:57.199432+0000","last_fullsized":"2024-05-08T23:26:57.199432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:12.365940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218406+0000","last_change":"2024-05-08T23:26:56.216615+0000","last_active":"2024-05-08T23:26:56.218406+0000","last_peered":"2024-05-08T23:26:56.218406+0000","last_clean":"2024-05-08T23:26:56.218406+0000","last_became_active":"2024-05-08T23:26:56.216204+0000","last_became_peered":"2024-05-08T23:26:56.216204+0000","last_unstale":"2024-05-08T23:26:56.218406+0000","last_undegraded":"2024-05-08T23:26:56.218406+0000","last_fullsized":"2024-05-08T23:26:56.218406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:04.765049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.221044+0000","last_change":"2024-05-08T23:26:56.219331+0000","last_active":"2024-05-08T23:26:56.221044+0000","last_peered":"2024-05-08T23:26:56.221044+0000","last_clean":"2024-05-08T23:26:56.221044+0000","last_became_active":"2024-05-08T23:26:56.219041+0000","last_became_peered":"2024-05-08T23:26:56.219041+0000","last_unstale":"2024-05-08T23:26:56.221044+0000","last_undegraded":"2024-05-08T23:26:56.221044+0000","last_fullsized":"2024-05-08T23:26:56.221044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:08.428124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198116+0000","last_change":"2024-05-08T23:26:56.219524+0000","last_active":"2024-05-08T23:26:57.198116+0000","last_peered":"2024-05-08T23:26:57.198116+0000","last_clean":"2024-05-08T23:26:57.198116+0000","last_became_active":"2024-05-08T23:26:56.218889+0000","last_became_peered":"2024-05-08T23:26:56.218889+0000","last_unstale":"2024-05-08T23:26:57.198116+0000","last_undegraded":"2024-05-08T23:26:57.198116+0000","last_fullsized":"2024-05-08T23:26:57.198116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:45.877888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.221112+0000","last_change":"2024-05-08T23:26:56.219043+0000","last_active":"2024-05-08T23:26:56.221112+0000","last_peered":"2024-05-08T23:26:56.221112+0000","last_clean":"2024-05-08T23:26:56.221112+0000","last_became_active":"2024-05-08T23:26:56.218412+0000","last_became_peered":"2024-05-08T23:26:56.218412+0000","last_unstale":"2024-05-08T23:26:56.221112+0000","last_undegraded":"2024-05-08T23:26:56.221112+0000","last_fullsized":"2024-05-08T23:26:56.221112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:06.646632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216620+0000","last_change":"2024-05-08T23:26:56.213328+0000","last_active":"2024-05-08T23:26:56.216620+0000","last_peered":"2024-05-08T23:26:56.216620+0000","last_clean":"2024-05-08T23:26:56.216620+0000","last_became_active":"2024-05-08T23:26:56.213035+0000","last_became_peered":"2024-05-08T23:26:56.213035+0000","last_unstale":"2024-05-08T23:26:56.216620+0000","last_undegraded":"2024-05-08T23:26:56.216620+0000","last_fullsized":"2024-05-08T23:26:56.216620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:36.414760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197691+0000","last_change":"2024-05-08T23:26:56.210542+0000","last_active":"2024-05-08T23:26:57.197691+0000","last_peered":"2024-05-08T23:26:57.197691+0000","last_clean":"2024-05-08T23:26:57.197691+0000","last_became_active":"2024-05-08T23:26:56.210262+0000","last_became_peered":"2024-05-08T23:26:56.210262+0000","last_unstale":"2024-05-08T23:26:57.197691+0000","last_undegraded":"2024-05-08T23:26:57.197691+0000","last_fullsized":"2024-05-08T23:26:57.197691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:12.290055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218666+0000","last_change":"2024-05-08T23:26:56.217496+0000","last_active":"2024-05-08T23:26:56.218666+0000","last_peered":"2024-05-08T23:26:56.218666+0000","last_clean":"2024-05-08T23:26:56.218666+0000","last_became_active":"2024-05-08T23:26:56.217166+0000","last_became_peered":"2024-05-08T23:26:56.217166+0000","last_unstale":"2024-05-08T23:26:56.218666+0000","last_undegraded":"2024-05-08T23:26:56.218666+0000","last_fullsized":"2024-05-08T23:26:56.218666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:42.595923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216883+0000","last_change":"2024-05-08T23:26:56.213403+0000","last_active":"2024-05-08T23:26:56.216883+0000","last_peered":"2024-05-08T23:26:56.216883+0000","last_clean":"2024-05-08T23:26:56.216883+0000","last_became_active":"2024-05-08T23:26:56.212964+0000","last_became_peered":"2024-05-08T23:26:56.212964+0000","last_unstale":"2024-05-08T23:26:56.216883+0000","last_undegraded":"2024-05-08T23:26:56.216883+0000","last_fullsized":"2024-05-08T23:26:56.216883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:18:40.017440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199007+0000","last_change":"2024-05-08T23:26:56.212567+0000","last_active":"2024-05-08T23:26:57.199007+0000","last_peered":"2024-05-08T23:26:57.199007+0000","last_clean":"2024-05-08T23:26:57.199007+0000","last_became_active":"2024-05-08T23:26:56.212261+0000","last_became_peered":"2024-05-08T23:26:56.212261+0000","last_unstale":"2024-05-08T23:26:57.199007+0000","last_undegraded":"2024-05-08T23:26:57.199007+0000","last_fullsized":"2024-05-08T23:26:57.199007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:42.659945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217539+0000","last_change":"2024-05-08T23:26:56.215606+0000","last_active":"2024-05-08T23:26:56.217539+0000","last_peered":"2024-05-08T23:26:56.217539+0000","last_clean":"2024-05-08T23:26:56.217539+0000","last_became_active":"2024-05-08T23:26:56.215360+0000","last_became_peered":"2024-05-08T23:26:56.215360+0000","last_unstale":"2024-05-08T23:26:56.217539+0000","last_undegraded":"2024-05-08T23:26:56.217539+0000","last_fullsized":"2024-05-08T23:26:56.217539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:00.263085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198570+0000","last_change":"2024-05-08T23:26:56.213744+0000","last_active":"2024-05-08T23:26:57.198570+0000","last_peered":"2024-05-08T23:26:57.198570+0000","last_clean":"2024-05-08T23:26:57.198570+0000","last_became_active":"2024-05-08T23:26:56.213385+0000","last_became_peered":"2024-05-08T23:26:56.213385+0000","last_unstale":"2024-05-08T23:26:57.198570+0000","last_undegraded":"2024-05-08T23:26:57.198570+0000","last_fullsized":"2024-05-08T23:26:57.198570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:08.428124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.215915+0000","last_change":"2024-05-08T23:26:56.209527+0000","last_active":"2024-05-08T23:26:56.215915+0000","last_peered":"2024-05-08T23:26:56.215915+0000","last_clean":"2024-05-08T23:26:56.215915+0000","last_became_active":"2024-05-08T23:26:56.209212+0000","last_became_peered":"2024-05-08T23:26:56.209212+0000","last_unstale":"2024-05-08T23:26:56.215915+0000","last_undegraded":"2024-05-08T23:26:56.215915+0000","last_fullsized":"2024-05-08T23:26:56.215915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.645733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197970+0000","last_change":"2024-05-08T23:26:56.220423+0000","last_active":"2024-05-08T23:26:57.197970+0000","last_peered":"2024-05-08T23:26:57.197970+0000","last_clean":"2024-05-08T23:26:57.197970+0000","last_became_active":"2024-05-08T23:26:56.219825+0000","last_became_peered":"2024-05-08T23:26:56.219825+0000","last_unstale":"2024-05-08T23:26:57.197970+0000","last_undegraded":"2024-05-08T23:26:57.197970+0000","last_fullsized":"2024-05-08T23:26:57.197970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:20.834836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.196769+0000","last_change":"2024-05-08T23:26:56.200689+0000","last_active":"2024-05-08T23:26:57.196769+0000","last_peered":"2024-05-08T23:26:57.196769+0000","last_clean":"2024-05-08T23:26:57.196769+0000","last_became_active":"2024-05-08T23:26:56.200395+0000","last_became_peered":"2024-05-08T23:26:56.200395+0000","last_unstale":"2024-05-08T23:26:57.196769+0000","last_undegraded":"2024-05-08T23:26:57.196769+0000","last_fullsized":"2024-05-08T23:26:57.196769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:21.483501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219794+0000","last_change":"2024-05-08T23:26:56.214376+0000","last_active":"2024-05-08T23:26:56.219794+0000","last_peered":"2024-05-08T23:26:56.219794+0000","last_clean":"2024-05-08T23:26:56.219794+0000","last_became_active":"2024-05-08T23:26:56.214093+0000","last_became_peered":"2024-05-08T23:26:56.214093+0000","last_unstale":"2024-05-08T23:26:56.219794+0000","last_undegraded":"2024-05-08T23:26:56.219794+0000","last_fullsized":"2024-05-08T23:26:56.219794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.645733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218069+0000","last_change":"2024-05-08T23:26:56.211972+0000","last_active":"2024-05-08T23:26:56.218069+0000","last_peered":"2024-05-08T23:26:56.218069+0000","last_clean":"2024-05-08T23:26:56.218069+0000","last_became_active":"2024-05-08T23:26:56.211691+0000","last_became_peered":"2024-05-08T23:26:56.211691+0000","last_unstale":"2024-05-08T23:26:56.218069+0000","last_undegraded":"2024-05-08T23:26:56.218069+0000","last_fullsized":"2024-05-08T23:26:56.218069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:13.472165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198300+0000","last_change":"2024-05-08T23:26:56.219800+0000","last_active":"2024-05-08T23:26:57.198300+0000","last_peered":"2024-05-08T23:26:57.198300+0000","last_clean":"2024-05-08T23:26:57.198300+0000","last_became_active":"2024-05-08T23:26:56.216894+0000","last_became_peered":"2024-05-08T23:26:56.216894+0000","last_unstale":"2024-05-08T23:26:57.198300+0000","last_undegraded":"2024-05-08T23:26:57.198300+0000","last_fullsized":"2024-05-08T23:26:57.198300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:53.396253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.225231+0000","last_change":"2024-05-08T23:26:56.211039+0000","last_active":"2024-05-08T23:26:56.225231+0000","last_peered":"2024-05-08T23:26:56.225231+0000","last_clean":"2024-05-08T23:26:56.225231+0000","last_became_active":"2024-05-08T23:26:56.210793+0000","last_became_peered":"2024-05-08T23:26:56.210793+0000","last_unstale":"2024-05-08T23:26:56.225231+0000","last_undegraded":"2024-05-08T23:26:56.225231+0000","last_fullsized":"2024-05-08T23:26:56.225231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:16.059931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198101+0000","last_change":"2024-05-08T23:26:56.219317+0000","last_active":"2024-05-08T23:26:57.198101+0000","last_peered":"2024-05-08T23:26:57.198101+0000","last_clean":"2024-05-08T23:26:57.198101+0000","last_became_active":"2024-05-08T23:26:56.215939+0000","last_became_peered":"2024-05-08T23:26:56.215939+0000","last_unstale":"2024-05-08T23:26:57.198101+0000","last_undegraded":"2024-05-08T23:26:57.198101+0000","last_fullsized":"2024-05-08T23:26:57.198101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:01.907735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198116+0000","last_change":"2024-05-08T23:26:56.218433+0000","last_active":"2024-05-08T23:26:57.198116+0000","last_peered":"2024-05-08T23:26:57.198116+0000","last_clean":"2024-05-08T23:26:57.198116+0000","last_became_active":"2024-05-08T23:26:56.213487+0000","last_became_peered":"2024-05-08T23:26:56.213487+0000","last_unstale":"2024-05-08T23:26:57.198116+0000","last_undegraded":"2024-05-08T23:26:57.198116+0000","last_fullsized":"2024-05-08T23:26:57.198116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:51.459201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.225138+0000","last_change":"2024-05-08T23:26:56.224183+0000","last_active":"2024-05-08T23:26:56.225138+0000","last_peered":"2024-05-08T23:26:56.225138+0000","last_clean":"2024-05-08T23:26:56.225138+0000","last_became_active":"2024-05-08T23:26:56.223847+0000","last_became_peered":"2024-05-08T23:26:56.223847+0000","last_unstale":"2024-05-08T23:26:56.225138+0000","last_undegraded":"2024-05-08T23:26:56.225138+0000","last_fullsized":"2024-05-08T23:26:56.225138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:19.238077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198153+0000","last_change":"2024-05-08T23:26:56.216039+0000","last_active":"2024-05-08T23:26:57.198153+0000","last_peered":"2024-05-08T23:26:57.198153+0000","last_clean":"2024-05-08T23:26:57.198153+0000","last_became_active":"2024-05-08T23:26:56.215113+0000","last_became_peered":"2024-05-08T23:26:56.215113+0000","last_unstale":"2024-05-08T23:26:57.198153+0000","last_undegraded":"2024-05-08T23:26:57.198153+0000","last_fullsized":"2024-05-08T23:26:57.198153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:42.595923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197844+0000","last_change":"2024-05-08T23:26:56.218951+0000","last_active":"2024-05-08T23:26:57.197844+0000","last_peered":"2024-05-08T23:26:57.197844+0000","last_clean":"2024-05-08T23:26:57.197844+0000","last_became_active":"2024-05-08T23:26:56.215106+0000","last_became_peered":"2024-05-08T23:26:56.215106+0000","last_unstale":"2024-05-08T23:26:57.197844+0000","last_undegraded":"2024-05-08T23:26:57.197844+0000","last_fullsized":"2024-05-08T23:26:57.197844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:41.139231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224567+0000","last_change":"2024-05-08T23:26:56.222512+0000","last_active":"2024-05-08T23:26:56.224567+0000","last_peered":"2024-05-08T23:26:56.224567+0000","last_clean":"2024-05-08T23:26:56.224567+0000","last_became_active":"2024-05-08T23:26:56.222221+0000","last_became_peered":"2024-05-08T23:26:56.222221+0000","last_unstale":"2024-05-08T23:26:56.224567+0000","last_undegraded":"2024-05-08T23:26:56.224567+0000","last_fullsized":"2024-05-08T23:26:56.224567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:31.227682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.220982+0000","last_change":"2024-05-08T23:26:56.219173+0000","last_active":"2024-05-08T23:26:56.220982+0000","last_peered":"2024-05-08T23:26:56.220982+0000","last_clean":"2024-05-08T23:26:56.220982+0000","last_became_active":"2024-05-08T23:26:56.218825+0000","last_became_peered":"2024-05-08T23:26:56.218825+0000","last_unstale":"2024-05-08T23:26:56.220982+0000","last_undegraded":"2024-05-08T23:26:56.220982+0000","last_fullsized":"2024-05-08T23:26:56.220982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:16.059931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217327+0000","last_change":"2024-05-08T23:26:56.212484+0000","last_active":"2024-05-08T23:26:56.217327+0000","last_peered":"2024-05-08T23:26:56.217327+0000","last_clean":"2024-05-08T23:26:56.217327+0000","last_became_active":"2024-05-08T23:26:56.212198+0000","last_became_peered":"2024-05-08T23:26:56.212198+0000","last_unstale":"2024-05-08T23:26:56.217327+0000","last_undegraded":"2024-05-08T23:26:56.217327+0000","last_fullsized":"2024-05-08T23:26:56.217327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:08.428124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198327+0000","last_change":"2024-05-08T23:26:56.214021+0000","last_active":"2024-05-08T23:26:57.198327+0000","last_peered":"2024-05-08T23:26:57.198327+0000","last_clean":"2024-05-08T23:26:57.198327+0000","last_became_active":"2024-05-08T23:26:56.213690+0000","last_became_peered":"2024-05-08T23:26:56.213690+0000","last_unstale":"2024-05-08T23:26:57.198327+0000","last_undegraded":"2024-05-08T23:26:57.198327+0000","last_fullsized":"2024-05-08T23:26:57.198327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.523398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217546+0000","last_change":"2024-05-08T23:26:56.214854+0000","last_active":"2024-05-08T23:26:56.217546+0000","last_peered":"2024-05-08T23:26:56.217546+0000","last_clean":"2024-05-08T23:26:56.217546+0000","last_became_active":"2024-05-08T23:26:56.214553+0000","last_became_peered":"2024-05-08T23:26:56.214553+0000","last_unstale":"2024-05-08T23:26:56.217546+0000","last_undegraded":"2024-05-08T23:26:56.217546+0000","last_fullsized":"2024-05-08T23:26:56.217546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:51.459201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.222697+0000","last_change":"2024-05-08T23:26:56.217991+0000","last_active":"2024-05-08T23:26:56.222697+0000","last_peered":"2024-05-08T23:26:56.222697+0000","last_clean":"2024-05-08T23:26:56.222697+0000","last_became_active":"2024-05-08T23:26:56.217716+0000","last_became_peered":"2024-05-08T23:26:56.217716+0000","last_unstale":"2024-05-08T23:26:56.222697+0000","last_undegraded":"2024-05-08T23:26:56.222697+0000","last_fullsized":"2024-05-08T23:26:56.222697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:16.287626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219666+0000","last_change":"2024-05-08T23:26:56.214607+0000","last_active":"2024-05-08T23:26:56.219666+0000","last_peered":"2024-05-08T23:26:56.219666+0000","last_clean":"2024-05-08T23:26:56.219666+0000","last_became_active":"2024-05-08T23:26:56.213810+0000","last_became_peered":"2024-05-08T23:26:56.213810+0000","last_unstale":"2024-05-08T23:26:56.219666+0000","last_undegraded":"2024-05-08T23:26:56.219666+0000","last_fullsized":"2024-05-08T23:26:56.219666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:25.098408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198060+0000","last_change":"2024-05-08T23:26:56.217811+0000","last_active":"2024-05-08T23:26:57.198060+0000","last_peered":"2024-05-08T23:26:57.198060+0000","last_clean":"2024-05-08T23:26:57.198060+0000","last_became_active":"2024-05-08T23:26:56.217395+0000","last_became_peered":"2024-05-08T23:26:56.217395+0000","last_unstale":"2024-05-08T23:26:57.198060+0000","last_undegraded":"2024-05-08T23:26:57.198060+0000","last_fullsized":"2024-05-08T23:26:57.198060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:13.472165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.221147+0000","last_change":"2024-05-08T23:26:56.219192+0000","last_active":"2024-05-08T23:26:56.221147+0000","last_peered":"2024-05-08T23:26:56.221147+0000","last_clean":"2024-05-08T23:26:56.221147+0000","last_became_active":"2024-05-08T23:26:56.218584+0000","last_became_peered":"2024-05-08T23:26:56.218584+0000","last_unstale":"2024-05-08T23:26:56.221147+0000","last_undegraded":"2024-05-08T23:26:56.221147+0000","last_fullsized":"2024-05-08T23:26:56.221147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:47.211615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199260+0000","last_change":"2024-05-08T23:26:55.755286+0000","last_active":"2024-05-08T23:26:57.199260+0000","last_peered":"2024-05-08T23:26:57.199260+0000","last_clean":"2024-05-08T23:26:57.199260+0000","last_became_active":"2024-05-08T23:26:55.755019+0000","last_became_peered":"2024-05-08T23:26:55.755019+0000","last_unstale":"2024-05-08T23:26:57.199260+0000","last_undegraded":"2024-05-08T23:26:57.199260+0000","last_fullsized":"2024-05-08T23:26:57.199260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:49.355512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199341+0000","last_change":"2024-05-08T23:26:56.214888+0000","last_active":"2024-05-08T23:26:57.199341+0000","last_peered":"2024-05-08T23:26:57.199341+0000","last_clean":"2024-05-08T23:26:57.199341+0000","last_became_active":"2024-05-08T23:26:56.214041+0000","last_became_peered":"2024-05-08T23:26:56.214041+0000","last_unstale":"2024-05-08T23:26:57.199341+0000","last_undegraded":"2024-05-08T23:26:57.199341+0000","last_fullsized":"2024-05-08T23:26:57.199341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:48.011872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.211279+0000","last_change":"2024-05-08T23:26:55.752137+0000","last_active":"2024-05-08T23:26:56.211279+0000","last_peered":"2024-05-08T23:26:56.211279+0000","last_clean":"2024-05-08T23:26:56.211279+0000","last_became_active":"2024-05-08T23:26:55.750793+0000","last_became_peered":"2024-05-08T23:26:55.750793+0000","last_unstale":"2024-05-08T23:26:56.211279+0000","last_undegraded":"2024-05-08T23:26:56.211279+0000","last_fullsized":"2024-05-08T23:26:56.211279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:02.909989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198433+0000","last_change":"2024-05-08T23:26:56.222787+0000","last_active":"2024-05-08T23:26:57.198433+0000","last_peered":"2024-05-08T23:26:57.198433+0000","last_clean":"2024-05-08T23:26:57.198433+0000","last_became_active":"2024-05-08T23:26:56.222614+0000","last_became_peered":"2024-05-08T23:26:56.222614+0000","last_unstale":"2024-05-08T23:26:57.198433+0000","last_undegraded":"2024-05-08T23:26:57.198433+0000","last_fullsized":"2024-05-08T23:26:57.198433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:22.981805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.229302+0000","last_change":"2024-05-08T23:26:55.754952+0000","last_active":"2024-05-08T23:26:56.229302+0000","last_peered":"2024-05-08T23:26:56.229302+0000","last_clean":"2024-05-08T23:26:56.229302+0000","last_became_active":"2024-05-08T23:26:55.754630+0000","last_became_peered":"2024-05-08T23:26:55.754630+0000","last_unstale":"2024-05-08T23:26:56.229302+0000","last_undegraded":"2024-05-08T23:26:56.229302+0000","last_fullsized":"2024-05-08T23:26:56.229302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:41.590210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217846+0000","last_change":"2024-05-08T23:26:56.215555+0000","last_active":"2024-05-08T23:26:56.217846+0000","last_peered":"2024-05-08T23:26:56.217846+0000","last_clean":"2024-05-08T23:26:56.217846+0000","last_became_active":"2024-05-08T23:26:56.215210+0000","last_became_peered":"2024-05-08T23:26:56.215210+0000","last_unstale":"2024-05-08T23:26:56.217846+0000","last_undegraded":"2024-05-08T23:26:56.217846+0000","last_fullsized":"2024-05-08T23:26:56.217846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:06.646632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.229207+0000","last_change":"2024-05-08T23:26:55.746957+0000","last_active":"2024-05-08T23:26:56.229207+0000","last_peered":"2024-05-08T23:26:56.229207+0000","last_clean":"2024-05-08T23:26:56.229207+0000","last_became_active":"2024-05-08T23:26:55.746643+0000","last_became_peered":"2024-05-08T23:26:55.746643+0000","last_unstale":"2024-05-08T23:26:56.229207+0000","last_undegraded":"2024-05-08T23:26:56.229207+0000","last_fullsized":"2024-05-08T23:26:56.229207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:01.593460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218123+0000","last_change":"2024-05-08T23:26:56.215663+0000","last_active":"2024-05-08T23:26:56.218123+0000","last_peered":"2024-05-08T23:26:56.218123+0000","last_clean":"2024-05-08T23:26:56.218123+0000","last_became_active":"2024-05-08T23:26:56.214686+0000","last_became_peered":"2024-05-08T23:26:56.214686+0000","last_unstale":"2024-05-08T23:26:56.218123+0000","last_undegraded":"2024-05-08T23:26:56.218123+0000","last_fullsized":"2024-05-08T23:26:56.218123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:36.048412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218479+0000","last_change":"2024-05-08T23:26:55.749375+0000","last_active":"2024-05-08T23:26:56.218479+0000","last_peered":"2024-05-08T23:26:56.218479+0000","last_clean":"2024-05-08T23:26:56.218479+0000","last_became_active":"2024-05-08T23:26:55.749066+0000","last_became_peered":"2024-05-08T23:26:55.749066+0000","last_unstale":"2024-05-08T23:26:56.218479+0000","last_undegraded":"2024-05-08T23:26:56.218479+0000","last_fullsized":"2024-05-08T23:26:56.218479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:55.744185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217764+0000","last_change":"2024-05-08T23:26:56.212898+0000","last_active":"2024-05-08T23:26:56.217764+0000","last_peered":"2024-05-08T23:26:56.217764+0000","last_clean":"2024-05-08T23:26:56.217764+0000","last_became_active":"2024-05-08T23:26:56.212567+0000","last_became_peered":"2024-05-08T23:26:56.212567+0000","last_unstale":"2024-05-08T23:26:56.217764+0000","last_undegraded":"2024-05-08T23:26:56.217764+0000","last_fullsized":"2024-05-08T23:26:56.217764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:22.981805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.210861+0000","last_change":"2024-05-08T23:26:55.751792+0000","last_active":"2024-05-08T23:26:56.210861+0000","last_peered":"2024-05-08T23:26:56.210861+0000","last_clean":"2024-05-08T23:26:56.210861+0000","last_became_active":"2024-05-08T23:26:55.751285+0000","last_became_peered":"2024-05-08T23:26:55.751285+0000","last_unstale":"2024-05-08T23:26:56.210861+0000","last_undegraded":"2024-05-08T23:26:56.210861+0000","last_fullsized":"2024-05-08T23:26:56.210861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:22.459643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217137+0000","last_change":"2024-05-08T23:26:56.211928+0000","last_active":"2024-05-08T23:26:56.217137+0000","last_peered":"2024-05-08T23:26:56.217137+0000","last_clean":"2024-05-08T23:26:56.217137+0000","last_became_active":"2024-05-08T23:26:56.211636+0000","last_became_peered":"2024-05-08T23:26:56.211636+0000","last_unstale":"2024-05-08T23:26:56.217137+0000","last_undegraded":"2024-05-08T23:26:56.217137+0000","last_fullsized":"2024-05-08T23:26:56.217137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.523398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.202869+0000","last_change":"2024-05-08T23:26:55.744642+0000","last_active":"2024-05-08T23:26:56.202869+0000","last_peered":"2024-05-08T23:26:56.202869+0000","last_clean":"2024-05-08T23:26:56.202869+0000","last_became_active":"2024-05-08T23:26:55.742524+0000","last_became_peered":"2024-05-08T23:26:55.742524+0000","last_unstale":"2024-05-08T23:26:56.202869+0000","last_undegraded":"2024-05-08T23:26:56.202869+0000","last_fullsized":"2024-05-08T23:26:56.202869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:42.732425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.196870+0000","last_change":"2024-05-08T23:26:56.201640+0000","last_active":"2024-05-08T23:26:57.196870+0000","last_peered":"2024-05-08T23:26:57.196870+0000","last_clean":"2024-05-08T23:26:57.196870+0000","last_became_active":"2024-05-08T23:26:56.201363+0000","last_became_peered":"2024-05-08T23:26:56.201363+0000","last_unstale":"2024-05-08T23:26:57.196870+0000","last_undegraded":"2024-05-08T23:26:57.196870+0000","last_fullsized":"2024-05-08T23:26:57.196870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.876263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.207492+0000","last_change":"2024-05-08T23:26:55.748872+0000","last_active":"2024-05-08T23:26:56.207492+0000","last_peered":"2024-05-08T23:26:56.207492+0000","last_clean":"2024-05-08T23:26:56.207492+0000","last_became_active":"2024-05-08T23:26:55.748542+0000","last_became_peered":"2024-05-08T23:26:55.748542+0000","last_unstale":"2024-05-08T23:26:56.207492+0000","last_undegraded":"2024-05-08T23:26:56.207492+0000","last_fullsized":"2024-05-08T23:26:56.207492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:08.275297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197425+0000","last_change":"2024-05-08T23:26:56.217280+0000","last_active":"2024-05-08T23:26:57.197425+0000","last_peered":"2024-05-08T23:26:57.197425+0000","last_clean":"2024-05-08T23:26:57.197425+0000","last_became_active":"2024-05-08T23:26:56.216956+0000","last_became_peered":"2024-05-08T23:26:56.216956+0000","last_unstale":"2024-05-08T23:26:57.197425+0000","last_undegraded":"2024-05-08T23:26:57.197425+0000","last_fullsized":"2024-05-08T23:26:57.197425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.771918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199218+0000","last_change":"2024-05-08T23:26:55.753320+0000","last_active":"2024-05-08T23:26:57.199218+0000","last_peered":"2024-05-08T23:26:57.199218+0000","last_clean":"2024-05-08T23:26:57.199218+0000","last_became_active":"2024-05-08T23:26:55.752922+0000","last_became_peered":"2024-05-08T23:26:55.752922+0000","last_unstale":"2024-05-08T23:26:57.199218+0000","last_undegraded":"2024-05-08T23:26:57.199218+0000","last_fullsized":"2024-05-08T23:26:57.199218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:33.504310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219355+0000","last_change":"2024-05-08T23:26:56.215783+0000","last_active":"2024-05-08T23:26:56.219355+0000","last_peered":"2024-05-08T23:26:56.219355+0000","last_clean":"2024-05-08T23:26:56.219355+0000","last_became_active":"2024-05-08T23:26:56.213244+0000","last_became_peered":"2024-05-08T23:26:56.213244+0000","last_unstale":"2024-05-08T23:26:56.219355+0000","last_undegraded":"2024-05-08T23:26:56.219355+0000","last_fullsized":"2024-05-08T23:26:56.219355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:08.846422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199108+0000","last_change":"2024-05-08T23:26:55.750296+0000","last_active":"2024-05-08T23:26:57.199108+0000","last_peered":"2024-05-08T23:26:57.199108+0000","last_clean":"2024-05-08T23:26:57.199108+0000","last_became_active":"2024-05-08T23:26:55.749975+0000","last_became_peered":"2024-05-08T23:26:55.749975+0000","last_unstale":"2024-05-08T23:26:57.199108+0000","last_undegraded":"2024-05-08T23:26:57.199108+0000","last_fullsized":"2024-05-08T23:26:57.199108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:08.997540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.226765+0000","last_change":"2024-05-08T23:26:56.225944+0000","last_active":"2024-05-08T23:26:56.226765+0000","last_peered":"2024-05-08T23:26:56.226765+0000","last_clean":"2024-05-08T23:26:56.226765+0000","last_became_active":"2024-05-08T23:26:56.225653+0000","last_became_peered":"2024-05-08T23:26:56.225653+0000","last_unstale":"2024-05-08T23:26:56.226765+0000","last_undegraded":"2024-05-08T23:26:56.226765+0000","last_fullsized":"2024-05-08T23:26:56.226765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:13.472165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198830+0000","last_change":"2024-05-08T23:26:55.754964+0000","last_active":"2024-05-08T23:26:57.198830+0000","last_peered":"2024-05-08T23:26:57.198830+0000","last_clean":"2024-05-08T23:26:57.198830+0000","last_became_active":"2024-05-08T23:26:55.754617+0000","last_became_peered":"2024-05-08T23:26:55.754617+0000","last_unstale":"2024-05-08T23:26:57.198830+0000","last_undegraded":"2024-05-08T23:26:57.198830+0000","last_fullsized":"2024-05-08T23:26:57.198830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:23.650422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.226161+0000","last_change":"2024-05-08T23:26:56.225752+0000","last_active":"2024-05-08T23:26:56.226161+0000","last_peered":"2024-05-08T23:26:56.226161+0000","last_clean":"2024-05-08T23:26:56.226161+0000","last_became_active":"2024-05-08T23:26:56.225459+0000","last_became_peered":"2024-05-08T23:26:56.225459+0000","last_unstale":"2024-05-08T23:26:56.226161+0000","last_undegraded":"2024-05-08T23:26:56.226161+0000","last_fullsized":"2024-05-08T23:26:56.226161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.523398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224128+0000","last_change":"2024-05-08T23:26:55.751568+0000","last_active":"2024-05-08T23:26:56.224128+0000","last_peered":"2024-05-08T23:26:56.224128+0000","last_clean":"2024-05-08T23:26:56.224128+0000","last_became_active":"2024-05-08T23:26:55.751290+0000","last_became_peered":"2024-05-08T23:26:55.751290+0000","last_unstale":"2024-05-08T23:26:56.224128+0000","last_undegraded":"2024-05-08T23:26:56.224128+0000","last_fullsized":"2024-05-08T23:26:56.224128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:08.997540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224452+0000","last_change":"2024-05-08T23:26:56.222628+0000","last_active":"2024-05-08T23:26:56.224452+0000","last_peered":"2024-05-08T23:26:56.224452+0000","last_clean":"2024-05-08T23:26:56.224452+0000","last_became_active":"2024-05-08T23:26:56.222380+0000","last_became_peered":"2024-05-08T23:26:56.222380+0000","last_unstale":"2024-05-08T23:26:56.224452+0000","last_undegraded":"2024-05-08T23:26:56.224452+0000","last_fullsized":"2024-05-08T23:26:56.224452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:25.098408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224869+0000","last_change":"2024-05-08T23:26:55.748991+0000","last_active":"2024-05-08T23:26:56.224869+0000","last_peered":"2024-05-08T23:26:56.224869+0000","last_clean":"2024-05-08T23:26:56.224869+0000","last_became_active":"2024-05-08T23:26:55.748691+0000","last_became_peered":"2024-05-08T23:26:55.748691+0000","last_unstale":"2024-05-08T23:26:56.224869+0000","last_undegraded":"2024-05-08T23:26:56.224869+0000","last_fullsized":"2024-05-08T23:26:56.224869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:29.409557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216895+0000","last_change":"2024-05-08T23:26:56.214475+0000","last_active":"2024-05-08T23:26:56.216895+0000","last_peered":"2024-05-08T23:26:56.216895+0000","last_clean":"2024-05-08T23:26:56.216895+0000","last_became_active":"2024-05-08T23:26:56.214164+0000","last_became_peered":"2024-05-08T23:26:56.214164+0000","last_unstale":"2024-05-08T23:26:56.216895+0000","last_undegraded":"2024-05-08T23:26:56.216895+0000","last_fullsized":"2024-05-08T23:26:56.216895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:25.098408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197110+0000","last_change":"2024-05-08T23:26:55.746305+0000","last_active":"2024-05-08T23:26:57.197110+0000","last_peered":"2024-05-08T23:26:57.197110+0000","last_clean":"2024-05-08T23:26:57.197110+0000","last_became_active":"2024-05-08T23:26:55.746010+0000","last_became_peered":"2024-05-08T23:26:55.746010+0000","last_unstale":"2024-05-08T23:26:57.197110+0000","last_undegraded":"2024-05-08T23:26:57.197110+0000","last_fullsized":"2024-05-08T23:26:57.197110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:46.205902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218482+0000","last_change":"2024-05-08T23:26:56.216764+0000","last_active":"2024-05-08T23:26:56.218482+0000","last_peered":"2024-05-08T23:26:56.218482+0000","last_clean":"2024-05-08T23:26:56.218482+0000","last_became_active":"2024-05-08T23:26:56.216273+0000","last_became_peered":"2024-05-08T23:26:56.216273+0000","last_unstale":"2024-05-08T23:26:56.218482+0000","last_undegraded":"2024-05-08T23:26:56.218482+0000","last_fullsized":"2024-05-08T23:26:56.218482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:25.496589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.201651+0000","last_change":"2024-05-08T23:26:55.747631+0000","last_active":"2024-05-08T23:26:56.201651+0000","last_peered":"2024-05-08T23:26:56.201651+0000","last_clean":"2024-05-08T23:26:56.201651+0000","last_became_active":"2024-05-08T23:26:55.747338+0000","last_became_peered":"2024-05-08T23:26:55.747338+0000","last_unstale":"2024-05-08T23:26:56.201651+0000","last_undegraded":"2024-05-08T23:26:56.201651+0000","last_fullsized":"2024-05-08T23:26:56.201651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:47.770925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218454+0000","last_change":"2024-05-08T23:26:56.217063+0000","last_active":"2024-05-08T23:26:56.218454+0000","last_peered":"2024-05-08T23:26:56.218454+0000","last_clean":"2024-05-08T23:26:56.218454+0000","last_became_active":"2024-05-08T23:26:56.216798+0000","last_became_peered":"2024-05-08T23:26:56.216798+0000","last_unstale":"2024-05-08T23:26:56.218454+0000","last_undegraded":"2024-05-08T23:26:56.218454+0000","last_fullsized":"2024-05-08T23:26:56.218454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.216913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.226721+0000","last_change":"2024-05-08T23:26:55.749893+0000","last_active":"2024-05-08T23:26:56.226721+0000","last_peered":"2024-05-08T23:26:56.226721+0000","last_clean":"2024-05-08T23:26:56.226721+0000","last_became_active":"2024-05-08T23:26:55.749596+0000","last_became_peered":"2024-05-08T23:26:55.749596+0000","last_unstale":"2024-05-08T23:26:56.226721+0000","last_undegraded":"2024-05-08T23:26:56.226721+0000","last_fullsized":"2024-05-08T23:26:56.226721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:35.409047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.208643+0000","last_change":"2024-05-08T23:26:56.204232+0000","last_active":"2024-05-08T23:26:56.208643+0000","last_peered":"2024-05-08T23:26:56.208643+0000","last_clean":"2024-05-08T23:26:56.208643+0000","last_became_active":"2024-05-08T23:26:56.203932+0000","last_became_peered":"2024-05-08T23:26:56.203932+0000","last_unstale":"2024-05-08T23:26:56.208643+0000","last_undegraded":"2024-05-08T23:26:56.208643+0000","last_fullsized":"2024-05-08T23:26:56.208643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:32.927014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199057+0000","last_change":"2024-05-08T23:26:55.747903+0000","last_active":"2024-05-08T23:26:57.199057+0000","last_peered":"2024-05-08T23:26:57.199057+0000","last_clean":"2024-05-08T23:26:57.199057+0000","last_became_active":"2024-05-08T23:26:55.747625+0000","last_became_peered":"2024-05-08T23:26:55.747625+0000","last_unstale":"2024-05-08T23:26:57.199057+0000","last_undegraded":"2024-05-08T23:26:57.199057+0000","last_fullsized":"2024-05-08T23:26:57.199057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:47.770925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199324+0000","last_change":"2024-05-08T23:26:56.217310+0000","last_active":"2024-05-08T23:26:57.199324+0000","last_peered":"2024-05-08T23:26:57.199324+0000","last_clean":"2024-05-08T23:26:57.199324+0000","last_became_active":"2024-05-08T23:26:56.217029+0000","last_became_peered":"2024-05-08T23:26:56.217029+0000","last_unstale":"2024-05-08T23:26:57.199324+0000","last_undegraded":"2024-05-08T23:26:57.199324+0000","last_fullsized":"2024-05-08T23:26:57.199324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:06.516103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224468+0000","last_change":"2024-05-08T23:26:55.749736+0000","last_active":"2024-05-08T23:26:56.224468+0000","last_peered":"2024-05-08T23:26:56.224468+0000","last_clean":"2024-05-08T23:26:56.224468+0000","last_became_active":"2024-05-08T23:26:55.749441+0000","last_became_peered":"2024-05-08T23:26:55.749441+0000","last_unstale":"2024-05-08T23:26:56.224468+0000","last_undegraded":"2024-05-08T23:26:56.224468+0000","last_fullsized":"2024-05-08T23:26:56.224468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:07.840709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.220478+0000","last_change":"2024-05-08T23:26:56.218056+0000","last_active":"2024-05-08T23:26:56.220478+0000","last_peered":"2024-05-08T23:26:56.220478+0000","last_clean":"2024-05-08T23:26:56.220478+0000","last_became_active":"2024-05-08T23:26:56.217773+0000","last_became_peered":"2024-05-08T23:26:56.217773+0000","last_unstale":"2024-05-08T23:26:56.220478+0000","last_undegraded":"2024-05-08T23:26:56.220478+0000","last_fullsized":"2024-05-08T23:26:56.220478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:16.059931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.207998+0000","last_change":"2024-05-08T23:26:55.742485+0000","last_active":"2024-05-08T23:26:56.207998+0000","last_peered":"2024-05-08T23:26:56.207998+0000","last_clean":"2024-05-08T23:26:56.207998+0000","last_became_active":"2024-05-08T23:26:55.742154+0000","last_became_peered":"2024-05-08T23:26:55.742154+0000","last_unstale":"2024-05-08T23:26:56.207998+0000","last_undegraded":"2024-05-08T23:26:56.207998+0000","last_fullsized":"2024-05-08T23:26:56.207998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:05.860831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197970+0000","last_change":"2024-05-08T23:26:56.220214+0000","last_active":"2024-05-08T23:26:57.197970+0000","last_peered":"2024-05-08T23:26:57.197970+0000","last_clean":"2024-05-08T23:26:57.197970+0000","last_became_active":"2024-05-08T23:26:56.219802+0000","last_became_peered":"2024-05-08T23:26:56.219802+0000","last_unstale":"2024-05-08T23:26:57.197970+0000","last_undegraded":"2024-05-08T23:26:57.197970+0000","last_fullsized":"2024-05-08T23:26:57.197970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:41.680125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.196984+0000","last_change":"2024-05-08T23:26:55.755256+0000","last_active":"2024-05-08T23:26:57.196984+0000","last_peered":"2024-05-08T23:26:57.196984+0000","last_clean":"2024-05-08T23:26:57.196984+0000","last_became_active":"2024-05-08T23:26:55.754980+0000","last_became_peered":"2024-05-08T23:26:55.754980+0000","last_unstale":"2024-05-08T23:26:57.196984+0000","last_undegraded":"2024-05-08T23:26:57.196984+0000","last_fullsized":"2024-05-08T23:26:57.196984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:34.570523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216613+0000","last_change":"2024-05-08T23:26:56.212708+0000","last_active":"2024-05-08T23:26:56.216613+0000","last_peered":"2024-05-08T23:26:56.216613+0000","last_clean":"2024-05-08T23:26:56.216613+0000","last_became_active":"2024-05-08T23:26:56.211885+0000","last_became_peered":"2024-05-08T23:26:56.211885+0000","last_unstale":"2024-05-08T23:26:56.216613+0000","last_undegraded":"2024-05-08T23:26:56.216613+0000","last_fullsized":"2024-05-08T23:26:56.216613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:06.516103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198090+0000","last_change":"2024-05-08T23:26:55.752322+0000","last_active":"2024-05-08T23:26:57.198090+0000","last_peered":"2024-05-08T23:26:57.198090+0000","last_clean":"2024-05-08T23:26:57.198090+0000","last_became_active":"2024-05-08T23:26:55.752041+0000","last_became_peered":"2024-05-08T23:26:55.752041+0000","last_unstale":"2024-05-08T23:26:57.198090+0000","last_undegraded":"2024-05-08T23:26:57.198090+0000","last_fullsized":"2024-05-08T23:26:57.198090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:03.229647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197691+0000","last_change":"2024-05-08T23:26:56.210610+0000","last_active":"2024-05-08T23:26:57.197691+0000","last_peered":"2024-05-08T23:26:57.197691+0000","last_clean":"2024-05-08T23:26:57.197691+0000","last_became_active":"2024-05-08T23:26:56.210287+0000","last_became_peered":"2024-05-08T23:26:56.210287+0000","last_unstale":"2024-05-08T23:26:57.197691+0000","last_undegraded":"2024-05-08T23:26:57.197691+0000","last_fullsized":"2024-05-08T23:26:57.197691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:52.123872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197792+0000","last_change":"2024-05-08T23:26:55.748394+0000","last_active":"2024-05-08T23:26:57.197792+0000","last_peered":"2024-05-08T23:26:57.197792+0000","last_clean":"2024-05-08T23:26:57.197792+0000","last_became_active":"2024-05-08T23:26:55.748139+0000","last_became_peered":"2024-05-08T23:26:55.748139+0000","last_unstale":"2024-05-08T23:26:57.197792+0000","last_undegraded":"2024-05-08T23:26:57.197792+0000","last_fullsized":"2024-05-08T23:26:57.197792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:58.003200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219665+0000","last_change":"2024-05-08T23:26:56.215669+0000","last_active":"2024-05-08T23:26:56.219665+0000","last_peered":"2024-05-08T23:26:56.219665+0000","last_clean":"2024-05-08T23:26:56.219665+0000","last_became_active":"2024-05-08T23:26:56.215345+0000","last_became_peered":"2024-05-08T23:26:56.215345+0000","last_unstale":"2024-05-08T23:26:56.219665+0000","last_undegraded":"2024-05-08T23:26:56.219665+0000","last_fullsized":"2024-05-08T23:26:56.219665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:48.011872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.196756+0000","last_change":"2024-05-08T23:26:55.753049+0000","last_active":"2024-05-08T23:26:57.196756+0000","last_peered":"2024-05-08T23:26:57.196756+0000","last_clean":"2024-05-08T23:26:57.196756+0000","last_became_active":"2024-05-08T23:26:55.752756+0000","last_became_peered":"2024-05-08T23:26:55.752756+0000","last_unstale":"2024-05-08T23:26:57.196756+0000","last_undegraded":"2024-05-08T23:26:57.196756+0000","last_fullsized":"2024-05-08T23:26:57.196756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.640021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.210025+0000","last_change":"2024-05-08T23:26:56.205854+0000","last_active":"2024-05-08T23:26:56.210025+0000","last_peered":"2024-05-08T23:26:56.210025+0000","last_clean":"2024-05-08T23:26:56.210025+0000","last_became_active":"2024-05-08T23:26:56.205575+0000","last_became_peered":"2024-05-08T23:26:56.205575+0000","last_unstale":"2024-05-08T23:26:56.210025+0000","last_undegraded":"2024-05-08T23:26:56.210025+0000","last_fullsized":"2024-05-08T23:26:56.210025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:32.927014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216090+0000","last_change":"2024-05-08T23:26:55.751835+0000","last_active":"2024-05-08T23:26:56.216090+0000","last_peered":"2024-05-08T23:26:56.216090+0000","last_clean":"2024-05-08T23:26:56.216090+0000","last_became_active":"2024-05-08T23:26:55.751562+0000","last_became_peered":"2024-05-08T23:26:55.751562+0000","last_unstale":"2024-05-08T23:26:56.216090+0000","last_undegraded":"2024-05-08T23:26:56.216090+0000","last_fullsized":"2024-05-08T23:26:56.216090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:40.067977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.221013+0000","last_change":"2024-05-08T23:26:56.219146+0000","last_active":"2024-05-08T23:26:56.221013+0000","last_peered":"2024-05-08T23:26:56.221013+0000","last_clean":"2024-05-08T23:26:56.221013+0000","last_became_active":"2024-05-08T23:26:56.218798+0000","last_became_peered":"2024-05-08T23:26:56.218798+0000","last_unstale":"2024-05-08T23:26:56.221013+0000","last_undegraded":"2024-05-08T23:26:56.221013+0000","last_fullsized":"2024-05-08T23:26:56.221013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:47.211615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.212933+0000","last_change":"2024-05-08T23:26:55.750013+0000","last_active":"2024-05-08T23:26:56.212933+0000","last_peered":"2024-05-08T23:26:56.212933+0000","last_clean":"2024-05-08T23:26:56.212933+0000","last_became_active":"2024-05-08T23:26:55.748574+0000","last_became_peered":"2024-05-08T23:26:55.748574+0000","last_unstale":"2024-05-08T23:26:56.212933+0000","last_undegraded":"2024-05-08T23:26:56.212933+0000","last_fullsized":"2024-05-08T23:26:56.212933+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.467464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198548+0000","last_change":"2024-05-08T23:26:56.214420+0000","last_active":"2024-05-08T23:26:57.198548+0000","last_peered":"2024-05-08T23:26:57.198548+0000","last_clean":"2024-05-08T23:26:57.198548+0000","last_became_active":"2024-05-08T23:26:56.214018+0000","last_became_peered":"2024-05-08T23:26:56.214018+0000","last_unstale":"2024-05-08T23:26:57.198548+0000","last_undegraded":"2024-05-08T23:26:57.198548+0000","last_fullsized":"2024-05-08T23:26:57.198548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:16.059931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198875+0000","last_change":"2024-05-08T23:26:55.755512+0000","last_active":"2024-05-08T23:26:57.198875+0000","last_peered":"2024-05-08T23:26:57.198875+0000","last_clean":"2024-05-08T23:26:57.198875+0000","last_became_active":"2024-05-08T23:26:55.755204+0000","last_became_peered":"2024-05-08T23:26:55.755204+0000","last_unstale":"2024-05-08T23:26:57.198875+0000","last_undegraded":"2024-05-08T23:26:57.198875+0000","last_fullsized":"2024-05-08T23:26:57.198875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:12.268144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224571+0000","last_change":"2024-05-08T23:26:56.224042+0000","last_active":"2024-05-08T23:26:56.224571+0000","last_peered":"2024-05-08T23:26:56.224571+0000","last_clean":"2024-05-08T23:26:56.224571+0000","last_became_active":"2024-05-08T23:26:56.223705+0000","last_became_peered":"2024-05-08T23:26:56.223705+0000","last_unstale":"2024-05-08T23:26:56.224571+0000","last_undegraded":"2024-05-08T23:26:56.224571+0000","last_fullsized":"2024-05-08T23:26:56.224571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:12.365940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198654+0000","last_change":"2024-05-08T23:26:55.751632+0000","last_active":"2024-05-08T23:26:57.198654+0000","last_peered":"2024-05-08T23:26:57.198654+0000","last_clean":"2024-05-08T23:26:57.198654+0000","last_became_active":"2024-05-08T23:26:55.751370+0000","last_became_peered":"2024-05-08T23:26:55.751370+0000","last_unstale":"2024-05-08T23:26:57.198654+0000","last_undegraded":"2024-05-08T23:26:57.198654+0000","last_fullsized":"2024-05-08T23:26:57.198654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:30.221969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216453+0000","last_change":"2024-05-08T23:26:56.212226+0000","last_active":"2024-05-08T23:26:56.216453+0000","last_peered":"2024-05-08T23:26:56.216453+0000","last_clean":"2024-05-08T23:26:56.216453+0000","last_became_active":"2024-05-08T23:26:56.211926+0000","last_became_peered":"2024-05-08T23:26:56.211926+0000","last_unstale":"2024-05-08T23:26:56.216453+0000","last_undegraded":"2024-05-08T23:26:56.216453+0000","last_fullsized":"2024-05-08T23:26:56.216453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:02.599173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.204598+0000","last_change":"2024-05-08T23:26:55.750628+0000","last_active":"2024-05-08T23:26:56.204598+0000","last_peered":"2024-05-08T23:26:56.204598+0000","last_clean":"2024-05-08T23:26:56.204598+0000","last_became_active":"2024-05-08T23:26:55.750358+0000","last_became_peered":"2024-05-08T23:26:55.750358+0000","last_unstale":"2024-05-08T23:26:56.204598+0000","last_undegraded":"2024-05-08T23:26:56.204598+0000","last_fullsized":"2024-05-08T23:26:56.204598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:05.510390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224480+0000","last_change":"2024-05-08T23:26:56.222698+0000","last_active":"2024-05-08T23:26:56.224480+0000","last_peered":"2024-05-08T23:26:56.224480+0000","last_clean":"2024-05-08T23:26:56.224480+0000","last_became_active":"2024-05-08T23:26:56.222236+0000","last_became_peered":"2024-05-08T23:26:56.222236+0000","last_unstale":"2024-05-08T23:26:56.224480+0000","last_undegraded":"2024-05-08T23:26:56.224480+0000","last_fullsized":"2024-05-08T23:26:56.224480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:08.428124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.202338+0000","last_change":"2024-05-08T23:26:55.751305+0000","last_active":"2024-05-08T23:26:56.202338+0000","last_peered":"2024-05-08T23:26:56.202338+0000","last_clean":"2024-05-08T23:26:56.202338+0000","last_became_active":"2024-05-08T23:26:55.751027+0000","last_became_peered":"2024-05-08T23:26:55.751027+0000","last_unstale":"2024-05-08T23:26:56.202338+0000","last_undegraded":"2024-05-08T23:26:56.202338+0000","last_fullsized":"2024-05-08T23:26:56.202338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:08.997540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219123+0000","last_change":"2024-05-08T23:26:56.216915+0000","last_active":"2024-05-08T23:26:56.219123+0000","last_peered":"2024-05-08T23:26:56.219123+0000","last_clean":"2024-05-08T23:26:56.219123+0000","last_became_active":"2024-05-08T23:26:56.216600+0000","last_became_peered":"2024-05-08T23:26:56.216600+0000","last_unstale":"2024-05-08T23:26:56.219123+0000","last_undegraded":"2024-05-08T23:26:56.219123+0000","last_fullsized":"2024-05-08T23:26:56.219123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:02.599173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198358+0000","last_change":"2024-05-08T23:26:55.753278+0000","last_active":"2024-05-08T23:26:57.198358+0000","last_peered":"2024-05-08T23:26:57.198358+0000","last_clean":"2024-05-08T23:26:57.198358+0000","last_became_active":"2024-05-08T23:26:55.752974+0000","last_became_peered":"2024-05-08T23:26:55.752974+0000","last_unstale":"2024-05-08T23:26:57.198358+0000","last_undegraded":"2024-05-08T23:26:57.198358+0000","last_fullsized":"2024-05-08T23:26:57.198358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198226+0000","last_change":"2024-05-08T23:26:56.216721+0000","last_active":"2024-05-08T23:26:57.198226+0000","last_peered":"2024-05-08T23:26:57.198226+0000","last_clean":"2024-05-08T23:26:57.198226+0000","last_became_active":"2024-05-08T23:26:56.214498+0000","last_became_peered":"2024-05-08T23:26:56.214498+0000","last_unstale":"2024-05-08T23:26:57.198226+0000","last_undegraded":"2024-05-08T23:26:57.198226+0000","last_fullsized":"2024-05-08T23:26:57.198226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:32.927014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.203684+0000","last_change":"2024-05-08T23:26:55.748328+0000","last_active":"2024-05-08T23:26:56.203684+0000","last_peered":"2024-05-08T23:26:56.203684+0000","last_clean":"2024-05-08T23:26:56.203684+0000","last_became_active":"2024-05-08T23:26:55.748033+0000","last_became_peered":"2024-05-08T23:26:55.748033+0000","last_unstale":"2024-05-08T23:26:56.203684+0000","last_undegraded":"2024-05-08T23:26:56.203684+0000","last_fullsized":"2024-05-08T23:26:56.203684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:47.770925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217505+0000","last_change":"2024-05-08T23:26:56.214935+0000","last_active":"2024-05-08T23:26:56.217505+0000","last_peered":"2024-05-08T23:26:56.217505+0000","last_clean":"2024-05-08T23:26:56.217505+0000","last_became_active":"2024-05-08T23:26:56.214578+0000","last_became_peered":"2024-05-08T23:26:56.214578+0000","last_unstale":"2024-05-08T23:26:56.217505+0000","last_undegraded":"2024-05-08T23:26:56.217505+0000","last_fullsized":"2024-05-08T23:26:56.217505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:31.870394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216530+0000","last_change":"2024-05-08T23:26:55.754914+0000","last_active":"2024-05-08T23:26:56.216530+0000","last_peered":"2024-05-08T23:26:56.216530+0000","last_clean":"2024-05-08T23:26:56.216530+0000","last_became_active":"2024-05-08T23:26:55.754650+0000","last_became_peered":"2024-05-08T23:26:55.754650+0000","last_unstale":"2024-05-08T23:26:56.216530+0000","last_undegraded":"2024-05-08T23:26:56.216530+0000","last_fullsized":"2024-05-08T23:26:56.216530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:18:39.011728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:26:55.746125+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:26:55.746125+0000","last_peered":"2024-05-08T23:26:55.746125+0000","last_clean":"2024-05-08T23:26:55.746125+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:26:55.746125+0000","last_undegraded":"2024-05-08T23:26:55.746125+0000","last_fullsized":"2024-05-08T23:26:55.746125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1617920,"data_stored":1028469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75700000000000001,"5min":1.038,"15min":1.038},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.376,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.58599999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60899999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.622},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.40699999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.29499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68000000000000005,"5min":0.87,"15min":0.87},"last":0.44800000000000001}]},{"osd":5,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73499999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.754,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77000000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":438725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68600000000000005}]},{"osd":1,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":1.115,"15min":1.115},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76500000000000001,"5min":1.052,"15min":1.052},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84399999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.58799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1032192,"data_stored":438225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.371,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56100000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.42299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.30599999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.435},{"interface":"front","average":{"1min":0.443,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":1.01,"15min":1.01},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":1.028,"15min":1.028},"last":0.39000000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":1.105,"15min":1.105},"last":0.58199999999999996}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.54000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1617920,"data_stored":1028433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48399999999999999,"15min":0.72599999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.52900000000000003,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":0.71399999999999997},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60799999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.51600000000000001,"15min":0.77800000000000002},"min":{"1min":0.312,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52900000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54800000000000004,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58099999999999996,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65400000000000003,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69099999999999995,"15min":0.93500000000000005},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.437},{"interface":"front","average":{"1min":0.624,"5min":0.70399999999999996,"15min":0.90800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.91300000000000003},"min":{"1min":0.50700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75700000000000001,"15min":1.0069999999999999},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73499999999999999,"15min":0.97499999999999998},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81799999999999995,"5min":7.734,"15min":7.734},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.46200000000000002}]},{"osd":7,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.96399999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.92200000000000004},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.83299999999999996,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.54300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":438613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.50700000000000001,"5min":0.621,"15min":0.621},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65800000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28399999999999997}]},{"osd":1,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44900000000000001,"15min":0.41299999999999998},"min":{"1min":0.33300000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.58499999999999996,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.435,"15min":0.42399999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.621,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56200000000000006,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.49099999999999999,"15min":0.46500000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.39800000000000002}]},{"osd":4,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64500000000000002,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90300000000000002,"5min":1.024,"15min":1.024},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.628,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46700000000000003}]},{"osd":5,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91600000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83199999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.47799999999999998}]},{"osd":6,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.51100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.89200000000000002,"5min":1.159,"15min":1.159},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53500000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69099999999999995,"15min":0.67200000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90000000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92500000000000004,"5min":1.194,"15min":1.194},"last":0.503}]}]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1032192,"data_stored":439595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.41599999999999998},"min":{"1min":0.308,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":1.46,"15min":1.46},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40400000000000003,"15min":0.41899999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.496,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.55800000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.32900000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60899999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.42899999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.55400000000000005,"15min":0.84299999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":11.964,"15min":11.964},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73899999999999999,"5min":1.593,"15min":1.593},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.629,"15min":0.65900000000000003},"min":{"1min":0.48399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78900000000000003,"5min":1.375,"15min":1.375},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85099999999999998,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.628}]},{"osd":5,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.50700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.94799999999999995,"5min":1.536,"15min":1.536},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.68400000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76900000000000002,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65400000000000003,"15min":0.68000000000000005},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89900000000000002,"5min":1.204,"15min":1.204},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69399999999999995,"15min":0.72899999999999998},"min":{"1min":0.57399999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90700000000000003,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":0.77000000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86099999999999999,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.58099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":1019904,"data_stored":438031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.64200000000000002},"min":{"1min":0.33700000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73399999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59499999999999997,"15min":0.72099999999999997},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.746,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.748,"5min":2.145,"15min":2.145},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60199999999999998,"15min":0.68400000000000005},"min":{"1min":0.36899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68500000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64700000000000002,"15min":0.78500000000000003},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81699999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.74299999999999999,"15min":1.1240000000000001},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.621}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.877,"15min":0.877},"last":0.497},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63400000000000001,"15min":0.68200000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73899999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.628},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77400000000000002,"5min":1.24,"15min":1.24},"last":0.55800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":439617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85199999999999998,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":0.89400000000000002},"min":{"1min":0.35199999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":8.798,"15min":8.798},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.77500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":4.71,"15min":4.71},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70699999999999996,"15min":0.96799999999999997},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0009999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.79100000000000004},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.82199999999999995,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.93000000000000005},"min":{"1min":0.46999999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78500000000000003,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73899999999999999,"15min":1.0109999999999999},"min":{"1min":0.51100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0489999999999999,"5min":10.427,"15min":10.427},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58599999999999997,"5min":0.874,"15min":0.874},"last":0.505},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85299999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.01},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:27:00.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:27:00.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.624+0000 7f4a9337f700 1 Processor -- start 2024-05-08T23:27:00.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.624+0000 7f4a9337f700 1 -- start start 2024-05-08T23:27:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9337f700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4a8c15db00 0x7f4a8c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9337f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4a8c0caa50 0x7f4a8c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9337f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caeb0 0x7f4a8c06f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9337f700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c15dee0 con 0x7f4a8c0caa50 2024-05-08T23:27:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9337f700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c0ccad0 con 0x7f4a8c15db00 2024-05-08T23:27:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9337f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c06ad20 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9091a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4a8c0caa50 0x7f4a8c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9091a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4a8c0caa50 0x7f4a8c06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37120/0 (socket says 172.21.15.5:37120) 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.625+0000 7f4a9091a700 1 -- 172.21.15.5:0/3970311742 learned_addr learned my addr 172.21.15.5:0/3970311742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a9191c700 1 --2- 172.21.15.5:0/3970311742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caeb0 0x7f4a8c06f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a9111b700 1 --2- 172.21.15.5:0/3970311742 >> v2:172.21.15.49:3300/0 conn(0x7f4a8c15db00 0x7f4a8c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a9091a700 1 -- 172.21.15.5:0/3970311742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caeb0 msgr2=0x7f4a8c06f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a9091a700 1 --2- 172.21.15.5:0/3970311742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caeb0 0x7f4a8c06f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a9091a700 1 -- 172.21.15.5:0/3970311742 >> v2:172.21.15.49:3300/0 conn(0x7f4a8c15db00 msgr2=0x7f4a8c0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a9091a700 1 --2- 172.21.15.5:0/3970311742 >> v2:172.21.15.49:3300/0 conn(0x7f4a8c15db00 0x7f4a8c0cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a9091a700 1 -- 172.21.15.5:0/3970311742 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c06b000 con 0x7f4a8c0caa50 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a9091a700 1 --2- 172.21.15.5:0/3970311742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caa50 0x7f4a8c06f280 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0cdf50 tx=0x7f4a80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff70e3a29b47c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a7ffff700 1 -- 172.21.15.5:0/3970311742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80011830 con 0x7f4a8c0caa50 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a7ffff700 1 -- 172.21.15.5:0/3970311742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a80011990 con 0x7f4a8c0caa50 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.626+0000 7f4a7ffff700 1 -- 172.21.15.5:0/3970311742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80011d20 con 0x7f4a8c0caa50 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 -- 172.21.15.5:0/3970311742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caa50 msgr2=0x7f4a8c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 --2- 172.21.15.5:0/3970311742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caa50 0x7f4a8c06f280 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0cdf50 tx=0x7f4a80009c00 comp rx=0 tx=0).stop 2024-05-08T23:27:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 -- 172.21.15.5:0/3970311742 shutdown_connections 2024-05-08T23:27:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 --2- 172.21.15.5:0/3970311742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caeb0 0x7f4a8c06f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 --2- 172.21.15.5:0/3970311742 >> v2:172.21.15.49:3300/0 conn(0x7f4a8c15db00 0x7f4a8c0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 --2- 172.21.15.5:0/3970311742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caa50 0x7f4a8c06f280 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 -- 172.21.15.5:0/3970311742 >> 172.21.15.5:0/3970311742 conn(0x7f4a8c099a50 msgr2=0x7f4a8c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 -- 172.21.15.5:0/3970311742 shutdown_connections 2024-05-08T23:27:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.627+0000 7f4a9337f700 1 -- 172.21.15.5:0/3970311742 wait complete. 2024-05-08T23:27:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.628+0000 7f4a9337f700 1 Processor -- start 2024-05-08T23:27:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.628+0000 7f4a9337f700 1 -- start start 2024-05-08T23:27:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9337f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caa50 0x7f4a8c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9337f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caeb0 0x7f4a8c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9337f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a8c15db00 0x7f4a8c0b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9337f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c155790 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9337f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0b46b0 con 0x7f4a8c15db00 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9337f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0b4990 con 0x7f4a8c0caa50 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9091a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caeb0 0x7f4a8c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9091a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caeb0 0x7f4a8c0b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37126/0 (socket says 172.21.15.5:37126) 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9091a700 1 -- 172.21.15.5:0/1539692290 learned_addr learned my addr 172.21.15.5:0/1539692290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9111b700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caa50 0x7f4a8c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.629+0000 7f4a9191c700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a8c15db00 0x7f4a8c0b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a9091a700 1 -- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caa50 msgr2=0x7f4a8c155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a9091a700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caa50 0x7f4a8c155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a9091a700 1 -- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a8c15db00 msgr2=0x7f4a8c0b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a9091a700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a8c15db00 0x7f4a8c0b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a9091a700 1 -- 172.21.15.5:0/1539692290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a80009550 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a9091a700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caeb0 0x7f4a8c0b39f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009c40 tx=0x7f4a80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a7dffb700 1 -- 172.21.15.5:0/1539692290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80004510 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a7dffb700 1 -- 172.21.15.5:0/1539692290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a800036a0 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c0b4c70 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a7dffb700 1 -- 172.21.15.5:0/1539692290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80003a50 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.630+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c0b5230 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.631+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c068000 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.632+0000 7f4a7dffb700 1 -- 172.21.15.5:0/1539692290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a80003bb0 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.632+0000 7f4a7dffb700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a7006ac90 0x7f4a7006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.632+0000 7f4a7dffb700 1 -- 172.21.15.5:0/1539692290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a80021030 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.633+0000 7f4a9111b700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a7006ac90 0x7f4a7006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.633+0000 7f4a9111b700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a7006ac90 0x7f4a7006d150 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0b4330 tx=0x7f4a88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.637+0000 7f4a7dffb700 1 -- 172.21.15.5:0/1539692290 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4a80055ad0 con 0x7f4a8c0caeb0 2024-05-08T23:27:00.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.847+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a8c0d1170 con 0x7f4a7006ac90 2024-05-08T23:27:00.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.856+0000 7f4a7dffb700 1 -- 172.21.15.5:0/1539692290 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289754 (secure 0 0 0) 0x7f4a88011750 con 0x7f4a7006ac90 2024-05-08T23:27:00.870 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a7006ac90 msgr2=0x7f4a7006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a7006ac90 0x7f4a7006d150 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0b4330 tx=0x7f4a88008040 comp rx=0 tx=0).stop 2024-05-08T23:27:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caeb0 msgr2=0x7f4a8c0b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caeb0 0x7f4a8c0b39f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009c40 tx=0x7f4a80004750 comp rx=0 tx=0).stop 2024-05-08T23:27:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 shutdown_connections 2024-05-08T23:27:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a7006ac90 0x7f4a7006d150 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a8c15db00 0x7f4a8c0b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4a8c0caa50 0x7f4a8c155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 --2- 172.21.15.5:0/1539692290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a8c0caeb0 0x7f4a8c0b39f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 >> 172.21.15.5:0/1539692290 conn(0x7f4a8c099a50 msgr2=0x7f4a8c06f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 shutdown_connections 2024-05-08T23:27:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:00.859+0000 7f4a9337f700 1 -- 172.21.15.5:0/1539692290 wait complete. 2024-05-08T23:27:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:27:00.874 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-05-08T23:26:59.379841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245832,"kb_used_data":9192,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754728888,"statfs":{"total":773094113280,"available":772842381312,"internally_reserved":0,"allocated":9412608,"data_stored":4689708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":242208309},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.652872"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199432+0000","last_change":"2024-05-08T23:26:56.214420+0000","last_active":"2024-05-08T23:26:57.199432+0000","last_peered":"2024-05-08T23:26:57.199432+0000","last_clean":"2024-05-08T23:26:57.199432+0000","last_became_active":"2024-05-08T23:26:56.213393+0000","last_became_peered":"2024-05-08T23:26:56.213393+0000","last_unstale":"2024-05-08T23:26:57.199432+0000","last_undegraded":"2024-05-08T23:26:57.199432+0000","last_fullsized":"2024-05-08T23:26:57.199432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:12.365940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218406+0000","last_change":"2024-05-08T23:26:56.216615+0000","last_active":"2024-05-08T23:26:56.218406+0000","last_peered":"2024-05-08T23:26:56.218406+0000","last_clean":"2024-05-08T23:26:56.218406+0000","last_became_active":"2024-05-08T23:26:56.216204+0000","last_became_peered":"2024-05-08T23:26:56.216204+0000","last_unstale":"2024-05-08T23:26:56.218406+0000","last_undegraded":"2024-05-08T23:26:56.218406+0000","last_fullsized":"2024-05-08T23:26:56.218406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:04.765049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.221044+0000","last_change":"2024-05-08T23:26:56.219331+0000","last_active":"2024-05-08T23:26:56.221044+0000","last_peered":"2024-05-08T23:26:56.221044+0000","last_clean":"2024-05-08T23:26:56.221044+0000","last_became_active":"2024-05-08T23:26:56.219041+0000","last_became_peered":"2024-05-08T23:26:56.219041+0000","last_unstale":"2024-05-08T23:26:56.221044+0000","last_undegraded":"2024-05-08T23:26:56.221044+0000","last_fullsized":"2024-05-08T23:26:56.221044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:08.428124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198116+0000","last_change":"2024-05-08T23:26:56.219524+0000","last_active":"2024-05-08T23:26:57.198116+0000","last_peered":"2024-05-08T23:26:57.198116+0000","last_clean":"2024-05-08T23:26:57.198116+0000","last_became_active":"2024-05-08T23:26:56.218889+0000","last_became_peered":"2024-05-08T23:26:56.218889+0000","last_unstale":"2024-05-08T23:26:57.198116+0000","last_undegraded":"2024-05-08T23:26:57.198116+0000","last_fullsized":"2024-05-08T23:26:57.198116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:45.877888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.221112+0000","last_change":"2024-05-08T23:26:56.219043+0000","last_active":"2024-05-08T23:26:56.221112+0000","last_peered":"2024-05-08T23:26:56.221112+0000","last_clean":"2024-05-08T23:26:56.221112+0000","last_became_active":"2024-05-08T23:26:56.218412+0000","last_became_peered":"2024-05-08T23:26:56.218412+0000","last_unstale":"2024-05-08T23:26:56.221112+0000","last_undegraded":"2024-05-08T23:26:56.221112+0000","last_fullsized":"2024-05-08T23:26:56.221112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:06.646632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216620+0000","last_change":"2024-05-08T23:26:56.213328+0000","last_active":"2024-05-08T23:26:56.216620+0000","last_peered":"2024-05-08T23:26:56.216620+0000","last_clean":"2024-05-08T23:26:56.216620+0000","last_became_active":"2024-05-08T23:26:56.213035+0000","last_became_peered":"2024-05-08T23:26:56.213035+0000","last_unstale":"2024-05-08T23:26:56.216620+0000","last_undegraded":"2024-05-08T23:26:56.216620+0000","last_fullsized":"2024-05-08T23:26:56.216620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:36.414760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197691+0000","last_change":"2024-05-08T23:26:56.210542+0000","last_active":"2024-05-08T23:26:57.197691+0000","last_peered":"2024-05-08T23:26:57.197691+0000","last_clean":"2024-05-08T23:26:57.197691+0000","last_became_active":"2024-05-08T23:26:56.210262+0000","last_became_peered":"2024-05-08T23:26:56.210262+0000","last_unstale":"2024-05-08T23:26:57.197691+0000","last_undegraded":"2024-05-08T23:26:57.197691+0000","last_fullsized":"2024-05-08T23:26:57.197691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:12.290055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218666+0000","last_change":"2024-05-08T23:26:56.217496+0000","last_active":"2024-05-08T23:26:56.218666+0000","last_peered":"2024-05-08T23:26:56.218666+0000","last_clean":"2024-05-08T23:26:56.218666+0000","last_became_active":"2024-05-08T23:26:56.217166+0000","last_became_peered":"2024-05-08T23:26:56.217166+0000","last_unstale":"2024-05-08T23:26:56.218666+0000","last_undegraded":"2024-05-08T23:26:56.218666+0000","last_fullsized":"2024-05-08T23:26:56.218666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:42.595923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216883+0000","last_change":"2024-05-08T23:26:56.213403+0000","last_active":"2024-05-08T23:26:56.216883+0000","last_peered":"2024-05-08T23:26:56.216883+0000","last_clean":"2024-05-08T23:26:56.216883+0000","last_became_active":"2024-05-08T23:26:56.212964+0000","last_became_peered":"2024-05-08T23:26:56.212964+0000","last_unstale":"2024-05-08T23:26:56.216883+0000","last_undegraded":"2024-05-08T23:26:56.216883+0000","last_fullsized":"2024-05-08T23:26:56.216883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:18:40.017440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199007+0000","last_change":"2024-05-08T23:26:56.212567+0000","last_active":"2024-05-08T23:26:57.199007+0000","last_peered":"2024-05-08T23:26:57.199007+0000","last_clean":"2024-05-08T23:26:57.199007+0000","last_became_active":"2024-05-08T23:26:56.212261+0000","last_became_peered":"2024-05-08T23:26:56.212261+0000","last_unstale":"2024-05-08T23:26:57.199007+0000","last_undegraded":"2024-05-08T23:26:57.199007+0000","last_fullsized":"2024-05-08T23:26:57.199007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:42.659945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217539+0000","last_change":"2024-05-08T23:26:56.215606+0000","last_active":"2024-05-08T23:26:56.217539+0000","last_peered":"2024-05-08T23:26:56.217539+0000","last_clean":"2024-05-08T23:26:56.217539+0000","last_became_active":"2024-05-08T23:26:56.215360+0000","last_became_peered":"2024-05-08T23:26:56.215360+0000","last_unstale":"2024-05-08T23:26:56.217539+0000","last_undegraded":"2024-05-08T23:26:56.217539+0000","last_fullsized":"2024-05-08T23:26:56.217539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:00.263085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198570+0000","last_change":"2024-05-08T23:26:56.213744+0000","last_active":"2024-05-08T23:26:57.198570+0000","last_peered":"2024-05-08T23:26:57.198570+0000","last_clean":"2024-05-08T23:26:57.198570+0000","last_became_active":"2024-05-08T23:26:56.213385+0000","last_became_peered":"2024-05-08T23:26:56.213385+0000","last_unstale":"2024-05-08T23:26:57.198570+0000","last_undegraded":"2024-05-08T23:26:57.198570+0000","last_fullsized":"2024-05-08T23:26:57.198570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:08.428124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.215915+0000","last_change":"2024-05-08T23:26:56.209527+0000","last_active":"2024-05-08T23:26:56.215915+0000","last_peered":"2024-05-08T23:26:56.215915+0000","last_clean":"2024-05-08T23:26:56.215915+0000","last_became_active":"2024-05-08T23:26:56.209212+0000","last_became_peered":"2024-05-08T23:26:56.209212+0000","last_unstale":"2024-05-08T23:26:56.215915+0000","last_undegraded":"2024-05-08T23:26:56.215915+0000","last_fullsized":"2024-05-08T23:26:56.215915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.645733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197970+0000","last_change":"2024-05-08T23:26:56.220423+0000","last_active":"2024-05-08T23:26:57.197970+0000","last_peered":"2024-05-08T23:26:57.197970+0000","last_clean":"2024-05-08T23:26:57.197970+0000","last_became_active":"2024-05-08T23:26:56.219825+0000","last_became_peered":"2024-05-08T23:26:56.219825+0000","last_unstale":"2024-05-08T23:26:57.197970+0000","last_undegraded":"2024-05-08T23:26:57.197970+0000","last_fullsized":"2024-05-08T23:26:57.197970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:20.834836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.196769+0000","last_change":"2024-05-08T23:26:56.200689+0000","last_active":"2024-05-08T23:26:57.196769+0000","last_peered":"2024-05-08T23:26:57.196769+0000","last_clean":"2024-05-08T23:26:57.196769+0000","last_became_active":"2024-05-08T23:26:56.200395+0000","last_became_peered":"2024-05-08T23:26:56.200395+0000","last_unstale":"2024-05-08T23:26:57.196769+0000","last_undegraded":"2024-05-08T23:26:57.196769+0000","last_fullsized":"2024-05-08T23:26:57.196769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:21.483501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219794+0000","last_change":"2024-05-08T23:26:56.214376+0000","last_active":"2024-05-08T23:26:56.219794+0000","last_peered":"2024-05-08T23:26:56.219794+0000","last_clean":"2024-05-08T23:26:56.219794+0000","last_became_active":"2024-05-08T23:26:56.214093+0000","last_became_peered":"2024-05-08T23:26:56.214093+0000","last_unstale":"2024-05-08T23:26:56.219794+0000","last_undegraded":"2024-05-08T23:26:56.219794+0000","last_fullsized":"2024-05-08T23:26:56.219794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.645733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218069+0000","last_change":"2024-05-08T23:26:56.211972+0000","last_active":"2024-05-08T23:26:56.218069+0000","last_peered":"2024-05-08T23:26:56.218069+0000","last_clean":"2024-05-08T23:26:56.218069+0000","last_became_active":"2024-05-08T23:26:56.211691+0000","last_became_peered":"2024-05-08T23:26:56.211691+0000","last_unstale":"2024-05-08T23:26:56.218069+0000","last_undegraded":"2024-05-08T23:26:56.218069+0000","last_fullsized":"2024-05-08T23:26:56.218069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:13.472165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198300+0000","last_change":"2024-05-08T23:26:56.219800+0000","last_active":"2024-05-08T23:26:57.198300+0000","last_peered":"2024-05-08T23:26:57.198300+0000","last_clean":"2024-05-08T23:26:57.198300+0000","last_became_active":"2024-05-08T23:26:56.216894+0000","last_became_peered":"2024-05-08T23:26:56.216894+0000","last_unstale":"2024-05-08T23:26:57.198300+0000","last_undegraded":"2024-05-08T23:26:57.198300+0000","last_fullsized":"2024-05-08T23:26:57.198300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:53.396253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.225231+0000","last_change":"2024-05-08T23:26:56.211039+0000","last_active":"2024-05-08T23:26:56.225231+0000","last_peered":"2024-05-08T23:26:56.225231+0000","last_clean":"2024-05-08T23:26:56.225231+0000","last_became_active":"2024-05-08T23:26:56.210793+0000","last_became_peered":"2024-05-08T23:26:56.210793+0000","last_unstale":"2024-05-08T23:26:56.225231+0000","last_undegraded":"2024-05-08T23:26:56.225231+0000","last_fullsized":"2024-05-08T23:26:56.225231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:16.059931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198101+0000","last_change":"2024-05-08T23:26:56.219317+0000","last_active":"2024-05-08T23:26:57.198101+0000","last_peered":"2024-05-08T23:26:57.198101+0000","last_clean":"2024-05-08T23:26:57.198101+0000","last_became_active":"2024-05-08T23:26:56.215939+0000","last_became_peered":"2024-05-08T23:26:56.215939+0000","last_unstale":"2024-05-08T23:26:57.198101+0000","last_undegraded":"2024-05-08T23:26:57.198101+0000","last_fullsized":"2024-05-08T23:26:57.198101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:01.907735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198116+0000","last_change":"2024-05-08T23:26:56.218433+0000","last_active":"2024-05-08T23:26:57.198116+0000","last_peered":"2024-05-08T23:26:57.198116+0000","last_clean":"2024-05-08T23:26:57.198116+0000","last_became_active":"2024-05-08T23:26:56.213487+0000","last_became_peered":"2024-05-08T23:26:56.213487+0000","last_unstale":"2024-05-08T23:26:57.198116+0000","last_undegraded":"2024-05-08T23:26:57.198116+0000","last_fullsized":"2024-05-08T23:26:57.198116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:51.459201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.225138+0000","last_change":"2024-05-08T23:26:56.224183+0000","last_active":"2024-05-08T23:26:56.225138+0000","last_peered":"2024-05-08T23:26:56.225138+0000","last_clean":"2024-05-08T23:26:56.225138+0000","last_became_active":"2024-05-08T23:26:56.223847+0000","last_became_peered":"2024-05-08T23:26:56.223847+0000","last_unstale":"2024-05-08T23:26:56.225138+0000","last_undegraded":"2024-05-08T23:26:56.225138+0000","last_fullsized":"2024-05-08T23:26:56.225138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:19.238077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198153+0000","last_change":"2024-05-08T23:26:56.216039+0000","last_active":"2024-05-08T23:26:57.198153+0000","last_peered":"2024-05-08T23:26:57.198153+0000","last_clean":"2024-05-08T23:26:57.198153+0000","last_became_active":"2024-05-08T23:26:56.215113+0000","last_became_peered":"2024-05-08T23:26:56.215113+0000","last_unstale":"2024-05-08T23:26:57.198153+0000","last_undegraded":"2024-05-08T23:26:57.198153+0000","last_fullsized":"2024-05-08T23:26:57.198153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:42.595923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197844+0000","last_change":"2024-05-08T23:26:56.218951+0000","last_active":"2024-05-08T23:26:57.197844+0000","last_peered":"2024-05-08T23:26:57.197844+0000","last_clean":"2024-05-08T23:26:57.197844+0000","last_became_active":"2024-05-08T23:26:56.215106+0000","last_became_peered":"2024-05-08T23:26:56.215106+0000","last_unstale":"2024-05-08T23:26:57.197844+0000","last_undegraded":"2024-05-08T23:26:57.197844+0000","last_fullsized":"2024-05-08T23:26:57.197844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:41.139231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224567+0000","last_change":"2024-05-08T23:26:56.222512+0000","last_active":"2024-05-08T23:26:56.224567+0000","last_peered":"2024-05-08T23:26:56.224567+0000","last_clean":"2024-05-08T23:26:56.224567+0000","last_became_active":"2024-05-08T23:26:56.222221+0000","last_became_peered":"2024-05-08T23:26:56.222221+0000","last_unstale":"2024-05-08T23:26:56.224567+0000","last_undegraded":"2024-05-08T23:26:56.224567+0000","last_fullsized":"2024-05-08T23:26:56.224567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:31.227682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.220982+0000","last_change":"2024-05-08T23:26:56.219173+0000","last_active":"2024-05-08T23:26:56.220982+0000","last_peered":"2024-05-08T23:26:56.220982+0000","last_clean":"2024-05-08T23:26:56.220982+0000","last_became_active":"2024-05-08T23:26:56.218825+0000","last_became_peered":"2024-05-08T23:26:56.218825+0000","last_unstale":"2024-05-08T23:26:56.220982+0000","last_undegraded":"2024-05-08T23:26:56.220982+0000","last_fullsized":"2024-05-08T23:26:56.220982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:16.059931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217327+0000","last_change":"2024-05-08T23:26:56.212484+0000","last_active":"2024-05-08T23:26:56.217327+0000","last_peered":"2024-05-08T23:26:56.217327+0000","last_clean":"2024-05-08T23:26:56.217327+0000","last_became_active":"2024-05-08T23:26:56.212198+0000","last_became_peered":"2024-05-08T23:26:56.212198+0000","last_unstale":"2024-05-08T23:26:56.217327+0000","last_undegraded":"2024-05-08T23:26:56.217327+0000","last_fullsized":"2024-05-08T23:26:56.217327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:08.428124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198327+0000","last_change":"2024-05-08T23:26:56.214021+0000","last_active":"2024-05-08T23:26:57.198327+0000","last_peered":"2024-05-08T23:26:57.198327+0000","last_clean":"2024-05-08T23:26:57.198327+0000","last_became_active":"2024-05-08T23:26:56.213690+0000","last_became_peered":"2024-05-08T23:26:56.213690+0000","last_unstale":"2024-05-08T23:26:57.198327+0000","last_undegraded":"2024-05-08T23:26:57.198327+0000","last_fullsized":"2024-05-08T23:26:57.198327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.523398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217546+0000","last_change":"2024-05-08T23:26:56.214854+0000","last_active":"2024-05-08T23:26:56.217546+0000","last_peered":"2024-05-08T23:26:56.217546+0000","last_clean":"2024-05-08T23:26:56.217546+0000","last_became_active":"2024-05-08T23:26:56.214553+0000","last_became_peered":"2024-05-08T23:26:56.214553+0000","last_unstale":"2024-05-08T23:26:56.217546+0000","last_undegraded":"2024-05-08T23:26:56.217546+0000","last_fullsized":"2024-05-08T23:26:56.217546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:51.459201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.222697+0000","last_change":"2024-05-08T23:26:56.217991+0000","last_active":"2024-05-08T23:26:56.222697+0000","last_peered":"2024-05-08T23:26:56.222697+0000","last_clean":"2024-05-08T23:26:56.222697+0000","last_became_active":"2024-05-08T23:26:56.217716+0000","last_became_peered":"2024-05-08T23:26:56.217716+0000","last_unstale":"2024-05-08T23:26:56.222697+0000","last_undegraded":"2024-05-08T23:26:56.222697+0000","last_fullsized":"2024-05-08T23:26:56.222697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:16.287626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219666+0000","last_change":"2024-05-08T23:26:56.214607+0000","last_active":"2024-05-08T23:26:56.219666+0000","last_peered":"2024-05-08T23:26:56.219666+0000","last_clean":"2024-05-08T23:26:56.219666+0000","last_became_active":"2024-05-08T23:26:56.213810+0000","last_became_peered":"2024-05-08T23:26:56.213810+0000","last_unstale":"2024-05-08T23:26:56.219666+0000","last_undegraded":"2024-05-08T23:26:56.219666+0000","last_fullsized":"2024-05-08T23:26:56.219666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:25.098408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198060+0000","last_change":"2024-05-08T23:26:56.217811+0000","last_active":"2024-05-08T23:26:57.198060+0000","last_peered":"2024-05-08T23:26:57.198060+0000","last_clean":"2024-05-08T23:26:57.198060+0000","last_became_active":"2024-05-08T23:26:56.217395+0000","last_became_peered":"2024-05-08T23:26:56.217395+0000","last_unstale":"2024-05-08T23:26:57.198060+0000","last_undegraded":"2024-05-08T23:26:57.198060+0000","last_fullsized":"2024-05-08T23:26:57.198060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:13.472165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.221147+0000","last_change":"2024-05-08T23:26:56.219192+0000","last_active":"2024-05-08T23:26:56.221147+0000","last_peered":"2024-05-08T23:26:56.221147+0000","last_clean":"2024-05-08T23:26:56.221147+0000","last_became_active":"2024-05-08T23:26:56.218584+0000","last_became_peered":"2024-05-08T23:26:56.218584+0000","last_unstale":"2024-05-08T23:26:56.221147+0000","last_undegraded":"2024-05-08T23:26:56.221147+0000","last_fullsized":"2024-05-08T23:26:56.221147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:47.211615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199260+0000","last_change":"2024-05-08T23:26:55.755286+0000","last_active":"2024-05-08T23:26:57.199260+0000","last_peered":"2024-05-08T23:26:57.199260+0000","last_clean":"2024-05-08T23:26:57.199260+0000","last_became_active":"2024-05-08T23:26:55.755019+0000","last_became_peered":"2024-05-08T23:26:55.755019+0000","last_unstale":"2024-05-08T23:26:57.199260+0000","last_undegraded":"2024-05-08T23:26:57.199260+0000","last_fullsized":"2024-05-08T23:26:57.199260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:49.355512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199341+0000","last_change":"2024-05-08T23:26:56.214888+0000","last_active":"2024-05-08T23:26:57.199341+0000","last_peered":"2024-05-08T23:26:57.199341+0000","last_clean":"2024-05-08T23:26:57.199341+0000","last_became_active":"2024-05-08T23:26:56.214041+0000","last_became_peered":"2024-05-08T23:26:56.214041+0000","last_unstale":"2024-05-08T23:26:57.199341+0000","last_undegraded":"2024-05-08T23:26:57.199341+0000","last_fullsized":"2024-05-08T23:26:57.199341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:48.011872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.211279+0000","last_change":"2024-05-08T23:26:55.752137+0000","last_active":"2024-05-08T23:26:56.211279+0000","last_peered":"2024-05-08T23:26:56.211279+0000","last_clean":"2024-05-08T23:26:56.211279+0000","last_became_active":"2024-05-08T23:26:55.750793+0000","last_became_peered":"2024-05-08T23:26:55.750793+0000","last_unstale":"2024-05-08T23:26:56.211279+0000","last_undegraded":"2024-05-08T23:26:56.211279+0000","last_fullsized":"2024-05-08T23:26:56.211279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:02.909989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198433+0000","last_change":"2024-05-08T23:26:56.222787+0000","last_active":"2024-05-08T23:26:57.198433+0000","last_peered":"2024-05-08T23:26:57.198433+0000","last_clean":"2024-05-08T23:26:57.198433+0000","last_became_active":"2024-05-08T23:26:56.222614+0000","last_became_peered":"2024-05-08T23:26:56.222614+0000","last_unstale":"2024-05-08T23:26:57.198433+0000","last_undegraded":"2024-05-08T23:26:57.198433+0000","last_fullsized":"2024-05-08T23:26:57.198433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:22.981805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.229302+0000","last_change":"2024-05-08T23:26:55.754952+0000","last_active":"2024-05-08T23:26:56.229302+0000","last_peered":"2024-05-08T23:26:56.229302+0000","last_clean":"2024-05-08T23:26:56.229302+0000","last_became_active":"2024-05-08T23:26:55.754630+0000","last_became_peered":"2024-05-08T23:26:55.754630+0000","last_unstale":"2024-05-08T23:26:56.229302+0000","last_undegraded":"2024-05-08T23:26:56.229302+0000","last_fullsized":"2024-05-08T23:26:56.229302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:41.590210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217846+0000","last_change":"2024-05-08T23:26:56.215555+0000","last_active":"2024-05-08T23:26:56.217846+0000","last_peered":"2024-05-08T23:26:56.217846+0000","last_clean":"2024-05-08T23:26:56.217846+0000","last_became_active":"2024-05-08T23:26:56.215210+0000","last_became_peered":"2024-05-08T23:26:56.215210+0000","last_unstale":"2024-05-08T23:26:56.217846+0000","last_undegraded":"2024-05-08T23:26:56.217846+0000","last_fullsized":"2024-05-08T23:26:56.217846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:06.646632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.229207+0000","last_change":"2024-05-08T23:26:55.746957+0000","last_active":"2024-05-08T23:26:56.229207+0000","last_peered":"2024-05-08T23:26:56.229207+0000","last_clean":"2024-05-08T23:26:56.229207+0000","last_became_active":"2024-05-08T23:26:55.746643+0000","last_became_peered":"2024-05-08T23:26:55.746643+0000","last_unstale":"2024-05-08T23:26:56.229207+0000","last_undegraded":"2024-05-08T23:26:56.229207+0000","last_fullsized":"2024-05-08T23:26:56.229207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:01.593460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218123+0000","last_change":"2024-05-08T23:26:56.215663+0000","last_active":"2024-05-08T23:26:56.218123+0000","last_peered":"2024-05-08T23:26:56.218123+0000","last_clean":"2024-05-08T23:26:56.218123+0000","last_became_active":"2024-05-08T23:26:56.214686+0000","last_became_peered":"2024-05-08T23:26:56.214686+0000","last_unstale":"2024-05-08T23:26:56.218123+0000","last_undegraded":"2024-05-08T23:26:56.218123+0000","last_fullsized":"2024-05-08T23:26:56.218123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:36.048412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218479+0000","last_change":"2024-05-08T23:26:55.749375+0000","last_active":"2024-05-08T23:26:56.218479+0000","last_peered":"2024-05-08T23:26:56.218479+0000","last_clean":"2024-05-08T23:26:56.218479+0000","last_became_active":"2024-05-08T23:26:55.749066+0000","last_became_peered":"2024-05-08T23:26:55.749066+0000","last_unstale":"2024-05-08T23:26:56.218479+0000","last_undegraded":"2024-05-08T23:26:56.218479+0000","last_fullsized":"2024-05-08T23:26:56.218479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:55.744185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217764+0000","last_change":"2024-05-08T23:26:56.212898+0000","last_active":"2024-05-08T23:26:56.217764+0000","last_peered":"2024-05-08T23:26:56.217764+0000","last_clean":"2024-05-08T23:26:56.217764+0000","last_became_active":"2024-05-08T23:26:56.212567+0000","last_became_peered":"2024-05-08T23:26:56.212567+0000","last_unstale":"2024-05-08T23:26:56.217764+0000","last_undegraded":"2024-05-08T23:26:56.217764+0000","last_fullsized":"2024-05-08T23:26:56.217764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:22.981805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.210861+0000","last_change":"2024-05-08T23:26:55.751792+0000","last_active":"2024-05-08T23:26:56.210861+0000","last_peered":"2024-05-08T23:26:56.210861+0000","last_clean":"2024-05-08T23:26:56.210861+0000","last_became_active":"2024-05-08T23:26:55.751285+0000","last_became_peered":"2024-05-08T23:26:55.751285+0000","last_unstale":"2024-05-08T23:26:56.210861+0000","last_undegraded":"2024-05-08T23:26:56.210861+0000","last_fullsized":"2024-05-08T23:26:56.210861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:22.459643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217137+0000","last_change":"2024-05-08T23:26:56.211928+0000","last_active":"2024-05-08T23:26:56.217137+0000","last_peered":"2024-05-08T23:26:56.217137+0000","last_clean":"2024-05-08T23:26:56.217137+0000","last_became_active":"2024-05-08T23:26:56.211636+0000","last_became_peered":"2024-05-08T23:26:56.211636+0000","last_unstale":"2024-05-08T23:26:56.217137+0000","last_undegraded":"2024-05-08T23:26:56.217137+0000","last_fullsized":"2024-05-08T23:26:56.217137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.523398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.202869+0000","last_change":"2024-05-08T23:26:55.744642+0000","last_active":"2024-05-08T23:26:56.202869+0000","last_peered":"2024-05-08T23:26:56.202869+0000","last_clean":"2024-05-08T23:26:56.202869+0000","last_became_active":"2024-05-08T23:26:55.742524+0000","last_became_peered":"2024-05-08T23:26:55.742524+0000","last_unstale":"2024-05-08T23:26:56.202869+0000","last_undegraded":"2024-05-08T23:26:56.202869+0000","last_fullsized":"2024-05-08T23:26:56.202869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:42.732425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.196870+0000","last_change":"2024-05-08T23:26:56.201640+0000","last_active":"2024-05-08T23:26:57.196870+0000","last_peered":"2024-05-08T23:26:57.196870+0000","last_clean":"2024-05-08T23:26:57.196870+0000","last_became_active":"2024-05-08T23:26:56.201363+0000","last_became_peered":"2024-05-08T23:26:56.201363+0000","last_unstale":"2024-05-08T23:26:57.196870+0000","last_undegraded":"2024-05-08T23:26:57.196870+0000","last_fullsized":"2024-05-08T23:26:57.196870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.876263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.207492+0000","last_change":"2024-05-08T23:26:55.748872+0000","last_active":"2024-05-08T23:26:56.207492+0000","last_peered":"2024-05-08T23:26:56.207492+0000","last_clean":"2024-05-08T23:26:56.207492+0000","last_became_active":"2024-05-08T23:26:55.748542+0000","last_became_peered":"2024-05-08T23:26:55.748542+0000","last_unstale":"2024-05-08T23:26:56.207492+0000","last_undegraded":"2024-05-08T23:26:56.207492+0000","last_fullsized":"2024-05-08T23:26:56.207492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:08.275297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197425+0000","last_change":"2024-05-08T23:26:56.217280+0000","last_active":"2024-05-08T23:26:57.197425+0000","last_peered":"2024-05-08T23:26:57.197425+0000","last_clean":"2024-05-08T23:26:57.197425+0000","last_became_active":"2024-05-08T23:26:56.216956+0000","last_became_peered":"2024-05-08T23:26:56.216956+0000","last_unstale":"2024-05-08T23:26:57.197425+0000","last_undegraded":"2024-05-08T23:26:57.197425+0000","last_fullsized":"2024-05-08T23:26:57.197425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.771918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199218+0000","last_change":"2024-05-08T23:26:55.753320+0000","last_active":"2024-05-08T23:26:57.199218+0000","last_peered":"2024-05-08T23:26:57.199218+0000","last_clean":"2024-05-08T23:26:57.199218+0000","last_became_active":"2024-05-08T23:26:55.752922+0000","last_became_peered":"2024-05-08T23:26:55.752922+0000","last_unstale":"2024-05-08T23:26:57.199218+0000","last_undegraded":"2024-05-08T23:26:57.199218+0000","last_fullsized":"2024-05-08T23:26:57.199218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:33.504310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219355+0000","last_change":"2024-05-08T23:26:56.215783+0000","last_active":"2024-05-08T23:26:56.219355+0000","last_peered":"2024-05-08T23:26:56.219355+0000","last_clean":"2024-05-08T23:26:56.219355+0000","last_became_active":"2024-05-08T23:26:56.213244+0000","last_became_peered":"2024-05-08T23:26:56.213244+0000","last_unstale":"2024-05-08T23:26:56.219355+0000","last_undegraded":"2024-05-08T23:26:56.219355+0000","last_fullsized":"2024-05-08T23:26:56.219355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:08.846422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199108+0000","last_change":"2024-05-08T23:26:55.750296+0000","last_active":"2024-05-08T23:26:57.199108+0000","last_peered":"2024-05-08T23:26:57.199108+0000","last_clean":"2024-05-08T23:26:57.199108+0000","last_became_active":"2024-05-08T23:26:55.749975+0000","last_became_peered":"2024-05-08T23:26:55.749975+0000","last_unstale":"2024-05-08T23:26:57.199108+0000","last_undegraded":"2024-05-08T23:26:57.199108+0000","last_fullsized":"2024-05-08T23:26:57.199108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:08.997540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.226765+0000","last_change":"2024-05-08T23:26:56.225944+0000","last_active":"2024-05-08T23:26:56.226765+0000","last_peered":"2024-05-08T23:26:56.226765+0000","last_clean":"2024-05-08T23:26:56.226765+0000","last_became_active":"2024-05-08T23:26:56.225653+0000","last_became_peered":"2024-05-08T23:26:56.225653+0000","last_unstale":"2024-05-08T23:26:56.226765+0000","last_undegraded":"2024-05-08T23:26:56.226765+0000","last_fullsized":"2024-05-08T23:26:56.226765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:13.472165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198830+0000","last_change":"2024-05-08T23:26:55.754964+0000","last_active":"2024-05-08T23:26:57.198830+0000","last_peered":"2024-05-08T23:26:57.198830+0000","last_clean":"2024-05-08T23:26:57.198830+0000","last_became_active":"2024-05-08T23:26:55.754617+0000","last_became_peered":"2024-05-08T23:26:55.754617+0000","last_unstale":"2024-05-08T23:26:57.198830+0000","last_undegraded":"2024-05-08T23:26:57.198830+0000","last_fullsized":"2024-05-08T23:26:57.198830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:23.650422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.226161+0000","last_change":"2024-05-08T23:26:56.225752+0000","last_active":"2024-05-08T23:26:56.226161+0000","last_peered":"2024-05-08T23:26:56.226161+0000","last_clean":"2024-05-08T23:26:56.226161+0000","last_became_active":"2024-05-08T23:26:56.225459+0000","last_became_peered":"2024-05-08T23:26:56.225459+0000","last_unstale":"2024-05-08T23:26:56.226161+0000","last_undegraded":"2024-05-08T23:26:56.226161+0000","last_fullsized":"2024-05-08T23:26:56.226161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.523398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224128+0000","last_change":"2024-05-08T23:26:55.751568+0000","last_active":"2024-05-08T23:26:56.224128+0000","last_peered":"2024-05-08T23:26:56.224128+0000","last_clean":"2024-05-08T23:26:56.224128+0000","last_became_active":"2024-05-08T23:26:55.751290+0000","last_became_peered":"2024-05-08T23:26:55.751290+0000","last_unstale":"2024-05-08T23:26:56.224128+0000","last_undegraded":"2024-05-08T23:26:56.224128+0000","last_fullsized":"2024-05-08T23:26:56.224128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:08.997540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224452+0000","last_change":"2024-05-08T23:26:56.222628+0000","last_active":"2024-05-08T23:26:56.224452+0000","last_peered":"2024-05-08T23:26:56.224452+0000","last_clean":"2024-05-08T23:26:56.224452+0000","last_became_active":"2024-05-08T23:26:56.222380+0000","last_became_peered":"2024-05-08T23:26:56.222380+0000","last_unstale":"2024-05-08T23:26:56.224452+0000","last_undegraded":"2024-05-08T23:26:56.224452+0000","last_fullsized":"2024-05-08T23:26:56.224452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:25.098408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224869+0000","last_change":"2024-05-08T23:26:55.748991+0000","last_active":"2024-05-08T23:26:56.224869+0000","last_peered":"2024-05-08T23:26:56.224869+0000","last_clean":"2024-05-08T23:26:56.224869+0000","last_became_active":"2024-05-08T23:26:55.748691+0000","last_became_peered":"2024-05-08T23:26:55.748691+0000","last_unstale":"2024-05-08T23:26:56.224869+0000","last_undegraded":"2024-05-08T23:26:56.224869+0000","last_fullsized":"2024-05-08T23:26:56.224869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:29.409557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216895+0000","last_change":"2024-05-08T23:26:56.214475+0000","last_active":"2024-05-08T23:26:56.216895+0000","last_peered":"2024-05-08T23:26:56.216895+0000","last_clean":"2024-05-08T23:26:56.216895+0000","last_became_active":"2024-05-08T23:26:56.214164+0000","last_became_peered":"2024-05-08T23:26:56.214164+0000","last_unstale":"2024-05-08T23:26:56.216895+0000","last_undegraded":"2024-05-08T23:26:56.216895+0000","last_fullsized":"2024-05-08T23:26:56.216895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:25.098408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197110+0000","last_change":"2024-05-08T23:26:55.746305+0000","last_active":"2024-05-08T23:26:57.197110+0000","last_peered":"2024-05-08T23:26:57.197110+0000","last_clean":"2024-05-08T23:26:57.197110+0000","last_became_active":"2024-05-08T23:26:55.746010+0000","last_became_peered":"2024-05-08T23:26:55.746010+0000","last_unstale":"2024-05-08T23:26:57.197110+0000","last_undegraded":"2024-05-08T23:26:57.197110+0000","last_fullsized":"2024-05-08T23:26:57.197110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:46.205902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218482+0000","last_change":"2024-05-08T23:26:56.216764+0000","last_active":"2024-05-08T23:26:56.218482+0000","last_peered":"2024-05-08T23:26:56.218482+0000","last_clean":"2024-05-08T23:26:56.218482+0000","last_became_active":"2024-05-08T23:26:56.216273+0000","last_became_peered":"2024-05-08T23:26:56.216273+0000","last_unstale":"2024-05-08T23:26:56.218482+0000","last_undegraded":"2024-05-08T23:26:56.218482+0000","last_fullsized":"2024-05-08T23:26:56.218482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:25.496589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.201651+0000","last_change":"2024-05-08T23:26:55.747631+0000","last_active":"2024-05-08T23:26:56.201651+0000","last_peered":"2024-05-08T23:26:56.201651+0000","last_clean":"2024-05-08T23:26:56.201651+0000","last_became_active":"2024-05-08T23:26:55.747338+0000","last_became_peered":"2024-05-08T23:26:55.747338+0000","last_unstale":"2024-05-08T23:26:56.201651+0000","last_undegraded":"2024-05-08T23:26:56.201651+0000","last_fullsized":"2024-05-08T23:26:56.201651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:47.770925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.218454+0000","last_change":"2024-05-08T23:26:56.217063+0000","last_active":"2024-05-08T23:26:56.218454+0000","last_peered":"2024-05-08T23:26:56.218454+0000","last_clean":"2024-05-08T23:26:56.218454+0000","last_became_active":"2024-05-08T23:26:56.216798+0000","last_became_peered":"2024-05-08T23:26:56.216798+0000","last_unstale":"2024-05-08T23:26:56.218454+0000","last_undegraded":"2024-05-08T23:26:56.218454+0000","last_fullsized":"2024-05-08T23:26:56.218454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.216913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.226721+0000","last_change":"2024-05-08T23:26:55.749893+0000","last_active":"2024-05-08T23:26:56.226721+0000","last_peered":"2024-05-08T23:26:56.226721+0000","last_clean":"2024-05-08T23:26:56.226721+0000","last_became_active":"2024-05-08T23:26:55.749596+0000","last_became_peered":"2024-05-08T23:26:55.749596+0000","last_unstale":"2024-05-08T23:26:56.226721+0000","last_undegraded":"2024-05-08T23:26:56.226721+0000","last_fullsized":"2024-05-08T23:26:56.226721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:35.409047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.208643+0000","last_change":"2024-05-08T23:26:56.204232+0000","last_active":"2024-05-08T23:26:56.208643+0000","last_peered":"2024-05-08T23:26:56.208643+0000","last_clean":"2024-05-08T23:26:56.208643+0000","last_became_active":"2024-05-08T23:26:56.203932+0000","last_became_peered":"2024-05-08T23:26:56.203932+0000","last_unstale":"2024-05-08T23:26:56.208643+0000","last_undegraded":"2024-05-08T23:26:56.208643+0000","last_fullsized":"2024-05-08T23:26:56.208643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:32.927014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199057+0000","last_change":"2024-05-08T23:26:55.747903+0000","last_active":"2024-05-08T23:26:57.199057+0000","last_peered":"2024-05-08T23:26:57.199057+0000","last_clean":"2024-05-08T23:26:57.199057+0000","last_became_active":"2024-05-08T23:26:55.747625+0000","last_became_peered":"2024-05-08T23:26:55.747625+0000","last_unstale":"2024-05-08T23:26:57.199057+0000","last_undegraded":"2024-05-08T23:26:57.199057+0000","last_fullsized":"2024-05-08T23:26:57.199057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:47.770925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.199324+0000","last_change":"2024-05-08T23:26:56.217310+0000","last_active":"2024-05-08T23:26:57.199324+0000","last_peered":"2024-05-08T23:26:57.199324+0000","last_clean":"2024-05-08T23:26:57.199324+0000","last_became_active":"2024-05-08T23:26:56.217029+0000","last_became_peered":"2024-05-08T23:26:56.217029+0000","last_unstale":"2024-05-08T23:26:57.199324+0000","last_undegraded":"2024-05-08T23:26:57.199324+0000","last_fullsized":"2024-05-08T23:26:57.199324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:06.516103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224468+0000","last_change":"2024-05-08T23:26:55.749736+0000","last_active":"2024-05-08T23:26:56.224468+0000","last_peered":"2024-05-08T23:26:56.224468+0000","last_clean":"2024-05-08T23:26:56.224468+0000","last_became_active":"2024-05-08T23:26:55.749441+0000","last_became_peered":"2024-05-08T23:26:55.749441+0000","last_unstale":"2024-05-08T23:26:56.224468+0000","last_undegraded":"2024-05-08T23:26:56.224468+0000","last_fullsized":"2024-05-08T23:26:56.224468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:07.840709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.220478+0000","last_change":"2024-05-08T23:26:56.218056+0000","last_active":"2024-05-08T23:26:56.220478+0000","last_peered":"2024-05-08T23:26:56.220478+0000","last_clean":"2024-05-08T23:26:56.220478+0000","last_became_active":"2024-05-08T23:26:56.217773+0000","last_became_peered":"2024-05-08T23:26:56.217773+0000","last_unstale":"2024-05-08T23:26:56.220478+0000","last_undegraded":"2024-05-08T23:26:56.220478+0000","last_fullsized":"2024-05-08T23:26:56.220478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:16.059931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.207998+0000","last_change":"2024-05-08T23:26:55.742485+0000","last_active":"2024-05-08T23:26:56.207998+0000","last_peered":"2024-05-08T23:26:56.207998+0000","last_clean":"2024-05-08T23:26:56.207998+0000","last_became_active":"2024-05-08T23:26:55.742154+0000","last_became_peered":"2024-05-08T23:26:55.742154+0000","last_unstale":"2024-05-08T23:26:56.207998+0000","last_undegraded":"2024-05-08T23:26:56.207998+0000","last_fullsized":"2024-05-08T23:26:56.207998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:05.860831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197970+0000","last_change":"2024-05-08T23:26:56.220214+0000","last_active":"2024-05-08T23:26:57.197970+0000","last_peered":"2024-05-08T23:26:57.197970+0000","last_clean":"2024-05-08T23:26:57.197970+0000","last_became_active":"2024-05-08T23:26:56.219802+0000","last_became_peered":"2024-05-08T23:26:56.219802+0000","last_unstale":"2024-05-08T23:26:57.197970+0000","last_undegraded":"2024-05-08T23:26:57.197970+0000","last_fullsized":"2024-05-08T23:26:57.197970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:41.680125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.196984+0000","last_change":"2024-05-08T23:26:55.755256+0000","last_active":"2024-05-08T23:26:57.196984+0000","last_peered":"2024-05-08T23:26:57.196984+0000","last_clean":"2024-05-08T23:26:57.196984+0000","last_became_active":"2024-05-08T23:26:55.754980+0000","last_became_peered":"2024-05-08T23:26:55.754980+0000","last_unstale":"2024-05-08T23:26:57.196984+0000","last_undegraded":"2024-05-08T23:26:57.196984+0000","last_fullsized":"2024-05-08T23:26:57.196984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:34.570523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216613+0000","last_change":"2024-05-08T23:26:56.212708+0000","last_active":"2024-05-08T23:26:56.216613+0000","last_peered":"2024-05-08T23:26:56.216613+0000","last_clean":"2024-05-08T23:26:56.216613+0000","last_became_active":"2024-05-08T23:26:56.211885+0000","last_became_peered":"2024-05-08T23:26:56.211885+0000","last_unstale":"2024-05-08T23:26:56.216613+0000","last_undegraded":"2024-05-08T23:26:56.216613+0000","last_fullsized":"2024-05-08T23:26:56.216613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:06.516103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198090+0000","last_change":"2024-05-08T23:26:55.752322+0000","last_active":"2024-05-08T23:26:57.198090+0000","last_peered":"2024-05-08T23:26:57.198090+0000","last_clean":"2024-05-08T23:26:57.198090+0000","last_became_active":"2024-05-08T23:26:55.752041+0000","last_became_peered":"2024-05-08T23:26:55.752041+0000","last_unstale":"2024-05-08T23:26:57.198090+0000","last_undegraded":"2024-05-08T23:26:57.198090+0000","last_fullsized":"2024-05-08T23:26:57.198090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:03.229647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197691+0000","last_change":"2024-05-08T23:26:56.210610+0000","last_active":"2024-05-08T23:26:57.197691+0000","last_peered":"2024-05-08T23:26:57.197691+0000","last_clean":"2024-05-08T23:26:57.197691+0000","last_became_active":"2024-05-08T23:26:56.210287+0000","last_became_peered":"2024-05-08T23:26:56.210287+0000","last_unstale":"2024-05-08T23:26:57.197691+0000","last_undegraded":"2024-05-08T23:26:57.197691+0000","last_fullsized":"2024-05-08T23:26:57.197691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:52.123872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.197792+0000","last_change":"2024-05-08T23:26:55.748394+0000","last_active":"2024-05-08T23:26:57.197792+0000","last_peered":"2024-05-08T23:26:57.197792+0000","last_clean":"2024-05-08T23:26:57.197792+0000","last_became_active":"2024-05-08T23:26:55.748139+0000","last_became_peered":"2024-05-08T23:26:55.748139+0000","last_unstale":"2024-05-08T23:26:57.197792+0000","last_undegraded":"2024-05-08T23:26:57.197792+0000","last_fullsized":"2024-05-08T23:26:57.197792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:58.003200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219665+0000","last_change":"2024-05-08T23:26:56.215669+0000","last_active":"2024-05-08T23:26:56.219665+0000","last_peered":"2024-05-08T23:26:56.219665+0000","last_clean":"2024-05-08T23:26:56.219665+0000","last_became_active":"2024-05-08T23:26:56.215345+0000","last_became_peered":"2024-05-08T23:26:56.215345+0000","last_unstale":"2024-05-08T23:26:56.219665+0000","last_undegraded":"2024-05-08T23:26:56.219665+0000","last_fullsized":"2024-05-08T23:26:56.219665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:48.011872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.196756+0000","last_change":"2024-05-08T23:26:55.753049+0000","last_active":"2024-05-08T23:26:57.196756+0000","last_peered":"2024-05-08T23:26:57.196756+0000","last_clean":"2024-05-08T23:26:57.196756+0000","last_became_active":"2024-05-08T23:26:55.752756+0000","last_became_peered":"2024-05-08T23:26:55.752756+0000","last_unstale":"2024-05-08T23:26:57.196756+0000","last_undegraded":"2024-05-08T23:26:57.196756+0000","last_fullsized":"2024-05-08T23:26:57.196756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.640021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.210025+0000","last_change":"2024-05-08T23:26:56.205854+0000","last_active":"2024-05-08T23:26:56.210025+0000","last_peered":"2024-05-08T23:26:56.210025+0000","last_clean":"2024-05-08T23:26:56.210025+0000","last_became_active":"2024-05-08T23:26:56.205575+0000","last_became_peered":"2024-05-08T23:26:56.205575+0000","last_unstale":"2024-05-08T23:26:56.210025+0000","last_undegraded":"2024-05-08T23:26:56.210025+0000","last_fullsized":"2024-05-08T23:26:56.210025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:32.927014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216090+0000","last_change":"2024-05-08T23:26:55.751835+0000","last_active":"2024-05-08T23:26:56.216090+0000","last_peered":"2024-05-08T23:26:56.216090+0000","last_clean":"2024-05-08T23:26:56.216090+0000","last_became_active":"2024-05-08T23:26:55.751562+0000","last_became_peered":"2024-05-08T23:26:55.751562+0000","last_unstale":"2024-05-08T23:26:56.216090+0000","last_undegraded":"2024-05-08T23:26:56.216090+0000","last_fullsized":"2024-05-08T23:26:56.216090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:40.067977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.221013+0000","last_change":"2024-05-08T23:26:56.219146+0000","last_active":"2024-05-08T23:26:56.221013+0000","last_peered":"2024-05-08T23:26:56.221013+0000","last_clean":"2024-05-08T23:26:56.221013+0000","last_became_active":"2024-05-08T23:26:56.218798+0000","last_became_peered":"2024-05-08T23:26:56.218798+0000","last_unstale":"2024-05-08T23:26:56.221013+0000","last_undegraded":"2024-05-08T23:26:56.221013+0000","last_fullsized":"2024-05-08T23:26:56.221013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:47.211615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.212933+0000","last_change":"2024-05-08T23:26:55.750013+0000","last_active":"2024-05-08T23:26:56.212933+0000","last_peered":"2024-05-08T23:26:56.212933+0000","last_clean":"2024-05-08T23:26:56.212933+0000","last_became_active":"2024-05-08T23:26:55.748574+0000","last_became_peered":"2024-05-08T23:26:55.748574+0000","last_unstale":"2024-05-08T23:26:56.212933+0000","last_undegraded":"2024-05-08T23:26:56.212933+0000","last_fullsized":"2024-05-08T23:26:56.212933+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.467464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198548+0000","last_change":"2024-05-08T23:26:56.214420+0000","last_active":"2024-05-08T23:26:57.198548+0000","last_peered":"2024-05-08T23:26:57.198548+0000","last_clean":"2024-05-08T23:26:57.198548+0000","last_became_active":"2024-05-08T23:26:56.214018+0000","last_became_peered":"2024-05-08T23:26:56.214018+0000","last_unstale":"2024-05-08T23:26:57.198548+0000","last_undegraded":"2024-05-08T23:26:57.198548+0000","last_fullsized":"2024-05-08T23:26:57.198548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:16.059931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198875+0000","last_change":"2024-05-08T23:26:55.755512+0000","last_active":"2024-05-08T23:26:57.198875+0000","last_peered":"2024-05-08T23:26:57.198875+0000","last_clean":"2024-05-08T23:26:57.198875+0000","last_became_active":"2024-05-08T23:26:55.755204+0000","last_became_peered":"2024-05-08T23:26:55.755204+0000","last_unstale":"2024-05-08T23:26:57.198875+0000","last_undegraded":"2024-05-08T23:26:57.198875+0000","last_fullsized":"2024-05-08T23:26:57.198875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:12.268144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224571+0000","last_change":"2024-05-08T23:26:56.224042+0000","last_active":"2024-05-08T23:26:56.224571+0000","last_peered":"2024-05-08T23:26:56.224571+0000","last_clean":"2024-05-08T23:26:56.224571+0000","last_became_active":"2024-05-08T23:26:56.223705+0000","last_became_peered":"2024-05-08T23:26:56.223705+0000","last_unstale":"2024-05-08T23:26:56.224571+0000","last_undegraded":"2024-05-08T23:26:56.224571+0000","last_fullsized":"2024-05-08T23:26:56.224571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:12.365940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198654+0000","last_change":"2024-05-08T23:26:55.751632+0000","last_active":"2024-05-08T23:26:57.198654+0000","last_peered":"2024-05-08T23:26:57.198654+0000","last_clean":"2024-05-08T23:26:57.198654+0000","last_became_active":"2024-05-08T23:26:55.751370+0000","last_became_peered":"2024-05-08T23:26:55.751370+0000","last_unstale":"2024-05-08T23:26:57.198654+0000","last_undegraded":"2024-05-08T23:26:57.198654+0000","last_fullsized":"2024-05-08T23:26:57.198654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:30.221969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216453+0000","last_change":"2024-05-08T23:26:56.212226+0000","last_active":"2024-05-08T23:26:56.216453+0000","last_peered":"2024-05-08T23:26:56.216453+0000","last_clean":"2024-05-08T23:26:56.216453+0000","last_became_active":"2024-05-08T23:26:56.211926+0000","last_became_peered":"2024-05-08T23:26:56.211926+0000","last_unstale":"2024-05-08T23:26:56.216453+0000","last_undegraded":"2024-05-08T23:26:56.216453+0000","last_fullsized":"2024-05-08T23:26:56.216453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:02.599173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.204598+0000","last_change":"2024-05-08T23:26:55.750628+0000","last_active":"2024-05-08T23:26:56.204598+0000","last_peered":"2024-05-08T23:26:56.204598+0000","last_clean":"2024-05-08T23:26:56.204598+0000","last_became_active":"2024-05-08T23:26:55.750358+0000","last_became_peered":"2024-05-08T23:26:55.750358+0000","last_unstale":"2024-05-08T23:26:56.204598+0000","last_undegraded":"2024-05-08T23:26:56.204598+0000","last_fullsized":"2024-05-08T23:26:56.204598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:05.510390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.224480+0000","last_change":"2024-05-08T23:26:56.222698+0000","last_active":"2024-05-08T23:26:56.224480+0000","last_peered":"2024-05-08T23:26:56.224480+0000","last_clean":"2024-05-08T23:26:56.224480+0000","last_became_active":"2024-05-08T23:26:56.222236+0000","last_became_peered":"2024-05-08T23:26:56.222236+0000","last_unstale":"2024-05-08T23:26:56.224480+0000","last_undegraded":"2024-05-08T23:26:56.224480+0000","last_fullsized":"2024-05-08T23:26:56.224480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:08.428124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.202338+0000","last_change":"2024-05-08T23:26:55.751305+0000","last_active":"2024-05-08T23:26:56.202338+0000","last_peered":"2024-05-08T23:26:56.202338+0000","last_clean":"2024-05-08T23:26:56.202338+0000","last_became_active":"2024-05-08T23:26:55.751027+0000","last_became_peered":"2024-05-08T23:26:55.751027+0000","last_unstale":"2024-05-08T23:26:56.202338+0000","last_undegraded":"2024-05-08T23:26:56.202338+0000","last_fullsized":"2024-05-08T23:26:56.202338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:08.997540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.219123+0000","last_change":"2024-05-08T23:26:56.216915+0000","last_active":"2024-05-08T23:26:56.219123+0000","last_peered":"2024-05-08T23:26:56.219123+0000","last_clean":"2024-05-08T23:26:56.219123+0000","last_became_active":"2024-05-08T23:26:56.216600+0000","last_became_peered":"2024-05-08T23:26:56.216600+0000","last_unstale":"2024-05-08T23:26:56.219123+0000","last_undegraded":"2024-05-08T23:26:56.219123+0000","last_fullsized":"2024-05-08T23:26:56.219123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:02.599173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198358+0000","last_change":"2024-05-08T23:26:55.753278+0000","last_active":"2024-05-08T23:26:57.198358+0000","last_peered":"2024-05-08T23:26:57.198358+0000","last_clean":"2024-05-08T23:26:57.198358+0000","last_became_active":"2024-05-08T23:26:55.752974+0000","last_became_peered":"2024-05-08T23:26:55.752974+0000","last_unstale":"2024-05-08T23:26:57.198358+0000","last_undegraded":"2024-05-08T23:26:57.198358+0000","last_fullsized":"2024-05-08T23:26:57.198358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:26:57.198226+0000","last_change":"2024-05-08T23:26:56.216721+0000","last_active":"2024-05-08T23:26:57.198226+0000","last_peered":"2024-05-08T23:26:57.198226+0000","last_clean":"2024-05-08T23:26:57.198226+0000","last_became_active":"2024-05-08T23:26:56.214498+0000","last_became_peered":"2024-05-08T23:26:56.214498+0000","last_unstale":"2024-05-08T23:26:57.198226+0000","last_undegraded":"2024-05-08T23:26:57.198226+0000","last_fullsized":"2024-05-08T23:26:57.198226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:32.927014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.203684+0000","last_change":"2024-05-08T23:26:55.748328+0000","last_active":"2024-05-08T23:26:56.203684+0000","last_peered":"2024-05-08T23:26:56.203684+0000","last_clean":"2024-05-08T23:26:56.203684+0000","last_became_active":"2024-05-08T23:26:55.748033+0000","last_became_peered":"2024-05-08T23:26:55.748033+0000","last_unstale":"2024-05-08T23:26:56.203684+0000","last_undegraded":"2024-05-08T23:26:56.203684+0000","last_fullsized":"2024-05-08T23:26:56.203684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:47.770925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.217505+0000","last_change":"2024-05-08T23:26:56.214935+0000","last_active":"2024-05-08T23:26:56.217505+0000","last_peered":"2024-05-08T23:26:56.217505+0000","last_clean":"2024-05-08T23:26:56.217505+0000","last_became_active":"2024-05-08T23:26:56.214578+0000","last_became_peered":"2024-05-08T23:26:56.214578+0000","last_unstale":"2024-05-08T23:26:56.217505+0000","last_undegraded":"2024-05-08T23:26:56.217505+0000","last_fullsized":"2024-05-08T23:26:56.217505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:55.726552+0000","last_clean_scrub_stamp":"2024-05-08T23:26:55.726552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:31.870394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:26:56.216530+0000","last_change":"2024-05-08T23:26:55.754914+0000","last_active":"2024-05-08T23:26:56.216530+0000","last_peered":"2024-05-08T23:26:56.216530+0000","last_clean":"2024-05-08T23:26:56.216530+0000","last_became_active":"2024-05-08T23:26:55.754650+0000","last_became_peered":"2024-05-08T23:26:55.754650+0000","last_unstale":"2024-05-08T23:26:56.216530+0000","last_undegraded":"2024-05-08T23:26:56.216530+0000","last_fullsized":"2024-05-08T23:26:56.216530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:26:54.720839+0000","last_clean_scrub_stamp":"2024-05-08T23:26:54.720839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:18:39.011728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:26:55.746125+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:26:55.746125+0000","last_peered":"2024-05-08T23:26:55.746125+0000","last_clean":"2024-05-08T23:26:55.746125+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:26:55.746125+0000","last_undegraded":"2024-05-08T23:26:55.746125+0000","last_fullsized":"2024-05-08T23:26:55.746125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1617920,"data_stored":1028469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75700000000000001,"5min":1.038,"15min":1.038},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.376,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.58599999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60899999999999999}]},{"osd":3,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.622},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.40699999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.29499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68000000000000005,"5min":0.87,"15min":0.87},"last":0.44800000000000001}]},{"osd":5,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73499999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.754,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77000000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":438725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68600000000000005}]},{"osd":1,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":1.115,"15min":1.115},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76500000000000001,"5min":1.052,"15min":1.052},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84399999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.58799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1032192,"data_stored":438225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.371,"15min":0.39400000000000002},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56100000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.42299999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.30599999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.435},{"interface":"front","average":{"1min":0.443,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65300000000000002,"5min":1.01,"15min":1.01},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46400000000000002,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":1.028,"15min":1.028},"last":0.39000000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.115},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":1.105,"15min":1.105},"last":0.58199999999999996}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.54000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1617920,"data_stored":1028433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48399999999999999,"15min":0.72599999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.52900000000000003,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":0.71399999999999997},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.60799999999999998,"5min":7.9889999999999999,"15min":7.9889999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.51600000000000001,"15min":0.77800000000000002},"min":{"1min":0.312,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":7.7869999999999999,"15min":7.7869999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.52900000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":8.0519999999999996,"15min":8.0519999999999996},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54800000000000004,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58099999999999996,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65400000000000003,"5min":7.6440000000000001,"15min":7.6440000000000001},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69099999999999995,"15min":0.93500000000000005},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80400000000000005,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.437},{"interface":"front","average":{"1min":0.624,"5min":0.70399999999999996,"15min":0.90800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72499999999999998,"15min":0.91300000000000003},"min":{"1min":0.50700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":7.1120000000000001,"15min":7.1120000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.75700000000000001,"15min":1.0069999999999999},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":7.6970000000000001,"15min":7.6970000000000001},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73499999999999999,"15min":0.97499999999999998},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81799999999999995,"5min":7.734,"15min":7.734},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71999999999999997,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.46200000000000002}]},{"osd":7,"last update":"Wed May 8 23:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.96399999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.92200000000000004},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.83299999999999996,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.54300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":438613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.50700000000000001,"5min":0.621,"15min":0.621},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65800000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28399999999999997}]},{"osd":1,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44900000000000001,"15min":0.41299999999999998},"min":{"1min":0.33300000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.58499999999999996,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.435,"15min":0.42399999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.621,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56200000000000006,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.49099999999999999,"15min":0.46500000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.39800000000000002}]},{"osd":4,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64500000000000002,"15min":0.63300000000000001},"min":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90300000000000002,"5min":1.024,"15min":1.024},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.628,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46700000000000003}]},{"osd":5,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91600000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83199999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.47799999999999998}]},{"osd":6,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.51100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.89200000000000002,"5min":1.159,"15min":1.159},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53500000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69099999999999995,"15min":0.67200000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90000000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92500000000000004,"5min":1.194,"15min":1.194},"last":0.503}]}]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1032192,"data_stored":439595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.41599999999999998},"min":{"1min":0.308,"5min":0.248,"15min":0.248},"max":{"1min":0.55600000000000005,"5min":1.46,"15min":1.46},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40400000000000003,"15min":0.41899999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.496,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.55800000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.32900000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60899999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.42899999999999999}]},{"osd":2,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.55400000000000005,"15min":0.84299999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":11.964,"15min":11.964},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73899999999999999,"5min":1.593,"15min":1.593},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.629,"15min":0.65900000000000003},"min":{"1min":0.48399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78900000000000003,"5min":1.375,"15min":1.375},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85099999999999998,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.628}]},{"osd":5,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.50700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.94799999999999995,"5min":1.536,"15min":1.536},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.68400000000000005},"min":{"1min":0.53400000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76900000000000002,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65400000000000003,"15min":0.68000000000000005},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89900000000000002,"5min":1.204,"15min":1.204},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69399999999999995,"15min":0.72899999999999998},"min":{"1min":0.57399999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90700000000000003,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed May 8 23:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":0.77000000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86099999999999999,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.58099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":1019904,"data_stored":438031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56999999999999995,"15min":0.64200000000000002},"min":{"1min":0.33700000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73399999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59499999999999997,"15min":0.72099999999999997},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.746,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.748,"5min":2.145,"15min":2.145},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60199999999999998,"15min":0.68400000000000005},"min":{"1min":0.36899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.68500000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64700000000000002,"15min":0.78500000000000003},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76700000000000002,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81699999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.74299999999999999,"15min":1.1240000000000001},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79300000000000004,"5min":12.127000000000001,"15min":12.127000000000001},"last":0.621}]},{"osd":5,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.877,"15min":0.877},"last":0.497},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63400000000000001,"15min":0.68200000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73899999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.628},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77400000000000002,"5min":1.24,"15min":1.24},"last":0.55800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":439617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.629,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85199999999999998,"5min":4.6710000000000003,"15min":4.6710000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":0.89400000000000002},"min":{"1min":0.35199999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":8.798,"15min":8.798},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.77500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":4.71,"15min":4.71},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70699999999999996,"15min":0.96799999999999997},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0009999999999999,"5min":4.8940000000000001,"15min":4.8940000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.79100000000000004},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.82199999999999995,"5min":4.8559999999999999,"15min":4.8559999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.93000000000000005},"min":{"1min":0.46999999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78500000000000003,"5min":8.8770000000000007,"15min":8.8770000000000007},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73899999999999999,"15min":1.0109999999999999},"min":{"1min":0.51100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0489999999999999,"5min":10.427,"15min":10.427},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58599999999999997,"5min":0.874,"15min":0.874},"last":0.505},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85299999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Wed May 8 23:26:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.01},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:27:00.880 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:27:00.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T23:27:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.067+0000 7fa2e4955700 1 Processor -- start 2024-05-08T23:27:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.067+0000 7fa2e4955700 1 -- start start 2024-05-08T23:27:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2e4955700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00b3a10 0x7fa2e00cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2e4955700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa2e00cdeb0 0x7fa2e00cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2e4955700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e0072b70 con 0x7fa2e00cbbb0 2024-05-08T23:27:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2e4955700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e00cc570 con 0x7fa2e00cdeb0 2024-05-08T23:27:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2e4955700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0072e50 con 0x7fa2e00b3a10 2024-05-08T23:27:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2ddd9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa2e00cbbb0 0x7fa2e00cd7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47108/0 (socket says 172.21.15.5:47108) 2024-05-08T23:27:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2ded9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa2e00cdeb0 0x7fa2e00cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2ddd9b700 1 -- 172.21.15.5:0/3460329146 learned_addr learned my addr 172.21.15.5:0/3460329146 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2de59c700 1 --2- 172.21.15.5:0/3460329146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00b3a10 0x7fa2e00cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2ded9d700 1 -- 172.21.15.5:0/3460329146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00b3a10 msgr2=0x7fa2e00cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2ded9d700 1 --2- 172.21.15.5:0/3460329146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00b3a10 0x7fa2e00cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2ded9d700 1 -- 172.21.15.5:0/3460329146 >> v1:172.21.15.5:6789/0 conn(0x7fa2e00cbbb0 legacy=0x7fa2e00cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2ded9d700 1 -- 172.21.15.5:0/3460329146 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00cef00 con 0x7fa2e00cdeb0 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.068+0000 7fa2dd59a700 1 -- 172.21.15.5:0/3460329146 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868528261 0 0) 0x7fa2e0072b70 con 0x7fa2e00cbbb0 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2ded9d700 1 --2- 172.21.15.5:0/3460329146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00cdeb0 0x7fa2e00cc030 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009d90 tx=0x7fa2d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107ac0bf941e0869 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2dd59a700 1 -- 172.21.15.5:0/3460329146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800e070 con 0x7fa2e00cdeb0 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2dd59a700 1 -- 172.21.15.5:0/3460329146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d800bb20 con 0x7fa2e00cdeb0 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2dd59a700 1 -- 172.21.15.5:0/3460329146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8012430 con 0x7fa2e00cdeb0 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2e4955700 1 -- 172.21.15.5:0/3460329146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00cdeb0 msgr2=0x7fa2e00cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2e4955700 1 --2- 172.21.15.5:0/3460329146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00cdeb0 0x7fa2e00cc030 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009d90 tx=0x7fa2d800b330 comp rx=0 tx=0).stop 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2e4955700 1 -- 172.21.15.5:0/3460329146 shutdown_connections 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2e4955700 1 --2- 172.21.15.5:0/3460329146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00cdeb0 0x7fa2e00cc030 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2e4955700 1 --2- 172.21.15.5:0/3460329146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00b3a10 0x7fa2e00cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.069+0000 7fa2e4955700 1 -- 172.21.15.5:0/3460329146 >> 172.21.15.5:0/3460329146 conn(0x7fa2e006a960 msgr2=0x7fa2e006ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.070+0000 7fa2e4955700 1 -- 172.21.15.5:0/3460329146 shutdown_connections 2024-05-08T23:27:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.070+0000 7fa2e4955700 1 -- 172.21.15.5:0/3460329146 wait complete. 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.071+0000 7fa2e4955700 1 Processor -- start 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.071+0000 7fa2e4955700 1 -- start start 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.072+0000 7fa2e4955700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00b3a10 0x7fa2e0098200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.072+0000 7fa2e4955700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e00cbbb0 0x7fa2e0152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.072+0000 7fa2e4955700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00cdeb0 0x7fa2e015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.072+0000 7fa2e4955700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0153860 con 0x7fa2e00b3a10 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.072+0000 7fa2e4955700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0098740 con 0x7fa2e00cbbb0 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.072+0000 7fa2e4955700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0152be0 con 0x7fa2e00cdeb0 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00b3a10 0x7fa2e0098200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00b3a10 0x7fa2e0098200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37156/0 (socket says 172.21.15.5:37156) 2024-05-08T23:27:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 -- 172.21.15.5:0/4289337038 learned_addr learned my addr 172.21.15.5:0/4289337038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2ded9d700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00cdeb0 0x7fa2e015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2ddd9b700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e00cbbb0 0x7fa2e0152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 -- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00cdeb0 msgr2=0x7fa2e015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00cdeb0 0x7fa2e015cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 -- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e00cbbb0 msgr2=0x7fa2e0152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e00cbbb0 0x7fa2e0152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 -- 172.21.15.5:0/4289337038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8009a70 con 0x7fa2e00b3a10 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2ded9d700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00cdeb0 0x7fa2e015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2ddd9b700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e00cbbb0 0x7fa2e0152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2de59c700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00b3a10 0x7fa2e0098200 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000e870 tx=0x7fa2d000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2ceffd700 1 -- 172.21.15.5:0/4289337038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0009590 con 0x7fa2e00b3a10 2024-05-08T23:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2ceffd700 1 -- 172.21.15.5:0/4289337038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d000ee70 con 0x7fa2e00b3a10 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e0152ec0 con 0x7fa2e00b3a10 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2ceffd700 1 -- 172.21.15.5:0/4289337038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000ca50 con 0x7fa2e00b3a10 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.073+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e0153480 con 0x7fa2e00b3a10 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.074+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e0153ad0 con 0x7fa2e00b3a10 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.075+0000 7fa2ceffd700 1 -- 172.21.15.5:0/4289337038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa2d000cbb0 con 0x7fa2e00b3a10 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.075+0000 7fa2ceffd700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2c006ac90 0x7fa2c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.076+0000 7fa2ceffd700 1 -- 172.21.15.5:0/4289337038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2d0089250 con 0x7fa2e00b3a10 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.076+0000 7fa2ddd9b700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2c006ac90 0x7fa2c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.076+0000 7fa2ddd9b700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2c006ac90 0x7fa2c006d150 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa2e006dab0 tx=0x7fa2d4007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.080+0000 7fa2ceffd700 1 -- 172.21.15.5:0/4289337038 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa2d00544c0 con 0x7fa2e00b3a10 2024-05-08T23:27:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.391+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa2e0074bc0 con 0x7fa2e00b3a10 2024-05-08T23:27:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.395+0000 7fa2ceffd700 1 -- 172.21.15.5:0/4289337038 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa2d000ce60 con 0x7fa2e00b3a10 2024-05-08T23:27:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2c006ac90 msgr2=0x7fa2c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2c006ac90 0x7fa2c006d150 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa2e006dab0 tx=0x7fa2d4007cc0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00b3a10 msgr2=0x7fa2e0098200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00b3a10 0x7fa2e0098200 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000e870 tx=0x7fa2d000ec30 comp rx=0 tx=0).stop 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 shutdown_connections 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa2c006ac90 0x7fa2c006d150 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2e00cdeb0 0x7fa2e015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e00b3a10 0x7fa2e0098200 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 --2- 172.21.15.5:0/4289337038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2e00cbbb0 0x7fa2e0152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 >> 172.21.15.5:0/4289337038 conn(0x7fa2e006a960 msgr2=0x7fa2e0141f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 shutdown_connections 2024-05-08T23:27:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.397+0000 7fa2e4955700 1 -- 172.21.15.5:0/4289337038 wait complete. 2024-05-08T23:27:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-08T23:27:01.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:01.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.549+0000 7fe8af877700 1 Processor -- start 2024-05-08T23:27:01.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.549+0000 7fe8af877700 1 -- start start 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8af877700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8a80b5630 0x7fe8a8166480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8af877700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a80cfe20 con 0x7fe8a815ef60 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8af877700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a809ad40 con 0x7fe8a80b2340 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8af877700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a80cd400 con 0x7fe8a80b5630 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8ade14700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8a80b5630 0x7fe8a8166480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8ad613700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe8a815ef60 0x7fe8a80cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47118/0 (socket says 172.21.15.5:47118) 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8ade14700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8a80b5630 0x7fe8a8166480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37172/0 (socket says 172.21.15.5:37172) 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8ad613700 1 -- 172.21.15.5:0/4124430453 learned_addr learned my addr 172.21.15.5:0/4124430453 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8ade14700 1 -- 172.21.15.5:0/4124430453 >> v1:172.21.15.49:6789/0 conn(0x7fe8a80b2340 legacy=0x7fe8a80cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8ade14700 1 -- 172.21.15.5:0/4124430453 >> v1:172.21.15.5:6789/0 conn(0x7fe8a815ef60 legacy=0x7fe8a80cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe89ffff700 1 -- 172.21.15.5:0/4124430453 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646278233 0 0) 0x7fe8a80cfe20 con 0x7fe8a815ef60 2024-05-08T23:27:01.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.550+0000 7fe8ade14700 1 -- 172.21.15.5:0/4124430453 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a80ccb60 con 0x7fe8a80b5630 2024-05-08T23:27:01.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe8ade14700 1 --2- 172.21.15.5:0/4124430453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a80b5630 0x7fe8a8166480 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009d90 tx=0x7fe8a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12862ae202ef754 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:01.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe89ffff700 1 -- 172.21.15.5:0/4124430453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000e070 con 0x7fe8a80b5630 2024-05-08T23:27:01.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe89ffff700 1 -- 172.21.15.5:0/4124430453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a000bbd0 con 0x7fe8a80b5630 2024-05-08T23:27:01.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe89ffff700 1 -- 172.21.15.5:0/4124430453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0012430 con 0x7fe8a80b5630 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe8af877700 1 -- 172.21.15.5:0/4124430453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a80b5630 msgr2=0x7fe8a8166480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe8af877700 1 --2- 172.21.15.5:0/4124430453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a80b5630 0x7fe8a8166480 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009d90 tx=0x7fe8a000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe8af877700 1 -- 172.21.15.5:0/4124430453 shutdown_connections 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe8af877700 1 --2- 172.21.15.5:0/4124430453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a80b5630 0x7fe8a8166480 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe8af877700 1 -- 172.21.15.5:0/4124430453 >> 172.21.15.5:0/4124430453 conn(0x7fe8a80b2f20 msgr2=0x7fe8a80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe8af877700 1 -- 172.21.15.5:0/4124430453 shutdown_connections 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.551+0000 7fe8af877700 1 -- 172.21.15.5:0/4124430453 wait complete. 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.553+0000 7fe8af877700 1 Processor -- start 2024-05-08T23:27:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.553+0000 7fe8af877700 1 -- start start 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.553+0000 7fe8af877700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a80b2340 0x7fe8a8153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.553+0000 7fe8af877700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8a80b5630 0x7fe8a80c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.553+0000 7fe8af877700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a815ef60 0x7fe8a815d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.553+0000 7fe8af877700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a8152dd0 con 0x7fe8a815ef60 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.553+0000 7fe8af877700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a80c5a60 con 0x7fe8a80b2340 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.553+0000 7fe8af877700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a80c5d40 con 0x7fe8a80b5630 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a815ef60 0x7fe8a815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a815ef60 0x7fe8a815d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37188/0 (socket says 172.21.15.5:37188) 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 -- 172.21.15.5:0/2620760467 learned_addr learned my addr 172.21.15.5:0/2620760467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ad613700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a80b2340 0x7fe8a8153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ace12700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8a80b5630 0x7fe8a80c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 -- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8a80b5630 msgr2=0x7fe8a80c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8a80b5630 0x7fe8a80c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 -- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a80b2340 msgr2=0x7fe8a8153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a80b2340 0x7fe8a8153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 -- 172.21.15.5:0/2620760467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a0009a70 con 0x7fe8a815ef60 2024-05-08T23:27:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8ade14700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a815ef60 0x7fe8a815d620 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0014040 tx=0x7fe8a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe89dffb700 1 -- 172.21.15.5:0/2620760467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a001b810 con 0x7fe8a815ef60 2024-05-08T23:27:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe89dffb700 1 -- 172.21.15.5:0/2620760467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0020070 con 0x7fe8a815ef60 2024-05-08T23:27:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a80c6020 con 0x7fe8a815ef60 2024-05-08T23:27:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.554+0000 7fe89dffb700 1 -- 172.21.15.5:0/2620760467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a001bbb0 con 0x7fe8a815ef60 2024-05-08T23:27:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.555+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a80c65e0 con 0x7fe8a815ef60 2024-05-08T23:27:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.555+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a8068000 con 0x7fe8a815ef60 2024-05-08T23:27:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.556+0000 7fe89dffb700 1 -- 172.21.15.5:0/2620760467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe8a001bd10 con 0x7fe8a815ef60 2024-05-08T23:27:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.557+0000 7fe89dffb700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe88c06ac90 0x7fe88c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.557+0000 7fe89dffb700 1 -- 172.21.15.5:0/2620760467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8a0022070 con 0x7fe8a815ef60 2024-05-08T23:27:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.557+0000 7fe8ad613700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe88c06ac90 0x7fe88c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:01.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.557+0000 7fe8ad613700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe88c06ac90 0x7fe88c06d150 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4007c40 tx=0x7fe8a4007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:01.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.561+0000 7fe89dffb700 1 -- 172.21.15.5:0/2620760467 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe8a0056510 con 0x7fe8a815ef60 2024-05-08T23:27:01.719 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:27:01.719+0000 7fb52a33c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:27:01.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.826+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8a80cc6b0 con 0x7fe8a815ef60 2024-05-08T23:27:01.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.826+0000 7fe89dffb700 1 -- 172.21.15.5:0/2620760467 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4730 (secure 0 0 0) 0x7fe8a0059b30 con 0x7fe8a815ef60 2024-05-08T23:27:01.827 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:01.827 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7336,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7755,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:26:59.207563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe88c06ac90 msgr2=0x7fe88c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe88c06ac90 0x7fe88c06d150 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4007c40 tx=0x7fe8a4007b90 comp rx=0 tx=0).stop 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a815ef60 msgr2=0x7fe8a815d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a815ef60 0x7fe8a815d620 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0014040 tx=0x7fe8a0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 shutdown_connections 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe88c06ac90 0x7fe88c06d150 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a815ef60 0x7fe8a815d620 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8a80b2340 0x7fe8a8153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 --2- 172.21.15.5:0/2620760467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8a80b5630 0x7fe8a80c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 >> 172.21.15.5:0/2620760467 conn(0x7fe8a80b2f20 msgr2=0x7fe8a80b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:01.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 shutdown_connections 2024-05-08T23:27:01.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:01.829+0000 7fe8af877700 1 -- 172.21.15.5:0/2620760467 wait complete. 2024-05-08T23:27:01.830 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 82 2024-05-08T23:27:02.245 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:26:56.191956+0000', 'modified': '2024-05-08T23:26:59.207563+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7758}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7758': {'gid': 7758, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/1093540582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 1093540582}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1093540582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:27:02.245 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:27:02.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T23:27:02.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.392+0000 7fb1d69a3700 1 Processor -- start 2024-05-08T23:27:02.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.392+0000 7fb1d69a3700 1 -- start start 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1d69a3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb1d015bbd0 0x7fb1d015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1d69a3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb1d00cc980 0x7fb1d00ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1d69a3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d00cfdc0 con 0x7fb1d015f110 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1d69a3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d00cefa0 con 0x7fb1d015bbd0 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1d69a3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d015c4f0 con 0x7fb1d00cc980 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1cffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb1d015bbd0 0x7fb1d015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1d4f40700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb1d015f110 0x7fb1d015f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47124/0 (socket says 172.21.15.5:47124) 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1d4f40700 1 -- 172.21.15.5:0/1083687030 learned_addr learned my addr 172.21.15.5:0/1083687030 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:02.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.393+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/1083687030 >> v2:172.21.15.49:3300/0 conn(0x7fb1d00cc980 0x7fb1d00ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1cffff700 1 -- 172.21.15.5:0/1083687030 >> v2:172.21.15.49:3300/0 conn(0x7fb1d00cc980 msgr2=0x7fb1d00ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1cffff700 1 --2- 172.21.15.5:0/1083687030 >> v2:172.21.15.49:3300/0 conn(0x7fb1d00cc980 0x7fb1d00ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1cffff700 1 -- 172.21.15.5:0/1083687030 >> v1:172.21.15.5:6789/0 conn(0x7fb1d015f110 legacy=0x7fb1d015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1cffff700 1 -- 172.21.15.5:0/1083687030 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00cf1b0 con 0x7fb1d015bbd0 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1ceffd700 1 -- 172.21.15.5:0/1083687030 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955027403 0 0) 0x7fb1d00cfdc0 con 0x7fb1d015f110 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1cffff700 1 --2- 172.21.15.5:0/1083687030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015bbd0 0x7fb1d015bfb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00c4e70 tx=0x7fb1c0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af521cd2b27ecd5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1ceffd700 1 -- 172.21.15.5:0/1083687030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0011830 con 0x7fb1d015bbd0 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1ceffd700 1 -- 172.21.15.5:0/1083687030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c0011990 con 0x7fb1d015bbd0 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.394+0000 7fb1ceffd700 1 -- 172.21.15.5:0/1083687030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0011d20 con 0x7fb1d015bbd0 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.395+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1083687030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015bbd0 msgr2=0x7fb1d015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.395+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1083687030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015bbd0 0x7fb1d015bfb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00c4e70 tx=0x7fb1c0009b20 comp rx=0 tx=0).stop 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.395+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1083687030 shutdown_connections 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.395+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1083687030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015bbd0 0x7fb1d015bfb0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.395+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1083687030 >> v2:172.21.15.49:3300/0 conn(0x7fb1d00cc980 0x7fb1d00ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.395+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1083687030 >> 172.21.15.5:0/1083687030 conn(0x7fb1d00b2ec0 msgr2=0x7fb1d00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.395+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1083687030 shutdown_connections 2024-05-08T23:27:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.395+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1083687030 wait complete. 2024-05-08T23:27:02.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.397+0000 7fb1d69a3700 1 Processor -- start 2024-05-08T23:27:02.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.397+0000 7fb1d69a3700 1 -- start start 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d69a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d00cc980 0x7fb1d0154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d69a3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1d015bbd0 0x7fb1d00c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d69a3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015f110 0x7fb1d0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d69a3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00b3d40 con 0x7fb1d015f110 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d69a3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00b3fe0 con 0x7fb1d00cc980 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d69a3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00b42c0 con 0x7fb1d015bbd0 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015f110 0x7fb1d0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015f110 0x7fb1d0158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37210/0 (socket says 172.21.15.5:37210) 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 -- 172.21.15.5:0/1516388281 learned_addr learned my addr 172.21.15.5:0/1516388281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1d015bbd0 0x7fb1d00c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1cffff700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d00cc980 0x7fb1d0154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 -- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1d015bbd0 msgr2=0x7fb1d00c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1d015bbd0 0x7fb1d00c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 -- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d00cc980 msgr2=0x7fb1d0154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d00cc980 0x7fb1d0154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 -- 172.21.15.5:0/1516388281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00094a0 con 0x7fb1d015f110 2024-05-08T23:27:02.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.398+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015f110 0x7fb1d0158ff0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800cba0 tx=0x7fb1c800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.399+0000 7fb1ccff9700 1 -- 172.21.15.5:0/1516388281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c800aaa0 con 0x7fb1d015f110 2024-05-08T23:27:02.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.399+0000 7fb1ccff9700 1 -- 172.21.15.5:0/1516388281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c800ac20 con 0x7fb1d015f110 2024-05-08T23:27:02.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.399+0000 7fb1ccff9700 1 -- 172.21.15.5:0/1516388281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80070d0 con 0x7fb1d015f110 2024-05-08T23:27:02.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.399+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1516388281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d00b45a0 con 0x7fb1d015f110 2024-05-08T23:27:02.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.399+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1516388281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d00b4b60 con 0x7fb1d015f110 2024-05-08T23:27:02.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.401+0000 7fb1ccff9700 1 -- 172.21.15.5:0/1516388281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb1c8007230 con 0x7fb1d015f110 2024-05-08T23:27:02.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.401+0000 7fb1ccff9700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1bc06ac90 0x7fb1bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.401+0000 7fb1ccff9700 1 -- 172.21.15.5:0/1516388281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1c8088db0 con 0x7fb1d015f110 2024-05-08T23:27:02.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.402+0000 7fb1cffff700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1bc06ac90 0x7fb1bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.402+0000 7fb1cffff700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1bc06ac90 0x7fb1bc06d150 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00090b0 tx=0x7fb1c0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.403+0000 7fb1e2b5db80 1 Processor -- start 2024-05-08T23:27:02.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.403+0000 7fb1e2b5db80 1 -- start start 2024-05-08T23:27:02.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dfeec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dfef1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff1ad80 0x55b8dff1b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfe47100 con 0x55b8dff13750 2024-05-08T23:27:02.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfe44ac0 con 0x55b8dfeee320 2024-05-08T23:27:02.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfe403b0 con 0x55b8dff1ad80 2024-05-08T23:27:02.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dfeec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dfeec0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37218/0 (socket says 172.21.15.5:37218) 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cffff700 1 -- 172.21.15.5:0/1792349216 learned_addr learned my addr 172.21.15.5:0/1792349216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff1ad80 0x55b8dff1b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dfef1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cffff700 1 -- 172.21.15.5:0/1792349216 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff1ad80 msgr2=0x55b8dff1b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cffff700 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff1ad80 0x55b8dff1b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cffff700 1 -- 172.21.15.5:0/1792349216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 msgr2=0x55b8dfef1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cffff700 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dfef1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.404+0000 7fb1cffff700 1 -- 172.21.15.5:0/1792349216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8dfcc1360 con 0x55b8dff13750 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.405+0000 7fb1cffff700 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dfeec0c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0072530 tx=0x7fb1c0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7caf9c3feb4a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.405+0000 7fb1b5ffb700 1 -- 172.21.15.5:0/1792349216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c002c070 con 0x55b8dff13750 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.405+0000 7fb1b5ffb700 1 -- 172.21.15.5:0/1792349216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c00036a0 con 0x55b8dff13750 2024-05-08T23:27:02.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.405+0000 7fb1b5ffb700 1 -- 172.21.15.5:0/1792349216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0020070 con 0x55b8dff13750 2024-05-08T23:27:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.405+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/1792349216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 msgr2=0x55b8dfeec0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.405+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dfeec0c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0072530 tx=0x7fb1c0010000 comp rx=0 tx=0).stop 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.406+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/1792349216 shutdown_connections 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.406+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff1ad80 0x55b8dff1b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.406+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dfeec0c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.406+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/1792349216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dfef1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.406+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/1792349216 >> 172.21.15.5:0/1792349216 conn(0x55b8dfe4e1c0 msgr2=0x55b8dff35d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.406+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/1792349216 shutdown_connections 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.406+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/1792349216 wait complete. 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.406+0000 7fb1e2b5db80 1 -- start start 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.407+0000 7fb1e2b5db80 1 Processor -- start 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dff1e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff3b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfddc510 con 0x55b8dff1ad80 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfddc1b0 con 0x55b8dfeee320 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfcc3650 con 0x55b8dff13750 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1e2b5db80 10 client.0 _ll_register_callbacks cb 0x55b8dfde5ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1d4f40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1d4f40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37232/0 (socket says 172.21.15.5:37232) 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1d4f40700 1 -- 172.21.15.5:0/3382027420 learned_addr learned my addr 172.21.15.5:0/3382027420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1cffff700 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dff1e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff3b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1d4f40700 1 -- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 msgr2=0x55b8dff3b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff3b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1d4f40700 1 -- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 msgr2=0x55b8dff1e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dff1e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1d4f40700 1 -- 172.21.15.5:0/3382027420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d00cf1b0 con 0x55b8dff1ad80 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.408+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff3b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3ba10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8057be0 tx=0x7fb1c80584c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a3f2567068b691 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1aeffd700 1 -- 172.21.15.5:0/3382027420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c805fdb0 con 0x55b8dff1ad80 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1aeffd700 1 -- 172.21.15.5:0/3382027420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1c805ba80 con 0x55b8dff1ad80 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/3382027420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8dfd99220 con 0x55b8dff1ad80 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1aeffd700 1 -- 172.21.15.5:0/3382027420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1c8060040 con 0x55b8dff1ad80 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 10 client.7899 fetch_fsmap learned FSMap version 82 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/3382027420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55b8dfc6db10 con 0x55b8dff1ad80 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1aeffd700 1 -- 172.21.15.5:0/3382027420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8dfd99220 con 0x55b8dff1ad80 2024-05-08T23:27:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1aeffd700 1 -- 172.21.15.5:0/3382027420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb1c8053a40 con 0x55b8dff1ad80 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 10 client.7899 fetch_fsmap finished waiting for FSMap version 82 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 10 client.7899 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 20 client.7899 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55b8dff1c6a0 0x55b8dff10810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 4 client.7899 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/3382027420 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8dfebf790 con 0x55b8dff1c6a0 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.409+0000 7fb1cffff700 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55b8dff1c6a0 0x55b8dff10810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.410+0000 7fb1cffff700 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55b8dff1c6a0 0x55b8dff10810 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed0c4943cf23848 server_cookie=61257adec2b70bb5 in_seq=0 out_seq=0 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.410+0000 7fb1aeffd700 10 client.7899 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.410+0000 7fb1aeffd700 1 -- 172.21.15.5:0/3382027420 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8dfebf790 con 0x55b8dff1c6a0 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.410+0000 7fb1aeffd700 10 client.7899 handle_command_reply: tid=0 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.410+0000 7fb1e2b5db80 1 client.7899 shutdown 2024-05-08T23:27:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/3382027420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 msgr2=0x55b8dff3ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3ba10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8057be0 tx=0x7fb1c80584c0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/3382027420 shutdown_connections 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55b8dff1c6a0 0x55b8dff10810 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3ba10 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dff1e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1aeffd700 0 client.7899 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/3382027420 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff3b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/3382027420 >> 172.21.15.5:0/3382027420 conn(0x55b8dfe4e1c0 msgr2=0x55b8dfef0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/3382027420 shutdown_connections 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/3382027420 wait complete. 2024-05-08T23:27:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.411+0000 7fb1e2b5db80 20 client.7899 trim_cache size 0 max 16384 2024-05-08T23:27:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.430+0000 7fb1e2b5db80 1 Processor -- start 2024-05-08T23:27:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.430+0000 7fb1e2b5db80 1 -- start start 2024-05-08T23:27:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.430+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dfee9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.430+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff35fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.430+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.430+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfeeaa40 con 0x55b8dff1ad80 2024-05-08T23:27:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.430+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8dff3d960 con 0x55b8dfeee320 2024-05-08T23:27:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.430+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8dff36500 con 0x55b8dff13750 2024-05-08T23:27:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37238/0 (socket says 172.21.15.5:37238) 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 -- 172.21.15.5:0/631946676 learned_addr learned my addr 172.21.15.5:0/631946676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1cffff700 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dfee9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff35fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 -- 172.21.15.5:0/631946676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 msgr2=0x55b8dff35fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff35fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 -- 172.21.15.5:0/631946676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 msgr2=0x55b8dfee9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dfee9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 -- 172.21.15.5:0/631946676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8dfe06510 con 0x55b8dff1ad80 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3bc10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8001d80 tx=0x7fb1c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738de87a84fda1af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1b67fc700 1 -- 172.21.15.5:0/631946676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8060040 con 0x55b8dff1ad80 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1b67fc700 1 -- 172.21.15.5:0/631946676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8057a20 con 0x55b8dff1ad80 2024-05-08T23:27:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.431+0000 7fb1b67fc700 1 -- 172.21.15.5:0/631946676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c805fdd0 con 0x55b8dff1ad80 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/631946676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 msgr2=0x55b8dff3bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3bc10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8001d80 tx=0x7fb1c8014000 comp rx=0 tx=0).stop 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/631946676 shutdown_connections 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff1ad80 0x55b8dff3bc10 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dfeee320 0x55b8dfee9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/631946676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dff13750 0x55b8dff35fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/631946676 >> 172.21.15.5:0/631946676 conn(0x55b8dfe4e1c0 msgr2=0x55b8dfee7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/631946676 shutdown_connections 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/631946676 wait complete. 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.432+0000 7fb1e2b5db80 1 -- start start 2024-05-08T23:27:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.433+0000 7fb1e2b5db80 1 Processor -- start 2024-05-08T23:27:02.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dfeee320 0x55b8dfee9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dff9e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1e2b5db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dff1ad80 0x55b8dff0ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8dff1ed80 con 0x55b8dff13750 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfeef5c0 con 0x55b8dff1ad80 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1e2b5db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8dfef2400 con 0x55b8dfeee320 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1e2b5db80 10 client.0 _ll_register_callbacks cb 0x55b8dfde5ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dff9e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dff9e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37242/0 (socket says 172.21.15.5:37242) 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 -- 172.21.15.5:0/2995697687 learned_addr learned my addr 172.21.15.5:0/2995697687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cffff700 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dfeee320 0x55b8dfee9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1d4f40700 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dff1ad80 0x55b8dff0ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 -- 172.21.15.5:0/2995697687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dfeee320 msgr2=0x55b8dfee9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dfeee320 0x55b8dfee9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 -- 172.21.15.5:0/2995697687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dff1ad80 msgr2=0x55b8dff0ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dff1ad80 0x55b8dff0ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 -- 172.21.15.5:0/2995697687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8dfc6db10 con 0x55b8dff13750 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1cf7fe700 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dff9e060 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00affb0 tx=0x7fb1c4017ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4e9ce2bf04c2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1aeffd700 1 -- 172.21.15.5:0/2995697687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c401c070 con 0x55b8dff13750 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1aeffd700 1 -- 172.21.15.5:0/2995697687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1c4004300 con 0x55b8dff13750 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.434+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/2995697687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1c00036a0 con 0x55b8dff13750 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1aeffd700 1 -- 172.21.15.5:0/2995697687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1c40044c0 con 0x55b8dff13750 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1e2b5db80 10 client.7905 fetch_fsmap learned FSMap version 82 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/2995697687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55b8dfcc5c80 con 0x55b8dff13750 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1aeffd700 1 -- 172.21.15.5:0/2995697687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c00036a0 con 0x55b8dff13750 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1aeffd700 1 -- 172.21.15.5:0/2995697687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb1c40036a0 con 0x55b8dff13750 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1e2b5db80 10 client.7905 fetch_fsmap finished waiting for FSMap version 82 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1e2b5db80 10 client.7905 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:27:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1e2b5db80 20 client.7905 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55b8dff32be0 0x55b8dff0f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1e2b5db80 4 client.7905 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/2995697687 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b8dfe38e70 con 0x55b8dff32be0 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1cffff700 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55b8dff32be0 0x55b8dff0f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1aeffd700 10 client.7905 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.435+0000 7fb1cffff700 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55b8dff32be0 0x55b8dff0f3b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0cb7d67dc018caa server_cookie=792d7a6d4ccd30c in_seq=0 out_seq=0 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1aeffd700 1 -- 172.21.15.5:0/2995697687 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b8dfe38e70 con 0x55b8dff32be0 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1aeffd700 10 client.7905 handle_command_reply: tid=0 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 client.7905 shutdown 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/2995697687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 msgr2=0x55b8dff9e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dff9e060 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00affb0 tx=0x7fb1c4017ae0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/2995697687 shutdown_connections 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55b8dff32be0 0x55b8dff0f3b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8dff1ad80 0x55b8dff0ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b8dfeee320 0x55b8dfee9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1aeffd700 0 client.7905 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 --2- 172.21.15.5:0/2995697687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8dff13750 0x55b8dff9e060 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/2995697687 >> 172.21.15.5:0/2995697687 conn(0x55b8dfe4e1c0 msgr2=0x55b8dff1b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/2995697687 shutdown_connections 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 1 -- 172.21.15.5:0/2995697687 wait complete. 2024-05-08T23:27:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1e2b5db80 20 client.7905 trim_cache size 0 max 16384 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7758, 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.2431165210000001, 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 82, 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 84, 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 11.494827924000001 2024-05-08T23:27:02.444 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1bc06ac90 msgr2=0x7fb1bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1bc06ac90 0x7fb1bc06d150 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00090b0 tx=0x7fb1c0008030 comp rx=0 tx=0).stop 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1516388281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015f110 msgr2=0x7fb1d0158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.436+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015f110 0x7fb1d0158ff0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800cba0 tx=0x7fb1c800ceb0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.437+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1516388281 shutdown_connections 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.437+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb1bc06ac90 0x7fb1bc06d150 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.437+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1d015f110 0x7fb1d0158ff0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.437+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1d00cc980 0x7fb1d0154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.437+0000 7fb1d69a3700 1 --2- 172.21.15.5:0/1516388281 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb1d015bbd0 0x7fb1d00c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.437+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1516388281 >> 172.21.15.5:0/1516388281 conn(0x7fb1d00b2ec0 msgr2=0x7fb1d0070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.437+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1516388281 shutdown_connections 2024-05-08T23:27:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.437+0000 7fb1d69a3700 1 -- 172.21.15.5:0/1516388281 wait complete. 2024-05-08T23:27:02.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:27:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.586+0000 7f47c4976700 1 Processor -- start 2024-05-08T23:27:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.586+0000 7f47c4976700 1 -- start start 2024-05-08T23:27:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.587+0000 7f47c4976700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f47c00b1260 0x7f47c015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.587+0000 7f47c4976700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c00cca10 con 0x7f47c00cbfc0 2024-05-08T23:27:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.587+0000 7f47c4976700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c015c080 con 0x7f47c00d1070 2024-05-08T23:27:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.587+0000 7f47c4976700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c015c740 con 0x7f47c00b1260 2024-05-08T23:27:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.587+0000 7f47bed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f47c00cbfc0 0x7f47c015f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47128/0 (socket says 172.21.15.5:47128) 2024-05-08T23:27:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.587+0000 7f47bed9d700 1 -- 172.21.15.5:0/2105840386 learned_addr learned my addr 172.21.15.5:0/2105840386 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47be59c700 1 --2- 172.21.15.5:0/2105840386 >> v2:172.21.15.49:3300/0 conn(0x7f47c00b1260 0x7f47c015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47bd59a700 1 -- 172.21.15.5:0/2105840386 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855156913 0 0) 0x7f47c00cca10 con 0x7f47c00cbfc0 2024-05-08T23:27:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47bd59a700 1 -- 172.21.15.5:0/2105840386 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47ac0035b0 con 0x7f47c00cbfc0 2024-05-08T23:27:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47be59c700 1 -- 172.21.15.5:0/2105840386 >> v1:172.21.15.49:6789/0 conn(0x7f47c00d1070 legacy=0x7f47c00b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47be59c700 1 -- 172.21.15.5:0/2105840386 >> v1:172.21.15.5:6789/0 conn(0x7f47c00cbfc0 legacy=0x7f47c015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47bd59a700 1 -- 172.21.15.5:0/2105840386 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3594608770 0 0) 0x7f47ac0035b0 con 0x7f47c00cbfc0 2024-05-08T23:27:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47be59c700 1 -- 172.21.15.5:0/2105840386 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c015c360 con 0x7f47c00b1260 2024-05-08T23:27:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47be59c700 1 --2- 172.21.15.5:0/2105840386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00b1260 0x7f47c015ba90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009d90 tx=0x7f47b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296d27a7d4d8dab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.588+0000 7f47bd59a700 1 -- 172.21.15.5:0/2105840386 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000e070 con 0x7f47c00b1260 2024-05-08T23:27:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47bd59a700 1 -- 172.21.15.5:0/2105840386 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0003a20 con 0x7f47c00b1260 2024-05-08T23:27:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47bd59a700 1 -- 172.21.15.5:0/2105840386 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8001670 con 0x7f47c00b1260 2024-05-08T23:27:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47c4976700 1 -- 172.21.15.5:0/2105840386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00b1260 msgr2=0x7f47c015ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47c4976700 1 --2- 172.21.15.5:0/2105840386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00b1260 0x7f47c015ba90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009d90 tx=0x7f47b000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47c4976700 1 -- 172.21.15.5:0/2105840386 shutdown_connections 2024-05-08T23:27:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47c4976700 1 --2- 172.21.15.5:0/2105840386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00b1260 0x7f47c015ba90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47c4976700 1 -- 172.21.15.5:0/2105840386 >> 172.21.15.5:0/2105840386 conn(0x7f47c0099a50 msgr2=0x7f47c00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47c4976700 1 -- 172.21.15.5:0/2105840386 shutdown_connections 2024-05-08T23:27:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.589+0000 7f47c4976700 1 -- 172.21.15.5:0/2105840386 wait complete. 2024-05-08T23:27:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.591+0000 7f47c4976700 1 Processor -- start 2024-05-08T23:27:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.591+0000 7f47c4976700 1 -- start start 2024-05-08T23:27:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47c4976700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b1260 0x7f47c0146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47c4976700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47c00cbfc0 0x7f47c01f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47c4976700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00d1070 0x7f47c006f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47c4976700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01f60f0 con 0x7f47c00b1260 2024-05-08T23:27:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47c4976700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c00c8ce0 con 0x7f47c00d1070 2024-05-08T23:27:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47c4976700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01f5e10 con 0x7f47c00cbfc0 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b1260 0x7f47c0146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b1260 0x7f47c0146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37248/0 (socket says 172.21.15.5:37248) 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 -- 172.21.15.5:0/2288371219 learned_addr learned my addr 172.21.15.5:0/2288371219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47bed9d700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00d1070 0x7f47c006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47bdd9b700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47c00cbfc0 0x7f47c01f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 -- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47c00cbfc0 msgr2=0x7f47c01f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47c00cbfc0 0x7f47c01f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 -- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00d1070 msgr2=0x7f47c006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00d1070 0x7f47c006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 -- 172.21.15.5:0/2288371219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0009a70 con 0x7f47c00b1260 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47be59c700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b1260 0x7f47c0146330 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f47b000b370 tx=0x7f47b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.592+0000 7f47b6ffd700 1 -- 172.21.15.5:0/2288371219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b00045c0 con 0x7f47c00b1260 2024-05-08T23:27:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.593+0000 7f47b6ffd700 1 -- 172.21.15.5:0/2288371219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0004740 con 0x7f47c00b1260 2024-05-08T23:27:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.593+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c00c8f80 con 0x7f47c00b1260 2024-05-08T23:27:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.593+0000 7f47b6ffd700 1 -- 172.21.15.5:0/2288371219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0012430 con 0x7f47c00b1260 2024-05-08T23:27:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.593+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c00c9540 con 0x7f47c00b1260 2024-05-08T23:27:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.594+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c0068000 con 0x7f47c00b1260 2024-05-08T23:27:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.594+0000 7f47b6ffd700 1 -- 172.21.15.5:0/2288371219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f47b0012590 con 0x7f47c00b1260 2024-05-08T23:27:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.595+0000 7f47b6ffd700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47a006ac90 0x7f47a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.595+0000 7f47b6ffd700 1 -- 172.21.15.5:0/2288371219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47b008a8a0 con 0x7f47c00b1260 2024-05-08T23:27:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.595+0000 7f47bdd9b700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47a006ac90 0x7f47a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.596+0000 7f47bdd9b700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47a006ac90 0x7f47a006d150 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f47c01578b0 tx=0x7f47a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.599+0000 7f47b6ffd700 1 -- 172.21.15.5:0/2288371219 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f47b0059520 con 0x7f47c00b1260 2024-05-08T23:27:02.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.870+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f47c006b740 con 0x7f47c00b1260 2024-05-08T23:27:02.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.870+0000 7f47b6ffd700 1 -- 172.21.15.5:0/2288371219 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f47b00590b0 con 0x7f47c00b1260 2024-05-08T23:27:02.870 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:27:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47a006ac90 msgr2=0x7f47a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47a006ac90 0x7f47a006d150 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f47c01578b0 tx=0x7f47a8009040 comp rx=0 tx=0).stop 2024-05-08T23:27:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b1260 msgr2=0x7f47c0146330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b1260 0x7f47c0146330 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f47b000b370 tx=0x7f47b0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 shutdown_connections 2024-05-08T23:27:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f47a006ac90 0x7f47a006d150 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47c00d1070 0x7f47c006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b1260 0x7f47c0146330 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 --2- 172.21.15.5:0/2288371219 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f47c00cbfc0 0x7f47c01f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 >> 172.21.15.5:0/2288371219 conn(0x7f47c0099a50 msgr2=0x7f47c0153770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 shutdown_connections 2024-05-08T23:27:02.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:02.873+0000 7f47c4976700 1 -- 172.21.15.5:0/2288371219 wait complete. 2024-05-08T23:27:02.886 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T23:27:02.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:27:02.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-08T23:27:02.887 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:27:02.887 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:27:02.887 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T23:27:02.887 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T23:27:02.887 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:27:02.887 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:27:02.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:02.888 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-08T23:27:02.907 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:27:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:27:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:27:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:27:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:27:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:27:02.907 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41894sec preferred_lft 41894sec 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-08T23:27:02.908 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::1c2e:b3ff:fe6b:56f/64 scope link 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::84a4:2aff:fe7c:ce1a/64 scope link 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether e2:3e:2c:7a:5d:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::e03e:2cff:fe7a:5db1/64 scope link 2024-05-08T23:27:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:27:02.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:02.910 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:27:02.979 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-08T23:27:02.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:02.980 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-08T23:27:03.046 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-08T23:27:03.046 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:27:03.046 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:27:03.046 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:27:03.046 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:27:03.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:03.047 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:27:03.047 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:27:03.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:27:03.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:27:03.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:27:03.047 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:27:03.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:03.171 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:27:03.191 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:27:03.192 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:27:03.192 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:27:03.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:03.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-08T23:27:03.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:27:03.268 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:27:03.330 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:27:03.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:27:03.331 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:27:03.393 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T23:27:03.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:03.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:27:03.491 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:27:03.492 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:27:03.513 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:27:03.513 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:27:03.513 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:27:03.513 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:27:03.513 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:27:03.514 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:27:03.514 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:27:03.514 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:27:03.532 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:27:03.532 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:27:03.532 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:27:03.532 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:27:03.532 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:27:03.532 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:27:03.532 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:27:03.532 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:27:03.533 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:27:03.533 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:27:03.533 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:27:03.535 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:27:03.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:03.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:27:03.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:03.568 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:27:03.663 INFO:teuthology.orchestra.run.smithi005.stdout:client6584 2024-05-08T23:27:03.663 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:27:03.664 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:03.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T23:27:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.875+0000 7f54390fd700 1 Processor -- start 2024-05-08T23:27:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.875+0000 7f54390fd700 1 -- start start 2024-05-08T23:27:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.876+0000 7f54390fd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f54340d3d20 0x7f54340b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.876+0000 7f54390fd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 0x7f54340b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.876+0000 7f54390fd700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54340cf610 con 0x7f54340cff30 2024-05-08T23:27:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.876+0000 7f54390fd700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54340d1fc0 con 0x7f54340d3d20 2024-05-08T23:27:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.876+0000 7f54390fd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54340d22a0 con 0x7f54340b3fc0 2024-05-08T23:27:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.876+0000 7f543259c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f54340d3d20 0x7f54340b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.876+0000 7f543259c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f54340d3d20 0x7f54340b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37256/0 (socket says 172.21.15.5:37256) 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.876+0000 7f543259c700 1 -- 172.21.15.5:0/2269827730 learned_addr learned my addr 172.21.15.5:0/2269827730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f543359e700 1 --2- 172.21.15.5:0/2269827730 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 0x7f54340b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f543259c700 1 -- 172.21.15.5:0/2269827730 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 msgr2=0x7f54340b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f543259c700 1 --2- 172.21.15.5:0/2269827730 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 0x7f54340b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f543259c700 1 -- 172.21.15.5:0/2269827730 >> v1:172.21.15.49:6789/0 conn(0x7f54340cff30 legacy=0x7f54340d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f543259c700 1 -- 172.21.15.5:0/2269827730 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54340d2580 con 0x7f54340d3d20 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f543259c700 1 --2- 172.21.15.5:0/2269827730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340d3d20 0x7f54340b3a80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f541c0098f0 tx=0x7f541c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17e76d1855fda4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f5431d9b700 1 -- 172.21.15.5:0/2269827730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c004230 con 0x7f54340d3d20 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f5431d9b700 1 -- 172.21.15.5:0/2269827730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c004390 con 0x7f54340d3d20 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.877+0000 7f5431d9b700 1 -- 172.21.15.5:0/2269827730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c0036a0 con 0x7f54340d3d20 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.878+0000 7f54390fd700 1 -- 172.21.15.5:0/2269827730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340d3d20 msgr2=0x7f54340b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.878+0000 7f54390fd700 1 --2- 172.21.15.5:0/2269827730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340d3d20 0x7f54340b3a80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f541c0098f0 tx=0x7f541c009c00 comp rx=0 tx=0).stop 2024-05-08T23:27:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.878+0000 7f54390fd700 1 -- 172.21.15.5:0/2269827730 shutdown_connections 2024-05-08T23:27:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.878+0000 7f54390fd700 1 --2- 172.21.15.5:0/2269827730 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 0x7f54340b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.878+0000 7f54390fd700 1 --2- 172.21.15.5:0/2269827730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340d3d20 0x7f54340b3a80 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.878+0000 7f54390fd700 1 -- 172.21.15.5:0/2269827730 >> 172.21.15.5:0/2269827730 conn(0x7f543408afe0 msgr2=0x7f54340d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.878+0000 7f54390fd700 1 -- 172.21.15.5:0/2269827730 shutdown_connections 2024-05-08T23:27:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.878+0000 7f54390fd700 1 -- 172.21.15.5:0/2269827730 wait complete. 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.879+0000 7f54390fd700 1 Processor -- start 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.879+0000 7f54390fd700 1 -- start start 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f54390fd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 0x7f543415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f54390fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340cff30 0x7f543415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f54390fd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54340d3d20 0x7f543415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f54390fd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543415b2f0 con 0x7f54340cff30 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f54390fd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543415f430 con 0x7f54340d3d20 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f54390fd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54340b7750 con 0x7f54340b3fc0 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f543259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340cff30 0x7f543415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f543259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340cff30 0x7f543415ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37268/0 (socket says 172.21.15.5:37268) 2024-05-08T23:27:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.880+0000 7f543259c700 1 -- 172.21.15.5:0/1450402681 learned_addr learned my addr 172.21.15.5:0/1450402681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f543359e700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54340d3d20 0x7f543415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f5432d9d700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 0x7f543415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f543259c700 1 -- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 msgr2=0x7f543415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f543259c700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 0x7f543415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f543259c700 1 -- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54340d3d20 msgr2=0x7f543415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f543259c700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54340d3d20 0x7f543415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f543259c700 1 -- 172.21.15.5:0/1450402681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c009550 con 0x7f54340cff30 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f543259c700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340cff30 0x7f543415ee30 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f541c000c00 tx=0x7f541c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f542b7fe700 1 -- 172.21.15.5:0/1450402681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00b860 con 0x7f54340cff30 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f542b7fe700 1 -- 172.21.15.5:0/1450402681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c00b9c0 con 0x7f54340cff30 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54340b7a30 con 0x7f54340cff30 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f542b7fe700 1 -- 172.21.15.5:0/1450402681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00bd70 con 0x7f54340cff30 2024-05-08T23:27:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.881+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54340b7ff0 con 0x7f54340cff30 2024-05-08T23:27:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.882+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5434074bc0 con 0x7f54340cff30 2024-05-08T23:27:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.883+0000 7f542b7fe700 1 -- 172.21.15.5:0/1450402681 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f541c01b440 con 0x7f54340cff30 2024-05-08T23:27:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.883+0000 7f542b7fe700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f541406ac90 0x7f541406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.883+0000 7f542b7fe700 1 -- 172.21.15.5:0/1450402681 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f541c08ab70 con 0x7f54340cff30 2024-05-08T23:27:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.883+0000 7f5432d9d700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f541406ac90 0x7f541406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.884+0000 7f5432d9d700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f541406ac90 0x7f541406d150 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f54341f6b50 tx=0x7f5424009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:03.888+0000 7f542b7fe700 1 -- 172.21.15.5:0/1450402681 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f541c055d40 con 0x7f54340cff30 2024-05-08T23:27:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.154+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f54340d4520 con 0x7f54340cff30 2024-05-08T23:27:04.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.226+0000 7f542b7fe700 1 -- 172.21.15.5:0/1450402681 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v83) v1 ==== 105+0+0 (secure 0 0 0) 0x7f541c059360 con 0x7f54340cff30 2024-05-08T23:27:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f541406ac90 msgr2=0x7f541406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f541406ac90 0x7f541406d150 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f54341f6b50 tx=0x7f5424009040 comp rx=0 tx=0).stop 2024-05-08T23:27:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340cff30 msgr2=0x7f543415ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340cff30 0x7f543415ee30 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f541c000c00 tx=0x7f541c004750 comp rx=0 tx=0).stop 2024-05-08T23:27:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 shutdown_connections 2024-05-08T23:27:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f541406ac90 0x7f541406d150 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54340d3d20 0x7f543415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f54340b3fc0 0x7f543415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 --2- 172.21.15.5:0/1450402681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54340cff30 0x7f543415ee30 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 >> 172.21.15.5:0/1450402681 conn(0x7f543408afe0 msgr2=0x7f54340d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 shutdown_connections 2024-05-08T23:27:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.230+0000 7f54390fd700 1 -- 172.21.15.5:0/1450402681 wait complete. 2024-05-08T23:27:04.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.387+0000 7f641f601700 1 Processor -- start 2024-05-08T23:27:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.388+0000 7f641f601700 1 -- start start 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641f601700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f641806f130 0x7f641806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641f601700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f641815c280 con 0x7f64180cecf0 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641f601700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f641806e780 con 0x7f64180cd4e0 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641f601700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641806ea60 con 0x7f641806f130 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641cb9c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f64180cecf0 0x7f641815c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47142/0 (socket says 172.21.15.5:47142) 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641cb9c700 1 -- 172.21.15.5:0/3889384047 learned_addr learned my addr 172.21.15.5:0/3889384047 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641db9e700 1 --2- 172.21.15.5:0/3889384047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f641806f130 0x7f641806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f640ffff700 1 -- 172.21.15.5:0/3889384047 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243610781 0 0) 0x7f641815c280 con 0x7f64180cecf0 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f640ffff700 1 -- 172.21.15.5:0/3889384047 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64040035b0 con 0x7f64180cecf0 2024-05-08T23:27:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f640ffff700 1 -- 172.21.15.5:0/3889384047 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438377595 0 0) 0x7f641806e780 con 0x7f64180cd4e0 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f640ffff700 1 -- 172.21.15.5:0/3889384047 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f641815c280 con 0x7f64180cd4e0 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641db9e700 1 -- 172.21.15.5:0/3889384047 >> v1:172.21.15.49:6789/0 conn(0x7f64180cd4e0 legacy=0x7f64180ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641db9e700 1 -- 172.21.15.5:0/3889384047 >> v1:172.21.15.5:6789/0 conn(0x7f64180cecf0 legacy=0x7f641815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.389+0000 7f641db9e700 1 -- 172.21.15.5:0/3889384047 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641806ed40 con 0x7f641806f130 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f641db9e700 1 --2- 172.21.15.5:0/3889384047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f641806f130 0x7f641806f910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6410009dc0 tx=0x7f641000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18bba6bf2c928ce1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f640ffff700 1 -- 172.21.15.5:0/3889384047 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641000e070 con 0x7f641806f130 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f640ffff700 1 -- 172.21.15.5:0/3889384047 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641000bb20 con 0x7f641806f130 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f640ffff700 1 -- 172.21.15.5:0/3889384047 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641000be90 con 0x7f641806f130 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f641f601700 1 -- 172.21.15.5:0/3889384047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f641806f130 msgr2=0x7f641806f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f641f601700 1 --2- 172.21.15.5:0/3889384047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f641806f130 0x7f641806f910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6410009dc0 tx=0x7f641000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f641f601700 1 -- 172.21.15.5:0/3889384047 shutdown_connections 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f641f601700 1 --2- 172.21.15.5:0/3889384047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f641806f130 0x7f641806f910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.390+0000 7f641f601700 1 -- 172.21.15.5:0/3889384047 >> 172.21.15.5:0/3889384047 conn(0x7f64180d0320 msgr2=0x7f64180d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.391+0000 7f641f601700 1 -- 172.21.15.5:0/3889384047 shutdown_connections 2024-05-08T23:27:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.391+0000 7f641f601700 1 -- 172.21.15.5:0/3889384047 wait complete. 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.392+0000 7f641f601700 1 Processor -- start 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.392+0000 7f641f601700 1 -- start start 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641f601700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f641806f130 0x7f641815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641f601700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64180cd4e0 0x7f6418157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641f601700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180cecf0 0x7f64180cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641f601700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64180c47e0 con 0x7f64180cecf0 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641f601700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64180c4a80 con 0x7f641806f130 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641f601700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641815a590 con 0x7f64180cd4e0 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641db9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180cecf0 0x7f64180cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641db9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180cecf0 0x7f64180cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37284/0 (socket says 172.21.15.5:37284) 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641db9e700 1 -- 172.21.15.5:0/3520965712 learned_addr learned my addr 172.21.15.5:0/3520965712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641cb9c700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64180cd4e0 0x7f6418157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641d39d700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f641806f130 0x7f641815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641db9e700 1 -- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64180cd4e0 msgr2=0x7f6418157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641db9e700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64180cd4e0 0x7f6418157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641db9e700 1 -- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f641806f130 msgr2=0x7f641815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641db9e700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f641806f130 0x7f641815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.393+0000 7f641db9e700 1 -- 172.21.15.5:0/3520965712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6410009a70 con 0x7f64180cecf0 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.394+0000 7f641db9e700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180cecf0 0x7f64180cacc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6410009fd0 tx=0x7f6410004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.394+0000 7f640dffb700 1 -- 172.21.15.5:0/3520965712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410003c20 con 0x7f64180cecf0 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.394+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641815a870 con 0x7f64180cecf0 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.394+0000 7f640dffb700 1 -- 172.21.15.5:0/3520965712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641001f070 con 0x7f64180cecf0 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.394+0000 7f640dffb700 1 -- 172.21.15.5:0/3520965712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64100098f0 con 0x7f64180cecf0 2024-05-08T23:27:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.394+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64180c4d60 con 0x7f64180cecf0 2024-05-08T23:27:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.396+0000 7f640dffb700 1 -- 172.21.15.5:0/3520965712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6410004030 con 0x7f64180cecf0 2024-05-08T23:27:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.396+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6418068000 con 0x7f64180cecf0 2024-05-08T23:27:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.396+0000 7f640dffb700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f63fc06ac90 0x7f63fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.397+0000 7f640dffb700 1 -- 172.21.15.5:0/3520965712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f641008ac00 con 0x7f64180cecf0 2024-05-08T23:27:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.397+0000 7f641d39d700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f63fc06ac90 0x7f63fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.397+0000 7f641d39d700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f63fc06ac90 0x7f63fc06d150 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64140016f0 tx=0x7f6414009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.401+0000 7f640dffb700 1 -- 172.21.15.5:0/3520965712 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6410091080 con 0x7f64180cecf0 2024-05-08T23:27:04.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.671+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f641806d440 con 0x7f64180cecf0 2024-05-08T23:27:04.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.672+0000 7f640dffb700 1 -- 172.21.15.5:0/3520965712 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f6410055df0 con 0x7f64180cecf0 2024-05-08T23:27:04.672 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:04.672 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7336,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:04.227681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7758,"mds_1":7755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:creating","state_seq":1,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:04.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f63fc06ac90 msgr2=0x7f63fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f63fc06ac90 0x7f63fc06d150 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64140016f0 tx=0x7f6414009040 comp rx=0 tx=0).stop 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180cecf0 msgr2=0x7f64180cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180cecf0 0x7f64180cacc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6410009fd0 tx=0x7f6410004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 shutdown_connections 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f63fc06ac90 0x7f63fc06d150 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180cecf0 0x7f64180cacc0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f641806f130 0x7f641815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 --2- 172.21.15.5:0/3520965712 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64180cd4e0 0x7f6418157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 >> 172.21.15.5:0/3520965712 conn(0x7f64180d0320 msgr2=0x7f641815c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 shutdown_connections 2024-05-08T23:27:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:04.674+0000 7f641f601700 1 -- 172.21.15.5:0/3520965712 wait complete. 2024-05-08T23:27:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 84 2024-05-08T23:27:05.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:26:56.191956+0000', 'modified': '2024-05-08T23:27:04.227681+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7758, 'mds_1': 7755}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7755': {'gid': 7755, 'name': 'c', 'rank': 1, 'incarnation': 84, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.5:6834/3718289875', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 3718289875}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 3718289875}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7758': {'gid': 7758, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/1093540582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 1093540582}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1093540582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:27:05.091 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7755:up:creating 2024-05-08T23:27:06.092 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.238+0000 7ff2ac10a700 1 Processor -- start 2024-05-08T23:27:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.238+0000 7ff2ac10a700 1 -- start start 2024-05-08T23:27:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2ac10a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2a40cdef0 0x7ff2a40cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2ac10a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40ce740 con 0x7ff2a40cc040 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2ac10a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40cb670 con 0x7ff2a415a630 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2ac10a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40cadc0 con 0x7ff2a40cdef0 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2a96a5700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2a40cdef0 0x7ff2a40cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2aa6a7700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff2a40cc040 0x7ff2a415e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47156/0 (socket says 172.21.15.5:47156) 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2aa6a7700 1 -- 172.21.15.5:0/3402968642 learned_addr learned my addr 172.21.15.5:0/3402968642 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2a96a5700 1 -- 172.21.15.5:0/3402968642 >> v1:172.21.15.49:6789/0 conn(0x7ff2a415a630 legacy=0x7ff2a40d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2a96a5700 1 -- 172.21.15.5:0/3402968642 >> v1:172.21.15.5:6789/0 conn(0x7ff2a40cc040 legacy=0x7ff2a415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2a96a5700 1 -- 172.21.15.5:0/3402968642 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40cb060 con 0x7ff2a40cdef0 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.239+0000 7ff2a8ea4700 1 -- 172.21.15.5:0/3402968642 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758113290 0 0) 0x7ff29c001490 con 0x7ff2a40cc040 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2a96a5700 1 --2- 172.21.15.5:0/3402968642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a40cdef0 0x7ff2a40cbb00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff294009840 tx=0x7ff294009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43292c377bc43a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2a8ea4700 1 -- 172.21.15.5:0/3402968642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294011830 con 0x7ff2a40cdef0 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2a8ea4700 1 -- 172.21.15.5:0/3402968642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff294011990 con 0x7ff2a40cdef0 2024-05-08T23:27:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2a8ea4700 1 -- 172.21.15.5:0/3402968642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294011d20 con 0x7ff2a40cdef0 2024-05-08T23:27:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2ac10a700 1 -- 172.21.15.5:0/3402968642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a40cdef0 msgr2=0x7ff2a40cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2ac10a700 1 --2- 172.21.15.5:0/3402968642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a40cdef0 0x7ff2a40cbb00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff294009840 tx=0x7ff294009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2ac10a700 1 -- 172.21.15.5:0/3402968642 shutdown_connections 2024-05-08T23:27:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2ac10a700 1 --2- 172.21.15.5:0/3402968642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a40cdef0 0x7ff2a40cbb00 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:06.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.240+0000 7ff2ac10a700 1 -- 172.21.15.5:0/3402968642 >> 172.21.15.5:0/3402968642 conn(0x7ff2a4099a50 msgr2=0x7ff2a40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:06.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.241+0000 7ff2ac10a700 1 -- 172.21.15.5:0/3402968642 shutdown_connections 2024-05-08T23:27:06.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.241+0000 7ff2ac10a700 1 -- 172.21.15.5:0/3402968642 wait complete. 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.242+0000 7ff2ac10a700 1 Processor -- start 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.242+0000 7ff2ac10a700 1 -- start start 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2ac10a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2a40cc040 0x7ff2a40b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2ac10a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2a40cdef0 0x7ff2a40b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2ac10a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a415a630 0x7ff2a415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2ac10a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a414f280 con 0x7ff2a415a630 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2ac10a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40b5d80 con 0x7ff2a40cc040 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2ac10a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a41f4d90 con 0x7ff2a40cdef0 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a415a630 0x7ff2a415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a415a630 0x7ff2a415cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37302/0 (socket says 172.21.15.5:37302) 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 -- 172.21.15.5:0/2438646766 learned_addr learned my addr 172.21.15.5:0/2438646766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2a96a5700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2a40cdef0 0x7ff2a40b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2a9ea6700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2a40cc040 0x7ff2a40b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 -- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2a40cdef0 msgr2=0x7ff2a40b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2a40cdef0 0x7ff2a40b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 -- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2a40cc040 msgr2=0x7ff2a40b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2a40cc040 0x7ff2a40b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 -- 172.21.15.5:0/2438646766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940094a0 con 0x7ff2a415a630 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2aa6a7700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a415a630 0x7ff2a415cda0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff29c002230 tx=0x7ff29c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff29a7fc700 1 -- 172.21.15.5:0/2438646766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00aaa0 con 0x7ff2a415a630 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff29a7fc700 1 -- 172.21.15.5:0/2438646766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c00ac20 con 0x7ff2a415a630 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a41f5070 con 0x7ff2a415a630 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.243+0000 7ff29a7fc700 1 -- 172.21.15.5:0/2438646766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0070d0 con 0x7ff2a415a630 2024-05-08T23:27:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.244+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40b5020 con 0x7ff2a415a630 2024-05-08T23:27:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.245+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a41f5590 con 0x7ff2a415a630 2024-05-08T23:27:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.245+0000 7ff29a7fc700 1 -- 172.21.15.5:0/2438646766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff29c007230 con 0x7ff2a415a630 2024-05-08T23:27:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.245+0000 7ff29a7fc700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff28806ac90 0x7ff28806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:06.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.246+0000 7ff29a7fc700 1 -- 172.21.15.5:0/2438646766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff29c088cd0 con 0x7ff2a415a630 2024-05-08T23:27:06.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.246+0000 7ff2a9ea6700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff28806ac90 0x7ff28806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:06.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.246+0000 7ff2a9ea6700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff28806ac90 0x7ff28806d150 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0007fd0 tx=0x7ff2a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.251+0000 7ff29a7fc700 1 -- 172.21.15.5:0/2438646766 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff2a41f5590 con 0x7ff2a415a630 2024-05-08T23:27:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.516+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2a40ce740 con 0x7ff2a415a630 2024-05-08T23:27:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.516+0000 7ff29a7fc700 1 -- 172.21.15.5:0/2438646766 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4758 (secure 0 0 0) 0x7ff2a40ce740 con 0x7ff2a415a630 2024-05-08T23:27:06.516 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:06.517 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:06.233541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff28806ac90 msgr2=0x7ff28806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff28806ac90 0x7ff28806d150 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0007fd0 tx=0x7ff2a0009040 comp rx=0 tx=0).stop 2024-05-08T23:27:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a415a630 msgr2=0x7ff2a415cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a415a630 0x7ff2a415cda0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff29c002230 tx=0x7ff29c00cee0 comp rx=0 tx=0).stop 2024-05-08T23:27:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 shutdown_connections 2024-05-08T23:27:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff28806ac90 0x7ff28806d150 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2a415a630 0x7ff2a415cda0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2a40cc040 0x7ff2a40b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 --2- 172.21.15.5:0/2438646766 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2a40cdef0 0x7ff2a40b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 >> 172.21.15.5:0/2438646766 conn(0x7ff2a4099a50 msgr2=0x7ff2a40b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 shutdown_connections 2024-05-08T23:27:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:06.519+0000 7ff2ac10a700 1 -- 172.21.15.5:0/2438646766 wait complete. 2024-05-08T23:27:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 87 2024-05-08T23:27:06.720 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:27:06.720+0000 7fb52a33c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:27:06.720 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:27:06.720+0000 7fb52a33c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T23:27:06.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:26:56.191956+0000', 'modified': '2024-05-08T23:27:06.233541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7758, 'mds_1': 7755, 'mds_2': 7336}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7336': {'gid': 7336, 'name': 'b', 'rank': 2, 'incarnation': 86, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.49:6837/1134209849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6835', 'nonce': 1134209849}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 1134209849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7755': {'gid': 7755, 'name': 'c', 'rank': 1, 'incarnation': 84, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.5:6834/3718289875', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 3718289875}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 3718289875}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7758': {'gid': 7758, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/1093540582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 1093540582}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1093540582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:27:06.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:27:06.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:27:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.083+0000 7f3aa2193700 1 Processor -- start 2024-05-08T23:27:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.083+0000 7f3aa2193700 1 -- start start 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3aa2193700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a9c0c6960 0x7f3a9c09a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3aa2193700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 0x7f3a9c0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3aa2193700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c0c54c0 con 0x7f3a9c15ee90 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3aa2193700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cf030 con 0x7f3a9c0c6960 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3aa2193700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cf310 con 0x7f3a9c0cbf00 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9affd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a9c0c6960 0x7f3a9c09a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9b7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3a9c15ee90 0x7f3a9c099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47158/0 (socket says 172.21.15.5:47158) 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9b7fe700 1 -- 172.21.15.5:0/2793467009 learned_addr learned my addr 172.21.15.5:0/2793467009 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/2793467009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 0x7f3a9c0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9affd700 1 -- 172.21.15.5:0/2793467009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 msgr2=0x7f3a9c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9affd700 1 --2- 172.21.15.5:0/2793467009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 0x7f3a9c0cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9affd700 1 -- 172.21.15.5:0/2793467009 >> v1:172.21.15.5:6789/0 conn(0x7f3a9c15ee90 legacy=0x7f3a9c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9affd700 1 -- 172.21.15.5:0/2793467009 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0cf5f0 con 0x7f3a9c0c6960 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.084+0000 7f3a9a7fc700 1 -- 172.21.15.5:0/2793467009 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59254973 0 0) 0x7f3a9c0c54c0 con 0x7f3a9c15ee90 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3a9affd700 1 --2- 172.21.15.5:0/2793467009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c0c6960 0x7f3a9c09a160 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009840 tx=0x7f3a84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68af43187200a235 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3a9a7fc700 1 -- 172.21.15.5:0/2793467009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84011830 con 0x7f3a9c0c6960 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3a9a7fc700 1 -- 172.21.15.5:0/2793467009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84011990 con 0x7f3a9c0c6960 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3a9a7fc700 1 -- 172.21.15.5:0/2793467009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84011d20 con 0x7f3a9c0c6960 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3aa2193700 1 -- 172.21.15.5:0/2793467009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c0c6960 msgr2=0x7f3a9c09a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3aa2193700 1 --2- 172.21.15.5:0/2793467009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c0c6960 0x7f3a9c09a160 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009840 tx=0x7f3a84009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3aa2193700 1 -- 172.21.15.5:0/2793467009 shutdown_connections 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3aa2193700 1 --2- 172.21.15.5:0/2793467009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 0x7f3a9c0cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3aa2193700 1 --2- 172.21.15.5:0/2793467009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c0c6960 0x7f3a9c09a160 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3aa2193700 1 -- 172.21.15.5:0/2793467009 >> 172.21.15.5:0/2793467009 conn(0x7f3a9c0b2f50 msgr2=0x7f3a9c0b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3aa2193700 1 -- 172.21.15.5:0/2793467009 shutdown_connections 2024-05-08T23:27:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.085+0000 7f3aa2193700 1 -- 172.21.15.5:0/2793467009 wait complete. 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.087+0000 7f3aa2193700 1 Processor -- start 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.087+0000 7f3aa2193700 1 -- start start 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3aa2193700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c0c6960 0x7f3a9c06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3aa2193700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 0x7f3a9c1f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3aa2193700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c15ee90 0x7f3a9c1f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3aa2193700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c159430 con 0x7f3a9c15ee90 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3aa2193700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1f3e00 con 0x7f3a9c0c6960 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3aa2193700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1f6b70 con 0x7f3a9c0cbf00 2024-05-08T23:27:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c15ee90 0x7f3a9c1f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c15ee90 0x7f3a9c1f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37322/0 (socket says 172.21.15.5:37322) 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 -- 172.21.15.5:0/139548303 learned_addr learned my addr 172.21.15.5:0/139548303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c0c6960 0x7f3a9c06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9affd700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 0x7f3a9c1f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 -- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 msgr2=0x7f3a9c1f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 0x7f3a9c1f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 -- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c0c6960 msgr2=0x7f3a9c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c0c6960 0x7f3a9c06d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 -- 172.21.15.5:0/139548303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840094a0 con 0x7f3a9c15ee90 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.088+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c15ee90 0x7f3a9c1f38c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400efc0 tx=0x7f3a9400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.089+0000 7f3a93fff700 1 -- 172.21.15.5:0/139548303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94009ea0 con 0x7f3a9c15ee90 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.089+0000 7f3a93fff700 1 -- 172.21.15.5:0/139548303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9400f040 con 0x7f3a9c15ee90 2024-05-08T23:27:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.089+0000 7f3aa2193700 1 -- 172.21.15.5:0/139548303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c1f6e50 con 0x7f3a9c15ee90 2024-05-08T23:27:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.089+0000 7f3a93fff700 1 -- 172.21.15.5:0/139548303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9400ced0 con 0x7f3a9c15ee90 2024-05-08T23:27:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.089+0000 7f3aa2193700 1 -- 172.21.15.5:0/139548303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c06bbf0 con 0x7f3a9c15ee90 2024-05-08T23:27:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.090+0000 7f3a93fff700 1 -- 172.21.15.5:0/139548303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a94014480 con 0x7f3a9c15ee90 2024-05-08T23:27:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.091+0000 7f3a93fff700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a7c06ac90 0x7f3a7c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.091+0000 7f3a93fff700 1 -- 172.21.15.5:0/139548303 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a94089a90 con 0x7f3a9c15ee90 2024-05-08T23:27:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.091+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a7c06ac90 0x7f3a7c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.092+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a7c06ac90 0x7f3a7c06d150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000f60 tx=0x7f3a8c007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.093+0000 7f3aae34db80 1 Processor -- start 2024-05-08T23:27:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.093+0000 7f3aae34db80 1 -- start start 2024-05-08T23:27:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 0x5591bab35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab5ed80 0x5591bab5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3aae34db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591baa8b100 con 0x5591bab57750 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3aae34db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591baa88ac0 con 0x5591bab5ed80 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3aae34db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591baa843b0 con 0x5591bab32320 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3a9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3a9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab300c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37328/0 (socket says 172.21.15.5:37328) 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.094+0000 7f3a9b7fe700 1 -- 172.21.15.5:0/3762872944 learned_addr learned my addr 172.21.15.5:0/3762872944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9affd700 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 0x5591bab35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab5ed80 0x5591bab5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9b7fe700 1 -- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 msgr2=0x5591bab35050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 0x5591bab35050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9b7fe700 1 -- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab5ed80 msgr2=0x5591bab5f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab5ed80 0x5591bab5f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9b7fe700 1 -- 172.21.15.5:0/3762872944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591ba905360 con 0x5591bab57750 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab5ed80 0x5591bab5f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab300c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c004960 tx=0x7f3a8c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85482d98f634cd7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a917fa700 1 -- 172.21.15.5:0/3762872944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c001290 con 0x5591bab57750 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a917fa700 1 -- 172.21.15.5:0/3762872944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c0013f0 con 0x5591bab57750 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.095+0000 7f3a917fa700 1 -- 172.21.15.5:0/3762872944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c001760 con 0x5591bab57750 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 -- 172.21.15.5:0/3762872944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 msgr2=0x5591bab300c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab300c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c004960 tx=0x7f3a8c013000 comp rx=0 tx=0).stop 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 -- 172.21.15.5:0/3762872944 shutdown_connections 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab5ed80 0x5591bab5f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab300c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3762872944 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 0x5591bab35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 -- 172.21.15.5:0/3762872944 >> 172.21.15.5:0/3762872944 conn(0x5591baa921c0 msgr2=0x5591bab79d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 -- 172.21.15.5:0/3762872944 shutdown_connections 2024-05-08T23:27:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.096+0000 7f3aae34db80 1 -- 172.21.15.5:0/3762872944 wait complete. 2024-05-08T23:27:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.097+0000 7f3aae34db80 1 -- start start 2024-05-08T23:27:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.097+0000 7f3aae34db80 1 Processor -- start 2024-05-08T23:27:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 0x5591bab88f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab57750 0x5591bab60930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab5ed80 0x5591bab7d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3aae34db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591baa20510 con 0x5591bab5ed80 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3aae34db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591baa201b0 con 0x5591bab57750 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3aae34db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591ba907650 con 0x5591bab32320 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3aae34db80 10 client.0 _ll_register_callbacks cb 0x5591baa29ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab5ed80 0x5591bab7d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3a9b7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 0x5591bab88f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab5ed80 0x5591bab7d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37334/0 (socket says 172.21.15.5:37334) 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3a9bfff700 1 -- 172.21.15.5:0/3594973535 learned_addr learned my addr 172.21.15.5:0/3594973535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.098+0000 7f3a9affd700 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab57750 0x5591bab60930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a9bfff700 1 -- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 msgr2=0x5591bab88f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 0x5591bab88f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a9bfff700 1 -- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab57750 msgr2=0x5591bab60930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab57750 0x5591bab60930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a9bfff700 1 -- 172.21.15.5:0/3594973535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c013550 con 0x5591bab5ed80 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab5ed80 0x5591bab7d6d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3a94058950 tx=0x7f3a94058f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37221aa263b7494 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a7a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/3594973535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94070370 con 0x5591bab5ed80 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/3594973535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a94060db0 con 0x5591bab5ed80 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3aae34db80 1 -- 172.21.15.5:0/3594973535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591ba9dd220 con 0x5591bab5ed80 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/3594973535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a94060920 con 0x5591bab5ed80 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3aae34db80 10 client.7947 fetch_fsmap learned FSMap version 87 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3aae34db80 1 -- 172.21.15.5:0/3594973535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5591ba8b1b10 con 0x5591bab5ed80 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.099+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/3594973535 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5591ba9dd220 con 0x5591bab5ed80 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/3594973535 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 87) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f3a940547b0 con 0x5591bab5ed80 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3aae34db80 10 client.7947 fetch_fsmap finished waiting for FSMap version 87 2024-05-08T23:27:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3aae34db80 10 client.7947 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3aae34db80 20 client.7947 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5591bab62c50 0x5591bab7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3aae34db80 4 client.7947 mds_command: new command op to 7336 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3aae34db80 1 -- 172.21.15.5:0/3594973535 --> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591bab03790 con 0x5591bab62c50 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5591bab62c50 0x5591bab7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5591bab62c50 0x5591bab7c120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a414d46a96ba9bc8 server_cookie=185f57ab556c74ba in_seq=0 out_seq=0 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.100+0000 7f3a7a7fc700 10 client.7947 ms_handle_connect on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/3594973535 <== mds.2 v2:172.21.15.49:6835/1134209849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5591bab03790 con 0x5591bab62c50 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3a7a7fc700 10 client.7947 handle_command_reply: tid=0 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 client.7947 shutdown 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 -- 172.21.15.5:0/3594973535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab5ed80 msgr2=0x5591bab7d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab5ed80 0x5591bab7d6d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3a94058950 tx=0x7f3a94058f00 comp rx=0 tx=0).stop 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 -- 172.21.15.5:0/3594973535 shutdown_connections 2024-05-08T23:27:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab5ed80 0x5591bab7d6d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab32320 0x5591bab88f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5591bab62c50 0x5591bab7c120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 --2- 172.21.15.5:0/3594973535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab57750 0x5591bab60930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3a7a7fc700 0 client.7947 ms_handle_reset on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 -- 172.21.15.5:0/3594973535 >> 172.21.15.5:0/3594973535 conn(0x5591baa921c0 msgr2=0x5591bab5f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 -- 172.21.15.5:0/3594973535 shutdown_connections 2024-05-08T23:27:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 1 -- 172.21.15.5:0/3594973535 wait complete. 2024-05-08T23:27:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.101+0000 7f3aae34db80 20 client.7947 trim_cache size 0 max 16384 2024-05-08T23:27:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.121+0000 7f3aae34db80 1 Processor -- start 2024-05-08T23:27:07.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.121+0000 7f3aae34db80 1 -- start start 2024-05-08T23:27:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.121+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 0x5591bab2dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.121+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab2e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.121+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591bab5ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3aae34db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bab27060 con 0x5591bab57750 2024-05-08T23:27:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3aae34db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bab7d2a0 con 0x5591bab32320 2024-05-08T23:27:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3aae34db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591bab88620 con 0x5591bab5ed80 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab2e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab2e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37340/0 (socket says 172.21.15.5:37340) 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 -- 172.21.15.5:0/4047220742 learned_addr learned my addr 172.21.15.5:0/4047220742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 0x5591bab2dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591bab5ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 -- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 msgr2=0x5591bab5ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591bab5ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 -- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 msgr2=0x5591bab2dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 0x5591bab2dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 -- 172.21.15.5:0/4047220742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591baa4a510 con 0x5591bab57750 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591bab5ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a9affd700 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab2e1d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400a010 tx=0x7f3a84010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435f95afe33b5eef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a91ffb700 1 -- 172.21.15.5:0/4047220742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84020040 con 0x5591bab57750 2024-05-08T23:27:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a91ffb700 1 -- 172.21.15.5:0/4047220742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84011d00 con 0x5591bab57750 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.122+0000 7f3a91ffb700 1 -- 172.21.15.5:0/4047220742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84004280 con 0x5591bab57750 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 -- 172.21.15.5:0/4047220742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 msgr2=0x5591bab2e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab2e1d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400a010 tx=0x7f3a84010000 comp rx=0 tx=0).stop 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 -- 172.21.15.5:0/4047220742 shutdown_connections 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591bab5ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 0x5591bab2dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 --2- 172.21.15.5:0/4047220742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab2e1d0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 -- 172.21.15.5:0/4047220742 >> 172.21.15.5:0/4047220742 conn(0x5591baa921c0 msgr2=0x5591bab88d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 -- 172.21.15.5:0/4047220742 shutdown_connections 2024-05-08T23:27:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.123+0000 7f3aae34db80 1 -- 172.21.15.5:0/4047220742 wait complete. 2024-05-08T23:27:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.124+0000 7f3aae34db80 1 -- start start 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.124+0000 7f3aae34db80 1 Processor -- start 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 0x5591bab32700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab32c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3aae34db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591babe38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3aae34db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bab7c500 con 0x5591bab57750 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3aae34db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591babe48f0 con 0x5591bab32320 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3aae34db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591babe4d10 con 0x5591bab5ed80 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3aae34db80 10 client.0 _ll_register_callbacks cb 0x5591baa29ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab32c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab32c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37348/0 (socket says 172.21.15.5:37348) 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 -- 172.21.15.5:0/1115323174 learned_addr learned my addr 172.21.15.5:0/1115323174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9bfff700 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591babe38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 0x5591bab32700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 -- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 msgr2=0x5591babe38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591babe38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 -- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 msgr2=0x5591bab32700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 0x5591bab32700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 -- 172.21.15.5:0/1115323174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c0cf5f0 con 0x5591bab57750 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a9affd700 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab32c40 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3a84013000 tx=0x7f3a840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6945fe473f5b89c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a7a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/1115323174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84020040 con 0x5591bab57750 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.125+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/1115323174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a84004440 con 0x5591bab57750 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 1 -- 172.21.15.5:0/1115323174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591baa7ce70 con 0x5591bab57750 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/1115323174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a840036a0 con 0x5591bab57750 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 10 client.7953 fetch_fsmap learned FSMap version 87 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 1 -- 172.21.15.5:0/1115323174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5591ba909c80 con 0x5591bab57750 2024-05-08T23:27:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/1115323174 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5591baa7ce70 con 0x5591bab57750 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/1115323174 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 87) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f3a84003800 con 0x5591bab57750 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 10 client.7953 fetch_fsmap finished waiting for FSMap version 87 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 10 client.7953 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 20 client.7953 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5591babe3e10 0x5591babe41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 4 client.7953 mds_command: new command op to 7336 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3aae34db80 1 -- 172.21.15.5:0/1115323174 --> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5591ba8d2ca0 con 0x5591babe3e10 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.126+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5591babe3e10 0x5591babe41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3a7a7fc700 10 client.7953 ms_handle_connect on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3a9b7fe700 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5591babe3e10 0x5591babe41f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7cd1d42174604cd server_cookie=76175bc296b6cf19 in_seq=0 out_seq=0 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3a7a7fc700 1 -- 172.21.15.5:0/1115323174 <== mds.2 v2:172.21.15.49:6835/1134209849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5591ba8d2ca0 con 0x5591babe3e10 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3a7a7fc700 10 client.7953 handle_command_reply: tid=0 2024-05-08T23:27:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 client.7953 shutdown 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 -- 172.21.15.5:0/1115323174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 msgr2=0x5591bab32c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab32c40 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3a84013000 tx=0x7f3a840047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 -- 172.21.15.5:0/1115323174 shutdown_connections 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5591bab5ed80 0x5591babe38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5591babe3e10 0x5591babe41f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5591bab32320 0x5591bab32700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 --2- 172.21.15.5:0/1115323174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591bab57750 0x5591bab32c40 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3a7a7fc700 0 client.7953 ms_handle_reset on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 -- 172.21.15.5:0/1115323174 >> 172.21.15.5:0/1115323174 conn(0x5591baa921c0 msgr2=0x5591bab88d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 -- 172.21.15.5:0/1115323174 shutdown_connections 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 1 -- 172.21.15.5:0/1115323174 wait complete. 2024-05-08T23:27:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.127+0000 7f3aae34db80 20 client.7953 trim_cache size 0 max 16384 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7336, 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 1.8943798569999999, 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 87, 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 84, 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:27:07.134 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 16.182717434000001 2024-05-08T23:27:07.135 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 -- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a7c06ac90 msgr2=0x7f3a7c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a7c06ac90 0x7f3a7c06d150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000f60 tx=0x7f3a8c007e70 comp rx=0 tx=0).stop 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 -- 172.21.15.5:0/139548303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c15ee90 msgr2=0x7f3a9c1f38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c15ee90 0x7f3a9c1f38c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400efc0 tx=0x7f3a9400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 -- 172.21.15.5:0/139548303 shutdown_connections 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a7c06ac90 0x7f3a7c06d150 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9c15ee90 0x7f3a9c1f38c0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c0c6960 0x7f3a9c06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 --2- 172.21.15.5:0/139548303 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3a9c0cbf00 0x7f3a9c1f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 -- 172.21.15.5:0/139548303 >> 172.21.15.5:0/139548303 conn(0x7f3a9c0b2f50 msgr2=0x7f3a9c073930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 -- 172.21.15.5:0/139548303 shutdown_connections 2024-05-08T23:27:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.128+0000 7f3aa2193700 1 -- 172.21.15.5:0/139548303 wait complete. 2024-05-08T23:27:07.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:27:07.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.279+0000 7fe9bc585700 1 Processor -- start 2024-05-08T23:27:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.279+0000 7fe9bc585700 1 -- start start 2024-05-08T23:27:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.280+0000 7fe9bc585700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9b4155da0 0x7fe9b406e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.280+0000 7fe9bc585700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b415f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.280+0000 7fe9bc585700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b406be40 con 0x7fe9b415f660 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.280+0000 7fe9bc585700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b406dcb0 con 0x7fe9b4155da0 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.280+0000 7fe9bc585700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b406e740 con 0x7fe9b406d4f0 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.280+0000 7fe9bab22700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe9b415f660 0x7fe9b415bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47170/0 (socket says 172.21.15.5:47170) 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.280+0000 7fe9bab22700 1 -- 172.21.15.5:0/471693604 learned_addr learned my addr 172.21.15.5:0/471693604 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9ba321700 1 --2- 172.21.15.5:0/471693604 >> v2:172.21.15.49:3300/0 conn(0x7fe9b4155da0 0x7fe9b406e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/471693604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b931f700 1 -- 172.21.15.5:0/471693604 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215398655 0 0) 0x7fe9b406be40 con 0x7fe9b415f660 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b931f700 1 -- 172.21.15.5:0/471693604 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a00035b0 con 0x7fe9b415f660 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b9b20700 1 -- 172.21.15.5:0/471693604 >> v2:172.21.15.49:3300/0 conn(0x7fe9b4155da0 msgr2=0x7fe9b406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/471693604 >> v2:172.21.15.49:3300/0 conn(0x7fe9b4155da0 0x7fe9b406e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b9b20700 1 -- 172.21.15.5:0/471693604 >> v1:172.21.15.5:6789/0 conn(0x7fe9b415f660 legacy=0x7fe9b415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b9b20700 1 -- 172.21.15.5:0/471693604 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b406c2d0 con 0x7fe9b406d4f0 2024-05-08T23:27:07.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/471693604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b415f120 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009840 tx=0x7fe9b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca84a4026a64042 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.281+0000 7fe9b931f700 1 -- 172.21.15.5:0/471693604 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0011830 con 0x7fe9b406d4f0 2024-05-08T23:27:07.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9b931f700 1 -- 172.21.15.5:0/471693604 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0011990 con 0x7fe9b406d4f0 2024-05-08T23:27:07.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9b931f700 1 -- 172.21.15.5:0/471693604 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0011d20 con 0x7fe9b406d4f0 2024-05-08T23:27:07.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9bc585700 1 -- 172.21.15.5:0/471693604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 msgr2=0x7fe9b415f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9bc585700 1 --2- 172.21.15.5:0/471693604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b415f120 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009840 tx=0x7fe9b0009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9bc585700 1 -- 172.21.15.5:0/471693604 shutdown_connections 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9bc585700 1 --2- 172.21.15.5:0/471693604 >> v2:172.21.15.49:3300/0 conn(0x7fe9b4155da0 0x7fe9b406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9bc585700 1 --2- 172.21.15.5:0/471693604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b415f120 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9bc585700 1 -- 172.21.15.5:0/471693604 >> 172.21.15.5:0/471693604 conn(0x7fe9b40cc580 msgr2=0x7fe9b40cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9bc585700 1 -- 172.21.15.5:0/471693604 shutdown_connections 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.282+0000 7fe9bc585700 1 -- 172.21.15.5:0/471693604 wait complete. 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.283+0000 7fe9bc585700 1 Processor -- start 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.284+0000 7fe9bc585700 1 -- start start 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.284+0000 7fe9bc585700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b41572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.284+0000 7fe9bc585700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4155da0 0x7fe9b40cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.284+0000 7fe9bc585700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b415f660 0x7fe9b40cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.284+0000 7fe9bc585700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4157810 con 0x7fe9b4155da0 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.284+0000 7fe9bc585700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40cde20 con 0x7fe9b415f660 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.284+0000 7fe9bc585700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40cec40 con 0x7fe9b406d4f0 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.284+0000 7fe9b9b20700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4155da0 0x7fe9b40cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9b9b20700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4155da0 0x7fe9b40cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37360/0 (socket says 172.21.15.5:37360) 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9b9b20700 1 -- 172.21.15.5:0/3875965805 learned_addr learned my addr 172.21.15.5:0/3875965805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9ba321700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b41572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9bab22700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b415f660 0x7fe9b40cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9b9b20700 1 -- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 msgr2=0x7fe9b41572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b41572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9b9b20700 1 -- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b415f660 msgr2=0x7fe9b40cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b415f660 0x7fe9b40cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9b9b20700 1 -- 172.21.15.5:0/3875965805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b00094a0 con 0x7fe9b4155da0 2024-05-08T23:27:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4155da0 0x7fe9b40cd8e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00090b0 tx=0x7fe9b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9a6ffd700 1 -- 172.21.15.5:0/3875965805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0004590 con 0x7fe9b4155da0 2024-05-08T23:27:07.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9a6ffd700 1 -- 172.21.15.5:0/3875965805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b001f070 con 0x7fe9b4155da0 2024-05-08T23:27:07.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9bc585700 1 -- 172.21.15.5:0/3875965805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b40ce030 con 0x7fe9b4155da0 2024-05-08T23:27:07.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9a6ffd700 1 -- 172.21.15.5:0/3875965805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00038e0 con 0x7fe9b4155da0 2024-05-08T23:27:07.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.285+0000 7fe9bc585700 1 -- 172.21.15.5:0/3875965805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b40ce8d0 con 0x7fe9b4155da0 2024-05-08T23:27:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.287+0000 7fe9a6ffd700 1 -- 172.21.15.5:0/3875965805 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe9b0003a40 con 0x7fe9b4155da0 2024-05-08T23:27:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.288+0000 7fe9a6ffd700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe99806acc0 0x7fe99806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.288+0000 7fe9a6ffd700 1 -- 172.21.15.5:0/3875965805 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9b0021070 con 0x7fe9b4155da0 2024-05-08T23:27:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.288+0000 7fe9ba321700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe99806acc0 0x7fe99806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.289+0000 7fe9ba321700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe99806acc0 0x7fe99806d180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009be0 tx=0x7fe9a8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.290+0000 7fe9c873fb80 1 Processor -- start 2024-05-08T23:27:07.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.290+0000 7fe9c873fb80 1 -- start start 2024-05-08T23:27:07.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.290+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c919750 0x56324c8f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c8f4320 0x56324c8f7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c921200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324c84d100 con 0x56324c919750 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324c84aac0 con 0x56324c8f4320 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56324c8463b0 con 0x56324c920d80 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c919750 0x56324c8f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c919750 0x56324c8f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37362/0 (socket says 172.21.15.5:37362) 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 -- 172.21.15.5:0/972705438 learned_addr learned my addr 172.21.15.5:0/972705438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9bab22700 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c921200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c8f4320 0x56324c8f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 -- 172.21.15.5:0/972705438 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 msgr2=0x56324c921200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c921200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 -- 172.21.15.5:0/972705438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c8f4320 msgr2=0x56324c8f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c8f4320 0x56324c8f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 -- 172.21.15.5:0/972705438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324c6c7360 con 0x56324c919750 2024-05-08T23:27:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe9ba321700 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c919750 0x56324c8f20c0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8004530 tx=0x7fe9a8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d117b38d91beb65e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe997fff700 1 -- 172.21.15.5:0/972705438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a801c040 con 0x56324c919750 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe997fff700 1 -- 172.21.15.5:0/972705438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8004560 con 0x56324c919750 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.291+0000 7fe997fff700 1 -- 172.21.15.5:0/972705438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8003ab0 con 0x56324c919750 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 -- 172.21.15.5:0/972705438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c919750 msgr2=0x56324c8f20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c919750 0x56324c8f20c0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8004530 tx=0x7fe9a8014000 comp rx=0 tx=0).stop 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 -- 172.21.15.5:0/972705438 shutdown_connections 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c921200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c919750 0x56324c8f20c0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/972705438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c8f4320 0x56324c8f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 -- 172.21.15.5:0/972705438 >> 172.21.15.5:0/972705438 conn(0x56324c8541c0 msgr2=0x56324c93bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 -- 172.21.15.5:0/972705438 shutdown_connections 2024-05-08T23:27:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.292+0000 7fe9c873fb80 1 -- 172.21.15.5:0/972705438 wait complete. 2024-05-08T23:27:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.293+0000 7fe9c873fb80 1 -- start start 2024-05-08T23:27:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.293+0000 7fe9c873fb80 1 Processor -- start 2024-05-08T23:27:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 0x56324c93fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c94e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324c7e2510 con 0x56324c8f4320 2024-05-08T23:27:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324c7e21b0 con 0x56324c919750 2024-05-08T23:27:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56324c6c9650 con 0x56324c920d80 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9c873fb80 10 client.0 _ll_register_callbacks cb 0x56324c7ebae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9ba321700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9ba321700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9244b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37374/0 (socket says 172.21.15.5:37374) 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.294+0000 7fe9ba321700 1 -- 172.21.15.5:0/1222331326 learned_addr learned my addr 172.21.15.5:0/1222331326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 0x56324c93fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9bab22700 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c94e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9ba321700 1 -- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 msgr2=0x56324c94e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9ba321700 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c94e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9ba321700 1 -- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 msgr2=0x56324c93fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9ba321700 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 0x56324c93fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9ba321700 1 -- 172.21.15.5:0/1222331326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a8004210 con 0x56324c8f4320 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9bab22700 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c94e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9ba321700 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9244b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8012010 tx=0x7fe9a80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a66d384e0139ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9957fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9957fa700 1 -- 172.21.15.5:0/1222331326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8022670 con 0x56324c8f4320 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9957fa700 1 -- 172.21.15.5:0/1222331326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9a8037ae0 con 0x56324c8f4320 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1222331326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56324c79f220 con 0x56324c8f4320 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9957fa700 1 -- 172.21.15.5:0/1222331326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9a8037ca0 con 0x56324c8f4320 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9c873fb80 10 client.7965 fetch_fsmap learned FSMap version 87 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1222331326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x56324c673b10 con 0x56324c8f4320 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9957fa700 1 -- 172.21.15.5:0/1222331326 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56324c79f220 con 0x56324c8f4320 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.295+0000 7fe9957fa700 1 -- 172.21.15.5:0/1222331326 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 87) v1 ==== 3138+0+0 (secure 0 0 0) 0x7fe9a80227d0 con 0x56324c8f4320 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9c873fb80 10 client.7965 fetch_fsmap finished waiting for FSMap version 87 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9c873fb80 10 client.7965 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9c873fb80 20 client.7965 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x56324c90f630 0x56324c93d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9c873fb80 4 client.7965 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1222331326 --> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56324c8c5790 con 0x56324c90f630 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x56324c90f630 0x56324c93d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x56324c90f630 0x56324c93d490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99f4a3edb09c9643 server_cookie=ec30fd5fd7d52229 in_seq=0 out_seq=0 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.296+0000 7fe9957fa700 10 client.7965 ms_handle_connect on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9957fa700 1 -- 172.21.15.5:0/1222331326 <== mds.1 v2:172.21.15.5:6832/3718289875 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56324c8c5790 con 0x56324c90f630 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9957fa700 10 client.7965 handle_command_reply: tid=0 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 client.7965 shutdown 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1222331326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 msgr2=0x56324c9244b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9244b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8012010 tx=0x7fe9a80047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1222331326 shutdown_connections 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c94e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x56324c90f630 0x56324c93d490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9244b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1222331326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 0x56324c93fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9957fa700 0 client.7965 ms_handle_reset on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1222331326 >> 172.21.15.5:0/1222331326 conn(0x56324c8541c0 msgr2=0x56324c8f61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1222331326 shutdown_connections 2024-05-08T23:27:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1222331326 wait complete. 2024-05-08T23:27:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.297+0000 7fe9c873fb80 20 client.7965 trim_cache size 0 max 16384 2024-05-08T23:27:07.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.316+0000 7fe9c873fb80 1 Processor -- start 2024-05-08T23:27:07.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.316+0000 7fe9c873fb80 1 -- start start 2024-05-08T23:27:07.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 0x56324c918620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c931710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324c8f3f90 con 0x56324c8f4320 2024-05-08T23:27:07.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324c940260 con 0x56324c919750 2024-05-08T23:27:07.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56324c8d3010 con 0x56324c920d80 2024-05-08T23:27:07.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9ba321700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9ba321700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9180e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37386/0 (socket says 172.21.15.5:37386) 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.317+0000 7fe9ba321700 1 -- 172.21.15.5:0/3935988958 learned_addr learned my addr 172.21.15.5:0/3935988958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 0x56324c918620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9bab22700 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c931710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9ba321700 1 -- 172.21.15.5:0/3935988958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 msgr2=0x56324c931710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9ba321700 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c931710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9ba321700 1 -- 172.21.15.5:0/3935988958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 msgr2=0x56324c918620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9ba321700 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 0x56324c918620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9ba321700 1 -- 172.21.15.5:0/3935988958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324c80c510 con 0x56324c8f4320 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9ba321700 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9180e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800b010 tx=0x7fe9a802f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27861fbf8f04d7b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9a4ff9700 1 -- 172.21.15.5:0/3935988958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8022670 con 0x56324c8f4320 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9a4ff9700 1 -- 172.21.15.5:0/3935988958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8031070 con 0x56324c8f4320 2024-05-08T23:27:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.318+0000 7fe9a4ff9700 1 -- 172.21.15.5:0/3935988958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8022ce0 con 0x56324c8f4320 2024-05-08T23:27:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 -- 172.21.15.5:0/3935988958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 msgr2=0x56324c9180e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9180e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800b010 tx=0x7fe9a802f070 comp rx=0 tx=0).stop 2024-05-08T23:27:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 -- 172.21.15.5:0/3935988958 shutdown_connections 2024-05-08T23:27:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c920d80 0x56324c931710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c9180e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/3935988958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c919750 0x56324c918620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 -- 172.21.15.5:0/3935988958 >> 172.21.15.5:0/3935988958 conn(0x56324c8541c0 msgr2=0x56324c8effe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 -- 172.21.15.5:0/3935988958 shutdown_connections 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 -- 172.21.15.5:0/3935988958 wait complete. 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 -- start start 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.319+0000 7fe9c873fb80 1 Processor -- start 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.320+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c8f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.320+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c919750 0x56324c8f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.320+0000 7fe9c873fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c920d80 0x56324c989c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.320+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324c9a8220 con 0x56324c8f4320 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.320+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324c98d8c0 con 0x56324c920d80 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.320+0000 7fe9c873fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56324c91dcd0 con 0x56324c919750 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.320+0000 7fe9c873fb80 10 client.0 _ll_register_callbacks cb 0x56324c7ebae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c8f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c8f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37390/0 (socket says 172.21.15.5:37390) 2024-05-08T23:27:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 -- 172.21.15.5:0/1071309023 learned_addr learned my addr 172.21.15.5:0/1071309023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9bab22700 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c920d80 0x56324c989c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c919750 0x56324c8f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 -- 172.21.15.5:0/1071309023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c919750 msgr2=0x56324c8f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c919750 0x56324c8f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 -- 172.21.15.5:0/1071309023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c920d80 msgr2=0x56324c989c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c920d80 0x56324c989c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 -- 172.21.15.5:0/1071309023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56324c673b10 con 0x56324c8f4320 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9ba321700 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c8f5880 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x56324c932d10 tx=0x7fe9a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b389bda5620476c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9957fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9957fa700 1 -- 172.21.15.5:0/1071309023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8022670 con 0x56324c8f4320 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9957fa700 1 -- 172.21.15.5:0/1071309023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9a80227d0 con 0x56324c8f4320 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1071309023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9a8004560 con 0x56324c8f4320 2024-05-08T23:27:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9957fa700 1 -- 172.21.15.5:0/1071309023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9a8040470 con 0x56324c8f4320 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9c873fb80 10 client.7971 fetch_fsmap learned FSMap version 87 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.321+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1071309023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x56324c6cbc80 con 0x56324c8f4320 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9957fa700 1 -- 172.21.15.5:0/1071309023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8004560 con 0x56324c8f4320 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9957fa700 1 -- 172.21.15.5:0/1071309023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 87) v1 ==== 3138+0+0 (secure 0 0 0) 0x7fe9a80405d0 con 0x56324c8f4320 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9c873fb80 10 client.7971 fetch_fsmap finished waiting for FSMap version 87 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9c873fb80 10 client.7971 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9c873fb80 20 client.7971 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x56324c95d9a0 0x56324c98a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9c873fb80 4 client.7971 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1071309023 --> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56324c83ee70 con 0x56324c95d9a0 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x56324c95d9a0 0x56324c98a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9b9b20700 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x56324c95d9a0 0x56324c98a170 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe1a7e1e8235e719 server_cookie=aa478003cf2f875d in_seq=0 out_seq=0 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9957fa700 10 client.7971 ms_handle_connect on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9957fa700 1 -- 172.21.15.5:0/1071309023 <== mds.1 v2:172.21.15.5:6832/3718289875 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56324c83ee70 con 0x56324c95d9a0 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9957fa700 10 client.7971 handle_command_reply: tid=0 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.322+0000 7fe9c873fb80 1 client.7971 shutdown 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1071309023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 msgr2=0x56324c8f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c8f5880 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x56324c932d10 tx=0x7fe9a8018000 comp rx=0 tx=0).stop 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1071309023 shutdown_connections 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56324c920d80 0x56324c989c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x56324c95d9a0 0x56324c98a170 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324c8f4320 0x56324c8f5880 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 --2- 172.21.15.5:0/1071309023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56324c919750 0x56324c8f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9957fa700 0 client.7971 ms_handle_reset on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1071309023 >> 172.21.15.5:0/1071309023 conn(0x56324c8541c0 msgr2=0x56324c9180e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1071309023 shutdown_connections 2024-05-08T23:27:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 1 -- 172.21.15.5:0/1071309023 wait complete. 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9c873fb80 20 client.7971 trim_cache size 0 max 16384 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7755, 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 3.0943413280000001, 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 87, 2024-05-08T23:27:07.329 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 84, 2024-05-08T23:27:07.330 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:27:07.330 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 16.381970186 2024-05-08T23:27:07.330 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:07.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9bc585700 1 -- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe99806acc0 msgr2=0x7fe99806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9bc585700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe99806acc0 0x7fe99806d180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009be0 tx=0x7fe9a8009210 comp rx=0 tx=0).stop 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9bc585700 1 -- 172.21.15.5:0/3875965805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4155da0 msgr2=0x7fe9b40cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.323+0000 7fe9bc585700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4155da0 0x7fe9b40cd8e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00090b0 tx=0x7fe9b0004750 comp rx=0 tx=0).stop 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.324+0000 7fe9bc585700 1 -- 172.21.15.5:0/3875965805 shutdown_connections 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.324+0000 7fe9bc585700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe99806acc0 0x7fe99806d180 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.324+0000 7fe9bc585700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b415f660 0x7fe9b40cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.324+0000 7fe9bc585700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9b406d4f0 0x7fe9b41572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.324+0000 7fe9bc585700 1 --2- 172.21.15.5:0/3875965805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4155da0 0x7fe9b40cd8e0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.324+0000 7fe9bc585700 1 -- 172.21.15.5:0/3875965805 >> 172.21.15.5:0/3875965805 conn(0x7fe9b40cc580 msgr2=0x7fe9b415cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.324+0000 7fe9bc585700 1 -- 172.21.15.5:0/3875965805 shutdown_connections 2024-05-08T23:27:07.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.324+0000 7fe9bc585700 1 -- 172.21.15.5:0/3875965805 wait complete. 2024-05-08T23:27:07.337 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T23:27:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.474+0000 7fa32d2e3700 1 Processor -- start 2024-05-08T23:27:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.474+0000 7fa32d2e3700 1 -- start start 2024-05-08T23:27:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa32d2e3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa328160f00 0x7fa3280b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa32d2e3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa3280d22d0 0x7fa3280ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa32d2e3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3280ce3f0 con 0x7fa3280b2f60 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa32d2e3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3280cdc50 con 0x7fa3280d22d0 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa32d2e3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3280b42c0 con 0x7fa328160f00 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa3277fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa3280d22d0 0x7fa3280ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa3267fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa3280b2f60 0x7fa3280d1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44428/0 (socket says 172.21.15.5:44428) 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa3277fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa3280d22d0 0x7fa3280ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48350/0 (socket says 172.21.15.5:48350) 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.475+0000 7fa3267fc700 1 -- 172.21.15.5:0/2191836503 learned_addr learned my addr 172.21.15.5:0/2191836503 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa326ffd700 1 --2- 172.21.15.5:0/2191836503 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa328160f00 0x7fa3280b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa3277fe700 1 -- 172.21.15.5:0/2191836503 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa328160f00 msgr2=0x7fa3280b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa3277fe700 1 --2- 172.21.15.5:0/2191836503 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa328160f00 0x7fa3280b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa3277fe700 1 -- 172.21.15.5:0/2191836503 >> v1:172.21.15.5:6789/0 conn(0x7fa3280b2f60 legacy=0x7fa3280d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa3277fe700 1 -- 172.21.15.5:0/2191836503 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3280b3740 con 0x7fa3280d22d0 2024-05-08T23:27:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa325ffb700 1 -- 172.21.15.5:0/2191836503 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614647914 0 0) 0x7fa3280ce3f0 con 0x7fa3280b2f60 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa326ffd700 1 --2- 172.21.15.5:0/2191836503 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa328160f00 0x7fa3280b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa3277fe700 1 --2- 172.21.15.5:0/2191836503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280d22d0 0x7fa3280ce860 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fa320009d90 tx=0x7fa32000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdf025a96adbc2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa325ffb700 1 -- 172.21.15.5:0/2191836503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32000e070 con 0x7fa3280d22d0 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa325ffb700 1 -- 172.21.15.5:0/2191836503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32000bbd0 con 0x7fa3280d22d0 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.476+0000 7fa325ffb700 1 -- 172.21.15.5:0/2191836503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320012430 con 0x7fa3280d22d0 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.477+0000 7fa32d2e3700 1 -- 172.21.15.5:0/2191836503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280d22d0 msgr2=0x7fa3280ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.477+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/2191836503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280d22d0 0x7fa3280ce860 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fa320009d90 tx=0x7fa32000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.477+0000 7fa32d2e3700 1 -- 172.21.15.5:0/2191836503 shutdown_connections 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.477+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/2191836503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280d22d0 0x7fa3280ce860 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.477+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/2191836503 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa328160f00 0x7fa3280b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.477+0000 7fa32d2e3700 1 -- 172.21.15.5:0/2191836503 >> 172.21.15.5:0/2191836503 conn(0x7fa32808afe0 msgr2=0x7fa3280d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.477+0000 7fa32d2e3700 1 -- 172.21.15.5:0/2191836503 shutdown_connections 2024-05-08T23:27:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.477+0000 7fa32d2e3700 1 -- 172.21.15.5:0/2191836503 wait complete. 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.478+0000 7fa32d2e3700 1 Processor -- start 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.478+0000 7fa32d2e3700 1 -- start start 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa32d2e3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280b2f60 0x7fa328158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa32d2e3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3280d22d0 0x7fa3280b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa32d2e3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328160f00 0x7fa32815d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa32d2e3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3280b1270 con 0x7fa3280b2f60 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa32d2e3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3280b1550 con 0x7fa328160f00 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa32d2e3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3280b1830 con 0x7fa3280d22d0 2024-05-08T23:27:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa326ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280b2f60 0x7fa328158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa326ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280b2f60 0x7fa328158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48356/0 (socket says 172.21.15.5:48356) 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.479+0000 7fa326ffd700 1 -- 172.21.15.5:0/3777236175 learned_addr learned my addr 172.21.15.5:0/3777236175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa3267fc700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3280d22d0 0x7fa3280b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa3277fe700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328160f00 0x7fa32815d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa326ffd700 1 -- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3280d22d0 msgr2=0x7fa3280b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa326ffd700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3280d22d0 0x7fa3280b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa326ffd700 1 -- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328160f00 msgr2=0x7fa32815d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa326ffd700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328160f00 0x7fa32815d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa326ffd700 1 -- 172.21.15.5:0/3777236175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320009a70 con 0x7fa3280b2f60 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa3277fe700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328160f00 0x7fa32815d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa3267fc700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3280d22d0 0x7fa3280b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa326ffd700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280b2f60 0x7fa328158a60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fa31800e7f0 tx=0x7fa31800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa31f7fe700 1 -- 172.21.15.5:0/3777236175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318005c20 con 0x7fa3280b2f60 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa31f7fe700 1 -- 172.21.15.5:0/3777236175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa318005d80 con 0x7fa3280b2f60 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa32d2e3700 1 -- 172.21.15.5:0/3777236175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3280b1b10 con 0x7fa3280b2f60 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa31f7fe700 1 -- 172.21.15.5:0/3777236175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318006270 con 0x7fa3280b2f60 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.480+0000 7fa32d2e3700 1 -- 172.21.15.5:0/3777236175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3280b2310 con 0x7fa3280b2f60 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.482+0000 7fa31f7fe700 1 -- 172.21.15.5:0/3777236175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa3180063d0 con 0x7fa3280b2f60 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.482+0000 7fa31f7fe700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa30806ac90 0x7fa30806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.482+0000 7fa31f7fe700 1 -- 172.21.15.5:0/3777236175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa318088e70 con 0x7fa3280b2f60 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.482+0000 7fa3267fc700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa30806ac90 0x7fa30806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.483+0000 7fa3267fc700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa30806ac90 0x7fa30806d150 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa310000f60 tx=0x7fa310009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.484+0000 7fa33949db80 1 Processor -- start 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.484+0000 7fa33949db80 1 -- start start 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa33949db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55927207d750 0x5592720560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa33949db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272058320 0x55927205b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa33949db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x559272085200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa33949db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559271fb1100 con 0x559272084d80 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa33949db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559271faeac0 con 0x559272058320 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa33949db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559271faa3b0 con 0x55927207d750 2024-05-08T23:27:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa3277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x559272085200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa3277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x559272085200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48368/0 (socket says 172.21.15.5:48368) 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa3277fe700 1 -- 172.21.15.5:0/382583711 learned_addr learned my addr 172.21.15.5:0/382583711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa326ffd700 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55927207d750 0x5592720560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.485+0000 7fa3267fc700 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272058320 0x55927205b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa3277fe700 1 -- 172.21.15.5:0/382583711 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55927207d750 msgr2=0x5592720560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa3277fe700 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55927207d750 0x5592720560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa3277fe700 1 -- 172.21.15.5:0/382583711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272058320 msgr2=0x55927205b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa3277fe700 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272058320 0x55927205b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa3277fe700 1 -- 172.21.15.5:0/382583711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559271e2b360 con 0x559272084d80 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa3277fe700 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x559272085200 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fa320013000 tx=0x7fa320010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebc0826fa88ae3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa31cff9700 1 -- 172.21.15.5:0/382583711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320003c20 con 0x559272084d80 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa31cff9700 1 -- 172.21.15.5:0/382583711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320021070 con 0x559272084d80 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.486+0000 7fa31cff9700 1 -- 172.21.15.5:0/382583711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200125e0 con 0x559272084d80 2024-05-08T23:27:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 -- 172.21.15.5:0/382583711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 msgr2=0x559272085200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x559272085200 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fa320013000 tx=0x7fa320010000 comp rx=0 tx=0).stop 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 -- 172.21.15.5:0/382583711 shutdown_connections 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x559272085200 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55927207d750 0x5592720560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 --2- 172.21.15.5:0/382583711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272058320 0x55927205b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 -- 172.21.15.5:0/382583711 >> 172.21.15.5:0/382583711 conn(0x559271fb81c0 msgr2=0x55927209fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 -- 172.21.15.5:0/382583711 shutdown_connections 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.487+0000 7fa33949db80 1 -- 172.21.15.5:0/382583711 wait complete. 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.488+0000 7fa33949db80 1 -- start start 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.488+0000 7fa33949db80 1 Processor -- start 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa33949db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x55927207c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa33949db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x5592720b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa33949db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 0x559272085160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa33949db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559271f46510 con 0x55927207d750 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa33949db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559271f461b0 con 0x559272084d80 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa33949db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559271e2d650 con 0x559272058320 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa33949db80 10 client.0 _ll_register_callbacks cb 0x559271f4fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x5592720b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x5592720b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48382/0 (socket says 172.21.15.5:48382) 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 -- 172.21.15.5:0/1418785027 learned_addr learned my addr 172.21.15.5:0/1418785027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa326ffd700 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x55927207c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3277fe700 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 0x559272085160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 -- 172.21.15.5:0/1418785027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 msgr2=0x55927207c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x55927207c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 -- 172.21.15.5:0/1418785027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 msgr2=0x559272085160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 0x559272085160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 -- 172.21.15.5:0/1418785027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3280b3740 con 0x55927207d750 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa3267fc700 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x5592720b2fb0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fa310007650 tx=0x7fa310001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e761258b184ebaa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa305ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa305ffb700 1 -- 172.21.15.5:0/1418785027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310013b10 con 0x55927207d750 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa305ffb700 1 -- 172.21.15.5:0/1418785027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa310013c70 con 0x55927207d750 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.489+0000 7fa33949db80 1 -- 172.21.15.5:0/1418785027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559271f03220 con 0x55927207d750 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa305ffb700 1 -- 172.21.15.5:0/1418785027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa310013e30 con 0x55927207d750 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa33949db80 10 client.7986 fetch_fsmap learned FSMap version 87 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa33949db80 1 -- 172.21.15.5:0/1418785027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x559271dd7b10 con 0x55927207d750 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa305ffb700 1 -- 172.21.15.5:0/1418785027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559271f03220 con 0x55927207d750 2024-05-08T23:27:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa305ffb700 1 -- 172.21.15.5:0/1418785027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 87) v1 ==== 3138+0+0 (secure 0 0 0) 0x7fa31001f070 con 0x55927207d750 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa33949db80 10 client.7986 fetch_fsmap finished waiting for FSMap version 87 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa33949db80 10 client.7986 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa33949db80 20 client.7986 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa33949db80 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5592720a17d0 0x5592720a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa33949db80 4 client.7986 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa33949db80 1 -- 172.21.15.5:0/1418785027 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559272029790 con 0x5592720a17d0 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa326ffd700 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5592720a17d0 0x5592720a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa305ffb700 10 client.7986 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.490+0000 7fa326ffd700 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5592720a17d0 0x5592720a1bb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b870d1d18bfda570 server_cookie=dff9dcf51a4ef5b3 in_seq=0 out_seq=0 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa305ffb700 1 -- 172.21.15.5:0/1418785027 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559272029790 con 0x5592720a17d0 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa305ffb700 10 client.7986 handle_command_reply: tid=0 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 client.7986 shutdown 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 -- 172.21.15.5:0/1418785027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 msgr2=0x5592720b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x5592720b2fb0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fa310007650 tx=0x7fa310001080 comp rx=0 tx=0).stop 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 -- 172.21.15.5:0/1418785027 shutdown_connections 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5592720a17d0 0x5592720a1bb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 0x559272085160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x55927207c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa305ffb700 0 client.7986 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 --2- 172.21.15.5:0/1418785027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x5592720b2fb0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 -- 172.21.15.5:0/1418785027 >> 172.21.15.5:0/1418785027 conn(0x559271fb81c0 msgr2=0x559272081fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 -- 172.21.15.5:0/1418785027 shutdown_connections 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 1 -- 172.21.15.5:0/1418785027 wait complete. 2024-05-08T23:27:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.491+0000 7fa33949db80 20 client.7986 trim_cache size 0 max 16384 2024-05-08T23:27:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.511+0000 7fa33949db80 1 Processor -- start 2024-05-08T23:27:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.511+0000 7fa33949db80 1 -- start start 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.511+0000 7fa33949db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x55927209f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.511+0000 7fa33949db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x559272055850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa33949db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 0x5592720a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa33949db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927209f520 con 0x55927207d750 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa33949db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559272055430 con 0x559272084d80 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa33949db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592720aea10 con 0x559272058320 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x559272055850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x559272055850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48398/0 (socket says 172.21.15.5:48398) 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 -- 172.21.15.5:0/4022987230 learned_addr learned my addr 172.21.15.5:0/4022987230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa326ffd700 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x55927209f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3277fe700 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 0x5592720a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 -- 172.21.15.5:0/4022987230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 msgr2=0x55927209f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x55927209f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 -- 172.21.15.5:0/4022987230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 msgr2=0x5592720a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 0x5592720a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 -- 172.21.15.5:0/4022987230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559271f70510 con 0x55927207d750 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa3267fc700 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x559272055850 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d250 tx=0x7fa310019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0b24f8ed24def1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa31d7fa700 1 -- 172.21.15.5:0/4022987230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310011c10 con 0x55927207d750 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa31d7fa700 1 -- 172.21.15.5:0/4022987230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa310018030 con 0x55927207d750 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.512+0000 7fa31d7fa700 1 -- 172.21.15.5:0/4022987230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3100139e0 con 0x55927207d750 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 -- 172.21.15.5:0/4022987230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 msgr2=0x559272055850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x559272055850 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d250 tx=0x7fa310019000 comp rx=0 tx=0).stop 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 -- 172.21.15.5:0/4022987230 shutdown_connections 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559272084d80 0x5592720a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x55927209f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 --2- 172.21.15.5:0/4022987230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55927207d750 0x559272055850 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 -- 172.21.15.5:0/4022987230 >> 172.21.15.5:0/4022987230 conn(0x559271fb81c0 msgr2=0x5592720813b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 -- 172.21.15.5:0/4022987230 shutdown_connections 2024-05-08T23:27:07.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.513+0000 7fa33949db80 1 -- 172.21.15.5:0/4022987230 wait complete. 2024-05-08T23:27:07.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.514+0000 7fa33949db80 1 -- start start 2024-05-08T23:27:07.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.514+0000 7fa33949db80 1 Processor -- start 2024-05-08T23:27:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa33949db80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x5592720eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa33949db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927207d750 0x5592720a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa33949db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x5592720a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa33949db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927210d1f0 con 0x559272084d80 2024-05-08T23:27:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa33949db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592720a0d70 con 0x55927207d750 2024-05-08T23:27:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa33949db80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592720a0b20 con 0x559272058320 2024-05-08T23:27:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa33949db80 10 client.0 _ll_register_callbacks cb 0x559271f4fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x5592720a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x5592720a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48402/0 (socket says 172.21.15.5:48402) 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 -- 172.21.15.5:0/2192514422 learned_addr learned my addr 172.21.15.5:0/2192514422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa326ffd700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x5592720eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3267fc700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927207d750 0x5592720a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 -- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 msgr2=0x5592720eebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x5592720eebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 -- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927207d750 msgr2=0x5592720a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927207d750 0x5592720a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 -- 172.21.15.5:0/2192514422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559271dd7b10 con 0x559272084d80 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa326ffd700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x5592720eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3267fc700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927207d750 0x5592720a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa3277fe700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x5592720a0560 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa32806b790 tx=0x7fa320004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1229ba749e41a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa305ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa305ffb700 1 -- 172.21.15.5:0/2192514422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320004240 con 0x559272084d80 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa305ffb700 1 -- 172.21.15.5:0/2192514422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa32002b040 con 0x559272084d80 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.515+0000 7fa33949db80 1 -- 172.21.15.5:0/2192514422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa320021070 con 0x559272084d80 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa305ffb700 1 -- 172.21.15.5:0/2192514422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3200043a0 con 0x559272084d80 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa33949db80 10 client.7992 fetch_fsmap learned FSMap version 87 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa33949db80 1 -- 172.21.15.5:0/2192514422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x559271e2fc80 con 0x559272084d80 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa305ffb700 1 -- 172.21.15.5:0/2192514422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320021070 con 0x559272084d80 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa305ffb700 1 -- 172.21.15.5:0/2192514422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 87) v1 ==== 3138+0+0 (secure 0 0 0) 0x7fa32002f070 con 0x559272084d80 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa33949db80 10 client.7992 fetch_fsmap finished waiting for FSMap version 87 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa33949db80 10 client.7992 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa33949db80 20 client.7992 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa33949db80 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55927207aeb0 0x55927207b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa33949db80 4 client.7992 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:27:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa33949db80 1 -- 172.21.15.5:0/2192514422 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559271fa2e70 con 0x55927207aeb0 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa326ffd700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55927207aeb0 0x55927207b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa326ffd700 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55927207aeb0 0x55927207b290 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58374c3a4bff597f server_cookie=a71f84d6f01923c4 in_seq=0 out_seq=0 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.516+0000 7fa305ffb700 10 client.7992 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa305ffb700 1 -- 172.21.15.5:0/2192514422 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x559271fa2e70 con 0x55927207aeb0 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa305ffb700 10 client.7992 handle_command_reply: tid=0 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 client.7992 shutdown 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 -- 172.21.15.5:0/2192514422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 msgr2=0x5592720a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x5592720a0560 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa32806b790 tx=0x7fa320004b40 comp rx=0 tx=0).stop 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 -- 172.21.15.5:0/2192514422 shutdown_connections 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55927207aeb0 0x55927207b290 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa305ffb700 0 client.7992 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559272084d80 0x5592720a0560 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x559272058320 0x5592720eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 --2- 172.21.15.5:0/2192514422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55927207d750 0x5592720a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 -- 172.21.15.5:0/2192514422 >> 172.21.15.5:0/2192514422 conn(0x559271fb81c0 msgr2=0x55927209f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 -- 172.21.15.5:0/2192514422 shutdown_connections 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 1 -- 172.21.15.5:0/2192514422 wait complete. 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa33949db80 20 client.7992 trim_cache size 0 max 16384 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7758, 2024-05-08T23:27:07.524 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:27:07.525 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:27:07.525 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:27:07.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 11.324107701999999, 2024-05-08T23:27:07.525 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 87, 2024-05-08T23:27:07.525 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 84, 2024-05-08T23:27:07.525 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:27:07.525 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 16.575819632000002 2024-05-08T23:27:07.525 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa32d2e3700 1 -- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa30806ac90 msgr2=0x7fa30806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa30806ac90 0x7fa30806d150 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa310000f60 tx=0x7fa310009040 comp rx=0 tx=0).stop 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa32d2e3700 1 -- 172.21.15.5:0/3777236175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280b2f60 msgr2=0x7fa328158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.517+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280b2f60 0x7fa328158a60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fa31800e7f0 tx=0x7fa31800ebb0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.518+0000 7fa32d2e3700 1 -- 172.21.15.5:0/3777236175 shutdown_connections 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.518+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa30806ac90 0x7fa30806d150 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.518+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa328160f00 0x7fa32815d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.518+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3280b2f60 0x7fa328158a60 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.518+0000 7fa32d2e3700 1 --2- 172.21.15.5:0/3777236175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa3280d22d0 0x7fa3280b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.518+0000 7fa32d2e3700 1 -- 172.21.15.5:0/3777236175 >> 172.21.15.5:0/3777236175 conn(0x7fa32808afe0 msgr2=0x7fa32806cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.518+0000 7fa32d2e3700 1 -- 172.21.15.5:0/3777236175 shutdown_connections 2024-05-08T23:27:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.518+0000 7fa32d2e3700 1 -- 172.21.15.5:0/3777236175 wait complete. 2024-05-08T23:27:07.531 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.667+0000 7f1319025700 1 Processor -- start 2024-05-08T23:27:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.667+0000 7f1319025700 1 -- start start 2024-05-08T23:27:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.668+0000 7f1319025700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f13140b03e0 0x7f131409a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.668+0000 7f1319025700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1314099a50 0x7f1314099f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.668+0000 7f1319025700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13140cc6b0 con 0x7f131415ed40 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.668+0000 7f1319025700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131409a440 con 0x7f13140b03e0 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.668+0000 7f1319025700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13140b14b0 con 0x7f1314099a50 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131359e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1314099a50 0x7f1314099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131259c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f13140b03e0 0x7f131409a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131259c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f13140b03e0 0x7f131409a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36718/0 (socket says 172.21.15.5:36718) 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f1312d9d700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f131415ed40 0x7f13140b0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:44340/0 (socket says 172.21.15.5:44340) 2024-05-08T23:27:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131259c700 1 -- 172.21.15.5:0/3508018611 learned_addr learned my addr 172.21.15.5:0/3508018611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131259c700 1 -- 172.21.15.5:0/3508018611 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1314099a50 msgr2=0x7f1314099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131259c700 1 --2- 172.21.15.5:0/3508018611 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1314099a50 0x7f1314099f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131259c700 1 -- 172.21.15.5:0/3508018611 >> v1:172.21.15.49:6789/0 conn(0x7f131415ed40 legacy=0x7f13140b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131259c700 1 -- 172.21.15.5:0/3508018611 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13140d0030 con 0x7f13140b03e0 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f1311d9b700 1 -- 172.21.15.5:0/3508018611 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531583388 0 0) 0x7f13140cc6b0 con 0x7f131415ed40 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f131259c700 1 --2- 172.21.15.5:0/3508018611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13140b03e0 0x7f131409a8b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f13080098f0 tx=0x7f1308009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e228393e74ef0009 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f1311d9b700 1 -- 172.21.15.5:0/3508018611 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308011830 con 0x7f13140b03e0 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f1311d9b700 1 -- 172.21.15.5:0/3508018611 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1308011990 con 0x7f13140b03e0 2024-05-08T23:27:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.669+0000 7f1311d9b700 1 -- 172.21.15.5:0/3508018611 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308011d20 con 0x7f13140b03e0 2024-05-08T23:27:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.670+0000 7f1319025700 1 -- 172.21.15.5:0/3508018611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13140b03e0 msgr2=0x7f131409a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.670+0000 7f1319025700 1 --2- 172.21.15.5:0/3508018611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13140b03e0 0x7f131409a8b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f13080098f0 tx=0x7f1308009c00 comp rx=0 tx=0).stop 2024-05-08T23:27:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.670+0000 7f1319025700 1 -- 172.21.15.5:0/3508018611 shutdown_connections 2024-05-08T23:27:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.670+0000 7f1319025700 1 --2- 172.21.15.5:0/3508018611 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1314099a50 0x7f1314099f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.670+0000 7f1319025700 1 --2- 172.21.15.5:0/3508018611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13140b03e0 0x7f131409a8b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.670+0000 7f1319025700 1 -- 172.21.15.5:0/3508018611 >> 172.21.15.5:0/3508018611 conn(0x7f13140b2f30 msgr2=0x7f13140b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.670+0000 7f1319025700 1 -- 172.21.15.5:0/3508018611 shutdown_connections 2024-05-08T23:27:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.670+0000 7f1319025700 1 -- 172.21.15.5:0/3508018611 wait complete. 2024-05-08T23:27:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.672+0000 7f1319025700 1 Processor -- start 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.672+0000 7f1319025700 1 -- start start 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.672+0000 7f1319025700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1314099a50 0x7f13140c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1319025700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13140b03e0 0x7f1314157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1319025700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131415ed40 0x7f1314157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1319025700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13140c9a50 con 0x7f1314099a50 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1319025700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13140c91b0 con 0x7f131415ed40 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1319025700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13140c9490 con 0x7f13140b03e0 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1314099a50 0x7f13140c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1314099a50 0x7f13140c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48418/0 (socket says 172.21.15.5:48418) 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 -- 172.21.15.5:0/3214242075 learned_addr learned my addr 172.21.15.5:0/3214242075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f131259c700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13140b03e0 0x7f1314157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f131359e700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131415ed40 0x7f1314157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 -- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13140b03e0 msgr2=0x7f1314157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13140b03e0 0x7f1314157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 -- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131415ed40 msgr2=0x7f1314157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131415ed40 0x7f1314157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 -- 172.21.15.5:0/3214242075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308009550 con 0x7f1314099a50 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f1312d9d700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1314099a50 0x7f13140c9e90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f13040011e0 tx=0x7f130400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f12fb7fe700 1 -- 172.21.15.5:0/3214242075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304004c80 con 0x7f1314099a50 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.673+0000 7f12fb7fe700 1 -- 172.21.15.5:0/3214242075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1304004e00 con 0x7f1314099a50 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.674+0000 7f12fb7fe700 1 -- 172.21.15.5:0/3214242075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304007310 con 0x7f1314099a50 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.674+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13140c9770 con 0x7f1314099a50 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.674+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13140ca3d0 con 0x7f1314099a50 2024-05-08T23:27:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.675+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13140ca5c0 con 0x7f1314099a50 2024-05-08T23:27:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.675+0000 7f12fb7fe700 1 -- 172.21.15.5:0/3214242075 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1304007470 con 0x7f1314099a50 2024-05-08T23:27:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.675+0000 7f12fb7fe700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f130006ac90 0x7f130006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.676+0000 7f12fb7fe700 1 -- 172.21.15.5:0/3214242075 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13040890c0 con 0x7f1314099a50 2024-05-08T23:27:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.676+0000 7f131259c700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f130006ac90 0x7f130006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.676+0000 7f131259c700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f130006ac90 0x7f130006d150 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1308014040 tx=0x7f1308008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.680+0000 7f12fb7fe700 1 -- 172.21.15.5:0/3214242075 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1304054290 con 0x7f1314099a50 2024-05-08T23:27:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.948+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1314068000 con 0x7f1314099a50 2024-05-08T23:27:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.948+0000 7f12fb7fe700 1 -- 172.21.15.5:0/3214242075 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4758 (secure 0 0 0) 0x7f13040578b0 con 0x7f1314099a50 2024-05-08T23:27:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:07.948 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:06.233541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.950+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f130006ac90 msgr2=0x7f130006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.950+0000 7f1319025700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f130006ac90 0x7f130006d150 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1308014040 tx=0x7f1308008030 comp rx=0 tx=0).stop 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.950+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1314099a50 msgr2=0x7f13140c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.950+0000 7f1319025700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1314099a50 0x7f13140c9e90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f13040011e0 tx=0x7f130400cae0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.950+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 shutdown_connections 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.950+0000 7f1319025700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f130006ac90 0x7f130006d150 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.950+0000 7f1319025700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131415ed40 0x7f1314157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.950+0000 7f1319025700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1314099a50 0x7f13140c9e90 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.951+0000 7f1319025700 1 --2- 172.21.15.5:0/3214242075 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f13140b03e0 0x7f1314157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.951+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 >> 172.21.15.5:0/3214242075 conn(0x7f13140b2f30 msgr2=0x7f13140cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.951+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 shutdown_connections 2024-05-08T23:27:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:07.951+0000 7f1319025700 1 -- 172.21.15.5:0/3214242075 wait complete. 2024-05-08T23:27:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 87 2024-05-08T23:27:07.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:07.965 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:27:07.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:07.994 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:27:08.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:08.060 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-05-08T23:27:08.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:08.124 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-05-08T23:27:08.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:08.188 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-05-08T23:27:08.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:27:08.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:27:08.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:27:08.489 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:27:08.490 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:27:08.490 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:27:10.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:27:10.655 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:27:10.655 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:27:10.655 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:27:12.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:27:12.821 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:27:12.822 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:27:12.822 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:27:14.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:27:14.988 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:27:14.988 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:27:14.989 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:27:16.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:27:17.154 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-05-08T23:27:17.155 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:27:17.155 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:27:17.155 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:27:17.312 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:27:17.313 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:27:17.313 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:27:19.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:27:19.478 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:27:19.479 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:27:19.479 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:27:21.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:27:21.646 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-08T23:27:21.647 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:27:21.647 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:27:21.647 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-08T23:27:21.804 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:27:21.805 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:27:21.805 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:27:21.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-05-08T23:27:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.954+0000 7efda6b59700 1 Processor -- start 2024-05-08T23:27:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.954+0000 7efda6b59700 1 -- start start 2024-05-08T23:27:21.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efda6b59700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efda015b4e0 0x7efda00b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efda6b59700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda00cf560 con 0x7efda015b8c0 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efda6b59700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda00ccff0 con 0x7efda00d1910 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efda6b59700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda00cefe0 con 0x7efda015b4e0 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efd9ffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efda015b8c0 0x7efda00d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33984/0 (socket says 172.21.15.5:33984) 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efd9ffff700 1 -- 172.21.15.5:0/951549692 learned_addr learned my addr 172.21.15.5:0/951549692 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efda48f5700 1 --2- 172.21.15.5:0/951549692 >> v2:172.21.15.49:3300/0 conn(0x7efda015b4e0 0x7efda00b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479986597 0 0) 0x7efda00cf560 con 0x7efda015b8c0 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd900035b0 con 0x7efda015b8c0 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.955+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750596953 0 0) 0x7efd900035b0 con 0x7efda015b8c0 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 >> v2:172.21.15.49:3300/0 conn(0x7efda015b4e0 msgr2=0x7efda00b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 --2- 172.21.15.5:0/951549692 >> v2:172.21.15.49:3300/0 conn(0x7efda015b4e0 0x7efda00b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 >> v1:172.21.15.49:6789/0 conn(0x7efda00d1910 legacy=0x7efda00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:21.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda00cd2d0 con 0x7efda015b8c0 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7efd8c001670 con 0x7efda015b8c0 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efda48f5700 1 --2- 172.21.15.5:0/951549692 >> v2:172.21.15.49:3300/0 conn(0x7efda015b4e0 0x7efda00b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 >> v1:172.21.15.5:6789/0 conn(0x7efda015b8c0 legacy=0x7efda00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90003a50 0x7efd90003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd90004850 0x7efd90004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efd90005210 0x7efd9000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda00cefe0 con 0x7efd90004850 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9000c110 con 0x7efd90003a50 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9000c2d0 con 0x7efd90005210 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9ffff700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd90004850 0x7efd90004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efda48f5700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90003a50 0x7efd90003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efda50f6700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efd90005210 0x7efd9000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9ffff700 1 -- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efd90005210 msgr2=0x7efd9000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9ffff700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efd90005210 0x7efd9000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9ffff700 1 -- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90003a50 msgr2=0x7efd90003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9ffff700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90003a50 0x7efd90003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9ffff700 1 -- 172.21.15.5:0/951549692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900035b0 con 0x7efd90004850 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efda48f5700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90003a50 0x7efd90003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efda50f6700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efd90005210 0x7efd9000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efda50f6700 1 -- 172.21.15.5:0/951549692 reap_dead start 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9ffff700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd90004850 0x7efd90004cd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c190 tx=0x7efd90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afa96f307af5f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c007b30 con 0x7efd90004850 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c00cd10 con 0x7efd90004850 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.956+0000 7efd9f7fe700 1 -- 172.21.15.5:0/951549692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c005ba0 con 0x7efd90004850 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.957+0000 7efda6b59700 1 -- 172.21.15.5:0/951549692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd90004850 msgr2=0x7efd90004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.957+0000 7efda6b59700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd90004850 0x7efd90004cd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c190 tx=0x7efd90005880 comp rx=0 tx=0).stop 2024-05-08T23:27:21.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.957+0000 7efda6b59700 1 -- 172.21.15.5:0/951549692 shutdown_connections 2024-05-08T23:27:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.957+0000 7efda6b59700 1 --2- 172.21.15.5:0/951549692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd90004850 0x7efd90004cd0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.957+0000 7efda6b59700 1 -- 172.21.15.5:0/951549692 >> 172.21.15.5:0/951549692 conn(0x7efda008afe0 msgr2=0x7efda008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.957+0000 7efda6b59700 1 -- 172.21.15.5:0/951549692 shutdown_connections 2024-05-08T23:27:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.957+0000 7efda6b59700 1 -- 172.21.15.5:0/951549692 wait complete. 2024-05-08T23:27:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.958+0000 7efda6b59700 1 Processor -- start 2024-05-08T23:27:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.958+0000 7efda6b59700 1 -- start start 2024-05-08T23:27:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.959+0000 7efda6b59700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90004850 0x7efda014f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.959+0000 7efda6b59700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda015bd80 0x7efda015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.959+0000 7efda6b59700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efda015e010 0x7efda015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.959+0000 7efda6b59700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda00b0b50 con 0x7efda015bd80 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.959+0000 7efda6b59700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda00c5ff0 con 0x7efd90004850 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.959+0000 7efda6b59700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda015cbd0 con 0x7efda015e010 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda015bd80 0x7efda015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda015bd80 0x7efda015c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59060/0 (socket says 172.21.15.5:59060) 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 -- 172.21.15.5:0/2595232219 learned_addr learned my addr 172.21.15.5:0/2595232219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efda48f5700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90004850 0x7efda014f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efda50f6700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efda015e010 0x7efda015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 -- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efda015e010 msgr2=0x7efda015e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efda015e010 0x7efda015e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 -- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90004850 msgr2=0x7efda014f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90004850 0x7efda014f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 -- 172.21.15.5:0/2595232219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda015c8f0 con 0x7efda015bd80 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9ffff700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda015bd80 0x7efda015c200 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000c00 tx=0x7efda00cf4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9d7fa700 1 -- 172.21.15.5:0/2595232219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c004c80 con 0x7efda015bd80 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9d7fa700 1 -- 172.21.15.5:0/2595232219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c004de0 con 0x7efda015bd80 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efd9d7fa700 1 -- 172.21.15.5:0/2595232219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c007310 con 0x7efda015bd80 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efda6b59700 1 -- 172.21.15.5:0/2595232219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda00c59a0 con 0x7efda015bd80 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.960+0000 7efda6b59700 1 -- 172.21.15.5:0/2595232219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda014fad0 con 0x7efda015bd80 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.962+0000 7efd9d7fa700 1 -- 172.21.15.5:0/2595232219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efd8c007470 con 0x7efda015bd80 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.963+0000 7efd9d7fa700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efd8406ac90 0x7efd8406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.963+0000 7efd9d7fa700 1 -- 172.21.15.5:0/2595232219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd8c08a470 con 0x7efda015bd80 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.963+0000 7efda48f5700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efd8406ac90 0x7efd8406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.963+0000 7efda48f5700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efd8406ac90 0x7efd8406d150 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efd94014400 tx=0x7efd94009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.965+0000 7efdb2d13b80 1 Processor -- start 2024-05-08T23:27:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.965+0000 7efdb2d13b80 1 -- start start 2024-05-08T23:27:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d777a0 0x5575e5d50110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5d550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d7edd0 0x5575e5d7f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575e5cab150 con 0x5575e5d52370 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575e5ca8b10 con 0x5575e5d7edd0 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575e5ca4400 con 0x5575e5d777a0 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5d550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5d550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59068/0 (socket says 172.21.15.5:59068) 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efda50f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d7edd0 0x5575e5d7f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 -- 172.21.15.5:0/976182118 learned_addr learned my addr 172.21.15.5:0/976182118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efda48f5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d777a0 0x5575e5d50110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 -- 172.21.15.5:0/976182118 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d777a0 msgr2=0x5575e5d50110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 --2- 172.21.15.5:0/976182118 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d777a0 0x5575e5d50110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 -- 172.21.15.5:0/976182118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d7edd0 msgr2=0x5575e5d7f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 --2- 172.21.15.5:0/976182118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d7edd0 0x5575e5d7f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 -- 172.21.15.5:0/976182118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575e5b253b0 con 0x5575e5d52370 2024-05-08T23:27:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efda50f6700 1 --2- 172.21.15.5:0/976182118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d7edd0 0x5575e5d7f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd9ffff700 1 --2- 172.21.15.5:0/976182118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5d550a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0592c0 tx=0x7efd8c058c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44274389cb24e84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd8a7fc700 1 -- 172.21.15.5:0/976182118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c05d0b0 con 0x5575e5d52370 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd8a7fc700 1 -- 172.21.15.5:0/976182118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c061670 con 0x5575e5d52370 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.966+0000 7efd8a7fc700 1 -- 172.21.15.5:0/976182118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0576d0 con 0x5575e5d52370 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 -- 172.21.15.5:0/976182118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 msgr2=0x5575e5d550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/976182118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5d550a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0592c0 tx=0x7efd8c058c90 comp rx=0 tx=0).stop 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 -- 172.21.15.5:0/976182118 shutdown_connections 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/976182118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d7edd0 0x5575e5d7f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/976182118 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d777a0 0x5575e5d50110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/976182118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5d550a0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 -- 172.21.15.5:0/976182118 >> 172.21.15.5:0/976182118 conn(0x5575e5cb2210 msgr2=0x5575e5d99dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 -- 172.21.15.5:0/976182118 shutdown_connections 2024-05-08T23:27:21.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.967+0000 7efdb2d13b80 1 -- 172.21.15.5:0/976182118 wait complete. 2024-05-08T23:27:21.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.968+0000 7efdb2d13b80 1 -- start start 2024-05-08T23:27:21.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.968+0000 7efdb2d13b80 1 Processor -- start 2024-05-08T23:27:21.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d52370 0x5575e5d766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d777a0 0x5575e5d9dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5d98aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575e5c40560 con 0x5575e5d777a0 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575e5c40200 con 0x5575e5d52370 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575e5b276a0 con 0x5575e5d7edd0 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efdb2d13b80 10 client.0 _ll_register_callbacks cb 0x5575e5c49b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d777a0 0x5575e5d9dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d777a0 0x5575e5d9dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59074/0 (socket says 172.21.15.5:59074) 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efd9ffff700 1 -- 172.21.15.5:0/4229578056 learned_addr learned my addr 172.21.15.5:0/4229578056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efda48f5700 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d52370 0x5575e5d766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efda50f6700 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5d98aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efd9ffff700 1 -- 172.21.15.5:0/4229578056 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 msgr2=0x5575e5d98aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efd9ffff700 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5d98aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efd9ffff700 1 -- 172.21.15.5:0/4229578056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d52370 msgr2=0x5575e5d766f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efd9ffff700 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d52370 0x5575e5d766f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.969+0000 7efd9ffff700 1 -- 172.21.15.5:0/4229578056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9800f040 con 0x5575e5d777a0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efd9ffff700 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d777a0 0x5575e5d9dd50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7efd8c05dfd0 tx=0x7efd8c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d73a796b2377fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efd7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efd7f7fe700 1 -- 172.21.15.5:0/4229578056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c071ea0 con 0x5575e5d777a0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efd7f7fe700 1 -- 172.21.15.5:0/4229578056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd8c05d0b0 con 0x5575e5d777a0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 1 -- 172.21.15.5:0/4229578056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575e5bfd270 con 0x5575e5d777a0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efd7f7fe700 1 -- 172.21.15.5:0/4229578056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd8c0576d0 con 0x5575e5d777a0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 10 client.8007 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 1 -- 172.21.15.5:0/4229578056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x7efda014fad0 con 0x5575e5d777a0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efd7f7fe700 1 -- 172.21.15.5:0/4229578056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5575e5bfd270 con 0x5575e5d777a0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efd7f7fe700 1 -- 172.21.15.5:0/4229578056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7efd8c08a070 con 0x5575e5d777a0 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 10 client.8007 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 10 client.8007 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 20 client.8007 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5575e5d9bc60 0x5575e5da8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 4 client.8007 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.970+0000 7efdb2d13b80 1 -- 172.21.15.5:0/4229578056 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575e5d237e0 con 0x5575e5d9bc60 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.971+0000 7efda48f5700 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5575e5d9bc60 0x5575e5da8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.971+0000 7efda48f5700 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5575e5d9bc60 0x5575e5da8da0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed82a82603d3d2bf server_cookie=6dcf3602c68a9d1a in_seq=0 out_seq=0 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.971+0000 7efd7f7fe700 10 client.8007 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.971+0000 7efd7f7fe700 1 -- 172.21.15.5:0/4229578056 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575e5d237e0 con 0x5575e5d9bc60 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.971+0000 7efd7f7fe700 10 client.8007 handle_command_reply: tid=0 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.971+0000 7efdb2d13b80 1 client.8007 shutdown 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 -- 172.21.15.5:0/4229578056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d777a0 msgr2=0x5575e5d9dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d777a0 0x5575e5d9dd50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7efd8c05dfd0 tx=0x7efd8c01d000 comp rx=0 tx=0).stop 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 -- 172.21.15.5:0/4229578056 shutdown_connections 2024-05-08T23:27:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5575e5d9bc60 0x5575e5da8da0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5d98aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d52370 0x5575e5d766f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/4229578056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d777a0 0x5575e5d9dd50 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efd7f7fe700 0 client.8007 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 -- 172.21.15.5:0/4229578056 >> 172.21.15.5:0/4229578056 conn(0x5575e5cb2210 msgr2=0x5575e5d7bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 -- 172.21.15.5:0/4229578056 shutdown_connections 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 1 -- 172.21.15.5:0/4229578056 wait complete. 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.972+0000 7efdb2d13b80 20 client.8007 trim_cache size 0 max 16384 2024-05-08T23:27:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.988+0000 7efdb2d13b80 1 Processor -- start 2024-05-08T23:27:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.988+0000 7efdb2d13b80 1 -- start start 2024-05-08T23:27:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.988+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dbb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d4ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5d7b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575e5d9b740 con 0x5575e5d52370 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575e5d7e4d0 con 0x5575e5d777a0 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575e5d50470 con 0x5575e5d7edd0 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dbb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dbb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59076/0 (socket says 172.21.15.5:59076) 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 -- 172.21.15.5:0/2840572409 learned_addr learned my addr 172.21.15.5:0/2840572409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efd9ffff700 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d4ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda50f6700 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5d7b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 -- 172.21.15.5:0/2840572409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 msgr2=0x5575e5d7b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5d7b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 -- 172.21.15.5:0/2840572409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 msgr2=0x5575e5d4ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d4ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 -- 172.21.15.5:0/2840572409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575e5ad1b60 con 0x5575e5d52370 2024-05-08T23:27:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efda48f5700 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dbb6e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7efd940043a0 tx=0x7efd94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b12d1c254f458 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efd8affd700 1 -- 172.21.15.5:0/2840572409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94028040 con 0x5575e5d52370 2024-05-08T23:27:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efd8affd700 1 -- 172.21.15.5:0/2840572409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94019070 con 0x5575e5d52370 2024-05-08T23:27:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.989+0000 7efd8affd700 1 -- 172.21.15.5:0/2840572409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94003c30 con 0x5575e5d52370 2024-05-08T23:27:21.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 -- 172.21.15.5:0/2840572409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 msgr2=0x5575e5dbb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:21.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dbb6e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7efd940043a0 tx=0x7efd94017000 comp rx=0 tx=0).stop 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 -- 172.21.15.5:0/2840572409 shutdown_connections 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5d7b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dbb6e0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/2840572409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d4ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 -- 172.21.15.5:0/2840572409 >> 172.21.15.5:0/2840572409 conn(0x5575e5cb2210 msgr2=0x5575e5d7ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 -- 172.21.15.5:0/2840572409 shutdown_connections 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.990+0000 7efdb2d13b80 1 -- 172.21.15.5:0/2840572409 wait complete. 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.991+0000 7efdb2d13b80 1 -- start start 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.991+0000 7efdb2d13b80 1 Processor -- start 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dfc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d76500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efdb2d13b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5e02f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575e5d76ed0 con 0x5575e5d52370 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575e5d97bc0 con 0x5575e5d777a0 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efdb2d13b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575e5d76a40 con 0x5575e5d7edd0 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efdb2d13b80 10 client.0 _ll_register_callbacks cb 0x5575e5c49b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dfc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dfc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59086/0 (socket says 172.21.15.5:59086) 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 -- 172.21.15.5:0/1610661165 learned_addr learned my addr 172.21.15.5:0/1610661165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efd9ffff700 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d76500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda50f6700 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5e02f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 -- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 msgr2=0x5575e5e02f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5e02f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 -- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 msgr2=0x5575e5d76500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d76500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 -- 172.21.15.5:0/1610661165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd94004040 con 0x5575e5d52370 2024-05-08T23:27:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efd9ffff700 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d76500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efda48f5700 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dfc730 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7efd940098d0 tx=0x7efd94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c82aa67dafa4206 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.992+0000 7efd7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efd7f7fe700 1 -- 172.21.15.5:0/1610661165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94028070 con 0x5575e5d52370 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efd7f7fe700 1 -- 172.21.15.5:0/1610661165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd9402d3a0 con 0x5575e5d52370 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 1 -- 172.21.15.5:0/1610661165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd8c061670 con 0x5575e5d52370 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efd7f7fe700 1 -- 172.21.15.5:0/1610661165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd9402d560 con 0x5575e5d52370 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 10 client.8013 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 1 -- 172.21.15.5:0/1610661165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5575e5c6a560 con 0x5575e5d52370 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efd7f7fe700 1 -- 172.21.15.5:0/1610661165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c061670 con 0x5575e5d52370 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efd7f7fe700 1 -- 172.21.15.5:0/1610661165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7efd9403c020 con 0x5575e5d52370 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 10 client.8013 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 10 client.8013 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 20 client.8013 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5575e5e03490 0x5575e5d82330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 4 client.8013 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:27:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efdb2d13b80 1 -- 172.21.15.5:0/1610661165 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5575e5c9cec0 con 0x5575e5e03490 2024-05-08T23:27:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.993+0000 7efd9ffff700 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5575e5e03490 0x5575e5d82330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.994+0000 7efd7f7fe700 10 client.8013 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:21.994+0000 7efd9ffff700 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5575e5e03490 0x5575e5d82330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b57802cce15489 server_cookie=5d7c21460aea185f in_seq=0 out_seq=0 2024-05-08T23:27:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.006+0000 7efd7f7fe700 1 -- 172.21.15.5:0/1610661165 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5575e5c9cec0 con 0x5575e5e03490 2024-05-08T23:27:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.006+0000 7efd7f7fe700 10 client.8013 handle_command_reply: tid=0 2024-05-08T23:27:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.006+0000 7efdb2d13b80 1 client.8013 shutdown 2024-05-08T23:27:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 -- 172.21.15.5:0/1610661165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 msgr2=0x5575e5dfc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dfc730 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7efd940098d0 tx=0x7efd94004750 comp rx=0 tx=0).stop 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 -- 172.21.15.5:0/1610661165 shutdown_connections 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5575e5e03490 0x5575e5d82330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5575e5d7edd0 0x5575e5e02f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575e5d52370 0x5575e5dfc730 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 --2- 172.21.15.5:0/1610661165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5575e5d777a0 0x5575e5d76500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efd7f7fe700 0 client.8013 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 -- 172.21.15.5:0/1610661165 >> 172.21.15.5:0/1610661165 conn(0x5575e5cb2210 msgr2=0x5575e5d7f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 -- 172.21.15.5:0/1610661165 shutdown_connections 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 1 -- 172.21.15.5:0/1610661165 wait complete. 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efdb2d13b80 20 client.8013 trim_cache size 0 max 16384 2024-05-08T23:27:22.007 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:22.008 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:27:22.008 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:27:22.008 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efda6b59700 1 -- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efd8406ac90 msgr2=0x7efd8406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efda6b59700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efd8406ac90 0x7efd8406d150 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efd94014400 tx=0x7efd94009210 comp rx=0 tx=0).stop 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efda6b59700 1 -- 172.21.15.5:0/2595232219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda015bd80 msgr2=0x7efda015c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.007+0000 7efda6b59700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda015bd80 0x7efda015c200 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000c00 tx=0x7efda00cf4d0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.008+0000 7efda6b59700 1 -- 172.21.15.5:0/2595232219 shutdown_connections 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.008+0000 7efda6b59700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7efd8406ac90 0x7efd8406d150 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.008+0000 7efda6b59700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7efda015e010 0x7efda015e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.008+0000 7efda6b59700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd90004850 0x7efda014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.008+0000 7efda6b59700 1 --2- 172.21.15.5:0/2595232219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda015bd80 0x7efda015c200 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.008+0000 7efda6b59700 1 -- 172.21.15.5:0/2595232219 >> 172.21.15.5:0/2595232219 conn(0x7efda008afe0 msgr2=0x7efda015f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.008+0000 7efda6b59700 1 -- 172.21.15.5:0/2595232219 shutdown_connections 2024-05-08T23:27:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.008+0000 7efda6b59700 1 -- 172.21.15.5:0/2595232219 wait complete. 2024-05-08T23:27:22.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 flush journal 2024-05-08T23:27:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.160+0000 7f7b6972d700 1 Processor -- start 2024-05-08T23:27:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.160+0000 7f7b6972d700 1 -- start start 2024-05-08T23:27:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b6972d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b640ca130 0x7f7b640cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b6972d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b640cd7c0 con 0x7f7b6415a7c0 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b6972d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b640cd1b0 con 0x7f7b640727b0 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b6972d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6415aba0 con 0x7f7b640ca130 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b62ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7b6415a7c0 0x7f7b640c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33996/0 (socket says 172.21.15.5:33996) 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2039236083 learned_addr learned my addr 172.21.15.5:0/2039236083 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2039236083 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b640ca130 0x7f7b640cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b61ffb700 1 -- 172.21.15.5:0/2039236083 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671479604 0 0) 0x7f7b640cd7c0 con 0x7f7b6415a7c0 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b61ffb700 1 -- 172.21.15.5:0/2039236083 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c0057c0 con 0x7f7b6415a7c0 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b61ffb700 1 -- 172.21.15.5:0/2039236083 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540700376 0 0) 0x7f7b640cd1b0 con 0x7f7b640727b0 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b61ffb700 1 -- 172.21.15.5:0/2039236083 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b640cd7c0 con 0x7f7b640727b0 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b627fc700 1 -- 172.21.15.5:0/2039236083 >> v1:172.21.15.49:6789/0 conn(0x7f7b640727b0 legacy=0x7f7b640b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b627fc700 1 -- 172.21.15.5:0/2039236083 >> v1:172.21.15.5:6789/0 conn(0x7f7b6415a7c0 legacy=0x7f7b640c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b627fc700 1 -- 172.21.15.5:0/2039236083 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b640b3e70 con 0x7f7b640ca130 2024-05-08T23:27:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.161+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2039236083 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b640ca130 0x7f7b640cdc50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009870 tx=0x7f7b58009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f336befad191712b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b61ffb700 1 -- 172.21.15.5:0/2039236083 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011830 con 0x7f7b640ca130 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b61ffb700 1 -- 172.21.15.5:0/2039236083 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b58011990 con 0x7f7b640ca130 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b61ffb700 1 -- 172.21.15.5:0/2039236083 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011d20 con 0x7f7b640ca130 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b6972d700 1 -- 172.21.15.5:0/2039236083 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b640ca130 msgr2=0x7f7b640cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b6972d700 1 --2- 172.21.15.5:0/2039236083 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b640ca130 0x7f7b640cdc50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009870 tx=0x7f7b58009b80 comp rx=0 tx=0).stop 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b6972d700 1 -- 172.21.15.5:0/2039236083 shutdown_connections 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b6972d700 1 --2- 172.21.15.5:0/2039236083 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b640ca130 0x7f7b640cdc50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b6972d700 1 -- 172.21.15.5:0/2039236083 >> 172.21.15.5:0/2039236083 conn(0x7f7b6406a960 msgr2=0x7f7b6406ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b6972d700 1 -- 172.21.15.5:0/2039236083 shutdown_connections 2024-05-08T23:27:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.162+0000 7f7b6972d700 1 -- 172.21.15.5:0/2039236083 wait complete. 2024-05-08T23:27:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.164+0000 7f7b6972d700 1 Processor -- start 2024-05-08T23:27:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.164+0000 7f7b6972d700 1 -- start start 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b6972d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b640727b0 0x7f7b640b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b6972d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b640ca130 0x7f7b641f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b6972d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b6415a7c0 0x7f7b6414bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b6972d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641490f0 con 0x7f7b640ca130 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b6972d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b640af1a0 con 0x7f7b640727b0 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b6972d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641f2ee0 con 0x7f7b6415a7c0 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b640ca130 0x7f7b641f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b640ca130 0x7f7b641f29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59096/0 (socket says 172.21.15.5:59096) 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 -- 172.21.15.5:0/2181289082 learned_addr learned my addr 172.21.15.5:0/2181289082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b640727b0 0x7f7b640b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b637fe700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b6415a7c0 0x7f7b6414bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 -- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b6415a7c0 msgr2=0x7f7b6414bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b6415a7c0 0x7f7b6414bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 -- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b640727b0 msgr2=0x7f7b640b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b640727b0 0x7f7b640b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 -- 172.21.15.5:0/2181289082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580094a0 con 0x7f7b640ca130 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b637fe700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b6415a7c0 0x7f7b6414bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b640ca130 0x7f7b641f29a0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7b580090b0 tx=0x7f7b58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.165+0000 7f7b4b7fe700 1 -- 172.21.15.5:0/2181289082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5801a760 con 0x7f7b640ca130 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.166+0000 7f7b4b7fe700 1 -- 172.21.15.5:0/2181289082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5801a8c0 con 0x7f7b640ca130 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.166+0000 7f7b6972d700 1 -- 172.21.15.5:0/2181289082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6414c0d0 con 0x7f7b640ca130 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.166+0000 7f7b4b7fe700 1 -- 172.21.15.5:0/2181289082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5801ac30 con 0x7f7b640ca130 2024-05-08T23:27:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.166+0000 7f7b6972d700 1 -- 172.21.15.5:0/2181289082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b640b15d0 con 0x7f7b640ca130 2024-05-08T23:27:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.167+0000 7f7b4b7fe700 1 -- 172.21.15.5:0/2181289082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7b58022070 con 0x7f7b640ca130 2024-05-08T23:27:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.167+0000 7f7b4b7fe700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b5006ac90 0x7f7b5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.168+0000 7f7b4b7fe700 1 -- 172.21.15.5:0/2181289082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b5808a440 con 0x7f7b640ca130 2024-05-08T23:27:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.168+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b5006ac90 0x7f7b5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.168+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b5006ac90 0x7f7b5006d150 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7b54007fd0 tx=0x7f7b54007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.170+0000 7f7b758e7b80 1 Processor -- start 2024-05-08T23:27:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.170+0000 7f7b758e7b80 1 -- start start 2024-05-08T23:27:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba747a8dd0 0x55ba747a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba7477b720 0x55ba7477e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a6570 0x55ba7477eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba746d5150 con 0x55ba747a8dd0 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba746d2b10 con 0x55ba747a6570 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba746ce400 con 0x55ba7477b720 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba747a8dd0 0x55ba747a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba747a8dd0 0x55ba747a91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59098/0 (socket says 172.21.15.5:59098) 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 -- 172.21.15.5:0/3401423168 learned_addr learned my addr 172.21.15.5:0/3401423168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b637fe700 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a6570 0x55ba7477eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b627fc700 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba7477b720 0x55ba7477e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 -- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba7477b720 msgr2=0x55ba7477e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba7477b720 0x55ba7477e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 -- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a6570 msgr2=0x55ba7477eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a6570 0x55ba7477eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 -- 172.21.15.5:0/3401423168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba7454f3b0 con 0x55ba747a8dd0 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b637fe700 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a6570 0x55ba7477eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.171+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba747a8dd0 0x55ba747a91b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7b54014610 tx=0x7f7b54012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29f733bda1219d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b48ff9700 1 -- 172.21.15.5:0/3401423168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54014d90 con 0x55ba747a8dd0 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b48ff9700 1 -- 172.21.15.5:0/3401423168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b54020410 con 0x55ba747a8dd0 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b48ff9700 1 -- 172.21.15.5:0/3401423168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b540207a0 con 0x55ba747a8dd0 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 -- 172.21.15.5:0/3401423168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba747a8dd0 msgr2=0x55ba747a91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba747a8dd0 0x55ba747a91b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7b54014610 tx=0x7f7b54012000 comp rx=0 tx=0).stop 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 -- 172.21.15.5:0/3401423168 shutdown_connections 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a6570 0x55ba7477eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba747a8dd0 0x55ba747a91b0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/3401423168 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba7477b720 0x55ba7477e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 -- 172.21.15.5:0/3401423168 >> 172.21.15.5:0/3401423168 conn(0x55ba746dc210 msgr2=0x55ba747c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 -- 172.21.15.5:0/3401423168 shutdown_connections 2024-05-08T23:27:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.172+0000 7f7b758e7b80 1 -- 172.21.15.5:0/3401423168 wait complete. 2024-05-08T23:27:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.173+0000 7f7b758e7b80 1 -- start start 2024-05-08T23:27:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.173+0000 7f7b758e7b80 1 Processor -- start 2024-05-08T23:27:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747a2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7466a560 con 0x55ba7477b720 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7466a200 con 0x55ba747a8dd0 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba745516a0 con 0x55ba747a6570 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b758e7b80 10 client.0 _ll_register_callbacks cb 0x55ba74673b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747a2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59110/0 (socket says 172.21.15.5:59110) 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2063373062 learned_addr learned my addr 172.21.15.5:0/2063373062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b637fe700 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2063373062 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 msgr2=0x55ba747ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2063373062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 msgr2=0x55ba747c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.174+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2063373062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c0057c0 con 0x55ba7477b720 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747a2710 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7b540011e0 tx=0x7f7b54000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d744b56665d19d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2063373062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54006c70 con 0x55ba7477b720 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2063373062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b5401c040 con 0x55ba7477b720 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2063373062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba74627270 con 0x55ba7477b720 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2063373062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b54020d00 con 0x55ba7477b720 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 10 client.8025 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2063373062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55ba744fbb60 con 0x55ba7477b720 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2063373062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba74627270 con 0x55ba7477b720 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2063373062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f7b54012070 con 0x55ba7477b720 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 10 client.8025 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 10 client.8025 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-05-08T23:27:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 20 client.8025 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55ba747c6480 0x55ba747df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 4 client.8025 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.175+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2063373062 --> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba7474d7e0 con 0x55ba747c6480 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.176+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55ba747c6480 0x55ba747df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.176+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55ba747c6480 0x55ba747df3b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d21570acb2de3f58 server_cookie=e5e06c944a72dcb6 in_seq=0 out_seq=0 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.176+0000 7f7b41ffb700 10 client.8025 ms_handle_connect on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.176+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2063373062 <== mds.1 v2:172.21.15.5:6832/3718289875 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba7474d7e0 con 0x55ba747c6480 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.176+0000 7f7b41ffb700 10 client.8025 handle_command_reply: tid=0 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.176+0000 7f7b758e7b80 1 client.8025 shutdown 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2063373062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 msgr2=0x55ba747a2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747a2710 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7b540011e0 tx=0x7f7b54000ed0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2063373062 shutdown_connections 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55ba747c6480 0x55ba747df3b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747a2710 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2063373062 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b41ffb700 0 client.8025 ms_handle_reset on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2063373062 >> 172.21.15.5:0/2063373062 conn(0x55ba746dc210 msgr2=0x55ba74778e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2063373062 shutdown_connections 2024-05-08T23:27:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2063373062 wait complete. 2024-05-08T23:27:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.177+0000 7f7b758e7b80 20 client.8025 trim_cache size 0 max 16384 2024-05-08T23:27:22.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.193+0000 7f7b758e7b80 1 Processor -- start 2024-05-08T23:27:22.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.193+0000 7f7b758e7b80 1 -- start start 2024-05-08T23:27:22.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba747c5f30 con 0x55ba7477b720 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba747b8d50 con 0x55ba747a8dd0 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba747c8210 con 0x55ba747a6570 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747ba000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59114/0 (socket says 172.21.15.5:59114) 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2215827359 learned_addr learned my addr 172.21.15.5:0/2215827359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b637fe700 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 msgr2=0x55ba747a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 msgr2=0x55ba747c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 -- 172.21.15.5:0/2215827359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba74694560 con 0x55ba7477b720 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b637fe700 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b627fc700 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747ba000 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f7b54001800 tx=0x7f7b5402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04c6ce3f1d295e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.194+0000 7f7b497fa700 1 -- 172.21.15.5:0/2215827359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b540149d0 con 0x55ba7477b720 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b497fa700 1 -- 172.21.15.5:0/2215827359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b54020b80 con 0x55ba7477b720 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b497fa700 1 -- 172.21.15.5:0/2215827359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b540014f0 con 0x55ba7477b720 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2215827359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 msgr2=0x55ba747ba000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747ba000 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f7b54001800 tx=0x7f7b5402c000 comp rx=0 tx=0).stop 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2215827359 shutdown_connections 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747ba000 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/2215827359 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2215827359 >> 172.21.15.5:0/2215827359 conn(0x55ba746dc210 msgr2=0x55ba747751e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2215827359 shutdown_connections 2024-05-08T23:27:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.195+0000 7f7b758e7b80 1 -- 172.21.15.5:0/2215827359 wait complete. 2024-05-08T23:27:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.196+0000 7f7b758e7b80 1 -- start start 2024-05-08T23:27:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.196+0000 7f7b758e7b80 1 Processor -- start 2024-05-08T23:27:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b758e7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba747a2dc0 con 0x55ba7477b720 2024-05-08T23:27:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba747a2fc0 con 0x55ba747a8dd0 2024-05-08T23:27:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b758e7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba747a3210 con 0x55ba747a6570 2024-05-08T23:27:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b758e7b80 10 client.0 _ll_register_callbacks cb 0x55ba74673b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747e51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59126/0 (socket says 172.21.15.5:59126) 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 -- 172.21.15.5:0/459789167 learned_addr learned my addr 172.21.15.5:0/459789167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b637fe700 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b627fc700 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 -- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 msgr2=0x55ba747e56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747e56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 -- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 msgr2=0x55ba747c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 -- 172.21.15.5:0/459789167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba744fbb60 con 0x55ba7477b720 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b627fc700 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b62ffd700 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747e51a0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7b54015fd0 tx=0x7f7b5400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f52d42d70860607 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b41ffb700 1 -- 172.21.15.5:0/459789167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54001580 con 0x55ba7477b720 2024-05-08T23:27:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.197+0000 7f7b41ffb700 1 -- 172.21.15.5:0/459789167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b54048ae0 con 0x55ba7477b720 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 1 -- 172.21.15.5:0/459789167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b54020410 con 0x55ba7477b720 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b41ffb700 1 -- 172.21.15.5:0/459789167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b54048ca0 con 0x55ba7477b720 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 10 client.8031 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 1 -- 172.21.15.5:0/459789167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55ba74553cd0 con 0x55ba7477b720 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b41ffb700 1 -- 172.21.15.5:0/459789167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b54020410 con 0x55ba7477b720 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b41ffb700 1 -- 172.21.15.5:0/459789167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f7b54012050 con 0x55ba7477b720 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 10 client.8031 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 10 client.8031 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 20 client.8031 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55ba7477dbc0 0x55ba747c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 4 client.8031 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b758e7b80 1 -- 172.21.15.5:0/459789167 --> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba746c6ec0 con 0x55ba7477dbc0 2024-05-08T23:27:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b627fc700 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55ba7477dbc0 0x55ba747c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b41ffb700 10 client.8031 ms_handle_connect on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:22.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.198+0000 7f7b627fc700 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55ba7477dbc0 0x55ba747c2de0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3983d352953f0ba4 server_cookie=2088a092d1aa0684 in_seq=0 out_seq=0 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b41ffb700 1 -- 172.21.15.5:0/459789167 <== mds.1 v2:172.21.15.5:6832/3718289875 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba746c6ec0 con 0x55ba7477dbc0 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b41ffb700 10 client.8031 handle_command_reply: tid=0 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 client.8031 shutdown 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 -- 172.21.15.5:0/459789167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 msgr2=0x55ba747e51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747e51a0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7b54015fd0 tx=0x7f7b5400a000 comp rx=0 tx=0).stop 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 -- 172.21.15.5:0/459789167 shutdown_connections 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ba747a8dd0 0x55ba747c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55ba7477dbc0 0x55ba747c2de0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba7477b720 0x55ba747e51a0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 --2- 172.21.15.5:0/459789167 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ba747a6570 0x55ba747e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b41ffb700 0 client.8031 ms_handle_reset on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:22.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 -- 172.21.15.5:0/459789167 >> 172.21.15.5:0/459789167 conn(0x55ba746dc210 msgr2=0x55ba747c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 -- 172.21.15.5:0/459789167 shutdown_connections 2024-05-08T23:27:22.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.206+0000 7f7b758e7b80 1 -- 172.21.15.5:0/459789167 wait complete. 2024-05-08T23:27:22.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b758e7b80 20 client.8031 trim_cache size 0 max 16384 2024-05-08T23:27:22.207 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:22.207 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:27:22.207 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:27:22.207 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:22.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 -- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b5006ac90 msgr2=0x7f7b5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b5006ac90 0x7f7b5006d150 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7b54007fd0 tx=0x7f7b54007e00 comp rx=0 tx=0).stop 2024-05-08T23:27:22.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 -- 172.21.15.5:0/2181289082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b640ca130 msgr2=0x7f7b641f29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b640ca130 0x7f7b641f29a0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7b580090b0 tx=0x7f7b58004750 comp rx=0 tx=0).stop 2024-05-08T23:27:22.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 -- 172.21.15.5:0/2181289082 shutdown_connections 2024-05-08T23:27:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b5006ac90 0x7f7b5006d150 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b6415a7c0 0x7f7b6414bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b640727b0 0x7f7b640b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 --2- 172.21.15.5:0/2181289082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b640ca130 0x7f7b641f29a0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 -- 172.21.15.5:0/2181289082 >> 172.21.15.5:0/2181289082 conn(0x7f7b6406a960 msgr2=0x7f7b6406ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 -- 172.21.15.5:0/2181289082 shutdown_connections 2024-05-08T23:27:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.207+0000 7f7b6972d700 1 -- 172.21.15.5:0/2181289082 wait complete. 2024-05-08T23:27:22.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 flush journal 2024-05-08T23:27:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.361+0000 7f8cd20cd700 1 Processor -- start 2024-05-08T23:27:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.361+0000 7f8cd20cd700 1 -- start start 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8cd20cd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc09a160 0x7f8ccc0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8cd20cd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8ccc0cff30 0x7f8ccc0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8cd20cd700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc0d04b0 con 0x7f8ccc15b710 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8cd20cd700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0b0910 con 0x7f8ccc0cff30 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8cd20cd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0b0bf0 con 0x7f8ccc09a160 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8ccb7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8ccc15b710 0x7f8ccc099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34010/0 (socket says 172.21.15.5:34010) 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/21966287 learned_addr learned my addr 172.21.15.5:0/21966287 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/21966287 >> v2:172.21.15.5:3300/0 conn(0x7f8ccc0cff30 0x7f8ccc0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.362+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/21966287 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc09a160 0x7f8ccc0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8cca7fc700 1 -- 172.21.15.5:0/21966287 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749546375 0 0) 0x7f8ccc0d04b0 con 0x7f8ccc15b710 2024-05-08T23:27:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8cca7fc700 1 -- 172.21.15.5:0/21966287 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb40035b0 con 0x7f8ccc15b710 2024-05-08T23:27:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8ccbfff700 1 -- 172.21.15.5:0/21966287 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc09a160 msgr2=0x7f8ccc0b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/21966287 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc09a160 0x7f8ccc0b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8ccbfff700 1 -- 172.21.15.5:0/21966287 >> v1:172.21.15.5:6789/0 conn(0x7f8ccc15b710 legacy=0x7f8ccc099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8ccbfff700 1 -- 172.21.15.5:0/21966287 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0b0ed0 con 0x7f8ccc0cff30 2024-05-08T23:27:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/21966287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc0cff30 0x7f8ccc0b1c10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009d90 tx=0x7f8cc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99422ba729b4b8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8cca7fc700 1 -- 172.21.15.5:0/21966287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400e070 con 0x7f8ccc0cff30 2024-05-08T23:27:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8cca7fc700 1 -- 172.21.15.5:0/21966287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc400bb20 con 0x7f8ccc0cff30 2024-05-08T23:27:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.363+0000 7f8cca7fc700 1 -- 172.21.15.5:0/21966287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400be90 con 0x7f8ccc0cff30 2024-05-08T23:27:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.364+0000 7f8cd20cd700 1 -- 172.21.15.5:0/21966287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc0cff30 msgr2=0x7f8ccc0b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.364+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/21966287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc0cff30 0x7f8ccc0b1c10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009d90 tx=0x7f8cc400b330 comp rx=0 tx=0).stop 2024-05-08T23:27:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.364+0000 7f8cd20cd700 1 -- 172.21.15.5:0/21966287 shutdown_connections 2024-05-08T23:27:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.364+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/21966287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc0cff30 0x7f8ccc0b1c10 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.364+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/21966287 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc09a160 0x7f8ccc0b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.364+0000 7f8cd20cd700 1 -- 172.21.15.5:0/21966287 >> 172.21.15.5:0/21966287 conn(0x7f8ccc0b2f60 msgr2=0x7f8ccc0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.364+0000 7f8cd20cd700 1 -- 172.21.15.5:0/21966287 shutdown_connections 2024-05-08T23:27:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.364+0000 7f8cd20cd700 1 -- 172.21.15.5:0/21966287 wait complete. 2024-05-08T23:27:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.365+0000 7f8cd20cd700 1 Processor -- start 2024-05-08T23:27:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.365+0000 7f8cd20cd700 1 -- start start 2024-05-08T23:27:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8cd20cd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc09a160 0x7f8ccc151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8cd20cd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc0cff30 0x7f8ccc14b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8cd20cd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ccc15b710 0x7f8ccc1f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8cd20cd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0b7070 con 0x7f8ccc09a160 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8cd20cd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc151a20 con 0x7f8ccc15b710 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8cd20cd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0c8d90 con 0x7f8ccc0cff30 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8ccb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc09a160 0x7f8ccc151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8ccb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc09a160 0x7f8ccc151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59152/0 (socket says 172.21.15.5:59152) 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/1739483981 learned_addr learned my addr 172.21.15.5:0/1739483981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ccc15b710 0x7f8ccc1f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.366+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc0cff30 0x7f8ccc14b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc0cff30 msgr2=0x7f8ccc14b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc0cff30 0x7f8ccc14b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ccc15b710 msgr2=0x7f8ccc1f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ccc15b710 0x7f8ccc1f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/1739483981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4009a70 con 0x7f8ccc09a160 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc09a160 0x7f8ccc151c20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00c420 tx=0x7f8cbc00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8cb3fff700 1 -- 172.21.15.5:0/1739483981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc005d70 con 0x7f8ccc09a160 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8cb3fff700 1 -- 172.21.15.5:0/1739483981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc0070d0 con 0x7f8ccc09a160 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8cd20cd700 1 -- 172.21.15.5:0/1739483981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc0cb490 con 0x7f8ccc09a160 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8cb3fff700 1 -- 172.21.15.5:0/1739483981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc007440 con 0x7f8ccc09a160 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.367+0000 7f8cd20cd700 1 -- 172.21.15.5:0/1739483981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc0cae40 con 0x7f8ccc09a160 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.369+0000 7f8cb3fff700 1 -- 172.21.15.5:0/1739483981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8cbc0075a0 con 0x7f8ccc09a160 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.369+0000 7f8cb3fff700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8cb806ac90 0x7f8cb806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.369+0000 7f8cb3fff700 1 -- 172.21.15.5:0/1739483981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cbc0885b0 con 0x7f8ccc09a160 2024-05-08T23:27:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.369+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8cb806ac90 0x7f8cb806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.370+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8cb806ac90 0x7f8cb806d150 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc15aa00 tx=0x7f8cc0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.371+0000 7f8cde287b80 1 Processor -- start 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.371+0000 7f8cde287b80 1 -- start start 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f0c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a63d0 0x55ac1f07e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8cde287b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1efd5150 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8cde287b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1efd2b10 con 0x55ac1f07c370 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8cde287b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac1efce400 con 0x55ac1f0a63d0 2024-05-08T23:27:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8ccb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8ccb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59164/0 (socket says 172.21.15.5:59164) 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/3755191591 learned_addr learned my addr 172.21.15.5:0/3755191591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f0c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.372+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a63d0 0x55ac1f07e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/3755191591 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a63d0 msgr2=0x55ac1f07e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a63d0 0x55ac1f07e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/3755191591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 msgr2=0x55ac1f0c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f0c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/3755191591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac1ee4f3b0 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0a93a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc086ae0 tx=0x7f8cbc057150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b253d55a728836c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8cb17fa700 1 -- 172.21.15.5:0/3755191591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc05b120 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8cb17fa700 1 -- 172.21.15.5:0/3755191591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc057590 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.373+0000 7f8cb17fa700 1 -- 172.21.15.5:0/3755191591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc05b310 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 -- 172.21.15.5:0/3755191591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 msgr2=0x55ac1f0a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0a93a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc086ae0 tx=0x7f8cbc057150 comp rx=0 tx=0).stop 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 -- 172.21.15.5:0/3755191591 shutdown_connections 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a63d0 0x55ac1f07e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0a93a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 --2- 172.21.15.5:0/3755191591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f0c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 -- 172.21.15.5:0/3755191591 >> 172.21.15.5:0/3755191591 conn(0x55ac1efdc210 msgr2=0x55ac1f0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 -- 172.21.15.5:0/3755191591 shutdown_connections 2024-05-08T23:27:22.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 -- 172.21.15.5:0/3755191591 wait complete. 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 -- start start 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.374+0000 7f8cde287b80 1 Processor -- start 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f09d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a63d0 0x55ac1f09ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 0x55ac1f0a0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8cde287b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1ef6a560 con 0x55ac1f0a63d0 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8cde287b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1ef6a200 con 0x55ac1f07c370 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8cde287b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac1ee516a0 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8cde287b80 10 client.0 _ll_register_callbacks cb 0x55ac1ef73b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8ccaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a63d0 0x55ac1f09ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.375+0000 7f8ccaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a63d0 0x55ac1f09ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59178/0 (socket says 172.21.15.5:59178) 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccaffd700 1 -- 172.21.15.5:0/1572810971 learned_addr learned my addr 172.21.15.5:0/1572810971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 0x55ac1f0a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f09d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccaffd700 1 -- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 msgr2=0x55ac1f0a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 0x55ac1f0a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccaffd700 1 -- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 msgr2=0x55ac1f09d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f09d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccaffd700 1 -- 172.21.15.5:0/1572810971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc0cae40 con 0x55ac1f0a63d0 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f09d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a63d0 0x55ac1f09ff50 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0015720 tx=0x7f8cc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d9ba55874bf001 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8caa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8caa7fc700 1 -- 172.21.15.5:0/1572810971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0004220 con 0x55ac1f0a63d0 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8caa7fc700 1 -- 172.21.15.5:0/1572810971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cc0004380 con 0x55ac1f0a63d0 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8cde287b80 1 -- 172.21.15.5:0/1572810971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac1ef27270 con 0x55ac1f0a63d0 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8caa7fc700 1 -- 172.21.15.5:0/1572810971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc0004540 con 0x55ac1f0a63d0 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8cde287b80 10 client.8046 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.376+0000 7f8cde287b80 1 -- 172.21.15.5:0/1572810971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55ac1edfbb60 con 0x55ac1f0a63d0 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8caa7fc700 1 -- 172.21.15.5:0/1572810971 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac1ef27270 con 0x55ac1f0a63d0 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8caa7fc700 1 -- 172.21.15.5:0/1572810971 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f8cc00036a0 con 0x55ac1f0a63d0 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8cde287b80 10 client.8046 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8cde287b80 10 client.8046 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8cde287b80 20 client.8046 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8cde287b80 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55ac1f0d0130 0x55ac1f0a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8cde287b80 4 client.8046 mds_command: new command op to 7336 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8cde287b80 1 -- 172.21.15.5:0/1572810971 --> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac1f04d7e0 con 0x55ac1f0d0130 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55ac1f0d0130 0x55ac1f0a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55ac1f0d0130 0x55ac1f0a2300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acded9be81da2b14 server_cookie=72dee2e8428c4ec8 in_seq=0 out_seq=0 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.377+0000 7f8caa7fc700 10 client.8046 ms_handle_connect on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8caa7fc700 1 -- 172.21.15.5:0/1572810971 <== mds.2 v2:172.21.15.49:6835/1134209849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac1f04d7e0 con 0x55ac1f0d0130 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8caa7fc700 10 client.8046 handle_command_reply: tid=0 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 client.8046 shutdown 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 -- 172.21.15.5:0/1572810971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a63d0 msgr2=0x55ac1f09ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a63d0 0x55ac1f09ff50 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0015720 tx=0x7f8cc0004750 comp rx=0 tx=0).stop 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 -- 172.21.15.5:0/1572810971 shutdown_connections 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 0x55ac1f0a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55ac1f0d0130 0x55ac1f0a2300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f07c370 0x55ac1f09d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 --2- 172.21.15.5:0/1572810971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a63d0 0x55ac1f09ff50 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 -- 172.21.15.5:0/1572810971 >> 172.21.15.5:0/1572810971 conn(0x55ac1efdc210 msgr2=0x55ac1f07e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8caa7fc700 0 client.8046 ms_handle_reset on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 -- 172.21.15.5:0/1572810971 shutdown_connections 2024-05-08T23:27:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 1 -- 172.21.15.5:0/1572810971 wait complete. 2024-05-08T23:27:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.378+0000 7f8cde287b80 20 client.8046 trim_cache size 0 max 16384 2024-05-08T23:27:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.394+0000 7f8cde287b80 1 Processor -- start 2024-05-08T23:27:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.394+0000 7f8cde287b80 1 -- start start 2024-05-08T23:27:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f07c370 0x55ac1f0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8cde287b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1f07b300 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8cde287b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1f0a6150 con 0x55ac1f0a63d0 2024-05-08T23:27:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8cde287b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac1f07aa80 con 0x55ac1f07c370 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8ccbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8ccbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59194/0 (socket says 172.21.15.5:59194) 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8ccbfff700 1 -- 172.21.15.5:0/3733900132 learned_addr learned my addr 172.21.15.5:0/3733900132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f07c370 0x55ac1f0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.395+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8ccbfff700 1 -- 172.21.15.5:0/3733900132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f07c370 msgr2=0x55ac1f0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f07c370 0x55ac1f0c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8ccbfff700 1 -- 172.21.15.5:0/3733900132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 msgr2=0x55ac1f0a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8ccbfff700 1 -- 172.21.15.5:0/3733900132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac1ef94560 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0c46f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b370 tx=0x7f8cc4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf728cf7954abe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8cb1ffb700 1 -- 172.21.15.5:0/3733900132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc402b070 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8cb1ffb700 1 -- 172.21.15.5:0/3733900132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc4003d60 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.396+0000 7f8cb1ffb700 1 -- 172.21.15.5:0/3733900132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4020040 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 -- 172.21.15.5:0/3733900132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 msgr2=0x55ac1f0c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0c46f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b370 tx=0x7f8cc4011000 comp rx=0 tx=0).stop 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 -- 172.21.15.5:0/3733900132 shutdown_connections 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f0a8fc0 0x55ac1f0c46f0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f07c370 0x55ac1f0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 --2- 172.21.15.5:0/3733900132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 -- 172.21.15.5:0/3733900132 >> 172.21.15.5:0/3733900132 conn(0x55ac1efdc210 msgr2=0x55ac1f0751e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 -- 172.21.15.5:0/3733900132 shutdown_connections 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 -- 172.21.15.5:0/3733900132 wait complete. 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 -- start start 2024-05-08T23:27:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.397+0000 7f8cde287b80 1 Processor -- start 2024-05-08T23:27:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f07c370 0x55ac1f1304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8cde287b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 0x55ac1f09fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8cde287b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1f09bc10 con 0x55ac1f07c370 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8cde287b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1f078e60 con 0x55ac1f0a63d0 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8cde287b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac1f0799d0 con 0x55ac1f0a8fc0 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8cde287b80 10 client.0 _ll_register_callbacks cb 0x7f8cc0010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8ccb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f07c370 0x55ac1f1304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8ccb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f07c370 0x55ac1f1304a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59208/0 (socket says 172.21.15.5:59208) 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.398+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/513451577 learned_addr learned my addr 172.21.15.5:0/513451577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccbfff700 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 0x55ac1f09fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 msgr2=0x55ac1f09fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 0x55ac1f09fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 msgr2=0x55ac1f0a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccb7fe700 1 -- 172.21.15.5:0/513451577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc0b0ed0 con 0x55ac1f07c370 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8ccb7fe700 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f07c370 0x55ac1f1304a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc057240 tx=0x7f8cbc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f2c281b9b51bd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8caa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8caa7fc700 1 -- 172.21.15.5:0/513451577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc0538d0 con 0x55ac1f07c370 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8caa7fc700 1 -- 172.21.15.5:0/513451577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cbc071030 con 0x55ac1f07c370 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8cde287b80 1 -- 172.21.15.5:0/513451577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cbc057590 con 0x55ac1f07c370 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8caa7fc700 1 -- 172.21.15.5:0/513451577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cbc05f8d0 con 0x55ac1f07c370 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8cde287b80 10 client.8052 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8cde287b80 1 -- 172.21.15.5:0/513451577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55ac1ee53cd0 con 0x55ac1f07c370 2024-05-08T23:27:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.399+0000 7f8caa7fc700 1 -- 172.21.15.5:0/513451577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc057590 con 0x55ac1f07c370 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8caa7fc700 1 -- 172.21.15.5:0/513451577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f8cbc05f440 con 0x55ac1f07c370 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8cde287b80 10 client.8052 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8cde287b80 10 client.8052 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8cde287b80 20 client.8052 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8cde287b80 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55ac1f0a0170 0x55ac1f0e0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8cde287b80 4 client.8052 mds_command: new command op to 7336 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8cde287b80 1 -- 172.21.15.5:0/513451577 --> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ac1efc6ec0 con 0x55ac1f0a0170 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55ac1f0a0170 0x55ac1f0e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8ccaffd700 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55ac1f0a0170 0x55ac1f0e0620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86024084e3acc8cd server_cookie=4c2d3d282b607a4c in_seq=0 out_seq=0 2024-05-08T23:27:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.400+0000 7f8caa7fc700 10 client.8052 ms_handle_connect on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.406+0000 7f8caa7fc700 1 -- 172.21.15.5:0/513451577 <== mds.2 v2:172.21.15.49:6835/1134209849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ac1efc6ec0 con 0x55ac1f0a0170 2024-05-08T23:27:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.406+0000 7f8caa7fc700 10 client.8052 handle_command_reply: tid=0 2024-05-08T23:27:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 client.8052 shutdown 2024-05-08T23:27:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 -- 172.21.15.5:0/513451577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f07c370 msgr2=0x55ac1f1304a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f07c370 0x55ac1f1304a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc057240 tx=0x7f8cbc000bc0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 -- 172.21.15.5:0/513451577 shutdown_connections 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55ac1f0a8fc0 0x55ac1f09fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1f07c370 0x55ac1f1304a0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55ac1f0a0170 0x55ac1f0e0620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 --2- 172.21.15.5:0/513451577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac1f0a63d0 0x55ac1f0a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8caa7fc700 0 client.8052 ms_handle_reset on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 -- 172.21.15.5:0/513451577 >> 172.21.15.5:0/513451577 conn(0x55ac1efdc210 msgr2=0x55ac1f0c4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 -- 172.21.15.5:0/513451577 shutdown_connections 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 1 -- 172.21.15.5:0/513451577 wait complete. 2024-05-08T23:27:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cde287b80 20 client.8052 trim_cache size 0 max 16384 2024-05-08T23:27:22.408 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:27:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:27:22.408 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cd20cd700 1 -- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8cb806ac90 msgr2=0x7f8cb806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8cb806ac90 0x7f8cb806d150 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc15aa00 tx=0x7f8cc0008040 comp rx=0 tx=0).stop 2024-05-08T23:27:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cd20cd700 1 -- 172.21.15.5:0/1739483981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc09a160 msgr2=0x7f8ccc151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.407+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc09a160 0x7f8ccc151c20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00c420 tx=0x7f8cbc00c7e0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.408+0000 7f8cd20cd700 1 -- 172.21.15.5:0/1739483981 shutdown_connections 2024-05-08T23:27:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.408+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8cb806ac90 0x7f8cb806d150 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.408+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ccc15b710 0x7f8ccc1f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.408+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc09a160 0x7f8ccc151c20 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.408+0000 7f8cd20cd700 1 --2- 172.21.15.5:0/1739483981 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ccc0cff30 0x7f8ccc14b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.408+0000 7f8cd20cd700 1 -- 172.21.15.5:0/1739483981 >> 172.21.15.5:0/1739483981 conn(0x7f8ccc0b2f60 msgr2=0x7f8ccc070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.408+0000 7f8cd20cd700 1 -- 172.21.15.5:0/1739483981 shutdown_connections 2024-05-08T23:27:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:22.408+0000 7f8cd20cd700 1 -- 172.21.15.5:0/1739483981 wait complete. 2024-05-08T23:27:22.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:22.420 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-05-08T23:27:22.447 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/ 2024-05-08T23:27:22.447 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4 2024-05-08T23:27:22.447 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5 2024-05-08T23:27:22.447 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6 2024-05-08T23:27:22.447 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7 2024-05-08T23:27:22.447 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-05-08T23:27:22.448 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:27:22.448 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:27:22.448 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:27:22.448 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:27:22.448 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-05-08T23:27:22.448 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:27:22.566 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-08T23:27:22.755 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:27:22.755 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:27:22.755 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:27:22.755 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:27:22.755 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-05-08T23:27:22.755 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:27:22.875 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-05-08T23:27:23.064 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:27:23.064 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:27:23.064 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:27:23.065 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:27:23.065 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-05-08T23:27:23.065 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:27:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-05-08T23:27:23.375 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:27:23.375 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:27:23.375 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:27:23.375 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:27:23.375 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-05-08T23:27:23.375 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:27:23.495 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627781 2024-05-08T23:27:23.684 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:27:23.684 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:27:23.684 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:27:23.684 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:27:23.684 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-05-08T23:27:23.684 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:27:23.806 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627782 2024-05-08T23:27:23.995 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:27:23.995 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:27:23.995 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:27:23.995 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:27:23.995 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-05-08T23:27:23.995 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:27:24.116 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627783 2024-05-08T23:27:24.305 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-05-08T23:27:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.399+0000 7f3b385bd080 1 Processor -- start 2024-05-08T23:27:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.399+0000 7f3b385bd080 1 -- start start 2024-05-08T23:27:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b385bd080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d9b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b385bd080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59da4b00 0x55cd59da94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b385bd080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59dab350 0x55cd59da99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b385bd080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd59c7c8b0 con 0x55cd59d1b2e0 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b385bd080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd59c781a0 con 0x55cd59da4b00 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b385bd080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd59c136d0 con 0x55cd59dab350 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2e68f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d9b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2e68f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d9b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59220/0 (socket says 172.21.15.5:59220) 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2e68f700 1 -- 172.21.15.5:0/3025570663 learned_addr learned my addr 172.21.15.5:0/3025570663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2ee90700 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59dab350 0x55cd59da99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2de8e700 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59da4b00 0x55cd59da94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2e68f700 1 -- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59dab350 msgr2=0x55cd59da99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2e68f700 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59dab350 0x55cd59da99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2e68f700 1 -- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59da4b00 msgr2=0x55cd59da94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2e68f700 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59da4b00 0x55cd59da94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.400+0000 7f3b2e68f700 1 -- 172.21.15.5:0/3025570663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2800a100 con 0x55cd59d1b2e0 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b2ee90700 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59dab350 0x55cd59da99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b2de8e700 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59da4b00 0x55cd59da94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b2e68f700 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d9b2e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000b2c0 tx=0x7f3b2000b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182c8b69c837e517 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b2ce8c700 1 -- 172.21.15.5:0/3025570663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20014070 con 0x55cd59d1b2e0 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b2ce8c700 1 -- 172.21.15.5:0/3025570663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2000f3f0 con 0x55cd59d1b2e0 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b2ce8c700 1 -- 172.21.15.5:0/3025570663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000f7a0 con 0x55cd59d1b2e0 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b385bd080 1 -- 172.21.15.5:0/3025570663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 msgr2=0x55cd59d9b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b385bd080 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d9b2e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000b2c0 tx=0x7f3b2000b5d0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b385bd080 1 -- 172.21.15.5:0/3025570663 shutdown_connections 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b385bd080 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59dab350 0x55cd59da99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b385bd080 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d9b2e0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b385bd080 1 --2- 172.21.15.5:0/3025570663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59da4b00 0x55cd59da94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b385bd080 1 -- 172.21.15.5:0/3025570663 >> 172.21.15.5:0/3025570663 conn(0x55cd59cfd420 msgr2=0x55cd59d01890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.401+0000 7f3b385bd080 1 -- 172.21.15.5:0/3025570663 shutdown_connections 2024-05-08T23:27:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.402+0000 7f3b385bd080 1 -- 172.21.15.5:0/3025570663 wait complete. 2024-05-08T23:27:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.402+0000 7f3b385bd080 1 Processor -- start 2024-05-08T23:27:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.402+0000 7f3b385bd080 1 -- start start 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b385bd080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d96fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b385bd080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59da4b00 0x55cd59d00d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b385bd080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59dab350 0x55cd59cfdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b385bd080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd59c13370 con 0x55cd59d1b2e0 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b385bd080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd59c0a680 con 0x55cd59dab350 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b385bd080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd59d01450 con 0x55cd59da4b00 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b2e68f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d96fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b2e68f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d96fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59234/0 (socket says 172.21.15.5:59234) 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b2e68f700 1 -- 172.21.15.5:0/1181696136 learned_addr learned my addr 172.21.15.5:0/1181696136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b2ee90700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59dab350 0x55cd59cfdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.403+0000 7f3b2de8e700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59da4b00 0x55cd59d00d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b2e68f700 1 -- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59da4b00 msgr2=0x55cd59d00d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b2e68f700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cd59da4b00 0x55cd59d00d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b2e68f700 1 -- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59dab350 msgr2=0x55cd59cfdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b2e68f700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cd59dab350 0x55cd59cfdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b2e68f700 1 -- 172.21.15.5:0/1181696136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20001ba0 con 0x55cd59d1b2e0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b2e68f700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d96fb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3b20000c00 tx=0x7f3b20000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b1f7fe700 1 -- 172.21.15.5:0/1181696136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20014070 con 0x55cd59d1b2e0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b1f7fe700 1 -- 172.21.15.5:0/1181696136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2001c850 con 0x55cd59d1b2e0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b1f7fe700 1 -- 172.21.15.5:0/1181696136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2001cc00 con 0x55cd59d1b2e0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd59c15ed0 con 0x55cd59d1b2e0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.404+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd59c15930 con 0x55cd59d1b2e0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.406+0000 7f3b1f7fe700 1 -- 172.21.15.5:0/1181696136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3b2000d040 con 0x55cd59d1b2e0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.406+0000 7f3b1f7fe700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b100695e0 0x7f3b1006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.406+0000 7f3b1f7fe700 1 -- 172.21.15.5:0/1181696136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b2000a040 con 0x55cd59d1b2e0 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.406+0000 7f3b385bd080 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x55cd59cfe500 0x55cd59d93b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.406+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 --> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] -- osd_op(unknown.0.0:1 12.12 12:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) v8 -- 0x55cd59cd3e60 con 0x55cd59cfe500 2024-05-08T23:27:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.407+0000 7f3b2de8e700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b100695e0 0x7f3b1006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.407+0000 7f3b2ee90700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x55cd59cfe500 0x55cd59d93b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.407+0000 7f3b2ee90700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x55cd59cfe500 0x55cd59d93b10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.407+0000 7f3b2de8e700 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b100695e0 0x7f3b1006baa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3b24000c00 tx=0x7f3b240038b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.408+0000 7f3b2ee90700 1 -- 172.21.15.5:0/1181696136 <== osd.3 v2:172.21.15.5:6808/47142 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v84'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3b2800fa80 con 0x55cd59cfe500 2024-05-08T23:27:24.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x55cd59cfe500 msgr2=0x55cd59d93b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x55cd59cfe500 0x55cd59d93b10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b100695e0 msgr2=0x7f3b1006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b100695e0 0x7f3b1006baa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3b24000c00 tx=0x7f3b240038b0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 msgr2=0x55cd59d96fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 --2- 172.21.15.5:0/1181696136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd59d1b2e0 0x55cd59d96fb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3b20000c00 tx=0x7f3b20000ed0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b2ee90700 1 -- 172.21.15.5:0/1181696136 reap_dead start 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 shutdown_connections 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 >> 172.21.15.5:0/1181696136 conn(0x55cd59cfd420 msgr2=0x55cd59d9b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 shutdown_connections 2024-05-08T23:27:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.409+0000 7f3b385bd080 1 -- 172.21.15.5:0/1181696136 wait complete. 2024-05-08T23:27:24.414 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-05-08T23:27:24.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.465+0000 7f4a6a677080 1 Processor -- start 2024-05-08T23:27:24.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.465+0000 7f4a6a677080 1 -- start start 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a6a677080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787570 0x563f81859ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a6a677080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787a50 0x563f817cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a6a677080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f817cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a6a677080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8172e1a0 con 0x563f8185be20 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a6a677080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f816c96d0 con 0x563f81787a50 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a6a677080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f816c9370 con 0x563f81787570 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f817cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f817cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59244/0 (socket says 172.21.15.5:59244) 2024-05-08T23:27:24.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 -- 172.21.15.5:0/173122254 learned_addr learned my addr 172.21.15.5:0/173122254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a5ff48700 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787a50 0x563f817cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60749700 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787570 0x563f81859ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 -- 172.21.15.5:0/173122254 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787570 msgr2=0x563f81859ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787570 0x563f81859ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 -- 172.21.15.5:0/173122254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787a50 msgr2=0x563f817cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787a50 0x563f817cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 -- 172.21.15.5:0/173122254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5800a100 con 0x563f8185be20 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.466+0000 7f4a60f4a700 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f817cf560 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800a880 tx=0x7f4a58005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9dbd37296febc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a5ef46700 1 -- 172.21.15.5:0/173122254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800a400 con 0x563f8185be20 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a5ef46700 1 -- 172.21.15.5:0/173122254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a58012660 con 0x563f8185be20 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a5ef46700 1 -- 172.21.15.5:0/173122254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58012a10 con 0x563f8185be20 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 -- 172.21.15.5:0/173122254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 msgr2=0x563f817cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f817cf560 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800a880 tx=0x7f4a58005d80 comp rx=0 tx=0).stop 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 -- 172.21.15.5:0/173122254 shutdown_connections 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f817cf560 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787570 0x563f81859ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 --2- 172.21.15.5:0/173122254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787a50 0x563f817cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 -- 172.21.15.5:0/173122254 >> 172.21.15.5:0/173122254 conn(0x563f817b22d0 msgr2=0x563f8185b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 -- 172.21.15.5:0/173122254 shutdown_connections 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.467+0000 7f4a6a677080 1 -- 172.21.15.5:0/173122254 wait complete. 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.468+0000 7f4a6a677080 1 Processor -- start 2024-05-08T23:27:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.468+0000 7f4a6a677080 1 -- start start 2024-05-08T23:27:24.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a6a677080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787570 0x563f8185a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a6a677080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787a50 0x563f818587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a6a677080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f81858d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a6a677080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f816c0680 con 0x563f8185be20 2024-05-08T23:27:24.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a6a677080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8185b190 con 0x563f81787570 2024-05-08T23:27:24.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a6a677080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f8185b620 con 0x563f81787a50 2024-05-08T23:27:24.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f81858d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f81858d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59254/0 (socket says 172.21.15.5:59254) 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 -- 172.21.15.5:0/2102748230 learned_addr learned my addr 172.21.15.5:0/2102748230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a5ff48700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787a50 0x563f818587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60749700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787570 0x563f8185a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 -- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787a50 msgr2=0x563f818587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563f81787a50 0x563f818587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 -- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787570 msgr2=0x563f8185a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f81787570 0x563f8185a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 -- 172.21.15.5:0/2102748230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5800a560 con 0x563f8185be20 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.469+0000 7f4a60f4a700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f81858d30 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x563f817b7940 tx=0x7f4a580033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.470+0000 7f4a5df44700 1 -- 172.21.15.5:0/2102748230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58009070 con 0x563f8185be20 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.470+0000 7f4a5df44700 1 -- 172.21.15.5:0/2102748230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5800dc10 con 0x563f8185be20 2024-05-08T23:27:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.470+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a50005d20 con 0x563f8185be20 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.470+0000 7f4a5df44700 1 -- 172.21.15.5:0/2102748230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58005a40 con 0x563f8185be20 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.470+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f816d88a0 con 0x563f8185be20 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.471+0000 7f4a5df44700 1 -- 172.21.15.5:0/2102748230 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a58026460 con 0x563f8185be20 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.472+0000 7f4a5df44700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a4c0695e0 0x7f4a4c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.472+0000 7f4a5df44700 1 -- 172.21.15.5:0/2102748230 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a5808a480 con 0x563f8185be20 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.472+0000 7f4a6a677080 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x563f817cf1b0 0x563f817afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.472+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 --> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] -- osd_op(unknown.0.0:1 12.12 12:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) v8 -- 0x563f818f8ca0 con 0x563f817cf1b0 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.472+0000 7f4a60749700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a4c0695e0 0x7f4a4c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.472+0000 7f4a5ff48700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x563f817cf1b0 0x563f817afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.472+0000 7f4a5ff48700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x563f817cf1b0 0x563f817afda0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.473+0000 7f4a60749700 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a4c0695e0 0x7f4a4c06baa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4a50000c00 tx=0x7f4a5000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a5ff48700 1 -- 172.21.15.5:0/2102748230 <== osd.3 v2:172.21.15.5:6808/47142 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v84'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4a5400e4f0 con 0x563f817cf1b0 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x563f817cf1b0 msgr2=0x563f817afda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x563f817cf1b0 0x563f817afda0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a4c0695e0 msgr2=0x7f4a4c06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4a4c0695e0 0x7f4a4c06baa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4a50000c00 tx=0x7f4a5000f000 comp rx=0 tx=0).stop 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 msgr2=0x563f81858d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 --2- 172.21.15.5:0/2102748230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f8185be20 0x563f81858d30 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x563f817b7940 tx=0x7f4a580033a0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a60f4a700 1 -- 172.21.15.5:0/2102748230 reap_dead start 2024-05-08T23:27:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 shutdown_connections 2024-05-08T23:27:24.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 >> 172.21.15.5:0/2102748230 conn(0x563f817b22d0 msgr2=0x563f817cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 shutdown_connections 2024-05-08T23:27:24.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.474+0000 7f4a6a677080 1 -- 172.21.15.5:0/2102748230 wait complete. 2024-05-08T23:27:24.478 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-05-08T23:27:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.529+0000 7f7c66e2b080 1 Processor -- start 2024-05-08T23:27:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.529+0000 7f7c66e2b080 1 -- start start 2024-05-08T23:27:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c66e2b080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f477624fa0 0x55f47764e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c66e2b080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47766ddc0 0x55f47766c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c66e2b080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 0x55f4776f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c66e2b080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4775ce8b0 con 0x55f47766ddc0 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c66e2b080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4775ca1a0 con 0x55f4776f7e10 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c66e2b080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4775656d0 con 0x55f477624fa0 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c546fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47766ddc0 0x55f47766c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c546fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47766ddc0 0x55f47766c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59276/0 (socket says 172.21.15.5:59276) 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c546fc700 1 -- 172.21.15.5:0/83844212 learned_addr learned my addr 172.21.15.5:0/83844212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c546fc700 1 -- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f477624fa0 msgr2=0x55f47764e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c5cefd700 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f477624fa0 0x55f47764e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c5d6fe700 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 0x55f4776f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c546fc700 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f477624fa0 0x55f47764e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.530+0000 7f7c546fc700 1 -- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 msgr2=0x55f4776f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c546fc700 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 0x55f4776f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c546fc700 1 -- 172.21.15.5:0/83844212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c005d20 con 0x55f47766ddc0 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c5cefd700 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f477624fa0 0x55f47764e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c5d6fe700 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 0x55f4776f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c546fc700 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47766ddc0 0x55f47766c160 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b2c0 tx=0x7f7c4400b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c69a5040618fe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c577fe700 1 -- 172.21.15.5:0/83844212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44014070 con 0x55f47766ddc0 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c577fe700 1 -- 172.21.15.5:0/83844212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4400f3f0 con 0x55f47766ddc0 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c577fe700 1 -- 172.21.15.5:0/83844212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400f7a0 con 0x55f47766ddc0 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c66e2b080 1 -- 172.21.15.5:0/83844212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47766ddc0 msgr2=0x55f47766c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c66e2b080 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47766ddc0 0x55f47766c160 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b2c0 tx=0x7f7c4400b5d0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c66e2b080 1 -- 172.21.15.5:0/83844212 shutdown_connections 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c66e2b080 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 0x55f4776f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c66e2b080 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f477624fa0 0x55f47764e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c66e2b080 1 --2- 172.21.15.5:0/83844212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47766ddc0 0x55f47766c160 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.531+0000 7f7c66e2b080 1 -- 172.21.15.5:0/83844212 >> 172.21.15.5:0/83844212 conn(0x55f477623ba0 msgr2=0x55f4776f6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.532+0000 7f7c66e2b080 1 -- 172.21.15.5:0/83844212 shutdown_connections 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.532+0000 7f7c66e2b080 1 -- 172.21.15.5:0/83844212 wait complete. 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.532+0000 7f7c66e2b080 1 Processor -- start 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.532+0000 7f7c66e2b080 1 -- start start 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c66e2b080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f477624fa0 0x55f4776e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c66e2b080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f47766ddc0 0x55f4776ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c66e2b080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 0x55f4776534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c66e2b080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f477565370 con 0x55f477624fa0 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c66e2b080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47755c680 con 0x55f4776f7e10 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c66e2b080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4776e9170 con 0x55f47766ddc0 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c5cefd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f477624fa0 0x55f4776e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c5cefd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f477624fa0 0x55f4776e2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59280/0 (socket says 172.21.15.5:59280) 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c5cefd700 1 -- 172.21.15.5:0/2736465754 learned_addr learned my addr 172.21.15.5:0/2736465754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c546fc700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f47766ddc0 0x55f4776ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c5d6fe700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 0x55f4776534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.533+0000 7f7c5cefd700 1 -- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f47766ddc0 msgr2=0x55f4776ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c5cefd700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f47766ddc0 0x55f4776ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c5cefd700 1 -- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 msgr2=0x55f4776534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c5cefd700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f4776f7e10 0x55f4776534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c5cefd700 1 -- 172.21.15.5:0/2736465754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c44001ba0 con 0x55f477624fa0 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c5cefd700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f477624fa0 0x55f4776e2a80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c008780 tx=0x7f7c4c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c567fc700 1 -- 172.21.15.5:0/2736465754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00a250 con 0x55f477624fa0 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c567fc700 1 -- 172.21.15.5:0/2736465754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c01e480 con 0x55f477624fa0 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c567fc700 1 -- 172.21.15.5:0/2736465754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c01e7f0 con 0x55f477624fa0 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f477567ed0 con 0x55f477624fa0 2024-05-08T23:27:24.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.534+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f477567930 con 0x55f477624fa0 2024-05-08T23:27:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.535+0000 7f7c567fc700 1 -- 172.21.15.5:0/2736465754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7c4c01e950 con 0x55f477624fa0 2024-05-08T23:27:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.536+0000 7f7c567fc700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c3c0695e0 0x7f7c3c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.536+0000 7f7c567fc700 1 -- 172.21.15.5:0/2736465754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c4c089f00 con 0x55f477624fa0 2024-05-08T23:27:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.536+0000 7f7c546fc700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c3c0695e0 0x7f7c3c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.536+0000 7f7c66e2b080 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55f4776e79d0 0x55f4776e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.536+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 12.5 12:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) v8 -- 0x55f4776e8330 con 0x55f4776e79d0 2024-05-08T23:27:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.536+0000 7f7c5d6fe700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55f4776e79d0 0x55f4776e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.537+0000 7f7c546fc700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c3c0695e0 0x7f7c3c06baa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b2c0 tx=0x7f7c44009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.537+0000 7f7c5d6fe700 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55f4776e79d0 0x55f4776e7df0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.538+0000 7f7c5d6fe700 1 -- 172.21.15.5:0/2736465754 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v84'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7c5800c700 con 0x55f4776e79d0 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.538+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55f4776e79d0 msgr2=0x55f4776e7df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.538+0000 7f7c66e2b080 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x55f4776e79d0 0x55f4776e7df0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.538+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c3c0695e0 msgr2=0x7f7c3c06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.538+0000 7f7c66e2b080 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c3c0695e0 0x7f7c3c06baa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b2c0 tx=0x7f7c44009000 comp rx=0 tx=0).stop 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.538+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f477624fa0 msgr2=0x55f4776e2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.538+0000 7f7c66e2b080 1 --2- 172.21.15.5:0/2736465754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f477624fa0 0x55f4776e2a80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c008780 tx=0x7f7c4c002fb0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.538+0000 7f7c5d6fe700 1 -- 172.21.15.5:0/2736465754 reap_dead start 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.539+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 shutdown_connections 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.539+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 >> 172.21.15.5:0/2736465754 conn(0x55f477623ba0 msgr2=0x55f477652bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.539+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 shutdown_connections 2024-05-08T23:27:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.539+0000 7f7c66e2b080 1 -- 172.21.15.5:0/2736465754 wait complete. 2024-05-08T23:27:24.543 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-05-08T23:27:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.593+0000 7fd317ef7080 1 Processor -- start 2024-05-08T23:27:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.593+0000 7fd317ef7080 1 -- start start 2024-05-08T23:27:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.593+0000 7fd317ef7080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9570 0x560f2c1012b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.593+0000 7fd317ef7080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c0b9a50 0x560f2c18c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.593+0000 7fd317ef7080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c18de20 0x560f2c18ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.593+0000 7fd317ef7080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2c0601a0 con 0x560f2c0b9a50 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.593+0000 7fd317ef7080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2bffb6d0 con 0x560f2c0b9570 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.593+0000 7fd317ef7080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f2bffb370 con 0x560f2c18de20 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c0b9a50 0x560f2c18c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c0b9a50 0x560f2c18c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59292/0 (socket says 172.21.15.5:59292) 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 -- 172.21.15.5:0/3708823009 learned_addr learned my addr 172.21.15.5:0/3708823009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30dfc9700 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9570 0x560f2c1012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30e7ca700 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c18de20 0x560f2c18ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 -- 172.21.15.5:0/3708823009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c18de20 msgr2=0x560f2c18ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c18de20 0x560f2c18ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 -- 172.21.15.5:0/3708823009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9570 msgr2=0x560f2c1012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9570 0x560f2c1012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 -- 172.21.15.5:0/3708823009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30800a1c0 con 0x560f2c0b9a50 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd30d7c8700 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c0b9a50 0x560f2c18c510 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd304001d60 tx=0x7fd30400b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6814b3a5adeb5158 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd2fffff700 1 -- 172.21.15.5:0/3708823009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400e4b0 con 0x560f2c0b9a50 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd2fffff700 1 -- 172.21.15.5:0/3708823009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30400e610 con 0x560f2c0b9a50 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.594+0000 7fd2fffff700 1 -- 172.21.15.5:0/3708823009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400e9c0 con 0x560f2c0b9a50 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 -- 172.21.15.5:0/3708823009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c0b9a50 msgr2=0x560f2c18c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c0b9a50 0x560f2c18c510 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd304001d60 tx=0x7fd30400b510 comp rx=0 tx=0).stop 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 -- 172.21.15.5:0/3708823009 shutdown_connections 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c18de20 0x560f2c18ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9570 0x560f2c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 --2- 172.21.15.5:0/3708823009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c0b9a50 0x560f2c18c510 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 -- 172.21.15.5:0/3708823009 >> 172.21.15.5:0/3708823009 conn(0x560f2c0e42d0 msgr2=0x560f2c18e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 -- 172.21.15.5:0/3708823009 shutdown_connections 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.595+0000 7fd317ef7080 1 -- 172.21.15.5:0/3708823009 wait complete. 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.596+0000 7fd317ef7080 1 Processor -- start 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.596+0000 7fd317ef7080 1 -- start start 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.596+0000 7fd317ef7080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c0b9570 0x560f2c1816f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.596+0000 7fd317ef7080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9a50 0x560f2c18b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.596+0000 7fd317ef7080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c18de20 0x560f2c18c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.596+0000 7fd317ef7080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2bff2680 con 0x560f2c18de20 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.596+0000 7fd317ef7080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2c0e3ca0 con 0x560f2c0b9a50 2024-05-08T23:27:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.596+0000 7fd317ef7080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f2c181c30 con 0x560f2c0b9570 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30e7ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c18de20 0x560f2c18c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30d7c8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9a50 0x560f2c18b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30d7c8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9a50 0x560f2c18b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:55466/0 (socket says 172.21.15.5:55466) 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30d7c8700 1 -- 172.21.15.5:0/2889906948 learned_addr learned my addr 172.21.15.5:0/2889906948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30dfc9700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c0b9570 0x560f2c1816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30e7ca700 1 -- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c0b9570 msgr2=0x560f2c1816f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30e7ca700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x560f2c0b9570 0x560f2c1816f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30e7ca700 1 -- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9a50 msgr2=0x560f2c18b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30e7ca700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f2c0b9a50 0x560f2c18b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30e7ca700 1 -- 172.21.15.5:0/2889906948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f2bffdb40 con 0x560f2c18de20 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd30e7ca700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c18de20 0x560f2c18c300 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd308012000 tx=0x7fd308003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd2feffd700 1 -- 172.21.15.5:0/2889906948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308010970 con 0x560f2c18de20 2024-05-08T23:27:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd2feffd700 1 -- 172.21.15.5:0/2889906948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd308020070 con 0x560f2c18de20 2024-05-08T23:27:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd2feffd700 1 -- 172.21.15.5:0/2889906948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308010d10 con 0x560f2c18de20 2024-05-08T23:27:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f2c00a8a0 con 0x560f2c18de20 2024-05-08T23:27:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.597+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f2bffded0 con 0x560f2c18de20 2024-05-08T23:27:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.599+0000 7fd2feffd700 1 -- 172.21.15.5:0/2889906948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd308022030 con 0x560f2c18de20 2024-05-08T23:27:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.599+0000 7fd2feffd700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd2f00695e0 0x7fd2f006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.599+0000 7fd2feffd700 1 -- 172.21.15.5:0/2889906948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd308089a80 con 0x560f2c18de20 2024-05-08T23:27:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.600+0000 7fd30dfc9700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd2f00695e0 0x7fd2f006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.600+0000 7fd317ef7080 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x560f2c101340 0x560f2c0fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.600+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 --> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] -- osd_op(unknown.0.0:1 12.9 12:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) v8 -- 0x560f2c0fd320 con 0x560f2c101340 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.600+0000 7fd30d7c8700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x560f2c101340 0x560f2c0fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.600+0000 7fd30dfc9700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd2f00695e0 0x7fd2f006baa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd300005400 tx=0x7fd30000c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.600+0000 7fd30d7c8700 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x560f2c101340 0x560f2c0fcde0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.601+0000 7fd30d7c8700 1 -- 172.21.15.5:0/2889906948 <== osd.2 v2:172.21.15.5:6800/47131 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v84'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd304013040 con 0x560f2c101340 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.601+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x560f2c101340 msgr2=0x560f2c0fcde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.601+0000 7fd317ef7080 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x560f2c101340 0x560f2c0fcde0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd2f00695e0 msgr2=0x7fd2f006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd317ef7080 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd2f00695e0 0x7fd2f006baa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd300005400 tx=0x7fd30000c040 comp rx=0 tx=0).stop 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c18de20 msgr2=0x560f2c18c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd317ef7080 1 --2- 172.21.15.5:0/2889906948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f2c18de20 0x560f2c18c300 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd308012000 tx=0x7fd308003330 comp rx=0 tx=0).stop 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd30e7ca700 1 -- 172.21.15.5:0/2889906948 reap_dead start 2024-05-08T23:27:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 shutdown_connections 2024-05-08T23:27:24.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 >> 172.21.15.5:0/2889906948 conn(0x560f2c0e42d0 msgr2=0x560f2c18e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 shutdown_connections 2024-05-08T23:27:24.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.602+0000 7fd317ef7080 1 -- 172.21.15.5:0/2889906948 wait complete. 2024-05-08T23:27:24.606 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-05-08T23:27:24.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.656+0000 7f9fc46da080 1 Processor -- start 2024-05-08T23:27:24.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.656+0000 7f9fc46da080 1 -- start start 2024-05-08T23:27:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fc46da080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa40039d0 0x557730027ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fc46da080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557730143af0 0x557730142a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fc46da080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577300b1400 0x557730142f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fc46da080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772ff8d8b0 con 0x7f9fa40039d0 2024-05-08T23:27:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fc46da080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772ff891a0 con 0x557730143af0 2024-05-08T23:27:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fc46da080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55772ff246d0 con 0x5577300b1400 2024-05-08T23:27:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fba7ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa40039d0 0x557730027ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fba7ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa40039d0 0x557730027ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59320/0 (socket says 172.21.15.5:59320) 2024-05-08T23:27:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fba7ac700 1 -- 172.21.15.5:0/4109070297 learned_addr learned my addr 172.21.15.5:0/4109070297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fbafad700 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577300b1400 0x557730142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.657+0000 7f9fb9fab700 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557730143af0 0x557730142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fba7ac700 1 -- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577300b1400 msgr2=0x557730142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fba7ac700 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577300b1400 0x557730142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fba7ac700 1 -- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557730143af0 msgr2=0x557730142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fba7ac700 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557730143af0 0x557730142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fba7ac700 1 -- 172.21.15.5:0/4109070297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55772ff26ed0 con 0x7f9fa40039d0 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fbafad700 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577300b1400 0x557730142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fba7ac700 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa40039d0 0x557730027ec0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00e350 tx=0x7f9fac00e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651e3d8b343d9b25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fb8fa9700 1 -- 172.21.15.5:0/4109070297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac016030 con 0x7f9fa40039d0 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fb8fa9700 1 -- 172.21.15.5:0/4109070297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac008070 con 0x7f9fa40039d0 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fb8fa9700 1 -- 172.21.15.5:0/4109070297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac001b80 con 0x7f9fa40039d0 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 -- 172.21.15.5:0/4109070297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa40039d0 msgr2=0x557730027ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa40039d0 0x557730027ec0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00e350 tx=0x7f9fac00e710 comp rx=0 tx=0).stop 2024-05-08T23:27:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 -- 172.21.15.5:0/4109070297 shutdown_connections 2024-05-08T23:27:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5577300b1400 0x557730142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa40039d0 0x557730027ec0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 --2- 172.21.15.5:0/4109070297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557730143af0 0x557730142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 -- 172.21.15.5:0/4109070297 >> 172.21.15.5:0/4109070297 conn(0x55772ffe2ba0 msgr2=0x557730026a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 -- 172.21.15.5:0/4109070297 shutdown_connections 2024-05-08T23:27:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.658+0000 7f9fc46da080 1 -- 172.21.15.5:0/4109070297 wait complete. 2024-05-08T23:27:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.659+0000 7f9fc46da080 1 Processor -- start 2024-05-08T23:27:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.659+0000 7f9fc46da080 1 -- start start 2024-05-08T23:27:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fc46da080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577300b1400 0x5577300b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fc46da080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557730143af0 0x557730011f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fc46da080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa40039d0 0x5577300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fc46da080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772ff24370 con 0x5577300b1400 2024-05-08T23:27:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fc46da080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772ff1b680 con 0x7f9fa40039d0 2024-05-08T23:27:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fc46da080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557730012470 con 0x557730143af0 2024-05-08T23:27:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fba7ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577300b1400 0x5577300b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fba7ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577300b1400 0x5577300b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59322/0 (socket says 172.21.15.5:59322) 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fba7ac700 1 -- 172.21.15.5:0/3574066896 learned_addr learned my addr 172.21.15.5:0/3574066896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fbafad700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa40039d0 0x5577300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fb9fab700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557730143af0 0x557730011f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fba7ac700 1 -- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557730143af0 msgr2=0x557730011f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fba7ac700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557730143af0 0x557730011f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fba7ac700 1 -- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa40039d0 msgr2=0x5577300a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fba7ac700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa40039d0 0x5577300a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.660+0000 7f9fba7ac700 1 -- 172.21.15.5:0/3574066896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac00a040 con 0x5577300b1400 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.661+0000 7f9fba7ac700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577300b1400 0x5577300b44f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00eb10 tx=0x7f9fac001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.661+0000 7f9fab7fe700 1 -- 172.21.15.5:0/3574066896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac011860 con 0x5577300b1400 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.661+0000 7f9fab7fe700 1 -- 172.21.15.5:0/3574066896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac0119e0 con 0x5577300b1400 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.661+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55772ff26930 con 0x5577300b1400 2024-05-08T23:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.661+0000 7f9fab7fe700 1 -- 172.21.15.5:0/3574066896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac011d90 con 0x5577300b1400 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.661+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577300afc10 con 0x5577300b1400 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.662+0000 7f9fab7fe700 1 -- 172.21.15.5:0/3574066896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9fac024070 con 0x5577300b1400 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.663+0000 7f9fab7fe700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f9c069610 0x7f9f9c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.663+0000 7f9fab7fe700 1 -- 172.21.15.5:0/3574066896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fac017070 con 0x5577300b1400 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.663+0000 7f9fb9fab700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f9c069610 0x7f9f9c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.663+0000 7f9fc46da080 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x5577300b00a0 0x5577300a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.663+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- osd_op(unknown.0.0:1 12.5 12:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) v8 -- 0x5577300aa350 con 0x5577300b00a0 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.663+0000 7f9fbafad700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x5577300b00a0 0x5577300a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.664+0000 7f9fb9fab700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f9c069610 0x7f9f9c06bad0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x5577300ad1e0 tx=0x7f9fb000c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.664+0000 7f9fbafad700 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x5577300b00a0 0x5577300a9e10 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fbafad700 1 -- 172.21.15.5:0/3574066896 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v84'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9fb40060d0 con 0x5577300b00a0 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x5577300b00a0 msgr2=0x5577300a9e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fc46da080 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x5577300b00a0 0x5577300a9e10 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f9c069610 msgr2=0x7f9f9c06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fc46da080 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f9c069610 0x7f9f9c06bad0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x5577300ad1e0 tx=0x7f9fb000c040 comp rx=0 tx=0).stop 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577300b1400 msgr2=0x5577300b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fc46da080 1 --2- 172.21.15.5:0/3574066896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577300b1400 0x5577300b44f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00eb10 tx=0x7f9fac001230 comp rx=0 tx=0).stop 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fbafad700 1 -- 172.21.15.5:0/3574066896 reap_dead start 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 shutdown_connections 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.665+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 >> 172.21.15.5:0/3574066896 conn(0x55772ffe2ba0 msgr2=0x55773002bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.666+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 shutdown_connections 2024-05-08T23:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.666+0000 7f9fc46da080 1 -- 172.21.15.5:0/3574066896 wait complete. 2024-05-08T23:27:24.670 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-05-08T23:27:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.720+0000 7f5aaf32a080 1 Processor -- start 2024-05-08T23:27:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.720+0000 7f5aaf32a080 1 -- start start 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aaf32a080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8897cbdd0 0x55f88992ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aaf32a080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f889814dc0 0x55f88989df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aaf32a080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f88989ee10 0x55f889811f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aaf32a080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8897711a0 con 0x55f8897cbdd0 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aaf32a080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88970c6d0 con 0x55f88989ee10 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aaf32a080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f88970c370 con 0x55f889814dc0 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8897cbdd0 0x55f88992ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8897cbdd0 0x55f88992ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59340/0 (socket says 172.21.15.5:59340) 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 -- 172.21.15.5:0/1495598319 learned_addr learned my addr 172.21.15.5:0/1495598319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa4bfb700 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f889814dc0 0x55f88989df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa5bfd700 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f88989ee10 0x55f889811f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 -- 172.21.15.5:0/1495598319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f889814dc0 msgr2=0x55f88989df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f889814dc0 0x55f88989df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 -- 172.21.15.5:0/1495598319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f88989ee10 msgr2=0x55f889811f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f88989ee10 0x55f889811f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 -- 172.21.15.5:0/1495598319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98005d20 con 0x55f8897cbdd0 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5aa53fc700 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8897cbdd0 0x55f88992ab40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800a440 tx=0x7f5a9800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96db3b154a9d7b00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5a977fe700 1 -- 172.21.15.5:0/1495598319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98008b60 con 0x55f8897cbdd0 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5a977fe700 1 -- 172.21.15.5:0/1495598319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a98008cc0 con 0x55f8897cbdd0 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.721+0000 7f5a977fe700 1 -- 172.21.15.5:0/1495598319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98012610 con 0x55f8897cbdd0 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 -- 172.21.15.5:0/1495598319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8897cbdd0 msgr2=0x55f88992ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8897cbdd0 0x55f88992ab40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800a440 tx=0x7f5a9800d000 comp rx=0 tx=0).stop 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 -- 172.21.15.5:0/1495598319 shutdown_connections 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f88989ee10 0x55f889811f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8897cbdd0 0x55f88992ab40 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 --2- 172.21.15.5:0/1495598319 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f889814dc0 0x55f88989df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 -- 172.21.15.5:0/1495598319 >> 172.21.15.5:0/1495598319 conn(0x55f8897caba0 msgr2=0x55f889813140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 -- 172.21.15.5:0/1495598319 shutdown_connections 2024-05-08T23:27:24.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.722+0000 7f5aaf32a080 1 -- 172.21.15.5:0/1495598319 wait complete. 2024-05-08T23:27:24.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.723+0000 7f5aaf32a080 1 Processor -- start 2024-05-08T23:27:24.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.723+0000 7f5aaf32a080 1 -- start start 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.723+0000 7f5aaf32a080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f8897cbdd0 0x55f88988e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aaf32a080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f889814dc0 0x55f889891820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aaf32a080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f88989ee10 0x55f889812180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aaf32a080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f889703680 con 0x55f889814dc0 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aaf32a080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8897cbc50 con 0x55f8897cbdd0 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aaf32a080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8897cb720 con 0x55f88989ee10 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f889814dc0 0x55f889891820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f889814dc0 0x55f889891820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59346/0 (socket says 172.21.15.5:59346) 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 -- 172.21.15.5:0/2509792391 learned_addr learned my addr 172.21.15.5:0/2509792391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa5bfd700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f88989ee10 0x55f889812180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa53fc700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f8897cbdd0 0x55f88988e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 -- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f88989ee10 msgr2=0x55f889812180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f88989ee10 0x55f889812180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 -- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f8897cbdd0 msgr2=0x55f88988e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f8897cbdd0 0x55f88988e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 -- 172.21.15.5:0/2509792391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f889735a40 con 0x55f889814dc0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aa4bfb700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f889814dc0 0x55f889891820 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c007e00 tx=0x7f5a9c006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5a967fc700 1 -- 172.21.15.5:0/2509792391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0133f0 con 0x55f889814dc0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5a967fc700 1 -- 172.21.15.5:0/2509792391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c00f040 con 0x55f889814dc0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f88971b8a0 con 0x55f889814dc0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.724+0000 7f5a967fc700 1 -- 172.21.15.5:0/2509792391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c013790 con 0x55f889814dc0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.725+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f88970eed0 con 0x55f889814dc0 2024-05-08T23:27:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.726+0000 7f5a967fc700 1 -- 172.21.15.5:0/2509792391 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5a9c0138f0 con 0x55f889814dc0 2024-05-08T23:27:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.726+0000 7f5a967fc700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a880695e0 0x7f5a8806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.726+0000 7f5a967fc700 1 -- 172.21.15.5:0/2509792391 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a9c086fa0 con 0x55f889814dc0 2024-05-08T23:27:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.727+0000 7f5aa53fc700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a880695e0 0x7f5a8806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.727+0000 7f5aaf32a080 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55f88988eac0 0x55f88988dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.727+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- osd_op(unknown.0.0:1 12.1d 12:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) v8 -- 0x55f8898126c0 con 0x55f88988eac0 2024-05-08T23:27:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.727+0000 7f5aa5bfd700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55f88988eac0 0x55f88988dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.727+0000 7f5aa53fc700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a880695e0 0x7f5a8806baa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800a8f0 tx=0x7f5a98027040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.727+0000 7f5aa5bfd700 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55f88988eac0 0x55f88988dd80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.728+0000 7f5aa5bfd700 1 -- 172.21.15.5:0/2509792391 <== osd.0 v2:172.21.15.5:6824/47134 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v84'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5aa0012330 con 0x55f88988eac0 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.728+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55f88988eac0 msgr2=0x55f88988dd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.728+0000 7f5aaf32a080 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55f88988eac0 0x55f88988dd80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a880695e0 msgr2=0x7f5a8806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aaf32a080 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a880695e0 0x7f5a8806baa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800a8f0 tx=0x7f5a98027040 comp rx=0 tx=0).stop 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f889814dc0 msgr2=0x55f889891820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aaf32a080 1 --2- 172.21.15.5:0/2509792391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f889814dc0 0x55f889891820 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c007e00 tx=0x7f5a9c006660 comp rx=0 tx=0).stop 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aa5bfd700 1 -- 172.21.15.5:0/2509792391 reap_dead start 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 shutdown_connections 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 >> 172.21.15.5:0/2509792391 conn(0x55f8897caba0 msgr2=0x55f88989e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 shutdown_connections 2024-05-08T23:27:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.729+0000 7f5aaf32a080 1 -- 172.21.15.5:0/2509792391 wait complete. 2024-05-08T23:27:24.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /d1/d2/d3 recursive,force 2024-05-08T23:27:24.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.875+0000 7f27000ca700 1 Processor -- start 2024-05-08T23:27:24.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.875+0000 7f27000ca700 1 -- start start 2024-05-08T23:27:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f27000ca700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26f815b8c0 0x7f26f80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f27000ca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26f80d1740 0x7f26f80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f27000ca700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f80cf560 con 0x7f26f815b4e0 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f27000ca700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f80cefe0 con 0x7f26f815b8c0 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f27000ca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80ccff0 con 0x7f26f80d1740 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fde66700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f26f815b4e0 0x7f26f80b1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34066/0 (socket says 172.21.15.5:34066) 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fde66700 1 -- 172.21.15.5:0/406185489 learned_addr learned my addr 172.21.15.5:0/406185489 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fd665700 1 --2- 172.21.15.5:0/406185489 >> v2:172.21.15.49:3300/0 conn(0x7f26f815b8c0 0x7f26f80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fe667700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26f80d1740 0x7f26f80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008534562 0 0) 0x7f26f80cf560 con 0x7f26f815b4e0 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e40035b0 con 0x7f26f815b4e0 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054046057 0 0) 0x7f26e40035b0 con 0x7f26f815b4e0 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26f80d1740 msgr2=0x7f26f80cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26f80d1740 0x7f26f80cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 >> v2:172.21.15.49:3300/0 conn(0x7f26f815b8c0 msgr2=0x7f26f80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 --2- 172.21.15.5:0/406185489 >> v2:172.21.15.49:3300/0 conn(0x7f26f815b8c0 0x7f26f80d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80cd2d0 con 0x7f26f815b4e0 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f26f4001670 con 0x7f26f815b4e0 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 >> v1:172.21.15.5:6789/0 conn(0x7f26f815b4e0 legacy=0x7f26f80b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.876+0000 7f26fce64700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a00 0x7f26e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fce64700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26f80d1740 0x7f26e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fce64700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e4004d90 0x7f26e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80cefe0 con 0x7f26e4003a00 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80ccff0 con 0x7f26e4004d90 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e400ba10 con 0x7f26f80d1740 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fde66700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a00 0x7f26e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fd665700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26f80d1740 0x7f26e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fe667700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e4004d90 0x7f26e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fde66700 1 -- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26f80d1740 msgr2=0x7f26e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fde66700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26f80d1740 0x7f26e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fde66700 1 -- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e4004d90 msgr2=0x7f26e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fde66700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e4004d90 0x7f26e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fde66700 1 -- 172.21.15.5:0/406185489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40035b0 con 0x7f26e4003a00 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fe667700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e4004d90 0x7f26e4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fde66700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a00 0x7f26e4003e20 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f26f400c3a0 tx=0x7f26e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76789a7b75eb9bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4007c60 con 0x7f26e4003a00 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f4005960 con 0x7f26e4003a00 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.877+0000 7f26fce64700 1 -- 172.21.15.5:0/406185489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f400cea0 con 0x7f26e4003a00 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.878+0000 7f27000ca700 1 -- 172.21.15.5:0/406185489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a00 msgr2=0x7f26e4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.878+0000 7f27000ca700 1 --2- 172.21.15.5:0/406185489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a00 0x7f26e4003e20 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f26f400c3a0 tx=0x7f26e4005880 comp rx=0 tx=0).stop 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.878+0000 7f26fe667700 1 -- 172.21.15.5:0/406185489 reap_dead start 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.878+0000 7f27000ca700 1 -- 172.21.15.5:0/406185489 shutdown_connections 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.878+0000 7f27000ca700 1 -- 172.21.15.5:0/406185489 >> 172.21.15.5:0/406185489 conn(0x7f26f808afe0 msgr2=0x7f26f808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.878+0000 7f27000ca700 1 -- 172.21.15.5:0/406185489 shutdown_connections 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.878+0000 7f27000ca700 1 -- 172.21.15.5:0/406185489 wait complete. 2024-05-08T23:27:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.879+0000 7f27000ca700 1 Processor -- start 2024-05-08T23:27:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.879+0000 7f27000ca700 1 -- start start 2024-05-08T23:27:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f27000ca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26e4003a00 0x7f26f815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f27000ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80b5c30 0x7f26f80cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f27000ca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26f80cb0a0 0x7f26f80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f27000ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80cb520 con 0x7f26f80b5c30 2024-05-08T23:27:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f27000ca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80b4b90 con 0x7f26f80cb0a0 2024-05-08T23:27:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f27000ca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80b4e70 con 0x7f26e4003a00 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f26fd665700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80b5c30 0x7f26f80cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f26fd665700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80b5c30 0x7f26f80cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59358/0 (socket says 172.21.15.5:59358) 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f26fd665700 1 -- 172.21.15.5:0/3240049283 learned_addr learned my addr 172.21.15.5:0/3240049283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f26fde66700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26e4003a00 0x7f26f815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.880+0000 7f26fe667700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26f80cb0a0 0x7f26f80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26fd665700 1 -- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26e4003a00 msgr2=0x7f26f815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26fd665700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26e4003a00 0x7f26f815e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26fd665700 1 -- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26f80cb0a0 msgr2=0x7f26f80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26fd665700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26f80cb0a0 0x7f26f80b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26fd665700 1 -- 172.21.15.5:0/3240049283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80b5150 con 0x7f26f80b5c30 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26fd665700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80b5c30 0x7f26f80cab60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f26e8011aa0 tx=0x7f26e8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26ee7fc700 1 -- 172.21.15.5:0/3240049283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800f940 con 0x7f26f80b5c30 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26ee7fc700 1 -- 172.21.15.5:0/3240049283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800faa0 con 0x7f26f80b5c30 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f27000ca700 1 -- 172.21.15.5:0/3240049283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f80b5430 con 0x7f26f80b5c30 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f26ee7fc700 1 -- 172.21.15.5:0/3240049283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800fe50 con 0x7f26f80b5c30 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.881+0000 7f27000ca700 1 -- 172.21.15.5:0/3240049283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f815e700 con 0x7f26f80b5c30 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.885+0000 7f270c284b80 1 Processor -- start 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.885+0000 7f270c284b80 1 -- start start 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f26ee7fc700 1 -- 172.21.15.5:0/3240049283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f26e8014440 con 0x7f26f80b5c30 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f270c284b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d8d61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f270c284b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d8db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f270c284b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d9052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f270c284b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480d8311e0 con 0x56480d8d8400 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f270c284b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480d82eba0 con 0x56480d8fd830 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f270c284b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56480d82a490 con 0x56480d904e60 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f26fde66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d8db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f26fde66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d8db130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59366/0 (socket says 172.21.15.5:59366) 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f26fde66700 1 -- 172.21.15.5:0/2447756877 learned_addr learned my addr 172.21.15.5:0/2447756877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f26fe667700 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d8d61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f26fd665700 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d9052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.886+0000 7f26ee7fc700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26dc06acc0 0x7f26dc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26ee7fc700 1 -- 172.21.15.5:0/3240049283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26e80898e0 con 0x7f26f80b5c30 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26fde66700 1 -- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 msgr2=0x56480d9052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26fde66700 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d9052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26fde66700 1 -- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 msgr2=0x56480d8d61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26fde66700 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d8d61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26fde66700 1 -- 172.21.15.5:0/2447756877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56480d6ab440 con 0x56480d8d8400 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26fd665700 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d9052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26fe667700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26dc06acc0 0x7f26dc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26fde66700 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d8db130 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f26f4015040 tx=0x7f26f400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b001ad75bc420631 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26db7fe700 1 -- 172.21.15.5:0/2447756877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f400cea0 con 0x56480d8d8400 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26db7fe700 1 -- 172.21.15.5:0/2447756877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f4001ad0 con 0x56480d8d8400 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.887+0000 7f26db7fe700 1 -- 172.21.15.5:0/2447756877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4001630 con 0x56480d8d8400 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 -- 172.21.15.5:0/2447756877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 msgr2=0x56480d8db130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f26fe667700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26dc06acc0 0x7f26dc06d180 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f26f80b3f30 tx=0x7f26f0004b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d8db130 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f26f4015040 tx=0x7f26f400f000 comp rx=0 tx=0).stop 2024-05-08T23:27:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 -- 172.21.15.5:0/2447756877 shutdown_connections 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d9052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d8d61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 --2- 172.21.15.5:0/2447756877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d8db130 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 -- 172.21.15.5:0/2447756877 >> 172.21.15.5:0/2447756877 conn(0x56480d8382a0 msgr2=0x56480d91fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 -- 172.21.15.5:0/2447756877 shutdown_connections 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.888+0000 7f270c284b80 1 -- 172.21.15.5:0/2447756877 wait complete. 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.889+0000 7f270c284b80 1 -- start start 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.889+0000 7f270c284b80 1 Processor -- start 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f270c284b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d92efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f270c284b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 0x56480d92fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f270c284b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d923570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f270c284b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480d7c65f0 con 0x56480d8d8400 2024-05-08T23:27:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f270c284b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480d7c6290 con 0x56480d904e60 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f270c284b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56480d6ad730 con 0x56480d8fd830 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f270c284b80 10 client.0 _ll_register_callbacks cb 0x56480d7cfbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d92efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d92efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59368/0 (socket says 172.21.15.5:59368) 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 -- 172.21.15.5:0/1044875855 learned_addr learned my addr 172.21.15.5:0/1044875855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fd665700 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 0x56480d92fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fe667700 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d923570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 -- 172.21.15.5:0/1044875855 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 msgr2=0x56480d92fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 0x56480d92fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 -- 172.21.15.5:0/1044875855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 msgr2=0x56480d923570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d923570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 -- 172.21.15.5:0/1044875855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e40035b0 con 0x56480d8d8400 2024-05-08T23:27:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26fde66700 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d92efd0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f26f4011000 tx=0x7f26f4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06bc8c1a6e6d52e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.890+0000 7f26d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1044875855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4007c00 con 0x56480d8d8400 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1044875855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26f4004870 con 0x56480d8d8400 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 1 -- 172.21.15.5:0/1044875855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56480d783300 con 0x56480d8d8400 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1044875855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f4004a30 con 0x56480d8d8400 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 10 client.8124 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 1 -- 172.21.15.5:0/1044875855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x56480d657bf0 con 0x56480d8d8400 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1044875855 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56480d783300 con 0x56480d8d8400 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1044875855 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f26f4025020 con 0x56480d8d8400 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 10 client.8124 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 10 client.8124 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 20 client.8124 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x56480d8fdc10 0x56480d9325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 4 client.8124 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f270c284b80 1 -- 172.21.15.5:0/1044875855 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56480d8a9870 con 0x56480d8fdc10 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.891+0000 7f26fd665700 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x56480d8fdc10 0x56480d9325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f26d8ff9700 10 client.8124 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f26fd665700 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x56480d8fdc10 0x56480d9325c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f881f410d57910f5 server_cookie=260bab1ef7f0fe49 in_seq=0 out_seq=0 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1044875855 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56480d8a9870 con 0x56480d8fdc10 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f26d8ff9700 10 client.8124 handle_command_reply: tid=0 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 client.8124 shutdown 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 -- 172.21.15.5:0/1044875855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 msgr2=0x56480d92efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d92efd0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f26f4011000 tx=0x7f26f4001a00 comp rx=0 tx=0).stop 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 -- 172.21.15.5:0/1044875855 shutdown_connections 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x56480d8fdc10 0x56480d9325c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d923570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d92efd0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f26d8ff9700 0 client.8124 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 --2- 172.21.15.5:0/1044875855 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 0x56480d92fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.892+0000 7f270c284b80 1 -- 172.21.15.5:0/1044875855 >> 172.21.15.5:0/1044875855 conn(0x56480d8382a0 msgr2=0x56480d9052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.893+0000 7f270c284b80 1 -- 172.21.15.5:0/1044875855 shutdown_connections 2024-05-08T23:27:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.893+0000 7f270c284b80 1 -- 172.21.15.5:0/1044875855 wait complete. 2024-05-08T23:27:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.893+0000 7f270c284b80 20 client.8124 trim_cache size 0 max 16384 2024-05-08T23:27:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.911+0000 7f270c284b80 1 Processor -- start 2024-05-08T23:27:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.911+0000 7f270c284b80 1 -- start start 2024-05-08T23:27:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f270c284b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f270c284b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d92edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f270c284b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d8d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f270c284b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480d8d16f0 con 0x56480d8d8400 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f270c284b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480d8d5850 con 0x56480d8fd830 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f270c284b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56480d907560 con 0x56480d904e60 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f26fde66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f26fde66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59370/0 (socket says 172.21.15.5:59370) 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f26fde66700 1 -- 172.21.15.5:0/1106024097 learned_addr learned my addr 172.21.15.5:0/1106024097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f26fe667700 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d8d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.912+0000 7f26fd665700 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d92edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26fde66700 1 -- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 msgr2=0x56480d8d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26fde66700 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d8d4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26fde66700 1 -- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 msgr2=0x56480d92edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26fde66700 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d92edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26fde66700 1 -- 172.21.15.5:0/1106024097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56480d8dc790 con 0x56480d8d8400 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26fe667700 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d8d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26fd665700 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d92edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26fde66700 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9078a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f26f4001a40 tx=0x7f26f402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7cc559e1117a35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26dbfff700 1 -- 172.21.15.5:0/1106024097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4004870 con 0x56480d8d8400 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26dbfff700 1 -- 172.21.15.5:0/1106024097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f40049d0 con 0x56480d8d8400 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f26dbfff700 1 -- 172.21.15.5:0/1106024097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40053c0 con 0x56480d8d8400 2024-05-08T23:27:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f270c284b80 1 -- 172.21.15.5:0/1106024097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 msgr2=0x56480d9078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.913+0000 7f270c284b80 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9078a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f26f4001a40 tx=0x7f26f402d070 comp rx=0 tx=0).stop 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 -- 172.21.15.5:0/1106024097 shutdown_connections 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d904e60 0x56480d8d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9078a0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 --2- 172.21.15.5:0/1106024097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d8fd830 0x56480d92edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 -- 172.21.15.5:0/1106024097 >> 172.21.15.5:0/1106024097 conn(0x56480d8382a0 msgr2=0x56480d8d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 -- 172.21.15.5:0/1106024097 shutdown_connections 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 -- 172.21.15.5:0/1106024097 wait complete. 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 -- start start 2024-05-08T23:27:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.914+0000 7f270c284b80 1 Processor -- start 2024-05-08T23:27:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f270c284b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f270c284b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 0x56480d9318d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f270c284b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d931e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f270c284b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480d917040 con 0x56480d8d8400 2024-05-08T23:27:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f270c284b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480d989800 con 0x56480d904e60 2024-05-08T23:27:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f270c284b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56480d907c10 con 0x56480d8fd830 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f270c284b80 10 client.0 _ll_register_callbacks cb 0x7f26f0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f26fde66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.915+0000 7f26fde66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9234d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59372/0 (socket says 172.21.15.5:59372) 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fde66700 1 -- 172.21.15.5:0/1793311742 learned_addr learned my addr 172.21.15.5:0/1793311742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fd665700 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 0x56480d9318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fe667700 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d931e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fde66700 1 -- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 msgr2=0x56480d9318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fde66700 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 0x56480d9318d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fde66700 1 -- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 msgr2=0x56480d931e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fde66700 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d931e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fde66700 1 -- 172.21.15.5:0/1793311742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56480d657bf0 con 0x56480d8d8400 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fe667700 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d931e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26fde66700 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9234d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004b60 tx=0x7f26f400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2d52e03685f6ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1793311742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4005540 con 0x56480d8d8400 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1793311742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26f40056a0 con 0x56480d8d8400 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f270c284b80 1 -- 172.21.15.5:0/1793311742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26f4001ad0 con 0x56480d8d8400 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1793311742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f4004be0 con 0x56480d8d8400 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f270c284b80 10 client.8130 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.916+0000 7f270c284b80 1 -- 172.21.15.5:0/1793311742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x56480d7f05f0 con 0x56480d8d8400 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1793311742 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f4001ad0 con 0x56480d8d8400 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1793311742 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f26f4025070 con 0x56480d8d8400 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f270c284b80 10 client.8130 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f270c284b80 10 client.8130 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f270c284b80 20 client.8130 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f270c284b80 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x56480d987ed0 0x56480d8d87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f270c284b80 4 client.8130 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f270c284b80 1 -- 172.21.15.5:0/1793311742 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) v1 -- 0x56480d822f50 con 0x56480d987ed0 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f26fd665700 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x56480d987ed0 0x56480d8d87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f26d8ff9700 10 client.8130 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.917+0000 7f26fd665700 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x56480d987ed0 0x56480d8d87e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=815586572679aa5b server_cookie=77ce6869411bfb6 in_seq=0 out_seq=0 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f26d8ff9700 1 -- 172.21.15.5:0/1793311742 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56480d822f50 con 0x56480d987ed0 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f26d8ff9700 10 client.8130 handle_command_reply: tid=0 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 client.8130 shutdown 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 -- 172.21.15.5:0/1793311742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 msgr2=0x56480d9234d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9234d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004b60 tx=0x7f26f400a000 comp rx=0 tx=0).stop 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 -- 172.21.15.5:0/1793311742 shutdown_connections 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x56480d987ed0 0x56480d8d87e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56480d904e60 0x56480d931e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480d8d8400 0x56480d9234d0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 --2- 172.21.15.5:0/1793311742 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56480d8fd830 0x56480d9318d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f26d8ff9700 0 client.8130 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 -- 172.21.15.5:0/1793311742 >> 172.21.15.5:0/1793311742 conn(0x56480d8382a0 msgr2=0x56480d8d4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.925 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:27:24.918+0000 7fb526b35700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-05-08T23:27:07.989312+0000 1=0+1) n(v3 rc2024-05-08T23:27:08.251306+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={6584=pAsLsXsFsx/pAsLsXsFsx@5},l=6584 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x564144076580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.989312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:08.251306+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.989312+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-05-08T23:27:08.251306+0000 6=0+6)","error_str":""},"return_code":-61} 2024-05-08T23:27:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 -- 172.21.15.5:0/1793311742 shutdown_connections 2024-05-08T23:27:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.918+0000 7f270c284b80 1 -- 172.21.15.5:0/1793311742 wait complete. 2024-05-08T23:27:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f270c284b80 20 client.8130 trim_cache size 0 max 16384 2024-05-08T23:27:24.926 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:24.926 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-08T23:27:24.926 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "4b4cf457-7a2d-427a-a3be-d52d0c5f729e", 2024-05-08T23:27:24.926 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-08T23:27:24.926 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:24.927 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:27:24.919+0000 7fb68e892700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1) n(v2 rc2024-05-08T23:27:08.251306+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6584=pAsLsXsFsx/pAsLsXsFsx@4},l=6584 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x561cc33d1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:08.251306+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-05-08T23:27:08.251306+0000 4=0+4)","error_str":""},"return_code":-61} 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 -- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26dc06acc0 msgr2=0x7f26dc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26dc06acc0 0x7f26dc06d180 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f26f80b3f30 tx=0x7f26f0004b40 comp rx=0 tx=0).stop 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 -- 172.21.15.5:0/3240049283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80b5c30 msgr2=0x7f26f80cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80b5c30 0x7f26f80cab60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f26e8011aa0 tx=0x7f26e8011db0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 -- 172.21.15.5:0/3240049283 shutdown_connections 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f26dc06acc0 0x7f26dc06d180 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26f80cb0a0 0x7f26f80b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f26e4003a00 0x7f26f815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 --2- 172.21.15.5:0/3240049283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80b5c30 0x7f26f80cab60 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 -- 172.21.15.5:0/3240049283 >> 172.21.15.5:0/3240049283 conn(0x7f26f808afe0 msgr2=0x7f26f80ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.919+0000 7f27000ca700 1 -- 172.21.15.5:0/3240049283 shutdown_connections 2024-05-08T23:27:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:24.920+0000 7f27000ca700 1 -- 172.21.15.5:0/3240049283 wait complete. 2024-05-08T23:27:24.929 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:27:24.920+0000 7fb52cb41700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-05-08T23:27:07.990312+0000 1=0+1) n(v4 rc2024-05-08T23:27:08.251306+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6584=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x564144076b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.990312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:08.251306+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.990312+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-05-08T23:27:08.251306+0000 5=0+5)","error_str":""},"return_code":-61} 2024-05-08T23:27:24.929 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:27:24.920+0000 7fb69489e700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1) n(v1 rc2024-05-08T23:27:08.251306+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6584=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x561cc24d4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:07.992312+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-05-08T23:27:08.251306+0000 3=0+3)","error_str":""},"return_code":-61} 2024-05-08T23:27:24.930 INFO:tasks.ceph.mds.b.smithi049.stderr:2024-05-08T23:27:24.919+0000 7f9eb1c5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-05-08T23:27:07.992312+0000 1=0+1) n(v0 rc2024-05-08T23:27:07.992312+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6584=pAsLsXsFsx/pAsLsXsFsx@4},l=6584 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x564fa03d6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.992312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:07.992312+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.992312+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-05-08T23:27:07.992312+0000 2=0+2)","error_str":""},"return_code":-61} 2024-05-08T23:27:24.930 INFO:tasks.ceph.mds.b.smithi049.stderr:2024-05-08T23:27:24.920+0000 7f9eb1c5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6584=pAsLsXsFsx/AsLsXsFsx@2},l=6584 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x564fa03d7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-05-08T23:27:24.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub pause 2024-05-08T23:27:25.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.070+0000 7fd675231700 1 Processor -- start 2024-05-08T23:27:25.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.070+0000 7fd675231700 1 -- start start 2024-05-08T23:27:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.071+0000 7fd675231700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd670154df0 0x7fd6700d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.071+0000 7fd675231700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6700aea70 0x7fd6700b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.071+0000 7fd675231700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67015add0 con 0x7fd6700d0890 2024-05-08T23:27:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.071+0000 7fd675231700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67015abe0 con 0x7fd6700aea70 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.071+0000 7fd675231700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700d0d10 con 0x7fd670154df0 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.071+0000 7fd66e59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd6700d0890 0x7fd6700b3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34070/0 (socket says 172.21.15.5:34070) 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.071+0000 7fd66f59e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6700aea70 0x7fd6700b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.071+0000 7fd66e59c700 1 -- 172.21.15.5:0/1775666082 learned_addr learned my addr 172.21.15.5:0/1775666082 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1775666082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd670154df0 0x7fd6700d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66dd9b700 1 -- 172.21.15.5:0/1775666082 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048193593 0 0) 0x7fd67015add0 con 0x7fd6700d0890 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66dd9b700 1 -- 172.21.15.5:0/1775666082 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd65c0057c0 con 0x7fd6700d0890 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66f59e700 1 -- 172.21.15.5:0/1775666082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd670154df0 msgr2=0x7fd6700d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1775666082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd670154df0 0x7fd6700d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66f59e700 1 -- 172.21.15.5:0/1775666082 >> v1:172.21.15.5:6789/0 conn(0x7fd6700d0890 legacy=0x7fd6700b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66f59e700 1 -- 172.21.15.5:0/1775666082 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67015b070 con 0x7fd6700aea70 2024-05-08T23:27:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1775666082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd670154df0 0x7fd6700d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1775666082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700aea70 0x7fd6700b4580 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd668009d90 tx=0x7fd66800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff5985ba9f6ea18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66dd9b700 1 -- 172.21.15.5:0/1775666082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800e070 con 0x7fd6700aea70 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66dd9b700 1 -- 172.21.15.5:0/1775666082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66800bb20 con 0x7fd6700aea70 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.072+0000 7fd66dd9b700 1 -- 172.21.15.5:0/1775666082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668012430 con 0x7fd6700aea70 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.073+0000 7fd675231700 1 -- 172.21.15.5:0/1775666082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700aea70 msgr2=0x7fd6700b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.073+0000 7fd675231700 1 --2- 172.21.15.5:0/1775666082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700aea70 0x7fd6700b4580 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd668009d90 tx=0x7fd66800b330 comp rx=0 tx=0).stop 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.073+0000 7fd675231700 1 -- 172.21.15.5:0/1775666082 shutdown_connections 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.073+0000 7fd675231700 1 --2- 172.21.15.5:0/1775666082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6700aea70 0x7fd6700b4580 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.073+0000 7fd675231700 1 --2- 172.21.15.5:0/1775666082 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd670154df0 0x7fd6700d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.073+0000 7fd675231700 1 -- 172.21.15.5:0/1775666082 >> 172.21.15.5:0/1775666082 conn(0x7fd670099a50 msgr2=0x7fd6700b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.073+0000 7fd675231700 1 -- 172.21.15.5:0/1775666082 shutdown_connections 2024-05-08T23:27:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.073+0000 7fd675231700 1 -- 172.21.15.5:0/1775666082 wait complete. 2024-05-08T23:27:25.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.074+0000 7fd675231700 1 Processor -- start 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.074+0000 7fd675231700 1 -- start start 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd675231700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6700aea70 0x7fd6700aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd675231700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0890 0x7fd670145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd675231700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670154df0 0x7fd6700b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd675231700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700af750 con 0x7fd670154df0 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd675231700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700afa30 con 0x7fd6700aea70 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd675231700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700afd10 con 0x7fd6700d0890 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd66f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670154df0 0x7fd6700b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd66f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670154df0 0x7fd6700b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59392/0 (socket says 172.21.15.5:59392) 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.075+0000 7fd66f59e700 1 -- 172.21.15.5:0/1883400959 learned_addr learned my addr 172.21.15.5:0/1883400959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd66e59c700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0890 0x7fd670145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6700aea70 0x7fd6700aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd66f59e700 1 -- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0890 msgr2=0x7fd670145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0890 0x7fd670145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd66f59e700 1 -- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6700aea70 msgr2=0x7fd6700aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6700aea70 0x7fd6700aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd66f59e700 1 -- 172.21.15.5:0/1883400959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668009a70 con 0x7fd670154df0 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670154df0 0x7fd6700b07f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd668014040 tx=0x7fd668004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd6677fe700 1 -- 172.21.15.5:0/1883400959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668012430 con 0x7fd670154df0 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd6677fe700 1 -- 172.21.15.5:0/1883400959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd668020070 con 0x7fd670154df0 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd675231700 1 -- 172.21.15.5:0/1883400959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6700afff0 con 0x7fd670154df0 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd6677fe700 1 -- 172.21.15.5:0/1883400959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668003ea0 con 0x7fd670154df0 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.076+0000 7fd675231700 1 -- 172.21.15.5:0/1883400959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6700af390 con 0x7fd670154df0 2024-05-08T23:27:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.078+0000 7fd6677fe700 1 -- 172.21.15.5:0/1883400959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd668004040 con 0x7fd670154df0 2024-05-08T23:27:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.078+0000 7fd6677fe700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.078+0000 7fd6677fe700 1 -- 172.21.15.5:0/1883400959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd668022070 con 0x7fd670154df0 2024-05-08T23:27:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.078+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.079+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c940 tx=0x7fd660008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.081+0000 7fd6813ebb80 1 Processor -- start 2024-05-08T23:27:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.081+0000 7fd6813ebb80 1 -- start start 2024-05-08T23:27:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 0x5608a2ef6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2efb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f25240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a2e51140 con 0x5608a2ef8360 2024-05-08T23:27:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a2e4eb00 con 0x5608a2f1d790 2024-05-08T23:27:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608a2e4a3f0 con 0x5608a2f24dc0 2024-05-08T23:27:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2efb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2efb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59406/0 (socket says 172.21.15.5:59406) 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66e59c700 1 -- 172.21.15.5:0/1435666813 learned_addr learned my addr 172.21.15.5:0/1435666813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 0x5608a2ef6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f25240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66e59c700 1 -- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 msgr2=0x5608a2f25240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66e59c700 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f25240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66e59c700 1 -- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 msgr2=0x5608a2ef6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66e59c700 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 0x5608a2ef6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66e59c700 1 -- 172.21.15.5:0/1435666813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a2ccb3a0 con 0x5608a2ef8360 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.082+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f25240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.083+0000 7fd66e59c700 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2efb090 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd658006db0 tx=0x7fd658007320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b2ed614b323c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.083+0000 7fd664ff9700 1 -- 172.21.15.5:0/1435666813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658006e40 con 0x5608a2ef8360 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.083+0000 7fd664ff9700 1 -- 172.21.15.5:0/1435666813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658001580 con 0x5608a2ef8360 2024-05-08T23:27:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.083+0000 7fd664ff9700 1 -- 172.21.15.5:0/1435666813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658001910 con 0x5608a2ef8360 2024-05-08T23:27:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.083+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1435666813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 msgr2=0x5608a2efb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.083+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2efb090 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd658006db0 tx=0x7fd658007320 comp rx=0 tx=0).stop 2024-05-08T23:27:25.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1435666813 shutdown_connections 2024-05-08T23:27:25.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f25240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 0x5608a2ef6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1435666813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2efb090 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1435666813 >> 172.21.15.5:0/1435666813 conn(0x5608a2e58200 msgr2=0x5608a2f3fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1435666813 shutdown_connections 2024-05-08T23:27:25.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1435666813 wait complete. 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 -- start start 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.084+0000 7fd6813ebb80 1 Processor -- start 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f1bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 0x5608a2f42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f1e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a2de6550 con 0x5608a2ef8360 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a2de61f0 con 0x5608a2f1d790 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608a2ccd690 con 0x5608a2f24dc0 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd6813ebb80 10 client.0 _ll_register_callbacks cb 0x5608a2defb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd66ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f1bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd66ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f1bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59420/0 (socket says 172.21.15.5:59420) 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd66ed9d700 1 -- 172.21.15.5:0/3151435877 learned_addr learned my addr 172.21.15.5:0/3151435877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd66f59e700 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f1e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.085+0000 7fd66e59c700 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 0x5608a2f42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd66ed9d700 1 -- 172.21.15.5:0/3151435877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 msgr2=0x5608a2f1e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f1e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd66ed9d700 1 -- 172.21.15.5:0/3151435877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 msgr2=0x5608a2f42830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 0x5608a2f42830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd66ed9d700 1 -- 172.21.15.5:0/3151435877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c0057c0 con 0x5608a2ef8360 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f1bfb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd660015440 tx=0x7fd660004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d533dc5f2d99eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd64dffb700 1 -- 172.21.15.5:0/3151435877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660023480 con 0x5608a2ef8360 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd64dffb700 1 -- 172.21.15.5:0/3151435877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd66001c040 con 0x5608a2ef8360 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd6813ebb80 1 -- 172.21.15.5:0/3151435877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608a2da3260 con 0x5608a2ef8360 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd64dffb700 1 -- 172.21.15.5:0/3151435877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6600235e0 con 0x5608a2ef8360 2024-05-08T23:27:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd6813ebb80 10 client.8145 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd6813ebb80 1 -- 172.21.15.5:0/3151435877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5608a2c77b50 con 0x5608a2ef8360 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd64dffb700 1 -- 172.21.15.5:0/3151435877 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5608a2da3260 con 0x5608a2ef8360 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd64dffb700 1 -- 172.21.15.5:0/3151435877 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fd660023740 con 0x5608a2ef8360 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd6813ebb80 10 client.8145 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd6813ebb80 10 client.8145 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd6813ebb80 20 client.8145 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.086+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5608a2f1eb90 0x5608a2f3f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.087+0000 7fd6813ebb80 4 client.8145 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.087+0000 7fd6813ebb80 1 -- 172.21.15.5:0/3151435877 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608a2ec97d0 con 0x5608a2f1eb90 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.087+0000 7fd66e59c700 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5608a2f1eb90 0x5608a2f3f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.087+0000 7fd64dffb700 10 client.8145 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.087+0000 7fd66e59c700 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5608a2f1eb90 0x5608a2f3f400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c64b7805f047d7b server_cookie=d60bd48b6d323edf in_seq=0 out_seq=0 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.087+0000 7fd64dffb700 1 -- 172.21.15.5:0/3151435877 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608a2ec97d0 con 0x5608a2f1eb90 2024-05-08T23:27:25.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.087+0000 7fd64dffb700 10 client.8145 handle_command_reply: tid=0 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 client.8145 shutdown 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 -- 172.21.15.5:0/3151435877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 msgr2=0x5608a2f1bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f1bfb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd660015440 tx=0x7fd660004750 comp rx=0 tx=0).stop 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 -- 172.21.15.5:0/3151435877 shutdown_connections 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5608a2f1eb90 0x5608a2f3f400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f24dc0 0x5608a2f1e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f1bfb0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/3151435877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f1d790 0x5608a2f42830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd64dffb700 0 client.8145 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 -- 172.21.15.5:0/3151435877 >> 172.21.15.5:0/3151435877 conn(0x5608a2e58200 msgr2=0x5608a2efa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 -- 172.21.15.5:0/3151435877 shutdown_connections 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 1 -- 172.21.15.5:0/3151435877 wait complete. 2024-05-08T23:27:25.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.088+0000 7fd6813ebb80 20 client.8145 trim_cache size 0 max 16384 2024-05-08T23:27:25.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.106+0000 7fd6813ebb80 1 Processor -- start 2024-05-08T23:27:25.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.106+0000 7fd6813ebb80 1 -- start start 2024-05-08T23:27:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f40710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 0x5608a2f222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f24dc0 0x5608a2ef3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a2ef51b0 con 0x5608a2ef8360 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a2ef6bb0 con 0x5608a2f24dc0 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608a2f22060 con 0x5608a2f1d790 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f40710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f40710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59422/0 (socket says 172.21.15.5:59422) 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66ed9d700 1 -- 172.21.15.5:0/4141067567 learned_addr learned my addr 172.21.15.5:0/4141067567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66e59c700 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 0x5608a2f222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66f59e700 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f24dc0 0x5608a2ef3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66ed9d700 1 -- 172.21.15.5:0/4141067567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 msgr2=0x5608a2f222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 0x5608a2f222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66ed9d700 1 -- 172.21.15.5:0/4141067567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f24dc0 msgr2=0x5608a2ef3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f24dc0 0x5608a2ef3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.107+0000 7fd66ed9d700 1 -- 172.21.15.5:0/4141067567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a2e10550 con 0x5608a2ef8360 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f40710 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd660004770 tx=0x7fd66001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19906cb44b92ec0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6657fa700 1 -- 172.21.15.5:0/4141067567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660023c00 con 0x5608a2ef8360 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6657fa700 1 -- 172.21.15.5:0/4141067567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660023d60 con 0x5608a2ef8360 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6657fa700 1 -- 172.21.15.5:0/4141067567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660023680 con 0x5608a2ef8360 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6813ebb80 1 -- 172.21.15.5:0/4141067567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 msgr2=0x5608a2f40710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f40710 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd660004770 tx=0x7fd66001b000 comp rx=0 tx=0).stop 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6813ebb80 1 -- 172.21.15.5:0/4141067567 shutdown_connections 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2f24dc0 0x5608a2ef3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2ef8360 0x5608a2f40710 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/4141067567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 0x5608a2f222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.108+0000 7fd6813ebb80 1 -- 172.21.15.5:0/4141067567 >> 172.21.15.5:0/4141067567 conn(0x5608a2e58200 msgr2=0x5608a2f1bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.109+0000 7fd6813ebb80 1 -- 172.21.15.5:0/4141067567 shutdown_connections 2024-05-08T23:27:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.109+0000 7fd6813ebb80 1 -- 172.21.15.5:0/4141067567 wait complete. 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.109+0000 7fd6813ebb80 1 -- start start 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.109+0000 7fd6813ebb80 1 Processor -- start 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2ef8360 0x5608a2fa7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 0x5608a2f27ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd6813ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2f24dc0 0x5608a2f1aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a2f28c40 con 0x5608a2f24dc0 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a2efb2b0 con 0x5608a2ef8360 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd6813ebb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608a2f289f0 con 0x5608a2f1d790 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd6813ebb80 10 client.0 _ll_register_callbacks cb 0x7fd660011bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2f24dc0 0x5608a2f1aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2f24dc0 0x5608a2f1aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59428/0 (socket says 172.21.15.5:59428) 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 -- 172.21.15.5:0/1847603704 learned_addr learned my addr 172.21.15.5:0/1847603704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2ef8360 0x5608a2fa7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66e59c700 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 0x5608a2f27ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 -- 172.21.15.5:0/1847603704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 msgr2=0x5608a2f27ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 0x5608a2f27ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 -- 172.21.15.5:0/1847603704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2ef8360 msgr2=0x5608a2fa7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2ef8360 0x5608a2fa7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 -- 172.21.15.5:0/1847603704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608a2c77b50 con 0x5608a2f24dc0 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd66f59e700 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2f24dc0 0x5608a2f1aa30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b680 tx=0x7fd66805a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9673dab3c669b9e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd64dffb700 1 -- 172.21.15.5:0/1847603704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668061f90 con 0x5608a2f24dc0 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd64dffb700 1 -- 172.21.15.5:0/1847603704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd66805ae40 con 0x5608a2f24dc0 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.111+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1847603704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608a2e42eb0 con 0x5608a2f24dc0 2024-05-08T23:27:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd64dffb700 1 -- 172.21.15.5:0/1847603704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd668077250 con 0x5608a2f24dc0 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd6813ebb80 10 client.8151 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1847603704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5608a2ccfcc0 con 0x5608a2f24dc0 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd64dffb700 1 -- 172.21.15.5:0/1847603704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5608a2e42eb0 con 0x5608a2f24dc0 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd64dffb700 1 -- 172.21.15.5:0/1847603704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fd66805eb80 con 0x5608a2f24dc0 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd6813ebb80 10 client.8151 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd6813ebb80 10 client.8151 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd6813ebb80 20 client.8151 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5608a2f284b0 0x5608a2f1af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd6813ebb80 4 client.8151 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1847603704 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5608a2c98ce0 con 0x5608a2f284b0 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5608a2f284b0 0x5608a2f1af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd66ed9d700 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5608a2f284b0 0x5608a2f1af70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a156a373f8b20c91 server_cookie=27dfc9dd8afd5eba in_seq=0 out_seq=0 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.112+0000 7fd64dffb700 10 client.8151 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd64dffb700 1 -- 172.21.15.5:0/1847603704 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5608a2c98ce0 con 0x5608a2f284b0 2024-05-08T23:27:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd64dffb700 10 client.8151 handle_command_reply: tid=0 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 client.8151 shutdown 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1847603704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2f24dc0 msgr2=0x5608a2f1aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2f24dc0 0x5608a2f1aa30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b680 tx=0x7fd66805a370 comp rx=0 tx=0).stop 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1847603704 shutdown_connections 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5608a2f284b0 0x5608a2f1af70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608a2f24dc0 0x5608a2f1aa30 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5608a2ef8360 0x5608a2fa7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd64dffb700 0 client.8151 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 --2- 172.21.15.5:0/1847603704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5608a2f1d790 0x5608a2f27ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1847603704 >> 172.21.15.5:0/1847603704 conn(0x5608a2e58200 msgr2=0x5608a2f1bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1847603704 shutdown_connections 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 1 -- 172.21.15.5:0/1847603704 wait complete. 2024-05-08T23:27:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.113+0000 7fd6813ebb80 20 client.8151 trim_cache size 0 max 16384 2024-05-08T23:27:25.119 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:27:25.119 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 -- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 msgr2=0x7fd65006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c940 tx=0x7fd660008040 comp rx=0 tx=0).stop 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 -- 172.21.15.5:0/1883400959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670154df0 msgr2=0x7fd6700b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670154df0 0x7fd6700b07f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd668014040 tx=0x7fd668004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 -- 172.21.15.5:0/1883400959 shutdown_connections 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd65006ac90 0x7fd65006d150 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670154df0 0x7fd6700b07f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6700aea70 0x7fd6700aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 --2- 172.21.15.5:0/1883400959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6700d0890 0x7fd670145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 -- 172.21.15.5:0/1883400959 >> 172.21.15.5:0/1883400959 conn(0x7fd670099a50 msgr2=0x7fd67006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 -- 172.21.15.5:0/1883400959 shutdown_connections 2024-05-08T23:27:25.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.114+0000 7fd675231700 1 -- 172.21.15.5:0/1883400959 wait complete. 2024-05-08T23:27:25.127 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 scrub status 2024-05-08T23:27:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.265+0000 7f6bab7eb700 1 Processor -- start 2024-05-08T23:27:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.266+0000 7f6bab7eb700 1 -- start start 2024-05-08T23:27:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.266+0000 7f6bab7eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba4099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.266+0000 7f6bab7eb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6ba4099f90 0x7f6ba40cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.266+0000 7f6bab7eb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba406d010 con 0x7f6ba415da60 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.266+0000 7f6bab7eb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba409a420 con 0x7f6ba4099f90 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.266+0000 7f6bab7eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba415de40 con 0x7f6ba409a790 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba9d88700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6ba4099f90 0x7f6ba40cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba8d86700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba4099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba9587700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6ba415da60 0x7f6ba40d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:53400/0 (socket says 172.21.15.5:53400) 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba9587700 1 -- 172.21.15.5:0/1284405894 learned_addr learned my addr 172.21.15.5:0/1284405894 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba9d88700 1 -- 172.21.15.5:0/1284405894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 msgr2=0x7f6ba4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/1284405894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba4099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba9d88700 1 -- 172.21.15.5:0/1284405894 >> v1:172.21.15.49:6789/0 conn(0x7f6ba415da60 legacy=0x7f6ba40d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba9d88700 1 -- 172.21.15.5:0/1284405894 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40d0b20 con 0x7f6ba4099f90 2024-05-08T23:27:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6b9bfff700 1 -- 172.21.15.5:0/1284405894 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138804528 0 0) 0x7f6ba406d010 con 0x7f6ba415da60 2024-05-08T23:27:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.267+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/1284405894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba4099f90 0x7f6ba40cc720 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009d90 tx=0x7f6b9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e43877a4ddc4d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6b9bfff700 1 -- 172.21.15.5:0/1284405894 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00e070 con 0x7f6ba4099f90 2024-05-08T23:27:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6b9bfff700 1 -- 172.21.15.5:0/1284405894 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c00bb20 con 0x7f6ba4099f90 2024-05-08T23:27:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6b9bfff700 1 -- 172.21.15.5:0/1284405894 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c012430 con 0x7f6ba4099f90 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6bab7eb700 1 -- 172.21.15.5:0/1284405894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba4099f90 msgr2=0x7f6ba40cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/1284405894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba4099f90 0x7f6ba40cc720 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009d90 tx=0x7f6b9c00b330 comp rx=0 tx=0).stop 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6bab7eb700 1 -- 172.21.15.5:0/1284405894 shutdown_connections 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/1284405894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba4099f90 0x7f6ba40cc720 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/1284405894 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba4099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6bab7eb700 1 -- 172.21.15.5:0/1284405894 >> 172.21.15.5:0/1284405894 conn(0x7f6ba40b2f60 msgr2=0x7f6ba40b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6bab7eb700 1 -- 172.21.15.5:0/1284405894 shutdown_connections 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.268+0000 7f6bab7eb700 1 -- 172.21.15.5:0/1284405894 wait complete. 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.270+0000 7f6bab7eb700 1 Processor -- start 2024-05-08T23:27:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.270+0000 7f6bab7eb700 1 -- start start 2024-05-08T23:27:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.270+0000 7f6bab7eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4099f90 0x7f6ba40c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.270+0000 7f6bab7eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba40b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.270+0000 7f6bab7eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba415da60 0x7f6ba41f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.270+0000 7f6bab7eb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40b4770 con 0x7f6ba4099f90 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.270+0000 7f6bab7eb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40b68b0 con 0x7f6ba415da60 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.270+0000 7f6bab7eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40b6b90 con 0x7f6ba409a790 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4099f90 0x7f6ba40c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba8d86700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba40b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4099f90 0x7f6ba40c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59432/0 (socket says 172.21.15.5:59432) 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 -- 172.21.15.5:0/2214033874 learned_addr learned my addr 172.21.15.5:0/2214033874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba415da60 0x7f6ba41f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 -- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 msgr2=0x7f6ba40b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba40b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 -- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba415da60 msgr2=0x7f6ba41f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba415da60 0x7f6ba41f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 -- 172.21.15.5:0/2214033874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c009a70 con 0x7f6ba4099f90 2024-05-08T23:27:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba40b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6ba9587700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4099f90 0x7f6ba40c5160 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c550 tx=0x7f6ba000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6b99ffb700 1 -- 172.21.15.5:0/2214033874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0004c80 con 0x7f6ba4099f90 2024-05-08T23:27:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6bab7eb700 1 -- 172.21.15.5:0/2214033874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba40b6e70 con 0x7f6ba4099f90 2024-05-08T23:27:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6b99ffb700 1 -- 172.21.15.5:0/2214033874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0004e00 con 0x7f6ba4099f90 2024-05-08T23:27:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.271+0000 7f6b99ffb700 1 -- 172.21.15.5:0/2214033874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0007310 con 0x7f6ba4099f90 2024-05-08T23:27:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.272+0000 7f6bab7eb700 1 -- 172.21.15.5:0/2214033874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba41f7ab0 con 0x7f6ba4099f90 2024-05-08T23:27:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.273+0000 7f6b99ffb700 1 -- 172.21.15.5:0/2214033874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6ba0007470 con 0x7f6ba4099f90 2024-05-08T23:27:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.273+0000 7f6b99ffb700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b8806ac90 0x7f6b8806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.274+0000 7f6b99ffb700 1 -- 172.21.15.5:0/2214033874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ba0088e90 con 0x7f6ba4099f90 2024-05-08T23:27:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.274+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b8806ac90 0x7f6b8806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.274+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b8806ac90 0x7f6b8806d150 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6ba414b770 tx=0x7f6b94009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.276+0000 7f6bb79a5b80 1 Processor -- start 2024-05-08T23:27:25.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.276+0000 7f6bb79a5b80 1 -- start start 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a337a0 0x55b227a0c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 0x55b227a110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227a3b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227967150 con 0x55b227a337a0 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227964b10 con 0x55b227a3add0 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b227960400 con 0x55b227a0e370 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9587700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a337a0 0x55b227a0c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9587700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a337a0 0x55b227a0c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59440/0 (socket says 172.21.15.5:59440) 2024-05-08T23:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9587700 1 -- 172.21.15.5:0/1436932907 learned_addr learned my addr 172.21.15.5:0/1436932907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 0x55b227a110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227a3b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9587700 1 -- 172.21.15.5:0/1436932907 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 msgr2=0x55b227a110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9587700 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 0x55b227a110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9587700 1 -- 172.21.15.5:0/1436932907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 msgr2=0x55b227a3b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9587700 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227a3b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.277+0000 7f6ba9587700 1 -- 172.21.15.5:0/1436932907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2277e13b0 con 0x55b227a337a0 2024-05-08T23:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.278+0000 7f6ba9587700 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a337a0 0x55b227a0c110 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f6ba005fc00 tx=0x7f6ba0001280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4820fa9339c2f813 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.279+0000 7f6b8effd700 1 -- 172.21.15.5:0/1436932907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0057d40 con 0x55b227a337a0 2024-05-08T23:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.279+0000 7f6b8effd700 1 -- 172.21.15.5:0/1436932907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba005bb70 con 0x55b227a337a0 2024-05-08T23:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.279+0000 7f6b8effd700 1 -- 172.21.15.5:0/1436932907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0060130 con 0x55b227a337a0 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/1436932907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a337a0 msgr2=0x55b227a0c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a337a0 0x55b227a0c110 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f6ba005fc00 tx=0x7f6ba0001280 comp rx=0 tx=0).stop 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/1436932907 shutdown_connections 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227a3b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a337a0 0x55b227a0c110 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/1436932907 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 0x55b227a110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/1436932907 >> 172.21.15.5:0/1436932907 conn(0x55b22796e210 msgr2=0x55b227a55dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/1436932907 shutdown_connections 2024-05-08T23:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.280+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/1436932907 wait complete. 2024-05-08T23:27:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.281+0000 7f6bb79a5b80 1 -- start start 2024-05-08T23:27:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.281+0000 7f6bb79a5b80 1 Processor -- start 2024-05-08T23:27:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 0x55b227a3ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 0x55b227a57720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a3add0 0x55b227a64540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2278fc560 con 0x55b227a3add0 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2278fc200 con 0x55b227a337a0 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2277e36a0 con 0x55b227a0e370 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6bb79a5b80 10 client.0 _ll_register_callbacks cb 0x55b227905b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6ba9d88700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a3add0 0x55b227a64540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6ba9d88700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a3add0 0x55b227a64540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59446/0 (socket says 172.21.15.5:59446) 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6ba9d88700 1 -- 172.21.15.5:0/4079284251 learned_addr learned my addr 172.21.15.5:0/4079284251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6ba9587700 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 0x55b227a3ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.282+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 0x55b227a57720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6ba9d88700 1 -- 172.21.15.5:0/4079284251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 msgr2=0x55b227a3ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 0x55b227a3ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6ba9d88700 1 -- 172.21.15.5:0/4079284251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 msgr2=0x55b227a57720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 0x55b227a57720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6ba9d88700 1 -- 172.21.15.5:0/4079284251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba41f7ab0 con 0x55b227a3add0 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a3add0 0x55b227a64540 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c013000 tx=0x7f6b9c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc2c297cfe38a42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6b83fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6b83fff700 1 -- 172.21.15.5:0/4079284251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c022070 con 0x55b227a3add0 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6b83fff700 1 -- 172.21.15.5:0/4079284251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b9c012aa0 con 0x55b227a3add0 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/4079284251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2278b9270 con 0x55b227a3add0 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6b83fff700 1 -- 172.21.15.5:0/4079284251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b9c012c60 con 0x55b227a3add0 2024-05-08T23:27:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6bb79a5b80 10 client.8160 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/4079284251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55b22778db60 con 0x55b227a3add0 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.283+0000 7f6b83fff700 1 -- 172.21.15.5:0/4079284251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b2278b9270 con 0x55b227a3add0 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6b83fff700 1 -- 172.21.15.5:0/4079284251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f6b9c004040 con 0x55b227a3add0 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6bb79a5b80 10 client.8160 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6bb79a5b80 10 client.8160 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6bb79a5b80 20 client.8160 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55b227a64c00 0x55b227a22660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6bb79a5b80 4 client.8160 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/4079284251 --> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2279df7e0 con 0x55b227a64c00 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6ba9587700 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55b227a64c00 0x55b227a22660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6ba9587700 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55b227a64c00 0x55b227a22660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa6fba07f0809a54 server_cookie=11b8deb9b969e399 in_seq=0 out_seq=0 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.284+0000 7f6b83fff700 10 client.8160 ms_handle_connect on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6b83fff700 1 -- 172.21.15.5:0/4079284251 <== mds.1 v2:172.21.15.5:6832/3718289875 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2279df7e0 con 0x55b227a64c00 2024-05-08T23:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6b83fff700 10 client.8160 handle_command_reply: tid=0 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 client.8160 shutdown 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/4079284251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a3add0 msgr2=0x55b227a64540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a3add0 0x55b227a64540 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c013000 tx=0x7f6b9c004b40 comp rx=0 tx=0).stop 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/4079284251 shutdown_connections 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55b227a64c00 0x55b227a22660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a3add0 0x55b227a64540 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a0e370 0x55b227a3ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/4079284251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 0x55b227a57720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6b83fff700 0 client.8160 ms_handle_reset on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/4079284251 >> 172.21.15.5:0/4079284251 conn(0x55b22796e210 msgr2=0x55b227a56070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/4079284251 shutdown_connections 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/4079284251 wait complete. 2024-05-08T23:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.285+0000 7f6bb79a5b80 20 client.8160 trim_cache size 0 max 16384 2024-05-08T23:27:25.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.303+0000 7f6bb79a5b80 1 Processor -- start 2024-05-08T23:27:25.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.303+0000 7f6bb79a5b80 1 -- start start 2024-05-08T23:27:25.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227a55ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 0x55b227a11340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a3add0 0x55b227a78fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227a0cc20 con 0x55b227a0e370 2024-05-08T23:27:25.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227a0db70 con 0x55b227a337a0 2024-05-08T23:27:25.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b227a59c30 con 0x55b227a3add0 2024-05-08T23:27:25.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6ba9587700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227a55ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6ba9587700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227a55ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59462/0 (socket says 172.21.15.5:59462) 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.304+0000 7f6ba9587700 1 -- 172.21.15.5:0/3152291165 learned_addr learned my addr 172.21.15.5:0/3152291165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a3add0 0x55b227a78fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 0x55b227a11340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6ba9587700 1 -- 172.21.15.5:0/3152291165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a3add0 msgr2=0x55b227a78fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6ba9587700 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a3add0 0x55b227a78fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6ba9587700 1 -- 172.21.15.5:0/3152291165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 msgr2=0x55b227a11340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6ba9587700 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 0x55b227a11340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6ba9587700 1 -- 172.21.15.5:0/3152291165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b227926560 con 0x55b227a0e370 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6ba9587700 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227a55ac0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0057cd0 tx=0x7f6ba005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d51229d7e2c0058 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6b8f7fe700 1 -- 172.21.15.5:0/3152291165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0057d40 con 0x55b227a0e370 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6b8f7fe700 1 -- 172.21.15.5:0/3152291165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0060130 con 0x55b227a0e370 2024-05-08T23:27:25.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.305+0000 7f6b8f7fe700 1 -- 172.21.15.5:0/3152291165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00560e0 con 0x55b227a0e370 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/3152291165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 msgr2=0x55b227a55ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227a55ac0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0057cd0 tx=0x7f6ba005a000 comp rx=0 tx=0).stop 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/3152291165 shutdown_connections 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a3add0 0x55b227a78fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227a55ac0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/3152291165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a337a0 0x55b227a11340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/3152291165 >> 172.21.15.5:0/3152291165 conn(0x55b22796e210 msgr2=0x55b227a56390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/3152291165 shutdown_connections 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/3152291165 wait complete. 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 -- start start 2024-05-08T23:27:25.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.306+0000 7f6bb79a5b80 1 Processor -- start 2024-05-08T23:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227abe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a337a0 0x55b227a09ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6bb79a5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227ac1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227ac20a0 con 0x55b227a0e370 2024-05-08T23:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227ac24d0 con 0x55b227a3add0 2024-05-08T23:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6bb79a5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b227a32ea0 con 0x55b227a337a0 2024-05-08T23:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6bb79a5b80 10 client.0 _ll_register_callbacks cb 0x7f6b9c003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6ba9587700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227abe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6ba9587700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227abe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59464/0 (socket says 172.21.15.5:59464) 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.307+0000 7f6ba9587700 1 -- 172.21.15.5:0/2395659070 learned_addr learned my addr 172.21.15.5:0/2395659070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227ac1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a337a0 0x55b227a09ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba9587700 1 -- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a337a0 msgr2=0x55b227a09ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba9587700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a337a0 0x55b227a09ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba9587700 1 -- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 msgr2=0x55b227ac1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba9587700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227ac1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba9587700 1 -- 172.21.15.5:0/2395659070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba0073040 con 0x55b227a0e370 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a337a0 0x55b227a09ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba9d88700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227ac1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6ba9587700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227abe070 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0054240 tx=0x7f6ba005a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550a9e7a042b9b18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6b83fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6b83fff700 1 -- 172.21.15.5:0/2395659070 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0055e80 con 0x55b227a0e370 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6b83fff700 1 -- 172.21.15.5:0/2395659070 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ba00560e0 con 0x55b227a0e370 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/2395659070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ba005bb70 con 0x55b227a0e370 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6b83fff700 1 -- 172.21.15.5:0/2395659070 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba0019260 con 0x55b227a0e370 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6bb79a5b80 10 client.8166 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/2395659070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55b2277e5cd0 con 0x55b227a0e370 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6b83fff700 1 -- 172.21.15.5:0/2395659070 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba005bb70 con 0x55b227a0e370 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6b83fff700 1 -- 172.21.15.5:0/2395659070 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f6ba0057fa0 con 0x55b227a0e370 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6bb79a5b80 10 client.8166 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6bb79a5b80 10 client.8166 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6bb79a5b80 20 client.8166 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.308+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55b227a3b550 0x55b227a3b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 4 client.8166 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/2395659070 --> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b227958ec0 con 0x55b227a3b550 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55b227a3b550 0x55b227a3b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6b83fff700 10 client.8166 ms_handle_connect on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6ba8d86700 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55b227a3b550 0x55b227a3b930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3e002623ffc2fbc server_cookie=f00faf57eb467fd1 in_seq=0 out_seq=0 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6b83fff700 1 -- 172.21.15.5:0/2395659070 <== mds.1 v2:172.21.15.5:6832/3718289875 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55b227958ec0 con 0x55b227a3b550 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6b83fff700 10 client.8166 handle_command_reply: tid=0 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 client.8166 shutdown 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/2395659070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 msgr2=0x55b227abe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227abe070 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0054240 tx=0x7f6ba005a070 comp rx=0 tx=0).stop 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/2395659070 shutdown_connections 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b227a3add0 0x55b227ac1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x55b227a3b550 0x55b227a3b930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227a0e370 0x55b227abe070 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 --2- 172.21.15.5:0/2395659070 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b227a337a0 0x55b227a09ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6b83fff700 0 client.8166 ms_handle_reset on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.309+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/2395659070 >> 172.21.15.5:0/2395659070 conn(0x55b22796e210 msgr2=0x55b227a55aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/2395659070 shutdown_connections 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bb79a5b80 1 -- 172.21.15.5:0/2395659070 wait complete. 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bb79a5b80 20 client.8166 trim_cache size 0 max 16384 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:27:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "4b4cf457-7a2d-427a-a3be-d52d0c5f729e": { 2024-05-08T23:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4b4cf457-7a2d-427a-a3be-d52d0c5f729e", 2024-05-08T23:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 -- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b8806ac90 msgr2=0x7f6b8806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b8806ac90 0x7f6b8806d150 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6ba414b770 tx=0x7f6b94009210 comp rx=0 tx=0).stop 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 -- 172.21.15.5:0/2214033874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4099f90 msgr2=0x7f6ba40c5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4099f90 0x7f6ba40c5160 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c550 tx=0x7f6ba000c860 comp rx=0 tx=0).stop 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 -- 172.21.15.5:0/2214033874 shutdown_connections 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6b8806ac90 0x7f6b8806d150 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba415da60 0x7f6ba41f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4099f90 0x7f6ba40c5160 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 --2- 172.21.15.5:0/2214033874 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6ba409a790 0x7f6ba40b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 -- 172.21.15.5:0/2214033874 >> 172.21.15.5:0/2214033874 conn(0x7f6ba40b2f60 msgr2=0x7f6ba40d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 -- 172.21.15.5:0/2214033874 shutdown_connections 2024-05-08T23:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.310+0000 7f6bab7eb700 1 -- 172.21.15.5:0/2214033874 wait complete. 2024-05-08T23:27:25.323 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:27:25.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 scrub status 2024-05-08T23:27:25.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.462+0000 7fbdaebbc700 1 Processor -- start 2024-05-08T23:27:25.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.462+0000 7fbdaebbc700 1 -- start start 2024-05-08T23:27:25.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdaebbc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbda815dc60 0x7fbda80ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdaebbc700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda80aed90 con 0x7fbda8072170 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdaebbc700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda806dcd0 con 0x7fbda80ae620 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdaebbc700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda806e290 con 0x7fbda815dc60 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdac958700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbda815dc60 0x7fbda80ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbda7fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbda8072170 0x7fbda80adf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34082/0 (socket says 172.21.15.5:34082) 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbda7fff700 1 -- 172.21.15.5:0/2075708908 learned_addr learned my addr 172.21.15.5:0/2075708908 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdac958700 1 -- 172.21.15.5:0/2075708908 >> v1:172.21.15.49:6789/0 conn(0x7fbda80ae620 legacy=0x7fbda806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdac958700 1 -- 172.21.15.5:0/2075708908 >> v1:172.21.15.5:6789/0 conn(0x7fbda8072170 legacy=0x7fbda80adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdac958700 1 -- 172.21.15.5:0/2075708908 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80b2aa0 con 0x7fbda815dc60 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbda77fe700 1 -- 172.21.15.5:0/2075708908 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313978618 0 0) 0x7fbd9c001110 con 0x7fbda8072170 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.464+0000 7fbdac958700 1 --2- 172.21.15.5:0/2075708908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda815dc60 0x7fbda80ad9d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009840 tx=0x7fbd98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bfaea91fcbbf78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbda77fe700 1 -- 172.21.15.5:0/2075708908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98011830 con 0x7fbda815dc60 2024-05-08T23:27:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbda77fe700 1 -- 172.21.15.5:0/2075708908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd98011990 con 0x7fbda815dc60 2024-05-08T23:27:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbda77fe700 1 -- 172.21.15.5:0/2075708908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98011d20 con 0x7fbda815dc60 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbdaebbc700 1 -- 172.21.15.5:0/2075708908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda815dc60 msgr2=0x7fbda80ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbdaebbc700 1 --2- 172.21.15.5:0/2075708908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda815dc60 0x7fbda80ad9d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009840 tx=0x7fbd98009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbdaebbc700 1 -- 172.21.15.5:0/2075708908 shutdown_connections 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbdaebbc700 1 --2- 172.21.15.5:0/2075708908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda815dc60 0x7fbda80ad9d0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbdaebbc700 1 -- 172.21.15.5:0/2075708908 >> 172.21.15.5:0/2075708908 conn(0x7fbda80b5a30 msgr2=0x7fbda80b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbdaebbc700 1 -- 172.21.15.5:0/2075708908 shutdown_connections 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.465+0000 7fbdaebbc700 1 -- 172.21.15.5:0/2075708908 wait complete. 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.466+0000 7fbdaebbc700 1 Processor -- start 2024-05-08T23:27:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.467+0000 7fbdaebbc700 1 -- start start 2024-05-08T23:27:25.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.467+0000 7fbdaebbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda8072170 0x7fbda81f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdaebbc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda80ae620 0x7fbda81567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdaebbc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbda815dc60 0x7fbda809b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdaebbc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8156ce0 con 0x7fbda80ae620 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdaebbc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda809a480 con 0x7fbda8072170 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdaebbc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda809a720 con 0x7fbda815dc60 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda80ae620 0x7fbda81567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda80ae620 0x7fbda81567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59486/0 (socket says 172.21.15.5:59486) 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 -- 172.21.15.5:0/1457730481 learned_addr learned my addr 172.21.15.5:0/1457730481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdac958700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda8072170 0x7fbda81f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdad159700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbda815dc60 0x7fbda809b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 -- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbda815dc60 msgr2=0x7fbda809b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbda815dc60 0x7fbda809b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 -- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda8072170 msgr2=0x7fbda81f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda8072170 0x7fbda81f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 -- 172.21.15.5:0/1457730481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd980094a0 con 0x7fbda80ae620 2024-05-08T23:27:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdad159700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbda815dc60 0x7fbda809b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda7fff700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda80ae620 0x7fbda81567a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00c520 tx=0x7fbd9c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda57fa700 1 -- 172.21.15.5:0/1457730481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c004c80 con 0x7fbda80ae620 2024-05-08T23:27:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda57fa700 1 -- 172.21.15.5:0/1457730481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c004e00 con 0x7fbda80ae620 2024-05-08T23:27:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbda57fa700 1 -- 172.21.15.5:0/1457730481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c007310 con 0x7fbda80ae620 2024-05-08T23:27:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.468+0000 7fbdaebbc700 1 -- 172.21.15.5:0/1457730481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda809aa00 con 0x7fbda80ae620 2024-05-08T23:27:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.469+0000 7fbdaebbc700 1 -- 172.21.15.5:0/1457730481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda809afc0 con 0x7fbda80ae620 2024-05-08T23:27:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.471+0000 7fbda57fa700 1 -- 172.21.15.5:0/1457730481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbd9c007470 con 0x7fbda80ae620 2024-05-08T23:27:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.471+0000 7fbda57fa700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbd9406ac90 0x7fbd9406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.471+0000 7fbda57fa700 1 -- 172.21.15.5:0/1457730481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd9c088e20 con 0x7fbda80ae620 2024-05-08T23:27:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.471+0000 7fbdac958700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbd9406ac90 0x7fbd9406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.472+0000 7fbdac958700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbd9406ac90 0x7fbd9406d150 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbd980090b0 tx=0x7fbd9801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.473+0000 7fbdbad76b80 1 Processor -- start 2024-05-08T23:27:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.473+0000 7fbdbad76b80 1 -- start start 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c07a0 0x5588a9999110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a999e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c7dd0 0x5588a99c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a98f4150 con 0x5588a999b370 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a98f1b10 con 0x5588a99c7dd0 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588a98ed400 con 0x5588a99c07a0 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a999e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a999e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59498/0 (socket says 172.21.15.5:59498) 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 -- 172.21.15.5:0/2422204620 learned_addr learned my addr 172.21.15.5:0/2422204620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdac958700 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c07a0 0x5588a9999110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdad159700 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c7dd0 0x5588a99c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 -- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c07a0 msgr2=0x5588a9999110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c07a0 0x5588a9999110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 -- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c7dd0 msgr2=0x5588a99c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c7dd0 0x5588a99c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 -- 172.21.15.5:0/2422204620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588a976e3b0 con 0x5588a999b370 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdac958700 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c07a0 0x5588a9999110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbdad159700 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c7dd0 0x5588a99c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.474+0000 7fbda7fff700 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a999e0a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c057c40 tx=0x7fbd9c057710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7b7ea4a3847c52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbd8e7fc700 1 -- 172.21.15.5:0/2422204620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0600a0 con 0x5588a999b370 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbd8e7fc700 1 -- 172.21.15.5:0/2422204620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c007720 con 0x5588a999b370 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbd8e7fc700 1 -- 172.21.15.5:0/2422204620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c056050 con 0x5588a999b370 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 -- 172.21.15.5:0/2422204620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 msgr2=0x5588a999e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a999e0a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c057c40 tx=0x7fbd9c057710 comp rx=0 tx=0).stop 2024-05-08T23:27:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 -- 172.21.15.5:0/2422204620 shutdown_connections 2024-05-08T23:27:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c7dd0 0x5588a99c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c07a0 0x5588a9999110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/2422204620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a999e0a0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 -- 172.21.15.5:0/2422204620 >> 172.21.15.5:0/2422204620 conn(0x5588a98fb210 msgr2=0x5588a99e2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 -- 172.21.15.5:0/2422204620 shutdown_connections 2024-05-08T23:27:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.475+0000 7fbdbad76b80 1 -- 172.21.15.5:0/2422204620 wait complete. 2024-05-08T23:27:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.476+0000 7fbdbad76b80 1 -- start start 2024-05-08T23:27:25.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.476+0000 7fbdbad76b80 1 Processor -- start 2024-05-08T23:27:25.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a999b370 0x5588a99f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c07a0 0x5588a99ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 0x5588a99e6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a9889560 con 0x5588a99c07a0 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a9889200 con 0x5588a999b370 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588a97706a0 con 0x5588a99c7dd0 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbdbad76b80 10 client.0 _ll_register_callbacks cb 0x5588a9892b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbda7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c07a0 0x5588a99ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbda7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c07a0 0x5588a99ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59512/0 (socket says 172.21.15.5:59512) 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.477+0000 7fbda7fff700 1 -- 172.21.15.5:0/528477297 learned_addr learned my addr 172.21.15.5:0/528477297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbda7fff700 1 -- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 msgr2=0x5588a99e6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbda7fff700 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 0x5588a99e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbda7fff700 1 -- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a999b370 msgr2=0x5588a99f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbdac958700 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a999b370 0x5588a99f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbda7fff700 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a999b370 0x5588a99f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbda7fff700 1 -- 172.21.15.5:0/528477297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda809afc0 con 0x5588a99c07a0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbdac958700 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a999b370 0x5588a99f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbda7fff700 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c07a0 0x5588a99ca010 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c000f60 tx=0x7fbd9c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0d5bd327dd36d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbd877fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbd877fe700 1 -- 172.21.15.5:0/528477297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0600a0 con 0x5588a99c07a0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbd877fe700 1 -- 172.21.15.5:0/528477297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd9c057f10 con 0x5588a99c07a0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbdbad76b80 1 -- 172.21.15.5:0/528477297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588a9846270 con 0x5588a99c07a0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbd877fe700 1 -- 172.21.15.5:0/528477297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd9c0580d0 con 0x5588a99c07a0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbdbad76b80 10 client.8181 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbdbad76b80 1 -- 172.21.15.5:0/528477297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5588a971ab60 con 0x5588a99c07a0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbd877fe700 1 -- 172.21.15.5:0/528477297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5588a9846270 con 0x5588a99c07a0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbd877fe700 1 -- 172.21.15.5:0/528477297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fbd9c0571a0 con 0x5588a99c07a0 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbdbad76b80 10 client.8181 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbdbad76b80 10 client.8181 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-05-08T23:27:25.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.478+0000 7fbdbad76b80 20 client.8181 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.479+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5588a99e42e0 0x5588a99e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.479+0000 7fbdbad76b80 4 client.8181 mds_command: new command op to 7336 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.479+0000 7fbdbad76b80 1 -- 172.21.15.5:0/528477297 --> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588a996c7e0 con 0x5588a99e42e0 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.479+0000 7fbdac958700 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5588a99e42e0 0x5588a99e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.479+0000 7fbd877fe700 10 client.8181 ms_handle_connect on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.479+0000 7fbdac958700 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5588a99e42e0 0x5588a99e6c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb76651aa5f98cd2 server_cookie=2a064e03dfbf4790 in_seq=0 out_seq=0 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbd877fe700 1 -- 172.21.15.5:0/528477297 <== mds.2 v2:172.21.15.49:6835/1134209849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588a996c7e0 con 0x5588a99e42e0 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbd877fe700 10 client.8181 handle_command_reply: tid=0 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 client.8181 shutdown 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 -- 172.21.15.5:0/528477297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c07a0 msgr2=0x5588a99ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c07a0 0x5588a99ca010 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c000f60 tx=0x7fbd9c000ed0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 -- 172.21.15.5:0/528477297 shutdown_connections 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 0x5588a99e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5588a99e42e0 0x5588a99e6c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a999b370 0x5588a99f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/528477297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c07a0 0x5588a99ca010 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbd877fe700 0 client.8181 ms_handle_reset on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:25.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 -- 172.21.15.5:0/528477297 >> 172.21.15.5:0/528477297 conn(0x5588a98fb210 msgr2=0x5588a999d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.480+0000 7fbdbad76b80 1 -- 172.21.15.5:0/528477297 shutdown_connections 2024-05-08T23:27:25.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.481+0000 7fbdbad76b80 1 -- 172.21.15.5:0/528477297 wait complete. 2024-05-08T23:27:25.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.481+0000 7fbdbad76b80 20 client.8181 trim_cache size 0 max 16384 2024-05-08T23:27:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.499+0000 7fbdbad76b80 1 Processor -- start 2024-05-08T23:27:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.499+0000 7fbdbad76b80 1 -- start start 2024-05-08T23:27:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a99e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a9996ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 0x5588a9997220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a99caf50 con 0x5588a999b370 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a99f2070 con 0x5588a99c07a0 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588a999e5c0 con 0x5588a99c7dd0 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a99e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a99e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59516/0 (socket says 172.21.15.5:59516) 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 -- 172.21.15.5:0/3257388987 learned_addr learned my addr 172.21.15.5:0/3257388987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdad159700 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 0x5588a9997220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbda7fff700 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a9996ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 -- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 msgr2=0x5588a9997220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 0x5588a9997220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 -- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 msgr2=0x5588a9996ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a9996ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 -- 172.21.15.5:0/3257388987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588a98b3560 con 0x5588a999b370 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbda7fff700 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a9996ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdad159700 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 0x5588a9997220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbdac958700 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a99e67f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800a010 tx=0x7fbd98010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58eac31ed5b6a5c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbd8effd700 1 -- 172.21.15.5:0/3257388987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9802a040 con 0x5588a999b370 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbd8effd700 1 -- 172.21.15.5:0/3257388987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd98025c10 con 0x5588a999b370 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.500+0000 7fbd8effd700 1 -- 172.21.15.5:0/3257388987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd980340b0 con 0x5588a999b370 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3257388987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 msgr2=0x5588a99e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a99e67f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800a010 tx=0x7fbd98010000 comp rx=0 tx=0).stop 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3257388987 shutdown_connections 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a99c7dd0 0x5588a9997220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a999b370 0x5588a99e67f0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3257388987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a9996ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3257388987 >> 172.21.15.5:0/3257388987 conn(0x5588a98fb210 msgr2=0x5588a99941e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3257388987 shutdown_connections 2024-05-08T23:27:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.501+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3257388987 wait complete. 2024-05-08T23:27:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.502+0000 7fbdbad76b80 1 -- start start 2024-05-08T23:27:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.502+0000 7fbdbad76b80 1 Processor -- start 2024-05-08T23:27:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a999b370 0x5588a9a04de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a99970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdbad76b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c7dd0 0x5588a99e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a99d87f0 con 0x5588a99c7dd0 2024-05-08T23:27:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a99e3ce0 con 0x5588a99c07a0 2024-05-08T23:27:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdbad76b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588a9997620 con 0x5588a999b370 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdbad76b80 10 client.0 _ll_register_callbacks cb 0x5588a9892b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdad159700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c7dd0 0x5588a99e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdad159700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c7dd0 0x5588a99e3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59532/0 (socket says 172.21.15.5:59532) 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.503+0000 7fbdad159700 1 -- 172.21.15.5:0/3704472336 learned_addr learned my addr 172.21.15.5:0/3704472336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdac958700 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a999b370 0x5588a9a04de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbda7fff700 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a99970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdad159700 1 -- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a999b370 msgr2=0x5588a9a04de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdad159700 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a999b370 0x5588a9a04de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdad159700 1 -- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 msgr2=0x5588a99970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdad159700 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a99970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdad159700 1 -- 172.21.15.5:0/3704472336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80b2aa0 con 0x5588a99c7dd0 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdad159700 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c7dd0 0x5588a99e3000 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbda00052b0 tx=0x7fbda0005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b951ca9dec5636f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbd877fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbd877fe700 1 -- 172.21.15.5:0/3704472336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda00116e0 con 0x5588a99c7dd0 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbd877fe700 1 -- 172.21.15.5:0/3704472336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbda0011840 con 0x5588a99c7dd0 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3704472336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd9c007720 con 0x5588a99c7dd0 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbd877fe700 1 -- 172.21.15.5:0/3704472336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbda0011a00 con 0x5588a99c7dd0 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdbad76b80 10 client.8187 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3704472336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5588a9772cd0 con 0x5588a99c7dd0 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbd877fe700 1 -- 172.21.15.5:0/3704472336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c007720 con 0x5588a99c7dd0 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.504+0000 7fbd877fe700 1 -- 172.21.15.5:0/3704472336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fbda0011b60 con 0x5588a99c7dd0 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdbad76b80 10 client.8187 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdbad76b80 10 client.8187 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdbad76b80 20 client.8187 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5588a99df730 0x5588a9a4aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdbad76b80 4 client.8187 mds_command: new command op to 7336 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3704472336 --> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588a98e5ec0 con 0x5588a99df730 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdac958700 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5588a99df730 0x5588a9a4aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbd877fe700 10 client.8187 ms_handle_connect on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdac958700 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5588a99df730 0x5588a9a4aa70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9e03d01fa0c746e server_cookie=211786831aa2901c in_seq=0 out_seq=0 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbd877fe700 1 -- 172.21.15.5:0/3704472336 <== mds.2 v2:172.21.15.49:6835/1134209849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5588a98e5ec0 con 0x5588a99df730 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbd877fe700 10 client.8187 handle_command_reply: tid=0 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.505+0000 7fbdbad76b80 1 client.8187 shutdown 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3704472336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c7dd0 msgr2=0x5588a99e3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c7dd0 0x5588a99e3000 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbda00052b0 tx=0x7fbda0005670 comp rx=0 tx=0).stop 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3704472336 shutdown_connections 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a99c7dd0 0x5588a99e3000 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5588a999b370 0x5588a9a04de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x5588a99df730 0x5588a9a4aa70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 --2- 172.21.15.5:0/3704472336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5588a99c07a0 0x5588a99970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbd877fe700 0 client.8187 ms_handle_reset on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3704472336 >> 172.21.15.5:0/3704472336 conn(0x5588a98fb210 msgr2=0x5588a99e67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3704472336 shutdown_connections 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 1 -- 172.21.15.5:0/3704472336 wait complete. 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdbad76b80 20 client.8187 trim_cache size 0 max 16384 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stdout: "4b4cf457-7a2d-427a-a3be-d52d0c5f729e": { 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:27:25.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4b4cf457-7a2d-427a-a3be-d52d0c5f729e", 2024-05-08T23:27:25.513 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:27:25.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:25.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:25.513 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdaebbc700 1 -- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbd9406ac90 msgr2=0x7fbd9406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdaebbc700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbd9406ac90 0x7fbd9406d150 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbd980090b0 tx=0x7fbd9801c040 comp rx=0 tx=0).stop 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdaebbc700 1 -- 172.21.15.5:0/1457730481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda80ae620 msgr2=0x7fbda81567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.506+0000 7fbdaebbc700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda80ae620 0x7fbda81567a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00c520 tx=0x7fbd9c00c830 comp rx=0 tx=0).stop 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.507+0000 7fbdaebbc700 1 -- 172.21.15.5:0/1457730481 shutdown_connections 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.507+0000 7fbdaebbc700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbd9406ac90 0x7fbd9406d150 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.507+0000 7fbdaebbc700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbda815dc60 0x7fbda809b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.507+0000 7fbdaebbc700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda8072170 0x7fbda81f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.507+0000 7fbdaebbc700 1 --2- 172.21.15.5:0/1457730481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbda80ae620 0x7fbda81567a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.507+0000 7fbdaebbc700 1 -- 172.21.15.5:0/1457730481 >> 172.21.15.5:0/1457730481 conn(0x7fbda80b5a30 msgr2=0x7fbda815bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.507+0000 7fbdaebbc700 1 -- 172.21.15.5:0/1457730481 shutdown_connections 2024-05-08T23:27:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.507+0000 7fbdaebbc700 1 -- 172.21.15.5:0/1457730481 wait complete. 2024-05-08T23:27:25.520 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:27:25.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T23:27:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.658+0000 7fa0270ed700 1 Processor -- start 2024-05-08T23:27:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.659+0000 7fa0270ed700 1 -- start start 2024-05-08T23:27:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.659+0000 7fa0270ed700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa02009a790 0x7fa020099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.659+0000 7fa0270ed700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02006d010 con 0x7fa020099f90 2024-05-08T23:27:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.659+0000 7fa0270ed700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02015de40 con 0x7fa02015da60 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.659+0000 7fa0270ed700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02009a420 con 0x7fa02009a790 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01ffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa02009a790 0x7fa020099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa02568a700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa020099f90 0x7fa0200cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34090/0 (socket says 172.21.15.5:34090) 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa02568a700 1 -- 172.21.15.5:0/679910445 learned_addr learned my addr 172.21.15.5:0/679910445 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01ffff700 1 -- 172.21.15.5:0/679910445 >> v1:172.21.15.49:6789/0 conn(0x7fa02015da60 legacy=0x7fa0200d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01ffff700 1 -- 172.21.15.5:0/679910445 >> v1:172.21.15.5:6789/0 conn(0x7fa020099f90 legacy=0x7fa0200cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01ffff700 1 -- 172.21.15.5:0/679910445 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200d0b20 con 0x7fa02009a790 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01f7fe700 1 -- 172.21.15.5:0/679910445 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992295850 0 0) 0x7fa02006d010 con 0x7fa020099f90 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01ffff700 1 --2- 172.21.15.5:0/679910445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02009a790 0x7fa020099a50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa014009840 tx=0x7fa014009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f3ca2a50ee3efb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01f7fe700 1 -- 172.21.15.5:0/679910445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014011830 con 0x7fa02009a790 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01f7fe700 1 -- 172.21.15.5:0/679910445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014011990 con 0x7fa02009a790 2024-05-08T23:27:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.660+0000 7fa01f7fe700 1 -- 172.21.15.5:0/679910445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014011d20 con 0x7fa02009a790 2024-05-08T23:27:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.661+0000 7fa0270ed700 1 -- 172.21.15.5:0/679910445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02009a790 msgr2=0x7fa020099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.661+0000 7fa0270ed700 1 --2- 172.21.15.5:0/679910445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02009a790 0x7fa020099a50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa014009840 tx=0x7fa014009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.661+0000 7fa0270ed700 1 -- 172.21.15.5:0/679910445 shutdown_connections 2024-05-08T23:27:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.661+0000 7fa0270ed700 1 --2- 172.21.15.5:0/679910445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02009a790 0x7fa020099a50 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.661+0000 7fa0270ed700 1 -- 172.21.15.5:0/679910445 >> 172.21.15.5:0/679910445 conn(0x7fa0200b2f60 msgr2=0x7fa0200b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.661+0000 7fa0270ed700 1 -- 172.21.15.5:0/679910445 shutdown_connections 2024-05-08T23:27:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.661+0000 7fa0270ed700 1 -- 172.21.15.5:0/679910445 wait complete. 2024-05-08T23:27:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.663+0000 7fa0270ed700 1 Processor -- start 2024-05-08T23:27:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.663+0000 7fa0270ed700 1 -- start start 2024-05-08T23:27:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa0270ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa020099f90 0x7fa020159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa0270ed700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa02009a790 0x7fa020150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa0270ed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02015da60 0x7fa0200b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa0270ed700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200b5560 con 0x7fa02015da60 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa0270ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200b48e0 con 0x7fa020099f90 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa0270ed700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200b4bc0 con 0x7fa02009a790 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa02568a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02015da60 0x7fa0200b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa02568a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02015da60 0x7fa0200b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59548/0 (socket says 172.21.15.5:59548) 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa02568a700 1 -- 172.21.15.5:0/1731929329 learned_addr learned my addr 172.21.15.5:0/1731929329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa01ffff700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa02009a790 0x7fa020150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.664+0000 7fa024e89700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa020099f90 0x7fa020159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa02568a700 1 -- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa02009a790 msgr2=0x7fa020150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa02568a700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa02009a790 0x7fa020150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa02568a700 1 -- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa020099f90 msgr2=0x7fa020159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa02568a700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa020099f90 0x7fa020159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa02568a700 1 -- 172.21.15.5:0/1731929329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0140094a0 con 0x7fa02015da60 2024-05-08T23:27:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa024e89700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa020099f90 0x7fa020159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa02568a700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02015da60 0x7fa0200b4090 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa01800cca0 tx=0x7fa01800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa01d7fa700 1 -- 172.21.15.5:0/1731929329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800ac20 con 0x7fa02015da60 2024-05-08T23:27:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa01d7fa700 1 -- 172.21.15.5:0/1731929329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01800ad80 con 0x7fa02015da60 2024-05-08T23:27:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa01d7fa700 1 -- 172.21.15.5:0/1731929329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018007310 con 0x7fa02015da60 2024-05-08T23:27:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa0270ed700 1 -- 172.21.15.5:0/1731929329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0200b4ea0 con 0x7fa02015da60 2024-05-08T23:27:25.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.665+0000 7fa0270ed700 1 -- 172.21.15.5:0/1731929329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0200b45d0 con 0x7fa02015da60 2024-05-08T23:27:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.667+0000 7fa01d7fa700 1 -- 172.21.15.5:0/1731929329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa018007470 con 0x7fa02015da60 2024-05-08T23:27:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.667+0000 7fa01d7fa700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa00c06ac90 0x7fa00c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.668+0000 7fa01d7fa700 1 -- 172.21.15.5:0/1731929329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa018088db0 con 0x7fa02015da60 2024-05-08T23:27:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.668+0000 7fa024e89700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa00c06ac90 0x7fa00c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.668+0000 7fa024e89700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa00c06ac90 0x7fa00c06d150 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa02015f640 tx=0x7fa010009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.670+0000 7fa0332a7b80 1 Processor -- start 2024-05-08T23:27:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.670+0000 7fa0332a7b80 1 -- start start 2024-05-08T23:27:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.670+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 0x5614e6419110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.670+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e641e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.670+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e6448250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.670+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e6374150 con 0x5614e641b370 2024-05-08T23:27:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.670+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e6371b10 con 0x5614e6447dd0 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.670+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e636d400 con 0x5614e64407a0 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e641e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e641e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59560/0 (socket says 172.21.15.5:59560) 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa024e89700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 0x5614e6419110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 -- 172.21.15.5:0/1243569154 learned_addr learned my addr 172.21.15.5:0/1243569154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa02568a700 1 --2- 172.21.15.5:0/1243569154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e6448250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 -- 172.21.15.5:0/1243569154 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 msgr2=0x5614e6419110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 --2- 172.21.15.5:0/1243569154 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 0x5614e6419110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 -- 172.21.15.5:0/1243569154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 msgr2=0x5614e6448250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 --2- 172.21.15.5:0/1243569154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e6448250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 -- 172.21.15.5:0/1243569154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e61ee3b0 con 0x5614e641b370 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa01ffff700 1 --2- 172.21.15.5:0/1243569154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e641e0a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa020072950 tx=0x7fa014010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447497aaf351a53f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa0067fc700 1 -- 172.21.15.5:0/1243569154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014020070 con 0x5614e641b370 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa0067fc700 1 -- 172.21.15.5:0/1243569154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014011800 con 0x5614e641b370 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.671+0000 7fa0067fc700 1 -- 172.21.15.5:0/1243569154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014011d00 con 0x5614e641b370 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1243569154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 msgr2=0x5614e641e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1243569154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e641e0a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa020072950 tx=0x7fa014010000 comp rx=0 tx=0).stop 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1243569154 shutdown_connections 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1243569154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e6448250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1243569154 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 0x5614e6419110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1243569154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e641e0a0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1243569154 >> 172.21.15.5:0/1243569154 conn(0x5614e637b210 msgr2=0x5614e6462dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1243569154 shutdown_connections 2024-05-08T23:27:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.672+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1243569154 wait complete. 2024-05-08T23:27:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.673+0000 7fa0332a7b80 1 -- start start 2024-05-08T23:27:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.673+0000 7fa0332a7b80 1 Processor -- start 2024-05-08T23:27:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e6464c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e64407a0 0x5614e6462a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e6447dd0 0x5614e643f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e6309560 con 0x5614e641b370 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e6309200 con 0x5614e64407a0 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e61f06a0 con 0x5614e6447dd0 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa0332a7b80 10 client.0 _ll_register_callbacks cb 0x5614e6312b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa024e89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e6464c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa024e89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e6464c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59574/0 (socket says 172.21.15.5:59574) 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa024e89700 1 -- 172.21.15.5:0/2175742241 learned_addr learned my addr 172.21.15.5:0/2175742241 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa01ffff700 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e64407a0 0x5614e6462a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa02568a700 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e6447dd0 0x5614e643f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa024e89700 1 -- 172.21.15.5:0/2175742241 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e6447dd0 msgr2=0x5614e643f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa024e89700 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e6447dd0 0x5614e643f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa024e89700 1 -- 172.21.15.5:0/2175742241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e64407a0 msgr2=0x5614e6462a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.674+0000 7fa024e89700 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e64407a0 0x5614e6462a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa024e89700 1 -- 172.21.15.5:0/2175742241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0200d0b20 con 0x5614e641b370 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa024e89700 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e6464c60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa010003c30 tx=0x7fa010001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23ac407974f44ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7f9fff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7f9fff7fe700 1 -- 172.21.15.5:0/2175742241 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010005540 con 0x5614e641b370 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7f9fff7fe700 1 -- 172.21.15.5:0/2175742241 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa010014ad0 con 0x5614e641b370 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 1 -- 172.21.15.5:0/2175742241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614e62c6270 con 0x5614e641b370 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7f9fff7fe700 1 -- 172.21.15.5:0/2175742241 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa010005330 con 0x5614e641b370 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 10 client.8202 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 1 -- 172.21.15.5:0/2175742241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5614e619ab60 con 0x5614e641b370 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7f9fff7fe700 1 -- 172.21.15.5:0/2175742241 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614e62c6270 con 0x5614e641b370 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7f9fff7fe700 1 -- 172.21.15.5:0/2175742241 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fa010020070 con 0x5614e641b370 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 10 client.8202 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 10 client.8202 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 20 client.8202 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5614e644b7a0 0x5614e64721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 4 client.8202 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.675+0000 7fa0332a7b80 1 -- 172.21.15.5:0/2175742241 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614e63ec7e0 con 0x5614e644b7a0 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.676+0000 7fa01ffff700 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5614e644b7a0 0x5614e64721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.676+0000 7fa01ffff700 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5614e644b7a0 0x5614e64721f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f406fb7c1cd3df6 server_cookie=c4822ac66987094a in_seq=0 out_seq=0 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.676+0000 7f9fff7fe700 10 client.8202 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.676+0000 7f9fff7fe700 1 -- 172.21.15.5:0/2175742241 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614e63ec7e0 con 0x5614e644b7a0 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.676+0000 7f9fff7fe700 10 client.8202 handle_command_reply: tid=0 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.676+0000 7fa0332a7b80 1 client.8202 shutdown 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 -- 172.21.15.5:0/2175742241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 msgr2=0x5614e6464c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e6464c60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa010003c30 tx=0x7fa010001080 comp rx=0 tx=0).stop 2024-05-08T23:27:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 -- 172.21.15.5:0/2175742241 shutdown_connections 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5614e644b7a0 0x5614e64721f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e6447dd0 0x5614e643f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e641b370 0x5614e6464c60 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7f9fff7fe700 0 client.8202 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/2175742241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e64407a0 0x5614e6462a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 -- 172.21.15.5:0/2175742241 >> 172.21.15.5:0/2175742241 conn(0x5614e637b210 msgr2=0x5614e6416ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 -- 172.21.15.5:0/2175742241 shutdown_connections 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 1 -- 172.21.15.5:0/2175742241 wait complete. 2024-05-08T23:27:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.677+0000 7fa0332a7b80 20 client.8202 trim_cache size 0 max 16384 2024-05-08T23:27:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.695+0000 7fa0332a7b80 1 Processor -- start 2024-05-08T23:27:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.695+0000 7fa0332a7b80 1 -- start start 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e641b370 0x5614e64189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 0x5614e6462670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e6447dd0 0x5614e641dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e6445720 con 0x5614e6447dd0 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e6445310 con 0x5614e641b370 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e641ed30 con 0x5614e64407a0 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e6447dd0 0x5614e641dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e6447dd0 0x5614e641dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59576/0 (socket says 172.21.15.5:59576) 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 -- 172.21.15.5:0/957077896 learned_addr learned my addr 172.21.15.5:0/957077896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa01ffff700 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 0x5614e6462670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa024e89700 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e641b370 0x5614e64189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 -- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 msgr2=0x5614e6462670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 0x5614e6462670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 -- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e641b370 msgr2=0x5614e64189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e641b370 0x5614e64189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 -- 172.21.15.5:0/957077896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e6471420 con 0x5614e6447dd0 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa024e89700 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e641b370 0x5614e64189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa02568a700 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e6447dd0 0x5614e641dd50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa0180541e0 tx=0x7fa018001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfadd39f1b8d4089 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.696+0000 7fa006ffd700 1 -- 172.21.15.5:0/957077896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018057ce0 con 0x5614e6447dd0 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa006ffd700 1 -- 172.21.15.5:0/957077896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01806f070 con 0x5614e6447dd0 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa006ffd700 1 -- 172.21.15.5:0/957077896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01805bb10 con 0x5614e6447dd0 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 -- 172.21.15.5:0/957077896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e6447dd0 msgr2=0x5614e641dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e6447dd0 0x5614e641dd50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa0180541e0 tx=0x7fa018001d40 comp rx=0 tx=0).stop 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 -- 172.21.15.5:0/957077896 shutdown_connections 2024-05-08T23:27:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e6447dd0 0x5614e641dd50 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e641b370 0x5614e64189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/957077896 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e64407a0 0x5614e6462670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 -- 172.21.15.5:0/957077896 >> 172.21.15.5:0/957077896 conn(0x5614e637b210 msgr2=0x5614e64141e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 -- 172.21.15.5:0/957077896 shutdown_connections 2024-05-08T23:27:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.697+0000 7fa0332a7b80 1 -- 172.21.15.5:0/957077896 wait complete. 2024-05-08T23:27:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.698+0000 7fa0332a7b80 1 -- start start 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.698+0000 7fa0332a7b80 1 Processor -- start 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e641b370 0x5614e6459100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e64407a0 0x5614e6416ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa0332a7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e64cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e64cbfc0 con 0x5614e64407a0 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e641be80 con 0x5614e6447dd0 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa0332a7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e6472b10 con 0x5614e641b370 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa0332a7b80 10 client.0 _ll_register_callbacks cb 0x5614e6312b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e64407a0 0x5614e6416ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e64407a0 0x5614e6416ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59584/0 (socket says 172.21.15.5:59584) 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa01ffff700 1 -- 172.21.15.5:0/1467992445 learned_addr learned my addr 172.21.15.5:0/1467992445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa024e89700 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e641b370 0x5614e6459100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa02568a700 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e64cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa01ffff700 1 -- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e641b370 msgr2=0x5614e6459100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa01ffff700 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e641b370 0x5614e6459100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa01ffff700 1 -- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 msgr2=0x5614e64cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa01ffff700 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e64cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa01ffff700 1 -- 172.21.15.5:0/1467992445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0200b45d0 con 0x5614e64407a0 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.699+0000 7fa02568a700 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e64cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa01ffff700 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e64407a0 0x5614e6416ce0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa014029040 tx=0x7fa0140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded3447e85244c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7f9fff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1467992445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014020040 con 0x5614e64407a0 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1467992445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0140042c0 con 0x5614e64407a0 2024-05-08T23:27:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1467992445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa014011800 con 0x5614e64407a0 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1467992445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa014004480 con 0x5614e64407a0 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 10 client.8208 fetch_fsmap learned FSMap version 89 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1467992445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5614e6333560 con 0x5614e64407a0 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1467992445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014011800 con 0x5614e64407a0 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1467992445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fa0140036a0 con 0x5614e64407a0 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 10 client.8208 fetch_fsmap finished waiting for FSMap version 89 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 10 client.8208 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 20 client.8208 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5614e641dd60 0x5614e641e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 4 client.8208 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1467992445 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614e6365ec0 con 0x5614e641dd60 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.700+0000 7fa024e89700 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5614e641dd60 0x5614e641e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa024e89700 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5614e641dd60 0x5614e641e170 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7ad1a94accc286 server_cookie=12af45a1174bd85b in_seq=0 out_seq=0 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7f9fff7fe700 10 client.8208 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1467992445 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5614e6365ec0 con 0x5614e641dd60 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7f9fff7fe700 10 client.8208 handle_command_reply: tid=0 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 client.8208 shutdown 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1467992445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e64407a0 msgr2=0x5614e6416ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e64407a0 0x5614e6416ce0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa014029040 tx=0x7fa0140047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1467992445 shutdown_connections 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x5614e641dd60 0x5614e641e170 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5614e6447dd0 0x5614e64cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5614e641b370 0x5614e6459100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 --2- 172.21.15.5:0/1467992445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614e64407a0 0x5614e6416ce0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7f9fff7fe700 0 client.8208 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1467992445 >> 172.21.15.5:0/1467992445 conn(0x5614e637b210 msgr2=0x5614e6463300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1467992445 shutdown_connections 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 1 -- 172.21.15.5:0/1467992445 wait complete. 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.701+0000 7fa0332a7b80 20 client.8208 trim_cache size 0 max 16384 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stdout: "4b4cf457-7a2d-427a-a3be-d52d0c5f729e": { 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4b4cf457-7a2d-427a-a3be-d52d0c5f729e", 2024-05-08T23:27:25.708 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:27:25.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:25.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:25.709 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 -- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa00c06ac90 msgr2=0x7fa00c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa00c06ac90 0x7fa00c06d150 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa02015f640 tx=0x7fa010009040 comp rx=0 tx=0).stop 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 -- 172.21.15.5:0/1731929329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02015da60 msgr2=0x7fa0200b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02015da60 0x7fa0200b4090 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa01800cca0 tx=0x7fa01800a060 comp rx=0 tx=0).stop 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 -- 172.21.15.5:0/1731929329 shutdown_connections 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa00c06ac90 0x7fa00c06d150 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa02015da60 0x7fa0200b4090 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa020099f90 0x7fa020159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 --2- 172.21.15.5:0/1731929329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa02009a790 0x7fa020150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 -- 172.21.15.5:0/1731929329 >> 172.21.15.5:0/1731929329 conn(0x7fa0200b2f60 msgr2=0x7fa0200b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 -- 172.21.15.5:0/1731929329 shutdown_connections 2024-05-08T23:27:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.702+0000 7fa0270ed700 1 -- 172.21.15.5:0/1731929329 wait complete. 2024-05-08T23:27:25.715 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:27:25.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.851+0000 7fe129aca700 1 Processor -- start 2024-05-08T23:27:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.851+0000 7fe129aca700 1 -- start start 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe129aca700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe1240cd4e0 0x7fe12406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe129aca700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1240cecf0 0x7fe12415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe129aca700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12415c280 con 0x7fe12406f130 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe129aca700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12406e780 con 0x7fe1240cecf0 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe129aca700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12406ea60 con 0x7fe1240cd4e0 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe123fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1240cecf0 0x7fe12415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe123fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1240cecf0 0x7fe12415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59598/0 (socket says 172.21.15.5:59598) 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe122ffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe12406f130 0x7fe1240ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34100/0 (socket says 172.21.15.5:34100) 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe123fff700 1 -- 172.21.15.5:0/1615421118 learned_addr learned my addr 172.21.15.5:0/1615421118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe1237fe700 1 --2- 172.21.15.5:0/1615421118 >> v2:172.21.15.49:3300/0 conn(0x7fe1240cd4e0 0x7fe12406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe123fff700 1 -- 172.21.15.5:0/1615421118 >> v2:172.21.15.49:3300/0 conn(0x7fe1240cd4e0 msgr2=0x7fe12406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe123fff700 1 --2- 172.21.15.5:0/1615421118 >> v2:172.21.15.49:3300/0 conn(0x7fe1240cd4e0 0x7fe12406f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe123fff700 1 -- 172.21.15.5:0/1615421118 >> v1:172.21.15.5:6789/0 conn(0x7fe12406f130 legacy=0x7fe1240ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe123fff700 1 -- 172.21.15.5:0/1615421118 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12406ed40 con 0x7fe1240cecf0 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe1227fc700 1 -- 172.21.15.5:0/1615421118 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170175032 0 0) 0x7fe12415c280 con 0x7fe12406f130 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.852+0000 7fe123fff700 1 --2- 172.21.15.5:0/1615421118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415c710 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe124072ef0 tx=0x7fe11c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a83251465e3eaed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe1227fc700 1 -- 172.21.15.5:0/1615421118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00e070 con 0x7fe1240cecf0 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe1227fc700 1 -- 172.21.15.5:0/1615421118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c00b910 con 0x7fe1240cecf0 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe1227fc700 1 -- 172.21.15.5:0/1615421118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00bcc0 con 0x7fe1240cecf0 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe129aca700 1 -- 172.21.15.5:0/1615421118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 msgr2=0x7fe12415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe129aca700 1 --2- 172.21.15.5:0/1615421118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415c710 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe124072ef0 tx=0x7fe11c00b330 comp rx=0 tx=0).stop 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe129aca700 1 -- 172.21.15.5:0/1615421118 shutdown_connections 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe129aca700 1 --2- 172.21.15.5:0/1615421118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415c710 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe129aca700 1 --2- 172.21.15.5:0/1615421118 >> v2:172.21.15.49:3300/0 conn(0x7fe1240cd4e0 0x7fe12406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe129aca700 1 -- 172.21.15.5:0/1615421118 >> 172.21.15.5:0/1615421118 conn(0x7fe1240d0320 msgr2=0x7fe1240d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe129aca700 1 -- 172.21.15.5:0/1615421118 shutdown_connections 2024-05-08T23:27:25.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.853+0000 7fe129aca700 1 -- 172.21.15.5:0/1615421118 wait complete. 2024-05-08T23:27:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.855+0000 7fe129aca700 1 Processor -- start 2024-05-08T23:27:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.855+0000 7fe129aca700 1 -- start start 2024-05-08T23:27:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe129aca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe12406f130 0x7fe124159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe129aca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1240cd4e0 0x7fe1240c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe129aca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe129aca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1241f4560 con 0x7fe1240cecf0 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe129aca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240c4c00 con 0x7fe1240cd4e0 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe129aca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240c3ff0 con 0x7fe12406f130 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59608/0 (socket says 172.21.15.5:59608) 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 -- 172.21.15.5:0/76004271 learned_addr learned my addr 172.21.15.5:0/76004271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe122ffd700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1240cd4e0 0x7fe1240c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe1237fe700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe12406f130 0x7fe124159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 -- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe12406f130 msgr2=0x7fe124159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe12406f130 0x7fe124159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 -- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1240cd4e0 msgr2=0x7fe1240c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1240cd4e0 0x7fe1240c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 -- 172.21.15.5:0/76004271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c009b30 con 0x7fe1240cecf0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe1237fe700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe12406f130 0x7fe124159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe123fff700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415a460 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe124070560 tx=0x7fe11c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe11bfff700 1 -- 172.21.15.5:0/76004271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c004830 con 0x7fe1240cecf0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe11bfff700 1 -- 172.21.15.5:0/76004271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c0099b0 con 0x7fe1240cecf0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe11bfff700 1 -- 172.21.15.5:0/76004271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c01a700 con 0x7fe1240cecf0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.856+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1240c42d0 con 0x7fe1240cecf0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.857+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1240c4890 con 0x7fe1240cecf0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.857+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe124068000 con 0x7fe1240cecf0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.858+0000 7fe11bfff700 1 -- 172.21.15.5:0/76004271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe11c01a860 con 0x7fe1240cecf0 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.858+0000 7fe11bfff700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe10406ac90 0x7fe10406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.858+0000 7fe11bfff700 1 -- 172.21.15.5:0/76004271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe11c08c720 con 0x7fe1240cecf0 2024-05-08T23:27:25.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.859+0000 7fe1237fe700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe10406ac90 0x7fe10406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.859+0000 7fe1237fe700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe10406ac90 0x7fe10406d150 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe12415af30 tx=0x7fe114008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:25.863+0000 7fe11bfff700 1 -- 172.21.15.5:0/76004271 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe11c057990 con 0x7fe1240cecf0 2024-05-08T23:27:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.130+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1240c4a50 con 0x7fe1240cecf0 2024-05-08T23:27:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.130+0000 7fe11bfff700 1 -- 172.21.15.5:0/76004271 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4762 (secure 0 0 0) 0x7fe11c05afb0 con 0x7fe1240cecf0 2024-05-08T23:27:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:26.131 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:21.265210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.132+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe10406ac90 msgr2=0x7fe10406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.132+0000 7fe129aca700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe10406ac90 0x7fe10406d150 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe12415af30 tx=0x7fe114008040 comp rx=0 tx=0).stop 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.132+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 msgr2=0x7fe12415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.132+0000 7fe129aca700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415a460 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe124070560 tx=0x7fe11c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.133+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 shutdown_connections 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.133+0000 7fe129aca700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe10406ac90 0x7fe10406d150 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.133+0000 7fe129aca700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1240cecf0 0x7fe12415a460 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.133+0000 7fe129aca700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe12406f130 0x7fe124159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.133+0000 7fe129aca700 1 --2- 172.21.15.5:0/76004271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1240cd4e0 0x7fe1240c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.133+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 >> 172.21.15.5:0/76004271 conn(0x7fe1240d0320 msgr2=0x7fe1240cbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.133+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 shutdown_connections 2024-05-08T23:27:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.133+0000 7fe129aca700 1 -- 172.21.15.5:0/76004271 wait complete. 2024-05-08T23:27:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 89 2024-05-08T23:27:26.146 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-05-08T23:27:26.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:27:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.281+0000 7fe36f309700 1 Processor -- start 2024-05-08T23:27:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.281+0000 7fe36f309700 1 -- start start 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36f309700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe3680d3f40 0x7fe368161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36f309700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe36815db80 0x7fe3680cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36f309700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe368162450 con 0x7fe36815d7a0 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36f309700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3680cee00 con 0x7fe36815db80 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36f309700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3680b27e0 con 0x7fe3680d3f40 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36d8a6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe36815db80 0x7fe3680cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36d0a5700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe36815d7a0 0x7fe3680b0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34110/0 (socket says 172.21.15.5:34110) 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36d0a5700 1 -- 172.21.15.5:0/1165686478 learned_addr learned my addr 172.21.15.5:0/1165686478 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36c8a4700 1 --2- 172.21.15.5:0/1165686478 >> v2:172.21.15.49:3300/0 conn(0x7fe3680d3f40 0x7fe368161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36d8a6700 1 -- 172.21.15.5:0/1165686478 >> v2:172.21.15.49:3300/0 conn(0x7fe3680d3f40 msgr2=0x7fe368161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36d8a6700 1 --2- 172.21.15.5:0/1165686478 >> v2:172.21.15.49:3300/0 conn(0x7fe3680d3f40 0x7fe368161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36d8a6700 1 -- 172.21.15.5:0/1165686478 >> v1:172.21.15.5:6789/0 conn(0x7fe36815d7a0 legacy=0x7fe3680b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.282+0000 7fe36d8a6700 1 -- 172.21.15.5:0/1165686478 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680d4400 con 0x7fe36815db80 2024-05-08T23:27:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.283+0000 7fe36d8a6700 1 --2- 172.21.15.5:0/1165686478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815db80 0x7fe3680cf290 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe3680d0de0 tx=0x7fe36000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf18f627a333950 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.284+0000 7fe35ffff700 1 -- 172.21.15.5:0/1165686478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36000e070 con 0x7fe36815db80 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.284+0000 7fe35ffff700 1 -- 172.21.15.5:0/1165686478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36000bb20 con 0x7fe36815db80 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.284+0000 7fe35ffff700 1 -- 172.21.15.5:0/1165686478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360012430 con 0x7fe36815db80 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.284+0000 7fe36f309700 1 -- 172.21.15.5:0/1165686478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815db80 msgr2=0x7fe3680cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.284+0000 7fe36f309700 1 --2- 172.21.15.5:0/1165686478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815db80 0x7fe3680cf290 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe3680d0de0 tx=0x7fe36000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.285+0000 7fe36f309700 1 -- 172.21.15.5:0/1165686478 shutdown_connections 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.285+0000 7fe36f309700 1 --2- 172.21.15.5:0/1165686478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815db80 0x7fe3680cf290 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.285+0000 7fe36f309700 1 --2- 172.21.15.5:0/1165686478 >> v2:172.21.15.49:3300/0 conn(0x7fe3680d3f40 0x7fe368161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.285+0000 7fe36f309700 1 -- 172.21.15.5:0/1165686478 >> 172.21.15.5:0/1165686478 conn(0x7fe36808afe0 msgr2=0x7fe3680d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.285+0000 7fe36f309700 1 -- 172.21.15.5:0/1165686478 shutdown_connections 2024-05-08T23:27:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.285+0000 7fe36f309700 1 -- 172.21.15.5:0/1165686478 wait complete. 2024-05-08T23:27:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.286+0000 7fe36f309700 1 Processor -- start 2024-05-08T23:27:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.286+0000 7fe36f309700 1 -- start start 2024-05-08T23:27:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36f309700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3680d3f40 0x7fe36814ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36f309700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815d7a0 0x7fe36815b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36f309700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe36815db80 0x7fe36815ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36f309700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680cb130 con 0x7fe36815d7a0 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36f309700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368160040 con 0x7fe36815db80 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36f309700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3680ca790 con 0x7fe3680d3f40 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815d7a0 0x7fe36815b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815d7a0 0x7fe36815b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59620/0 (socket says 172.21.15.5:59620) 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 -- 172.21.15.5:0/1022041100 learned_addr learned my addr 172.21.15.5:0/1022041100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36d8a6700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe36815db80 0x7fe36815ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36d0a5700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3680d3f40 0x7fe36814ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 -- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3680d3f40 msgr2=0x7fe36814ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3680d3f40 0x7fe36814ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 -- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe36815db80 msgr2=0x7fe36815ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe36815db80 0x7fe36815ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 -- 172.21.15.5:0/1022041100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360009a70 con 0x7fe36815d7a0 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36d8a6700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe36815db80 0x7fe36815ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.287+0000 7fe36c8a4700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815d7a0 0x7fe36815b4f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe35800e900 tx=0x7fe35800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.288+0000 7fe35dffb700 1 -- 172.21.15.5:0/1022041100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358009650 con 0x7fe36815d7a0 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.288+0000 7fe35dffb700 1 -- 172.21.15.5:0/1022041100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35800c850 con 0x7fe36815d7a0 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.288+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3680caa70 con 0x7fe36815d7a0 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.288+0000 7fe35dffb700 1 -- 172.21.15.5:0/1022041100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800cbc0 con 0x7fe36815d7a0 2024-05-08T23:27:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.288+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36814cf90 con 0x7fe36815d7a0 2024-05-08T23:27:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.289+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368068000 con 0x7fe36815d7a0 2024-05-08T23:27:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.289+0000 7fe35dffb700 1 -- 172.21.15.5:0/1022041100 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe35800cd20 con 0x7fe36815d7a0 2024-05-08T23:27:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.290+0000 7fe35dffb700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe34c06ac90 0x7fe34c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.290+0000 7fe35dffb700 1 -- 172.21.15.5:0/1022041100 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3580897a0 con 0x7fe36815d7a0 2024-05-08T23:27:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.290+0000 7fe36d0a5700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe34c06ac90 0x7fe34c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.291+0000 7fe36d0a5700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe34c06ac90 0x7fe34c06d150 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe3681f6940 tx=0x7fe364009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.295+0000 7fe35dffb700 1 -- 172.21.15.5:0/1022041100 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe358054970 con 0x7fe36815d7a0 2024-05-08T23:27:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.603+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe3680d0e30 con 0x7fe36815d7a0 2024-05-08T23:27:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.603+0000 7fe35dffb700 1 -- 172.21.15.5:0/1022041100 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1595 (secure 0 0 0) 0x7fe358017650 con 0x7fe36815d7a0 2024-05-08T23:27:26.603 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:26.604 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":401,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":68,"data_bytes":693403,"bytes_used":252055552,"bytes_avail":772842057728,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":8530,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":90,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7336},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7755},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7758}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-05-08T23:25:41.360892+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:27:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe34c06ac90 msgr2=0x7fe34c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe34c06ac90 0x7fe34c06d150 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe3681f6940 tx=0x7fe364009040 comp rx=0 tx=0).stop 2024-05-08T23:27:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815d7a0 msgr2=0x7fe36815b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815d7a0 0x7fe36815b4f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe35800e900 tx=0x7fe35800ecc0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 shutdown_connections 2024-05-08T23:27:26.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe34c06ac90 0x7fe34c06d150 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe36815db80 0x7fe36815ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe3680d3f40 0x7fe36814ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 --2- 172.21.15.5:0/1022041100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe36815d7a0 0x7fe36815b4f0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 >> 172.21.15.5:0/1022041100 conn(0x7fe36808afe0 msgr2=0x7fe36815cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 shutdown_connections 2024-05-08T23:27:26.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:26.606+0000 7fe36f309700 1 -- 172.21.15.5:0/1022041100 wait complete. 2024-05-08T23:27:26.619 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-05-08T23:27:27.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.768+0000 7fc8ae976700 1 Processor -- start 2024-05-08T23:27:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.769+0000 7fc8ae976700 1 -- start start 2024-05-08T23:27:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.769+0000 7fc8ae976700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc8a80cf840 0x7fc8a80af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.769+0000 7fc8ae976700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a806a960 0x7fc8a815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.769+0000 7fc8ae976700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a80cd980 con 0x7fc8a80d0eb0 2024-05-08T23:27:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.769+0000 7fc8ae976700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a80ccdb0 con 0x7fc8a80cf840 2024-05-08T23:27:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.769+0000 7fc8ae976700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80cd050 con 0x7fc8a806a960 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a7fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc8a80cf840 0x7fc8a80af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8acf13700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc8a80d0eb0 0x7fc8a806dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51968/0 (socket says 172.21.15.5:51968) 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8acf13700 1 -- 172.21.15.5:0/3895552656 learned_addr learned my addr 172.21.15.5:0/3895552656 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a77fe700 1 --2- 172.21.15.5:0/3895552656 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a806a960 0x7fc8a815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a6ffd700 1 -- 172.21.15.5:0/3895552656 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309131703 0 0) 0x7fc8a80cd980 con 0x7fc8a80d0eb0 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a6ffd700 1 -- 172.21.15.5:0/3895552656 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8940035b0 con 0x7fc8a80d0eb0 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a7fff700 1 -- 172.21.15.5:0/3895552656 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a806a960 msgr2=0x7fc8a815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a7fff700 1 --2- 172.21.15.5:0/3895552656 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a806a960 0x7fc8a815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a7fff700 1 -- 172.21.15.5:0/3895552656 >> v1:172.21.15.5:6789/0 conn(0x7fc8a80d0eb0 legacy=0x7fc8a806dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a7fff700 1 -- 172.21.15.5:0/3895552656 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80cd330 con 0x7fc8a80cf840 2024-05-08T23:27:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a7fff700 1 --2- 172.21.15.5:0/3895552656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80af440 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc898009870 tx=0x7fc898009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1837222d39b8b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a6ffd700 1 -- 172.21.15.5:0/3895552656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898011830 con 0x7fc8a80cf840 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a6ffd700 1 -- 172.21.15.5:0/3895552656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc898011990 con 0x7fc8a80cf840 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.770+0000 7fc8a6ffd700 1 -- 172.21.15.5:0/3895552656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898011d20 con 0x7fc8a80cf840 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.771+0000 7fc8ae976700 1 -- 172.21.15.5:0/3895552656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 msgr2=0x7fc8a80af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.771+0000 7fc8ae976700 1 --2- 172.21.15.5:0/3895552656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80af440 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc898009870 tx=0x7fc898009b80 comp rx=0 tx=0).stop 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.771+0000 7fc8ae976700 1 -- 172.21.15.5:0/3895552656 shutdown_connections 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.771+0000 7fc8ae976700 1 --2- 172.21.15.5:0/3895552656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80af440 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.771+0000 7fc8ae976700 1 --2- 172.21.15.5:0/3895552656 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a806a960 0x7fc8a815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.771+0000 7fc8ae976700 1 -- 172.21.15.5:0/3895552656 >> 172.21.15.5:0/3895552656 conn(0x7fc8a80d0320 msgr2=0x7fc8a80d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.771+0000 7fc8ae976700 1 -- 172.21.15.5:0/3895552656 shutdown_connections 2024-05-08T23:27:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.771+0000 7fc8ae976700 1 -- 172.21.15.5:0/3895552656 wait complete. 2024-05-08T23:27:27.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.772+0000 7fc8ae976700 1 Processor -- start 2024-05-08T23:27:27.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.773+0000 7fc8ae976700 1 -- start start 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.773+0000 7fc8ae976700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8a806a960 0x7fc8a80b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.773+0000 7fc8ae976700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.773+0000 7fc8ae976700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a80d0eb0 0x7fc8a80afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.773+0000 7fc8ae976700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80b12f0 con 0x7fc8a80cf840 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.773+0000 7fc8ae976700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80b0670 con 0x7fc8a806a960 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.773+0000 7fc8ae976700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80b0950 con 0x7fc8a80d0eb0 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34894/0 (socket says 172.21.15.5:34894) 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 -- 172.21.15.5:0/1545661807 learned_addr learned my addr 172.21.15.5:0/1545661807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8acf13700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a80d0eb0 0x7fc8a80afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a7fff700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8a806a960 0x7fc8a80b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 -- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a80d0eb0 msgr2=0x7fc8a80afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a80d0eb0 0x7fc8a80afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 -- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8a806a960 msgr2=0x7fc8a80b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8a806a960 0x7fc8a80b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 -- 172.21.15.5:0/1545661807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8980094a0 con 0x7fc8a80cf840 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a77fe700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80b4e30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e980 tx=0x7fc89000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a4ff9700 1 -- 172.21.15.5:0/1545661807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890009710 con 0x7fc8a80cf840 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a4ff9700 1 -- 172.21.15.5:0/1545661807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89000c850 con 0x7fc8a80cf840 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a80b0c30 con 0x7fc8a80cf840 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8a4ff9700 1 -- 172.21.15.5:0/1545661807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89000cbe0 con 0x7fc8a80cf840 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.774+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a81580c0 con 0x7fc8a80cf840 2024-05-08T23:27:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.775+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a8068000 con 0x7fc8a80cf840 2024-05-08T23:27:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.776+0000 7fc8a4ff9700 1 -- 172.21.15.5:0/1545661807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc89000cd40 con 0x7fc8a80cf840 2024-05-08T23:27:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.776+0000 7fc8a4ff9700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806ac90 0x7fc88806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.776+0000 7fc8a4ff9700 1 -- 172.21.15.5:0/1545661807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc890089390 con 0x7fc8a80cf840 2024-05-08T23:27:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.777+0000 7fc8a7fff700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806ac90 0x7fc88806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:27.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.777+0000 7fc8a7fff700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806ac90 0x7fc88806d150 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc8980090b0 tx=0x7fc898008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:27.781+0000 7fc8a4ff9700 1 -- 172.21.15.5:0/1545661807 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc8900546c0 con 0x7fc8a80cf840 2024-05-08T23:27:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.049+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8a80b62a0 con 0x7fc8a80cf840 2024-05-08T23:27:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.050+0000 7fc8a4ff9700 1 -- 172.21.15.5:0/1545661807 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4764 (secure 0 0 0) 0x7fc890057ce0 con 0x7fc8a80cf840 2024-05-08T23:27:28.050 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:28.050 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:26.278562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806ac90 msgr2=0x7fc88806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806ac90 0x7fc88806d150 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc8980090b0 tx=0x7fc898008030 comp rx=0 tx=0).stop 2024-05-08T23:27:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 msgr2=0x7fc8a80b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80b4e30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e980 tx=0x7fc89000ed40 comp rx=0 tx=0).stop 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 shutdown_connections 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806ac90 0x7fc88806d150 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc8a80d0eb0 0x7fc8a80afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8a806a960 0x7fc8a80b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 --2- 172.21.15.5:0/1545661807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80cf840 0x7fc8a80b4e30 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 >> 172.21.15.5:0/1545661807 conn(0x7fc8a80d0320 msgr2=0x7fc8a815c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 shutdown_connections 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.052+0000 7fc8ae976700 1 -- 172.21.15.5:0/1545661807 wait complete. 2024-05-08T23:27:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 90 2024-05-08T23:27:28.066 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-05-08T23:27:28.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:27:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.200+0000 7ff026e57700 1 Processor -- start 2024-05-08T23:27:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.201+0000 7ff026e57700 1 -- start start 2024-05-08T23:27:28.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.201+0000 7ff026e57700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 0x7ff020161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.201+0000 7ff026e57700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff02015db80 0x7ff0200cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.201+0000 7ff026e57700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff020162450 con 0x7ff02015d7a0 2024-05-08T23:27:28.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.201+0000 7ff026e57700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0200cee00 con 0x7ff02015db80 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.201+0000 7ff026e57700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200b27e0 con 0x7ff0200d3f40 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff0253f4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff02015db80 0x7ff0200cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff024bf3700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff02015d7a0 0x7ff0200b0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51976/0 (socket says 172.21.15.5:51976) 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff024bf3700 1 -- 172.21.15.5:0/439542467 learned_addr learned my addr 172.21.15.5:0/439542467 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff01ffff700 1 --2- 172.21.15.5:0/439542467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 0x7ff020161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff0253f4700 1 -- 172.21.15.5:0/439542467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 msgr2=0x7ff020161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff0253f4700 1 --2- 172.21.15.5:0/439542467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 0x7ff020161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff0253f4700 1 -- 172.21.15.5:0/439542467 >> v1:172.21.15.5:6789/0 conn(0x7ff02015d7a0 legacy=0x7ff0200b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff0253f4700 1 -- 172.21.15.5:0/439542467 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0200d4400 con 0x7ff02015db80 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff01f7fe700 1 -- 172.21.15.5:0/439542467 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366617806 0 0) 0x7ff010001110 con 0x7ff02015d7a0 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff0253f4700 1 --2- 172.21.15.5:0/439542467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff0200cf290 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff018009d90 tx=0x7ff01800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d19299ba9ce75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff01f7fe700 1 -- 172.21.15.5:0/439542467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01800e070 con 0x7ff02015db80 2024-05-08T23:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff01f7fe700 1 -- 172.21.15.5:0/439542467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01800bb20 con 0x7ff02015db80 2024-05-08T23:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.202+0000 7ff01f7fe700 1 -- 172.21.15.5:0/439542467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018012430 con 0x7ff02015db80 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.203+0000 7ff026e57700 1 -- 172.21.15.5:0/439542467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 msgr2=0x7ff0200cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.203+0000 7ff026e57700 1 --2- 172.21.15.5:0/439542467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff0200cf290 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff018009d90 tx=0x7ff01800b330 comp rx=0 tx=0).stop 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.203+0000 7ff026e57700 1 -- 172.21.15.5:0/439542467 shutdown_connections 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.203+0000 7ff026e57700 1 --2- 172.21.15.5:0/439542467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff0200cf290 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.203+0000 7ff026e57700 1 --2- 172.21.15.5:0/439542467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 0x7ff020161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.203+0000 7ff026e57700 1 -- 172.21.15.5:0/439542467 >> 172.21.15.5:0/439542467 conn(0x7ff02008afe0 msgr2=0x7ff0200d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.203+0000 7ff026e57700 1 -- 172.21.15.5:0/439542467 shutdown_connections 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.203+0000 7ff026e57700 1 -- 172.21.15.5:0/439542467 wait complete. 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.204+0000 7ff026e57700 1 Processor -- start 2024-05-08T23:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.204+0000 7ff026e57700 1 -- start start 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.205+0000 7ff026e57700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 0x7ff02015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.205+0000 7ff026e57700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff02015d7a0 0x7ff0200d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.205+0000 7ff026e57700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff02014c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.205+0000 7ff026e57700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02015df90 con 0x7ff02015db80 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.205+0000 7ff026e57700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02015e830 con 0x7ff02015d7a0 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.205+0000 7ff026e57700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200c9da0 con 0x7ff0200d3f40 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff02014c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff02014c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34906/0 (socket says 172.21.15.5:34906) 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 -- 172.21.15.5:0/4047414231 learned_addr learned my addr 172.21.15.5:0/4047414231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff01ffff700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff02015d7a0 0x7ff0200d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff024bf3700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 0x7ff02015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 -- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 msgr2=0x7ff02015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 0x7ff02015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 -- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff02015d7a0 msgr2=0x7ff0200d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff02015d7a0 0x7ff0200d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 -- 172.21.15.5:0/4047414231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff018009a70 con 0x7ff02015db80 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff0253f4700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff02014c5b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff02008ba90 tx=0x7ff018004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff01d7fa700 1 -- 172.21.15.5:0/4047414231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018012430 con 0x7ff02015db80 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff01d7fa700 1 -- 172.21.15.5:0/4047414231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018020070 con 0x7ff02015db80 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02015e1e0 con 0x7ff02015db80 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff01d7fa700 1 -- 172.21.15.5:0/4047414231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018004280 con 0x7ff02015db80 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.206+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0200d2100 con 0x7ff02015db80 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.207+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0200d22c0 con 0x7ff02015db80 2024-05-08T23:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.208+0000 7ff01d7fa700 1 -- 172.21.15.5:0/4047414231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff0180043e0 con 0x7ff02015db80 2024-05-08T23:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.208+0000 7ff01d7fa700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff00c06ac90 0x7ff00c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.208+0000 7ff01d7fa700 1 -- 172.21.15.5:0/4047414231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff018022070 con 0x7ff02015db80 2024-05-08T23:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.209+0000 7ff024bf3700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff00c06ac90 0x7ff00c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.209+0000 7ff024bf3700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff00c06ac90 0x7ff00c06d150 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff010000f60 tx=0x7ff010007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.214+0000 7ff01d7fa700 1 -- 172.21.15.5:0/4047414231 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff018056620 con 0x7ff02015db80 2024-05-08T23:27:28.283 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:27:28.283+0000 7f8a4c50b700 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T23:27:28.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.530+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff0200b2490 con 0x7ff02015db80 2024-05-08T23:27:28.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.530+0000 7ff01d7fa700 1 -- 172.21.15.5:0/4047414231 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1965 (secure 0 0 0) 0x7ff018059c40 con 0x7ff02015db80 2024-05-08T23:27:28.531 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:28.531 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"2 MDSs report damaged metadata","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":403,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700348,"bytes_used":256286720,"bytes_avail":772837826560,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9896,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":91,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7336},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7755},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7758}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-05-08T23:27:27.387490+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:27:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.532+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff00c06ac90 msgr2=0x7ff00c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.532+0000 7ff026e57700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff00c06ac90 0x7ff00c06d150 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff010000f60 tx=0x7ff010007c90 comp rx=0 tx=0).stop 2024-05-08T23:27:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 msgr2=0x7ff02014c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff02014c5b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff02008ba90 tx=0x7ff018004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 shutdown_connections 2024-05-08T23:27:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff00c06ac90 0x7ff00c06d150 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff02015db80 0x7ff02014c5b0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0200d3f40 0x7ff02015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 --2- 172.21.15.5:0/4047414231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff02015d7a0 0x7ff0200d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 >> 172.21.15.5:0/4047414231 conn(0x7ff02008afe0 msgr2=0x7ff02015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 shutdown_connections 2024-05-08T23:27:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.533+0000 7ff026e57700 1 -- 172.21.15.5:0/4047414231 wait complete. 2024-05-08T23:27:28.546 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:27:28.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub resume 2024-05-08T23:27:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.685+0000 7ff03e73d700 1 Processor -- start 2024-05-08T23:27:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.685+0000 7ff03e73d700 1 -- start start 2024-05-08T23:27:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff03e73d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff0381580a0 0x7ff03806d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff03e73d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff03806c080 0x7ff03806da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff03e73d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03806e3f0 con 0x7ff038157cc0 2024-05-08T23:27:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff03e73d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0380cf7f0 con 0x7ff0381580a0 2024-05-08T23:27:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff03e73d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03806e110 con 0x7ff03806c080 2024-05-08T23:27:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff0377fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff0381580a0 0x7ff03806d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff03ccda700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff03806c080 0x7ff03806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff0377fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff0381580a0 0x7ff03806d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58094/0 (socket says 172.21.15.5:58094) 2024-05-08T23:27:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff03ccda700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff03806c080 0x7ff03806da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:57926/0 (socket says 172.21.15.5:57926) 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.686+0000 7ff0377fe700 1 -- 172.21.15.5:0/2758316849 learned_addr learned my addr 172.21.15.5:0/2758316849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff0377fe700 1 -- 172.21.15.5:0/2758316849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff03806c080 msgr2=0x7ff03806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff0377fe700 1 --2- 172.21.15.5:0/2758316849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff03806c080 0x7ff03806da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff0377fe700 1 -- 172.21.15.5:0/2758316849 >> v1:172.21.15.49:6789/0 conn(0x7ff038157cc0 legacy=0x7ff03806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff0377fe700 1 -- 172.21.15.5:0/2758316849 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03815e020 con 0x7ff0381580a0 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff03ccda700 1 --2- 172.21.15.5:0/2758316849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff03806c080 0x7ff03806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff0377fe700 1 --2- 172.21.15.5:0/2758316849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0381580a0 0x7ff03806d500 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0098f0 tx=0x7ff02c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72351a2eafc11a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff036ffd700 1 -- 172.21.15.5:0/2758316849 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c004230 con 0x7ff0381580a0 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff036ffd700 1 -- 172.21.15.5:0/2758316849 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c004390 con 0x7ff0381580a0 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.687+0000 7ff036ffd700 1 -- 172.21.15.5:0/2758316849 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0036a0 con 0x7ff0381580a0 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.688+0000 7ff03e73d700 1 -- 172.21.15.5:0/2758316849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0381580a0 msgr2=0x7ff03806d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.688+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2758316849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0381580a0 0x7ff03806d500 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0098f0 tx=0x7ff02c009c00 comp rx=0 tx=0).stop 2024-05-08T23:27:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.688+0000 7ff03e73d700 1 -- 172.21.15.5:0/2758316849 shutdown_connections 2024-05-08T23:27:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.688+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2758316849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff03806c080 0x7ff03806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.688+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2758316849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0381580a0 0x7ff03806d500 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.688+0000 7ff03e73d700 1 -- 172.21.15.5:0/2758316849 >> 172.21.15.5:0/2758316849 conn(0x7ff0380713c0 msgr2=0x7ff0380717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.688+0000 7ff03e73d700 1 -- 172.21.15.5:0/2758316849 shutdown_connections 2024-05-08T23:27:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.688+0000 7ff03e73d700 1 -- 172.21.15.5:0/2758316849 wait complete. 2024-05-08T23:27:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.690+0000 7ff03e73d700 1 Processor -- start 2024-05-08T23:27:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.690+0000 7ff03e73d700 1 -- start start 2024-05-08T23:27:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff03e73d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff03806c080 0x7ff03815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff03e73d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038157cc0 0x7ff0381f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff03e73d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0381580a0 0x7ff038155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff03e73d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038146fc0 con 0x7ff03806c080 2024-05-08T23:27:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff03e73d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0381472a0 con 0x7ff038157cc0 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff03e73d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff038147580 con 0x7ff0381580a0 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff037fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff03806c080 0x7ff03815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff037fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff03806c080 0x7ff03815d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34916/0 (socket says 172.21.15.5:34916) 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff037fff700 1 -- 172.21.15.5:0/2871980108 learned_addr learned my addr 172.21.15.5:0/2871980108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff0377fe700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038157cc0 0x7ff0381f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff03ccda700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0381580a0 0x7ff038155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.691+0000 7ff037fff700 1 -- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0381580a0 msgr2=0x7ff038155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff037fff700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0381580a0 0x7ff038155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff037fff700 1 -- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038157cc0 msgr2=0x7ff0381f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff037fff700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038157cc0 0x7ff0381f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff037fff700 1 -- 172.21.15.5:0/2871980108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030009a70 con 0x7ff03806c080 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff037fff700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff03806c080 0x7ff03815d4f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff02800c750 tx=0x7ff02800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff034ff9700 1 -- 172.21.15.5:0/2871980108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0280054e0 con 0x7ff03806c080 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff03e73d700 1 -- 172.21.15.5:0/2871980108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c009550 con 0x7ff03806c080 2024-05-08T23:27:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff034ff9700 1 -- 172.21.15.5:0/2871980108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff028003680 con 0x7ff03806c080 2024-05-08T23:27:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff034ff9700 1 -- 172.21.15.5:0/2871980108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028003a30 con 0x7ff03806c080 2024-05-08T23:27:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.692+0000 7ff03e73d700 1 -- 172.21.15.5:0/2871980108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff038146c20 con 0x7ff03806c080 2024-05-08T23:27:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.694+0000 7ff034ff9700 1 -- 172.21.15.5:0/2871980108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff028003b90 con 0x7ff03806c080 2024-05-08T23:27:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.694+0000 7ff034ff9700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff02406ac90 0x7ff02406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.694+0000 7ff034ff9700 1 -- 172.21.15.5:0/2871980108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff028088880 con 0x7ff03806c080 2024-05-08T23:27:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.694+0000 7ff0377fe700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff02406ac90 0x7ff02406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.695+0000 7ff0377fe700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff02406ac90 0x7ff02406d150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009130 tx=0x7ff02c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.696+0000 7ff04a8f7b80 1 Processor -- start 2024-05-08T23:27:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.696+0000 7ff04a8f7b80 1 -- start start 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed14697a0 0x55aed1442110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed14470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed1471250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed139d150 con 0x55aed1444370 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed139ab10 con 0x55aed14697a0 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aed1396400 con 0x55aed1470dd0 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff0377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed14470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff0377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed14470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34918/0 (socket says 172.21.15.5:34918) 2024-05-08T23:27:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff0377fe700 1 -- 172.21.15.5:0/3260520935 learned_addr learned my addr 172.21.15.5:0/3260520935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff037fff700 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed14697a0 0x55aed1442110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff03ccda700 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed1471250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff0377fe700 1 -- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 msgr2=0x55aed1471250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.697+0000 7ff0377fe700 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed1471250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff0377fe700 1 -- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed14697a0 msgr2=0x55aed1442110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff0377fe700 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed14697a0 0x55aed1442110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff0377fe700 1 -- 172.21.15.5:0/3260520935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed12173b0 con 0x55aed1444370 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff037fff700 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed14697a0 0x55aed1442110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff0377fe700 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed14470a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff038066cc0 tx=0x7ff02c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd95c997b37881b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff01dffb700 1 -- 172.21.15.5:0/3260520935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c020070 con 0x55aed1444370 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff01dffb700 1 -- 172.21.15.5:0/3260520935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c0045d0 con 0x55aed1444370 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff01dffb700 1 -- 172.21.15.5:0/3260520935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c012640 con 0x55aed1444370 2024-05-08T23:27:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/3260520935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 msgr2=0x55aed14470a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.698+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed14470a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff038066cc0 tx=0x7ff02c010000 comp rx=0 tx=0).stop 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.699+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/3260520935 shutdown_connections 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.699+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed1471250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.699+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed14697a0 0x55aed1442110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.699+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/3260520935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed14470a0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.699+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/3260520935 >> 172.21.15.5:0/3260520935 conn(0x55aed13a4210 msgr2=0x55aed148bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.699+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/3260520935 shutdown_connections 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.699+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/3260520935 wait complete. 2024-05-08T23:27:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.699+0000 7ff04a8f7b80 1 -- start start 2024-05-08T23:27:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.700+0000 7ff04a8f7b80 1 Processor -- start 2024-05-08T23:27:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 0x55aed148adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed1467980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed1332560 con 0x55aed14697a0 2024-05-08T23:27:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed1332200 con 0x55aed1444370 2024-05-08T23:27:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aed12196a0 con 0x55aed1470dd0 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff04a8f7b80 10 client.0 _ll_register_callbacks cb 0x55aed133bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff0377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff0377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34926/0 (socket says 172.21.15.5:34926) 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff0377fe700 1 -- 172.21.15.5:0/438201251 learned_addr learned my addr 172.21.15.5:0/438201251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff03ccda700 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed1467980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff0377fe700 1 -- 172.21.15.5:0/438201251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 msgr2=0x55aed1467980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff0377fe700 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed1467980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff0377fe700 1 -- 172.21.15.5:0/438201251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 msgr2=0x55aed148adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff037fff700 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 0x55aed148adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff0377fe700 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 0x55aed148adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.701+0000 7ff0377fe700 1 -- 172.21.15.5:0/438201251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff038146c20 con 0x55aed14697a0 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff0377fe700 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467440 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009f50 tx=0x7ff02c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e7d68165ca540e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff016ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff016ffd700 1 -- 172.21.15.5:0/438201251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c012a50 con 0x55aed14697a0 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff016ffd700 1 -- 172.21.15.5:0/438201251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff02c012bb0 con 0x55aed14697a0 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/438201251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aed14157e0 con 0x55aed14697a0 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff016ffd700 1 -- 172.21.15.5:0/438201251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02c012d70 con 0x55aed14697a0 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 10 client.8250 fetch_fsmap learned FSMap version 91 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/438201251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x7ff03815e020 con 0x55aed14697a0 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff016ffd700 1 -- 172.21.15.5:0/438201251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aed14157e0 con 0x55aed14697a0 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff016ffd700 1 -- 172.21.15.5:0/438201251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 91) v1 ==== 3154+0+0 (secure 0 0 0) 0x7ff02c032430 con 0x55aed14697a0 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 10 client.8250 fetch_fsmap finished waiting for FSMap version 91 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 10 client.8250 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 20 client.8250 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55aed148ff70 0x55aed1489d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 4 client.8250 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.702+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/438201251 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aed138eec0 con 0x55aed148ff70 2024-05-08T23:27:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.703+0000 7ff037fff700 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55aed148ff70 0x55aed1489d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.703+0000 7ff037fff700 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55aed148ff70 0x55aed1489d40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a317f8b3e93472de server_cookie=bdeb4be9cddf7fa4 in_seq=0 out_seq=0 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.703+0000 7ff016ffd700 10 client.8250 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.703+0000 7ff016ffd700 1 -- 172.21.15.5:0/438201251 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aed138eec0 con 0x55aed148ff70 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.703+0000 7ff016ffd700 10 client.8250 handle_command_reply: tid=0 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.703+0000 7ff04a8f7b80 1 client.8250 shutdown 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/438201251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 msgr2=0x55aed1467440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467440 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009f50 tx=0x7ff02c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/438201251 shutdown_connections 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55aed148ff70 0x55aed1489d40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed1467980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 0x55aed148adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff016ffd700 0 client.8250 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/438201251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467440 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/438201251 >> 172.21.15.5:0/438201251 conn(0x55aed13a4210 msgr2=0x55aed143fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/438201251 shutdown_connections 2024-05-08T23:27:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/438201251 wait complete. 2024-05-08T23:27:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.704+0000 7ff04a8f7b80 20 client.8250 trim_cache size 0 max 16384 2024-05-08T23:27:28.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.722+0000 7ff04a8f7b80 1 Processor -- start 2024-05-08T23:27:28.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.722+0000 7ff04a8f7b80 1 -- start start 2024-05-08T23:27:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.722+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed143d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.722+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed14697a0 0x55aed14ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.722+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1470dd0 0x55aed149e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.722+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed1446360 con 0x55aed1444370 2024-05-08T23:27:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.722+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed14680b0 con 0x55aed1470dd0 2024-05-08T23:27:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.722+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aed1441c20 con 0x55aed14697a0 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed143d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed143d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34938/0 (socket says 172.21.15.5:34938) 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 -- 172.21.15.5:0/4111335324 learned_addr learned my addr 172.21.15.5:0/4111335324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff03ccda700 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1470dd0 0x55aed149e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff0377fe700 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed14697a0 0x55aed14ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 -- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed14697a0 msgr2=0x55aed14ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed14697a0 0x55aed14ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 -- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1470dd0 msgr2=0x55aed149e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1470dd0 0x55aed149e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 -- 172.21.15.5:0/4111335324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed11c3b60 con 0x55aed1444370 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff0377fe700 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed14697a0 0x55aed14ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff037fff700 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed143d390 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff02800cdb0 tx=0x7ff028076040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e898ca2b6bc7e48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff01e7fc700 1 -- 172.21.15.5:0/4111335324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02806f180 con 0x55aed1444370 2024-05-08T23:27:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff01e7fc700 1 -- 172.21.15.5:0/4111335324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02805b560 con 0x55aed1444370 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.723+0000 7ff01e7fc700 1 -- 172.21.15.5:0/4111335324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02805fb20 con 0x55aed1444370 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/4111335324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 msgr2=0x55aed143d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed143d390 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff02800cdb0 tx=0x7ff028076040 comp rx=0 tx=0).stop 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/4111335324 shutdown_connections 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1470dd0 0x55aed149e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed1444370 0x55aed143d390 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/4111335324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed14697a0 0x55aed14ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/4111335324 >> 172.21.15.5:0/4111335324 conn(0x55aed13a4210 msgr2=0x55aed1440b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/4111335324 shutdown_connections 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.724+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/4111335324 wait complete. 2024-05-08T23:27:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.725+0000 7ff04a8f7b80 1 -- start start 2024-05-08T23:27:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.725+0000 7ff04a8f7b80 1 Processor -- start 2024-05-08T23:27:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 0x55aed1467250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff04a8f7b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed14f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed1445e00 con 0x55aed14697a0 2024-05-08T23:27:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed1445a40 con 0x55aed1444370 2024-05-08T23:27:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff04a8f7b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aed14f68e0 con 0x55aed1470dd0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff04a8f7b80 10 client.0 _ll_register_callbacks cb 0x55aed133bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34950/0 (socket says 172.21.15.5:34950) 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 -- 172.21.15.5:0/2401466255 learned_addr learned my addr 172.21.15.5:0/2401466255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff037fff700 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 0x55aed1467250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff03ccda700 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed14f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 -- 172.21.15.5:0/2401466255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 msgr2=0x55aed14f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed14f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 -- 172.21.15.5:0/2401466255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 msgr2=0x55aed1467250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 0x55aed1467250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 -- 172.21.15.5:0/2401466255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c009550 con 0x55aed14697a0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff0377fe700 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467790 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff02c000c00 tx=0x7ff02c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9a6ce7e0619466 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff016ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.726+0000 7ff016ffd700 1 -- 172.21.15.5:0/2401466255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c003f60 con 0x55aed14697a0 2024-05-08T23:27:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff016ffd700 1 -- 172.21.15.5:0/2401466255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff02c032940 con 0x55aed14697a0 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/2401466255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aed12ef270 con 0x55aed14697a0 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff016ffd700 1 -- 172.21.15.5:0/2401466255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02c005a20 con 0x55aed14697a0 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 10 client.8256 fetch_fsmap learned FSMap version 91 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/2401466255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55aed135c560 con 0x55aed14697a0 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff016ffd700 1 -- 172.21.15.5:0/2401466255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c04b020 con 0x55aed14697a0 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff016ffd700 1 -- 172.21.15.5:0/2401466255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 91) v1 ==== 3154+0+0 (secure 0 0 0) 0x7ff02c005b80 con 0x55aed14697a0 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 10 client.8256 fetch_fsmap finished waiting for FSMap version 91 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 10 client.8256 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 20 client.8256 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55aed14f4dc0 0x55aed14f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 4 client.8256 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/2401466255 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55aed11e4cf0 con 0x55aed14f4dc0 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff037fff700 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55aed14f4dc0 0x55aed14f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.727+0000 7ff016ffd700 10 client.8256 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff037fff700 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55aed14f4dc0 0x55aed14f59d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9eaee6355e84d19 server_cookie=2a1a448d1caca63e in_seq=0 out_seq=0 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff016ffd700 1 -- 172.21.15.5:0/2401466255 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55aed11e4cf0 con 0x55aed14f4dc0 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff016ffd700 10 client.8256 handle_command_reply: tid=0 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 client.8256 shutdown 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/2401466255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 msgr2=0x55aed1467790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467790 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff02c000c00 tx=0x7ff02c011000 comp rx=0 tx=0).stop 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/2401466255 shutdown_connections 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55aed14f4dc0 0x55aed14f59d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55aed1470dd0 0x55aed14f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aed1444370 0x55aed1467250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 --2- 172.21.15.5:0/2401466255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aed14697a0 0x55aed1467790 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/2401466255 >> 172.21.15.5:0/2401466255 conn(0x55aed13a4210 msgr2=0x55aed143fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff016ffd700 0 client.8256 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/2401466255 shutdown_connections 2024-05-08T23:27:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 1 -- 172.21.15.5:0/2401466255 wait complete. 2024-05-08T23:27:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.728+0000 7ff04a8f7b80 20 client.8256 trim_cache size 0 max 16384 2024-05-08T23:27:28.734 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:28.734 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:27:28.734 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 -- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff02406ac90 msgr2=0x7ff02406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff02406ac90 0x7ff02406d150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009130 tx=0x7ff02c008030 comp rx=0 tx=0).stop 2024-05-08T23:27:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 -- 172.21.15.5:0/2871980108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff03806c080 msgr2=0x7ff03815d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff03806c080 0x7ff03815d4f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff02800c750 tx=0x7ff02800ca60 comp rx=0 tx=0).stop 2024-05-08T23:27:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 -- 172.21.15.5:0/2871980108 shutdown_connections 2024-05-08T23:27:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff02406ac90 0x7ff02406d150 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0381580a0 0x7ff038155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff03806c080 0x7ff03815d4f0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 --2- 172.21.15.5:0/2871980108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038157cc0 0x7ff0381f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 -- 172.21.15.5:0/2871980108 >> 172.21.15.5:0/2871980108 conn(0x7ff0380713c0 msgr2=0x7ff0380b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 -- 172.21.15.5:0/2871980108 shutdown_connections 2024-05-08T23:27:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.729+0000 7ff03e73d700 1 -- 172.21.15.5:0/2871980108 wait complete. 2024-05-08T23:27:28.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 scrub status 2024-05-08T23:27:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.880+0000 7f5856413700 1 Processor -- start 2024-05-08T23:27:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.880+0000 7f5856413700 1 -- start start 2024-05-08T23:27:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f5856413700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500d3d20 0x7f58500b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f5856413700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58500b3fc0 0x7f58500b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f5856413700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58500cf610 con 0x7f58500cff30 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f5856413700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58500d1fc0 con 0x7f58500b3fc0 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f5856413700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58500d22a0 con 0x7f58500d3d20 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f584ffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f58500cff30 0x7f58500d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51992/0 (socket says 172.21.15.5:51992) 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f584ffff700 1 -- 172.21.15.5:0/2970165994 learned_addr learned my addr 172.21.15.5:0/2970165994 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f58549b0700 1 --2- 172.21.15.5:0/2970165994 >> v2:172.21.15.49:3300/0 conn(0x7f58500b3fc0 0x7f58500b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f584f7fe700 1 --2- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500d3d20 0x7f58500b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f584effd700 1 -- 172.21.15.5:0/2970165994 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147580255 0 0) 0x7f58500cf610 con 0x7f58500cff30 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.881+0000 7f584effd700 1 -- 172.21.15.5:0/2970165994 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f583c0035b0 con 0x7f58500cff30 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f58549b0700 1 -- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500d3d20 msgr2=0x7f58500b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f584effd700 1 -- 172.21.15.5:0/2970165994 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731201515 0 0) 0x7f583c0035b0 con 0x7f58500cff30 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f58549b0700 1 --2- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500d3d20 0x7f58500b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f58549b0700 1 -- 172.21.15.5:0/2970165994 >> v1:172.21.15.5:6789/0 conn(0x7f58500cff30 legacy=0x7f58500d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f58549b0700 1 -- 172.21.15.5:0/2970165994 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58500d2580 con 0x7f58500b3fc0 2024-05-08T23:27:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f584f7fe700 1 --2- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500d3d20 0x7f58500b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f58549b0700 1 --2- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500b3fc0 0x7f58500b4450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5848009d90 tx=0x7f584800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ed43f61460578e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f584effd700 1 -- 172.21.15.5:0/2970165994 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800e070 con 0x7f58500b3fc0 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f584effd700 1 -- 172.21.15.5:0/2970165994 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5848003a20 con 0x7f58500b3fc0 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.882+0000 7f584effd700 1 -- 172.21.15.5:0/2970165994 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840001670 con 0x7f58500b3fc0 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.883+0000 7f5856413700 1 -- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500b3fc0 msgr2=0x7f58500b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.883+0000 7f5856413700 1 --2- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500b3fc0 0x7f58500b4450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5848009d90 tx=0x7f584800b330 comp rx=0 tx=0).stop 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.883+0000 7f5856413700 1 -- 172.21.15.5:0/2970165994 shutdown_connections 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.883+0000 7f5856413700 1 --2- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500b3fc0 0x7f58500b4450 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.883+0000 7f5856413700 1 --2- 172.21.15.5:0/2970165994 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500d3d20 0x7f58500b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.883+0000 7f5856413700 1 -- 172.21.15.5:0/2970165994 >> 172.21.15.5:0/2970165994 conn(0x7f585008afe0 msgr2=0x7f58500d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.883+0000 7f5856413700 1 -- 172.21.15.5:0/2970165994 shutdown_connections 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.883+0000 7f5856413700 1 -- 172.21.15.5:0/2970165994 wait complete. 2024-05-08T23:27:28.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.884+0000 7f5856413700 1 Processor -- start 2024-05-08T23:27:28.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.884+0000 7f5856413700 1 -- start start 2024-05-08T23:27:28.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f5856413700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58500b3fc0 0x7f585015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f5856413700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500cff30 0x7f58500cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f5856413700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500d3d20 0x7f58500b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f5856413700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500cbdf0 con 0x7f58500b3fc0 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f5856413700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500cb220 con 0x7f58500d3d20 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f5856413700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58500cb4c0 con 0x7f58500cff30 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f584ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58500b3fc0 0x7f585015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f584ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58500b3fc0 0x7f585015aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34956/0 (socket says 172.21.15.5:34956) 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f584ffff700 1 -- 172.21.15.5:0/3440881182 learned_addr learned my addr 172.21.15.5:0/3440881182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.885+0000 7f584f7fe700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500cff30 0x7f58500cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f58549b0700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500d3d20 0x7f58500b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584ffff700 1 -- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500cff30 msgr2=0x7f58500cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584ffff700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500cff30 0x7f58500cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584ffff700 1 -- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500d3d20 msgr2=0x7f58500b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584ffff700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500d3d20 0x7f58500b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584ffff700 1 -- 172.21.15.5:0/3440881182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380094a0 con 0x7f58500b3fc0 2024-05-08T23:27:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f58549b0700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500d3d20 0x7f58500b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584ffff700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58500b3fc0 0x7f585015aeb0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f58400011e0 tx=0x7f584000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584cff9700 1 -- 172.21.15.5:0/3440881182 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840007560 con 0x7f58500b3fc0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584cff9700 1 -- 172.21.15.5:0/3440881182 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58400076c0 con 0x7f58500b3fc0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f5856413700 1 -- 172.21.15.5:0/3440881182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5848009a70 con 0x7f58500b3fc0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f584cff9700 1 -- 172.21.15.5:0/3440881182 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840004a90 con 0x7f58500b3fc0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.886+0000 7f5856413700 1 -- 172.21.15.5:0/3440881182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58500cba80 con 0x7f58500b3fc0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.887+0000 7f584cff9700 1 -- 172.21.15.5:0/3440881182 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5840004bf0 con 0x7f58500b3fc0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.888+0000 7f584cff9700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f583006ac90 0x7f583006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.888+0000 7f584cff9700 1 -- 172.21.15.5:0/3440881182 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5840088740 con 0x7f58500b3fc0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.888+0000 7f584f7fe700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f583006ac90 0x7f583006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.889+0000 7f584f7fe700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f583006ac90 0x7f583006d150 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f58500b7760 tx=0x7f5838008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.891+0000 7f58625cdb80 1 Processor -- start 2024-05-08T23:27:28.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.891+0000 7f58625cdb80 1 -- start start 2024-05-08T23:27:28.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98f810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98fab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58625cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c98ed7150 con 0x558c98faadd0 2024-05-08T23:27:28.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58625cdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c98ed4b10 con 0x558c98f7e370 2024-05-08T23:27:28.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58625cdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c98ed0400 con 0x558c98fa37a0 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58549b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98fab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58549b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98fab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34970/0 (socket says 172.21.15.5:34970) 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58549b0700 1 -- 172.21.15.5:0/1064683255 learned_addr learned my addr 172.21.15.5:0/1064683255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f584ffff700 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f584f7fe700 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98f810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58549b0700 1 -- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 msgr2=0x558c98f7c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58549b0700 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58549b0700 1 -- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 msgr2=0x558c98f810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58549b0700 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98f810a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f58549b0700 1 -- 172.21.15.5:0/1064683255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c98d513b0 con 0x558c98faadd0 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.892+0000 7f584ffff700 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f58549b0700 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98fab250 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f585008bb00 tx=0x7f5848010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6f753b6b36abba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f5845ffb700 1 -- 172.21.15.5:0/1064683255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800ba20 con 0x558c98faadd0 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f5845ffb700 1 -- 172.21.15.5:0/1064683255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584800bb80 con 0x558c98faadd0 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f5845ffb700 1 -- 172.21.15.5:0/1064683255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848004030 con 0x558c98faadd0 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f58625cdb80 1 -- 172.21.15.5:0/1064683255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 msgr2=0x558c98fab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f58625cdb80 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98fab250 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f585008bb00 tx=0x7f5848010000 comp rx=0 tx=0).stop 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f58625cdb80 1 -- 172.21.15.5:0/1064683255 shutdown_connections 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f58625cdb80 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98fab250 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f58625cdb80 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f58625cdb80 1 --2- 172.21.15.5:0/1064683255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98f810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.893+0000 7f58625cdb80 1 -- 172.21.15.5:0/1064683255 >> 172.21.15.5:0/1064683255 conn(0x558c98ede210 msgr2=0x558c98fc5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.894+0000 7f58625cdb80 1 -- 172.21.15.5:0/1064683255 shutdown_connections 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.894+0000 7f58625cdb80 1 -- 172.21.15.5:0/1064683255 wait complete. 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.894+0000 7f58625cdb80 1 -- start start 2024-05-08T23:27:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.895+0000 7f58625cdb80 1 Processor -- start 2024-05-08T23:27:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.895+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98fd6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.895+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98fa37a0 0x558c98fabf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.895+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98faadd0 0x558c98fd7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.895+0000 7f58625cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c98e6c560 con 0x558c98fa37a0 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.895+0000 7f58625cdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c98e6c200 con 0x558c98f7e370 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.895+0000 7f58625cdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c98d536a0 con 0x558c98faadd0 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.895+0000 7f58625cdb80 10 client.0 _ll_register_callbacks cb 0x558c98e75b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98fa37a0 0x558c98fabf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98fa37a0 0x558c98fabf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34982/0 (socket says 172.21.15.5:34982) 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 -- 172.21.15.5:0/2425548039 learned_addr learned my addr 172.21.15.5:0/2425548039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584ffff700 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98fd6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f58549b0700 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98faadd0 0x558c98fd7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 -- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98faadd0 msgr2=0x558c98fd7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98faadd0 0x558c98fd7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 -- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 msgr2=0x558c98fd6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98fd6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 -- 172.21.15.5:0/2425548039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58500d2580 con 0x558c98fa37a0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584ffff700 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98fd6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f584f7fe700 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98fa37a0 0x558c98fabf70 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5838015750 tx=0x7f58380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da572c7ab8f93d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f582effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f582effd700 1 -- 172.21.15.5:0/2425548039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838004440 con 0x558c98fa37a0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f582effd700 1 -- 172.21.15.5:0/2425548039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58380036a0 con 0x558c98fa37a0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f58625cdb80 1 -- 172.21.15.5:0/2425548039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c98e29270 con 0x558c98fa37a0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f582effd700 1 -- 172.21.15.5:0/2425548039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58380045a0 con 0x558c98fa37a0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.896+0000 7f58625cdb80 10 client.8268 fetch_fsmap learned FSMap version 91 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f58625cdb80 1 -- 172.21.15.5:0/2425548039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x558c98cfdb60 con 0x558c98fa37a0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f582effd700 1 -- 172.21.15.5:0/2425548039 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c98e29270 con 0x558c98fa37a0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f582effd700 1 -- 172.21.15.5:0/2425548039 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 91) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f5838003860 con 0x558c98fa37a0 2024-05-08T23:27:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f58625cdb80 10 client.8268 fetch_fsmap finished waiting for FSMap version 91 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f58625cdb80 10 client.8268 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f58625cdb80 20 client.8268 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f58625cdb80 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x558c98fd8030 0x558c98fc7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f58625cdb80 4 client.8268 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f58625cdb80 1 -- 172.21.15.5:0/2425548039 --> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c98f4f7e0 con 0x558c98fd8030 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f584ffff700 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x558c98fd8030 0x558c98fc7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f582effd700 10 client.8268 ms_handle_connect on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.897+0000 7f584ffff700 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x558c98fd8030 0x558c98fc7740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2eb1d087fd1c9282 server_cookie=a859365cd09dc61c in_seq=0 out_seq=0 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f582effd700 1 -- 172.21.15.5:0/2425548039 <== mds.1 v2:172.21.15.5:6832/3718289875 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c98f4f7e0 con 0x558c98fd8030 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f582effd700 10 client.8268 handle_command_reply: tid=0 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 client.8268 shutdown 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 -- 172.21.15.5:0/2425548039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98fa37a0 msgr2=0x558c98fabf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98fa37a0 0x558c98fabf70 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5838015750 tx=0x7f58380047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 -- 172.21.15.5:0/2425548039 shutdown_connections 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98faadd0 0x558c98fd7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98fd6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x558c98fd8030 0x558c98fc7740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 --2- 172.21.15.5:0/2425548039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98fa37a0 0x558c98fabf70 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 -- 172.21.15.5:0/2425548039 >> 172.21.15.5:0/2425548039 conn(0x558c98ede210 msgr2=0x558c98f80310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f582effd700 0 client.8268 ms_handle_reset on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 -- 172.21.15.5:0/2425548039 shutdown_connections 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 1 -- 172.21.15.5:0/2425548039 wait complete. 2024-05-08T23:27:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.898+0000 7f58625cdb80 20 client.8268 trim_cache size 0 max 16384 2024-05-08T23:27:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.917+0000 7f58625cdb80 1 Processor -- start 2024-05-08T23:27:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.917+0000 7f58625cdb80 1 -- start start 2024-05-08T23:27:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.917+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98faeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.917+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.917+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f80420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.917+0000 7f58625cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c98fe7210 con 0x558c98faadd0 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.917+0000 7f58625cdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c98f7d840 con 0x558c98f7e370 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.917+0000 7f58625cdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c98f80960 con 0x558c98fa37a0 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f80420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f80420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34994/0 (socket says 172.21.15.5:34994) 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 -- 172.21.15.5:0/3142671767 learned_addr learned my addr 172.21.15.5:0/3142671767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f584ffff700 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98faeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f584f7fe700 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 -- 172.21.15.5:0/3142671767 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 msgr2=0x558c98f7d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 -- 172.21.15.5:0/3142671767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 msgr2=0x558c98faeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98faeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 -- 172.21.15.5:0/3142671767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c98e96560 con 0x558c98faadd0 2024-05-08T23:27:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58549b0700 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f80420 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f5848022000 tx=0x7f5848012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4576dd4be8153b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58467fc700 1 -- 172.21.15.5:0/3142671767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848026680 con 0x558c98faadd0 2024-05-08T23:27:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58467fc700 1 -- 172.21.15.5:0/3142671767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58480267e0 con 0x558c98faadd0 2024-05-08T23:27:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.918+0000 7f58467fc700 1 -- 172.21.15.5:0/3142671767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848026b70 con 0x558c98faadd0 2024-05-08T23:27:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 -- 172.21.15.5:0/3142671767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 msgr2=0x558c98f80420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f80420 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f5848022000 tx=0x7f5848012000 comp rx=0 tx=0).stop 2024-05-08T23:27:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 -- 172.21.15.5:0/3142671767 shutdown_connections 2024-05-08T23:27:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f80420 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98f7e370 0x558c98faeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 --2- 172.21.15.5:0/3142671767 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98fa37a0 0x558c98f7d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 -- 172.21.15.5:0/3142671767 >> 172.21.15.5:0/3142671767 conn(0x558c98ede210 msgr2=0x558c98fa2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 -- 172.21.15.5:0/3142671767 shutdown_connections 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.919+0000 7f58625cdb80 1 -- 172.21.15.5:0/3142671767 wait complete. 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.920+0000 7f58625cdb80 1 -- start start 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.920+0000 7f58625cdb80 1 Processor -- start 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98f7e370 0x558c9902bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98fa37a0 0x558c98f79ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58625cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f7a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58625cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9902c2b0 con 0x558c98faadd0 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58625cdb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c98fc9920 con 0x558c98fa37a0 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58625cdb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c98f7d400 con 0x558c98f7e370 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58625cdb80 10 client.0 _ll_register_callbacks cb 0x558c98e75b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58549b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f7a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58549b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f7a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34996/0 (socket says 172.21.15.5:34996) 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58549b0700 1 -- 172.21.15.5:0/26686379 learned_addr learned my addr 172.21.15.5:0/26686379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f584f7fe700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98fa37a0 0x558c98f79ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58549b0700 1 -- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98f7e370 msgr2=0x558c9902bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f584ffff700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98f7e370 0x558c9902bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.921+0000 7f58549b0700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98f7e370 0x558c9902bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58549b0700 1 -- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98fa37a0 msgr2=0x558c98f79ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58549b0700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98fa37a0 0x558c98f79ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58549b0700 1 -- 172.21.15.5:0/26686379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c98d513b0 con 0x558c98faadd0 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f584f7fe700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98fa37a0 0x558c98f79ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f584ffff700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98f7e370 0x558c9902bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58549b0700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f7a220 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5848003bb0 tx=0x7f5848004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73c94bf77a3c700 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f582effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f582effd700 1 -- 172.21.15.5:0/26686379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848012430 con 0x558c98faadd0 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f582effd700 1 -- 172.21.15.5:0/26686379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5848021070 con 0x558c98faadd0 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 1 -- 172.21.15.5:0/26686379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f584800bb80 con 0x558c98faadd0 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f582effd700 1 -- 172.21.15.5:0/26686379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5848012590 con 0x558c98faadd0 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 10 client.8274 fetch_fsmap learned FSMap version 91 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 1 -- 172.21.15.5:0/26686379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x558c98d55cd0 con 0x558c98faadd0 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f582effd700 1 -- 172.21.15.5:0/26686379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584800bb80 con 0x558c98faadd0 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f582effd700 1 -- 172.21.15.5:0/26686379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 91) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f584802d070 con 0x558c98faadd0 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 10 client.8274 fetch_fsmap finished waiting for FSMap version 91 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 10 client.8274 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 20 client.8274 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x558c98f80420 0x558c9902e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 4 client.8274 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.922+0000 7f58625cdb80 1 -- 172.21.15.5:0/26686379 --> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c98ec8ec0 con 0x558c98f80420 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f584ffff700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x558c98f80420 0x558c9902e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f582effd700 10 client.8274 ms_handle_connect on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f584ffff700 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x558c98f80420 0x558c9902e890 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4501becc5724d859 server_cookie=3257e97589e15367 in_seq=0 out_seq=0 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f582effd700 1 -- 172.21.15.5:0/26686379 <== mds.1 v2:172.21.15.5:6832/3718289875 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x558c98ec8ec0 con 0x558c98f80420 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f582effd700 10 client.8274 handle_command_reply: tid=0 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 client.8274 shutdown 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 -- 172.21.15.5:0/26686379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 msgr2=0x558c98f7a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f7a220 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5848003bb0 tx=0x7f5848004b40 comp rx=0 tx=0).stop 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 -- 172.21.15.5:0/26686379 shutdown_connections 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.5:6832/3718289875,v1:172.21.15.5:6834/3718289875] conn(0x558c98f80420 0x558c9902e890 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c98faadd0 0x558c98f7a220 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558c98f7e370 0x558c9902bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 --2- 172.21.15.5:0/26686379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558c98fa37a0 0x558c98f79ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f582effd700 0 client.8274 ms_handle_reset on v2:172.21.15.5:6832/3718289875 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.923+0000 7f58625cdb80 1 -- 172.21.15.5:0/26686379 >> 172.21.15.5:0/26686379 conn(0x558c98ede210 msgr2=0x558c98fa2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f58625cdb80 1 -- 172.21.15.5:0/26686379 shutdown_connections 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f58625cdb80 1 -- 172.21.15.5:0/26686379 wait complete. 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f58625cdb80 20 client.8274 trim_cache size 0 max 16384 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stdout: "4b4cf457-7a2d-427a-a3be-d52d0c5f729e": { 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:27:28.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4b4cf457-7a2d-427a-a3be-d52d0c5f729e", 2024-05-08T23:27:28.931 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:27:28.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:28.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:28.931 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 -- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f583006ac90 msgr2=0x7f583006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f583006ac90 0x7f583006d150 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f58500b7760 tx=0x7f5838008040 comp rx=0 tx=0).stop 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 -- 172.21.15.5:0/3440881182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58500b3fc0 msgr2=0x7f585015aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58500b3fc0 0x7f585015aeb0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f58400011e0 tx=0x7f584000cb00 comp rx=0 tx=0).stop 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 -- 172.21.15.5:0/3440881182 shutdown_connections 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f583006ac90 0x7f583006d150 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58500d3d20 0x7f58500b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58500b3fc0 0x7f585015aeb0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 --2- 172.21.15.5:0/3440881182 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f58500cff30 0x7f58500cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 -- 172.21.15.5:0/3440881182 >> 172.21.15.5:0/3440881182 conn(0x7f585008afe0 msgr2=0x7f585015f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 -- 172.21.15.5:0/3440881182 shutdown_connections 2024-05-08T23:27:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:28.924+0000 7f5856413700 1 -- 172.21.15.5:0/3440881182 wait complete. 2024-05-08T23:27:28.938 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:27:28.938 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 scrub status 2024-05-08T23:27:29.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.075+0000 7f12d1c75700 1 Processor -- start 2024-05-08T23:27:29.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.075+0000 7f12d1c75700 1 -- start start 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12d1c75700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12cc06bd60 0x7f12cc0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12d1c75700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f12cc0cdc90 0x7f12cc0b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12d1c75700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0b1200 con 0x7f12cc0d13a0 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12d1c75700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc0cfcc0 con 0x7f12cc06bd60 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12d1c75700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc0ce5c0 con 0x7f12cc0cdc90 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12caffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12cc06bd60 0x7f12cc0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12caffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12cc06bd60 0x7f12cc0cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35000/0 (socket says 172.21.15.5:35000) 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12caffd700 1 -- 172.21.15.5:0/2095212300 learned_addr learned my addr 172.21.15.5:0/2095212300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12cbfff700 1 --2- 172.21.15.5:0/2095212300 >> v2:172.21.15.49:3300/0 conn(0x7f12cc0cdc90 0x7f12cc0b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12caffd700 1 -- 172.21.15.5:0/2095212300 >> v2:172.21.15.49:3300/0 conn(0x7f12cc0cdc90 msgr2=0x7f12cc0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12caffd700 1 --2- 172.21.15.5:0/2095212300 >> v2:172.21.15.49:3300/0 conn(0x7f12cc0cdc90 0x7f12cc0b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12caffd700 1 -- 172.21.15.5:0/2095212300 >> v1:172.21.15.49:6789/0 conn(0x7f12cc0d13a0 legacy=0x7f12cc0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12caffd700 1 -- 172.21.15.5:0/2095212300 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0ce8a0 con 0x7f12cc06bd60 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.076+0000 7f12cbfff700 1 --2- 172.21.15.5:0/2095212300 >> v2:172.21.15.49:3300/0 conn(0x7f12cc0cdc90 0x7f12cc0b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12caffd700 1 --2- 172.21.15.5:0/2095212300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc06bd60 0x7f12cc0cf780 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0b2eb0 tx=0x7f12c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f931bfe5fd97fc3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12ca7fc700 1 -- 172.21.15.5:0/2095212300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000b870 con 0x7f12cc06bd60 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12ca7fc700 1 -- 172.21.15.5:0/2095212300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000b9d0 con 0x7f12cc06bd60 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12ca7fc700 1 -- 172.21.15.5:0/2095212300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000bd60 con 0x7f12cc06bd60 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12d1c75700 1 -- 172.21.15.5:0/2095212300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc06bd60 msgr2=0x7f12cc0cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12d1c75700 1 --2- 172.21.15.5:0/2095212300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc06bd60 0x7f12cc0cf780 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0b2eb0 tx=0x7f12c0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12d1c75700 1 -- 172.21.15.5:0/2095212300 shutdown_connections 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12d1c75700 1 --2- 172.21.15.5:0/2095212300 >> v2:172.21.15.49:3300/0 conn(0x7f12cc0cdc90 0x7f12cc0b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12d1c75700 1 --2- 172.21.15.5:0/2095212300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc06bd60 0x7f12cc0cf780 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.077+0000 7f12d1c75700 1 -- 172.21.15.5:0/2095212300 >> 172.21.15.5:0/2095212300 conn(0x7f12cc08afe0 msgr2=0x7f12cc08b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.078+0000 7f12d1c75700 1 -- 172.21.15.5:0/2095212300 shutdown_connections 2024-05-08T23:27:29.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.078+0000 7f12d1c75700 1 -- 172.21.15.5:0/2095212300 wait complete. 2024-05-08T23:27:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.079+0000 7f12d1c75700 1 Processor -- start 2024-05-08T23:27:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.079+0000 7f12d1c75700 1 -- start start 2024-05-08T23:27:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12d1c75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12cc06bd60 0x7f12cc0c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12d1c75700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12cc0cdc90 0x7f12cc1f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12d1c75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc0d13a0 0x7f12cc156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12d1c75700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc157ff0 con 0x7f12cc0d13a0 2024-05-08T23:27:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12d1c75700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0ca570 con 0x7f12cc06bd60 2024-05-08T23:27:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12d1c75700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0c9f60 con 0x7f12cc0cdc90 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc0d13a0 0x7f12cc156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc0d13a0 0x7f12cc156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35016/0 (socket says 172.21.15.5:35016) 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cbfff700 1 -- 172.21.15.5:0/1871507441 learned_addr learned my addr 172.21.15.5:0/1871507441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12caffd700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12cc0cdc90 0x7f12cc1f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12cc06bd60 0x7f12cc0c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cbfff700 1 -- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12cc0cdc90 msgr2=0x7f12cc1f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cbfff700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12cc0cdc90 0x7f12cc1f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cbfff700 1 -- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12cc06bd60 msgr2=0x7f12cc0c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cbfff700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12cc06bd60 0x7f12cc0c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12cbfff700 1 -- 172.21.15.5:0/1871507441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c0009550 con 0x7f12cc0d13a0 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.080+0000 7f12caffd700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12cc0cdc90 0x7f12cc1f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.081+0000 7f12cbfff700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc0d13a0 0x7f12cc156e50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009fa0 tx=0x7f12c400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.081+0000 7f12b3fff700 1 -- 172.21.15.5:0/1871507441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4003a20 con 0x7f12cc0d13a0 2024-05-08T23:27:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.081+0000 7f12b3fff700 1 -- 172.21.15.5:0/1871507441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400f040 con 0x7f12cc0d13a0 2024-05-08T23:27:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.081+0000 7f12d1c75700 1 -- 172.21.15.5:0/1871507441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc0c84e0 con 0x7f12cc0d13a0 2024-05-08T23:27:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.081+0000 7f12b3fff700 1 -- 172.21.15.5:0/1871507441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4003de0 con 0x7f12cc0d13a0 2024-05-08T23:27:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.081+0000 7f12d1c75700 1 -- 172.21.15.5:0/1871507441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc0c8aa0 con 0x7f12cc0d13a0 2024-05-08T23:27:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.082+0000 7f12b3fff700 1 -- 172.21.15.5:0/1871507441 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f12c4004030 con 0x7f12cc0d13a0 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.083+0000 7f12b3fff700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12b806ac90 0x7f12b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.083+0000 7f12b3fff700 1 -- 172.21.15.5:0/1871507441 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12c40895f0 con 0x7f12cc0d13a0 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.083+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12b806ac90 0x7f12b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.084+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12b806ac90 0x7f12b806d150 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000f60 tx=0x7f12bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.085+0000 7f12dde2fb80 1 Processor -- start 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.085+0000 7f12dde2fb80 1 -- start start 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 0x55b73ee1b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee4a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ed76150 con 0x55b73ee1d370 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ed73b10 con 0x55b73ee427a0 2024-05-08T23:27:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b73ed6f400 con 0x55b73ee49dd0 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35024/0 (socket says 172.21.15.5:35024) 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 -- 172.21.15.5:0/1274911105 learned_addr learned my addr 172.21.15.5:0/1274911105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 0x55b73ee1b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12cbfff700 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee4a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 -- 172.21.15.5:0/1274911105 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 msgr2=0x55b73ee4a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee4a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 -- 172.21.15.5:0/1274911105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 msgr2=0x55b73ee1b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 0x55b73ee1b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 -- 172.21.15.5:0/1274911105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b73ebf03b0 con 0x55b73ee1d370 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.086+0000 7f12caffd700 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee200a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f12c0013000 tx=0x7f12c0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c461b534c3fb989a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.087+0000 7f12b17fa700 1 -- 172.21.15.5:0/1274911105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000bb40 con 0x55b73ee1d370 2024-05-08T23:27:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.087+0000 7f12b17fa700 1 -- 172.21.15.5:0/1274911105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000bcc0 con 0x55b73ee1d370 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.087+0000 7f12b17fa700 1 -- 172.21.15.5:0/1274911105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0004280 con 0x55b73ee1d370 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.087+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1274911105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 msgr2=0x55b73ee200a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.087+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee200a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f12c0013000 tx=0x7f12c0010000 comp rx=0 tx=0).stop 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.087+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1274911105 shutdown_connections 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.087+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee4a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.088+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 0x55b73ee1b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.088+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1274911105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee200a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.088+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1274911105 >> 172.21.15.5:0/1274911105 conn(0x55b73ed7d210 msgr2=0x55b73ee64dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.088+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1274911105 shutdown_connections 2024-05-08T23:27:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.088+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1274911105 wait complete. 2024-05-08T23:27:29.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.088+0000 7f12dde2fb80 1 -- start start 2024-05-08T23:27:29.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.088+0000 7f12dde2fb80 1 Processor -- start 2024-05-08T23:27:29.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee64f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee427a0 0x55b73ee404b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee6c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ed0b560 con 0x55b73ee427a0 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ed0b200 con 0x55b73ee1d370 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b73ebf26a0 con 0x55b73ee49dd0 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12dde2fb80 10 client.0 _ll_register_callbacks cb 0x55b73ed14b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee427a0 0x55b73ee404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee427a0 0x55b73ee404b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35040/0 (socket says 172.21.15.5:35040) 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 -- 172.21.15.5:0/625801501 learned_addr learned my addr 172.21.15.5:0/625801501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12cbfff700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee6c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee64f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 -- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 msgr2=0x55b73ee6c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee6c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 -- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 msgr2=0x55b73ee64f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee64f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 -- 172.21.15.5:0/625801501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc0ce8a0 con 0x55b73ee427a0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12cbfff700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee6c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee64f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12caffd700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee427a0 0x55b73ee404b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009130 tx=0x7f12c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5016fc7cea112cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.090+0000 7f12aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12aa7fc700 1 -- 172.21.15.5:0/625801501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0004550 con 0x55b73ee427a0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12aa7fc700 1 -- 172.21.15.5:0/625801501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12c00036a0 con 0x55b73ee427a0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 1 -- 172.21.15.5:0/625801501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b73ecc8270 con 0x55b73ee427a0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12aa7fc700 1 -- 172.21.15.5:0/625801501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12c0003860 con 0x55b73ee427a0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 10 client.8286 fetch_fsmap learned FSMap version 91 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 1 -- 172.21.15.5:0/625801501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55b73eb9cb60 con 0x55b73ee427a0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12aa7fc700 1 -- 172.21.15.5:0/625801501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b73ecc8270 con 0x55b73ee427a0 2024-05-08T23:27:29.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12aa7fc700 1 -- 172.21.15.5:0/625801501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 91) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f12c00039c0 con 0x55b73ee427a0 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 10 client.8286 fetch_fsmap finished waiting for FSMap version 91 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 10 client.8286 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 20 client.8286 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55b73ee62850 0x55b73ee62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 4 client.8286 mds_command: new command op to 7336 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12dde2fb80 1 -- 172.21.15.5:0/625801501 --> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b73edee7e0 con 0x55b73ee62850 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.091+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55b73ee62850 0x55b73ee62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.092+0000 7f12aa7fc700 10 client.8286 ms_handle_connect on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.092+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55b73ee62850 0x55b73ee62c60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50b4039bf1e33879 server_cookie=dcaed712429c15b4 in_seq=0 out_seq=0 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.092+0000 7f12aa7fc700 1 -- 172.21.15.5:0/625801501 <== mds.2 v2:172.21.15.49:6835/1134209849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b73edee7e0 con 0x55b73ee62850 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.092+0000 7f12aa7fc700 10 client.8286 handle_command_reply: tid=0 2024-05-08T23:27:29.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.092+0000 7f12dde2fb80 1 client.8286 shutdown 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 -- 172.21.15.5:0/625801501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee427a0 msgr2=0x55b73ee404b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee427a0 0x55b73ee404b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009130 tx=0x7f12c00047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 -- 172.21.15.5:0/625801501 shutdown_connections 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee6c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55b73ee62850 0x55b73ee62c60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee64f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/625801501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee427a0 0x55b73ee404b0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12aa7fc700 0 client.8286 ms_handle_reset on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 -- 172.21.15.5:0/625801501 >> 172.21.15.5:0/625801501 conn(0x55b73ed7d210 msgr2=0x55b73ee1f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 -- 172.21.15.5:0/625801501 shutdown_connections 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 1 -- 172.21.15.5:0/625801501 wait complete. 2024-05-08T23:27:29.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.093+0000 7f12dde2fb80 20 client.8286 trim_cache size 0 max 16384 2024-05-08T23:27:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.111+0000 7f12dde2fb80 1 Processor -- start 2024-05-08T23:27:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.111+0000 7f12dde2fb80 1 -- start start 2024-05-08T23:27:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee1b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee427a0 0x55b73ee40c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee49dd0 0x55b73ee86d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ee19f50 con 0x55b73ee49dd0 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ee411d0 con 0x55b73ee1d370 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b73ee6a3d0 con 0x55b73ee427a0 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee49dd0 0x55b73ee86d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee49dd0 0x55b73ee86d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35050/0 (socket says 172.21.15.5:35050) 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.112+0000 7f12cbfff700 1 -- 172.21.15.5:0/2083856329 learned_addr learned my addr 172.21.15.5:0/2083856329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee1b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12caffd700 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee427a0 0x55b73ee40c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12cbfff700 1 -- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee427a0 msgr2=0x55b73ee40c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12cbfff700 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee427a0 0x55b73ee40c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12cbfff700 1 -- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 msgr2=0x55b73ee1b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12cbfff700 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee1b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12cbfff700 1 -- 172.21.15.5:0/2083856329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b73ed35560 con 0x55b73ee49dd0 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee1b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12cbfff700 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee49dd0 0x55b73ee86d50 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f12c4058380 tx=0x7f12c4058af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1646eba8bb17f25b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12b1ffb700 1 -- 172.21.15.5:0/2083856329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c405c220 con 0x55b73ee49dd0 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12b1ffb700 1 -- 172.21.15.5:0/2083856329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c405e040 con 0x55b73ee49dd0 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.113+0000 7f12b1ffb700 1 -- 172.21.15.5:0/2083856329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40607e0 con 0x55b73ee49dd0 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 -- 172.21.15.5:0/2083856329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee49dd0 msgr2=0x55b73ee86d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee49dd0 0x55b73ee86d50 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f12c4058380 tx=0x7f12c4058af0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 -- 172.21.15.5:0/2083856329 shutdown_connections 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee49dd0 0x55b73ee86d50 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee1d370 0x55b73ee1b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/2083856329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee427a0 0x55b73ee40c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 -- 172.21.15.5:0/2083856329 >> 172.21.15.5:0/2083856329 conn(0x55b73ed7d210 msgr2=0x55b73ee1af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 -- 172.21.15.5:0/2083856329 shutdown_connections 2024-05-08T23:27:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 -- 172.21.15.5:0/2083856329 wait complete. 2024-05-08T23:27:29.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 -- start start 2024-05-08T23:27:29.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.114+0000 7f12dde2fb80 1 Processor -- start 2024-05-08T23:27:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee6b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 0x55b73ee62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12dde2fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ee19a10 con 0x55b73ee1d370 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ee64da0 con 0x55b73ee427a0 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12dde2fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b73eed19b0 con 0x55b73ee49dd0 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12dde2fb80 10 client.0 _ll_register_callbacks cb 0x55b73ed14b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee6b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee6b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35058/0 (socket says 172.21.15.5:35058) 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.115+0000 7f12cb7fe700 1 -- 172.21.15.5:0/1321536094 learned_addr learned my addr 172.21.15.5:0/1321536094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12cbfff700 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12caffd700 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 0x55b73ee62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12cb7fe700 1 -- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 msgr2=0x55b73ee631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12cb7fe700 1 -- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 msgr2=0x55b73ee62c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 0x55b73ee62c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12cb7fe700 1 -- 172.21.15.5:0/1321536094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc0c8aa0 con 0x55b73ee1d370 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12cb7fe700 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee6b190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f12bc003770 tx=0x7f12bc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4da53395fb4efc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12aa7fc700 1 -- 172.21.15.5:0/1321536094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc025280 con 0x55b73ee1d370 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12aa7fc700 1 -- 172.21.15.5:0/1321536094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12bc0253e0 con 0x55b73ee1d370 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1321536094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12c000bcc0 con 0x55b73ee1d370 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12aa7fc700 1 -- 172.21.15.5:0/1321536094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12bc0255a0 con 0x55b73ee1d370 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12dde2fb80 10 client.8292 fetch_fsmap learned FSMap version 91 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1321536094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55b73ebf4cd0 con 0x55b73ee1d370 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.116+0000 7f12aa7fc700 1 -- 172.21.15.5:0/1321536094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000bcc0 con 0x55b73ee1d370 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12aa7fc700 1 -- 172.21.15.5:0/1321536094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 91) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f12bc025700 con 0x55b73ee1d370 2024-05-08T23:27:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12dde2fb80 10 client.8292 fetch_fsmap finished waiting for FSMap version 91 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12dde2fb80 10 client.8292 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12dde2fb80 20 client.8292 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55b73ee47d00 0x55b73eece190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12dde2fb80 4 client.8292 mds_command: new command op to 7336 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1321536094 --> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b73ed67ec0 con 0x55b73ee47d00 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12caffd700 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55b73ee47d00 0x55b73eece190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12aa7fc700 10 client.8292 ms_handle_connect on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.117+0000 7f12caffd700 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55b73ee47d00 0x55b73eece190 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15563eb6cd061d70 server_cookie=1446a36e020da34 in_seq=0 out_seq=0 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12aa7fc700 1 -- 172.21.15.5:0/1321536094 <== mds.2 v2:172.21.15.49:6835/1134209849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55b73ed67ec0 con 0x55b73ee47d00 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12aa7fc700 10 client.8292 handle_command_reply: tid=0 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 client.8292 shutdown 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1321536094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 msgr2=0x55b73ee6b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee6b190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f12bc003770 tx=0x7f12bc000ed0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1321536094 shutdown_connections 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55b73ee49dd0 0x55b73ee631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b73ee1d370 0x55b73ee6b190 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:6835/1134209849,v1:172.21.15.49:6837/1134209849] conn(0x55b73ee47d00 0x55b73eece190 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 --2- 172.21.15.5:0/1321536094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b73ee427a0 0x55b73ee62c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12aa7fc700 0 client.8292 ms_handle_reset on v2:172.21.15.49:6835/1134209849 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1321536094 >> 172.21.15.5:0/1321536094 conn(0x55b73ed7d210 msgr2=0x55b73ee1af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1321536094 shutdown_connections 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 1 -- 172.21.15.5:0/1321536094 wait complete. 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12dde2fb80 20 client.8292 trim_cache size 0 max 16384 2024-05-08T23:27:29.124 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout: "4b4cf457-7a2d-427a-a3be-d52d0c5f729e": { 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4b4cf457-7a2d-427a-a3be-d52d0c5f729e", 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:29.125 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12d1c75700 1 -- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12b806ac90 msgr2=0x7f12b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.118+0000 7f12d1c75700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12b806ac90 0x7f12b806d150 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000f60 tx=0x7f12bc009040 comp rx=0 tx=0).stop 2024-05-08T23:27:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 -- 172.21.15.5:0/1871507441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc0d13a0 msgr2=0x7f12cc156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc0d13a0 0x7f12cc156e50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009fa0 tx=0x7f12c400d460 comp rx=0 tx=0).stop 2024-05-08T23:27:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 -- 172.21.15.5:0/1871507441 shutdown_connections 2024-05-08T23:27:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f12b806ac90 0x7f12b806d150 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc0d13a0 0x7f12cc156e50 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12cc06bd60 0x7f12cc0c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 --2- 172.21.15.5:0/1871507441 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12cc0cdc90 0x7f12cc1f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 -- 172.21.15.5:0/1871507441 >> 172.21.15.5:0/1871507441 conn(0x7f12cc08afe0 msgr2=0x7f12cc0cd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 -- 172.21.15.5:0/1871507441 shutdown_connections 2024-05-08T23:27:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.119+0000 7f12d1c75700 1 -- 172.21.15.5:0/1871507441 wait complete. 2024-05-08T23:27:29.132 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:27:29.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T23:27:29.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.271+0000 7f0348d85700 1 Processor -- start 2024-05-08T23:27:29.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.271+0000 7f0348d85700 1 -- start start 2024-05-08T23:27:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0348d85700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03440d13a0 0x7f03440d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0348d85700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03440cdc90 0x7f03440b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0348d85700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03440b1200 con 0x7f034406bd60 2024-05-08T23:27:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0348d85700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03440d1cc0 con 0x7f03440cdc90 2024-05-08T23:27:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0348d85700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440ce5c0 con 0x7f03440d13a0 2024-05-08T23:27:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0341d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f034406bd60 0x7f03440cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52008/0 (socket says 172.21.15.5:52008) 2024-05-08T23:27:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0341d9b700 1 -- 172.21.15.5:0/4182702580 learned_addr learned my addr 172.21.15.5:0/4182702580 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0342d9d700 1 --2- 172.21.15.5:0/4182702580 >> v2:172.21.15.5:3300/0 conn(0x7f03440cdc90 0x7f03440b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f034259c700 1 --2- 172.21.15.5:0/4182702580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03440d13a0 0x7f03440d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f034159a700 1 -- 172.21.15.5:0/4182702580 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003962085 0 0) 0x7f03440b1200 con 0x7f034406bd60 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f034159a700 1 -- 172.21.15.5:0/4182702580 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f032c0035b0 con 0x7f034406bd60 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0342d9d700 1 -- 172.21.15.5:0/4182702580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03440d13a0 msgr2=0x7f03440d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0342d9d700 1 --2- 172.21.15.5:0/4182702580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03440d13a0 0x7f03440d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.272+0000 7f0342d9d700 1 -- 172.21.15.5:0/4182702580 >> v1:172.21.15.5:6789/0 conn(0x7f034406bd60 legacy=0x7f03440cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.273+0000 7f0342d9d700 1 -- 172.21.15.5:0/4182702580 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440ce8a0 con 0x7f03440cdc90 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.273+0000 7f0342d9d700 1 --2- 172.21.15.5:0/4182702580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440cdc90 0x7f03440b19c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f033c009d90 tx=0x7f033c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3324e2eda8c9d8e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.273+0000 7f034159a700 1 -- 172.21.15.5:0/4182702580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c00e070 con 0x7f03440cdc90 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.273+0000 7f034159a700 1 -- 172.21.15.5:0/4182702580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c00bb20 con 0x7f03440cdc90 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.273+0000 7f034159a700 1 -- 172.21.15.5:0/4182702580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c012430 con 0x7f03440cdc90 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.273+0000 7f0348d85700 1 -- 172.21.15.5:0/4182702580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440cdc90 msgr2=0x7f03440b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.273+0000 7f0348d85700 1 --2- 172.21.15.5:0/4182702580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440cdc90 0x7f03440b19c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f033c009d90 tx=0x7f033c00b330 comp rx=0 tx=0).stop 2024-05-08T23:27:29.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.274+0000 7f0348d85700 1 -- 172.21.15.5:0/4182702580 shutdown_connections 2024-05-08T23:27:29.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.274+0000 7f0348d85700 1 --2- 172.21.15.5:0/4182702580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440cdc90 0x7f03440b19c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.274+0000 7f0348d85700 1 --2- 172.21.15.5:0/4182702580 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03440d13a0 0x7f03440d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.274+0000 7f0348d85700 1 -- 172.21.15.5:0/4182702580 >> 172.21.15.5:0/4182702580 conn(0x7f034408afe0 msgr2=0x7f034408b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.274+0000 7f0348d85700 1 -- 172.21.15.5:0/4182702580 shutdown_connections 2024-05-08T23:27:29.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.274+0000 7f0348d85700 1 -- 172.21.15.5:0/4182702580 wait complete. 2024-05-08T23:27:29.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.275+0000 7f0348d85700 1 Processor -- start 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.275+0000 7f0348d85700 1 -- start start 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0348d85700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f034406bd60 0x7f03441f0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0348d85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03440cdc90 0x7f03440c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0348d85700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440d13a0 0x7f03440b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0348d85700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03441f0ea0 con 0x7f03440d13a0 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0348d85700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440b5410 con 0x7f03440cdc90 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0348d85700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440c7740 con 0x7f034406bd60 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0342d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440d13a0 0x7f03440b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0342d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440d13a0 0x7f03440b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35082/0 (socket says 172.21.15.5:35082) 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0342d9d700 1 -- 172.21.15.5:0/4005166549 learned_addr learned my addr 172.21.15.5:0/4005166549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f034259c700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f034406bd60 0x7f03441f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0341d9b700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03440cdc90 0x7f03440c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0342d9d700 1 -- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f034406bd60 msgr2=0x7f03441f0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0342d9d700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f034406bd60 0x7f03441f0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0342d9d700 1 -- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03440cdc90 msgr2=0x7f03440c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0342d9d700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03440cdc90 0x7f03440c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0342d9d700 1 -- 172.21.15.5:0/4005166549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c009a70 con 0x7f03440d13a0 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f0341d9b700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03440cdc90 0x7f03440c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.276+0000 7f034259c700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f034406bd60 0x7f03441f0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.277+0000 7f0342d9d700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440d13a0 0x7f03440b6630 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f03440cef90 tx=0x7f033c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.277+0000 7f0332ffd700 1 -- 172.21.15.5:0/4005166549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c0129f0 con 0x7f03440d13a0 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.277+0000 7f0332ffd700 1 -- 172.21.15.5:0/4005166549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c020070 con 0x7f03440d13a0 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.277+0000 7f0348d85700 1 -- 172.21.15.5:0/4005166549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03441f3fb0 con 0x7f03440d13a0 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.277+0000 7f0332ffd700 1 -- 172.21.15.5:0/4005166549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c012db0 con 0x7f03440d13a0 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.277+0000 7f0348d85700 1 -- 172.21.15.5:0/4005166549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03440b50a0 con 0x7f03440d13a0 2024-05-08T23:27:29.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.279+0000 7f0332ffd700 1 -- 172.21.15.5:0/4005166549 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f033c004040 con 0x7f03440d13a0 2024-05-08T23:27:29.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.279+0000 7f0332ffd700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f032406ac90 0x7f032406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.279+0000 7f0332ffd700 1 -- 172.21.15.5:0/4005166549 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f033c022070 con 0x7f03440d13a0 2024-05-08T23:27:29.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.279+0000 7f034259c700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f032406ac90 0x7f032406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.280+0000 7f034259c700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f032406ac90 0x7f032406d150 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0334009ac0 tx=0x7f0334008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.281+0000 7f0354f3fb80 1 Processor -- start 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.281+0000 7f0354f3fb80 1 -- start start 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc93110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fc980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcc2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fbee150 con 0x55cb1fcba7a0 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fbebb10 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fbe7400 con 0x55cb1fc95370 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f034259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc93110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f034259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc93110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35084/0 (socket says 172.21.15.5:35084) 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f034259c700 1 -- 172.21.15.5:0/1569836784 learned_addr learned my addr 172.21.15.5:0/1569836784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0341d9b700 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fc980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f034259c700 1 -- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 msgr2=0x55cb1fc980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0342d9d700 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcc2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f034259c700 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fc980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f034259c700 1 -- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 msgr2=0x55cb1fcc2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f034259c700 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcc2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f034259c700 1 -- 172.21.15.5:0/1569836784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb1fa683b0 con 0x55cb1fcba7a0 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0341d9b700 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fc980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.282+0000 7f0342d9d700 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcc2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.283+0000 7f034259c700 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc93110 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0334010950 tx=0x7f0334013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacf21e35e39f2a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.283+0000 7f0323fff700 1 -- 172.21.15.5:0/1569836784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334004230 con 0x55cb1fcba7a0 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.283+0000 7f0323fff700 1 -- 172.21.15.5:0/1569836784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033401a070 con 0x55cb1fcba7a0 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.283+0000 7f0323fff700 1 -- 172.21.15.5:0/1569836784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334004590 con 0x55cb1fcba7a0 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1569836784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 msgr2=0x55cb1fc93110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc93110 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0334010950 tx=0x7f0334013000 comp rx=0 tx=0).stop 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1569836784 shutdown_connections 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcc2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc93110 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1569836784 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fc980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1569836784 >> 172.21.15.5:0/1569836784 conn(0x55cb1fbf5210 msgr2=0x55cb1fcdcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1569836784 shutdown_connections 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1569836784 wait complete. 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 -- start start 2024-05-08T23:27:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.284+0000 7f0354f3fb80 1 Processor -- start 2024-05-08T23:27:29.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.285+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fcdf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.285+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fce0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.285+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcb9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fb83560 con 0x55cb1fcba7a0 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fb83200 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fa6a6a0 con 0x55cb1fc95370 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0354f3fb80 10 client.0 _ll_register_callbacks cb 0x55cb1fb8cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fce0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fce0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35088/0 (socket says 172.21.15.5:35088) 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 -- 172.21.15.5:0/1409342495 learned_addr learned my addr 172.21.15.5:0/1409342495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0342d9d700 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcb9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f034259c700 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fcdf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 -- 172.21.15.5:0/1409342495 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 msgr2=0x55cb1fcdf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fcdf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 -- 172.21.15.5:0/1409342495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 msgr2=0x55cb1fcb9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcb9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 -- 172.21.15.5:0/1409342495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033400e050 con 0x55cb1fcba7a0 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0341d9b700 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fce0a20 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f033800e1c0 tx=0x7f033800e580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3550b7bf375cd5d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f03217fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f03217fa700 1 -- 172.21.15.5:0/1409342495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338005ae0 con 0x55cb1fcba7a0 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f03217fa700 1 -- 172.21.15.5:0/1409342495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0338005c40 con 0x55cb1fcba7a0 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.286+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1409342495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb1fb40270 con 0x55cb1fcba7a0 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f03217fa700 1 -- 172.21.15.5:0/1409342495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f033800eec0 con 0x55cb1fcba7a0 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f0354f3fb80 10 client.8307 fetch_fsmap learned FSMap version 91 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1409342495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55cb1fa14b60 con 0x55cb1fcba7a0 2024-05-08T23:27:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f03217fa700 1 -- 172.21.15.5:0/1409342495 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb1fb40270 con 0x55cb1fcba7a0 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f03217fa700 1 -- 172.21.15.5:0/1409342495 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 91) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f03380070d0 con 0x55cb1fcba7a0 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f0354f3fb80 10 client.8307 fetch_fsmap finished waiting for FSMap version 91 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f0354f3fb80 10 client.8307 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f0354f3fb80 20 client.8307 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55cb1fcc3460 0x55cb1fcb7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f0354f3fb80 4 client.8307 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1409342495 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb1fc667e0 con 0x55cb1fcc3460 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f034259c700 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55cb1fcc3460 0x55cb1fcb7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f03217fa700 10 client.8307 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.287+0000 7f034259c700 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55cb1fcc3460 0x55cb1fcb7e90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81e83b70638dcf2b server_cookie=37a5a22fd7e02e41 in_seq=0 out_seq=0 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f03217fa700 1 -- 172.21.15.5:0/1409342495 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb1fc667e0 con 0x55cb1fcc3460 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f03217fa700 10 client.8307 handle_command_reply: tid=0 2024-05-08T23:27:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 client.8307 shutdown 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1409342495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 msgr2=0x55cb1fce0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fce0a20 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f033800e1c0 tx=0x7f033800e580 comp rx=0 tx=0).stop 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1409342495 shutdown_connections 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55cb1fcc3460 0x55cb1fcb7e90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fcb9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fc95370 0x55cb1fcdf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/1409342495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fce0a20 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f03217fa700 0 client.8307 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1409342495 >> 172.21.15.5:0/1409342495 conn(0x55cb1fbf5210 msgr2=0x55cb1fcc2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1409342495 shutdown_connections 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.288+0000 7f0354f3fb80 1 -- 172.21.15.5:0/1409342495 wait complete. 2024-05-08T23:27:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.289+0000 7f0354f3fb80 20 client.8307 trim_cache size 0 max 16384 2024-05-08T23:27:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.308+0000 7f0354f3fb80 1 Processor -- start 2024-05-08T23:27:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.308+0000 7f0354f3fb80 1 -- start start 2024-05-08T23:27:29.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.308+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.308+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc92fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.308+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcc1dd0 0x55cb1fc972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.308+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fc98e80 con 0x55cb1fcba7a0 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.308+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fcd3a10 con 0x55cb1fc95370 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.308+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fcc3960 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc92fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc92fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35102/0 (socket says 172.21.15.5:35102) 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 -- 172.21.15.5:0/3186855163 learned_addr learned my addr 172.21.15.5:0/3186855163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f034259c700 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0342d9d700 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcc1dd0 0x55cb1fc972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 -- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcc1dd0 msgr2=0x55cb1fc972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcc1dd0 0x55cb1fc972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 -- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 msgr2=0x55cb1fcffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 -- 172.21.15.5:0/3186855163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb1fbad560 con 0x55cb1fcba7a0 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f034259c700 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0341d9b700 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc92fa0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0338001350 tx=0x7f033800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb8fe790db4db6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0330ff9700 1 -- 172.21.15.5:0/3186855163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033800e980 con 0x55cb1fcba7a0 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0330ff9700 1 -- 172.21.15.5:0/3186855163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033800d040 con 0x55cb1fcba7a0 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.309+0000 7f0330ff9700 1 -- 172.21.15.5:0/3186855163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338007680 con 0x55cb1fcba7a0 2024-05-08T23:27:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3186855163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 msgr2=0x55cb1fc92fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc92fa0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0338001350 tx=0x7f033800c000 comp rx=0 tx=0).stop 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3186855163 shutdown_connections 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcc1dd0 0x55cb1fc972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3186855163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcba7a0 0x55cb1fc92fa0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3186855163 >> 172.21.15.5:0/3186855163 conn(0x55cb1fbf5210 msgr2=0x55cb1fce0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3186855163 shutdown_connections 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.310+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3186855163 wait complete. 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.311+0000 7f0354f3fb80 1 -- start start 2024-05-08T23:27:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.311+0000 7f0354f3fb80 1 Processor -- start 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcbe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcba7a0 0x55cb1fd3f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0354f3fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fd2b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fc93280 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fd2cb40 con 0x55cb1fc95370 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0354f3fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb1fcc3bf0 con 0x55cb1fcba7a0 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0354f3fb80 10 client.0 _ll_register_callbacks cb 0x55cb1fb8cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fd2b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fd2b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35118/0 (socket says 172.21.15.5:35118) 2024-05-08T23:27:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 -- 172.21.15.5:0/3541298518 learned_addr learned my addr 172.21.15.5:0/3541298518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f034259c700 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcbe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0341d9b700 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcba7a0 0x55cb1fd3f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 -- 172.21.15.5:0/3541298518 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcba7a0 msgr2=0x55cb1fd3f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcba7a0 0x55cb1fd3f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 -- 172.21.15.5:0/3541298518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 msgr2=0x55cb1fcbe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcbe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 -- 172.21.15.5:0/3541298518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb1fa14b60 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f0342d9d700 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fd2b810 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f033c009540 tx=0x7f033c05a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed150418492f3c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.312+0000 7f03217fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f03217fa700 1 -- 172.21.15.5:0/3541298518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c062000 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f03217fa700 1 -- 172.21.15.5:0/3541298518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f033c055c90 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3541298518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f033401a070 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f03217fa700 1 -- 172.21.15.5:0/3541298518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f033c089260 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 10 client.8313 fetch_fsmap learned FSMap version 92 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3541298518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x55cb1fa6ccd0 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f03217fa700 1 -- 172.21.15.5:0/3541298518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c076020 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f03217fa700 1 -- 172.21.15.5:0/3541298518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 92) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f033c055e50 con 0x55cb1fcc1dd0 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 10 client.8313 fetch_fsmap finished waiting for FSMap version 92 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 10 client.8313 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 20 client.8313 populate_metadata read hostname 'smithi005' 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55cb1fd2bd50 0x55cb1fcdf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 4 client.8313 mds_command: new command op to 7758 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3541298518 --> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cb1fbdfec0 con 0x55cb1fd2bd50 2024-05-08T23:27:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.313+0000 7f034259c700 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55cb1fd2bd50 0x55cb1fcdf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f03217fa700 10 client.8313 ms_handle_connect on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f034259c700 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55cb1fd2bd50 0x55cb1fcdf630 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=978514eb662ef58a server_cookie=6f727b1d4c196997 in_seq=0 out_seq=0 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f03217fa700 1 -- 172.21.15.5:0/3541298518 <== mds.0 v2:172.21.15.5:6833/1093540582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55cb1fbdfec0 con 0x55cb1fd2bd50 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f03217fa700 10 client.8313 handle_command_reply: tid=0 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 client.8313 shutdown 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3541298518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcc1dd0 msgr2=0x55cb1fd2b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fd2b810 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f033c009540 tx=0x7f033c05a3e0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3541298518 shutdown_connections 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.5:6833/1093540582,v1:172.21.15.5:6835/1093540582] conn(0x55cb1fd2bd50 0x55cb1fcdf630 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb1fcc1dd0 0x55cb1fd2b810 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cb1fc95370 0x55cb1fcbe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f03217fa700 0 client.8313 ms_handle_reset on v2:172.21.15.5:6833/1093540582 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 --2- 172.21.15.5:0/3541298518 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cb1fcba7a0 0x55cb1fd3f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3541298518 >> 172.21.15.5:0/3541298518 conn(0x55cb1fbf5210 msgr2=0x55cb1fcc2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3541298518 shutdown_connections 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 1 -- 172.21.15.5:0/3541298518 wait complete. 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.314+0000 7f0354f3fb80 20 client.8313 trim_cache size 0 max 16384 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stdout: "4b4cf457-7a2d-427a-a3be-d52d0c5f729e": { 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:27:29.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4b4cf457-7a2d-427a-a3be-d52d0c5f729e", 2024-05-08T23:27:29.323 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:27:29.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:29.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:29.323 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 -- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f032406ac90 msgr2=0x7f032406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f032406ac90 0x7f032406d150 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0334009ac0 tx=0x7f0334008040 comp rx=0 tx=0).stop 2024-05-08T23:27:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 -- 172.21.15.5:0/4005166549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440d13a0 msgr2=0x7f03440b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440d13a0 0x7f03440b6630 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f03440cef90 tx=0x7f033c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 -- 172.21.15.5:0/4005166549 shutdown_connections 2024-05-08T23:27:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f032406ac90 0x7f032406d150 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03440d13a0 0x7f03440b6630 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f034406bd60 0x7f03441f0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 --2- 172.21.15.5:0/4005166549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03440cdc90 0x7f03440c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 -- 172.21.15.5:0/4005166549 >> 172.21.15.5:0/4005166549 conn(0x7f034408afe0 msgr2=0x7f03440b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 -- 172.21.15.5:0/4005166549 shutdown_connections 2024-05-08T23:27:29.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.315+0000 7f0348d85700 1 -- 172.21.15.5:0/4005166549 wait complete. 2024-05-08T23:27:29.328 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:27:29.329 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.465+0000 7f795d01b700 1 Processor -- start 2024-05-08T23:27:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.465+0000 7f795d01b700 1 -- start start 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795d01b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7958161030 0x7f7958099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795d01b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580b2480 0x7f7958099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795d01b700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580d0840 con 0x7f795809a4d0 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795d01b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795806ccc0 con 0x7f7958161030 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795d01b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795806c600 con 0x7f79580b2480 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795759e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f795809a4d0 0x7f79580cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52024/0 (socket says 172.21.15.5:52024) 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795759e700 1 -- 172.21.15.5:0/3797967726 learned_addr learned my addr 172.21.15.5:0/3797967726 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f7956d9d700 1 --2- 172.21.15.5:0/3797967726 >> v2:172.21.15.49:3300/0 conn(0x7f7958161030 0x7f7958099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795659c700 1 --2- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580b2480 0x7f7958099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f7955d9b700 1 -- 172.21.15.5:0/3797967726 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921312615 0 0) 0x7f79580d0840 con 0x7f795809a4d0 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f7955d9b700 1 -- 172.21.15.5:0/3797967726 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79400035b0 con 0x7f795809a4d0 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f7956d9d700 1 -- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580b2480 msgr2=0x7f7958099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f7956d9d700 1 --2- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580b2480 0x7f7958099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f7956d9d700 1 -- 172.21.15.5:0/3797967726 >> v1:172.21.15.5:6789/0 conn(0x7f795809a4d0 legacy=0x7f79580cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f7956d9d700 1 -- 172.21.15.5:0/3797967726 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795806c8e0 con 0x7f7958161030 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.466+0000 7f795659c700 1 --2- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580b2480 0x7f7958099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f7956d9d700 1 --2- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958161030 0x7f7958099a50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f794c009840 tx=0x7f794c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=562895db2c6c02ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f7955d9b700 1 -- 172.21.15.5:0/3797967726 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c011830 con 0x7f7958161030 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f7955d9b700 1 -- 172.21.15.5:0/3797967726 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c011990 con 0x7f7958161030 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f7955d9b700 1 -- 172.21.15.5:0/3797967726 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c011d00 con 0x7f7958161030 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f795d01b700 1 -- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958161030 msgr2=0x7f7958099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f795d01b700 1 --2- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958161030 0x7f7958099a50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f794c009840 tx=0x7f794c009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f795d01b700 1 -- 172.21.15.5:0/3797967726 shutdown_connections 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f795d01b700 1 --2- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958161030 0x7f7958099a50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f795d01b700 1 --2- 172.21.15.5:0/3797967726 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580b2480 0x7f7958099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f795d01b700 1 -- 172.21.15.5:0/3797967726 >> 172.21.15.5:0/3797967726 conn(0x7f79580b2f20 msgr2=0x7f79580b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f795d01b700 1 -- 172.21.15.5:0/3797967726 shutdown_connections 2024-05-08T23:27:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.467+0000 7f795d01b700 1 -- 172.21.15.5:0/3797967726 wait complete. 2024-05-08T23:27:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.469+0000 7f795d01b700 1 Processor -- start 2024-05-08T23:27:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.469+0000 7f795d01b700 1 -- start start 2024-05-08T23:27:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795d01b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795809a4d0 0x7f7958155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795d01b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580b2480 0x7f79580c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795d01b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7958161030 0x7f795806ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795d01b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580b6370 con 0x7f795809a4d0 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795d01b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580b6650 con 0x7f79580b2480 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795d01b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79580b6930 con 0x7f7958161030 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f7956d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795809a4d0 0x7f7958155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f7956d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795809a4d0 0x7f7958155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35130/0 (socket says 172.21.15.5:35130) 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f7956d9d700 1 -- 172.21.15.5:0/1209670567 learned_addr learned my addr 172.21.15.5:0/1209670567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795659c700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580b2480 0x7f79580c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795759e700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7958161030 0x7f795806ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f7956d9d700 1 -- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7958161030 msgr2=0x7f795806ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f7956d9d700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7958161030 0x7f795806ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f7956d9d700 1 -- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580b2480 msgr2=0x7f79580c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f7956d9d700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580b2480 0x7f79580c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f7956d9d700 1 -- 172.21.15.5:0/1209670567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480094a0 con 0x7f795809a4d0 2024-05-08T23:27:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.470+0000 7f795659c700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580b2480 0x7f79580c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.471+0000 7f7956d9d700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795809a4d0 0x7f7958155aa0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f794c0090b0 tx=0x7f794c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.471+0000 7f793f7fe700 1 -- 172.21.15.5:0/1209670567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c0036a0 con 0x7f795809a4d0 2024-05-08T23:27:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.471+0000 7f793f7fe700 1 -- 172.21.15.5:0/1209670567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c01f070 con 0x7f795809a4d0 2024-05-08T23:27:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.471+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c0094a0 con 0x7f795809a4d0 2024-05-08T23:27:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.471+0000 7f793f7fe700 1 -- 172.21.15.5:0/1209670567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c003a40 con 0x7f795809a4d0 2024-05-08T23:27:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.471+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7958155fe0 con 0x7f795809a4d0 2024-05-08T23:27:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.472+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7958156370 con 0x7f795809a4d0 2024-05-08T23:27:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.473+0000 7f793f7fe700 1 -- 172.21.15.5:0/1209670567 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f794c003ba0 con 0x7f795809a4d0 2024-05-08T23:27:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.473+0000 7f793f7fe700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f794406ac90 0x7f794406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.473+0000 7f793f7fe700 1 -- 172.21.15.5:0/1209670567 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f794c021070 con 0x7f795809a4d0 2024-05-08T23:27:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.473+0000 7f795659c700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f794406ac90 0x7f794406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.474+0000 7f795659c700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f794406ac90 0x7f794406d150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f79580d0b10 tx=0x7f7948008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.478+0000 7f793f7fe700 1 -- 172.21.15.5:0/1209670567 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f794c055e30 con 0x7f795809a4d0 2024-05-08T23:27:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.748+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7958160b70 con 0x7f795809a4d0 2024-05-08T23:27:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.748+0000 7f793f7fe700 1 -- 172.21.15.5:0/1209670567 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7f794c015020 con 0x7f795809a4d0 2024-05-08T23:27:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:29.749 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f794406ac90 msgr2=0x7f794406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f794406ac90 0x7f794406d150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f79580d0b10 tx=0x7f7948008040 comp rx=0 tx=0).stop 2024-05-08T23:27:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795809a4d0 msgr2=0x7f7958155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795809a4d0 0x7f7958155aa0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f794c0090b0 tx=0x7f794c004750 comp rx=0 tx=0).stop 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 shutdown_connections 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f794406ac90 0x7f794406d150 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7958161030 0x7f795806ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795809a4d0 0x7f7958155aa0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 --2- 172.21.15.5:0/1209670567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580b2480 0x7f79580c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 >> 172.21.15.5:0/1209670567 conn(0x7f79580b2f20 msgr2=0x7f795815b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 shutdown_connections 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.751+0000 7f795d01b700 1 -- 172.21.15.5:0/1209670567 wait complete. 2024-05-08T23:27:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:29.765 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-05-08T23:27:29.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:27:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.900+0000 7f983a27e700 1 Processor -- start 2024-05-08T23:27:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.900+0000 7f983a27e700 1 -- start start 2024-05-08T23:27:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f983a27e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f983415ecf0 0x7f98340ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f983a27e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f983406adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f983a27e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98340d0cd0 con 0x7f98340d0fe0 2024-05-08T23:27:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f983a27e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983406b2f0 con 0x7f983415ecf0 2024-05-08T23:27:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f983a27e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983415f0d0 con 0x7f983406a960 2024-05-08T23:27:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f9833fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f983415ecf0 0x7f98340ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f98337fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f98340d0fe0 0x7f98340ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52034/0 (socket says 172.21.15.5:52034) 2024-05-08T23:27:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f98337fe700 1 -- 172.21.15.5:0/3678821511 learned_addr learned my addr 172.21.15.5:0/3678821511 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f983881b700 1 --2- 172.21.15.5:0/3678821511 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f983406adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f9833fff700 1 -- 172.21.15.5:0/3678821511 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 msgr2=0x7f983406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f9833fff700 1 --2- 172.21.15.5:0/3678821511 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f983406adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f9833fff700 1 -- 172.21.15.5:0/3678821511 >> v1:172.21.15.5:6789/0 conn(0x7f98340d0fe0 legacy=0x7f98340ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f9833fff700 1 -- 172.21.15.5:0/3678821511 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340cc300 con 0x7f983415ecf0 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f9832ffd700 1 -- 172.21.15.5:0/3678821511 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582333248 0 0) 0x7f98340d0cd0 con 0x7f98340d0fe0 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.901+0000 7f983881b700 1 --2- 172.21.15.5:0/3678821511 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f983406adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f9833fff700 1 --2- 172.21.15.5:0/3678821511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f983415ecf0 0x7f98340ce990 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9824009840 tx=0x7f9824009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e045f8a2a5189e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f9832ffd700 1 -- 172.21.15.5:0/3678821511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824011830 con 0x7f983415ecf0 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f9832ffd700 1 -- 172.21.15.5:0/3678821511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9824011990 con 0x7f983415ecf0 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f9832ffd700 1 -- 172.21.15.5:0/3678821511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824011d20 con 0x7f983415ecf0 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f983a27e700 1 -- 172.21.15.5:0/3678821511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f983415ecf0 msgr2=0x7f98340ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f983a27e700 1 --2- 172.21.15.5:0/3678821511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f983415ecf0 0x7f98340ce990 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9824009840 tx=0x7f9824009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f983a27e700 1 -- 172.21.15.5:0/3678821511 shutdown_connections 2024-05-08T23:27:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f983a27e700 1 --2- 172.21.15.5:0/3678821511 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f983406adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f983a27e700 1 --2- 172.21.15.5:0/3678821511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f983415ecf0 0x7f98340ce990 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f983a27e700 1 -- 172.21.15.5:0/3678821511 >> 172.21.15.5:0/3678821511 conn(0x7f9834099a50 msgr2=0x7f98340d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:29.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f983a27e700 1 -- 172.21.15.5:0/3678821511 shutdown_connections 2024-05-08T23:27:29.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.902+0000 7f983a27e700 1 -- 172.21.15.5:0/3678821511 wait complete. 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.904+0000 7f983a27e700 1 Processor -- start 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.904+0000 7f983a27e700 1 -- start start 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f983a27e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f98341f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f983a27e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98340d0fe0 0x7f98340c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f983a27e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f983415ecf0 0x7f983415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f983a27e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98340b1390 con 0x7f98340d0fe0 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f983a27e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98341f43b0 con 0x7f983415ecf0 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f983a27e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98340b15e0 con 0x7f983406a960 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98340d0fe0 0x7f98340c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98340d0fe0 0x7f98340c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35148/0 (socket says 172.21.15.5:35148) 2024-05-08T23:27:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 -- 172.21.15.5:0/2206515442 learned_addr learned my addr 172.21.15.5:0/2206515442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f9833fff700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f98341f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f983881b700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f983415ecf0 0x7f983415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 -- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 msgr2=0x7f98341f0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f98341f0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 -- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f983415ecf0 msgr2=0x7f983415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f983415ecf0 0x7f983415ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 -- 172.21.15.5:0/2206515442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240094a0 con 0x7f98340d0fe0 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.905+0000 7f98337fe700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98340d0fe0 0x7f98340c6720 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f982800c550 tx=0x7f982800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.906+0000 7f9830ff9700 1 -- 172.21.15.5:0/2206515442 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828004c80 con 0x7f98340d0fe0 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.906+0000 7f9830ff9700 1 -- 172.21.15.5:0/2206515442 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9828004e00 con 0x7f98340d0fe0 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.906+0000 7f9830ff9700 1 -- 172.21.15.5:0/2206515442 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828007310 con 0x7f98340d0fe0 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.906+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98340b18c0 con 0x7f98340d0fe0 2024-05-08T23:27:29.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.906+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98340b1e80 con 0x7f98340d0fe0 2024-05-08T23:27:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.907+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9834068000 con 0x7f98340d0fe0 2024-05-08T23:27:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.908+0000 7f9830ff9700 1 -- 172.21.15.5:0/2206515442 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9828007470 con 0x7f98340d0fe0 2024-05-08T23:27:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.908+0000 7f9830ff9700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f982006ac90 0x7f982006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.908+0000 7f9830ff9700 1 -- 172.21.15.5:0/2206515442 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9828089470 con 0x7f98340d0fe0 2024-05-08T23:27:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.908+0000 7f9833fff700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f982006ac90 0x7f982006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.909+0000 7f9833fff700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f982006ac90 0x7f982006d150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f98240090b0 tx=0x7f9824008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:29.912+0000 7f9830ff9700 1 -- 172.21.15.5:0/2206515442 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9828054730 con 0x7f98340d0fe0 2024-05-08T23:27:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.224+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f98340ad200 con 0x7f98340d0fe0 2024-05-08T23:27:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.224+0000 7f9830ff9700 1 -- 172.21.15.5:0/2206515442 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1965 (secure 0 0 0) 0x7f9828057d50 con 0x7f98340d0fe0 2024-05-08T23:27:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:30.225 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":404,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700348,"bytes_used":256286720,"bytes_avail":772837826560,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9896,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7336},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7755},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7758}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-05-08T23:27:27.387490+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f982006ac90 msgr2=0x7f982006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f982006ac90 0x7f982006d150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f98240090b0 tx=0x7f9824008030 comp rx=0 tx=0).stop 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98340d0fe0 msgr2=0x7f98340c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98340d0fe0 0x7f98340c6720 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f982800c550 tx=0x7f982800c860 comp rx=0 tx=0).stop 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 shutdown_connections 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f982006ac90 0x7f982006d150 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f983415ecf0 0x7f983415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f983406a960 0x7f98341f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 --2- 172.21.15.5:0/2206515442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98340d0fe0 0x7f98340c6720 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.226+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 >> 172.21.15.5:0/2206515442 conn(0x7f9834099a50 msgr2=0x7f98340aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.227+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 shutdown_connections 2024-05-08T23:27:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:30.227+0000 7f983a27e700 1 -- 172.21.15.5:0/2206515442 wait complete. 2024-05-08T23:27:30.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:27:31.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.385+0000 7fd60e444700 1 Processor -- start 2024-05-08T23:27:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.385+0000 7fd60e444700 1 -- start start 2024-05-08T23:27:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.386+0000 7fd60e444700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd60815e830 0x7fd6080ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.386+0000 7fd60e444700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6080d0a90 con 0x7fd6080d0f20 2024-05-08T23:27:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.386+0000 7fd60e444700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60806b720 con 0x7fd60806b0e0 2024-05-08T23:27:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.386+0000 7fd60e444700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60806b510 con 0x7fd60815e830 2024-05-08T23:27:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd607fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd60815e830 0x7fd6080ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd6077fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd6080d0f20 0x7fd60806a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52040/0 (socket says 172.21.15.5:52040) 2024-05-08T23:27:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd6077fe700 1 -- 172.21.15.5:0/1288080214 learned_addr learned my addr 172.21.15.5:0/1288080214 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd607fff700 1 -- 172.21.15.5:0/1288080214 >> v1:172.21.15.49:6789/0 conn(0x7fd60806b0e0 legacy=0x7fd608157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd607fff700 1 -- 172.21.15.5:0/1288080214 >> v1:172.21.15.5:6789/0 conn(0x7fd6080d0f20 legacy=0x7fd60806a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd607fff700 1 -- 172.21.15.5:0/1288080214 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080d0440 con 0x7fd60815e830 2024-05-08T23:27:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd607fff700 1 --2- 172.21.15.5:0/1288080214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd60815e830 0x7fd6080ceb50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009840 tx=0x7fd5f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5669ffdd3893adc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd606ffd700 1 -- 172.21.15.5:0/1288080214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8011830 con 0x7fd60815e830 2024-05-08T23:27:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd606ffd700 1 -- 172.21.15.5:0/1288080214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8011990 con 0x7fd60815e830 2024-05-08T23:27:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.387+0000 7fd606ffd700 1 -- 172.21.15.5:0/1288080214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8011d20 con 0x7fd60815e830 2024-05-08T23:27:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.388+0000 7fd60e444700 1 -- 172.21.15.5:0/1288080214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd60815e830 msgr2=0x7fd6080ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.388+0000 7fd60e444700 1 --2- 172.21.15.5:0/1288080214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd60815e830 0x7fd6080ceb50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009840 tx=0x7fd5f8009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.388+0000 7fd60e444700 1 -- 172.21.15.5:0/1288080214 shutdown_connections 2024-05-08T23:27:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.388+0000 7fd60e444700 1 --2- 172.21.15.5:0/1288080214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd60815e830 0x7fd6080ceb50 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.388+0000 7fd60e444700 1 -- 172.21.15.5:0/1288080214 >> 172.21.15.5:0/1288080214 conn(0x7fd6080b5810 msgr2=0x7fd6080b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.388+0000 7fd60e444700 1 -- 172.21.15.5:0/1288080214 shutdown_connections 2024-05-08T23:27:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.388+0000 7fd60e444700 1 -- 172.21.15.5:0/1288080214 wait complete. 2024-05-08T23:27:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.389+0000 7fd60e444700 1 Processor -- start 2024-05-08T23:27:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.390+0000 7fd60e444700 1 -- start start 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.390+0000 7fd60e444700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd60806b0e0 0x7fd6080c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.390+0000 7fd60e444700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6080d0f20 0x7fd608152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.390+0000 7fd60e444700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd60815e830 0x7fd6081e7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.390+0000 7fd60e444700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b0ee0 con 0x7fd6080d0f20 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.390+0000 7fd60e444700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd608152d40 con 0x7fd60815e830 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.390+0000 7fd60e444700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b08d0 con 0x7fd60806b0e0 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6080d0f20 0x7fd608152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6080d0f20 0x7fd608152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35174/0 (socket says 172.21.15.5:35174) 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd607fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd60806b0e0 0x7fd6080c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 -- 172.21.15.5:0/1767980540 learned_addr learned my addr 172.21.15.5:0/1767980540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd60c9e1700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd60815e830 0x7fd6081e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 -- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd60806b0e0 msgr2=0x7fd6080c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd60806b0e0 0x7fd6080c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 -- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd60815e830 msgr2=0x7fd6081e7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd60815e830 0x7fd6081e7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 -- 172.21.15.5:0/1767980540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f80094a0 con 0x7fd6080d0f20 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd6077fe700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6080d0f20 0x7fd608152800 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c520 tx=0x7fd5fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd604ff9700 1 -- 172.21.15.5:0/1767980540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc005d70 con 0x7fd6080d0f20 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd604ff9700 1 -- 172.21.15.5:0/1767980540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc0070d0 con 0x7fd6080d0f20 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6081e7c50 con 0x7fd6080d0f20 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd604ff9700 1 -- 172.21.15.5:0/1767980540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc007440 con 0x7fd6080d0f20 2024-05-08T23:27:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.391+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6081e7f30 con 0x7fd6080d0f20 2024-05-08T23:27:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.392+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6080d00d0 con 0x7fd6080d0f20 2024-05-08T23:27:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.393+0000 7fd604ff9700 1 -- 172.21.15.5:0/1767980540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd5fc0075a0 con 0x7fd6080d0f20 2024-05-08T23:27:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.393+0000 7fd604ff9700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5f406ac90 0x7fd5f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.393+0000 7fd604ff9700 1 -- 172.21.15.5:0/1767980540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5fc088690 con 0x7fd6080d0f20 2024-05-08T23:27:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.393+0000 7fd607fff700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5f406ac90 0x7fd5f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.394+0000 7fd607fff700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5f406ac90 0x7fd5f406d150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80090b0 tx=0x7fd5f801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.398+0000 7fd604ff9700 1 -- 172.21.15.5:0/1767980540 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd5fc053890 con 0x7fd6080d0f20 2024-05-08T23:27:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.666+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd608068000 con 0x7fd6080d0f20 2024-05-08T23:27:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.667+0000 7fd604ff9700 1 -- 172.21.15.5:0/1767980540 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7fd5fc056eb0 con 0x7fd6080d0f20 2024-05-08T23:27:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:31.667 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5f406ac90 msgr2=0x7fd5f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5f406ac90 0x7fd5f406d150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80090b0 tx=0x7fd5f801c040 comp rx=0 tx=0).stop 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6080d0f20 msgr2=0x7fd608152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6080d0f20 0x7fd608152800 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c520 tx=0x7fd5fc00c830 comp rx=0 tx=0).stop 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 shutdown_connections 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5f406ac90 0x7fd5f406d150 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd60815e830 0x7fd6081e7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd60806b0e0 0x7fd6080c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 --2- 172.21.15.5:0/1767980540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6080d0f20 0x7fd608152800 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 >> 172.21.15.5:0/1767980540 conn(0x7fd6080b5810 msgr2=0x7fd60806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 shutdown_connections 2024-05-08T23:27:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.669+0000 7fd60e444700 1 -- 172.21.15.5:0/1767980540 wait complete. 2024-05-08T23:27:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:31.683 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-05-08T23:27:31.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:27:31.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.818+0000 7f9ffd905700 1 Processor -- start 2024-05-08T23:27:31.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.818+0000 7f9ffd905700 1 -- start start 2024-05-08T23:27:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ffd905700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ff815d780 0x7f9ff8161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ffd905700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9ff815db60 0x7f9ff81619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ffd905700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff806c9c0 con 0x7f9ff806c450 2024-05-08T23:27:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ffd905700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff806cca0 con 0x7f9ff815db60 2024-05-08T23:27:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ffd905700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff806cf80 con 0x7f9ff815d780 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ff67fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9ff815db60 0x7f9ff81619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ff77fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9ff806c450 0x7f9ff80ce3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52054/0 (socket says 172.21.15.5:52054) 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ff77fe700 1 -- 172.21.15.5:0/933319353 learned_addr learned my addr 172.21.15.5:0/933319353 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.819+0000 7f9ff6ffd700 1 --2- 172.21.15.5:0/933319353 >> v2:172.21.15.49:3300/0 conn(0x7f9ff815d780 0x7f9ff8161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff67fc700 1 -- 172.21.15.5:0/933319353 >> v2:172.21.15.49:3300/0 conn(0x7f9ff815d780 msgr2=0x7f9ff8161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff67fc700 1 --2- 172.21.15.5:0/933319353 >> v2:172.21.15.49:3300/0 conn(0x7f9ff815d780 0x7f9ff8161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff67fc700 1 -- 172.21.15.5:0/933319353 >> v1:172.21.15.5:6789/0 conn(0x7f9ff806c450 legacy=0x7f9ff80ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff67fc700 1 -- 172.21.15.5:0/933319353 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff806d260 con 0x7f9ff815db60 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff5ffb700 1 -- 172.21.15.5:0/933319353 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246169506 0 0) 0x7f9ff806c9c0 con 0x7f9ff806c450 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff67fc700 1 --2- 172.21.15.5:0/933319353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815db60 0x7f9ff81619b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9ff808ba20 tx=0x7f9fe0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a847d55f675c15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff5ffb700 1 -- 172.21.15.5:0/933319353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0011830 con 0x7f9ff815db60 2024-05-08T23:27:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff5ffb700 1 -- 172.21.15.5:0/933319353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe0011990 con 0x7f9ff815db60 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.820+0000 7f9ff5ffb700 1 -- 172.21.15.5:0/933319353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0011d20 con 0x7f9ff815db60 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.821+0000 7f9ffd905700 1 -- 172.21.15.5:0/933319353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815db60 msgr2=0x7f9ff81619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.821+0000 7f9ffd905700 1 --2- 172.21.15.5:0/933319353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815db60 0x7f9ff81619b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9ff808ba20 tx=0x7f9fe0009b20 comp rx=0 tx=0).stop 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.821+0000 7f9ffd905700 1 -- 172.21.15.5:0/933319353 shutdown_connections 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.821+0000 7f9ffd905700 1 --2- 172.21.15.5:0/933319353 >> v2:172.21.15.49:3300/0 conn(0x7f9ff815d780 0x7f9ff8161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.821+0000 7f9ffd905700 1 --2- 172.21.15.5:0/933319353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815db60 0x7f9ff81619b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.821+0000 7f9ffd905700 1 -- 172.21.15.5:0/933319353 >> 172.21.15.5:0/933319353 conn(0x7f9ff808afe0 msgr2=0x7f9ff80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.821+0000 7f9ffd905700 1 -- 172.21.15.5:0/933319353 shutdown_connections 2024-05-08T23:27:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.821+0000 7f9ffd905700 1 -- 172.21.15.5:0/933319353 wait complete. 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.822+0000 7f9ffd905700 1 Processor -- start 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.823+0000 7f9ffd905700 1 -- start start 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ffd905700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ff806c450 0x7f9ff80b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ffd905700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815d780 0x7f9ff80ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ffd905700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff815db60 0x7f9ff80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ffd905700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80b4ba0 con 0x7f9ff815d780 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ffd905700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80b4e40 con 0x7f9ff815db60 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ffd905700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80b5120 con 0x7f9ff806c450 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815d780 0x7f9ff80ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815d780 0x7f9ff80ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35198/0 (socket says 172.21.15.5:35198) 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 -- 172.21.15.5:0/1838139589 learned_addr learned my addr 172.21.15.5:0/1838139589 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff77fe700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff815db60 0x7f9ff80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff6ffd700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ff806c450 0x7f9ff80b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 -- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ff806c450 msgr2=0x7f9ff80b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ff806c450 0x7f9ff80b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 -- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff815db60 msgr2=0x7f9ff80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff815db60 0x7f9ff80cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 -- 172.21.15.5:0/1838139589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe00094a0 con 0x7f9ff815d780 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.824+0000 7f9ff67fc700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815d780 0x7f9ff80ccc00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009b60 tx=0x7f9fe0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.825+0000 7f9feffff700 1 -- 172.21.15.5:0/1838139589 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00036a0 con 0x7f9ff815d780 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.825+0000 7f9feffff700 1 -- 172.21.15.5:0/1838139589 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe001e070 con 0x7f9ff815d780 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.825+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff80b5400 con 0x7f9ff815d780 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.825+0000 7f9feffff700 1 -- 172.21.15.5:0/1838139589 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0003a60 con 0x7f9ff815d780 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.825+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff80b59c0 con 0x7f9ff815d780 2024-05-08T23:27:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.826+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff80678f0 con 0x7f9ff815d780 2024-05-08T23:27:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.826+0000 7f9feffff700 1 -- 172.21.15.5:0/1838139589 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9fe0003bc0 con 0x7f9ff815d780 2024-05-08T23:27:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.827+0000 7f9feffff700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9fd806ac90 0x7f9fd806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.827+0000 7f9feffff700 1 -- 172.21.15.5:0/1838139589 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fe008a280 con 0x7f9ff815d780 2024-05-08T23:27:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.827+0000 7f9ff6ffd700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9fd806ac90 0x7f9fd806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.828+0000 7f9ff6ffd700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9fd806ac90 0x7f9fd806d150 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009ab0 tx=0x7f9fe8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:31.831+0000 7f9feffff700 1 -- 172.21.15.5:0/1838139589 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9fe00554f0 con 0x7f9ff815d780 2024-05-08T23:27:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.142+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9ff806da60 con 0x7f9ff815d780 2024-05-08T23:27:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.142+0000 7f9feffff700 1 -- 172.21.15.5:0/1838139589 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1965 (secure 0 0 0) 0x7f9fe0058b10 con 0x7f9ff815d780 2024-05-08T23:27:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:32.143 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":406,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700348,"bytes_used":256286720,"bytes_avail":772837826560,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6654,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7336},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7755},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7758}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-05-08T23:27:27.387490+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:27:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9fd806ac90 msgr2=0x7f9fd806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9fd806ac90 0x7f9fd806d150 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009ab0 tx=0x7f9fe8008040 comp rx=0 tx=0).stop 2024-05-08T23:27:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815d780 msgr2=0x7f9ff80ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815d780 0x7f9ff80ccc00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009b60 tx=0x7f9fe0004750 comp rx=0 tx=0).stop 2024-05-08T23:27:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 shutdown_connections 2024-05-08T23:27:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9fd806ac90 0x7f9fd806d150 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff815db60 0x7f9ff80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9ff806c450 0x7f9ff80b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 --2- 172.21.15.5:0/1838139589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff815d780 0x7f9ff80ccc00 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 >> 172.21.15.5:0/1838139589 conn(0x7f9ff808afe0 msgr2=0x7f9ff80b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 shutdown_connections 2024-05-08T23:27:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:32.145+0000 7f9ffd905700 1 -- 172.21.15.5:0/1838139589 wait complete. 2024-05-08T23:27:32.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:27:33.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.309+0000 7fb0f793d700 1 Processor -- start 2024-05-08T23:27:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.309+0000 7fb0f793d700 1 -- start start 2024-05-08T23:27:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f793d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb0f006e6a0 0x7fb0f006d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f793d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f00b5860 0x7fb0f006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f793d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f0163130 con 0x7fb0f006beb0 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f793d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f006c340 con 0x7fb0f006e6a0 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f793d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f0162310 con 0x7fb0f00b5860 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f56d9700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb0f006e6a0 0x7fb0f006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f4ed8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f00b5860 0x7fb0f006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f5eda700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb0f006beb0 0x7fb0f0165900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:55472/0 (socket says 172.21.15.5:55472) 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f56d9700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb0f006e6a0 0x7fb0f006d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58334/0 (socket says 172.21.15.5:58334) 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f4ed8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f00b5860 0x7fb0f006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:58182/0 (socket says 172.21.15.5:58182) 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.310+0000 7fb0f5eda700 1 -- 172.21.15.5:0/1951752413 learned_addr learned my addr 172.21.15.5:0/1951752413 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0f4ed8700 1 -- 172.21.15.5:0/1951752413 >> v2:172.21.15.49:3300/0 conn(0x7fb0f006e6a0 msgr2=0x7fb0f006d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0f4ed8700 1 --2- 172.21.15.5:0/1951752413 >> v2:172.21.15.49:3300/0 conn(0x7fb0f006e6a0 0x7fb0f006d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0f4ed8700 1 -- 172.21.15.5:0/1951752413 >> v1:172.21.15.49:6789/0 conn(0x7fb0f006beb0 legacy=0x7fb0f0165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0f4ed8700 1 -- 172.21.15.5:0/1951752413 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f006ea80 con 0x7fb0f00b5860 2024-05-08T23:27:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0f56d9700 1 --2- 172.21.15.5:0/1951752413 >> v2:172.21.15.49:3300/0 conn(0x7fb0f006e6a0 0x7fb0f006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0f4ed8700 1 --2- 172.21.15.5:0/1951752413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f00b5860 0x7fb0f006b970 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009840 tx=0x7fb0e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=866c9553ebcf7bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0e7fff700 1 -- 172.21.15.5:0/1951752413 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0011830 con 0x7fb0f00b5860 2024-05-08T23:27:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0e7fff700 1 -- 172.21.15.5:0/1951752413 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e0011990 con 0x7fb0f00b5860 2024-05-08T23:27:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.311+0000 7fb0e7fff700 1 -- 172.21.15.5:0/1951752413 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0011d20 con 0x7fb0f00b5860 2024-05-08T23:27:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.312+0000 7fb0f793d700 1 -- 172.21.15.5:0/1951752413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f00b5860 msgr2=0x7fb0f006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.312+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1951752413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f00b5860 0x7fb0f006b970 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009840 tx=0x7fb0e0009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.312+0000 7fb0f793d700 1 -- 172.21.15.5:0/1951752413 shutdown_connections 2024-05-08T23:27:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.312+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1951752413 >> v2:172.21.15.49:3300/0 conn(0x7fb0f006e6a0 0x7fb0f006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.312+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1951752413 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f00b5860 0x7fb0f006b970 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.312+0000 7fb0f793d700 1 -- 172.21.15.5:0/1951752413 >> 172.21.15.5:0/1951752413 conn(0x7fb0f0071350 msgr2=0x7fb0f0071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.312+0000 7fb0f793d700 1 -- 172.21.15.5:0/1951752413 shutdown_connections 2024-05-08T23:27:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.312+0000 7fb0f793d700 1 -- 172.21.15.5:0/1951752413 wait complete. 2024-05-08T23:27:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.313+0000 7fb0f793d700 1 Processor -- start 2024-05-08T23:27:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.313+0000 7fb0f793d700 1 -- start start 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f793d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0f006beb0 0x7fb0f0150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f793d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f006e6a0 0x7fb0f015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f793d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0f00b5860 0x7fb0f00b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f793d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f0151210 con 0x7fb0f006beb0 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f793d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f015e210 con 0x7fb0f00b5860 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f793d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f015e4f0 con 0x7fb0f006e6a0 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f56d9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0f006beb0 0x7fb0f0150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f56d9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0f006beb0 0x7fb0f0150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35214/0 (socket says 172.21.15.5:35214) 2024-05-08T23:27:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f56d9700 1 -- 172.21.15.5:0/1150447211 learned_addr learned my addr 172.21.15.5:0/1150447211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f5eda700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0f00b5860 0x7fb0f00b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f4ed8700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f006e6a0 0x7fb0f015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f56d9700 1 -- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f006e6a0 msgr2=0x7fb0f015d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f56d9700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f006e6a0 0x7fb0f015d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f56d9700 1 -- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0f00b5860 msgr2=0x7fb0f00b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f56d9700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0f00b5860 0x7fb0f00b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f56d9700 1 -- 172.21.15.5:0/1150447211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0094a0 con 0x7fb0f006beb0 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f5eda700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0f00b5860 0x7fb0f00b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.314+0000 7fb0f4ed8700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f006e6a0 0x7fb0f015d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.315+0000 7fb0f56d9700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0f006beb0 0x7fb0f0150cd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00ebe0 tx=0x7fb0ec009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.315+0000 7fb0e5ffb700 1 -- 172.21.15.5:0/1150447211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00c910 con 0x7fb0f006beb0 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.315+0000 7fb0e5ffb700 1 -- 172.21.15.5:0/1150447211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec00ca70 con 0x7fb0f006beb0 2024-05-08T23:27:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.315+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e00094a0 con 0x7fb0f006beb0 2024-05-08T23:27:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.315+0000 7fb0e5ffb700 1 -- 172.21.15.5:0/1150447211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00cde0 con 0x7fb0f006beb0 2024-05-08T23:27:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.315+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f015eab0 con 0x7fb0f006beb0 2024-05-08T23:27:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.316+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f00cc340 con 0x7fb0f006beb0 2024-05-08T23:27:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.317+0000 7fb0e5ffb700 1 -- 172.21.15.5:0/1150447211 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb0ec013470 con 0x7fb0f006beb0 2024-05-08T23:27:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.317+0000 7fb0e5ffb700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0d406ac90 0x7fb0d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.317+0000 7fb0e5ffb700 1 -- 172.21.15.5:0/1150447211 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0ec0894f0 con 0x7fb0f006beb0 2024-05-08T23:27:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.317+0000 7fb0f4ed8700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0d406ac90 0x7fb0d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.318+0000 7fb0f4ed8700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0d406ac90 0x7fb0d406d150 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009fd0 tx=0x7fb0e0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.322+0000 7fb0e5ffb700 1 -- 172.21.15.5:0/1150447211 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb0ec0546e0 con 0x7fb0f006beb0 2024-05-08T23:27:33.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.590+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0f00b2700 con 0x7fb0f006beb0 2024-05-08T23:27:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.590+0000 7fb0e5ffb700 1 -- 172.21.15.5:0/1150447211 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7fb0ec057d00 con 0x7fb0f006beb0 2024-05-08T23:27:33.590 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:33.590 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0d406ac90 msgr2=0x7fb0d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0d406ac90 0x7fb0d406d150 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009fd0 tx=0x7fb0e0008030 comp rx=0 tx=0).stop 2024-05-08T23:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0f006beb0 msgr2=0x7fb0f0150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0f006beb0 0x7fb0f0150cd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00ebe0 tx=0x7fb0ec009ec0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 shutdown_connections 2024-05-08T23:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb0d406ac90 0x7fb0d406d150 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0f00b5860 0x7fb0f00b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0f006beb0 0x7fb0f0150cd0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 --2- 172.21.15.5:0/1150447211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb0f006e6a0 0x7fb0f015d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 >> 172.21.15.5:0/1150447211 conn(0x7fb0f0071350 msgr2=0x7fb0f0160080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 shutdown_connections 2024-05-08T23:27:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.593+0000 7fb0f793d700 1 -- 172.21.15.5:0/1150447211 wait complete. 2024-05-08T23:27:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:33.606 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-05-08T23:27:33.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:27:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.742+0000 7fc7c0f06700 1 Processor -- start 2024-05-08T23:27:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.742+0000 7fc7c0f06700 1 -- start start 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7c0f06700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc06e4d0 0x7fc7bc15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7c0f06700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7bc0b3990 0x7fc7bc06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7c0f06700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7bc0b4a80 con 0x7fc7bc0af030 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7c0f06700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7bc15ede0 con 0x7fc7bc0b3990 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7c0f06700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc06d800 con 0x7fc7bc06e4d0 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7ba59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc7bc0af030 0x7fc7bc06ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52064/0 (socket says 172.21.15.5:52064) 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7ba59c700 1 -- 172.21.15.5:0/2303873030 learned_addr learned my addr 172.21.15.5:0/2303873030 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7b9d9b700 1 --2- 172.21.15.5:0/2303873030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc06e4d0 0x7fc7bc15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7bad9d700 1 --2- 172.21.15.5:0/2303873030 >> v2:172.21.15.49:3300/0 conn(0x7fc7bc0b3990 0x7fc7bc06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7b959a700 1 -- 172.21.15.5:0/2303873030 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367772367 0 0) 0x7fc7bc0b4a80 con 0x7fc7bc0af030 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.743+0000 7fc7b959a700 1 -- 172.21.15.5:0/2303873030 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a40035b0 con 0x7fc7bc0af030 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7b9d9b700 1 -- 172.21.15.5:0/2303873030 >> v2:172.21.15.49:3300/0 conn(0x7fc7bc0b3990 msgr2=0x7fc7bc06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7b9d9b700 1 --2- 172.21.15.5:0/2303873030 >> v2:172.21.15.49:3300/0 conn(0x7fc7bc0b3990 0x7fc7bc06a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7b9d9b700 1 -- 172.21.15.5:0/2303873030 >> v1:172.21.15.5:6789/0 conn(0x7fc7bc0af030 legacy=0x7fc7bc06ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7b9d9b700 1 -- 172.21.15.5:0/2303873030 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc0b3e70 con 0x7fc7bc06e4d0 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7bad9d700 1 --2- 172.21.15.5:0/2303873030 >> v2:172.21.15.49:3300/0 conn(0x7fc7bc0b3990 0x7fc7bc06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7b9d9b700 1 --2- 172.21.15.5:0/2303873030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc06e4d0 0x7fc7bc15e8a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009840 tx=0x7fc7b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ddb9b88421291b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7b959a700 1 -- 172.21.15.5:0/2303873030 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0011830 con 0x7fc7bc06e4d0 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7b959a700 1 -- 172.21.15.5:0/2303873030 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b0011990 con 0x7fc7bc06e4d0 2024-05-08T23:27:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.744+0000 7fc7b959a700 1 -- 172.21.15.5:0/2303873030 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0011d20 con 0x7fc7bc06e4d0 2024-05-08T23:27:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.745+0000 7fc7c0f06700 1 -- 172.21.15.5:0/2303873030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc06e4d0 msgr2=0x7fc7bc15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.745+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/2303873030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc06e4d0 0x7fc7bc15e8a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009840 tx=0x7fc7b0009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.745+0000 7fc7c0f06700 1 -- 172.21.15.5:0/2303873030 shutdown_connections 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.745+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/2303873030 >> v2:172.21.15.49:3300/0 conn(0x7fc7bc0b3990 0x7fc7bc06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.745+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/2303873030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc06e4d0 0x7fc7bc15e8a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.745+0000 7fc7c0f06700 1 -- 172.21.15.5:0/2303873030 >> 172.21.15.5:0/2303873030 conn(0x7fc7bc0cfce0 msgr2=0x7fc7bc0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.745+0000 7fc7c0f06700 1 -- 172.21.15.5:0/2303873030 shutdown_connections 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.745+0000 7fc7c0f06700 1 -- 172.21.15.5:0/2303873030 wait complete. 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.746+0000 7fc7c0f06700 1 Processor -- start 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.746+0000 7fc7c0f06700 1 -- start start 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7c0f06700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7bc06e4d0 0x7fc7bc0ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7c0f06700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc0af030 0x7fc7bc150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7c0f06700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7bc0b3990 0x7fc7bc155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7c0f06700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0ce8d0 con 0x7fc7bc0b3990 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7c0f06700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc150f70 con 0x7fc7bc06e4d0 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7c0f06700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc1558d0 con 0x7fc7bc0af030 2024-05-08T23:27:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7bad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7bc0b3990 0x7fc7bc155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7bad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7bc0b3990 0x7fc7bc155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35224/0 (socket says 172.21.15.5:35224) 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7bad9d700 1 -- 172.21.15.5:0/1279826212 learned_addr learned my addr 172.21.15.5:0/1279826212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.747+0000 7fc7b9d9b700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc0af030 0x7fc7bc150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7ba59c700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7bc06e4d0 0x7fc7bc0ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7bad9d700 1 -- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc0af030 msgr2=0x7fc7bc150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7bad9d700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc0af030 0x7fc7bc150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7bad9d700 1 -- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7bc06e4d0 msgr2=0x7fc7bc0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7bad9d700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7bc06e4d0 0x7fc7bc0ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7bad9d700 1 -- 172.21.15.5:0/1279826212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4009a70 con 0x7fc7bc0b3990 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7bad9d700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7bc0b3990 0x7fc7bc155d60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009ea0 tx=0x7fc7b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7aaffd700 1 -- 172.21.15.5:0/1279826212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b400cde0 con 0x7fc7bc0b3990 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7aaffd700 1 -- 172.21.15.5:0/1279826212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b400f040 con 0x7fc7bc0b3990 2024-05-08T23:27:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b00094a0 con 0x7fc7bc0b3990 2024-05-08T23:27:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7aaffd700 1 -- 172.21.15.5:0/1279826212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40146c0 con 0x7fc7bc0b3990 2024-05-08T23:27:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.748+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc0ce5f0 con 0x7fc7bc0b3990 2024-05-08T23:27:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.749+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc155a60 con 0x7fc7bc0b3990 2024-05-08T23:27:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.750+0000 7fc7aaffd700 1 -- 172.21.15.5:0/1279826212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc7b4014820 con 0x7fc7bc0b3990 2024-05-08T23:27:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.750+0000 7fc7aaffd700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc79c06acc0 0x7fc79c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.751+0000 7fc7aaffd700 1 -- 172.21.15.5:0/1279826212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7b408a6c0 con 0x7fc7bc0b3990 2024-05-08T23:27:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.751+0000 7fc7ba59c700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc79c06acc0 0x7fc79c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.751+0000 7fc7ba59c700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc79c06acc0 0x7fc79c06d180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0011e0 tx=0x7fc7ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:33.755+0000 7fc7aaffd700 1 -- 172.21.15.5:0/1279826212 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc7b40557f0 con 0x7fc7bc0b3990 2024-05-08T23:27:34.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.067+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc7bc068000 con 0x7fc7bc0b3990 2024-05-08T23:27:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.068+0000 7fc7aaffd700 1 -- 172.21.15.5:0/1279826212 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1965 (secure 0 0 0) 0x7fc7b40187c0 con 0x7fc7bc0b3990 2024-05-08T23:27:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:34.068 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":408,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700348,"bytes_used":256286720,"bytes_avail":772837826560,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6654,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7336},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7755},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7758}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-05-08T23:27:27.387490+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:27:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc79c06acc0 msgr2=0x7fc79c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc79c06acc0 0x7fc79c06d180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0011e0 tx=0x7fc7ac009040 comp rx=0 tx=0).stop 2024-05-08T23:27:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7bc0b3990 msgr2=0x7fc7bc155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7bc0b3990 0x7fc7bc155d60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009ea0 tx=0x7fc7b400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:27:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 shutdown_connections 2024-05-08T23:27:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc79c06acc0 0x7fc79c06d180 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7bc0b3990 0x7fc7bc155d60 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7bc06e4d0 0x7fc7bc0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 --2- 172.21.15.5:0/1279826212 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7bc0af030 0x7fc7bc150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 >> 172.21.15.5:0/1279826212 conn(0x7fc7bc0cfce0 msgr2=0x7fc7bc06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 shutdown_connections 2024-05-08T23:27:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:34.070+0000 7fc7c0f06700 1 -- 172.21.15.5:0/1279826212 wait complete. 2024-05-08T23:27:34.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:27:35.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:35.123 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:27:35.123+0000 7f8a4c50b700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T23:27:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.230+0000 7fa227dad700 1 Processor -- start 2024-05-08T23:27:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.230+0000 7fa227dad700 1 -- start start 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa227dad700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa22006e6a0 0x7fa22006d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa227dad700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa2200b5860 0x7fa22006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa227dad700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa220163130 con 0x7fa22006beb0 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa227dad700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa220162310 con 0x7fa2200b5860 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa227dad700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22006c2e0 con 0x7fa22006e6a0 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa225348700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa2200b5860 0x7fa22006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa225348700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa2200b5860 0x7fa22006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35226/0 (socket says 172.21.15.5:35226) 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa225348700 1 -- 172.21.15.5:0/3109888362 learned_addr learned my addr 172.21.15.5:0/3109888362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa225b49700 1 --2- 172.21.15.5:0/3109888362 >> v2:172.21.15.49:3300/0 conn(0x7fa22006e6a0 0x7fa22006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa225348700 1 -- 172.21.15.5:0/3109888362 >> v2:172.21.15.49:3300/0 conn(0x7fa22006e6a0 msgr2=0x7fa22006d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa225348700 1 --2- 172.21.15.5:0/3109888362 >> v2:172.21.15.49:3300/0 conn(0x7fa22006e6a0 0x7fa22006d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa225348700 1 -- 172.21.15.5:0/3109888362 >> v1:172.21.15.49:6789/0 conn(0x7fa22006beb0 legacy=0x7fa220165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.231+0000 7fa225348700 1 -- 172.21.15.5:0/3109888362 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22006ea80 con 0x7fa2200b5860 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa225348700 1 --2- 172.21.15.5:0/3109888362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2200b5860 0x7fa22006b970 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa22006cf30 tx=0x7fa210009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff578d371c961099 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa224b47700 1 -- 172.21.15.5:0/3109888362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210011830 con 0x7fa2200b5860 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa224b47700 1 -- 172.21.15.5:0/3109888362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa210011990 con 0x7fa2200b5860 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa224b47700 1 -- 172.21.15.5:0/3109888362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210011d20 con 0x7fa2200b5860 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa227dad700 1 -- 172.21.15.5:0/3109888362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2200b5860 msgr2=0x7fa22006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa227dad700 1 --2- 172.21.15.5:0/3109888362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2200b5860 0x7fa22006b970 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa22006cf30 tx=0x7fa210009bd0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa227dad700 1 -- 172.21.15.5:0/3109888362 shutdown_connections 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa227dad700 1 --2- 172.21.15.5:0/3109888362 >> v2:172.21.15.49:3300/0 conn(0x7fa22006e6a0 0x7fa22006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa227dad700 1 --2- 172.21.15.5:0/3109888362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2200b5860 0x7fa22006b970 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.232+0000 7fa227dad700 1 -- 172.21.15.5:0/3109888362 >> 172.21.15.5:0/3109888362 conn(0x7fa220071350 msgr2=0x7fa220071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.233+0000 7fa227dad700 1 -- 172.21.15.5:0/3109888362 shutdown_connections 2024-05-08T23:27:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.233+0000 7fa227dad700 1 -- 172.21.15.5:0/3109888362 wait complete. 2024-05-08T23:27:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.234+0000 7fa227dad700 1 Processor -- start 2024-05-08T23:27:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.234+0000 7fa227dad700 1 -- start start 2024-05-08T23:27:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa227dad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22006beb0 0x7fa22015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa227dad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22006e6a0 0x7fa2200a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa227dad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2200b5860 0x7fa2200a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa227dad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22015f090 con 0x7fa22006e6a0 2024-05-08T23:27:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa227dad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22015dff0 con 0x7fa22006beb0 2024-05-08T23:27:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa227dad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22015e2d0 con 0x7fa2200b5860 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa225348700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22006e6a0 0x7fa2200a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa225348700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22006e6a0 0x7fa2200a8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35230/0 (socket says 172.21.15.5:35230) 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.235+0000 7fa225348700 1 -- 172.21.15.5:0/2632733766 learned_addr learned my addr 172.21.15.5:0/2632733766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa225b49700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22006beb0 0x7fa22015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa22634a700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2200b5860 0x7fa2200a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa225348700 1 -- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2200b5860 msgr2=0x7fa2200a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa225348700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2200b5860 0x7fa2200a9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa225348700 1 -- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22006beb0 msgr2=0x7fa22015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa225348700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22006beb0 0x7fa22015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa225348700 1 -- 172.21.15.5:0/2632733766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa210009550 con 0x7fa22006e6a0 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa225348700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22006e6a0 0x7fa2200a8cf0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa210000c00 tx=0x7fa210004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa2167fc700 1 -- 172.21.15.5:0/2632733766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210004570 con 0x7fa22006e6a0 2024-05-08T23:27:35.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa2167fc700 1 -- 172.21.15.5:0/2632733766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21001f070 con 0x7fa22006e6a0 2024-05-08T23:27:35.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22015e5b0 con 0x7fa22006e6a0 2024-05-08T23:27:35.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa2167fc700 1 -- 172.21.15.5:0/2632733766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2100038e0 con 0x7fa22006e6a0 2024-05-08T23:27:35.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.236+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22015eb70 con 0x7fa22006e6a0 2024-05-08T23:27:35.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.237+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2200b2700 con 0x7fa22006e6a0 2024-05-08T23:27:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.238+0000 7fa2167fc700 1 -- 172.21.15.5:0/2632733766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa210021030 con 0x7fa22006e6a0 2024-05-08T23:27:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.238+0000 7fa2167fc700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa20406ac90 0x7fa20406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.238+0000 7fa2167fc700 1 -- 172.21.15.5:0/2632733766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa21008a780 con 0x7fa22006e6a0 2024-05-08T23:27:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.238+0000 7fa225b49700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa20406ac90 0x7fa20406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.239+0000 7fa225b49700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa20406ac90 0x7fa20406d150 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009b70 tx=0x7fa21c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:35.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.243+0000 7fa2167fc700 1 -- 172.21.15.5:0/2632733766 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa210055b10 con 0x7fa22006e6a0 2024-05-08T23:27:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.508+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa220068000 con 0x7fa22006e6a0 2024-05-08T23:27:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.508+0000 7fa2167fc700 1 -- 172.21.15.5:0/2632733766 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7fa210015090 con 0x7fa22006e6a0 2024-05-08T23:27:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:35.509 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa20406ac90 msgr2=0x7fa20406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa20406ac90 0x7fa20406d150 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009b70 tx=0x7fa21c008040 comp rx=0 tx=0).stop 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22006e6a0 msgr2=0x7fa2200a8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22006e6a0 0x7fa2200a8cf0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa210000c00 tx=0x7fa210004750 comp rx=0 tx=0).stop 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 shutdown_connections 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa20406ac90 0x7fa20406d150 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa2200b5860 0x7fa2200a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22006beb0 0x7fa22015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 --2- 172.21.15.5:0/2632733766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22006e6a0 0x7fa2200a8cf0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 >> 172.21.15.5:0/2632733766 conn(0x7fa220071350 msgr2=0x7fa220071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 shutdown_connections 2024-05-08T23:27:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.511+0000 7fa227dad700 1 -- 172.21.15.5:0/2632733766 wait complete. 2024-05-08T23:27:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:35.525 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-05-08T23:27:35.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:27:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.702+0000 7fdcad90d700 1 Processor -- start 2024-05-08T23:27:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.702+0000 7fdcad90d700 1 -- start start 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdcad90d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdca815d780 0x7fdca8161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdcad90d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca806c450 0x7fdca81619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdcad90d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca806c9c0 con 0x7fdca815db60 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdcad90d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca806cca0 con 0x7fdca815d780 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdcad90d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca806cf80 con 0x7fdca806c450 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca67fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdca815db60 0x7fdca80ce3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52068/0 (socket says 172.21.15.5:52068) 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca67fc700 1 -- 172.21.15.5:0/2915617538 learned_addr learned my addr 172.21.15.5:0/2915617538 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca6ffd700 1 --2- 172.21.15.5:0/2915617538 >> v2:172.21.15.49:3300/0 conn(0x7fdca815d780 0x7fdca8161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca77fe700 1 --2- 172.21.15.5:0/2915617538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca806c450 0x7fdca81619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca5ffb700 1 -- 172.21.15.5:0/2915617538 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040315868 0 0) 0x7fdca806c9c0 con 0x7fdca815db60 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca5ffb700 1 -- 172.21.15.5:0/2915617538 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc900035b0 con 0x7fdca815db60 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca77fe700 1 -- 172.21.15.5:0/2915617538 >> v2:172.21.15.49:3300/0 conn(0x7fdca815d780 msgr2=0x7fdca8161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca77fe700 1 --2- 172.21.15.5:0/2915617538 >> v2:172.21.15.49:3300/0 conn(0x7fdca815d780 0x7fdca8161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.703+0000 7fdca77fe700 1 -- 172.21.15.5:0/2915617538 >> v1:172.21.15.5:6789/0 conn(0x7fdca815db60 legacy=0x7fdca80ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.704+0000 7fdca77fe700 1 -- 172.21.15.5:0/2915617538 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca806d260 con 0x7fdca806c450 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.704+0000 7fdca77fe700 1 --2- 172.21.15.5:0/2915617538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca806c450 0x7fdca81619b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009d90 tx=0x7fdca000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1189ccad159204a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.704+0000 7fdca5ffb700 1 -- 172.21.15.5:0/2915617538 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000e070 con 0x7fdca806c450 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.704+0000 7fdca5ffb700 1 -- 172.21.15.5:0/2915617538 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca000bb20 con 0x7fdca806c450 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.704+0000 7fdca5ffb700 1 -- 172.21.15.5:0/2915617538 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000be90 con 0x7fdca806c450 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.705+0000 7fdcad90d700 1 -- 172.21.15.5:0/2915617538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca806c450 msgr2=0x7fdca81619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.705+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2915617538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca806c450 0x7fdca81619b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009d90 tx=0x7fdca000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.705+0000 7fdcad90d700 1 -- 172.21.15.5:0/2915617538 shutdown_connections 2024-05-08T23:27:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.705+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2915617538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca806c450 0x7fdca81619b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.705+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2915617538 >> v2:172.21.15.49:3300/0 conn(0x7fdca815d780 0x7fdca8161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.705+0000 7fdcad90d700 1 -- 172.21.15.5:0/2915617538 >> 172.21.15.5:0/2915617538 conn(0x7fdca808afe0 msgr2=0x7fdca80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.705+0000 7fdcad90d700 1 -- 172.21.15.5:0/2915617538 shutdown_connections 2024-05-08T23:27:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.705+0000 7fdcad90d700 1 -- 172.21.15.5:0/2915617538 wait complete. 2024-05-08T23:27:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.706+0000 7fdcad90d700 1 Processor -- start 2024-05-08T23:27:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.706+0000 7fdcad90d700 1 -- start start 2024-05-08T23:27:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdcad90d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca806c450 0x7fdca8151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdcad90d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca815d780 0x7fdca80ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdcad90d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca815db60 0x7fdca80cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdcad90d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b6de0 con 0x7fdca806c450 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdcad90d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80c7640 con 0x7fdca815db60 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdcad90d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80c81c0 con 0x7fdca815d780 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdca6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca806c450 0x7fdca8151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdca6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca806c450 0x7fdca8151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35240/0 (socket says 172.21.15.5:35240) 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdca6ffd700 1 -- 172.21.15.5:0/2017907428 learned_addr learned my addr 172.21.15.5:0/2017907428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdca77fe700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca815db60 0x7fdca80cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdca67fc700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca815d780 0x7fdca80ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdca6ffd700 1 -- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca815d780 msgr2=0x7fdca80ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdca6ffd700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca815d780 0x7fdca80ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.707+0000 7fdca6ffd700 1 -- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca815db60 msgr2=0x7fdca80cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdca6ffd700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca815db60 0x7fdca80cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdca6ffd700 1 -- 172.21.15.5:0/2017907428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0009a70 con 0x7fdca806c450 2024-05-08T23:27:35.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdca77fe700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca815db60 0x7fdca80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdca6ffd700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca806c450 0x7fdca8151a10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800e9b0 tx=0x7fdc9800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdc8ffff700 1 -- 172.21.15.5:0/2017907428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800c850 con 0x7fdca806c450 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdc8ffff700 1 -- 172.21.15.5:0/2017907428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9800c9b0 con 0x7fdca806c450 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca8151f50 con 0x7fdca806c450 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdc8ffff700 1 -- 172.21.15.5:0/2017907428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800cd20 con 0x7fdca806c450 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.708+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca80c7b70 con 0x7fdca806c450 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.709+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca80678f0 con 0x7fdca806c450 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.709+0000 7fdc8ffff700 1 -- 172.21.15.5:0/2017907428 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdc98013470 con 0x7fdca806c450 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.710+0000 7fdc8ffff700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdc9406acc0 0x7fdc9406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.710+0000 7fdc8ffff700 1 -- 172.21.15.5:0/2017907428 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc980894b0 con 0x7fdca806c450 2024-05-08T23:27:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.710+0000 7fdca67fc700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdc9406acc0 0x7fdc9406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.711+0000 7fdca67fc700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdc9406acc0 0x7fdc9406d180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdca8155420 tx=0x7fdc9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:35.715+0000 7fdc8ffff700 1 -- 172.21.15.5:0/2017907428 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdc98054610 con 0x7fdca806c450 2024-05-08T23:27:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.026+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdca80b9460 con 0x7fdca806c450 2024-05-08T23:27:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.026+0000 7fdc8ffff700 1 -- 172.21.15.5:0/2017907428 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1983 (secure 0 0 0) 0x7fdc98057c30 con 0x7fdca806c450 2024-05-08T23:27:36.026 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:36.026 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":705854,"bytes_used":256303104,"bytes_avail":772837810176,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7336,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7336},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7755},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7758}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-05-08T23:27:33.388874+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdc9406acc0 msgr2=0x7fdc9406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdc9406acc0 0x7fdc9406d180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdca8155420 tx=0x7fdc9c009040 comp rx=0 tx=0).stop 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca806c450 msgr2=0x7fdca8151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca806c450 0x7fdca8151a10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800e9b0 tx=0x7fdc9800ed70 comp rx=0 tx=0).stop 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 shutdown_connections 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdc9406acc0 0x7fdc9406d180 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca815db60 0x7fdca80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca806c450 0x7fdca8151a10 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 --2- 172.21.15.5:0/2017907428 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdca815d780 0x7fdca80ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 >> 172.21.15.5:0/2017907428 conn(0x7fdca808afe0 msgr2=0x7fdca80d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 shutdown_connections 2024-05-08T23:27:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:36.028+0000 7fdcad90d700 1 -- 172.21.15.5:0/2017907428 wait complete. 2024-05-08T23:27:36.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-05-08T23:27:37.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:37.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.187+0000 7f32314ce700 1 Processor -- start 2024-05-08T23:27:37.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.187+0000 7f32314ce700 1 -- start start 2024-05-08T23:27:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.188+0000 7f32314ce700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c15a380 0x7f322c0cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.188+0000 7f32314ce700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f322c0cf3c0 0x7f322c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.188+0000 7f32314ce700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0b2b10 con 0x7f322c06f4a0 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.188+0000 7f32314ce700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0ceba0 con 0x7f322c0cf3c0 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.188+0000 7f32314ce700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c06f950 con 0x7f322c15a380 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322a7fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f322c0cf3c0 0x7f322c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322affd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c15a380 0x7f322c0cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322a7fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f322c0cf3c0 0x7f322c06ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:58404/0 (socket says 172.21.15.5:58404) 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322a7fc700 1 -- 172.21.15.5:0/762965037 learned_addr learned my addr 172.21.15.5:0/762965037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322affd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c15a380 0x7f322c0cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:58248/0 (socket says 172.21.15.5:58248) 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322b7fe700 1 --1- 172.21.15.5:0/762965037 >> v1:172.21.15.49:6789/0 conn(0x7f322c06f4a0 0x7f322c15e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:55496/0 (socket says 172.21.15.5:55496) 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322a7fc700 1 -- 172.21.15.5:0/762965037 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c15a380 msgr2=0x7f322c0cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322a7fc700 1 --2- 172.21.15.5:0/762965037 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c15a380 0x7f322c0cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322a7fc700 1 -- 172.21.15.5:0/762965037 >> v1:172.21.15.49:6789/0 conn(0x7f322c06f4a0 legacy=0x7f322c15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322a7fc700 1 -- 172.21.15.5:0/762965037 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0ce340 con 0x7f322c0cf3c0 2024-05-08T23:27:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f3229ffb700 1 -- 172.21.15.5:0/762965037 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193409569 0 0) 0x7f322c0b2b10 con 0x7f322c06f4a0 2024-05-08T23:27:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322affd700 1 --2- 172.21.15.5:0/762965037 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c15a380 0x7f322c0cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.189+0000 7f322a7fc700 1 --2- 172.21.15.5:0/762965037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c0cf3c0 0x7f322c06ef60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f321c0098f0 tx=0x7f321c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b5d2f73186bbb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f3229ffb700 1 -- 172.21.15.5:0/762965037 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c011830 con 0x7f322c0cf3c0 2024-05-08T23:27:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f3229ffb700 1 -- 172.21.15.5:0/762965037 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c011990 con 0x7f322c0cf3c0 2024-05-08T23:27:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f3229ffb700 1 -- 172.21.15.5:0/762965037 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c011d20 con 0x7f322c0cf3c0 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f32314ce700 1 -- 172.21.15.5:0/762965037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c0cf3c0 msgr2=0x7f322c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f32314ce700 1 --2- 172.21.15.5:0/762965037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c0cf3c0 0x7f322c06ef60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f321c0098f0 tx=0x7f321c009c00 comp rx=0 tx=0).stop 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f32314ce700 1 -- 172.21.15.5:0/762965037 shutdown_connections 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f32314ce700 1 --2- 172.21.15.5:0/762965037 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c15a380 0x7f322c0cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f32314ce700 1 --2- 172.21.15.5:0/762965037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c0cf3c0 0x7f322c06ef60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f32314ce700 1 -- 172.21.15.5:0/762965037 >> 172.21.15.5:0/762965037 conn(0x7f322c0cfce0 msgr2=0x7f322c0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f32314ce700 1 -- 172.21.15.5:0/762965037 shutdown_connections 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.190+0000 7f32314ce700 1 -- 172.21.15.5:0/762965037 wait complete. 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.192+0000 7f32314ce700 1 Processor -- start 2024-05-08T23:27:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.192+0000 7f32314ce700 1 -- start start 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f32314ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c06f4a0 0x7f322c0c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f32314ce700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c0cf3c0 0x7f322c0af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f32314ce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c15a380 0x7f322c0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f32314ce700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0cae90 con 0x7f322c15a380 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f32314ce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0ca210 con 0x7f322c06f4a0 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f32314ce700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0ca4f0 con 0x7f322c0cf3c0 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c15a380 0x7f322c0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c15a380 0x7f322c0afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35256/0 (socket says 172.21.15.5:35256) 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 -- 172.21.15.5:0/3877244250 learned_addr learned my addr 172.21.15.5:0/3877244250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322affd700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c06f4a0 0x7f322c0c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322a7fc700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c0cf3c0 0x7f322c0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 -- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c0cf3c0 msgr2=0x7f322c0af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c0cf3c0 0x7f322c0af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 -- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c06f4a0 msgr2=0x7f322c0c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c06f4a0 0x7f322c0c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 -- 172.21.15.5:0/3877244250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321400b740 con 0x7f322c15a380 2024-05-08T23:27:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.193+0000 7f322b7fe700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c15a380 0x7f322c0afd90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f3224002230 tx=0x7f322400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.194+0000 7f32237fe700 1 -- 172.21.15.5:0/3877244250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322400aac0 con 0x7f322c15a380 2024-05-08T23:27:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.194+0000 7f32237fe700 1 -- 172.21.15.5:0/3877244250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322400ac40 con 0x7f322c15a380 2024-05-08T23:27:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.194+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c009550 con 0x7f322c15a380 2024-05-08T23:27:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.194+0000 7f32237fe700 1 -- 172.21.15.5:0/3877244250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32240070d0 con 0x7f322c15a380 2024-05-08T23:27:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.194+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c0caab0 con 0x7f322c15a380 2024-05-08T23:27:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.195+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c068000 con 0x7f322c15a380 2024-05-08T23:27:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.195+0000 7f32237fe700 1 -- 172.21.15.5:0/3877244250 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3224007250 con 0x7f322c15a380 2024-05-08T23:27:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.196+0000 7f32237fe700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320c06ac90 0x7f320c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.196+0000 7f32237fe700 1 -- 172.21.15.5:0/3877244250 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3224088d80 con 0x7f322c15a380 2024-05-08T23:27:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.196+0000 7f322affd700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320c06ac90 0x7f320c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.196+0000 7f322affd700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320c06ac90 0x7f320c06d150 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f321400be50 tx=0x7f321400b420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:37.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.200+0000 7f32237fe700 1 -- 172.21.15.5:0/3877244250 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3224053f60 con 0x7f322c15a380 2024-05-08T23:27:37.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.472+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f322c06fbb0 con 0x7f322c15a380 2024-05-08T23:27:37.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.473+0000 7f32237fe700 1 -- 172.21.15.5:0/3877244250 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7f3224018020 con 0x7f322c15a380 2024-05-08T23:27:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:37.473 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320c06ac90 msgr2=0x7f320c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320c06ac90 0x7f320c06d150 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f321400be50 tx=0x7f321400b420 comp rx=0 tx=0).stop 2024-05-08T23:27:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c15a380 msgr2=0x7f322c0afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c15a380 0x7f322c0afd90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f3224002230 tx=0x7f322400cf50 comp rx=0 tx=0).stop 2024-05-08T23:27:37.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 shutdown_connections 2024-05-08T23:27:37.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f320c06ac90 0x7f320c06d150 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c15a380 0x7f322c0afd90 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f322c06f4a0 0x7f322c0c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 --2- 172.21.15.5:0/3877244250 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f322c0cf3c0 0x7f322c0af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 >> 172.21.15.5:0/3877244250 conn(0x7f322c0cfce0 msgr2=0x7f322c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:37.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 shutdown_connections 2024-05-08T23:27:37.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.475+0000 7f32314ce700 1 -- 172.21.15.5:0/3877244250 wait complete. 2024-05-08T23:27:37.479 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:37.489 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-05-08T23:27:37.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:27:37.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.624+0000 7f9db613a700 1 Processor -- start 2024-05-08T23:27:37.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.624+0000 7f9db613a700 1 -- start start 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9db613a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9db006a960 0x7f9db015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9db613a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db00cd980 con 0x7f9db00cf840 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9db613a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db00ccdb0 con 0x7f9db00d0eb0 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9db613a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db00cd050 con 0x7f9db006a960 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9daeffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9db006a960 0x7f9db015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9daf7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9db00cf840 0x7f9db00af440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58878/0 (socket says 172.21.15.5:58878) 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9daeffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9db006a960 0x7f9db015d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57250/0 (socket says 172.21.15.5:57250) 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9daf7fe700 1 -- 172.21.15.5:0/2753361501 learned_addr learned my addr 172.21.15.5:0/2753361501 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9daeffd700 1 -- 172.21.15.5:0/2753361501 >> v1:172.21.15.49:6789/0 conn(0x7f9db00d0eb0 legacy=0x7f9db0161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.625+0000 7f9daeffd700 1 -- 172.21.15.5:0/2753361501 >> v1:172.21.15.5:6789/0 conn(0x7f9db00cf840 legacy=0x7f9db00af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9daeffd700 1 -- 172.21.15.5:0/2753361501 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db00cd330 con 0x7f9db006a960 2024-05-08T23:27:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9dae7fc700 1 -- 172.21.15.5:0/2753361501 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817519234 0 0) 0x7f9db00cd980 con 0x7f9db00cf840 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9daeffd700 1 --2- 172.21.15.5:0/2753361501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db006a960 0x7f9db015d4f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009840 tx=0x7f9da4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584e05e07390f592 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9dae7fc700 1 -- 172.21.15.5:0/2753361501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4004230 con 0x7f9db006a960 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9dae7fc700 1 -- 172.21.15.5:0/2753361501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da4004390 con 0x7f9db006a960 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9dae7fc700 1 -- 172.21.15.5:0/2753361501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40036a0 con 0x7f9db006a960 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9db613a700 1 -- 172.21.15.5:0/2753361501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db006a960 msgr2=0x7f9db015d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9db613a700 1 --2- 172.21.15.5:0/2753361501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db006a960 0x7f9db015d4f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009840 tx=0x7f9da4009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9db613a700 1 -- 172.21.15.5:0/2753361501 shutdown_connections 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9db613a700 1 --2- 172.21.15.5:0/2753361501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db006a960 0x7f9db015d4f0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.626+0000 7f9db613a700 1 -- 172.21.15.5:0/2753361501 >> 172.21.15.5:0/2753361501 conn(0x7f9db00d0320 msgr2=0x7f9db00d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.627+0000 7f9db613a700 1 -- 172.21.15.5:0/2753361501 shutdown_connections 2024-05-08T23:27:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.627+0000 7f9db613a700 1 -- 172.21.15.5:0/2753361501 wait complete. 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.628+0000 7f9db613a700 1 Processor -- start 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.628+0000 7f9db613a700 1 -- start start 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9db613a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db006a960 0x7f9db015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9db613a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db00cf840 0x7f9db01605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9db613a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9db00d0eb0 0x7f9db0160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9db613a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0161050 con 0x7f9db00cf840 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9db613a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db015e720 con 0x7f9db006a960 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9db613a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db015e9c0 con 0x7f9db00d0eb0 2024-05-08T23:27:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db00cf840 0x7f9db01605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db00cf840 0x7f9db01605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57258/0 (socket says 172.21.15.5:57258) 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daeffd700 1 -- 172.21.15.5:0/3278030881 learned_addr learned my addr 172.21.15.5:0/3278030881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daf7fe700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db006a960 0x7f9db015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daffff700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9db00d0eb0 0x7f9db0160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daeffd700 1 -- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9db00d0eb0 msgr2=0x7f9db0160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daeffd700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9db00d0eb0 0x7f9db0160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daeffd700 1 -- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db006a960 msgr2=0x7f9db015e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daeffd700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db006a960 0x7f9db015e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.629+0000 7f9daeffd700 1 -- 172.21.15.5:0/3278030881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40094a0 con 0x7f9db00cf840 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.630+0000 7f9daeffd700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db00cf840 0x7f9db01605d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9da40090b0 tx=0x7f9da4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.630+0000 7f9d97fff700 1 -- 172.21.15.5:0/3278030881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4011830 con 0x7f9db00cf840 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.630+0000 7f9d97fff700 1 -- 172.21.15.5:0/3278030881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da401e070 con 0x7f9db00cf840 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.630+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db015eca0 con 0x7f9db00cf840 2024-05-08T23:27:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.630+0000 7f9d97fff700 1 -- 172.21.15.5:0/3278030881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4011bb0 con 0x7f9db00cf840 2024-05-08T23:27:37.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.630+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db00b2f20 con 0x7f9db00cf840 2024-05-08T23:27:37.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.631+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db015f1c0 con 0x7f9db00cf840 2024-05-08T23:27:37.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.631+0000 7f9d97fff700 1 -- 172.21.15.5:0/3278030881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9da4011d10 con 0x7f9db00cf840 2024-05-08T23:27:37.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.632+0000 7f9d97fff700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9d9c06ac90 0x7f9d9c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.632+0000 7f9d97fff700 1 -- 172.21.15.5:0/3278030881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9da408a730 con 0x7f9db00cf840 2024-05-08T23:27:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.632+0000 7f9daf7fe700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9d9c06ac90 0x7f9d9c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.632+0000 7f9daf7fe700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9d9c06ac90 0x7f9d9c06d150 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9da0000f60 tx=0x7f9da0007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.636+0000 7f9d97fff700 1 -- 172.21.15.5:0/3278030881 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9da4055900 con 0x7f9db00cf840 2024-05-08T23:27:37.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.950+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9db0068000 con 0x7f9db00cf840 2024-05-08T23:27:37.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.950+0000 7f9d97fff700 1 -- 172.21.15.5:0/3278030881 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1964 (secure 0 0 0) 0x7f9da40207c0 con 0x7f9db00cf840 2024-05-08T23:27:37.950 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:37.950 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":412,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":705854,"bytes_used":256303104,"bytes_avail":772837810176,"bytes_total":773094113280,"write_bytes_sec":2047,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7336},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7755},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7758}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-05-08T23:27:33.388874+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9d9c06ac90 msgr2=0x7f9d9c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9d9c06ac90 0x7f9d9c06d150 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9da0000f60 tx=0x7f9da0007c00 comp rx=0 tx=0).stop 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db00cf840 msgr2=0x7f9db01605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db00cf840 0x7f9db01605d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9da40090b0 tx=0x7f9da4004750 comp rx=0 tx=0).stop 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 shutdown_connections 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9d9c06ac90 0x7f9d9c06d150 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9db00d0eb0 0x7f9db0160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db006a960 0x7f9db015e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 --2- 172.21.15.5:0/3278030881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9db00cf840 0x7f9db01605d0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.952+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 >> 172.21.15.5:0/3278030881 conn(0x7f9db00d0320 msgr2=0x7f9db015c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.953+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 shutdown_connections 2024-05-08T23:27:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:37.953+0000 7f9db613a700 1 -- 172.21.15.5:0/3278030881 wait complete. 2024-05-08T23:27:37.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-05-08T23:27:38.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:27:38.306 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:27:38.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.112+0000 7f9e0490a700 1 Processor -- start 2024-05-08T23:27:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.112+0000 7f9e0490a700 1 -- start start 2024-05-08T23:27:39.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9e0490a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e0006a960 0x7f9e000ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9e0490a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e000cdbb0 con 0x7f9e000cf760 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9e0490a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e000cd310 con 0x7f9e0006b140 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9e0490a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000cd5f0 con 0x7f9e0006a960 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9e000cf760 0x7f9e000cbe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58894/0 (socket says 172.21.15.5:58894) 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfed9d700 1 -- 172.21.15.5:0/742424760 learned_addr learned my addr 172.21.15.5:0/742424760 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40876748 0 0) 0x7f9e000cdbb0 con 0x7f9e000cf760 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfdd9b700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e0006a960 0x7f9e000ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de80035b0 con 0x7f9e000cf760 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1323136807 0 0) 0x7f9de80035b0 con 0x7f9e000cf760 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e0006a960 msgr2=0x7f9e000ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e0006a960 0x7f9e000ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 >> v1:172.21.15.49:6789/0 conn(0x7f9e0006b140 legacy=0x7f9e0015bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000cd8d0 con 0x7f9e000cf760 2024-05-08T23:27:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f9df80019f0 con 0x7f9e000cf760 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.113+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 >> v1:172.21.15.5:6789/0 conn(0x7f9e000cf760 legacy=0x7f9e000cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e0006a960 0x7f9de8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de8003fb0 0x7f9de8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d90 0x7f9de8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000cd5f0 con 0x7f9de8003fb0 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de800bba0 con 0x7f9de8004d90 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de800bd60 con 0x7f9e0006a960 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfdd9b700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de8003fb0 0x7f9de8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfe59c700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e0006a960 0x7f9de8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfed9d700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d90 0x7f9de8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfdd9b700 1 -- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e0006a960 msgr2=0x7f9de8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfdd9b700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9e0006a960 0x7f9de8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfdd9b700 1 -- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d90 msgr2=0x7f9de8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfdd9b700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d90 0x7f9de8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfdd9b700 1 -- 172.21.15.5:0/742424760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de80035b0 con 0x7f9de8003fb0 2024-05-08T23:27:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfdd9b700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de8003fb0 0x7f9de8004850 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009b60 tx=0x7f9de8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be7b4796255185e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400e6c0 con 0x7f9de8003fb0 2024-05-08T23:27:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df400e820 con 0x7f9de8003fb0 2024-05-08T23:27:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.114+0000 7f9dfd59a700 1 -- 172.21.15.5:0/742424760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400eb90 con 0x7f9de8003fb0 2024-05-08T23:27:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.115+0000 7f9e0490a700 1 -- 172.21.15.5:0/742424760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de8003fb0 msgr2=0x7f9de8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.115+0000 7f9e0490a700 1 --2- 172.21.15.5:0/742424760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9de8003fb0 0x7f9de8004850 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009b60 tx=0x7f9de8005880 comp rx=0 tx=0).stop 2024-05-08T23:27:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.115+0000 7f9dfed9d700 1 -- 172.21.15.5:0/742424760 reap_dead start 2024-05-08T23:27:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.115+0000 7f9e0490a700 1 -- 172.21.15.5:0/742424760 shutdown_connections 2024-05-08T23:27:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.115+0000 7f9e0490a700 1 -- 172.21.15.5:0/742424760 >> 172.21.15.5:0/742424760 conn(0x7f9e000d0320 msgr2=0x7f9e000d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.115+0000 7f9e0490a700 1 -- 172.21.15.5:0/742424760 shutdown_connections 2024-05-08T23:27:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.115+0000 7f9e0490a700 1 -- 172.21.15.5:0/742424760 wait complete. 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.117+0000 7f9e0490a700 1 Processor -- start 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.117+0000 7f9e0490a700 1 -- start start 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.117+0000 7f9e0490a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de8003fb0 0x7f9e000c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.117+0000 7f9e0490a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e001f49f0 0x7f9e001f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.117+0000 7f9e0490a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e001f4e70 0x7f9e001f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.117+0000 7f9e0490a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001f45c0 con 0x7f9e001f4e70 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.117+0000 7f9e0490a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0009bf20 con 0x7f9e001f49f0 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.117+0000 7f9e0490a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0009c200 con 0x7f9de8003fb0 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e001f4e70 0x7f9e001f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e001f4e70 0x7f9e001f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57278/0 (socket says 172.21.15.5:57278) 2024-05-08T23:27:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 -- 172.21.15.5:0/2544601313 learned_addr learned my addr 172.21.15.5:0/2544601313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfe59c700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de8003fb0 0x7f9e000c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfdd9b700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e001f49f0 0x7f9e001f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 -- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de8003fb0 msgr2=0x7f9e000c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de8003fb0 0x7f9e000c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 -- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e001f49f0 msgr2=0x7f9e001f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e001f49f0 0x7f9e001f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 -- 172.21.15.5:0/2544601313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000c3cc0 con 0x7f9e001f4e70 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9dfed9d700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e001f4e70 0x7f9e001f4080 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9e000cba60 tx=0x7f9df800a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9deeffd700 1 -- 172.21.15.5:0/2544601313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800ac50 con 0x7f9e001f4e70 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9deeffd700 1 -- 172.21.15.5:0/2544601313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df800adb0 con 0x7f9e001f4e70 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0009c4e0 con 0x7f9e001f4e70 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9deeffd700 1 -- 172.21.15.5:0/2544601313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8013630 con 0x7f9e001f4e70 2024-05-08T23:27:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.118+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0009caa0 con 0x7f9e001f4e70 2024-05-08T23:27:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.119+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0009cc60 con 0x7f9e001f4e70 2024-05-08T23:27:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.120+0000 7f9deeffd700 1 -- 172.21.15.5:0/2544601313 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9df8013790 con 0x7f9e001f4e70 2024-05-08T23:27:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.120+0000 7f9deeffd700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9de006ac90 0x7f9de006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.121+0000 7f9deeffd700 1 -- 172.21.15.5:0/2544601313 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9df8089dd0 con 0x7f9e001f4e70 2024-05-08T23:27:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.121+0000 7f9dfe59c700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9de006ac90 0x7f9de006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.121+0000 7f9dfe59c700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9de006ac90 0x7f9de006d150 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9df0007fd0 tx=0x7f9df0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.125+0000 7f9deeffd700 1 -- 172.21.15.5:0/2544601313 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9df8054fa0 con 0x7f9e001f4e70 2024-05-08T23:27:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.393+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e00068000 con 0x7f9e001f4e70 2024-05-08T23:27:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.394+0000 7f9deeffd700 1 -- 172.21.15.5:0/2544601313 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7f9df8017070 con 0x7f9e001f4e70 2024-05-08T23:27:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:39.394 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9de006ac90 msgr2=0x7f9de006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9de006ac90 0x7f9de006d150 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9df0007fd0 tx=0x7f9df0009040 comp rx=0 tx=0).stop 2024-05-08T23:27:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e001f4e70 msgr2=0x7f9e001f4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e001f4e70 0x7f9e001f4080 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9e000cba60 tx=0x7f9df800a5f0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 shutdown_connections 2024-05-08T23:27:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9de006ac90 0x7f9de006d150 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e001f4e70 0x7f9e001f4080 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9de8003fb0 0x7f9e000c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 --2- 172.21.15.5:0/2544601313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e001f49f0 0x7f9e001f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 >> 172.21.15.5:0/2544601313 conn(0x7f9e000d0320 msgr2=0x7f9e000ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 shutdown_connections 2024-05-08T23:27:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.396+0000 7f9e0490a700 1 -- 172.21.15.5:0/2544601313 wait complete. 2024-05-08T23:27:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:39.410 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-05-08T23:27:39.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:27:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.544+0000 7fe74e099700 1 Processor -- start 2024-05-08T23:27:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.544+0000 7fe74e099700 1 -- start start 2024-05-08T23:27:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.545+0000 7fe74e099700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7480b2480 0x7fe74809a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.545+0000 7fe74e099700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe7480cbbc0 0x7fe7480cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.545+0000 7fe74e099700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7480d0840 con 0x7fe748161030 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.545+0000 7fe74e099700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74806ccc0 con 0x7fe7480cbbc0 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.545+0000 7fe74e099700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480d0660 con 0x7fe7480b2480 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.545+0000 7fe7477fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe748161030 0x7fe748099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58906/0 (socket says 172.21.15.5:58906) 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.545+0000 7fe7477fe700 1 -- 172.21.15.5:0/582560619 learned_addr learned my addr 172.21.15.5:0/582560619 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe747fff700 1 --2- 172.21.15.5:0/582560619 >> v2:172.21.15.49:3300/0 conn(0x7fe7480cbbc0 0x7fe7480cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe746ffd700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7480b2480 0x7fe74809a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377367463 0 0) 0x7fe7480d0840 con 0x7fe748161030 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7340035b0 con 0x7fe748161030 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2018041353 0 0) 0x7fe7340035b0 con 0x7fe748161030 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7480b2480 msgr2=0x7fe74809a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7480b2480 0x7fe74809a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 >> v2:172.21.15.49:3300/0 conn(0x7fe7480cbbc0 msgr2=0x7fe7480cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 --2- 172.21.15.5:0/582560619 >> v2:172.21.15.49:3300/0 conn(0x7fe7480cbbc0 0x7fe7480cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74806c600 con 0x7fe748161030 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fe738001670 con 0x7fe748161030 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 >> v1:172.21.15.5:6789/0 conn(0x7fe748161030 legacy=0x7fe748099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7480b2480 0x7fe734003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe734003f80 0x7fe734004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe734004d90 0x7fe734005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74806ccc0 con 0x7fe734004d90 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480d0660 con 0x7fe734003f80 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73400ba10 con 0x7fe7480b2480 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe747fff700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe734004d90 0x7fe734005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe746ffd700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe734003f80 0x7fe734004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.546+0000 7fe7477fe700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7480b2480 0x7fe734003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe747fff700 1 -- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7480b2480 msgr2=0x7fe734003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe747fff700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7480b2480 0x7fe734003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe747fff700 1 -- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe734003f80 msgr2=0x7fe734004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe747fff700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe734003f80 0x7fe734004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe747fff700 1 -- 172.21.15.5:0/582560619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7340035b0 con 0x7fe734004d90 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe747fff700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe734004d90 0x7fe734005210 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fe740009fc0 tx=0x7fe734005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52859827b77bcecf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000ebd0 con 0x7fe734004d90 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74000f040 con 0x7fe734004d90 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe7467fc700 1 -- 172.21.15.5:0/582560619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740014480 con 0x7fe734004d90 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe74e099700 1 -- 172.21.15.5:0/582560619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe734004d90 msgr2=0x7fe734005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe74e099700 1 --2- 172.21.15.5:0/582560619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe734004d90 0x7fe734005210 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fe740009fc0 tx=0x7fe734005880 comp rx=0 tx=0).stop 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.547+0000 7fe747fff700 1 -- 172.21.15.5:0/582560619 reap_dead start 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.548+0000 7fe74e099700 1 -- 172.21.15.5:0/582560619 shutdown_connections 2024-05-08T23:27:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.548+0000 7fe74e099700 1 -- 172.21.15.5:0/582560619 >> 172.21.15.5:0/582560619 conn(0x7fe7480b2f20 msgr2=0x7fe7480b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.548+0000 7fe74e099700 1 -- 172.21.15.5:0/582560619 shutdown_connections 2024-05-08T23:27:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.548+0000 7fe74e099700 1 -- 172.21.15.5:0/582560619 wait complete. 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.549+0000 7fe74e099700 1 Processor -- start 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.550+0000 7fe74e099700 1 -- start start 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.550+0000 7fe74e099700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7481483f0 0x7fe7480c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe74e099700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7481487d0 0x7fe7480c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe74e099700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7480b5570 0x7fe7480b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe74e099700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b6f20 con 0x7fe7480b5570 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe74e099700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b62a0 con 0x7fe7481483f0 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe74e099700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b6580 con 0x7fe7481487d0 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7480b5570 0x7fe7480b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7480b5570 0x7fe7480b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57286/0 (socket says 172.21.15.5:57286) 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 -- 172.21.15.5:0/324295671 learned_addr learned my addr 172.21.15.5:0/324295671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe7477fe700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7481483f0 0x7fe7480c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe746ffd700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7481487d0 0x7fe7480c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 -- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7481487d0 msgr2=0x7fe7480c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7481487d0 0x7fe7480c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 -- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7481483f0 msgr2=0x7fe7480c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7481483f0 0x7fe7480c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 -- 172.21.15.5:0/324295671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7480b6860 con 0x7fe7480b5570 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe747fff700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7480b5570 0x7fe7480b5a00 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe740016000 tx=0x7fe740009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe73ffff700 1 -- 172.21.15.5:0/324295671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740007000 con 0x7fe7480b5570 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe73ffff700 1 -- 172.21.15.5:0/324295671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe740004450 con 0x7fe7480b5570 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7480b6b40 con 0x7fe7480b5570 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.551+0000 7fe73ffff700 1 -- 172.21.15.5:0/324295671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740004800 con 0x7fe7480b5570 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.552+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74814add0 con 0x7fe7480b5570 2024-05-08T23:27:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.552+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe748148d40 con 0x7fe7480b5570 2024-05-08T23:27:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.553+0000 7fe73ffff700 1 -- 172.21.15.5:0/324295671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe740014480 con 0x7fe7480b5570 2024-05-08T23:27:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.553+0000 7fe73ffff700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe72806ac90 0x7fe72806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.553+0000 7fe73ffff700 1 -- 172.21.15.5:0/324295671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe74008a1f0 con 0x7fe7480b5570 2024-05-08T23:27:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.554+0000 7fe7477fe700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe72806ac90 0x7fe72806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.554+0000 7fe7477fe700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe72806ac90 0x7fe72806d150 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe7380048d0 tx=0x7fe738007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.558+0000 7fe73ffff700 1 -- 172.21.15.5:0/324295671 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe748148d40 con 0x7fe7480b5570 2024-05-08T23:27:39.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.879+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe748148d40 con 0x7fe7480b5570 2024-05-08T23:27:39.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.880+0000 7fe73ffff700 1 -- 172.21.15.5:0/324295671 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1870 (secure 0 0 0) 0x7fe748148d40 con 0x7fe7480b5570 2024-05-08T23:27:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:39.880 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":414,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":705854,"bytes_used":256303104,"bytes_avail":772837810176,"bytes_total":773094113280,"write_bytes_sec":2047,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":7336},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7755},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7758}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-05-08T23:27:37.389930+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:27:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe72806ac90 msgr2=0x7fe72806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe72806ac90 0x7fe72806d150 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe7380048d0 tx=0x7fe738007980 comp rx=0 tx=0).stop 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7480b5570 msgr2=0x7fe7480b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7480b5570 0x7fe7480b5a00 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe740016000 tx=0x7fe740009750 comp rx=0 tx=0).stop 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 shutdown_connections 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe72806ac90 0x7fe72806d150 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7480b5570 0x7fe7480b5a00 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7481483f0 0x7fe7480c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 --2- 172.21.15.5:0/324295671 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7481487d0 0x7fe7480c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 >> 172.21.15.5:0/324295671 conn(0x7fe7480b2f20 msgr2=0x7fe74815de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 shutdown_connections 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:39.882+0000 7fe74e099700 1 -- 172.21.15.5:0/324295671 wait complete. 2024-05-08T23:27:39.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/d1/d2/d3]'} 2024-05-08T23:27:39.896 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:27:39.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:27:39.946 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:27:39.991 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:27:39.991 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:27:39.991 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:40.012 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:27:40.012 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:40.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:27:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.260+0000 7f9ce4711700 1 Processor -- start 2024-05-08T23:27:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.260+0000 7f9ce4711700 1 -- start start 2024-05-08T23:27:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.261+0000 7f9ce4711700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9cdc0ccbd0 0x7f9cdc168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.261+0000 7f9ce4711700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cdc09a870 con 0x7f9cdc0b2480 2024-05-08T23:27:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.261+0000 7f9ce4711700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cdc0d0840 con 0x7f9cdc161030 2024-05-08T23:27:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.261+0000 7f9ce4711700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cdc06ccc0 con 0x7f9cdc0ccbd0 2024-05-08T23:27:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.261+0000 7f9ce2cae700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9cdc0ccbd0 0x7f9cdc168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.261+0000 7f9ce1cac700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9cdc0b2480 0x7f9cdc09a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58914/0 (socket says 172.21.15.5:58914) 2024-05-08T23:27:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.261+0000 7f9ce1cac700 1 -- 172.21.15.5:0/774939756 learned_addr learned my addr 172.21.15.5:0/774939756 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce2cae700 1 -- 172.21.15.5:0/774939756 >> v1:172.21.15.49:6789/0 conn(0x7f9cdc161030 legacy=0x7f9cdc099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce2cae700 1 -- 172.21.15.5:0/774939756 >> v1:172.21.15.5:6789/0 conn(0x7f9cdc0b2480 legacy=0x7f9cdc09a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce14ab700 1 -- 172.21.15.5:0/774939756 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302341439 0 0) 0x7f9cdc09a870 con 0x7f9cdc0b2480 2024-05-08T23:27:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce2cae700 1 -- 172.21.15.5:0/774939756 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc06c600 con 0x7f9cdc0ccbd0 2024-05-08T23:27:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce2cae700 1 --2- 172.21.15.5:0/774939756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc0ccbd0 0x7f9cdc168580 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009d90 tx=0x7f9cd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad21fd42706fc08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce14ab700 1 -- 172.21.15.5:0/774939756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd400e070 con 0x7f9cdc0ccbd0 2024-05-08T23:27:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce14ab700 1 -- 172.21.15.5:0/774939756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd400bbd0 con 0x7f9cdc0ccbd0 2024-05-08T23:27:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce14ab700 1 -- 172.21.15.5:0/774939756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4012430 con 0x7f9cdc0ccbd0 2024-05-08T23:27:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce4711700 1 -- 172.21.15.5:0/774939756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc0ccbd0 msgr2=0x7f9cdc168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.262+0000 7f9ce4711700 1 --2- 172.21.15.5:0/774939756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc0ccbd0 0x7f9cdc168580 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009d90 tx=0x7f9cd400b330 comp rx=0 tx=0).stop 2024-05-08T23:27:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.263+0000 7f9ce4711700 1 -- 172.21.15.5:0/774939756 shutdown_connections 2024-05-08T23:27:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.263+0000 7f9ce4711700 1 --2- 172.21.15.5:0/774939756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc0ccbd0 0x7f9cdc168580 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.263+0000 7f9ce4711700 1 -- 172.21.15.5:0/774939756 >> 172.21.15.5:0/774939756 conn(0x7f9cdc0b2f20 msgr2=0x7f9cdc0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.263+0000 7f9ce4711700 1 -- 172.21.15.5:0/774939756 shutdown_connections 2024-05-08T23:27:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.263+0000 7f9ce4711700 1 -- 172.21.15.5:0/774939756 wait complete. 2024-05-08T23:27:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.264+0000 7f9ce4711700 1 Processor -- start 2024-05-08T23:27:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.264+0000 7f9ce4711700 1 -- start start 2024-05-08T23:27:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.265+0000 7f9ce4711700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cdc0b2480 0x7f9cdc14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.265+0000 7f9ce4711700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9cdc0ccbd0 0x7f9cdc156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.265+0000 7f9ce4711700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc161030 0x7f9cdc0c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.265+0000 7f9ce4711700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc0d0660 con 0x7f9cdc161030 2024-05-08T23:27:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.265+0000 7f9ce4711700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc1500e0 con 0x7f9cdc0b2480 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.265+0000 7f9ce4711700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc1553a0 con 0x7f9cdc0ccbd0 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc161030 0x7f9cdc0c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc161030 0x7f9cdc0c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57308/0 (socket says 172.21.15.5:57308) 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 -- 172.21.15.5:0/3131175508 learned_addr learned my addr 172.21.15.5:0/3131175508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce24ad700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cdc0b2480 0x7f9cdc14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce1cac700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9cdc0ccbd0 0x7f9cdc156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 -- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9cdc0ccbd0 msgr2=0x7f9cdc156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9cdc0ccbd0 0x7f9cdc156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 -- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cdc0b2480 msgr2=0x7f9cdc14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cdc0b2480 0x7f9cdc14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 -- 172.21.15.5:0/3131175508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd4009a70 con 0x7f9cdc161030 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce1cac700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9cdc0ccbd0 0x7f9cdc156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce2cae700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc161030 0x7f9cdc0c7110 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4014040 tx=0x7f9cd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9cd2ffd700 1 -- 172.21.15.5:0/3131175508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4012430 con 0x7f9cdc161030 2024-05-08T23:27:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9cd2ffd700 1 -- 172.21.15.5:0/3131175508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd4020070 con 0x7f9cdc161030 2024-05-08T23:27:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc155680 con 0x7f9cdc161030 2024-05-08T23:27:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9cd2ffd700 1 -- 172.21.15.5:0/3131175508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4004280 con 0x7f9cdc161030 2024-05-08T23:27:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.266+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc155c40 con 0x7f9cdc161030 2024-05-08T23:27:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.267+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cdc068000 con 0x7f9cdc161030 2024-05-08T23:27:40.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.268+0000 7f9cd2ffd700 1 -- 172.21.15.5:0/3131175508 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9cd40043e0 con 0x7f9cdc161030 2024-05-08T23:27:40.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.268+0000 7f9cd2ffd700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cc006ac90 0x7f9cc006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.268+0000 7f9cd2ffd700 1 -- 172.21.15.5:0/3131175508 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cd4022070 con 0x7f9cdc161030 2024-05-08T23:27:40.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.269+0000 7f9ce24ad700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cc006ac90 0x7f9cc006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.269+0000 7f9ce24ad700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cc006ac90 0x7f9cc006d150 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000f60 tx=0x7f9cd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:40.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.273+0000 7f9cd2ffd700 1 -- 172.21.15.5:0/3131175508 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9cd40567b0 con 0x7f9cdc161030 2024-05-08T23:27:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.492+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cdc0c7ec0 con 0x7f9cdc161030 2024-05-08T23:27:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.493+0000 7f9cd2ffd700 1 -- 172.21.15.5:0/3131175508 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15996 (secure 0 0 0) 0x7f9cd401ae60 con 0x7f9cdc161030 2024-05-08T23:27:40.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:40.493 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:57.193438+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:54.037450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:55.162938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cc006ac90 msgr2=0x7f9cc006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cc006ac90 0x7f9cc006d150 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000f60 tx=0x7f9cd8009040 comp rx=0 tx=0).stop 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc161030 msgr2=0x7f9cdc0c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc161030 0x7f9cdc0c7110 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4014040 tx=0x7f9cd4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 shutdown_connections 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9cc006ac90 0x7f9cc006d150 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cdc161030 0x7f9cdc0c7110 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cdc0b2480 0x7f9cdc14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 --2- 172.21.15.5:0/3131175508 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9cdc0ccbd0 0x7f9cdc156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 >> 172.21.15.5:0/3131175508 conn(0x7f9cdc0b2f20 msgr2=0x7f9cdc158e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 shutdown_connections 2024-05-08T23:27:40.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.495+0000 7f9ce4711700 1 -- 172.21.15.5:0/3131175508 wait complete. 2024-05-08T23:27:40.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:26:54.037450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:26:55.162938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:27:40.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:27:40.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.648+0000 7f2ab6e50700 1 Processor -- start 2024-05-08T23:27:40.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.648+0000 7f2ab6e50700 1 -- start start 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2ab6e50700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2ab006f8c0 0x7f2ab006a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2ab6e50700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00b3870 con 0x7f2ab015d880 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2ab6e50700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00b4ee0 con 0x7f2ab00af990 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2ab6e50700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00b32f0 con 0x7f2ab006f8c0 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2ab4bec700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2ab015d880 0x7f2ab00ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58918/0 (socket says 172.21.15.5:58918) 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2ab4bec700 1 -- 172.21.15.5:0/2181866153 learned_addr learned my addr 172.21.15.5:0/2181866153 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2ab53ed700 1 --2- 172.21.15.5:0/2181866153 >> v2:172.21.15.49:3300/0 conn(0x7f2ab006f8c0 0x7f2ab006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413801185 0 0) 0x7f2ab00b3870 con 0x7f2ab015d880 2024-05-08T23:27:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a980035b0 con 0x7f2ab015d880 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.650+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 636870009 0 0) 0x7f2a980035b0 con 0x7f2ab015d880 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 >> v2:172.21.15.49:3300/0 conn(0x7f2ab006f8c0 msgr2=0x7f2ab006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 --2- 172.21.15.5:0/2181866153 >> v2:172.21.15.49:3300/0 conn(0x7f2ab006f8c0 0x7f2ab006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 >> v1:172.21.15.49:6789/0 conn(0x7f2ab00af990 legacy=0x7f2ab006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00b5180 con 0x7f2ab015d880 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f2aa0001670 con 0x7f2ab015d880 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 >> v1:172.21.15.5:6789/0 conn(0x7f2ab015d880 legacy=0x7f2ab00ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98003a00 0x7f2a98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a98004850 0x7f2a98004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a98005210 0x7f2a9800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00b32f0 con 0x7f2a98005210 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9800c0b0 con 0x7f2a98004850 2024-05-08T23:27:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9800c270 con 0x7f2a98003a00 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab53ed700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a98005210 0x7f2a9800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaffff700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a98004850 0x7f2a98004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab4bec700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98003a00 0x7f2a98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab53ed700 1 -- 172.21.15.5:0/2181866153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98003a00 msgr2=0x7f2a98003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab53ed700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98003a00 0x7f2a98003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab53ed700 1 -- 172.21.15.5:0/2181866153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a98004850 msgr2=0x7f2a98004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab53ed700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a98004850 0x7f2a98004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab53ed700 1 -- 172.21.15.5:0/2181866153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a980035b0 con 0x7f2a98005210 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab53ed700 1 -- 172.21.15.5:0/2181866153 reap_dead start 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2ab53ed700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a98005210 0x7f2a9800ba10 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009fc0 tx=0x7f2a98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d40091a792bf349 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.651+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8003a20 con 0x7f2a98005210 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa800f040 con 0x7f2a98005210 2024-05-08T23:27:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2aaf7fe700 1 -- 172.21.15.5:0/2181866153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8003de0 con 0x7f2a98005210 2024-05-08T23:27:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2181866153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a98005210 msgr2=0x7f2a9800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2ab6e50700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a98005210 0x7f2a9800ba10 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009fc0 tx=0x7f2a98005880 comp rx=0 tx=0).stop 2024-05-08T23:27:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2181866153 shutdown_connections 2024-05-08T23:27:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2ab6e50700 1 --2- 172.21.15.5:0/2181866153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a98005210 0x7f2a9800ba10 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2181866153 >> 172.21.15.5:0/2181866153 conn(0x7f2ab00b5810 msgr2=0x7f2ab00b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2181866153 shutdown_connections 2024-05-08T23:27:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.652+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2181866153 wait complete. 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.653+0000 7f2ab6e50700 1 Processor -- start 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.654+0000 7f2ab6e50700 1 -- start start 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.654+0000 7f2ab6e50700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98005210 0x7f2ab0143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.654+0000 7f2ab6e50700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ab0151ef0 0x7f2ab01569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.654+0000 7f2ab6e50700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0156f30 0x7f2ab015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.654+0000 7f2ab6e50700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0155200 con 0x7f2ab0156f30 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.654+0000 7f2ab6e50700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab01573c0 con 0x7f2ab0151ef0 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.654+0000 7f2ab6e50700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0144430 con 0x7f2a98005210 2024-05-08T23:27:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0156f30 0x7f2ab015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0156f30 0x7f2ab015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57324/0 (socket says 172.21.15.5:57324) 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab4bec700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98005210 0x7f2ab0143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2aaffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ab0151ef0 0x7f2ab01569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 -- 172.21.15.5:0/2603489185 learned_addr learned my addr 172.21.15.5:0/2603489185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 -- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98005210 msgr2=0x7f2ab0143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98005210 0x7f2ab0143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 -- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ab0151ef0 msgr2=0x7f2ab01569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ab0151ef0 0x7f2ab01569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 -- 172.21.15.5:0/2603489185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab0154610 con 0x7f2ab0156f30 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab53ed700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0156f30 0x7f2ab015a990 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00d13d0 tx=0x7f2aa8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2aad7fa700 1 -- 172.21.15.5:0/2603489185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8003e10 con 0x7f2ab0156f30 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2aad7fa700 1 -- 172.21.15.5:0/2603489185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa8004230 con 0x7f2ab0156f30 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab0154860 con 0x7f2ab0156f30 2024-05-08T23:27:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2aad7fa700 1 -- 172.21.15.5:0/2603489185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80045e0 con 0x7f2ab0156f30 2024-05-08T23:27:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.655+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab0154e20 con 0x7f2ab0156f30 2024-05-08T23:27:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.656+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab0155340 con 0x7f2ab0156f30 2024-05-08T23:27:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.657+0000 7f2aad7fa700 1 -- 172.21.15.5:0/2603489185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2aa8004740 con 0x7f2ab0156f30 2024-05-08T23:27:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.657+0000 7f2aad7fa700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a9c06ac90 0x7f2a9c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.657+0000 7f2aad7fa700 1 -- 172.21.15.5:0/2603489185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aa808a500 con 0x7f2ab0156f30 2024-05-08T23:27:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.657+0000 7f2ab4bec700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a9c06ac90 0x7f2a9c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.658+0000 7f2ab4bec700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a9c06ac90 0x7f2a9c06d150 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa0000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.662+0000 7f2aad7fa700 1 -- 172.21.15.5:0/2603489185 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2ab0155340 con 0x7f2ab0156f30 2024-05-08T23:27:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.878+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ab0155340 con 0x7f2ab0156f30 2024-05-08T23:27:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.878+0000 7f2aad7fa700 1 -- 172.21.15.5:0/2603489185 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ab0155340 con 0x7f2ab0156f30 2024-05-08T23:27:40.878 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:27:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.880+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a9c06ac90 msgr2=0x7f2a9c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.880+0000 7f2ab6e50700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a9c06ac90 0x7f2a9c06d150 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa0000ed0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.880+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0156f30 msgr2=0x7f2ab015a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.880+0000 7f2ab6e50700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0156f30 0x7f2ab015a990 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00d13d0 tx=0x7f2aa8009750 comp rx=0 tx=0).stop 2024-05-08T23:27:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.881+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 shutdown_connections 2024-05-08T23:27:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.881+0000 7f2ab6e50700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2a9c06ac90 0x7f2a9c06d150 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.881+0000 7f2ab6e50700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0156f30 0x7f2ab015a990 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.881+0000 7f2ab6e50700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2a98005210 0x7f2ab0143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.881+0000 7f2ab6e50700 1 --2- 172.21.15.5:0/2603489185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ab0151ef0 0x7f2ab01569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.881+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 >> 172.21.15.5:0/2603489185 conn(0x7f2ab00b5810 msgr2=0x7f2ab00b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.881+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 shutdown_connections 2024-05-08T23:27:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:40.881+0000 7f2ab6e50700 1 -- 172.21.15.5:0/2603489185 wait complete. 2024-05-08T23:27:40.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:27:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.035+0000 7f545a0fd700 1 Processor -- start 2024-05-08T23:27:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.035+0000 7f545a0fd700 1 -- start start 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.035+0000 7f545a0fd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545409a790 0x7f5454099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f545a0fd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5454099f90 0x7f54540cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f545a0fd700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545406d010 con 0x7f545415da60 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f545a0fd700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545409a420 con 0x7f5454099f90 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f545a0fd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545415de40 con 0x7f545409a790 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f5453fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5454099f90 0x7f54540cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f5452ffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545409a790 0x7f5454099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f54537fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f545415da60 0x7f54540d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60210/0 (socket says 172.21.15.5:60210) 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f54537fe700 1 -- 172.21.15.5:0/1719311538 learned_addr learned my addr 172.21.15.5:0/1719311538 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f5452ffd700 1 -- 172.21.15.5:0/1719311538 >> v2:172.21.15.49:3300/0 conn(0x7f5454099f90 msgr2=0x7f54540cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f5452ffd700 1 --2- 172.21.15.5:0/1719311538 >> v2:172.21.15.49:3300/0 conn(0x7f5454099f90 0x7f54540cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f5452ffd700 1 -- 172.21.15.5:0/1719311538 >> v1:172.21.15.49:6789/0 conn(0x7f545415da60 legacy=0x7f54540d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f54527fc700 1 -- 172.21.15.5:0/1719311538 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695727015 0 0) 0x7f545406d010 con 0x7f545415da60 2024-05-08T23:27:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.036+0000 7f5452ffd700 1 -- 172.21.15.5:0/1719311538 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540d0b20 con 0x7f545409a790 2024-05-08T23:27:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f5452ffd700 1 --2- 172.21.15.5:0/1719311538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545409a790 0x7f5454099a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5448009840 tx=0x7f5448009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50604502ff25d4e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f54527fc700 1 -- 172.21.15.5:0/1719311538 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448011830 con 0x7f545409a790 2024-05-08T23:27:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f54527fc700 1 -- 172.21.15.5:0/1719311538 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448011990 con 0x7f545409a790 2024-05-08T23:27:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f54527fc700 1 -- 172.21.15.5:0/1719311538 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448011d20 con 0x7f545409a790 2024-05-08T23:27:41.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f545a0fd700 1 -- 172.21.15.5:0/1719311538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545409a790 msgr2=0x7f5454099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f545a0fd700 1 --2- 172.21.15.5:0/1719311538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545409a790 0x7f5454099a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5448009840 tx=0x7f5448009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:41.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f545a0fd700 1 -- 172.21.15.5:0/1719311538 shutdown_connections 2024-05-08T23:27:41.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f545a0fd700 1 --2- 172.21.15.5:0/1719311538 >> v2:172.21.15.49:3300/0 conn(0x7f5454099f90 0x7f54540cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f545a0fd700 1 --2- 172.21.15.5:0/1719311538 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545409a790 0x7f5454099a50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f545a0fd700 1 -- 172.21.15.5:0/1719311538 >> 172.21.15.5:0/1719311538 conn(0x7f54540b2f60 msgr2=0x7f54540b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.037+0000 7f545a0fd700 1 -- 172.21.15.5:0/1719311538 shutdown_connections 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.038+0000 7f545a0fd700 1 -- 172.21.15.5:0/1719311538 wait complete. 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.039+0000 7f545a0fd700 1 Processor -- start 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.039+0000 7f545a0fd700 1 -- start start 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f545a0fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5454099f90 0x7f54540c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f545a0fd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f545409a790 0x7f5454152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f545a0fd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545415da60 0x7f54540b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f545a0fd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540c8280 con 0x7f5454099f90 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f545a0fd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545415f400 con 0x7f545409a790 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f545a0fd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54541f36d0 con 0x7f545415da60 2024-05-08T23:27:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f54537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5454099f90 0x7f54540c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f54537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5454099f90 0x7f54540c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57326/0 (socket says 172.21.15.5:57326) 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f54537fe700 1 -- 172.21.15.5:0/719172643 learned_addr learned my addr 172.21.15.5:0/719172643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f5453fff700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545415da60 0x7f54540b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f5452ffd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f545409a790 0x7f5454152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f54537fe700 1 -- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545415da60 msgr2=0x7f54540b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f54537fe700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545415da60 0x7f54540b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f54537fe700 1 -- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f545409a790 msgr2=0x7f5454152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f54537fe700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f545409a790 0x7f5454152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.040+0000 7f54537fe700 1 -- 172.21.15.5:0/719172643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480094a0 con 0x7f5454099f90 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.041+0000 7f54537fe700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5454099f90 0x7f54540c79a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f544400c550 tx=0x7f544400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.041+0000 7f543bfff700 1 -- 172.21.15.5:0/719172643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444004c80 con 0x7f5454099f90 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.041+0000 7f543bfff700 1 -- 172.21.15.5:0/719172643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5444004e00 con 0x7f5454099f90 2024-05-08T23:27:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.041+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54540c7fa0 con 0x7f5454099f90 2024-05-08T23:27:41.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.041+0000 7f543bfff700 1 -- 172.21.15.5:0/719172643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444007310 con 0x7f5454099f90 2024-05-08T23:27:41.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.041+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545415f980 con 0x7f5454099f90 2024-05-08T23:27:41.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.042+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5454074bc0 con 0x7f5454099f90 2024-05-08T23:27:41.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.042+0000 7f543bfff700 1 -- 172.21.15.5:0/719172643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5444007470 con 0x7f5454099f90 2024-05-08T23:27:41.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.043+0000 7f543bfff700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f544006acc0 0x7f544006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.043+0000 7f543bfff700 1 -- 172.21.15.5:0/719172643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5444088e30 con 0x7f5454099f90 2024-05-08T23:27:41.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.043+0000 7f5452ffd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f544006acc0 0x7f544006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.044+0000 7f5452ffd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f544006acc0 0x7f544006d180 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f54480090b0 tx=0x7f5448008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.048+0000 7f543bfff700 1 -- 172.21.15.5:0/719172643 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f54440540a0 con 0x7f5454099f90 2024-05-08T23:27:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.266+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54540d1250 con 0x7f5454099f90 2024-05-08T23:27:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.266+0000 7f543bfff700 1 -- 172.21.15.5:0/719172643 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54440576c0 con 0x7f5454099f90 2024-05-08T23:27:41.266 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:27:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f544006acc0 msgr2=0x7f544006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f544006acc0 0x7f544006d180 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f54480090b0 tx=0x7f5448008030 comp rx=0 tx=0).stop 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5454099f90 msgr2=0x7f54540c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5454099f90 0x7f54540c79a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f544400c550 tx=0x7f544400c860 comp rx=0 tx=0).stop 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 shutdown_connections 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f544006acc0 0x7f544006d180 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f545415da60 0x7f54540b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5454099f90 0x7f54540c79a0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 --2- 172.21.15.5:0/719172643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f545409a790 0x7f5454152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 >> 172.21.15.5:0/719172643 conn(0x7f54540b2f60 msgr2=0x7f54540cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 shutdown_connections 2024-05-08T23:27:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.269+0000 7f545a0fd700 1 -- 172.21.15.5:0/719172643 wait complete. 2024-05-08T23:27:41.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:27:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.422+0000 7fa0b04ad700 1 Processor -- start 2024-05-08T23:27:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.423+0000 7fa0b04ad700 1 -- start start 2024-05-08T23:27:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0b04ad700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa0a809a790 0x7fa0a8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0b04ad700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a806d010 con 0x7fa0a815da60 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0b04ad700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a815de40 con 0x7fa0a8099f90 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0b04ad700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a809a420 con 0x7fa0a809a790 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ada48700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa0a809a790 0x7fa0a8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ae249700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa0a815da60 0x7fa0a80d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58924/0 (socket says 172.21.15.5:58924) 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ae249700 1 -- 172.21.15.5:0/3198090294 learned_addr learned my addr 172.21.15.5:0/3198090294 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ada48700 1 -- 172.21.15.5:0/3198090294 >> v1:172.21.15.49:6789/0 conn(0x7fa0a8099f90 legacy=0x7fa0a80cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ada48700 1 -- 172.21.15.5:0/3198090294 >> v1:172.21.15.5:6789/0 conn(0x7fa0a815da60 legacy=0x7fa0a80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ada48700 1 -- 172.21.15.5:0/3198090294 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a80d0b20 con 0x7fa0a809a790 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ad247700 1 -- 172.21.15.5:0/3198090294 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737118311 0 0) 0x7fa0a806d010 con 0x7fa0a815da60 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ada48700 1 --2- 172.21.15.5:0/3198090294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a809a790 0x7fa0a8099a50 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa098009840 tx=0x7fa098009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6ab21eda002972 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.424+0000 7fa0ad247700 1 -- 172.21.15.5:0/3198090294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098004230 con 0x7fa0a809a790 2024-05-08T23:27:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0ad247700 1 -- 172.21.15.5:0/3198090294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098004390 con 0x7fa0a809a790 2024-05-08T23:27:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0ad247700 1 -- 172.21.15.5:0/3198090294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980036a0 con 0x7fa0a809a790 2024-05-08T23:27:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0b04ad700 1 -- 172.21.15.5:0/3198090294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a809a790 msgr2=0x7fa0a8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0b04ad700 1 --2- 172.21.15.5:0/3198090294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a809a790 0x7fa0a8099a50 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa098009840 tx=0x7fa098009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0b04ad700 1 -- 172.21.15.5:0/3198090294 shutdown_connections 2024-05-08T23:27:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0b04ad700 1 --2- 172.21.15.5:0/3198090294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a809a790 0x7fa0a8099a50 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0b04ad700 1 -- 172.21.15.5:0/3198090294 >> 172.21.15.5:0/3198090294 conn(0x7fa0a80b2f60 msgr2=0x7fa0a80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0b04ad700 1 -- 172.21.15.5:0/3198090294 shutdown_connections 2024-05-08T23:27:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.425+0000 7fa0b04ad700 1 -- 172.21.15.5:0/3198090294 wait complete. 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0b04ad700 1 Processor -- start 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0b04ad700 1 -- start start 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0b04ad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a8099f90 0x7fa0a815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0b04ad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0a809a790 0x7fa0a81f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0b04ad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0a815da60 0x7fa0a80c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0b04ad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a815c080 con 0x7fa0a8099f90 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0b04ad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8160ef0 con 0x7fa0a815da60 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0b04ad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80c70e0 con 0x7fa0a809a790 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.427+0000 7fa0ae249700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a8099f90 0x7fa0a815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ae249700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a8099f90 0x7fa0a815bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57338/0 (socket says 172.21.15.5:57338) 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ae249700 1 -- 172.21.15.5:0/919561158 learned_addr learned my addr 172.21.15.5:0/919561158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ada48700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0a809a790 0x7fa0a81f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ae249700 1 -- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0a809a790 msgr2=0x7fa0a81f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ae249700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0a809a790 0x7fa0a81f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ae249700 1 -- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0a815da60 msgr2=0x7fa0a80c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0aea4a700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0a815da60 0x7fa0a80c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ae249700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0a815da60 0x7fa0a80c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ae249700 1 -- 172.21.15.5:0/919561158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0980094a0 con 0x7fa0a8099f90 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0aea4a700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0a815da60 0x7fa0a80c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0ae249700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a8099f90 0x7fa0a815bb40 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400c550 tx=0x7fa0a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa09effd700 1 -- 172.21.15.5:0/919561158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4004c80 con 0x7fa0a8099f90 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa09effd700 1 -- 172.21.15.5:0/919561158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a4004e00 con 0x7fa0a8099f90 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a80c5bc0 con 0x7fa0a8099f90 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa09effd700 1 -- 172.21.15.5:0/919561158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4007310 con 0x7fa0a8099f90 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.428+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a80c58e0 con 0x7fa0a8099f90 2024-05-08T23:27:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.429+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a8074bc0 con 0x7fa0a8099f90 2024-05-08T23:27:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.430+0000 7fa09effd700 1 -- 172.21.15.5:0/919561158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa0a4007470 con 0x7fa0a8099f90 2024-05-08T23:27:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.430+0000 7fa09effd700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa08c06ac90 0x7fa08c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.430+0000 7fa09effd700 1 -- 172.21.15.5:0/919561158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0a4089040 con 0x7fa0a8099f90 2024-05-08T23:27:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.431+0000 7fa0ada48700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa08c06ac90 0x7fa08c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.431+0000 7fa0ada48700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa08c06ac90 0x7fa08c06d150 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa0a815c6d0 tx=0x7fa098008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.435+0000 7fa09effd700 1 -- 172.21.15.5:0/919561158 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa0a4054220 con 0x7fa0a8099f90 2024-05-08T23:27:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.652+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0a80d1250 con 0x7fa0a8099f90 2024-05-08T23:27:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.652+0000 7fa09effd700 1 -- 172.21.15.5:0/919561158 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0a4057840 con 0x7fa0a8099f90 2024-05-08T23:27:41.652 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:27:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.654+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa08c06ac90 msgr2=0x7fa08c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.654+0000 7fa0b04ad700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa08c06ac90 0x7fa08c06d150 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa0a815c6d0 tx=0x7fa098008030 comp rx=0 tx=0).stop 2024-05-08T23:27:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a8099f90 msgr2=0x7fa0a815bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a8099f90 0x7fa0a815bb40 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400c550 tx=0x7fa0a400c910 comp rx=0 tx=0).stop 2024-05-08T23:27:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 shutdown_connections 2024-05-08T23:27:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa08c06ac90 0x7fa08c06d150 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0a815da60 0x7fa0a80c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a8099f90 0x7fa0a815bb40 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 --2- 172.21.15.5:0/919561158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0a809a790 0x7fa0a81f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 >> 172.21.15.5:0/919561158 conn(0x7fa0a80b2f60 msgr2=0x7fa0a80d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 shutdown_connections 2024-05-08T23:27:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.655+0000 7fa0b04ad700 1 -- 172.21.15.5:0/919561158 wait complete. 2024-05-08T23:27:41.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.805+0000 7f7205e39700 1 Processor -- start 2024-05-08T23:27:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.805+0000 7f7205e39700 1 -- start start 2024-05-08T23:27:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.806+0000 7f7205e39700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f720015b900 0x7f72000cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.806+0000 7f7205e39700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7200099a50 0x7f7200099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.806+0000 7f7205e39700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000b1410 con 0x7f720009a410 2024-05-08T23:27:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.806+0000 7f7205e39700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720009a8a0 con 0x7f7200099a50 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.806+0000 7f7205e39700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72000b0e30 con 0x7f720015b900 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.806+0000 7f71fffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f720009a410 0x7f72000cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58934/0 (socket says 172.21.15.5:58934) 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.806+0000 7f71fffff700 1 -- 172.21.15.5:0/1777007023 learned_addr learned my addr 172.21.15.5:0/1777007023 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71ff7fe700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f720015b900 0x7f72000cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71feffd700 1 --2- 172.21.15.5:0/1777007023 >> v2:172.21.15.49:3300/0 conn(0x7f7200099a50 0x7f7200099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206847878 0 0) 0x7f72000b1410 con 0x7f720009a410 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e80035b0 con 0x7f720009a410 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 694639180 0 0) 0x7f71e80035b0 con 0x7f720009a410 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f720015b900 msgr2=0x7f72000cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f720015b900 0x7f72000cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 >> v2:172.21.15.49:3300/0 conn(0x7f7200099a50 msgr2=0x7f7200099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 --2- 172.21.15.5:0/1777007023 >> v2:172.21.15.49:3300/0 conn(0x7f7200099a50 0x7f7200099ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000cffd0 con 0x7f720009a410 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f71f80019f0 con 0x7f720009a410 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 >> v1:172.21.15.5:6789/0 conn(0x7f720009a410 legacy=0x7f72000cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8003a00 0x7f71e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f720015b900 0x7f71e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8004d90 0x7f71e8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720009a8a0 con 0x7f71e8004d90 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72000b0e30 con 0x7f71e8003a00 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e800ba10 con 0x7f720015b900 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71fffff700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8004d90 0x7f71e8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71feffd700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f720015b900 0x7f71e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.807+0000 7f71ff7fe700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8003a00 0x7f71e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fffff700 1 -- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f720015b900 msgr2=0x7f71e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fffff700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f720015b900 0x7f71e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fffff700 1 -- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8003a00 msgr2=0x7f71e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fffff700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71e8003a00 0x7f71e8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fffff700 1 -- 172.21.15.5:0/1777007023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80035b0 con 0x7f71e8004d90 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fffff700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8004d90 0x7f71e8005210 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f71f800a860 tx=0x7f71e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49725ea9c1816b6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8012600 con 0x7f71e8004d90 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8012780 con 0x7f71e8004d90 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.808+0000 7f71fe7fc700 1 -- 172.21.15.5:0/1777007023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8012b30 con 0x7f71e8004d90 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.809+0000 7f7205e39700 1 -- 172.21.15.5:0/1777007023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8004d90 msgr2=0x7f71e8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.809+0000 7f7205e39700 1 --2- 172.21.15.5:0/1777007023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8004d90 0x7f71e8005210 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f71f800a860 tx=0x7f71e8005880 comp rx=0 tx=0).stop 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.809+0000 7f71fffff700 1 -- 172.21.15.5:0/1777007023 reap_dead start 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.809+0000 7f7205e39700 1 -- 172.21.15.5:0/1777007023 shutdown_connections 2024-05-08T23:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.809+0000 7f7205e39700 1 -- 172.21.15.5:0/1777007023 >> 172.21.15.5:0/1777007023 conn(0x7f72000b2f20 msgr2=0x7f72000b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.809+0000 7f7205e39700 1 -- 172.21.15.5:0/1777007023 shutdown_connections 2024-05-08T23:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.809+0000 7f7205e39700 1 -- 172.21.15.5:0/1777007023 wait complete. 2024-05-08T23:27:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.810+0000 7f7205e39700 1 Processor -- start 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.810+0000 7f7205e39700 1 -- start start 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f7205e39700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7200150850 0x7f72000b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f7205e39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7200150c30 0x7f72001593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f7205e39700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720015bce0 0x7f7200159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f7205e39700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72000b5ca0 con 0x7f720015bce0 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f7205e39700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72000b6900 con 0x7f7200150c30 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f7205e39700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72000b5fd0 con 0x7f7200150850 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720015bce0 0x7f7200159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720015bce0 0x7f7200159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57366/0 (socket says 172.21.15.5:57366) 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71fffff700 1 -- 172.21.15.5:0/1911122703 learned_addr learned my addr 172.21.15.5:0/1911122703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71ff7fe700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7200150850 0x7f72000b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71feffd700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7200150c30 0x7f72001593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71fffff700 1 -- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7200150850 msgr2=0x7f72000b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71fffff700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7200150850 0x7f72000b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71fffff700 1 -- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7200150c30 msgr2=0x7f72001593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71fffff700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7200150c30 0x7f72001593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.811+0000 7f71fffff700 1 -- 172.21.15.5:0/1911122703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000b62b0 con 0x7f720015bce0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.812+0000 7f71fffff700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720015bce0 0x7f7200159900 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f71f8012000 tx=0x7f71f8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.812+0000 7f71e7fff700 1 -- 172.21.15.5:0/1911122703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f801d5f0 con 0x7f720015bce0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.812+0000 7f71e7fff700 1 -- 172.21.15.5:0/1911122703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f801d750 con 0x7f720015bce0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.812+0000 7f71e7fff700 1 -- 172.21.15.5:0/1911122703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f801db00 con 0x7f720015bce0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.812+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72000b6590 con 0x7f720015bce0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.812+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72000b6d90 con 0x7f720015bce0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.813+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7200068000 con 0x7f720015bce0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.813+0000 7f71e7fff700 1 -- 172.21.15.5:0/1911122703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f71f8019070 con 0x7f720015bce0 2024-05-08T23:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.814+0000 7f71e7fff700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71ec06ac90 0x7f71ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.814+0000 7f71e7fff700 1 -- 172.21.15.5:0/1911122703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71f808aa80 con 0x7f720015bce0 2024-05-08T23:27:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.814+0000 7f71ff7fe700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71ec06ac90 0x7f71ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.815+0000 7f71ff7fe700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71ec06ac90 0x7f71ec06d150 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009a20 tx=0x7f71f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:41.819+0000 7f71e7fff700 1 -- 172.21.15.5:0/1911122703 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f71f8055c70 con 0x7f720015bce0 2024-05-08T23:27:42.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.053+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f72000b6a40 con 0x7f720015bce0 2024-05-08T23:27:42.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.053+0000 7f71e7fff700 1 -- 172.21.15.5:0/1911122703 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v84) v1 ==== 80+0+0 (secure 0 0 0) 0x7f71f8017080 con 0x7f720015bce0 2024-05-08T23:27:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71ec06ac90 msgr2=0x7f71ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71ec06ac90 0x7f71ec06d150 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009a20 tx=0x7f71f0008040 comp rx=0 tx=0).stop 2024-05-08T23:27:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720015bce0 msgr2=0x7f7200159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720015bce0 0x7f7200159900 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f71f8012000 tx=0x7f71f8001d80 comp rx=0 tx=0).stop 2024-05-08T23:27:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 shutdown_connections 2024-05-08T23:27:42.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f71ec06ac90 0x7f71ec06d150 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720015bce0 0x7f7200159900 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7200150850 0x7f72000b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 --2- 172.21.15.5:0/1911122703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7200150c30 0x7f72001593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 >> 172.21.15.5:0/1911122703 conn(0x7f72000b2f20 msgr2=0x7f72000b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:42.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 shutdown_connections 2024-05-08T23:27:42.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.056+0000 7f7205e39700 1 -- 172.21.15.5:0/1911122703 wait complete. 2024-05-08T23:27:42.057 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:27:42.069 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:27:42.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:42.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T23:27:42.112 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:27:42.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:27:42.112 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:27:42.173 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:27:42.173 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:42.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:27:42.196 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:27:42.196 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:42.235 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:27:42.235 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:27:42.236 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:42.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:27:42.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.421+0000 7f4152a60700 1 Processor -- start 2024-05-08T23:27:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.422+0000 7f4152a60700 1 -- start start 2024-05-08T23:27:42.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.422+0000 7f4152a60700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c0d3f40 0x7f414c161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.422+0000 7f4152a60700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f414c15db80 0x7f414c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.422+0000 7f4152a60700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414c162450 con 0x7f414c15d7a0 2024-05-08T23:27:42.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.422+0000 7f4152a60700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414c0cee00 con 0x7f414c15db80 2024-05-08T23:27:42.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.422+0000 7f4152a60700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c0b27e0 con 0x7f414c0d3f40 2024-05-08T23:27:42.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f4150ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f414c15db80 0x7f414c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f4150ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f414c15db80 0x7f414c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57376/0 (socket says 172.21.15.5:57376) 2024-05-08T23:27:42.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f4150ffd700 1 -- 172.21.15.5:0/1335734097 learned_addr learned my addr 172.21.15.5:0/1335734097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f414b7fe700 1 --2- 172.21.15.5:0/1335734097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c0d3f40 0x7f414c161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f4150ffd700 1 -- 172.21.15.5:0/1335734097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c0d3f40 msgr2=0x7f414c161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f4150ffd700 1 --2- 172.21.15.5:0/1335734097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c0d3f40 0x7f414c161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f4150ffd700 1 -- 172.21.15.5:0/1335734097 >> v1:172.21.15.49:6789/0 conn(0x7f414c15d7a0 legacy=0x7f414c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f4150ffd700 1 -- 172.21.15.5:0/1335734097 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c0d4400 con 0x7f414c15db80 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f4150ffd700 1 --2- 172.21.15.5:0/1335734097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c0cf290 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f4144009e50 tx=0x7f414400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939b15a206e889c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f414affd700 1 -- 172.21.15.5:0/1335734097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400e070 con 0x7f414c15db80 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f414affd700 1 -- 172.21.15.5:0/1335734097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414400b9c0 con 0x7f414c15db80 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.423+0000 7f414affd700 1 -- 172.21.15.5:0/1335734097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400bd70 con 0x7f414c15db80 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.424+0000 7f4152a60700 1 -- 172.21.15.5:0/1335734097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 msgr2=0x7f414c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.424+0000 7f4152a60700 1 --2- 172.21.15.5:0/1335734097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c0cf290 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f4144009e50 tx=0x7f414400b330 comp rx=0 tx=0).stop 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.424+0000 7f4152a60700 1 -- 172.21.15.5:0/1335734097 shutdown_connections 2024-05-08T23:27:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.424+0000 7f4152a60700 1 --2- 172.21.15.5:0/1335734097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c0cf290 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.424+0000 7f4152a60700 1 --2- 172.21.15.5:0/1335734097 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c0d3f40 0x7f414c161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.424+0000 7f4152a60700 1 -- 172.21.15.5:0/1335734097 >> 172.21.15.5:0/1335734097 conn(0x7f414c08afe0 msgr2=0x7f414c0d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.424+0000 7f4152a60700 1 -- 172.21.15.5:0/1335734097 shutdown_connections 2024-05-08T23:27:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.424+0000 7f4152a60700 1 -- 172.21.15.5:0/1335734097 wait complete. 2024-05-08T23:27:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.425+0000 7f4152a60700 1 Processor -- start 2024-05-08T23:27:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.426+0000 7f4152a60700 1 -- start start 2024-05-08T23:27:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.426+0000 7f4152a60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f414c0d3f40 0x7f414c15e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.426+0000 7f4152a60700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c15d7a0 0x7f414c159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.426+0000 7f4152a60700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.426+0000 7f4152a60700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c15eee0 con 0x7f414c15db80 2024-05-08T23:27:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.426+0000 7f4152a60700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c15a3f0 con 0x7f414c0d3f40 2024-05-08T23:27:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.426+0000 7f4152a60700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c15f0d0 con 0x7f414c15d7a0 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c1603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57378/0 (socket says 172.21.15.5:57378) 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 -- 172.21.15.5:0/2998954330 learned_addr learned my addr 172.21.15.5:0/2998954330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f414b7fe700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c15d7a0 0x7f414c159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f414bfff700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f414c0d3f40 0x7f414c15e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 -- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c15d7a0 msgr2=0x7f414c159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c15d7a0 0x7f414c159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 -- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f414c0d3f40 msgr2=0x7f414c15e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f414c0d3f40 0x7f414c15e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 -- 172.21.15.5:0/2998954330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4144009b30 con 0x7f414c15db80 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4150ffd700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c1603c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f4144012000 tx=0x7f4144004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4148ff9700 1 -- 172.21.15.5:0/2998954330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144004800 con 0x7f414c15db80 2024-05-08T23:27:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4148ff9700 1 -- 172.21.15.5:0/2998954330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144004960 con 0x7f414c15db80 2024-05-08T23:27:42.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c15f3b0 con 0x7f414c15db80 2024-05-08T23:27:42.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4148ff9700 1 -- 172.21.15.5:0/2998954330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414401c680 con 0x7f414c15db80 2024-05-08T23:27:42.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.427+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c0c7430 con 0x7f414c15db80 2024-05-08T23:27:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.429+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414c0d33c0 con 0x7f414c15db80 2024-05-08T23:27:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.429+0000 7f4148ff9700 1 -- 172.21.15.5:0/2998954330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f414401c7e0 con 0x7f414c15db80 2024-05-08T23:27:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.429+0000 7f4148ff9700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f413806ac90 0x7f413806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.429+0000 7f4148ff9700 1 -- 172.21.15.5:0/2998954330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f414408ab40 con 0x7f414c15db80 2024-05-08T23:27:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.430+0000 7f414bfff700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f413806ac90 0x7f413806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.430+0000 7f414bfff700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f413806ac90 0x7f413806d150 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f414c0c66b0 tx=0x7f413c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:42.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.434+0000 7f4148ff9700 1 -- 172.21.15.5:0/2998954330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4144055db0 con 0x7f414c15db80 2024-05-08T23:27:42.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.702+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f414c068000 con 0x7f414c15db80 2024-05-08T23:27:42.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.702+0000 7f4148ff9700 1 -- 172.21.15.5:0/2998954330 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7f41440593d0 con 0x7f414c15db80 2024-05-08T23:27:42.702 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:42.703 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:42.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f413806ac90 msgr2=0x7f413806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f413806ac90 0x7f413806d150 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f414c0c66b0 tx=0x7f413c009040 comp rx=0 tx=0).stop 2024-05-08T23:27:42.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 msgr2=0x7f414c1603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c1603c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f4144012000 tx=0x7f4144004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 shutdown_connections 2024-05-08T23:27:42.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f413806ac90 0x7f413806d150 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f414c15db80 0x7f414c1603c0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f414c0d3f40 0x7f414c15e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 --2- 172.21.15.5:0/2998954330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f414c15d7a0 0x7f414c159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 >> 172.21.15.5:0/2998954330 conn(0x7f414c08afe0 msgr2=0x7f414c06bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:42.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 shutdown_connections 2024-05-08T23:27:42.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.705+0000 7f4152a60700 1 -- 172.21.15.5:0/2998954330 wait complete. 2024-05-08T23:27:42.707 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:42.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:27:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.855+0000 7f995aaaf700 1 Processor -- start 2024-05-08T23:27:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.855+0000 7f995aaaf700 1 -- start start 2024-05-08T23:27:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.856+0000 7f995aaaf700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 0x7f995415b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.856+0000 7f995aaaf700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540cd910 con 0x7f995406d510 2024-05-08T23:27:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.856+0000 7f995aaaf700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540cdbf0 con 0x7f995406f270 2024-05-08T23:27:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.856+0000 7f995aaaf700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540cded0 con 0x7f995406ee90 2024-05-08T23:27:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.856+0000 7f995904c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f995406d510 0x7f995406d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58946/0 (socket says 172.21.15.5:58946) 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.856+0000 7f995904c700 1 -- 172.21.15.5:0/2414468066 learned_addr learned my addr 172.21.15.5:0/2414468066 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f995884b700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 0x7f995415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14544601 0 0) 0x7f99540cd910 con 0x7f995406d510 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99440057c0 con 0x7f995406d510 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283114398 0 0) 0x7f99440057c0 con 0x7f995406d510 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 msgr2=0x7f995415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 0x7f995415b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 >> v1:172.21.15.49:6789/0 conn(0x7f995406f270 legacy=0x7f99540cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540ce1b0 con 0x7f995406d510 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f994c0019f0 con 0x7f995406d510 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f995884b700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 0x7f995415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 >> v1:172.21.15.5:6789/0 conn(0x7f995406d510 legacy=0x7f995406d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f9944006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 0x7f9944006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9944006fa0 0x7f9944007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540cded0 con 0x7f9944005c40 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994400dc20 con 0x7f9944006fa0 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994400dd90 con 0x7f995406ee90 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f995884b700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f9944006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f9953fff700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 0x7f9944006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f995904c700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9944006fa0 0x7f9944007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.857+0000 7f995884b700 1 -- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 msgr2=0x7f9944006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995884b700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 0x7f9944006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995884b700 1 -- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9944006fa0 msgr2=0x7f9944007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995884b700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9944006fa0 0x7f9944007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995884b700 1 -- 172.21.15.5:0/2414468066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99480094a0 con 0x7f9944005c40 2024-05-08T23:27:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f9953fff700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995406ee90 0x7f9944006a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995884b700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f9944006020 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9948009d30 tx=0x7f9944007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5226c40615851699 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948009e80 con 0x7f9944005c40 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994800e9a0 con 0x7f9944005c40 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f99537fe700 1 -- 172.21.15.5:0/2414468066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994800ed10 con 0x7f9944005c40 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995aaaf700 1 -- 172.21.15.5:0/2414468066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 msgr2=0x7f9944006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995aaaf700 1 --2- 172.21.15.5:0/2414468066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f9944006020 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9948009d30 tx=0x7f9944007a90 comp rx=0 tx=0).stop 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995904c700 1 -- 172.21.15.5:0/2414468066 reap_dead start 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995aaaf700 1 -- 172.21.15.5:0/2414468066 shutdown_connections 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.858+0000 7f995aaaf700 1 -- 172.21.15.5:0/2414468066 >> 172.21.15.5:0/2414468066 conn(0x7f99540cfce0 msgr2=0x7f99540d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.859+0000 7f995aaaf700 1 -- 172.21.15.5:0/2414468066 shutdown_connections 2024-05-08T23:27:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.859+0000 7f995aaaf700 1 -- 172.21.15.5:0/2414468066 wait complete. 2024-05-08T23:27:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.860+0000 7f995aaaf700 1 Processor -- start 2024-05-08T23:27:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.860+0000 7f995aaaf700 1 -- start start 2024-05-08T23:27:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995aaaf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f995415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995aaaf700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995415d580 0x7f99540b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995aaaf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954140fa0 0x7f9954141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995aaaf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995415c5f0 con 0x7f9944005c40 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995aaaf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9954155370 con 0x7f9954140fa0 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995aaaf700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9954155610 con 0x7f995415d580 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f995415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f995415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57388/0 (socket says 172.21.15.5:57388) 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 -- 172.21.15.5:0/3067332737 learned_addr learned my addr 172.21.15.5:0/3067332737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f9953fff700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995415d580 0x7f99540b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995904c700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954140fa0 0x7f9954141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 -- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995415d580 msgr2=0x7f99540b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995415d580 0x7f99540b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 -- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954140fa0 msgr2=0x7f9954141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954140fa0 0x7f9954141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 -- 172.21.15.5:0/3067332737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99541558f0 con 0x7f9944005c40 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995904c700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954140fa0 0x7f9954141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.861+0000 7f995884b700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f995415bee0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f99440069e0 tx=0x7f99480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.862+0000 7f99517fa700 1 -- 172.21.15.5:0/3067332737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948021070 con 0x7f9944005c40 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.862+0000 7f99517fa700 1 -- 172.21.15.5:0/3067332737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9948009e80 con 0x7f9944005c40 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.862+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9954155bd0 con 0x7f9944005c40 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.862+0000 7f99517fa700 1 -- 172.21.15.5:0/3067332737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99480038c0 con 0x7f9944005c40 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.862+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9954141960 con 0x7f9944005c40 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.863+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9954068000 con 0x7f9944005c40 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.863+0000 7f99517fa700 1 -- 172.21.15.5:0/3067332737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9948025070 con 0x7f9944005c40 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.864+0000 7f99517fa700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f993806ac90 0x7f993806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.864+0000 7f99517fa700 1 -- 172.21.15.5:0/3067332737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f994808a520 con 0x7f9944005c40 2024-05-08T23:27:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.864+0000 7f9953fff700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f993806ac90 0x7f993806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.864+0000 7f9953fff700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f993806ac90 0x7f993806d150 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9940000c00 tx=0x7f9940009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:42.868+0000 7f99517fa700 1 -- 172.21.15.5:0/3067332737 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9948055810 con 0x7f9944005c40 2024-05-08T23:27:43.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.088+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99540cc310 con 0x7f9944005c40 2024-05-08T23:27:43.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.089+0000 7f99517fa700 1 -- 172.21.15.5:0/3067332737 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15996 (secure 0 0 0) 0x7f9948058e30 con 0x7f9944005c40 2024-05-08T23:27:43.090 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:43.090 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:57.193438+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:54.037450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:55.162938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:27:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f993806ac90 msgr2=0x7f993806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f993806ac90 0x7f993806d150 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9940000c00 tx=0x7f9940009040 comp rx=0 tx=0).stop 2024-05-08T23:27:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 msgr2=0x7f995415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f995415bee0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f99440069e0 tx=0x7f99480047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 shutdown_connections 2024-05-08T23:27:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f993806ac90 0x7f993806d150 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954140fa0 0x7f9954141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9944005c40 0x7f995415bee0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 --2- 172.21.15.5:0/3067332737 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f995415d580 0x7f99540b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 >> 172.21.15.5:0/3067332737 conn(0x7f99540cfce0 msgr2=0x7f9954158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 shutdown_connections 2024-05-08T23:27:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.092+0000 7f995aaaf700 1 -- 172.21.15.5:0/3067332737 wait complete. 2024-05-08T23:27:43.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:26:54.037450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:26:55.162938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:27:43.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:27:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.248+0000 7f7c65518700 1 Processor -- start 2024-05-08T23:27:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.248+0000 7f7c65518700 1 -- start start 2024-05-08T23:27:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c65518700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c600aea70 0x7f7c600b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c65518700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6015add0 con 0x7f7c60154df0 2024-05-08T23:27:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c65518700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6015b070 con 0x7f7c600d0a60 2024-05-08T23:27:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c65518700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6015abe0 con 0x7f7c600aea70 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c5effd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c60154df0 0x7f7c600d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58952/0 (socket says 172.21.15.5:58952) 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c5effd700 1 -- 172.21.15.5:0/1834685118 learned_addr learned my addr 172.21.15.5:0/1834685118 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c5e7fc700 1 --2- 172.21.15.5:0/1834685118 >> v2:172.21.15.49:3300/0 conn(0x7f7c600aea70 0x7f7c600b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c5dffb700 1 -- 172.21.15.5:0/1834685118 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287082720 0 0) 0x7f7c6015add0 con 0x7f7c60154df0 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.249+0000 7f7c5dffb700 1 -- 172.21.15.5:0/1834685118 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c4c0057c0 con 0x7f7c60154df0 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5dffb700 1 -- 172.21.15.5:0/1834685118 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134665529 0 0) 0x7f7c6015b070 con 0x7f7c600d0a60 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5dffb700 1 -- 172.21.15.5:0/1834685118 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6015add0 con 0x7f7c600d0a60 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5dffb700 1 -- 172.21.15.5:0/1834685118 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2615665105 0 0) 0x7f7c4c0057c0 con 0x7f7c60154df0 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5e7fc700 1 -- 172.21.15.5:0/1834685118 >> v1:172.21.15.49:6789/0 conn(0x7f7c600d0a60 legacy=0x7f7c600b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5e7fc700 1 -- 172.21.15.5:0/1834685118 >> v1:172.21.15.5:6789/0 conn(0x7f7c60154df0 legacy=0x7f7c600d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5e7fc700 1 -- 172.21.15.5:0/1834685118 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6015b350 con 0x7f7c600aea70 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5e7fc700 1 --2- 172.21.15.5:0/1834685118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c600aea70 0x7f7c600b3e70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009840 tx=0x7f7c48009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55553a88c79864bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5dffb700 1 -- 172.21.15.5:0/1834685118 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500015f0 con 0x7f7c600aea70 2024-05-08T23:27:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5dffb700 1 -- 172.21.15.5:0/1834685118 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c48004030 con 0x7f7c600aea70 2024-05-08T23:27:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.250+0000 7f7c5dffb700 1 -- 172.21.15.5:0/1834685118 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c480043c0 con 0x7f7c600aea70 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.251+0000 7f7c65518700 1 -- 172.21.15.5:0/1834685118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c600aea70 msgr2=0x7f7c600b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.251+0000 7f7c65518700 1 --2- 172.21.15.5:0/1834685118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c600aea70 0x7f7c600b3e70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009840 tx=0x7f7c48009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.251+0000 7f7c65518700 1 -- 172.21.15.5:0/1834685118 shutdown_connections 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.251+0000 7f7c65518700 1 --2- 172.21.15.5:0/1834685118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c600aea70 0x7f7c600b3e70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.251+0000 7f7c65518700 1 -- 172.21.15.5:0/1834685118 >> 172.21.15.5:0/1834685118 conn(0x7f7c60099a50 msgr2=0x7f7c600b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.251+0000 7f7c65518700 1 -- 172.21.15.5:0/1834685118 shutdown_connections 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.251+0000 7f7c65518700 1 -- 172.21.15.5:0/1834685118 wait complete. 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.252+0000 7f7c65518700 1 Processor -- start 2024-05-08T23:27:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c65518700 1 -- start start 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c65518700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c600aea70 0x7f7c601f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c65518700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c600d0a60 0x7f7c601f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c65518700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c60154df0 0x7f7c600ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c65518700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c601f4050 con 0x7f7c600aea70 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c65518700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6014ec50 con 0x7f7c60154df0 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c65518700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6014e040 con 0x7f7c600d0a60 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c5effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c600aea70 0x7f7c601f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c5effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c600aea70 0x7f7c601f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57396/0 (socket says 172.21.15.5:57396) 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.253+0000 7f7c5effd700 1 -- 172.21.15.5:0/2960827456 learned_addr learned my addr 172.21.15.5:0/2960827456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c5e7fc700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c600d0a60 0x7f7c601f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c5f7fe700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c60154df0 0x7f7c600ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c5effd700 1 -- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c600d0a60 msgr2=0x7f7c601f04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c5effd700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c600d0a60 0x7f7c601f04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c5effd700 1 -- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c60154df0 msgr2=0x7f7c600ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c5effd700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c60154df0 0x7f7c600ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c5effd700 1 -- 172.21.15.5:0/2960827456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c480094a0 con 0x7f7c600aea70 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c5effd700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c600aea70 0x7f7c601f3b10 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c450 tx=0x7f7c5000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c577fe700 1 -- 172.21.15.5:0/2960827456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004c80 con 0x7f7c600aea70 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c577fe700 1 -- 172.21.15.5:0/2960827456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50004e00 con 0x7f7c600aea70 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6014e320 con 0x7f7c600aea70 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c577fe700 1 -- 172.21.15.5:0/2960827456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50007310 con 0x7f7c600aea70 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.254+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6014e8e0 con 0x7f7c600aea70 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.255+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c60074bc0 con 0x7f7c600aea70 2024-05-08T23:27:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.256+0000 7f7c577fe700 1 -- 172.21.15.5:0/2960827456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7c50007470 con 0x7f7c600aea70 2024-05-08T23:27:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.256+0000 7f7c577fe700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c4006acc0 0x7f7c4006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.256+0000 7f7c577fe700 1 -- 172.21.15.5:0/2960827456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c50089050 con 0x7f7c600aea70 2024-05-08T23:27:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.257+0000 7f7c5e7fc700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c4006acc0 0x7f7c4006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.257+0000 7f7c5e7fc700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c4006acc0 0x7f7c4006d180 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7c480090b0 tx=0x7f7c48008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.261+0000 7f7c577fe700 1 -- 172.21.15.5:0/2960827456 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7c50054230 con 0x7f7c600aea70 2024-05-08T23:27:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.478+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c601f09e0 con 0x7f7c600aea70 2024-05-08T23:27:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.478+0000 7f7c577fe700 1 -- 172.21.15.5:0/2960827456 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c50057850 con 0x7f7c600aea70 2024-05-08T23:27:43.478 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:27:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.480+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c4006acc0 msgr2=0x7f7c4006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.480+0000 7f7c65518700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c4006acc0 0x7f7c4006d180 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7c480090b0 tx=0x7f7c48008030 comp rx=0 tx=0).stop 2024-05-08T23:27:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.480+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c600aea70 msgr2=0x7f7c601f3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.480+0000 7f7c65518700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c600aea70 0x7f7c601f3b10 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c450 tx=0x7f7c5000c810 comp rx=0 tx=0).stop 2024-05-08T23:27:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.481+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 shutdown_connections 2024-05-08T23:27:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.481+0000 7f7c65518700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7c4006acc0 0x7f7c4006d180 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.481+0000 7f7c65518700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c60154df0 0x7f7c600ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.481+0000 7f7c65518700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c600aea70 0x7f7c601f3b10 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.481+0000 7f7c65518700 1 --2- 172.21.15.5:0/2960827456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7c600d0a60 0x7f7c601f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.481+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 >> 172.21.15.5:0/2960827456 conn(0x7f7c60099a50 msgr2=0x7f7c600b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.481+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 shutdown_connections 2024-05-08T23:27:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.481+0000 7f7c65518700 1 -- 172.21.15.5:0/2960827456 wait complete. 2024-05-08T23:27:43.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:27:43.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.633+0000 7f4e85bc3700 1 Processor -- start 2024-05-08T23:27:43.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.633+0000 7f4e85bc3700 1 -- start start 2024-05-08T23:27:43.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.634+0000 7f4e85bc3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e800d13a0 0x7f4e800d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.634+0000 7f4e85bc3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4e800cdc90 0x7f4e800b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.634+0000 7f4e85bc3700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e800b1200 con 0x7f4e8006bd60 2024-05-08T23:27:43.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.634+0000 7f4e85bc3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e800d1cc0 con 0x7f4e800d13a0 2024-05-08T23:27:43.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.634+0000 7f4e85bc3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e800ce5c0 con 0x7f4e800cdc90 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.634+0000 7f4e7f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e800d13a0 0x7f4e800d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.634+0000 7f4e7f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e800d13a0 0x7f4e800d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57408/0 (socket says 172.21.15.5:57408) 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.634+0000 7f4e7f7fe700 1 -- 172.21.15.5:0/2808586642 learned_addr learned my addr 172.21.15.5:0/2808586642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7ffff700 1 --2- 172.21.15.5:0/2808586642 >> v2:172.21.15.49:3300/0 conn(0x7f4e800cdc90 0x7f4e800b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7f7fe700 1 -- 172.21.15.5:0/2808586642 >> v2:172.21.15.49:3300/0 conn(0x7f4e800cdc90 msgr2=0x7f4e800b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7f7fe700 1 --2- 172.21.15.5:0/2808586642 >> v2:172.21.15.49:3300/0 conn(0x7f4e800cdc90 0x7f4e800b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7f7fe700 1 -- 172.21.15.5:0/2808586642 >> v1:172.21.15.49:6789/0 conn(0x7f4e8006bd60 legacy=0x7f4e800cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7f7fe700 1 -- 172.21.15.5:0/2808586642 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800ce8a0 con 0x7f4e800d13a0 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7f7fe700 1 --2- 172.21.15.5:0/2808586642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e800d13a0 0x7f4e800d1780 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4e800cfe90 tx=0x7f4e70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351958f9d1dc6008 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7e7fc700 1 -- 172.21.15.5:0/2808586642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004230 con 0x7f4e800d13a0 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7e7fc700 1 -- 172.21.15.5:0/2808586642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70004390 con 0x7f4e800d13a0 2024-05-08T23:27:43.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.635+0000 7f4e7e7fc700 1 -- 172.21.15.5:0/2808586642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e700036a0 con 0x7f4e800d13a0 2024-05-08T23:27:43.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.636+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2808586642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e800d13a0 msgr2=0x7f4e800d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.636+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2808586642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e800d13a0 0x7f4e800d1780 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4e800cfe90 tx=0x7f4e70009bd0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.636+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2808586642 shutdown_connections 2024-05-08T23:27:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.636+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2808586642 >> v2:172.21.15.49:3300/0 conn(0x7f4e800cdc90 0x7f4e800b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.636+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2808586642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e800d13a0 0x7f4e800d1780 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.636+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2808586642 >> 172.21.15.5:0/2808586642 conn(0x7f4e8008afe0 msgr2=0x7f4e8008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.636+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2808586642 shutdown_connections 2024-05-08T23:27:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.636+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2808586642 wait complete. 2024-05-08T23:27:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.637+0000 7f4e85bc3700 1 Processor -- start 2024-05-08T23:27:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.637+0000 7f4e85bc3700 1 -- start start 2024-05-08T23:27:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e85bc3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e8006bd60 0x7f4e8014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e85bc3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e800cdc90 0x7f4e800b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e85bc3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4e800d13a0 0x7f4e800b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e85bc3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8015ffd0 con 0x7f4e8006bd60 2024-05-08T23:27:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e85bc3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8009be90 con 0x7f4e800cdc90 2024-05-08T23:27:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e85bc3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8009c170 con 0x7f4e800d13a0 2024-05-08T23:27:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e8006bd60 0x7f4e8014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e8006bd60 0x7f4e8014f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57414/0 (socket says 172.21.15.5:57414) 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e7f7fe700 1 -- 172.21.15.5:0/2688304193 learned_addr learned my addr 172.21.15.5:0/2688304193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e7ffff700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4e800d13a0 0x7f4e800b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.638+0000 7f4e7effd700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e800cdc90 0x7f4e800b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e7f7fe700 1 -- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4e800d13a0 msgr2=0x7f4e800b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e7f7fe700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4e800d13a0 0x7f4e800b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e7f7fe700 1 -- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e800cdc90 msgr2=0x7f4e800b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e7f7fe700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e800cdc90 0x7f4e800b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e7f7fe700 1 -- 172.21.15.5:0/2688304193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70009550 con 0x7f4e8006bd60 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e7effd700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e800cdc90 0x7f4e800b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e7f7fe700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e8006bd60 0x7f4e8014f760 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009130 tx=0x7f4e70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e77fff700 1 -- 172.21.15.5:0/2688304193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70003c90 con 0x7f4e8006bd60 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e77fff700 1 -- 172.21.15.5:0/2688304193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7001f070 con 0x7f4e8006bd60 2024-05-08T23:27:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8009c450 con 0x7f4e8006bd60 2024-05-08T23:27:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e77fff700 1 -- 172.21.15.5:0/2688304193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70011870 con 0x7f4e8006bd60 2024-05-08T23:27:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.639+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8009ca10 con 0x7f4e8006bd60 2024-05-08T23:27:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.640+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8009ced0 con 0x7f4e8006bd60 2024-05-08T23:27:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.641+0000 7f4e77fff700 1 -- 172.21.15.5:0/2688304193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4e70021030 con 0x7f4e8006bd60 2024-05-08T23:27:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.641+0000 7f4e77fff700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4e6006ac90 0x7f4e6006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.641+0000 7f4e77fff700 1 -- 172.21.15.5:0/2688304193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e7008a7e0 con 0x7f4e8006bd60 2024-05-08T23:27:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.641+0000 7f4e7effd700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4e6006ac90 0x7f4e6006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.642+0000 7f4e7effd700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4e6006ac90 0x7f4e6006d150 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4e800afa80 tx=0x7f4e68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.646+0000 7f4e77fff700 1 -- 172.21.15.5:0/2688304193 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4e8009ced0 con 0x7f4e8006bd60 2024-05-08T23:27:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.862+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e8009ced0 con 0x7f4e8006bd60 2024-05-08T23:27:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.863+0000 7f4e77fff700 1 -- 172.21.15.5:0/2688304193 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e8009ced0 con 0x7f4e8006bd60 2024-05-08T23:27:43.863 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:27:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4e6006ac90 msgr2=0x7f4e6006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4e6006ac90 0x7f4e6006d150 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4e800afa80 tx=0x7f4e68008040 comp rx=0 tx=0).stop 2024-05-08T23:27:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e8006bd60 msgr2=0x7f4e8014f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e8006bd60 0x7f4e8014f760 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009130 tx=0x7f4e70004750 comp rx=0 tx=0).stop 2024-05-08T23:27:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 shutdown_connections 2024-05-08T23:27:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4e6006ac90 0x7f4e6006d150 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4e800d13a0 0x7f4e800b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e8006bd60 0x7f4e8014f760 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 --2- 172.21.15.5:0/2688304193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e800cdc90 0x7f4e800b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 >> 172.21.15.5:0/2688304193 conn(0x7f4e8008afe0 msgr2=0x7f4e8008b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 shutdown_connections 2024-05-08T23:27:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:43.865+0000 7f4e85bc3700 1 -- 172.21.15.5:0/2688304193 wait complete. 2024-05-08T23:27:43.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.021+0000 7f0e82950700 1 Processor -- start 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.021+0000 7f0e82950700 1 -- start start 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e82950700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c0cbf50 0x7f0e7c161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e82950700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0cf970 con 0x7f0e7c15e760 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e82950700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0cfc50 con 0x7f0e7c162240 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e82950700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0cff30 con 0x7f0e7c0cbf50 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e80eed700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0e7c15e760 0x7f0e7c06a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58964/0 (socket says 172.21.15.5:58964) 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e80eed700 1 -- 172.21.15.5:0/3433107421 learned_addr learned my addr 172.21.15.5:0/3433107421 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7bfff700 1 --2- 172.21.15.5:0/3433107421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c0cbf50 0x7f0e7c161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7affd700 1 -- 172.21.15.5:0/3433107421 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141656479 0 0) 0x7f0e7c0cf970 con 0x7f0e7c15e760 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7affd700 1 -- 172.21.15.5:0/3433107421 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e640035b0 con 0x7f0e7c15e760 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7affd700 1 -- 172.21.15.5:0/3433107421 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311123380 0 0) 0x7f0e7c0cfc50 con 0x7f0e7c162240 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7affd700 1 -- 172.21.15.5:0/3433107421 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e7c0cf970 con 0x7f0e7c162240 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7affd700 1 -- 172.21.15.5:0/3433107421 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1880259594 0 0) 0x7f0e640035b0 con 0x7f0e7c15e760 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7bfff700 1 -- 172.21.15.5:0/3433107421 >> v1:172.21.15.49:6789/0 conn(0x7f0e7c162240 legacy=0x7f0e7c15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7bfff700 1 -- 172.21.15.5:0/3433107421 >> v1:172.21.15.5:6789/0 conn(0x7f0e7c15e760 legacy=0x7f0e7c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.022+0000 7f0e7bfff700 1 -- 172.21.15.5:0/3433107421 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c1617e0 con 0x7f0e7c0cbf50 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e7bfff700 1 --2- 172.21.15.5:0/3433107421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c0cbf50 0x7f0e7c161d00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009870 tx=0x7f0e6c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a290fb0b2e61e61f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e7affd700 1 -- 172.21.15.5:0/3433107421 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e740019f0 con 0x7f0e7c0cbf50 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e7affd700 1 -- 172.21.15.5:0/3433107421 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c004030 con 0x7f0e7c0cbf50 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e7affd700 1 -- 172.21.15.5:0/3433107421 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0043c0 con 0x7f0e7c0cbf50 2024-05-08T23:27:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e82950700 1 -- 172.21.15.5:0/3433107421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c0cbf50 msgr2=0x7f0e7c161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e82950700 1 --2- 172.21.15.5:0/3433107421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c0cbf50 0x7f0e7c161d00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009870 tx=0x7f0e6c009b80 comp rx=0 tx=0).stop 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e82950700 1 -- 172.21.15.5:0/3433107421 shutdown_connections 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e82950700 1 --2- 172.21.15.5:0/3433107421 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c0cbf50 0x7f0e7c161d00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.023+0000 7f0e82950700 1 -- 172.21.15.5:0/3433107421 >> 172.21.15.5:0/3433107421 conn(0x7f0e7c099a50 msgr2=0x7f0e7c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.024+0000 7f0e82950700 1 -- 172.21.15.5:0/3433107421 shutdown_connections 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.024+0000 7f0e82950700 1 -- 172.21.15.5:0/3433107421 wait complete. 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.025+0000 7f0e82950700 1 Processor -- start 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.025+0000 7f0e82950700 1 -- start start 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e82950700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0cbf50 0x7f0e7c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e82950700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c162240 0x7f0e7c0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e82950700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e7c0b3430 0x7f0e7c159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e82950700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0b3940 con 0x7f0e7c0b3430 2024-05-08T23:27:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e82950700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0b3c20 con 0x7f0e7c0cbf50 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e82950700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0b3f00 con 0x7f0e7c162240 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e80eed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e7c0b3430 0x7f0e7c159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e80eed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e7c0b3430 0x7f0e7c159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57424/0 (socket says 172.21.15.5:57424) 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e80eed700 1 -- 172.21.15.5:0/632043185 learned_addr learned my addr 172.21.15.5:0/632043185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e7b7fe700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c162240 0x7f0e7c0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e7bfff700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0cbf50 0x7f0e7c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e80eed700 1 -- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c162240 msgr2=0x7f0e7c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e80eed700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c162240 0x7f0e7c0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e80eed700 1 -- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0cbf50 msgr2=0x7f0e7c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e80eed700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0cbf50 0x7f0e7c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.026+0000 7f0e80eed700 1 -- 172.21.15.5:0/632043185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c0094a0 con 0x7f0e7c0b3430 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.027+0000 7f0e80eed700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e7c0b3430 0x7f0e7c159580 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f0e74001560 tx=0x7f0e7400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.027+0000 7f0e78ff9700 1 -- 172.21.15.5:0/632043185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400aa80 con 0x7f0e7c0b3430 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.027+0000 7f0e78ff9700 1 -- 172.21.15.5:0/632043185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7400ac00 con 0x7f0e7c0b3430 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.027+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c0b41e0 con 0x7f0e7c0b3430 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.027+0000 7f0e78ff9700 1 -- 172.21.15.5:0/632043185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e740070d0 con 0x7f0e7c0b3430 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.027+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c0b46f0 con 0x7f0e7c0b3430 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.028+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c0aff60 con 0x7f0e7c0b3430 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.028+0000 7f0e78ff9700 1 -- 172.21.15.5:0/632043185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e74007230 con 0x7f0e7c0b3430 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.029+0000 7f0e78ff9700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e6806ac90 0x7f0e6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.029+0000 7f0e78ff9700 1 -- 172.21.15.5:0/632043185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e74089130 con 0x7f0e7c0b3430 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.029+0000 7f0e7bfff700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e6806ac90 0x7f0e6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.030+0000 7f0e7bfff700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e6806ac90 0x7f0e6806d150 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0097c0 tx=0x7f0e6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.034+0000 7f0e78ff9700 1 -- 172.21.15.5:0/632043185 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0e7c0aff60 con 0x7f0e7c0b3430 2024-05-08T23:27:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.256+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e7c0aff60 con 0x7f0e7c0b3430 2024-05-08T23:27:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.256+0000 7f0e78ff9700 1 -- 172.21.15.5:0/632043185 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e7c0aff60 con 0x7f0e7c0b3430 2024-05-08T23:27:44.256 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.258+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e6806ac90 msgr2=0x7f0e6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.258+0000 7f0e82950700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e6806ac90 0x7f0e6806d150 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0097c0 tx=0x7f0e6c008030 comp rx=0 tx=0).stop 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.258+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e7c0b3430 msgr2=0x7f0e7c159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.258+0000 7f0e82950700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e7c0b3430 0x7f0e7c159580 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f0e74001560 tx=0x7f0e7400cf50 comp rx=0 tx=0).stop 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.258+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 shutdown_connections 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.258+0000 7f0e82950700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0e6806ac90 0x7f0e6806d150 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.258+0000 7f0e82950700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e7c0b3430 0x7f0e7c159580 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.258+0000 7f0e82950700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0cbf50 0x7f0e7c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.259+0000 7f0e82950700 1 --2- 172.21.15.5:0/632043185 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0e7c162240 0x7f0e7c0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.259+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 >> 172.21.15.5:0/632043185 conn(0x7f0e7c099a50 msgr2=0x7f0e7c0737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.259+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 shutdown_connections 2024-05-08T23:27:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.259+0000 7f0e82950700 1 -- 172.21.15.5:0/632043185 wait complete. 2024-05-08T23:27:44.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.409+0000 7fd221330700 1 Processor -- start 2024-05-08T23:27:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.409+0000 7fd221330700 1 -- start start 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd221330700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd21c15d9f0 0x7fd21c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd221330700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c0af4a0 0x7fd21c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd221330700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21c0cfb50 con 0x7fd21c06aea0 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd221330700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21c06b330 con 0x7fd21c15d9f0 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd221330700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0cf560 con 0x7fd21c0af4a0 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd21b7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd21c06aea0 0x7fd21c06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58974/0 (socket says 172.21.15.5:58974) 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd21b7fe700 1 -- 172.21.15.5:0/122269340 learned_addr learned my addr 172.21.15.5:0/122269340 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd21a7fc700 1 --2- 172.21.15.5:0/122269340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c0af4a0 0x7fd21c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd21affd700 1 --2- 172.21.15.5:0/122269340 >> v2:172.21.15.49:3300/0 conn(0x7fd21c15d9f0 0x7fd21c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd219ffb700 1 -- 172.21.15.5:0/122269340 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806199113 0 0) 0x7fd21c0cfb50 con 0x7fd21c06aea0 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.410+0000 7fd219ffb700 1 -- 172.21.15.5:0/122269340 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2040035b0 con 0x7fd21c06aea0 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd21a7fc700 1 -- 172.21.15.5:0/122269340 >> v2:172.21.15.49:3300/0 conn(0x7fd21c15d9f0 msgr2=0x7fd21c0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd21a7fc700 1 --2- 172.21.15.5:0/122269340 >> v2:172.21.15.49:3300/0 conn(0x7fd21c15d9f0 0x7fd21c0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd21a7fc700 1 -- 172.21.15.5:0/122269340 >> v1:172.21.15.5:6789/0 conn(0x7fd21c06aea0 legacy=0x7fd21c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd21a7fc700 1 -- 172.21.15.5:0/122269340 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21c15ddd0 con 0x7fd21c0af4a0 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd21affd700 1 --2- 172.21.15.5:0/122269340 >> v2:172.21.15.49:3300/0 conn(0x7fd21c15d9f0 0x7fd21c0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd21a7fc700 1 --2- 172.21.15.5:0/122269340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c0af4a0 0x7fd21c06a960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd210009840 tx=0x7fd210009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571ffd0f0dbf334d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd219ffb700 1 -- 172.21.15.5:0/122269340 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210011830 con 0x7fd21c0af4a0 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd219ffb700 1 -- 172.21.15.5:0/122269340 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd210011990 con 0x7fd21c0af4a0 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.411+0000 7fd219ffb700 1 -- 172.21.15.5:0/122269340 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210011d20 con 0x7fd21c0af4a0 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.412+0000 7fd221330700 1 -- 172.21.15.5:0/122269340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c0af4a0 msgr2=0x7fd21c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.412+0000 7fd221330700 1 --2- 172.21.15.5:0/122269340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c0af4a0 0x7fd21c06a960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd210009840 tx=0x7fd210009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.412+0000 7fd221330700 1 -- 172.21.15.5:0/122269340 shutdown_connections 2024-05-08T23:27:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.412+0000 7fd221330700 1 --2- 172.21.15.5:0/122269340 >> v2:172.21.15.49:3300/0 conn(0x7fd21c15d9f0 0x7fd21c0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.412+0000 7fd221330700 1 --2- 172.21.15.5:0/122269340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c0af4a0 0x7fd21c06a960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.412+0000 7fd221330700 1 -- 172.21.15.5:0/122269340 >> 172.21.15.5:0/122269340 conn(0x7fd21c0d02f0 msgr2=0x7fd21c0d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.412+0000 7fd221330700 1 -- 172.21.15.5:0/122269340 shutdown_connections 2024-05-08T23:27:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.412+0000 7fd221330700 1 -- 172.21.15.5:0/122269340 wait complete. 2024-05-08T23:27:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.413+0000 7fd221330700 1 Processor -- start 2024-05-08T23:27:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.413+0000 7fd221330700 1 -- start start 2024-05-08T23:27:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd221330700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd21c06aea0 0x7fd21c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd221330700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd21c0af4a0 0x7fd21c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd221330700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c15d9f0 0x7fd21c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd221330700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c158b80 con 0x7fd21c0af4a0 2024-05-08T23:27:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd221330700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c157fb0 con 0x7fd21c06aea0 2024-05-08T23:27:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd221330700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c158250 con 0x7fd21c15d9f0 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd21c0af4a0 0x7fd21c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd21c0af4a0 0x7fd21c15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57434/0 (socket says 172.21.15.5:57434) 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21a7fc700 1 -- 172.21.15.5:0/1146892462 learned_addr learned my addr 172.21.15.5:0/1146892462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21b7fe700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c15d9f0 0x7fd21c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21affd700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd21c06aea0 0x7fd21c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21a7fc700 1 -- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c15d9f0 msgr2=0x7fd21c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21a7fc700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c15d9f0 0x7fd21c15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21a7fc700 1 -- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd21c06aea0 msgr2=0x7fd21c155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21a7fc700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd21c06aea0 0x7fd21c155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.414+0000 7fd21a7fc700 1 -- 172.21.15.5:0/1146892462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c0094a0 con 0x7fd21c0af4a0 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.415+0000 7fd21a7fc700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd21c0af4a0 0x7fd21c15c360 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd2100090b0 tx=0x7fd210004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.415+0000 7fd2037fe700 1 -- 172.21.15.5:0/1146892462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100036a0 con 0x7fd21c0af4a0 2024-05-08T23:27:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.415+0000 7fd2037fe700 1 -- 172.21.15.5:0/1146892462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21001e070 con 0x7fd21c0af4a0 2024-05-08T23:27:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.415+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2100094a0 con 0x7fd21c0af4a0 2024-05-08T23:27:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.415+0000 7fd2037fe700 1 -- 172.21.15.5:0/1146892462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210003a40 con 0x7fd21c0af4a0 2024-05-08T23:27:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.415+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c158810 con 0x7fd21c0af4a0 2024-05-08T23:27:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.416+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c158d30 con 0x7fd21c0af4a0 2024-05-08T23:27:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.416+0000 7fd2037fe700 1 -- 172.21.15.5:0/1146892462 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd210003ba0 con 0x7fd21c0af4a0 2024-05-08T23:27:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.417+0000 7fd2037fe700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd20806ac90 0x7fd20806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.417+0000 7fd2037fe700 1 -- 172.21.15.5:0/1146892462 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd21008a110 con 0x7fd21c0af4a0 2024-05-08T23:27:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.417+0000 7fd21affd700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd20806ac90 0x7fd20806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.418+0000 7fd21affd700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd20806ac90 0x7fd20806d150 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0cc1c0 tx=0x7fd20c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.422+0000 7fd2037fe700 1 -- 172.21.15.5:0/1146892462 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd21c158d30 con 0x7fd21c0af4a0 2024-05-08T23:27:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.686+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd21c0678f0 con 0x7fd21c0af4a0 2024-05-08T23:27:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.687+0000 7fd2037fe700 1 -- 172.21.15.5:0/1146892462 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7fd210055300 con 0x7fd21c0af4a0 2024-05-08T23:27:44.687 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:44.687 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd20806ac90 msgr2=0x7fd20806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd20806ac90 0x7fd20806d150 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0cc1c0 tx=0x7fd20c009210 comp rx=0 tx=0).stop 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd21c0af4a0 msgr2=0x7fd21c15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd21c0af4a0 0x7fd21c15c360 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd2100090b0 tx=0x7fd210004750 comp rx=0 tx=0).stop 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 shutdown_connections 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd20806ac90 0x7fd20806d150 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd21c15d9f0 0x7fd21c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd21c06aea0 0x7fd21c155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 --2- 172.21.15.5:0/1146892462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd21c0af4a0 0x7fd21c15c360 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 >> 172.21.15.5:0/1146892462 conn(0x7fd21c0d02f0 msgr2=0x7fd21c0cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 shutdown_connections 2024-05-08T23:27:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.689+0000 7fd221330700 1 -- 172.21.15.5:0/1146892462 wait complete. 2024-05-08T23:27:44.691 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:44.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:27:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.839+0000 7f2f0e4e6700 1 Processor -- start 2024-05-08T23:27:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.839+0000 7f2f0e4e6700 1 -- start start 2024-05-08T23:27:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.840+0000 7f2f0e4e6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2f0815ef60 0x7f2f080b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.840+0000 7f2f0e4e6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f080cf000 0x7f2f080cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.840+0000 7f2f0e4e6700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f080cfe10 con 0x7f2f080cf9a0 2024-05-08T23:27:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.840+0000 7f2f0e4e6700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080cd400 con 0x7f2f080cf000 2024-05-08T23:27:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.840+0000 7f2f0e4e6700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0809ad40 con 0x7f2f0815ef60 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.840+0000 7f2f077fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f080cf000 0x7f2f080cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.840+0000 7f2f0ca83700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2f080cf9a0 0x7f2f08099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58988/0 (socket says 172.21.15.5:58988) 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f0ca83700 1 -- 172.21.15.5:0/3810868939 learned_addr learned my addr 172.21.15.5:0/3810868939 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f07fff700 1 --2- 172.21.15.5:0/3810868939 >> v2:172.21.15.49:3300/0 conn(0x7f2f0815ef60 0x7f2f080b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f077fe700 1 -- 172.21.15.5:0/3810868939 >> v2:172.21.15.49:3300/0 conn(0x7f2f0815ef60 msgr2=0x7f2f080b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f077fe700 1 --2- 172.21.15.5:0/3810868939 >> v2:172.21.15.49:3300/0 conn(0x7f2f0815ef60 0x7f2f080b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f077fe700 1 -- 172.21.15.5:0/3810868939 >> v1:172.21.15.5:6789/0 conn(0x7f2f080cf9a0 legacy=0x7f2f08099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f077fe700 1 -- 172.21.15.5:0/3810868939 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080ccb60 con 0x7f2f080cf000 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f06ffd700 1 -- 172.21.15.5:0/3810868939 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279590893 0 0) 0x7f2f080cfe10 con 0x7f2f080cf9a0 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f077fe700 1 --2- 172.21.15.5:0/3810868939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f080cf000 0x7f2f080cf460 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2f0806e820 tx=0x7f2efc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f073ccacc6cd39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f06ffd700 1 -- 172.21.15.5:0/3810868939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc011830 con 0x7f2f080cf000 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f06ffd700 1 -- 172.21.15.5:0/3810868939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc011990 con 0x7f2f080cf000 2024-05-08T23:27:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.841+0000 7f2f06ffd700 1 -- 172.21.15.5:0/3810868939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc011d20 con 0x7f2f080cf000 2024-05-08T23:27:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.842+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3810868939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f080cf000 msgr2=0x7f2f080cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.842+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3810868939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f080cf000 0x7f2f080cf460 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2f0806e820 tx=0x7f2efc009b20 comp rx=0 tx=0).stop 2024-05-08T23:27:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.842+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3810868939 shutdown_connections 2024-05-08T23:27:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.842+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3810868939 >> v2:172.21.15.49:3300/0 conn(0x7f2f0815ef60 0x7f2f080b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.842+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3810868939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f080cf000 0x7f2f080cf460 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.842+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3810868939 >> 172.21.15.5:0/3810868939 conn(0x7f2f080b2f20 msgr2=0x7f2f080b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.842+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3810868939 shutdown_connections 2024-05-08T23:27:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.842+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3810868939 wait complete. 2024-05-08T23:27:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.843+0000 7f2f0e4e6700 1 Processor -- start 2024-05-08T23:27:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.844+0000 7f2f0e4e6700 1 -- start start 2024-05-08T23:27:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.844+0000 7f2f0e4e6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f080cf000 0x7f2f081587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.844+0000 7f2f0e4e6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f080cf9a0 0x7f2f0809b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.844+0000 7f2f0e4e6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0815ef60 0x7f2f080b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.844+0000 7f2f0e4e6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0815ab90 con 0x7f2f0815ef60 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.844+0000 7f2f0e4e6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08158d00 con 0x7f2f080cf9a0 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.844+0000 7f2f0e4e6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0809b7a0 con 0x7f2f080cf000 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.844+0000 7f2f0ca83700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0815ef60 0x7f2f080b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0ca83700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0815ef60 0x7f2f080b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57446/0 (socket says 172.21.15.5:57446) 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0ca83700 1 -- 172.21.15.5:0/3164785615 learned_addr learned my addr 172.21.15.5:0/3164785615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f077fe700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f080cf9a0 0x7f2f0809b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f07fff700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f080cf000 0x7f2f081587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0ca83700 1 -- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f080cf000 msgr2=0x7f2f081587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0ca83700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f080cf000 0x7f2f081587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0ca83700 1 -- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f080cf9a0 msgr2=0x7f2f0809b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0ca83700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f080cf9a0 0x7f2f0809b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0ca83700 1 -- 172.21.15.5:0/3164785615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc0094a0 con 0x7f2f0815ef60 2024-05-08T23:27:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0ca83700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0815ef60 0x7f2f080b5ab0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000cb60 tx=0x7f2f0000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f04ff9700 1 -- 172.21.15.5:0/3164785615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000ab50 con 0x7f2f0815ef60 2024-05-08T23:27:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f04ff9700 1 -- 172.21.15.5:0/3164785615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000acb0 con 0x7f2f0815ef60 2024-05-08T23:27:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0809ba40 con 0x7f2f0815ef60 2024-05-08T23:27:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f04ff9700 1 -- 172.21.15.5:0/3164785615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00007310 con 0x7f2f0815ef60 2024-05-08T23:27:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.845+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0809c000 con 0x7f2f0815ef60 2024-05-08T23:27:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.846+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f08068000 con 0x7f2f0815ef60 2024-05-08T23:27:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.847+0000 7f2f04ff9700 1 -- 172.21.15.5:0/3164785615 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2f00007470 con 0x7f2f0815ef60 2024-05-08T23:27:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.847+0000 7f2f04ff9700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef406ac90 0x7f2ef406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.847+0000 7f2f04ff9700 1 -- 172.21.15.5:0/3164785615 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f00088d80 con 0x7f2f0815ef60 2024-05-08T23:27:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.848+0000 7f2f07fff700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef406ac90 0x7f2ef406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.848+0000 7f2f07fff700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef406ac90 0x7f2ef406d150 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009b70 tx=0x7f2ef8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:44.852+0000 7f2f04ff9700 1 -- 172.21.15.5:0/3164785615 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2f00053ff0 con 0x7f2f0815ef60 2024-05-08T23:27:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.072+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f080b4230 con 0x7f2f0815ef60 2024-05-08T23:27:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.074+0000 7f2f04ff9700 1 -- 172.21.15.5:0/3164785615 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15996 (secure 0 0 0) 0x7f2f00018020 con 0x7f2f0815ef60 2024-05-08T23:27:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:45.074 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:57.193438+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:54.037450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:55.162938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef406ac90 msgr2=0x7f2ef406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef406ac90 0x7f2ef406d150 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009b70 tx=0x7f2ef8008040 comp rx=0 tx=0).stop 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0815ef60 msgr2=0x7f2f080b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0815ef60 0x7f2f080b5ab0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000cb60 tx=0x7f2f0000cf20 comp rx=0 tx=0).stop 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 shutdown_connections 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2ef406ac90 0x7f2ef406d150 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f0815ef60 0x7f2f080b5ab0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2f080cf000 0x7f2f081587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 --2- 172.21.15.5:0/3164785615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f080cf9a0 0x7f2f0809b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 >> 172.21.15.5:0/3164785615 conn(0x7f2f080b2f20 msgr2=0x7f2f0806ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 shutdown_connections 2024-05-08T23:27:45.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.076+0000 7f2f0e4e6700 1 -- 172.21.15.5:0/3164785615 wait complete. 2024-05-08T23:27:45.090 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:27:45.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:45.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.225+0000 7f0d00447700 1 Processor -- start 2024-05-08T23:27:45.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.226+0000 7f0d00447700 1 -- start start 2024-05-08T23:27:45.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.226+0000 7f0d00447700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf80d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0d00447700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0cf80af860 0x7f0cf80ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0d00447700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf80cfbd0 con 0x7f0cf80c5720 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0d00447700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf80cf440 con 0x7f0cf80af860 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0d00447700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80cb780 con 0x7f0cf815a750 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfd9e2700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0cf80af860 0x7f0cf80ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfe9e4700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0cf80c5720 0x7f0cf80cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58998/0 (socket says 172.21.15.5:58998) 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfe9e4700 1 -- 172.21.15.5:0/1517965046 learned_addr learned my addr 172.21.15.5:0/1517965046 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfe1e3700 1 --2- 172.21.15.5:0/1517965046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf80d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfd1e1700 1 -- 172.21.15.5:0/1517965046 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395720822 0 0) 0x7f0cf80cfbd0 con 0x7f0cf80c5720 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfd1e1700 1 -- 172.21.15.5:0/1517965046 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce40035b0 con 0x7f0cf80c5720 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfd9e2700 1 -- 172.21.15.5:0/1517965046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 msgr2=0x7f0cf80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:45.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfd9e2700 1 --2- 172.21.15.5:0/1517965046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf80d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfd9e2700 1 -- 172.21.15.5:0/1517965046 >> v1:172.21.15.5:6789/0 conn(0x7f0cf80c5720 legacy=0x7f0cf80cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfd9e2700 1 -- 172.21.15.5:0/1517965046 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf80cf650 con 0x7f0cf80af860 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfe1e3700 1 --2- 172.21.15.5:0/1517965046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf80d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.227+0000 7f0cfd9e2700 1 --2- 172.21.15.5:0/1517965046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf80ad260 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009840 tx=0x7f0ce8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2aa049de58e505 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0cfd1e1700 1 -- 172.21.15.5:0/1517965046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8011830 con 0x7f0cf80af860 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0cfd1e1700 1 -- 172.21.15.5:0/1517965046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce8011990 con 0x7f0cf80af860 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0cfd1e1700 1 -- 172.21.15.5:0/1517965046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8011d20 con 0x7f0cf80af860 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0d00447700 1 -- 172.21.15.5:0/1517965046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 msgr2=0x7f0cf80ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0d00447700 1 --2- 172.21.15.5:0/1517965046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf80ad260 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009840 tx=0x7f0ce8009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0d00447700 1 -- 172.21.15.5:0/1517965046 shutdown_connections 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0d00447700 1 --2- 172.21.15.5:0/1517965046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf80d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0d00447700 1 --2- 172.21.15.5:0/1517965046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf80ad260 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0d00447700 1 -- 172.21.15.5:0/1517965046 >> 172.21.15.5:0/1517965046 conn(0x7f0cf80d03a0 msgr2=0x7f0cf80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0d00447700 1 -- 172.21.15.5:0/1517965046 shutdown_connections 2024-05-08T23:27:45.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.228+0000 7f0d00447700 1 -- 172.21.15.5:0/1517965046 wait complete. 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.230+0000 7f0d00447700 1 Processor -- start 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.230+0000 7f0d00447700 1 -- start start 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0d00447700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf814dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0d00447700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cf80c5720 0x7f0cf81592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0d00447700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf8159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0d00447700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80c9c30 con 0x7f0cf80af860 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0d00447700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80afdb0 con 0x7f0cf80c5720 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0d00447700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80b0090 con 0x7f0cf815a750 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf814dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf814dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57478/0 (socket says 172.21.15.5:57478) 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 -- 172.21.15.5:0/116281548 learned_addr learned my addr 172.21.15.5:0/116281548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 -- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 msgr2=0x7f0cf8159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfd9e2700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cf80c5720 0x7f0cf81592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe9e4700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf8159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf8159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 -- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cf80c5720 msgr2=0x7f0cf81592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cf80c5720 0x7f0cf81592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 -- 172.21.15.5:0/116281548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80094a0 con 0x7f0cf80af860 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0cfe1e3700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf814dba0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400e810 tx=0x7f0cf400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0ceeffd700 1 -- 172.21.15.5:0/116281548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4013610 con 0x7f0cf80af860 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0ceeffd700 1 -- 172.21.15.5:0/116281548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf4013770 con 0x7f0cf80af860 2024-05-08T23:27:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0ceeffd700 1 -- 172.21.15.5:0/116281548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4013b20 con 0x7f0cf80af860 2024-05-08T23:27:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.231+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf80b0370 con 0x7f0cf80af860 2024-05-08T23:27:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.232+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf814e0e0 con 0x7f0cf80af860 2024-05-08T23:27:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.233+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf8068000 con 0x7f0cf80af860 2024-05-08T23:27:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.233+0000 7f0ceeffd700 1 -- 172.21.15.5:0/116281548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0cf4013c80 con 0x7f0cf80af860 2024-05-08T23:27:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.234+0000 7f0ceeffd700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0cdc06ac90 0x7f0cdc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.234+0000 7f0ceeffd700 1 -- 172.21.15.5:0/116281548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cf4088fd0 con 0x7f0cf80af860 2024-05-08T23:27:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.234+0000 7f0cfd9e2700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0cdc06ac90 0x7f0cdc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.234+0000 7f0cfd9e2700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0cdc06ac90 0x7f0cdc06d150 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0cf814f990 tx=0x7f0ce8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.239+0000 7f0ceeffd700 1 -- 172.21.15.5:0/116281548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0cf40541b0 con 0x7f0cf80af860 2024-05-08T23:27:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.520+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cf8071ed0 con 0x7f0cf80af860 2024-05-08T23:27:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.521+0000 7f0ceeffd700 1 -- 172.21.15.5:0/116281548 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7f0cf40577d0 con 0x7f0cf80af860 2024-05-08T23:27:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:45.521 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0cdc06ac90 msgr2=0x7f0cdc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0cdc06ac90 0x7f0cdc06d150 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0cf814f990 tx=0x7f0ce8008030 comp rx=0 tx=0).stop 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 msgr2=0x7f0cf814dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf814dba0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400e810 tx=0x7f0cf400ebd0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 shutdown_connections 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0cdc06ac90 0x7f0cdc06d150 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cf815a750 0x7f0cf8159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf80af860 0x7f0cf814dba0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 --2- 172.21.15.5:0/116281548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cf80c5720 0x7f0cf81592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 >> 172.21.15.5:0/116281548 conn(0x7f0cf80d03a0 msgr2=0x7f0cf806b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 shutdown_connections 2024-05-08T23:27:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.523+0000 7f0d00447700 1 -- 172.21.15.5:0/116281548 wait complete. 2024-05-08T23:27:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:45.537 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:45.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.672+0000 7ff077c1d700 1 Processor -- start 2024-05-08T23:27:45.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.672+0000 7ff077c1d700 1 -- start start 2024-05-08T23:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.673+0000 7ff077c1d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff07015deb0 0x7ff070164b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.673+0000 7ff077c1d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07006d910 con 0x7ff0700cf720 2024-05-08T23:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.673+0000 7ff077c1d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07006ba30 con 0x7ff07015e420 2024-05-08T23:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.673+0000 7ff077c1d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07006d5e0 con 0x7ff07015deb0 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.673+0000 7ff0761ba700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff07015deb0 0x7ff070164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.673+0000 7ff0759b9700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff0700cf720 0x7ff07006b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59004/0 (socket says 172.21.15.5:59004) 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0759b9700 1 -- 172.21.15.5:0/71924566 learned_addr learned my addr 172.21.15.5:0/71924566 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0761ba700 1 -- 172.21.15.5:0/71924566 >> v1:172.21.15.49:6789/0 conn(0x7ff07015e420 legacy=0x7ff07015d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0761ba700 1 -- 172.21.15.5:0/71924566 >> v1:172.21.15.5:6789/0 conn(0x7ff0700cf720 legacy=0x7ff07006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0761ba700 1 -- 172.21.15.5:0/71924566 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07006bc80 con 0x7ff07015deb0 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0749b7700 1 -- 172.21.15.5:0/71924566 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234828816 0 0) 0x7ff07006d910 con 0x7ff0700cf720 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0761ba700 1 --2- 172.21.15.5:0/71924566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015deb0 0x7ff070164b30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff068009d90 tx=0x7ff06800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b281089d20b798ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0749b7700 1 -- 172.21.15.5:0/71924566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06800e070 con 0x7ff07015deb0 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0749b7700 1 -- 172.21.15.5:0/71924566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06800bbd0 con 0x7ff07015deb0 2024-05-08T23:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.674+0000 7ff0749b7700 1 -- 172.21.15.5:0/71924566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068012430 con 0x7ff07015deb0 2024-05-08T23:27:45.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.675+0000 7ff077c1d700 1 -- 172.21.15.5:0/71924566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015deb0 msgr2=0x7ff070164b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.675+0000 7ff077c1d700 1 --2- 172.21.15.5:0/71924566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015deb0 0x7ff070164b30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff068009d90 tx=0x7ff06800b330 comp rx=0 tx=0).stop 2024-05-08T23:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.675+0000 7ff077c1d700 1 -- 172.21.15.5:0/71924566 shutdown_connections 2024-05-08T23:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.675+0000 7ff077c1d700 1 --2- 172.21.15.5:0/71924566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015deb0 0x7ff070164b30 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.675+0000 7ff077c1d700 1 -- 172.21.15.5:0/71924566 >> 172.21.15.5:0/71924566 conn(0x7ff0700cfce0 msgr2=0x7ff0700d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.675+0000 7ff077c1d700 1 -- 172.21.15.5:0/71924566 shutdown_connections 2024-05-08T23:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.675+0000 7ff077c1d700 1 -- 172.21.15.5:0/71924566 wait complete. 2024-05-08T23:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.677+0000 7ff077c1d700 1 Processor -- start 2024-05-08T23:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.677+0000 7ff077c1d700 1 -- start start 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff077c1d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0700cf720 0x7ff0701536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff077c1d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff07015deb0 0x7ff07014d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff077c1d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015e420 0x7ff07014d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff077c1d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff070156350 con 0x7ff07015e420 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff077c1d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0701556d0 con 0x7ff07015deb0 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff077c1d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0701559b0 con 0x7ff0700cf720 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015e420 0x7ff07014d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015e420 0x7ff07014d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57502/0 (socket says 172.21.15.5:57502) 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 -- 172.21.15.5:0/2861599270 learned_addr learned my addr 172.21.15.5:0/2861599270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0759b9700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0700cf720 0x7ff0701536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0751b8700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff07015deb0 0x7ff07014d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 -- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0700cf720 msgr2=0x7ff0701536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0700cf720 0x7ff0701536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 -- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff07015deb0 msgr2=0x7ff07014d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff07015deb0 0x7ff07014d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 -- 172.21.15.5:0/2861599270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff068009a70 con 0x7ff07015e420 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0751b8700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff07015deb0 0x7ff07014d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0759b9700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0700cf720 0x7ff0701536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.678+0000 7ff0761ba700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015e420 0x7ff07014d730 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff068014040 tx=0x7ff068004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.679+0000 7ff0627fc700 1 -- 172.21.15.5:0/2861599270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06801b810 con 0x7ff07015e420 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.679+0000 7ff0627fc700 1 -- 172.21.15.5:0/2861599270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068020070 con 0x7ff07015e420 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.679+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff070155c90 con 0x7ff07015e420 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.679+0000 7ff0627fc700 1 -- 172.21.15.5:0/2861599270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06801bbf0 con 0x7ff07015e420 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.679+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0700a84b0 con 0x7ff07015e420 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.680+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff070156490 con 0x7ff07015e420 2024-05-08T23:27:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.680+0000 7ff0627fc700 1 -- 172.21.15.5:0/2861599270 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff06801bd50 con 0x7ff07015e420 2024-05-08T23:27:45.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.680+0000 7ff0627fc700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff05406ac90 0x7ff05406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:45.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.681+0000 7ff0627fc700 1 -- 172.21.15.5:0/2861599270 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff068022070 con 0x7ff07015e420 2024-05-08T23:27:45.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.681+0000 7ff0759b9700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff05406ac90 0x7ff05406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:45.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.681+0000 7ff0759b9700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff05406ac90 0x7ff05406d150 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff064000f60 tx=0x7ff064009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:45.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.685+0000 7ff0627fc700 1 -- 172.21.15.5:0/2861599270 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff070156490 con 0x7ff07015e420 2024-05-08T23:27:45.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.958+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff070156490 con 0x7ff07015e420 2024-05-08T23:27:45.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.958+0000 7ff0627fc700 1 -- 172.21.15.5:0/2861599270 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4767 (secure 0 0 0) 0x7ff070156490 con 0x7ff07015e420 2024-05-08T23:27:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:45.959 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:26:56.191956+0000","modified":"2024-05-08T23:27:29.289507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7758,"mds_1":7755,"mds_2":7336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7336":{"gid":7336,"name":"b","rank":2,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.49:6837/1134209849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1134209849},{"type":"v1","addr":"172.21.15.49:6837","nonce":1134209849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7755":{"gid":7755,"name":"c","rank":1,"incarnation":84,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/3718289875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3718289875},{"type":"v1","addr":"172.21.15.5:6834","nonce":3718289875}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7758":{"gid":7758,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1093540582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1093540582},{"type":"v1","addr":"172.21.15.5:6835","nonce":1093540582}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T23:27:45.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff05406ac90 msgr2=0x7ff05406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff05406ac90 0x7ff05406d150 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff064000f60 tx=0x7ff064009040 comp rx=0 tx=0).stop 2024-05-08T23:27:45.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015e420 msgr2=0x7ff07014d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:45.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015e420 0x7ff07014d730 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff068014040 tx=0x7ff068004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 shutdown_connections 2024-05-08T23:27:45.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff05406ac90 0x7ff05406d150 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07015e420 0x7ff07014d730 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff0700cf720 0x7ff0701536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 --2- 172.21.15.5:0/2861599270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff07015deb0 0x7ff07014d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:45.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 >> 172.21.15.5:0/2861599270 conn(0x7ff0700cfce0 msgr2=0x7ff0700af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:45.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 shutdown_connections 2024-05-08T23:27:45.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:45.961+0000 7ff077c1d700 1 -- 172.21.15.5:0/2861599270 wait complete. 2024-05-08T23:27:45.962 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-08T23:27:45.975 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:27:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.110+0000 7f0657e9f700 1 Processor -- start 2024-05-08T23:27:46.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.111+0000 7f0657e9f700 1 -- start start 2024-05-08T23:27:46.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.111+0000 7f0657e9f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06500cecf0 0x7f065006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.111+0000 7f0657e9f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f065006f130 0x7f065015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.111+0000 7f0657e9f700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065015c280 con 0x7f06500cd4e0 2024-05-08T23:27:46.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.111+0000 7f0657e9f700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065006e780 con 0x7f06500cecf0 2024-05-08T23:27:46.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.111+0000 7f0657e9f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065006ea60 con 0x7f065006f130 2024-05-08T23:27:46.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065543a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06500cecf0 0x7f065006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065543a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06500cecf0 0x7f065006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57508/0 (socket says 172.21.15.5:57508) 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065543a700 1 -- 172.21.15.5:0/200907086 learned_addr learned my addr 172.21.15.5:0/200907086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065643c700 1 --2- 172.21.15.5:0/200907086 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f065006f130 0x7f065015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065543a700 1 -- 172.21.15.5:0/200907086 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f065006f130 msgr2=0x7f065015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065543a700 1 --2- 172.21.15.5:0/200907086 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f065006f130 0x7f065015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065543a700 1 -- 172.21.15.5:0/200907086 >> v1:172.21.15.49:6789/0 conn(0x7f06500cd4e0 legacy=0x7f06500ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065543a700 1 -- 172.21.15.5:0/200907086 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065006ed40 con 0x7f06500cecf0 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f065543a700 1 --2- 172.21.15.5:0/200907086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06500cecf0 0x7f065006f910 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f06400098f0 tx=0x7f0640009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89db825cad2b5d30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f0654c39700 1 -- 172.21.15.5:0/200907086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640011830 con 0x7f06500cecf0 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f0654c39700 1 -- 172.21.15.5:0/200907086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0640011990 con 0x7f06500cecf0 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.112+0000 7f0654c39700 1 -- 172.21.15.5:0/200907086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640011d20 con 0x7f06500cecf0 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.113+0000 7f0657e9f700 1 -- 172.21.15.5:0/200907086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06500cecf0 msgr2=0x7f065006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.113+0000 7f0657e9f700 1 --2- 172.21.15.5:0/200907086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06500cecf0 0x7f065006f910 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f06400098f0 tx=0x7f0640009c00 comp rx=0 tx=0).stop 2024-05-08T23:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.113+0000 7f0657e9f700 1 -- 172.21.15.5:0/200907086 shutdown_connections 2024-05-08T23:27:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.113+0000 7f0657e9f700 1 --2- 172.21.15.5:0/200907086 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f065006f130 0x7f065015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.113+0000 7f0657e9f700 1 --2- 172.21.15.5:0/200907086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06500cecf0 0x7f065006f910 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.113+0000 7f0657e9f700 1 -- 172.21.15.5:0/200907086 >> 172.21.15.5:0/200907086 conn(0x7f06500d0320 msgr2=0x7f06500d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.113+0000 7f0657e9f700 1 -- 172.21.15.5:0/200907086 shutdown_connections 2024-05-08T23:27:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.113+0000 7f0657e9f700 1 -- 172.21.15.5:0/200907086 wait complete. 2024-05-08T23:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.114+0000 7f0657e9f700 1 Processor -- start 2024-05-08T23:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.114+0000 7f0657e9f700 1 -- start start 2024-05-08T23:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0657e9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065006f130 0x7f0650151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0657e9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06500cd4e0 0x7f06500c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0657e9f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06500cecf0 0x7f06500c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0657e9f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06500c4af0 con 0x7f065006f130 2024-05-08T23:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0657e9f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06500c4dd0 con 0x7f06500cd4e0 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0657e9f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06500c50b0 con 0x7f06500cecf0 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0655c3b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065006f130 0x7f0650151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0655c3b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065006f130 0x7f0650151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57524/0 (socket says 172.21.15.5:57524) 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.115+0000 7f0655c3b700 1 -- 172.21.15.5:0/1391241602 learned_addr learned my addr 172.21.15.5:0/1391241602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f065543a700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06500cd4e0 0x7f06500c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f065643c700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06500cecf0 0x7f06500c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f0655c3b700 1 -- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06500cecf0 msgr2=0x7f06500c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f0655c3b700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06500cecf0 0x7f06500c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f0655c3b700 1 -- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06500cd4e0 msgr2=0x7f06500c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f0655c3b700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06500cd4e0 0x7f06500c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f0655c3b700 1 -- 172.21.15.5:0/1391241602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0640009550 con 0x7f065006f130 2024-05-08T23:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f0655c3b700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065006f130 0x7f0650151e20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f064c001ed0 tx=0x7f064c0021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f06467fc700 1 -- 172.21.15.5:0/1391241602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c003c00 con 0x7f065006f130 2024-05-08T23:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f06467fc700 1 -- 172.21.15.5:0/1391241602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c003d60 con 0x7f065006f130 2024-05-08T23:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06500c5390 con 0x7f065006f130 2024-05-08T23:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f06467fc700 1 -- 172.21.15.5:0/1391241602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c00ab40 con 0x7f065006f130 2024-05-08T23:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.116+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0650152360 con 0x7f065006f130 2024-05-08T23:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.117+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0650068000 con 0x7f065006f130 2024-05-08T23:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.118+0000 7f06467fc700 1 -- 172.21.15.5:0/1391241602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f064c00aca0 con 0x7f065006f130 2024-05-08T23:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.118+0000 7f06467fc700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f063406ac90 0x7f063406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.118+0000 7f06467fc700 1 -- 172.21.15.5:0/1391241602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f064c089360 con 0x7f065006f130 2024-05-08T23:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.119+0000 7f065543a700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f063406ac90 0x7f063406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.119+0000 7f065543a700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f063406ac90 0x7f063406d150 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0640009c40 tx=0x7f0640008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.123+0000 7f06467fc700 1 -- 172.21.15.5:0/1391241602 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f064c054540 con 0x7f065006f130 2024-05-08T23:27:46.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.339+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f065006d440 con 0x7f065006f130 2024-05-08T23:27:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.341+0000 7f06467fc700 1 -- 172.21.15.5:0/1391241602 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15996 (secure 0 0 0) 0x7f064c057b60 con 0x7f065006f130 2024-05-08T23:27:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:46.341 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:26:57.193438+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:26:54.037450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T23:26:55.162938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f063406ac90 msgr2=0x7f063406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f063406ac90 0x7f063406d150 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0640009c40 tx=0x7f0640008030 comp rx=0 tx=0).stop 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065006f130 msgr2=0x7f0650151e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065006f130 0x7f0650151e20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f064c001ed0 tx=0x7f064c0021e0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 shutdown_connections 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f063406ac90 0x7f063406d150 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06500cecf0 0x7f06500c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065006f130 0x7f0650151e20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 --2- 172.21.15.5:0/1391241602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06500cd4e0 0x7f06500c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.343+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 >> 172.21.15.5:0/1391241602 conn(0x7f06500d0320 msgr2=0x7f06500d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.344+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 shutdown_connections 2024-05-08T23:27:46.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.344+0000 7f0657e9f700 1 -- 172.21.15.5:0/1391241602 wait complete. 2024-05-08T23:27:46.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:27:46.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.495+0000 7f80a2285700 1 Processor -- start 2024-05-08T23:27:46.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.495+0000 7f80a2285700 1 -- start start 2024-05-08T23:27:46.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.496+0000 7f80a2285700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c0c94c0 0x7f809c142960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.496+0000 7f80a2285700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f809c146a10 0x7f809c142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.496+0000 7f80a2285700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c161a80 con 0x7f809c0c90e0 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.496+0000 7f80a2285700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809c06a960 con 0x7f809c146a10 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.496+0000 7f80a2285700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c161d60 con 0x7f809c0c94c0 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f80a0822700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f809c146a10 0x7f809c142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f809b7fe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c0c94c0 0x7f809c142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f80a0822700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f809c146a10 0x7f809c142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34240/0 (socket says 172.21.15.5:34240) 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f80a0822700 1 -- 172.21.15.5:0/3136995218 learned_addr learned my addr 172.21.15.5:0/3136995218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f809bfff700 1 --1- 172.21.15.5:0/3136995218 >> v1:172.21.15.49:6789/0 conn(0x7f809c0c90e0 0x7f809c1612c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60294/0 (socket says 172.21.15.5:60294) 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f80a0822700 1 -- 172.21.15.5:0/3136995218 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c0c94c0 msgr2=0x7f809c142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:46.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f80a0822700 1 --2- 172.21.15.5:0/3136995218 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c0c94c0 0x7f809c142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f80a0822700 1 -- 172.21.15.5:0/3136995218 >> v1:172.21.15.49:6789/0 conn(0x7f809c0c90e0 legacy=0x7f809c1612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f80a0822700 1 -- 172.21.15.5:0/3136995218 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c162040 con 0x7f809c146a10 2024-05-08T23:27:46.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f80a0822700 1 --2- 172.21.15.5:0/3136995218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c146a10 0x7f809c142ea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8094009e50 tx=0x7f809400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=432f55e3a4a389c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.497+0000 7f809affd700 1 -- 172.21.15.5:0/3136995218 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400e070 con 0x7f809c146a10 2024-05-08T23:27:46.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f809affd700 1 -- 172.21.15.5:0/3136995218 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809400b910 con 0x7f809c146a10 2024-05-08T23:27:46.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f809affd700 1 -- 172.21.15.5:0/3136995218 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400bcc0 con 0x7f809c146a10 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f80a2285700 1 -- 172.21.15.5:0/3136995218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c146a10 msgr2=0x7f809c142ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f80a2285700 1 --2- 172.21.15.5:0/3136995218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c146a10 0x7f809c142ea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8094009e50 tx=0x7f809400b330 comp rx=0 tx=0).stop 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f80a2285700 1 -- 172.21.15.5:0/3136995218 shutdown_connections 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f80a2285700 1 --2- 172.21.15.5:0/3136995218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c146a10 0x7f809c142ea0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f80a2285700 1 --2- 172.21.15.5:0/3136995218 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c0c94c0 0x7f809c142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f80a2285700 1 -- 172.21.15.5:0/3136995218 >> 172.21.15.5:0/3136995218 conn(0x7f809c0cc580 msgr2=0x7f809c0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f80a2285700 1 -- 172.21.15.5:0/3136995218 shutdown_connections 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.498+0000 7f80a2285700 1 -- 172.21.15.5:0/3136995218 wait complete. 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.499+0000 7f80a2285700 1 Processor -- start 2024-05-08T23:27:46.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.500+0000 7f80a2285700 1 -- start start 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.500+0000 7f80a2285700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c0c90e0 0x7f809c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.500+0000 7f80a2285700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809c0c94c0 0x7f809c143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.500+0000 7f80a2285700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c146a10 0x7f809c13eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.500+0000 7f80a2285700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c15d4d0 con 0x7f809c0c94c0 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.500+0000 7f80a2285700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1444e0 con 0x7f809c0c90e0 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.500+0000 7f80a2285700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1447c0 con 0x7f809c146a10 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809c0c94c0 0x7f809c143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c0c90e0 0x7f809c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809c0c94c0 0x7f809c143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57540/0 (socket says 172.21.15.5:57540) 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 -- 172.21.15.5:0/235170843 learned_addr learned my addr 172.21.15.5:0/235170843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f80a0822700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c146a10 0x7f809c13eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 -- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c146a10 msgr2=0x7f809c13eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c146a10 0x7f809c13eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 -- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c0c90e0 msgr2=0x7f809c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c0c90e0 0x7f809c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 -- 172.21.15.5:0/235170843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094009b30 con 0x7f809c0c94c0 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809bfff700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c0c90e0 0x7f809c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f809b7fe700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809c0c94c0 0x7f809c143a20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f809000e9b0 tx=0x7f809000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f8098ff9700 1 -- 172.21.15.5:0/235170843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090009710 con 0x7f809c0c94c0 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c144aa0 con 0x7f809c0c94c0 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f8098ff9700 1 -- 172.21.15.5:0/235170843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809000ee50 con 0x7f809c0c94c0 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f8098ff9700 1 -- 172.21.15.5:0/235170843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000ca50 con 0x7f809c0c94c0 2024-05-08T23:27:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.501+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c0b5590 con 0x7f809c0c94c0 2024-05-08T23:27:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.502+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c15ee60 con 0x7f809c0c94c0 2024-05-08T23:27:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.503+0000 7f8098ff9700 1 -- 172.21.15.5:0/235170843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f809000cbb0 con 0x7f809c0c94c0 2024-05-08T23:27:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.503+0000 7f8098ff9700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f808806ac90 0x7f808806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.503+0000 7f8098ff9700 1 -- 172.21.15.5:0/235170843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80900892d0 con 0x7f809c0c94c0 2024-05-08T23:27:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.504+0000 7f809bfff700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f808806ac90 0x7f808806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.504+0000 7f809bfff700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f808806ac90 0x7f808806d150 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f808c000f60 tx=0x7f808c007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.508+0000 7f8098ff9700 1 -- 172.21.15.5:0/235170843 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f809c15ee60 con 0x7f809c0c94c0 2024-05-08T23:27:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.773+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f809c15ee60 con 0x7f809c0c94c0 2024-05-08T23:27:46.774 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:27:46.774+0000 7f8a49d06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:27:46.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.781+0000 7f8098ff9700 1 -- 172.21.15.5:0/235170843 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) v1 ==== 161+0+0 (secure 0 0 0) 0x7f809c15ee60 con 0x7f809c0c94c0 2024-05-08T23:27:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f808806ac90 msgr2=0x7f808806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f808806ac90 0x7f808806d150 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f808c000f60 tx=0x7f808c007ed0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809c0c94c0 msgr2=0x7f809c143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809c0c94c0 0x7f809c143a20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f809000e9b0 tx=0x7f809000ecc0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 shutdown_connections 2024-05-08T23:27:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f808806ac90 0x7f808806d150 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f809c146a10 0x7f809c13eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809c0c90e0 0x7f809c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 --2- 172.21.15.5:0/235170843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809c0c94c0 0x7f809c143a20 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 >> 172.21.15.5:0/235170843 conn(0x7f809c0cc580 msgr2=0x7f809c140e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:46.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 shutdown_connections 2024-05-08T23:27:46.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.784+0000 7f80a2285700 1 -- 172.21.15.5:0/235170843 wait complete. 2024-05-08T23:27:46.785 INFO:tasks.ceph.mds.a.smithi005.stderr: -9811> 2024-05-08T23:27:06.720+0000 7fb52a33c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:27:46.785 INFO:tasks.ceph.mds.a.smithi005.stderr: -9809> 2024-05-08T23:27:06.720+0000 7fb52a33c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T23:27:46.786 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:27:46.787 INFO:tasks.ceph.mds.b.smithi049.stderr: -1017> 2024-05-08T23:27:24.919+0000 7f9eb1c5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-05-08T23:27:07.992312+0000 1=0+1) n(v0 rc2024-05-08T23:27:07.992312+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6584=pAsLsXsFsx/pAsLsXsFsx@4},l=6584 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x564fa03d6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.992312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:07.992312+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.992312+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-05-08T23:27:07.992312+0000 2=0+2)","error_str":""},"return_code":-61} 2024-05-08T23:27:46.787 INFO:tasks.ceph.mds.b.smithi049.stderr: -1002> 2024-05-08T23:27:24.920+0000 7f9eb1c5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6584=pAsLsXsFsx/AsLsXsFsx@2},l=6584 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x564fa03d7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-05-08T23:27:46.788 INFO:tasks.ceph.mds.c.smithi005.stderr: -1117> 2024-05-08T23:27:24.919+0000 7fb68e892700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1) n(v2 rc2024-05-08T23:27:08.251306+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6584=pAsLsXsFsx/pAsLsXsFsx@4},l=6584 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x561cc33d1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:08.251306+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-05-08T23:27:08.251306+0000 4=0+4)","error_str":""},"return_code":-61} 2024-05-08T23:27:46.788 INFO:tasks.ceph.mds.c.smithi005.stderr: -997> 2024-05-08T23:27:24.920+0000 7fb69489e700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1) n(v1 rc2024-05-08T23:27:08.251306+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6584=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x561cc24d4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:07.992312+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.991312+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-05-08T23:27:08.251306+0000 3=0+3)","error_str":""},"return_code":-61} 2024-05-08T23:27:46.794 INFO:tasks.ceph.mds.a.smithi005.stderr: -2363> 2024-05-08T23:27:24.918+0000 7fb526b35700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-05-08T23:27:07.989312+0000 1=0+1) n(v3 rc2024-05-08T23:27:08.251306+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={6584=pAsLsXsFsx/pAsLsXsFsx@5},l=6584 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x564144076580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.989312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:08.251306+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.989312+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-05-08T23:27:08.251306+0000 6=0+6)","error_str":""},"return_code":-61} 2024-05-08T23:27:46.794 INFO:tasks.ceph.mds.a.smithi005.stderr: -2264> 2024-05-08T23:27:24.920+0000 7fb52cb41700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-05-08T23:27:07.990312+0000 1=0+1) n(v4 rc2024-05-08T23:27:08.251306+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6584=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x564144076b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:27:07.990312+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:27:08.251306+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-05-08T23:27:07.990312+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-05-08T23:27:08.251306+0000 5=0+5)","error_str":""},"return_code":-61} 2024-05-08T23:27:46.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:27:46.816 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:27:46.818 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:27:46.825 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:27:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.941+0000 7fdacf0a0700 1 Processor -- start 2024-05-08T23:27:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.941+0000 7fdacf0a0700 1 -- start start 2024-05-08T23:27:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.942+0000 7fdacf0a0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b8c0 0x7fdac80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.942+0000 7fdacf0a0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdac80d1740 0x7fdac80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.942+0000 7fdacf0a0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80cf560 con 0x7fdac815b4e0 2024-05-08T23:27:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.942+0000 7fdacf0a0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac80cefe0 con 0x7fdac80d1740 2024-05-08T23:27:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.942+0000 7fdacf0a0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80d1bb0 con 0x7fdac815b8c0 2024-05-08T23:27:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdacd63d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdac80d1740 0x7fdac80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac7fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b8c0 0x7fdac80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdacce3c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdac815b4e0 0x7fdac80b1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60308/0 (socket says 172.21.15.5:60308) 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdacd63d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdac80d1740 0x7fdac80cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34300/0 (socket says 172.21.15.5:34300) 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdacce3c700 1 -- 172.21.15.5:0/1763466354 learned_addr learned my addr 172.21.15.5:0/1763466354 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac7fff700 1 -- 172.21.15.5:0/1763466354 >> v2:172.21.15.49:3300/0 conn(0x7fdac80d1740 msgr2=0x7fdac80cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac7fff700 1 --2- 172.21.15.5:0/1763466354 >> v2:172.21.15.49:3300/0 conn(0x7fdac80d1740 0x7fdac80cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac7fff700 1 -- 172.21.15.5:0/1763466354 >> v1:172.21.15.49:6789/0 conn(0x7fdac815b4e0 legacy=0x7fdac80b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac7fff700 1 -- 172.21.15.5:0/1763466354 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80ccff0 con 0x7fdac815b8c0 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac7fff700 1 --2- 172.21.15.5:0/1763466354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b8c0 0x7fdac80d1200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009840 tx=0x7fdab4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e71c9f1c73cffe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac77fe700 1 -- 172.21.15.5:0/1763466354 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4011830 con 0x7fdac815b8c0 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac77fe700 1 -- 172.21.15.5:0/1763466354 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab4011990 con 0x7fdac815b8c0 2024-05-08T23:27:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.943+0000 7fdac77fe700 1 -- 172.21.15.5:0/1763466354 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4011d20 con 0x7fdac815b8c0 2024-05-08T23:27:46.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.944+0000 7fdacf0a0700 1 -- 172.21.15.5:0/1763466354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b8c0 msgr2=0x7fdac80d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.944+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/1763466354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b8c0 0x7fdac80d1200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009840 tx=0x7fdab4009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.944+0000 7fdacf0a0700 1 -- 172.21.15.5:0/1763466354 shutdown_connections 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.944+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/1763466354 >> v2:172.21.15.49:3300/0 conn(0x7fdac80d1740 0x7fdac80cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.944+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/1763466354 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b8c0 0x7fdac80d1200 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.944+0000 7fdacf0a0700 1 -- 172.21.15.5:0/1763466354 >> 172.21.15.5:0/1763466354 conn(0x7fdac808afe0 msgr2=0x7fdac808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.944+0000 7fdacf0a0700 1 -- 172.21.15.5:0/1763466354 shutdown_connections 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.944+0000 7fdacf0a0700 1 -- 172.21.15.5:0/1763466354 wait complete. 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.945+0000 7fdacf0a0700 1 Processor -- start 2024-05-08T23:27:46.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.945+0000 7fdacf0a0700 1 -- start start 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.946+0000 7fdacf0a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac80d1740 0x7fdac81f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.946+0000 7fdacf0a0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b4e0 0x7fdac81f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.946+0000 7fdacf0a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac815b8c0 0x7fdac815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.946+0000 7fdacf0a0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac815e330 con 0x7fdac815b8c0 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.946+0000 7fdacf0a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac815e610 con 0x7fdac80d1740 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.946+0000 7fdacf0a0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac81f4530 con 0x7fdac815b4e0 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdacd63d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac815b8c0 0x7fdac815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdacce3c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac80d1740 0x7fdac81f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac7fff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b4e0 0x7fdac81f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdacd63d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac815b8c0 0x7fdac815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57582/0 (socket says 172.21.15.5:57582) 2024-05-08T23:27:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdacd63d700 1 -- 172.21.15.5:0/2277704036 learned_addr learned my addr 172.21.15.5:0/2277704036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac7fff700 1 -- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac80d1740 msgr2=0x7fdac81f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac7fff700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac80d1740 0x7fdac81f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac7fff700 1 -- 172.21.15.5:0/2277704036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac815b8c0 msgr2=0x7fdac815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac7fff700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac815b8c0 0x7fdac815dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac7fff700 1 -- 172.21.15.5:0/2277704036 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab40094a0 con 0x7fdac815b4e0 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdacd63d700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac815b8c0 0x7fdac815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac7fff700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b4e0 0x7fdac81f0890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdab4016000 tx=0x7fdab4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac57fa700 1 -- 172.21.15.5:0/2277704036 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40036a0 con 0x7fdac815b4e0 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac57fa700 1 -- 172.21.15.5:0/2277704036 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab401e070 con 0x7fdac815b4e0 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdac57fa700 1 -- 172.21.15.5:0/2277704036 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4003a60 con 0x7fdac815b4e0 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.947+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac81f4810 con 0x7fdac815b4e0 2024-05-08T23:27:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.948+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac81f0dd0 con 0x7fdac815b4e0 2024-05-08T23:27:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.948+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac8074bc0 con 0x7fdac815b4e0 2024-05-08T23:27:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.949+0000 7fdac57fa700 1 -- 172.21.15.5:0/2277704036 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdab4003bc0 con 0x7fdac815b4e0 2024-05-08T23:27:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.949+0000 7fdac57fa700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdaac06ac90 0x7fdaac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.950+0000 7fdac57fa700 1 -- 172.21.15.5:0/2277704036 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fdab408a410 con 0x7fdac815b4e0 2024-05-08T23:27:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.950+0000 7fdacce3c700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdaac06ac90 0x7fdaac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.950+0000 7fdacce3c700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdaac06ac90 0x7fdaac06d150 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdabc005910 tx=0x7fdabc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:46.955+0000 7fdac57fa700 1 -- 172.21.15.5:0/2277704036 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdab40554e0 con 0x7fdac815b4e0 2024-05-08T23:27:47.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.221+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdac80b0fe0 con 0x7fdac815b4e0 2024-05-08T23:27:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.784+0000 7fdac57fa700 1 -- 172.21.15.5:0/2277704036 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdab4058b00 con 0x7fdac815b4e0 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdaac06ac90 msgr2=0x7fdaac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdaac06ac90 0x7fdaac06d150 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdabc005910 tx=0x7fdabc009040 comp rx=0 tx=0).stop 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b4e0 msgr2=0x7fdac81f0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b4e0 0x7fdac81f0890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdab4016000 tx=0x7fdab4004750 comp rx=0 tx=0).stop 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 shutdown_connections 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdaac06ac90 0x7fdaac06d150 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac815b8c0 0x7fdac815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac80d1740 0x7fdac81f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 --2- 172.21.15.5:0/2277704036 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdac815b4e0 0x7fdac81f0890 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.787+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 >> 172.21.15.5:0/2277704036 conn(0x7fdac808afe0 msgr2=0x7fdac806f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.788+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 shutdown_connections 2024-05-08T23:27:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.788+0000 7fdacf0a0700 1 -- 172.21.15.5:0/2277704036 wait complete. 2024-05-08T23:27:47.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:27:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.992+0000 7fc72fba6700 1 Processor -- start 2024-05-08T23:27:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.993+0000 7fc72fba6700 1 -- start start 2024-05-08T23:27:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.993+0000 7fc72fba6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 0x7fc7280b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.993+0000 7fc72fba6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7280b3fc0 0x7fc7280b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.993+0000 7fc72fba6700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7280cf610 con 0x7fc7280cff30 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.993+0000 7fc72fba6700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7280d1fc0 con 0x7fc7280b3fc0 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.993+0000 7fc72fba6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280d22a0 con 0x7fc7280d3d20 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72d942700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc7280cff30 0x7fc7280d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43894/0 (socket says 172.21.15.5:43894) 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72d942700 1 -- 172.21.15.5:0/1193144646 learned_addr learned my addr 172.21.15.5:0/1193144646 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72e143700 1 --2- 172.21.15.5:0/1193144646 >> v2:172.21.15.49:3300/0 conn(0x7fc7280b3fc0 0x7fc7280b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72d141700 1 --2- 172.21.15.5:0/1193144646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 0x7fc7280b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72c940700 1 -- 172.21.15.5:0/1193144646 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385889910 0 0) 0x7fc7280cf610 con 0x7fc7280cff30 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72c940700 1 -- 172.21.15.5:0/1193144646 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7140035b0 con 0x7fc7280cff30 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72e143700 1 -- 172.21.15.5:0/1193144646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 msgr2=0x7fc7280b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72e143700 1 --2- 172.21.15.5:0/1193144646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 0x7fc7280b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72e143700 1 -- 172.21.15.5:0/1193144646 >> v1:172.21.15.5:6789/0 conn(0x7fc7280cff30 legacy=0x7fc7280d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72c940700 1 -- 172.21.15.5:0/1193144646 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1798727059 0 0) 0x7fc7140035b0 con 0x7fc7280cff30 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.994+0000 7fc72e143700 1 -- 172.21.15.5:0/1193144646 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280d2580 con 0x7fc7280b3fc0 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72e143700 1 --2- 172.21.15.5:0/1193144646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 0x7fc7280b4450 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc720009d90 tx=0x7fc72000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab4b8241277f30a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72c940700 1 -- 172.21.15.5:0/1193144646 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000e070 con 0x7fc7280b3fc0 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72c940700 1 -- 172.21.15.5:0/1193144646 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc720003a20 con 0x7fc7280b3fc0 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72c940700 1 -- 172.21.15.5:0/1193144646 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724001670 con 0x7fc7280b3fc0 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72fba6700 1 -- 172.21.15.5:0/1193144646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 msgr2=0x7fc7280b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1193144646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 0x7fc7280b4450 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc720009d90 tx=0x7fc72000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72fba6700 1 -- 172.21.15.5:0/1193144646 shutdown_connections 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1193144646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 0x7fc7280b4450 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1193144646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 0x7fc7280b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72fba6700 1 -- 172.21.15.5:0/1193144646 >> 172.21.15.5:0/1193144646 conn(0x7fc72808afe0 msgr2=0x7fc7280d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72fba6700 1 -- 172.21.15.5:0/1193144646 shutdown_connections 2024-05-08T23:27:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.995+0000 7fc72fba6700 1 -- 172.21.15.5:0/1193144646 wait complete. 2024-05-08T23:27:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.997+0000 7fc72fba6700 1 Processor -- start 2024-05-08T23:27:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.997+0000 7fc72fba6700 1 -- start start 2024-05-08T23:27:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.997+0000 7fc72fba6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 0x7fc7280b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.997+0000 7fc72fba6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7280cff30 0x7fc7280b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.997+0000 7fc72fba6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 0x7fc7280b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.997+0000 7fc72fba6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280b7b60 con 0x7fc7280cff30 2024-05-08T23:27:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72fba6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280b7e40 con 0x7fc7280b3fc0 2024-05-08T23:27:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72fba6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280b8120 con 0x7fc7280d3d20 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7280cff30 0x7fc7280b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7280cff30 0x7fc7280b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33410/0 (socket says 172.21.15.5:33410) 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d942700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 0x7fc7280b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 -- 172.21.15.5:0/1959351834 learned_addr learned my addr 172.21.15.5:0/1959351834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72e143700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 0x7fc7280b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 -- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 msgr2=0x7fc7280b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 0x7fc7280b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 -- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 msgr2=0x7fc7280b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 0x7fc7280b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 -- 172.21.15.5:0/1959351834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720009a70 con 0x7fc7280cff30 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d942700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 0x7fc7280b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72d141700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7280cff30 0x7fc7280b7280 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc71800e870 tx=0x7fc71800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc71e7fc700 1 -- 172.21.15.5:0/1959351834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718009590 con 0x7fc7280cff30 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc71e7fc700 1 -- 172.21.15.5:0/1959351834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc718013680 con 0x7fc7280cff30 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7280b8400 con 0x7fc7280cff30 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.998+0000 7fc71e7fc700 1 -- 172.21.15.5:0/1959351834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180139f0 con 0x7fc7280cff30 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.999+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7280b77c0 con 0x7fc7280cff30 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:47.999+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7280d3210 con 0x7fc7280cff30 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.000+0000 7fc71e7fc700 1 -- 172.21.15.5:0/1959351834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc718013b50 con 0x7fc7280cff30 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.001+0000 7fc71e7fc700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc70c06ac90 0x7fc70c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.001+0000 7fc71e7fc700 1 -- 172.21.15.5:0/1959351834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc7180892f0 con 0x7fc7280cff30 2024-05-08T23:27:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.001+0000 7fc72d942700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc70c06ac90 0x7fc70c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.001+0000 7fc72d942700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc70c06ac90 0x7fc70c06d150 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc724007fd0 tx=0x7fc724001230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.006+0000 7fc71e7fc700 1 -- 172.21.15.5:0/1959351834 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc7180543d0 con 0x7fc7280cff30 2024-05-08T23:27:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.223+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc728074bc0 con 0x7fc7280cff30 2024-05-08T23:27:48.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.784+0000 7fc71e7fc700 1 -- 172.21.15.5:0/1959351834 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc7180579f0 con 0x7fc7280cff30 2024-05-08T23:27:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc70c06ac90 msgr2=0x7fc70c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc70c06ac90 0x7fc70c06d150 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc724007fd0 tx=0x7fc724001230 comp rx=0 tx=0).stop 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7280cff30 msgr2=0x7fc7280b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7280cff30 0x7fc7280b7280 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc71800e870 tx=0x7fc71800eb80 comp rx=0 tx=0).stop 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 shutdown_connections 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc70c06ac90 0x7fc70c06d150 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc7280d3d20 0x7fc7280b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7280b3fc0 0x7fc7280b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 --2- 172.21.15.5:0/1959351834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7280cff30 0x7fc7280b7280 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 >> 172.21.15.5:0/1959351834 conn(0x7fc72808afe0 msgr2=0x7fc7280d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.789+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 shutdown_connections 2024-05-08T23:27:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.790+0000 7fc72fba6700 1 -- 172.21.15.5:0/1959351834 wait complete. 2024-05-08T23:27:48.791 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:27:48.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.957+0000 7fa7492c2700 1 Processor -- start 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.957+0000 7fa7492c2700 1 -- start start 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.958+0000 7fa7492c2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa74406b070 0x7fa744168490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.958+0000 7fa7492c2700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa744145040 con 0x7fa7440b4d50 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.958+0000 7fa7492c2700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7440b16e0 con 0x7fa7440b0d80 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.958+0000 7fa7492c2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7440b19c0 con 0x7fa74406b070 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.958+0000 7fa7427fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa7440b4d50 0x7fa74406a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43910/0 (socket says 172.21.15.5:43910) 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa7427fc700 1 -- 172.21.15.5:0/1445460346 learned_addr learned my addr 172.21.15.5:0/1445460346 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa7437fe700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa74406b070 0x7fa744168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254742397 0 0) 0x7fa744145040 con 0x7fa7440b4d50 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa72c0035b0 con 0x7fa7440b4d50 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916269212 0 0) 0x7fa7440b16e0 con 0x7fa7440b0d80 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa744145040 con 0x7fa7440b0d80 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3388635253 0 0) 0x7fa72c0035b0 con 0x7fa7440b4d50 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa74406b070 msgr2=0x7fa744168490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa74406b070 0x7fa744168490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 >> v1:172.21.15.49:6789/0 conn(0x7fa7440b0d80 legacy=0x7fa7440b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7440b1ca0 con 0x7fa7440b4d50 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fa738001670 con 0x7fa7440b4d50 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 >> v1:172.21.15.5:6789/0 conn(0x7fa7440b4d50 legacy=0x7fa74406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa72c003bc0 0x7fa72c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa74406b070 0x7fa72c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa72c004d90 0x7fa72c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7440b19c0 con 0x7fa72c004d90 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c00bac0 con 0x7fa72c003bc0 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c00bc80 con 0x7fa74406b070 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa7437fe700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa72c004d90 0x7fa72c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa7427fc700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa74406b070 0x7fa72c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.959+0000 7fa742ffd700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa72c003bc0 0x7fa72c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa7437fe700 1 -- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa74406b070 msgr2=0x7fa72c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa7437fe700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa74406b070 0x7fa72c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa7437fe700 1 -- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa72c003bc0 msgr2=0x7fa72c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa7437fe700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa72c003bc0 0x7fa72c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa7437fe700 1 -- 172.21.15.5:0/1445460346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0035b0 con 0x7fa72c004d90 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa7437fe700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa72c004d90 0x7fa72c005210 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009fc0 tx=0x7fa72c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49c9334f49d1a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00ec30 con 0x7fa72c004d90 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c00f040 con 0x7fa72c004d90 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.960+0000 7fa741ffb700 1 -- 172.21.15.5:0/1445460346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c014480 con 0x7fa72c004d90 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.961+0000 7fa7492c2700 1 -- 172.21.15.5:0/1445460346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa72c004d90 msgr2=0x7fa72c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.961+0000 7fa7492c2700 1 --2- 172.21.15.5:0/1445460346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa72c004d90 0x7fa72c005210 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009fc0 tx=0x7fa72c005880 comp rx=0 tx=0).stop 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.961+0000 7fa7437fe700 1 -- 172.21.15.5:0/1445460346 reap_dead start 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.961+0000 7fa7492c2700 1 -- 172.21.15.5:0/1445460346 shutdown_connections 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.961+0000 7fa7492c2700 1 -- 172.21.15.5:0/1445460346 >> 172.21.15.5:0/1445460346 conn(0x7fa744099a50 msgr2=0x7fa7440b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.961+0000 7fa7492c2700 1 -- 172.21.15.5:0/1445460346 shutdown_connections 2024-05-08T23:27:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.961+0000 7fa7492c2700 1 -- 172.21.15.5:0/1445460346 wait complete. 2024-05-08T23:27:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.962+0000 7fa7492c2700 1 Processor -- start 2024-05-08T23:27:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.962+0000 7fa7492c2700 1 -- start start 2024-05-08T23:27:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7492c2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7441f0940 0x7fa7441f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7492c2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7440aecc0 0x7fa7441f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7492c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7441f6cb0 0x7fa7440cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7492c2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7441f7160 con 0x7fa7441f6cb0 2024-05-08T23:27:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7492c2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa744132b20 con 0x7fa7440aecc0 2024-05-08T23:27:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7492c2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa744132dc0 con 0x7fa7441f0940 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7441f6cb0 0x7fa7440cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7441f6cb0 0x7fa7440cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33430/0 (socket says 172.21.15.5:33430) 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.963+0000 7fa7437fe700 1 -- 172.21.15.5:0/3610881221 learned_addr learned my addr 172.21.15.5:0/3610881221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7427fc700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7440aecc0 0x7fa7441f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa742ffd700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7441f0940 0x7fa7441f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7437fe700 1 -- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7441f0940 msgr2=0x7fa7441f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7437fe700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7441f0940 0x7fa7441f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7437fe700 1 -- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7440aecc0 msgr2=0x7fa7441f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7437fe700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7440aecc0 0x7fa7441f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7437fe700 1 -- 172.21.15.5:0/3610881221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7440af4f0 con 0x7fa7441f6cb0 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7437fe700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7441f6cb0 0x7fa7440cf060 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa73c016000 tx=0x7fa73c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa72b7fe700 1 -- 172.21.15.5:0/3610881221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c007000 con 0x7fa7441f6cb0 2024-05-08T23:27:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa72b7fe700 1 -- 172.21.15.5:0/3610881221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c022070 con 0x7fa7441f6cb0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7440af7d0 con 0x7fa7441f6cb0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa72b7fe700 1 -- 172.21.15.5:0/3610881221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c029480 con 0x7fa7441f6cb0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.964+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7440af140 con 0x7fa7441f6cb0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.965+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa744074bc0 con 0x7fa7441f6cb0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.966+0000 7fa72b7fe700 1 -- 172.21.15.5:0/3610881221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa73c01e430 con 0x7fa7441f6cb0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.966+0000 7fa72b7fe700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa73006ac90 0x7fa73006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.966+0000 7fa72b7fe700 1 -- 172.21.15.5:0/3610881221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5529+0+0 (secure 0 0 0) 0x7fa73c024070 con 0x7fa7441f6cb0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.966+0000 7fa742ffd700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa73006ac90 0x7fa73006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.967+0000 7fa742ffd700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa73006ac90 0x7fa73006d150 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa7441329f0 tx=0x7fa734009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:48.971+0000 7fa72b7fe700 1 -- 172.21.15.5:0/3610881221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa73c0565a0 con 0x7fa7441f6cb0 2024-05-08T23:27:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.186+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa744132f00 con 0x7fa7441f6cb0 2024-05-08T23:27:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.788+0000 7fa72b7fe700 1 -- 172.21.15.5:0/3610881221 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa73c059bc0 con 0x7fa7441f6cb0 2024-05-08T23:27:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa73006ac90 msgr2=0x7fa73006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa73006ac90 0x7fa73006d150 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa7441329f0 tx=0x7fa734009040 comp rx=0 tx=0).stop 2024-05-08T23:27:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7441f6cb0 msgr2=0x7fa7440cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7441f6cb0 0x7fa7440cf060 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa73c016000 tx=0x7fa73c009750 comp rx=0 tx=0).stop 2024-05-08T23:27:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 shutdown_connections 2024-05-08T23:27:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa73006ac90 0x7fa73006d150 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7441f6cb0 0x7fa7440cf060 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa7441f0940 0x7fa7441f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 --2- 172.21.15.5:0/3610881221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7440aecc0 0x7fa7441f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.794+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 >> 172.21.15.5:0/3610881221 conn(0x7fa744099a50 msgr2=0x7fa74406f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.795+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 shutdown_connections 2024-05-08T23:27:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.795+0000 7fa7492c2700 1 -- 172.21.15.5:0/3610881221 wait complete. 2024-05-08T23:27:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-08T23:27:49.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T23:27:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.953+0000 7f4db0173700 1 Processor -- start 2024-05-08T23:27:49.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.954+0000 7f4db0173700 1 -- start start 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.954+0000 7f4db0173700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80d3d20 0x7f4da80b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.954+0000 7f4db0173700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4da80b3fc0 0x7f4da80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.954+0000 7f4db0173700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80cf610 con 0x7f4da80cff30 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.954+0000 7f4db0173700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da80d1fc0 con 0x7f4da80b3fc0 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.954+0000 7f4db0173700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80d22a0 con 0x7f4da80d3d20 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dae710700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4da80b3fc0 0x7f4da80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dad70e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80d3d20 0x7f4da80b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dadf0f700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4da80cff30 0x7f4da80d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60808/0 (socket says 172.21.15.5:60808) 2024-05-08T23:27:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dadf0f700 1 -- 172.21.15.5:0/1836209564 learned_addr learned my addr 172.21.15.5:0/1836209564 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dae710700 1 -- 172.21.15.5:0/1836209564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80d3d20 msgr2=0x7f4da80b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dae710700 1 --2- 172.21.15.5:0/1836209564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80d3d20 0x7f4da80b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dae710700 1 -- 172.21.15.5:0/1836209564 >> v1:172.21.15.49:6789/0 conn(0x7f4da80cff30 legacy=0x7f4da80d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dae710700 1 -- 172.21.15.5:0/1836209564 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80d2580 con 0x7f4da80b3fc0 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dad70e700 1 --2- 172.21.15.5:0/1836209564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80d3d20 0x7f4da80b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.955+0000 7f4dae710700 1 --2- 172.21.15.5:0/1836209564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80b3fc0 0x7f4da80b4450 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009d90 tx=0x7f4da000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77d56c96da446bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4dacf0d700 1 -- 172.21.15.5:0/1836209564 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da000e070 con 0x7f4da80b3fc0 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4dacf0d700 1 -- 172.21.15.5:0/1836209564 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da0003a20 con 0x7f4da80b3fc0 2024-05-08T23:27:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4dacf0d700 1 -- 172.21.15.5:0/1836209564 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0003dd0 con 0x7f4da80b3fc0 2024-05-08T23:27:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4db0173700 1 -- 172.21.15.5:0/1836209564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80b3fc0 msgr2=0x7f4da80b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4db0173700 1 --2- 172.21.15.5:0/1836209564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80b3fc0 0x7f4da80b4450 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009d90 tx=0x7f4da000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4db0173700 1 -- 172.21.15.5:0/1836209564 shutdown_connections 2024-05-08T23:27:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4db0173700 1 --2- 172.21.15.5:0/1836209564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80b3fc0 0x7f4da80b4450 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4db0173700 1 --2- 172.21.15.5:0/1836209564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80d3d20 0x7f4da80b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4db0173700 1 -- 172.21.15.5:0/1836209564 >> 172.21.15.5:0/1836209564 conn(0x7f4da808afe0 msgr2=0x7f4da80d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4db0173700 1 -- 172.21.15.5:0/1836209564 shutdown_connections 2024-05-08T23:27:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.956+0000 7f4db0173700 1 -- 172.21.15.5:0/1836209564 wait complete. 2024-05-08T23:27:49.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.958+0000 7f4db0173700 1 Processor -- start 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.958+0000 7f4db0173700 1 -- start start 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4db0173700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80b3fc0 0x7f4da81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4db0173700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4da80cff30 0x7f4da80b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4db0173700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80d3d20 0x7f4da80b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4db0173700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80cb640 con 0x7f4da80cff30 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4db0173700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80cb920 con 0x7f4da80d3d20 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4db0173700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80cbc00 con 0x7f4da80b3fc0 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4da80cff30 0x7f4da80b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4da80cff30 0x7f4da80b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33440/0 (socket says 172.21.15.5:33440) 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 -- 172.21.15.5:0/3629654438 learned_addr learned my addr 172.21.15.5:0/3629654438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dadf0f700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80b3fc0 0x7f4da81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dae710700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80d3d20 0x7f4da80b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:49.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 -- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80b3fc0 msgr2=0x7f4da81577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80b3fc0 0x7f4da81577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 -- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80d3d20 msgr2=0x7f4da80b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80d3d20 0x7f4da80b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 -- 172.21.15.5:0/3629654438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d980094a0 con 0x7f4da80cff30 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4dad70e700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4da80cff30 0x7f4da80b50e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800ebe0 tx=0x7f4d98009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.959+0000 7f4d9e7fc700 1 -- 172.21.15.5:0/3629654438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800c8b0 con 0x7f4da80cff30 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.960+0000 7f4d9e7fc700 1 -- 172.21.15.5:0/3629654438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9800ca10 con 0x7f4da80cff30 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.960+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da0009a70 con 0x7f4da80cff30 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.960+0000 7f4d9e7fc700 1 -- 172.21.15.5:0/3629654438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800cda0 con 0x7f4da80cff30 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.960+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da80cc1c0 con 0x7f4da80cff30 2024-05-08T23:27:49.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.961+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da80cc530 con 0x7f4da80cff30 2024-05-08T23:27:49.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.961+0000 7f4d9e7fc700 1 -- 172.21.15.5:0/3629654438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4d98013470 con 0x7f4da80cff30 2024-05-08T23:27:49.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.962+0000 7f4d9e7fc700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4d8c06ac90 0x7f4d8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:49.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.962+0000 7f4d9e7fc700 1 -- 172.21.15.5:0/3629654438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4d98089260 con 0x7f4da80cff30 2024-05-08T23:27:49.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.962+0000 7f4dadf0f700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4d8c06ac90 0x7f4d8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:49.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.962+0000 7f4dadf0f700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4d8c06ac90 0x7f4d8c06d150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4da4000f60 tx=0x7f4da4007d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:49.966+0000 7f4d9e7fc700 1 -- 172.21.15.5:0/3629654438 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4d98054690 con 0x7f4da80cff30 2024-05-08T23:27:50.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.185+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4da815fea0 con 0x7f4da80cff30 2024-05-08T23:27:50.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.189+0000 7f4d9e7fc700 1 -- 172.21.15.5:0/3629654438 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4d98057cb0 con 0x7f4da80cff30 2024-05-08T23:27:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4d8c06ac90 msgr2=0x7f4d8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4d8c06ac90 0x7f4d8c06d150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4da4000f60 tx=0x7f4da4007d90 comp rx=0 tx=0).stop 2024-05-08T23:27:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4da80cff30 msgr2=0x7f4da80b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4da80cff30 0x7f4da80b50e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800ebe0 tx=0x7f4d98009e20 comp rx=0 tx=0).stop 2024-05-08T23:27:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 shutdown_connections 2024-05-08T23:27:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4d8c06ac90 0x7f4d8c06d150 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4da80d3d20 0x7f4da80b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4da80b3fc0 0x7f4da81577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 --2- 172.21.15.5:0/3629654438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4da80cff30 0x7f4da80b50e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 >> 172.21.15.5:0/3629654438 conn(0x7f4da808afe0 msgr2=0x7f4da80d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:50.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.191+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 shutdown_connections 2024-05-08T23:27:50.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.192+0000 7f4db0173700 1 -- 172.21.15.5:0/3629654438 wait complete. 2024-05-08T23:27:50.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-05-08T23:27:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.342+0000 7f7e9fbe0700 1 Processor -- start 2024-05-08T23:27:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.342+0000 7f7e9fbe0700 1 -- start start 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9fbe0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e980af030 0x7f7e9815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9fbe0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e980b4a80 con 0x7f7e980b3990 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9fbe0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9806d800 con 0x7f7e9806e4d0 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9fbe0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9815ede0 con 0x7f7e980af030 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9d17b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7e980b3990 0x7f7e9806ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43916/0 (socket says 172.21.15.5:43916) 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9d17b700 1 -- 172.21.15.5:0/1090076488 learned_addr learned my addr 172.21.15.5:0/1090076488 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9d97c700 1 --2- 172.21.15.5:0/1090076488 >> v2:172.21.15.49:3300/0 conn(0x7f7e980af030 0x7f7e9815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9c97a700 1 -- 172.21.15.5:0/1090076488 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487359442 0 0) 0x7f7e980b4a80 con 0x7f7e980b3990 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9c97a700 1 -- 172.21.15.5:0/1090076488 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e840035b0 con 0x7f7e980b3990 2024-05-08T23:27:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9c97a700 1 -- 172.21.15.5:0/1090076488 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 497474380 0 0) 0x7f7e840035b0 con 0x7f7e980b3990 2024-05-08T23:27:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9d97c700 1 -- 172.21.15.5:0/1090076488 >> v1:172.21.15.49:6789/0 conn(0x7f7e9806e4d0 legacy=0x7f7e9806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9d97c700 1 -- 172.21.15.5:0/1090076488 >> v1:172.21.15.5:6789/0 conn(0x7f7e980b3990 legacy=0x7f7e9806ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.343+0000 7f7e9d97c700 1 -- 172.21.15.5:0/1090076488 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980b3e70 con 0x7f7e980af030 2024-05-08T23:27:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.344+0000 7f7e9d97c700 1 --2- 172.21.15.5:0/1090076488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980af030 0x7f7e9815e8a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009840 tx=0x7f7e94009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26ced4ad0ecbce4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.344+0000 7f7e9c97a700 1 -- 172.21.15.5:0/1090076488 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88001670 con 0x7f7e980af030 2024-05-08T23:27:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.344+0000 7f7e9c97a700 1 -- 172.21.15.5:0/1090076488 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e94004030 con 0x7f7e980af030 2024-05-08T23:27:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.344+0000 7f7e9c97a700 1 -- 172.21.15.5:0/1090076488 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940043c0 con 0x7f7e980af030 2024-05-08T23:27:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.344+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/1090076488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980af030 msgr2=0x7f7e9815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.344+0000 7f7e9fbe0700 1 --2- 172.21.15.5:0/1090076488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980af030 0x7f7e9815e8a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009840 tx=0x7f7e94009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.345+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/1090076488 shutdown_connections 2024-05-08T23:27:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.345+0000 7f7e9fbe0700 1 --2- 172.21.15.5:0/1090076488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980af030 0x7f7e9815e8a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.345+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/1090076488 >> 172.21.15.5:0/1090076488 conn(0x7f7e980cfce0 msgr2=0x7f7e980d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.345+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/1090076488 shutdown_connections 2024-05-08T23:27:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.345+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/1090076488 wait complete. 2024-05-08T23:27:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.346+0000 7f7e9fbe0700 1 Processor -- start 2024-05-08T23:27:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.346+0000 7f7e9fbe0700 1 -- start start 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9fbe0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9806e4d0 0x7f7e980cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9fbe0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7e980af030 0x7f7e980c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9fbe0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980b3990 0x7f7e980cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9fbe0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980ccf30 con 0x7f7e9806e4d0 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9fbe0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980cc320 con 0x7f7e980b3990 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9fbe0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980cc600 con 0x7f7e980af030 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9806e4d0 0x7f7e980cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9806e4d0 0x7f7e980cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33442/0 (socket says 172.21.15.5:33442) 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 -- 172.21.15.5:0/3528816041 learned_addr learned my addr 172.21.15.5:0/3528816041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9e17d700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980b3990 0x7f7e980cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d17b700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7e980af030 0x7f7e980c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 -- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7e980af030 msgr2=0x7f7e980c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7e980af030 0x7f7e980c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 -- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980b3990 msgr2=0x7f7e980cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980b3990 0x7f7e980cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 -- 172.21.15.5:0/3528816041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e940094a0 con 0x7f7e9806e4d0 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.347+0000 7f7e9d97c700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9806e4d0 0x7f7e980cb630 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009b90 tx=0x7f7e94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.348+0000 7f7e8e7fc700 1 -- 172.21.15.5:0/3528816041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94011630 con 0x7f7e9806e4d0 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.348+0000 7f7e8e7fc700 1 -- 172.21.15.5:0/3528816041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9401f070 con 0x7f7e9806e4d0 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.348+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e980cc8e0 con 0x7f7e9806e4d0 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.348+0000 7f7e8e7fc700 1 -- 172.21.15.5:0/3528816041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94011a10 con 0x7f7e9806e4d0 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.348+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e980cbf40 con 0x7f7e9806e4d0 2024-05-08T23:27:50.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.349+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e98068000 con 0x7f7e9806e4d0 2024-05-08T23:27:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.349+0000 7f7e8e7fc700 1 -- 172.21.15.5:0/3528816041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7e94011b70 con 0x7f7e9806e4d0 2024-05-08T23:27:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.350+0000 7f7e8e7fc700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7e7c06ac90 0x7f7e7c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.350+0000 7f7e8e7fc700 1 -- 172.21.15.5:0/3528816041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7e94021070 con 0x7f7e9806e4d0 2024-05-08T23:27:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.350+0000 7f7e9d17b700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7e7c06ac90 0x7f7e7c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.350+0000 7f7e9d17b700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7e7c06ac90 0x7f7e7c06d150 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e9806c0c0 tx=0x7f7e88007bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.355+0000 7f7e8e7fc700 1 -- 172.21.15.5:0/3528816041 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7e94055e10 con 0x7f7e9806e4d0 2024-05-08T23:27:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.662+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) v1 -- 0x7f7e9815e0d0 con 0x7f7e9806e4d0 2024-05-08T23:27:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.787+0000 7f7e8e7fc700 1 -- 172.21.15.5:0/3528816041 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v351) v1 ==== 154+0+0 (secure 0 0 0) 0x7f7e94059430 con 0x7f7e9806e4d0 2024-05-08T23:27:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7e7c06ac90 msgr2=0x7f7e7c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7e7c06ac90 0x7f7e7c06d150 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e9806c0c0 tx=0x7f7e88007bc0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9806e4d0 msgr2=0x7f7e980cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9806e4d0 0x7f7e980cb630 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009b90 tx=0x7f7e94004750 comp rx=0 tx=0).stop 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 shutdown_connections 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7e7c06ac90 0x7f7e7c06d150 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e980b3990 0x7f7e980cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9806e4d0 0x7f7e980cb630 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 --2- 172.21.15.5:0/3528816041 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7e980af030 0x7f7e980c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 >> 172.21.15.5:0/3528816041 conn(0x7f7e980cfce0 msgr2=0x7f7e98161180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 shutdown_connections 2024-05-08T23:27:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.790+0000 7f7e9fbe0700 1 -- 172.21.15.5:0/3528816041 wait complete. 2024-05-08T23:27:50.805 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-05-08T23:27:50.806 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-05-08T23:27:50.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:27:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.946+0000 7f405290e700 1 Processor -- start 2024-05-08T23:27:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.946+0000 7f405290e700 1 -- start start 2024-05-08T23:27:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f405290e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0aef20 0x7f404c15c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f405290e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f404c15f230 0x7f404c0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f405290e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c0cd3b0 con 0x7f404c0cca70 2024-05-08T23:27:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f405290e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404c141df0 con 0x7f404c15f230 2024-05-08T23:27:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f405290e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0ccf00 con 0x7f404c0aef20 2024-05-08T23:27:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f404c15f230 0x7f404c0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f4050eab700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f404c0cca70 0x7f404c0ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43932/0 (socket says 172.21.15.5:43932) 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f404c15f230 0x7f404c0cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33444/0 (socket says 172.21.15.5:33444) 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f4050eab700 1 -- 172.21.15.5:0/3997688775 learned_addr learned my addr 172.21.15.5:0/3997688775 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404bfff700 1 --2- 172.21.15.5:0/3997688775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0aef20 0x7f404c15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404b7fe700 1 -- 172.21.15.5:0/3997688775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0aef20 msgr2=0x7f404c15c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404b7fe700 1 --2- 172.21.15.5:0/3997688775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0aef20 0x7f404c15c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404b7fe700 1 -- 172.21.15.5:0/3997688775 >> v1:172.21.15.5:6789/0 conn(0x7f404c0cca70 legacy=0x7f404c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404b7fe700 1 -- 172.21.15.5:0/3997688775 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c0af300 con 0x7f404c15f230 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404bfff700 1 --2- 172.21.15.5:0/3997688775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0aef20 0x7f404c15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.947+0000 7f404affd700 1 -- 172.21.15.5:0/3997688775 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306227126 0 0) 0x7f404c0cd3b0 con 0x7f404c0cca70 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f404b7fe700 1 --2- 172.21.15.5:0/3997688775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c0cc530 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4040009840 tx=0x7f4040009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ac7b77150aeebf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f404affd700 1 -- 172.21.15.5:0/3997688775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040011830 con 0x7f404c15f230 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f404affd700 1 -- 172.21.15.5:0/3997688775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040011990 con 0x7f404c15f230 2024-05-08T23:27:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f404affd700 1 -- 172.21.15.5:0/3997688775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040011d20 con 0x7f404c15f230 2024-05-08T23:27:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f405290e700 1 -- 172.21.15.5:0/3997688775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 msgr2=0x7f404c0cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f405290e700 1 --2- 172.21.15.5:0/3997688775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c0cc530 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4040009840 tx=0x7f4040009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f405290e700 1 -- 172.21.15.5:0/3997688775 shutdown_connections 2024-05-08T23:27:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f405290e700 1 --2- 172.21.15.5:0/3997688775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0aef20 0x7f404c15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f405290e700 1 --2- 172.21.15.5:0/3997688775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c0cc530 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.948+0000 7f405290e700 1 -- 172.21.15.5:0/3997688775 >> 172.21.15.5:0/3997688775 conn(0x7f404c0cfce0 msgr2=0x7f404c0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.949+0000 7f405290e700 1 -- 172.21.15.5:0/3997688775 shutdown_connections 2024-05-08T23:27:50.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.949+0000 7f405290e700 1 -- 172.21.15.5:0/3997688775 wait complete. 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.950+0000 7f405290e700 1 Processor -- start 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.950+0000 7f405290e700 1 -- start start 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f405290e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f404c0aef20 0x7f404c136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f405290e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0cca70 0x7f404c071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f405290e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f405290e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0b4b90 con 0x7f404c15f230 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f405290e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0b2f90 con 0x7f404c0aef20 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f405290e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0b3250 con 0x7f404c0cca70 2024-05-08T23:27:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33460/0 (socket says 172.21.15.5:33460) 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 -- 172.21.15.5:0/1818749869 learned_addr learned my addr 172.21.15.5:0/1818749869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f404bfff700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f404c0aef20 0x7f404c136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f404b7fe700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0cca70 0x7f404c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 -- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0cca70 msgr2=0x7f404c071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0cca70 0x7f404c071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 -- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f404c0aef20 msgr2=0x7f404c136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f404c0aef20 0x7f404c136eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 -- 172.21.15.5:0/1818749869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40400094a0 con 0x7f404c15f230 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.951+0000 7f4050eab700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c155500 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f4044002230 tx=0x7f404400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.952+0000 7f4048ff9700 1 -- 172.21.15.5:0/1818749869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404400ac20 con 0x7f404c15f230 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.952+0000 7f4048ff9700 1 -- 172.21.15.5:0/1818749869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404400ada0 con 0x7f404c15f230 2024-05-08T23:27:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.952+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c0b3530 con 0x7f404c15f230 2024-05-08T23:27:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.952+0000 7f4048ff9700 1 -- 172.21.15.5:0/1818749869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044007310 con 0x7f404c15f230 2024-05-08T23:27:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.952+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c0b2c10 con 0x7f404c15f230 2024-05-08T23:27:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.953+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c0b3b40 con 0x7f404c15f230 2024-05-08T23:27:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.953+0000 7f4048ff9700 1 -- 172.21.15.5:0/1818749869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4044007470 con 0x7f404c15f230 2024-05-08T23:27:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.954+0000 7f4048ff9700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f403806ac90 0x7f403806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.954+0000 7f4048ff9700 1 -- 172.21.15.5:0/1818749869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4044088f10 con 0x7f404c15f230 2024-05-08T23:27:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.954+0000 7f404bfff700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f403806ac90 0x7f403806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.954+0000 7f404bfff700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f403806ac90 0x7f403806d150 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:50.958+0000 7f4048ff9700 1 -- 172.21.15.5:0/1818749869 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f40440543d0 con 0x7f404c15f230 2024-05-08T23:27:51.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.179+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f404c068000 con 0x7f404c15f230 2024-05-08T23:27:51.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.180+0000 7f4048ff9700 1 -- 172.21.15.5:0/1818749869 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+12679 (secure 0 0 0) 0x7f4044018090 con 0x7f404c15f230 2024-05-08T23:27:51.180 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:51.180 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:27:49.783341+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/1093540582":"2024-05-09T23:27:46.775109+0000","172.21.15.5:6833/1093540582":"2024-05-09T23:27:46.775109+0000","172.21.15.5:6834/3718289875":"2024-05-09T23:27:46.775104+0000","172.21.15.5:6832/3718289875":"2024-05-09T23:27:46.775104+0000","172.21.15.49:6837/1134209849":"2024-05-09T23:27:46.775087+0000","172.21.15.49:6835/1134209849":"2024-05-09T23:27:46.775087+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:27:51.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f403806ac90 msgr2=0x7f403806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f403806ac90 0x7f403806d150 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c008040 comp rx=0 tx=0).stop 2024-05-08T23:27:51.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 msgr2=0x7f404c155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c155500 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f4044002230 tx=0x7f404400a060 comp rx=0 tx=0).stop 2024-05-08T23:27:51.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 shutdown_connections 2024-05-08T23:27:51.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f403806ac90 0x7f403806d150 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404c15f230 0x7f404c155500 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f404c0aef20 0x7f404c136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 --2- 172.21.15.5:0/1818749869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f404c0cca70 0x7f404c071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 >> 172.21.15.5:0/1818749869 conn(0x7f404c0cfce0 msgr2=0x7f404c06ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:51.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 shutdown_connections 2024-05-08T23:27:51.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.182+0000 7f405290e700 1 -- 172.21.15.5:0/1818749869 wait complete. 2024-05-08T23:27:51.196 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:27:51.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:27:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.336+0000 7f308ac45700 1 Processor -- start 2024-05-08T23:27:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.336+0000 7f308ac45700 1 -- start start 2024-05-08T23:27:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f308ac45700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f30840b50d0 0x7f308406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f308ac45700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30840b5550 0x7f308406aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f308ac45700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3084099ce0 con 0x7f308415d8c0 2024-05-08T23:27:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f308ac45700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308406fde0 con 0x7f30840b50d0 2024-05-08T23:27:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f308ac45700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308406f260 con 0x7f30840b5550 2024-05-08T23:27:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f3083fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f30840b50d0 0x7f308406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f30889e1700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f308415d8c0 0x7f30840ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43940/0 (socket says 172.21.15.5:43940) 2024-05-08T23:27:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f3083fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f30840b50d0 0x7f308406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33476/0 (socket says 172.21.15.5:33476) 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.337+0000 7f30889e1700 1 -- 172.21.15.5:0/3332925759 learned_addr learned my addr 172.21.15.5:0/3332925759 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f3083fff700 1 -- 172.21.15.5:0/3332925759 >> v2:172.21.15.49:3300/0 conn(0x7f30840b5550 msgr2=0x7f308406aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f30891e2700 1 --2- 172.21.15.5:0/3332925759 >> v2:172.21.15.49:3300/0 conn(0x7f30840b5550 0x7f308406aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f3083fff700 1 --2- 172.21.15.5:0/3332925759 >> v2:172.21.15.49:3300/0 conn(0x7f30840b5550 0x7f308406aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f3083fff700 1 -- 172.21.15.5:0/3332925759 >> v1:172.21.15.5:6789/0 conn(0x7f308415d8c0 legacy=0x7f30840ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f3083fff700 1 -- 172.21.15.5:0/3332925759 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308406f540 con 0x7f30840b50d0 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f30891e2700 1 --2- 172.21.15.5:0/3332925759 >> v2:172.21.15.49:3300/0 conn(0x7f30840b5550 0x7f308406aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f3083fff700 1 --2- 172.21.15.5:0/3332925759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b50d0 0x7f308406a960 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f30840b4650 tx=0x7f3078009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcf189da3766643 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f30837fe700 1 -- 172.21.15.5:0/3332925759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078011830 con 0x7f30840b50d0 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f30837fe700 1 -- 172.21.15.5:0/3332925759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3078011990 con 0x7f30840b50d0 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.338+0000 7f30837fe700 1 -- 172.21.15.5:0/3332925759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078011d20 con 0x7f30840b50d0 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.339+0000 7f308ac45700 1 -- 172.21.15.5:0/3332925759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b50d0 msgr2=0x7f308406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.339+0000 7f308ac45700 1 --2- 172.21.15.5:0/3332925759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b50d0 0x7f308406a960 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f30840b4650 tx=0x7f3078009b20 comp rx=0 tx=0).stop 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.339+0000 7f308ac45700 1 -- 172.21.15.5:0/3332925759 shutdown_connections 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.339+0000 7f308ac45700 1 --2- 172.21.15.5:0/3332925759 >> v2:172.21.15.49:3300/0 conn(0x7f30840b5550 0x7f308406aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.339+0000 7f308ac45700 1 --2- 172.21.15.5:0/3332925759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b50d0 0x7f308406a960 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.339+0000 7f308ac45700 1 -- 172.21.15.5:0/3332925759 >> 172.21.15.5:0/3332925759 conn(0x7f30840b5a30 msgr2=0x7f30840b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.339+0000 7f308ac45700 1 -- 172.21.15.5:0/3332925759 shutdown_connections 2024-05-08T23:27:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.339+0000 7f308ac45700 1 -- 172.21.15.5:0/3332925759 wait complete. 2024-05-08T23:27:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.341+0000 7f308ac45700 1 Processor -- start 2024-05-08T23:27:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.341+0000 7f308ac45700 1 -- start start 2024-05-08T23:27:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f308ac45700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30840b50d0 0x7f308415f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f308ac45700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b5550 0x7f30841e7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f308ac45700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f308415d8c0 0x7f308414ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f308ac45700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084160db0 con 0x7f30840b5550 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f308ac45700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308415fa40 con 0x7f30840b50d0 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f308ac45700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308415fd10 con 0x7f308415d8c0 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f3083fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b5550 0x7f30841e7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f3083fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b5550 0x7f30841e7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33478/0 (socket says 172.21.15.5:33478) 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f3083fff700 1 -- 172.21.15.5:0/3585572103 learned_addr learned my addr 172.21.15.5:0/3585572103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f30889e1700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30840b50d0 0x7f308415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f30891e2700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f308415d8c0 0x7f308414ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f3083fff700 1 -- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f308415d8c0 msgr2=0x7f308414ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f3083fff700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f308415d8c0 0x7f308414ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f3083fff700 1 -- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30840b50d0 msgr2=0x7f308415f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f3083fff700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30840b50d0 0x7f308415f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.342+0000 7f3083fff700 1 -- 172.21.15.5:0/3585572103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30780094a0 con 0x7f30840b5550 2024-05-08T23:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.343+0000 7f3083fff700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b5550 0x7f30841e7900 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f30780090b0 tx=0x7f3078004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.343+0000 7f30817fa700 1 -- 172.21.15.5:0/3585572103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30780036a0 con 0x7f30840b5550 2024-05-08T23:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.343+0000 7f30817fa700 1 -- 172.21.15.5:0/3585572103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307801f070 con 0x7f30840b5550 2024-05-08T23:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.343+0000 7f30817fa700 1 -- 172.21.15.5:0/3585572103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078003a60 con 0x7f30840b5550 2024-05-08T23:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.343+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308415fff0 con 0x7f30840b5550 2024-05-08T23:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.343+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841605b0 con 0x7f30840b5550 2024-05-08T23:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.344+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3084074bc0 con 0x7f30840b5550 2024-05-08T23:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.344+0000 7f30817fa700 1 -- 172.21.15.5:0/3585572103 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3078003bc0 con 0x7f30840b5550 2024-05-08T23:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.345+0000 7f30817fa700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f307006ac90 0x7f307006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.345+0000 7f30817fa700 1 -- 172.21.15.5:0/3585572103 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f3078021070 con 0x7f30840b5550 2024-05-08T23:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.345+0000 7f30889e1700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f307006ac90 0x7f307006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.346+0000 7f30889e1700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f307006ac90 0x7f307006d150 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3074005910 tx=0x7f3074009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:51.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.350+0000 7f30817fa700 1 -- 172.21.15.5:0/3585572103 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3078055ed0 con 0x7f30840b5550 2024-05-08T23:27:51.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.563+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f308406b640 con 0x7f30840b5550 2024-05-08T23:27:51.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.564+0000 7f30817fa700 1 -- 172.21.15.5:0/3585572103 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f307802b020 con 0x7f30840b5550 2024-05-08T23:27:51.564 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:27:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f307006ac90 msgr2=0x7f307006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f307006ac90 0x7f307006d150 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3074005910 tx=0x7f3074009040 comp rx=0 tx=0).stop 2024-05-08T23:27:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b5550 msgr2=0x7f30841e7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b5550 0x7f30841e7900 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f30780090b0 tx=0x7f3078004750 comp rx=0 tx=0).stop 2024-05-08T23:27:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 shutdown_connections 2024-05-08T23:27:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f307006ac90 0x7f307006d150 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f308415d8c0 0x7f308414ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30840b50d0 0x7f308415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 --2- 172.21.15.5:0/3585572103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30840b5550 0x7f30841e7900 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 >> 172.21.15.5:0/3585572103 conn(0x7f30840b5a30 msgr2=0x7f30841593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 shutdown_connections 2024-05-08T23:27:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.566+0000 7f308ac45700 1 -- 172.21.15.5:0/3585572103 wait complete. 2024-05-08T23:27:51.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-05-08T23:27:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.715+0000 7fdaff1ca700 1 Processor -- start 2024-05-08T23:27:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.716+0000 7fdaff1ca700 1 -- start start 2024-05-08T23:27:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.716+0000 7fdaff1ca700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdaf815ef60 0x7fdaf80b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.716+0000 7fdaff1ca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf80cf000 0x7fdaf80cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.716+0000 7fdaff1ca700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdaf80cf9a0 0x7fdaf8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.716+0000 7fdaff1ca700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf80cd400 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.716+0000 7fdaff1ca700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf80cfe30 con 0x7fdaf815ef60 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.716+0000 7fdaff1ca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf809ad40 con 0x7fdaf80cf000 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdaf80cf9a0 0x7fdaf8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdaf80cf9a0 0x7fdaf8099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33492/0 (socket says 172.21.15.5:33492) 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 -- 172.21.15.5:0/533573454 learned_addr learned my addr 172.21.15.5:0/533573454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafcf66700 1 --2- 172.21.15.5:0/533573454 >> v2:172.21.15.49:3300/0 conn(0x7fdaf815ef60 0x7fdaf80b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdaf7fff700 1 --2- 172.21.15.5:0/533573454 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf80cf000 0x7fdaf80cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 -- 172.21.15.5:0/533573454 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf80cf000 msgr2=0x7fdaf80cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 --2- 172.21.15.5:0/533573454 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf80cf000 0x7fdaf80cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 -- 172.21.15.5:0/533573454 >> v2:172.21.15.49:3300/0 conn(0x7fdaf815ef60 msgr2=0x7fdaf80b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 --2- 172.21.15.5:0/533573454 >> v2:172.21.15.49:3300/0 conn(0x7fdaf815ef60 0x7fdaf80b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 -- 172.21.15.5:0/533573454 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf80ccb60 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdafd767700 1 --2- 172.21.15.5:0/533573454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf8099a50 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fdaf806e820 tx=0x7fdaf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405246d9320ace4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdaf77fe700 1 -- 172.21.15.5:0/533573454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000e070 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdaf77fe700 1 -- 172.21.15.5:0/533573454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf000b910 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.717+0000 7fdaf77fe700 1 -- 172.21.15.5:0/533573454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000bcc0 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 -- 172.21.15.5:0/533573454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 msgr2=0x7fdaf8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/533573454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf8099a50 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fdaf806e820 tx=0x7fdaf000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 -- 172.21.15.5:0/533573454 shutdown_connections 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/533573454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf8099a50 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/533573454 >> v2:172.21.15.49:3300/0 conn(0x7fdaf815ef60 0x7fdaf80b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/533573454 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf80cf000 0x7fdaf80cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 -- 172.21.15.5:0/533573454 >> 172.21.15.5:0/533573454 conn(0x7fdaf80b2f20 msgr2=0x7fdaf80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 -- 172.21.15.5:0/533573454 shutdown_connections 2024-05-08T23:27:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.718+0000 7fdaff1ca700 1 -- 172.21.15.5:0/533573454 wait complete. 2024-05-08T23:27:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.719+0000 7fdaff1ca700 1 Processor -- start 2024-05-08T23:27:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.719+0000 7fdaff1ca700 1 -- start start 2024-05-08T23:27:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaff1ca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf80cf000 0x7fdaf815b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaff1ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf81f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaff1ca700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf815ef60 0x7fdaf80c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaff1ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80c78c0 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaff1ca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80c4e30 con 0x7fdaf80cf000 2024-05-08T23:27:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaff1ca700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf815bd80 con 0x7fdaf815ef60 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaf7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf81f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaf7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf81f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33506/0 (socket says 172.21.15.5:33506) 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.720+0000 7fdaf7fff700 1 -- 172.21.15.5:0/4170984951 learned_addr learned my addr 172.21.15.5:0/4170984951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdafcf66700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf80cf000 0x7fdaf815b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdafd767700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf815ef60 0x7fdaf80c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf7fff700 1 -- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf815ef60 msgr2=0x7fdaf80c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf7fff700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf815ef60 0x7fdaf80c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf7fff700 1 -- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf80cf000 msgr2=0x7fdaf815b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf7fff700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf80cf000 0x7fdaf815b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf7fff700 1 -- 172.21.15.5:0/4170984951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0009b30 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdafcf66700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf80cf000 0x7fdaf815b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf7fff700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf81f0d50 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e9b0 tx=0x7fdae000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf57fa700 1 -- 172.21.15.5:0/4170984951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0009710 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf57fa700 1 -- 172.21.15.5:0/4170984951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae000c850 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaf57fa700 1 -- 172.21.15.5:0/4170984951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000cbc0 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf80c50d0 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.721+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf80c5690 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.722+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf8067bc0 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.723+0000 7fdaf57fa700 1 -- 172.21.15.5:0/4170984951 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdae000cd20 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.723+0000 7fdaf57fa700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdad806ac90 0x7fdad806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.723+0000 7fdaf57fa700 1 -- 172.21.15.5:0/4170984951 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fdae0089530 con 0x7fdaf80cf9a0 2024-05-08T23:27:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.724+0000 7fdafcf66700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdad806ac90 0x7fdad806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.724+0000 7fdafcf66700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdad806ac90 0x7fdad806d150 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009b10 tx=0x7fdae8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:51.728+0000 7fdaf57fa700 1 -- 172.21.15.5:0/4170984951 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdae00549f0 con 0x7fdaf80cf9a0 2024-05-08T23:27:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.046+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) v1 -- 0x7fdaf8068000 con 0x7fdaf80cf9a0 2024-05-08T23:27:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.793+0000 7fdaf57fa700 1 -- 172.21.15.5:0/4170984951 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v353) v1 ==== 168+0+0 (secure 0 0 0) 0x7fdae0058010 con 0x7fdaf80cf9a0 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.796+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdad806ac90 msgr2=0x7fdad806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.796+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdad806ac90 0x7fdad806d150 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009b10 tx=0x7fdae8008040 comp rx=0 tx=0).stop 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.796+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 msgr2=0x7fdaf81f0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.796+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf81f0d50 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e9b0 tx=0x7fdae000ed70 comp rx=0 tx=0).stop 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.797+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 shutdown_connections 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.797+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdad806ac90 0x7fdad806d150 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.797+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdaf815ef60 0x7fdaf80c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.797+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaf80cf000 0x7fdaf815b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.797+0000 7fdaff1ca700 1 --2- 172.21.15.5:0/4170984951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf80cf9a0 0x7fdaf81f0d50 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.797+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 >> 172.21.15.5:0/4170984951 conn(0x7fdaf80b2f20 msgr2=0x7fdaf815cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.797+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 shutdown_connections 2024-05-08T23:27:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.797+0000 7fdaff1ca700 1 -- 172.21.15.5:0/4170984951 wait complete. 2024-05-08T23:27:52.814 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T23:27:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.960+0000 7f88840db700 1 Processor -- start 2024-05-08T23:27:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.960+0000 7f88840db700 1 -- start start 2024-05-08T23:27:52.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f88840db700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f887c06b070 0x7f887c0b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:52.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f88840db700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c162d40 0x7f887c1633f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:52.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f88840db700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0b16e0 con 0x7f887c14b930 2024-05-08T23:27:52.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f88840db700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c1631b0 con 0x7f887c06b070 2024-05-08T23:27:52.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f88840db700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0b1980 con 0x7f887c162d40 2024-05-08T23:27:52.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f8881676700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f887c06b070 0x7f887c0b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:52.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f8882678700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c162d40 0x7f887c1633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:52.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f8881676700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f887c06b070 0x7f887c0b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56260/0 (socket says 172.21.15.5:56260) 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f8881e77700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f887c14b930 0x7f887c06a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:60840/0 (socket says 172.21.15.5:60840) 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.961+0000 7f8881676700 1 -- 172.21.15.5:0/1322028771 learned_addr learned my addr 172.21.15.5:0/1322028771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8882678700 1 -- 172.21.15.5:0/1322028771 >> v2:172.21.15.49:3300/0 conn(0x7f887c06b070 msgr2=0x7f887c0b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8882678700 1 --2- 172.21.15.5:0/1322028771 >> v2:172.21.15.49:3300/0 conn(0x7f887c06b070 0x7f887c0b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8882678700 1 -- 172.21.15.5:0/1322028771 >> v1:172.21.15.49:6789/0 conn(0x7f887c14b930 legacy=0x7f887c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8882678700 1 -- 172.21.15.5:0/1322028771 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0b1c60 con 0x7f887c162d40 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8881676700 1 --2- 172.21.15.5:0/1322028771 >> v2:172.21.15.49:3300/0 conn(0x7f887c06b070 0x7f887c0b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8882678700 1 --2- 172.21.15.5:0/1322028771 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c162d40 0x7f887c1633f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8874009d90 tx=0x7f887400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f62a1c06b4aa1c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8880e75700 1 -- 172.21.15.5:0/1322028771 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400e070 con 0x7f887c162d40 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8880e75700 1 -- 172.21.15.5:0/1322028771 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887400bb20 con 0x7f887c162d40 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.962+0000 7f8880e75700 1 -- 172.21.15.5:0/1322028771 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874012430 con 0x7f887c162d40 2024-05-08T23:27:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.963+0000 7f88840db700 1 -- 172.21.15.5:0/1322028771 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c162d40 msgr2=0x7f887c1633f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:52.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.963+0000 7f88840db700 1 --2- 172.21.15.5:0/1322028771 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c162d40 0x7f887c1633f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8874009d90 tx=0x7f887400b330 comp rx=0 tx=0).stop 2024-05-08T23:27:52.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.963+0000 7f88840db700 1 -- 172.21.15.5:0/1322028771 shutdown_connections 2024-05-08T23:27:52.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.963+0000 7f88840db700 1 --2- 172.21.15.5:0/1322028771 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c162d40 0x7f887c1633f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.963+0000 7f88840db700 1 --2- 172.21.15.5:0/1322028771 >> v2:172.21.15.49:3300/0 conn(0x7f887c06b070 0x7f887c0b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.963+0000 7f88840db700 1 -- 172.21.15.5:0/1322028771 >> 172.21.15.5:0/1322028771 conn(0x7f887c099a50 msgr2=0x7f887c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:52.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.963+0000 7f88840db700 1 -- 172.21.15.5:0/1322028771 shutdown_connections 2024-05-08T23:27:52.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.963+0000 7f88840db700 1 -- 172.21.15.5:0/1322028771 wait complete. 2024-05-08T23:27:52.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.964+0000 7f88840db700 1 Processor -- start 2024-05-08T23:27:52.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.964+0000 7f88840db700 1 -- start start 2024-05-08T23:27:52.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f88840db700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c06b070 0x7f887c147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:52.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f88840db700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c14b930 0x7f887c1e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:52.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f88840db700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c162d40 0x7f887c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:52.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f88840db700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1e8a50 con 0x7f887c14b930 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f88840db700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1eb300 con 0x7f887c162d40 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f88840db700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1e9f30 con 0x7f887c06b070 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f8881676700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c14b930 0x7f887c1e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f8881676700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c14b930 0x7f887c1e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33514/0 (socket says 172.21.15.5:33514) 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.965+0000 7f8881676700 1 -- 172.21.15.5:0/1470021687 learned_addr learned my addr 172.21.15.5:0/1470021687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f8882678700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c162d40 0x7f887c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f8881e77700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c06b070 0x7f887c147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f8881676700 1 -- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c06b070 msgr2=0x7f887c147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f8881676700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c06b070 0x7f887c147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f8881676700 1 -- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c162d40 msgr2=0x7f887c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f8881676700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c162d40 0x7f887c0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:52.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f8881676700 1 -- 172.21.15.5:0/1470021687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8878009550 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f8881676700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c14b930 0x7f887c1e99f0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f887800ebe0 tx=0x7f887800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f88727fc700 1 -- 172.21.15.5:0/1470021687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887800c850 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f88727fc700 1 -- 172.21.15.5:0/1470021687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887800c9b0 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8874009a70 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f88727fc700 1 -- 172.21.15.5:0/1470021687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887800cd40 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.966+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c1e8f10 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.967+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c074bc0 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.968+0000 7f88727fc700 1 -- 172.21.15.5:0/1470021687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8878013470 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.968+0000 7f88727fc700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f886006acc0 0x7f886006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.968+0000 7f88727fc700 1 -- 172.21.15.5:0/1470021687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8878089300 con 0x7f887c14b930 2024-05-08T23:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.968+0000 7f8881e77700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f886006acc0 0x7f886006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:52.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.969+0000 7f8881e77700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f886006acc0 0x7f886006d180 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f886c000f60 tx=0x7f886c007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:52.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:52.973+0000 7f88727fc700 1 -- 172.21.15.5:0/1470021687 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f88780546a0 con 0x7f887c14b930 2024-05-08T23:27:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.191+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f887c073c90 con 0x7f887c14b930 2024-05-08T23:27:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.195+0000 7f88727fc700 1 -- 172.21.15.5:0/1470021687 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8878057cc0 con 0x7f887c14b930 2024-05-08T23:27:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f886006acc0 msgr2=0x7f886006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f886006acc0 0x7f886006d180 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f886c000f60 tx=0x7f886c007ed0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c14b930 msgr2=0x7f887c1e99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c14b930 0x7f887c1e99f0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f887800ebe0 tx=0x7f887800eef0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 shutdown_connections 2024-05-08T23:27:53.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f886006acc0 0x7f886006d180 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c162d40 0x7f887c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f887c06b070 0x7f887c147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 --2- 172.21.15.5:0/1470021687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c14b930 0x7f887c1e99f0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 >> 172.21.15.5:0/1470021687 conn(0x7f887c099a50 msgr2=0x7f887c149fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:53.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 shutdown_connections 2024-05-08T23:27:53.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.197+0000 7f88840db700 1 -- 172.21.15.5:0/1470021687 wait complete. 2024-05-08T23:27:53.211 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:27:53.211 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:27:53.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:27:53.309 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:27:53.310 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:27:53.310 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:53.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:27:53.331 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:27:53.331 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:53.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:27:53.352 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:27:53.353 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:53.394 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:27:53.394 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:27:53.394 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:27:53.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:27:53.415 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T23:27:53.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.580+0000 7f9f6e359700 1 Processor -- start 2024-05-08T23:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.581+0000 7f9f6e359700 1 -- start start 2024-05-08T23:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.581+0000 7f9f6e359700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f6815a630 0x7f9f680d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.581+0000 7f9f6e359700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9f680cdef0 0x7f9f680cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.581+0000 7f9f6e359700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9f680cc040 0x7f9f6815e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.581+0000 7f9f6e359700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f680ce740 con 0x7f9f680cdef0 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.581+0000 7f9f6e359700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f680cb670 con 0x7f9f680cc040 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.581+0000 7f9f6e359700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680cadc0 con 0x7f9f6815a630 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9f680cdef0 0x7f9f680cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9f680cdef0 0x7f9f680cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33522/0 (socket says 172.21.15.5:33522) 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 -- 172.21.15.5:0/2048259564 learned_addr learned my addr 172.21.15.5:0/2048259564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f6c8f6700 1 --2- 172.21.15.5:0/2048259564 >> v2:172.21.15.49:3300/0 conn(0x7f9f680cc040 0x7f9f6815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f67fff700 1 --2- 172.21.15.5:0/2048259564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f6815a630 0x7f9f680d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 -- 172.21.15.5:0/2048259564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f6815a630 msgr2=0x7f9f680d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 --2- 172.21.15.5:0/2048259564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f6815a630 0x7f9f680d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 -- 172.21.15.5:0/2048259564 >> v2:172.21.15.49:3300/0 conn(0x7f9f680cc040 msgr2=0x7f9f6815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 --2- 172.21.15.5:0/2048259564 >> v2:172.21.15.49:3300/0 conn(0x7f9f680cc040 0x7f9f6815e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 -- 172.21.15.5:0/2048259564 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680cb060 con 0x7f9f680cdef0 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f677fe700 1 --2- 172.21.15.5:0/2048259564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f680cdef0 0x7f9f680cbb00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f9f6809e420 tx=0x7f9f5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c460d3905045c21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f66ffd700 1 -- 172.21.15.5:0/2048259564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00f8a0 con 0x7f9f680cdef0 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f66ffd700 1 -- 172.21.15.5:0/2048259564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c00fa00 con 0x7f9f680cdef0 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.582+0000 7f9f66ffd700 1 -- 172.21.15.5:0/2048259564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00fd90 con 0x7f9f680cdef0 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 -- 172.21.15.5:0/2048259564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f680cdef0 msgr2=0x7f9f680cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 --2- 172.21.15.5:0/2048259564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f680cdef0 0x7f9f680cbb00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f9f6809e420 tx=0x7f9f5c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 -- 172.21.15.5:0/2048259564 shutdown_connections 2024-05-08T23:27:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 --2- 172.21.15.5:0/2048259564 >> v2:172.21.15.49:3300/0 conn(0x7f9f680cc040 0x7f9f6815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 --2- 172.21.15.5:0/2048259564 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f6815a630 0x7f9f680d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 --2- 172.21.15.5:0/2048259564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f680cdef0 0x7f9f680cbb00 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 -- 172.21.15.5:0/2048259564 >> 172.21.15.5:0/2048259564 conn(0x7f9f68099a50 msgr2=0x7f9f680d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 -- 172.21.15.5:0/2048259564 shutdown_connections 2024-05-08T23:27:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.583+0000 7f9f6e359700 1 -- 172.21.15.5:0/2048259564 wait complete. 2024-05-08T23:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.584+0000 7f9f6e359700 1 Processor -- start 2024-05-08T23:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6e359700 1 -- start start 2024-05-08T23:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6e359700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f680cc040 0x7f9f6815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6e359700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cdef0 0x7f9f680c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6e359700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f6815a630 0x7f9f680c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6e359700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680c33b0 con 0x7f9f6815a630 2024-05-08T23:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6e359700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680c8520 con 0x7f9f680cdef0 2024-05-08T23:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6e359700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680c8800 con 0x7f9f680cc040 2024-05-08T23:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6c8f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f6815a630 0x7f9f680c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6c8f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f6815a630 0x7f9f680c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33536/0 (socket says 172.21.15.5:33536) 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.585+0000 7f9f6c8f6700 1 -- 172.21.15.5:0/3282347828 learned_addr learned my addr 172.21.15.5:0/3282347828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f677fe700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cdef0 0x7f9f680c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f67fff700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f680cc040 0x7f9f6815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f6c8f6700 1 -- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f680cc040 msgr2=0x7f9f6815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f6c8f6700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f680cc040 0x7f9f6815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f6c8f6700 1 -- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cdef0 msgr2=0x7f9f680c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f6c8f6700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cdef0 0x7f9f680c2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f6c8f6700 1 -- 172.21.15.5:0/3282347828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c009550 con 0x7f9f6815a630 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f6c8f6700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f6815a630 0x7f9f680c7bb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000eed0 tx=0x7f9f6000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f64ff9700 1 -- 172.21.15.5:0/3282347828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60010040 con 0x7f9f6815a630 2024-05-08T23:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f64ff9700 1 -- 172.21.15.5:0/3282347828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6000bc80 con 0x7f9f6815a630 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f680c8ae0 con 0x7f9f6815a630 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f64ff9700 1 -- 172.21.15.5:0/3282347828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60009e40 con 0x7f9f6815a630 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.586+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f680c90a0 con 0x7f9f6815a630 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.587+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f680c9290 con 0x7f9f6815a630 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.588+0000 7f9f64ff9700 1 -- 172.21.15.5:0/3282347828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9f60014400 con 0x7f9f6815a630 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.588+0000 7f9f64ff9700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f5406ac90 0x7f9f5406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.589+0000 7f9f64ff9700 1 -- 172.21.15.5:0/3282347828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9f60089380 con 0x7f9f6815a630 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.589+0000 7f9f67fff700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f5406ac90 0x7f9f5406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.589+0000 7f9f67fff700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f5406ac90 0x7f9f5406d150 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009b40 tx=0x7f9f58009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.593+0000 7f9f64ff9700 1 -- 172.21.15.5:0/3282347828 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9f60054840 con 0x7f9f6815a630 2024-05-08T23:27:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.865+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f68068000 con 0x7f9f6815a630 2024-05-08T23:27:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.865+0000 7f9f64ff9700 1 -- 172.21.15.5:0/3282347828 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f9f60018de0 con 0x7f9f6815a630 2024-05-08T23:27:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:53.866 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/2918399549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2918399549},{"type":"v1","addr":"172.21.15.49:6836","nonce":2918399549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6983,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1419257670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":1419257670},{"type":"v1","addr":"172.21.15.49:6837","nonce":1419257670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7954,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3629473431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3629473431},{"type":"v1","addr":"172.21.15.5:6835","nonce":3629473431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/3112121847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3112121847},{"type":"v1","addr":"172.21.15.5:6833","nonce":3112121847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[]} 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f5406ac90 msgr2=0x7f9f5406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f5406ac90 0x7f9f5406d150 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009b40 tx=0x7f9f58009210 comp rx=0 tx=0).stop 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f6815a630 msgr2=0x7f9f680c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f6815a630 0x7f9f680c7bb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000eed0 tx=0x7f9f6000b390 comp rx=0 tx=0).stop 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 shutdown_connections 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9f5406ac90 0x7f9f5406d150 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f6815a630 0x7f9f680c7bb0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9f680cc040 0x7f9f6815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 --2- 172.21.15.5:0/3282347828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cdef0 0x7f9f680c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 >> 172.21.15.5:0/3282347828 conn(0x7f9f68099a50 msgr2=0x7f9f680d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 shutdown_connections 2024-05-08T23:27:53.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:53.868+0000 7f9f6e359700 1 -- 172.21.15.5:0/3282347828 wait complete. 2024-05-08T23:27:53.872 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 94 2024-05-08T23:27:53.883 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:27:53.883 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T23:27:53.883 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:27:53.883 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:27:53.883 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:27:53.883 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T23:27:53.884 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:27:53.884 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:27:53.884 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:27:53.884 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T23:27:53.884 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:27:53.884 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:27:53.884 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:27:53.884 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T23:27:53.885 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:27:53.885 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:27:53.896 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:27:53.896 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:27:53.897 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:27:53.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:27:53.898 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:27:53.898 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:27:53.898 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:27:53.898 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:27:53.938 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:27:53.939 DEBUG:teuthology.parallel:result is None 2024-05-08T23:27:53.939 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:27:53.940 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:27:53.940 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:27:53.940 DEBUG:teuthology.parallel:result is None 2024-05-08T23:27:53.940 DEBUG:teuthology.parallel:result is None 2024-05-08T23:27:53.941 DEBUG:teuthology.parallel:result is None 2024-05-08T23:27:53.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T23:27:54.029 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:27:54.030 INFO:tasks.ceph.mds.d.smithi049.stdout:starting mds.d at 2024-05-08T23:27:54.031 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:27:54.031 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:27:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.089+0000 7f7ba08ee700 1 Processor -- start 2024-05-08T23:27:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.089+0000 7f7ba08ee700 1 -- start start 2024-05-08T23:27:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.090+0000 7f7ba08ee700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7b9c15b900 0x7f7b9c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.090+0000 7f7ba08ee700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c09a610 0x7f7b9c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.090+0000 7f7ba08ee700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c0b1410 con 0x7f7b9c099a50 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.090+0000 7f7ba08ee700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c0b0e30 con 0x7f7b9c15b900 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.090+0000 7f7ba08ee700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0cffd0 con 0x7f7b9c09a610 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9a59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7b9c15b900 0x7f7b9c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9a59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7b9c15b900 0x7f7b9c0cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33592/0 (socket says 172.21.15.5:33592) 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9a59c700 1 -- 172.21.15.5:0/711902448 learned_addr learned my addr 172.21.15.5:0/711902448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9ad9d700 1 --2- 172.21.15.5:0/711902448 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c09a610 0x7f7b9c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9a59c700 1 -- 172.21.15.5:0/711902448 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c09a610 msgr2=0x7f7b9c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9a59c700 1 --2- 172.21.15.5:0/711902448 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c09a610 0x7f7b9c0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9a59c700 1 -- 172.21.15.5:0/711902448 >> v1:172.21.15.49:6789/0 conn(0x7f7b9c099a50 legacy=0x7f7b9c099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9a59c700 1 -- 172.21.15.5:0/711902448 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0b10a0 con 0x7f7b9c15b900 2024-05-08T23:27:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9a59c700 1 --2- 172.21.15.5:0/711902448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c15b900 0x7f7b9c0cf9e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0098f0 tx=0x7f7b8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b1ade9e95c0bfa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9959a700 1 -- 172.21.15.5:0/711902448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c011830 con 0x7f7b9c15b900 2024-05-08T23:27:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9959a700 1 -- 172.21.15.5:0/711902448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c011990 con 0x7f7b9c15b900 2024-05-08T23:27:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.091+0000 7f7b9959a700 1 -- 172.21.15.5:0/711902448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c011d20 con 0x7f7b9c15b900 2024-05-08T23:27:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.092+0000 7f7ba08ee700 1 -- 172.21.15.5:0/711902448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c15b900 msgr2=0x7f7b9c0cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.092+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/711902448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c15b900 0x7f7b9c0cf9e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0098f0 tx=0x7f7b8c009c00 comp rx=0 tx=0).stop 2024-05-08T23:27:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.092+0000 7f7ba08ee700 1 -- 172.21.15.5:0/711902448 shutdown_connections 2024-05-08T23:27:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.092+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/711902448 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c09a610 0x7f7b9c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.092+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/711902448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c15b900 0x7f7b9c0cf9e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.092+0000 7f7ba08ee700 1 -- 172.21.15.5:0/711902448 >> 172.21.15.5:0/711902448 conn(0x7f7b9c0b2f20 msgr2=0x7f7b9c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.092+0000 7f7ba08ee700 1 -- 172.21.15.5:0/711902448 shutdown_connections 2024-05-08T23:27:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.092+0000 7f7ba08ee700 1 -- 172.21.15.5:0/711902448 wait complete. 2024-05-08T23:27:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7ba08ee700 1 Processor -- start 2024-05-08T23:27:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7ba08ee700 1 -- start start 2024-05-08T23:27:54.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7ba08ee700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c099a50 0x7f7b9c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:54.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7ba08ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c09a610 0x7f7b9c0b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:54.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7ba08ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b9c15b900 0x7f7b9c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:54.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7ba08ee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c15cbd0 con 0x7f7b9c09a610 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7ba08ee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c15ceb0 con 0x7f7b9c15b900 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7ba08ee700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c15d190 con 0x7f7b9c099a50 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.094+0000 7f7b99d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c09a610 0x7f7b9c0b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b99d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c09a610 0x7f7b9c0b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33600/0 (socket says 172.21.15.5:33600) 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b99d9b700 1 -- 172.21.15.5:0/2273843649 learned_addr learned my addr 172.21.15.5:0/2273843649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b9a59c700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c099a50 0x7f7b9c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b9ad9d700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b9c15b900 0x7f7b9c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b99d9b700 1 -- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c099a50 msgr2=0x7f7b9c1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b99d9b700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c099a50 0x7f7b9c1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b99d9b700 1 -- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b9c15b900 msgr2=0x7f7b9c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b99d9b700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b9c15b900 0x7f7b9c0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b99d9b700 1 -- 172.21.15.5:0/2273843649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c009550 con 0x7f7b9c09a610 2024-05-08T23:27:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b99d9b700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c09a610 0x7f7b9c0b5ea0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000c4e0 tx=0x7f7b9000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b8affd700 1 -- 172.21.15.5:0/2273843649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900052d0 con 0x7f7b9c09a610 2024-05-08T23:27:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b8affd700 1 -- 172.21.15.5:0/2273843649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b90005430 con 0x7f7b9c09a610 2024-05-08T23:27:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c15d470 con 0x7f7b9c09a610 2024-05-08T23:27:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7b8affd700 1 -- 172.21.15.5:0/2273843649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900038c0 con 0x7f7b9c09a610 2024-05-08T23:27:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.095+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c0b63e0 con 0x7f7b9c09a610 2024-05-08T23:27:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.096+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c068000 con 0x7f7b9c09a610 2024-05-08T23:27:54.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.097+0000 7f7b8affd700 1 -- 172.21.15.5:0/2273843649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7b90003a20 con 0x7f7b9c09a610 2024-05-08T23:27:54.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.097+0000 7f7b8affd700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b7c06ac90 0x7f7b7c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:54.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.097+0000 7f7b8affd700 1 -- 172.21.15.5:0/2273843649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7b90088610 con 0x7f7b9c09a610 2024-05-08T23:27:54.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.098+0000 7f7b9a59c700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b7c06ac90 0x7f7b7c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:54.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.098+0000 7f7b9a59c700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b7c06ac90 0x7f7b7c06d150 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c014040 tx=0x7f7b8c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:54.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.102+0000 7f7b8affd700 1 -- 172.21.15.5:0/2273843649 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7b900539c0 con 0x7f7b9c09a610 2024-05-08T23:27:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:54.337+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7b9c0b35e0 con 0x7f7b9c09a610 2024-05-08T23:27:55.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.198+0000 7f7b8affd700 1 -- 172.21.15.5:0/2273843649 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7b90056fe0 con 0x7f7b9c09a610 2024-05-08T23:27:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b7c06ac90 msgr2=0x7f7b7c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b7c06ac90 0x7f7b7c06d150 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c014040 tx=0x7f7b8c008030 comp rx=0 tx=0).stop 2024-05-08T23:27:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c09a610 msgr2=0x7f7b9c0b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c09a610 0x7f7b9c0b5ea0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000c4e0 tx=0x7f7b9000c8a0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 shutdown_connections 2024-05-08T23:27:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7b7c06ac90 0x7f7b7c06d150 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b9c15b900 0x7f7b9c0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7b9c099a50 0x7f7b9c1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 --2- 172.21.15.5:0/2273843649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c09a610 0x7f7b9c0b5ea0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 >> 172.21.15.5:0/2273843649 conn(0x7f7b9c0b2f20 msgr2=0x7f7b9c070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 shutdown_connections 2024-05-08T23:27:55.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.201+0000 7f7ba08ee700 1 -- 172.21.15.5:0/2273843649 wait complete. 2024-05-08T23:27:55.202 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-08T23:27:55.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T23:27:55.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.358+0000 7fa5a0fb3700 1 Processor -- start 2024-05-08T23:27:55.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.358+0000 7fa5a0fb3700 1 -- start start 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa5a0fb3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa59c0cecf0 0x7fa59c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa5a0fb3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa59c06f130 0x7fa59c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa5a0fb3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59c15c280 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa5a0fb3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c06e780 con 0x7fa59c0cecf0 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa5a0fb3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c06ea60 con 0x7fa59c06f130 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa599d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa59c0cecf0 0x7fa59c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa59a59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa59c0cd4e0 0x7fa59c0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43956/0 (socket says 172.21.15.5:43956) 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa59a59c700 1 -- 172.21.15.5:0/1461951764 learned_addr learned my addr 172.21.15.5:0/1461951764 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa59ad9d700 1 --2- 172.21.15.5:0/1461951764 >> v2:172.21.15.49:3300/0 conn(0x7fa59c06f130 0x7fa59c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa599d9b700 1 -- 172.21.15.5:0/1461951764 >> v2:172.21.15.49:3300/0 conn(0x7fa59c06f130 msgr2=0x7fa59c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa599d9b700 1 --2- 172.21.15.5:0/1461951764 >> v2:172.21.15.49:3300/0 conn(0x7fa59c06f130 0x7fa59c15c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa599d9b700 1 -- 172.21.15.5:0/1461951764 >> v1:172.21.15.5:6789/0 conn(0x7fa59c0cd4e0 legacy=0x7fa59c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa599d9b700 1 -- 172.21.15.5:0/1461951764 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c06ed40 con 0x7fa59c0cecf0 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.359+0000 7fa59ad9d700 1 --2- 172.21.15.5:0/1461951764 >> v2:172.21.15.49:3300/0 conn(0x7fa59c06f130 0x7fa59c15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.360+0000 7fa599d9b700 1 --2- 172.21.15.5:0/1461951764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cecf0 0x7fa59c06f910 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fa59c15f520 tx=0x7fa58c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f85e0b1bec1f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.360+0000 7fa59959a700 1 -- 172.21.15.5:0/1461951764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00f8a0 con 0x7fa59c0cecf0 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.360+0000 7fa59959a700 1 -- 172.21.15.5:0/1461951764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00fa00 con 0x7fa59c0cecf0 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.360+0000 7fa59959a700 1 -- 172.21.15.5:0/1461951764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00fd90 con 0x7fa59c0cecf0 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.360+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/1461951764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cecf0 msgr2=0x7fa59c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.360+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/1461951764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cecf0 0x7fa59c06f910 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fa59c15f520 tx=0x7fa58c009b20 comp rx=0 tx=0).stop 2024-05-08T23:27:55.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.361+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/1461951764 shutdown_connections 2024-05-08T23:27:55.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.361+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/1461951764 >> v2:172.21.15.49:3300/0 conn(0x7fa59c06f130 0x7fa59c15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.361+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/1461951764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cecf0 0x7fa59c06f910 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.361+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/1461951764 >> 172.21.15.5:0/1461951764 conn(0x7fa59c0d0320 msgr2=0x7fa59c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.361+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/1461951764 shutdown_connections 2024-05-08T23:27:55.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.361+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/1461951764 wait complete. 2024-05-08T23:27:55.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.362+0000 7fa5a0fb3700 1 Processor -- start 2024-05-08T23:27:55.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.362+0000 7fa5a0fb3700 1 -- start start 2024-05-08T23:27:55.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa5a0fb3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59c06f130 0x7fa59c0c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa5a0fb3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cd4e0 0x7fa59c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa5a0fb3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa59c0cecf0 0x7fa59c0ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa5a0fb3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c158eb0 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa5a0fb3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c159190 con 0x7fa59c0cecf0 2024-05-08T23:27:55.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa5a0fb3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c159470 con 0x7fa59c06f130 2024-05-08T23:27:55.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa599d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cd4e0 0x7fa59c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa599d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cd4e0 0x7fa59c0b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33616/0 (socket says 172.21.15.5:33616) 2024-05-08T23:27:55.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa599d9b700 1 -- 172.21.15.5:0/294603073 learned_addr learned my addr 172.21.15.5:0/294603073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:55.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa59a59c700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59c06f130 0x7fa59c0c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa59ad9d700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa59c0cecf0 0x7fa59c0ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa599d9b700 1 -- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59c06f130 msgr2=0x7fa59c0c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa599d9b700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59c06f130 0x7fa59c0c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa599d9b700 1 -- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa59c0cecf0 msgr2=0x7fa59c0ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa599d9b700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa59c0cecf0 0x7fa59c0ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.363+0000 7fa599d9b700 1 -- 172.21.15.5:0/294603073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c0094a0 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.364+0000 7fa599d9b700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cd4e0 0x7fa59c0b57a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa58c000c00 tx=0x7fa58c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.364+0000 7fa58affd700 1 -- 172.21.15.5:0/294603073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c01f070 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.364+0000 7fa58affd700 1 -- 172.21.15.5:0/294603073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c0036a0 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.364+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c159750 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.364+0000 7fa58affd700 1 -- 172.21.15.5:0/294603073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c003a30 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.364+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c0b3120 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.365+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c068000 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.365+0000 7fa58affd700 1 -- 172.21.15.5:0/294603073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa58c003b90 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.366+0000 7fa58affd700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57c06ac90 0x7fa57c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.366+0000 7fa58affd700 1 -- 172.21.15.5:0/294603073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa58c08bc40 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.366+0000 7fa59a59c700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57c06ac90 0x7fa57c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.367+0000 7fa59a59c700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57c06ac90 0x7fa57c06d150 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa590005910 tx=0x7fa590009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.371+0000 7fa58affd700 1 -- 172.21.15.5:0/294603073 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa58c057180 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.679+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa59c06d440 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.680+0000 7fa58affd700 1 -- 172.21.15.5:0/294603073 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa58c05a7a0 con 0x7fa59c0cd4e0 2024-05-08T23:27:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:55.680 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:27:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-08T23:27:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm4+KLORAAJGma8zcoGLift7K5wyvWlA==", 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm5iMNDBAANT+UrtTUBpYyQ9dzJyS01Q==", 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmRixkBhAAk9s24ERPJ/6nCJY3g7QppA==", 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmPmurExAAxIzDzbLZ3s35XVJrvYiVCA==", 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-08T23:27:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCrCDxmI4/bNBAAzNXcuLQCS9gaAJ0r4iYKlA==", 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCuCDxmMS6HNhAAcTP6nStED06bdM4MtbEPjQ==", 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCxCDxmDGv5NRAAljFQZXLGWwt+UXZ+HCdx0Q==", 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-08T23:27:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC0CDxmL+5LNRAAVPySIUtICBwIokJL9fHFtw==", 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC5CDxm/F15BhAAEx9mKA9izSmb+xyxjEi9FA==", 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:27:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC8CDxm0zAYBhAAjHT+TZFqk5MWvd0P5LmwfA==", 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC/CDxmPcbiBRAAOmWNgaQGi6YGKTp8SJekUg==", 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDCCDxm9FsDBRAA/lfmrpmL4UO3s/tJJ4o6yQ==", 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-08T23:27:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm9tLAGBAAK8Lq89gJ902XqzXYAbYfkg==", 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmCStoIBAAhraObJ5LS9LtF97+hIcTFQ==", 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxm2NzHHBAAUbeD58MntIoiwT8LekjIlw==", 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCqCDxmyvveIxAATEn7nOPHqtqSbdfQid25HA==", 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-08T23:27:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm3ZIxDxAAM8M5YUfK9G2PkTzU4wY/+w==", 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-08T23:27:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxmntlgNRAA7JOIS+0NL1KmSSZhZjXlVQ==", 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpCDxm1dPQMBAAhu0CK6DcucQteuTo7qN2IQ==", 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-08T23:27:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-08T23:27:55.694 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57c06ac90 msgr2=0x7fa57c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57c06ac90 0x7fa57c06d150 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa590005910 tx=0x7fa590009040 comp rx=0 tx=0).stop 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cd4e0 msgr2=0x7fa59c0b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cd4e0 0x7fa59c0b57a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa58c000c00 tx=0x7fa58c004750 comp rx=0 tx=0).stop 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 shutdown_connections 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57c06ac90 0x7fa57c06d150 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa59c0cecf0 0x7fa59c0ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59c06f130 0x7fa59c0c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.682+0000 7fa5a0fb3700 1 --2- 172.21.15.5:0/294603073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0cd4e0 0x7fa59c0b57a0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.683+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 >> 172.21.15.5:0/294603073 conn(0x7fa59c0d0320 msgr2=0x7fa59c0d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.683+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 shutdown_connections 2024-05-08T23:27:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.683+0000 7fa5a0fb3700 1 -- 172.21.15.5:0/294603073 wait complete. 2024-05-08T23:27:55.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:27:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.833+0000 7f8726793700 1 Processor -- start 2024-05-08T23:27:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.833+0000 7f8726793700 1 -- start start 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.834+0000 7f8726793700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f872015ba30 0x7f8720099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.834+0000 7f8726793700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87200cf2f0 con 0x7f8720099f90 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.834+0000 7f8726793700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87200cfd00 con 0x7f87200cd0c0 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.834+0000 7f8726793700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872009a410 con 0x7f872015ba30 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.834+0000 7f871f7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8720099f90 0x7f87200b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43960/0 (socket says 172.21.15.5:43960) 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.834+0000 7f871f7fe700 1 -- 172.21.15.5:0/2600357831 learned_addr learned my addr 172.21.15.5:0/2600357831 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871ffff700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f872015ba30 0x7f8720099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79139804 0 0) 0x7f87200cf2f0 con 0x7f8720099f90 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f870c0035b0 con 0x7f8720099f90 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568097496 0 0) 0x7f870c0035b0 con 0x7f8720099f90 2024-05-08T23:27:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f872015ba30 msgr2=0x7f8720099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f872015ba30 0x7f8720099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 >> v1:172.21.15.49:6789/0 conn(0x7f87200cd0c0 legacy=0x7f87200b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200d02c0 con 0x7f8720099f90 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f8708001670 con 0x7f8720099f90 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 >> v1:172.21.15.5:6789/0 conn(0x7f8720099f90 legacy=0x7f87200b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f870c003a30 0x7f870c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f872015ba30 0x7f870c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f870c004d90 0x7f870c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872009a410 con 0x7f870c004d90 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c00bac0 con 0x7f870c003a30 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c00bc80 con 0x7f872015ba30 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f8724d30700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f870c004d90 0x7f870c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871ffff700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f870c003a30 0x7f870c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f871f7fe700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f872015ba30 0x7f870c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f8724d30700 1 -- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f872015ba30 msgr2=0x7f870c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f8724d30700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f872015ba30 0x7f870c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f8724d30700 1 -- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f870c003a30 msgr2=0x7f870c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f8724d30700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f870c003a30 0x7f870c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.835+0000 7f8724d30700 1 -- 172.21.15.5:0/2600357831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c0035b0 con 0x7f870c004d90 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f8724d30700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f870c004d90 0x7f870c005210 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f871800c160 tx=0x7f870c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a530fb7f50dd6658 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718001db0 con 0x7f870c004d90 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871800cca0 con 0x7f870c004d90 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f871effd700 1 -- 172.21.15.5:0/2600357831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718003c40 con 0x7f870c004d90 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f8726793700 1 -- 172.21.15.5:0/2600357831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f870c004d90 msgr2=0x7f870c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f8726793700 1 --2- 172.21.15.5:0/2600357831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f870c004d90 0x7f870c005210 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f871800c160 tx=0x7f870c005880 comp rx=0 tx=0).stop 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f8724d30700 1 -- 172.21.15.5:0/2600357831 reap_dead start 2024-05-08T23:27:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f8726793700 1 -- 172.21.15.5:0/2600357831 shutdown_connections 2024-05-08T23:27:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f8726793700 1 -- 172.21.15.5:0/2600357831 >> 172.21.15.5:0/2600357831 conn(0x7f87200b2ee0 msgr2=0x7f87200b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f8726793700 1 -- 172.21.15.5:0/2600357831 shutdown_connections 2024-05-08T23:27:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.836+0000 7f8726793700 1 -- 172.21.15.5:0/2600357831 wait complete. 2024-05-08T23:27:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.838+0000 7f8726793700 1 Processor -- start 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.838+0000 7f8726793700 1 -- start start 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f8726793700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8720158ec0 0x7f87200b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f8726793700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f872006d610 0x7f872006ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f8726793700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8720159780 0x7f8720159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f8726793700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872006bfb0 con 0x7f872006d610 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f8726793700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872015a140 con 0x7f8720158ec0 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f8726793700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87200c84c0 con 0x7f8720159780 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f872006d610 0x7f872006ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f872006d610 0x7f872006ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33644/0 (socket says 172.21.15.5:33644) 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 -- 172.21.15.5:0/658566013 learned_addr learned my addr 172.21.15.5:0/658566013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871ffff700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8720158ec0 0x7f87200b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f8724d30700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8720159780 0x7f8720159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 -- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8720159780 msgr2=0x7f8720159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8720159780 0x7f8720159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 -- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8720158ec0 msgr2=0x7f87200b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8720158ec0 0x7f87200b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 -- 172.21.15.5:0/658566013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872006da90 con 0x7f872006d610 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.839+0000 7f871f7fe700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f872006d610 0x7f872006ba70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f87080072c0 tx=0x7f8708007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.840+0000 7f871cff9700 1 -- 172.21.15.5:0/658566013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870800f870 con 0x7f872006d610 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.840+0000 7f871cff9700 1 -- 172.21.15.5:0/658566013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870800f9d0 con 0x7f872006d610 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.840+0000 7f871cff9700 1 -- 172.21.15.5:0/658566013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870800fd80 con 0x7f872006d610 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.840+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87201592a0 con 0x7f872006d610 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.840+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872006c480 con 0x7f872006d610 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.841+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8720074bf0 con 0x7f872006d610 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.841+0000 7f871cff9700 1 -- 172.21.15.5:0/658566013 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f870801d460 con 0x7f872006d610 2024-05-08T23:27:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.842+0000 7f871cff9700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870006ac90 0x7f870006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.842+0000 7f871cff9700 1 -- 172.21.15.5:0/658566013 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8708089000 con 0x7f872006d610 2024-05-08T23:27:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.842+0000 7f871ffff700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870006ac90 0x7f870006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.842+0000 7f871ffff700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870006ac90 0x7f870006d150 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8710009ba0 tx=0x7f8710008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:55.846+0000 7f871cff9700 1 -- 172.21.15.5:0/658566013 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f87080545c0 con 0x7f872006d610 2024-05-08T23:27:56.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.066+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8720068000 con 0x7f872006d610 2024-05-08T23:27:56.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.067+0000 7f871cff9700 1 -- 172.21.15.5:0/658566013 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+12294 (secure 0 0 0) 0x7f8708057be0 con 0x7f872006d610 2024-05-08T23:27:56.067 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:56.067 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:27:55.194813+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:27:56.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870006ac90 msgr2=0x7f870006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870006ac90 0x7f870006d150 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8710009ba0 tx=0x7f8710008040 comp rx=0 tx=0).stop 2024-05-08T23:27:56.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f872006d610 msgr2=0x7f872006ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f872006d610 0x7f872006ba70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f87080072c0 tx=0x7f8708007680 comp rx=0 tx=0).stop 2024-05-08T23:27:56.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 shutdown_connections 2024-05-08T23:27:56.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f870006ac90 0x7f870006d150 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8720159780 0x7f8720159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8720158ec0 0x7f87200b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 --2- 172.21.15.5:0/658566013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f872006d610 0x7f872006ba70 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 >> 172.21.15.5:0/658566013 conn(0x7f87200b2ee0 msgr2=0x7f87200aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.069+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 shutdown_connections 2024-05-08T23:27:56.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.070+0000 7f8726793700 1 -- 172.21.15.5:0/658566013 wait complete. 2024-05-08T23:27:56.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:27:56.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:27:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.226+0000 7f0cbeb47700 1 Processor -- start 2024-05-08T23:27:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.226+0000 7f0cbeb47700 1 -- start start 2024-05-08T23:27:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cbeb47700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0cb815b4e0 0x7f0cb80b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cbeb47700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb80cf560 con 0x7f0cb80d1910 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cbeb47700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb80ccff0 con 0x7f0cb815b8c0 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cbeb47700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb80cefe0 con 0x7f0cb815b4e0 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cbd0e4700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0cb80d1910 0x7f0cb80cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43976/0 (socket says 172.21.15.5:43976) 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cbd0e4700 1 -- 172.21.15.5:0/834576627 learned_addr learned my addr 172.21.15.5:0/834576627 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/834576627 >> v2:172.21.15.49:3300/0 conn(0x7f0cb815b4e0 0x7f0cb80b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192368317 0 0) 0x7f0cb80cf560 con 0x7f0cb80d1910 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ca80035b0 con 0x7f0cb80d1910 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354507037 0 0) 0x7f0cb80ccff0 con 0x7f0cb815b8c0 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.227+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cb80cf560 con 0x7f0cb815b8c0 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1454191708 0 0) 0x7f0ca80035b0 con 0x7f0cb80d1910 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 >> v2:172.21.15.49:3300/0 conn(0x7f0cb815b4e0 msgr2=0x7f0cb80b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 --2- 172.21.15.5:0/834576627 >> v2:172.21.15.49:3300/0 conn(0x7f0cb815b4e0 0x7f0cb80b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 >> v1:172.21.15.49:6789/0 conn(0x7f0cb815b8c0 legacy=0x7f0cb80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb80cd2d0 con 0x7f0cb80d1910 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f0cb00019f0 con 0x7f0cb80d1910 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 >> v1:172.21.15.5:6789/0 conn(0x7f0cb80d1910 legacy=0x7f0cb80cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ca8003be0 0x7f0ca8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8004850 0x7f0ca8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca8005210 0x7f0ca800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb80cefe0 con 0x7f0ca8003be0 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca800c0e0 con 0x7f0ca8005210 2024-05-08T23:27:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca800c2a0 con 0x7f0ca8004850 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/834576627 >> v2:172.21.15.49:3300/0 conn(0x7f0cb815b4e0 0x7f0cb80b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbd0e4700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca8005210 0x7f0ca800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ca8003be0 0x7f0ca8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cb7fff700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8004850 0x7f0ca8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbc8e3700 1 -- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8004850 msgr2=0x7f0ca8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8004850 0x7f0ca8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbc8e3700 1 -- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca8005210 msgr2=0x7f0ca800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca8005210 0x7f0ca800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbc8e3700 1 -- 172.21.15.5:0/834576627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80035b0 con 0x7f0ca8003be0 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbd0e4700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca8005210 0x7f0ca800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbd0e4700 1 -- 172.21.15.5:0/834576627 reap_dead start 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.228+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ca8003be0 0x7f0ca8003fe0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009d10 tx=0x7f0ca8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f22a4b81a83aef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac009e60 con 0x7f0ca8003be0 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac00e9a0 con 0x7f0ca8003be0 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cb77fe700 1 -- 172.21.15.5:0/834576627 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00ed30 con 0x7f0ca8003be0 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cbeb47700 1 -- 172.21.15.5:0/834576627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ca8003be0 msgr2=0x7f0ca8003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cbeb47700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ca8003be0 0x7f0ca8003fe0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009d10 tx=0x7f0ca8005880 comp rx=0 tx=0).stop 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cbeb47700 1 -- 172.21.15.5:0/834576627 shutdown_connections 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cbeb47700 1 --2- 172.21.15.5:0/834576627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ca8003be0 0x7f0ca8003fe0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cbeb47700 1 -- 172.21.15.5:0/834576627 >> 172.21.15.5:0/834576627 conn(0x7f0cb808afe0 msgr2=0x7f0cb808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cbeb47700 1 -- 172.21.15.5:0/834576627 shutdown_connections 2024-05-08T23:27:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.229+0000 7f0cbeb47700 1 -- 172.21.15.5:0/834576627 wait complete. 2024-05-08T23:27:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.231+0000 7f0cbeb47700 1 Processor -- start 2024-05-08T23:27:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.231+0000 7f0cbeb47700 1 -- start start 2024-05-08T23:27:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.232+0000 7f0cbeb47700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8003be0 0x7f0cb8157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.232+0000 7f0cbeb47700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cb80c6750 0x7f0cb8148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.232+0000 7f0cbeb47700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb806edd0 0x7f0cb806f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.232+0000 7f0cbeb47700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb806f7c0 con 0x7f0cb806edd0 2024-05-08T23:27:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.232+0000 7f0cbeb47700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb80c77e0 con 0x7f0cb80c6750 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.232+0000 7f0cbeb47700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb8148950 con 0x7f0ca8003be0 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.232+0000 7f0cbd0e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb806edd0 0x7f0cb806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbd0e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb806edd0 0x7f0cb806f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33668/0 (socket says 172.21.15.5:33668) 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbd0e4700 1 -- 172.21.15.5:0/2926488621 learned_addr learned my addr 172.21.15.5:0/2926488621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cb7fff700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cb80c6750 0x7f0cb8148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8003be0 0x7f0cb8157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbd0e4700 1 -- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8003be0 msgr2=0x7f0cb8157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbd0e4700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8003be0 0x7f0cb8157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbd0e4700 1 -- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cb80c6750 msgr2=0x7f0cb8148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbd0e4700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cb80c6750 0x7f0cb8148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbd0e4700 1 -- 172.21.15.5:0/2926488621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb80c6bd0 con 0x7f0cb806edd0 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbd0e4700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb806edd0 0x7f0cb806f280 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80cef60 tx=0x7f0cb000a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cb57fa700 1 -- 172.21.15.5:0/2926488621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0016070 con 0x7f0cb806edd0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cb57fa700 1 -- 172.21.15.5:0/2926488621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb000ae80 con 0x7f0cb806edd0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb80c6eb0 con 0x7f0cb806edd0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cb57fa700 1 -- 172.21.15.5:0/2926488621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00075b0 con 0x7f0cb806edd0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.233+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb80c7470 con 0x7f0cb806edd0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.234+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb80c7990 con 0x7f0cb806edd0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.235+0000 7f0cb57fa700 1 -- 172.21.15.5:0/2926488621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0cb0007710 con 0x7f0cb806edd0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.235+0000 7f0cb57fa700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0c9c06ac90 0x7f0c9c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.235+0000 7f0cb57fa700 1 -- 172.21.15.5:0/2926488621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0cb008d240 con 0x7f0cb806edd0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.235+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0c9c06ac90 0x7f0c9c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.236+0000 7f0cbc8e3700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0c9c06ac90 0x7f0c9c06d150 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0090b0 tx=0x7f0cac01e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.240+0000 7f0cb57fa700 1 -- 172.21.15.5:0/2926488621 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0cb80c7990 con 0x7f0cb806edd0 2024-05-08T23:27:56.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.454+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0cb80b17a0 con 0x7f0cb806edd0 2024-05-08T23:27:56.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.454+0000 7f0cb57fa700 1 -- 172.21.15.5:0/2926488621 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0cb0058770 con 0x7f0cb806edd0 2024-05-08T23:27:56.454 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0c9c06ac90 msgr2=0x7f0c9c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0c9c06ac90 0x7f0c9c06d150 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0090b0 tx=0x7f0cac01e040 comp rx=0 tx=0).stop 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb806edd0 msgr2=0x7f0cb806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb806edd0 0x7f0cb806f280 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80cef60 tx=0x7f0cb000a9c0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 shutdown_connections 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0c9c06ac90 0x7f0c9c06d150 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb806edd0 0x7f0cb806f280 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0ca8003be0 0x7f0cb8157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 --2- 172.21.15.5:0/2926488621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cb80c6750 0x7f0cb8148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.456+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 >> 172.21.15.5:0/2926488621 conn(0x7f0cb808afe0 msgr2=0x7f0cb806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.457+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 shutdown_connections 2024-05-08T23:27:56.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.457+0000 7f0cbeb47700 1 -- 172.21.15.5:0/2926488621 wait complete. 2024-05-08T23:27:56.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:27:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.606+0000 7f3080e3e700 1 Processor -- start 2024-05-08T23:27:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.607+0000 7f3080e3e700 1 -- start start 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.607+0000 7f3080e3e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c161030 0x7f307c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.607+0000 7f3080e3e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f307c0b2480 0x7f307c099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.607+0000 7f3080e3e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f307c09a4d0 0x7f307c0cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.607+0000 7f3080e3e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c06ccc0 con 0x7f307c09a4d0 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.607+0000 7f3080e3e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0d0840 con 0x7f307c0b2480 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.607+0000 7f3080e3e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c06c600 con 0x7f307c161030 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f307c09a4d0 0x7f307c0cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f307c09a4d0 0x7f307c0cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33682/0 (socket says 172.21.15.5:33682) 2024-05-08T23:27:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 -- 172.21.15.5:0/2128935145 learned_addr learned my addr 172.21.15.5:0/2128935145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f3071d9b700 1 --2- 172.21.15.5:0/2128935145 >> v2:172.21.15.49:3300/0 conn(0x7f307c0b2480 0x7f307c099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307a59c700 1 --2- 172.21.15.5:0/2128935145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c161030 0x7f307c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 -- 172.21.15.5:0/2128935145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c161030 msgr2=0x7f307c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 --2- 172.21.15.5:0/2128935145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c161030 0x7f307c099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 -- 172.21.15.5:0/2128935145 >> v2:172.21.15.49:3300/0 conn(0x7f307c0b2480 msgr2=0x7f307c099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 --2- 172.21.15.5:0/2128935145 >> v2:172.21.15.49:3300/0 conn(0x7f307c0b2480 0x7f307c099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 -- 172.21.15.5:0/2128935145 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c06c8e0 con 0x7f307c09a4d0 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f3071d9b700 1 --2- 172.21.15.5:0/2128935145 >> v2:172.21.15.49:3300/0 conn(0x7f307c0b2480 0x7f307c099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f307ad9d700 1 --2- 172.21.15.5:0/2128935145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c09a4d0 0x7f307c0cf650 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f307c0676a0 tx=0x7f307400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9e2e86d78a3faf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f3079d9b700 1 -- 172.21.15.5:0/2128935145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400e070 con 0x7f307c09a4d0 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f3079d9b700 1 -- 172.21.15.5:0/2128935145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400b9c0 con 0x7f307c09a4d0 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.608+0000 7f3079d9b700 1 -- 172.21.15.5:0/2128935145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400bd70 con 0x7f307c09a4d0 2024-05-08T23:27:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 -- 172.21.15.5:0/2128935145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c09a4d0 msgr2=0x7f307c0cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 --2- 172.21.15.5:0/2128935145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c09a4d0 0x7f307c0cf650 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f307c0676a0 tx=0x7f307400b330 comp rx=0 tx=0).stop 2024-05-08T23:27:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 -- 172.21.15.5:0/2128935145 shutdown_connections 2024-05-08T23:27:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 --2- 172.21.15.5:0/2128935145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c09a4d0 0x7f307c0cf650 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 --2- 172.21.15.5:0/2128935145 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c161030 0x7f307c099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 --2- 172.21.15.5:0/2128935145 >> v2:172.21.15.49:3300/0 conn(0x7f307c0b2480 0x7f307c099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 -- 172.21.15.5:0/2128935145 >> 172.21.15.5:0/2128935145 conn(0x7f307c0b2f20 msgr2=0x7f307c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 -- 172.21.15.5:0/2128935145 shutdown_connections 2024-05-08T23:27:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.609+0000 7f3080e3e700 1 -- 172.21.15.5:0/2128935145 wait complete. 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.611+0000 7f3080e3e700 1 Processor -- start 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.611+0000 7f3080e3e700 1 -- start start 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f3080e3e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c09a4d0 0x7f307c06fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f3080e3e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f307c0b2480 0x7f307c0c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f3080e3e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c161030 0x7f307c1f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f3080e3e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0ca300 con 0x7f307c161030 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f3080e3e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0ca720 con 0x7f307c0b2480 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f3080e3e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c9680 con 0x7f307c09a4d0 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f307ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c161030 0x7f307c1f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f307ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c161030 0x7f307c1f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33684/0 (socket says 172.21.15.5:33684) 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.612+0000 7f307ad9d700 1 -- 172.21.15.5:0/1795242547 learned_addr learned my addr 172.21.15.5:0/1795242547 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f3071d9b700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f307c0b2480 0x7f307c0c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f307a59c700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c09a4d0 0x7f307c06fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f307ad9d700 1 -- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c09a4d0 msgr2=0x7f307c06fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f307ad9d700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c09a4d0 0x7f307c06fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f307ad9d700 1 -- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f307c0b2480 msgr2=0x7f307c0c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f307ad9d700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f307c0b2480 0x7f307c0c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f307ad9d700 1 -- 172.21.15.5:0/1795242547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009b30 con 0x7f307c161030 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f307ad9d700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c161030 0x7f307c1f3510 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f307c072530 tx=0x7f3074004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f30737fe700 1 -- 172.21.15.5:0/1795242547 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740099b0 con 0x7f307c161030 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f30737fe700 1 -- 172.21.15.5:0/1795242547 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30740092e0 con 0x7f307c161030 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c0c9960 con 0x7f307c161030 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f30737fe700 1 -- 172.21.15.5:0/1795242547 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307401c640 con 0x7f307c161030 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.613+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c0c9f20 con 0x7f307c161030 2024-05-08T23:27:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.614+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c068000 con 0x7f307c161030 2024-05-08T23:27:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.615+0000 7f30737fe700 1 -- 172.21.15.5:0/1795242547 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f307401c7a0 con 0x7f307c161030 2024-05-08T23:27:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.615+0000 7f30737fe700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f305c06ac90 0x7f305c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.615+0000 7f30737fe700 1 -- 172.21.15.5:0/1795242547 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f307408ada0 con 0x7f307c161030 2024-05-08T23:27:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.616+0000 7f307a59c700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f305c06ac90 0x7f305c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.616+0000 7f307a59c700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f305c06ac90 0x7f305c06d150 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f306c00b9b0 tx=0x7f306c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.620+0000 7f30737fe700 1 -- 172.21.15.5:0/1795242547 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3074056240 con 0x7f307c161030 2024-05-08T23:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.838+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f307c0d0f50 con 0x7f307c161030 2024-05-08T23:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.839+0000 7f30737fe700 1 -- 172.21.15.5:0/1795242547 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3074059860 con 0x7f307c161030 2024-05-08T23:27:56.839 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:56.839 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-08T23:27:56.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-08T23:27:56.840 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-08T23:27:56.840 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-08T23:27:56.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:27:56.840 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.841+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f305c06ac90 msgr2=0x7f305c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.841+0000 7f3080e3e700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f305c06ac90 0x7f305c06d150 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f306c00b9b0 tx=0x7f306c008040 comp rx=0 tx=0).stop 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.841+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c161030 msgr2=0x7f307c1f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.841+0000 7f3080e3e700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c161030 0x7f307c1f3510 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f307c072530 tx=0x7f3074004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.842+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 shutdown_connections 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.842+0000 7f3080e3e700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f305c06ac90 0x7f305c06d150 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.842+0000 7f3080e3e700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c161030 0x7f307c1f3510 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.842+0000 7f3080e3e700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f307c09a4d0 0x7f307c06fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.842+0000 7f3080e3e700 1 --2- 172.21.15.5:0/1795242547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f307c0b2480 0x7f307c0c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.842+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 >> 172.21.15.5:0/1795242547 conn(0x7f307c0b2f20 msgr2=0x7f307c1600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.842+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 shutdown_connections 2024-05-08T23:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.842+0000 7f3080e3e700 1 -- 172.21.15.5:0/1795242547 wait complete. 2024-05-08T23:27:56.855 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:27:56.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:27:56.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.995+0000 7fd60088c700 1 Processor -- start 2024-05-08T23:27:56.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.995+0000 7fd60088c700 1 -- start start 2024-05-08T23:27:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.996+0000 7fd60088c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd5fc0cbf50 0x7fd5fc161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.996+0000 7fd60088c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc162240 0x7fd5fc15e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.996+0000 7fd60088c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd5fc0d0f10 0x7fd5fc15e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.996+0000 7fd60088c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cf970 con 0x7fd5fc0cbf50 2024-05-08T23:27:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.996+0000 7fd60088c700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cfc50 con 0x7fd5fc0d0f10 2024-05-08T23:27:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.996+0000 7fd60088c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cff30 con 0x7fd5fc162240 2024-05-08T23:27:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.996+0000 7fd5fa59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd5fc0cbf50 0x7fd5fc161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fa59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd5fc0cbf50 0x7fd5fc161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33692/0 (socket says 172.21.15.5:33692) 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fa59c700 1 -- 172.21.15.5:0/1295958891 learned_addr learned my addr 172.21.15.5:0/1295958891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fad9d700 1 --2- 172.21.15.5:0/1295958891 >> v2:172.21.15.49:3300/0 conn(0x7fd5fc0d0f10 0x7fd5fc15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5f9d9b700 1 --2- 172.21.15.5:0/1295958891 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc162240 0x7fd5fc15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fa59c700 1 -- 172.21.15.5:0/1295958891 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc162240 msgr2=0x7fd5fc15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fa59c700 1 --2- 172.21.15.5:0/1295958891 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc162240 0x7fd5fc15e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fa59c700 1 -- 172.21.15.5:0/1295958891 >> v2:172.21.15.49:3300/0 conn(0x7fd5fc0d0f10 msgr2=0x7fd5fc15e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fa59c700 1 --2- 172.21.15.5:0/1295958891 >> v2:172.21.15.49:3300/0 conn(0x7fd5fc0d0f10 0x7fd5fc15e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fa59c700 1 -- 172.21.15.5:0/1295958891 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc1617e0 con 0x7fd5fc0cbf50 2024-05-08T23:27:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5fa59c700 1 --2- 172.21.15.5:0/1295958891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc0cbf50 0x7fd5fc161d00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc072c50 tx=0x7fd5ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab9b09d1f0f7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5f959a700 1 -- 172.21.15.5:0/1295958891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00b870 con 0x7fd5fc0cbf50 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5f959a700 1 -- 172.21.15.5:0/1295958891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec00b9d0 con 0x7fd5fc0cbf50 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.997+0000 7fd5f959a700 1 -- 172.21.15.5:0/1295958891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00bd60 con 0x7fd5fc0cbf50 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 -- 172.21.15.5:0/1295958891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc0cbf50 msgr2=0x7fd5fc161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 --2- 172.21.15.5:0/1295958891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc0cbf50 0x7fd5fc161d00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc072c50 tx=0x7fd5ec009bd0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 -- 172.21.15.5:0/1295958891 shutdown_connections 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 --2- 172.21.15.5:0/1295958891 >> v2:172.21.15.49:3300/0 conn(0x7fd5fc0d0f10 0x7fd5fc15e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 --2- 172.21.15.5:0/1295958891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc0cbf50 0x7fd5fc161d00 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 --2- 172.21.15.5:0/1295958891 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc162240 0x7fd5fc15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 -- 172.21.15.5:0/1295958891 >> 172.21.15.5:0/1295958891 conn(0x7fd5fc099a50 msgr2=0x7fd5fc0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 -- 172.21.15.5:0/1295958891 shutdown_connections 2024-05-08T23:27:57.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:56.998+0000 7fd60088c700 1 -- 172.21.15.5:0/1295958891 wait complete. 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd60088c700 1 Processor -- start 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd60088c700 1 -- start start 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd60088c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc0cbf50 0x7fd5fc0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd60088c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc0d0f10 0x7fd5fc159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd60088c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc162240 0x7fd5fc1e9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd60088c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b3f40 con 0x7fd5fc162240 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd60088c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc1e9bf0 con 0x7fd5fc0cbf50 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd60088c700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b4190 con 0x7fd5fc0d0f10 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.000+0000 7fd5fad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc162240 0x7fd5fc1e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc162240 0x7fd5fc1e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33698/0 (socket says 172.21.15.5:33698) 2024-05-08T23:27:57.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fad9d700 1 -- 172.21.15.5:0/2354362976 learned_addr learned my addr 172.21.15.5:0/2354362976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fa59c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc0cbf50 0x7fd5fc0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5f9d9b700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc0d0f10 0x7fd5fc159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fad9d700 1 -- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc0d0f10 msgr2=0x7fd5fc159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fad9d700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc0d0f10 0x7fd5fc159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fad9d700 1 -- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc0cbf50 msgr2=0x7fd5fc0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fad9d700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc0cbf50 0x7fd5fc0b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fad9d700 1 -- 172.21.15.5:0/2354362976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec009550 con 0x7fd5fc162240 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fa59c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc0cbf50 0x7fd5fc0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5fad9d700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc162240 0x7fd5fc1e9280 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4007e40 tx=0x7fd5f400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5f2ffd700 1 -- 172.21.15.5:0/2354362976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400ede0 con 0x7fd5fc162240 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5f2ffd700 1 -- 172.21.15.5:0/2354362976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4014420 con 0x7fd5fc162240 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc06e030 con 0x7fd5fc162240 2024-05-08T23:27:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd5f2ffd700 1 -- 172.21.15.5:0/2354362976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40147d0 con 0x7fd5fc162240 2024-05-08T23:27:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.001+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0b4750 con 0x7fd5fc162240 2024-05-08T23:27:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.002+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc15b9c0 con 0x7fd5fc162240 2024-05-08T23:27:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.003+0000 7fd5f2ffd700 1 -- 172.21.15.5:0/2354362976 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd5f4014930 con 0x7fd5fc162240 2024-05-08T23:27:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.003+0000 7fd5f2ffd700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5dc06ac90 0x7fd5dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.003+0000 7fd5f2ffd700 1 -- 172.21.15.5:0/2354362976 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd5f4089700 con 0x7fd5fc162240 2024-05-08T23:27:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.004+0000 7fd5fa59c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5dc06ac90 0x7fd5dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.004+0000 7fd5fa59c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5dc06ac90 0x7fd5dc06d150 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec014040 tx=0x7fd5ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:57.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.008+0000 7fd5f2ffd700 1 -- 172.21.15.5:0/2354362976 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd5f4054c30 con 0x7fd5fc162240 2024-05-08T23:27:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.227+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd5fc0b4970 con 0x7fd5fc162240 2024-05-08T23:27:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.808+0000 7fd5f2ffd700 1 -- 172.21.15.5:0/2354362976 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd5f4058250 con 0x7fd5fc162240 2024-05-08T23:27:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5dc06ac90 msgr2=0x7fd5dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5dc06ac90 0x7fd5dc06d150 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec014040 tx=0x7fd5ec008030 comp rx=0 tx=0).stop 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc162240 msgr2=0x7fd5fc1e9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc162240 0x7fd5fc1e9280 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4007e40 tx=0x7fd5f400e3c0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 shutdown_connections 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd5dc06ac90 0x7fd5dc06d150 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5fc162240 0x7fd5fc1e9280 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5fc0cbf50 0x7fd5fc0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 --2- 172.21.15.5:0/2354362976 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd5fc0d0f10 0x7fd5fc159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.815+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 >> 172.21.15.5:0/2354362976 conn(0x7fd5fc099a50 msgr2=0x7fd5fc160d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.816+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 shutdown_connections 2024-05-08T23:27:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.816+0000 7fd60088c700 1 -- 172.21.15.5:0/2354362976 wait complete. 2024-05-08T23:27:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-08T23:27:57.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:27:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.979+0000 7fbf7e821700 1 Processor -- start 2024-05-08T23:27:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.979+0000 7fbf7e821700 1 -- start start 2024-05-08T23:27:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.980+0000 7fbf7e821700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf780b0d80 0x7fbf780b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.980+0000 7fbf7e821700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf781454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.980+0000 7fbf7e821700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf78145040 con 0x7fbf780b4d50 2024-05-08T23:27:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.980+0000 7fbf7e821700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf780b38c0 con 0x7fbf780b0d80 2024-05-08T23:27:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.980+0000 7fbf7e821700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780b16e0 con 0x7fbf7806b070 2024-05-08T23:27:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf77fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf780b0d80 0x7fbf780b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf777fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf780b4d50 0x7fbf7806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34388/0 (socket says 172.21.15.5:34388) 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf777fe700 1 -- 172.21.15.5:0/2277066590 learned_addr learned my addr 172.21.15.5:0/2277066590 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf7cdbe700 1 --2- 172.21.15.5:0/2277066590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf781454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf77fff700 1 -- 172.21.15.5:0/2277066590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 msgr2=0x7fbf781454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf77fff700 1 --2- 172.21.15.5:0/2277066590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf781454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf77fff700 1 -- 172.21.15.5:0/2277066590 >> v1:172.21.15.5:6789/0 conn(0x7fbf780b4d50 legacy=0x7fbf7806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf76ffd700 1 -- 172.21.15.5:0/2277066590 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282383195 0 0) 0x7fbf78145040 con 0x7fbf780b4d50 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf77fff700 1 -- 172.21.15.5:0/2277066590 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf780b19c0 con 0x7fbf780b0d80 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf7cdbe700 1 --2- 172.21.15.5:0/2277066590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf781454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf77fff700 1 --2- 172.21.15.5:0/2277066590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b0d80 0x7fbf780b3380 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009840 tx=0x7fbf68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bacba2ac332e401 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf76ffd700 1 -- 172.21.15.5:0/2277066590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011830 con 0x7fbf780b0d80 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf76ffd700 1 -- 172.21.15.5:0/2277066590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68011990 con 0x7fbf780b0d80 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.981+0000 7fbf76ffd700 1 -- 172.21.15.5:0/2277066590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011d20 con 0x7fbf780b0d80 2024-05-08T23:27:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.982+0000 7fbf7e821700 1 -- 172.21.15.5:0/2277066590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b0d80 msgr2=0x7fbf780b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.982+0000 7fbf7e821700 1 --2- 172.21.15.5:0/2277066590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b0d80 0x7fbf780b3380 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009840 tx=0x7fbf68009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.982+0000 7fbf7e821700 1 -- 172.21.15.5:0/2277066590 shutdown_connections 2024-05-08T23:27:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.982+0000 7fbf7e821700 1 --2- 172.21.15.5:0/2277066590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf781454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.982+0000 7fbf7e821700 1 --2- 172.21.15.5:0/2277066590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b0d80 0x7fbf780b3380 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.982+0000 7fbf7e821700 1 -- 172.21.15.5:0/2277066590 >> 172.21.15.5:0/2277066590 conn(0x7fbf78099a50 msgr2=0x7fbf780b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.982+0000 7fbf7e821700 1 -- 172.21.15.5:0/2277066590 shutdown_connections 2024-05-08T23:27:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.982+0000 7fbf7e821700 1 -- 172.21.15.5:0/2277066590 wait complete. 2024-05-08T23:27:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.984+0000 7fbf7e821700 1 Processor -- start 2024-05-08T23:27:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.984+0000 7fbf7e821700 1 -- start start 2024-05-08T23:27:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7e821700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf7815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7e821700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf780b0d80 0x7fbf7815f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7e821700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b4d50 0x7fbf7815fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7e821700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7815d5c0 con 0x7fbf780b4d50 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7e821700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf78140b70 con 0x7fbf780b0d80 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7e821700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf78140e50 con 0x7fbf7806b070 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7cdbe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b4d50 0x7fbf7815fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7cdbe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b4d50 0x7fbf7815fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58876/0 (socket says 172.21.15.5:58876) 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7cdbe700 1 -- 172.21.15.5:0/182186247 learned_addr learned my addr 172.21.15.5:0/182186247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf77fff700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf7815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf777fe700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf780b0d80 0x7fbf7815f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7cdbe700 1 -- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 msgr2=0x7fbf7815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7cdbe700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf7815d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7cdbe700 1 -- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf780b0d80 msgr2=0x7fbf7815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7cdbe700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf780b0d80 0x7fbf7815f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.985+0000 7fbf7cdbe700 1 -- 172.21.15.5:0/182186247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680094a0 con 0x7fbf780b4d50 2024-05-08T23:27:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.986+0000 7fbf7cdbe700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b4d50 0x7fbf7815fd60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fbf70007ef0 tx=0x7fbf7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.986+0000 7fbf74ff9700 1 -- 172.21.15.5:0/182186247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000dd80 con 0x7fbf780b4d50 2024-05-08T23:27:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.986+0000 7fbf74ff9700 1 -- 172.21.15.5:0/182186247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7000f040 con 0x7fbf780b4d50 2024-05-08T23:27:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.986+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf78141130 con 0x7fbf780b4d50 2024-05-08T23:27:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.986+0000 7fbf74ff9700 1 -- 172.21.15.5:0/182186247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70003c60 con 0x7fbf780b4d50 2024-05-08T23:27:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.986+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf781f75a0 con 0x7fbf780b4d50 2024-05-08T23:27:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.987+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf78074bc0 con 0x7fbf780b4d50 2024-05-08T23:27:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.988+0000 7fbf74ff9700 1 -- 172.21.15.5:0/182186247 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbf70004030 con 0x7fbf780b4d50 2024-05-08T23:27:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.988+0000 7fbf74ff9700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf6406ac90 0x7fbf6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.988+0000 7fbf74ff9700 1 -- 172.21.15.5:0/182186247 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbf700891f0 con 0x7fbf780b4d50 2024-05-08T23:27:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.988+0000 7fbf77fff700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf6406ac90 0x7fbf6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.989+0000 7fbf77fff700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf6406ac90 0x7fbf6406d150 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbf680090b0 tx=0x7fbf68008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:57.993+0000 7fbf74ff9700 1 -- 172.21.15.5:0/182186247 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbf700545c0 con 0x7fbf780b4d50 2024-05-08T23:27:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.210+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbf7806ae30 con 0x7fbf780b4d50 2024-05-08T23:27:58.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.814+0000 7fbf74ff9700 1 -- 172.21.15.5:0/182186247 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbf70057be0 con 0x7fbf780b4d50 2024-05-08T23:27:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.820+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf6406ac90 msgr2=0x7fbf6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.820+0000 7fbf7e821700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf6406ac90 0x7fbf6406d150 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbf680090b0 tx=0x7fbf68008030 comp rx=0 tx=0).stop 2024-05-08T23:27:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.820+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b4d50 msgr2=0x7fbf7815fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.820+0000 7fbf7e821700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b4d50 0x7fbf7815fd60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fbf70007ef0 tx=0x7fbf7000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.822+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 shutdown_connections 2024-05-08T23:27:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.822+0000 7fbf7e821700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbf6406ac90 0x7fbf6406d150 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.822+0000 7fbf7e821700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780b4d50 0x7fbf7815fd60 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.822+0000 7fbf7e821700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf7806b070 0x7fbf7815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.822+0000 7fbf7e821700 1 --2- 172.21.15.5:0/182186247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf780b0d80 0x7fbf7815f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.822+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 >> 172.21.15.5:0/182186247 conn(0x7fbf78099a50 msgr2=0x7fbf78145c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.823+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 shutdown_connections 2024-05-08T23:27:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.825+0000 7fbf7e821700 1 -- 172.21.15.5:0/182186247 wait complete. 2024-05-08T23:27:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-08T23:27:58.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:27:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.990+0000 7f198ad27700 1 Processor -- start 2024-05-08T23:27:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.990+0000 7f198ad27700 1 -- start start 2024-05-08T23:27:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f198ad27700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f198415b8c0 0x7f19840d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f198ad27700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 0x7f19840cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f198ad27700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19840cf560 con 0x7f198415b4e0 2024-05-08T23:27:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f198ad27700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19840cefe0 con 0x7f198415b8c0 2024-05-08T23:27:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f198ad27700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19840ccff0 con 0x7f19840d1740 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1983fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f198415b8c0 0x7f19840d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1988ac3700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f198415b4e0 0x7f19840b1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34390/0 (socket says 172.21.15.5:34390) 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1988ac3700 1 -- 172.21.15.5:0/3021802520 learned_addr learned my addr 172.21.15.5:0/3021802520 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1983fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f198415b8c0 0x7f19840d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58892/0 (socket says 172.21.15.5:58892) 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f19892c4700 1 --2- 172.21.15.5:0/3021802520 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 0x7f19840cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1983fff700 1 -- 172.21.15.5:0/3021802520 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 msgr2=0x7f19840cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1983fff700 1 --2- 172.21.15.5:0/3021802520 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 0x7f19840cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1983fff700 1 -- 172.21.15.5:0/3021802520 >> v1:172.21.15.5:6789/0 conn(0x7f198415b4e0 legacy=0x7f19840b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1983fff700 1 -- 172.21.15.5:0/3021802520 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19840cd2d0 con 0x7f198415b8c0 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f19837fe700 1 -- 172.21.15.5:0/3021802520 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152392946 0 0) 0x7f19840cf560 con 0x7f198415b4e0 2024-05-08T23:27:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.991+0000 7f1983fff700 1 --2- 172.21.15.5:0/3021802520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b8c0 0x7f19840d1200 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f196c009840 tx=0x7f196c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d6db15823b2305 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f19837fe700 1 -- 172.21.15.5:0/3021802520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c011830 con 0x7f198415b8c0 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f19837fe700 1 -- 172.21.15.5:0/3021802520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c011990 con 0x7f198415b8c0 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f19837fe700 1 -- 172.21.15.5:0/3021802520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c011d20 con 0x7f198415b8c0 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f198ad27700 1 -- 172.21.15.5:0/3021802520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b8c0 msgr2=0x7f19840d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f198ad27700 1 --2- 172.21.15.5:0/3021802520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b8c0 0x7f19840d1200 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f196c009840 tx=0x7f196c009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f198ad27700 1 -- 172.21.15.5:0/3021802520 shutdown_connections 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f198ad27700 1 --2- 172.21.15.5:0/3021802520 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 0x7f19840cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f198ad27700 1 --2- 172.21.15.5:0/3021802520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b8c0 0x7f19840d1200 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f198ad27700 1 -- 172.21.15.5:0/3021802520 >> 172.21.15.5:0/3021802520 conn(0x7f198408afe0 msgr2=0x7f198408b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f198ad27700 1 -- 172.21.15.5:0/3021802520 shutdown_connections 2024-05-08T23:27:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.992+0000 7f198ad27700 1 -- 172.21.15.5:0/3021802520 wait complete. 2024-05-08T23:27:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.994+0000 7f198ad27700 1 Processor -- start 2024-05-08T23:27:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.994+0000 7f198ad27700 1 -- start start 2024-05-08T23:27:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f198ad27700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 0x7f198415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f198ad27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b4e0 0x7f19840c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f198ad27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f198415b8c0 0x7f198414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f198ad27700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1984153450 con 0x7f198415b4e0 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f198ad27700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198414fed0 con 0x7f198415b8c0 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f198ad27700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19840c5dd0 con 0x7f19840d1740 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b4e0 0x7f19840c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b4e0 0x7f19840c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58898/0 (socket says 172.21.15.5:58898) 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 -- 172.21.15.5:0/3103384644 learned_addr learned my addr 172.21.15.5:0/3103384644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1988ac3700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 0x7f198415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f19892c4700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f198415b8c0 0x7f198414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 -- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 msgr2=0x7f198415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 0x7f198415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 -- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f198415b8c0 msgr2=0x7f198414f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f198415b8c0 0x7f198414f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 -- 172.21.15.5:0/3103384644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0094a0 con 0x7f198415b4e0 2024-05-08T23:27:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f19892c4700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f198415b8c0 0x7f198414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.995+0000 7f1983fff700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b4e0 0x7f19840c9530 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f196c009fd0 tx=0x7f196c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.996+0000 7f19817fa700 1 -- 172.21.15.5:0/3103384644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0036a0 con 0x7f198415b4e0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.996+0000 7f19817fa700 1 -- 172.21.15.5:0/3103384644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c01e070 con 0x7f198415b4e0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.996+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1984153170 con 0x7f198415b4e0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.996+0000 7f19817fa700 1 -- 172.21.15.5:0/3103384644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c003a40 con 0x7f198415b4e0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.996+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19840c5a60 con 0x7f198415b4e0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.997+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19840c55e0 con 0x7f198415b4e0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.997+0000 7f19817fa700 1 -- 172.21.15.5:0/3103384644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f196c003ba0 con 0x7f198415b4e0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.998+0000 7f19817fa700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f196406ac90 0x7f196406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.998+0000 7f19817fa700 1 -- 172.21.15.5:0/3103384644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f196c08a250 con 0x7f198415b4e0 2024-05-08T23:27:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.998+0000 7f1988ac3700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f196406ac90 0x7f196406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:58.999+0000 7f1988ac3700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f196406ac90 0x7f196406d150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1974007fd0 tx=0x7f1974007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.003+0000 7f19817fa700 1 -- 172.21.15.5:0/3103384644 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f19840c55e0 con 0x7f198415b4e0 2024-05-08T23:27:59.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.269+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f19840c55e0 con 0x7f198415b4e0 2024-05-08T23:27:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.276+0000 7f19817fa700 1 -- 172.21.15.5:0/3103384644 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v96) v1 ==== 174+0+0 (secure 0 0 0) 0x7f19840c55e0 con 0x7f198415b4e0 2024-05-08T23:27:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.282+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f196406ac90 msgr2=0x7f196406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.282+0000 7f198ad27700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f196406ac90 0x7f196406d150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1974007fd0 tx=0x7f1974007de0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.282+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b4e0 msgr2=0x7f19840c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.282+0000 7f198ad27700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b4e0 0x7f19840c9530 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f196c009fd0 tx=0x7f196c004750 comp rx=0 tx=0).stop 2024-05-08T23:27:59.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.283+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 shutdown_connections 2024-05-08T23:27:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.283+0000 7f198ad27700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f196406ac90 0x7f196406d150 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.283+0000 7f198ad27700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f198415b8c0 0x7f198414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.283+0000 7f198ad27700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f19840d1740 0x7f198415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.283+0000 7f198ad27700 1 --2- 172.21.15.5:0/3103384644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198415b4e0 0x7f19840c9530 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.283+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 >> 172.21.15.5:0/3103384644 conn(0x7f198408afe0 msgr2=0x7f19840af250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.286+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 shutdown_connections 2024-05-08T23:27:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.287+0000 7f198ad27700 1 -- 172.21.15.5:0/3103384644 wait complete. 2024-05-08T23:27:59.303 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 14 and data pool 15 2024-05-08T23:27:59.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:27:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.448+0000 7ffa3ed01700 1 Processor -- start 2024-05-08T23:27:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.448+0000 7ffa3ed01700 1 -- start start 2024-05-08T23:27:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.449+0000 7ffa3ed01700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa38168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.449+0000 7ffa3ed01700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa3809a870 con 0x7ffa380b2480 2024-05-08T23:27:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.449+0000 7ffa3ed01700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa380d0840 con 0x7ffa38161030 2024-05-08T23:27:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.449+0000 7ffa3ed01700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3806ccc0 con 0x7ffa380ccbd0 2024-05-08T23:27:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.449+0000 7ffa37fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ffa380b2480 0x7ffa3809a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34404/0 (socket says 172.21.15.5:34404) 2024-05-08T23:27:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.449+0000 7ffa37fff700 1 -- 172.21.15.5:0/767641593 learned_addr learned my addr 172.21.15.5:0/767641593 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa3d29e700 1 --2- 172.21.15.5:0/767641593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa38168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa377fe700 1 -- 172.21.15.5:0/767641593 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426014318 0 0) 0x7ffa3809a870 con 0x7ffa380b2480 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa377fe700 1 -- 172.21.15.5:0/767641593 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa240035b0 con 0x7ffa380b2480 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa377fe700 1 -- 172.21.15.5:0/767641593 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353984396 0 0) 0x7ffa380d0840 con 0x7ffa38161030 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa377fe700 1 -- 172.21.15.5:0/767641593 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa3809a870 con 0x7ffa38161030 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa3d29e700 1 -- 172.21.15.5:0/767641593 >> v1:172.21.15.49:6789/0 conn(0x7ffa38161030 legacy=0x7ffa38099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa3d29e700 1 -- 172.21.15.5:0/767641593 >> v1:172.21.15.5:6789/0 conn(0x7ffa380b2480 legacy=0x7ffa3809a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa3d29e700 1 -- 172.21.15.5:0/767641593 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3806c600 con 0x7ffa380ccbd0 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa3d29e700 1 --2- 172.21.15.5:0/767641593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa38168580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009dc0 tx=0x7ffa3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f3e815484585e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa377fe700 1 -- 172.21.15.5:0/767641593 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000e070 con 0x7ffa380ccbd0 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.450+0000 7ffa377fe700 1 -- 172.21.15.5:0/767641593 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3000bb20 con 0x7ffa380ccbd0 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.451+0000 7ffa377fe700 1 -- 172.21.15.5:0/767641593 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000be90 con 0x7ffa380ccbd0 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.451+0000 7ffa3ed01700 1 -- 172.21.15.5:0/767641593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 msgr2=0x7ffa38168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.451+0000 7ffa3ed01700 1 --2- 172.21.15.5:0/767641593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa38168580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009dc0 tx=0x7ffa3000b330 comp rx=0 tx=0).stop 2024-05-08T23:27:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.451+0000 7ffa3ed01700 1 -- 172.21.15.5:0/767641593 shutdown_connections 2024-05-08T23:27:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.451+0000 7ffa3ed01700 1 --2- 172.21.15.5:0/767641593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa38168580 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.451+0000 7ffa3ed01700 1 -- 172.21.15.5:0/767641593 >> 172.21.15.5:0/767641593 conn(0x7ffa380b2f20 msgr2=0x7ffa380b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.451+0000 7ffa3ed01700 1 -- 172.21.15.5:0/767641593 shutdown_connections 2024-05-08T23:27:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.451+0000 7ffa3ed01700 1 -- 172.21.15.5:0/767641593 wait complete. 2024-05-08T23:27:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.453+0000 7ffa3ed01700 1 Processor -- start 2024-05-08T23:27:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.453+0000 7ffa3ed01700 1 -- start start 2024-05-08T23:27:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3ed01700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa380b2480 0x7ffa380b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3ed01700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa380c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3ed01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa38161030 0x7ffa3814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3ed01700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa380d0660 con 0x7ffa38161030 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3ed01700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3814c350 con 0x7ffa380b2480 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3ed01700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3814c5b0 con 0x7ffa380ccbd0 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3d29e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa38161030 0x7ffa3814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3d29e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa38161030 0x7ffa3814efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58908/0 (socket says 172.21.15.5:58908) 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3d29e700 1 -- 172.21.15.5:0/628180940 learned_addr learned my addr 172.21.15.5:0/628180940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa37fff700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa380c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.454+0000 7ffa3ca9d700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa380b2480 0x7ffa380b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa3d29e700 1 -- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 msgr2=0x7ffa380c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa3d29e700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa380c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa3d29e700 1 -- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa380b2480 msgr2=0x7ffa380b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa3d29e700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa380b2480 0x7ffa380b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa3d29e700 1 -- 172.21.15.5:0/628180940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa30009a70 con 0x7ffa38161030 2024-05-08T23:27:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa3d29e700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa38161030 0x7ffa3814efd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ffa30014040 tx=0x7ffa30004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa357fa700 1 -- 172.21.15.5:0/628180940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300048c0 con 0x7ffa38161030 2024-05-08T23:27:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa357fa700 1 -- 172.21.15.5:0/628180940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa30020070 con 0x7ffa38161030 2024-05-08T23:27:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa380cab20 con 0x7ffa38161030 2024-05-08T23:27:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa357fa700 1 -- 172.21.15.5:0/628180940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300276c0 con 0x7ffa38161030 2024-05-08T23:27:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.455+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa380cb0a0 con 0x7ffa38161030 2024-05-08T23:27:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.456+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa38068000 con 0x7ffa38161030 2024-05-08T23:27:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.457+0000 7ffa357fa700 1 -- 172.21.15.5:0/628180940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffa30027820 con 0x7ffa38161030 2024-05-08T23:27:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.457+0000 7ffa357fa700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa1806ac90 0x7ffa1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.457+0000 7ffa357fa700 1 -- 172.21.15.5:0/628180940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa30022070 con 0x7ffa38161030 2024-05-08T23:27:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.457+0000 7ffa3ca9d700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa1806ac90 0x7ffa1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.458+0000 7ffa3ca9d700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa1806ac90 0x7ffa1806d150 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffa28001830 tx=0x7ffa28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.462+0000 7ffa357fa700 1 -- 172.21.15.5:0/628180940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffa30056460 con 0x7ffa38161030 2024-05-08T23:27:59.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.677+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa3806bd20 con 0x7ffa38161030 2024-05-08T23:27:59.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.679+0000 7ffa357fa700 1 -- 172.21.15.5:0/628180940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+15957 (secure 0 0 0) 0x7ffa30059a80 con 0x7ffa38161030 2024-05-08T23:27:59.679 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:27:59.679 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":95,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:27:59.271421+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:27:57.260067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T23:27:58.243420+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:27:59.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa1806ac90 msgr2=0x7ffa1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa1806ac90 0x7ffa1806d150 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffa28001830 tx=0x7ffa28009040 comp rx=0 tx=0).stop 2024-05-08T23:27:59.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa38161030 msgr2=0x7ffa3814efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa38161030 0x7ffa3814efd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ffa30014040 tx=0x7ffa30004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 shutdown_connections 2024-05-08T23:27:59.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ffa1806ac90 0x7ffa1806d150 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa38161030 0x7ffa3814efd0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa380b2480 0x7ffa380b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 --2- 172.21.15.5:0/628180940 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffa380ccbd0 0x7ffa380c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 >> 172.21.15.5:0/628180940 conn(0x7ffa380b2f20 msgr2=0x7ffa3806f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:59.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 shutdown_connections 2024-05-08T23:27:59.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.681+0000 7ffa3ed01700 1 -- 172.21.15.5:0/628180940 wait complete. 2024-05-08T23:27:59.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:27:59.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.833+0000 7fc22a572700 1 Processor -- start 2024-05-08T23:27:59.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.833+0000 7fc22a572700 1 -- start start 2024-05-08T23:27:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc22a572700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc22415db00 0x7fc2240cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc22a572700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc22406f990 0x7fc22406e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc22a572700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22406ad20 con 0x7fc2240caa50 2024-05-08T23:27:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc22a572700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22415dee0 con 0x7fc22415db00 2024-05-08T23:27:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc22a572700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240ccad0 con 0x7fc22406f990 2024-05-08T23:27:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc223fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc22415db00 0x7fc2240cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc2237fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc2240caa50 0x7fc22406f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34418/0 (socket says 172.21.15.5:34418) 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc2237fe700 1 -- 172.21.15.5:0/3247357092 learned_addr learned my addr 172.21.15.5:0/3247357092 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.834+0000 7fc228b0f700 1 --2- 172.21.15.5:0/3247357092 >> v2:172.21.15.49:3300/0 conn(0x7fc22406f990 0x7fc22406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.835+0000 7fc223fff700 1 -- 172.21.15.5:0/3247357092 >> v2:172.21.15.49:3300/0 conn(0x7fc22406f990 msgr2=0x7fc22406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.835+0000 7fc223fff700 1 --2- 172.21.15.5:0/3247357092 >> v2:172.21.15.49:3300/0 conn(0x7fc22406f990 0x7fc22406e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.835+0000 7fc223fff700 1 -- 172.21.15.5:0/3247357092 >> v1:172.21.15.5:6789/0 conn(0x7fc2240caa50 legacy=0x7fc22406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.835+0000 7fc223fff700 1 -- 172.21.15.5:0/3247357092 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22406b000 con 0x7fc22415db00 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.835+0000 7fc223fff700 1 --2- 172.21.15.5:0/3247357092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc22415db00 0x7fc2240cc590 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc224099f70 tx=0x7fc214009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a219a7bffed5f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.835+0000 7fc222ffd700 1 -- 172.21.15.5:0/3247357092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214011830 con 0x7fc22415db00 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.835+0000 7fc222ffd700 1 -- 172.21.15.5:0/3247357092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214011990 con 0x7fc22415db00 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.835+0000 7fc222ffd700 1 -- 172.21.15.5:0/3247357092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214011d20 con 0x7fc22415db00 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.836+0000 7fc22a572700 1 -- 172.21.15.5:0/3247357092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc22415db00 msgr2=0x7fc2240cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.836+0000 7fc22a572700 1 --2- 172.21.15.5:0/3247357092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc22415db00 0x7fc2240cc590 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc224099f70 tx=0x7fc214009b50 comp rx=0 tx=0).stop 2024-05-08T23:27:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.836+0000 7fc22a572700 1 -- 172.21.15.5:0/3247357092 shutdown_connections 2024-05-08T23:27:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.836+0000 7fc22a572700 1 --2- 172.21.15.5:0/3247357092 >> v2:172.21.15.49:3300/0 conn(0x7fc22406f990 0x7fc22406e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.836+0000 7fc22a572700 1 --2- 172.21.15.5:0/3247357092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc22415db00 0x7fc2240cc590 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.836+0000 7fc22a572700 1 -- 172.21.15.5:0/3247357092 >> 172.21.15.5:0/3247357092 conn(0x7fc224099a50 msgr2=0x7fc2240d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.836+0000 7fc22a572700 1 -- 172.21.15.5:0/3247357092 shutdown_connections 2024-05-08T23:27:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.836+0000 7fc22a572700 1 -- 172.21.15.5:0/3247357092 wait complete. 2024-05-08T23:27:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.837+0000 7fc22a572700 1 Processor -- start 2024-05-08T23:27:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.837+0000 7fc22a572700 1 -- start start 2024-05-08T23:27:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc22a572700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc22406f990 0x7fc224155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc22a572700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2240caa50 0x7fc2240c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc22a572700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc22415db00 0x7fc2240c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc22a572700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240b4d90 con 0x7fc2240caa50 2024-05-08T23:27:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc22a572700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240b3cf0 con 0x7fc22415db00 2024-05-08T23:27:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc22a572700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240b3fd0 con 0x7fc22406f990 2024-05-08T23:27:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc2237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2240caa50 0x7fc2240c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc2237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2240caa50 0x7fc2240c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58920/0 (socket says 172.21.15.5:58920) 2024-05-08T23:27:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc2237fe700 1 -- 172.21.15.5:0/2586669067 learned_addr learned my addr 172.21.15.5:0/2586669067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:27:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc223fff700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc22406f990 0x7fc224155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc228b0f700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc22415db00 0x7fc2240c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc2237fe700 1 -- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc22406f990 msgr2=0x7fc224155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc2237fe700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc22406f990 0x7fc224155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc2237fe700 1 -- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc22415db00 msgr2=0x7fc2240c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:59.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc2237fe700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc22415db00 0x7fc2240c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:59.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.838+0000 7fc2237fe700 1 -- 172.21.15.5:0/2586669067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2140094a0 con 0x7fc2240caa50 2024-05-08T23:27:59.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.839+0000 7fc2237fe700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2240caa50 0x7fc2240c96a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c520 tx=0x7fc21800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:59.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.839+0000 7fc220ff9700 1 -- 172.21.15.5:0/2586669067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218005d70 con 0x7fc2240caa50 2024-05-08T23:27:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.839+0000 7fc220ff9700 1 -- 172.21.15.5:0/2586669067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2180070d0 con 0x7fc2240caa50 2024-05-08T23:27:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.839+0000 7fc220ff9700 1 -- 172.21.15.5:0/2586669067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218007440 con 0x7fc2240caa50 2024-05-08T23:27:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.839+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2240b42b0 con 0x7fc2240caa50 2024-05-08T23:27:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.839+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2240b4870 con 0x7fc2240caa50 2024-05-08T23:27:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.840+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2240b6300 con 0x7fc2240caa50 2024-05-08T23:27:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.841+0000 7fc220ff9700 1 -- 172.21.15.5:0/2586669067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc2180075a0 con 0x7fc2240caa50 2024-05-08T23:27:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.841+0000 7fc220ff9700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc21006ac90 0x7fc21006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.841+0000 7fc220ff9700 1 -- 172.21.15.5:0/2586669067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc218088a60 con 0x7fc2240caa50 2024-05-08T23:27:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.841+0000 7fc223fff700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc21006ac90 0x7fc21006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.842+0000 7fc223fff700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc21006ac90 0x7fc21006d150 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc2140090b0 tx=0x7fc21401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:27:59.846+0000 7fc220ff9700 1 -- 172.21.15.5:0/2586669067 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc218053dd0 con 0x7fc2240caa50 2024-05-08T23:28:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.062+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc224068000 con 0x7fc2240caa50 2024-05-08T23:28:00.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.064+0000 7fc220ff9700 1 -- 172.21.15.5:0/2586669067 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+15957 (secure 0 0 0) 0x7fc218021090 con 0x7fc2240caa50 2024-05-08T23:28:00.064 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":95,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:27:59.271421+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:27:57.260067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T23:27:58.243420+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:28:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc21006ac90 msgr2=0x7fc21006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc21006ac90 0x7fc21006d150 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc2140090b0 tx=0x7fc21401c040 comp rx=0 tx=0).stop 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2240caa50 msgr2=0x7fc2240c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2240caa50 0x7fc2240c96a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c520 tx=0x7fc21800c830 comp rx=0 tx=0).stop 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 shutdown_connections 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc21006ac90 0x7fc21006d150 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc22415db00 0x7fc2240c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc22406f990 0x7fc224155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 --2- 172.21.15.5:0/2586669067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2240caa50 0x7fc2240c96a0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 >> 172.21.15.5:0/2586669067 conn(0x7fc224099a50 msgr2=0x7fc2240b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 shutdown_connections 2024-05-08T23:28:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.066+0000 7fc22a572700 1 -- 172.21.15.5:0/2586669067 wait complete. 2024-05-08T23:28:00.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:28:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.222+0000 7fc8a1c92700 1 Processor -- start 2024-05-08T23:28:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.222+0000 7fc8a1c92700 1 -- start start 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.222+0000 7fc8a1c92700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc89c0cff30 0x7fc89c0d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc8a1c92700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0b4a90 0x7fc89c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc8a1c92700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0d0850 con 0x7fc89c0d3d20 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc8a1c92700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0cf610 con 0x7fc89c0cff30 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc8a1c92700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0d1fc0 con 0x7fc89c0b4a90 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89b7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc89c0cff30 0x7fc89c0d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89bfff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0b4a90 0x7fc89c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89affd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc89c0d3d20 0x7fc89c0ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:56902/0 (socket says 172.21.15.5:56902) 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89affd700 1 -- 172.21.15.5:0/210033786 learned_addr learned my addr 172.21.15.5:0/210033786 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89a7fc700 1 -- 172.21.15.5:0/210033786 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830358342 0 0) 0x7fc89c0d0850 con 0x7fc89c0d3d20 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89a7fc700 1 -- 172.21.15.5:0/210033786 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8840035b0 con 0x7fc89c0d3d20 2024-05-08T23:28:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89bfff700 1 -- 172.21.15.5:0/210033786 >> v2:172.21.15.49:3300/0 conn(0x7fc89c0cff30 msgr2=0x7fc89c0d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89bfff700 1 --2- 172.21.15.5:0/210033786 >> v2:172.21.15.49:3300/0 conn(0x7fc89c0cff30 0x7fc89c0d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89bfff700 1 -- 172.21.15.5:0/210033786 >> v1:172.21.15.49:6789/0 conn(0x7fc89c0d3d20 legacy=0x7fc89c0ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89bfff700 1 -- 172.21.15.5:0/210033786 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0d22a0 con 0x7fc89c0b4a90 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.223+0000 7fc89b7fe700 1 --2- 172.21.15.5:0/210033786 >> v2:172.21.15.49:3300/0 conn(0x7fc89c0cff30 0x7fc89c0d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.224+0000 7fc89bfff700 1 --2- 172.21.15.5:0/210033786 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0b4a90 0x7fc89c0ce970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc894009d90 tx=0x7fc89400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c866e9169a9a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.224+0000 7fc89a7fc700 1 -- 172.21.15.5:0/210033786 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400e070 con 0x7fc89c0b4a90 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.224+0000 7fc89a7fc700 1 -- 172.21.15.5:0/210033786 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89400bb20 con 0x7fc89c0b4a90 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.224+0000 7fc89a7fc700 1 -- 172.21.15.5:0/210033786 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894012430 con 0x7fc89c0b4a90 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.224+0000 7fc8a1c92700 1 -- 172.21.15.5:0/210033786 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0b4a90 msgr2=0x7fc89c0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.224+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/210033786 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0b4a90 0x7fc89c0ce970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc894009d90 tx=0x7fc89400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.225+0000 7fc8a1c92700 1 -- 172.21.15.5:0/210033786 shutdown_connections 2024-05-08T23:28:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.225+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/210033786 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0b4a90 0x7fc89c0ce970 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.225+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/210033786 >> v2:172.21.15.49:3300/0 conn(0x7fc89c0cff30 0x7fc89c0d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.225+0000 7fc8a1c92700 1 -- 172.21.15.5:0/210033786 >> 172.21.15.5:0/210033786 conn(0x7fc89c08afe0 msgr2=0x7fc89c0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.225+0000 7fc8a1c92700 1 -- 172.21.15.5:0/210033786 shutdown_connections 2024-05-08T23:28:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.225+0000 7fc8a1c92700 1 -- 172.21.15.5:0/210033786 wait complete. 2024-05-08T23:28:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.226+0000 7fc8a1c92700 1 Processor -- start 2024-05-08T23:28:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.226+0000 7fc8a1c92700 1 -- start start 2024-05-08T23:28:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc8a1c92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc89c0b4a90 0x7fc89c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc8a1c92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89c0cff30 0x7fc89c14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc8a1c92700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0d3d20 0x7fc89c0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc8a1c92700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b6ab0 con 0x7fc89c0cff30 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc8a1c92700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b7f10 con 0x7fc89c0b4a90 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc8a1c92700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c14d800 con 0x7fc89c0d3d20 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89c0cff30 0x7fc89c14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89c0cff30 0x7fc89c14d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58928/0 (socket says 172.21.15.5:58928) 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc89c0b4a90 0x7fc89c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89affd700 1 -- 172.21.15.5:0/3740754951 learned_addr learned my addr 172.21.15.5:0/3740754951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89bfff700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0d3d20 0x7fc89c0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89affd700 1 -- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0d3d20 msgr2=0x7fc89c0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89affd700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0d3d20 0x7fc89c0c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89affd700 1 -- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc89c0b4a90 msgr2=0x7fc89c0b8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89affd700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc89c0b4a90 0x7fc89c0b8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89affd700 1 -- 172.21.15.5:0/3740754951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0094a0 con 0x7fc89c0cff30 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.227+0000 7fc89b7fe700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc89c0b4a90 0x7fc89c0b8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.228+0000 7fc89affd700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89c0cff30 0x7fc89c14d2c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc89000c700 tx=0x7fc89000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.228+0000 7fc883fff700 1 -- 172.21.15.5:0/3740754951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890005d70 con 0x7fc89c0cff30 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.228+0000 7fc883fff700 1 -- 172.21.15.5:0/3740754951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8900070d0 con 0x7fc89c0cff30 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.228+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc894009a70 con 0x7fc89c0cff30 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.228+0000 7fc883fff700 1 -- 172.21.15.5:0/3740754951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890007460 con 0x7fc89c0cff30 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.228+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c0b81b0 con 0x7fc89c0cff30 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.229+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c074bc0 con 0x7fc89c0cff30 2024-05-08T23:28:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.230+0000 7fc883fff700 1 -- 172.21.15.5:0/3740754951 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc8900075c0 con 0x7fc89c0cff30 2024-05-08T23:28:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.230+0000 7fc883fff700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806acc0 0x7fc88806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.230+0000 7fc883fff700 1 -- 172.21.15.5:0/3740754951 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc890088810 con 0x7fc89c0cff30 2024-05-08T23:28:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.230+0000 7fc89b7fe700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806acc0 0x7fc88806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.231+0000 7fc89b7fe700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806acc0 0x7fc88806d180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0114b0 tx=0x7fc88c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.235+0000 7fc883fff700 1 -- 172.21.15.5:0/3740754951 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc890053970 con 0x7fc89c0cff30 2024-05-08T23:28:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:00.507+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc89c0d4520 con 0x7fc89c0cff30 2024-05-08T23:28:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.280+0000 7fc883fff700 1 -- 172.21.15.5:0/3740754951 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc890056f90 con 0x7fc89c0cff30 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806acc0 msgr2=0x7fc88806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806acc0 0x7fc88806d180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0114b0 tx=0x7fc88c009210 comp rx=0 tx=0).stop 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89c0cff30 msgr2=0x7fc89c14d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89c0cff30 0x7fc89c14d2c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc89000c700 tx=0x7fc89000ca10 comp rx=0 tx=0).stop 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 shutdown_connections 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc88806acc0 0x7fc88806d180 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc89c0d3d20 0x7fc89c0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc89c0b4a90 0x7fc89c0b8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 --2- 172.21.15.5:0/3740754951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89c0cff30 0x7fc89c14d2c0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 >> 172.21.15.5:0/3740754951 conn(0x7fc89c08afe0 msgr2=0x7fc89c0cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:01.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 shutdown_connections 2024-05-08T23:28:01.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.283+0000 7fc8a1c92700 1 -- 172.21.15.5:0/3740754951 wait complete. 2024-05-08T23:28:01.299 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:28:01.299 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:28:01.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.442+0000 7f8a749fc700 1 Processor -- start 2024-05-08T23:28:01.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.442+0000 7f8a749fc700 1 -- start start 2024-05-08T23:28:01.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.443+0000 7f8a749fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8a7006bd60 0x7f8a700cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.443+0000 7f8a749fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a700cdc90 0x7f8a700b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.443+0000 7f8a749fc700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a700b1200 con 0x7f8a700d13a0 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a749fc700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700cfcc0 con 0x7f8a7006bd60 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a749fc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700ce5c0 con 0x7f8a700cdc90 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6e59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8a700d13a0 0x7f8a700d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34424/0 (socket says 172.21.15.5:34424) 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6e59c700 1 -- 172.21.15.5:0/1265218084 learned_addr learned my addr 172.21.15.5:0/1265218084 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6dd9b700 1 --2- 172.21.15.5:0/1265218084 >> v2:172.21.15.49:3300/0 conn(0x7f8a7006bd60 0x7f8a700cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6ed9d700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a700cdc90 0x7f8a700b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979091139 0 0) 0x7f8a700b1200 con 0x7f8a700d13a0 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a580035b0 con 0x7f8a700d13a0 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2693955022 0 0) 0x7f8a580035b0 con 0x7f8a700d13a0 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a700cdc90 msgr2=0x7f8a700b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a700cdc90 0x7f8a700b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 >> v2:172.21.15.49:3300/0 conn(0x7f8a7006bd60 msgr2=0x7f8a700cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 --2- 172.21.15.5:0/1265218084 >> v2:172.21.15.49:3300/0 conn(0x7f8a7006bd60 0x7f8a700cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700ce8a0 con 0x7f8a700d13a0 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f8a60001670 con 0x7f8a700d13a0 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 >> v1:172.21.15.5:6789/0 conn(0x7f8a700d13a0 legacy=0x7f8a700d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a58003a00 0x7f8a58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a700cdc90 0x7f8a58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58004d90 0x7f8a58005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700cfcc0 con 0x7f8a58004d90 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700ce5c0 con 0x7f8a58003a00 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.444+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5800ba10 con 0x7f8a700cdc90 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6ed9d700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58004d90 0x7f8a58005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6dd9b700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a700cdc90 0x7f8a58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6e59c700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a58003a00 0x7f8a58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6ed9d700 1 -- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a700cdc90 msgr2=0x7f8a58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6ed9d700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a700cdc90 0x7f8a58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6ed9d700 1 -- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a58003a00 msgr2=0x7f8a58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6ed9d700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a58003a00 0x7f8a58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6ed9d700 1 -- 172.21.15.5:0/1265218084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580035b0 con 0x7f8a58004d90 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6ed9d700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58004d90 0x7f8a58005210 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800e170 tx=0x7f8a58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab2490372e74924 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800ec90 con 0x7f8a58004d90 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6800f040 con 0x7f8a58004d90 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.445+0000 7f8a6d59a700 1 -- 172.21.15.5:0/1265218084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68014480 con 0x7f8a58004d90 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.446+0000 7f8a749fc700 1 -- 172.21.15.5:0/1265218084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58004d90 msgr2=0x7f8a58005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.446+0000 7f8a749fc700 1 --2- 172.21.15.5:0/1265218084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58004d90 0x7f8a58005210 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800e170 tx=0x7f8a58005880 comp rx=0 tx=0).stop 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.446+0000 7f8a6ed9d700 1 -- 172.21.15.5:0/1265218084 reap_dead start 2024-05-08T23:28:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.446+0000 7f8a749fc700 1 -- 172.21.15.5:0/1265218084 shutdown_connections 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.446+0000 7f8a749fc700 1 -- 172.21.15.5:0/1265218084 >> 172.21.15.5:0/1265218084 conn(0x7f8a7008afe0 msgr2=0x7f8a7008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.446+0000 7f8a749fc700 1 -- 172.21.15.5:0/1265218084 shutdown_connections 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.446+0000 7f8a749fc700 1 -- 172.21.15.5:0/1265218084 wait complete. 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.448+0000 7f8a749fc700 1 Processor -- start 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.448+0000 7f8a749fc700 1 -- start start 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a749fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a58004d90 0x7f8a700c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a749fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a700c7d80 0x7f8a7014d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a749fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a7014d6e0 0x7f8a7014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a749fc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700ca540 con 0x7f8a700c7d80 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a749fc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7015ee00 con 0x7f8a7014d6e0 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a749fc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7014e0d0 con 0x7f8a58004d90 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a700c7d80 0x7f8a7014d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a700c7d80 0x7f8a7014d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58952/0 (socket says 172.21.15.5:58952) 2024-05-08T23:28:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6dd9b700 1 -- 172.21.15.5:0/2352178005 learned_addr learned my addr 172.21.15.5:0/2352178005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6e59c700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a58004d90 0x7f8a700c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6ed9d700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a7014d6e0 0x7f8a7014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6dd9b700 1 -- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a58004d90 msgr2=0x7f8a700c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6dd9b700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a58004d90 0x7f8a700c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6dd9b700 1 -- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a7014d6e0 msgr2=0x7f8a7014db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6dd9b700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a7014d6e0 0x7f8a7014db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.449+0000 7f8a6dd9b700 1 -- 172.21.15.5:0/2352178005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700c8200 con 0x7f8a700c7d80 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.450+0000 7f8a6dd9b700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a700c7d80 0x7f8a7014d1a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8a64011ad0 tx=0x7f8a64011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.450+0000 7f8a5effd700 1 -- 172.21.15.5:0/2352178005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400f8e0 con 0x7f8a700c7d80 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.450+0000 7f8a5effd700 1 -- 172.21.15.5:0/2352178005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6400fa40 con 0x7f8a700c7d80 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.450+0000 7f8a5effd700 1 -- 172.21.15.5:0/2352178005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400fdf0 con 0x7f8a700c7d80 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.450+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7015e4a0 con 0x7f8a700c7d80 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.450+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7015e740 con 0x7f8a700c7d80 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.451+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7015ef40 con 0x7f8a700c7d80 2024-05-08T23:28:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.452+0000 7f8a5effd700 1 -- 172.21.15.5:0/2352178005 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8a64014440 con 0x7f8a700c7d80 2024-05-08T23:28:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.452+0000 7f8a5effd700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a5006ac90 0x7f8a5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.452+0000 7f8a5effd700 1 -- 172.21.15.5:0/2352178005 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a64089890 con 0x7f8a700c7d80 2024-05-08T23:28:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.452+0000 7f8a6e59c700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a5006ac90 0x7f8a5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.453+0000 7f8a6e59c700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a5006ac90 0x7f8a5006d150 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8a60000fb0 tx=0x7f8a60001e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.457+0000 7f8a5effd700 1 -- 172.21.15.5:0/2352178005 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8a7015ef40 con 0x7f8a700c7d80 2024-05-08T23:28:01.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:01.727+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8a7015ef40 con 0x7f8a700c7d80 2024-05-08T23:28:02.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.285+0000 7f8a5effd700 1 -- 172.21.15.5:0/2352178005 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8a7015ef40 con 0x7f8a700c7d80 2024-05-08T23:28:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a5006ac90 msgr2=0x7f8a5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a5006ac90 0x7f8a5006d150 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8a60000fb0 tx=0x7f8a60001e00 comp rx=0 tx=0).stop 2024-05-08T23:28:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a700c7d80 msgr2=0x7f8a7014d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a700c7d80 0x7f8a7014d1a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8a64011ad0 tx=0x7f8a64011de0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 shutdown_connections 2024-05-08T23:28:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a5006ac90 0x7f8a5006d150 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a7014d6e0 0x7f8a7014db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a58004d90 0x7f8a700c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 --2- 172.21.15.5:0/2352178005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a700c7d80 0x7f8a7014d1a0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 >> 172.21.15.5:0/2352178005 conn(0x7f8a7008afe0 msgr2=0x7f8a7008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 shutdown_connections 2024-05-08T23:28:02.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.288+0000 7f8a749fc700 1 -- 172.21.15.5:0/2352178005 wait complete. 2024-05-08T23:28:02.304 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:28:02.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.446+0000 7ff8276e8700 1 Processor -- start 2024-05-08T23:28:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.446+0000 7ff8276e8700 1 -- start start 2024-05-08T23:28:02.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff8276e8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8200cd4e0 0x7ff82006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff8276e8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff82006f130 0x7ff8200ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff8276e8700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82015c280 con 0x7ff8200ceb20 2024-05-08T23:28:02.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff8276e8700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8200cefb0 con 0x7ff8200cd4e0 2024-05-08T23:28:02.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff8276e8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82006e780 con 0x7ff82006f130 2024-05-08T23:28:02.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff825484700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8200cd4e0 0x7ff82006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff825c85700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff8200ceb20 0x7ff82015e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34426/0 (socket says 172.21.15.5:34426) 2024-05-08T23:28:02.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff825c85700 1 -- 172.21.15.5:0/717312749 learned_addr learned my addr 172.21.15.5:0/717312749 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:02.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff825484700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8200cd4e0 0x7ff82006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58968/0 (socket says 172.21.15.5:58968) 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff824c83700 1 --2- 172.21.15.5:0/717312749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff82006f130 0x7ff8200ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff825484700 1 -- 172.21.15.5:0/717312749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff82006f130 msgr2=0x7ff8200ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff825484700 1 --2- 172.21.15.5:0/717312749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff82006f130 0x7ff8200ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff825484700 1 -- 172.21.15.5:0/717312749 >> v1:172.21.15.5:6789/0 conn(0x7ff8200ceb20 legacy=0x7ff82015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff825484700 1 -- 172.21.15.5:0/717312749 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82006ea60 con 0x7ff8200cd4e0 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.447+0000 7ff817fff700 1 -- 172.21.15.5:0/717312749 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651499183 0 0) 0x7ff82015c280 con 0x7ff8200ceb20 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff825484700 1 --2- 172.21.15.5:0/717312749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200cd4e0 0x7ff82006f910 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009840 tx=0x7ff81c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b917781e8836c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff817fff700 1 -- 172.21.15.5:0/717312749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c011830 con 0x7ff8200cd4e0 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff817fff700 1 -- 172.21.15.5:0/717312749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c011990 con 0x7ff8200cd4e0 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff817fff700 1 -- 172.21.15.5:0/717312749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c011d20 con 0x7ff8200cd4e0 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff8276e8700 1 -- 172.21.15.5:0/717312749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200cd4e0 msgr2=0x7ff82006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff8276e8700 1 --2- 172.21.15.5:0/717312749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200cd4e0 0x7ff82006f910 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009840 tx=0x7ff81c009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff8276e8700 1 -- 172.21.15.5:0/717312749 shutdown_connections 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff8276e8700 1 --2- 172.21.15.5:0/717312749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200cd4e0 0x7ff82006f910 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff8276e8700 1 --2- 172.21.15.5:0/717312749 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff82006f130 0x7ff8200ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff8276e8700 1 -- 172.21.15.5:0/717312749 >> 172.21.15.5:0/717312749 conn(0x7ff8200d0320 msgr2=0x7ff8200d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.448+0000 7ff8276e8700 1 -- 172.21.15.5:0/717312749 shutdown_connections 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.449+0000 7ff8276e8700 1 -- 172.21.15.5:0/717312749 wait complete. 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.450+0000 7ff8276e8700 1 Processor -- start 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.450+0000 7ff8276e8700 1 -- start start 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff8276e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff82006f130 0x7ff8201f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff8276e8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8200cd4e0 0x7ff820158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff8276e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200ceb20 0x7ff8200c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff8276e8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200b0090 con 0x7ff8200ceb20 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff8276e8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200afd40 con 0x7ff82006f130 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff8276e8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201f6f00 con 0x7ff8200cd4e0 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200ceb20 0x7ff8200c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200ceb20 0x7ff8200c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58982/0 (socket says 172.21.15.5:58982) 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 -- 172.21.15.5:0/47893406 learned_addr learned my addr 172.21.15.5:0/47893406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825484700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff82006f130 0x7ff8201f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff824c83700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8200cd4e0 0x7ff820158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 -- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8200cd4e0 msgr2=0x7ff820158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8200cd4e0 0x7ff820158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 -- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff82006f130 msgr2=0x7ff8201f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff82006f130 0x7ff8201f66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 -- 172.21.15.5:0/47893406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0094a0 con 0x7ff8200ceb20 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff825c85700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200ceb20 0x7ff8200c7a60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff81800cba0 tx=0x7ff81800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:02.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff815ffb700 1 -- 172.21.15.5:0/47893406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800aaa0 con 0x7ff8200ceb20 2024-05-08T23:28:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.451+0000 7ff815ffb700 1 -- 172.21.15.5:0/47893406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81800ac20 con 0x7ff8200ceb20 2024-05-08T23:28:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.452+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8201f71e0 con 0x7ff8200ceb20 2024-05-08T23:28:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.452+0000 7ff815ffb700 1 -- 172.21.15.5:0/47893406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8180070d0 con 0x7ff8200ceb20 2024-05-08T23:28:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.452+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201f6c10 con 0x7ff8200ceb20 2024-05-08T23:28:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.453+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8200b0240 con 0x7ff8200ceb20 2024-05-08T23:28:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.453+0000 7ff815ffb700 1 -- 172.21.15.5:0/47893406 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff818007230 con 0x7ff8200ceb20 2024-05-08T23:28:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.454+0000 7ff815ffb700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff80406acc0 0x7ff80406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.454+0000 7ff815ffb700 1 -- 172.21.15.5:0/47893406 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff818088ff0 con 0x7ff8200ceb20 2024-05-08T23:28:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.454+0000 7ff825484700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff80406acc0 0x7ff80406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.455+0000 7ff825484700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff80406acc0 0x7ff80406d180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0090b0 tx=0x7ff81c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.459+0000 7ff815ffb700 1 -- 172.21.15.5:0/47893406 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff8200b0240 con 0x7ff8200ceb20 2024-05-08T23:28:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.726+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff820068000 con 0x7ff8200ceb20 2024-05-08T23:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.727+0000 7ff815ffb700 1 -- 172.21.15.5:0/47893406 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4727 (secure 0 0 0) 0x7ff818018090 con 0x7ff8200ceb20 2024-05-08T23:28:02.727 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:02.727 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":99,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7052,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7058,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:02.279622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff80406acc0 msgr2=0x7ff80406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff80406acc0 0x7ff80406d180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0090b0 tx=0x7ff81c008030 comp rx=0 tx=0).stop 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200ceb20 msgr2=0x7ff8200c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200ceb20 0x7ff8200c7a60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff81800cba0 tx=0x7ff81800ceb0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 shutdown_connections 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff80406acc0 0x7ff80406d180 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8200ceb20 0x7ff8200c7a60 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff82006f130 0x7ff8201f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 --2- 172.21.15.5:0/47893406 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff8200cd4e0 0x7ff820158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 >> 172.21.15.5:0/47893406 conn(0x7ff8200d0320 msgr2=0x7ff8200d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 shutdown_connections 2024-05-08T23:28:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.729+0000 7ff8276e8700 1 -- 172.21.15.5:0/47893406 wait complete. 2024-05-08T23:28:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 99 2024-05-08T23:28:02.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:28:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.879+0000 7f4f5eaec700 1 Processor -- start 2024-05-08T23:28:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.879+0000 7f4f5eaec700 1 -- start start 2024-05-08T23:28:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f5eaec700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 0x7f4f5806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f5eaec700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4f5806fa10 0x7f4f580b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f5eaec700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580cef60 con 0x7f4f5815e310 2024-05-08T23:28:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f5eaec700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5815eee0 con 0x7f4f5806fa10 2024-05-08T23:28:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f5eaec700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580ce980 con 0x7f4f5815dcc0 2024-05-08T23:28:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f5d089700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4f5815e310 0x7f4f580adb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34430/0 (socket says 172.21.15.5:34430) 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f5d089700 1 -- 172.21.15.5:0/2285849678 learned_addr learned my addr 172.21.15.5:0/2285849678 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f5c888700 1 --2- 172.21.15.5:0/2285849678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 0x7f4f5806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f57fff700 1 --2- 172.21.15.5:0/2285849678 >> v2:172.21.15.49:3300/0 conn(0x7f4f5806fa10 0x7f4f580b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f577fe700 1 -- 172.21.15.5:0/2285849678 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569443230 0 0) 0x7f4f580cef60 con 0x7f4f5815e310 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.881+0000 7f4f577fe700 1 -- 172.21.15.5:0/2285849678 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f400035b0 con 0x7f4f5815e310 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f57fff700 1 -- 172.21.15.5:0/2285849678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 msgr2=0x7f4f5806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f57fff700 1 --2- 172.21.15.5:0/2285849678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 0x7f4f5806ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f57fff700 1 -- 172.21.15.5:0/2285849678 >> v1:172.21.15.5:6789/0 conn(0x7f4f5815e310 legacy=0x7f4f580adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f577fe700 1 -- 172.21.15.5:0/2285849678 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1432557908 0 0) 0x7f4f400035b0 con 0x7f4f5815e310 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f57fff700 1 -- 172.21.15.5:0/2285849678 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5815e0a0 con 0x7f4f5806fa10 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f57fff700 1 --2- 172.21.15.5:0/2285849678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5806fa10 0x7f4f580b26d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009840 tx=0x7f4f4c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c76ed8c75f1ebbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f577fe700 1 -- 172.21.15.5:0/2285849678 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f500019f0 con 0x7f4f5806fa10 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f577fe700 1 -- 172.21.15.5:0/2285849678 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c004030 con 0x7f4f5806fa10 2024-05-08T23:28:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.882+0000 7f4f577fe700 1 -- 172.21.15.5:0/2285849678 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0043a0 con 0x7f4f5806fa10 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.883+0000 7f4f5eaec700 1 -- 172.21.15.5:0/2285849678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5806fa10 msgr2=0x7f4f580b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.883+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/2285849678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5806fa10 0x7f4f580b26d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009840 tx=0x7f4f4c009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.883+0000 7f4f5eaec700 1 -- 172.21.15.5:0/2285849678 shutdown_connections 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.883+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/2285849678 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 0x7f4f5806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.883+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/2285849678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5806fa10 0x7f4f580b26d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.883+0000 7f4f5eaec700 1 -- 172.21.15.5:0/2285849678 >> 172.21.15.5:0/2285849678 conn(0x7f4f58099a50 msgr2=0x7f4f580b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.883+0000 7f4f5eaec700 1 -- 172.21.15.5:0/2285849678 shutdown_connections 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.883+0000 7f4f5eaec700 1 -- 172.21.15.5:0/2285849678 wait complete. 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.884+0000 7f4f5eaec700 1 Processor -- start 2024-05-08T23:28:02.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.885+0000 7f4f5eaec700 1 -- start start 2024-05-08T23:28:02.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5eaec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5806fa10 0x7f4f580cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5eaec700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 0x7f4f580c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5eaec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5815e310 0x7f4f580c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5eaec700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580cc290 con 0x7f4f5806fa10 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5eaec700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580ccc60 con 0x7f4f5815e310 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5eaec700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580cb680 con 0x7f4f5815dcc0 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5806fa10 0x7f4f580cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5806fa10 0x7f4f580cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58992/0 (socket says 172.21.15.5:58992) 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 -- 172.21.15.5:0/264021122 learned_addr learned my addr 172.21.15.5:0/264021122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5d089700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5815e310 0x7f4f580c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f57fff700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 0x7f4f580c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 -- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 msgr2=0x7f4f580c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 0x7f4f580c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 -- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5815e310 msgr2=0x7f4f580c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5815e310 0x7f4f580c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 -- 172.21.15.5:0/264021122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c0094a0 con 0x7f4f5806fa10 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.886+0000 7f4f5c888700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5806fa10 0x7f4f580cc720 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800e9b0 tx=0x7f4f4800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.887+0000 7f4f557fa700 1 -- 172.21.15.5:0/264021122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48009710 con 0x7f4f5806fa10 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.887+0000 7f4f557fa700 1 -- 172.21.15.5:0/264021122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4800ee50 con 0x7f4f5806fa10 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.887+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f580b2d90 con 0x7f4f5806fa10 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.887+0000 7f4f557fa700 1 -- 172.21.15.5:0/264021122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800ca50 con 0x7f4f5806fa10 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.887+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f580cbc40 con 0x7f4f5806fa10 2024-05-08T23:28:02.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.888+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f58068000 con 0x7f4f5806fa10 2024-05-08T23:28:02.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.888+0000 7f4f557fa700 1 -- 172.21.15.5:0/264021122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4f4800cbb0 con 0x7f4f5806fa10 2024-05-08T23:28:02.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.889+0000 7f4f557fa700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f4406ac90 0x7f4f4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:02.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.889+0000 7f4f557fa700 1 -- 172.21.15.5:0/264021122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f480897c0 con 0x7f4f5806fa10 2024-05-08T23:28:02.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.889+0000 7f4f57fff700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f4406ac90 0x7f4f4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:02.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.889+0000 7f4f57fff700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f4406ac90 0x7f4f4406d150 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0090b0 tx=0x7f4f4c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:02.893+0000 7f4f557fa700 1 -- 172.21.15.5:0/264021122 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4f48054890 con 0x7f4f5806fa10 2024-05-08T23:28:03.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.114+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f580cd980 con 0x7f4f5806fa10 2024-05-08T23:28:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.115+0000 7f4f557fa700 1 -- 172.21.15.5:0/264021122 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+15944 (secure 0 0 0) 0x7f4f4800ce60 con 0x7f4f5806fa10 2024-05-08T23:28:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:03.195 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:28:00.273637+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:27:57.260067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T23:27:58.243420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f4406ac90 msgr2=0x7f4f4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f4406ac90 0x7f4f4406d150 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0090b0 tx=0x7f4f4c008030 comp rx=0 tx=0).stop 2024-05-08T23:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5806fa10 msgr2=0x7f4f580cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5806fa10 0x7f4f580cc720 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800e9b0 tx=0x7f4f4800ecc0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 shutdown_connections 2024-05-08T23:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f4406ac90 0x7f4f4406d150 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f5815e310 0x7f4f580c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5806fa10 0x7f4f580cc720 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 --2- 172.21.15.5:0/264021122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f5815dcc0 0x7f4f580c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 >> 172.21.15.5:0/264021122 conn(0x7f4f58099a50 msgr2=0x7f4f5806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 shutdown_connections 2024-05-08T23:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.117+0000 7f4f5eaec700 1 -- 172.21.15.5:0/264021122 wait complete. 2024-05-08T23:28:03.198 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:28:03.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:28:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.339+0000 7f063e46e700 1 Processor -- start 2024-05-08T23:28:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.339+0000 7f063e46e700 1 -- start start 2024-05-08T23:28:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.340+0000 7f063e46e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f063806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.340+0000 7f063e46e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063815c280 con 0x7f06380cecf0 2024-05-08T23:28:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.340+0000 7f063e46e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063806e780 con 0x7f063806f130 2024-05-08T23:28:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.340+0000 7f063e46e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063806ea60 con 0x7f06380cd4e0 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.340+0000 7f063ca0b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f06380cecf0 0x7f063815c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34438/0 (socket says 172.21.15.5:34438) 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.340+0000 7f063ca0b700 1 -- 172.21.15.5:0/965412459 learned_addr learned my addr 172.21.15.5:0/965412459 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.340+0000 7f0637fff700 1 --2- 172.21.15.5:0/965412459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f063806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0636ffd700 1 -- 172.21.15.5:0/965412459 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145112092 0 0) 0x7f063815c280 con 0x7f06380cecf0 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0636ffd700 1 -- 172.21.15.5:0/965412459 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06200035b0 con 0x7f06380cecf0 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0636ffd700 1 -- 172.21.15.5:0/965412459 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570861816 0 0) 0x7f063806e780 con 0x7f063806f130 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0636ffd700 1 -- 172.21.15.5:0/965412459 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063815c280 con 0x7f063806f130 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0637fff700 1 -- 172.21.15.5:0/965412459 >> v1:172.21.15.49:6789/0 conn(0x7f063806f130 legacy=0x7f06380ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0637fff700 1 -- 172.21.15.5:0/965412459 >> v1:172.21.15.5:6789/0 conn(0x7f06380cecf0 legacy=0x7f063815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0637fff700 1 -- 172.21.15.5:0/965412459 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063806ed40 con 0x7f06380cd4e0 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0637fff700 1 --2- 172.21.15.5:0/965412459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f063806f910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0628009870 tx=0x7f0628009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ece83d5b5290f86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0636ffd700 1 -- 172.21.15.5:0/965412459 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628011830 con 0x7f06380cd4e0 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0636ffd700 1 -- 172.21.15.5:0/965412459 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0628011990 con 0x7f06380cd4e0 2024-05-08T23:28:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.341+0000 7f0636ffd700 1 -- 172.21.15.5:0/965412459 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628011d20 con 0x7f06380cd4e0 2024-05-08T23:28:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.342+0000 7f063e46e700 1 -- 172.21.15.5:0/965412459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 msgr2=0x7f063806f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.342+0000 7f063e46e700 1 --2- 172.21.15.5:0/965412459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f063806f910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0628009870 tx=0x7f0628009b80 comp rx=0 tx=0).stop 2024-05-08T23:28:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.342+0000 7f063e46e700 1 -- 172.21.15.5:0/965412459 shutdown_connections 2024-05-08T23:28:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.342+0000 7f063e46e700 1 --2- 172.21.15.5:0/965412459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f063806f910 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.342+0000 7f063e46e700 1 -- 172.21.15.5:0/965412459 >> 172.21.15.5:0/965412459 conn(0x7f06380d0320 msgr2=0x7f06380d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.342+0000 7f063e46e700 1 -- 172.21.15.5:0/965412459 shutdown_connections 2024-05-08T23:28:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.342+0000 7f063e46e700 1 -- 172.21.15.5:0/965412459 wait complete. 2024-05-08T23:28:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.343+0000 7f063e46e700 1 Processor -- start 2024-05-08T23:28:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.343+0000 7f063e46e700 1 -- start start 2024-05-08T23:28:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063e46e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f063806f130 0x7f06380cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063e46e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f06380c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063e46e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06380cecf0 0x7f06380b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063e46e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06381500f0 con 0x7f06380cecf0 2024-05-08T23:28:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063e46e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06380cbbf0 con 0x7f063806f130 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063e46e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06380cad00 con 0x7f06380cd4e0 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063ca0b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06380cecf0 0x7f06380b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063ca0b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06380cecf0 0x7f06380b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58998/0 (socket says 172.21.15.5:58998) 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.344+0000 7f063ca0b700 1 -- 172.21.15.5:0/1336988233 learned_addr learned my addr 172.21.15.5:0/1336988233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f0637fff700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f063806f130 0x7f06380cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f06377fe700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f06380c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f063ca0b700 1 -- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 msgr2=0x7f06380c2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f063ca0b700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f06380c2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f063ca0b700 1 -- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f063806f130 msgr2=0x7f06380cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f063ca0b700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f063806f130 0x7f06380cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f063ca0b700 1 -- 172.21.15.5:0/1336988233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06280094a0 con 0x7f06380cecf0 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f06377fe700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f06380c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f063ca0b700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06380cecf0 0x7f06380b0630 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f063000caf0 tx=0x7f063000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f0634ff9700 1 -- 172.21.15.5:0/1336988233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000aad0 con 0x7f06380cecf0 2024-05-08T23:28:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f0634ff9700 1 -- 172.21.15.5:0/1336988233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063000ac30 con 0x7f06380cecf0 2024-05-08T23:28:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06380cafe0 con 0x7f06380cecf0 2024-05-08T23:28:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f0634ff9700 1 -- 172.21.15.5:0/1336988233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630012400 con 0x7f06380cecf0 2024-05-08T23:28:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.345+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06380cb5a0 con 0x7f06380cecf0 2024-05-08T23:28:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.346+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0638068000 con 0x7f06380cecf0 2024-05-08T23:28:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.347+0000 7f0634ff9700 1 -- 172.21.15.5:0/1336988233 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f063001b3b0 con 0x7f06380cecf0 2024-05-08T23:28:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.347+0000 7f0634ff9700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f062406ac90 0x7f062406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.347+0000 7f0634ff9700 1 -- 172.21.15.5:0/1336988233 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0630089070 con 0x7f06380cecf0 2024-05-08T23:28:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.347+0000 7f0637fff700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f062406ac90 0x7f062406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.348+0000 7f0637fff700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f062406ac90 0x7f062406d150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f06280097c0 tx=0x7f062801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.352+0000 7f0634ff9700 1 -- 172.21.15.5:0/1336988233 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0630054260 con 0x7f06380cecf0 2024-05-08T23:28:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.563+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06380d0e40 con 0x7f062406ac90 2024-05-08T23:28:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.572+0000 7f0634ff9700 1 -- 172.21.15.5:0/1336988233 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289737 (secure 0 0 0) 0x7f0628011630 con 0x7f062406ac90 2024-05-08T23:28:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f062406ac90 msgr2=0x7f062406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f062406ac90 0x7f062406d150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f06280097c0 tx=0x7f062801c040 comp rx=0 tx=0).stop 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06380cecf0 msgr2=0x7f06380b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06380cecf0 0x7f06380b0630 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f063000caf0 tx=0x7f063000ceb0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 shutdown_connections 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f062406ac90 0x7f062406d150 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06380cecf0 0x7f06380b0630 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f063806f130 0x7f06380cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 --2- 172.21.15.5:0/1336988233 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f06380cd4e0 0x7f06380c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.575+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 >> 172.21.15.5:0/1336988233 conn(0x7f06380d0320 msgr2=0x7f06380ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.576+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 shutdown_connections 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.576+0000 7f063e46e700 1 -- 172.21.15.5:0/1336988233 wait complete. 2024-05-08T23:28:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:28:03.589 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-05-08T23:28:03.396006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259416,"kb_used_data":10468,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715304,"statfs":{"total":773094113280,"available":772828471296,"internally_reserved":0,"allocated":10719232,"data_stored":5281456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":254791221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.583521"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971135+0000","last_change":"2024-05-08T23:27:59.298932+0000","last_active":"2024-05-08T23:28:01.971135+0000","last_peered":"2024-05-08T23:28:01.971135+0000","last_clean":"2024-05-08T23:28:01.971135+0000","last_became_active":"2024-05-08T23:27:59.298658+0000","last_became_peered":"2024-05-08T23:27:59.298658+0000","last_unstale":"2024-05-08T23:28:01.971135+0000","last_undegraded":"2024-05-08T23:28:01.971135+0000","last_fullsized":"2024-05-08T23:28:01.971135+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:16.213838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280105+0000","last_change":"2024-05-08T23:27:59.302217+0000","last_active":"2024-05-08T23:28:00.280105+0000","last_peered":"2024-05-08T23:28:00.280105+0000","last_clean":"2024-05-08T23:28:00.280105+0000","last_became_active":"2024-05-08T23:27:59.301337+0000","last_became_peered":"2024-05-08T23:27:59.301337+0000","last_unstale":"2024-05-08T23:28:00.280105+0000","last_undegraded":"2024-05-08T23:28:00.280105+0000","last_fullsized":"2024-05-08T23:28:00.280105+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:09.983712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971133+0000","last_change":"2024-05-08T23:27:59.299073+0000","last_active":"2024-05-08T23:28:01.971133+0000","last_peered":"2024-05-08T23:28:01.971133+0000","last_clean":"2024-05-08T23:28:01.971133+0000","last_became_active":"2024-05-08T23:27:59.298788+0000","last_became_peered":"2024-05-08T23:27:59.298788+0000","last_unstale":"2024-05-08T23:28:01.971133+0000","last_undegraded":"2024-05-08T23:28:01.971133+0000","last_fullsized":"2024-05-08T23:28:01.971133+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:26.005470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914829+0000","last_change":"2024-05-08T23:27:59.293241+0000","last_active":"2024-05-08T23:28:01.914829+0000","last_peered":"2024-05-08T23:28:01.914829+0000","last_clean":"2024-05-08T23:28:01.914829+0000","last_became_active":"2024-05-08T23:27:59.292812+0000","last_became_peered":"2024-05-08T23:27:59.292812+0000","last_unstale":"2024-05-08T23:28:01.914829+0000","last_undegraded":"2024-05-08T23:28:01.914829+0000","last_fullsized":"2024-05-08T23:28:01.914829+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:08.872414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971096+0000","last_change":"2024-05-08T23:27:59.299007+0000","last_active":"2024-05-08T23:28:01.971096+0000","last_peered":"2024-05-08T23:28:01.971096+0000","last_clean":"2024-05-08T23:28:01.971096+0000","last_became_active":"2024-05-08T23:27:59.298460+0000","last_became_peered":"2024-05-08T23:27:59.298460+0000","last_unstale":"2024-05-08T23:28:01.971096+0000","last_undegraded":"2024-05-08T23:28:01.971096+0000","last_fullsized":"2024-05-08T23:28:01.971096+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:04.189948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970533+0000","last_change":"2024-05-08T23:27:59.296875+0000","last_active":"2024-05-08T23:28:01.970533+0000","last_peered":"2024-05-08T23:28:01.970533+0000","last_clean":"2024-05-08T23:28:01.970533+0000","last_became_active":"2024-05-08T23:27:59.296235+0000","last_became_peered":"2024-05-08T23:27:59.296235+0000","last_unstale":"2024-05-08T23:28:01.970533+0000","last_undegraded":"2024-05-08T23:28:01.970533+0000","last_fullsized":"2024-05-08T23:28:01.970533+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:04.189948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970314+0000","last_change":"2024-05-08T23:27:59.296574+0000","last_active":"2024-05-08T23:28:01.970314+0000","last_peered":"2024-05-08T23:28:01.970314+0000","last_clean":"2024-05-08T23:28:01.970314+0000","last_became_active":"2024-05-08T23:27:59.296082+0000","last_became_peered":"2024-05-08T23:27:59.296082+0000","last_unstale":"2024-05-08T23:28:01.970314+0000","last_undegraded":"2024-05-08T23:28:01.970314+0000","last_fullsized":"2024-05-08T23:28:01.970314+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:20.161397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280233+0000","last_change":"2024-05-08T23:27:59.299388+0000","last_active":"2024-05-08T23:28:00.280233+0000","last_peered":"2024-05-08T23:28:00.280233+0000","last_clean":"2024-05-08T23:28:00.280233+0000","last_became_active":"2024-05-08T23:27:59.292296+0000","last_became_peered":"2024-05-08T23:27:59.292296+0000","last_unstale":"2024-05-08T23:28:00.280233+0000","last_undegraded":"2024-05-08T23:28:00.280233+0000","last_fullsized":"2024-05-08T23:28:00.280233+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:57.640839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281087+0000","last_change":"2024-05-08T23:27:59.293245+0000","last_active":"2024-05-08T23:28:00.281087+0000","last_peered":"2024-05-08T23:28:00.281087+0000","last_clean":"2024-05-08T23:28:00.281087+0000","last_became_active":"2024-05-08T23:27:59.291724+0000","last_became_peered":"2024-05-08T23:27:59.291724+0000","last_unstale":"2024-05-08T23:28:00.281087+0000","last_undegraded":"2024-05-08T23:28:00.281087+0000","last_fullsized":"2024-05-08T23:28:00.281087+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:36.456980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971041+0000","last_change":"2024-05-08T23:27:59.301109+0000","last_active":"2024-05-08T23:28:01.971041+0000","last_peered":"2024-05-08T23:28:01.971041+0000","last_clean":"2024-05-08T23:28:01.971041+0000","last_became_active":"2024-05-08T23:27:59.300828+0000","last_became_peered":"2024-05-08T23:27:59.300828+0000","last_unstale":"2024-05-08T23:28:01.971041+0000","last_undegraded":"2024-05-08T23:28:01.971041+0000","last_fullsized":"2024-05-08T23:28:01.971041+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.892981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970993+0000","last_change":"2024-05-08T23:27:59.295420+0000","last_active":"2024-05-08T23:28:01.970993+0000","last_peered":"2024-05-08T23:28:01.970993+0000","last_clean":"2024-05-08T23:28:01.970993+0000","last_became_active":"2024-05-08T23:27:59.295115+0000","last_became_peered":"2024-05-08T23:27:59.295115+0000","last_unstale":"2024-05-08T23:28:01.970993+0000","last_undegraded":"2024-05-08T23:28:01.970993+0000","last_fullsized":"2024-05-08T23:28:01.970993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:33.408974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280605+0000","last_change":"2024-05-08T23:27:59.295103+0000","last_active":"2024-05-08T23:28:00.280605+0000","last_peered":"2024-05-08T23:28:00.280605+0000","last_clean":"2024-05-08T23:28:00.280605+0000","last_became_active":"2024-05-08T23:27:59.294780+0000","last_became_peered":"2024-05-08T23:27:59.294780+0000","last_unstale":"2024-05-08T23:28:00.280605+0000","last_undegraded":"2024-05-08T23:28:00.280605+0000","last_fullsized":"2024-05-08T23:28:00.280605+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:50.816404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914515+0000","last_change":"2024-05-08T23:27:59.294306+0000","last_active":"2024-05-08T23:28:01.914515+0000","last_peered":"2024-05-08T23:28:01.914515+0000","last_clean":"2024-05-08T23:28:01.914515+0000","last_became_active":"2024-05-08T23:27:59.293793+0000","last_became_peered":"2024-05-08T23:27:59.293793+0000","last_unstale":"2024-05-08T23:28:01.914515+0000","last_undegraded":"2024-05-08T23:28:01.914515+0000","last_fullsized":"2024-05-08T23:28:01.914515+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:51.283363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685501+0000","last_change":"2024-05-08T23:27:59.289470+0000","last_active":"2024-05-08T23:28:01.685501+0000","last_peered":"2024-05-08T23:28:01.685501+0000","last_clean":"2024-05-08T23:28:01.685501+0000","last_became_active":"2024-05-08T23:27:59.289210+0000","last_became_peered":"2024-05-08T23:27:59.289210+0000","last_unstale":"2024-05-08T23:28:01.685501+0000","last_undegraded":"2024-05-08T23:28:01.685501+0000","last_fullsized":"2024-05-08T23:28:01.685501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:06.890524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914457+0000","last_change":"2024-05-08T23:27:59.297560+0000","last_active":"2024-05-08T23:28:01.914457+0000","last_peered":"2024-05-08T23:28:01.914457+0000","last_clean":"2024-05-08T23:28:01.914457+0000","last_became_active":"2024-05-08T23:27:59.297301+0000","last_became_peered":"2024-05-08T23:27:59.297301+0000","last_unstale":"2024-05-08T23:28:01.914457+0000","last_undegraded":"2024-05-08T23:28:01.914457+0000","last_fullsized":"2024-05-08T23:28:01.914457+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:39.006316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970432+0000","last_change":"2024-05-08T23:27:59.301058+0000","last_active":"2024-05-08T23:28:01.970432+0000","last_peered":"2024-05-08T23:28:01.970432+0000","last_clean":"2024-05-08T23:28:01.970432+0000","last_became_active":"2024-05-08T23:27:59.300625+0000","last_became_peered":"2024-05-08T23:27:59.300625+0000","last_unstale":"2024-05-08T23:28:01.970432+0000","last_undegraded":"2024-05-08T23:28:01.970432+0000","last_fullsized":"2024-05-08T23:28:01.970432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:54.092024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280085+0000","last_change":"2024-05-08T23:27:59.304310+0000","last_active":"2024-05-08T23:28:00.280085+0000","last_peered":"2024-05-08T23:28:00.280085+0000","last_clean":"2024-05-08T23:28:00.280085+0000","last_became_active":"2024-05-08T23:27:59.303975+0000","last_became_peered":"2024-05-08T23:27:59.303975+0000","last_unstale":"2024-05-08T23:28:00.280085+0000","last_undegraded":"2024-05-08T23:28:00.280085+0000","last_fullsized":"2024-05-08T23:28:00.280085+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:09.983807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914752+0000","last_change":"2024-05-08T23:27:59.290442+0000","last_active":"2024-05-08T23:28:01.914752+0000","last_peered":"2024-05-08T23:28:01.914752+0000","last_clean":"2024-05-08T23:28:01.914752+0000","last_became_active":"2024-05-08T23:27:59.290132+0000","last_became_peered":"2024-05-08T23:27:59.290132+0000","last_unstale":"2024-05-08T23:28:01.914752+0000","last_undegraded":"2024-05-08T23:28:01.914752+0000","last_fullsized":"2024-05-08T23:28:01.914752+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:28.263816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685717+0000","last_change":"2024-05-08T23:27:59.305713+0000","last_active":"2024-05-08T23:28:01.685717+0000","last_peered":"2024-05-08T23:28:01.685717+0000","last_clean":"2024-05-08T23:28:01.685717+0000","last_became_active":"2024-05-08T23:27:59.305440+0000","last_became_peered":"2024-05-08T23:27:59.305440+0000","last_unstale":"2024-05-08T23:28:01.685717+0000","last_undegraded":"2024-05-08T23:28:01.685717+0000","last_fullsized":"2024-05-08T23:28:01.685717+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:10.417479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280711+0000","last_change":"2024-05-08T23:27:59.290612+0000","last_active":"2024-05-08T23:28:00.280711+0000","last_peered":"2024-05-08T23:28:00.280711+0000","last_clean":"2024-05-08T23:28:00.280711+0000","last_became_active":"2024-05-08T23:27:59.290301+0000","last_became_peered":"2024-05-08T23:27:59.290301+0000","last_unstale":"2024-05-08T23:28:00.280711+0000","last_undegraded":"2024-05-08T23:28:00.280711+0000","last_fullsized":"2024-05-08T23:28:00.280711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:54.092024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685698+0000","last_change":"2024-05-08T23:27:59.297631+0000","last_active":"2024-05-08T23:28:01.685698+0000","last_peered":"2024-05-08T23:28:01.685698+0000","last_clean":"2024-05-08T23:28:01.685698+0000","last_became_active":"2024-05-08T23:27:59.297346+0000","last_became_peered":"2024-05-08T23:27:59.297346+0000","last_unstale":"2024-05-08T23:28:01.685698+0000","last_undegraded":"2024-05-08T23:28:01.685698+0000","last_fullsized":"2024-05-08T23:28:01.685698+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.206273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279766+0000","last_change":"2024-05-08T23:27:59.300694+0000","last_active":"2024-05-08T23:28:00.279766+0000","last_peered":"2024-05-08T23:28:00.279766+0000","last_clean":"2024-05-08T23:28:00.279766+0000","last_became_active":"2024-05-08T23:27:59.297402+0000","last_became_peered":"2024-05-08T23:27:59.297402+0000","last_unstale":"2024-05-08T23:28:00.279766+0000","last_undegraded":"2024-05-08T23:28:00.279766+0000","last_fullsized":"2024-05-08T23:28:00.279766+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:18:46.413950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279773+0000","last_change":"2024-05-08T23:27:59.300443+0000","last_active":"2024-05-08T23:28:00.279773+0000","last_peered":"2024-05-08T23:28:00.279773+0000","last_clean":"2024-05-08T23:28:00.279773+0000","last_became_active":"2024-05-08T23:27:59.297121+0000","last_became_peered":"2024-05-08T23:27:59.297121+0000","last_unstale":"2024-05-08T23:28:00.279773+0000","last_undegraded":"2024-05-08T23:28:00.279773+0000","last_fullsized":"2024-05-08T23:28:00.279773+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:24.154883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970380+0000","last_change":"2024-05-08T23:27:59.297228+0000","last_active":"2024-05-08T23:28:01.970380+0000","last_peered":"2024-05-08T23:28:01.970380+0000","last_clean":"2024-05-08T23:28:01.970380+0000","last_became_active":"2024-05-08T23:27:59.296600+0000","last_became_peered":"2024-05-08T23:27:59.296600+0000","last_unstale":"2024-05-08T23:28:01.970380+0000","last_undegraded":"2024-05-08T23:28:01.970380+0000","last_fullsized":"2024-05-08T23:28:01.970380+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:50.816404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971124+0000","last_change":"2024-05-08T23:27:59.305421+0000","last_active":"2024-05-08T23:28:01.971124+0000","last_peered":"2024-05-08T23:28:01.971124+0000","last_clean":"2024-05-08T23:28:01.971124+0000","last_became_active":"2024-05-08T23:27:59.305178+0000","last_became_peered":"2024-05-08T23:27:59.305178+0000","last_unstale":"2024-05-08T23:28:01.971124+0000","last_undegraded":"2024-05-08T23:28:01.971124+0000","last_fullsized":"2024-05-08T23:28:01.971124+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:36.456980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280204+0000","last_change":"2024-05-08T23:27:59.301844+0000","last_active":"2024-05-08T23:28:00.280204+0000","last_peered":"2024-05-08T23:28:00.280204+0000","last_clean":"2024-05-08T23:28:00.280204+0000","last_became_active":"2024-05-08T23:27:59.297764+0000","last_became_peered":"2024-05-08T23:27:59.297764+0000","last_unstale":"2024-05-08T23:28:00.280204+0000","last_undegraded":"2024-05-08T23:28:00.280204+0000","last_fullsized":"2024-05-08T23:28:00.280204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:03.367146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970783+0000","last_change":"2024-05-08T23:27:59.294147+0000","last_active":"2024-05-08T23:28:01.970783+0000","last_peered":"2024-05-08T23:28:01.970783+0000","last_clean":"2024-05-08T23:28:01.970783+0000","last_became_active":"2024-05-08T23:27:59.293854+0000","last_became_peered":"2024-05-08T23:27:59.293854+0000","last_unstale":"2024-05-08T23:28:01.970783+0000","last_undegraded":"2024-05-08T23:28:01.970783+0000","last_fullsized":"2024-05-08T23:28:01.970783+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:57.504940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970794+0000","last_change":"2024-05-08T23:27:59.300795+0000","last_active":"2024-05-08T23:28:01.970794+0000","last_peered":"2024-05-08T23:28:01.970794+0000","last_clean":"2024-05-08T23:28:01.970794+0000","last_became_active":"2024-05-08T23:27:59.300416+0000","last_became_peered":"2024-05-08T23:27:59.300416+0000","last_unstale":"2024-05-08T23:28:01.970794+0000","last_undegraded":"2024-05-08T23:28:01.970794+0000","last_fullsized":"2024-05-08T23:28:01.970794+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:59.844824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279884+0000","last_change":"2024-05-08T23:27:59.304016+0000","last_active":"2024-05-08T23:28:00.279884+0000","last_peered":"2024-05-08T23:28:00.279884+0000","last_clean":"2024-05-08T23:28:00.279884+0000","last_became_active":"2024-05-08T23:27:59.303682+0000","last_became_peered":"2024-05-08T23:27:59.303682+0000","last_unstale":"2024-05-08T23:28:00.279884+0000","last_undegraded":"2024-05-08T23:28:00.279884+0000","last_fullsized":"2024-05-08T23:28:00.279884+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:46.774681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685393+0000","last_change":"2024-05-08T23:27:59.306434+0000","last_active":"2024-05-08T23:28:01.685393+0000","last_peered":"2024-05-08T23:28:01.685393+0000","last_clean":"2024-05-08T23:28:01.685393+0000","last_became_active":"2024-05-08T23:27:59.306284+0000","last_became_peered":"2024-05-08T23:27:59.306284+0000","last_unstale":"2024-05-08T23:28:01.685393+0000","last_undegraded":"2024-05-08T23:28:01.685393+0000","last_fullsized":"2024-05-08T23:28:01.685393+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:13.425324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914449+0000","last_change":"2024-05-08T23:27:59.289224+0000","last_active":"2024-05-08T23:28:01.914449+0000","last_peered":"2024-05-08T23:28:01.914449+0000","last_clean":"2024-05-08T23:28:01.914449+0000","last_became_active":"2024-05-08T23:27:59.288961+0000","last_became_peered":"2024-05-08T23:27:59.288961+0000","last_unstale":"2024-05-08T23:28:01.914449+0000","last_undegraded":"2024-05-08T23:28:01.914449+0000","last_fullsized":"2024-05-08T23:28:01.914449+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:41.298532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970721+0000","last_change":"2024-05-08T23:27:59.300321+0000","last_active":"2024-05-08T23:28:01.970721+0000","last_peered":"2024-05-08T23:28:01.970721+0000","last_clean":"2024-05-08T23:28:01.970721+0000","last_became_active":"2024-05-08T23:27:59.300041+0000","last_became_peered":"2024-05-08T23:27:59.300041+0000","last_unstale":"2024-05-08T23:28:01.970721+0000","last_undegraded":"2024-05-08T23:28:01.970721+0000","last_fullsized":"2024-05-08T23:28:01.970721+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:10.417479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685812+0000","last_change":"2024-05-08T23:27:59.296600+0000","last_active":"2024-05-08T23:28:01.685812+0000","last_peered":"2024-05-08T23:28:01.685812+0000","last_clean":"2024-05-08T23:28:01.685812+0000","last_became_active":"2024-05-08T23:27:59.296350+0000","last_became_peered":"2024-05-08T23:27:59.296350+0000","last_unstale":"2024-05-08T23:28:01.685812+0000","last_undegraded":"2024-05-08T23:28:01.685812+0000","last_fullsized":"2024-05-08T23:28:01.685812+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:17.263706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"95'4","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280181+0000","last_change":"2024-05-08T23:27:58.827026+0000","last_active":"2024-05-08T23:28:00.280181+0000","last_peered":"2024-05-08T23:28:00.280181+0000","last_clean":"2024-05-08T23:28:00.280181+0000","last_became_active":"2024-05-08T23:27:58.826560+0000","last_became_peered":"2024-05-08T23:27:58.826560+0000","last_unstale":"2024-05-08T23:28:00.280181+0000","last_undegraded":"2024-05-08T23:28:00.280181+0000","last_fullsized":"2024-05-08T23:28:00.280181+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:03.184914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914901+0000","last_change":"2024-05-08T23:27:59.294072+0000","last_active":"2024-05-08T23:28:01.914901+0000","last_peered":"2024-05-08T23:28:01.914901+0000","last_clean":"2024-05-08T23:28:01.914901+0000","last_became_active":"2024-05-08T23:27:59.293814+0000","last_became_peered":"2024-05-08T23:27:59.293814+0000","last_unstale":"2024-05-08T23:28:01.914901+0000","last_undegraded":"2024-05-08T23:28:01.914901+0000","last_fullsized":"2024-05-08T23:28:01.914901+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:50.816404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970650+0000","last_change":"2024-05-08T23:27:58.832976+0000","last_active":"2024-05-08T23:28:01.970650+0000","last_peered":"2024-05-08T23:28:01.970650+0000","last_clean":"2024-05-08T23:28:01.970650+0000","last_became_active":"2024-05-08T23:27:58.832709+0000","last_became_peered":"2024-05-08T23:27:58.832709+0000","last_unstale":"2024-05-08T23:28:01.970650+0000","last_undegraded":"2024-05-08T23:28:01.970650+0000","last_fullsized":"2024-05-08T23:28:01.970650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:00.882978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971127+0000","last_change":"2024-05-08T23:27:59.294970+0000","last_active":"2024-05-08T23:28:01.971127+0000","last_peered":"2024-05-08T23:28:01.971127+0000","last_clean":"2024-05-08T23:28:01.971127+0000","last_became_active":"2024-05-08T23:27:59.294663+0000","last_became_peered":"2024-05-08T23:27:59.294663+0000","last_unstale":"2024-05-08T23:28:01.971127+0000","last_undegraded":"2024-05-08T23:28:01.971127+0000","last_fullsized":"2024-05-08T23:28:01.971127+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:20.759506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280041+0000","last_change":"2024-05-08T23:27:58.827416+0000","last_active":"2024-05-08T23:28:00.280041+0000","last_peered":"2024-05-08T23:28:00.280041+0000","last_clean":"2024-05-08T23:28:00.280041+0000","last_became_active":"2024-05-08T23:27:58.825207+0000","last_became_peered":"2024-05-08T23:27:58.825207+0000","last_unstale":"2024-05-08T23:28:00.280041+0000","last_undegraded":"2024-05-08T23:28:00.280041+0000","last_fullsized":"2024-05-08T23:28:00.280041+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:11.269051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281150+0000","last_change":"2024-05-08T23:27:59.291143+0000","last_active":"2024-05-08T23:28:00.281150+0000","last_peered":"2024-05-08T23:28:00.281150+0000","last_clean":"2024-05-08T23:28:00.281150+0000","last_became_active":"2024-05-08T23:27:59.290820+0000","last_became_peered":"2024-05-08T23:27:59.290820+0000","last_unstale":"2024-05-08T23:28:00.281150+0000","last_undegraded":"2024-05-08T23:28:00.281150+0000","last_fullsized":"2024-05-08T23:28:00.281150+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:33.408974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280172+0000","last_change":"2024-05-08T23:27:58.829959+0000","last_active":"2024-05-08T23:28:00.280172+0000","last_peered":"2024-05-08T23:28:00.280172+0000","last_clean":"2024-05-08T23:28:00.280172+0000","last_became_active":"2024-05-08T23:27:58.829523+0000","last_became_peered":"2024-05-08T23:27:58.829523+0000","last_unstale":"2024-05-08T23:28:00.280172+0000","last_undegraded":"2024-05-08T23:28:00.280172+0000","last_fullsized":"2024-05-08T23:28:00.280172+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:39.653675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280966+0000","last_change":"2024-05-08T23:27:59.291794+0000","last_active":"2024-05-08T23:28:00.280966+0000","last_peered":"2024-05-08T23:28:00.280966+0000","last_clean":"2024-05-08T23:28:00.280966+0000","last_became_active":"2024-05-08T23:27:59.291501+0000","last_became_peered":"2024-05-08T23:27:59.291501+0000","last_unstale":"2024-05-08T23:28:00.280966+0000","last_undegraded":"2024-05-08T23:28:00.280966+0000","last_fullsized":"2024-05-08T23:28:00.280966+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:56.514212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971150+0000","last_change":"2024-05-08T23:27:58.831902+0000","last_active":"2024-05-08T23:28:01.971150+0000","last_peered":"2024-05-08T23:28:01.971150+0000","last_clean":"2024-05-08T23:28:01.971150+0000","last_became_active":"2024-05-08T23:27:58.831632+0000","last_became_peered":"2024-05-08T23:27:58.831632+0000","last_unstale":"2024-05-08T23:28:01.971150+0000","last_undegraded":"2024-05-08T23:28:01.971150+0000","last_fullsized":"2024-05-08T23:28:01.971150+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:35.903078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970397+0000","last_change":"2024-05-08T23:27:59.294808+0000","last_active":"2024-05-08T23:28:01.970397+0000","last_peered":"2024-05-08T23:28:01.970397+0000","last_clean":"2024-05-08T23:28:01.970397+0000","last_became_active":"2024-05-08T23:27:59.294156+0000","last_became_peered":"2024-05-08T23:27:59.294156+0000","last_unstale":"2024-05-08T23:28:01.970397+0000","last_undegraded":"2024-05-08T23:28:01.970397+0000","last_fullsized":"2024-05-08T23:28:01.970397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:12.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279874+0000","last_change":"2024-05-08T23:27:58.836811+0000","last_active":"2024-05-08T23:28:00.279874+0000","last_peered":"2024-05-08T23:28:00.279874+0000","last_clean":"2024-05-08T23:28:00.279874+0000","last_became_active":"2024-05-08T23:27:58.836478+0000","last_became_peered":"2024-05-08T23:27:58.836478+0000","last_unstale":"2024-05-08T23:28:00.279874+0000","last_undegraded":"2024-05-08T23:28:00.279874+0000","last_fullsized":"2024-05-08T23:28:00.279874+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:19.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914488+0000","last_change":"2024-05-08T23:27:59.290913+0000","last_active":"2024-05-08T23:28:01.914488+0000","last_peered":"2024-05-08T23:28:01.914488+0000","last_clean":"2024-05-08T23:28:01.914488+0000","last_became_active":"2024-05-08T23:27:59.290569+0000","last_became_peered":"2024-05-08T23:27:59.290569+0000","last_unstale":"2024-05-08T23:28:01.914488+0000","last_undegraded":"2024-05-08T23:28:01.914488+0000","last_fullsized":"2024-05-08T23:28:01.914488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:06.902107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970626+0000","last_change":"2024-05-08T23:27:58.836437+0000","last_active":"2024-05-08T23:28:01.970626+0000","last_peered":"2024-05-08T23:28:01.970626+0000","last_clean":"2024-05-08T23:28:01.970626+0000","last_became_active":"2024-05-08T23:27:58.836227+0000","last_became_peered":"2024-05-08T23:27:58.836227+0000","last_unstale":"2024-05-08T23:28:01.970626+0000","last_undegraded":"2024-05-08T23:28:01.970626+0000","last_fullsized":"2024-05-08T23:28:01.970626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:55.509178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280372+0000","last_change":"2024-05-08T23:27:59.292738+0000","last_active":"2024-05-08T23:28:00.280372+0000","last_peered":"2024-05-08T23:28:00.280372+0000","last_clean":"2024-05-08T23:28:00.280372+0000","last_became_active":"2024-05-08T23:27:59.292426+0000","last_became_peered":"2024-05-08T23:27:59.292426+0000","last_unstale":"2024-05-08T23:28:00.280372+0000","last_undegraded":"2024-05-08T23:28:00.280372+0000","last_fullsized":"2024-05-08T23:28:00.280372+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:12.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970866+0000","last_change":"2024-05-08T23:27:58.831459+0000","last_active":"2024-05-08T23:28:01.970866+0000","last_peered":"2024-05-08T23:28:01.970866+0000","last_clean":"2024-05-08T23:28:01.970866+0000","last_became_active":"2024-05-08T23:27:58.830916+0000","last_became_peered":"2024-05-08T23:27:58.830916+0000","last_unstale":"2024-05-08T23:28:01.970866+0000","last_undegraded":"2024-05-08T23:28:01.970866+0000","last_fullsized":"2024-05-08T23:28:01.970866+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:06:04.332019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280311+0000","last_change":"2024-05-08T23:27:59.301586+0000","last_active":"2024-05-08T23:28:00.280311+0000","last_peered":"2024-05-08T23:28:00.280311+0000","last_clean":"2024-05-08T23:28:00.280311+0000","last_became_active":"2024-05-08T23:27:59.297345+0000","last_became_peered":"2024-05-08T23:27:59.297345+0000","last_unstale":"2024-05-08T23:28:00.280311+0000","last_undegraded":"2024-05-08T23:28:00.280311+0000","last_fullsized":"2024-05-08T23:28:00.280311+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:06.902107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971258+0000","last_change":"2024-05-08T23:27:58.832163+0000","last_active":"2024-05-08T23:28:01.971258+0000","last_peered":"2024-05-08T23:28:01.971258+0000","last_clean":"2024-05-08T23:28:01.971258+0000","last_became_active":"2024-05-08T23:27:58.831828+0000","last_became_peered":"2024-05-08T23:27:58.831828+0000","last_unstale":"2024-05-08T23:28:01.971258+0000","last_undegraded":"2024-05-08T23:28:01.971258+0000","last_fullsized":"2024-05-08T23:28:01.971258+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:16.262919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281199+0000","last_change":"2024-05-08T23:27:59.290199+0000","last_active":"2024-05-08T23:28:00.281199+0000","last_peered":"2024-05-08T23:28:00.281199+0000","last_clean":"2024-05-08T23:28:00.281199+0000","last_became_active":"2024-05-08T23:27:59.289811+0000","last_became_peered":"2024-05-08T23:27:59.289811+0000","last_unstale":"2024-05-08T23:28:00.281199+0000","last_undegraded":"2024-05-08T23:28:00.281199+0000","last_fullsized":"2024-05-08T23:28:00.281199+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:40.658709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914771+0000","last_change":"2024-05-08T23:27:58.837681+0000","last_active":"2024-05-08T23:28:01.914771+0000","last_peered":"2024-05-08T23:28:01.914771+0000","last_clean":"2024-05-08T23:28:01.914771+0000","last_became_active":"2024-05-08T23:27:58.837379+0000","last_became_peered":"2024-05-08T23:27:58.837379+0000","last_unstale":"2024-05-08T23:28:01.914771+0000","last_undegraded":"2024-05-08T23:28:01.914771+0000","last_fullsized":"2024-05-08T23:28:01.914771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:03.184914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970927+0000","last_change":"2024-05-08T23:27:59.297921+0000","last_active":"2024-05-08T23:28:01.970927+0000","last_peered":"2024-05-08T23:28:01.970927+0000","last_clean":"2024-05-08T23:28:01.970927+0000","last_became_active":"2024-05-08T23:27:59.297584+0000","last_became_peered":"2024-05-08T23:27:59.297584+0000","last_unstale":"2024-05-08T23:28:01.970927+0000","last_undegraded":"2024-05-08T23:28:01.970927+0000","last_fullsized":"2024-05-08T23:28:01.970927+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:33.408974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"95'3","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971106+0000","last_change":"2024-05-08T23:27:58.829370+0000","last_active":"2024-05-08T23:28:01.971106+0000","last_peered":"2024-05-08T23:28:01.971106+0000","last_clean":"2024-05-08T23:28:01.971106+0000","last_became_active":"2024-05-08T23:27:58.829043+0000","last_became_peered":"2024-05-08T23:27:58.829043+0000","last_unstale":"2024-05-08T23:28:01.971106+0000","last_undegraded":"2024-05-08T23:28:01.971106+0000","last_fullsized":"2024-05-08T23:28:01.971106+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:20.060441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970884+0000","last_change":"2024-05-08T23:27:59.299100+0000","last_active":"2024-05-08T23:28:01.970884+0000","last_peered":"2024-05-08T23:28:01.970884+0000","last_clean":"2024-05-08T23:28:01.970884+0000","last_became_active":"2024-05-08T23:27:59.298732+0000","last_became_peered":"2024-05-08T23:27:59.298732+0000","last_unstale":"2024-05-08T23:28:01.970884+0000","last_undegraded":"2024-05-08T23:28:01.970884+0000","last_fullsized":"2024-05-08T23:28:01.970884+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:36.456980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"95'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280992+0000","last_change":"2024-05-08T23:27:58.828019+0000","last_active":"2024-05-08T23:28:00.280992+0000","last_peered":"2024-05-08T23:28:00.280992+0000","last_clean":"2024-05-08T23:28:00.280992+0000","last_became_active":"2024-05-08T23:27:58.827686+0000","last_became_peered":"2024-05-08T23:27:58.827686+0000","last_unstale":"2024-05-08T23:28:00.280992+0000","last_undegraded":"2024-05-08T23:28:00.280992+0000","last_fullsized":"2024-05-08T23:28:00.280992+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:57.801469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914526+0000","last_change":"2024-05-08T23:27:59.293374+0000","last_active":"2024-05-08T23:28:01.914526+0000","last_peered":"2024-05-08T23:28:01.914526+0000","last_clean":"2024-05-08T23:28:01.914526+0000","last_became_active":"2024-05-08T23:27:59.292953+0000","last_became_peered":"2024-05-08T23:27:59.292953+0000","last_unstale":"2024-05-08T23:28:01.914526+0000","last_undegraded":"2024-05-08T23:28:01.914526+0000","last_fullsized":"2024-05-08T23:28:01.914526+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:24.207488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685724+0000","last_change":"2024-05-08T23:27:58.828114+0000","last_active":"2024-05-08T23:28:01.685724+0000","last_peered":"2024-05-08T23:28:01.685724+0000","last_clean":"2024-05-08T23:28:01.685724+0000","last_became_active":"2024-05-08T23:27:58.827758+0000","last_became_peered":"2024-05-08T23:27:58.827758+0000","last_unstale":"2024-05-08T23:28:01.685724+0000","last_undegraded":"2024-05-08T23:28:01.685724+0000","last_fullsized":"2024-05-08T23:28:01.685724+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:33.675717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970950+0000","last_change":"2024-05-08T23:27:59.306251+0000","last_active":"2024-05-08T23:28:01.970950+0000","last_peered":"2024-05-08T23:28:01.970950+0000","last_clean":"2024-05-08T23:28:01.970950+0000","last_became_active":"2024-05-08T23:27:59.305973+0000","last_became_peered":"2024-05-08T23:27:59.305973+0000","last_unstale":"2024-05-08T23:28:01.970950+0000","last_undegraded":"2024-05-08T23:28:01.970950+0000","last_fullsized":"2024-05-08T23:28:01.970950+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:13.425324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685647+0000","last_change":"2024-05-08T23:27:58.831736+0000","last_active":"2024-05-08T23:28:01.685647+0000","last_peered":"2024-05-08T23:28:01.685647+0000","last_clean":"2024-05-08T23:28:01.685647+0000","last_became_active":"2024-05-08T23:27:58.831437+0000","last_became_peered":"2024-05-08T23:27:58.831437+0000","last_unstale":"2024-05-08T23:28:01.685647+0000","last_undegraded":"2024-05-08T23:28:01.685647+0000","last_fullsized":"2024-05-08T23:28:01.685647+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:00.717795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280372+0000","last_change":"2024-05-08T23:27:59.287910+0000","last_active":"2024-05-08T23:28:00.280372+0000","last_peered":"2024-05-08T23:28:00.280372+0000","last_clean":"2024-05-08T23:28:00.280372+0000","last_became_active":"2024-05-08T23:27:59.287565+0000","last_became_peered":"2024-05-08T23:27:59.287565+0000","last_unstale":"2024-05-08T23:28:00.280372+0000","last_undegraded":"2024-05-08T23:28:00.280372+0000","last_fullsized":"2024-05-08T23:28:00.280372+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.892981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280478+0000","last_change":"2024-05-08T23:27:58.827223+0000","last_active":"2024-05-08T23:28:00.280478+0000","last_peered":"2024-05-08T23:28:00.280478+0000","last_clean":"2024-05-08T23:28:00.280478+0000","last_became_active":"2024-05-08T23:27:58.826938+0000","last_became_peered":"2024-05-08T23:27:58.826938+0000","last_unstale":"2024-05-08T23:28:00.280478+0000","last_undegraded":"2024-05-08T23:28:00.280478+0000","last_fullsized":"2024-05-08T23:28:00.280478+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:45.066176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685472+0000","last_change":"2024-05-08T23:27:59.296033+0000","last_active":"2024-05-08T23:28:01.685472+0000","last_peered":"2024-05-08T23:28:01.685472+0000","last_clean":"2024-05-08T23:28:01.685472+0000","last_became_active":"2024-05-08T23:27:59.295720+0000","last_became_peered":"2024-05-08T23:27:59.295720+0000","last_unstale":"2024-05-08T23:28:01.685472+0000","last_undegraded":"2024-05-08T23:28:01.685472+0000","last_fullsized":"2024-05-08T23:28:01.685472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:49:01.343367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"95'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971315+0000","last_change":"2024-05-08T23:27:58.831236+0000","last_active":"2024-05-08T23:28:01.971315+0000","last_peered":"2024-05-08T23:28:01.971315+0000","last_clean":"2024-05-08T23:28:01.971315+0000","last_became_active":"2024-05-08T23:27:58.830601+0000","last_became_peered":"2024-05-08T23:27:58.830601+0000","last_unstale":"2024-05-08T23:28:01.971315+0000","last_undegraded":"2024-05-08T23:28:01.971315+0000","last_fullsized":"2024-05-08T23:28:01.971315+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:54.719865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971183+0000","last_change":"2024-05-08T23:27:59.293621+0000","last_active":"2024-05-08T23:28:01.971183+0000","last_peered":"2024-05-08T23:28:01.971183+0000","last_clean":"2024-05-08T23:28:01.971183+0000","last_became_active":"2024-05-08T23:27:59.293356+0000","last_became_peered":"2024-05-08T23:27:59.293356+0000","last_unstale":"2024-05-08T23:28:01.971183+0000","last_undegraded":"2024-05-08T23:28:01.971183+0000","last_fullsized":"2024-05-08T23:28:01.971183+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:40.658709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281217+0000","last_change":"2024-05-08T23:27:58.829032+0000","last_active":"2024-05-08T23:28:00.281217+0000","last_peered":"2024-05-08T23:28:00.281217+0000","last_clean":"2024-05-08T23:28:00.281217+0000","last_became_active":"2024-05-08T23:27:58.828705+0000","last_became_peered":"2024-05-08T23:27:58.828705+0000","last_unstale":"2024-05-08T23:28:00.281217+0000","last_undegraded":"2024-05-08T23:28:00.281217+0000","last_fullsized":"2024-05-08T23:28:00.281217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:58.162125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280875+0000","last_change":"2024-05-08T23:27:59.286820+0000","last_active":"2024-05-08T23:28:00.280875+0000","last_peered":"2024-05-08T23:28:00.280875+0000","last_clean":"2024-05-08T23:28:00.280875+0000","last_became_active":"2024-05-08T23:27:59.286534+0000","last_became_peered":"2024-05-08T23:27:59.286534+0000","last_unstale":"2024-05-08T23:28:00.280875+0000","last_undegraded":"2024-05-08T23:28:00.280875+0000","last_fullsized":"2024-05-08T23:28:00.280875+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:56.836101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970982+0000","last_change":"2024-05-08T23:27:58.823276+0000","last_active":"2024-05-08T23:28:01.970982+0000","last_peered":"2024-05-08T23:28:01.970982+0000","last_clean":"2024-05-08T23:28:01.970982+0000","last_became_active":"2024-05-08T23:27:58.822987+0000","last_became_peered":"2024-05-08T23:27:58.822987+0000","last_unstale":"2024-05-08T23:28:01.970982+0000","last_undegraded":"2024-05-08T23:28:01.970982+0000","last_fullsized":"2024-05-08T23:28:01.970982+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:00.882978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971051+0000","last_change":"2024-05-08T23:27:59.295770+0000","last_active":"2024-05-08T23:28:01.971051+0000","last_peered":"2024-05-08T23:28:01.971051+0000","last_clean":"2024-05-08T23:28:01.971051+0000","last_became_active":"2024-05-08T23:27:59.295429+0000","last_became_peered":"2024-05-08T23:27:59.295429+0000","last_unstale":"2024-05-08T23:28:01.971051+0000","last_undegraded":"2024-05-08T23:28:01.971051+0000","last_fullsized":"2024-05-08T23:28:01.971051+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.892981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280244+0000","last_change":"2024-05-08T23:27:58.829778+0000","last_active":"2024-05-08T23:28:00.280244+0000","last_peered":"2024-05-08T23:28:00.280244+0000","last_clean":"2024-05-08T23:28:00.280244+0000","last_became_active":"2024-05-08T23:27:58.829181+0000","last_became_peered":"2024-05-08T23:27:58.829181+0000","last_unstale":"2024-05-08T23:28:00.280244+0000","last_undegraded":"2024-05-08T23:28:00.280244+0000","last_fullsized":"2024-05-08T23:28:00.280244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:58.839790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685515+0000","last_change":"2024-05-08T23:27:59.292518+0000","last_active":"2024-05-08T23:28:01.685515+0000","last_peered":"2024-05-08T23:28:01.685515+0000","last_clean":"2024-05-08T23:28:01.685515+0000","last_became_active":"2024-05-08T23:27:59.292224+0000","last_became_peered":"2024-05-08T23:27:59.292224+0000","last_unstale":"2024-05-08T23:28:01.685515+0000","last_undegraded":"2024-05-08T23:28:01.685515+0000","last_fullsized":"2024-05-08T23:28:01.685515+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:20.759506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279908+0000","last_change":"2024-05-08T23:27:58.827556+0000","last_active":"2024-05-08T23:28:00.279908+0000","last_peered":"2024-05-08T23:28:00.279908+0000","last_clean":"2024-05-08T23:28:00.279908+0000","last_became_active":"2024-05-08T23:27:58.825365+0000","last_became_peered":"2024-05-08T23:27:58.825365+0000","last_unstale":"2024-05-08T23:28:00.279908+0000","last_undegraded":"2024-05-08T23:28:00.279908+0000","last_fullsized":"2024-05-08T23:28:00.279908+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:20.792869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685616+0000","last_change":"2024-05-08T23:27:59.304356+0000","last_active":"2024-05-08T23:28:01.685616+0000","last_peered":"2024-05-08T23:28:01.685616+0000","last_clean":"2024-05-08T23:28:01.685616+0000","last_became_active":"2024-05-08T23:27:59.304085+0000","last_became_peered":"2024-05-08T23:27:59.304085+0000","last_unstale":"2024-05-08T23:28:01.685616+0000","last_undegraded":"2024-05-08T23:28:01.685616+0000","last_fullsized":"2024-05-08T23:28:01.685616+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:15.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281105+0000","last_change":"2024-05-08T23:27:58.832192+0000","last_active":"2024-05-08T23:28:00.281105+0000","last_peered":"2024-05-08T23:28:00.281105+0000","last_clean":"2024-05-08T23:28:00.281105+0000","last_became_active":"2024-05-08T23:27:58.831880+0000","last_became_peered":"2024-05-08T23:27:58.831880+0000","last_unstale":"2024-05-08T23:28:00.281105+0000","last_undegraded":"2024-05-08T23:28:00.281105+0000","last_fullsized":"2024-05-08T23:28:00.281105+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:12.627757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685436+0000","last_change":"2024-05-08T23:27:59.291895+0000","last_active":"2024-05-08T23:28:01.685436+0000","last_peered":"2024-05-08T23:28:01.685436+0000","last_clean":"2024-05-08T23:28:01.685436+0000","last_became_active":"2024-05-08T23:27:59.291597+0000","last_became_peered":"2024-05-08T23:27:59.291597+0000","last_unstale":"2024-05-08T23:28:01.685436+0000","last_undegraded":"2024-05-08T23:28:01.685436+0000","last_fullsized":"2024-05-08T23:28:01.685436+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:08.063795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281193+0000","last_change":"2024-05-08T23:27:58.825191+0000","last_active":"2024-05-08T23:28:00.281193+0000","last_peered":"2024-05-08T23:28:00.281193+0000","last_clean":"2024-05-08T23:28:00.281193+0000","last_became_active":"2024-05-08T23:27:58.824897+0000","last_became_peered":"2024-05-08T23:27:58.824897+0000","last_unstale":"2024-05-08T23:28:00.281193+0000","last_undegraded":"2024-05-08T23:28:00.281193+0000","last_fullsized":"2024-05-08T23:28:00.281193+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:27.916838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970514+0000","last_change":"2024-05-08T23:27:59.301272+0000","last_active":"2024-05-08T23:28:01.970514+0000","last_peered":"2024-05-08T23:28:01.970514+0000","last_clean":"2024-05-08T23:28:01.970514+0000","last_became_active":"2024-05-08T23:27:59.300973+0000","last_became_peered":"2024-05-08T23:27:59.300973+0000","last_unstale":"2024-05-08T23:28:01.970514+0000","last_undegraded":"2024-05-08T23:28:01.970514+0000","last_fullsized":"2024-05-08T23:28:01.970514+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:57.504940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279771+0000","last_change":"2024-05-08T23:27:58.835661+0000","last_active":"2024-05-08T23:28:00.279771+0000","last_peered":"2024-05-08T23:28:00.279771+0000","last_clean":"2024-05-08T23:28:00.279771+0000","last_became_active":"2024-05-08T23:27:58.835336+0000","last_became_peered":"2024-05-08T23:27:58.835336+0000","last_unstale":"2024-05-08T23:28:00.279771+0000","last_undegraded":"2024-05-08T23:28:00.279771+0000","last_fullsized":"2024-05-08T23:28:00.279771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:53.927547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280374+0000","last_change":"2024-05-08T23:27:59.302039+0000","last_active":"2024-05-08T23:28:00.280374+0000","last_peered":"2024-05-08T23:28:00.280374+0000","last_clean":"2024-05-08T23:28:00.280374+0000","last_became_active":"2024-05-08T23:27:59.300994+0000","last_became_peered":"2024-05-08T23:27:59.300994+0000","last_unstale":"2024-05-08T23:28:00.280374+0000","last_undegraded":"2024-05-08T23:28:00.280374+0000","last_fullsized":"2024-05-08T23:28:00.280374+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:10.376057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281167+0000","last_change":"2024-05-08T23:27:58.827693+0000","last_active":"2024-05-08T23:28:00.281167+0000","last_peered":"2024-05-08T23:28:00.281167+0000","last_clean":"2024-05-08T23:28:00.281167+0000","last_became_active":"2024-05-08T23:27:58.827394+0000","last_became_peered":"2024-05-08T23:27:58.827394+0000","last_unstale":"2024-05-08T23:28:00.281167+0000","last_undegraded":"2024-05-08T23:28:00.281167+0000","last_fullsized":"2024-05-08T23:28:00.281167+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:15.208804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971204+0000","last_change":"2024-05-08T23:27:59.294527+0000","last_active":"2024-05-08T23:28:01.971204+0000","last_peered":"2024-05-08T23:28:01.971204+0000","last_clean":"2024-05-08T23:28:01.971204+0000","last_became_active":"2024-05-08T23:27:59.294255+0000","last_became_peered":"2024-05-08T23:27:59.294255+0000","last_unstale":"2024-05-08T23:28:01.971204+0000","last_undegraded":"2024-05-08T23:28:01.971204+0000","last_fullsized":"2024-05-08T23:28:01.971204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:15.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280417+0000","last_change":"2024-05-08T23:27:58.825434+0000","last_active":"2024-05-08T23:28:00.280417+0000","last_peered":"2024-05-08T23:28:00.280417+0000","last_clean":"2024-05-08T23:28:00.280417+0000","last_became_active":"2024-05-08T23:27:58.825108+0000","last_became_peered":"2024-05-08T23:27:58.825108+0000","last_unstale":"2024-05-08T23:28:00.280417+0000","last_undegraded":"2024-05-08T23:28:00.280417+0000","last_fullsized":"2024-05-08T23:28:00.280417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:56.499907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971233+0000","last_change":"2024-05-08T23:27:59.299313+0000","last_active":"2024-05-08T23:28:01.971233+0000","last_peered":"2024-05-08T23:28:01.971233+0000","last_clean":"2024-05-08T23:28:01.971233+0000","last_became_active":"2024-05-08T23:27:59.298879+0000","last_became_peered":"2024-05-08T23:27:59.298879+0000","last_unstale":"2024-05-08T23:28:01.971233+0000","last_undegraded":"2024-05-08T23:28:01.971233+0000","last_fullsized":"2024-05-08T23:28:01.971233+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:37.758468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280143+0000","last_change":"2024-05-08T23:27:58.829411+0000","last_active":"2024-05-08T23:28:00.280143+0000","last_peered":"2024-05-08T23:28:00.280143+0000","last_clean":"2024-05-08T23:28:00.280143+0000","last_became_active":"2024-05-08T23:27:58.829095+0000","last_became_peered":"2024-05-08T23:27:58.829095+0000","last_unstale":"2024-05-08T23:28:00.280143+0000","last_undegraded":"2024-05-08T23:28:00.280143+0000","last_fullsized":"2024-05-08T23:28:00.280143+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:31.394993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970893+0000","last_change":"2024-05-08T23:27:59.298501+0000","last_active":"2024-05-08T23:28:01.970893+0000","last_peered":"2024-05-08T23:28:01.970893+0000","last_clean":"2024-05-08T23:28:01.970893+0000","last_became_active":"2024-05-08T23:27:59.298229+0000","last_became_peered":"2024-05-08T23:27:59.298229+0000","last_unstale":"2024-05-08T23:28:01.970893+0000","last_undegraded":"2024-05-08T23:28:01.970893+0000","last_fullsized":"2024-05-08T23:28:01.970893+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:21.797902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971027+0000","last_change":"2024-05-08T23:27:58.820714+0000","last_active":"2024-05-08T23:28:01.971027+0000","last_peered":"2024-05-08T23:28:01.971027+0000","last_clean":"2024-05-08T23:28:01.971027+0000","last_became_active":"2024-05-08T23:27:58.820379+0000","last_became_peered":"2024-05-08T23:27:58.820379+0000","last_unstale":"2024-05-08T23:28:01.971027+0000","last_undegraded":"2024-05-08T23:28:01.971027+0000","last_fullsized":"2024-05-08T23:28:01.971027+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:29.171114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685718+0000","last_change":"2024-05-08T23:27:59.294148+0000","last_active":"2024-05-08T23:28:01.685718+0000","last_peered":"2024-05-08T23:28:01.685718+0000","last_clean":"2024-05-08T23:28:01.685718+0000","last_became_active":"2024-05-08T23:27:59.293847+0000","last_became_peered":"2024-05-08T23:27:59.293847+0000","last_unstale":"2024-05-08T23:28:01.685718+0000","last_undegraded":"2024-05-08T23:28:01.685718+0000","last_fullsized":"2024-05-08T23:28:01.685718+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:19.303740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279975+0000","last_change":"2024-05-08T23:27:58.828891+0000","last_active":"2024-05-08T23:28:00.279975+0000","last_peered":"2024-05-08T23:28:00.279975+0000","last_clean":"2024-05-08T23:28:00.279975+0000","last_became_active":"2024-05-08T23:27:58.828141+0000","last_became_peered":"2024-05-08T23:27:58.828141+0000","last_unstale":"2024-05-08T23:28:00.279975+0000","last_undegraded":"2024-05-08T23:28:00.279975+0000","last_fullsized":"2024-05-08T23:28:00.279975+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:25.887947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280608+0000","last_change":"2024-05-08T23:27:59.287397+0000","last_active":"2024-05-08T23:28:00.280608+0000","last_peered":"2024-05-08T23:28:00.280608+0000","last_clean":"2024-05-08T23:28:00.280608+0000","last_became_active":"2024-05-08T23:27:59.287104+0000","last_became_peered":"2024-05-08T23:27:59.287104+0000","last_unstale":"2024-05-08T23:28:00.280608+0000","last_undegraded":"2024-05-08T23:28:00.280608+0000","last_fullsized":"2024-05-08T23:28:00.280608+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:59.844824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971576+0000","last_change":"2024-05-08T23:27:58.832142+0000","last_active":"2024-05-08T23:28:01.971576+0000","last_peered":"2024-05-08T23:28:01.971576+0000","last_clean":"2024-05-08T23:28:01.971576+0000","last_became_active":"2024-05-08T23:27:58.831837+0000","last_became_peered":"2024-05-08T23:27:58.831837+0000","last_unstale":"2024-05-08T23:28:01.971576+0000","last_undegraded":"2024-05-08T23:28:01.971576+0000","last_fullsized":"2024-05-08T23:28:01.971576+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:53.692460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685400+0000","last_change":"2024-05-08T23:27:59.295239+0000","last_active":"2024-05-08T23:28:01.685400+0000","last_peered":"2024-05-08T23:28:01.685400+0000","last_clean":"2024-05-08T23:28:01.685400+0000","last_became_active":"2024-05-08T23:27:59.294968+0000","last_became_peered":"2024-05-08T23:27:59.294968+0000","last_unstale":"2024-05-08T23:28:01.685400+0000","last_undegraded":"2024-05-08T23:28:01.685400+0000","last_fullsized":"2024-05-08T23:28:01.685400+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:25.869287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914815+0000","last_change":"2024-05-08T23:27:58.833182+0000","last_active":"2024-05-08T23:28:01.914815+0000","last_peered":"2024-05-08T23:28:01.914815+0000","last_clean":"2024-05-08T23:28:01.914815+0000","last_became_active":"2024-05-08T23:27:58.832862+0000","last_became_peered":"2024-05-08T23:27:58.832862+0000","last_unstale":"2024-05-08T23:28:01.914815+0000","last_undegraded":"2024-05-08T23:28:01.914815+0000","last_fullsized":"2024-05-08T23:28:01.914815+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:19.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970658+0000","last_change":"2024-05-08T23:27:59.298861+0000","last_active":"2024-05-08T23:28:01.970658+0000","last_peered":"2024-05-08T23:28:01.970658+0000","last_clean":"2024-05-08T23:28:01.970658+0000","last_became_active":"2024-05-08T23:27:59.298539+0000","last_became_peered":"2024-05-08T23:27:59.298539+0000","last_unstale":"2024-05-08T23:28:01.970658+0000","last_undegraded":"2024-05-08T23:28:01.970658+0000","last_fullsized":"2024-05-08T23:28:01.970658+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:12.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685527+0000","last_change":"2024-05-08T23:27:58.819783+0000","last_active":"2024-05-08T23:28:01.685527+0000","last_peered":"2024-05-08T23:28:01.685527+0000","last_clean":"2024-05-08T23:28:01.685527+0000","last_became_active":"2024-05-08T23:27:58.819459+0000","last_became_peered":"2024-05-08T23:27:58.819459+0000","last_unstale":"2024-05-08T23:28:01.685527+0000","last_undegraded":"2024-05-08T23:28:01.685527+0000","last_fullsized":"2024-05-08T23:28:01.685527+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:30.876071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685861+0000","last_change":"2024-05-08T23:27:59.291886+0000","last_active":"2024-05-08T23:28:01.685861+0000","last_peered":"2024-05-08T23:28:01.685861+0000","last_clean":"2024-05-08T23:28:01.685861+0000","last_became_active":"2024-05-08T23:27:59.291619+0000","last_became_peered":"2024-05-08T23:27:59.291619+0000","last_unstale":"2024-05-08T23:28:01.685861+0000","last_undegraded":"2024-05-08T23:28:01.685861+0000","last_fullsized":"2024-05-08T23:28:01.685861+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:26.251204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970584+0000","last_change":"2024-05-08T23:27:58.827644+0000","last_active":"2024-05-08T23:28:01.970584+0000","last_peered":"2024-05-08T23:28:01.970584+0000","last_clean":"2024-05-08T23:28:01.970584+0000","last_became_active":"2024-05-08T23:27:58.827360+0000","last_became_peered":"2024-05-08T23:27:58.827360+0000","last_unstale":"2024-05-08T23:28:01.970584+0000","last_undegraded":"2024-05-08T23:28:01.970584+0000","last_fullsized":"2024-05-08T23:28:01.970584+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:45.066176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-08T23:27:59.285266+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:27:59.285266+0000","last_peered":"2024-05-08T23:27:59.285266+0000","last_clean":"2024-05-08T23:27:59.285266+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:27:59.285266+0000","last_undegraded":"2024-05-08T23:27:59.285266+0000","last_fullsized":"2024-05-08T23:27:59.285266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1777664,"data_stored":1098542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.53500000000000003},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80300000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.59899999999999998,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76600000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":1.038,"15min":1.038},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.55700000000000005},"min":{"1min":0.42899999999999999,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.83699999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67600000000000005}]},{"osd":2,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.54800000000000004},"min":{"1min":0.46100000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.625,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78400000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60699999999999998}]},{"osd":3,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.53800000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77800000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63700000000000001,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.70599999999999996,"5min":0.78100000000000003,"15min":0.85499999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77000000000000002,"5min":0.87,"15min":0.87},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75800000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.372,"15min":0.34100000000000003},"max":{"1min":0.78500000000000003,"5min":0.81399999999999995,"15min":0.93500000000000005},"last":0.626}]},{"osd":6,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84599999999999997,"5min":0.86199999999999999,"15min":0.89700000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41499999999999998,"5min":0.33600000000000002,"15min":0.314},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1196032,"data_stored":515014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57099999999999995,"15min":0.53500000000000003},"min":{"1min":0.44,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66400000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58899999999999997,"15min":0.55500000000000005},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.753,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73199999999999998}]},{"osd":1,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.88200000000000001,"15min":0.94999999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.86899999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72999999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.85699999999999998,"15min":1.115},"last":0.66100000000000003}]},{"osd":3,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.46000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78000000000000003,"5min":0.878,"15min":1.0229999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.98899999999999999,"15min":1.0009999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52100000000000002,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.746,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53800000000000003,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.56100000000000005},"min":{"1min":0.34399999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":1.052},"last":0.499},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.60199999999999998},"min":{"1min":0.38100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75900000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73499999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.372,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.67500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":508062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.91000000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.42199999999999999},"min":{"1min":0.28199999999999997,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.55200000000000005,"5min":0.63500000000000001,"15min":0.93100000000000005},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39700000000000002,"15min":0.41199999999999998},"min":{"1min":0.25700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.88400000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42299999999999999,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50600000000000001,"5min":0.61399999999999999,"15min":0.96999999999999997},"last":0.32100000000000001}]},{"osd":3,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.61799999999999999,"5min":0.65800000000000003,"15min":1.01},"last":0.436},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.47499999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59599999999999997,"5min":0.68200000000000005,"15min":1.028},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.39200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.873,"5min":1.004,"15min":1.1479999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66000000000000003,"15min":0.67700000000000005},"min":{"1min":0.51600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96399999999999997,"5min":1.0169999999999999,"15min":1.1579999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.627,"15min":0.64100000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.89000000000000001,"15min":1.0489999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64300000000000002,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.115},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64900000000000002,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.2050000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":1.105},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.69399999999999995},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84999999999999998,"5min":0.89200000000000002,"15min":1.1359999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.47499999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.0609999999999999},"last":0.78500000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574936,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1785856,"data_stored":1105312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.69299999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.53900000000000003,"5min":0.94099999999999995,"15min":7.7610000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40999999999999998,"15min":0.68600000000000005},"min":{"1min":0.249,"5min":0.249,"15min":0.24399999999999999},"max":{"1min":0.52100000000000002,"5min":0.65800000000000003,"15min":7.9889999999999999},"last":0.33900000000000002}]},{"osd":2,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41699999999999998,"15min":0.745},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.627,"5min":0.88500000000000001,"15min":7.7869999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.71899999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.627,"5min":0.71399999999999997,"15min":8.0519999999999996},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.32200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.50900000000000001,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.48499999999999999,"15min":0.76500000000000001},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59699999999999998,"5min":1.9770000000000001,"15min":7.6440000000000001},"last":0.437}]},{"osd":4,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.90500000000000003},"min":{"1min":0.41899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":7.9139999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61899999999999999,"15min":0.879},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96199999999999997,"5min":2.069,"15min":7.1879999999999997},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64600000000000002,"15min":0.88600000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83499999999999996,"5min":2.0350000000000001,"15min":7.1120000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66500000000000004,"15min":0.97699999999999998},"min":{"1min":0.503,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":7.6970000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.94399999999999995},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89200000000000002,"5min":1.1759999999999999,"15min":7.734},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.92400000000000004},"min":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.88,"15min":8.0220000000000002},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.93600000000000005},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.872,"5min":1.5369999999999999,"15min":7.9539999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.89500000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91300000000000003,"5min":3.1139999999999999,"15min":7.1600000000000001},"last":0.62}]}]},{"osd":2,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1191936,"data_stored":514396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.56399999999999995,"5min":0.621,"15min":0.621},"last":0.442},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57599999999999996,"5min":0.65800000000000003,"15min":0.68400000000000005},"last":0.373}]},{"osd":1,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45500000000000002,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.61799999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.35399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42299999999999999}]},{"osd":3,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55900000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.495,"15min":0.46600000000000003},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60499999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.40500000000000003}]},{"osd":4,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.434,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.77200000000000002,"5min":1.024,"15min":1.024},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85499999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.46300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79500000000000004,"5min":0.97899999999999998,"15min":1.0549999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65300000000000002,"15min":0.67400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85499999999999998,"5min":0.89200000000000002,"15min":1.159},"last":0.499},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.50700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53500000000000003}]},{"osd":7,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.67200000000000004},"min":{"1min":0.52100000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89700000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91900000000000004,"5min":0.997,"15min":1.194},"last":0.55400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1191936,"data_stored":509648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.41299999999999998},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":1.46},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":1.2749999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.28699999999999998,"15min":0.27700000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45000000000000001,"15min":0.46800000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.5649999999999999},"last":0.442}]},{"osd":2,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45200000000000001,"15min":0.80900000000000005},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":11.964},"last":0.63},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63300000000000001,"5min":0.73899999999999999,"15min":1.593},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63,"15min":0.65900000000000003},"min":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.71199999999999997},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.214,"5min":1.214,"15min":2.4590000000000001},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64600000000000002,"15min":0.66500000000000004},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":1.536},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65800000000000003,"15min":0.68200000000000005},"min":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.3100000000000001},"last":0.89700000000000002}]},{"osd":6,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64700000000000002,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.5029999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67300000000000004,"15min":0.69899999999999995},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68600000000000005,"15min":0.76300000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":2.5230000000000001},"last":0.91000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574937,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1208320,"data_stored":516050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54900000000000004,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":2.0209999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55700000000000005,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":4.0960000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.57799999999999996,"15min":0.66400000000000003},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":2.145},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.67600000000000005},"min":{"1min":0.40500000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.85299999999999998,"5min":0.88500000000000001,"15min":2.2530000000000001},"last":0.621}]},{"osd":2,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57599999999999996,"15min":0.67600000000000005},"min":{"1min":0.42199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.3769999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.77000000000000002},"min":{"1min":0.44400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":4.0389999999999997},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.0800000000000001},"last":0.439},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":1.079},"min":{"1min":0.47099999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":12.127000000000001},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.5,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79700000000000004,"5min":0.80200000000000005,"15min":0.877},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.61499999999999999},"min":{"1min":0.378,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.93700000000000006},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.624,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57599999999999996,"15min":0.624},"min":{"1min":0.34799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60799999999999998,"15min":0.68899999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.877,"5min":0.877,"15min":1.24},"last":0.68300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1191936,"data_stored":514432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.69999999999999996},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.96099999999999997,"15min":4.6710000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59199999999999997,"15min":0.86599999999999999},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.30499999999999999},"max":{"1min":0.77200000000000002,"5min":0.93400000000000005,"15min":8.798},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57599999999999996,"15min":0.75600000000000001},"min":{"1min":0.41999999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":4.71},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60999999999999999,"15min":0.93600000000000005},"min":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0189999999999999,"15min":8.6880000000000006},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.746},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":1.002,"15min":4.8940000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.77700000000000002},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.443},"max":{"1min":0.81100000000000005,"5min":1.0349999999999999,"15min":4.8559999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.90200000000000002},"min":{"1min":0.5,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79200000000000004,"5min":0.91800000000000004,"15min":8.8770000000000007},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.97699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":1.214,"15min":10.427},"last":0.626}]},{"osd":4,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.496,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68100000000000005,"5min":0.874,"15min":0.874},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.56899999999999995},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72399999999999998,"5min":0.86799999999999999,"15min":0.95899999999999996},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.57499999999999996},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88400000000000001,"5min":1.141,"15min":1.141},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48599999999999999}]},{"osd":7,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34799999999999998},"max":{"1min":0.82999999999999996,"5min":1.01,"15min":1.01},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":1.135,"15min":1.135},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:28:03.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:28:03.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.768+0000 7fdff27f4700 1 Processor -- start 2024-05-08T23:28:03.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.768+0000 7fdff27f4700 1 -- start start 2024-05-08T23:28:03.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.769+0000 7fdff27f4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 0x7fdfec099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.769+0000 7fdff27f4700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdfec099f90 0x7fdfec06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.769+0000 7fdff27f4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec09a3e0 con 0x7fdfec15ee40 2024-05-08T23:28:03.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.769+0000 7fdff27f4700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec0cd910 con 0x7fdfec099f90 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.769+0000 7fdff27f4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec06f810 con 0x7fdfec0cc370 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.769+0000 7fdfebfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdfec15ee40 0x7fdfec0cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34452/0 (socket says 172.21.15.5:34452) 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.769+0000 7fdfebfff700 1 -- 172.21.15.5:0/770461875 learned_addr learned my addr 172.21.15.5:0/770461875 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeb7fe700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 0x7fdfec099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdff0d91700 1 --2- 172.21.15.5:0/770461875 >> v2:172.21.15.49:3300/0 conn(0x7fdfec099f90 0x7fdfec06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803711035 0 0) 0x7fdfdc001110 con 0x7fdfec15ee40 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd40035b0 con 0x7fdfec15ee40 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 457142142 0 0) 0x7fdfd40035b0 con 0x7fdfec15ee40 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 msgr2=0x7fdfec099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 0x7fdfec099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 >> v2:172.21.15.49:3300/0 conn(0x7fdfec099f90 msgr2=0x7fdfec06fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 --2- 172.21.15.5:0/770461875 >> v2:172.21.15.49:3300/0 conn(0x7fdfec099f90 0x7fdfec06fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec06eee0 con 0x7fdfec15ee40 2024-05-08T23:28:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fdfdc001800 con 0x7fdfec15ee40 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeb7fe700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 0x7fdfec099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 >> v1:172.21.15.5:6789/0 conn(0x7fdfec15ee40 legacy=0x7fdfec0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd4003a00 0x7fdfd4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd4004850 0x7fdfd4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 0x7fdfd4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec06f810 con 0x7fdfd4003a00 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd400ba10 con 0x7fdfd4004850 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd400bb50 con 0x7fdfec0cc370 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfebfff700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd4003a00 0x7fdfd4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdfeb7fe700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd4004850 0x7fdfd4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.770+0000 7fdff0d91700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 0x7fdfd4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfebfff700 1 -- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 msgr2=0x7fdfd4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfebfff700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfec0cc370 0x7fdfd4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfebfff700 1 -- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd4004850 msgr2=0x7fdfd4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfebfff700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd4004850 0x7fdfd4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfebfff700 1 -- 172.21.15.5:0/770461875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe00094a0 con 0x7fdfd4003a00 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfebfff700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd4003a00 0x7fdfd4003e20 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c190 tx=0x7fdfd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356556e64ac3cc33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc007d00 con 0x7fdfd4003a00 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc00cc70 con 0x7fdfd4003a00 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdfeaffd700 1 -- 172.21.15.5:0/770461875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc005960 con 0x7fdfd4003a00 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdff27f4700 1 -- 172.21.15.5:0/770461875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd4003a00 msgr2=0x7fdfd4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdff27f4700 1 --2- 172.21.15.5:0/770461875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd4003a00 0x7fdfd4003e20 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c190 tx=0x7fdfd4005880 comp rx=0 tx=0).stop 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdff0d91700 1 -- 172.21.15.5:0/770461875 reap_dead start 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdff27f4700 1 -- 172.21.15.5:0/770461875 shutdown_connections 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.771+0000 7fdff27f4700 1 -- 172.21.15.5:0/770461875 >> 172.21.15.5:0/770461875 conn(0x7fdfec0b3010 msgr2=0x7fdfec0b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.772+0000 7fdff27f4700 1 -- 172.21.15.5:0/770461875 shutdown_connections 2024-05-08T23:28:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.772+0000 7fdff27f4700 1 -- 172.21.15.5:0/770461875 wait complete. 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.773+0000 7fdff27f4700 1 Processor -- start 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.773+0000 7fdff27f4700 1 -- start start 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff27f4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfd4003a00 0x7fdfec0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff27f4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfec0af230 0x7fdfec0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff27f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfec0caa40 0x7fdfec0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff27f4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0cb810 con 0x7fdfec0caa40 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff27f4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec149240 con 0x7fdfec0af230 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff27f4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0caf70 con 0x7fdfd4003a00 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff0d91700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfec0caa40 0x7fdfec0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff0d91700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfec0caa40 0x7fdfec0c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59016/0 (socket says 172.21.15.5:59016) 2024-05-08T23:28:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff0d91700 1 -- 172.21.15.5:0/3741584808 learned_addr learned my addr 172.21.15.5:0/3741584808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdfeb7fe700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfec0af230 0x7fdfec0c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdfebfff700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfd4003a00 0x7fdfec0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff0d91700 1 -- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfd4003a00 msgr2=0x7fdfec0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff0d91700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfd4003a00 0x7fdfec0ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff0d91700 1 -- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfec0af230 msgr2=0x7fdfec0c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff0d91700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfec0af230 0x7fdfec0c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.774+0000 7fdff0d91700 1 -- 172.21.15.5:0/3741584808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0cb250 con 0x7fdfec0caa40 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.775+0000 7fdff0d91700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfec0caa40 0x7fdfec0c8b30 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0723c0 tx=0x7fdfe4011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.775+0000 7fdfe8ff9700 1 -- 172.21.15.5:0/3741584808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400e040 con 0x7fdfec0caa40 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.775+0000 7fdfe8ff9700 1 -- 172.21.15.5:0/3741584808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe400f040 con 0x7fdfec0caa40 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.775+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec0cb530 con 0x7fdfec0caa40 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.775+0000 7fdfe8ff9700 1 -- 172.21.15.5:0/3741584808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400fe40 con 0x7fdfec0caa40 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.775+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec148c30 con 0x7fdfec0caa40 2024-05-08T23:28:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.776+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec1493f0 con 0x7fdfec0caa40 2024-05-08T23:28:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.776+0000 7fdfe8ff9700 1 -- 172.21.15.5:0/3741584808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdfe4022460 con 0x7fdfec0caa40 2024-05-08T23:28:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.777+0000 7fdfe8ff9700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfd806ac90 0x7fdfd806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.777+0000 7fdfe8ff9700 1 -- 172.21.15.5:0/3741584808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfe4089700 con 0x7fdfec0caa40 2024-05-08T23:28:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.777+0000 7fdfebfff700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfd806ac90 0x7fdfd806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.778+0000 7fdfebfff700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfd806ac90 0x7fdfd806d150 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc015040 tx=0x7fdfdc00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.782+0000 7fdfe8ff9700 1 -- 172.21.15.5:0/3741584808 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdfec1493f0 con 0x7fdfec0caa40 2024-05-08T23:28:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:03.993+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfec06f4a0 con 0x7fdfd806ac90 2024-05-08T23:28:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.002+0000 7fdfe8ff9700 1 -- 172.21.15.5:0/3741584808 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289737 (secure 0 0 0) 0x7fdfdc023070 con 0x7fdfd806ac90 2024-05-08T23:28:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfd806ac90 msgr2=0x7fdfd806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfd806ac90 0x7fdfd806d150 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc015040 tx=0x7fdfdc00a000 comp rx=0 tx=0).stop 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfec0caa40 msgr2=0x7fdfec0c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfec0caa40 0x7fdfec0c8b30 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0723c0 tx=0x7fdfe4011f60 comp rx=0 tx=0).stop 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 shutdown_connections 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdfd806ac90 0x7fdfd806d150 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfec0caa40 0x7fdfec0c8b30 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdfd4003a00 0x7fdfec0ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 --2- 172.21.15.5:0/3741584808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfec0af230 0x7fdfec0c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 >> 172.21.15.5:0/3741584808 conn(0x7fdfec0b3010 msgr2=0x7fdfec0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:04.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 shutdown_connections 2024-05-08T23:28:04.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.006+0000 7fdff27f4700 1 -- 172.21.15.5:0/3741584808 wait complete. 2024-05-08T23:28:04.010 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:28:04.020 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-05-08T23:28:03.396006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259416,"kb_used_data":10468,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715304,"statfs":{"total":773094113280,"available":772828471296,"internally_reserved":0,"allocated":10719232,"data_stored":5281456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":254791221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.583521"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971135+0000","last_change":"2024-05-08T23:27:59.298932+0000","last_active":"2024-05-08T23:28:01.971135+0000","last_peered":"2024-05-08T23:28:01.971135+0000","last_clean":"2024-05-08T23:28:01.971135+0000","last_became_active":"2024-05-08T23:27:59.298658+0000","last_became_peered":"2024-05-08T23:27:59.298658+0000","last_unstale":"2024-05-08T23:28:01.971135+0000","last_undegraded":"2024-05-08T23:28:01.971135+0000","last_fullsized":"2024-05-08T23:28:01.971135+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:16.213838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280105+0000","last_change":"2024-05-08T23:27:59.302217+0000","last_active":"2024-05-08T23:28:00.280105+0000","last_peered":"2024-05-08T23:28:00.280105+0000","last_clean":"2024-05-08T23:28:00.280105+0000","last_became_active":"2024-05-08T23:27:59.301337+0000","last_became_peered":"2024-05-08T23:27:59.301337+0000","last_unstale":"2024-05-08T23:28:00.280105+0000","last_undegraded":"2024-05-08T23:28:00.280105+0000","last_fullsized":"2024-05-08T23:28:00.280105+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:09.983712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971133+0000","last_change":"2024-05-08T23:27:59.299073+0000","last_active":"2024-05-08T23:28:01.971133+0000","last_peered":"2024-05-08T23:28:01.971133+0000","last_clean":"2024-05-08T23:28:01.971133+0000","last_became_active":"2024-05-08T23:27:59.298788+0000","last_became_peered":"2024-05-08T23:27:59.298788+0000","last_unstale":"2024-05-08T23:28:01.971133+0000","last_undegraded":"2024-05-08T23:28:01.971133+0000","last_fullsized":"2024-05-08T23:28:01.971133+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:26.005470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914829+0000","last_change":"2024-05-08T23:27:59.293241+0000","last_active":"2024-05-08T23:28:01.914829+0000","last_peered":"2024-05-08T23:28:01.914829+0000","last_clean":"2024-05-08T23:28:01.914829+0000","last_became_active":"2024-05-08T23:27:59.292812+0000","last_became_peered":"2024-05-08T23:27:59.292812+0000","last_unstale":"2024-05-08T23:28:01.914829+0000","last_undegraded":"2024-05-08T23:28:01.914829+0000","last_fullsized":"2024-05-08T23:28:01.914829+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:08.872414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971096+0000","last_change":"2024-05-08T23:27:59.299007+0000","last_active":"2024-05-08T23:28:01.971096+0000","last_peered":"2024-05-08T23:28:01.971096+0000","last_clean":"2024-05-08T23:28:01.971096+0000","last_became_active":"2024-05-08T23:27:59.298460+0000","last_became_peered":"2024-05-08T23:27:59.298460+0000","last_unstale":"2024-05-08T23:28:01.971096+0000","last_undegraded":"2024-05-08T23:28:01.971096+0000","last_fullsized":"2024-05-08T23:28:01.971096+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:04.189948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970533+0000","last_change":"2024-05-08T23:27:59.296875+0000","last_active":"2024-05-08T23:28:01.970533+0000","last_peered":"2024-05-08T23:28:01.970533+0000","last_clean":"2024-05-08T23:28:01.970533+0000","last_became_active":"2024-05-08T23:27:59.296235+0000","last_became_peered":"2024-05-08T23:27:59.296235+0000","last_unstale":"2024-05-08T23:28:01.970533+0000","last_undegraded":"2024-05-08T23:28:01.970533+0000","last_fullsized":"2024-05-08T23:28:01.970533+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:04.189948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970314+0000","last_change":"2024-05-08T23:27:59.296574+0000","last_active":"2024-05-08T23:28:01.970314+0000","last_peered":"2024-05-08T23:28:01.970314+0000","last_clean":"2024-05-08T23:28:01.970314+0000","last_became_active":"2024-05-08T23:27:59.296082+0000","last_became_peered":"2024-05-08T23:27:59.296082+0000","last_unstale":"2024-05-08T23:28:01.970314+0000","last_undegraded":"2024-05-08T23:28:01.970314+0000","last_fullsized":"2024-05-08T23:28:01.970314+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:20.161397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280233+0000","last_change":"2024-05-08T23:27:59.299388+0000","last_active":"2024-05-08T23:28:00.280233+0000","last_peered":"2024-05-08T23:28:00.280233+0000","last_clean":"2024-05-08T23:28:00.280233+0000","last_became_active":"2024-05-08T23:27:59.292296+0000","last_became_peered":"2024-05-08T23:27:59.292296+0000","last_unstale":"2024-05-08T23:28:00.280233+0000","last_undegraded":"2024-05-08T23:28:00.280233+0000","last_fullsized":"2024-05-08T23:28:00.280233+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:57.640839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281087+0000","last_change":"2024-05-08T23:27:59.293245+0000","last_active":"2024-05-08T23:28:00.281087+0000","last_peered":"2024-05-08T23:28:00.281087+0000","last_clean":"2024-05-08T23:28:00.281087+0000","last_became_active":"2024-05-08T23:27:59.291724+0000","last_became_peered":"2024-05-08T23:27:59.291724+0000","last_unstale":"2024-05-08T23:28:00.281087+0000","last_undegraded":"2024-05-08T23:28:00.281087+0000","last_fullsized":"2024-05-08T23:28:00.281087+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:36.456980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971041+0000","last_change":"2024-05-08T23:27:59.301109+0000","last_active":"2024-05-08T23:28:01.971041+0000","last_peered":"2024-05-08T23:28:01.971041+0000","last_clean":"2024-05-08T23:28:01.971041+0000","last_became_active":"2024-05-08T23:27:59.300828+0000","last_became_peered":"2024-05-08T23:27:59.300828+0000","last_unstale":"2024-05-08T23:28:01.971041+0000","last_undegraded":"2024-05-08T23:28:01.971041+0000","last_fullsized":"2024-05-08T23:28:01.971041+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.892981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970993+0000","last_change":"2024-05-08T23:27:59.295420+0000","last_active":"2024-05-08T23:28:01.970993+0000","last_peered":"2024-05-08T23:28:01.970993+0000","last_clean":"2024-05-08T23:28:01.970993+0000","last_became_active":"2024-05-08T23:27:59.295115+0000","last_became_peered":"2024-05-08T23:27:59.295115+0000","last_unstale":"2024-05-08T23:28:01.970993+0000","last_undegraded":"2024-05-08T23:28:01.970993+0000","last_fullsized":"2024-05-08T23:28:01.970993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:33.408974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280605+0000","last_change":"2024-05-08T23:27:59.295103+0000","last_active":"2024-05-08T23:28:00.280605+0000","last_peered":"2024-05-08T23:28:00.280605+0000","last_clean":"2024-05-08T23:28:00.280605+0000","last_became_active":"2024-05-08T23:27:59.294780+0000","last_became_peered":"2024-05-08T23:27:59.294780+0000","last_unstale":"2024-05-08T23:28:00.280605+0000","last_undegraded":"2024-05-08T23:28:00.280605+0000","last_fullsized":"2024-05-08T23:28:00.280605+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:50.816404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914515+0000","last_change":"2024-05-08T23:27:59.294306+0000","last_active":"2024-05-08T23:28:01.914515+0000","last_peered":"2024-05-08T23:28:01.914515+0000","last_clean":"2024-05-08T23:28:01.914515+0000","last_became_active":"2024-05-08T23:27:59.293793+0000","last_became_peered":"2024-05-08T23:27:59.293793+0000","last_unstale":"2024-05-08T23:28:01.914515+0000","last_undegraded":"2024-05-08T23:28:01.914515+0000","last_fullsized":"2024-05-08T23:28:01.914515+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:51.283363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685501+0000","last_change":"2024-05-08T23:27:59.289470+0000","last_active":"2024-05-08T23:28:01.685501+0000","last_peered":"2024-05-08T23:28:01.685501+0000","last_clean":"2024-05-08T23:28:01.685501+0000","last_became_active":"2024-05-08T23:27:59.289210+0000","last_became_peered":"2024-05-08T23:27:59.289210+0000","last_unstale":"2024-05-08T23:28:01.685501+0000","last_undegraded":"2024-05-08T23:28:01.685501+0000","last_fullsized":"2024-05-08T23:28:01.685501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:06.890524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914457+0000","last_change":"2024-05-08T23:27:59.297560+0000","last_active":"2024-05-08T23:28:01.914457+0000","last_peered":"2024-05-08T23:28:01.914457+0000","last_clean":"2024-05-08T23:28:01.914457+0000","last_became_active":"2024-05-08T23:27:59.297301+0000","last_became_peered":"2024-05-08T23:27:59.297301+0000","last_unstale":"2024-05-08T23:28:01.914457+0000","last_undegraded":"2024-05-08T23:28:01.914457+0000","last_fullsized":"2024-05-08T23:28:01.914457+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:39.006316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970432+0000","last_change":"2024-05-08T23:27:59.301058+0000","last_active":"2024-05-08T23:28:01.970432+0000","last_peered":"2024-05-08T23:28:01.970432+0000","last_clean":"2024-05-08T23:28:01.970432+0000","last_became_active":"2024-05-08T23:27:59.300625+0000","last_became_peered":"2024-05-08T23:27:59.300625+0000","last_unstale":"2024-05-08T23:28:01.970432+0000","last_undegraded":"2024-05-08T23:28:01.970432+0000","last_fullsized":"2024-05-08T23:28:01.970432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:54.092024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280085+0000","last_change":"2024-05-08T23:27:59.304310+0000","last_active":"2024-05-08T23:28:00.280085+0000","last_peered":"2024-05-08T23:28:00.280085+0000","last_clean":"2024-05-08T23:28:00.280085+0000","last_became_active":"2024-05-08T23:27:59.303975+0000","last_became_peered":"2024-05-08T23:27:59.303975+0000","last_unstale":"2024-05-08T23:28:00.280085+0000","last_undegraded":"2024-05-08T23:28:00.280085+0000","last_fullsized":"2024-05-08T23:28:00.280085+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:09.983807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914752+0000","last_change":"2024-05-08T23:27:59.290442+0000","last_active":"2024-05-08T23:28:01.914752+0000","last_peered":"2024-05-08T23:28:01.914752+0000","last_clean":"2024-05-08T23:28:01.914752+0000","last_became_active":"2024-05-08T23:27:59.290132+0000","last_became_peered":"2024-05-08T23:27:59.290132+0000","last_unstale":"2024-05-08T23:28:01.914752+0000","last_undegraded":"2024-05-08T23:28:01.914752+0000","last_fullsized":"2024-05-08T23:28:01.914752+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:28.263816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685717+0000","last_change":"2024-05-08T23:27:59.305713+0000","last_active":"2024-05-08T23:28:01.685717+0000","last_peered":"2024-05-08T23:28:01.685717+0000","last_clean":"2024-05-08T23:28:01.685717+0000","last_became_active":"2024-05-08T23:27:59.305440+0000","last_became_peered":"2024-05-08T23:27:59.305440+0000","last_unstale":"2024-05-08T23:28:01.685717+0000","last_undegraded":"2024-05-08T23:28:01.685717+0000","last_fullsized":"2024-05-08T23:28:01.685717+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:10.417479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280711+0000","last_change":"2024-05-08T23:27:59.290612+0000","last_active":"2024-05-08T23:28:00.280711+0000","last_peered":"2024-05-08T23:28:00.280711+0000","last_clean":"2024-05-08T23:28:00.280711+0000","last_became_active":"2024-05-08T23:27:59.290301+0000","last_became_peered":"2024-05-08T23:27:59.290301+0000","last_unstale":"2024-05-08T23:28:00.280711+0000","last_undegraded":"2024-05-08T23:28:00.280711+0000","last_fullsized":"2024-05-08T23:28:00.280711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:54.092024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685698+0000","last_change":"2024-05-08T23:27:59.297631+0000","last_active":"2024-05-08T23:28:01.685698+0000","last_peered":"2024-05-08T23:28:01.685698+0000","last_clean":"2024-05-08T23:28:01.685698+0000","last_became_active":"2024-05-08T23:27:59.297346+0000","last_became_peered":"2024-05-08T23:27:59.297346+0000","last_unstale":"2024-05-08T23:28:01.685698+0000","last_undegraded":"2024-05-08T23:28:01.685698+0000","last_fullsized":"2024-05-08T23:28:01.685698+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.206273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279766+0000","last_change":"2024-05-08T23:27:59.300694+0000","last_active":"2024-05-08T23:28:00.279766+0000","last_peered":"2024-05-08T23:28:00.279766+0000","last_clean":"2024-05-08T23:28:00.279766+0000","last_became_active":"2024-05-08T23:27:59.297402+0000","last_became_peered":"2024-05-08T23:27:59.297402+0000","last_unstale":"2024-05-08T23:28:00.279766+0000","last_undegraded":"2024-05-08T23:28:00.279766+0000","last_fullsized":"2024-05-08T23:28:00.279766+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:18:46.413950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279773+0000","last_change":"2024-05-08T23:27:59.300443+0000","last_active":"2024-05-08T23:28:00.279773+0000","last_peered":"2024-05-08T23:28:00.279773+0000","last_clean":"2024-05-08T23:28:00.279773+0000","last_became_active":"2024-05-08T23:27:59.297121+0000","last_became_peered":"2024-05-08T23:27:59.297121+0000","last_unstale":"2024-05-08T23:28:00.279773+0000","last_undegraded":"2024-05-08T23:28:00.279773+0000","last_fullsized":"2024-05-08T23:28:00.279773+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:24.154883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970380+0000","last_change":"2024-05-08T23:27:59.297228+0000","last_active":"2024-05-08T23:28:01.970380+0000","last_peered":"2024-05-08T23:28:01.970380+0000","last_clean":"2024-05-08T23:28:01.970380+0000","last_became_active":"2024-05-08T23:27:59.296600+0000","last_became_peered":"2024-05-08T23:27:59.296600+0000","last_unstale":"2024-05-08T23:28:01.970380+0000","last_undegraded":"2024-05-08T23:28:01.970380+0000","last_fullsized":"2024-05-08T23:28:01.970380+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:50.816404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971124+0000","last_change":"2024-05-08T23:27:59.305421+0000","last_active":"2024-05-08T23:28:01.971124+0000","last_peered":"2024-05-08T23:28:01.971124+0000","last_clean":"2024-05-08T23:28:01.971124+0000","last_became_active":"2024-05-08T23:27:59.305178+0000","last_became_peered":"2024-05-08T23:27:59.305178+0000","last_unstale":"2024-05-08T23:28:01.971124+0000","last_undegraded":"2024-05-08T23:28:01.971124+0000","last_fullsized":"2024-05-08T23:28:01.971124+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:36.456980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280204+0000","last_change":"2024-05-08T23:27:59.301844+0000","last_active":"2024-05-08T23:28:00.280204+0000","last_peered":"2024-05-08T23:28:00.280204+0000","last_clean":"2024-05-08T23:28:00.280204+0000","last_became_active":"2024-05-08T23:27:59.297764+0000","last_became_peered":"2024-05-08T23:27:59.297764+0000","last_unstale":"2024-05-08T23:28:00.280204+0000","last_undegraded":"2024-05-08T23:28:00.280204+0000","last_fullsized":"2024-05-08T23:28:00.280204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:03.367146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970783+0000","last_change":"2024-05-08T23:27:59.294147+0000","last_active":"2024-05-08T23:28:01.970783+0000","last_peered":"2024-05-08T23:28:01.970783+0000","last_clean":"2024-05-08T23:28:01.970783+0000","last_became_active":"2024-05-08T23:27:59.293854+0000","last_became_peered":"2024-05-08T23:27:59.293854+0000","last_unstale":"2024-05-08T23:28:01.970783+0000","last_undegraded":"2024-05-08T23:28:01.970783+0000","last_fullsized":"2024-05-08T23:28:01.970783+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:57.504940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970794+0000","last_change":"2024-05-08T23:27:59.300795+0000","last_active":"2024-05-08T23:28:01.970794+0000","last_peered":"2024-05-08T23:28:01.970794+0000","last_clean":"2024-05-08T23:28:01.970794+0000","last_became_active":"2024-05-08T23:27:59.300416+0000","last_became_peered":"2024-05-08T23:27:59.300416+0000","last_unstale":"2024-05-08T23:28:01.970794+0000","last_undegraded":"2024-05-08T23:28:01.970794+0000","last_fullsized":"2024-05-08T23:28:01.970794+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:59.844824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279884+0000","last_change":"2024-05-08T23:27:59.304016+0000","last_active":"2024-05-08T23:28:00.279884+0000","last_peered":"2024-05-08T23:28:00.279884+0000","last_clean":"2024-05-08T23:28:00.279884+0000","last_became_active":"2024-05-08T23:27:59.303682+0000","last_became_peered":"2024-05-08T23:27:59.303682+0000","last_unstale":"2024-05-08T23:28:00.279884+0000","last_undegraded":"2024-05-08T23:28:00.279884+0000","last_fullsized":"2024-05-08T23:28:00.279884+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:46.774681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685393+0000","last_change":"2024-05-08T23:27:59.306434+0000","last_active":"2024-05-08T23:28:01.685393+0000","last_peered":"2024-05-08T23:28:01.685393+0000","last_clean":"2024-05-08T23:28:01.685393+0000","last_became_active":"2024-05-08T23:27:59.306284+0000","last_became_peered":"2024-05-08T23:27:59.306284+0000","last_unstale":"2024-05-08T23:28:01.685393+0000","last_undegraded":"2024-05-08T23:28:01.685393+0000","last_fullsized":"2024-05-08T23:28:01.685393+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:13.425324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914449+0000","last_change":"2024-05-08T23:27:59.289224+0000","last_active":"2024-05-08T23:28:01.914449+0000","last_peered":"2024-05-08T23:28:01.914449+0000","last_clean":"2024-05-08T23:28:01.914449+0000","last_became_active":"2024-05-08T23:27:59.288961+0000","last_became_peered":"2024-05-08T23:27:59.288961+0000","last_unstale":"2024-05-08T23:28:01.914449+0000","last_undegraded":"2024-05-08T23:28:01.914449+0000","last_fullsized":"2024-05-08T23:28:01.914449+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:41.298532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970721+0000","last_change":"2024-05-08T23:27:59.300321+0000","last_active":"2024-05-08T23:28:01.970721+0000","last_peered":"2024-05-08T23:28:01.970721+0000","last_clean":"2024-05-08T23:28:01.970721+0000","last_became_active":"2024-05-08T23:27:59.300041+0000","last_became_peered":"2024-05-08T23:27:59.300041+0000","last_unstale":"2024-05-08T23:28:01.970721+0000","last_undegraded":"2024-05-08T23:28:01.970721+0000","last_fullsized":"2024-05-08T23:28:01.970721+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:10.417479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685812+0000","last_change":"2024-05-08T23:27:59.296600+0000","last_active":"2024-05-08T23:28:01.685812+0000","last_peered":"2024-05-08T23:28:01.685812+0000","last_clean":"2024-05-08T23:28:01.685812+0000","last_became_active":"2024-05-08T23:27:59.296350+0000","last_became_peered":"2024-05-08T23:27:59.296350+0000","last_unstale":"2024-05-08T23:28:01.685812+0000","last_undegraded":"2024-05-08T23:28:01.685812+0000","last_fullsized":"2024-05-08T23:28:01.685812+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:17.263706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"95'4","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280181+0000","last_change":"2024-05-08T23:27:58.827026+0000","last_active":"2024-05-08T23:28:00.280181+0000","last_peered":"2024-05-08T23:28:00.280181+0000","last_clean":"2024-05-08T23:28:00.280181+0000","last_became_active":"2024-05-08T23:27:58.826560+0000","last_became_peered":"2024-05-08T23:27:58.826560+0000","last_unstale":"2024-05-08T23:28:00.280181+0000","last_undegraded":"2024-05-08T23:28:00.280181+0000","last_fullsized":"2024-05-08T23:28:00.280181+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:03.184914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914901+0000","last_change":"2024-05-08T23:27:59.294072+0000","last_active":"2024-05-08T23:28:01.914901+0000","last_peered":"2024-05-08T23:28:01.914901+0000","last_clean":"2024-05-08T23:28:01.914901+0000","last_became_active":"2024-05-08T23:27:59.293814+0000","last_became_peered":"2024-05-08T23:27:59.293814+0000","last_unstale":"2024-05-08T23:28:01.914901+0000","last_undegraded":"2024-05-08T23:28:01.914901+0000","last_fullsized":"2024-05-08T23:28:01.914901+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:50.816404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970650+0000","last_change":"2024-05-08T23:27:58.832976+0000","last_active":"2024-05-08T23:28:01.970650+0000","last_peered":"2024-05-08T23:28:01.970650+0000","last_clean":"2024-05-08T23:28:01.970650+0000","last_became_active":"2024-05-08T23:27:58.832709+0000","last_became_peered":"2024-05-08T23:27:58.832709+0000","last_unstale":"2024-05-08T23:28:01.970650+0000","last_undegraded":"2024-05-08T23:28:01.970650+0000","last_fullsized":"2024-05-08T23:28:01.970650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:00.882978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971127+0000","last_change":"2024-05-08T23:27:59.294970+0000","last_active":"2024-05-08T23:28:01.971127+0000","last_peered":"2024-05-08T23:28:01.971127+0000","last_clean":"2024-05-08T23:28:01.971127+0000","last_became_active":"2024-05-08T23:27:59.294663+0000","last_became_peered":"2024-05-08T23:27:59.294663+0000","last_unstale":"2024-05-08T23:28:01.971127+0000","last_undegraded":"2024-05-08T23:28:01.971127+0000","last_fullsized":"2024-05-08T23:28:01.971127+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:20.759506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280041+0000","last_change":"2024-05-08T23:27:58.827416+0000","last_active":"2024-05-08T23:28:00.280041+0000","last_peered":"2024-05-08T23:28:00.280041+0000","last_clean":"2024-05-08T23:28:00.280041+0000","last_became_active":"2024-05-08T23:27:58.825207+0000","last_became_peered":"2024-05-08T23:27:58.825207+0000","last_unstale":"2024-05-08T23:28:00.280041+0000","last_undegraded":"2024-05-08T23:28:00.280041+0000","last_fullsized":"2024-05-08T23:28:00.280041+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:11.269051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281150+0000","last_change":"2024-05-08T23:27:59.291143+0000","last_active":"2024-05-08T23:28:00.281150+0000","last_peered":"2024-05-08T23:28:00.281150+0000","last_clean":"2024-05-08T23:28:00.281150+0000","last_became_active":"2024-05-08T23:27:59.290820+0000","last_became_peered":"2024-05-08T23:27:59.290820+0000","last_unstale":"2024-05-08T23:28:00.281150+0000","last_undegraded":"2024-05-08T23:28:00.281150+0000","last_fullsized":"2024-05-08T23:28:00.281150+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:33.408974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280172+0000","last_change":"2024-05-08T23:27:58.829959+0000","last_active":"2024-05-08T23:28:00.280172+0000","last_peered":"2024-05-08T23:28:00.280172+0000","last_clean":"2024-05-08T23:28:00.280172+0000","last_became_active":"2024-05-08T23:27:58.829523+0000","last_became_peered":"2024-05-08T23:27:58.829523+0000","last_unstale":"2024-05-08T23:28:00.280172+0000","last_undegraded":"2024-05-08T23:28:00.280172+0000","last_fullsized":"2024-05-08T23:28:00.280172+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:39.653675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280966+0000","last_change":"2024-05-08T23:27:59.291794+0000","last_active":"2024-05-08T23:28:00.280966+0000","last_peered":"2024-05-08T23:28:00.280966+0000","last_clean":"2024-05-08T23:28:00.280966+0000","last_became_active":"2024-05-08T23:27:59.291501+0000","last_became_peered":"2024-05-08T23:27:59.291501+0000","last_unstale":"2024-05-08T23:28:00.280966+0000","last_undegraded":"2024-05-08T23:28:00.280966+0000","last_fullsized":"2024-05-08T23:28:00.280966+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:56.514212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971150+0000","last_change":"2024-05-08T23:27:58.831902+0000","last_active":"2024-05-08T23:28:01.971150+0000","last_peered":"2024-05-08T23:28:01.971150+0000","last_clean":"2024-05-08T23:28:01.971150+0000","last_became_active":"2024-05-08T23:27:58.831632+0000","last_became_peered":"2024-05-08T23:27:58.831632+0000","last_unstale":"2024-05-08T23:28:01.971150+0000","last_undegraded":"2024-05-08T23:28:01.971150+0000","last_fullsized":"2024-05-08T23:28:01.971150+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:35.903078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970397+0000","last_change":"2024-05-08T23:27:59.294808+0000","last_active":"2024-05-08T23:28:01.970397+0000","last_peered":"2024-05-08T23:28:01.970397+0000","last_clean":"2024-05-08T23:28:01.970397+0000","last_became_active":"2024-05-08T23:27:59.294156+0000","last_became_peered":"2024-05-08T23:27:59.294156+0000","last_unstale":"2024-05-08T23:28:01.970397+0000","last_undegraded":"2024-05-08T23:28:01.970397+0000","last_fullsized":"2024-05-08T23:28:01.970397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:12.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279874+0000","last_change":"2024-05-08T23:27:58.836811+0000","last_active":"2024-05-08T23:28:00.279874+0000","last_peered":"2024-05-08T23:28:00.279874+0000","last_clean":"2024-05-08T23:28:00.279874+0000","last_became_active":"2024-05-08T23:27:58.836478+0000","last_became_peered":"2024-05-08T23:27:58.836478+0000","last_unstale":"2024-05-08T23:28:00.279874+0000","last_undegraded":"2024-05-08T23:28:00.279874+0000","last_fullsized":"2024-05-08T23:28:00.279874+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:19.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914488+0000","last_change":"2024-05-08T23:27:59.290913+0000","last_active":"2024-05-08T23:28:01.914488+0000","last_peered":"2024-05-08T23:28:01.914488+0000","last_clean":"2024-05-08T23:28:01.914488+0000","last_became_active":"2024-05-08T23:27:59.290569+0000","last_became_peered":"2024-05-08T23:27:59.290569+0000","last_unstale":"2024-05-08T23:28:01.914488+0000","last_undegraded":"2024-05-08T23:28:01.914488+0000","last_fullsized":"2024-05-08T23:28:01.914488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:06.902107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970626+0000","last_change":"2024-05-08T23:27:58.836437+0000","last_active":"2024-05-08T23:28:01.970626+0000","last_peered":"2024-05-08T23:28:01.970626+0000","last_clean":"2024-05-08T23:28:01.970626+0000","last_became_active":"2024-05-08T23:27:58.836227+0000","last_became_peered":"2024-05-08T23:27:58.836227+0000","last_unstale":"2024-05-08T23:28:01.970626+0000","last_undegraded":"2024-05-08T23:28:01.970626+0000","last_fullsized":"2024-05-08T23:28:01.970626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:55.509178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280372+0000","last_change":"2024-05-08T23:27:59.292738+0000","last_active":"2024-05-08T23:28:00.280372+0000","last_peered":"2024-05-08T23:28:00.280372+0000","last_clean":"2024-05-08T23:28:00.280372+0000","last_became_active":"2024-05-08T23:27:59.292426+0000","last_became_peered":"2024-05-08T23:27:59.292426+0000","last_unstale":"2024-05-08T23:28:00.280372+0000","last_undegraded":"2024-05-08T23:28:00.280372+0000","last_fullsized":"2024-05-08T23:28:00.280372+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:12.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970866+0000","last_change":"2024-05-08T23:27:58.831459+0000","last_active":"2024-05-08T23:28:01.970866+0000","last_peered":"2024-05-08T23:28:01.970866+0000","last_clean":"2024-05-08T23:28:01.970866+0000","last_became_active":"2024-05-08T23:27:58.830916+0000","last_became_peered":"2024-05-08T23:27:58.830916+0000","last_unstale":"2024-05-08T23:28:01.970866+0000","last_undegraded":"2024-05-08T23:28:01.970866+0000","last_fullsized":"2024-05-08T23:28:01.970866+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:06:04.332019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280311+0000","last_change":"2024-05-08T23:27:59.301586+0000","last_active":"2024-05-08T23:28:00.280311+0000","last_peered":"2024-05-08T23:28:00.280311+0000","last_clean":"2024-05-08T23:28:00.280311+0000","last_became_active":"2024-05-08T23:27:59.297345+0000","last_became_peered":"2024-05-08T23:27:59.297345+0000","last_unstale":"2024-05-08T23:28:00.280311+0000","last_undegraded":"2024-05-08T23:28:00.280311+0000","last_fullsized":"2024-05-08T23:28:00.280311+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:06.902107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971258+0000","last_change":"2024-05-08T23:27:58.832163+0000","last_active":"2024-05-08T23:28:01.971258+0000","last_peered":"2024-05-08T23:28:01.971258+0000","last_clean":"2024-05-08T23:28:01.971258+0000","last_became_active":"2024-05-08T23:27:58.831828+0000","last_became_peered":"2024-05-08T23:27:58.831828+0000","last_unstale":"2024-05-08T23:28:01.971258+0000","last_undegraded":"2024-05-08T23:28:01.971258+0000","last_fullsized":"2024-05-08T23:28:01.971258+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:16.262919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281199+0000","last_change":"2024-05-08T23:27:59.290199+0000","last_active":"2024-05-08T23:28:00.281199+0000","last_peered":"2024-05-08T23:28:00.281199+0000","last_clean":"2024-05-08T23:28:00.281199+0000","last_became_active":"2024-05-08T23:27:59.289811+0000","last_became_peered":"2024-05-08T23:27:59.289811+0000","last_unstale":"2024-05-08T23:28:00.281199+0000","last_undegraded":"2024-05-08T23:28:00.281199+0000","last_fullsized":"2024-05-08T23:28:00.281199+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:40.658709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914771+0000","last_change":"2024-05-08T23:27:58.837681+0000","last_active":"2024-05-08T23:28:01.914771+0000","last_peered":"2024-05-08T23:28:01.914771+0000","last_clean":"2024-05-08T23:28:01.914771+0000","last_became_active":"2024-05-08T23:27:58.837379+0000","last_became_peered":"2024-05-08T23:27:58.837379+0000","last_unstale":"2024-05-08T23:28:01.914771+0000","last_undegraded":"2024-05-08T23:28:01.914771+0000","last_fullsized":"2024-05-08T23:28:01.914771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:03.184914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970927+0000","last_change":"2024-05-08T23:27:59.297921+0000","last_active":"2024-05-08T23:28:01.970927+0000","last_peered":"2024-05-08T23:28:01.970927+0000","last_clean":"2024-05-08T23:28:01.970927+0000","last_became_active":"2024-05-08T23:27:59.297584+0000","last_became_peered":"2024-05-08T23:27:59.297584+0000","last_unstale":"2024-05-08T23:28:01.970927+0000","last_undegraded":"2024-05-08T23:28:01.970927+0000","last_fullsized":"2024-05-08T23:28:01.970927+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:33.408974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"95'3","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971106+0000","last_change":"2024-05-08T23:27:58.829370+0000","last_active":"2024-05-08T23:28:01.971106+0000","last_peered":"2024-05-08T23:28:01.971106+0000","last_clean":"2024-05-08T23:28:01.971106+0000","last_became_active":"2024-05-08T23:27:58.829043+0000","last_became_peered":"2024-05-08T23:27:58.829043+0000","last_unstale":"2024-05-08T23:28:01.971106+0000","last_undegraded":"2024-05-08T23:28:01.971106+0000","last_fullsized":"2024-05-08T23:28:01.971106+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:20.060441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970884+0000","last_change":"2024-05-08T23:27:59.299100+0000","last_active":"2024-05-08T23:28:01.970884+0000","last_peered":"2024-05-08T23:28:01.970884+0000","last_clean":"2024-05-08T23:28:01.970884+0000","last_became_active":"2024-05-08T23:27:59.298732+0000","last_became_peered":"2024-05-08T23:27:59.298732+0000","last_unstale":"2024-05-08T23:28:01.970884+0000","last_undegraded":"2024-05-08T23:28:01.970884+0000","last_fullsized":"2024-05-08T23:28:01.970884+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:36.456980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"95'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280992+0000","last_change":"2024-05-08T23:27:58.828019+0000","last_active":"2024-05-08T23:28:00.280992+0000","last_peered":"2024-05-08T23:28:00.280992+0000","last_clean":"2024-05-08T23:28:00.280992+0000","last_became_active":"2024-05-08T23:27:58.827686+0000","last_became_peered":"2024-05-08T23:27:58.827686+0000","last_unstale":"2024-05-08T23:28:00.280992+0000","last_undegraded":"2024-05-08T23:28:00.280992+0000","last_fullsized":"2024-05-08T23:28:00.280992+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:57.801469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914526+0000","last_change":"2024-05-08T23:27:59.293374+0000","last_active":"2024-05-08T23:28:01.914526+0000","last_peered":"2024-05-08T23:28:01.914526+0000","last_clean":"2024-05-08T23:28:01.914526+0000","last_became_active":"2024-05-08T23:27:59.292953+0000","last_became_peered":"2024-05-08T23:27:59.292953+0000","last_unstale":"2024-05-08T23:28:01.914526+0000","last_undegraded":"2024-05-08T23:28:01.914526+0000","last_fullsized":"2024-05-08T23:28:01.914526+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:24.207488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685724+0000","last_change":"2024-05-08T23:27:58.828114+0000","last_active":"2024-05-08T23:28:01.685724+0000","last_peered":"2024-05-08T23:28:01.685724+0000","last_clean":"2024-05-08T23:28:01.685724+0000","last_became_active":"2024-05-08T23:27:58.827758+0000","last_became_peered":"2024-05-08T23:27:58.827758+0000","last_unstale":"2024-05-08T23:28:01.685724+0000","last_undegraded":"2024-05-08T23:28:01.685724+0000","last_fullsized":"2024-05-08T23:28:01.685724+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:33.675717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970950+0000","last_change":"2024-05-08T23:27:59.306251+0000","last_active":"2024-05-08T23:28:01.970950+0000","last_peered":"2024-05-08T23:28:01.970950+0000","last_clean":"2024-05-08T23:28:01.970950+0000","last_became_active":"2024-05-08T23:27:59.305973+0000","last_became_peered":"2024-05-08T23:27:59.305973+0000","last_unstale":"2024-05-08T23:28:01.970950+0000","last_undegraded":"2024-05-08T23:28:01.970950+0000","last_fullsized":"2024-05-08T23:28:01.970950+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:13.425324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685647+0000","last_change":"2024-05-08T23:27:58.831736+0000","last_active":"2024-05-08T23:28:01.685647+0000","last_peered":"2024-05-08T23:28:01.685647+0000","last_clean":"2024-05-08T23:28:01.685647+0000","last_became_active":"2024-05-08T23:27:58.831437+0000","last_became_peered":"2024-05-08T23:27:58.831437+0000","last_unstale":"2024-05-08T23:28:01.685647+0000","last_undegraded":"2024-05-08T23:28:01.685647+0000","last_fullsized":"2024-05-08T23:28:01.685647+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:00.717795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280372+0000","last_change":"2024-05-08T23:27:59.287910+0000","last_active":"2024-05-08T23:28:00.280372+0000","last_peered":"2024-05-08T23:28:00.280372+0000","last_clean":"2024-05-08T23:28:00.280372+0000","last_became_active":"2024-05-08T23:27:59.287565+0000","last_became_peered":"2024-05-08T23:27:59.287565+0000","last_unstale":"2024-05-08T23:28:00.280372+0000","last_undegraded":"2024-05-08T23:28:00.280372+0000","last_fullsized":"2024-05-08T23:28:00.280372+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.892981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280478+0000","last_change":"2024-05-08T23:27:58.827223+0000","last_active":"2024-05-08T23:28:00.280478+0000","last_peered":"2024-05-08T23:28:00.280478+0000","last_clean":"2024-05-08T23:28:00.280478+0000","last_became_active":"2024-05-08T23:27:58.826938+0000","last_became_peered":"2024-05-08T23:27:58.826938+0000","last_unstale":"2024-05-08T23:28:00.280478+0000","last_undegraded":"2024-05-08T23:28:00.280478+0000","last_fullsized":"2024-05-08T23:28:00.280478+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:45.066176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685472+0000","last_change":"2024-05-08T23:27:59.296033+0000","last_active":"2024-05-08T23:28:01.685472+0000","last_peered":"2024-05-08T23:28:01.685472+0000","last_clean":"2024-05-08T23:28:01.685472+0000","last_became_active":"2024-05-08T23:27:59.295720+0000","last_became_peered":"2024-05-08T23:27:59.295720+0000","last_unstale":"2024-05-08T23:28:01.685472+0000","last_undegraded":"2024-05-08T23:28:01.685472+0000","last_fullsized":"2024-05-08T23:28:01.685472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:49:01.343367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"95'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971315+0000","last_change":"2024-05-08T23:27:58.831236+0000","last_active":"2024-05-08T23:28:01.971315+0000","last_peered":"2024-05-08T23:28:01.971315+0000","last_clean":"2024-05-08T23:28:01.971315+0000","last_became_active":"2024-05-08T23:27:58.830601+0000","last_became_peered":"2024-05-08T23:27:58.830601+0000","last_unstale":"2024-05-08T23:28:01.971315+0000","last_undegraded":"2024-05-08T23:28:01.971315+0000","last_fullsized":"2024-05-08T23:28:01.971315+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:54.719865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971183+0000","last_change":"2024-05-08T23:27:59.293621+0000","last_active":"2024-05-08T23:28:01.971183+0000","last_peered":"2024-05-08T23:28:01.971183+0000","last_clean":"2024-05-08T23:28:01.971183+0000","last_became_active":"2024-05-08T23:27:59.293356+0000","last_became_peered":"2024-05-08T23:27:59.293356+0000","last_unstale":"2024-05-08T23:28:01.971183+0000","last_undegraded":"2024-05-08T23:28:01.971183+0000","last_fullsized":"2024-05-08T23:28:01.971183+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:40.658709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281217+0000","last_change":"2024-05-08T23:27:58.829032+0000","last_active":"2024-05-08T23:28:00.281217+0000","last_peered":"2024-05-08T23:28:00.281217+0000","last_clean":"2024-05-08T23:28:00.281217+0000","last_became_active":"2024-05-08T23:27:58.828705+0000","last_became_peered":"2024-05-08T23:27:58.828705+0000","last_unstale":"2024-05-08T23:28:00.281217+0000","last_undegraded":"2024-05-08T23:28:00.281217+0000","last_fullsized":"2024-05-08T23:28:00.281217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:58.162125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280875+0000","last_change":"2024-05-08T23:27:59.286820+0000","last_active":"2024-05-08T23:28:00.280875+0000","last_peered":"2024-05-08T23:28:00.280875+0000","last_clean":"2024-05-08T23:28:00.280875+0000","last_became_active":"2024-05-08T23:27:59.286534+0000","last_became_peered":"2024-05-08T23:27:59.286534+0000","last_unstale":"2024-05-08T23:28:00.280875+0000","last_undegraded":"2024-05-08T23:28:00.280875+0000","last_fullsized":"2024-05-08T23:28:00.280875+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:56.836101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970982+0000","last_change":"2024-05-08T23:27:58.823276+0000","last_active":"2024-05-08T23:28:01.970982+0000","last_peered":"2024-05-08T23:28:01.970982+0000","last_clean":"2024-05-08T23:28:01.970982+0000","last_became_active":"2024-05-08T23:27:58.822987+0000","last_became_peered":"2024-05-08T23:27:58.822987+0000","last_unstale":"2024-05-08T23:28:01.970982+0000","last_undegraded":"2024-05-08T23:28:01.970982+0000","last_fullsized":"2024-05-08T23:28:01.970982+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:00.882978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971051+0000","last_change":"2024-05-08T23:27:59.295770+0000","last_active":"2024-05-08T23:28:01.971051+0000","last_peered":"2024-05-08T23:28:01.971051+0000","last_clean":"2024-05-08T23:28:01.971051+0000","last_became_active":"2024-05-08T23:27:59.295429+0000","last_became_peered":"2024-05-08T23:27:59.295429+0000","last_unstale":"2024-05-08T23:28:01.971051+0000","last_undegraded":"2024-05-08T23:28:01.971051+0000","last_fullsized":"2024-05-08T23:28:01.971051+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.892981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280244+0000","last_change":"2024-05-08T23:27:58.829778+0000","last_active":"2024-05-08T23:28:00.280244+0000","last_peered":"2024-05-08T23:28:00.280244+0000","last_clean":"2024-05-08T23:28:00.280244+0000","last_became_active":"2024-05-08T23:27:58.829181+0000","last_became_peered":"2024-05-08T23:27:58.829181+0000","last_unstale":"2024-05-08T23:28:00.280244+0000","last_undegraded":"2024-05-08T23:28:00.280244+0000","last_fullsized":"2024-05-08T23:28:00.280244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:58.839790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685515+0000","last_change":"2024-05-08T23:27:59.292518+0000","last_active":"2024-05-08T23:28:01.685515+0000","last_peered":"2024-05-08T23:28:01.685515+0000","last_clean":"2024-05-08T23:28:01.685515+0000","last_became_active":"2024-05-08T23:27:59.292224+0000","last_became_peered":"2024-05-08T23:27:59.292224+0000","last_unstale":"2024-05-08T23:28:01.685515+0000","last_undegraded":"2024-05-08T23:28:01.685515+0000","last_fullsized":"2024-05-08T23:28:01.685515+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:20.759506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279908+0000","last_change":"2024-05-08T23:27:58.827556+0000","last_active":"2024-05-08T23:28:00.279908+0000","last_peered":"2024-05-08T23:28:00.279908+0000","last_clean":"2024-05-08T23:28:00.279908+0000","last_became_active":"2024-05-08T23:27:58.825365+0000","last_became_peered":"2024-05-08T23:27:58.825365+0000","last_unstale":"2024-05-08T23:28:00.279908+0000","last_undegraded":"2024-05-08T23:28:00.279908+0000","last_fullsized":"2024-05-08T23:28:00.279908+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:20.792869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685616+0000","last_change":"2024-05-08T23:27:59.304356+0000","last_active":"2024-05-08T23:28:01.685616+0000","last_peered":"2024-05-08T23:28:01.685616+0000","last_clean":"2024-05-08T23:28:01.685616+0000","last_became_active":"2024-05-08T23:27:59.304085+0000","last_became_peered":"2024-05-08T23:27:59.304085+0000","last_unstale":"2024-05-08T23:28:01.685616+0000","last_undegraded":"2024-05-08T23:28:01.685616+0000","last_fullsized":"2024-05-08T23:28:01.685616+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:15.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281105+0000","last_change":"2024-05-08T23:27:58.832192+0000","last_active":"2024-05-08T23:28:00.281105+0000","last_peered":"2024-05-08T23:28:00.281105+0000","last_clean":"2024-05-08T23:28:00.281105+0000","last_became_active":"2024-05-08T23:27:58.831880+0000","last_became_peered":"2024-05-08T23:27:58.831880+0000","last_unstale":"2024-05-08T23:28:00.281105+0000","last_undegraded":"2024-05-08T23:28:00.281105+0000","last_fullsized":"2024-05-08T23:28:00.281105+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:12.627757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685436+0000","last_change":"2024-05-08T23:27:59.291895+0000","last_active":"2024-05-08T23:28:01.685436+0000","last_peered":"2024-05-08T23:28:01.685436+0000","last_clean":"2024-05-08T23:28:01.685436+0000","last_became_active":"2024-05-08T23:27:59.291597+0000","last_became_peered":"2024-05-08T23:27:59.291597+0000","last_unstale":"2024-05-08T23:28:01.685436+0000","last_undegraded":"2024-05-08T23:28:01.685436+0000","last_fullsized":"2024-05-08T23:28:01.685436+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:08.063795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281193+0000","last_change":"2024-05-08T23:27:58.825191+0000","last_active":"2024-05-08T23:28:00.281193+0000","last_peered":"2024-05-08T23:28:00.281193+0000","last_clean":"2024-05-08T23:28:00.281193+0000","last_became_active":"2024-05-08T23:27:58.824897+0000","last_became_peered":"2024-05-08T23:27:58.824897+0000","last_unstale":"2024-05-08T23:28:00.281193+0000","last_undegraded":"2024-05-08T23:28:00.281193+0000","last_fullsized":"2024-05-08T23:28:00.281193+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:27.916838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970514+0000","last_change":"2024-05-08T23:27:59.301272+0000","last_active":"2024-05-08T23:28:01.970514+0000","last_peered":"2024-05-08T23:28:01.970514+0000","last_clean":"2024-05-08T23:28:01.970514+0000","last_became_active":"2024-05-08T23:27:59.300973+0000","last_became_peered":"2024-05-08T23:27:59.300973+0000","last_unstale":"2024-05-08T23:28:01.970514+0000","last_undegraded":"2024-05-08T23:28:01.970514+0000","last_fullsized":"2024-05-08T23:28:01.970514+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:57.504940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279771+0000","last_change":"2024-05-08T23:27:58.835661+0000","last_active":"2024-05-08T23:28:00.279771+0000","last_peered":"2024-05-08T23:28:00.279771+0000","last_clean":"2024-05-08T23:28:00.279771+0000","last_became_active":"2024-05-08T23:27:58.835336+0000","last_became_peered":"2024-05-08T23:27:58.835336+0000","last_unstale":"2024-05-08T23:28:00.279771+0000","last_undegraded":"2024-05-08T23:28:00.279771+0000","last_fullsized":"2024-05-08T23:28:00.279771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:53.927547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280374+0000","last_change":"2024-05-08T23:27:59.302039+0000","last_active":"2024-05-08T23:28:00.280374+0000","last_peered":"2024-05-08T23:28:00.280374+0000","last_clean":"2024-05-08T23:28:00.280374+0000","last_became_active":"2024-05-08T23:27:59.300994+0000","last_became_peered":"2024-05-08T23:27:59.300994+0000","last_unstale":"2024-05-08T23:28:00.280374+0000","last_undegraded":"2024-05-08T23:28:00.280374+0000","last_fullsized":"2024-05-08T23:28:00.280374+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:10.376057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.281167+0000","last_change":"2024-05-08T23:27:58.827693+0000","last_active":"2024-05-08T23:28:00.281167+0000","last_peered":"2024-05-08T23:28:00.281167+0000","last_clean":"2024-05-08T23:28:00.281167+0000","last_became_active":"2024-05-08T23:27:58.827394+0000","last_became_peered":"2024-05-08T23:27:58.827394+0000","last_unstale":"2024-05-08T23:28:00.281167+0000","last_undegraded":"2024-05-08T23:28:00.281167+0000","last_fullsized":"2024-05-08T23:28:00.281167+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:15.208804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971204+0000","last_change":"2024-05-08T23:27:59.294527+0000","last_active":"2024-05-08T23:28:01.971204+0000","last_peered":"2024-05-08T23:28:01.971204+0000","last_clean":"2024-05-08T23:28:01.971204+0000","last_became_active":"2024-05-08T23:27:59.294255+0000","last_became_peered":"2024-05-08T23:27:59.294255+0000","last_unstale":"2024-05-08T23:28:01.971204+0000","last_undegraded":"2024-05-08T23:28:01.971204+0000","last_fullsized":"2024-05-08T23:28:01.971204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:15.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280417+0000","last_change":"2024-05-08T23:27:58.825434+0000","last_active":"2024-05-08T23:28:00.280417+0000","last_peered":"2024-05-08T23:28:00.280417+0000","last_clean":"2024-05-08T23:28:00.280417+0000","last_became_active":"2024-05-08T23:27:58.825108+0000","last_became_peered":"2024-05-08T23:27:58.825108+0000","last_unstale":"2024-05-08T23:28:00.280417+0000","last_undegraded":"2024-05-08T23:28:00.280417+0000","last_fullsized":"2024-05-08T23:28:00.280417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:56.499907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971233+0000","last_change":"2024-05-08T23:27:59.299313+0000","last_active":"2024-05-08T23:28:01.971233+0000","last_peered":"2024-05-08T23:28:01.971233+0000","last_clean":"2024-05-08T23:28:01.971233+0000","last_became_active":"2024-05-08T23:27:59.298879+0000","last_became_peered":"2024-05-08T23:27:59.298879+0000","last_unstale":"2024-05-08T23:28:01.971233+0000","last_undegraded":"2024-05-08T23:28:01.971233+0000","last_fullsized":"2024-05-08T23:28:01.971233+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:37.758468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280143+0000","last_change":"2024-05-08T23:27:58.829411+0000","last_active":"2024-05-08T23:28:00.280143+0000","last_peered":"2024-05-08T23:28:00.280143+0000","last_clean":"2024-05-08T23:28:00.280143+0000","last_became_active":"2024-05-08T23:27:58.829095+0000","last_became_peered":"2024-05-08T23:27:58.829095+0000","last_unstale":"2024-05-08T23:28:00.280143+0000","last_undegraded":"2024-05-08T23:28:00.280143+0000","last_fullsized":"2024-05-08T23:28:00.280143+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:31.394993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970893+0000","last_change":"2024-05-08T23:27:59.298501+0000","last_active":"2024-05-08T23:28:01.970893+0000","last_peered":"2024-05-08T23:28:01.970893+0000","last_clean":"2024-05-08T23:28:01.970893+0000","last_became_active":"2024-05-08T23:27:59.298229+0000","last_became_peered":"2024-05-08T23:27:59.298229+0000","last_unstale":"2024-05-08T23:28:01.970893+0000","last_undegraded":"2024-05-08T23:28:01.970893+0000","last_fullsized":"2024-05-08T23:28:01.970893+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:21.797902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971027+0000","last_change":"2024-05-08T23:27:58.820714+0000","last_active":"2024-05-08T23:28:01.971027+0000","last_peered":"2024-05-08T23:28:01.971027+0000","last_clean":"2024-05-08T23:28:01.971027+0000","last_became_active":"2024-05-08T23:27:58.820379+0000","last_became_peered":"2024-05-08T23:27:58.820379+0000","last_unstale":"2024-05-08T23:28:01.971027+0000","last_undegraded":"2024-05-08T23:28:01.971027+0000","last_fullsized":"2024-05-08T23:28:01.971027+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:29.171114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685718+0000","last_change":"2024-05-08T23:27:59.294148+0000","last_active":"2024-05-08T23:28:01.685718+0000","last_peered":"2024-05-08T23:28:01.685718+0000","last_clean":"2024-05-08T23:28:01.685718+0000","last_became_active":"2024-05-08T23:27:59.293847+0000","last_became_peered":"2024-05-08T23:27:59.293847+0000","last_unstale":"2024-05-08T23:28:01.685718+0000","last_undegraded":"2024-05-08T23:28:01.685718+0000","last_fullsized":"2024-05-08T23:28:01.685718+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:19.303740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.279975+0000","last_change":"2024-05-08T23:27:58.828891+0000","last_active":"2024-05-08T23:28:00.279975+0000","last_peered":"2024-05-08T23:28:00.279975+0000","last_clean":"2024-05-08T23:28:00.279975+0000","last_became_active":"2024-05-08T23:27:58.828141+0000","last_became_peered":"2024-05-08T23:27:58.828141+0000","last_unstale":"2024-05-08T23:28:00.279975+0000","last_undegraded":"2024-05-08T23:28:00.279975+0000","last_fullsized":"2024-05-08T23:28:00.279975+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:25.887947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:00.280608+0000","last_change":"2024-05-08T23:27:59.287397+0000","last_active":"2024-05-08T23:28:00.280608+0000","last_peered":"2024-05-08T23:28:00.280608+0000","last_clean":"2024-05-08T23:28:00.280608+0000","last_became_active":"2024-05-08T23:27:59.287104+0000","last_became_peered":"2024-05-08T23:27:59.287104+0000","last_unstale":"2024-05-08T23:28:00.280608+0000","last_undegraded":"2024-05-08T23:28:00.280608+0000","last_fullsized":"2024-05-08T23:28:00.280608+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:59.844824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.971576+0000","last_change":"2024-05-08T23:27:58.832142+0000","last_active":"2024-05-08T23:28:01.971576+0000","last_peered":"2024-05-08T23:28:01.971576+0000","last_clean":"2024-05-08T23:28:01.971576+0000","last_became_active":"2024-05-08T23:27:58.831837+0000","last_became_peered":"2024-05-08T23:27:58.831837+0000","last_unstale":"2024-05-08T23:28:01.971576+0000","last_undegraded":"2024-05-08T23:28:01.971576+0000","last_fullsized":"2024-05-08T23:28:01.971576+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:53.692460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685400+0000","last_change":"2024-05-08T23:27:59.295239+0000","last_active":"2024-05-08T23:28:01.685400+0000","last_peered":"2024-05-08T23:28:01.685400+0000","last_clean":"2024-05-08T23:28:01.685400+0000","last_became_active":"2024-05-08T23:27:59.294968+0000","last_became_peered":"2024-05-08T23:27:59.294968+0000","last_unstale":"2024-05-08T23:28:01.685400+0000","last_undegraded":"2024-05-08T23:28:01.685400+0000","last_fullsized":"2024-05-08T23:28:01.685400+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:25.869287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.914815+0000","last_change":"2024-05-08T23:27:58.833182+0000","last_active":"2024-05-08T23:28:01.914815+0000","last_peered":"2024-05-08T23:28:01.914815+0000","last_clean":"2024-05-08T23:28:01.914815+0000","last_became_active":"2024-05-08T23:27:58.832862+0000","last_became_peered":"2024-05-08T23:27:58.832862+0000","last_unstale":"2024-05-08T23:28:01.914815+0000","last_undegraded":"2024-05-08T23:28:01.914815+0000","last_fullsized":"2024-05-08T23:28:01.914815+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:19.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970658+0000","last_change":"2024-05-08T23:27:59.298861+0000","last_active":"2024-05-08T23:28:01.970658+0000","last_peered":"2024-05-08T23:28:01.970658+0000","last_clean":"2024-05-08T23:28:01.970658+0000","last_became_active":"2024-05-08T23:27:59.298539+0000","last_became_peered":"2024-05-08T23:27:59.298539+0000","last_unstale":"2024-05-08T23:28:01.970658+0000","last_undegraded":"2024-05-08T23:28:01.970658+0000","last_fullsized":"2024-05-08T23:28:01.970658+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:12.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685527+0000","last_change":"2024-05-08T23:27:58.819783+0000","last_active":"2024-05-08T23:28:01.685527+0000","last_peered":"2024-05-08T23:28:01.685527+0000","last_clean":"2024-05-08T23:28:01.685527+0000","last_became_active":"2024-05-08T23:27:58.819459+0000","last_became_peered":"2024-05-08T23:27:58.819459+0000","last_unstale":"2024-05-08T23:28:01.685527+0000","last_undegraded":"2024-05-08T23:28:01.685527+0000","last_fullsized":"2024-05-08T23:28:01.685527+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:30.876071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.685861+0000","last_change":"2024-05-08T23:27:59.291886+0000","last_active":"2024-05-08T23:28:01.685861+0000","last_peered":"2024-05-08T23:28:01.685861+0000","last_clean":"2024-05-08T23:28:01.685861+0000","last_became_active":"2024-05-08T23:27:59.291619+0000","last_became_peered":"2024-05-08T23:27:59.291619+0000","last_unstale":"2024-05-08T23:28:01.685861+0000","last_undegraded":"2024-05-08T23:28:01.685861+0000","last_fullsized":"2024-05-08T23:28:01.685861+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:58.806407+0000","last_clean_scrub_stamp":"2024-05-08T23:27:58.806407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:26.251204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970584+0000","last_change":"2024-05-08T23:27:58.827644+0000","last_active":"2024-05-08T23:28:01.970584+0000","last_peered":"2024-05-08T23:28:01.970584+0000","last_clean":"2024-05-08T23:28:01.970584+0000","last_became_active":"2024-05-08T23:27:58.827360+0000","last_became_peered":"2024-05-08T23:27:58.827360+0000","last_unstale":"2024-05-08T23:28:01.970584+0000","last_undegraded":"2024-05-08T23:28:01.970584+0000","last_fullsized":"2024-05-08T23:28:01.970584+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:27:57.801374+0000","last_clean_scrub_stamp":"2024-05-08T23:27:57.801374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:45.066176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-08T23:27:59.285266+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:27:59.285266+0000","last_peered":"2024-05-08T23:27:59.285266+0000","last_clean":"2024-05-08T23:27:59.285266+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:27:59.285266+0000","last_undegraded":"2024-05-08T23:27:59.285266+0000","last_fullsized":"2024-05-08T23:27:59.285266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1777664,"data_stored":1098542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.53500000000000003},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80300000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.59899999999999998,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76600000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":1.038,"15min":1.038},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.55700000000000005},"min":{"1min":0.42899999999999999,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.83699999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67600000000000005}]},{"osd":2,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.54800000000000004},"min":{"1min":0.46100000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.625,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78400000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60699999999999998}]},{"osd":3,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.53800000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77800000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63700000000000001,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.70599999999999996,"5min":0.78100000000000003,"15min":0.85499999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77000000000000002,"5min":0.87,"15min":0.87},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75800000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.372,"15min":0.34100000000000003},"max":{"1min":0.78500000000000003,"5min":0.81399999999999995,"15min":0.93500000000000005},"last":0.626}]},{"osd":6,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84599999999999997,"5min":0.86199999999999999,"15min":0.89700000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41499999999999998,"5min":0.33600000000000002,"15min":0.314},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1196032,"data_stored":515014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57099999999999995,"15min":0.53500000000000003},"min":{"1min":0.44,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66400000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58899999999999997,"15min":0.55500000000000005},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.753,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73199999999999998}]},{"osd":1,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.88200000000000001,"15min":0.94999999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.81599999999999995,"15min":0.86899999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72999999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69099999999999995,"5min":0.85699999999999998,"15min":1.115},"last":0.66100000000000003}]},{"osd":3,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.46000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78000000000000003,"5min":0.878,"15min":1.0229999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.98899999999999999,"15min":1.0009999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52100000000000002,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.746,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53800000000000003,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.56100000000000005},"min":{"1min":0.34399999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":1.052},"last":0.499},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.60199999999999998},"min":{"1min":0.38100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75900000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73499999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.372,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.67500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":508062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.91000000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.42199999999999999},"min":{"1min":0.28199999999999997,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.55200000000000005,"5min":0.63500000000000001,"15min":0.93100000000000005},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39700000000000002,"15min":0.41199999999999998},"min":{"1min":0.25700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.88400000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42299999999999999,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50600000000000001,"5min":0.61399999999999999,"15min":0.96999999999999997},"last":0.32100000000000001}]},{"osd":3,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.49399999999999999},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.61799999999999999,"5min":0.65800000000000003,"15min":1.01},"last":0.436},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.47499999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59599999999999997,"5min":0.68200000000000005,"15min":1.028},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.39200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.873,"5min":1.004,"15min":1.1479999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66000000000000003,"15min":0.67700000000000005},"min":{"1min":0.51600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96399999999999997,"5min":1.0169999999999999,"15min":1.1579999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.627,"15min":0.64100000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.89000000000000001,"15min":1.0489999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64300000000000002,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.115},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64900000000000002,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.2050000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":1.105},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed May 8 23:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.69399999999999995},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84999999999999998,"5min":0.89200000000000002,"15min":1.1359999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.47499999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.0609999999999999},"last":0.78500000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574936,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1785856,"data_stored":1105312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.69299999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.53900000000000003,"5min":0.94099999999999995,"15min":7.7610000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40999999999999998,"15min":0.68600000000000005},"min":{"1min":0.249,"5min":0.249,"15min":0.24399999999999999},"max":{"1min":0.52100000000000002,"5min":0.65800000000000003,"15min":7.9889999999999999},"last":0.33900000000000002}]},{"osd":2,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41699999999999998,"15min":0.745},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.627,"5min":0.88500000000000001,"15min":7.7869999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.71899999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.627,"5min":0.71399999999999997,"15min":8.0519999999999996},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.32200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.50900000000000001,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.48499999999999999,"15min":0.76500000000000001},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59699999999999998,"5min":1.9770000000000001,"15min":7.6440000000000001},"last":0.437}]},{"osd":4,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.90500000000000003},"min":{"1min":0.41899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":7.9139999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61899999999999999,"15min":0.879},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96199999999999997,"5min":2.069,"15min":7.1879999999999997},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64600000000000002,"15min":0.88600000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83499999999999996,"5min":2.0350000000000001,"15min":7.1120000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66500000000000004,"15min":0.97699999999999998},"min":{"1min":0.503,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":7.6970000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.94399999999999995},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89200000000000002,"5min":1.1759999999999999,"15min":7.734},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.92400000000000004},"min":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.88,"15min":8.0220000000000002},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.93600000000000005},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.872,"5min":1.5369999999999999,"15min":7.9539999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.89500000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91300000000000003,"5min":3.1139999999999999,"15min":7.1600000000000001},"last":0.62}]}]},{"osd":2,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1191936,"data_stored":514396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.56399999999999995,"5min":0.621,"15min":0.621},"last":0.442},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57599999999999996,"5min":0.65800000000000003,"15min":0.68400000000000005},"last":0.373}]},{"osd":1,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45500000000000002,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.61799999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.35399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42299999999999999}]},{"osd":3,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55900000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.495,"15min":0.46600000000000003},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60499999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.40500000000000003}]},{"osd":4,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.434,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.77200000000000002,"5min":1.024,"15min":1.024},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85499999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.46300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79500000000000004,"5min":0.97899999999999998,"15min":1.0549999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65300000000000002,"15min":0.67400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85499999999999998,"5min":0.89200000000000002,"15min":1.159},"last":0.499},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.50700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53500000000000003}]},{"osd":7,"last update":"Wed May 8 23:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.67200000000000004},"min":{"1min":0.52100000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89700000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91900000000000004,"5min":0.997,"15min":1.194},"last":0.55400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1191936,"data_stored":509648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.41299999999999998},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":1.46},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":1.2749999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.28699999999999998,"15min":0.27700000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45000000000000001,"15min":0.46800000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.5649999999999999},"last":0.442}]},{"osd":2,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45200000000000001,"15min":0.80900000000000005},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":11.964},"last":0.63},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63300000000000001,"5min":0.73899999999999999,"15min":1.593},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63,"15min":0.65900000000000003},"min":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.71199999999999997},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.214,"5min":1.214,"15min":2.4590000000000001},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64600000000000002,"15min":0.66500000000000004},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":1.536},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65800000000000003,"15min":0.68200000000000005},"min":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.3100000000000001},"last":0.89700000000000002}]},{"osd":6,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64700000000000002,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.5029999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Wed May 8 23:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67300000000000004,"15min":0.69899999999999995},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68600000000000005,"15min":0.76300000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":2.5230000000000001},"last":0.91000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574937,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1208320,"data_stored":516050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54900000000000004,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":2.0209999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55700000000000005,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":4.0960000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.57799999999999996,"15min":0.66400000000000003},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":2.145},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.67600000000000005},"min":{"1min":0.40500000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.85299999999999998,"5min":0.88500000000000001,"15min":2.2530000000000001},"last":0.621}]},{"osd":2,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57599999999999996,"15min":0.67600000000000005},"min":{"1min":0.42199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.3769999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.77000000000000002},"min":{"1min":0.44400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":4.0389999999999997},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.0800000000000001},"last":0.439},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":1.079},"min":{"1min":0.47099999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":12.127000000000001},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.5,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79700000000000004,"5min":0.80200000000000005,"15min":0.877},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.61499999999999999},"min":{"1min":0.378,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.93700000000000006},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.624,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed May 8 23:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57599999999999996,"15min":0.624},"min":{"1min":0.34799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60799999999999998,"15min":0.68899999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.877,"5min":0.877,"15min":1.24},"last":0.68300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1191936,"data_stored":514432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.69999999999999996},"min":{"1min":0.42399999999999999,"5min":0.32700000000000001,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.96099999999999997,"15min":4.6710000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59199999999999997,"15min":0.86599999999999999},"min":{"1min":0.39300000000000002,"5min":0.35199999999999998,"15min":0.30499999999999999},"max":{"1min":0.77200000000000002,"5min":0.93400000000000005,"15min":8.798},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57599999999999996,"15min":0.75600000000000001},"min":{"1min":0.41999999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":4.71},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60999999999999999,"15min":0.93600000000000005},"min":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0189999999999999,"15min":8.6880000000000006},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.746},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":1.002,"15min":4.8940000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.77700000000000002},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.443},"max":{"1min":0.81100000000000005,"5min":1.0349999999999999,"15min":4.8559999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.90200000000000002},"min":{"1min":0.5,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79200000000000004,"5min":0.91800000000000004,"15min":8.8770000000000007},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.97699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":1.214,"15min":10.427},"last":0.626}]},{"osd":4,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.496,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68100000000000005,"5min":0.874,"15min":0.874},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.56899999999999995},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.72399999999999998,"5min":0.86799999999999999,"15min":0.95899999999999996},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.57499999999999996},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88400000000000001,"5min":1.141,"15min":1.141},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48599999999999999}]},{"osd":7,"last update":"Wed May 8 23:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34799999999999998},"max":{"1min":0.82999999999999996,"5min":1.01,"15min":1.01},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":1.135,"15min":1.135},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:28:04.024 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:28:04.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T23:28:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.206+0000 7fd1014ae700 1 Processor -- start 2024-05-08T23:28:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.206+0000 7fd1014ae700 1 -- start start 2024-05-08T23:28:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.206+0000 7fd1014ae700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0fc161c00 0x7fd0fc168820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.206+0000 7fd1014ae700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc0b49b0 con 0x7fd0fc15d700 2024-05-08T23:28:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.206+0000 7fd1014ae700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc0b1150 con 0x7fd0fc15dae0 2024-05-08T23:28:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.206+0000 7fd1014ae700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc0b3da0 con 0x7fd0fc161c00 2024-05-08T23:28:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0fb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0fc161c00 0x7fd0fc168820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0faffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0fc15d700 0x7fd0fc0b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34468/0 (socket says 172.21.15.5:34468) 2024-05-08T23:28:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0faffd700 1 -- 172.21.15.5:0/2725905957 learned_addr learned my addr 172.21.15.5:0/2725905957 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0fb7fe700 1 --2- 172.21.15.5:0/2725905957 >> v2:172.21.15.5:3300/0 conn(0x7fd0fc161c00 0x7fd0fc168820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59018/0 (socket says 172.21.15.5:59018) 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0fb7fe700 1 -- 172.21.15.5:0/2725905957 >> v1:172.21.15.49:6789/0 conn(0x7fd0fc15dae0 legacy=0x7fd0fc1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0fb7fe700 1 -- 172.21.15.5:0/2725905957 >> v1:172.21.15.5:6789/0 conn(0x7fd0fc15d700 legacy=0x7fd0fc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0f9ffb700 1 -- 172.21.15.5:0/2725905957 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389443118 0 0) 0x7fd0fc0b49b0 con 0x7fd0fc15d700 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0fb7fe700 1 -- 172.21.15.5:0/2725905957 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc0b4080 con 0x7fd0fc161c00 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0fb7fe700 1 --2- 172.21.15.5:0/2725905957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc161c00 0x7fd0fc168820 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009d90 tx=0x7fd0f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e91092badca77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0f9ffb700 1 -- 172.21.15.5:0/2725905957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400e070 con 0x7fd0fc161c00 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0f9ffb700 1 -- 172.21.15.5:0/2725905957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f400bb90 con 0x7fd0fc161c00 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.207+0000 7fd0f9ffb700 1 -- 172.21.15.5:0/2725905957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4003a20 con 0x7fd0fc161c00 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.208+0000 7fd1014ae700 1 -- 172.21.15.5:0/2725905957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc161c00 msgr2=0x7fd0fc168820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.208+0000 7fd1014ae700 1 --2- 172.21.15.5:0/2725905957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc161c00 0x7fd0fc168820 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009d90 tx=0x7fd0f400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.208+0000 7fd1014ae700 1 -- 172.21.15.5:0/2725905957 shutdown_connections 2024-05-08T23:28:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.208+0000 7fd1014ae700 1 --2- 172.21.15.5:0/2725905957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc161c00 0x7fd0fc168820 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.208+0000 7fd1014ae700 1 -- 172.21.15.5:0/2725905957 >> 172.21.15.5:0/2725905957 conn(0x7fd0fc08afe0 msgr2=0x7fd0fc0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.208+0000 7fd1014ae700 1 -- 172.21.15.5:0/2725905957 shutdown_connections 2024-05-08T23:28:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.208+0000 7fd1014ae700 1 -- 172.21.15.5:0/2725905957 wait complete. 2024-05-08T23:28:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.209+0000 7fd1014ae700 1 Processor -- start 2024-05-08T23:28:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd1014ae700 1 -- start start 2024-05-08T23:28:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd1014ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0fc15d700 0x7fd0fc1f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd1014ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc15dae0 0x7fd0fc15e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd1014ae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0fc161c00 0x7fd0fc06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd1014ae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc1f6760 con 0x7fd0fc15dae0 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd1014ae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc15e280 con 0x7fd0fc15d700 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd1014ae700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc1f37f0 con 0x7fd0fc161c00 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd0fa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc15dae0 0x7fd0fc15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd0fa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc15dae0 0x7fd0fc15e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59032/0 (socket says 172.21.15.5:59032) 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.210+0000 7fd0fa7fc700 1 -- 172.21.15.5:0/2864465903 learned_addr learned my addr 172.21.15.5:0/2864465903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0faffd700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0fc15d700 0x7fd0fc1f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0fb7fe700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0fc161c00 0x7fd0fc06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0fa7fc700 1 -- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0fc161c00 msgr2=0x7fd0fc06ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0fa7fc700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0fc161c00 0x7fd0fc06ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0fa7fc700 1 -- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0fc15d700 msgr2=0x7fd0fc1f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0fa7fc700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0fc15d700 0x7fd0fc1f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0fa7fc700 1 -- 172.21.15.5:0/2864465903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009a70 con 0x7fd0fc15dae0 2024-05-08T23:28:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0faffd700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0fc15d700 0x7fd0fc1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0fb7fe700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0fc161c00 0x7fd0fc06ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0fa7fc700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc15dae0 0x7fd0fc15e580 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000c4d0 tx=0x7fd0f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0e37fe700 1 -- 172.21.15.5:0/2864465903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0011790 con 0x7fd0fc15dae0 2024-05-08T23:28:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0e37fe700 1 -- 172.21.15.5:0/2864465903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f00118f0 con 0x7fd0fc15dae0 2024-05-08T23:28:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0fc1f3ad0 con 0x7fd0fc15dae0 2024-05-08T23:28:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd0e37fe700 1 -- 172.21.15.5:0/2864465903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0011ca0 con 0x7fd0fc15dae0 2024-05-08T23:28:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.211+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0fc1f4090 con 0x7fd0fc15dae0 2024-05-08T23:28:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.212+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0fc1f45d0 con 0x7fd0fc15dae0 2024-05-08T23:28:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.213+0000 7fd0e37fe700 1 -- 172.21.15.5:0/2864465903 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd0f001c460 con 0x7fd0fc15dae0 2024-05-08T23:28:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.213+0000 7fd0e37fe700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0e806ac90 0x7fd0e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.213+0000 7fd0e37fe700 1 -- 172.21.15.5:0/2864465903 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0f0089100 con 0x7fd0fc15dae0 2024-05-08T23:28:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.213+0000 7fd0faffd700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0e806ac90 0x7fd0e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.214+0000 7fd0faffd700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0e806ac90 0x7fd0e806d150 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0016e0 tx=0x7fd0ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:04.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.218+0000 7fd0e37fe700 1 -- 172.21.15.5:0/2864465903 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd0fc1f45d0 con 0x7fd0fc15dae0 2024-05-08T23:28:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.524+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd0fc1f45d0 con 0x7fd0fc15dae0 2024-05-08T23:28:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.528+0000 7fd0e37fe700 1 -- 172.21.15.5:0/2864465903 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd0fc1f45d0 con 0x7fd0fc15dae0 2024-05-08T23:28:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0e806ac90 msgr2=0x7fd0e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0e806ac90 0x7fd0e806d150 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0016e0 tx=0x7fd0ec009040 comp rx=0 tx=0).stop 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc15dae0 msgr2=0x7fd0fc15e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc15dae0 0x7fd0fc15e580 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000c4d0 tx=0x7fd0f000c890 comp rx=0 tx=0).stop 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 shutdown_connections 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd0e806ac90 0x7fd0e806d150 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd0fc161c00 0x7fd0fc06ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0fc15d700 0x7fd0fc1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 --2- 172.21.15.5:0/2864465903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0fc15dae0 0x7fd0fc15e580 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 >> 172.21.15.5:0/2864465903 conn(0x7fd0fc08afe0 msgr2=0x7fd0fc15b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 shutdown_connections 2024-05-08T23:28:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.531+0000 7fd1014ae700 1 -- 172.21.15.5:0/2864465903 wait complete. 2024-05-08T23:28:04.533 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-08T23:28:04.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.681+0000 7f838dae4700 1 Processor -- start 2024-05-08T23:28:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.681+0000 7f838dae4700 1 -- start start 2024-05-08T23:28:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.682+0000 7f838dae4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f838815db50 0x7f83880d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.682+0000 7f838dae4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880cea40 con 0x7f838815d770 2024-05-08T23:28:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.682+0000 7f838dae4700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880ce190 con 0x7f83880d0590 2024-05-08T23:28:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.682+0000 7f838dae4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880d0a20 con 0x7f838815db50 2024-05-08T23:28:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.682+0000 7f83877fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f838815d770 0x7f83880ceed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34484/0 (socket says 172.21.15.5:34484) 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.682+0000 7f83877fe700 1 -- 172.21.15.5:0/3899550196 learned_addr learned my addr 172.21.15.5:0/3899550196 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f8386ffd700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f838815db50 0x7f83880d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733977756 0 0) 0x7f83880cea40 con 0x7f838815d770 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83700035b0 con 0x7f838815d770 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630272224 0 0) 0x7f83880ce190 con 0x7f83880d0590 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83880cea40 con 0x7f83880d0590 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2401227802 0 0) 0x7f83700035b0 con 0x7f838815d770 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f838815db50 msgr2=0x7f83880d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f838815db50 0x7f83880d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 >> v1:172.21.15.49:6789/0 conn(0x7f83880d0590 legacy=0x7f838806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880ce410 con 0x7f838815d770 2024-05-08T23:28:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f8378001670 con 0x7f838815d770 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 >> v1:172.21.15.5:6789/0 conn(0x7f838815d770 legacy=0x7f83880ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f838815db50 0x7f8370003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004120 0x7f8370004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8370004d90 0x7f8370005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83880d0a20 con 0x7f8370004d90 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837000baf0 con 0x7f8370004120 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837000bcb0 con 0x7f838815db50 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f8387fff700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8370004d90 0x7f8370005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f8386ffd700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004120 0x7f8370004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f83877fe700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f838815db50 0x7f8370003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f8387fff700 1 -- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f838815db50 msgr2=0x7f8370003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f8387fff700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f838815db50 0x7f8370003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.683+0000 7f8387fff700 1 -- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004120 msgr2=0x7f8370004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f8387fff700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004120 0x7f8370004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f8387fff700 1 -- 172.21.15.5:0/3899550196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83700035b0 con 0x7f8370004d90 2024-05-08T23:28:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f8387fff700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8370004d90 0x7f8370005210 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f838000c2f0 tx=0x7f8370005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1910367e0a9295c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380001f30 con 0x7f8370004d90 2024-05-08T23:28:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838000ce90 con 0x7f8370004d90 2024-05-08T23:28:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f83867fc700 1 -- 172.21.15.5:0/3899550196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380003c40 con 0x7f8370004d90 2024-05-08T23:28:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f838dae4700 1 -- 172.21.15.5:0/3899550196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8370004d90 msgr2=0x7f8370005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f838dae4700 1 --2- 172.21.15.5:0/3899550196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8370004d90 0x7f8370005210 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f838000c2f0 tx=0x7f8370005880 comp rx=0 tx=0).stop 2024-05-08T23:28:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f8387fff700 1 -- 172.21.15.5:0/3899550196 reap_dead start 2024-05-08T23:28:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f838dae4700 1 -- 172.21.15.5:0/3899550196 shutdown_connections 2024-05-08T23:28:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f838dae4700 1 -- 172.21.15.5:0/3899550196 >> 172.21.15.5:0/3899550196 conn(0x7f838808d070 msgr2=0x7f838809c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f838dae4700 1 -- 172.21.15.5:0/3899550196 shutdown_connections 2024-05-08T23:28:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.684+0000 7f838dae4700 1 -- 172.21.15.5:0/3899550196 wait complete. 2024-05-08T23:28:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.686+0000 7f838dae4700 1 Processor -- start 2024-05-08T23:28:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.686+0000 7f838dae4700 1 -- start start 2024-05-08T23:28:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f838dae4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004d90 0x7f83881f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f838dae4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838815bda0 0x7f838815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f838dae4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83880b8880 0x7f83880b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f838dae4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838815c760 con 0x7f838815bda0 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f838dae4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83881f41d0 con 0x7f8370004d90 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f838dae4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880b9230 con 0x7f83880b8880 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8386ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838815bda0 0x7f838815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8386ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838815bda0 0x7f838815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59046/0 (socket says 172.21.15.5:59046) 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8386ffd700 1 -- 172.21.15.5:0/2093899031 learned_addr learned my addr 172.21.15.5:0/2093899031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f83877fe700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004d90 0x7f83881f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8387fff700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83880b8880 0x7f83880b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8386ffd700 1 -- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83880b8880 msgr2=0x7f83880b8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8386ffd700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83880b8880 0x7f83880b8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8386ffd700 1 -- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004d90 msgr2=0x7f83881f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8386ffd700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004d90 0x7f83881f3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.687+0000 7f8386ffd700 1 -- 172.21.15.5:0/2093899031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880caa60 con 0x7f838815bda0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.688+0000 7f8386ffd700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838815bda0 0x7f838815c220 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f837c0119c0 tx=0x7f837c011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.688+0000 7f836ffff700 1 -- 172.21.15.5:0/2093899031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c004240 con 0x7f838815bda0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.688+0000 7f836ffff700 1 -- 172.21.15.5:0/2093899031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c0043a0 con 0x7f838815bda0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.688+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83880cad40 con 0x7f838815bda0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.688+0000 7f836ffff700 1 -- 172.21.15.5:0/2093899031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c00f730 con 0x7f838815bda0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.688+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83880cb300 con 0x7f838815bda0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.689+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8388068000 con 0x7f838815bda0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.689+0000 7f836ffff700 1 -- 172.21.15.5:0/2093899031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f837c00f890 con 0x7f838815bda0 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.690+0000 7f836ffff700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f837406ac90 0x7f837406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.690+0000 7f836ffff700 1 -- 172.21.15.5:0/2093899031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f837c0898a0 con 0x7f838815bda0 2024-05-08T23:28:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.690+0000 7f83877fe700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f837406ac90 0x7f837406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.691+0000 7f83877fe700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f837406ac90 0x7f837406d150 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8378001eb0 tx=0x7f8378009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.694+0000 7f836ffff700 1 -- 172.21.15.5:0/2093899031 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f837c054a80 con 0x7f838815bda0 2024-05-08T23:28:04.801 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:28:04.801+0000 7fa0ab37b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:28:04.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.959+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f838806bbb0 con 0x7f838815bda0 2024-05-08T23:28:04.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.960+0000 7f836ffff700 1 -- 172.21.15.5:0/2093899031 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4727 (secure 0 0 0) 0x7f837c019020 con 0x7f838815bda0 2024-05-08T23:28:04.960 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:04.960 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":99,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7052,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7058,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:02.279622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:04.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f837406ac90 msgr2=0x7f837406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f837406ac90 0x7f837406d150 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8378001eb0 tx=0x7f8378009040 comp rx=0 tx=0).stop 2024-05-08T23:28:04.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838815bda0 msgr2=0x7f838815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:04.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838815bda0 0x7f838815c220 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f837c0119c0 tx=0x7f837c011d80 comp rx=0 tx=0).stop 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 shutdown_connections 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f837406ac90 0x7f837406d150 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f83880b8880 0x7f83880b8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8370004d90 0x7f83881f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 --2- 172.21.15.5:0/2093899031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838815bda0 0x7f838815c220 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 >> 172.21.15.5:0/2093899031 conn(0x7f838808d070 msgr2=0x7f83880b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 shutdown_connections 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:04.962+0000 7f838dae4700 1 -- 172.21.15.5:0/2093899031 wait complete. 2024-05-08T23:28:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 99 2024-05-08T23:28:05.379 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:27:59.271749+0000', 'modified': '2024-05-08T23:28:02.279622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8592}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8592': {'gid': 8592, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6834/1510899872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 1510899872}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 1510899872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:28:05.379 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:28:05.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:28:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.525+0000 7ff88c846700 1 Processor -- start 2024-05-08T23:28:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.525+0000 7ff88c846700 1 -- start start 2024-05-08T23:28:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff88c846700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff88815b900 0x7ff8880cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff88c846700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff888099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff88c846700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff88809a410 0x7ff8880cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff88c846700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8880b1410 con 0x7ff88809a410 2024-05-08T23:28:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff88c846700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88809a8a0 con 0x7ff88815b900 2024-05-08T23:28:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff88c846700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880b0e30 con 0x7ff888099a50 2024-05-08T23:28:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff886d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff88809a410 0x7ff8880cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff886d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff88809a410 0x7ff8880cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59054/0 (socket says 172.21.15.5:59054) 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff886d9d700 1 -- 172.21.15.5:0/1771627672 learned_addr learned my addr 172.21.15.5:0/1771627672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff885d9b700 1 --2- 172.21.15.5:0/1771627672 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff888099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.526+0000 7ff88659c700 1 --2- 172.21.15.5:0/1771627672 >> v2:172.21.15.49:3300/0 conn(0x7ff88815b900 0x7ff8880cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff886d9d700 1 -- 172.21.15.5:0/1771627672 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 msgr2=0x7ff888099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff886d9d700 1 --2- 172.21.15.5:0/1771627672 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff888099ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff886d9d700 1 -- 172.21.15.5:0/1771627672 >> v2:172.21.15.49:3300/0 conn(0x7ff88815b900 msgr2=0x7ff8880cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff886d9d700 1 --2- 172.21.15.5:0/1771627672 >> v2:172.21.15.49:3300/0 conn(0x7ff88815b900 0x7ff8880cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff886d9d700 1 -- 172.21.15.5:0/1771627672 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880cffd0 con 0x7ff88809a410 2024-05-08T23:28:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff886d9d700 1 --2- 172.21.15.5:0/1771627672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880cbf40 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff888072130 tx=0x7ff88000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fa0db2054a629f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff88559a700 1 -- 172.21.15.5:0/1771627672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000e070 con 0x7ff88809a410 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff88559a700 1 -- 172.21.15.5:0/1771627672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88000b910 con 0x7ff88809a410 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.527+0000 7ff88559a700 1 -- 172.21.15.5:0/1771627672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000bcc0 con 0x7ff88809a410 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 -- 172.21.15.5:0/1771627672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 msgr2=0x7ff8880cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 --2- 172.21.15.5:0/1771627672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880cbf40 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff888072130 tx=0x7ff88000b330 comp rx=0 tx=0).stop 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 -- 172.21.15.5:0/1771627672 shutdown_connections 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 --2- 172.21.15.5:0/1771627672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880cbf40 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 --2- 172.21.15.5:0/1771627672 >> v2:172.21.15.49:3300/0 conn(0x7ff88815b900 0x7ff8880cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 --2- 172.21.15.5:0/1771627672 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff888099ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 -- 172.21.15.5:0/1771627672 >> 172.21.15.5:0/1771627672 conn(0x7ff8880b2f20 msgr2=0x7ff8880b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 -- 172.21.15.5:0/1771627672 shutdown_connections 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.528+0000 7ff88c846700 1 -- 172.21.15.5:0/1771627672 wait complete. 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.529+0000 7ff88c846700 1 Processor -- start 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.529+0000 7ff88c846700 1 -- start start 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff88c846700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff88806c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff88c846700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff88c846700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff88815b900 0x7ff8880c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff88c846700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88806d3b0 con 0x7ff88809a410 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff88c846700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880b6f30 con 0x7ff88815b900 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff88c846700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880c8730 con 0x7ff888099a50 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff885d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff885d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59062/0 (socket says 172.21.15.5:59062) 2024-05-08T23:28:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff885d9b700 1 -- 172.21.15.5:0/858963412 learned_addr learned my addr 172.21.15.5:0/858963412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff886d9d700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff88815b900 0x7ff8880c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.530+0000 7ff88659c700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff88806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff885d9b700 1 -- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 msgr2=0x7ff88806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff885d9b700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff88806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff885d9b700 1 -- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff88815b900 msgr2=0x7ff8880c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff885d9b700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff88815b900 0x7ff8880c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff885d9b700 1 -- 172.21.15.5:0/858963412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff880009b30 con 0x7ff88809a410 2024-05-08T23:28:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff88659c700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff88806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff885d9b700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880b69f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e870 tx=0x7ff87000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff87effd700 1 -- 172.21.15.5:0/858963412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870013680 con 0x7ff88809a410 2024-05-08T23:28:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff87effd700 1 -- 172.21.15.5:0/858963412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8700137e0 con 0x7ff88809a410 2024-05-08T23:28:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff87effd700 1 -- 172.21.15.5:0/858963412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870013b90 con 0x7ff88809a410 2024-05-08T23:28:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff88c846700 1 -- 172.21.15.5:0/858963412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8880c7790 con 0x7ff88809a410 2024-05-08T23:28:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.531+0000 7ff88c846700 1 -- 172.21.15.5:0/858963412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88806cdf0 con 0x7ff88809a410 2024-05-08T23:28:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.533+0000 7ff87effd700 1 -- 172.21.15.5:0/858963412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff870013cf0 con 0x7ff88809a410 2024-05-08T23:28:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.534+0000 7ff87effd700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff86806ac90 0x7ff86806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.534+0000 7ff87effd700 1 -- 172.21.15.5:0/858963412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8700891c0 con 0x7ff88809a410 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.534+0000 7ff88659c700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff86806ac90 0x7ff86806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.535+0000 7ff88659c700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff86806ac90 0x7ff86806d150 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff878009b90 tx=0x7ff878008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.536+0000 7ff898a00b80 1 Processor -- start 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.536+0000 7ff898a00b80 1 -- start start 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 0x55c90d0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0b5320 0x55c90d0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0e1d80 0x55c90d0e2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff898a00b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90d00e100 con 0x55c90d0b5320 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff898a00b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90d00bac0 con 0x55c90d0da750 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff898a00b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c90d0073b0 con 0x55c90d0e1d80 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0b5320 0x55c90d0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0b5320 0x55c90d0b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59070/0 (socket says 172.21.15.5:59070) 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 -- 172.21.15.5:0/1814329551 learned_addr learned my addr 172.21.15.5:0/1814329551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff886d9d700 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0e1d80 0x55c90d0e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff88659c700 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 0x55c90d0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 -- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0e1d80 msgr2=0x55c90d0e2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0e1d80 0x55c90d0e2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 -- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 msgr2=0x55c90d0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 0x55c90d0b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 -- 172.21.15.5:0/1814329551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c90ce88360 con 0x55c90d0b5320 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff886d9d700 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0e1d80 0x55c90d0e2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.537+0000 7ff885d9b700 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0b5320 0x55c90d0b8050 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff87005ff30 tx=0x7ff870057a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb02565860df9fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff867fff700 1 -- 172.21.15.5:0/1814329551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8700580d0 con 0x55c90d0b5320 2024-05-08T23:28:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff867fff700 1 -- 172.21.15.5:0/1814329551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87005bea0 con 0x55c90d0b5320 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff867fff700 1 -- 172.21.15.5:0/1814329551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870060460 con 0x55c90d0b5320 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff898a00b80 1 -- 172.21.15.5:0/1814329551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0b5320 msgr2=0x55c90d0b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0b5320 0x55c90d0b8050 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff87005ff30 tx=0x7ff870057a50 comp rx=0 tx=0).stop 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff898a00b80 1 -- 172.21.15.5:0/1814329551 shutdown_connections 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0e1d80 0x55c90d0e2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 0x55c90d0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1814329551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0b5320 0x55c90d0b8050 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.538+0000 7ff898a00b80 1 -- 172.21.15.5:0/1814329551 >> 172.21.15.5:0/1814329551 conn(0x55c90d0151c0 msgr2=0x55c90d0fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.539+0000 7ff898a00b80 1 -- 172.21.15.5:0/1814329551 shutdown_connections 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.539+0000 7ff898a00b80 1 -- 172.21.15.5:0/1814329551 wait complete. 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.539+0000 7ff898a00b80 1 -- start start 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.539+0000 7ff898a00b80 1 Processor -- start 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.540+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0b5320 0x55c90d0e5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.540+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0da750 0x55c90d0e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.540+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.540+0000 7ff898a00b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90cfa3510 con 0x55c90d0e1d80 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.540+0000 7ff898a00b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90cfa31b0 con 0x55c90d0b5320 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.540+0000 7ff898a00b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c90ce8a650 con 0x55c90d0da750 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.540+0000 7ff898a00b80 10 client.0 _ll_register_callbacks cb 0x55c90cfacae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.540+0000 7ff886d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff886d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0d9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59082/0 (socket says 172.21.15.5:59082) 2024-05-08T23:28:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff886d9d700 1 -- 172.21.15.5:0/2471489698 learned_addr learned my addr 172.21.15.5:0/2471489698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff88659c700 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0b5320 0x55c90d0e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff885d9b700 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0da750 0x55c90d0e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff886d9d700 1 -- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0da750 msgr2=0x55c90d0e42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff886d9d700 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0da750 0x55c90d0e42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff886d9d700 1 -- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0b5320 msgr2=0x55c90d0e5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff886d9d700 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0b5320 0x55c90d0e5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff886d9d700 1 -- 172.21.15.5:0/2471489698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88806cdf0 con 0x55c90d0e1d80 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff885d9b700 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0da750 0x55c90d0e42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff886d9d700 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0d9570 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff880003c50 tx=0x7ff880004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c8b35d73d2e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff8657fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff8657fa700 1 -- 172.21.15.5:0/2471489698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880004450 con 0x55c90d0e1d80 2024-05-08T23:28:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff8657fa700 1 -- 172.21.15.5:0/2471489698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8800045b0 con 0x55c90d0e1d80 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff898a00b80 1 -- 172.21.15.5:0/2471489698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c90cf60220 con 0x55c90d0e1d80 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff8657fa700 1 -- 172.21.15.5:0/2471489698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8800092e0 con 0x55c90d0e1d80 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff898a00b80 10 client.8730 fetch_fsmap learned FSMap version 99 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.541+0000 7ff898a00b80 1 -- 172.21.15.5:0/2471489698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55c90ce34b10 con 0x55c90d0e1d80 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff8657fa700 1 -- 172.21.15.5:0/2471489698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c90cf60220 con 0x55c90d0e1d80 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff8657fa700 1 -- 172.21.15.5:0/2471489698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 99) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff88000b3a0 con 0x55c90d0e1d80 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff898a00b80 10 client.8730 fetch_fsmap finished waiting for FSMap version 99 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff898a00b80 10 client.8730 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff898a00b80 20 client.8730 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff898a00b80 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55c90d0d8a20 0x55c90d0d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff898a00b80 4 client.8730 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff898a00b80 1 -- 172.21.15.5:0/2471489698 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c90d086790 con 0x55c90d0d8a20 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff88659c700 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55c90d0d8a20 0x55c90d0d8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff8657fa700 10 client.8730 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.542+0000 7ff88659c700 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55c90d0d8a20 0x55c90d0d8e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126923601f7eae49 server_cookie=797f0e0222e71918 in_seq=0 out_seq=0 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff8657fa700 1 -- 172.21.15.5:0/2471489698 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c90d086790 con 0x55c90d0d8a20 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff8657fa700 10 client.8730 handle_command_reply: tid=0 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 client.8730 shutdown 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 -- 172.21.15.5:0/2471489698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 msgr2=0x55c90d0d9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0d9570 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff880003c50 tx=0x7ff880004b40 comp rx=0 tx=0).stop 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 -- 172.21.15.5:0/2471489698 shutdown_connections 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55c90d0d8a20 0x55c90d0d8e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0d9570 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0b5320 0x55c90d0e5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff8657fa700 0 client.8730 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:05.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 --2- 172.21.15.5:0/2471489698 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0da750 0x55c90d0e42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 -- 172.21.15.5:0/2471489698 >> 172.21.15.5:0/2471489698 conn(0x55c90d0151c0 msgr2=0x55c90d0b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:05.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 -- 172.21.15.5:0/2471489698 shutdown_connections 2024-05-08T23:28:05.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 1 -- 172.21.15.5:0/2471489698 wait complete. 2024-05-08T23:28:05.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.543+0000 7ff898a00b80 20 client.8730 trim_cache size 0 max 16384 2024-05-08T23:28:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff898a00b80 1 Processor -- start 2024-05-08T23:28:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff898a00b80 1 -- start start 2024-05-08T23:28:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 0x55c90d0e4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 0x55c90d0fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff898a00b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90d0dae00 con 0x55c90d0e1d80 2024-05-08T23:28:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff898a00b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90d1167c0 con 0x55c90d0da750 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff898a00b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c90d0e23b0 con 0x55c90d0b5320 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.563+0000 7ff886d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff886d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0fd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59100/0 (socket says 172.21.15.5:59100) 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff886d9d700 1 -- 172.21.15.5:0/531151794 learned_addr learned my addr 172.21.15.5:0/531151794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff885d9b700 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 0x55c90d0fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff88659c700 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 0x55c90d0e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff886d9d700 1 -- 172.21.15.5:0/531151794 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 msgr2=0x55c90d0e4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff886d9d700 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 0x55c90d0e4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff886d9d700 1 -- 172.21.15.5:0/531151794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 msgr2=0x55c90d0fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff886d9d700 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 0x55c90d0fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff886d9d700 1 -- 172.21.15.5:0/531151794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c90cfcd510 con 0x55c90d0e1d80 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff886d9d700 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0fd4f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff88000d010 tx=0x7ff88001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc15358c78cfc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff87cff9700 1 -- 172.21.15.5:0/531151794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8800047c0 con 0x55c90d0e1d80 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff87cff9700 1 -- 172.21.15.5:0/531151794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88000b330 con 0x55c90d0e1d80 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.564+0000 7ff87cff9700 1 -- 172.21.15.5:0/531151794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880004250 con 0x55c90d0e1d80 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 -- 172.21.15.5:0/531151794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 msgr2=0x55c90d0fd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0fd4f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff88000d010 tx=0x7ff88001f040 comp rx=0 tx=0).stop 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 -- 172.21.15.5:0/531151794 shutdown_connections 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0e1d80 0x55c90d0fd4f0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 0x55c90d0e4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 --2- 172.21.15.5:0/531151794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0da750 0x55c90d0fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 -- 172.21.15.5:0/531151794 >> 172.21.15.5:0/531151794 conn(0x55c90d0151c0 msgr2=0x55c90d0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 -- 172.21.15.5:0/531151794 shutdown_connections 2024-05-08T23:28:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 -- 172.21.15.5:0/531151794 wait complete. 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 -- start start 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.565+0000 7ff898a00b80 1 Processor -- start 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.566+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 0x55c90d0f5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.566+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0da750 0x55c90d0d7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.566+0000 7ff898a00b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0e1d80 0x55c90d0f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.566+0000 7ff898a00b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90d0f65b0 con 0x55c90d0da750 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.566+0000 7ff898a00b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90d0f80a0 con 0x55c90d0e1d80 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.566+0000 7ff898a00b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c90d0f62d0 con 0x55c90d0b5320 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.566+0000 7ff898a00b80 10 client.0 _ll_register_callbacks cb 0x55c90cfacae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0da750 0x55c90d0d7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0da750 0x55c90d0d7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59104/0 (socket says 172.21.15.5:59104) 2024-05-08T23:28:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 -- 172.21.15.5:0/1897917481 learned_addr learned my addr 172.21.15.5:0/1897917481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff886d9d700 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0e1d80 0x55c90d0f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff88659c700 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 0x55c90d0f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 -- 172.21.15.5:0/1897917481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 msgr2=0x55c90d0f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 0x55c90d0f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 -- 172.21.15.5:0/1897917481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0e1d80 msgr2=0x55c90d0f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0e1d80 0x55c90d0f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 -- 172.21.15.5:0/1897917481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c90ce34b10 con 0x55c90d0da750 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff885d9b700 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0da750 0x55c90d0d7eb0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff8700877d0 tx=0x7ff870089000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44037a0770746908 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff8657fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff8657fa700 1 -- 172.21.15.5:0/1897917481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8700580d0 con 0x55c90d0da750 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff8657fa700 1 -- 172.21.15.5:0/1897917481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff870076e40 con 0x55c90d0da750 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff898a00b80 1 -- 172.21.15.5:0/1897917481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff87005bea0 con 0x55c90d0da750 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff8657fa700 1 -- 172.21.15.5:0/1897917481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff870060460 con 0x55c90d0da750 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff898a00b80 10 client.8736 fetch_fsmap learned FSMap version 99 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.567+0000 7ff898a00b80 1 -- 172.21.15.5:0/1897917481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55c90ce8cc80 con 0x55c90d0da750 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff8657fa700 1 -- 172.21.15.5:0/1897917481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87005bea0 con 0x55c90d0da750 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff8657fa700 1 -- 172.21.15.5:0/1897917481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 99) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff8700872f0 con 0x55c90d0da750 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff898a00b80 10 client.8736 fetch_fsmap finished waiting for FSMap version 99 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff898a00b80 10 client.8736 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff898a00b80 20 client.8736 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55c90d0f4240 0x55c90d0f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff898a00b80 4 client.8736 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff898a00b80 1 -- 172.21.15.5:0/1897917481 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c90cfffe70 con 0x55c90d0f4240 2024-05-08T23:28:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff88659c700 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55c90d0f4240 0x55c90d0f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff8657fa700 10 client.8736 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff88659c700 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55c90d0f4240 0x55c90d0f4620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08ecc7fdbf0036b server_cookie=1b7ee0775f8a66e6 in_seq=0 out_seq=0 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff8657fa700 1 -- 172.21.15.5:0/1897917481 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c90cfffe70 con 0x55c90d0f4240 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff8657fa700 10 client.8736 handle_command_reply: tid=0 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.568+0000 7ff898a00b80 1 client.8736 shutdown 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 -- 172.21.15.5:0/1897917481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0da750 msgr2=0x55c90d0d7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0da750 0x55c90d0d7eb0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff8700877d0 tx=0x7ff870089000 comp rx=0 tx=0).stop 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 -- 172.21.15.5:0/1897917481 shutdown_connections 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55c90d0f4240 0x55c90d0f4620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c90d0e1d80 0x55c90d0f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55c90d0b5320 0x55c90d0f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff8657fa700 0 client.8736 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 --2- 172.21.15.5:0/1897917481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90d0da750 0x55c90d0d7eb0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 -- 172.21.15.5:0/1897917481 >> 172.21.15.5:0/1897917481 conn(0x55c90d0151c0 msgr2=0x55c90d0b80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 -- 172.21.15.5:0/1897917481 shutdown_connections 2024-05-08T23:28:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 1 -- 172.21.15.5:0/1897917481 wait complete. 2024-05-08T23:28:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff898a00b80 20 client.8736 trim_cache size 0 max 16384 2024-05-08T23:28:05.578 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:28:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:28:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 8592, 2024-05-08T23:28:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:28:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:28:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:28:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.2959816760000002, 2024-05-08T23:28:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 99, 2024-05-08T23:28:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-05-08T23:28:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-05-08T23:28:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 11.535471469999999 2024-05-08T23:28:05.580 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 -- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff86806ac90 msgr2=0x7ff86806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff86806ac90 0x7ff86806d150 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff878009b90 tx=0x7ff878008040 comp rx=0 tx=0).stop 2024-05-08T23:28:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 -- 172.21.15.5:0/858963412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 msgr2=0x7ff8880b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880b69f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e870 tx=0x7ff87000eb80 comp rx=0 tx=0).stop 2024-05-08T23:28:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 -- 172.21.15.5:0/858963412 shutdown_connections 2024-05-08T23:28:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff86806ac90 0x7ff86806d150 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff88815b900 0x7ff8880c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff888099a50 0x7ff88806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 --2- 172.21.15.5:0/858963412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88809a410 0x7ff8880b69f0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.569+0000 7ff88c846700 1 -- 172.21.15.5:0/858963412 >> 172.21.15.5:0/858963412 conn(0x7ff8880b2f20 msgr2=0x7ff8880b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.570+0000 7ff88c846700 1 -- 172.21.15.5:0/858963412 shutdown_connections 2024-05-08T23:28:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.570+0000 7ff88c846700 1 -- 172.21.15.5:0/858963412 wait complete. 2024-05-08T23:28:05.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:28:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.721+0000 7fab3842d700 1 Processor -- start 2024-05-08T23:28:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.721+0000 7fab3842d700 1 -- start start 2024-05-08T23:28:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab3842d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3015ee40 0x7fab300cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab3842d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fab300cc370 0x7fab30099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab3842d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3009a3e0 con 0x7fab30099f90 2024-05-08T23:28:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab3842d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3006f810 con 0x7fab300cc370 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab3842d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3006eee0 con 0x7fab3015ee40 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab359c8700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fab300cc370 0x7fab30099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab361c9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3015ee40 0x7fab300cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab369ca700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fab30099f90 0x7fab300cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:56936/0 (socket says 172.21.15.5:56936) 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab359c8700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fab300cc370 0x7fab30099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:47050/0 (socket says 172.21.15.5:47050) 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab369ca700 1 -- 172.21.15.5:0/1761148694 learned_addr learned my addr 172.21.15.5:0/1761148694 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab359c8700 1 -- 172.21.15.5:0/1761148694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3015ee40 msgr2=0x7fab300cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab359c8700 1 --2- 172.21.15.5:0/1761148694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3015ee40 0x7fab300cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.723+0000 7fab359c8700 1 -- 172.21.15.5:0/1761148694 >> v1:172.21.15.49:6789/0 conn(0x7fab30099f90 legacy=0x7fab300cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.724+0000 7fab359c8700 1 -- 172.21.15.5:0/1761148694 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3006f1c0 con 0x7fab300cc370 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.724+0000 7fab361c9700 1 --2- 172.21.15.5:0/1761148694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3015ee40 0x7fab300cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.724+0000 7fab359c8700 1 --2- 172.21.15.5:0/1761148694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab300cc370 0x7fab30099a50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fab20009840 tx=0x7fab20009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d72688bd691fec75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.724+0000 7fab351c7700 1 -- 172.21.15.5:0/1761148694 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20011830 con 0x7fab300cc370 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.724+0000 7fab351c7700 1 -- 172.21.15.5:0/1761148694 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab20011990 con 0x7fab300cc370 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.724+0000 7fab351c7700 1 -- 172.21.15.5:0/1761148694 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20011d20 con 0x7fab300cc370 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.724+0000 7fab3842d700 1 -- 172.21.15.5:0/1761148694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab300cc370 msgr2=0x7fab30099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.724+0000 7fab3842d700 1 --2- 172.21.15.5:0/1761148694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab300cc370 0x7fab30099a50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fab20009840 tx=0x7fab20009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.725+0000 7fab3842d700 1 -- 172.21.15.5:0/1761148694 shutdown_connections 2024-05-08T23:28:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.725+0000 7fab3842d700 1 --2- 172.21.15.5:0/1761148694 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab3015ee40 0x7fab300cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.725+0000 7fab3842d700 1 --2- 172.21.15.5:0/1761148694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab300cc370 0x7fab30099a50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.725+0000 7fab3842d700 1 -- 172.21.15.5:0/1761148694 >> 172.21.15.5:0/1761148694 conn(0x7fab300b3010 msgr2=0x7fab300b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.725+0000 7fab3842d700 1 -- 172.21.15.5:0/1761148694 shutdown_connections 2024-05-08T23:28:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.725+0000 7fab3842d700 1 -- 172.21.15.5:0/1761148694 wait complete. 2024-05-08T23:28:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.726+0000 7fab3842d700 1 Processor -- start 2024-05-08T23:28:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.726+0000 7fab3842d700 1 -- start start 2024-05-08T23:28:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab3842d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30099f90 0x7fab300b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab3842d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab300cc370 0x7fab300b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab3842d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3015ee40 0x7fab300b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab3842d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab300c85f0 con 0x7fab3015ee40 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab3842d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab300c98a0 con 0x7fab30099f90 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab3842d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab300c8820 con 0x7fab300cc370 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab369ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3015ee40 0x7fab300b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab369ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3015ee40 0x7fab300b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59116/0 (socket says 172.21.15.5:59116) 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab369ca700 1 -- 172.21.15.5:0/1385253876 learned_addr learned my addr 172.21.15.5:0/1385253876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab361c9700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30099f90 0x7fab300b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab359c8700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab300cc370 0x7fab300b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab369ca700 1 -- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab300cc370 msgr2=0x7fab300b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab369ca700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab300cc370 0x7fab300b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab369ca700 1 -- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30099f90 msgr2=0x7fab300b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab369ca700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30099f90 0x7fab300b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab369ca700 1 -- 172.21.15.5:0/1385253876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c0094a0 con 0x7fab3015ee40 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.727+0000 7fab359c8700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab300cc370 0x7fab300b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.728+0000 7fab369ca700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3015ee40 0x7fab300b4830 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fab2800ca80 tx=0x7fab2800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.728+0000 7fab26ffd700 1 -- 172.21.15.5:0/1385253876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2800a9e0 con 0x7fab3015ee40 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.728+0000 7fab26ffd700 1 -- 172.21.15.5:0/1385253876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2800ab40 con 0x7fab3015ee40 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.728+0000 7fab26ffd700 1 -- 172.21.15.5:0/1385253876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28012400 con 0x7fab3015ee40 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.728+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab200094a0 con 0x7fab3015ee40 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.728+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab300c8b90 con 0x7fab3015ee40 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.729+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab30068000 con 0x7fab3015ee40 2024-05-08T23:28:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.729+0000 7fab26ffd700 1 -- 172.21.15.5:0/1385253876 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fab2801b3b0 con 0x7fab3015ee40 2024-05-08T23:28:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.730+0000 7fab26ffd700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab1406ac90 0x7fab1406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.730+0000 7fab26ffd700 1 -- 172.21.15.5:0/1385253876 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab28088cd0 con 0x7fab3015ee40 2024-05-08T23:28:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.730+0000 7fab361c9700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab1406ac90 0x7fab1406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.731+0000 7fab361c9700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab1406ac90 0x7fab1406d150 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009d70 tx=0x7fab2c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:05.735+0000 7fab26ffd700 1 -- 172.21.15.5:0/1385253876 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fab28053f60 con 0x7fab3015ee40 2024-05-08T23:28:06.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.005+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fab3009a3e0 con 0x7fab3015ee40 2024-05-08T23:28:06.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.005+0000 7fab26ffd700 1 -- 172.21.15.5:0/1385253876 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7fab3009a3e0 con 0x7fab3015ee40 2024-05-08T23:28:06.005 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:28:06.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab1406ac90 msgr2=0x7fab1406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab1406ac90 0x7fab1406d150 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009d70 tx=0x7fab2c009210 comp rx=0 tx=0).stop 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3015ee40 msgr2=0x7fab300b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3015ee40 0x7fab300b4830 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fab2800ca80 tx=0x7fab2800cd90 comp rx=0 tx=0).stop 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 shutdown_connections 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fab1406ac90 0x7fab1406d150 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3015ee40 0x7fab300b4830 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30099f90 0x7fab300b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 --2- 172.21.15.5:0/1385253876 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fab300cc370 0x7fab300b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 >> 172.21.15.5:0/1385253876 conn(0x7fab300b3010 msgr2=0x7fab300cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.007+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 shutdown_connections 2024-05-08T23:28:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:06.008+0000 7fab3842d700 1 -- 172.21.15.5:0/1385253876 wait complete. 2024-05-08T23:28:06.021 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T23:28:06.021 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:28:06.022 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-08T23:28:06.022 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:28:06.022 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:28:06.022 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T23:28:06.022 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T23:28:06.022 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:28:06.022 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:28:06.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.022 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-08T23:28:06.043 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:28:06.043 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:28:06.043 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:28:06.044 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:28:06.044 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-08T23:28:06.044 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:28:06.044 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:28:06.044 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-08T23:28:06.044 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-08T23:28:06.045 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:28:06.045 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-08T23:28:06.045 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-08T23:28:06.045 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:28:06.045 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:28:06.045 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-08T23:28:06.045 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:28:06.045 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41831sec preferred_lft 41831sec 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:28:06.046 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::1c2e:b3ff:fe6b:56f/64 scope link 2024-05-08T23:28:06.047 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:28:06.047 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:28:06.047 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 86:a4:2a:7c:ce:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:28:06.047 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::84a4:2aff:fe7c:ce1a/64 scope link 2024-05-08T23:28:06.047 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:28:06.047 INFO:teuthology.orchestra.run.smithi005.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:28:06.047 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether e2:3e:2c:7a:5d:b1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:28:06.048 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::e03e:2cff:fe7a:5db1/64 scope link 2024-05-08T23:28:06.048 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:28:06.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.048 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:28:06.115 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-08T23:28:06.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.117 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-08T23:28:06.181 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-08T23:28:06.181 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:28:06.181 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:28:06.182 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:28:06.182 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:28:06.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.182 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-05-08T23:28:06.182 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-08T23:28:06.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:28:06.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:28:06.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:28:06.183 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-08T23:28:06.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.309 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:28:06.329 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:28:06.329 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:28:06.329 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:28:06.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.330 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-08T23:28:06.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:28:06.410 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:28:06.472 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:28:06.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:28:06.472 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:28:06.534 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T23:28:06.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:28:06.632 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:28:06.632 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:28:06.652 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:28:06.653 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:28:06.653 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:28:06.653 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:28:06.653 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:28:06.653 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:28:06.653 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:28:06.653 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:28:06.672 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:28:06.672 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:28:06.672 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.49:3301" 2024-05-08T23:28:06.672 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@4fd8ca56-5ac1-431d-b985-b50a13a8130a.cephfs=/ 2024-05-08T23:28:06.672 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.49:3301" will pass to kernel 2024-05-08T23:28:06.672 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:28:06.672 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:28:06.673 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:28:06.673 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790" 2024-05-08T23:28:06.673 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.49:6789,172.21.15.49:6790:/ 2024-05-08T23:28:06.673 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=4fd8ca56-5ac1-431d-b985-b50a13a8130a" will pass to kernel 2024-05-08T23:28:06.675 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:28:06.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.675 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:28:06.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:06.707 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:28:06.801 INFO:teuthology.orchestra.run.smithi005.stdout:client8161 2024-05-08T23:28:06.801 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:28:06.801 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:06.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T23:28:07.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.011+0000 7f5c223e4700 1 Processor -- start 2024-05-08T23:28:07.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.012+0000 7f5c223e4700 1 -- start start 2024-05-08T23:28:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.012+0000 7f5c223e4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b4d50 0x7f5c1c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.012+0000 7f5c223e4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c1c06aea0 0x7f5c1c1454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.012+0000 7f5c223e4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c145040 con 0x7f5c1c0b0d80 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.012+0000 7f5c223e4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c06b330 con 0x7f5c1c06aea0 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.012+0000 7f5c223e4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0b16e0 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c20981700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c1c06aea0 0x7f5c1c1454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c20981700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c1c06aea0 0x7f5c1c1454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59118/0 (socket says 172.21.15.5:59118) 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c20981700 1 -- 172.21.15.5:0/3988742576 learned_addr learned my addr 172.21.15.5:0/3988742576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c1bfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5c1c0b0d80 0x7f5c1c0b3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34490/0 (socket says 172.21.15.5:34490) 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c1b7fe700 1 --2- 172.21.15.5:0/3988742576 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b4d50 0x7f5c1c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c20981700 1 -- 172.21.15.5:0/3988742576 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b4d50 msgr2=0x7f5c1c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c20981700 1 --2- 172.21.15.5:0/3988742576 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b4d50 0x7f5c1c06a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c20981700 1 -- 172.21.15.5:0/3988742576 >> v1:172.21.15.5:6789/0 conn(0x7f5c1c0b0d80 legacy=0x7f5c1c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c20981700 1 -- 172.21.15.5:0/3988742576 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c0b19c0 con 0x7f5c1c06aea0 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c1affd700 1 -- 172.21.15.5:0/3988742576 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074131304 0 0) 0x7f5c1c145040 con 0x7f5c1c0b0d80 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c20981700 1 --2- 172.21.15.5:0/3988742576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c06aea0 0x7f5c1c1454d0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009e50 tx=0x7f5c1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b67a57f35bb17e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c1affd700 1 -- 172.21.15.5:0/3988742576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400e070 con 0x7f5c1c06aea0 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c1affd700 1 -- 172.21.15.5:0/3988742576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1400b9c0 con 0x7f5c1c06aea0 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.013+0000 7f5c1affd700 1 -- 172.21.15.5:0/3988742576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400bd70 con 0x7f5c1c06aea0 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.014+0000 7f5c223e4700 1 -- 172.21.15.5:0/3988742576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c06aea0 msgr2=0x7f5c1c1454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.014+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3988742576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c06aea0 0x7f5c1c1454d0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009e50 tx=0x7f5c1400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.014+0000 7f5c223e4700 1 -- 172.21.15.5:0/3988742576 shutdown_connections 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.014+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3988742576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c06aea0 0x7f5c1c1454d0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.014+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3988742576 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b4d50 0x7f5c1c06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.014+0000 7f5c223e4700 1 -- 172.21.15.5:0/3988742576 >> 172.21.15.5:0/3988742576 conn(0x7f5c1c099a50 msgr2=0x7f5c1c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.014+0000 7f5c223e4700 1 -- 172.21.15.5:0/3988742576 shutdown_connections 2024-05-08T23:28:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.014+0000 7f5c223e4700 1 -- 172.21.15.5:0/3988742576 wait complete. 2024-05-08T23:28:07.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.015+0000 7f5c223e4700 1 Processor -- start 2024-05-08T23:28:07.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.016+0000 7f5c223e4700 1 -- start start 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.016+0000 7f5c223e4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c06aea0 0x7f5c1c15d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.016+0000 7f5c223e4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b0d80 0x7f5c1c0af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.016+0000 7f5c223e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c0b4d50 0x7f5c1c0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.016+0000 7f5c223e4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1f2fb0 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.016+0000 7f5c223e4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0ae170 con 0x7f5c1c06aea0 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.016+0000 7f5c223e4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0ae450 con 0x7f5c1c0b0d80 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c0b4d50 0x7f5c1c0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c0b4d50 0x7f5c1c0afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59124/0 (socket says 172.21.15.5:59124) 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 -- 172.21.15.5:0/3091396412 learned_addr learned my addr 172.21.15.5:0/3091396412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c1bfff700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c06aea0 0x7f5c1c15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c1b7fe700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b0d80 0x7f5c1c0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 -- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b0d80 msgr2=0x7f5c1c0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b0d80 0x7f5c1c0af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 -- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c06aea0 msgr2=0x7f5c1c15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c06aea0 0x7f5c1c15d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 -- 172.21.15.5:0/3091396412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14009b30 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c20981700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c0b4d50 0x7f5c1c0afad0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0715e0 tx=0x7f5c14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c18ff9700 1 -- 172.21.15.5:0/3091396412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14003ea0 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c18ff9700 1 -- 172.21.15.5:0/3091396412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c14004920 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c0ae730 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c18ff9700 1 -- 172.21.15.5:0/3091396412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1401c680 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.017+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c15d550 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.018+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c074bc0 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.019+0000 7f5c18ff9700 1 -- 172.21.15.5:0/3091396412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5c1401c7e0 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.019+0000 7f5c18ff9700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bfc06acc0 0x7f5bfc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.020+0000 7f5c18ff9700 1 -- 172.21.15.5:0/3091396412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c1408ab50 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.020+0000 7f5c1bfff700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bfc06acc0 0x7f5bfc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:07.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.020+0000 7f5c1bfff700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bfc06acc0 0x7f5bfc06d180 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c007fd0 tx=0x7f5c0c007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:07.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.025+0000 7f5c18ff9700 1 -- 172.21.15.5:0/3091396412 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5c14055d30 con 0x7f5c1c0b4d50 2024-05-08T23:28:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:07.295+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f5c1c0b2130 con 0x7f5c1c0b4d50 2024-05-08T23:28:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.302+0000 7f5c18ff9700 1 -- 172.21.15.5:0/3091396412 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v100) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5c14059350 con 0x7f5c1c0b4d50 2024-05-08T23:28:08.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bfc06acc0 msgr2=0x7f5bfc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:08.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bfc06acc0 0x7f5bfc06d180 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c007fd0 tx=0x7f5c0c007de0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c0b4d50 msgr2=0x7f5c1c0afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c0b4d50 0x7f5c1c0afad0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0715e0 tx=0x7f5c14004ad0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 shutdown_connections 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bfc06acc0 0x7f5bfc06d180 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c0b4d50 0x7f5c1c0afad0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c06aea0 0x7f5c1c15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 --2- 172.21.15.5:0/3091396412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c1c0b0d80 0x7f5c1c0af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 >> 172.21.15.5:0/3091396412 conn(0x7f5c1c099a50 msgr2=0x7f5c1c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.305+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 shutdown_connections 2024-05-08T23:28:08.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.306+0000 7f5c223e4700 1 -- 172.21.15.5:0/3091396412 wait complete. 2024-05-08T23:28:08.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:08.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:28:08.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:28:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.464+0000 7f8c20162700 1 Processor -- start 2024-05-08T23:28:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.465+0000 7f8c20162700 1 -- start start 2024-05-08T23:28:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.465+0000 7f8c20162700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c1815ef60 0x7f8c180b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.465+0000 7f8c20162700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf000 0x7f8c180cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.465+0000 7f8c20162700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8c180cf9a0 0x7f8c18099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.465+0000 7f8c20162700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c180cd400 con 0x7f8c1815ef60 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.465+0000 7f8c20162700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c180cfe30 con 0x7f8c180cf9a0 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.465+0000 7f8c20162700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1809ad40 con 0x7f8c180cf000 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c1815ef60 0x7f8c180b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c1815ef60 0x7f8c180b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37498/0 (socket says 172.21.15.5:37498) 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 -- 172.21.15.5:0/1611115591 learned_addr learned my addr 172.21.15.5:0/1611115591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1e6ff700 1 --2- 172.21.15.5:0/1611115591 >> v2:172.21.15.49:3300/0 conn(0x7f8c180cf9a0 0x7f8c18099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1d6fd700 1 --2- 172.21.15.5:0/1611115591 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf000 0x7f8c180cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 -- 172.21.15.5:0/1611115591 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf000 msgr2=0x7f8c180cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 --2- 172.21.15.5:0/1611115591 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf000 0x7f8c180cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 -- 172.21.15.5:0/1611115591 >> v2:172.21.15.49:3300/0 conn(0x7f8c180cf9a0 msgr2=0x7f8c18099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 --2- 172.21.15.5:0/1611115591 >> v2:172.21.15.49:3300/0 conn(0x7f8c180cf9a0 0x7f8c18099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 -- 172.21.15.5:0/1611115591 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180ccb60 con 0x7f8c1815ef60 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1defe700 1 --2- 172.21.15.5:0/1611115591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c1815ef60 0x7f8c180b2340 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8c18072200 tx=0x7f8c14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6371e7dd88950cc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1cefc700 1 -- 172.21.15.5:0/1611115591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14011830 con 0x7f8c1815ef60 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1cefc700 1 -- 172.21.15.5:0/1611115591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c14011990 con 0x7f8c1815ef60 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.466+0000 7f8c1cefc700 1 -- 172.21.15.5:0/1611115591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14011d20 con 0x7f8c1815ef60 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 -- 172.21.15.5:0/1611115591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c1815ef60 msgr2=0x7f8c180b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 --2- 172.21.15.5:0/1611115591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c1815ef60 0x7f8c180b2340 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8c18072200 tx=0x7f8c14009bd0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 -- 172.21.15.5:0/1611115591 shutdown_connections 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 --2- 172.21.15.5:0/1611115591 >> v2:172.21.15.49:3300/0 conn(0x7f8c180cf9a0 0x7f8c18099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 --2- 172.21.15.5:0/1611115591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c1815ef60 0x7f8c180b2340 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 --2- 172.21.15.5:0/1611115591 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf000 0x7f8c180cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 -- 172.21.15.5:0/1611115591 >> 172.21.15.5:0/1611115591 conn(0x7f8c180b2f20 msgr2=0x7f8c180b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 -- 172.21.15.5:0/1611115591 shutdown_connections 2024-05-08T23:28:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.467+0000 7f8c20162700 1 -- 172.21.15.5:0/1611115591 wait complete. 2024-05-08T23:28:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.468+0000 7f8c20162700 1 Processor -- start 2024-05-08T23:28:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.468+0000 7f8c20162700 1 -- start start 2024-05-08T23:28:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c20162700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c180cf000 0x7f8c181527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c20162700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf9a0 0x7f8c18155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c20162700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c1815ef60 0x7f8c18155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c20162700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18154280 con 0x7f8c180cf000 2024-05-08T23:28:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c20162700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18157c50 con 0x7f8c1815ef60 2024-05-08T23:28:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c20162700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18157f30 con 0x7f8c180cf9a0 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c1defe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c180cf000 0x7f8c181527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c1defe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c180cf000 0x7f8c181527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37514/0 (socket says 172.21.15.5:37514) 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.469+0000 7f8c1defe700 1 -- 172.21.15.5:0/3548763684 learned_addr learned my addr 172.21.15.5:0/3548763684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1e6ff700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c1815ef60 0x7f8c18155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1d6fd700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf9a0 0x7f8c18155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1defe700 1 -- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf9a0 msgr2=0x7f8c18155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1defe700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf9a0 0x7f8c18155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1defe700 1 -- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c1815ef60 msgr2=0x7f8c18155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1defe700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c1815ef60 0x7f8c18155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1defe700 1 -- 172.21.15.5:0/3548763684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14009550 con 0x7f8c180cf000 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1e6ff700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c1815ef60 0x7f8c18155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c1defe700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c180cf000 0x7f8c181527b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009c10 tx=0x7f8c14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c0e7fc700 1 -- 172.21.15.5:0/3548763684 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14004510 con 0x7f8c180cf000 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c0e7fc700 1 -- 172.21.15.5:0/3548763684 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1401e070 con 0x7f8c180cf000 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c18158210 con 0x7f8c180cf000 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c0e7fc700 1 -- 172.21.15.5:0/3548763684 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140038e0 con 0x7f8c180cf000 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.470+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c18154560 con 0x7f8c180cf000 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.471+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c18158420 con 0x7f8c180cf000 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.472+0000 7f8c0e7fc700 1 -- 172.21.15.5:0/3548763684 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8c14003a40 con 0x7f8c180cf000 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.472+0000 7f8c0e7fc700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8bfc06ac90 0x7f8bfc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.472+0000 7f8c0e7fc700 1 -- 172.21.15.5:0/3548763684 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c1408a0e0 con 0x7f8c180cf000 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.473+0000 7f8c1d6fd700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8bfc06ac90 0x7f8bfc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.473+0000 7f8c1d6fd700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8bfc06ac90 0x7f8bfc06d150 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ab0 tx=0x7f8c08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.477+0000 7f8c0e7fc700 1 -- 172.21.15.5:0/3548763684 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8c14055350 con 0x7f8c180cf000 2024-05-08T23:28:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.744+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c18068000 con 0x7f8c180cf000 2024-05-08T23:28:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.744+0000 7f8c0e7fc700 1 -- 172.21.15.5:0/3548763684 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4746 (secure 0 0 0) 0x7f8c14058970 con 0x7f8c180cf000 2024-05-08T23:28:08.744 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:08.744 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":101,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7052,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:08.302198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8592,"mds_1":7058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:creating","state_seq":1,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8bfc06ac90 msgr2=0x7f8bfc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8bfc06ac90 0x7f8bfc06d150 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ab0 tx=0x7f8c08008040 comp rx=0 tx=0).stop 2024-05-08T23:28:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c180cf000 msgr2=0x7f8c181527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c180cf000 0x7f8c181527b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009c10 tx=0x7f8c14004750 comp rx=0 tx=0).stop 2024-05-08T23:28:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 shutdown_connections 2024-05-08T23:28:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8bfc06ac90 0x7f8bfc06d150 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c1815ef60 0x7f8c18155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c180cf000 0x7f8c181527b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 --2- 172.21.15.5:0/3548763684 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c180cf9a0 0x7f8c18155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 >> 172.21.15.5:0/3548763684 conn(0x7f8c180b2f20 msgr2=0x7f8c1815d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 shutdown_connections 2024-05-08T23:28:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:08.747+0000 7f8c20162700 1 -- 172.21.15.5:0/3548763684 wait complete. 2024-05-08T23:28:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 101 2024-05-08T23:28:09.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:27:59.271749+0000', 'modified': '2024-05-08T23:28:08.302198+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8592, 'mds_1': 7058}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7058': {'gid': 7058, 'name': 'a', 'rank': 1, 'incarnation': 101, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.5:6835/1828661194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 1828661194}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1828661194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_8592': {'gid': 8592, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6834/1510899872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 1510899872}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 1510899872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:28:09.163 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7058:up:creating 2024-05-08T23:28:09.802 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:28:09.802+0000 7fa0ab37b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:28:10.163 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:10.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.309+0000 7fa9d38eb700 1 Processor -- start 2024-05-08T23:28:10.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.309+0000 7fa9d38eb700 1 -- start start 2024-05-08T23:28:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d38eb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9cc06a960 0x7fa9cc06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d38eb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc0b3170 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d38eb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc0cc5e0 con 0x7fa9cc0b5ba0 2024-05-08T23:28:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d38eb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9cc06b350 con 0x7fa9cc06a960 2024-05-08T23:28:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d1e88700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa9cc15cfa0 0x7fa9cc15d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49504/0 (socket says 172.21.15.5:49504) 2024-05-08T23:28:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d1e88700 1 -- 172.21.15.5:0/3363796717 learned_addr learned my addr 172.21.15.5:0/3363796717 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d0e86700 1 --2- 172.21.15.5:0/3363796717 >> v2:172.21.15.49:3300/0 conn(0x7fa9cc06a960 0x7fa9cc06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d0e86700 1 -- 172.21.15.5:0/3363796717 >> v1:172.21.15.49:6789/0 conn(0x7fa9cc0b5ba0 legacy=0x7fa9cc0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d0e86700 1 -- 172.21.15.5:0/3363796717 >> v1:172.21.15.5:6789/0 conn(0x7fa9cc15cfa0 legacy=0x7fa9cc15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.311+0000 7fa9d0e86700 1 -- 172.21.15.5:0/3363796717 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0cc000 con 0x7fa9cc06a960 2024-05-08T23:28:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.312+0000 7fa9d0e86700 1 --2- 172.21.15.5:0/3363796717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 0x7fa9cc06ae10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009840 tx=0x7fa9bc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f63d293079d6f86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.313+0000 7fa9c3fff700 1 -- 172.21.15.5:0/3363796717 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc011830 con 0x7fa9cc06a960 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.313+0000 7fa9c3fff700 1 -- 172.21.15.5:0/3363796717 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc011990 con 0x7fa9cc06a960 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.313+0000 7fa9c3fff700 1 -- 172.21.15.5:0/3363796717 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc011d20 con 0x7fa9cc06a960 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.313+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3363796717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 msgr2=0x7fa9cc06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.313+0000 7fa9d38eb700 1 --2- 172.21.15.5:0/3363796717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 0x7fa9cc06ae10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009840 tx=0x7fa9bc009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.313+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3363796717 shutdown_connections 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.313+0000 7fa9d38eb700 1 --2- 172.21.15.5:0/3363796717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 0x7fa9cc06ae10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.313+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3363796717 >> 172.21.15.5:0/3363796717 conn(0x7fa9cc099a50 msgr2=0x7fa9cc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.314+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3363796717 shutdown_connections 2024-05-08T23:28:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.314+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3363796717 wait complete. 2024-05-08T23:28:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.315+0000 7fa9d38eb700 1 Processor -- start 2024-05-08T23:28:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.315+0000 7fa9d38eb700 1 -- start start 2024-05-08T23:28:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d38eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 0x7fa9cc152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d38eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa9cc0b5ba0 0x7fa9cc1e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d38eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc15cfa0 0x7fa9cc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d38eb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1530b0 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d38eb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc153e60 con 0x7fa9cc06a960 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d38eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1e8750 con 0x7fa9cc0b5ba0 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d1e88700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc15cfa0 0x7fa9cc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d1e88700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc15cfa0 0x7fa9cc158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37526/0 (socket says 172.21.15.5:37526) 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d1687700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 0x7fa9cc152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d1e88700 1 -- 172.21.15.5:0/3012584001 learned_addr learned my addr 172.21.15.5:0/3012584001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.316+0000 7fa9d0e86700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa9cc0b5ba0 0x7fa9cc1e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9d1e88700 1 -- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa9cc0b5ba0 msgr2=0x7fa9cc1e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9d1e88700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa9cc0b5ba0 0x7fa9cc1e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9d1e88700 1 -- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 msgr2=0x7fa9cc152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9d1e88700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 0x7fa9cc152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9d1e88700 1 -- 172.21.15.5:0/3012584001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0094a0 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9d1e88700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc15cfa0 0x7fa9cc158ff0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400ca80 tx=0x7fa9c400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9c1ffb700 1 -- 172.21.15.5:0/3012584001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400a9e0 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9c1ffb700 1 -- 172.21.15.5:0/3012584001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400ab40 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9c1ffb700 1 -- 172.21.15.5:0/3012584001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4012400 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.317+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc1e8a30 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.318+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc1537e0 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.318+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc0ce5a0 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.319+0000 7fa9c1ffb700 1 -- 172.21.15.5:0/3012584001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa9c401b3b0 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.319+0000 7fa9c1ffb700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa9b006ac90 0x7fa9b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.319+0000 7fa9c1ffb700 1 -- 172.21.15.5:0/3012584001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9c4088dc0 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.320+0000 7fa9d1687700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa9b006ac90 0x7fa9b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.320+0000 7fa9d1687700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa9b006ac90 0x7fa9b006d150 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8007fd0 tx=0x7fa9c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.324+0000 7fa9c1ffb700 1 -- 172.21.15.5:0/3012584001 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa9c4053fb0 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.589+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9cc067a80 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.589+0000 7fa9c1ffb700 1 -- 172.21.15.5:0/3012584001 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4759 (secure 0 0 0) 0x7fa9c4017090 con 0x7fa9cc15cfa0 2024-05-08T23:28:10.590 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:10.590 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:10.308722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa9b006ac90 msgr2=0x7fa9b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa9b006ac90 0x7fa9b006d150 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8007fd0 tx=0x7fa9c8009040 comp rx=0 tx=0).stop 2024-05-08T23:28:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc15cfa0 msgr2=0x7fa9cc158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc15cfa0 0x7fa9cc158ff0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400ca80 tx=0x7fa9c400cd90 comp rx=0 tx=0).stop 2024-05-08T23:28:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 shutdown_connections 2024-05-08T23:28:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa9b006ac90 0x7fa9b006d150 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:10.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc15cfa0 0x7fa9cc158ff0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:10.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9cc06a960 0x7fa9cc152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:10.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 --2- 172.21.15.5:0/3012584001 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa9cc0b5ba0 0x7fa9cc1e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:10.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 >> 172.21.15.5:0/3012584001 conn(0x7fa9cc099a50 msgr2=0x7fa9cc0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:10.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 shutdown_connections 2024-05-08T23:28:10.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:10.592+0000 7fa9d38eb700 1 -- 172.21.15.5:0/3012584001 wait complete. 2024-05-08T23:28:10.593 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 104 2024-05-08T23:28:11.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:27:59.271749+0000', 'modified': '2024-05-08T23:28:10.308722+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8592, 'mds_1': 7058, 'mds_2': 7052}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7052': {'gid': 7052, 'name': 'b', 'rank': 2, 'incarnation': 103, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.49:6837/873094942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6835', 'nonce': 873094942}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 873094942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7058': {'gid': 7058, 'name': 'a', 'rank': 1, 'incarnation': 101, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.5:6835/1828661194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 1828661194}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1828661194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_8592': {'gid': 8592, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6834/1510899872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 1510899872}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 1510899872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:28:11.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:28:11.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:28:11.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.157+0000 7f1d7f941700 1 Processor -- start 2024-05-08T23:28:11.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.157+0000 7f1d7f941700 1 -- start start 2024-05-08T23:28:11.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7f941700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d7815b900 0x7f1d780cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7f941700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1d78099a50 0x7f1d78099f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7f941700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7809a8d0 con 0x7f1d7809a440 2024-05-08T23:28:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7f941700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d780b1410 con 0x7f1d7815b900 2024-05-08T23:28:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7f941700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d780b0e30 con 0x7f1d78099a50 2024-05-08T23:28:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7d6dd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d7815b900 0x7f1d780cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7d6dd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d7815b900 0x7f1d780cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37538/0 (socket says 172.21.15.5:37538) 2024-05-08T23:28:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/3931423728 learned_addr learned my addr 172.21.15.5:0/3931423728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.158+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/3931423728 >> v2:172.21.15.49:3300/0 conn(0x7f1d78099a50 0x7f1d78099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.159+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/3931423728 >> v2:172.21.15.49:3300/0 conn(0x7f1d78099a50 msgr2=0x7f1d78099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.159+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/3931423728 >> v2:172.21.15.49:3300/0 conn(0x7f1d78099a50 0x7f1d78099f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.159+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/3931423728 >> v1:172.21.15.49:6789/0 conn(0x7f1d7809a440 legacy=0x7f1d780cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.159+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/3931423728 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d780cffd0 con 0x7f1d7815b900 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.159+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/3931423728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d7815b900 0x7f1d780cf9e0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f1d780d0a70 tx=0x7f1d74009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9888161cbaa9a363 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.159+0000 7f1d6ffff700 1 -- 172.21.15.5:0/3931423728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74011830 con 0x7f1d7815b900 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.159+0000 7f1d6ffff700 1 -- 172.21.15.5:0/3931423728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d74011990 con 0x7f1d7815b900 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.159+0000 7f1d6ffff700 1 -- 172.21.15.5:0/3931423728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74011d20 con 0x7f1d7815b900 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.160+0000 7f1d7f941700 1 -- 172.21.15.5:0/3931423728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d7815b900 msgr2=0x7f1d780cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.160+0000 7f1d7f941700 1 --2- 172.21.15.5:0/3931423728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d7815b900 0x7f1d780cf9e0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f1d780d0a70 tx=0x7f1d74009bd0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.160+0000 7f1d7f941700 1 -- 172.21.15.5:0/3931423728 shutdown_connections 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.160+0000 7f1d7f941700 1 --2- 172.21.15.5:0/3931423728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d7815b900 0x7f1d780cf9e0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.160+0000 7f1d7f941700 1 --2- 172.21.15.5:0/3931423728 >> v2:172.21.15.49:3300/0 conn(0x7f1d78099a50 0x7f1d78099f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.160+0000 7f1d7f941700 1 -- 172.21.15.5:0/3931423728 >> 172.21.15.5:0/3931423728 conn(0x7f1d780b2f20 msgr2=0x7f1d780b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.160+0000 7f1d7f941700 1 -- 172.21.15.5:0/3931423728 shutdown_connections 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.160+0000 7f1d7f941700 1 -- 172.21.15.5:0/3931423728 wait complete. 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.161+0000 7f1d7f941700 1 Processor -- start 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.161+0000 7f1d7f941700 1 -- start start 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7f941700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78099a50 0x7f1d780c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7f941700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d7809a440 0x7f1d78154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7f941700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1d7815b900 0x7f1d7806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7f941700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7806ba70 con 0x7f1d78099a50 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7f941700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780ca580 con 0x7f1d7809a440 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7f941700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780ca860 con 0x7f1d7815b900 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7d6dd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78099a50 0x7f1d780c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7d6dd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78099a50 0x7f1d780c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37546/0 (socket says 172.21.15.5:37546) 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/840528158 learned_addr learned my addr 172.21.15.5:0/840528158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d7809a440 0x7f1d78154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7dede700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1d7815b900 0x7f1d7806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1d7815b900 msgr2=0x7f1d7806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1d7815b900 0x7f1d7806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d7809a440 msgr2=0x7f1d78154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d7809a440 0x7f1d78154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.162+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/840528158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74009550 con 0x7f1d78099a50 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.163+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78099a50 0x7f1d780c8500 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009c10 tx=0x7f1d74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.163+0000 7f1d6dffb700 1 -- 172.21.15.5:0/840528158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74004510 con 0x7f1d78099a50 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.163+0000 7f1d6dffb700 1 -- 172.21.15.5:0/840528158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7401e070 con 0x7f1d78099a50 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.163+0000 7f1d7f941700 1 -- 172.21.15.5:0/840528158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d780cab40 con 0x7f1d78099a50 2024-05-08T23:28:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.163+0000 7f1d6dffb700 1 -- 172.21.15.5:0/840528158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d740038e0 con 0x7f1d78099a50 2024-05-08T23:28:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.163+0000 7f1d7f941700 1 -- 172.21.15.5:0/840528158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d780c8a40 con 0x7f1d78099a50 2024-05-08T23:28:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.165+0000 7f1d6dffb700 1 -- 172.21.15.5:0/840528158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1d74003a40 con 0x7f1d78099a50 2024-05-08T23:28:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.165+0000 7f1d6dffb700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d5c06ac90 0x7f1d5c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.165+0000 7f1d6dffb700 1 -- 172.21.15.5:0/840528158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d7408a090 con 0x7f1d78099a50 2024-05-08T23:28:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.166+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d5c06ac90 0x7f1d5c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.166+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d5c06ac90 0x7f1d5c06d150 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1d78070c70 tx=0x7f1d68009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.167+0000 7f1d8bafbb80 1 Processor -- start 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.167+0000 7f1d8bafbb80 1 -- start start 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e746750 0x56070e71f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e721320 0x56070e724050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e74e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070e67a100 con 0x56070e74dd80 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070e677ac0 con 0x56070e721320 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56070e6733b0 con 0x56070e746750 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7dede700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e74e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7dede700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e74e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37550/0 (socket says 172.21.15.5:37550) 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7dede700 1 -- 172.21.15.5:0/2500788736 learned_addr learned my addr 172.21.15.5:0/2500788736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e721320 0x56070e724050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e746750 0x56070e71f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7dede700 1 -- 172.21.15.5:0/2500788736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e746750 msgr2=0x56070e71f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e746750 0x56070e71f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7dede700 1 -- 172.21.15.5:0/2500788736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e721320 msgr2=0x56070e724050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e721320 0x56070e724050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.168+0000 7f1d7dede700 1 -- 172.21.15.5:0/2500788736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56070e4f4360 con 0x56070e74dd80 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e74e200 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f1d70002170 tx=0x7f1d70001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da01fb4f882fc397 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d62ffd700 1 -- 172.21.15.5:0/2500788736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000fc70 con 0x56070e74dd80 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d62ffd700 1 -- 172.21.15.5:0/2500788736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7000fdd0 con 0x56070e74dd80 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d62ffd700 1 -- 172.21.15.5:0/2500788736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70003c00 con 0x56070e74dd80 2024-05-08T23:28:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2500788736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 msgr2=0x56070e74e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e74e200 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f1d70002170 tx=0x7f1d70001a70 comp rx=0 tx=0).stop 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2500788736 shutdown_connections 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e74e200 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e746750 0x56070e71f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2500788736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e721320 0x56070e724050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2500788736 >> 172.21.15.5:0/2500788736 conn(0x56070e6811c0 msgr2=0x56070e768d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.169+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2500788736 shutdown_connections 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.170+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2500788736 wait complete. 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.170+0000 7f1d8bafbb80 1 -- start start 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.170+0000 7f1d8bafbb80 1 Processor -- start 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 0x56070e750750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e769060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070e60f510 con 0x56070e74dd80 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070e60f1b0 con 0x56070e746750 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56070e4f6650 con 0x56070e721320 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d8bafbb80 10 client.0 _ll_register_callbacks cb 0x56070e618ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7dede700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7dede700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7695a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37558/0 (socket says 172.21.15.5:37558) 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7dede700 1 -- 172.21.15.5:0/2305018661 learned_addr learned my addr 172.21.15.5:0/2305018661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e769060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 0x56070e750750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7dede700 1 -- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 msgr2=0x56070e750750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 0x56070e750750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7dede700 1 -- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 msgr2=0x56070e769060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e769060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7dede700 1 -- 172.21.15.5:0/2305018661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7809a8d0 con 0x56070e74dd80 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.172+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e769060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7695a0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f1d70001f80 tx=0x7f1d70001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93496ab224d45013 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d57fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d57fff700 1 -- 172.21.15.5:0/2305018661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70005560 con 0x56070e74dd80 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d57fff700 1 -- 172.21.15.5:0/2305018661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d700056c0 con 0x56070e74dd80 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2305018661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56070e5cc220 con 0x56070e74dd80 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d57fff700 1 -- 172.21.15.5:0/2305018661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d70005880 con 0x56070e74dd80 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 10 client.8769 fetch_fsmap learned FSMap version 104 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2305018661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x56070e4a0b10 con 0x56070e74dd80 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d57fff700 1 -- 172.21.15.5:0/2305018661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56070e5cc220 con 0x56070e74dd80 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d57fff700 1 -- 172.21.15.5:0/2305018661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f1d70012070 con 0x56070e74dd80 2024-05-08T23:28:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 10 client.8769 fetch_fsmap finished waiting for FSMap version 104 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 10 client.8769 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 20 client.8769 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56070e77bfb0 0x56070e74efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 4 client.8769 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.173+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2305018661 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56070e6f2790 con 0x56070e77bfb0 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.174+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56070e77bfb0 0x56070e74efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.174+0000 7f1d57fff700 10 client.8769 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.174+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56070e77bfb0 0x56070e74efb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77ba74fc05c487e6 server_cookie=8a4433b1de2d60d3 in_seq=0 out_seq=0 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d57fff700 1 -- 172.21.15.5:0/2305018661 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56070e6f2790 con 0x56070e77bfb0 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d57fff700 10 client.8769 handle_command_reply: tid=0 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 client.8769 shutdown 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2305018661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 msgr2=0x56070e7695a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7695a0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f1d70001f80 tx=0x7f1d70001250 comp rx=0 tx=0).stop 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2305018661 shutdown_connections 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56070e77bfb0 0x56070e74efb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7695a0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 0x56070e750750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d57fff700 0 client.8769 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2305018661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e769060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2305018661 >> 172.21.15.5:0/2305018661 conn(0x56070e6811c0 msgr2=0x56070e71d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2305018661 shutdown_connections 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2305018661 wait complete. 2024-05-08T23:28:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.175+0000 7f1d8bafbb80 20 client.8769 trim_cache size 0 max 16384 2024-05-08T23:28:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.194+0000 7f1d8bafbb80 1 Processor -- start 2024-05-08T23:28:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.194+0000 7f1d8bafbb80 1 -- start start 2024-05-08T23:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.195+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e721320 0x56070e723eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.195+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e777420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.195+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e74dd80 0x56070e777960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.195+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070e71e730 con 0x56070e721320 2024-05-08T23:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.195+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070e71ddf0 con 0x56070e746750 2024-05-08T23:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.195+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56070e724b40 con 0x56070e74dd80 2024-05-08T23:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e721320 0x56070e723eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e721320 0x56070e723eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37560/0 (socket says 172.21.15.5:37560) 2024-05-08T23:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/3219632959 learned_addr learned my addr 172.21.15.5:0/3219632959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e777420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7dede700 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e74dd80 0x56070e777960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/3219632959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e74dd80 msgr2=0x56070e777960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e74dd80 0x56070e777960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/3219632959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 msgr2=0x56070e777420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e777420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 -- 172.21.15.5:0/3219632959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7806ca10 con 0x56070e721320 2024-05-08T23:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e721320 0x56070e723eb0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f1d74058f50 tx=0x7f1d74088520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e32a6cf8602c3e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d637fe700 1 -- 172.21.15.5:0/3219632959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7405cdf0 con 0x56070e721320 2024-05-08T23:28:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d637fe700 1 -- 172.21.15.5:0/3219632959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d740613b0 con 0x56070e721320 2024-05-08T23:28:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.196+0000 7f1d637fe700 1 -- 172.21.15.5:0/3219632959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74010e60 con 0x56070e721320 2024-05-08T23:28:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/3219632959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e721320 msgr2=0x56070e723eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e721320 0x56070e723eb0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f1d74058f50 tx=0x7f1d74088520 comp rx=0 tx=0).stop 2024-05-08T23:28:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/3219632959 shutdown_connections 2024-05-08T23:28:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e74dd80 0x56070e777960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e721320 0x56070e723eb0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/3219632959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e777420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/3219632959 >> 172.21.15.5:0/3219632959 conn(0x56070e6811c0 msgr2=0x56070e74a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/3219632959 shutdown_connections 2024-05-08T23:28:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/3219632959 wait complete. 2024-05-08T23:28:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.197+0000 7f1d8bafbb80 1 -- start start 2024-05-08T23:28:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.198+0000 7f1d8bafbb80 1 Processor -- start 2024-05-08T23:28:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.198+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 0x56070e71f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.198+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e768eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.198+0000 7f1d8bafbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070e7d40c0 con 0x56070e74dd80 2024-05-08T23:28:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070e77a410 con 0x56070e746750 2024-05-08T23:28:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d8bafbb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56070e77a660 con 0x56070e721320 2024-05-08T23:28:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d8bafbb80 10 client.0 _ll_register_callbacks cb 0x7f1d68009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37572/0 (socket says 172.21.15.5:37572) 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 -- 172.21.15.5:0/2830871431 learned_addr learned my addr 172.21.15.5:0/2830871431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 0x56070e71f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e768eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 -- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 msgr2=0x56070e71f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 0x56070e71f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 -- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 msgr2=0x56070e768eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e768eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 -- 172.21.15.5:0/2830871431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d780c8a40 con 0x56070e74dd80 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7cedc700 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e768eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d7dede700 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7693f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x56070e74a920 tx=0x7f1d70013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f0a21bb8854675 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d57fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d57fff700 1 -- 172.21.15.5:0/2830871431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7001f620 con 0x56070e74dd80 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d57fff700 1 -- 172.21.15.5:0/2830871431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d70001a70 con 0x56070e74dd80 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.199+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2830871431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d7000fdd0 con 0x56070e74dd80 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d57fff700 1 -- 172.21.15.5:0/2830871431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d70001790 con 0x56070e74dd80 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d8bafbb80 10 client.8775 fetch_fsmap learned FSMap version 104 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2830871431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x56070e639510 con 0x56070e74dd80 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d57fff700 1 -- 172.21.15.5:0/2830871431 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7000fdd0 con 0x56070e74dd80 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d57fff700 1 -- 172.21.15.5:0/2830871431 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f1d70012050 con 0x56070e74dd80 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d8bafbb80 10 client.8775 fetch_fsmap finished waiting for FSMap version 104 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d8bafbb80 10 client.8775 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:28:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d8bafbb80 20 client.8775 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56070e76b650 0x56070e76ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d8bafbb80 4 client.8775 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2830871431 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56070e66be70 con 0x56070e76b650 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.200+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56070e76b650 0x56070e76ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d57fff700 10 client.8775 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d7d6dd700 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56070e76b650 0x56070e76ba30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5b6b4bbc54fde0d server_cookie=35c5d5d8c79e962f in_seq=0 out_seq=0 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d57fff700 1 -- 172.21.15.5:0/2830871431 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56070e66be70 con 0x56070e76b650 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d57fff700 10 client.8775 handle_command_reply: tid=0 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 client.8775 shutdown 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2830871431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 msgr2=0x56070e7693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7693f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x56070e74a920 tx=0x7f1d70013000 comp rx=0 tx=0).stop 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2830871431 shutdown_connections 2024-05-08T23:28:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56070e76b650 0x56070e76ba30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56070e74dd80 0x56070e7693f0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56070e721320 0x56070e71f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d57fff700 0 client.8775 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 --2- 172.21.15.5:0/2830871431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56070e746750 0x56070e768eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2830871431 >> 172.21.15.5:0/2830871431 conn(0x56070e6811c0 msgr2=0x56070e750f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2830871431 shutdown_connections 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 1 -- 172.21.15.5:0/2830871431 wait complete. 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.201+0000 7f1d8bafbb80 20 client.8775 trim_cache size 0 max 16384 2024-05-08T23:28:11.214 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7052, 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 1.8933655309999999, 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 104, 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-05-08T23:28:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-05-08T23:28:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 17.169729840999999 2024-05-08T23:28:11.216 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 -- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d5c06ac90 msgr2=0x7f1d5c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d5c06ac90 0x7f1d5c06d150 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1d78070c70 tx=0x7f1d68009210 comp rx=0 tx=0).stop 2024-05-08T23:28:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 -- 172.21.15.5:0/840528158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78099a50 msgr2=0x7f1d780c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78099a50 0x7f1d780c8500 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009c10 tx=0x7f1d74004750 comp rx=0 tx=0).stop 2024-05-08T23:28:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 -- 172.21.15.5:0/840528158 shutdown_connections 2024-05-08T23:28:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f1d5c06ac90 0x7f1d5c06d150 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1d7815b900 0x7f1d7806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78099a50 0x7f1d780c8500 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 --2- 172.21.15.5:0/840528158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d7809a440 0x7f1d78154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 -- 172.21.15.5:0/840528158 >> 172.21.15.5:0/840528158 conn(0x7f1d780b2f20 msgr2=0x7f1d780b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 -- 172.21.15.5:0/840528158 shutdown_connections 2024-05-08T23:28:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.202+0000 7f1d7f941700 1 -- 172.21.15.5:0/840528158 wait complete. 2024-05-08T23:28:11.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T23:28:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.357+0000 7f143ed96700 1 Processor -- start 2024-05-08T23:28:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.357+0000 7f143ed96700 1 -- start start 2024-05-08T23:28:11.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f143ed96700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14380d3f40 0x7f1438161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f143ed96700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1438162450 con 0x7f143815d7a0 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f143ed96700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380cee00 con 0x7f143815db80 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f143ed96700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14380b27e0 con 0x7f14380d3f40 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f1437fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14380d3f40 0x7f1438161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f143cb32700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f143815d7a0 0x7f14380b0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49520/0 (socket says 172.21.15.5:49520) 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f143cb32700 1 -- 172.21.15.5:0/872593042 learned_addr learned my addr 172.21.15.5:0/872593042 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f1437fff700 1 -- 172.21.15.5:0/872593042 >> v1:172.21.15.49:6789/0 conn(0x7f143815db80 legacy=0x7f14380cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f1437fff700 1 -- 172.21.15.5:0/872593042 >> v1:172.21.15.5:6789/0 conn(0x7f143815d7a0 legacy=0x7f14380b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f1437fff700 1 -- 172.21.15.5:0/872593042 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380d4400 con 0x7f14380d3f40 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f1437fff700 1 --2- 172.21.15.5:0/872593042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14380d3f40 0x7f1438161f10 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f1424009820 tx=0x7f1424009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e3445441a952f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.358+0000 7f14377fe700 1 -- 172.21.15.5:0/872593042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424005e30 con 0x7f14380d3f40 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f14377fe700 1 -- 172.21.15.5:0/872593042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424004030 con 0x7f14380d3f40 2024-05-08T23:28:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f14377fe700 1 -- 172.21.15.5:0/872593042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240043e0 con 0x7f14380d3f40 2024-05-08T23:28:11.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f143ed96700 1 -- 172.21.15.5:0/872593042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14380d3f40 msgr2=0x7f1438161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f143ed96700 1 --2- 172.21.15.5:0/872593042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14380d3f40 0x7f1438161f10 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f1424009820 tx=0x7f1424009b30 comp rx=0 tx=0).stop 2024-05-08T23:28:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f143ed96700 1 -- 172.21.15.5:0/872593042 shutdown_connections 2024-05-08T23:28:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f143ed96700 1 --2- 172.21.15.5:0/872593042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14380d3f40 0x7f1438161f10 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f143ed96700 1 -- 172.21.15.5:0/872593042 >> 172.21.15.5:0/872593042 conn(0x7f143808afe0 msgr2=0x7f14380d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f143ed96700 1 -- 172.21.15.5:0/872593042 shutdown_connections 2024-05-08T23:28:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.359+0000 7f143ed96700 1 -- 172.21.15.5:0/872593042 wait complete. 2024-05-08T23:28:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.361+0000 7f143ed96700 1 Processor -- start 2024-05-08T23:28:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.361+0000 7f143ed96700 1 -- start start 2024-05-08T23:28:11.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143ed96700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14380d3f40 0x7f14381f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143ed96700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f143815d7a0 0x7f143815f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143ed96700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f143815db80 0x7f143815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143ed96700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381f4250 con 0x7f143815db80 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143ed96700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380ca790 con 0x7f143815d7a0 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143ed96700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14380caa70 con 0x7f14380d3f40 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143d333700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f143815db80 0x7f143815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143d333700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f143815db80 0x7f143815fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37580/0 (socket says 172.21.15.5:37580) 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.362+0000 7f143d333700 1 -- 172.21.15.5:0/2291536106 learned_addr learned my addr 172.21.15.5:0/2291536106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f1437fff700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f143815d7a0 0x7f143815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143cb32700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14380d3f40 0x7f14381f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143d333700 1 -- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14380d3f40 msgr2=0x7f14381f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143d333700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14380d3f40 0x7f14381f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143d333700 1 -- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f143815d7a0 msgr2=0x7f143815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143d333700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f143815d7a0 0x7f143815f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143d333700 1 -- 172.21.15.5:0/2291536106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1424009430 con 0x7f143815db80 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f1437fff700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f143815d7a0 0x7f143815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143d333700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f143815db80 0x7f143815fd90 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f143000cb50 tx=0x7f143000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f14357fa700 1 -- 172.21.15.5:0/2291536106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430003c00 con 0x7f143815db80 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f14357fa700 1 -- 172.21.15.5:0/2291536106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430003d60 con 0x7f143815db80 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143ed96700 1 -- 172.21.15.5:0/2291536106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14380cad50 con 0x7f143815db80 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f14357fa700 1 -- 172.21.15.5:0/2291536106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000ab70 con 0x7f143815db80 2024-05-08T23:28:11.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.363+0000 7f143ed96700 1 -- 172.21.15.5:0/2291536106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14381f49f0 con 0x7f143815db80 2024-05-08T23:28:11.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.365+0000 7f14357fa700 1 -- 172.21.15.5:0/2291536106 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f143000acd0 con 0x7f143815db80 2024-05-08T23:28:11.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.365+0000 7f14357fa700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f141c06ac90 0x7f141c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.366+0000 7f14357fa700 1 -- 172.21.15.5:0/2291536106 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1430089430 con 0x7f143815db80 2024-05-08T23:28:11.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.366+0000 7f143cb32700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f141c06ac90 0x7f141c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.366+0000 7f143cb32700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f141c06ac90 0x7f141c06d150 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f142c000f60 tx=0x7f142c007d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.368+0000 7f144af50b80 1 Processor -- start 2024-05-08T23:28:11.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.368+0000 7f144af50b80 1 -- start start 2024-05-08T23:28:11.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.368+0000 7f144af50b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd611d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.368+0000 7f144af50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6122050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.368+0000 7f144af50b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 0x562fd614c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.368+0000 7f144af50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd6078100 con 0x562fd611f320 2024-05-08T23:28:11.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.368+0000 7f144af50b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd6075ac0 con 0x562fd614bd80 2024-05-08T23:28:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.368+0000 7f144af50b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562fd60713b0 con 0x562fd6144750 2024-05-08T23:28:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6122050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6122050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37592/0 (socket says 172.21.15.5:37592) 2024-05-08T23:28:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 -- 172.21.15.5:0/2824433103 learned_addr learned my addr 172.21.15.5:0/2824433103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f143cb32700 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd611d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f143d333700 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 0x562fd614c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 -- 172.21.15.5:0/2824433103 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 msgr2=0x562fd611d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd611d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 -- 172.21.15.5:0/2824433103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 msgr2=0x562fd614c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 0x562fd614c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 -- 172.21.15.5:0/2824433103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fd5ef2360 con 0x562fd611f320 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f1437fff700 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6122050 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f1424004000 tx=0x7f1424010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c0474d6e1305ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f14227fc700 1 -- 172.21.15.5:0/2824433103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240036a0 con 0x562fd611f320 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f14227fc700 1 -- 172.21.15.5:0/2824433103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424003800 con 0x562fd611f320 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.369+0000 7f14227fc700 1 -- 172.21.15.5:0/2824433103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424003b70 con 0x562fd611f320 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 -- 172.21.15.5:0/2824433103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 msgr2=0x562fd6122050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6122050 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f1424004000 tx=0x7f1424010000 comp rx=0 tx=0).stop 2024-05-08T23:28:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 -- 172.21.15.5:0/2824433103 shutdown_connections 2024-05-08T23:28:11.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 0x562fd614c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd611d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 --2- 172.21.15.5:0/2824433103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6122050 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 -- 172.21.15.5:0/2824433103 >> 172.21.15.5:0/2824433103 conn(0x562fd607f1c0 msgr2=0x562fd6166d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 -- 172.21.15.5:0/2824433103 shutdown_connections 2024-05-08T23:28:11.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.370+0000 7f144af50b80 1 -- 172.21.15.5:0/2824433103 wait complete. 2024-05-08T23:28:11.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.371+0000 7f144af50b80 1 -- start start 2024-05-08T23:28:11.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.371+0000 7f144af50b80 1 Processor -- start 2024-05-08T23:28:11.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f144af50b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd611f320 0x562fd61673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f144af50b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd6144750 0x562fd6176600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f144af50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd614bd80 0x562fd6169e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f144af50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd600d510 con 0x562fd614bd80 2024-05-08T23:28:11.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f144af50b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd600d1b0 con 0x562fd6144750 2024-05-08T23:28:11.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f144af50b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562fd5ef4650 con 0x562fd611f320 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f144af50b80 10 client.0 _ll_register_callbacks cb 0x562fd6016ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143d333700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd614bd80 0x562fd6169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143d333700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd614bd80 0x562fd6169e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37604/0 (socket says 172.21.15.5:37604) 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143d333700 1 -- 172.21.15.5:0/546704781 learned_addr learned my addr 172.21.15.5:0/546704781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f1437fff700 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd6144750 0x562fd6176600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143cb32700 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd611f320 0x562fd61673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143d333700 1 -- 172.21.15.5:0/546704781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd611f320 msgr2=0x562fd61673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143d333700 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd611f320 0x562fd61673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143d333700 1 -- 172.21.15.5:0/546704781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd6144750 msgr2=0x562fd6176600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143d333700 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd6144750 0x562fd6176600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.372+0000 7f143d333700 1 -- 172.21.15.5:0/546704781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14380d4400 con 0x562fd614bd80 2024-05-08T23:28:11.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f143d333700 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd614bd80 0x562fd6169e20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f14300878c0 tx=0x7f1430058230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f3a109dbd70c25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f14177fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f14177fe700 1 -- 172.21.15.5:0/546704781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430060620 con 0x562fd614bd80 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f14177fe700 1 -- 172.21.15.5:0/546704781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14300890b0 con 0x562fd614bd80 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 1 -- 172.21.15.5:0/546704781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fd5fca220 con 0x562fd614bd80 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f14177fe700 1 -- 172.21.15.5:0/546704781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1430089270 con 0x562fd614bd80 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 10 client.8790 fetch_fsmap learned FSMap version 104 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 1 -- 172.21.15.5:0/546704781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x562fd5e9eb10 con 0x562fd614bd80 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f14177fe700 1 -- 172.21.15.5:0/546704781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562fd5fca220 con 0x562fd614bd80 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f14177fe700 1 -- 172.21.15.5:0/546704781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f1430058490 con 0x562fd614bd80 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 10 client.8790 fetch_fsmap finished waiting for FSMap version 104 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 10 client.8790 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 20 client.8790 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x562fd61686f0 0x562fd6168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 4 client.8790 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f144af50b80 1 -- 172.21.15.5:0/546704781 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fd60f0790 con 0x562fd61686f0 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.373+0000 7f143cb32700 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x562fd61686f0 0x562fd6168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.374+0000 7f14177fe700 10 client.8790 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.374+0000 7f143cb32700 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x562fd61686f0 0x562fd6168ad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=204b20a1c76ab79b server_cookie=3048e54bc268e0cc in_seq=0 out_seq=0 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.374+0000 7f14177fe700 1 -- 172.21.15.5:0/546704781 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562fd60f0790 con 0x562fd61686f0 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.374+0000 7f14177fe700 10 client.8790 handle_command_reply: tid=0 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.374+0000 7f144af50b80 1 client.8790 shutdown 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 -- 172.21.15.5:0/546704781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd614bd80 msgr2=0x562fd6169e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd614bd80 0x562fd6169e20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f14300878c0 tx=0x7f1430058230 comp rx=0 tx=0).stop 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 -- 172.21.15.5:0/546704781 shutdown_connections 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x562fd61686f0 0x562fd6168ad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd614bd80 0x562fd6169e20 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd611f320 0x562fd61673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f14177fe700 0 client.8790 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:11.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 --2- 172.21.15.5:0/546704781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd6144750 0x562fd6176600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 -- 172.21.15.5:0/546704781 >> 172.21.15.5:0/546704781 conn(0x562fd607f1c0 msgr2=0x562fd6148ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 -- 172.21.15.5:0/546704781 shutdown_connections 2024-05-08T23:28:11.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 1 -- 172.21.15.5:0/546704781 wait complete. 2024-05-08T23:28:11.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.375+0000 7f144af50b80 20 client.8790 trim_cache size 0 max 16384 2024-05-08T23:28:11.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.394+0000 7f144af50b80 1 Processor -- start 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.394+0000 7f144af50b80 1 -- start start 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f144af50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6178f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f144af50b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd61794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f144af50b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 0x562fd611aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f144af50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd6122290 con 0x562fd611f320 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f144af50b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd614eaa0 con 0x562fd614bd80 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f144af50b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562fd614cd00 con 0x562fd6144750 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f143cb32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6178f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f143cb32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6178f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37612/0 (socket says 172.21.15.5:37612) 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f143cb32700 1 -- 172.21.15.5:0/1892816861 learned_addr learned my addr 172.21.15.5:0/1892816861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f143d333700 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 0x562fd611aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.395+0000 7f1437fff700 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd61794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f143cb32700 1 -- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 msgr2=0x562fd61794c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f143cb32700 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd61794c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f143cb32700 1 -- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 msgr2=0x562fd611aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f143cb32700 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 0x562fd611aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f143cb32700 1 -- 172.21.15.5:0/1892816861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fd6037510 con 0x562fd611f320 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f1437fff700 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd61794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f143cb32700 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6178f80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f142c004b90 tx=0x7f142c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455d09cd28ea0a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f1422ffd700 1 -- 172.21.15.5:0/1892816861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c001d00 con 0x562fd611f320 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f1422ffd700 1 -- 172.21.15.5:0/1892816861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c004f50 con 0x562fd611f320 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f1422ffd700 1 -- 172.21.15.5:0/1892816861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0050c0 con 0x562fd611f320 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f144af50b80 1 -- 172.21.15.5:0/1892816861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 msgr2=0x562fd6178f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.396+0000 7f144af50b80 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6178f80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f142c004b90 tx=0x7f142c00c000 comp rx=0 tx=0).stop 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 -- 172.21.15.5:0/1892816861 shutdown_connections 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd614bd80 0x562fd611aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd611f320 0x562fd6178f80 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 --2- 172.21.15.5:0/1892816861 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd6144750 0x562fd61794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 -- 172.21.15.5:0/1892816861 >> 172.21.15.5:0/1892816861 conn(0x562fd607f1c0 msgr2=0x562fd611c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 -- 172.21.15.5:0/1892816861 shutdown_connections 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 -- 172.21.15.5:0/1892816861 wait complete. 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 -- start start 2024-05-08T23:28:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.397+0000 7f144af50b80 1 Processor -- start 2024-05-08T23:28:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f144af50b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd611f320 0x562fd61cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f144af50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd6144750 0x562fd6147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f144af50b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd614bd80 0x562fd6166510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f144af50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd6118510 con 0x562fd6144750 2024-05-08T23:28:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f144af50b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd6147d20 con 0x562fd611f320 2024-05-08T23:28:11.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f144af50b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x562fd614c3f0 con 0x562fd614bd80 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f144af50b80 10 client.0 _ll_register_callbacks cb 0x562fd6016ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f1437fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd6144750 0x562fd6147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f1437fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd6144750 0x562fd6147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37618/0 (socket says 172.21.15.5:37618) 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f1437fff700 1 -- 172.21.15.5:0/3437876045 learned_addr learned my addr 172.21.15.5:0/3437876045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.398+0000 7f143cb32700 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd611f320 0x562fd61cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f143d333700 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd614bd80 0x562fd6166510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f1437fff700 1 -- 172.21.15.5:0/3437876045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd614bd80 msgr2=0x562fd6166510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f1437fff700 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd614bd80 0x562fd6166510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f1437fff700 1 -- 172.21.15.5:0/3437876045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd611f320 msgr2=0x562fd61cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f1437fff700 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd611f320 0x562fd61cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f1437fff700 1 -- 172.21.15.5:0/3437876045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c00f040 con 0x562fd6144750 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f1437fff700 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd6144750 0x562fd6147230 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f143808ba90 tx=0x7f14240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5740587527adf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f14177fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f14177fe700 1 -- 172.21.15.5:0/3437876045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424003b80 con 0x562fd6144750 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f14177fe700 1 -- 172.21.15.5:0/3437876045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1424003ce0 con 0x562fd6144750 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f144af50b80 1 -- 172.21.15.5:0/3437876045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1424003800 con 0x562fd6144750 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f14177fe700 1 -- 172.21.15.5:0/3437876045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1424003ea0 con 0x562fd6144750 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f144af50b80 10 client.8796 fetch_fsmap learned FSMap version 104 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.399+0000 7f144af50b80 1 -- 172.21.15.5:0/3437876045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x562fd5ef6c80 con 0x562fd6144750 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f14177fe700 1 -- 172.21.15.5:0/3437876045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424003800 con 0x562fd6144750 2024-05-08T23:28:11.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f14177fe700 1 -- 172.21.15.5:0/3437876045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f1424024020 con 0x562fd6144750 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f144af50b80 10 client.8796 fetch_fsmap finished waiting for FSMap version 104 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f144af50b80 10 client.8796 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f144af50b80 20 client.8796 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f144af50b80 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x562fd614edf0 0x562fd614f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f144af50b80 4 client.8796 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f144af50b80 1 -- 172.21.15.5:0/3437876045 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562fd6069e70 con 0x562fd614edf0 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f143cb32700 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x562fd614edf0 0x562fd614f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f14177fe700 10 client.8796 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f143cb32700 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x562fd614edf0 0x562fd614f1d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6e9a2d02ddd2eb5 server_cookie=b778b26d4c34658a in_seq=0 out_seq=0 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f14177fe700 1 -- 172.21.15.5:0/3437876045 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562fd6069e70 con 0x562fd614edf0 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f14177fe700 10 client.8796 handle_command_reply: tid=0 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.400+0000 7f144af50b80 1 client.8796 shutdown 2024-05-08T23:28:11.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 -- 172.21.15.5:0/3437876045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd6144750 msgr2=0x562fd6147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd6144750 0x562fd6147230 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f143808ba90 tx=0x7f14240047c0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 -- 172.21.15.5:0/3437876045 shutdown_connections 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x562fd614edf0 0x562fd614f1d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x562fd614bd80 0x562fd6166510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562fd611f320 0x562fd61cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f14177fe700 0 client.8796 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 --2- 172.21.15.5:0/3437876045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fd6144750 0x562fd6147230 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 -- 172.21.15.5:0/3437876045 >> 172.21.15.5:0/3437876045 conn(0x562fd607f1c0 msgr2=0x562fd61671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 -- 172.21.15.5:0/3437876045 shutdown_connections 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 1 -- 172.21.15.5:0/3437876045 wait complete. 2024-05-08T23:28:11.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f144af50b80 20 client.8796 trim_cache size 0 max 16384 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7058, 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 3.097727951, 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 104, 2024-05-08T23:28:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-05-08T23:28:11.410 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-05-08T23:28:11.410 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 17.366621384999998 2024-05-08T23:28:11.410 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 -- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f141c06ac90 msgr2=0x7f141c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f141c06ac90 0x7f141c06d150 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f142c000f60 tx=0x7f142c007d90 comp rx=0 tx=0).stop 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 -- 172.21.15.5:0/2291536106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f143815db80 msgr2=0x7f143815fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f143815db80 0x7f143815fd90 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f143000cb50 tx=0x7f143000cf10 comp rx=0 tx=0).stop 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 -- 172.21.15.5:0/2291536106 shutdown_connections 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f141c06ac90 0x7f141c06d150 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f143815db80 0x7f143815fd90 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14380d3f40 0x7f14381f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 --2- 172.21.15.5:0/2291536106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f143815d7a0 0x7f143815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.401+0000 7f143ed96700 1 -- 172.21.15.5:0/2291536106 >> 172.21.15.5:0/2291536106 conn(0x7f143808afe0 msgr2=0x7f143806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.402+0000 7f143ed96700 1 -- 172.21.15.5:0/2291536106 shutdown_connections 2024-05-08T23:28:11.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.402+0000 7f143ed96700 1 -- 172.21.15.5:0/2291536106 wait complete. 2024-05-08T23:28:11.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:28:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.552+0000 7f27e8a58700 1 Processor -- start 2024-05-08T23:28:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.552+0000 7f27e8a58700 1 -- start start 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e8a58700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f27e415b900 0x7f27e40cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e8a58700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f27e4099a50 0x7f27e4099f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e8a58700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e409a440 0x7f27e40cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e8a58700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e409a8d0 con 0x7f27e4099a50 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e8a58700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e40b1410 con 0x7f27e415b900 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e8a58700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40b0e30 con 0x7f27e409a440 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e1d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f27e4099a50 0x7f27e4099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e1d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f27e4099a50 0x7f27e4099f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37622/0 (socket says 172.21.15.5:37622) 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e1d9b700 1 -- 172.21.15.5:0/1869299342 learned_addr learned my addr 172.21.15.5:0/1869299342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e259c700 1 --2- 172.21.15.5:0/1869299342 >> v2:172.21.15.49:3300/0 conn(0x7f27e415b900 0x7f27e40cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/1869299342 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e409a440 0x7f27e40cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e1d9b700 1 -- 172.21.15.5:0/1869299342 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e409a440 msgr2=0x7f27e40cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/1869299342 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e409a440 0x7f27e40cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e1d9b700 1 -- 172.21.15.5:0/1869299342 >> v2:172.21.15.49:3300/0 conn(0x7f27e415b900 msgr2=0x7f27e40cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/1869299342 >> v2:172.21.15.49:3300/0 conn(0x7f27e415b900 0x7f27e40cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.553+0000 7f27e1d9b700 1 -- 172.21.15.5:0/1869299342 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40cffd0 con 0x7f27e4099a50 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/1869299342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e4099a50 0x7f27e4099f00 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f27e4072130 tx=0x7f27d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f80f3e8784c233 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e159a700 1 -- 172.21.15.5:0/1869299342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8011830 con 0x7f27e4099a50 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e159a700 1 -- 172.21.15.5:0/1869299342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8011990 con 0x7f27e4099a50 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e159a700 1 -- 172.21.15.5:0/1869299342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8011d20 con 0x7f27e4099a50 2024-05-08T23:28:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e8a58700 1 -- 172.21.15.5:0/1869299342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e4099a50 msgr2=0x7f27e4099f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e8a58700 1 --2- 172.21.15.5:0/1869299342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e4099a50 0x7f27e4099f00 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f27e4072130 tx=0x7f27d8009bd0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e8a58700 1 -- 172.21.15.5:0/1869299342 shutdown_connections 2024-05-08T23:28:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e8a58700 1 --2- 172.21.15.5:0/1869299342 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e409a440 0x7f27e40cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e8a58700 1 --2- 172.21.15.5:0/1869299342 >> v2:172.21.15.49:3300/0 conn(0x7f27e415b900 0x7f27e40cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e8a58700 1 --2- 172.21.15.5:0/1869299342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e4099a50 0x7f27e4099f00 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.554+0000 7f27e8a58700 1 -- 172.21.15.5:0/1869299342 >> 172.21.15.5:0/1869299342 conn(0x7f27e40b2f20 msgr2=0x7f27e40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.555+0000 7f27e8a58700 1 -- 172.21.15.5:0/1869299342 shutdown_connections 2024-05-08T23:28:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.555+0000 7f27e8a58700 1 -- 172.21.15.5:0/1869299342 wait complete. 2024-05-08T23:28:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.556+0000 7f27e8a58700 1 Processor -- start 2024-05-08T23:28:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.556+0000 7f27e8a58700 1 -- start start 2024-05-08T23:28:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e8a58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e4099a50 0x7f27e40c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e8a58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e409a440 0x7f27e406c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e8a58700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e415b900 0x7f27e4155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e8a58700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e406c0a0 con 0x7f27e409a440 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e8a58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e406ca00 con 0x7f27e4099a50 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e8a58700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e4155b90 con 0x7f27e415b900 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e409a440 0x7f27e406c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e409a440 0x7f27e406c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37624/0 (socket says 172.21.15.5:37624) 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 -- 172.21.15.5:0/3893059295 learned_addr learned my addr 172.21.15.5:0/3893059295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e259c700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e4099a50 0x7f27e40c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e415b900 0x7f27e4155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 -- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e415b900 msgr2=0x7f27e4155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e415b900 0x7f27e4155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 -- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e4099a50 msgr2=0x7f27e40c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e4099a50 0x7f27e40c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 -- 172.21.15.5:0/3893059295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8009550 con 0x7f27e409a440 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.557+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e409a440 0x7f27e406c4c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f27d8014040 tx=0x7f27d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.558+0000 7f27d2ffd700 1 -- 172.21.15.5:0/3893059295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8004510 con 0x7f27e409a440 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.558+0000 7f27d2ffd700 1 -- 172.21.15.5:0/3893059295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d801e070 con 0x7f27e409a440 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.558+0000 7f27e8a58700 1 -- 172.21.15.5:0/3893059295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e406ba70 con 0x7f27e409a440 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.558+0000 7f27d2ffd700 1 -- 172.21.15.5:0/3893059295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80038e0 con 0x7f27e409a440 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.558+0000 7f27e8a58700 1 -- 172.21.15.5:0/3893059295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e406d8c0 con 0x7f27e409a440 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.559+0000 7f27d2ffd700 1 -- 172.21.15.5:0/3893059295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f27d8003a40 con 0x7f27e409a440 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.560+0000 7f27d2ffd700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f27c406ac90 0x7f27c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.560+0000 7f27d2ffd700 1 -- 172.21.15.5:0/3893059295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27d808a180 con 0x7f27e409a440 2024-05-08T23:28:11.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.560+0000 7f27e259c700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f27c406ac90 0x7f27c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.560+0000 7f27e259c700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f27c406ac90 0x7f27c406d150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009ad0 tx=0x7f27d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.562+0000 7f27f4c12b80 1 Processor -- start 2024-05-08T23:28:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.562+0000 7f27f4c12b80 1 -- start start 2024-05-08T23:28:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989c8320 0x5599989cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989f4d80 0x5599989f5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559998921100 con 0x5599989c8320 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999891eac0 con 0x5599989f4d80 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55999891a3b0 con 0x5599989ed750 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989c8320 0x5599989cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989c8320 0x5599989cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37628/0 (socket says 172.21.15.5:37628) 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 -- 172.21.15.5:0/466627803 learned_addr learned my addr 172.21.15.5:0/466627803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989f4d80 0x5599989f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e259c700 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 -- 172.21.15.5:0/466627803 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 msgr2=0x5599989c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 -- 172.21.15.5:0/466627803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989f4d80 msgr2=0x5599989f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989f4d80 0x5599989f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 -- 172.21.15.5:0/466627803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999879b360 con 0x5599989c8320 2024-05-08T23:28:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.563+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989c8320 0x5599989cb050 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f27d8006e30 tx=0x7f27d80092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4025e551f12e96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27c3fff700 1 -- 172.21.15.5:0/466627803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d805ce40 con 0x5599989c8320 2024-05-08T23:28:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27c3fff700 1 -- 172.21.15.5:0/466627803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8061400 con 0x5599989c8320 2024-05-08T23:28:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27c3fff700 1 -- 172.21.15.5:0/466627803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80573b0 con 0x5599989c8320 2024-05-08T23:28:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27f4c12b80 1 -- 172.21.15.5:0/466627803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989c8320 msgr2=0x5599989cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989c8320 0x5599989cb050 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f27d8006e30 tx=0x7f27d80092c0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27f4c12b80 1 -- 172.21.15.5:0/466627803 shutdown_connections 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989f4d80 0x5599989f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/466627803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989c8320 0x5599989cb050 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.564+0000 7f27f4c12b80 1 -- 172.21.15.5:0/466627803 >> 172.21.15.5:0/466627803 conn(0x5599989281c0 msgr2=0x559998a0fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.565+0000 7f27f4c12b80 1 -- 172.21.15.5:0/466627803 shutdown_connections 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.565+0000 7f27f4c12b80 1 -- 172.21.15.5:0/466627803 wait complete. 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.565+0000 7f27f4c12b80 1 -- start start 2024-05-08T23:28:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.565+0000 7f27f4c12b80 1 Processor -- start 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.566+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x5599989f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.566+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.566+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.566+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599988b6510 con 0x5599989f4d80 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.566+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599988b61b0 con 0x5599989c8320 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.566+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55999879d650 con 0x5599989ed750 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.566+0000 7f27f4c12b80 10 client.0 _ll_register_callbacks cb 0x5599988bfae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.566+0000 7f27e2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a23220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37640/0 (socket says 172.21.15.5:37640) 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e2d9d700 1 -- 172.21.15.5:0/1305898353 learned_addr learned my addr 172.21.15.5:0/1305898353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e259c700 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x5599989f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e2d9d700 1 -- 172.21.15.5:0/1305898353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 msgr2=0x5599989ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e2d9d700 1 -- 172.21.15.5:0/1305898353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 msgr2=0x5599989f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x5599989f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e2d9d700 1 -- 172.21.15.5:0/1305898353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e406d8c0 con 0x5599989f4d80 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a23220 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f27dc004320 tx=0x7f27dc004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41193f70bcc43164 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27c17fa700 1 -- 172.21.15.5:0/1305898353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0077a0 con 0x5599989f4d80 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27c17fa700 1 -- 172.21.15.5:0/1305898353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27dc007900 con 0x5599989f4d80 2024-05-08T23:28:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27f4c12b80 1 -- 172.21.15.5:0/1305898353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559998873220 con 0x5599989f4d80 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27c17fa700 1 -- 172.21.15.5:0/1305898353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27dc007ac0 con 0x5599989f4d80 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27f4c12b80 10 client.8808 fetch_fsmap learned FSMap version 104 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.567+0000 7f27f4c12b80 1 -- 172.21.15.5:0/1305898353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x559998747b10 con 0x5599989f4d80 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27c17fa700 1 -- 172.21.15.5:0/1305898353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559998873220 con 0x5599989f4d80 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27c17fa700 1 -- 172.21.15.5:0/1305898353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f27dc007c20 con 0x5599989f4d80 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27f4c12b80 10 client.8808 fetch_fsmap finished waiting for FSMap version 104 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27f4c12b80 10 client.8808 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27f4c12b80 20 client.8808 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x559998a12720 0x559998a12b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27f4c12b80 4 client.8808 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27f4c12b80 1 -- 172.21.15.5:0/1305898353 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559998999790 con 0x559998a12720 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27e259c700 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x559998a12720 0x559998a12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27c17fa700 10 client.8808 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.568+0000 7f27e259c700 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x559998a12720 0x559998a12b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e0ade6dff1e11b server_cookie=7cd31d871d24a7f4 in_seq=0 out_seq=0 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27c17fa700 1 -- 172.21.15.5:0/1305898353 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559998999790 con 0x559998a12720 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27c17fa700 10 client.8808 handle_command_reply: tid=0 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 client.8808 shutdown 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 -- 172.21.15.5:0/1305898353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 msgr2=0x559998a23220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a23220 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f27dc004320 tx=0x7f27dc004630 comp rx=0 tx=0).stop 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 -- 172.21.15.5:0/1305898353 shutdown_connections 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x559998a12720 0x559998a12b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a23220 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x5599989f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27c17fa700 0 client.8808 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/1305898353 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 -- 172.21.15.5:0/1305898353 >> 172.21.15.5:0/1305898353 conn(0x5599989281c0 msgr2=0x5599989f1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 -- 172.21.15.5:0/1305898353 shutdown_connections 2024-05-08T23:28:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 1 -- 172.21.15.5:0/1305898353 wait complete. 2024-05-08T23:28:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.569+0000 7f27f4c12b80 20 client.8808 trim_cache size 0 max 16384 2024-05-08T23:28:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.589+0000 7f27f4c12b80 1 Processor -- start 2024-05-08T23:28:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.589+0000 7f27f4c12b80 1 -- start start 2024-05-08T23:28:11.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.589+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x5599989f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.589+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.589+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.589+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599989cb6c0 con 0x5599989f4d80 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.589+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599989c7590 con 0x5599989c8320 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.589+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599989eac30 con 0x5599989ed750 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a31ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37648/0 (socket says 172.21.15.5:37648) 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 -- 172.21.15.5:0/2612360214 learned_addr learned my addr 172.21.15.5:0/2612360214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e259c700 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x5599989f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 -- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 msgr2=0x5599989ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 -- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 msgr2=0x5599989f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x5599989f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 -- 172.21.15.5:0/2612360214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599988e0510 con 0x5599989f4d80 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a31ee0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007080 tx=0x7f27dc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee5ecf796bbd639 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27d0ff9700 1 -- 172.21.15.5:0/2612360214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc009410 con 0x5599989f4d80 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27d0ff9700 1 -- 172.21.15.5:0/2612360214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27dc00f040 con 0x5599989f4d80 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.590+0000 7f27d0ff9700 1 -- 172.21.15.5:0/2612360214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0079e0 con 0x5599989f4d80 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 -- 172.21.15.5:0/2612360214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 msgr2=0x559998a31ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a31ee0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007080 tx=0x7f27dc013000 comp rx=0 tx=0).stop 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 -- 172.21.15.5:0/2612360214 shutdown_connections 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989f4d80 0x559998a31ee0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x5599989f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/2612360214 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989ed750 0x5599989ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 -- 172.21.15.5:0/2612360214 >> 172.21.15.5:0/2612360214 conn(0x5599989281c0 msgr2=0x5599989c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 -- 172.21.15.5:0/2612360214 shutdown_connections 2024-05-08T23:28:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 -- 172.21.15.5:0/2612360214 wait complete. 2024-05-08T23:28:11.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.591+0000 7f27f4c12b80 1 -- start start 2024-05-08T23:28:11.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.592+0000 7f27f4c12b80 1 Processor -- start 2024-05-08T23:28:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.592+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x559998a17530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.592+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989ed750 0x5599989ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.592+0000 7f27f4c12b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989f4d80 0x5599989ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.592+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599989e7350 con 0x5599989ed750 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.592+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599989f5660 con 0x5599989c8320 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27f4c12b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x559998a16d60 con 0x5599989f4d80 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27f4c12b80 10 client.0 _ll_register_callbacks cb 0x5599988bfae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989ed750 0x5599989ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989ed750 0x5599989ec9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37654/0 (socket says 172.21.15.5:37654) 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 -- 172.21.15.5:0/3458482565 learned_addr learned my addr 172.21.15.5:0/3458482565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e2d9d700 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989f4d80 0x5599989ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e259c700 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x559998a17530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 -- 172.21.15.5:0/3458482565 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989f4d80 msgr2=0x5599989ecf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989f4d80 0x5599989ecf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 -- 172.21.15.5:0/3458482565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 msgr2=0x559998a17530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x559998a17530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 -- 172.21.15.5:0/3458482565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559998747b10 con 0x5599989ed750 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27e1d9b700 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989ed750 0x5599989ec9c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f27d8003fd0 tx=0x7f27d808a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f319c8a878bf4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27c17fa700 1 -- 172.21.15.5:0/3458482565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d805ce40 con 0x5599989ed750 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27c17fa700 1 -- 172.21.15.5:0/3458482565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27d80573b0 con 0x5599989ed750 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27f4c12b80 1 -- 172.21.15.5:0/3458482565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27d8061400 con 0x5599989ed750 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27c17fa700 1 -- 172.21.15.5:0/3458482565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27d8055720 con 0x5599989ed750 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27f4c12b80 10 client.8814 fetch_fsmap learned FSMap version 104 2024-05-08T23:28:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.593+0000 7f27f4c12b80 1 -- 172.21.15.5:0/3458482565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x55999879fc80 con 0x5599989ed750 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27c17fa700 1 -- 172.21.15.5:0/3458482565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8061400 con 0x5599989ed750 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27c17fa700 1 -- 172.21.15.5:0/3458482565 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) v1 ==== 3138+0+0 (secure 0 0 0) 0x7f27d8059270 con 0x5599989ed750 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27f4c12b80 10 client.8814 fetch_fsmap finished waiting for FSMap version 104 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27f4c12b80 10 client.8814 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27f4c12b80 20 client.8814 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x559998a232c0 0x559998a05710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27f4c12b80 4 client.8814 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27f4c12b80 1 -- 172.21.15.5:0/3458482565 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559998912e70 con 0x559998a232c0 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27e259c700 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x559998a232c0 0x559998a05710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27c17fa700 10 client.8814 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27e259c700 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x559998a232c0 0x559998a05710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6137ccb727e47ddd server_cookie=c214c9583d0d9301 in_seq=0 out_seq=0 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27c17fa700 1 -- 172.21.15.5:0/3458482565 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559998912e70 con 0x559998a232c0 2024-05-08T23:28:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27c17fa700 10 client.8814 handle_command_reply: tid=0 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.594+0000 7f27f4c12b80 1 client.8814 shutdown 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 -- 172.21.15.5:0/3458482565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989ed750 msgr2=0x5599989ec9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989ed750 0x5599989ec9c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f27d8003fd0 tx=0x7f27d808a000 comp rx=0 tx=0).stop 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 -- 172.21.15.5:0/3458482565 shutdown_connections 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x559998a232c0 0x559998a05710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5599989f4d80 0x5599989ecf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5599989c8320 0x559998a17530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 --2- 172.21.15.5:0/3458482565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599989ed750 0x5599989ec9c0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27c17fa700 0 client.8814 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 -- 172.21.15.5:0/3458482565 >> 172.21.15.5:0/3458482565 conn(0x5599989281c0 msgr2=0x5599989ec0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 -- 172.21.15.5:0/3458482565 shutdown_connections 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 1 -- 172.21.15.5:0/3458482565 wait complete. 2024-05-08T23:28:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27f4c12b80 20 client.8814 trim_cache size 0 max 16384 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4fd8ca56-5ac1-431d-b985-b50a13a8130a", 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 8592, 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 12.321954444999999, 2024-05-08T23:28:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 104, 2024-05-08T23:28:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-05-08T23:28:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-05-08T23:28:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 17.561445560999999 2024-05-08T23:28:11.603 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 -- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f27c406ac90 msgr2=0x7f27c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f27c406ac90 0x7f27c406d150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009ad0 tx=0x7f27d4008040 comp rx=0 tx=0).stop 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 -- 172.21.15.5:0/3893059295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e409a440 msgr2=0x7f27e406c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e409a440 0x7f27e406c4c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f27d8014040 tx=0x7f27d8004750 comp rx=0 tx=0).stop 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 -- 172.21.15.5:0/3893059295 shutdown_connections 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f27c406ac90 0x7f27c406d150 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f27e415b900 0x7f27e4155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e4099a50 0x7f27e40c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 --2- 172.21.15.5:0/3893059295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27e409a440 0x7f27e406c4c0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.595+0000 7f27e8a58700 1 -- 172.21.15.5:0/3893059295 >> 172.21.15.5:0/3893059295 conn(0x7f27e40b2f20 msgr2=0x7f27e40b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.596+0000 7f27e8a58700 1 -- 172.21.15.5:0/3893059295 shutdown_connections 2024-05-08T23:28:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.596+0000 7f27e8a58700 1 -- 172.21.15.5:0/3893059295 wait complete. 2024-05-08T23:28:11.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.745+0000 7fe6a3ddb700 1 Processor -- start 2024-05-08T23:28:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.745+0000 7fe6a3ddb700 1 -- start start 2024-05-08T23:28:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a3ddb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe69c0cd4e0 0x7fe69c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a3ddb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c15c280 con 0x7fe69c0cecf0 2024-05-08T23:28:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a3ddb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c06e780 con 0x7fe69c06f130 2024-05-08T23:28:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a3ddb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69c06ea60 con 0x7fe69c0cd4e0 2024-05-08T23:28:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a2378700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe69c0cecf0 0x7fe69c15c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49526/0 (socket says 172.21.15.5:49526) 2024-05-08T23:28:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a2378700 1 -- 172.21.15.5:0/1392598095 learned_addr learned my addr 172.21.15.5:0/1392598095 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a1b77700 1 --2- 172.21.15.5:0/1392598095 >> v2:172.21.15.49:3300/0 conn(0x7fe69c0cd4e0 0x7fe69c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a0b75700 1 -- 172.21.15.5:0/1392598095 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616987263 0 0) 0x7fe69c15c280 con 0x7fe69c0cecf0 2024-05-08T23:28:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.746+0000 7fe6a0b75700 1 -- 172.21.15.5:0/1392598095 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6880035b0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.747+0000 7fe6a1b77700 1 -- 172.21.15.5:0/1392598095 >> v1:172.21.15.49:6789/0 conn(0x7fe69c06f130 legacy=0x7fe69c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.747+0000 7fe6a0b75700 1 -- 172.21.15.5:0/1392598095 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859307469 0 0) 0x7fe6880035b0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.747+0000 7fe6a1b77700 1 -- 172.21.15.5:0/1392598095 >> v1:172.21.15.5:6789/0 conn(0x7fe69c0cecf0 legacy=0x7fe69c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.747+0000 7fe6a1b77700 1 -- 172.21.15.5:0/1392598095 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c06ed40 con 0x7fe69c0cd4e0 2024-05-08T23:28:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.747+0000 7fe6a1b77700 1 --2- 172.21.15.5:0/1392598095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 0x7fe69c06f910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe698009840 tx=0x7fe698009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38421f2c19492b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.747+0000 7fe6a0b75700 1 -- 172.21.15.5:0/1392598095 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6940019f0 con 0x7fe69c0cd4e0 2024-05-08T23:28:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.747+0000 7fe6a0b75700 1 -- 172.21.15.5:0/1392598095 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe698004030 con 0x7fe69c0cd4e0 2024-05-08T23:28:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.747+0000 7fe6a0b75700 1 -- 172.21.15.5:0/1392598095 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6980043c0 con 0x7fe69c0cd4e0 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.748+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/1392598095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 msgr2=0x7fe69c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.748+0000 7fe6a3ddb700 1 --2- 172.21.15.5:0/1392598095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 0x7fe69c06f910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe698009840 tx=0x7fe698009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.748+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/1392598095 shutdown_connections 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.748+0000 7fe6a3ddb700 1 --2- 172.21.15.5:0/1392598095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 0x7fe69c06f910 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.748+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/1392598095 >> 172.21.15.5:0/1392598095 conn(0x7fe69c0d0320 msgr2=0x7fe69c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.748+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/1392598095 shutdown_connections 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.748+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/1392598095 wait complete. 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.749+0000 7fe6a3ddb700 1 Processor -- start 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.749+0000 7fe6a3ddb700 1 -- start start 2024-05-08T23:28:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a3ddb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69c06f130 0x7fe69c06af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a3ddb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 0x7fe69c0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a3ddb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe69c0cecf0 0x7fe69c0b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a3ddb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0b4ec0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a3ddb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0b50f0 con 0x7fe69c0cd4e0 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a3ddb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0b53d0 con 0x7fe69c06f130 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a2378700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe69c0cecf0 0x7fe69c0b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a2378700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe69c0cecf0 0x7fe69c0b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37656/0 (socket says 172.21.15.5:37656) 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a2378700 1 -- 172.21.15.5:0/3660624831 learned_addr learned my addr 172.21.15.5:0/3660624831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a1b77700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69c06f130 0x7fe69c06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a1376700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 0x7fe69c0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a2378700 1 -- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69c06f130 msgr2=0x7fe69c06af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a2378700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69c06f130 0x7fe69c06af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a2378700 1 -- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 msgr2=0x7fe69c0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a2378700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 0x7fe69c0b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a2378700 1 -- 172.21.15.5:0/3660624831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980094a0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.750+0000 7fe6a1b77700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69c06f130 0x7fe69c06af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.751+0000 7fe6a2378700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe69c0cecf0 0x7fe69c0b1d20 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fe69400cc00 tx=0x7fe69400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.751+0000 7fe68e7fc700 1 -- 172.21.15.5:0/3660624831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69400ab20 con 0x7fe69c0cecf0 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.751+0000 7fe68e7fc700 1 -- 172.21.15.5:0/3660624831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69400ac80 con 0x7fe69c0cecf0 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.751+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c0b56b0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.751+0000 7fe68e7fc700 1 -- 172.21.15.5:0/3660624831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6940070d0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.751+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c0c4bd0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.752+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69c0c4df0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.752+0000 7fe68e7fc700 1 -- 172.21.15.5:0/3660624831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe694007230 con 0x7fe69c0cecf0 2024-05-08T23:28:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.753+0000 7fe68e7fc700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe68006ac90 0x7fe68006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.753+0000 7fe68e7fc700 1 -- 172.21.15.5:0/3660624831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6940891a0 con 0x7fe69c0cecf0 2024-05-08T23:28:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.753+0000 7fe6a1b77700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe68006ac90 0x7fe68006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.754+0000 7fe6a1b77700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe68006ac90 0x7fe68006d150 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe698009b90 tx=0x7fe698008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:11.758+0000 7fe68e7fc700 1 -- 172.21.15.5:0/3660624831 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe694054380 con 0x7fe69c0cecf0 2024-05-08T23:28:12.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.022+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe69c068000 con 0x7fe69c0cecf0 2024-05-08T23:28:12.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.023+0000 7fe68e7fc700 1 -- 172.21.15.5:0/3660624831 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4759 (secure 0 0 0) 0x7fe6940579a0 con 0x7fe69c0cecf0 2024-05-08T23:28:12.023 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:12.023 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:10.308722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe68006ac90 msgr2=0x7fe68006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe68006ac90 0x7fe68006d150 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe698009b90 tx=0x7fe698008030 comp rx=0 tx=0).stop 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe69c0cecf0 msgr2=0x7fe69c0b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe69c0cecf0 0x7fe69c0b1d20 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fe69400cc00 tx=0x7fe69400a060 comp rx=0 tx=0).stop 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 shutdown_connections 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe68006ac90 0x7fe68006d150 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe69c0cecf0 0x7fe69c0b1d20 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe69c06f130 0x7fe69c06af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 --2- 172.21.15.5:0/3660624831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0cd4e0 0x7fe69c0b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 >> 172.21.15.5:0/3660624831 conn(0x7fe69c0d0320 msgr2=0x7fe69c0d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 shutdown_connections 2024-05-08T23:28:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:12.025+0000 7fe6a3ddb700 1 -- 172.21.15.5:0/3660624831 wait complete. 2024-05-08T23:28:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 104 2024-05-08T23:28:12.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:12.039 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:28:12.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:12.070 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-05-08T23:28:12.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:12.136 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-05-08T23:28:12.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:12.198 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-05-08T23:28:12.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:12.263 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-05-08T23:28:12.327 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:28:12.532 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:28:12.533 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:28:12.533 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:28:14.533 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:28:14.700 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:28:14.700 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:28:14.700 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:28:14.802 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:28:14.802+0000 7fa0ab37b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T23:28:16.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:28:16.865 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:28:16.865 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:28:16.865 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:28:18.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:28:19.032 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-08T23:28:19.033 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:28:19.033 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:28:21.034 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-08T23:28:21.197 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2024-05-08T23:28:21.198 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:28:21.198 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-05-08T23:28:21.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:28:21.355 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-05-08T23:28:21.356 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:28:21.356 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:28:23.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:28:23.520 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-05-08T23:28:23.520 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:28:23.521 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:28:25.522 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-08T23:28:25.684 INFO:teuthology.orchestra.run.smithi005.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-08T23:28:25.685 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-08T23:28:25.685 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:28:25.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-08T23:28:25.842 INFO:teuthology.orchestra.run.smithi049.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-08T23:28:25.843 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-08T23:28:25.843 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-05-08T23:28:25.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 flush journal 2024-05-08T23:28:25.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.989+0000 7feb9e43b700 1 Processor -- start 2024-05-08T23:28:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.989+0000 7feb9e43b700 1 -- start start 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.990+0000 7feb9e43b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb98145040 0x7feb980b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.990+0000 7feb9e43b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7feb980d0a60 0x7feb980afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.990+0000 7feb9e43b700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb980af6e0 con 0x7feb9815fbb0 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.990+0000 7feb9e43b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb980b6460 con 0x7feb98145040 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.990+0000 7feb9e43b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb980aeb50 con 0x7feb980d0a60 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.990+0000 7feb97fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb98145040 0x7feb980b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.990+0000 7feb977fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7feb9815fbb0 0x7feb980d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33630/0 (socket says 172.21.15.5:33630) 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.990+0000 7feb977fe700 1 -- 172.21.15.5:0/3097356721 learned_addr learned my addr 172.21.15.5:0/3097356721 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/3097356721 >> v2:172.21.15.49:3300/0 conn(0x7feb980d0a60 0x7feb980afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb97fff700 1 -- 172.21.15.5:0/3097356721 >> v2:172.21.15.49:3300/0 conn(0x7feb980d0a60 msgr2=0x7feb980afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb97fff700 1 --2- 172.21.15.5:0/3097356721 >> v2:172.21.15.49:3300/0 conn(0x7feb980d0a60 0x7feb980afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb97fff700 1 -- 172.21.15.5:0/3097356721 >> v1:172.21.15.5:6789/0 conn(0x7feb9815fbb0 legacy=0x7feb980d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb97fff700 1 -- 172.21.15.5:0/3097356721 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb980aedd0 con 0x7feb98145040 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb96ffd700 1 -- 172.21.15.5:0/3097356721 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809096984 0 0) 0x7feb980af6e0 con 0x7feb9815fbb0 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/3097356721 >> v2:172.21.15.49:3300/0 conn(0x7feb980d0a60 0x7feb980afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb97fff700 1 --2- 172.21.15.5:0/3097356721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb98145040 0x7feb980b5f20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7feb9809a0a0 tx=0x7feb88009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462f6f218f9592c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb96ffd700 1 -- 172.21.15.5:0/3097356721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88011830 con 0x7feb98145040 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb96ffd700 1 -- 172.21.15.5:0/3097356721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb88011990 con 0x7feb98145040 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.991+0000 7feb96ffd700 1 -- 172.21.15.5:0/3097356721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88011d20 con 0x7feb98145040 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.992+0000 7feb9e43b700 1 -- 172.21.15.5:0/3097356721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb98145040 msgr2=0x7feb980b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.992+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3097356721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb98145040 0x7feb980b5f20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7feb9809a0a0 tx=0x7feb88009b20 comp rx=0 tx=0).stop 2024-05-08T23:28:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.992+0000 7feb9e43b700 1 -- 172.21.15.5:0/3097356721 shutdown_connections 2024-05-08T23:28:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.992+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3097356721 >> v2:172.21.15.49:3300/0 conn(0x7feb980d0a60 0x7feb980afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.992+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3097356721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb98145040 0x7feb980b5f20 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.992+0000 7feb9e43b700 1 -- 172.21.15.5:0/3097356721 >> 172.21.15.5:0/3097356721 conn(0x7feb98099a50 msgr2=0x7feb980b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.992+0000 7feb9e43b700 1 -- 172.21.15.5:0/3097356721 shutdown_connections 2024-05-08T23:28:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.992+0000 7feb9e43b700 1 -- 172.21.15.5:0/3097356721 wait complete. 2024-05-08T23:28:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.993+0000 7feb9e43b700 1 Processor -- start 2024-05-08T23:28:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.993+0000 7feb9e43b700 1 -- start start 2024-05-08T23:28:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9e43b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb980d0a60 0x7feb9813b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9e43b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb98145040 0x7feb98139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9e43b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb9815fbb0 0x7feb980cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9e43b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9813cca0 con 0x7feb9815fbb0 2024-05-08T23:28:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9e43b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9813c020 con 0x7feb980d0a60 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9e43b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9813c300 con 0x7feb98145040 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9c9d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb9815fbb0 0x7feb980cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9c9d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb9815fbb0 0x7feb980cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54352/0 (socket says 172.21.15.5:54352) 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.994+0000 7feb9c9d8700 1 -- 172.21.15.5:0/3319327100 learned_addr learned my addr 172.21.15.5:0/3319327100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb977fe700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb98145040 0x7feb98139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb97fff700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb980d0a60 0x7feb9813b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb9c9d8700 1 -- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb98145040 msgr2=0x7feb98139e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb98145040 0x7feb98139e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb9c9d8700 1 -- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb980d0a60 msgr2=0x7feb9813b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb980d0a60 0x7feb9813b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb9c9d8700 1 -- 172.21.15.5:0/3319327100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb880094a0 con 0x7feb9815fbb0 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb9815fbb0 0x7feb980cec60 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7feb90007ef0 tx=0x7feb9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb94ff9700 1 -- 172.21.15.5:0/3319327100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90003a20 con 0x7feb9815fbb0 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb94ff9700 1 -- 172.21.15.5:0/3319327100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9000f040 con 0x7feb9815fbb0 2024-05-08T23:28:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb9e43b700 1 -- 172.21.15.5:0/3319327100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9813c5e0 con 0x7feb9815fbb0 2024-05-08T23:28:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb94ff9700 1 -- 172.21.15.5:0/3319327100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90003dc0 con 0x7feb9815fbb0 2024-05-08T23:28:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.995+0000 7feb9e43b700 1 -- 172.21.15.5:0/3319327100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9813d0c0 con 0x7feb9815fbb0 2024-05-08T23:28:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.997+0000 7feb94ff9700 1 -- 172.21.15.5:0/3319327100 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feb90004030 con 0x7feb9815fbb0 2024-05-08T23:28:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.997+0000 7feb94ff9700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb8406ac90 0x7feb8406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.997+0000 7feb94ff9700 1 -- 172.21.15.5:0/3319327100 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb90089500 con 0x7feb9815fbb0 2024-05-08T23:28:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.997+0000 7feb97fff700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb8406ac90 0x7feb8406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.998+0000 7feb97fff700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb8406ac90 0x7feb8406d150 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7feb880090b0 tx=0x7feb88008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.999+0000 7febaa5f5b80 1 Processor -- start 2024-05-08T23:28:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:25.999+0000 7febaa5f5b80 1 -- start start 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 0x564f48e17110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 0x564f48e1c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e45dd0 0x564f48e46250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f48d72150 con 0x564f48e45dd0 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f48d6fb10 con 0x564f48e3e7a0 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f48d6b400 con 0x564f48e19370 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb9c9d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e45dd0 0x564f48e46250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb9c9d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e45dd0 0x564f48e46250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54358/0 (socket says 172.21.15.5:54358) 2024-05-08T23:28:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb9c9d8700 1 -- 172.21.15.5:0/690430593 learned_addr learned my addr 172.21.15.5:0/690430593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb97fff700 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 0x564f48e17110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb977fe700 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 0x564f48e1c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb9c9d8700 1 -- 172.21.15.5:0/690430593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 msgr2=0x564f48e1c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 0x564f48e1c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb9c9d8700 1 -- 172.21.15.5:0/690430593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 msgr2=0x564f48e17110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 0x564f48e17110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.000+0000 7feb9c9d8700 1 -- 172.21.15.5:0/690430593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f48bec3b0 con 0x564f48e45dd0 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.001+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e45dd0 0x564f48e46250 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7feb90087a50 tx=0x7feb90057fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4f22bcc3a8da6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.001+0000 7feb7dffb700 1 -- 172.21.15.5:0/690430593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9005c1d0 con 0x564f48e45dd0 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.001+0000 7feb7dffb700 1 -- 172.21.15.5:0/690430593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb90060790 con 0x564f48e45dd0 2024-05-08T23:28:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.001+0000 7feb7dffb700 1 -- 172.21.15.5:0/690430593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90058600 con 0x564f48e45dd0 2024-05-08T23:28:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.001+0000 7febaa5f5b80 1 -- 172.21.15.5:0/690430593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e45dd0 msgr2=0x564f48e46250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.001+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e45dd0 0x564f48e46250 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7feb90087a50 tx=0x7feb90057fe0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.002+0000 7febaa5f5b80 1 -- 172.21.15.5:0/690430593 shutdown_connections 2024-05-08T23:28:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.002+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e45dd0 0x564f48e46250 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.002+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 0x564f48e17110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.002+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/690430593 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 0x564f48e1c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.002+0000 7febaa5f5b80 1 -- 172.21.15.5:0/690430593 >> 172.21.15.5:0/690430593 conn(0x564f48d79210 msgr2=0x564f48e60dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.002+0000 7febaa5f5b80 1 -- 172.21.15.5:0/690430593 shutdown_connections 2024-05-08T23:28:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.002+0000 7febaa5f5b80 1 -- 172.21.15.5:0/690430593 wait complete. 2024-05-08T23:28:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.002+0000 7febaa5f5b80 1 -- start start 2024-05-08T23:28:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.003+0000 7febaa5f5b80 1 Processor -- start 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 0x564f48e647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e45dd0 0x564f48e64d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f48d07560 con 0x564f48e19370 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f48d07200 con 0x564f48e3e7a0 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f48bee6a0 con 0x564f48e45dd0 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7febaa5f5b80 10 client.0 _ll_register_callbacks cb 0x564f48d10b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7feb97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7feb97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e62c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54368/0 (socket says 172.21.15.5:54368) 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7feb97fff700 1 -- 172.21.15.5:0/1834388410 learned_addr learned my addr 172.21.15.5:0/1834388410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7feb977fe700 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 0x564f48e647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e45dd0 0x564f48e64d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.004+0000 7feb97fff700 1 -- 172.21.15.5:0/1834388410 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e45dd0 msgr2=0x564f48e64d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb97fff700 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e45dd0 0x564f48e64d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb97fff700 1 -- 172.21.15.5:0/1834388410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 msgr2=0x564f48e647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb97fff700 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 0x564f48e647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb97fff700 1 -- 172.21.15.5:0/1834388410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9813d0c0 con 0x564f48e19370 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb97fff700 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e62c60 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7feb980b5790 tx=0x7feb8800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f91715e6d0f377 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb76ffd700 1 -- 172.21.15.5:0/1834388410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8801f070 con 0x564f48e19370 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb76ffd700 1 -- 172.21.15.5:0/1834388410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb88003a70 con 0x564f48e19370 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 1 -- 172.21.15.5:0/1834388410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f48cc4270 con 0x564f48e19370 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb76ffd700 1 -- 172.21.15.5:0/1834388410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb88003c30 con 0x564f48e19370 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 10 client.8835 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 1 -- 172.21.15.5:0/1834388410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x564f48b98b60 con 0x564f48e19370 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb76ffd700 1 -- 172.21.15.5:0/1834388410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f48cc4270 con 0x564f48e19370 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7feb76ffd700 1 -- 172.21.15.5:0/1834388410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7feb8802f070 con 0x564f48e19370 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 10 client.8835 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 10 client.8835 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 20 client.8835 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x564f48e70df0 0x564f48e483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 4 client.8835 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.005+0000 7febaa5f5b80 1 -- 172.21.15.5:0/1834388410 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f48dea7e0 con 0x564f48e70df0 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.006+0000 7feb977fe700 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x564f48e70df0 0x564f48e483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.006+0000 7feb977fe700 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x564f48e70df0 0x564f48e483c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f62c87537222a749 server_cookie=df0a42a1b9139e1a in_seq=0 out_seq=0 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.006+0000 7feb76ffd700 10 client.8835 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.006+0000 7feb76ffd700 1 -- 172.21.15.5:0/1834388410 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f48dea7e0 con 0x564f48e70df0 2024-05-08T23:28:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.006+0000 7feb76ffd700 10 client.8835 handle_command_reply: tid=0 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 client.8835 shutdown 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 -- 172.21.15.5:0/1834388410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 msgr2=0x564f48e62c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e62c60 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7feb980b5790 tx=0x7feb8800e000 comp rx=0 tx=0).stop 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 -- 172.21.15.5:0/1834388410 shutdown_connections 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x564f48e70df0 0x564f48e483c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e45dd0 0x564f48e64d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e62c60 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7feb76ffd700 0 client.8835 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/1834388410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e3e7a0 0x564f48e647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 -- 172.21.15.5:0/1834388410 >> 172.21.15.5:0/1834388410 conn(0x564f48d79210 msgr2=0x564f48e461f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 -- 172.21.15.5:0/1834388410 shutdown_connections 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 1 -- 172.21.15.5:0/1834388410 wait complete. 2024-05-08T23:28:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.007+0000 7febaa5f5b80 20 client.8835 trim_cache size 0 max 16384 2024-05-08T23:28:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.023+0000 7febaa5f5b80 1 Processor -- start 2024-05-08T23:28:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.023+0000 7febaa5f5b80 1 -- start start 2024-05-08T23:28:26.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e3e7a0 0x564f48e483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 0x564f48e62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f48e71640 con 0x564f48e19370 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f48e1c750 con 0x564f48e45dd0 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f48e10840 con 0x564f48e3e7a0 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e730f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54380/0 (socket says 172.21.15.5:54380) 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 -- 172.21.15.5:0/27555419 learned_addr learned my addr 172.21.15.5:0/27555419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 0x564f48e62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb977fe700 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e3e7a0 0x564f48e483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 -- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e3e7a0 msgr2=0x564f48e483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e3e7a0 0x564f48e483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 -- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 msgr2=0x564f48e62c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 0x564f48e62c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 -- 172.21.15.5:0/27555419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f48d31560 con 0x564f48e19370 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb977fe700 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e3e7a0 0x564f48e483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb97fff700 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e730f0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7feb88021040 tx=0x7feb8801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dd866459d10c3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.024+0000 7feb7e7fc700 1 -- 172.21.15.5:0/27555419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88025af0 con 0x564f48e19370 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7feb7e7fc700 1 -- 172.21.15.5:0/27555419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8802c040 con 0x564f48e19370 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7feb7e7fc700 1 -- 172.21.15.5:0/27555419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88025e90 con 0x564f48e19370 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 -- 172.21.15.5:0/27555419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 msgr2=0x564f48e730f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e730f0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7feb88021040 tx=0x7feb8801f000 comp rx=0 tx=0).stop 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 -- 172.21.15.5:0/27555419 shutdown_connections 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 0x564f48e62c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e19370 0x564f48e730f0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/27555419 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e3e7a0 0x564f48e483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 -- 172.21.15.5:0/27555419 >> 172.21.15.5:0/27555419 conn(0x564f48d79210 msgr2=0x564f48e16410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 -- 172.21.15.5:0/27555419 shutdown_connections 2024-05-08T23:28:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.025+0000 7febaa5f5b80 1 -- 172.21.15.5:0/27555419 wait complete. 2024-05-08T23:28:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.026+0000 7febaa5f5b80 1 -- start start 2024-05-08T23:28:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.026+0000 7febaa5f5b80 1 Processor -- start 2024-05-08T23:28:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 0x564f48e1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e3e7a0 0x564f48ec8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7febaa5f5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 0x564f48e420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f48ec9090 con 0x564f48e3e7a0 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f48e42b30 con 0x564f48e45dd0 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7febaa5f5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f48e60610 con 0x564f48e19370 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7febaa5f5b80 10 client.0 _ll_register_callbacks cb 0x564f48d10b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e3e7a0 0x564f48ec8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e3e7a0 0x564f48ec8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54386/0 (socket says 172.21.15.5:54386) 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 -- 172.21.15.5:0/3164719881 learned_addr learned my addr 172.21.15.5:0/3164719881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb97fff700 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 0x564f48e1a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb9c9d8700 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 0x564f48e420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 -- 172.21.15.5:0/3164719881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 msgr2=0x564f48e1a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 0x564f48e1a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 -- 172.21.15.5:0/3164719881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 msgr2=0x564f48e420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 0x564f48e420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 -- 172.21.15.5:0/3164719881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f48b98b60 con 0x564f48e3e7a0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb977fe700 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e3e7a0 0x564f48ec8b50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7feb98144340 tx=0x7feb8c007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0873f7edccdfe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.027+0000 7feb76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7feb76ffd700 1 -- 172.21.15.5:0/3164719881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c004a80 con 0x564f48e3e7a0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7feb76ffd700 1 -- 172.21.15.5:0/3164719881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb8c021020 con 0x564f48e3e7a0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 1 -- 172.21.15.5:0/3164719881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb90060790 con 0x564f48e3e7a0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7feb76ffd700 1 -- 172.21.15.5:0/3164719881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb8c004be0 con 0x564f48e3e7a0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 10 client.8841 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 1 -- 172.21.15.5:0/3164719881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x564f48bf0cd0 con 0x564f48e3e7a0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7feb76ffd700 1 -- 172.21.15.5:0/3164719881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb90060790 con 0x564f48e3e7a0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7feb76ffd700 1 -- 172.21.15.5:0/3164719881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7feb8c005050 con 0x564f48e3e7a0 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 10 client.8841 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 10 client.8841 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 20 client.8841 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x564f48e1b420 0x564f48e62370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 4 client.8841 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:28:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7febaa5f5b80 1 -- 172.21.15.5:0/3164719881 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f48d63ec0 con 0x564f48e1b420 2024-05-08T23:28:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.028+0000 7feb97fff700 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x564f48e1b420 0x564f48e62370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.029+0000 7feb76ffd700 10 client.8841 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.029+0000 7feb97fff700 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x564f48e1b420 0x564f48e62370 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64b7c3d3268defe server_cookie=62637365d4cdf901 in_seq=0 out_seq=0 2024-05-08T23:28:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7feb76ffd700 1 -- 172.21.15.5:0/3164719881 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f48d63ec0 con 0x564f48e1b420 2024-05-08T23:28:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7feb76ffd700 10 client.8841 handle_command_reply: tid=0 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 client.8841 shutdown 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 -- 172.21.15.5:0/3164719881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e3e7a0 msgr2=0x564f48ec8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e3e7a0 0x564f48ec8b50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7feb98144340 tx=0x7feb8c007e40 comp rx=0 tx=0).stop 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 -- 172.21.15.5:0/3164719881 shutdown_connections 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x564f48e1b420 0x564f48e62370 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f48e45dd0 0x564f48e420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x564f48e19370 0x564f48e1a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 --2- 172.21.15.5:0/3164719881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f48e3e7a0 0x564f48ec8b50 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7feb76ffd700 0 client.8841 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 -- 172.21.15.5:0/3164719881 >> 172.21.15.5:0/3164719881 conn(0x564f48d79210 msgr2=0x564f48e16410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 -- 172.21.15.5:0/3164719881 shutdown_connections 2024-05-08T23:28:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 1 -- 172.21.15.5:0/3164719881 wait complete. 2024-05-08T23:28:26.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.041+0000 7febaa5f5b80 20 client.8841 trim_cache size 0 max 16384 2024-05-08T23:28:26.043 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:26.043 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:28:26.043 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:28:26.043 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 -- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb8406ac90 msgr2=0x7feb8406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb8406ac90 0x7feb8406d150 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7feb880090b0 tx=0x7feb88008030 comp rx=0 tx=0).stop 2024-05-08T23:28:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 -- 172.21.15.5:0/3319327100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb9815fbb0 msgr2=0x7feb980cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb9815fbb0 0x7feb980cec60 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7feb90007ef0 tx=0x7feb9000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 -- 172.21.15.5:0/3319327100 shutdown_connections 2024-05-08T23:28:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb8406ac90 0x7feb8406d150 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb9815fbb0 0x7feb980cec60 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb980d0a60 0x7feb9813b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 --2- 172.21.15.5:0/3319327100 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb98145040 0x7feb98139e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 -- 172.21.15.5:0/3319327100 >> 172.21.15.5:0/3319327100 conn(0x7feb98099a50 msgr2=0x7feb981435e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 -- 172.21.15.5:0/3319327100 shutdown_connections 2024-05-08T23:28:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.042+0000 7feb9e43b700 1 -- 172.21.15.5:0/3319327100 wait complete. 2024-05-08T23:28:26.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 flush journal 2024-05-08T23:28:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.194+0000 7f6d4e81a700 1 Processor -- start 2024-05-08T23:28:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.194+0000 7f6d4e81a700 1 -- start start 2024-05-08T23:28:26.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.195+0000 7f6d4e81a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d4815b670 0x7f6d4815e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.195+0000 7f6d4e81a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d480af3b0 0x7f6d480ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.195+0000 7f6d4e81a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4815ee70 con 0x7f6d48162170 2024-05-08T23:28:26.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.195+0000 7f6d4e81a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4806d680 con 0x7f6d4815b670 2024-05-08T23:28:26.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.195+0000 7f6d4e81a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d480afc10 con 0x7f6d480af3b0 2024-05-08T23:28:26.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.195+0000 7f6d47fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d4815b670 0x7f6d4815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.195+0000 7f6d477fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6d48162170 0x7f6d480cf4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33642/0 (socket says 172.21.15.5:33642) 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.195+0000 7f6d477fe700 1 -- 172.21.15.5:0/2697487362 learned_addr learned my addr 172.21.15.5:0/2697487362 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/2697487362 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d480af3b0 0x7f6d480ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d47fff700 1 -- 172.21.15.5:0/2697487362 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d480af3b0 msgr2=0x7f6d480ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d47fff700 1 --2- 172.21.15.5:0/2697487362 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d480af3b0 0x7f6d480ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d47fff700 1 -- 172.21.15.5:0/2697487362 >> v1:172.21.15.5:6789/0 conn(0x7f6d48162170 legacy=0x7f6d480cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d47fff700 1 -- 172.21.15.5:0/2697487362 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d480af930 con 0x7f6d4815b670 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d46ffd700 1 -- 172.21.15.5:0/2697487362 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573892084 0 0) 0x7f6d4815ee70 con 0x7f6d48162170 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d47fff700 1 --2- 172.21.15.5:0/2697487362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4815b670 0x7f6d4815e930 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009840 tx=0x7f6d38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bc07efb66a724c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d46ffd700 1 -- 172.21.15.5:0/2697487362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38011830 con 0x7f6d4815b670 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d46ffd700 1 -- 172.21.15.5:0/2697487362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d38011990 con 0x7f6d4815b670 2024-05-08T23:28:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.196+0000 7f6d46ffd700 1 -- 172.21.15.5:0/2697487362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38011d20 con 0x7f6d4815b670 2024-05-08T23:28:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.197+0000 7f6d4e81a700 1 -- 172.21.15.5:0/2697487362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4815b670 msgr2=0x7f6d4815e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.197+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/2697487362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4815b670 0x7f6d4815e930 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009840 tx=0x7f6d38009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.197+0000 7f6d4e81a700 1 -- 172.21.15.5:0/2697487362 shutdown_connections 2024-05-08T23:28:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.197+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/2697487362 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d480af3b0 0x7f6d480ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.197+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/2697487362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4815b670 0x7f6d4815e930 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.197+0000 7f6d4e81a700 1 -- 172.21.15.5:0/2697487362 >> 172.21.15.5:0/2697487362 conn(0x7f6d48099a50 msgr2=0x7f6d480b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.197+0000 7f6d4e81a700 1 -- 172.21.15.5:0/2697487362 shutdown_connections 2024-05-08T23:28:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.197+0000 7f6d4e81a700 1 -- 172.21.15.5:0/2697487362 wait complete. 2024-05-08T23:28:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.198+0000 7f6d4e81a700 1 Processor -- start 2024-05-08T23:28:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.198+0000 7f6d4e81a700 1 -- start start 2024-05-08T23:28:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4e81a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d480af3b0 0x7f6d481f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4e81a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d4815b670 0x7f6d481518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4e81a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48162170 0x7f6d4815f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4e81a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d48151df0 con 0x7f6d48162170 2024-05-08T23:28:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4e81a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d481f42e0 con 0x7f6d480af3b0 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4e81a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d480c8790 con 0x7f6d4815b670 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4cdb7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48162170 0x7f6d4815f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4cdb7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48162170 0x7f6d4815f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54410/0 (socket says 172.21.15.5:54410) 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.199+0000 7f6d4cdb7700 1 -- 172.21.15.5:0/4277364331 learned_addr learned my addr 172.21.15.5:0/4277364331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d477fe700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d4815b670 0x7f6d481518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d47fff700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d480af3b0 0x7f6d481f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d4cdb7700 1 -- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d4815b670 msgr2=0x7f6d481518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d4815b670 0x7f6d481518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d4cdb7700 1 -- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d480af3b0 msgr2=0x7f6d481f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d480af3b0 0x7f6d481f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d4cdb7700 1 -- 172.21.15.5:0/4277364331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380094a0 con 0x7f6d48162170 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48162170 0x7f6d4815f540 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f6d40007e40 tx=0x7f6d4000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d44ff9700 1 -- 172.21.15.5:0/4277364331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000eea0 con 0x7f6d48162170 2024-05-08T23:28:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d44ff9700 1 -- 172.21.15.5:0/4277364331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d40014420 con 0x7f6d48162170 2024-05-08T23:28:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d4e81a700 1 -- 172.21.15.5:0/4277364331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d480c8a70 con 0x7f6d48162170 2024-05-08T23:28:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d44ff9700 1 -- 172.21.15.5:0/4277364331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400147b0 con 0x7f6d48162170 2024-05-08T23:28:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.200+0000 7f6d4e81a700 1 -- 172.21.15.5:0/4277364331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d480c9030 con 0x7f6d48162170 2024-05-08T23:28:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.202+0000 7f6d44ff9700 1 -- 172.21.15.5:0/4277364331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d40014910 con 0x7f6d48162170 2024-05-08T23:28:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.202+0000 7f6d44ff9700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d3406ac90 0x7f6d3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.202+0000 7f6d44ff9700 1 -- 172.21.15.5:0/4277364331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d40089b50 con 0x7f6d48162170 2024-05-08T23:28:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.202+0000 7f6d47fff700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d3406ac90 0x7f6d3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.203+0000 7f6d47fff700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d3406ac90 0x7f6d3406d150 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d380090b0 tx=0x7f6d38008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.204+0000 7f6d5a9d4b80 1 Processor -- start 2024-05-08T23:28:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.204+0000 7f6d5a9d4b80 1 -- start start 2024-05-08T23:28:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c646110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 0x55f05c64b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c675250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05c5a1150 con 0x55f05c66d7a0 2024-05-08T23:28:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05c59eb10 con 0x55f05c648370 2024-05-08T23:28:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f05c59a400 con 0x55f05c674dd0 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c646110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c646110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54418/0 (socket says 172.21.15.5:54418) 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d47fff700 1 -- 172.21.15.5:0/3650055600 learned_addr learned my addr 172.21.15.5:0/3650055600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c675250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.205+0000 7f6d477fe700 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 0x55f05c64b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d47fff700 1 -- 172.21.15.5:0/3650055600 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 msgr2=0x55f05c675250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c675250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d47fff700 1 -- 172.21.15.5:0/3650055600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 msgr2=0x55f05c64b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 0x55f05c64b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d47fff700 1 -- 172.21.15.5:0/3650055600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f05c41b3b0 con 0x55f05c66d7a0 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c646110 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f6d48071380 tx=0x7f6d38010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d63edcde1ba612 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d2dffb700 1 -- 172.21.15.5:0/3650055600 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3802b070 con 0x55f05c66d7a0 2024-05-08T23:28:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d2dffb700 1 -- 172.21.15.5:0/3650055600 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d380036a0 con 0x55f05c66d7a0 2024-05-08T23:28:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.206+0000 7f6d2dffb700 1 -- 172.21.15.5:0/3650055600 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3801f070 con 0x55f05c66d7a0 2024-05-08T23:28:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3650055600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 msgr2=0x55f05c646110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c646110 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f6d48071380 tx=0x7f6d38010000 comp rx=0 tx=0).stop 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3650055600 shutdown_connections 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c675250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c646110 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3650055600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 0x55f05c64b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3650055600 >> 172.21.15.5:0/3650055600 conn(0x55f05c5a8210 msgr2=0x55f05c68fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3650055600 shutdown_connections 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3650055600 wait complete. 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.207+0000 7f6d5a9d4b80 1 -- start start 2024-05-08T23:28:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.208+0000 7f6d5a9d4b80 1 Processor -- start 2024-05-08T23:28:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c648370 0x55f05c678780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c674dd0 0x55f05c695a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05c536560 con 0x55f05c66d7a0 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05c536200 con 0x55f05c674dd0 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f05c41d6a0 con 0x55f05c648370 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d5a9d4b80 10 client.0 _ll_register_callbacks cb 0x55f05c53fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54420/0 (socket says 172.21.15.5:54420) 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d477fe700 1 -- 172.21.15.5:0/3008558790 learned_addr learned my addr 172.21.15.5:0/3008558790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c648370 0x55f05c678780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.209+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c674dd0 0x55f05c695a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d477fe700 1 -- 172.21.15.5:0/3008558790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c648370 msgr2=0x55f05c678780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d477fe700 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c648370 0x55f05c678780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d477fe700 1 -- 172.21.15.5:0/3008558790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c674dd0 msgr2=0x55f05c695a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d477fe700 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c674dd0 0x55f05c695a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d477fe700 1 -- 172.21.15.5:0/3008558790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d480af930 con 0x55f05c66d7a0 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d477fe700 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695520 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c007710 tx=0x7f6d3c001430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ed38d3dcaa5221 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d26ffd700 1 -- 172.21.15.5:0/3008558790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c001ba0 con 0x55f05c66d7a0 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d26ffd700 1 -- 172.21.15.5:0/3008558790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d3c001d00 con 0x55f05c66d7a0 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3008558790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f05c4f3270 con 0x55f05c66d7a0 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d26ffd700 1 -- 172.21.15.5:0/3008558790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d3c015470 con 0x55f05c66d7a0 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d5a9d4b80 10 client.8856 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3008558790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x55f05c3c7b60 con 0x55f05c66d7a0 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d26ffd700 1 -- 172.21.15.5:0/3008558790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f05c4f3270 con 0x55f05c66d7a0 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.210+0000 7f6d26ffd700 1 -- 172.21.15.5:0/3008558790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f6d3c0155d0 con 0x55f05c66d7a0 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d5a9d4b80 10 client.8856 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d5a9d4b80 10 client.8856 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d5a9d4b80 20 client.8856 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x55f05c66c490 0x55f05c676460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d5a9d4b80 4 client.8856 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3008558790 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f05c6197e0 con 0x55f05c66c490 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x55f05c66c490 0x55f05c676460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x55f05c66c490 0x55f05c676460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44f74252f66af255 server_cookie=6c3e98bdd2431833 in_seq=0 out_seq=0 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.211+0000 7f6d26ffd700 10 client.8856 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d26ffd700 1 -- 172.21.15.5:0/3008558790 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f05c6197e0 con 0x55f05c66c490 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d26ffd700 10 client.8856 handle_command_reply: tid=0 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 client.8856 shutdown 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3008558790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 msgr2=0x55f05c695520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695520 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c007710 tx=0x7f6d3c001430 comp rx=0 tx=0).stop 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3008558790 shutdown_connections 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x55f05c66c490 0x55f05c676460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c674dd0 0x55f05c695a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c648370 0x55f05c678780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d26ffd700 0 client.8856 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/3008558790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695520 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3008558790 >> 172.21.15.5:0/3008558790 conn(0x55f05c5a8210 msgr2=0x55f05c64a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3008558790 shutdown_connections 2024-05-08T23:28:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/3008558790 wait complete. 2024-05-08T23:28:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.212+0000 7f6d5a9d4b80 20 client.8856 trim_cache size 0 max 16384 2024-05-08T23:28:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.228+0000 7f6d5a9d4b80 1 Processor -- start 2024-05-08T23:28:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.228+0000 7f6d5a9d4b80 1 -- start start 2024-05-08T23:28:26.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c648370 0x55f05c6751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c66d7a0 0x55f05c6953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c695900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05c694b30 con 0x55f05c648370 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05c6786a0 con 0x55f05c66d7a0 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f05c678210 con 0x55f05c674dd0 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c648370 0x55f05c6751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c648370 0x55f05c6751b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54436/0 (socket says 172.21.15.5:54436) 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 -- 172.21.15.5:0/1227098044 learned_addr learned my addr 172.21.15.5:0/1227098044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c695900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d477fe700 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c66d7a0 0x55f05c6953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 -- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 msgr2=0x55f05c695900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c695900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 -- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c66d7a0 msgr2=0x55f05c6953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c66d7a0 0x55f05c6953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 -- 172.21.15.5:0/1227098044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f05c560560 con 0x55f05c648370 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c695900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d477fe700 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c66d7a0 0x55f05c6953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d47fff700 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c648370 0x55f05c6751b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f6d38011cb0 tx=0x7f6d3801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18355a104cae6d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d2e7fc700 1 -- 172.21.15.5:0/1227098044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38040ea0 con 0x55f05c648370 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.229+0000 7f6d2e7fc700 1 -- 172.21.15.5:0/1227098044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d38003a90 con 0x55f05c648370 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d2e7fc700 1 -- 172.21.15.5:0/1227098044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3802b040 con 0x55f05c648370 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/1227098044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c648370 msgr2=0x55f05c6751b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c648370 0x55f05c6751b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f6d38011cb0 tx=0x7f6d3801f000 comp rx=0 tx=0).stop 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/1227098044 shutdown_connections 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c695900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c648370 0x55f05c6751b0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/1227098044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c66d7a0 0x55f05c6953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/1227098044 >> 172.21.15.5:0/1227098044 conn(0x55f05c5a8210 msgr2=0x55f05c6411e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/1227098044 shutdown_connections 2024-05-08T23:28:26.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.230+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/1227098044 wait complete. 2024-05-08T23:28:26.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.231+0000 7f6d5a9d4b80 1 -- start start 2024-05-08T23:28:26.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.231+0000 7f6d5a9d4b80 1 Processor -- start 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 0x55f05c6952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d5a9d4b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c6fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05c66b590 con 0x55f05c66d7a0 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05c6788b0 con 0x55f05c648370 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d5a9d4b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f05c649120 con 0x55f05c674dd0 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d5a9d4b80 10 client.0 _ll_register_callbacks cb 0x55f05c53fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54438/0 (socket says 172.21.15.5:54438) 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 -- 172.21.15.5:0/2460822428 learned_addr learned my addr 172.21.15.5:0/2460822428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d4cdb7700 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c6fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d47fff700 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 0x55f05c6952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 -- 172.21.15.5:0/2460822428 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 msgr2=0x55f05c6fbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c6fbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 -- 172.21.15.5:0/2460822428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 msgr2=0x55f05c6952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 0x55f05c6952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 -- 172.21.15.5:0/2460822428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f05c41b3b0 con 0x55f05c66d7a0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d477fe700 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695820 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0076a0 tx=0x7f6d3c001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b02639767a9aa6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.232+0000 7f6d26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d26ffd700 1 -- 172.21.15.5:0/2460822428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c02c070 con 0x55f05c66d7a0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d26ffd700 1 -- 172.21.15.5:0/2460822428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d3c015b40 con 0x55f05c66d7a0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/2460822428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d380036a0 con 0x55f05c66d7a0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d26ffd700 1 -- 172.21.15.5:0/2460822428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d3c015680 con 0x55f05c66d7a0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 10 client.8862 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/2460822428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x55f05c41fcd0 con 0x55f05c66d7a0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d26ffd700 1 -- 172.21.15.5:0/2460822428 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d380036a0 con 0x55f05c66d7a0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d26ffd700 1 -- 172.21.15.5:0/2460822428 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f6d3c004870 con 0x55f05c66d7a0 2024-05-08T23:28:26.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 10 client.8862 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:26.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 10 client.8862 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:26.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 20 client.8862 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:26.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x55f05c68fbb0 0x55f05c6fc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 4 client.8862 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:28:26.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/2460822428 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f05c592ec0 con 0x55f05c68fbb0 2024-05-08T23:28:26.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.233+0000 7f6d47fff700 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x55f05c68fbb0 0x55f05c6fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.234+0000 7f6d26ffd700 10 client.8862 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:26.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.234+0000 7f6d47fff700 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x55f05c68fbb0 0x55f05c6fc520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94f2ae8ebd7c06aa server_cookie=83169762f401bda0 in_seq=0 out_seq=0 2024-05-08T23:28:26.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.241+0000 7f6d26ffd700 1 -- 172.21.15.5:0/2460822428 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f05c592ec0 con 0x55f05c68fbb0 2024-05-08T23:28:26.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.241+0000 7f6d26ffd700 10 client.8862 handle_command_reply: tid=0 2024-05-08T23:28:26.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.241+0000 7f6d5a9d4b80 1 client.8862 shutdown 2024-05-08T23:28:26.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.241+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/2460822428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 msgr2=0x55f05c695820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.241+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695820 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0076a0 tx=0x7f6d3c001080 comp rx=0 tx=0).stop 2024-05-08T23:28:26.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/2460822428 shutdown_connections 2024-05-08T23:28:26.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x55f05c68fbb0 0x55f05c6fc520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55f05c674dd0 0x55f05c6fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f05c648370 0x55f05c6952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 1 --2- 172.21.15.5:0/2460822428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f05c66d7a0 0x55f05c695820 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d26ffd700 0 client.8862 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/2460822428 >> 172.21.15.5:0/2460822428 conn(0x55f05c5a8210 msgr2=0x55f05c678d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/2460822428 shutdown_connections 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 1 -- 172.21.15.5:0/2460822428 wait complete. 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d5a9d4b80 20 client.8862 trim_cache size 0 max 16384 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:26.243 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:28:26.244 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:28:26.244 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d4e81a700 1 -- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d3406ac90 msgr2=0x7f6d3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d3406ac90 0x7f6d3406d150 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d380090b0 tx=0x7f6d38008030 comp rx=0 tx=0).stop 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d4e81a700 1 -- 172.21.15.5:0/4277364331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48162170 msgr2=0x7f6d4815f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48162170 0x7f6d4815f540 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f6d40007e40 tx=0x7f6d4000e3c0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d4e81a700 1 -- 172.21.15.5:0/4277364331 shutdown_connections 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.242+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6d3406ac90 0x7f6d3406d150 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.243+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48162170 0x7f6d4815f540 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.243+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d480af3b0 0x7f6d481f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.243+0000 7f6d4e81a700 1 --2- 172.21.15.5:0/4277364331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6d4815b670 0x7f6d481518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.243+0000 7f6d4e81a700 1 -- 172.21.15.5:0/4277364331 >> 172.21.15.5:0/4277364331 conn(0x7f6d48099a50 msgr2=0x7f6d480cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.243+0000 7f6d4e81a700 1 -- 172.21.15.5:0/4277364331 shutdown_connections 2024-05-08T23:28:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.243+0000 7f6d4e81a700 1 -- 172.21.15.5:0/4277364331 wait complete. 2024-05-08T23:28:26.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 flush journal 2024-05-08T23:28:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.395+0000 7fd39c31b700 1 Processor -- start 2024-05-08T23:28:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.395+0000 7fd39c31b700 1 -- start start 2024-05-08T23:28:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.396+0000 7fd39c31b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd394099a50 0x7fd3940d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.396+0000 7fd39c31b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd394099e30 0x7fd39409a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.396+0000 7fd39c31b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd39406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.396+0000 7fd39c31b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39406f930 con 0x7fd394099e30 2024-05-08T23:28:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd39c31b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3940cfdf0 con 0x7fd394099a50 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd39c31b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940d00d0 con 0x7fd3940cba90 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd394099e30 0x7fd39409a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd394099e30 0x7fd39409a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54442/0 (socket says 172.21.15.5:54442) 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 -- 172.21.15.5:0/2591326877 learned_addr learned my addr 172.21.15.5:0/2591326877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/2591326877 >> v2:172.21.15.49:3300/0 conn(0x7fd394099a50 0x7fd3940d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/2591326877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd39406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 -- 172.21.15.5:0/2591326877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 msgr2=0x7fd39406fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 --2- 172.21.15.5:0/2591326877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd39406fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 -- 172.21.15.5:0/2591326877 >> v2:172.21.15.49:3300/0 conn(0x7fd394099a50 msgr2=0x7fd3940d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 --2- 172.21.15.5:0/2591326877 >> v2:172.21.15.49:3300/0 conn(0x7fd394099a50 0x7fd3940d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 -- 172.21.15.5:0/2591326877 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940d03b0 con 0x7fd394099e30 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/2591326877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd39406fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3998b6700 1 --2- 172.21.15.5:0/2591326877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099e30 0x7fd39409a790 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd39406eb80 tx=0x7fd384009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b2e509e4edc0a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3990b5700 1 -- 172.21.15.5:0/2591326877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400b870 con 0x7fd394099e30 2024-05-08T23:28:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3990b5700 1 -- 172.21.15.5:0/2591326877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38400b9d0 con 0x7fd394099e30 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.397+0000 7fd3990b5700 1 -- 172.21.15.5:0/2591326877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400bd60 con 0x7fd394099e30 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 -- 172.21.15.5:0/2591326877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099e30 msgr2=0x7fd39409a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2591326877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099e30 0x7fd39409a790 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd39406eb80 tx=0x7fd384009bd0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 -- 172.21.15.5:0/2591326877 shutdown_connections 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2591326877 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd39406fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2591326877 >> v2:172.21.15.49:3300/0 conn(0x7fd394099a50 0x7fd3940d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2591326877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099e30 0x7fd39409a790 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 -- 172.21.15.5:0/2591326877 >> 172.21.15.5:0/2591326877 conn(0x7fd3940b2f30 msgr2=0x7fd3940b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 -- 172.21.15.5:0/2591326877 shutdown_connections 2024-05-08T23:28:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.398+0000 7fd39c31b700 1 -- 172.21.15.5:0/2591326877 wait complete. 2024-05-08T23:28:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.399+0000 7fd39c31b700 1 Processor -- start 2024-05-08T23:28:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39c31b700 1 -- start start 2024-05-08T23:28:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39c31b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099a50 0x7fd39415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39c31b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394099e30 0x7fd3940b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39c31b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd3941586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39c31b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940b5a40 con 0x7fd394099a50 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39c31b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394158bf0 con 0x7fd394099e30 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39c31b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940b5ce0 con 0x7fd3940cba90 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39a0b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099a50 0x7fd39415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39a0b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099a50 0x7fd39415c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54452/0 (socket says 172.21.15.5:54452) 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.400+0000 7fd39a0b7700 1 -- 172.21.15.5:0/2785336960 learned_addr learned my addr 172.21.15.5:0/2785336960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd3998b6700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394099e30 0x7fd3940b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd3941586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39a0b7700 1 -- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 msgr2=0x7fd3941586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd3941586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39a0b7700 1 -- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394099e30 msgr2=0x7fd3940b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394099e30 0x7fd3940b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39a0b7700 1 -- 172.21.15.5:0/2785336960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd384009550 con 0x7fd394099a50 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099a50 0x7fd39415c370 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd39000e810 tx=0x7fd39000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd38affd700 1 -- 172.21.15.5:0/2785336960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390013610 con 0x7fd394099a50 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd38affd700 1 -- 172.21.15.5:0/2785336960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd390013770 con 0x7fd394099a50 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39c31b700 1 -- 172.21.15.5:0/2785336960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3940b3f50 con 0x7fd394099a50 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd38affd700 1 -- 172.21.15.5:0/2785336960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390013b00 con 0x7fd394099a50 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.401+0000 7fd39c31b700 1 -- 172.21.15.5:0/2785336960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3940b4510 con 0x7fd394099a50 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.403+0000 7fd38affd700 1 -- 172.21.15.5:0/2785336960 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd390013c60 con 0x7fd394099a50 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.403+0000 7fd38affd700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd37806ac90 0x7fd37806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.403+0000 7fd38affd700 1 -- 172.21.15.5:0/2785336960 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd390088f60 con 0x7fd394099a50 2024-05-08T23:28:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.404+0000 7fd3998b6700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd37806ac90 0x7fd37806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.404+0000 7fd3998b6700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd37806ac90 0x7fd37806d150 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd384000c00 tx=0x7fd384008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.405+0000 7fd3a84d5b80 1 Processor -- start 2024-05-08T23:28:26.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.406+0000 7fd3a84d5b80 1 -- start start 2024-05-08T23:28:26.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.406+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aebe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.406+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.406+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.406+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327ae19150 con 0x56327aec0370 2024-05-08T23:28:26.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.406+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327ae16b10 con 0x56327aeecdd0 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.406+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56327ae12400 con 0x56327aee57a0 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54468/0 (socket says 172.21.15.5:54468) 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 -- 172.21.15.5:0/1376967570 learned_addr learned my addr 172.21.15.5:0/1376967570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aebe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 -- 172.21.15.5:0/1376967570 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 msgr2=0x56327aebe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aebe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 -- 172.21.15.5:0/1376967570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 msgr2=0x56327aeed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeed250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 -- 172.21.15.5:0/1376967570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56327ac933b0 con 0x56327aec0370 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd3998b6700 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec30a0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd3940745f0 tx=0x7fd384010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6668dd7d470aae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd377fff700 1 -- 172.21.15.5:0/1376967570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840038a0 con 0x56327aec0370 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd377fff700 1 -- 172.21.15.5:0/1376967570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384003a00 con 0x56327aec0370 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.407+0000 7fd377fff700 1 -- 172.21.15.5:0/1376967570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384003d70 con 0x56327aec0370 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1376967570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 msgr2=0x56327aec30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec30a0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd3940745f0 tx=0x7fd384010000 comp rx=0 tx=0).stop 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1376967570 shutdown_connections 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aebe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1376967570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec30a0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1376967570 >> 172.21.15.5:0/1376967570 conn(0x56327ae20210 msgr2=0x56327af07dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1376967570 shutdown_connections 2024-05-08T23:28:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.408+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1376967570 wait complete. 2024-05-08T23:28:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.409+0000 7fd3a84d5b80 1 -- start start 2024-05-08T23:28:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.409+0000 7fd3a84d5b80 1 Processor -- start 2024-05-08T23:28:26.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aec0370 0x56327af09c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aeecdd0 0x56327af0bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327adae560 con 0x56327aeecdd0 2024-05-08T23:28:26.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327adae200 con 0x56327aec0370 2024-05-08T23:28:26.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56327ac956a0 con 0x56327aee57a0 2024-05-08T23:28:26.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd3a84d5b80 10 client.0 _ll_register_callbacks cb 0x56327adb7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:26.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd39a8b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aeecdd0 0x56327af0bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd39a8b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aeecdd0 0x56327af0bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54482/0 (socket says 172.21.15.5:54482) 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd39a8b8700 1 -- 172.21.15.5:0/1725476191 learned_addr learned my addr 172.21.15.5:0/1725476191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aec0370 0x56327af09c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.410+0000 7fd3998b6700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd39a8b8700 1 -- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 msgr2=0x56327aee9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd39a8b8700 1 -- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aec0370 msgr2=0x56327af09c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aec0370 0x56327af09c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd39a8b8700 1 -- 172.21.15.5:0/1725476191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3940d03b0 con 0x56327aeecdd0 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3998b6700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aec0370 0x56327af09c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aeecdd0 0x56327af0bcc0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00fed0 tx=0x7fd38c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c60892eda2bdf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3757fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3757fa700 1 -- 172.21.15.5:0/1725476191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c014070 con 0x56327aeecdd0 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3757fa700 1 -- 172.21.15.5:0/1725476191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd38c01aad0 con 0x56327aeecdd0 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1725476191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56327ad6b270 con 0x56327aeecdd0 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3757fa700 1 -- 172.21.15.5:0/1725476191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd38c00ee60 con 0x56327aeecdd0 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 10 client.8874 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1725476191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x56327ac3fb60 con 0x56327aeecdd0 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3757fa700 1 -- 172.21.15.5:0/1725476191 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56327ad6b270 con 0x56327aeecdd0 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3757fa700 1 -- 172.21.15.5:0/1725476191 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fd38c01acd0 con 0x56327aeecdd0 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 10 client.8874 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 10 client.8874 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 20 client.8874 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56327af08610 0x56327aeedf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 4 client.8874 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.411+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1725476191 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56327ae917e0 con 0x56327af08610 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.412+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56327af08610 0x56327aeedf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.412+0000 7fd3757fa700 10 client.8874 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.412+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56327af08610 0x56327aeedf70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c74093bfa45b45d server_cookie=e18cde3fc8767659 in_seq=0 out_seq=0 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3757fa700 1 -- 172.21.15.5:0/1725476191 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56327ae917e0 con 0x56327af08610 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3757fa700 10 client.8874 handle_command_reply: tid=0 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 client.8874 shutdown 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1725476191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aeecdd0 msgr2=0x56327af0bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aeecdd0 0x56327af0bcc0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00fed0 tx=0x7fd38c00e6c0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1725476191 shutdown_connections 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56327af08610 0x56327aeedf70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aeecdd0 0x56327af0bcc0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aec0370 0x56327af09c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/1725476191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3757fa700 0 client.8874 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:26.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1725476191 >> 172.21.15.5:0/1725476191 conn(0x56327ae20210 msgr2=0x56327aebc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1725476191 shutdown_connections 2024-05-08T23:28:26.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/1725476191 wait complete. 2024-05-08T23:28:26.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.413+0000 7fd3a84d5b80 20 client.8874 trim_cache size 0 max 16384 2024-05-08T23:28:26.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.429+0000 7fd3a84d5b80 1 Processor -- start 2024-05-08T23:28:26.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.429+0000 7fd3a84d5b80 1 -- start start 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327af171c0 con 0x56327aec0370 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327aefe9c0 con 0x56327aeecdd0 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56327aeea860 con 0x56327aee57a0 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54490/0 (socket says 172.21.15.5:54490) 2024-05-08T23:28:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 -- 172.21.15.5:0/3568987221 learned_addr learned my addr 172.21.15.5:0/3568987221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd3998b6700 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 -- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 msgr2=0x56327aee9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 -- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 msgr2=0x56327aeea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 -- 172.21.15.5:0/3568987221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56327add8560 con 0x56327aec0370 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd3998b6700 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec2db0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fd3900542f0 tx=0x7fd390057c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab77e8c1ecfeaf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd388da4700 1 -- 172.21.15.5:0/3568987221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900601e0 con 0x56327aec0370 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd388da4700 1 -- 172.21.15.5:0/3568987221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd390055f30 con 0x56327aec0370 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.430+0000 7fd388da4700 1 -- 172.21.15.5:0/3568987221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390056190 con 0x56327aec0370 2024-05-08T23:28:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3568987221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 msgr2=0x56327aec2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec2db0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fd3900542f0 tx=0x7fd390057c70 comp rx=0 tx=0).stop 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3568987221 shutdown_connections 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aeea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aec0370 0x56327aec2db0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3568987221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aee57a0 0x56327aee9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3568987221 >> 172.21.15.5:0/3568987221 conn(0x56327ae20210 msgr2=0x56327aee91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3568987221 shutdown_connections 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.431+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3568987221 wait complete. 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.432+0000 7fd3a84d5b80 1 -- start start 2024-05-08T23:28:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.432+0000 7fd3a84d5b80 1 Processor -- start 2024-05-08T23:28:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.433+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aec0370 0x56327af740d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.433+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aee57a0 0x56327aee8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.433+0000 7fd3a84d5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aee86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.433+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327af09c60 con 0x56327aee57a0 2024-05-08T23:28:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.433+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327af75a40 con 0x56327aeecdd0 2024-05-08T23:28:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.433+0000 7fd3a84d5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56327aebd790 con 0x56327aec0370 2024-05-08T23:28:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.433+0000 7fd3a84d5b80 10 client.0 _ll_register_callbacks cb 0x56327adb7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aee57a0 0x56327aee8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aee57a0 0x56327aee8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54502/0 (socket says 172.21.15.5:54502) 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 -- 172.21.15.5:0/3746122033 learned_addr learned my addr 172.21.15.5:0/3746122033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aec0370 0x56327af740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd39a8b8700 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aee86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 -- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aec0370 msgr2=0x56327af740d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aec0370 0x56327af740d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 -- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 msgr2=0x56327aee86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aee86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 -- 172.21.15.5:0/3746122033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3940b4510 con 0x56327aee57a0 2024-05-08T23:28:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3998b6700 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aee57a0 0x56327aee8180 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b7a0 tx=0x7fd3840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd632b0a1db3f20e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3757fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3757fa700 1 -- 172.21.15.5:0/3746122033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384003d70 con 0x56327aee57a0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3757fa700 1 -- 172.21.15.5:0/3746122033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd384004560 con 0x56327aee57a0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3746122033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd384003a00 con 0x56327aee57a0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3757fa700 1 -- 172.21.15.5:0/3746122033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd384025460 con 0x56327aee57a0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3a84d5b80 10 client.8880 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.434+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3746122033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x56327ac97cd0 con 0x56327aee57a0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3757fa700 1 -- 172.21.15.5:0/3746122033 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384003a00 con 0x56327aee57a0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3757fa700 1 -- 172.21.15.5:0/3746122033 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7fd3840255c0 con 0x56327aee57a0 2024-05-08T23:28:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3a84d5b80 10 client.8880 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3a84d5b80 10 client.8880 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3a84d5b80 20 client.8880 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56327aee9dc0 0x56327af06ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3a84d5b80 4 client.8880 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:28:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3746122033 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56327ae0aec0 con 0x56327aee9dc0 2024-05-08T23:28:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56327aee9dc0 0x56327af06ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd3757fa700 10 client.8880 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.435+0000 7fd39a0b7700 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56327aee9dc0 0x56327af06ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b5031e28eb602b3 server_cookie=13cba3686d7c0baa in_seq=0 out_seq=0 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3757fa700 1 -- 172.21.15.5:0/3746122033 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56327ae0aec0 con 0x56327aee9dc0 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3757fa700 10 client.8880 handle_command_reply: tid=0 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 client.8880 shutdown 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3746122033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aee57a0 msgr2=0x56327aee8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aee57a0 0x56327aee8180 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b7a0 tx=0x7fd3840047c0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3746122033 shutdown_connections 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x56327aee9dc0 0x56327af06ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56327aeecdd0 0x56327aee86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56327aec0370 0x56327af740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3757fa700 0 client.8880 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 --2- 172.21.15.5:0/3746122033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56327aee57a0 0x56327aee8180 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3746122033 >> 172.21.15.5:0/3746122033 conn(0x56327ae20210 msgr2=0x56327aebcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3746122033 shutdown_connections 2024-05-08T23:28:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.442+0000 7fd3a84d5b80 1 -- 172.21.15.5:0/3746122033 wait complete. 2024-05-08T23:28:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd3a84d5b80 20 client.8880 trim_cache size 0 max 16384 2024-05-08T23:28:26.444 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:26.444 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-08T23:28:26.445 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:28:26.445 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 -- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd37806ac90 msgr2=0x7fd37806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd37806ac90 0x7fd37806d150 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd384000c00 tx=0x7fd384008030 comp rx=0 tx=0).stop 2024-05-08T23:28:26.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 -- 172.21.15.5:0/2785336960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099a50 msgr2=0x7fd39415c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099a50 0x7fd39415c370 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd39000e810 tx=0x7fd39000ebd0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 -- 172.21.15.5:0/2785336960 shutdown_connections 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd37806ac90 0x7fd37806d150 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd3940cba90 0x7fd3941586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd394099a50 0x7fd39415c370 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 --2- 172.21.15.5:0/2785336960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394099e30 0x7fd3940b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 -- 172.21.15.5:0/2785336960 >> 172.21.15.5:0/2785336960 conn(0x7fd3940b2f30 msgr2=0x7fd3940b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 -- 172.21.15.5:0/2785336960 shutdown_connections 2024-05-08T23:28:26.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:26.443+0000 7fd39c31b700 1 -- 172.21.15.5:0/2785336960 wait complete. 2024-05-08T23:28:26.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:26.457 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-05-08T23:28:26.486 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/ 2024-05-08T23:28:26.486 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4 2024-05-08T23:28:26.486 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5 2024-05-08T23:28:26.486 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6 2024-05-08T23:28:26.486 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7 2024-05-08T23:28:26.486 INFO:teuthology.orchestra.run.smithi005.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-05-08T23:28:26.487 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:28:26.487 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:28:26.487 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:28:26.487 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:28:26.487 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-05-08T23:28:26.487 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:28:26.603 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-08T23:28:26.792 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:28:26.793 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:28:26.793 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:28:26.793 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:28:26.793 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-05-08T23:28:26.793 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:28:26.914 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-05-08T23:28:27.103 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:28:27.103 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:28:27.103 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:28:27.103 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:28:27.103 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-05-08T23:28:27.104 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:28:27.223 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-05-08T23:28:27.412 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:28:27.412 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:28:27.413 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:28:27.413 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:28:27.413 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-05-08T23:28:27.413 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:28:27.535 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627781 2024-05-08T23:28:27.723 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:28:27.724 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:28:27.724 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:28:27.724 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:28:27.724 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-05-08T23:28:27.724 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:28:27.845 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627782 2024-05-08T23:28:28.034 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:28:28.034 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-08T23:28:28.035 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-08T23:28:28.035 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-08T23:28:28.035 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-05-08T23:28:28.035 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-08T23:28:28.157 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627783 2024-05-08T23:28:28.346 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-05-08T23:28:28.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.441+0000 7f501c0fd080 1 Processor -- start 2024-05-08T23:28:28.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.441+0000 7f501c0fd080 1 -- start start 2024-05-08T23:28:28.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f501c0fd080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a0245950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f501c0fd080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a01cddc0 0x55a3a0256e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f501c0fd080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a0257e10 0x55a3a01cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f501c0fd080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a012a1a0 con 0x55a3a01cddc0 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f501c0fd080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a00c56d0 con 0x55a3a0184dd0 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f501c0fd080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3a00c5370 con 0x55a3a0257e10 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f500bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a01cddc0 0x55a3a0256e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f500bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a01cddc0 0x55a3a0256e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47372/0 (socket says 172.21.15.5:47372) 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f500bfff700 1 -- 172.21.15.5:0/497300534 learned_addr learned my addr 172.21.15.5:0/497300534 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f50121cf700 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a0245950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.442+0000 7f50129d0700 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a0257e10 0x55a3a01cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f500bfff700 1 -- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a0257e10 msgr2=0x55a3a01cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f500bfff700 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a0257e10 0x55a3a01cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f500bfff700 1 -- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 msgr2=0x55a3a0245950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f500bfff700 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a0245950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f500bfff700 1 -- 172.21.15.5:0/497300534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004005de0 con 0x55a3a01cddc0 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f50121cf700 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a0245950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f500bfff700 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a01cddc0 0x55a3a0256e80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0098b0 tx=0x7f4ffc005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f651a61afbe323 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f50111cd700 1 -- 172.21.15.5:0/497300534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc00f040 con 0x55a3a01cddc0 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f50111cd700 1 -- 172.21.15.5:0/497300534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc008a30 con 0x55a3a01cddc0 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.443+0000 7f50111cd700 1 -- 172.21.15.5:0/497300534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc008de0 con 0x55a3a01cddc0 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 -- 172.21.15.5:0/497300534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a01cddc0 msgr2=0x55a3a0256e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a01cddc0 0x55a3a0256e80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0098b0 tx=0x7f4ffc005a00 comp rx=0 tx=0).stop 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 -- 172.21.15.5:0/497300534 shutdown_connections 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a0257e10 0x55a3a01cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a0245950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 --2- 172.21.15.5:0/497300534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a01cddc0 0x55a3a0256e80 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 -- 172.21.15.5:0/497300534 >> 172.21.15.5:0/497300534 conn(0x55a3a0183ba0 msgr2=0x55a3a01cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 -- 172.21.15.5:0/497300534 shutdown_connections 2024-05-08T23:28:28.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 -- 172.21.15.5:0/497300534 wait complete. 2024-05-08T23:28:28.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.444+0000 7f501c0fd080 1 Processor -- start 2024-05-08T23:28:28.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.445+0000 7f501c0fd080 1 -- start start 2024-05-08T23:28:28.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.445+0000 7f501c0fd080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a02530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.445+0000 7f501c0fd080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a01cddc0 0x55a3a0235920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.445+0000 7f501c0fd080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0257e10 0x55a3a0235e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.445+0000 7f501c0fd080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a00bc680 con 0x55a3a0257e10 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.445+0000 7f501c0fd080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a01cbc00 con 0x55a3a0184dd0 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.445+0000 7f501c0fd080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3a0230c40 con 0x55a3a01cddc0 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50129d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0257e10 0x55a3a0235e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50129d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0257e10 0x55a3a0235e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47376/0 (socket says 172.21.15.5:47376) 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50129d0700 1 -- 172.21.15.5:0/3623234732 learned_addr learned my addr 172.21.15.5:0/3623234732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50121cf700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a02530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f500bfff700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a01cddc0 0x55a3a0235920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50121cf700 1 -- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a01cddc0 msgr2=0x55a3a0235920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50121cf700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55a3a01cddc0 0x55a3a0235920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50121cf700 1 -- 172.21.15.5:0/3623234732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0257e10 msgr2=0x55a3a0235e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50121cf700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0257e10 0x55a3a0235e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50121cf700 1 -- 172.21.15.5:0/3623234732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc009510 con 0x55a3a0184dd0 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50129d0700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0257e10 0x55a3a0235e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.446+0000 7f50121cf700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a02530f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5004014000 tx=0x7f5004002fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.447+0000 7f500b7fe700 1 -- 172.21.15.5:0/3623234732 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004008690 con 0x55a3a0184dd0 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.447+0000 7f500b7fe700 1 -- 172.21.15.5:0/3623234732 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500401d830 con 0x55a3a0184dd0 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.447+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a3a00d48a0 con 0x55a3a0184dd0 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.447+0000 7f500b7fe700 1 -- 172.21.15.5:0/3623234732 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500401dbe0 con 0x55a3a0184dd0 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.447+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3a00c7ed0 con 0x55a3a0184dd0 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.448+0000 7f500b7fe700 1 -- 172.21.15.5:0/3623234732 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f500401dd40 con 0x55a3a0184dd0 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.449+0000 7f500b7fe700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ff40695e0 0x7f4ff406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.449+0000 7f500b7fe700 1 -- 172.21.15.5:0/3623234732 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5004089780 con 0x55a3a0184dd0 2024-05-08T23:28:28.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.449+0000 7f500bfff700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ff40695e0 0x7f4ff406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.449+0000 7f501c0fd080 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x55a3a0253630 0x55a3a0237f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.449+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 --> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] -- osd_op(unknown.0.0:1 14.12 14:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x55a3a01cc3e0 con 0x55a3a0253630 2024-05-08T23:28:28.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.449+0000 7f50129d0700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x55a3a0253630 0x55a3a0237f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.449+0000 7f50129d0700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x55a3a0253630 0x55a3a0237f00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.449+0000 7f500bfff700 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ff40695e0 0x7f4ff406baa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55a3a0254130 tx=0x7f4ffc01f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f50129d0700 1 -- 172.21.15.5:0/3623234732 <== osd.2 v2:172.21.15.5:6800/47131 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v96'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f500c003c10 con 0x55a3a0253630 2024-05-08T23:28:28.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x55a3a0253630 msgr2=0x55a3a0237f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x55a3a0253630 0x55a3a0237f00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ff40695e0 msgr2=0x7f4ff406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4ff40695e0 0x7f4ff406baa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55a3a0254130 tx=0x7f4ffc01f040 comp rx=0 tx=0).stop 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 msgr2=0x55a3a02530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 --2- 172.21.15.5:0/3623234732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a3a0184dd0 0x55a3a02530f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5004014000 tx=0x7f5004002fb0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f50129d0700 1 -- 172.21.15.5:0/3623234732 reap_dead start 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 shutdown_connections 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 >> 172.21.15.5:0/3623234732 conn(0x55a3a0183ba0 msgr2=0x55a3a0184180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 shutdown_connections 2024-05-08T23:28:28.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.451+0000 7f501c0fd080 1 -- 172.21.15.5:0/3623234732 wait complete. 2024-05-08T23:28:28.454 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-05-08T23:28:28.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.506+0000 7fef662ee080 1 Processor -- start 2024-05-08T23:28:28.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.506+0000 7fef662ee080 1 -- start start 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef662ee080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea51beba0 0x558ea524bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef662ee080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea5234a30 0x558ea51bc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef662ee080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea51a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef662ee080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea50b86d0 con 0x558ea524b5b0 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef662ee080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea50b8370 con 0x558ea5234a30 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef662ee080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ea50af680 con 0x558ea51beba0 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea51a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea51a3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47394/0 (socket says 172.21.15.5:47394) 2024-05-08T23:28:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 -- 172.21.15.5:0/619566273 learned_addr learned my addr 172.21.15.5:0/619566273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef53fff700 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea5234a30 0x558ea51bc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5c3c0700 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea51beba0 0x558ea524bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 -- 172.21.15.5:0/619566273 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea51beba0 msgr2=0x558ea524bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea51beba0 0x558ea524bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 -- 172.21.15.5:0/619566273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea5234a30 msgr2=0x558ea51bc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea5234a30 0x558ea51bc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 -- 172.21.15.5:0/619566273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c005de0 con 0x558ea524b5b0 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.507+0000 7fef5cbc1700 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea51a3be0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fef540027d0 tx=0x7fef54002ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c3e17eb304eab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef5b3be700 1 -- 172.21.15.5:0/619566273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54012680 con 0x558ea524b5b0 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef5b3be700 1 -- 172.21.15.5:0/619566273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef54012800 con 0x558ea524b5b0 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef5b3be700 1 -- 172.21.15.5:0/619566273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54012bb0 con 0x558ea524b5b0 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 -- 172.21.15.5:0/619566273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 msgr2=0x558ea51a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea51a3be0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fef540027d0 tx=0x7fef54002ae0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 -- 172.21.15.5:0/619566273 shutdown_connections 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea51a3be0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea51beba0 0x558ea524bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 --2- 172.21.15.5:0/619566273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea5234a30 0x558ea51bc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 -- 172.21.15.5:0/619566273 >> 172.21.15.5:0/619566273 conn(0x558ea5175ac0 msgr2=0x558ea51bd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 -- 172.21.15.5:0/619566273 shutdown_connections 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.508+0000 7fef662ee080 1 -- 172.21.15.5:0/619566273 wait complete. 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.509+0000 7fef662ee080 1 Processor -- start 2024-05-08T23:28:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.509+0000 7fef662ee080 1 -- start start 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef662ee080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea51beba0 0x558ea524a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef662ee080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea5234a30 0x558ea51bb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef662ee080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea5230310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef662ee080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea524ad40 con 0x558ea524b5b0 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef662ee080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea522d300 con 0x558ea51beba0 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef662ee080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ea522d5e0 con 0x558ea5234a30 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5cbc1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea5230310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5cbc1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea5230310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47406/0 (socket says 172.21.15.5:47406) 2024-05-08T23:28:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5c3c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea51beba0 0x558ea524a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5cbc1700 1 -- 172.21.15.5:0/1716965231 learned_addr learned my addr 172.21.15.5:0/1716965231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef53fff700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea5234a30 0x558ea51bb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5cbc1700 1 -- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea5234a30 msgr2=0x558ea51bb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5cbc1700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558ea5234a30 0x558ea51bb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5cbc1700 1 -- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea51beba0 msgr2=0x558ea524a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5cbc1700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ea51beba0 0x558ea524a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.510+0000 7fef5cbc1700 1 -- 172.21.15.5:0/1716965231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef540024b0 con 0x558ea524b5b0 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.511+0000 7fef5cbc1700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea5230310 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fef54005d20 tx=0x7fef540033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.511+0000 7fef5a3bc700 1 -- 172.21.15.5:0/1716965231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5401b9d0 con 0x558ea524b5b0 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.511+0000 7fef5a3bc700 1 -- 172.21.15.5:0/1716965231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5401bb50 con 0x558ea524b5b0 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.511+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558ea50e1a40 con 0x558ea524b5b0 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.511+0000 7fef5a3bc700 1 -- 172.21.15.5:0/1716965231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54025410 con 0x558ea524b5b0 2024-05-08T23:28:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.511+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ea50c78a0 con 0x558ea524b5b0 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.512+0000 7fef5a3bc700 1 -- 172.21.15.5:0/1716965231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fef54023070 con 0x558ea524b5b0 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.513+0000 7fef5a3bc700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fef3c0695e0 0x7fef3c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.513+0000 7fef5a3bc700 1 -- 172.21.15.5:0/1716965231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef54089a70 con 0x558ea524b5b0 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.513+0000 7fef5c3c0700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fef3c0695e0 0x7fef3c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.513+0000 7fef662ee080 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x558ea522cd90 0x558ea522e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.513+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 --> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] -- osd_op(unknown.0.0:1 14.12 14:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x558ea522e8e0 con 0x558ea522cd90 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.513+0000 7fef53fff700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x558ea522cd90 0x558ea522e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.513+0000 7fef5c3c0700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fef3c0695e0 0x7fef3c06baa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fef4c016040 tx=0x7fef4c010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.513+0000 7fef53fff700 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x558ea522cd90 0x558ea522e3a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef53fff700 1 -- 172.21.15.5:0/1716965231 <== osd.2 v2:172.21.15.5:6800/47131 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v96'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef44012040 con 0x558ea522cd90 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x558ea522cd90 msgr2=0x558ea522e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x558ea522cd90 0x558ea522e3a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fef3c0695e0 msgr2=0x7fef3c06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fef3c0695e0 0x7fef3c06baa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fef4c016040 tx=0x7fef4c010000 comp rx=0 tx=0).stop 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 msgr2=0x558ea5230310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 --2- 172.21.15.5:0/1716965231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ea524b5b0 0x558ea5230310 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fef54005d20 tx=0x7fef540033a0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef5cbc1700 1 -- 172.21.15.5:0/1716965231 reap_dead start 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 shutdown_connections 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 >> 172.21.15.5:0/1716965231 conn(0x558ea5175ac0 msgr2=0x558ea51bd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 shutdown_connections 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.515+0000 7fef662ee080 1 -- 172.21.15.5:0/1716965231 wait complete. 2024-05-08T23:28:28.519 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-05-08T23:28:28.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.569+0000 7fcb5acfc080 1 Processor -- start 2024-05-08T23:28:28.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.569+0000 7fcb5acfc080 1 -- start start 2024-05-08T23:28:28.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb5acfc080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9705d0 0x55cfea973220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb5acfc080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 0x55cfea8ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb5acfc080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9709b0 0x55cfea8ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb5acfc080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfea8538b0 con 0x55cfea9709b0 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb5acfc080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfea84f1a0 con 0x55cfea9711d0 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb5acfc080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfea7ea6d0 con 0x55cfea9705d0 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9709b0 0x55cfea8ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9709b0 0x55cfea8ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47418/0 (socket says 172.21.15.5:47418) 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 -- 172.21.15.5:0/1989996360 learned_addr learned my addr 172.21.15.5:0/1989996360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb505cd700 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 0x55cfea8ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb50dce700 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9705d0 0x55cfea973220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 -- 172.21.15.5:0/1989996360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9705d0 msgr2=0x55cfea973220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9705d0 0x55cfea973220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 -- 172.21.15.5:0/1989996360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 msgr2=0x55cfea8ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 0x55cfea8ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 -- 172.21.15.5:0/1989996360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfea7eced0 con 0x55cfea9709b0 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb515cf700 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9709b0 0x55cfea8ab650 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fcb38004090 tx=0x7fcb48008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2944d93ba910eb54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb4f5cb700 1 -- 172.21.15.5:0/1989996360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480105a0 con 0x55cfea9709b0 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb4f5cb700 1 -- 172.21.15.5:0/1989996360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb48010870 con 0x55cfea9709b0 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.570+0000 7fcb4f5cb700 1 -- 172.21.15.5:0/1989996360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48010c20 con 0x55cfea9709b0 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 -- 172.21.15.5:0/1989996360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9709b0 msgr2=0x55cfea8ab650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9709b0 0x55cfea8ab650 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fcb38004090 tx=0x7fcb48008030 comp rx=0 tx=0).stop 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 -- 172.21.15.5:0/1989996360 shutdown_connections 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9709b0 0x55cfea8ab650 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9705d0 0x55cfea973220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 --2- 172.21.15.5:0/1989996360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 0x55cfea8ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 -- 172.21.15.5:0/1989996360 >> 172.21.15.5:0/1989996360 conn(0x55cfea8d4d60 msgr2=0x55cfea8d6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 -- 172.21.15.5:0/1989996360 shutdown_connections 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.571+0000 7fcb5acfc080 1 -- 172.21.15.5:0/1989996360 wait complete. 2024-05-08T23:28:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.572+0000 7fcb5acfc080 1 Processor -- start 2024-05-08T23:28:28.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.572+0000 7fcb5acfc080 1 -- start start 2024-05-08T23:28:28.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.572+0000 7fcb5acfc080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9705d0 0x55cfea8ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.572+0000 7fcb5acfc080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9709b0 0x55cfea96cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.572+0000 7fcb5acfc080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 0x55cfea8e94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb5acfc080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfea7ea370 con 0x55cfea9705d0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb5acfc080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfea7e1680 con 0x55cfea9711d0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb5acfc080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfea971950 con 0x55cfea9709b0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9705d0 0x55cfea8ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9705d0 0x55cfea8ec3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47428/0 (socket says 172.21.15.5:47428) 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 -- 172.21.15.5:0/3120475830 learned_addr learned my addr 172.21.15.5:0/3120475830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb515cf700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 0x55cfea8e94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb505cd700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9709b0 0x55cfea96cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 -- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9709b0 msgr2=0x55cfea96cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cfea9709b0 0x55cfea96cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 -- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 msgr2=0x55cfea8e94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfea9711d0 0x55cfea8e94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 -- 172.21.15.5:0/3120475830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44005d20 con 0x55cfea9705d0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb50dce700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9705d0 0x55cfea8ec3a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fcb400017a0 tx=0x7fcb4000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb4e5c9700 1 -- 172.21.15.5:0/3120475830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40014070 con 0x55cfea9705d0 2024-05-08T23:28:28.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb4e5c9700 1 -- 172.21.15.5:0/3120475830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4000e720 con 0x55cfea9705d0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cfea7ec930 con 0x55cfea9705d0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb4e5c9700 1 -- 172.21.15.5:0/3120475830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000eab0 con 0x55cfea9705d0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.573+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfea971d70 con 0x55cfea9705d0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.575+0000 7fcb4e5c9700 1 -- 172.21.15.5:0/3120475830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcb4000ec10 con 0x55cfea9705d0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.575+0000 7fcb4e5c9700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb3c0695e0 0x7fcb3c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.575+0000 7fcb4e5c9700 1 -- 172.21.15.5:0/3120475830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb40004040 con 0x55cfea9705d0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.576+0000 7fcb505cd700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb3c0695e0 0x7fcb3c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.576+0000 7fcb5acfc080 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55cfea8eef60 0x55cfeaa16fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.576+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- osd_op(unknown.0.0:1 14.5 14:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x55cfea96d280 con 0x55cfea8eef60 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.576+0000 7fcb515cf700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55cfea8eef60 0x55cfeaa16fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.576+0000 7fcb505cd700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb3c0695e0 0x7fcb3c06baa0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x55cfea8ec8e0 tx=0x7fcb4400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.576+0000 7fcb515cf700 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55cfea8eef60 0x55cfeaa16fd0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb515cf700 1 -- 172.21.15.5:0/3120475830 <== osd.0 v2:172.21.15.5:6824/47134 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v96'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb48025040 con 0x55cfea8eef60 2024-05-08T23:28:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55cfea8eef60 msgr2=0x55cfeaa16fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55cfea8eef60 0x55cfeaa16fd0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb3c0695e0 msgr2=0x7fcb3c06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcb3c0695e0 0x7fcb3c06baa0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x55cfea8ec8e0 tx=0x7fcb4400c040 comp rx=0 tx=0).stop 2024-05-08T23:28:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9705d0 msgr2=0x55cfea8ec3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 --2- 172.21.15.5:0/3120475830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cfea9705d0 0x55cfea8ec3a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fcb400017a0 tx=0x7fcb4000f730 comp rx=0 tx=0).stop 2024-05-08T23:28:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb515cf700 1 -- 172.21.15.5:0/3120475830 reap_dead start 2024-05-08T23:28:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 shutdown_connections 2024-05-08T23:28:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 >> 172.21.15.5:0/3120475830 conn(0x55cfea8d4d60 msgr2=0x55cfea8d6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 shutdown_connections 2024-05-08T23:28:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.578+0000 7fcb5acfc080 1 -- 172.21.15.5:0/3120475830 wait complete. 2024-05-08T23:28:28.581 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-05-08T23:28:28.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8b44d4080 1 Processor -- start 2024-05-08T23:28:28.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8b44d4080 1 -- start start 2024-05-08T23:28:28.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8b44d4080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d03fa0 0x5609b0ddaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8b44d4080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d32610 0x5609b0d04af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8b44d4080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d4a310 0x5609b0d05030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8b44d4080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b0ca91a0 con 0x5609b0d32610 2024-05-08T23:28:28.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8b44d4080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b0c446d0 con 0x5609b0d03fa0 2024-05-08T23:28:28.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8b44d4080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609b0c44370 con 0x5609b0d4a310 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.631+0000 7fb8a9da5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d32610 0x5609b0d04af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8aa5a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d03fa0 0x5609b0ddaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a9da5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d32610 0x5609b0d04af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47444/0 (socket says 172.21.15.5:47444) 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a9da5700 1 -- 172.21.15.5:0/1505141196 learned_addr learned my addr 172.21.15.5:0/1505141196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8aada7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d4a310 0x5609b0d05030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a9da5700 1 -- 172.21.15.5:0/1505141196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d4a310 msgr2=0x5609b0d05030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a9da5700 1 --2- 172.21.15.5:0/1505141196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d4a310 0x5609b0d05030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a9da5700 1 -- 172.21.15.5:0/1505141196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d03fa0 msgr2=0x5609b0ddaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a9da5700 1 --2- 172.21.15.5:0/1505141196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d03fa0 0x5609b0ddaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a9da5700 1 -- 172.21.15.5:0/1505141196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a400a1c0 con 0x5609b0d32610 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8aa5a6700 1 --2- 172.21.15.5:0/1505141196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d03fa0 0x5609b0ddaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a9da5700 1 --2- 172.21.15.5:0/1505141196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d32610 0x5609b0d04af0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0002de0 tx=0x7fb8a000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfa97561be36f12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a8da3700 1 -- 172.21.15.5:0/1505141196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a000bd60 con 0x5609b0d32610 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a8da3700 1 -- 172.21.15.5:0/1505141196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0008030 con 0x5609b0d32610 2024-05-08T23:28:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.632+0000 7fb8a8da3700 1 -- 172.21.15.5:0/1505141196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a000f640 con 0x5609b0d32610 2024-05-08T23:28:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 -- 172.21.15.5:0/1505141196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d32610 msgr2=0x5609b0d04af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 --2- 172.21.15.5:0/1505141196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d32610 0x5609b0d04af0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0002de0 tx=0x7fb8a000b4c0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 -- 172.21.15.5:0/1505141196 shutdown_connections 2024-05-08T23:28:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 --2- 172.21.15.5:0/1505141196 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d4a310 0x5609b0d05030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 --2- 172.21.15.5:0/1505141196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d03fa0 0x5609b0ddaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 --2- 172.21.15.5:0/1505141196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d32610 0x5609b0d04af0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 -- 172.21.15.5:0/1505141196 >> 172.21.15.5:0/1505141196 conn(0x5609b0d02ba0 msgr2=0x5609b0d4c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 -- 172.21.15.5:0/1505141196 shutdown_connections 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 -- 172.21.15.5:0/1505141196 wait complete. 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 Processor -- start 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.633+0000 7fb8b44d4080 1 -- start start 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8b44d4080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d03fa0 0x5609b0e70050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8b44d4080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d32610 0x5609b0dca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8b44d4080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d4a310 0x5609b0dcf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8b44d4080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b0c3b680 con 0x5609b0d03fa0 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8b44d4080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b0ddb740 con 0x5609b0d4a310 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8b44d4080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609b0dcab90 con 0x5609b0d32610 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8aa5a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d03fa0 0x5609b0e70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8aa5a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d03fa0 0x5609b0e70050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47446/0 (socket says 172.21.15.5:47446) 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8aa5a6700 1 -- 172.21.15.5:0/3858049799 learned_addr learned my addr 172.21.15.5:0/3858049799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8aada7700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d4a310 0x5609b0dcf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.634+0000 7fb8a9da5700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d32610 0x5609b0dca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb8aa5a6700 1 -- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d32610 msgr2=0x5609b0dca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb8aa5a6700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5609b0d32610 0x5609b0dca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb8aa5a6700 1 -- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d4a310 msgr2=0x5609b0dcf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb8aa5a6700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5609b0d4a310 0x5609b0dcf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb8aa5a6700 1 -- 172.21.15.5:0/3858049799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c005da0 con 0x5609b0d03fa0 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb8aa5a6700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d03fa0 0x5609b0e70050 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0063b0 tx=0x7fb89c0066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb89b7fe700 1 -- 172.21.15.5:0/3858049799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c010040 con 0x5609b0d03fa0 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb89b7fe700 1 -- 172.21.15.5:0/3858049799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c015480 con 0x5609b0d03fa0 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a000a040 con 0x5609b0d03fa0 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb89b7fe700 1 -- 172.21.15.5:0/3858049799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c015830 con 0x5609b0d03fa0 2024-05-08T23:28:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.635+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609b0c538a0 con 0x5609b0d03fa0 2024-05-08T23:28:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.637+0000 7fb89b7fe700 1 -- 172.21.15.5:0/3858049799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb89c005020 con 0x5609b0d03fa0 2024-05-08T23:28:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.637+0000 7fb89b7fe700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb88c069610 0x7fb88c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.637+0000 7fb89b7fe700 1 -- 172.21.15.5:0/3858049799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb89c087280 con 0x5609b0d03fa0 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.637+0000 7fb8a9da5700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb88c069610 0x7fb88c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.637+0000 7fb8b44d4080 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x5609b0dcf9a0 0x5609b0dcd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.637+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 --> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] -- osd_op(unknown.0.0:1 14.9 14:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5609b0dcde60 con 0x5609b0dcf9a0 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.638+0000 7fb8aada7700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x5609b0dcf9a0 0x5609b0dcd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.638+0000 7fb8a9da5700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb88c069610 0x7fb88c06bad0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x5609b0d043c0 tx=0x7fb8a001d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.638+0000 7fb8aada7700 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x5609b0dcf9a0 0x5609b0dcd920 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.639+0000 7fb8aada7700 1 -- 172.21.15.5:0/3858049799 <== osd.5 v2:172.21.15.49:6800/47399 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v96'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb8a400eaa0 con 0x5609b0dcf9a0 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.639+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x5609b0dcf9a0 msgr2=0x5609b0dcd920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.639+0000 7fb8b44d4080 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x5609b0dcf9a0 0x5609b0dcd920 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb88c069610 msgr2=0x7fb88c06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8b44d4080 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb88c069610 0x7fb88c06bad0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x5609b0d043c0 tx=0x7fb8a001d040 comp rx=0 tx=0).stop 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d03fa0 msgr2=0x5609b0e70050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8b44d4080 1 --2- 172.21.15.5:0/3858049799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609b0d03fa0 0x5609b0e70050 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0063b0 tx=0x7fb89c0066c0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8aada7700 1 -- 172.21.15.5:0/3858049799 reap_dead start 2024-05-08T23:28:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 shutdown_connections 2024-05-08T23:28:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 >> 172.21.15.5:0/3858049799 conn(0x5609b0d02ba0 msgr2=0x5609b0d47590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 shutdown_connections 2024-05-08T23:28:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.640+0000 7fb8b44d4080 1 -- 172.21.15.5:0/3858049799 wait complete. 2024-05-08T23:28:28.643 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-05-08T23:28:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.692+0000 7f3a9e4eb080 1 Processor -- start 2024-05-08T23:28:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.692+0000 7f3a9e4eb080 1 -- start start 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a9e4eb080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e318820fa0 0x55e31881e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a9e4eb080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318869dc0 0x55e3188e8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a9e4eb080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e3188f3e10 0x55e318866f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a9e4eb080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3187c61a0 con 0x55e318869dc0 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a9e4eb080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3187616d0 con 0x55e318820fa0 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a9e4eb080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e318761370 con 0x55e3188f3e10 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318869dc0 0x55e3188e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a945bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e318820fa0 0x55e31881e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318869dc0 0x55e3188e8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47452/0 (socket says 172.21.15.5:47452) 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 -- 172.21.15.5:0/1178503221 learned_addr learned my addr 172.21.15.5:0/1178503221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a94dbe700 1 --2- 172.21.15.5:0/1178503221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e3188f3e10 0x55e318866f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 -- 172.21.15.5:0/1178503221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e3188f3e10 msgr2=0x55e318866f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 --2- 172.21.15.5:0/1178503221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e3188f3e10 0x55e318866f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 -- 172.21.15.5:0/1178503221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e318820fa0 msgr2=0x55e31881e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 --2- 172.21.15.5:0/1178503221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e318820fa0 0x55e31881e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 -- 172.21.15.5:0/1178503221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88005d20 con 0x55e318869dc0 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a93dbc700 1 --2- 172.21.15.5:0/1178503221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318869dc0 0x55e3188e8430 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800a440 tx=0x7f3a8800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a5b49c0555c2b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a92dba700 1 -- 172.21.15.5:0/1178503221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88008a00 con 0x55e318869dc0 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a92dba700 1 -- 172.21.15.5:0/1178503221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88008bb0 con 0x55e318869dc0 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.693+0000 7f3a92dba700 1 -- 172.21.15.5:0/1178503221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8801c410 con 0x55e318869dc0 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1178503221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318869dc0 msgr2=0x55e3188e8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 --2- 172.21.15.5:0/1178503221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318869dc0 0x55e3188e8430 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800a440 tx=0x7f3a8800d000 comp rx=0 tx=0).stop 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1178503221 shutdown_connections 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 --2- 172.21.15.5:0/1178503221 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e3188f3e10 0x55e318866f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 --2- 172.21.15.5:0/1178503221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e318820fa0 0x55e31881e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 --2- 172.21.15.5:0/1178503221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318869dc0 0x55e3188e8430 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1178503221 >> 172.21.15.5:0/1178503221 conn(0x55e31881fba0 msgr2=0x55e3188f30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1178503221 shutdown_connections 2024-05-08T23:28:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.694+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1178503221 wait complete. 2024-05-08T23:28:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.695+0000 7f3a9e4eb080 1 Processor -- start 2024-05-08T23:28:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.695+0000 7f3a9e4eb080 1 -- start start 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.695+0000 7f3a9e4eb080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318820fa0 0x55e3188dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.695+0000 7f3a9e4eb080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e318869dc0 0x55e3188f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.695+0000 7f3a9e4eb080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3188f3e10 0x55e3188e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.695+0000 7f3a9e4eb080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e318758680 con 0x55e318820fa0 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.695+0000 7f3a9e4eb080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3188f75e0 con 0x55e3188f3e10 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.695+0000 7f3a9e4eb080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3188f2d40 con 0x55e318869dc0 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318820fa0 0x55e3188dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318820fa0 0x55e3188dbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47462/0 (socket says 172.21.15.5:47462) 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 -- 172.21.15.5:0/1599388888 learned_addr learned my addr 172.21.15.5:0/1599388888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a93dbc700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e318869dc0 0x55e3188f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a94dbe700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3188f3e10 0x55e3188e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 -- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e318869dc0 msgr2=0x55e3188f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e318869dc0 0x55e3188f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 -- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3188f3e10 msgr2=0x55e3188e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3188f3e10 0x55e3188e8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 -- 172.21.15.5:0/1599388888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e31878aa40 con 0x55e318820fa0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a945bd700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318820fa0 0x55e3188dbf50 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f3a84001d60 tx=0x7f3a8400c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a91db8700 1 -- 172.21.15.5:0/1599388888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400f450 con 0x55e318820fa0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a91db8700 1 -- 172.21.15.5:0/1599388888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8400f5b0 con 0x55e318820fa0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e3187708a0 con 0x55e318820fa0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a91db8700 1 -- 172.21.15.5:0/1599388888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400f960 con 0x55e318820fa0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.696+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e318763ed0 con 0x55e318820fa0 2024-05-08T23:28:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.698+0000 7f3a91db8700 1 -- 172.21.15.5:0/1599388888 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a84015020 con 0x55e318820fa0 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.698+0000 7f3a91db8700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a800695e0 0x7f3a8006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.698+0000 7f3a91db8700 1 -- 172.21.15.5:0/1599388888 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a84088360 con 0x55e318820fa0 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.698+0000 7f3a93dbc700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a800695e0 0x7f3a8006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.698+0000 7f3a9e4eb080 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55e3188f7f70 0x55e3188e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.698+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- osd_op(unknown.0.0:1 14.5 14:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x55e3188e8b70 con 0x55e3188f7f70 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.699+0000 7f3a94dbe700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55e3188f7f70 0x55e3188e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.699+0000 7f3a93dbc700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a800695e0 0x7f3a8006baa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3a88023000 tx=0x7f3a88008000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.699+0000 7f3a94dbe700 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55e3188f7f70 0x55e3188e42b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a94dbe700 1 -- 172.21.15.5:0/1599388888 <== osd.0 v2:172.21.15.5:6824/47134 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v96'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3a8c002ac0 con 0x55e3188f7f70 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55e3188f7f70 msgr2=0x55e3188e42b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55e3188f7f70 0x55e3188e42b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a800695e0 msgr2=0x7f3a8006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3a800695e0 0x7f3a8006baa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3a88023000 tx=0x7f3a88008000 comp rx=0 tx=0).stop 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318820fa0 msgr2=0x55e3188dbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 --2- 172.21.15.5:0/1599388888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e318820fa0 0x55e3188dbf50 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f3a84001d60 tx=0x7f3a8400c640 comp rx=0 tx=0).stop 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a94dbe700 1 -- 172.21.15.5:0/1599388888 reap_dead start 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 shutdown_connections 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 >> 172.21.15.5:0/1599388888 conn(0x55e31881fba0 msgr2=0x55e3188f1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 shutdown_connections 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.700+0000 7f3a9e4eb080 1 -- 172.21.15.5:0/1599388888 wait complete. 2024-05-08T23:28:28.705 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-05-08T23:28:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.754+0000 7fe0f5fbf080 1 Processor -- start 2024-05-08T23:28:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.754+0000 7fe0f5fbf080 1 -- start start 2024-05-08T23:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0f5fbf080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f0300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0f5fbf080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778efb7480 0x55778f03fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0f5fbf080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778f043150 0x55778f040760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0f5fbf080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778ef151a0 con 0x55778ef98d80 2024-05-08T23:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0f5fbf080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778eeb06d0 con 0x55778f043150 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0f5fbf080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55778eeb0370 con 0x55778efb7480 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0ec091700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f0300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0ec091700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f0300d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47470/0 (socket says 172.21.15.5:47470) 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0ec091700 1 -- 172.21.15.5:0/4168360423 learned_addr learned my addr 172.21.15.5:0/4168360423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0ec892700 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778f043150 0x55778f040760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0eb890700 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778efb7480 0x55778f03fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.755+0000 7fe0ec091700 1 -- 172.21.15.5:0/4168360423 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778efb7480 msgr2=0x55778f03fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0ec091700 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778efb7480 0x55778f03fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0ec091700 1 -- 172.21.15.5:0/4168360423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778f043150 msgr2=0x55778f040760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0ec091700 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778f043150 0x55778f040760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0ec091700 1 -- 172.21.15.5:0/4168360423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e400a100 con 0x55778ef98d80 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0ec091700 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f0300d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4001c00 tx=0x7fe0d400b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cbc9fa6145a2ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0ea88e700 1 -- 172.21.15.5:0/4168360423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400bdb0 con 0x55778ef98d80 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0ea88e700 1 -- 172.21.15.5:0/4168360423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d4001d50 con 0x55778ef98d80 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0ea88e700 1 -- 172.21.15.5:0/4168360423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400e6b0 con 0x55778ef98d80 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/4168360423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 msgr2=0x55778f0300d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0f5fbf080 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f0300d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4001c00 tx=0x7fe0d400b5d0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/4168360423 shutdown_connections 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0f5fbf080 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778f043150 0x55778f040760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0f5fbf080 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f0300d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0f5fbf080 1 --2- 172.21.15.5:0/4168360423 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778efb7480 0x55778f03fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/4168360423 >> 172.21.15.5:0/4168360423 conn(0x55778ef9ad60 msgr2=0x55778ef6ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.756+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/4168360423 shutdown_connections 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.757+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/4168360423 wait complete. 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.757+0000 7fe0f5fbf080 1 Processor -- start 2024-05-08T23:28:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.757+0000 7fe0f5fbf080 1 -- start start 2024-05-08T23:28:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0f5fbf080 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f02dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0f5fbf080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778efb7480 0x55778f040760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0f5fbf080 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778f043150 0x55778f040ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0f5fbf080 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778eea7680 con 0x55778ef98d80 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0f5fbf080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778f035ee0 con 0x55778efb7480 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0f5fbf080 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55778f035260 con 0x55778f043150 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec091700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f02dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec091700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f02dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47474/0 (socket says 172.21.15.5:47474) 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec091700 1 -- 172.21.15.5:0/1959166638 learned_addr learned my addr 172.21.15.5:0/1959166638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0eb890700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778efb7480 0x55778f040760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec892700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778f043150 0x55778f040ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec091700 1 -- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778f043150 msgr2=0x55778f040ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec091700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55778f043150 0x55778f040ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec091700 1 -- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778efb7480 msgr2=0x55778f040760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec091700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55778efb7480 0x55778f040760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.758+0000 7fe0ec091700 1 -- 172.21.15.5:0/1959166638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d8001e10 con 0x55778ef98d80 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.759+0000 7fe0ec091700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f02dee0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x55778f034430 tx=0x7fe0d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.759+0000 7fe0e988c700 1 -- 172.21.15.5:0/1959166638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d401e070 con 0x55778ef98d80 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.759+0000 7fe0e988c700 1 -- 172.21.15.5:0/1959166638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d4000fe0 con 0x55778ef98d80 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.759+0000 7fe0e988c700 1 -- 172.21.15.5:0/1959166638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40011d0 con 0x55778ef98d80 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.759+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55778eebf8a0 con 0x55778ef98d80 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.759+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55778eeb2ed0 con 0x55778ef98d80 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.760+0000 7fe0e988c700 1 -- 172.21.15.5:0/1959166638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe0d402c440 con 0x55778ef98d80 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.761+0000 7fe0e988c700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0cc0695e0 0x7fe0cc06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.761+0000 7fe0e988c700 1 -- 172.21.15.5:0/1959166638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0d4089560 con 0x55778ef98d80 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.761+0000 7fe0eb890700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0cc0695e0 0x7fe0cc06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.761+0000 7fe0f5fbf080 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55778f034ce0 0x55778f0d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.761+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- osd_op(unknown.0.0:1 14.1d 14:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x55778f0d68f0 con 0x55778f034ce0 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.761+0000 7fe0ec892700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55778f034ce0 0x55778f0d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.762+0000 7fe0eb890700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0cc0695e0 0x7fe0cc06baa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55778f047a00 tx=0x7fe0dc00c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.762+0000 7fe0ec892700 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55778f034ce0 0x55778f0d63b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0ec892700 1 -- 172.21.15.5:0/1959166638 <== osd.0 v2:172.21.15.5:6824/47134 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v96'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe0e400f590 con 0x55778f034ce0 2024-05-08T23:28:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55778f034ce0 msgr2=0x55778f0d63b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x55778f034ce0 0x55778f0d63b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0cc0695e0 msgr2=0x7fe0cc06baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe0cc0695e0 0x7fe0cc06baa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55778f047a00 tx=0x7fe0dc00c040 comp rx=0 tx=0).stop 2024-05-08T23:28:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 msgr2=0x55778f02dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 --2- 172.21.15.5:0/1959166638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ef98d80 0x55778f02dee0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x55778f034430 tx=0x7fe0d4000bc0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0ec892700 1 -- 172.21.15.5:0/1959166638 reap_dead start 2024-05-08T23:28:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 shutdown_connections 2024-05-08T23:28:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 >> 172.21.15.5:0/1959166638 conn(0x55778ef9ad60 msgr2=0x55778f041860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 shutdown_connections 2024-05-08T23:28:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.763+0000 7fe0f5fbf080 1 -- 172.21.15.5:0/1959166638 wait complete. 2024-05-08T23:28:28.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub start /d1/d2/d3 recursive,force 2024-05-08T23:28:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.908+0000 7f7d4f57b700 1 Processor -- start 2024-05-08T23:28:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.908+0000 7f7d4f57b700 1 -- start start 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4f57b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7d48161ff0 0x7f7d4806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4f57b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d4806beb0 0x7f7d480ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4f57b700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4806c2e0 con 0x7f7d4815d900 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4f57b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4815dce0 con 0x7f7d4806beb0 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4f57b700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d480ce5b0 con 0x7f7d48161ff0 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4db18700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d4806beb0 0x7f7d480ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4db18700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d4806beb0 0x7f7d480ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47486/0 (socket says 172.21.15.5:47486) 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4db18700 1 -- 172.21.15.5:0/3146435848 learned_addr learned my addr 172.21.15.5:0/3146435848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/3146435848 >> v2:172.21.15.49:3300/0 conn(0x7f7d48161ff0 0x7f7d4806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4db18700 1 -- 172.21.15.5:0/3146435848 >> v2:172.21.15.49:3300/0 conn(0x7f7d48161ff0 msgr2=0x7f7d4806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4db18700 1 --2- 172.21.15.5:0/3146435848 >> v2:172.21.15.49:3300/0 conn(0x7f7d48161ff0 0x7f7d4806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4db18700 1 -- 172.21.15.5:0/3146435848 >> v1:172.21.15.49:6789/0 conn(0x7f7d4815d900 legacy=0x7f7d480cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.909+0000 7f7d4db18700 1 -- 172.21.15.5:0/3146435848 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d48161510 con 0x7f7d4806beb0 2024-05-08T23:28:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d4db18700 1 --2- 172.21.15.5:0/3146435848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4806beb0 0x7f7d480ada00 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7d4806eaa0 tx=0x7f7d4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c8293ae0f88bef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d3ffff700 1 -- 172.21.15.5:0/3146435848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000e070 con 0x7f7d4806beb0 2024-05-08T23:28:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d3ffff700 1 -- 172.21.15.5:0/3146435848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4000b910 con 0x7f7d4806beb0 2024-05-08T23:28:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d3ffff700 1 -- 172.21.15.5:0/3146435848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000bcc0 con 0x7f7d4806beb0 2024-05-08T23:28:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d4f57b700 1 -- 172.21.15.5:0/3146435848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4806beb0 msgr2=0x7f7d480ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/3146435848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4806beb0 0x7f7d480ada00 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7d4806eaa0 tx=0x7f7d4000b330 comp rx=0 tx=0).stop 2024-05-08T23:28:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d4f57b700 1 -- 172.21.15.5:0/3146435848 shutdown_connections 2024-05-08T23:28:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/3146435848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4806beb0 0x7f7d480ada00 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/3146435848 >> v2:172.21.15.49:3300/0 conn(0x7f7d48161ff0 0x7f7d4806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.910+0000 7f7d4f57b700 1 -- 172.21.15.5:0/3146435848 >> 172.21.15.5:0/3146435848 conn(0x7f7d480713c0 msgr2=0x7f7d480717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.911+0000 7f7d4f57b700 1 -- 172.21.15.5:0/3146435848 shutdown_connections 2024-05-08T23:28:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.911+0000 7f7d4f57b700 1 -- 172.21.15.5:0/3146435848 wait complete. 2024-05-08T23:28:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.912+0000 7f7d4f57b700 1 Processor -- start 2024-05-08T23:28:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.912+0000 7f7d4f57b700 1 -- start start 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4f57b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d4806beb0 0x7f7d48149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4f57b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4815d900 0x7f7d48155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4f57b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d48161ff0 0x7f7d480af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4f57b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481543f0 con 0x7f7d4815d900 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4f57b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481546d0 con 0x7f7d4806beb0 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4f57b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481549b0 con 0x7f7d48161ff0 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4815d900 0x7f7d48155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4815d900 0x7f7d48155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47502/0 (socket says 172.21.15.5:47502) 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 -- 172.21.15.5:0/392628634 learned_addr learned my addr 172.21.15.5:0/392628634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4d317700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d4806beb0 0x7f7d48149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4db18700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d48161ff0 0x7f7d480af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 -- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d48161ff0 msgr2=0x7f7d480af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d48161ff0 0x7f7d480af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 -- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d4806beb0 msgr2=0x7f7d48149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d4806beb0 0x7f7d48149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 -- 172.21.15.5:0/392628634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d40009b30 con 0x7f7d4815d900 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.913+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4815d900 0x7f7d48155770 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800e870 tx=0x7f7d3800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.914+0000 7f7d3dffb700 1 -- 172.21.15.5:0/392628634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38009590 con 0x7f7d4815d900 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.914+0000 7f7d3dffb700 1 -- 172.21.15.5:0/392628634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800ee70 con 0x7f7d4815d900 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.914+0000 7f7d4f57b700 1 -- 172.21.15.5:0/392628634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d48154c90 con 0x7f7d4815d900 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.914+0000 7f7d3dffb700 1 -- 172.21.15.5:0/392628634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800ca50 con 0x7f7d4815d900 2024-05-08T23:28:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.914+0000 7f7d4f57b700 1 -- 172.21.15.5:0/392628634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d48155250 con 0x7f7d4815d900 2024-05-08T23:28:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.915+0000 7f7d3dffb700 1 -- 172.21.15.5:0/392628634 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7d3800cbb0 con 0x7f7d4815d900 2024-05-08T23:28:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.916+0000 7f7d3dffb700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d2c06ac90 0x7f7d2c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.916+0000 7f7d3dffb700 1 -- 172.21.15.5:0/392628634 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d38089370 con 0x7f7d4815d900 2024-05-08T23:28:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.916+0000 7f7d4d317700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d2c06ac90 0x7f7d2c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.917+0000 7f7d4d317700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d2c06ac90 0x7f7d2c06d150 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7d44000f60 tx=0x7f7d44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.918+0000 7f7d5b735b80 1 Processor -- start 2024-05-08T23:28:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.918+0000 7f7d5b735b80 1 -- start start 2024-05-08T23:28:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb566a830 0x55bcb56431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb5648130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb56722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb559e1e0 con 0x55bcb5671e60 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb559bba0 con 0x55bcb5645400 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcb5597490 con 0x55bcb566a830 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb56722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb56722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47512/0 (socket says 172.21.15.5:47512) 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 -- 172.21.15.5:0/4096328260 learned_addr learned my addr 172.21.15.5:0/4096328260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4d317700 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb566a830 0x55bcb56431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb5648130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 -- 172.21.15.5:0/4096328260 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb566a830 msgr2=0x55bcb56431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb566a830 0x55bcb56431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 -- 172.21.15.5:0/4096328260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 msgr2=0x55bcb5648130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb5648130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 -- 172.21.15.5:0/4096328260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcb5418440 con 0x55bcb5671e60 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.919+0000 7f7d4db18700 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb56722e0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7d48066ce0 tx=0x7f7d40010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98245a4d94ba2a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d32ffd700 1 -- 172.21.15.5:0/4096328260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000bc80 con 0x55bcb5671e60 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d32ffd700 1 -- 172.21.15.5:0/4096328260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d400099c0 con 0x55bcb5671e60 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d32ffd700 1 -- 172.21.15.5:0/4096328260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40003c40 con 0x55bcb5671e60 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 -- 172.21.15.5:0/4096328260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 msgr2=0x55bcb56722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb56722e0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7d48066ce0 tx=0x7f7d40010000 comp rx=0 tx=0).stop 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 -- 172.21.15.5:0/4096328260 shutdown_connections 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb56722e0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb566a830 0x55bcb56431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/4096328260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb5648130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 -- 172.21.15.5:0/4096328260 >> 172.21.15.5:0/4096328260 conn(0x55bcb55a52a0 msgr2=0x55bcb568ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 -- 172.21.15.5:0/4096328260 shutdown_connections 2024-05-08T23:28:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.920+0000 7f7d5b735b80 1 -- 172.21.15.5:0/4096328260 wait complete. 2024-05-08T23:28:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.921+0000 7f7d5b735b80 1 -- start start 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.921+0000 7f7d5b735b80 1 Processor -- start 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 0x55bcb569ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb568d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5671e60 0x55bcb568d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb55335f0 con 0x55bcb566a830 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb5533290 con 0x55bcb5671e60 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcb541a730 con 0x55bcb5645400 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d5b735b80 10 client.0 _ll_register_callbacks cb 0x55bcb553cbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d4cb16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb568d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d4cb16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb568d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47524/0 (socket says 172.21.15.5:47524) 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d4cb16700 1 -- 172.21.15.5:0/3229593808 learned_addr learned my addr 172.21.15.5:0/3229593808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d4db18700 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5671e60 0x55bcb568d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.922+0000 7f7d4d317700 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 0x55bcb569ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d4cb16700 1 -- 172.21.15.5:0/3229593808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 msgr2=0x55bcb569ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 0x55bcb569ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d4cb16700 1 -- 172.21.15.5:0/3229593808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5671e60 msgr2=0x55bcb568d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5671e60 0x55bcb568d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d4cb16700 1 -- 172.21.15.5:0/3229593808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4000be70 con 0x55bcb566a830 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb568d140 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7d380600d0 tx=0x7f7d380589c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b84a212334869db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d27fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d27fff700 1 -- 172.21.15.5:0/3229593808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38060600 con 0x55bcb566a830 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d27fff700 1 -- 172.21.15.5:0/3229593808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d3805a070 con 0x55bcb566a830 2024-05-08T23:28:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3229593808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcb54f0300 con 0x55bcb566a830 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d27fff700 1 -- 172.21.15.5:0/3229593808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d38057670 con 0x55bcb566a830 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d5b735b80 10 client.8946 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3229593808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x55bcb53c4bf0 con 0x55bcb566a830 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d27fff700 1 -- 172.21.15.5:0/3229593808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bcb54f0300 con 0x55bcb566a830 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d27fff700 1 -- 172.21.15.5:0/3229593808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f7d38060170 con 0x55bcb566a830 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d5b735b80 10 client.8946 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.923+0000 7f7d5b735b80 10 client.8946 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.924+0000 7f7d5b735b80 20 client.8946 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.924+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55bcb5673000 0x55bcb56733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.924+0000 7f7d5b735b80 4 client.8946 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.924+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3229593808 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcb5616870 con 0x55bcb5673000 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.924+0000 7f7d4d317700 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55bcb5673000 0x55bcb56733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.924+0000 7f7d27fff700 10 client.8946 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.924+0000 7f7d4d317700 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55bcb5673000 0x55bcb56733e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50323a4ae2927af server_cookie=c18678bbc0aa2fb2 in_seq=0 out_seq=0 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d27fff700 1 -- 172.21.15.5:0/3229593808 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bcb5616870 con 0x55bcb5673000 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d27fff700 10 client.8946 handle_command_reply: tid=0 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 client.8946 shutdown 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3229593808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 msgr2=0x55bcb568d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb568d140 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7d380600d0 tx=0x7f7d380589c0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3229593808 shutdown_connections 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55bcb5673000 0x55bcb56733e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5671e60 0x55bcb568d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 0x55bcb569ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d27fff700 0 client.8946 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3229593808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb568d140 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3229593808 >> 172.21.15.5:0/3229593808 conn(0x55bcb55a52a0 msgr2=0x55bcb56413c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3229593808 shutdown_connections 2024-05-08T23:28:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3229593808 wait complete. 2024-05-08T23:28:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.925+0000 7f7d5b735b80 20 client.8946 trim_cache size 0 max 16384 2024-05-08T23:28:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.944+0000 7f7d5b735b80 1 Processor -- start 2024-05-08T23:28:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.944+0000 7f7d5b735b80 1 -- start start 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 0x55bcb5669e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb566a830 0x55bcb56aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb566efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb5646e70 con 0x55bcb5671e60 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb56692f0 con 0x55bcb566a830 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcb56ae940 con 0x55bcb5645400 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb566efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb566efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47540/0 (socket says 172.21.15.5:47540) 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 -- 172.21.15.5:0/1258658027 learned_addr learned my addr 172.21.15.5:0/1258658027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb566a830 0x55bcb56aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4d317700 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 0x55bcb5669e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 -- 172.21.15.5:0/1258658027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 msgr2=0x55bcb5669e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 0x55bcb5669e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 -- 172.21.15.5:0/1258658027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb566a830 msgr2=0x55bcb56aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb566a830 0x55bcb56aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 -- 172.21.15.5:0/1258658027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcb555d5f0 con 0x55bcb5671e60 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d4db18700 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb566efd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009fd0 tx=0x7f7d40009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9654fc0df18e6bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d337fe700 1 -- 172.21.15.5:0/1258658027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4002a040 con 0x55bcb5671e60 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d337fe700 1 -- 172.21.15.5:0/1258658027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4000bca0 con 0x55bcb5671e60 2024-05-08T23:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.945+0000 7f7d337fe700 1 -- 172.21.15.5:0/1258658027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40003c60 con 0x55bcb5671e60 2024-05-08T23:28:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 -- 172.21.15.5:0/1258658027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 msgr2=0x55bcb566efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb566efd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009fd0 tx=0x7f7d40009810 comp rx=0 tx=0).stop 2024-05-08T23:28:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 -- 172.21.15.5:0/1258658027 shutdown_connections 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb5671e60 0x55bcb566efd0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5645400 0x55bcb5669e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/1258658027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb566a830 0x55bcb56aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 -- 172.21.15.5:0/1258658027 >> 172.21.15.5:0/1258658027 conn(0x55bcb55a52a0 msgr2=0x55bcb563e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 -- 172.21.15.5:0/1258658027 shutdown_connections 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.946+0000 7f7d5b735b80 1 -- 172.21.15.5:0/1258658027 wait complete. 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.947+0000 7f7d5b735b80 1 -- start start 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.947+0000 7f7d5b735b80 1 Processor -- start 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb56a5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb566d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d5b735b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5671e60 0x55bcb569ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb568fc40 con 0x55bcb566a830 2024-05-08T23:28:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb56457e0 con 0x55bcb5645400 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d5b735b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcb5646bc0 con 0x55bcb5671e60 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d5b735b80 10 client.0 _ll_register_callbacks cb 0x55bcb553cbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d4cb16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb566d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d4cb16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb566d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47552/0 (socket says 172.21.15.5:47552) 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d4cb16700 1 -- 172.21.15.5:0/3678728153 learned_addr learned my addr 172.21.15.5:0/3678728153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d4d317700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb56a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.948+0000 7f7d4db18700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5671e60 0x55bcb569ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d4cb16700 1 -- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5671e60 msgr2=0x55bcb569ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5671e60 0x55bcb569ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d4cb16700 1 -- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 msgr2=0x55bcb56a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb56a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d4cb16700 1 -- 172.21.15.5:0/3678728153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcb5418440 con 0x55bcb566a830 2024-05-08T23:28:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d4d317700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb56a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d4db18700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5671e60 0x55bcb569ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d4cb16700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb566d390 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7d38054ba0 tx=0x7f7d3801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa70c50819a2642 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d27fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d27fff700 1 -- 172.21.15.5:0/3678728153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d380544c0 con 0x55bcb566a830 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d27fff700 1 -- 172.21.15.5:0/3678728153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d3807a070 con 0x55bcb566a830 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3678728153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d400099c0 con 0x55bcb566a830 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d27fff700 1 -- 172.21.15.5:0/3678728153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d380877e0 con 0x55bcb566a830 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 10 client.8952 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3678728153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x55bcb541cd60 con 0x55bcb566a830 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d27fff700 1 -- 172.21.15.5:0/3678728153 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d400099c0 con 0x55bcb566a830 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d27fff700 1 -- 172.21.15.5:0/3678728153 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f7d38053f70 con 0x55bcb566a830 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 10 client.8952 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 10 client.8952 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 20 client.8952 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55bcb569cba0 0x55bcb56ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 4 client.8952 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.949+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3678728153 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) v1 -- 0x55bcb558ff50 con 0x55bcb569cba0 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.950+0000 7f7d4d317700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55bcb569cba0 0x55bcb56ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.950+0000 7f7d27fff700 10 client.8952 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.950+0000 7f7d4d317700 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55bcb569cba0 0x55bcb56ecbc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7372b8d613c0ae86 server_cookie=8c8e08504f51f47d in_seq=0 out_seq=0 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d27fff700 1 -- 172.21.15.5:0/3678728153 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bcb558ff50 con 0x55bcb569cba0 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d27fff700 10 client.8952 handle_command_reply: tid=0 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 client.8952 shutdown 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3678728153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 msgr2=0x55bcb566d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb566d390 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7d38054ba0 tx=0x7f7d3801e040 comp rx=0 tx=0).stop 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3678728153 shutdown_connections 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55bcb569cba0 0x55bcb56ecbc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55bcb5671e60 0x55bcb569ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bcb5645400 0x55bcb56a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d27fff700 0 client.8952 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 --2- 172.21.15.5:0/3678728153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb566a830 0x55bcb566d390 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3678728153 >> 172.21.15.5:0/3678728153 conn(0x55bcb55a52a0 msgr2=0x55bcb5669e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.957 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:28:28.951+0000 7fa0a7b74700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1) n(v3 rc2024-05-08T23:28:12.325938+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={8161=pAsLsXsFsx/pAsLsXsFsx@5},l=8161 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ab1daee580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.325938+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-05-08T23:28:12.325938+0000 6=0+6)","error_str":""},"return_code":-61} 2024-05-08T23:28:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3678728153 shutdown_connections 2024-05-08T23:28:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 1 -- 172.21.15.5:0/3678728153 wait complete. 2024-05-08T23:28:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f7d5b735b80 20 client.8952 trim_cache size 0 max 16384 2024-05-08T23:28:28.958 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:28:28.951+0000 7f59c8b28700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-05-08T23:28:12.067944+0000 1=0+1) n(v2 rc2024-05-08T23:28:12.325938+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={8161=pAsLsXsFsx/pAsLsXsFsx@4},l=8161 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x557432a06b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.067944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.325938+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.067944+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-05-08T23:28:12.325938+0000 4=0+4)","error_str":""},"return_code":-61} 2024-05-08T23:28:28.959 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:28.959 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-08T23:28:28.959 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:28.959 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-08T23:28:28.959 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:28.959 INFO:tasks.ceph.mds.b.smithi049.stderr:2024-05-08T23:28:28.952+0000 7f7187517700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1) n(v0 rc2024-05-08T23:28:12.068944+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={8161=pAsLsXsFsx/pAsLsXsFsx@4},l=8161 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x564e1f5e7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.068944+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-05-08T23:28:12.068944+0000 2=0+2)","error_str":""},"return_code":-61} 2024-05-08T23:28:28.959 INFO:tasks.ceph.mds.b.smithi049.stderr:2024-05-08T23:28:28.952+0000 7f7187517700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={8161=pAsLsXsFsx/AsLsXsFsx@2},l=8161 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x564e1f5e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-05-08T23:28:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 -- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d2c06ac90 msgr2=0x7f7d2c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d2c06ac90 0x7f7d2c06d150 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7d44000f60 tx=0x7f7d44009040 comp rx=0 tx=0).stop 2024-05-08T23:28:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 -- 172.21.15.5:0/392628634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4815d900 msgr2=0x7f7d48155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4815d900 0x7f7d48155770 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800e870 tx=0x7f7d3800ec30 comp rx=0 tx=0).stop 2024-05-08T23:28:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 -- 172.21.15.5:0/392628634 shutdown_connections 2024-05-08T23:28:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7d2c06ac90 0x7f7d2c06d150 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7d48161ff0 0x7f7d480af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d4806beb0 0x7f7d48149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 --2- 172.21.15.5:0/392628634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d4815d900 0x7f7d48155770 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 -- 172.21.15.5:0/392628634 >> 172.21.15.5:0/392628634 conn(0x7f7d480713c0 msgr2=0x7f7d4806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 -- 172.21.15.5:0/392628634 shutdown_connections 2024-05-08T23:28:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:28.952+0000 7f7d4f57b700 1 -- 172.21.15.5:0/392628634 wait complete. 2024-05-08T23:28:28.961 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-08T23:28:28.953+0000 7fa0adb80700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1) n(v4 rc2024-05-08T23:28:12.325938+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={8161=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55ab1daeeb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.325938+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-05-08T23:28:12.325938+0000 5=0+5)","error_str":""},"return_code":-61} 2024-05-08T23:28:28.961 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-05-08T23:28:28.953+0000 7f59ceb34700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1) n(v1 rc2024-05-08T23:28:12.325938+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={8161=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x557432a07080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.068944+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-05-08T23:28:12.325938+0000 3=0+3)","error_str":""},"return_code":-61} 2024-05-08T23:28:28.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub pause 2024-05-08T23:28:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.106+0000 7f795fac5700 1 Processor -- start 2024-05-08T23:28:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.106+0000 7f795fac5700 1 -- start start 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795fac5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580aea70 0x7f79580b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795fac5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79580d0a60 0x7f79580b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795fac5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795815add0 con 0x7f7958154df0 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795fac5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795815abe0 con 0x7f79580d0a60 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795fac5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795815b070 con 0x7f79580aea70 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795d060700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580aea70 0x7f79580b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795e062700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79580d0a60 0x7f79580b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795d861700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7958154df0 0x7f79580d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:42876/0 (socket says 172.21.15.5:42876) 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795e062700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79580d0a60 0x7f79580b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:60522/0 (socket says 172.21.15.5:60522) 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795d060700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580aea70 0x7f79580b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:49594/0 (socket says 172.21.15.5:49594) 2024-05-08T23:28:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.107+0000 7f795d861700 1 -- 172.21.15.5:0/1616345754 learned_addr learned my addr 172.21.15.5:0/1616345754 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795e062700 1 -- 172.21.15.5:0/1616345754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580aea70 msgr2=0x7f79580b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795e062700 1 --2- 172.21.15.5:0/1616345754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580aea70 0x7f79580b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795e062700 1 -- 172.21.15.5:0/1616345754 >> v1:172.21.15.49:6789/0 conn(0x7f7958154df0 legacy=0x7f79580d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795e062700 1 -- 172.21.15.5:0/1616345754 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795815b350 con 0x7f79580d0a60 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795c85f700 1 -- 172.21.15.5:0/1616345754 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640615610 0 0) 0x7f795815add0 con 0x7f7958154df0 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795d060700 1 --2- 172.21.15.5:0/1616345754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580aea70 0x7f79580b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795e062700 1 --2- 172.21.15.5:0/1616345754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580d0a60 0x7f79580b43b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7950009d90 tx=0x7f795000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2450e433e428f9ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795c85f700 1 -- 172.21.15.5:0/1616345754 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795000e070 con 0x7f79580d0a60 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795c85f700 1 -- 172.21.15.5:0/1616345754 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795000bb20 con 0x7f79580d0a60 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.108+0000 7f795c85f700 1 -- 172.21.15.5:0/1616345754 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950012430 con 0x7f79580d0a60 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.109+0000 7f795fac5700 1 -- 172.21.15.5:0/1616345754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580d0a60 msgr2=0x7f79580b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.109+0000 7f795fac5700 1 --2- 172.21.15.5:0/1616345754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580d0a60 0x7f79580b43b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7950009d90 tx=0x7f795000b330 comp rx=0 tx=0).stop 2024-05-08T23:28:29.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.109+0000 7f795fac5700 1 -- 172.21.15.5:0/1616345754 shutdown_connections 2024-05-08T23:28:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.109+0000 7f795fac5700 1 --2- 172.21.15.5:0/1616345754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79580d0a60 0x7f79580b43b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.109+0000 7f795fac5700 1 --2- 172.21.15.5:0/1616345754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580aea70 0x7f79580b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.109+0000 7f795fac5700 1 -- 172.21.15.5:0/1616345754 >> 172.21.15.5:0/1616345754 conn(0x7f7958099a50 msgr2=0x7f79580b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.109+0000 7f795fac5700 1 -- 172.21.15.5:0/1616345754 shutdown_connections 2024-05-08T23:28:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.109+0000 7f795fac5700 1 -- 172.21.15.5:0/1616345754 wait complete. 2024-05-08T23:28:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.110+0000 7f795fac5700 1 Processor -- start 2024-05-08T23:28:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.110+0000 7f795fac5700 1 -- start start 2024-05-08T23:28:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795fac5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79580aea70 0x7f795814dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795fac5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580d0a60 0x7f795813c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795fac5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958154df0 0x7f795809bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795fac5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795809c370 con 0x7f79580aea70 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795fac5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795809c650 con 0x7f7958154df0 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795fac5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795809c930 con 0x7f79580d0a60 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795d861700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79580aea70 0x7f795814dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795d861700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79580aea70 0x7f795814dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47556/0 (socket says 172.21.15.5:47556) 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795d861700 1 -- 172.21.15.5:0/3282863481 learned_addr learned my addr 172.21.15.5:0/3282863481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795e062700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958154df0 0x7f795809bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795d060700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580d0a60 0x7f795813c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.111+0000 7f795d861700 1 -- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580d0a60 msgr2=0x7f795813c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795d861700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580d0a60 0x7f795813c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795d861700 1 -- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958154df0 msgr2=0x7f795809bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795d861700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958154df0 0x7f795809bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795d861700 1 -- 172.21.15.5:0/3282863481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540094a0 con 0x7f79580aea70 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795e062700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958154df0 0x7f795809bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795d060700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580d0a60 0x7f795813c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795d861700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79580aea70 0x7f795814dcf0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c550 tx=0x7f794c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f794a7fc700 1 -- 172.21.15.5:0/3282863481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c004c80 con 0x7f79580aea70 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f794a7fc700 1 -- 172.21.15.5:0/3282863481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c004e00 con 0x7f79580aea70 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795fac5700 1 -- 172.21.15.5:0/3282863481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7950009a70 con 0x7f79580aea70 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f794a7fc700 1 -- 172.21.15.5:0/3282863481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c007310 con 0x7f79580aea70 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.112+0000 7f795fac5700 1 -- 172.21.15.5:0/3282863481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795809cef0 con 0x7f79580aea70 2024-05-08T23:28:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.114+0000 7f794a7fc700 1 -- 172.21.15.5:0/3282863481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f794c007470 con 0x7f79580aea70 2024-05-08T23:28:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.114+0000 7f794a7fc700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f793c06ac90 0x7f793c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.114+0000 7f794a7fc700 1 -- 172.21.15.5:0/3282863481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f794c0890d0 con 0x7f79580aea70 2024-05-08T23:28:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.114+0000 7f795d060700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f793c06ac90 0x7f793c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.115+0000 7f795d060700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f793c06ac90 0x7f793c06d150 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f795809e5b0 tx=0x7f7954008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.116+0000 7f796bc7fb80 1 Processor -- start 2024-05-08T23:28:29.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.116+0000 7f796bc7fb80 1 -- start start 2024-05-08T23:28:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 0x56414da1b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414d9ee360 0x56414da37240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da183c0 0x56414d9f0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d947140 con 0x56414da183c0 2024-05-08T23:28:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d944b00 con 0x56414d9ee360 2024-05-08T23:28:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56414d9403f0 con 0x56414da1afb0 2024-05-08T23:28:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f795e062700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da183c0 0x56414d9f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f795e062700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da183c0 0x56414d9f0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47570/0 (socket says 172.21.15.5:47570) 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f795d060700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414d9ee360 0x56414da37240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f795e062700 1 -- 172.21.15.5:0/199483533 learned_addr learned my addr 172.21.15.5:0/199483533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.117+0000 7f795d861700 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 0x56414da1b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f795d060700 1 -- 172.21.15.5:0/199483533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 msgr2=0x56414da1b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f795d060700 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 0x56414da1b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f795d060700 1 -- 172.21.15.5:0/199483533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da183c0 msgr2=0x56414d9f0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f795d060700 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da183c0 0x56414d9f0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f795d060700 1 -- 172.21.15.5:0/199483533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414d7c13a0 con 0x56414d9ee360 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f795e062700 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da183c0 0x56414d9f0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f795d060700 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414d9ee360 0x56414da37240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7954010b20 tx=0x7f7954013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0bacc8e0cb190f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f793b7fe700 1 -- 172.21.15.5:0/199483533 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954015b00 con 0x56414d9ee360 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f793b7fe700 1 -- 172.21.15.5:0/199483533 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795401a070 con 0x56414d9ee360 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.118+0000 7f793b7fe700 1 -- 172.21.15.5:0/199483533 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954015e60 con 0x56414d9ee360 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 -- 172.21.15.5:0/199483533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414d9ee360 msgr2=0x56414da37240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414d9ee360 0x56414da37240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7954010b20 tx=0x7f7954013000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 -- 172.21.15.5:0/199483533 shutdown_connections 2024-05-08T23:28:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da183c0 0x56414d9f0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 0x56414da1b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/199483533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414d9ee360 0x56414da37240 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 -- 172.21.15.5:0/199483533 >> 172.21.15.5:0/199483533 conn(0x56414d94e200 msgr2=0x56414da35dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 -- 172.21.15.5:0/199483533 shutdown_connections 2024-05-08T23:28:29.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.119+0000 7f796bc7fb80 1 -- 172.21.15.5:0/199483533 wait complete. 2024-05-08T23:28:29.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.120+0000 7f796bc7fb80 1 -- start start 2024-05-08T23:28:29.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.120+0000 7f796bc7fb80 1 Processor -- start 2024-05-08T23:28:29.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da11040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414d9fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 0x56414da14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d8dc550 con 0x56414d9ee360 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d8dc1f0 con 0x56414da183c0 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56414d7c3690 con 0x56414da1afb0 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f796bc7fb80 10 client.0 _ll_register_callbacks cb 0x56414d8e5b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da11040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da11040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47582/0 (socket says 172.21.15.5:47582) 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 -- 172.21.15.5:0/4060867339 learned_addr learned my addr 172.21.15.5:0/4060867339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d060700 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414d9fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795e062700 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 0x56414da14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 -- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 msgr2=0x56414da14e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 0x56414da14e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 -- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 msgr2=0x56414d9fbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414d9fbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 -- 172.21.15.5:0/4060867339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795815b350 con 0x56414d9ee360 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d060700 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414d9fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f795d861700 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da11040 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c520 tx=0x7f794c057ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf974d1a6e3aa7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.121+0000 7f7938ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f7938ff9700 1 -- 172.21.15.5:0/4060867339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c058350 con 0x56414d9ee360 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f7938ff9700 1 -- 172.21.15.5:0/4060867339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f794c0602e0 con 0x56414d9ee360 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 1 -- 172.21.15.5:0/4060867339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56414d899260 con 0x56414d9ee360 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f7938ff9700 1 -- 172.21.15.5:0/4060867339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f794c0573e0 con 0x56414d9ee360 2024-05-08T23:28:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 10 client.8961 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 1 -- 172.21.15.5:0/4060867339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x7f795400e050 con 0x56414d9ee360 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f7938ff9700 1 -- 172.21.15.5:0/4060867339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56414d899260 con 0x56414d9ee360 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f7938ff9700 1 -- 172.21.15.5:0/4060867339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f794c05fe50 con 0x56414d9ee360 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 10 client.8961 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 10 client.8961 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 20 client.8961 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56414da1a800 0x56414da120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 4 client.8961 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f796bc7fb80 1 -- 172.21.15.5:0/4060867339 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56414d9bf7d0 con 0x56414da1a800 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.122+0000 7f795d060700 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56414da1a800 0x56414da120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f7938ff9700 10 client.8961 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f795d060700 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56414da1a800 0x56414da120f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79e84a0b1b38314 server_cookie=303db64f86221643 in_seq=0 out_seq=0 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f7938ff9700 1 -- 172.21.15.5:0/4060867339 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56414d9bf7d0 con 0x56414da1a800 2024-05-08T23:28:29.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f7938ff9700 10 client.8961 handle_command_reply: tid=0 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 client.8961 shutdown 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 -- 172.21.15.5:0/4060867339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 msgr2=0x56414da11040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da11040 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c520 tx=0x7f794c057ef0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 -- 172.21.15.5:0/4060867339 shutdown_connections 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56414da1a800 0x56414da120f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da1afb0 0x56414da14e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da11040 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f7938ff9700 0 client.8961 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/4060867339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414d9fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.123+0000 7f796bc7fb80 1 -- 172.21.15.5:0/4060867339 >> 172.21.15.5:0/4060867339 conn(0x56414d94e200 msgr2=0x56414da36660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.124+0000 7f796bc7fb80 1 -- 172.21.15.5:0/4060867339 shutdown_connections 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.124+0000 7f796bc7fb80 1 -- 172.21.15.5:0/4060867339 wait complete. 2024-05-08T23:28:29.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.124+0000 7f796bc7fb80 20 client.8961 trim_cache size 0 max 16384 2024-05-08T23:28:29.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.142+0000 7f796bc7fb80 1 Processor -- start 2024-05-08T23:28:29.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.142+0000 7f796bc7fb80 1 -- start start 2024-05-08T23:28:29.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.142+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414d9ee360 0x56414da17a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414da53770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da1afb0 0x56414da14940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414da17f50 con 0x56414da1afb0 2024-05-08T23:28:29.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d9f1bf0 con 0x56414da183c0 2024-05-08T23:28:29.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56414d9f1070 con 0x56414d9ee360 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da1afb0 0x56414da14940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da1afb0 0x56414da14940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47596/0 (socket says 172.21.15.5:47596) 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 -- 172.21.15.5:0/1690380951 learned_addr learned my addr 172.21.15.5:0/1690380951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795d060700 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414da53770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795d861700 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414d9ee360 0x56414da17a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 -- 172.21.15.5:0/1690380951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414d9ee360 msgr2=0x56414da17a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414d9ee360 0x56414da17a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 -- 172.21.15.5:0/1690380951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 msgr2=0x56414da53770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414da53770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 -- 172.21.15.5:0/1690380951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414d76db50 con 0x56414da1afb0 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f795e062700 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da1afb0 0x56414da14940 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f79580676a0 tx=0x7f7950010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8693919b6a2cff09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f793bfff700 1 -- 172.21.15.5:0/1690380951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950022070 con 0x56414da1afb0 2024-05-08T23:28:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f793bfff700 1 -- 172.21.15.5:0/1690380951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7950003c60 con 0x56414da1afb0 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.143+0000 7f793bfff700 1 -- 172.21.15.5:0/1690380951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500125e0 con 0x56414da1afb0 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 -- 172.21.15.5:0/1690380951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da1afb0 msgr2=0x56414da14940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da1afb0 0x56414da14940 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f79580676a0 tx=0x7f7950010000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 -- 172.21.15.5:0/1690380951 shutdown_connections 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414da1afb0 0x56414da14940 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414d9ee360 0x56414da17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/1690380951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da183c0 0x56414da53770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 -- 172.21.15.5:0/1690380951 >> 172.21.15.5:0/1690380951 conn(0x56414d94e200 msgr2=0x56414da36420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 -- 172.21.15.5:0/1690380951 shutdown_connections 2024-05-08T23:28:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.144+0000 7f796bc7fb80 1 -- 172.21.15.5:0/1690380951 wait complete. 2024-05-08T23:28:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.145+0000 7f796bc7fb80 1 -- start start 2024-05-08T23:28:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.145+0000 7f796bc7fb80 1 Processor -- start 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da56190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da183c0 0x56414da98d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f796bc7fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da1afb0 0x56414da53480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414da99c30 con 0x56414d9ee360 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414da56ac0 con 0x56414da1afb0 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f796bc7fb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56414da1d760 con 0x56414da183c0 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f796bc7fb80 10 client.0 _ll_register_callbacks cb 0x56414d8e5b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da56190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da56190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47608/0 (socket says 172.21.15.5:47608) 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 -- 172.21.15.5:0/3867710680 learned_addr learned my addr 172.21.15.5:0/3867710680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795e062700 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da1afb0 0x56414da53480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 -- 172.21.15.5:0/3867710680 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da183c0 msgr2=0x56414da98d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da183c0 0x56414da98d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 -- 172.21.15.5:0/3867710680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da1afb0 msgr2=0x56414da53480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da1afb0 0x56414da53480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 -- 172.21.15.5:0/3867710680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7950003a20 con 0x56414d9ee360 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795e062700 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da1afb0 0x56414da53480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f795d861700 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da56190 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f794c057bf0 tx=0x7f794c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3750c830d3cbad34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.146+0000 7f7938ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f7938ff9700 1 -- 172.21.15.5:0/3867710680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00e040 con 0x56414d9ee360 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f7938ff9700 1 -- 172.21.15.5:0/3867710680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f794c0602e0 con 0x56414d9ee360 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 1 -- 172.21.15.5:0/3867710680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f795401a070 con 0x56414d9ee360 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f7938ff9700 1 -- 172.21.15.5:0/3867710680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f794c007720 con 0x56414d9ee360 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 10 client.8967 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 1 -- 172.21.15.5:0/3867710680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x56414d906550 con 0x56414d9ee360 2024-05-08T23:28:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f7938ff9700 1 -- 172.21.15.5:0/3867710680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795401a070 con 0x56414d9ee360 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f7938ff9700 1 -- 172.21.15.5:0/3867710680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f794c058150 con 0x56414d9ee360 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 10 client.8967 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 10 client.8967 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 20 client.8967 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56414da539c0 0x56414d9faab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 4 client.8967 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f796bc7fb80 1 -- 172.21.15.5:0/3867710680 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x56414d938eb0 con 0x56414da539c0 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.147+0000 7f795d060700 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56414da539c0 0x56414d9faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f795d060700 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56414da539c0 0x56414d9faab0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0dea52d2ee51c55 server_cookie=82cca254063a39fe in_seq=0 out_seq=0 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f7938ff9700 10 client.8967 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f7938ff9700 1 -- 172.21.15.5:0/3867710680 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56414d938eb0 con 0x56414da539c0 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f7938ff9700 10 client.8967 handle_command_reply: tid=0 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 client.8967 shutdown 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 -- 172.21.15.5:0/3867710680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 msgr2=0x56414da56190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da56190 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f794c057bf0 tx=0x7f794c000ed0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 -- 172.21.15.5:0/3867710680 shutdown_connections 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56414da539c0 0x56414d9faab0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414da1afb0 0x56414da53480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414d9ee360 0x56414da56190 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f7938ff9700 0 client.8967 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 --2- 172.21.15.5:0/3867710680 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56414da183c0 0x56414da98d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 -- 172.21.15.5:0/3867710680 >> 172.21.15.5:0/3867710680 conn(0x56414d94e200 msgr2=0x56414da14940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 -- 172.21.15.5:0/3867710680 shutdown_connections 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 1 -- 172.21.15.5:0/3867710680 wait complete. 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.148+0000 7f796bc7fb80 20 client.8967 trim_cache size 0 max 16384 2024-05-08T23:28:29.154 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:29.155 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:28:29.155 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:29.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 -- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f793c06ac90 msgr2=0x7f793c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f793c06ac90 0x7f793c06d150 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f795809e5b0 tx=0x7f7954008040 comp rx=0 tx=0).stop 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 -- 172.21.15.5:0/3282863481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79580aea70 msgr2=0x7f795814dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79580aea70 0x7f795814dcf0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c550 tx=0x7f794c00c910 comp rx=0 tx=0).stop 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 -- 172.21.15.5:0/3282863481 shutdown_connections 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f793c06ac90 0x7f793c06d150 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7958154df0 0x7f795809bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79580aea70 0x7f795814dcf0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 --2- 172.21.15.5:0/3282863481 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f79580d0a60 0x7f795813c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 -- 172.21.15.5:0/3282863481 >> 172.21.15.5:0/3282863481 conn(0x7f7958099a50 msgr2=0x7f795806e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 -- 172.21.15.5:0/3282863481 shutdown_connections 2024-05-08T23:28:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.149+0000 7f795fac5700 1 -- 172.21.15.5:0/3282863481 wait complete. 2024-05-08T23:28:29.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 scrub status 2024-05-08T23:28:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.300+0000 7f495e4d4700 1 Processor -- start 2024-05-08T23:28:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.301+0000 7f495e4d4700 1 -- start start 2024-05-08T23:28:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.301+0000 7f495e4d4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f49580cc590 0x7f495806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.301+0000 7f495e4d4700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f495806f7c0 0x7f495806e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.301+0000 7f495e4d4700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49580cb160 con 0x7f495815db00 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.301+0000 7f495e4d4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495806fc50 con 0x7f49580cc590 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.301+0000 7f495e4d4700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495815dee0 con 0x7f495806f7c0 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f49577fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f49580cc590 0x7f495806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f49577fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f49580cc590 0x7f495806f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47620/0 (socket says 172.21.15.5:47620) 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f49577fe700 1 -- 172.21.15.5:0/2073661215 learned_addr learned my addr 172.21.15.5:0/2073661215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f495ca71700 1 --2- 172.21.15.5:0/2073661215 >> v2:172.21.15.49:3300/0 conn(0x7f495806f7c0 0x7f495806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f49577fe700 1 -- 172.21.15.5:0/2073661215 >> v2:172.21.15.49:3300/0 conn(0x7f495806f7c0 msgr2=0x7f495806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f49577fe700 1 --2- 172.21.15.5:0/2073661215 >> v2:172.21.15.49:3300/0 conn(0x7f495806f7c0 0x7f495806e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f49577fe700 1 -- 172.21.15.5:0/2073661215 >> v1:172.21.15.49:6789/0 conn(0x7f495815db00 legacy=0x7f49580caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f49577fe700 1 -- 172.21.15.5:0/2073661215 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495806ad20 con 0x7f49580cc590 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f49577fe700 1 --2- 172.21.15.5:0/2073661215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f495806f280 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f495806ba20 tx=0x7f4940009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1243c21bf3eac47b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f4956ffd700 1 -- 172.21.15.5:0/2073661215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940011830 con 0x7f49580cc590 2024-05-08T23:28:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f4956ffd700 1 -- 172.21.15.5:0/2073661215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940011990 con 0x7f49580cc590 2024-05-08T23:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.302+0000 7f4956ffd700 1 -- 172.21.15.5:0/2073661215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940011d20 con 0x7f49580cc590 2024-05-08T23:28:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.303+0000 7f495e4d4700 1 -- 172.21.15.5:0/2073661215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 msgr2=0x7f495806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.303+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2073661215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f495806f280 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f495806ba20 tx=0x7f4940009c00 comp rx=0 tx=0).stop 2024-05-08T23:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.303+0000 7f495e4d4700 1 -- 172.21.15.5:0/2073661215 shutdown_connections 2024-05-08T23:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.303+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2073661215 >> v2:172.21.15.49:3300/0 conn(0x7f495806f7c0 0x7f495806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.303+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2073661215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f495806f280 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.303+0000 7f495e4d4700 1 -- 172.21.15.5:0/2073661215 >> 172.21.15.5:0/2073661215 conn(0x7f4958099a50 msgr2=0x7f49580d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.303+0000 7f495e4d4700 1 -- 172.21.15.5:0/2073661215 shutdown_connections 2024-05-08T23:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.303+0000 7f495e4d4700 1 -- 172.21.15.5:0/2073661215 wait complete. 2024-05-08T23:28:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.305+0000 7f495e4d4700 1 Processor -- start 2024-05-08T23:28:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.305+0000 7f495e4d4700 1 -- start start 2024-05-08T23:28:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f495e4d4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495806f7c0 0x7f4958156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f495e4d4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f49580afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f495e4d4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f495815db00 0x7f49581f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f495e4d4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4958157400 con 0x7f49580cc590 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f495e4d4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49580ca070 con 0x7f495806f7c0 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f495e4d4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49581f3ac0 con 0x7f495815db00 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f49577fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f49580afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f49577fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f49580afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47630/0 (socket says 172.21.15.5:47630) 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f49577fe700 1 -- 172.21.15.5:0/2087801695 learned_addr learned my addr 172.21.15.5:0/2087801695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f4957fff700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495806f7c0 0x7f4958156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f495ca71700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f495815db00 0x7f49581f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f49577fe700 1 -- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f495815db00 msgr2=0x7f49581f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f49577fe700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f495815db00 0x7f49581f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f49577fe700 1 -- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495806f7c0 msgr2=0x7f4958156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f49577fe700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495806f7c0 0x7f4958156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.306+0000 7f49577fe700 1 -- 172.21.15.5:0/2087801695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940009550 con 0x7f49580cc590 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.307+0000 7f49577fe700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f49580afd60 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4940009870 tx=0x7f4940004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.307+0000 7f4954ff9700 1 -- 172.21.15.5:0/2087801695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49400045b0 con 0x7f49580cc590 2024-05-08T23:28:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.307+0000 7f4954ff9700 1 -- 172.21.15.5:0/2087801695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494001f070 con 0x7f49580cc590 2024-05-08T23:28:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.307+0000 7f495e4d4700 1 -- 172.21.15.5:0/2087801695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49581f3da0 con 0x7f49580cc590 2024-05-08T23:28:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.307+0000 7f4954ff9700 1 -- 172.21.15.5:0/2087801695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49400038a0 con 0x7f49580cc590 2024-05-08T23:28:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.307+0000 7f495e4d4700 1 -- 172.21.15.5:0/2087801695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49580b44e0 con 0x7f49580cc590 2024-05-08T23:28:29.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.308+0000 7f4954ff9700 1 -- 172.21.15.5:0/2087801695 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4940021030 con 0x7f49580cc590 2024-05-08T23:28:29.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.309+0000 7f4954ff9700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f493806ac90 0x7f493806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.309+0000 7f4954ff9700 1 -- 172.21.15.5:0/2087801695 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f494008a960 con 0x7f49580cc590 2024-05-08T23:28:29.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.309+0000 7f4957fff700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f493806ac90 0x7f493806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.310+0000 7f4957fff700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f493806ac90 0x7f493806d150 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4948000f60 tx=0x7f4948009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.311+0000 7f496a68eb80 1 Processor -- start 2024-05-08T23:28:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.311+0000 7f496a68eb80 1 -- start start 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x5602523b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f496a68eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025230e150 con 0x5602523da7a0 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f496a68eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025230bb10 con 0x5602523b5370 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f496a68eb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x560252307400 con 0x5602523e1dd0 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f4957fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x5602523b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f4957fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x5602523b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47646/0 (socket says 172.21.15.5:47646) 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f4957fff700 1 -- 172.21.15.5:0/175586990 learned_addr learned my addr 172.21.15.5:0/175586990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f49577fe700 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f495ca71700 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f4957fff700 1 -- 172.21.15.5:0/175586990 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 msgr2=0x5602523e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f4957fff700 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f4957fff700 1 -- 172.21.15.5:0/175586990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 msgr2=0x5602523b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f4957fff700 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.312+0000 7f4957fff700 1 -- 172.21.15.5:0/175586990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602521883b0 con 0x5602523da7a0 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.313+0000 7f4957fff700 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x5602523b3110 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4948005330 tx=0x7f4948011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c75d2fd8507149f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.313+0000 7f494dffb700 1 -- 172.21.15.5:0/175586990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49480050d0 con 0x5602523da7a0 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.313+0000 7f494dffb700 1 -- 172.21.15.5:0/175586990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948003dd0 con 0x5602523da7a0 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.313+0000 7f494dffb700 1 -- 172.21.15.5:0/175586990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948021650 con 0x5602523da7a0 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.313+0000 7f496a68eb80 1 -- 172.21.15.5:0/175586990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 msgr2=0x5602523b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.313+0000 7f496a68eb80 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x5602523b3110 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4948005330 tx=0x7f4948011000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.314+0000 7f496a68eb80 1 -- 172.21.15.5:0/175586990 shutdown_connections 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.314+0000 7f496a68eb80 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.314+0000 7f496a68eb80 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x5602523b3110 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.314+0000 7f496a68eb80 1 --2- 172.21.15.5:0/175586990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.314+0000 7f496a68eb80 1 -- 172.21.15.5:0/175586990 >> 172.21.15.5:0/175586990 conn(0x560252315210 msgr2=0x5602523fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.314+0000 7f496a68eb80 1 -- 172.21.15.5:0/175586990 shutdown_connections 2024-05-08T23:28:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.314+0000 7f496a68eb80 1 -- 172.21.15.5:0/175586990 wait complete. 2024-05-08T23:28:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.314+0000 7f496a68eb80 1 -- start start 2024-05-08T23:28:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.315+0000 7f496a68eb80 1 Processor -- start 2024-05-08T23:28:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523da7a0 0x5602523e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523e1dd0 0x5602523e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f496a68eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602522a3560 con 0x5602523e1dd0 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f496a68eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602522a3200 con 0x5602523b5370 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f496a68eb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56025218a6a0 con 0x5602523da7a0 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f496a68eb80 10 client.0 _ll_register_callbacks cb 0x5602522acb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f495ca71700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523e1dd0 0x5602523e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f495ca71700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523e1dd0 0x5602523e2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47658/0 (socket says 172.21.15.5:47658) 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f495ca71700 1 -- 172.21.15.5:0/2951027939 learned_addr learned my addr 172.21.15.5:0/2951027939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f49577fe700 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523da7a0 0x5602523e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f4957fff700 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f495ca71700 1 -- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523da7a0 msgr2=0x5602523e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.316+0000 7f495ca71700 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523da7a0 0x5602523e4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f495ca71700 1 -- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 msgr2=0x5602523d7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f495ca71700 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523d7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f495ca71700 1 -- 172.21.15.5:0/2951027939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494800f040 con 0x5602523e1dd0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f4957fff700 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523d7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f495ca71700 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523e1dd0 0x5602523e2f70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4950004690 tx=0x7f4950003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb68c49ec8fd010 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f4936ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f4936ffd700 1 -- 172.21.15.5:0/2951027939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950003d60 con 0x5602523e1dd0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f4936ffd700 1 -- 172.21.15.5:0/2951027939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4950007950 con 0x5602523e1dd0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 1 -- 172.21.15.5:0/2951027939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560252260270 con 0x5602523e1dd0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f4936ffd700 1 -- 172.21.15.5:0/2951027939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4950007b10 con 0x5602523e1dd0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 10 client.8979 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 1 -- 172.21.15.5:0/2951027939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x560252134b60 con 0x5602523e1dd0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f4936ffd700 1 -- 172.21.15.5:0/2951027939 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560252260270 con 0x5602523e1dd0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f4936ffd700 1 -- 172.21.15.5:0/2951027939 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f4950014070 con 0x5602523e1dd0 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 10 client.8979 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 10 client.8979 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 20 client.8979 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x5602523e34b0 0x5602523fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 4 client.8979 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.317+0000 7f496a68eb80 1 -- 172.21.15.5:0/2951027939 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602523867e0 con 0x5602523e34b0 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.318+0000 7f4957fff700 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x5602523e34b0 0x5602523fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.318+0000 7f4936ffd700 10 client.8979 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.318+0000 7f4957fff700 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x5602523e34b0 0x5602523fe250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32b5b66da4e17b77 server_cookie=5931c0be3e0494a8 in_seq=0 out_seq=0 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.318+0000 7f4936ffd700 1 -- 172.21.15.5:0/2951027939 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602523867e0 con 0x5602523e34b0 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.318+0000 7f4936ffd700 10 client.8979 handle_command_reply: tid=0 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 client.8979 shutdown 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 -- 172.21.15.5:0/2951027939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523e1dd0 msgr2=0x5602523e2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523e1dd0 0x5602523e2f70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4950004690 tx=0x7f4950003ec0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 -- 172.21.15.5:0/2951027939 shutdown_connections 2024-05-08T23:28:29.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x5602523e34b0 0x5602523fe250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523e1dd0 0x5602523e2f70 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523d7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 --2- 172.21.15.5:0/2951027939 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523da7a0 0x5602523e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f4936ffd700 0 client.8979 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 -- 172.21.15.5:0/2951027939 >> 172.21.15.5:0/2951027939 conn(0x560252315210 msgr2=0x5602523b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 -- 172.21.15.5:0/2951027939 shutdown_connections 2024-05-08T23:28:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 1 -- 172.21.15.5:0/2951027939 wait complete. 2024-05-08T23:28:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.319+0000 7f496a68eb80 20 client.8979 trim_cache size 0 max 16384 2024-05-08T23:28:29.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.337+0000 7f496a68eb80 1 Processor -- start 2024-05-08T23:28:29.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.337+0000 7f496a68eb80 1 -- start start 2024-05-08T23:28:29.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523b5370 0x56025241ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523da7a0 0x5602523fd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f496a68eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602523b96b0 con 0x5602523b5370 2024-05-08T23:28:29.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f496a68eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602523b1d00 con 0x5602523da7a0 2024-05-08T23:28:29.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f496a68eb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602523b3840 con 0x5602523e1dd0 2024-05-08T23:28:29.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f4957fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523b5370 0x56025241ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f4957fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523b5370 0x56025241ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47674/0 (socket says 172.21.15.5:47674) 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f4957fff700 1 -- 172.21.15.5:0/3219440187 learned_addr learned my addr 172.21.15.5:0/3219440187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f49577fe700 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523da7a0 0x5602523fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f495ca71700 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f4957fff700 1 -- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 msgr2=0x5602523e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f4957fff700 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f4957fff700 1 -- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523da7a0 msgr2=0x5602523fd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f4957fff700 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523da7a0 0x5602523fd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f4957fff700 1 -- 172.21.15.5:0/3219440187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602523b1e50 con 0x5602523b5370 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.338+0000 7f495ca71700 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f4957fff700 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523b5370 0x56025241ecf0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4948007c10 tx=0x7f4948005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302166d7014873e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f494e7fc700 1 -- 172.21.15.5:0/3219440187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948028040 con 0x5602523b5370 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f494e7fc700 1 -- 172.21.15.5:0/3219440187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948019070 con 0x5602523b5370 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f494e7fc700 1 -- 172.21.15.5:0/3219440187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494802f5d0 con 0x5602523b5370 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f496a68eb80 1 -- 172.21.15.5:0/3219440187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523b5370 msgr2=0x56025241ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523b5370 0x56025241ecf0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4948007c10 tx=0x7f4948005000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f496a68eb80 1 -- 172.21.15.5:0/3219440187 shutdown_connections 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523b5370 0x56025241ecf0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3219440187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523da7a0 0x5602523fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.339+0000 7f496a68eb80 1 -- 172.21.15.5:0/3219440187 >> 172.21.15.5:0/3219440187 conn(0x560252315210 msgr2=0x5602523b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.340+0000 7f496a68eb80 1 -- 172.21.15.5:0/3219440187 shutdown_connections 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.340+0000 7f496a68eb80 1 -- 172.21.15.5:0/3219440187 wait complete. 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.340+0000 7f496a68eb80 1 -- start start 2024-05-08T23:28:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.340+0000 7f496a68eb80 1 Processor -- start 2024-05-08T23:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x560252463120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f496a68eb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f496a68eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602523ff100 con 0x5602523da7a0 2024-05-08T23:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f496a68eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560252463660 con 0x5602523b5370 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f496a68eb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602524660f0 con 0x5602523e1dd0 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f496a68eb80 10 client.0 _ll_register_callbacks cb 0x5602522acb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f49577fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x560252463120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f49577fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x560252463120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47682/0 (socket says 172.21.15.5:47682) 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.341+0000 7f49577fe700 1 -- 172.21.15.5:0/3786769026 learned_addr learned my addr 172.21.15.5:0/3786769026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f4957fff700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f495ca71700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f49577fe700 1 -- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 msgr2=0x5602523b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f49577fe700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f49577fe700 1 -- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 msgr2=0x5602523e5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f49577fe700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523e5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f49577fe700 1 -- 172.21.15.5:0/3786769026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602521883b0 con 0x5602523da7a0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f4957fff700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f495ca71700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f49577fe700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x560252463120 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f4940009c40 tx=0x7f4940059f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13c0d31561e95bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f4936ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f4936ffd700 1 -- 172.21.15.5:0/3786769026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940061ad0 con 0x5602523da7a0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f4936ffd700 1 -- 172.21.15.5:0/3786769026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4940059940 con 0x5602523da7a0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 1 -- 172.21.15.5:0/3786769026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4948003dd0 con 0x5602523da7a0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f4936ffd700 1 -- 172.21.15.5:0/3786769026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4940059b00 con 0x5602523da7a0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 10 client.8985 fetch_fsmap learned FSMap version 106 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 1 -- 172.21.15.5:0/3786769026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x5602522cd560 con 0x5602523da7a0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f4936ffd700 1 -- 172.21.15.5:0/3786769026 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948003dd0 con 0x5602523da7a0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f4936ffd700 1 -- 172.21.15.5:0/3786769026 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 3150+0+0 (secure 0 0 0) 0x7f4940058bd0 con 0x5602523da7a0 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 10 client.8985 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T23:28:29.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 10 client.8985 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 20 client.8985 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x560252465bf0 0x5602523b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 4 client.8985 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.342+0000 7f496a68eb80 1 -- 172.21.15.5:0/3786769026 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5602522ffec0 con 0x560252465bf0 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f4957fff700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x560252465bf0 0x5602523b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f4957fff700 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x560252465bf0 0x5602523b5c90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e62f6aaccd72c07 server_cookie=c5ae874a73b1964c in_seq=0 out_seq=0 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f4936ffd700 10 client.8985 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f4936ffd700 1 -- 172.21.15.5:0/3786769026 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5602522ffec0 con 0x560252465bf0 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f4936ffd700 10 client.8985 handle_command_reply: tid=0 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 client.8985 shutdown 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 -- 172.21.15.5:0/3786769026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 msgr2=0x560252463120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x560252463120 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f4940009c40 tx=0x7f4940059f10 comp rx=0 tx=0).stop 2024-05-08T23:28:29.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 -- 172.21.15.5:0/3786769026 shutdown_connections 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x560252465bf0 0x5602523b5c90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5602523e1dd0 0x5602523b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602523b5370 0x5602523e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f4936ffd700 0 client.8985 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 --2- 172.21.15.5:0/3786769026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602523da7a0 0x560252463120 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 -- 172.21.15.5:0/3786769026 >> 172.21.15.5:0/3786769026 conn(0x560252315210 msgr2=0x5602523b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 -- 172.21.15.5:0/3786769026 shutdown_connections 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.343+0000 7f496a68eb80 1 -- 172.21.15.5:0/3786769026 wait complete. 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f496a68eb80 20 client.8985 trim_cache size 0 max 16384 2024-05-08T23:28:29.350 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:29.351 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 -- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f493806ac90 msgr2=0x7f493806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f493806ac90 0x7f493806d150 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4948000f60 tx=0x7f4948009040 comp rx=0 tx=0).stop 2024-05-08T23:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 -- 172.21.15.5:0/2087801695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 msgr2=0x7f49580afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f49580afd60 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4940009870 tx=0x7f4940004750 comp rx=0 tx=0).stop 2024-05-08T23:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 -- 172.21.15.5:0/2087801695 shutdown_connections 2024-05-08T23:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f493806ac90 0x7f493806d150 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f495815db00 0x7f49581f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495806f7c0 0x7f4958156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 --2- 172.21.15.5:0/2087801695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49580cc590 0x7f49580afd60 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 -- 172.21.15.5:0/2087801695 >> 172.21.15.5:0/2087801695 conn(0x7f4958099a50 msgr2=0x7f495806f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 -- 172.21.15.5:0/2087801695 shutdown_connections 2024-05-08T23:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.344+0000 7f495e4d4700 1 -- 172.21.15.5:0/2087801695 wait complete. 2024-05-08T23:28:29.353 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:28:29.351+0000 7f8a4c50b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T23:28:29.358 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:29.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 scrub status 2024-05-08T23:28:29.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.496+0000 7f8e1f5f7700 1 Processor -- start 2024-05-08T23:28:29.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.496+0000 7f8e1f5f7700 1 -- start start 2024-05-08T23:28:29.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e1f5f7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 0x7f8e180b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e1f5f7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e18145040 con 0x7f8e180b4d50 2024-05-08T23:28:29.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e1f5f7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e180b38c0 con 0x7f8e1806b070 2024-05-08T23:28:29.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e1f5f7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e180b16e0 con 0x7f8e180b0d80 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e1cb92700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8e180b4d50 0x7f8e1806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46976/0 (socket says 172.21.15.5:46976) 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e1cb92700 1 -- 172.21.15.5:0/64461788 learned_addr learned my addr 172.21.15.5:0/64461788 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e1d393700 1 --2- 172.21.15.5:0/64461788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 0x7f8e180b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e0ffff700 1 -- 172.21.15.5:0/64461788 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797727991 0 0) 0x7f8e18145040 con 0x7f8e180b4d50 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e0ffff700 1 -- 172.21.15.5:0/64461788 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e040035b0 con 0x7f8e180b4d50 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.497+0000 7f8e0ffff700 1 -- 172.21.15.5:0/64461788 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771671164 0 0) 0x7f8e180b38c0 con 0x7f8e1806b070 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e0ffff700 1 -- 172.21.15.5:0/64461788 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e18145040 con 0x7f8e1806b070 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e1d393700 1 -- 172.21.15.5:0/64461788 >> v1:172.21.15.49:6789/0 conn(0x7f8e1806b070 legacy=0x7f8e181454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e0ffff700 1 -- 172.21.15.5:0/64461788 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 460259340 0 0) 0x7f8e040035b0 con 0x7f8e180b4d50 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e1d393700 1 -- 172.21.15.5:0/64461788 >> v1:172.21.15.5:6789/0 conn(0x7f8e180b4d50 legacy=0x7f8e1806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e1d393700 1 -- 172.21.15.5:0/64461788 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e180b19c0 con 0x7f8e180b0d80 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e1d393700 1 --2- 172.21.15.5:0/64461788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 0x7f8e180b3380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009870 tx=0x7f8e14009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68182dc37669f808 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e0ffff700 1 -- 172.21.15.5:0/64461788 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08001670 con 0x7f8e180b0d80 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e0ffff700 1 -- 172.21.15.5:0/64461788 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e14004030 con 0x7f8e180b0d80 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.498+0000 7f8e0ffff700 1 -- 172.21.15.5:0/64461788 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140043c0 con 0x7f8e180b0d80 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.499+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/64461788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 msgr2=0x7f8e180b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.499+0000 7f8e1f5f7700 1 --2- 172.21.15.5:0/64461788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 0x7f8e180b3380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009870 tx=0x7f8e14009b80 comp rx=0 tx=0).stop 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.499+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/64461788 shutdown_connections 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.499+0000 7f8e1f5f7700 1 --2- 172.21.15.5:0/64461788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 0x7f8e180b3380 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.499+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/64461788 >> 172.21.15.5:0/64461788 conn(0x7f8e18099a50 msgr2=0x7f8e180b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.499+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/64461788 shutdown_connections 2024-05-08T23:28:29.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.499+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/64461788 wait complete. 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.500+0000 7f8e1f5f7700 1 Processor -- start 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.500+0000 7f8e1f5f7700 1 -- start start 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1f5f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e1806b070 0x7f8e18151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1f5f7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 0x7f8e1809dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1f5f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e180b4d50 0x7f8e1813e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1f5f7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18151f70 con 0x7f8e1806b070 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1f5f7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181f3c30 con 0x7f8e180b4d50 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1f5f7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18152d70 con 0x7f8e180b0d80 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1d393700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e1806b070 0x7f8e18151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1d393700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e1806b070 0x7f8e18151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47690/0 (socket says 172.21.15.5:47690) 2024-05-08T23:28:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1d393700 1 -- 172.21.15.5:0/4115660542 learned_addr learned my addr 172.21.15.5:0/4115660542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1db94700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e180b4d50 0x7f8e1813e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1d393700 1 -- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 msgr2=0x7f8e1809dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1d393700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 0x7f8e1809dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1d393700 1 -- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e180b4d50 msgr2=0x7f8e1813e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1d393700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e180b4d50 0x7f8e1813e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.501+0000 7f8e1d393700 1 -- 172.21.15.5:0/4115660542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e140094a0 con 0x7f8e1806b070 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.502+0000 7f8e1d393700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e1806b070 0x7f8e18151a30 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8e140090b0 tx=0x7f8e14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.502+0000 7f8e0dffb700 1 -- 172.21.15.5:0/4115660542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14011630 con 0x7f8e1806b070 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.502+0000 7f8e0dffb700 1 -- 172.21.15.5:0/4115660542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1401f070 con 0x7f8e1806b070 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.502+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/4115660542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1809b520 con 0x7f8e1806b070 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.502+0000 7f8e0dffb700 1 -- 172.21.15.5:0/4115660542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140119d0 con 0x7f8e1806b070 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.502+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/4115660542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e18152160 con 0x7f8e1806b070 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.503+0000 7f8e0dffb700 1 -- 172.21.15.5:0/4115660542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8e14011b30 con 0x7f8e1806b070 2024-05-08T23:28:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.504+0000 7f8e0dffb700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dfc06ac90 0x7f8dfc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.504+0000 7f8e0dffb700 1 -- 172.21.15.5:0/4115660542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e14021070 con 0x7f8e1806b070 2024-05-08T23:28:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.504+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dfc06ac90 0x7f8dfc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.505+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dfc06ac90 0x7f8dfc06d150 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8e080017a0 tx=0x7f8e08001290 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.506+0000 7f8e2b7b1b80 1 Processor -- start 2024-05-08T23:28:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.506+0000 7f8e2b7b1b80 1 -- start start 2024-05-08T23:28:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabde89110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabde8e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 0x55eabdeb8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eabdde4150 con 0x55eabde8b370 2024-05-08T23:28:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eabdde1b10 con 0x55eabdeb7dd0 2024-05-08T23:28:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eabdddd400 con 0x55eabdeb07a0 2024-05-08T23:28:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e1cb92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabde8e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e1cb92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabde8e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47694/0 (socket says 172.21.15.5:47694) 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e1cb92700 1 -- 172.21.15.5:0/2101782935 learned_addr learned my addr 172.21.15.5:0/2101782935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e1db94700 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 0x55eabdeb8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.507+0000 7f8e1d393700 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabde89110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e1cb92700 1 -- 172.21.15.5:0/2101782935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 msgr2=0x55eabde89110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabde89110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e1cb92700 1 -- 172.21.15.5:0/2101782935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 msgr2=0x55eabdeb8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 0x55eabdeb8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e1cb92700 1 -- 172.21.15.5:0/2101782935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eabdc5e3b0 con 0x55eabde8b370 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabde8e0a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8e180cffb0 tx=0x7f8e0800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8212ff9fd24ed392 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e02ffd700 1 -- 172.21.15.5:0/2101782935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0801a070 con 0x55eabde8b370 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e02ffd700 1 -- 172.21.15.5:0/2101782935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e08006030 con 0x55eabde8b370 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.508+0000 7f8e02ffd700 1 -- 172.21.15.5:0/2101782935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e080063a0 con 0x55eabde8b370 2024-05-08T23:28:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2101782935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 msgr2=0x55eabde8e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabde8e0a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8e180cffb0 tx=0x7f8e0800f000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2101782935 shutdown_connections 2024-05-08T23:28:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 0x55eabdeb8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabde89110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/2101782935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabde8e0a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2101782935 >> 172.21.15.5:0/2101782935 conn(0x55eabddeb210 msgr2=0x55eabded2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2101782935 shutdown_connections 2024-05-08T23:28:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.509+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2101782935 wait complete. 2024-05-08T23:28:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.510+0000 7f8e2b7b1b80 1 -- start start 2024-05-08T23:28:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.510+0000 7f8e2b7b1b80 1 Processor -- start 2024-05-08T23:28:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabdeaf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabdeba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 0x55eabded31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eabdd79560 con 0x55eabde8b370 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eabdd79200 con 0x55eabdeb7dd0 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eabdc606a0 con 0x55eabdeb07a0 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e1d393700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabdeaf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e2b7b1b80 10 client.0 _ll_register_callbacks cb 0x55eabdd82b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e1d393700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabdeaf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47704/0 (socket says 172.21.15.5:47704) 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.511+0000 7f8e1d393700 1 -- 172.21.15.5:0/409353699 learned_addr learned my addr 172.21.15.5:0/409353699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabdeba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e1db94700 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 0x55eabded31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e1d393700 1 -- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 msgr2=0x55eabdeba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e1d393700 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabdeba650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e1d393700 1 -- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 msgr2=0x55eabded31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e1d393700 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 0x55eabded31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e1d393700 1 -- 172.21.15.5:0/409353699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e080069d0 con 0x55eabde8b370 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e1d393700 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabdeaf330 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009ed0 tx=0x7f8e140104c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa2136e54ccf49b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8df7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8df7fff700 1 -- 172.21.15.5:0/409353699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1405d920 con 0x55eabde8b370 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8df7fff700 1 -- 172.21.15.5:0/409353699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e14061ee0 con 0x55eabde8b370 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/409353699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eabdd36270 con 0x55eabde8b370 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8df7fff700 1 -- 172.21.15.5:0/409353699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e14057e90 con 0x55eabde8b370 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e2b7b1b80 10 client.8997 fetch_fsmap learned FSMap version 107 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/409353699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55eabdc0ab60 con 0x55eabde8b370 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.512+0000 7f8df7fff700 1 -- 172.21.15.5:0/409353699 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eabdd36270 con 0x55eabde8b370 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8df7fff700 1 -- 172.21.15.5:0/409353699 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 107) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f8e14059d50 con 0x55eabde8b370 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8e2b7b1b80 10 client.8997 fetch_fsmap finished waiting for FSMap version 107 2024-05-08T23:28:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8e2b7b1b80 10 client.8997 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8e2b7b1b80 20 client.8997 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x55eabdebab90 0x55eabdee5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8e2b7b1b80 4 client.8997 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/409353699 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eabde5c7e0 con 0x55eabdebab90 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x55eabdebab90 0x55eabdee5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8df7fff700 10 client.8997 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.513+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x55eabdebab90 0x55eabdee5630 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80f2e29c902de68d server_cookie=d86c3267ed15388c in_seq=0 out_seq=0 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8df7fff700 1 -- 172.21.15.5:0/409353699 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eabde5c7e0 con 0x55eabdebab90 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8df7fff700 10 client.8997 handle_command_reply: tid=0 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 client.8997 shutdown 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/409353699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 msgr2=0x55eabdeaf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabdeaf330 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009ed0 tx=0x7f8e140104c0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/409353699 shutdown_connections 2024-05-08T23:28:29.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x55eabdebab90 0x55eabdee5630 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabdeb7dd0 0x55eabded31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabde8b370 0x55eabdeaf330 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8df7fff700 0 client.8997 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/409353699 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabdeba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/409353699 >> 172.21.15.5:0/409353699 conn(0x55eabddeb210 msgr2=0x55eabde87450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/409353699 shutdown_connections 2024-05-08T23:28:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/409353699 wait complete. 2024-05-08T23:28:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.514+0000 7f8e2b7b1b80 20 client.8997 trim_cache size 0 max 16384 2024-05-08T23:28:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.533+0000 7f8e2b7b1b80 1 Processor -- start 2024-05-08T23:28:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.533+0000 7f8e2b7b1b80 1 -- start start 2024-05-08T23:28:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabdeb81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb07a0 0x55eabdeb5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb7dd0 0x55eabde8dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eabde8dba0 con 0x55eabdeb07a0 2024-05-08T23:28:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eabdebc0b0 con 0x55eabde8b370 2024-05-08T23:28:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eabdeb4470 con 0x55eabdeb7dd0 2024-05-08T23:28:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1cb92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb07a0 0x55eabdeb5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1cb92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb07a0 0x55eabdeb5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47714/0 (socket says 172.21.15.5:47714) 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1cb92700 1 -- 172.21.15.5:0/2867720450 learned_addr learned my addr 172.21.15.5:0/2867720450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1d393700 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabdeb81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1db94700 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb7dd0 0x55eabde8dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1cb92700 1 -- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb7dd0 msgr2=0x55eabde8dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb7dd0 0x55eabde8dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1cb92700 1 -- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 msgr2=0x55eabdeb81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabdeb81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1cb92700 1 -- 172.21.15.5:0/2867720450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eabdda3560 con 0x55eabdeb07a0 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.534+0000 7f8e1d393700 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabdeb81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb07a0 0x55eabdeb5060 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8e181f7210 tx=0x7f8e08007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb9565b78c48c96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e037fe700 1 -- 172.21.15.5:0/2867720450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08024040 con 0x55eabdeb07a0 2024-05-08T23:28:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e037fe700 1 -- 172.21.15.5:0/2867720450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e08005c20 con 0x55eabdeb07a0 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e037fe700 1 -- 172.21.15.5:0/2867720450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0802a380 con 0x55eabdeb07a0 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2867720450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb07a0 msgr2=0x55eabdeb5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb07a0 0x55eabdeb5060 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8e181f7210 tx=0x7f8e08007000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2867720450 shutdown_connections 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb7dd0 0x55eabde8dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabdeb81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/2867720450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb07a0 0x55eabdeb5060 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2867720450 >> 172.21.15.5:0/2867720450 conn(0x55eabddeb210 msgr2=0x55eabdeaf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2867720450 shutdown_connections 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.535+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/2867720450 wait complete. 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.536+0000 7f8e2b7b1b80 1 -- start start 2024-05-08T23:28:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.536+0000 7f8e2b7b1b80 1 Processor -- start 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabde8e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabdf3b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e2b7b1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb7dd0 0x55eabdf3be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eabdf3b4e0 con 0x55eabdeb7dd0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eabdee5a90 con 0x55eabde8b370 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e2b7b1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eabded21e0 con 0x55eabdeb07a0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e2b7b1b80 10 client.0 _ll_register_callbacks cb 0x55eabdd82b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e1db94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb7dd0 0x55eabdf3be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e1db94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb7dd0 0x55eabdf3be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47726/0 (socket says 172.21.15.5:47726) 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e1db94700 1 -- 172.21.15.5:0/688740330 learned_addr learned my addr 172.21.15.5:0/688740330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e1d393700 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabde8e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.537+0000 7f8e1cb92700 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabdf3b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e1db94700 1 -- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 msgr2=0x55eabdf3b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e1db94700 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabdf3b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e1db94700 1 -- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 msgr2=0x55eabde8e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e1db94700 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabde8e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e1db94700 1 -- 172.21.15.5:0/688740330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eabdc5e3b0 con 0x55eabdeb7dd0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e1db94700 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb7dd0 0x55eabdf3be80 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x55eabdebc210 tx=0x7f8e100055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9948a1abf9bc287e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8df7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8df7fff700 1 -- 172.21.15.5:0/688740330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10016830 con 0x55eabdeb7dd0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8df7fff700 1 -- 172.21.15.5:0/688740330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e10016990 con 0x55eabdeb7dd0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/688740330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e08006030 con 0x55eabdeb7dd0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8df7fff700 1 -- 172.21.15.5:0/688740330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e10016b50 con 0x55eabdeb7dd0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 10 client.9003 fetch_fsmap learned FSMap version 107 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/688740330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55eabdc62cd0 con 0x55eabdeb7dd0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8df7fff700 1 -- 172.21.15.5:0/688740330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e08006030 con 0x55eabdeb7dd0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8df7fff700 1 -- 172.21.15.5:0/688740330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 107) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f8e10016cb0 con 0x55eabdeb7dd0 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 10 client.9003 fetch_fsmap finished waiting for FSMap version 107 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 10 client.9003 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 20 client.9003 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x55eabdeb54c0 0x55eabdf3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 4 client.9003 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.538+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/688740330 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eabddd5ec0 con 0x55eabdeb54c0 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e1d393700 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x55eabdeb54c0 0x55eabdf3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8df7fff700 10 client.9003 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e1d393700 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x55eabdeb54c0 0x55eabdf3eb00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7c5b4422a67a0f4 server_cookie=750aaf24ccb84fff in_seq=0 out_seq=0 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8df7fff700 1 -- 172.21.15.5:0/688740330 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55eabddd5ec0 con 0x55eabdeb54c0 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8df7fff700 10 client.9003 handle_command_reply: tid=0 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 client.9003 shutdown 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/688740330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb7dd0 msgr2=0x55eabdf3be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb7dd0 0x55eabdf3be80 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x55eabdebc210 tx=0x7f8e100055f0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/688740330 shutdown_connections 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x55eabdeb54c0 0x55eabdf3eb00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eabdeb7dd0 0x55eabdf3be80 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eabde8b370 0x55eabde8e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 --2- 172.21.15.5:0/688740330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55eabdeb07a0 0x55eabdf3b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8df7fff700 0 client.9003 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.539+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/688740330 >> 172.21.15.5:0/688740330 conn(0x55eabddeb210 msgr2=0x55eabdeb81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/688740330 shutdown_connections 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e2b7b1b80 1 -- 172.21.15.5:0/688740330 wait complete. 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e2b7b1b80 20 client.9003 trim_cache size 0 max 16384 2024-05-08T23:28:29.546 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dfc06ac90 msgr2=0x7f8dfc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dfc06ac90 0x7f8dfc06d150 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8e080017a0 tx=0x7f8e08001290 comp rx=0 tx=0).stop 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/4115660542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e1806b070 msgr2=0x7f8e18151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e1806b070 0x7f8e18151a30 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8e140090b0 tx=0x7f8e14004750 comp rx=0 tx=0).stop 2024-05-08T23:28:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/4115660542 shutdown_connections 2024-05-08T23:28:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8dfc06ac90 0x7f8dfc06d150 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e180b4d50 0x7f8e1813e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e1806b070 0x7f8e18151a30 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 --2- 172.21.15.5:0/4115660542 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8e180b0d80 0x7f8e1809dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/4115660542 >> 172.21.15.5:0/4115660542 conn(0x7f8e18099a50 msgr2=0x7f8e181447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/4115660542 shutdown_connections 2024-05-08T23:28:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.540+0000 7f8e1f5f7700 1 -- 172.21.15.5:0/4115660542 wait complete. 2024-05-08T23:28:29.554 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:29.554 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T23:28:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.694+0000 7fe4f4937700 1 Processor -- start 2024-05-08T23:28:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.694+0000 7fe4f4937700 1 -- start start 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.695+0000 7fe4f4937700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe4f006bd60 0x7fe4f00cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.695+0000 7fe4f4937700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00b1200 con 0x7fe4f00d13a0 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.695+0000 7fe4f4937700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00cfcc0 con 0x7fe4f00cdc90 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.695+0000 7fe4f4937700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00ce5c0 con 0x7fe4f006bd60 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.695+0000 7fe4edd9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe4f006bd60 0x7fe4f00cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.695+0000 7fe4ee59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe4f00d13a0 0x7fe4f00d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46986/0 (socket says 172.21.15.5:46986) 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.695+0000 7fe4ee59c700 1 -- 172.21.15.5:0/3841019203 learned_addr learned my addr 172.21.15.5:0/3841019203 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.695+0000 7fe4edd9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe4f006bd60 0x7fe4f00cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47740/0 (socket says 172.21.15.5:47740) 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4edd9b700 1 -- 172.21.15.5:0/3841019203 >> v1:172.21.15.49:6789/0 conn(0x7fe4f00cdc90 legacy=0x7fe4f00b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4edd9b700 1 -- 172.21.15.5:0/3841019203 >> v1:172.21.15.5:6789/0 conn(0x7fe4f00d13a0 legacy=0x7fe4f00d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4edd9b700 1 -- 172.21.15.5:0/3841019203 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00ce8a0 con 0x7fe4f006bd60 2024-05-08T23:28:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/3841019203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f00cf780 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009840 tx=0x7fe4d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d9962fc6c1be15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4ed59a700 1 -- 172.21.15.5:0/3841019203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8004230 con 0x7fe4f006bd60 2024-05-08T23:28:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4ed59a700 1 -- 172.21.15.5:0/3841019203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d8004390 con 0x7fe4f006bd60 2024-05-08T23:28:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4ed59a700 1 -- 172.21.15.5:0/3841019203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80036a0 con 0x7fe4f006bd60 2024-05-08T23:28:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4f4937700 1 -- 172.21.15.5:0/3841019203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 msgr2=0x7fe4f00cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.696+0000 7fe4f4937700 1 --2- 172.21.15.5:0/3841019203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f00cf780 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009840 tx=0x7fe4d8009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.697+0000 7fe4f4937700 1 -- 172.21.15.5:0/3841019203 shutdown_connections 2024-05-08T23:28:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.697+0000 7fe4f4937700 1 --2- 172.21.15.5:0/3841019203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f00cf780 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.697+0000 7fe4f4937700 1 -- 172.21.15.5:0/3841019203 >> 172.21.15.5:0/3841019203 conn(0x7fe4f008afe0 msgr2=0x7fe4f008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.697+0000 7fe4f4937700 1 -- 172.21.15.5:0/3841019203 shutdown_connections 2024-05-08T23:28:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.697+0000 7fe4f4937700 1 -- 172.21.15.5:0/3841019203 wait complete. 2024-05-08T23:28:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.698+0000 7fe4f4937700 1 Processor -- start 2024-05-08T23:28:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.698+0000 7fe4f4937700 1 -- start start 2024-05-08T23:28:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4f4937700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f01f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4f4937700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f00cdc90 0x7fe4f00c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4f4937700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe4f00d13a0 0x7fe4f0156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4f4937700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00c9200 con 0x7fe4f006bd60 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4f4937700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00ae620 con 0x7fe4f00cdc90 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4f4937700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00ae900 con 0x7fe4f00d13a0 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f01f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f01f3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47756/0 (socket says 172.21.15.5:47756) 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4ee59c700 1 -- 172.21.15.5:0/275879063 learned_addr learned my addr 172.21.15.5:0/275879063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f00cdc90 0x7fe4f00c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4ee59c700 1 -- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe4f00d13a0 msgr2=0x7fe4f0156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe4f00d13a0 0x7fe4f0156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe4f00d13a0 0x7fe4f0156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4ee59c700 1 -- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f00cdc90 msgr2=0x7fe4f00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f00cdc90 0x7fe4f00c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4ee59c700 1 -- 172.21.15.5:0/275879063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d80094a0 con 0x7fe4f006bd60 2024-05-08T23:28:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f00cdc90 0x7fe4f00c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.699+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe4f00d13a0 0x7fe4f0156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.700+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f01f3780 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000c490 tx=0x7fe4e000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.700+0000 7fe4e6ffd700 1 -- 172.21.15.5:0/275879063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0005d70 con 0x7fe4f006bd60 2024-05-08T23:28:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.700+0000 7fe4e6ffd700 1 -- 172.21.15.5:0/275879063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e00070d0 con 0x7fe4f006bd60 2024-05-08T23:28:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.700+0000 7fe4e6ffd700 1 -- 172.21.15.5:0/275879063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0007460 con 0x7fe4f006bd60 2024-05-08T23:28:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.700+0000 7fe4f4937700 1 -- 172.21.15.5:0/275879063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f00aebe0 con 0x7fe4f006bd60 2024-05-08T23:28:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.700+0000 7fe4f4937700 1 -- 172.21.15.5:0/275879063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01f3cc0 con 0x7fe4f006bd60 2024-05-08T23:28:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.701+0000 7fe4e6ffd700 1 -- 172.21.15.5:0/275879063 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe4e00075c0 con 0x7fe4f006bd60 2024-05-08T23:28:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.702+0000 7fe4e6ffd700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe4d006ac90 0x7fe4d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.702+0000 7fe4e6ffd700 1 -- 172.21.15.5:0/275879063 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4e0088830 con 0x7fe4f006bd60 2024-05-08T23:28:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.702+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe4d006ac90 0x7fe4d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.703+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe4d006ac90 0x7fe4d006d150 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00c8e40 tx=0x7fe4d8013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.704+0000 7fe500af1b80 1 Processor -- start 2024-05-08T23:28:29.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.704+0000 7fe500af1b80 1 -- start start 2024-05-08T23:28:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d37937a0 0x5645d376c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d37710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d379add0 0x5645d379b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe500af1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d36c7150 con 0x5645d37937a0 2024-05-08T23:28:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe500af1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d36c4b10 con 0x5645d379add0 2024-05-08T23:28:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe500af1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645d36c0400 con 0x5645d376e370 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d37937a0 0x5645d376c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d37937a0 0x5645d376c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47762/0 (socket says 172.21.15.5:47762) 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4ee59c700 1 -- 172.21.15.5:0/324742590 learned_addr learned my addr 172.21.15.5:0/324742590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d379add0 0x5645d379b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d37710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4ee59c700 1 -- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 msgr2=0x5645d37710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d37710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4ee59c700 1 -- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d379add0 msgr2=0x5645d379b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d379add0 0x5645d379b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4ee59c700 1 -- 172.21.15.5:0/324742590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645d35413b0 con 0x5645d37937a0 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.705+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d379add0 0x5645d379b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.706+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d37937a0 0x5645d376c110 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000cc50 tx=0x7fe4e0057100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7210a4a1800e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.706+0000 7fe4cffff700 1 -- 172.21.15.5:0/324742590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e005b500 con 0x5645d37937a0 2024-05-08T23:28:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.706+0000 7fe4cffff700 1 -- 172.21.15.5:0/324742590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e005fac0 con 0x5645d37937a0 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.706+0000 7fe4cffff700 1 -- 172.21.15.5:0/324742590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e005b170 con 0x5645d37937a0 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.706+0000 7fe500af1b80 1 -- 172.21.15.5:0/324742590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d37937a0 msgr2=0x5645d376c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.706+0000 7fe500af1b80 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d37937a0 0x5645d376c110 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000cc50 tx=0x7fe4e0057100 comp rx=0 tx=0).stop 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 -- 172.21.15.5:0/324742590 shutdown_connections 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d379add0 0x5645d379b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d37937a0 0x5645d376c110 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 --2- 172.21.15.5:0/324742590 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d37710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 -- 172.21.15.5:0/324742590 >> 172.21.15.5:0/324742590 conn(0x5645d36ce210 msgr2=0x5645d37b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 -- 172.21.15.5:0/324742590 shutdown_connections 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 -- 172.21.15.5:0/324742590 wait complete. 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 -- start start 2024-05-08T23:28:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.707+0000 7fe500af1b80 1 Processor -- start 2024-05-08T23:28:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.708+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d376e370 0x5645d37c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.708+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d37937a0 0x5645d379e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.708+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.708+0000 7fe500af1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d365c560 con 0x5645d379add0 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.708+0000 7fe500af1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d365c200 con 0x5645d376e370 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.708+0000 7fe500af1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645d35436a0 con 0x5645d37937a0 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.708+0000 7fe500af1b80 10 client.0 _ll_register_callbacks cb 0x5645d3665b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37b83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47778/0 (socket says 172.21.15.5:47778) 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4ee59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d376e370 0x5645d37c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 -- 172.21.15.5:0/1028297337 learned_addr learned my addr 172.21.15.5:0/1028297337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d37937a0 0x5645d379e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 -- 172.21.15.5:0/1028297337 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d37937a0 msgr2=0x5645d379e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d37937a0 0x5645d379e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 -- 172.21.15.5:0/1028297337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d376e370 msgr2=0x5645d37c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d376e370 0x5645d37c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 -- 172.21.15.5:0/1028297337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01f3cc0 con 0x5645d379add0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37b83b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800eae0 tx=0x7fe4e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3213c204964d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/1028297337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800cb20 con 0x5645d379add0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/1028297337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4e8008040 con 0x5645d379add0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe500af1b80 1 -- 172.21.15.5:0/1028297337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645d3619270 con 0x5645d379add0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/1028297337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e800cc80 con 0x5645d379add0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe500af1b80 10 client.9018 fetch_fsmap learned FSMap version 107 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.709+0000 7fe500af1b80 1 -- 172.21.15.5:0/1028297337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5645d34edb60 con 0x5645d379add0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/1028297337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645d3619270 con 0x5645d379add0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/1028297337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 107) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fe4e8003a00 con 0x5645d379add0 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe500af1b80 10 client.9018 fetch_fsmap finished waiting for FSMap version 107 2024-05-08T23:28:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe500af1b80 10 client.9018 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe500af1b80 20 client.9018 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe500af1b80 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5645d379bf70 0x5645d379ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe500af1b80 4 client.9018 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe500af1b80 1 -- 172.21.15.5:0/1028297337 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645d373f7e0 con 0x5645d379bf70 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5645d379bf70 0x5645d379ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe4cd7fa700 10 client.9018 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.710+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5645d379bf70 0x5645d379ca50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78b18b51bdd82f6 server_cookie=8a660105340433f3 in_seq=0 out_seq=0 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/1028297337 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645d373f7e0 con 0x5645d379bf70 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe4cd7fa700 10 client.9018 handle_command_reply: tid=0 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 client.9018 shutdown 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 -- 172.21.15.5:0/1028297337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 msgr2=0x5645d37b83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37b83b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800eae0 tx=0x7fe4e800edf0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 -- 172.21.15.5:0/1028297337 shutdown_connections 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5645d379bf70 0x5645d379ca50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37b83b0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d376e370 0x5645d37c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 --2- 172.21.15.5:0/1028297337 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d37937a0 0x5645d379e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe4cd7fa700 0 client.9018 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 -- 172.21.15.5:0/1028297337 >> 172.21.15.5:0/1028297337 conn(0x5645d36ce210 msgr2=0x5645d37b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 -- 172.21.15.5:0/1028297337 shutdown_connections 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 1 -- 172.21.15.5:0/1028297337 wait complete. 2024-05-08T23:28:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.711+0000 7fe500af1b80 20 client.9018 trim_cache size 0 max 16384 2024-05-08T23:28:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.731+0000 7fe500af1b80 1 Processor -- start 2024-05-08T23:28:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.731+0000 7fe500af1b80 1 -- start start 2024-05-08T23:28:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.731+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d379dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.731+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 0x5645d37b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.731+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.731+0000 7fe500af1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d3767e00 con 0x5645d379add0 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.731+0000 7fe500af1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d376ded0 con 0x5645d37937a0 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.731+0000 7fe500af1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645d37b60c0 con 0x5645d376e370 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47786/0 (socket says 172.21.15.5:47786) 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 -- 172.21.15.5:0/2175484820 learned_addr learned my addr 172.21.15.5:0/2175484820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 0x5645d37b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d379dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 -- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 msgr2=0x5645d379dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d379dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 -- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 msgr2=0x5645d37b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 0x5645d37b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 -- 172.21.15.5:0/2175484820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645d3686560 con 0x5645d379add0 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d379dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37c7bf0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8007aa0 tx=0x7fe4e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aec7878aa0ac98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4e4ff9700 1 -- 172.21.15.5:0/2175484820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8003a00 con 0x5645d379add0 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4e4ff9700 1 -- 172.21.15.5:0/2175484820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e802b070 con 0x5645d379add0 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.732+0000 7fe4e4ff9700 1 -- 172.21.15.5:0/2175484820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800c920 con 0x5645d379add0 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 -- 172.21.15.5:0/2175484820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 msgr2=0x5645d37c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37c7bf0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8007aa0 tx=0x7fe4e8014000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 -- 172.21.15.5:0/2175484820 shutdown_connections 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d37c7bf0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d379dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2175484820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 0x5645d37b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 -- 172.21.15.5:0/2175484820 >> 172.21.15.5:0/2175484820 conn(0x5645d36ce210 msgr2=0x5645d37b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 -- 172.21.15.5:0/2175484820 shutdown_connections 2024-05-08T23:28:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 -- 172.21.15.5:0/2175484820 wait complete. 2024-05-08T23:28:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.733+0000 7fe500af1b80 1 -- start start 2024-05-08T23:28:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.734+0000 7fe500af1b80 1 Processor -- start 2024-05-08T23:28:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d3804c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 0x5645d3822c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe500af1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d381ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe500af1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d37c50d0 con 0x5645d379add0 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe500af1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d3822850 con 0x5645d37937a0 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe500af1b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645d381f960 con 0x5645d376e370 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe500af1b80 10 client.0 _ll_register_callbacks cb 0x5645d3665b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe4eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d381ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe4eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d381ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47792/0 (socket says 172.21.15.5:47792) 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe4eed9d700 1 -- 172.21.15.5:0/2222313415 learned_addr learned my addr 172.21.15.5:0/2222313415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.735+0000 7fe4edd9b700 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 0x5645d3822c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d3804c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4eed9d700 1 -- 172.21.15.5:0/2222313415 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 msgr2=0x5645d3804c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d3804c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4eed9d700 1 -- 172.21.15.5:0/2222313415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 msgr2=0x5645d3822c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 0x5645d3822c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4eed9d700 1 -- 172.21.15.5:0/2222313415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645d34edb60 con 0x5645d379add0 2024-05-08T23:28:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4eed9d700 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d381ec90 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8002230 tx=0x7fe4e8004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6f578b27fc7117 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/2222313415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800c920 con 0x5645d379add0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/2222313415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4e802d070 con 0x5645d379add0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe500af1b80 1 -- 172.21.15.5:0/2222313415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4e005fac0 con 0x5645d379add0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/2222313415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e800ca80 con 0x5645d379add0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe500af1b80 10 client.9024 fetch_fsmap learned FSMap version 107 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe500af1b80 1 -- 172.21.15.5:0/2222313415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5645d3545cd0 con 0x5645d379add0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.736+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/2222313415 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e005fac0 con 0x5645d379add0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/2222313415 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 107) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fe4e800cbe0 con 0x5645d379add0 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 10 client.9024 fetch_fsmap finished waiting for FSMap version 107 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 10 client.9024 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 20 client.9024 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5645d37d7e10 0x5645d37c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 4 client.9024 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 1 -- 172.21.15.5:0/2222313415 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5645d36b8ec0 con 0x5645d37d7e10 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5645d37d7e10 0x5645d37c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe4cd7fa700 10 client.9024 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe4ee59c700 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5645d37d7e10 0x5645d37c7b50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbf664c20aed1f server_cookie=aecf2faf8d4d57bb in_seq=0 out_seq=0 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe4cd7fa700 1 -- 172.21.15.5:0/2222313415 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5645d36b8ec0 con 0x5645d37d7e10 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe4cd7fa700 10 client.9024 handle_command_reply: tid=0 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 1 client.9024 shutdown 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 1 -- 172.21.15.5:0/2222313415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 msgr2=0x5645d381ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.737+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d381ec90 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8002230 tx=0x7fe4e8004b00 comp rx=0 tx=0).stop 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 1 -- 172.21.15.5:0/2222313415 shutdown_connections 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5645d37d7e10 0x5645d37c7b50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645d379add0 0x5645d381ec90 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5645d376e370 0x5645d3804c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 1 --2- 172.21.15.5:0/2222313415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5645d37937a0 0x5645d3822c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4cd7fa700 0 client.9024 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 1 -- 172.21.15.5:0/2222313415 >> 172.21.15.5:0/2222313415 conn(0x5645d36ce210 msgr2=0x5645d37b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 1 -- 172.21.15.5:0/2222313415 shutdown_connections 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 1 -- 172.21.15.5:0/2222313415 wait complete. 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe500af1b80 20 client.9024 trim_cache size 0 max 16384 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:29.744 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 -- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe4d006ac90 msgr2=0x7fe4d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe4d006ac90 0x7fe4d006d150 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00c8e40 tx=0x7fe4d8013040 comp rx=0 tx=0).stop 2024-05-08T23:28:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 -- 172.21.15.5:0/275879063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 msgr2=0x7fe4f01f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f01f3780 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000c490 tx=0x7fe4e000c850 comp rx=0 tx=0).stop 2024-05-08T23:28:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 -- 172.21.15.5:0/275879063 shutdown_connections 2024-05-08T23:28:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe4d006ac90 0x7fe4d006d150 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe4f00d13a0 0x7fe4f0156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f006bd60 0x7fe4f01f3780 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 --2- 172.21.15.5:0/275879063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4f00cdc90 0x7fe4f00c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 -- 172.21.15.5:0/275879063 >> 172.21.15.5:0/275879063 conn(0x7fe4f008afe0 msgr2=0x7fe4f015e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 -- 172.21.15.5:0/275879063 shutdown_connections 2024-05-08T23:28:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.738+0000 7fe4f4937700 1 -- 172.21.15.5:0/275879063 wait complete. 2024-05-08T23:28:29.752 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:29.752 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.889+0000 7f5ad994f700 1 Processor -- start 2024-05-08T23:28:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.889+0000 7f5ad994f700 1 -- start start 2024-05-08T23:28:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad994f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5ad406b810 0x7f5ad40cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad994f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad40d0880 con 0x7f5ad406bc70 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad994f700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad40d0b60 con 0x7f5ad415d9c0 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad994f700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad40ce5b0 con 0x7f5ad406b810 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad27fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5ad406b810 0x7f5ad40cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5ad406bc70 0x7f5ad40d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47002/0 (socket says 172.21.15.5:47002) 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad37fe700 1 -- 172.21.15.5:0/43348251 learned_addr learned my addr 172.21.15.5:0/43348251 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad27fc700 1 -- 172.21.15.5:0/43348251 >> v1:172.21.15.49:6789/0 conn(0x7f5ad415d9c0 legacy=0x7f5ad40cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad27fc700 1 -- 172.21.15.5:0/43348251 >> v1:172.21.15.5:6789/0 conn(0x7f5ad406bc70 legacy=0x7f5ad40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad27fc700 1 -- 172.21.15.5:0/43348251 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad40ce890 con 0x7f5ad406b810 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad1ffb700 1 -- 172.21.15.5:0/43348251 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113954202 0 0) 0x7f5ad40d0880 con 0x7f5ad406bc70 2024-05-08T23:28:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad27fc700 1 --2- 172.21.15.5:0/43348251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad406b810 0x7f5ad40cf090 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009820 tx=0x7f5ac8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4a8cca3d9b52b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.890+0000 7f5ad1ffb700 1 -- 172.21.15.5:0/43348251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8005e30 con 0x7f5ad406b810 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad1ffb700 1 -- 172.21.15.5:0/43348251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac8006030 con 0x7f5ad406b810 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad1ffb700 1 -- 172.21.15.5:0/43348251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac80063e0 con 0x7f5ad406b810 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad994f700 1 -- 172.21.15.5:0/43348251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad406b810 msgr2=0x7f5ad40cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad994f700 1 --2- 172.21.15.5:0/43348251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad406b810 0x7f5ad40cf090 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009820 tx=0x7f5ac8009b30 comp rx=0 tx=0).stop 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad994f700 1 -- 172.21.15.5:0/43348251 shutdown_connections 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad994f700 1 --2- 172.21.15.5:0/43348251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad406b810 0x7f5ad40cf090 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad994f700 1 -- 172.21.15.5:0/43348251 >> 172.21.15.5:0/43348251 conn(0x7f5ad408afe0 msgr2=0x7f5ad40d3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad994f700 1 -- 172.21.15.5:0/43348251 shutdown_connections 2024-05-08T23:28:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.891+0000 7f5ad994f700 1 -- 172.21.15.5:0/43348251 wait complete. 2024-05-08T23:28:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.892+0000 7f5ad994f700 1 Processor -- start 2024-05-08T23:28:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.893+0000 7f5ad994f700 1 -- start start 2024-05-08T23:28:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.893+0000 7f5ad994f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad406b810 0x7f5ad4157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.893+0000 7f5ad994f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5ad406bc70 0x7f5ad40cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.893+0000 7f5ad994f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad415d9c0 0x7f5ad40b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.893+0000 7f5ad994f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40ca7b0 con 0x7f5ad415d9c0 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad994f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40caa90 con 0x7f5ad406b810 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad994f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40cad70 con 0x7f5ad406bc70 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad415d9c0 0x7f5ad40b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad415d9c0 0x7f5ad40b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47804/0 (socket says 172.21.15.5:47804) 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad406b810 0x7f5ad4157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 -- 172.21.15.5:0/2723187756 learned_addr learned my addr 172.21.15.5:0/2723187756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad27fc700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5ad406bc70 0x7f5ad40cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 -- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5ad406bc70 msgr2=0x7f5ad40cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5ad406bc70 0x7f5ad40cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 -- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad406b810 msgr2=0x7f5ad4157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad406b810 0x7f5ad4157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 -- 172.21.15.5:0/2723187756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac8009430 con 0x7f5ad415d9c0 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad2ffd700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad406b810 0x7f5ad4157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad37fe700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad415d9c0 0x7f5ad40b12c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5acc002230 tx=0x7f5acc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5abbfff700 1 -- 172.21.15.5:0/2723187756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00aac0 con 0x7f5ad415d9c0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5abbfff700 1 -- 172.21.15.5:0/2723187756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc00ac40 con 0x7f5ad415d9c0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.894+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad40cb050 con 0x7f5ad415d9c0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.895+0000 7f5abbfff700 1 -- 172.21.15.5:0/2723187756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0070d0 con 0x7f5ad415d9c0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.895+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad40cc690 con 0x7f5ad415d9c0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.895+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad40ca8f0 con 0x7f5ad415d9c0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.896+0000 7f5abbfff700 1 -- 172.21.15.5:0/2723187756 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5acc007250 con 0x7f5ad415d9c0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.897+0000 7f5abbfff700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ac006ac90 0x7f5ac006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.897+0000 7f5abbfff700 1 -- 172.21.15.5:0/2723187756 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5acc089000 con 0x7f5ad415d9c0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.897+0000 7f5ad2ffd700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ac006ac90 0x7f5ac006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.897+0000 7f5ad2ffd700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ac006ac90 0x7f5ac006d150 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40b0740 tx=0x7f5ac4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:29.902+0000 7f5abbfff700 1 -- 172.21.15.5:0/2723187756 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5ad40ca8f0 con 0x7f5ad415d9c0 2024-05-08T23:28:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.169+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ad40ca8f0 con 0x7f5ad415d9c0 2024-05-08T23:28:30.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.170+0000 7f5abbfff700 1 -- 172.21.15.5:0/2723187756 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4766 (secure 0 0 0) 0x7f5ad40ca8f0 con 0x7f5ad415d9c0 2024-05-08T23:28:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:30.170 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":107,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:29.352928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ac006ac90 msgr2=0x7f5ac006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ac006ac90 0x7f5ac006d150 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40b0740 tx=0x7f5ac4009040 comp rx=0 tx=0).stop 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad415d9c0 msgr2=0x7f5ad40b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad415d9c0 0x7f5ad40b12c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5acc002230 tx=0x7f5acc00cf50 comp rx=0 tx=0).stop 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 shutdown_connections 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5ac006ac90 0x7f5ac006d150 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ad415d9c0 0x7f5ad40b12c0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad406b810 0x7f5ad4157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 --2- 172.21.15.5:0/2723187756 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5ad406bc70 0x7f5ad40cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 >> 172.21.15.5:0/2723187756 conn(0x7f5ad408afe0 msgr2=0x7f5ad40d35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 shutdown_connections 2024-05-08T23:28:30.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.172+0000 7f5ad994f700 1 -- 172.21.15.5:0/2723187756 wait complete. 2024-05-08T23:28:30.174 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 107 2024-05-08T23:28:30.186 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'b'] 2024-05-08T23:28:30.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:28:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.321+0000 7fcb0346b700 1 Processor -- start 2024-05-08T23:28:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.321+0000 7fcb0346b700 1 -- start start 2024-05-08T23:28:30.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcb0346b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc0d0eb0 0x7fcafc1688c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcb0346b700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcafc0cd980 con 0x7fcafc06a960 2024-05-08T23:28:30.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcb0346b700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcafc0ccdb0 con 0x7fcafc0cf840 2024-05-08T23:28:30.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcb0346b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0cd050 con 0x7fcafc0d0eb0 2024-05-08T23:28:30.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcb00a06700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcafc06a960 0x7fcafc15d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47004/0 (socket says 172.21.15.5:47004) 2024-05-08T23:28:30.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcb00a06700 1 -- 172.21.15.5:0/2445548571 learned_addr learned my addr 172.21.15.5:0/2445548571 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcb01a08700 1 --2- 172.21.15.5:0/2445548571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc0d0eb0 0x7fcafc1688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcaf3fff700 1 -- 172.21.15.5:0/2445548571 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135435129 0 0) 0x7fcafc0cd980 con 0x7fcafc06a960 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcaf3fff700 1 -- 172.21.15.5:0/2445548571 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae80035b0 con 0x7fcafc06a960 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcaf3fff700 1 -- 172.21.15.5:0/2445548571 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303004373 0 0) 0x7fcafc0ccdb0 con 0x7fcafc0cf840 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.322+0000 7fcaf3fff700 1 -- 172.21.15.5:0/2445548571 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcafc0cd980 con 0x7fcafc0cf840 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcaf3fff700 1 -- 172.21.15.5:0/2445548571 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3811939667 0 0) 0x7fcae80035b0 con 0x7fcafc06a960 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcb01a08700 1 -- 172.21.15.5:0/2445548571 >> v1:172.21.15.49:6789/0 conn(0x7fcafc0cf840 legacy=0x7fcafc0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcb01a08700 1 -- 172.21.15.5:0/2445548571 >> v1:172.21.15.5:6789/0 conn(0x7fcafc06a960 legacy=0x7fcafc15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcb01a08700 1 -- 172.21.15.5:0/2445548571 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0cd330 con 0x7fcafc0d0eb0 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcb01a08700 1 --2- 172.21.15.5:0/2445548571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc0d0eb0 0x7fcafc1688c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009dc0 tx=0x7fcaf400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da473f6f8e1aced server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcaf3fff700 1 -- 172.21.15.5:0/2445548571 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400e070 con 0x7fcafc0d0eb0 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcaf3fff700 1 -- 172.21.15.5:0/2445548571 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf4003a20 con 0x7fcafc0d0eb0 2024-05-08T23:28:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcaf3fff700 1 -- 172.21.15.5:0/2445548571 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec001670 con 0x7fcafc0d0eb0 2024-05-08T23:28:30.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcb0346b700 1 -- 172.21.15.5:0/2445548571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc0d0eb0 msgr2=0x7fcafc1688c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.323+0000 7fcb0346b700 1 --2- 172.21.15.5:0/2445548571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc0d0eb0 0x7fcafc1688c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009dc0 tx=0x7fcaf400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:30.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.324+0000 7fcb0346b700 1 -- 172.21.15.5:0/2445548571 shutdown_connections 2024-05-08T23:28:30.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.324+0000 7fcb0346b700 1 --2- 172.21.15.5:0/2445548571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc0d0eb0 0x7fcafc1688c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.324+0000 7fcb0346b700 1 -- 172.21.15.5:0/2445548571 >> 172.21.15.5:0/2445548571 conn(0x7fcafc0d0320 msgr2=0x7fcafc0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.324+0000 7fcb0346b700 1 -- 172.21.15.5:0/2445548571 shutdown_connections 2024-05-08T23:28:30.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.324+0000 7fcb0346b700 1 -- 172.21.15.5:0/2445548571 wait complete. 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.325+0000 7fcb0346b700 1 Processor -- start 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.325+0000 7fcb0346b700 1 -- start start 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb0346b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc06a960 0x7fcafc14a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb0346b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcafc0cf840 0x7fcafc14b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb0346b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcafc0d0eb0 0x7fcafc15bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb0346b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc14add0 con 0x7fcafc0cf840 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb0346b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0b4550 con 0x7fcafc0d0eb0 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb0346b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0b3fd0 con 0x7fcafc06a960 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb00a06700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcafc0cf840 0x7fcafc14b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb00a06700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcafc0cf840 0x7fcafc14b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47806/0 (socket says 172.21.15.5:47806) 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb00a06700 1 -- 172.21.15.5:0/1336592853 learned_addr learned my addr 172.21.15.5:0/1336592853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb01a08700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcafc0d0eb0 0x7fcafc15bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb01207700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc06a960 0x7fcafc14a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb00a06700 1 -- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc06a960 msgr2=0x7fcafc14a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb00a06700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc06a960 0x7fcafc14a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb00a06700 1 -- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcafc0d0eb0 msgr2=0x7fcafc15bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb00a06700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcafc0d0eb0 0x7fcafc15bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.326+0000 7fcb00a06700 1 -- 172.21.15.5:0/1336592853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4009a70 con 0x7fcafc0cf840 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.327+0000 7fcb00a06700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcafc0cf840 0x7fcafc14b240 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00c750 tx=0x7fcaec00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.327+0000 7fcaf1ffb700 1 -- 172.21.15.5:0/1336592853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec004d40 con 0x7fcafc0cf840 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.327+0000 7fcaf1ffb700 1 -- 172.21.15.5:0/1336592853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec0070d0 con 0x7fcafc0cf840 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.327+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcafc15df00 con 0x7fcafc0cf840 2024-05-08T23:28:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.327+0000 7fcaf1ffb700 1 -- 172.21.15.5:0/1336592853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec004ea0 con 0x7fcafc0cf840 2024-05-08T23:28:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.327+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcafc0b4a80 con 0x7fcafc0cf840 2024-05-08T23:28:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.328+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcafc06f330 con 0x7fcafc0cf840 2024-05-08T23:28:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.328+0000 7fcaf1ffb700 1 -- 172.21.15.5:0/1336592853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcaec007460 con 0x7fcafc0cf840 2024-05-08T23:28:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.329+0000 7fcaf1ffb700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcae006ac90 0x7fcae006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.329+0000 7fcaf1ffb700 1 -- 172.21.15.5:0/1336592853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaec0890c0 con 0x7fcafc0cf840 2024-05-08T23:28:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.329+0000 7fcb01207700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcae006ac90 0x7fcae006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.329+0000 7fcb01207700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcae006ac90 0x7fcae006d150 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80011e0 tx=0x7fcaf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.333+0000 7fcaf1ffb700 1 -- 172.21.15.5:0/1336592853 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcaec0542b0 con 0x7fcafc0cf840 2024-05-08T23:28:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.643+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcafc0d0120 con 0x7fcafc0cf840 2024-05-08T23:28:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.643+0000 7fcaf1ffb700 1 -- 172.21.15.5:0/1336592853 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1714 (secure 0 0 0) 0x7fcaec0578d0 con 0x7fcafc0cf840 2024-05-08T23:28:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:30.644 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":465,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":699774,"bytes_used":265920512,"bytes_avail":772828192768,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9639,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":107,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7052},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":7058},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8592}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-05-08T23:27:53.393882+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:28:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcae006ac90 msgr2=0x7fcae006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcae006ac90 0x7fcae006d150 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80011e0 tx=0x7fcaf8009040 comp rx=0 tx=0).stop 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcafc0cf840 msgr2=0x7fcafc14b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcafc0cf840 0x7fcafc14b240 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00c750 tx=0x7fcaec00ca60 comp rx=0 tx=0).stop 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 shutdown_connections 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fcae006ac90 0x7fcae006d150 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcafc0d0eb0 0x7fcafc15bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcafc06a960 0x7fcafc14a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 --2- 172.21.15.5:0/1336592853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcafc0cf840 0x7fcafc14b240 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 >> 172.21.15.5:0/1336592853 conn(0x7fcafc0d0320 msgr2=0x7fcafc0afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 shutdown_connections 2024-05-08T23:28:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:30.646+0000 7fcb0346b700 1 -- 172.21.15.5:0/1336592853 wait complete. 2024-05-08T23:28:30.660 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-05-08T23:28:31.661 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.806+0000 7f8cac5cb700 1 Processor -- start 2024-05-08T23:28:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.806+0000 7f8cac5cb700 1 -- start start 2024-05-08T23:28:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8cac5cb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ca415d7a0 0x7f8ca4161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8cac5cb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca40d3f40 0x7f8ca40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8cac5cb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca4162450 con 0x7f8ca415db80 2024-05-08T23:28:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8cac5cb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca40cee00 con 0x7f8ca415d7a0 2024-05-08T23:28:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8cac5cb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40b27e0 con 0x7f8ca40d3f40 2024-05-08T23:28:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8ca9b66700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca40d3f40 0x7f8ca40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8caa367700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ca415d7a0 0x7f8ca4161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8caab68700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8ca415db80 0x7f8ca40cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:42918/0 (socket says 172.21.15.5:42918) 2024-05-08T23:28:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8caab68700 1 -- 172.21.15.5:0/3451594889 learned_addr learned my addr 172.21.15.5:0/3451594889 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.807+0000 7f8ca9b66700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca40d3f40 0x7f8ca40b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:49840/0 (socket says 172.21.15.5:49840) 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8caa367700 1 -- 172.21.15.5:0/3451594889 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca40d3f40 msgr2=0x7f8ca40b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8caa367700 1 --2- 172.21.15.5:0/3451594889 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca40d3f40 0x7f8ca40b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8caa367700 1 -- 172.21.15.5:0/3451594889 >> v1:172.21.15.49:6789/0 conn(0x7f8ca415db80 legacy=0x7f8ca40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8caa367700 1 -- 172.21.15.5:0/3451594889 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40d4400 con 0x7f8ca415d7a0 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8ca9365700 1 -- 172.21.15.5:0/3451594889 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532486204 0 0) 0x7f8c9c001490 con 0x7f8ca415db80 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8ca9b66700 1 --2- 172.21.15.5:0/3451594889 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca40d3f40 0x7f8ca40b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8caa367700 1 --2- 172.21.15.5:0/3451594889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415d7a0 0x7f8ca4161f10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009840 tx=0x7f8ca0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b171b50e7a91853 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8ca9365700 1 -- 172.21.15.5:0/3451594889 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0011830 con 0x7f8ca415d7a0 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8ca9365700 1 -- 172.21.15.5:0/3451594889 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca0011990 con 0x7f8ca415d7a0 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.808+0000 7f8ca9365700 1 -- 172.21.15.5:0/3451594889 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0011d20 con 0x7f8ca415d7a0 2024-05-08T23:28:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.809+0000 7f8cac5cb700 1 -- 172.21.15.5:0/3451594889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415d7a0 msgr2=0x7f8ca4161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.809+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/3451594889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415d7a0 0x7f8ca4161f10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009840 tx=0x7f8ca0009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.809+0000 7f8cac5cb700 1 -- 172.21.15.5:0/3451594889 shutdown_connections 2024-05-08T23:28:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.809+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/3451594889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415d7a0 0x7f8ca4161f10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.809+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/3451594889 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca40d3f40 0x7f8ca40b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.809+0000 7f8cac5cb700 1 -- 172.21.15.5:0/3451594889 >> 172.21.15.5:0/3451594889 conn(0x7f8ca408afe0 msgr2=0x7f8ca40d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.809+0000 7f8cac5cb700 1 -- 172.21.15.5:0/3451594889 shutdown_connections 2024-05-08T23:28:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.809+0000 7f8cac5cb700 1 -- 172.21.15.5:0/3451594889 wait complete. 2024-05-08T23:28:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.810+0000 7f8cac5cb700 1 Processor -- start 2024-05-08T23:28:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.810+0000 7f8cac5cb700 1 -- start start 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8cac5cb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca40d3f40 0x7f8ca41584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8cac5cb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca415d7a0 0x7f8ca40b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8cac5cb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415db80 0x7f8ca4155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8cac5cb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40c82d0 con 0x7f8ca40d3f40 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8cac5cb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40c64b0 con 0x7f8ca415db80 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8cac5cb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40c7650 con 0x7f8ca415d7a0 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8caa367700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca40d3f40 0x7f8ca41584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8caa367700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca40d3f40 0x7f8ca41584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47816/0 (socket says 172.21.15.5:47816) 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8caa367700 1 -- 172.21.15.5:0/1933460487 learned_addr learned my addr 172.21.15.5:0/1933460487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8caab68700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415db80 0x7f8ca4155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.811+0000 7f8ca9b66700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca415d7a0 0x7f8ca40b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8caa367700 1 -- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca415d7a0 msgr2=0x7f8ca40b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8caa367700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca415d7a0 0x7f8ca40b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8caa367700 1 -- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415db80 msgr2=0x7f8ca4155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8caa367700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415db80 0x7f8ca4155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8caa367700 1 -- 172.21.15.5:0/1933460487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94009430 con 0x7f8ca40d3f40 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8caa367700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca40d3f40 0x7f8ca41584e0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00090b0 tx=0x7f8ca0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8c9affd700 1 -- 172.21.15.5:0/1933460487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0004550 con 0x7f8ca40d3f40 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8c9affd700 1 -- 172.21.15.5:0/1933460487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca001e070 con 0x7f8ca40d3f40 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca00094a0 con 0x7f8ca40d3f40 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8c9affd700 1 -- 172.21.15.5:0/1933460487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00038e0 con 0x7f8ca40d3f40 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.812+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca40c7c10 con 0x7f8ca40d3f40 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.813+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca40cf950 con 0x7f8ca40d3f40 2024-05-08T23:28:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.814+0000 7f8c9affd700 1 -- 172.21.15.5:0/1933460487 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8ca0003a40 con 0x7f8ca40d3f40 2024-05-08T23:28:31.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.814+0000 7f8c9affd700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c8806ac90 0x7f8c8806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:31.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.814+0000 7f8c9affd700 1 -- 172.21.15.5:0/1933460487 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ca008a190 con 0x7f8ca40d3f40 2024-05-08T23:28:31.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.814+0000 7f8ca9b66700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c8806ac90 0x7f8c8806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:31.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.815+0000 7f8ca9b66700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c8806ac90 0x7f8c8806d150 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4161000 tx=0x7f8c940091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:31.819+0000 7f8c9affd700 1 -- 172.21.15.5:0/1933460487 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8ca0055400 con 0x7f8ca40d3f40 2024-05-08T23:28:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.085+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ca4068000 con 0x7f8ca40d3f40 2024-05-08T23:28:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.085+0000 7f8c9affd700 1 -- 172.21.15.5:0/1933460487 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4767 (secure 0 0 0) 0x7f8ca0058a20 con 0x7f8ca40d3f40 2024-05-08T23:28:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:32.085 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":108,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:31.361777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c8806ac90 msgr2=0x7f8c8806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c8806ac90 0x7f8c8806d150 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4161000 tx=0x7f8c940091c0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca40d3f40 msgr2=0x7f8ca41584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca40d3f40 0x7f8ca41584e0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00090b0 tx=0x7f8ca0004750 comp rx=0 tx=0).stop 2024-05-08T23:28:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 shutdown_connections 2024-05-08T23:28:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8c8806ac90 0x7f8c8806d150 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ca415db80 0x7f8ca4155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca40d3f40 0x7f8ca41584e0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 --2- 172.21.15.5:0/1933460487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8ca415d7a0 0x7f8ca40b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 >> 172.21.15.5:0/1933460487 conn(0x7f8ca408afe0 msgr2=0x7f8ca40d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 shutdown_connections 2024-05-08T23:28:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.088+0000 7f8cac5cb700 1 -- 172.21.15.5:0/1933460487 wait complete. 2024-05-08T23:28:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 108 2024-05-08T23:28:32.101 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'b'] 2024-05-08T23:28:32.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:28:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.237+0000 7fe9f1dda700 1 Processor -- start 2024-05-08T23:28:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.237+0000 7fe9f1dda700 1 -- start start 2024-05-08T23:28:32.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9f1dda700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 0x7fe9ec0af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9f1dda700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9ec0d0eb0 0x7fe9ec161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9f1dda700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ec0cd980 con 0x7fe9ec06a960 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9f1dda700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ec0ccdb0 con 0x7fe9ec0d0eb0 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9f1dda700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0cd050 con 0x7fe9ec0cf840 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9eaffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe9ec06a960 0x7fe9ec15d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47014/0 (socket says 172.21.15.5:47014) 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9eaffd700 1 -- 172.21.15.5:0/229963574 learned_addr learned my addr 172.21.15.5:0/229963574 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9ebfff700 1 --2- 172.21.15.5:0/229963574 >> v2:172.21.15.49:3300/0 conn(0x7fe9ec0d0eb0 0x7fe9ec161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9eb7fe700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 0x7fe9ec0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481163182 0 0) 0x7fe9ec0cd980 con 0x7fe9ec06a960 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.238+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9d40035b0 con 0x7fe9ec06a960 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 241962736 0 0) 0x7fe9d40035b0 con 0x7fe9ec06a960 2024-05-08T23:28:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 msgr2=0x7fe9ec0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 0x7fe9ec0af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 >> v2:172.21.15.49:3300/0 conn(0x7fe9ec0d0eb0 msgr2=0x7fe9ec161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 --2- 172.21.15.5:0/229963574 >> v2:172.21.15.49:3300/0 conn(0x7fe9ec0d0eb0 0x7fe9ec161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec0cd330 con 0x7fe9ec06a960 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fe9e0001670 con 0x7fe9ec06a960 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eb7fe700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 0x7fe9ec0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ebfff700 1 --2- 172.21.15.5:0/229963574 >> v2:172.21.15.49:3300/0 conn(0x7fe9ec0d0eb0 0x7fe9ec161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 >> v1:172.21.15.5:6789/0 conn(0x7fe9ec06a960 legacy=0x7fe9ec15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 0x7fe9d4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d4003f90 0x7fe9d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d4004d90 0x7fe9d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0ccdb0 con 0x7fe9d4003f90 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0cd050 con 0x7fe9d4004d90 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d400ba10 con 0x7fe9ec0cf840 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eaffd700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d4003f90 0x7fe9d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eb7fe700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 0x7fe9d4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9ebfff700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d4004d90 0x7fe9d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eaffd700 1 -- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 msgr2=0x7fe9d4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eaffd700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9ec0cf840 0x7fe9d4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eaffd700 1 -- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d4004d90 msgr2=0x7fe9d4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eaffd700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d4004d90 0x7fe9d4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eaffd700 1 -- 172.21.15.5:0/229963574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0094a0 con 0x7fe9d4003f90 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.239+0000 7fe9eaffd700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d4003f90 0x7fe9d4004850 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000c160 tx=0x7fe9d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ba8800ef87931f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0007b70 con 0x7fe9d4003f90 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e000cd00 con 0x7fe9d4003f90 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9ea7fc700 1 -- 172.21.15.5:0/229963574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0005ba0 con 0x7fe9d4003f90 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9f1dda700 1 -- 172.21.15.5:0/229963574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d4003f90 msgr2=0x7fe9d4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9f1dda700 1 --2- 172.21.15.5:0/229963574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9d4003f90 0x7fe9d4004850 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000c160 tx=0x7fe9d4005880 comp rx=0 tx=0).stop 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9ebfff700 1 -- 172.21.15.5:0/229963574 reap_dead start 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9f1dda700 1 -- 172.21.15.5:0/229963574 shutdown_connections 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9f1dda700 1 -- 172.21.15.5:0/229963574 >> 172.21.15.5:0/229963574 conn(0x7fe9ec0d0320 msgr2=0x7fe9ec0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9f1dda700 1 -- 172.21.15.5:0/229963574 shutdown_connections 2024-05-08T23:28:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.240+0000 7fe9f1dda700 1 -- 172.21.15.5:0/229963574 wait complete. 2024-05-08T23:28:32.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.242+0000 7fe9f1dda700 1 Processor -- start 2024-05-08T23:28:32.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.242+0000 7fe9f1dda700 1 -- start start 2024-05-08T23:28:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9f1dda700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d4003f90 0x7fe9ec09ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9f1dda700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ec09be30 0x7fe9ec1f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9f1dda700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9ec09d620 0x7fe9ec09dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9f1dda700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec09d200 con 0x7fe9ec09be30 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9f1dda700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec09c2b0 con 0x7fe9ec09d620 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9f1dda700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec09c580 con 0x7fe9d4003f90 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ec09be30 0x7fe9ec1f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ec09be30 0x7fe9ec1f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47832/0 (socket says 172.21.15.5:47832) 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.243+0000 7fe9eaffd700 1 -- 172.21.15.5:0/776471586 learned_addr learned my addr 172.21.15.5:0/776471586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9eb7fe700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d4003f90 0x7fe9ec09ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9ebfff700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9ec09d620 0x7fe9ec09dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9eaffd700 1 -- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d4003f90 msgr2=0x7fe9ec09ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9eaffd700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d4003f90 0x7fe9ec09ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9eaffd700 1 -- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9ec09d620 msgr2=0x7fe9ec09dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9eaffd700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9ec09d620 0x7fe9ec09dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9eaffd700 1 -- 172.21.15.5:0/776471586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec15df00 con 0x7fe9ec09be30 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9eb7fe700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d4003f90 0x7fe9ec09ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9eaffd700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ec09be30 0x7fe9ec1f2a00 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4004700 tx=0x7fe9e0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9d3fff700 1 -- 172.21.15.5:0/776471586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000ce70 con 0x7fe9ec09be30 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9d3fff700 1 -- 172.21.15.5:0/776471586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e00070d0 con 0x7fe9ec09be30 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec09c860 con 0x7fe9ec09be30 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9d3fff700 1 -- 172.21.15.5:0/776471586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0007460 con 0x7fe9ec09be30 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.244+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec09ce20 con 0x7fe9ec09be30 2024-05-08T23:28:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.245+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec068000 con 0x7fe9ec09be30 2024-05-08T23:28:32.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.246+0000 7fe9d3fff700 1 -- 172.21.15.5:0/776471586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe9e0019030 con 0x7fe9ec09be30 2024-05-08T23:28:32.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.246+0000 7fe9d3fff700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9d806ac90 0x7fe9d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.246+0000 7fe9d3fff700 1 -- 172.21.15.5:0/776471586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9e008a980 con 0x7fe9ec09be30 2024-05-08T23:28:32.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.247+0000 7fe9eb7fe700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9d806ac90 0x7fe9d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.247+0000 7fe9eb7fe700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9d806ac90 0x7fe9d806d150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0045d0 tx=0x7fe9dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.251+0000 7fe9d3fff700 1 -- 172.21.15.5:0/776471586 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe9e0055bf0 con 0x7fe9ec09be30 2024-05-08T23:28:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.559+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe9ec0b62a0 con 0x7fe9ec09be30 2024-05-08T23:28:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.559+0000 7fe9d3fff700 1 -- 172.21.15.5:0/776471586 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1925 (secure 0 0 0) 0x7fe9e0059210 con 0x7fe9ec09be30 2024-05-08T23:28:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:32.560 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":467,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":699774,"bytes_used":265920512,"bytes_avail":772828192768,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9639,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":109,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7052},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":7058},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8592}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-05-08T23:28:31.403877+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:28:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.561+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9d806ac90 msgr2=0x7fe9d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.561+0000 7fe9f1dda700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9d806ac90 0x7fe9d806d150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0045d0 tx=0x7fe9dc008040 comp rx=0 tx=0).stop 2024-05-08T23:28:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.561+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ec09be30 msgr2=0x7fe9ec1f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.561+0000 7fe9f1dda700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ec09be30 0x7fe9ec1f2a00 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4004700 tx=0x7fe9e0001930 comp rx=0 tx=0).stop 2024-05-08T23:28:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.562+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 shutdown_connections 2024-05-08T23:28:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.562+0000 7fe9f1dda700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe9d806ac90 0x7fe9d806d150 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.562+0000 7fe9f1dda700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9ec09d620 0x7fe9ec09dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.562+0000 7fe9f1dda700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe9d4003f90 0x7fe9ec09ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.562+0000 7fe9f1dda700 1 --2- 172.21.15.5:0/776471586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ec09be30 0x7fe9ec1f2a00 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.562+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 >> 172.21.15.5:0/776471586 conn(0x7fe9ec0d0320 msgr2=0x7fe9ec0b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.562+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 shutdown_connections 2024-05-08T23:28:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.562+0000 7fe9f1dda700 1 -- 172.21.15.5:0/776471586 wait complete. 2024-05-08T23:28:32.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:28:32.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub abort 2024-05-08T23:28:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.714+0000 7f0b330a8700 1 Processor -- start 2024-05-08T23:28:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.714+0000 7f0b330a8700 1 -- start start 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.715+0000 7f0b330a8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c0ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.715+0000 7f0b330a8700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0b2c072170 0x7f0b2c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b330a8700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c0ae8e0 con 0x7f0b2c0ae450 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b330a8700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2c0aed90 con 0x7f0b2c072170 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b330a8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c06dcd0 con 0x7f0b2c15dc60 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b2bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0b2c072170 0x7f0b2c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b30e44700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c0ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b31645700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0b2c0ae450 0x7f0b2c0b3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:42928/0 (socket says 172.21.15.5:42928) 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b31645700 1 -- 172.21.15.5:0/1549534397 learned_addr learned my addr 172.21.15.5:0/1549534397 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b2b7fe700 1 -- 172.21.15.5:0/1549534397 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459420845 0 0) 0x7f0b2c0ae8e0 con 0x7f0b2c0ae450 2024-05-08T23:28:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b30e44700 1 -- 172.21.15.5:0/1549534397 >> v2:172.21.15.49:3300/0 conn(0x7f0b2c072170 msgr2=0x7f0b2c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b30e44700 1 --2- 172.21.15.5:0/1549534397 >> v2:172.21.15.49:3300/0 conn(0x7f0b2c072170 0x7f0b2c0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b30e44700 1 -- 172.21.15.5:0/1549534397 >> v1:172.21.15.49:6789/0 conn(0x7f0b2c0ae450 legacy=0x7f0b2c0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.716+0000 7f0b30e44700 1 -- 172.21.15.5:0/1549534397 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c06e290 con 0x7f0b2c15dc60 2024-05-08T23:28:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b30e44700 1 --2- 172.21.15.5:0/1549534397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c0ad9d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009840 tx=0x7f0b1c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35240a824dcc8b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b2b7fe700 1 -- 172.21.15.5:0/1549534397 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c011830 con 0x7f0b2c15dc60 2024-05-08T23:28:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b2b7fe700 1 -- 172.21.15.5:0/1549534397 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c011990 con 0x7f0b2c15dc60 2024-05-08T23:28:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b2b7fe700 1 -- 172.21.15.5:0/1549534397 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c011d20 con 0x7f0b2c15dc60 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b330a8700 1 -- 172.21.15.5:0/1549534397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 msgr2=0x7f0b2c0ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b330a8700 1 --2- 172.21.15.5:0/1549534397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c0ad9d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009840 tx=0x7f0b1c009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b330a8700 1 -- 172.21.15.5:0/1549534397 shutdown_connections 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b330a8700 1 --2- 172.21.15.5:0/1549534397 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c0ad9d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b330a8700 1 --2- 172.21.15.5:0/1549534397 >> v2:172.21.15.49:3300/0 conn(0x7f0b2c072170 0x7f0b2c0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b330a8700 1 -- 172.21.15.5:0/1549534397 >> 172.21.15.5:0/1549534397 conn(0x7f0b2c0b5a30 msgr2=0x7f0b2c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b330a8700 1 -- 172.21.15.5:0/1549534397 shutdown_connections 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.717+0000 7f0b330a8700 1 -- 172.21.15.5:0/1549534397 wait complete. 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.719+0000 7f0b330a8700 1 Processor -- start 2024-05-08T23:28:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.719+0000 7f0b330a8700 1 -- start start 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b330a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b2c072170 0x7f0b2c150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b330a8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2c0ae450 0x7f0b2c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b330a8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c1e5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b330a8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c15b670 con 0x7f0b2c0ae450 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b330a8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c1e55b0 con 0x7f0b2c072170 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b330a8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c14ec90 con 0x7f0b2c15dc60 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2c0ae450 0x7f0b2c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2c0ae450 0x7f0b2c15b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47848/0 (socket says 172.21.15.5:47848) 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 -- 172.21.15.5:0/3835966052 learned_addr learned my addr 172.21.15.5:0/3835966052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b30e44700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b2c072170 0x7f0b2c150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b31645700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c1e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 -- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 msgr2=0x7f0b2c1e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c1e5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 -- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b2c072170 msgr2=0x7f0b2c150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b2c072170 0x7f0b2c150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 -- 172.21.15.5:0/3835966052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c0094a0 con 0x7f0b2c0ae450 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.720+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2c0ae450 0x7f0b2c15b130 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000e9b0 tx=0x7f0b2000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.721+0000 7f0b297fa700 1 -- 172.21.15.5:0/3835966052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20009710 con 0x7f0b2c0ae450 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.721+0000 7f0b297fa700 1 -- 172.21.15.5:0/3835966052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2000ee50 con 0x7f0b2c0ae450 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.721+0000 7f0b330a8700 1 -- 172.21.15.5:0/3835966052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c15b8a0 con 0x7f0b2c0ae450 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.721+0000 7f0b297fa700 1 -- 172.21.15.5:0/3835966052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000ca50 con 0x7f0b2c0ae450 2024-05-08T23:28:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.721+0000 7f0b330a8700 1 -- 172.21.15.5:0/3835966052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c15be60 con 0x7f0b2c0ae450 2024-05-08T23:28:32.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.722+0000 7f0b297fa700 1 -- 172.21.15.5:0/3835966052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0b2000cbb0 con 0x7f0b2c0ae450 2024-05-08T23:28:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.723+0000 7f0b297fa700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b1806ac90 0x7f0b1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.723+0000 7f0b297fa700 1 -- 172.21.15.5:0/3835966052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b20089240 con 0x7f0b2c0ae450 2024-05-08T23:28:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.723+0000 7f0b30e44700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b1806ac90 0x7f0b1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.724+0000 7f0b30e44700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b1806ac90 0x7f0b1806d150 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009fd0 tx=0x7f0b1c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.725+0000 7f0b3f262b80 1 Processor -- start 2024-05-08T23:28:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.725+0000 7f0b3f262b80 1 -- start start 2024-05-08T23:28:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be04c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be051090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bdfa7140 con 0x5561be04e360 2024-05-08T23:28:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bdfa4b00 con 0x5561be073790 2024-05-08T23:28:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561bdfa03f0 con 0x5561be07adc0 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b2bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be051090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b2bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be051090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47862/0 (socket says 172.21.15.5:47862) 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b2bfff700 1 -- 172.21.15.5:0/4246384653 learned_addr learned my addr 172.21.15.5:0/4246384653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b30e44700 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be04c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b31645700 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b2bfff700 1 -- 172.21.15.5:0/4246384653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 msgr2=0x5561be07b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b2bfff700 1 -- 172.21.15.5:0/4246384653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 msgr2=0x5561be04c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be04c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.726+0000 7f0b2bfff700 1 -- 172.21.15.5:0/4246384653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561bde213a0 con 0x5561be04e360 2024-05-08T23:28:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.727+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be051090 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f0b200877a0 tx=0x7f0b2000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2047d74e17f1caf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.727+0000 7f0b127fc700 1 -- 172.21.15.5:0/4246384653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2005bfa0 con 0x5561be04e360 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.727+0000 7f0b127fc700 1 -- 172.21.15.5:0/4246384653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b20060560 con 0x5561be04e360 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.727+0000 7f0b127fc700 1 -- 172.21.15.5:0/4246384653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2005e0c0 con 0x5561be04e360 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.727+0000 7f0b3f262b80 1 -- 172.21.15.5:0/4246384653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 msgr2=0x5561be051090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.727+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be051090 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f0b200877a0 tx=0x7f0b2000cf60 comp rx=0 tx=0).stop 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.728+0000 7f0b3f262b80 1 -- 172.21.15.5:0/4246384653 shutdown_connections 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.728+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.728+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be04c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.728+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/4246384653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be051090 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.728+0000 7f0b3f262b80 1 -- 172.21.15.5:0/4246384653 >> 172.21.15.5:0/4246384653 conn(0x5561bdfae200 msgr2=0x5561be095dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.728+0000 7f0b3f262b80 1 -- 172.21.15.5:0/4246384653 shutdown_connections 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.728+0000 7f0b3f262b80 1 -- 172.21.15.5:0/4246384653 wait complete. 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.728+0000 7f0b3f262b80 1 -- start start 2024-05-08T23:28:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.729+0000 7f0b3f262b80 1 Processor -- start 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.729+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be04e360 0x5561be07e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.729+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be097240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.729+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be0998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.729+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bdf3c550 con 0x5561be07adc0 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.729+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bdf3c1f0 con 0x5561be073790 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.729+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561bde23690 con 0x5561be04e360 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.729+0000 7f0b3f262b80 10 client.0 _ll_register_callbacks cb 0x5561bdf45b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be0998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be0998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47874/0 (socket says 172.21.15.5:47874) 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 -- 172.21.15.5:0/3923120853 learned_addr learned my addr 172.21.15.5:0/3923120853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be097240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b30e44700 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be04e360 0x5561be07e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 -- 172.21.15.5:0/3923120853 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be04e360 msgr2=0x5561be07e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be04e360 0x5561be07e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 -- 172.21.15.5:0/3923120853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 msgr2=0x5561be097240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be097240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 -- 172.21.15.5:0/3923120853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c15be60 con 0x5561be07adc0 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b31645700 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be0998c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f0b24007bd0 tx=0x7f0b240125a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e223f885c69e994 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/3923120853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24012c90 con 0x5561be07adc0 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/3923120853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b24012df0 con 0x5561be07adc0 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b3f262b80 1 -- 172.21.15.5:0/3923120853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561bdef9260 con 0x5561be07adc0 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/3923120853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b24015440 con 0x5561be07adc0 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b3f262b80 10 client.9057 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.730+0000 7f0b3f262b80 1 -- 172.21.15.5:0/3923120853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x5561bddcdb50 con 0x5561be07adc0 2024-05-08T23:28:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/3923120853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561bdef9260 con 0x5561be07adc0 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/3923120853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f0b240155a0 con 0x5561be07adc0 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b3f262b80 10 client.9057 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b3f262b80 10 client.9057 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b3f262b80 20 client.9057 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5561be097780 0x5561be09d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b3f262b80 4 client.9057 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b3f262b80 1 -- 172.21.15.5:0/3923120853 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561be01f7d0 con 0x5561be097780 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b30e44700 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5561be097780 0x5561be09d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b0b7fe700 10 client.9057 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.731+0000 7f0b30e44700 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5561be097780 0x5561be09d270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed34bb1e0d02361f server_cookie=cf12c52d7656ac8c in_seq=0 out_seq=0 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/3923120853 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561be01f7d0 con 0x5561be097780 2024-05-08T23:28:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b0b7fe700 10 client.9057 handle_command_reply: tid=0 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 client.9057 shutdown 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 -- 172.21.15.5:0/3923120853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 msgr2=0x5561be0998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be0998c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f0b24007bd0 tx=0x7f0b240125a0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 -- 172.21.15.5:0/3923120853 shutdown_connections 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5561be097780 0x5561be09d270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be0998c0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be04e360 0x5561be07e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b0b7fe700 0 client.9057 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/3923120853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be097240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 -- 172.21.15.5:0/3923120853 >> 172.21.15.5:0/3923120853 conn(0x5561bdfae200 msgr2=0x5561be050300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 -- 172.21.15.5:0/3923120853 shutdown_connections 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 1 -- 172.21.15.5:0/3923120853 wait complete. 2024-05-08T23:28:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.732+0000 7f0b3f262b80 20 client.9057 trim_cache size 0 max 16384 2024-05-08T23:28:32.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.750+0000 7f0b3f262b80 1 Processor -- start 2024-05-08T23:28:32.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.750+0000 7f0b3f262b80 1 -- start start 2024-05-08T23:28:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be09dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561be09db30 con 0x5561be04e360 2024-05-08T23:28:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561be09a490 con 0x5561be073790 2024-05-08T23:28:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561be052330 con 0x5561be07adc0 2024-05-08T23:28:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b30e44700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be09dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b30e44700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be09dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47890/0 (socket says 172.21.15.5:47890) 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b30e44700 1 -- 172.21.15.5:0/56786379 learned_addr learned my addr 172.21.15.5:0/56786379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b31645700 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b30e44700 1 -- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 msgr2=0x5561be07b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b30e44700 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b30e44700 1 -- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 msgr2=0x5561be07b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b30e44700 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be07b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b30e44700 1 -- 172.21.15.5:0/56786379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561bdf66550 con 0x5561be04e360 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.751+0000 7f0b31645700 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b30e44700 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be09dff0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0090b0 tx=0x7f0b1c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b153ab9012fd0ab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b12ffd700 1 -- 172.21.15.5:0/56786379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0038a0 con 0x5561be04e360 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b12ffd700 1 -- 172.21.15.5:0/56786379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c03a070 con 0x5561be04e360 2024-05-08T23:28:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b12ffd700 1 -- 172.21.15.5:0/56786379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c003c20 con 0x5561be04e360 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 -- 172.21.15.5:0/56786379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 msgr2=0x5561be09dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be09dff0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0090b0 tx=0x7f0b1c010000 comp rx=0 tx=0).stop 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 -- 172.21.15.5:0/56786379 shutdown_connections 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be07adc0 0x5561be07b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be04e360 0x5561be09dff0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/56786379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be073790 0x5561be07b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 -- 172.21.15.5:0/56786379 >> 172.21.15.5:0/56786379 conn(0x5561bdfae200 msgr2=0x5561be04ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 -- 172.21.15.5:0/56786379 shutdown_connections 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.752+0000 7f0b3f262b80 1 -- 172.21.15.5:0/56786379 wait complete. 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.753+0000 7f0b3f262b80 1 -- start start 2024-05-08T23:28:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.753+0000 7f0b3f262b80 1 Processor -- start 2024-05-08T23:28:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be04e360 0x5561be08b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be073790 0x5561be0506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b3f262b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be049cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561be09e1a0 con 0x5561be07adc0 2024-05-08T23:28:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561be072670 con 0x5561be04e360 2024-05-08T23:28:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b3f262b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561be09e3f0 con 0x5561be073790 2024-05-08T23:28:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b3f262b80 10 client.0 _ll_register_callbacks cb 0x7f0b24007460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b31645700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be049cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b31645700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be049cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47906/0 (socket says 172.21.15.5:47906) 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b31645700 1 -- 172.21.15.5:0/1384119885 learned_addr learned my addr 172.21.15.5:0/1384119885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b30e44700 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be04e360 0x5561be08b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.754+0000 7f0b2bfff700 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be073790 0x5561be0506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b31645700 1 -- 172.21.15.5:0/1384119885 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be073790 msgr2=0x5561be0506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b31645700 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be073790 0x5561be0506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b31645700 1 -- 172.21.15.5:0/1384119885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be04e360 msgr2=0x5561be08b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b31645700 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be04e360 0x5561be08b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b31645700 1 -- 172.21.15.5:0/1384119885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c06e290 con 0x5561be07adc0 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b31645700 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be049cd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f0b240129b0 tx=0x7f0b24001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62fb0c1313f3691 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/1384119885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24015640 con 0x5561be07adc0 2024-05-08T23:28:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/1384119885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b2402fad0 con 0x5561be07adc0 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 1 -- 172.21.15.5:0/1384119885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b20060560 con 0x5561be07adc0 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/1384119885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b2402fc90 con 0x5561be07adc0 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 10 client.9063 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 1 -- 172.21.15.5:0/1384119885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x5561bde25cc0 con 0x5561be07adc0 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/1384119885 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b20060560 con 0x5561be07adc0 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/1384119885 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f0b2402f020 con 0x5561be07adc0 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 10 client.9063 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 10 client.9063 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 20 client.9063 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5561be04c510 0x5561be050cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 4 client.9063 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.755+0000 7f0b3f262b80 1 -- 172.21.15.5:0/1384119885 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5561bdf98eb0 con 0x5561be04c510 2024-05-08T23:28:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b30e44700 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5561be04c510 0x5561be050cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b0b7fe700 10 client.9063 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b30e44700 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5561be04c510 0x5561be050cb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c5e0272ce97781 server_cookie=66f7ce2892220312 in_seq=0 out_seq=0 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b0b7fe700 1 -- 172.21.15.5:0/1384119885 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5561bdf98eb0 con 0x5561be04c510 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b0b7fe700 10 client.9063 handle_command_reply: tid=0 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 client.9063 shutdown 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 -- 172.21.15.5:0/1384119885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 msgr2=0x5561be049cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be049cd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f0b240129b0 tx=0x7f0b24001400 comp rx=0 tx=0).stop 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 -- 172.21.15.5:0/1384119885 shutdown_connections 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x5561be04c510 0x5561be050cb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561be07adc0 0x5561be049cd0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5561be04e360 0x5561be08b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b0b7fe700 0 client.9063 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 --2- 172.21.15.5:0/1384119885 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x5561be073790 0x5561be0506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.756+0000 7f0b3f262b80 1 -- 172.21.15.5:0/1384119885 >> 172.21.15.5:0/1384119885 conn(0x5561bdfae200 msgr2=0x5561be07b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b3f262b80 1 -- 172.21.15.5:0/1384119885 shutdown_connections 2024-05-08T23:28:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b3f262b80 1 -- 172.21.15.5:0/1384119885 wait complete. 2024-05-08T23:28:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b3f262b80 20 client.9063 trim_cache size 0 max 16384 2024-05-08T23:28:32.763 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:28:32.763 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 -- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b1806ac90 msgr2=0x7f0b1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b1806ac90 0x7f0b1806d150 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009fd0 tx=0x7f0b1c01c040 comp rx=0 tx=0).stop 2024-05-08T23:28:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 -- 172.21.15.5:0/3835966052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2c0ae450 msgr2=0x7f0b2c15b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2c0ae450 0x7f0b2c15b130 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000e9b0 tx=0x7f0b2000ecc0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 -- 172.21.15.5:0/3835966052 shutdown_connections 2024-05-08T23:28:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0b1806ac90 0x7f0b1806d150 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0b2c15dc60 0x7f0b2c1e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b2c072170 0x7f0b2c150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 --2- 172.21.15.5:0/3835966052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2c0ae450 0x7f0b2c15b130 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 -- 172.21.15.5:0/3835966052 >> 172.21.15.5:0/3835966052 conn(0x7f0b2c0b5a30 msgr2=0x7f0b2c154e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 -- 172.21.15.5:0/3835966052 shutdown_connections 2024-05-08T23:28:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.757+0000 7f0b330a8700 1 -- 172.21.15.5:0/3835966052 wait complete. 2024-05-08T23:28:32.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 scrub status 2024-05-08T23:28:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.908+0000 7fb992ae1700 1 Processor -- start 2024-05-08T23:28:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.909+0000 7fb992ae1700 1 -- start start 2024-05-08T23:28:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.909+0000 7fb992ae1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.909+0000 7fb992ae1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c0b1200 con 0x7fb98c06bd60 2024-05-08T23:28:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.909+0000 7fb992ae1700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c0d1cc0 con 0x7fb98c0cdc90 2024-05-08T23:28:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.909+0000 7fb992ae1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c0ce5c0 con 0x7fb98c0d13a0 2024-05-08T23:28:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb983fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb98c06bd60 0x7fb98c0cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47024/0 (socket says 172.21.15.5:47024) 2024-05-08T23:28:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb983fff700 1 -- 172.21.15.5:0/518339706 learned_addr learned my addr 172.21.15.5:0/518339706 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb99087d700 1 --2- 172.21.15.5:0/518339706 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb9837fe700 1 -- 172.21.15.5:0/518339706 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762328651 0 0) 0x7fb98c0b1200 con 0x7fb98c06bd60 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb9837fe700 1 -- 172.21.15.5:0/518339706 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9740035b0 con 0x7fb98c06bd60 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb9837fe700 1 -- 172.21.15.5:0/518339706 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130195097 0 0) 0x7fb98c0d1cc0 con 0x7fb98c0cdc90 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb9837fe700 1 -- 172.21.15.5:0/518339706 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb98c0b1200 con 0x7fb98c0cdc90 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb9837fe700 1 -- 172.21.15.5:0/518339706 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1919898163 0 0) 0x7fb9740035b0 con 0x7fb98c06bd60 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb99087d700 1 -- 172.21.15.5:0/518339706 >> v1:172.21.15.49:6789/0 conn(0x7fb98c0cdc90 legacy=0x7fb98c0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb99087d700 1 -- 172.21.15.5:0/518339706 >> v1:172.21.15.5:6789/0 conn(0x7fb98c06bd60 legacy=0x7fb98c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb99087d700 1 -- 172.21.15.5:0/518339706 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c0ce8a0 con 0x7fb98c0d13a0 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.910+0000 7fb99087d700 1 --2- 172.21.15.5:0/518339706 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0d1780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb988009870 tx=0x7fb988009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf118603d2a82e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb9837fe700 1 -- 172.21.15.5:0/518339706 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c001670 con 0x7fb98c0d13a0 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb9837fe700 1 -- 172.21.15.5:0/518339706 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb988004030 con 0x7fb98c0d13a0 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb9837fe700 1 -- 172.21.15.5:0/518339706 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9880043c0 con 0x7fb98c0d13a0 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb992ae1700 1 -- 172.21.15.5:0/518339706 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 msgr2=0x7fb98c0d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb992ae1700 1 --2- 172.21.15.5:0/518339706 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0d1780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb988009870 tx=0x7fb988009b80 comp rx=0 tx=0).stop 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb992ae1700 1 -- 172.21.15.5:0/518339706 shutdown_connections 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb992ae1700 1 --2- 172.21.15.5:0/518339706 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0d1780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb992ae1700 1 -- 172.21.15.5:0/518339706 >> 172.21.15.5:0/518339706 conn(0x7fb98c08afe0 msgr2=0x7fb98c08b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb992ae1700 1 -- 172.21.15.5:0/518339706 shutdown_connections 2024-05-08T23:28:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.911+0000 7fb992ae1700 1 -- 172.21.15.5:0/518339706 wait complete. 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.913+0000 7fb992ae1700 1 Processor -- start 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.913+0000 7fb992ae1700 1 -- start start 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb992ae1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98c06bd60 0x7fb98c0b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb992ae1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c0cdc90 0x7fb98c156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb992ae1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb992ae1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c0b4520 con 0x7fb98c06bd60 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb992ae1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c0b4800 con 0x7fb98c0cdc90 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb992ae1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c0b4ae0 con 0x7fb98c0d13a0 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98c06bd60 0x7fb98c0b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98c06bd60 0x7fb98c0b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47914/0 (socket says 172.21.15.5:47914) 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 -- 172.21.15.5:0/3031711895 learned_addr learned my addr 172.21.15.5:0/3031711895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb983fff700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c0cdc90 0x7fb98c156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 -- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 msgr2=0x7fb98c0b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99107e700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 -- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c0cdc90 msgr2=0x7fb98c156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c0cdc90 0x7fb98c156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 -- 172.21.15.5:0/3031711895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880094a0 con 0x7fb98c06bd60 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb983fff700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c0cdc90 0x7fb98c156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99107e700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb99087d700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98c06bd60 0x7fb98c0b55c0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb988000c00 tx=0x7fb988004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb9817fa700 1 -- 172.21.15.5:0/3031711895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988011630 con 0x7fb98c06bd60 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb9817fa700 1 -- 172.21.15.5:0/3031711895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98801f070 con 0x7fb98c06bd60 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb992ae1700 1 -- 172.21.15.5:0/3031711895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c0b4dc0 con 0x7fb98c06bd60 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.914+0000 7fb9817fa700 1 -- 172.21.15.5:0/3031711895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9880119f0 con 0x7fb98c06bd60 2024-05-08T23:28:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.915+0000 7fb992ae1700 1 -- 172.21.15.5:0/3031711895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c157420 con 0x7fb98c06bd60 2024-05-08T23:28:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.916+0000 7fb9817fa700 1 -- 172.21.15.5:0/3031711895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb988011b50 con 0x7fb98c06bd60 2024-05-08T23:28:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.916+0000 7fb9817fa700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97806ac90 0x7fb97806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.917+0000 7fb9817fa700 1 -- 172.21.15.5:0/3031711895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb988021070 con 0x7fb98c06bd60 2024-05-08T23:28:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.917+0000 7fb983fff700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97806ac90 0x7fb97806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.917+0000 7fb983fff700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97806ac90 0x7fb97806d150 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0c9780 tx=0x7fb97c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.918+0000 7fb99ec9bb80 1 Processor -- start 2024-05-08T23:28:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99ec9bb80 1 -- start start 2024-05-08T23:28:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 0x558d997a9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 0x558d997ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d99704150 con 0x558d997d7dd0 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d99701b10 con 0x558d997d07a0 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d996fd400 con 0x558d997ab370 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99107e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99107e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997d8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47928/0 (socket says 172.21.15.5:47928) 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.919+0000 7fb99107e700 1 -- 172.21.15.5:0/3344786403 learned_addr learned my addr 172.21.15.5:0/3344786403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb99087d700 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 0x558d997a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb983fff700 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 0x558d997ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb99107e700 1 -- 172.21.15.5:0/3344786403 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 msgr2=0x558d997ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb99107e700 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 0x558d997ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb99107e700 1 -- 172.21.15.5:0/3344786403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 msgr2=0x558d997a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb99107e700 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 0x558d997a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb99107e700 1 -- 172.21.15.5:0/3344786403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d9957e3b0 con 0x558d997d7dd0 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb99107e700 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997d8250 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb984002190 tx=0x7fb98400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0dc0da0fab051f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb9727fc700 1 -- 172.21.15.5:0/3344786403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400dea0 con 0x558d997d7dd0 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb9727fc700 1 -- 172.21.15.5:0/3344786403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9840123f0 con 0x558d997d7dd0 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.920+0000 7fb9727fc700 1 -- 172.21.15.5:0/3344786403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984012760 con 0x558d997d7dd0 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3344786403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 msgr2=0x558d997d8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997d8250 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb984002190 tx=0x7fb98400d430 comp rx=0 tx=0).stop 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3344786403 shutdown_connections 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997d8250 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 0x558d997a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3344786403 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 0x558d997ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3344786403 >> 172.21.15.5:0/3344786403 conn(0x558d9970b210 msgr2=0x558d997f2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3344786403 shutdown_connections 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.921+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3344786403 wait complete. 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.922+0000 7fb99ec9bb80 1 -- start start 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.922+0000 7fb99ec9bb80 1 Processor -- start 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997f1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 0x558d997f3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 0x558d99801e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d99699560 con 0x558d997ab370 2024-05-08T23:28:32.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d99699200 con 0x558d997d7dd0 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d995806a0 con 0x558d997d07a0 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99ec9bb80 10 client.0 _ll_register_callbacks cb 0x558d996a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997f1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47936/0 (socket says 172.21.15.5:47936) 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 -- 172.21.15.5:0/3008029112 learned_addr learned my addr 172.21.15.5:0/3008029112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99107e700 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 0x558d99801e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb983fff700 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 0x558d997f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 -- 172.21.15.5:0/3008029112 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 msgr2=0x558d997f3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 0x558d997f3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 -- 172.21.15.5:0/3008029112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 msgr2=0x558d99801e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 0x558d99801e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 -- 172.21.15.5:0/3008029112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c0b1200 con 0x558d997ab370 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb99087d700 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997f1260 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fb988059b50 tx=0x7fb9880046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c625a0a7881982a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.923+0000 7fb96b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb96b7fe700 1 -- 172.21.15.5:0/3008029112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988062000 con 0x558d997ab370 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb96b7fe700 1 -- 172.21.15.5:0/3008029112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb988059e70 con 0x558d997ab370 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3008029112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d99656270 con 0x558d997ab370 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb96b7fe700 1 -- 172.21.15.5:0/3008029112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb98805a030 con 0x558d997ab370 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 10 client.9075 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3008029112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x558d9952ab60 con 0x558d997ab370 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb96b7fe700 1 -- 172.21.15.5:0/3008029112 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558d99656270 con 0x558d997ab370 2024-05-08T23:28:32.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb96b7fe700 1 -- 172.21.15.5:0/3008029112 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fb988059100 con 0x558d997ab370 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 10 client.9075 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 10 client.9075 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 20 client.9075 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x558d997f6620 0x558d997f4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 4 client.9075 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3008029112 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d9977c7e0 con 0x558d997f6620 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.924+0000 7fb983fff700 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x558d997f6620 0x558d997f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb983fff700 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x558d997f6620 0x558d997f4250 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7460cdbb2d7c950 server_cookie=730d2c53ff7f8a8a in_seq=0 out_seq=0 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb96b7fe700 10 client.9075 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:32.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb96b7fe700 1 -- 172.21.15.5:0/3008029112 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d9977c7e0 con 0x558d997f6620 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb96b7fe700 10 client.9075 handle_command_reply: tid=0 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb99ec9bb80 1 client.9075 shutdown 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3008029112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 msgr2=0x558d997f1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997f1260 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fb988059b50 tx=0x7fb9880046e0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3008029112 shutdown_connections 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.925+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x558d997f6620 0x558d997f4250 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.926+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 0x558d99801e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.926+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997f1260 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.926+0000 7fb96b7fe700 0 client.9075 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.926+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/3008029112 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 0x558d997f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.926+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3008029112 >> 172.21.15.5:0/3008029112 conn(0x558d9970b210 msgr2=0x558d997d81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.926+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3008029112 shutdown_connections 2024-05-08T23:28:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.926+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/3008029112 wait complete. 2024-05-08T23:28:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.926+0000 7fb99ec9bb80 20 client.9075 trim_cache size 0 max 16384 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.944+0000 7fb99ec9bb80 1 Processor -- start 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.944+0000 7fb99ec9bb80 1 -- start start 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 0x558d997a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 0x558d997adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d997a9870 con 0x558d997ab370 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d997f3460 con 0x558d997d7dd0 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d997f4250 con 0x558d997d07a0 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47946/0 (socket says 172.21.15.5:47946) 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 -- 172.21.15.5:0/1790118845 learned_addr learned my addr 172.21.15.5:0/1790118845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 -- 172.21.15.5:0/1790118845 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 msgr2=0x558d997a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 --2- 172.21.15.5:0/1790118845 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 0x558d997a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 -- 172.21.15.5:0/1790118845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 msgr2=0x558d997adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99107e700 1 --2- 172.21.15.5:0/1790118845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 0x558d997adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 --2- 172.21.15.5:0/1790118845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 0x558d997adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 -- 172.21.15.5:0/1790118845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d996c3560 con 0x558d997ab370 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb99087d700 1 --2- 172.21.15.5:0/1790118845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997d0000 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fb988056110 tx=0x7fb98806e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a64359f6872fef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb972ffd700 1 -- 172.21.15.5:0/1790118845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988059c10 con 0x558d997ab370 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb972ffd700 1 -- 172.21.15.5:0/1790118845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb988062000 con 0x558d997ab370 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.945+0000 7fb972ffd700 1 -- 172.21.15.5:0/1790118845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988059100 con 0x558d997ab370 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/1790118845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 msgr2=0x558d997d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/1790118845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997d0000 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fb988056110 tx=0x7fb98806e000 comp rx=0 tx=0).stop 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/1790118845 shutdown_connections 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/1790118845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d7dd0 0x558d997adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/1790118845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997ab370 0x558d997d0000 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/1790118845 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997d07a0 0x558d997a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/1790118845 >> 172.21.15.5:0/1790118845 conn(0x558d9970b210 msgr2=0x558d99801ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/1790118845 shutdown_connections 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.946+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/1790118845 wait complete. 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.947+0000 7fb99ec9bb80 1 -- start start 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.947+0000 7fb99ec9bb80 1 Processor -- start 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 0x558d997f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 0x558d9985b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99ec9bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997f4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d997ae4e0 con 0x558d997d7dd0 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9985b880 con 0x558d997d07a0 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99ec9bb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d9985bcb0 con 0x558d997ab370 2024-05-08T23:28:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99ec9bb80 10 client.0 _ll_register_callbacks cb 0x558d996a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47956/0 (socket says 172.21.15.5:47956) 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 -- 172.21.15.5:0/2956676499 learned_addr learned my addr 172.21.15.5:0/2956676499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb983fff700 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 0x558d9985b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99087d700 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 0x558d997f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 -- 172.21.15.5:0/2956676499 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 msgr2=0x558d997f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 0x558d997f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 -- 172.21.15.5:0/2956676499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 msgr2=0x558d9985b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 0x558d9985b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 -- 172.21.15.5:0/2956676499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d9952ab60 con 0x558d997d7dd0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb99107e700 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997f4940 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fb984015040 tx=0x7fb984010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f63e326f01e6930 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.948+0000 7fb96b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb96b7fe700 1 -- 172.21.15.5:0/2956676499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984021040 con 0x558d997d7dd0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb96b7fe700 1 -- 172.21.15.5:0/2956676499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9840050b0 con 0x558d997d7dd0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/2956676499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9840123f0 con 0x558d997d7dd0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb96b7fe700 1 -- 172.21.15.5:0/2956676499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb984005270 con 0x558d997d7dd0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 10 client.9081 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/2956676499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x558d99582cd0 con 0x558d997d7dd0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb96b7fe700 1 -- 172.21.15.5:0/2956676499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9840123f0 con 0x558d997d7dd0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb96b7fe700 1 -- 172.21.15.5:0/2956676499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fb9840053d0 con 0x558d997d7dd0 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 10 client.9081 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 10 client.9081 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 20 client.9081 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x558d997f4e80 0x558d997d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 4 client.9081 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/2956676499 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558d996f5ec0 con 0x558d997f4e80 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.949+0000 7fb99087d700 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x558d997f4e80 0x558d997d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99087d700 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x558d997f4e80 0x558d997d3920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fa07ff4a5e1790b server_cookie=58b92cc367705c9e in_seq=0 out_seq=0 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb96b7fe700 10 client.9081 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb96b7fe700 1 -- 172.21.15.5:0/2956676499 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x558d996f5ec0 con 0x558d997f4e80 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb96b7fe700 10 client.9081 handle_command_reply: tid=0 2024-05-08T23:28:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 client.9081 shutdown 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/2956676499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 msgr2=0x558d997f4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997f4940 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fb984015040 tx=0x7fb984010000 comp rx=0 tx=0).stop 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/2956676499 shutdown_connections 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x558d997f4e80 0x558d997d3920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d997d7dd0 0x558d997f4940 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d997ab370 0x558d997f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb96b7fe700 0 client.9081 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 --2- 172.21.15.5:0/2956676499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d997d07a0 0x558d9985b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/2956676499 >> 172.21.15.5:0/2956676499 conn(0x558d9970b210 msgr2=0x558d997cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/2956676499 shutdown_connections 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 1 -- 172.21.15.5:0/2956676499 wait complete. 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.950+0000 7fb99ec9bb80 20 client.9081 trim_cache size 0 max 16384 2024-05-08T23:28:32.958 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:32.959 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 -- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97806ac90 msgr2=0x7fb97806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97806ac90 0x7fb97806d150 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0c9780 tx=0x7fb97c009040 comp rx=0 tx=0).stop 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 -- 172.21.15.5:0/3031711895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98c06bd60 msgr2=0x7fb98c0b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98c06bd60 0x7fb98c0b55c0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb988000c00 tx=0x7fb988004750 comp rx=0 tx=0).stop 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 -- 172.21.15.5:0/3031711895 shutdown_connections 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb97806ac90 0x7fb97806d150 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb98c0d13a0 0x7fb98c0b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98c06bd60 0x7fb98c0b55c0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 --2- 172.21.15.5:0/3031711895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb98c0cdc90 0x7fb98c156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 -- 172.21.15.5:0/3031711895 >> 172.21.15.5:0/3031711895 conn(0x7fb98c08afe0 msgr2=0x7fb98c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 -- 172.21.15.5:0/3031711895 shutdown_connections 2024-05-08T23:28:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:32.951+0000 7fb992ae1700 1 -- 172.21.15.5:0/3031711895 wait complete. 2024-05-08T23:28:32.964 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:32.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 scrub status 2024-05-08T23:28:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.103+0000 7f10aa7fb700 1 Processor -- start 2024-05-08T23:28:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.103+0000 7f10aa7fb700 1 -- start start 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10aa7fb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 0x7f10a40b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10aa7fb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40b4c70 con 0x7f10a40b0d00 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10aa7fb700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a415aab0 con 0x7f10a415a6d0 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10aa7fb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b2c40 con 0x7f10a40ae050 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10a37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f10a40b0d00 0x7f10a40b32f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47032/0 (socket says 172.21.15.5:47032) 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10a37fe700 1 -- 172.21.15.5:0/1120351191 learned_addr learned my addr 172.21.15.5:0/1120351191 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 0x7f10a40b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521582221 0 0) 0x7f10a40b4c70 con 0x7f10a40b0d00 2024-05-08T23:28:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.104+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10900035b0 con 0x7f10a40b0d00 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003240181 0 0) 0x7f10a415aab0 con 0x7f10a415a6d0 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a40b4c70 con 0x7f10a415a6d0 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3679172574 0 0) 0x7f10900035b0 con 0x7f10a40b0d00 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 msgr2=0x7f10a40b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 0x7f10a40b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 >> v1:172.21.15.49:6789/0 conn(0x7f10a415a6d0 legacy=0x7f10a40ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40b4990 con 0x7f10a40b0d00 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f1094001670 con 0x7f10a40b0d00 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 0x7f10a40b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 >> v1:172.21.15.5:6789/0 conn(0x7f10a40b0d00 legacy=0x7f10a40b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1090003bc0 0x7f1090003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 0x7f1090004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1090004d90 0x7f1090005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b2c40 con 0x7f1090004d90 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109000dd00 con 0x7f1090003bc0 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109000dec0 con 0x7f10a40ae050 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1090004d90 0x7f1090005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a3fff700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1090003bc0 0x7f1090003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a37fe700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 0x7f1090004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a8d98700 1 -- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 msgr2=0x7f1090004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a40ae050 0x7f1090004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a8d98700 1 -- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1090003bc0 msgr2=0x7f1090003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1090003bc0 0x7f1090003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a8d98700 1 -- 172.21.15.5:0/1120351191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c009a70 con 0x7f1090004d90 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.105+0000 7f10a3fff700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1090003bc0 0x7f1090003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1090004d90 0x7f1090005210 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f109c00e350 tx=0x7f1090005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c6dec28d7a98d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00edf0 con 0x7f1090004d90 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c00f040 con 0x7f1090004d90 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10a2ffd700 1 -- 172.21.15.5:0/1120351191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c0146a0 con 0x7f1090004d90 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10aa7fb700 1 -- 172.21.15.5:0/1120351191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1090004d90 msgr2=0x7f1090005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10aa7fb700 1 --2- 172.21.15.5:0/1120351191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1090004d90 0x7f1090005210 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f109c00e350 tx=0x7f1090005880 comp rx=0 tx=0).stop 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10a8d98700 1 -- 172.21.15.5:0/1120351191 reap_dead start 2024-05-08T23:28:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10aa7fb700 1 -- 172.21.15.5:0/1120351191 shutdown_connections 2024-05-08T23:28:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10aa7fb700 1 -- 172.21.15.5:0/1120351191 >> 172.21.15.5:0/1120351191 conn(0x7f10a40b5a30 msgr2=0x7f10a40b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10aa7fb700 1 -- 172.21.15.5:0/1120351191 shutdown_connections 2024-05-08T23:28:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.106+0000 7f10aa7fb700 1 -- 172.21.15.5:0/1120351191 wait complete. 2024-05-08T23:28:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.108+0000 7f10aa7fb700 1 Processor -- start 2024-05-08T23:28:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.108+0000 7f10aa7fb700 1 -- start start 2024-05-08T23:28:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10aa7fb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a414c1c0 0x7f10a40cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10aa7fb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a414c5a0 0x7f10a40cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10aa7fb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10a4159c80 0x7f10a415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10aa7fb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b4050 con 0x7f10a4159c80 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10aa7fb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a415c4d0 con 0x7f10a414c5a0 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10aa7fb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a415c7b0 con 0x7f10a414c1c0 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a8d98700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10a4159c80 0x7f10a415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a414c5a0 0x7f10a40cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a8d98700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10a4159c80 0x7f10a415aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47980/0 (socket says 172.21.15.5:47980) 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a8d98700 1 -- 172.21.15.5:0/147467072 learned_addr learned my addr 172.21.15.5:0/147467072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a3fff700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a414c1c0 0x7f10a40cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a8d98700 1 -- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a414c1c0 msgr2=0x7f10a40cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a8d98700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a414c1c0 0x7f10a40cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a8d98700 1 -- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a414c5a0 msgr2=0x7f10a40cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a8d98700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a414c5a0 0x7f10a40cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.109+0000 7f10a8d98700 1 -- 172.21.15.5:0/147467072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a415ca90 con 0x7f10a4159c80 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.110+0000 7f10a8d98700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10a4159c80 0x7f10a415aab0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f109c002230 tx=0x7f109c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.110+0000 7f10a0ff9700 1 -- 172.21.15.5:0/147467072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c009410 con 0x7f10a4159c80 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.110+0000 7f10a0ff9700 1 -- 172.21.15.5:0/147467072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c015cd0 con 0x7f10a4159c80 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.110+0000 7f10a0ff9700 1 -- 172.21.15.5:0/147467072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c01d660 con 0x7f10a4159c80 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.110+0000 7f10aa7fb700 1 -- 172.21.15.5:0/147467072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a415cd70 con 0x7f10a4159c80 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.110+0000 7f10aa7fb700 1 -- 172.21.15.5:0/147467072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a415d330 con 0x7f10a4159c80 2024-05-08T23:28:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.112+0000 7f10a0ff9700 1 -- 172.21.15.5:0/147467072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f109c01d7c0 con 0x7f10a4159c80 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.112+0000 7f10a0ff9700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f108406ac90 0x7f108406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.112+0000 7f10a0ff9700 1 -- 172.21.15.5:0/147467072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f109c08a9a0 con 0x7f10a4159c80 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.112+0000 7f10a3fff700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f108406ac90 0x7f108406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.113+0000 7f10a3fff700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f108406ac90 0x7f108406d150 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f108c007fd0 tx=0x7f108c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.114+0000 7f10b69b5b80 1 Processor -- start 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.114+0000 7f10b69b5b80 1 -- start start 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bda7a0 0x561663bb3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 0x561663bb80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561663b0e150 con 0x561663bda7a0 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561663b0bb10 con 0x561663be1dd0 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561663b07400 con 0x561663bb5370 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bda7a0 0x561663bb3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bda7a0 0x561663bb3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47990/0 (socket says 172.21.15.5:47990) 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10a8d98700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10a3fff700 1 -- 172.21.15.5:0/3440562809 learned_addr learned my addr 172.21.15.5:0/3440562809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.115+0000 7f10a37fe700 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 0x561663bb80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f10a3fff700 1 -- 172.21.15.5:0/3440562809 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 msgr2=0x561663bb80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f10a3fff700 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 0x561663bb80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f10a3fff700 1 -- 172.21.15.5:0/3440562809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 msgr2=0x561663be2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f10a3fff700 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f10a3fff700 1 -- 172.21.15.5:0/3440562809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616639883b0 con 0x561663bda7a0 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f10a8d98700 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f10a3fff700 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bda7a0 0x561663bb3110 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f108c0129b0 tx=0x7f108c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e843726ca710593e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f1099ffb700 1 -- 172.21.15.5:0/3440562809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0157d0 con 0x561663bda7a0 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f1099ffb700 1 -- 172.21.15.5:0/3440562809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c015930 con 0x561663bda7a0 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.116+0000 7f1099ffb700 1 -- 172.21.15.5:0/3440562809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c015ca0 con 0x561663bda7a0 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3440562809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bda7a0 msgr2=0x561663bb3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bda7a0 0x561663bb3110 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f108c0129b0 tx=0x7f108c015000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3440562809 shutdown_connections 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bda7a0 0x561663bb3110 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3440562809 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 0x561663bb80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3440562809 >> 172.21.15.5:0/3440562809 conn(0x561663b15210 msgr2=0x561663bfcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3440562809 shutdown_connections 2024-05-08T23:28:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3440562809 wait complete. 2024-05-08T23:28:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.117+0000 7f10b69b5b80 1 -- start start 2024-05-08T23:28:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.118+0000 7f10b69b5b80 1 Processor -- start 2024-05-08T23:28:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663c0bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bda7a0 0x561663bd9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561663aa3560 con 0x561663bb5370 2024-05-08T23:28:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561663aa3200 con 0x561663be1dd0 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56166398a6a0 con 0x561663bda7a0 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10b69b5b80 10 client.0 _ll_register_callbacks cb 0x561663aacb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663c0bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663c0bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48004/0 (socket says 172.21.15.5:48004) 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 -- 172.21.15.5:0/3778346514 learned_addr learned my addr 172.21.15.5:0/3778346514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a8d98700 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a37fe700 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bda7a0 0x561663bd9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 -- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bda7a0 msgr2=0x561663bd9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bda7a0 0x561663bd9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 -- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 msgr2=0x561663be53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 -- 172.21.15.5:0/3778346514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a415d330 con 0x561663bb5370 2024-05-08T23:28:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10a3fff700 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663c0bfb0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f108c009010 tx=0x7f108c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dadae3cb45f5dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f1082ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f1082ffd700 1 -- 172.21.15.5:0/3778346514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0157d0 con 0x561663bb5370 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f1082ffd700 1 -- 172.21.15.5:0/3778346514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f108c015e10 con 0x561663bb5370 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.119+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3778346514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561663a60270 con 0x561663bb5370 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f1082ffd700 1 -- 172.21.15.5:0/3778346514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f108c021ba0 con 0x561663bb5370 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10b69b5b80 10 client.9093 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3778346514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x7f10a40b4c70 con 0x561663bb5370 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f1082ffd700 1 -- 172.21.15.5:0/3778346514 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561663a60270 con 0x561663bb5370 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f1082ffd700 1 -- 172.21.15.5:0/3778346514 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f108c021d00 con 0x561663bb5370 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10b69b5b80 10 client.9093 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10b69b5b80 10 client.9093 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10b69b5b80 20 client.9093 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x561663be2f70 0x561663bfc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10b69b5b80 4 client.9093 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3778346514 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561663b867e0 con 0x561663be2f70 2024-05-08T23:28:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.120+0000 7f10a37fe700 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x561663be2f70 0x561663bfc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.121+0000 7f1082ffd700 10 client.9093 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.121+0000 7f10a37fe700 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x561663be2f70 0x561663bfc180 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2212f2078b5472eb server_cookie=4169792ae17f0f7a in_seq=0 out_seq=0 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.121+0000 7f1082ffd700 1 -- 172.21.15.5:0/3778346514 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561663b867e0 con 0x561663be2f70 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.121+0000 7f1082ffd700 10 client.9093 handle_command_reply: tid=0 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.121+0000 7f10b69b5b80 1 client.9093 shutdown 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3778346514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 msgr2=0x561663c0bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663c0bfb0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f108c009010 tx=0x7f108c0012d0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3778346514 shutdown_connections 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x561663be2f70 0x561663bfc180 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663be1dd0 0x561663be53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663c0bfb0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f1082ffd700 0 client.9093 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/3778346514 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bda7a0 0x561663bd9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3778346514 >> 172.21.15.5:0/3778346514 conn(0x561663b15210 msgr2=0x561663bdefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3778346514 shutdown_connections 2024-05-08T23:28:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 1 -- 172.21.15.5:0/3778346514 wait complete. 2024-05-08T23:28:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.122+0000 7f10b69b5b80 20 client.9093 trim_cache size 0 max 16384 2024-05-08T23:28:33.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.140+0000 7f10b69b5b80 1 Processor -- start 2024-05-08T23:28:33.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.140+0000 7f10b69b5b80 1 -- start start 2024-05-08T23:28:33.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 0x561663bb9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663bd9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663be1dd0 0x561663bd9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561663be4f50 con 0x561663be1dd0 2024-05-08T23:28:33.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561663bfc350 con 0x561663bda7a0 2024-05-08T23:28:33.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561663bb2170 con 0x561663bb5370 2024-05-08T23:28:33.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663be1dd0 0x561663bd9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663be1dd0 0x561663bd9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48016/0 (socket says 172.21.15.5:48016) 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 -- 172.21.15.5:0/1163546779 learned_addr learned my addr 172.21.15.5:0/1163546779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a37fe700 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663bd9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a3fff700 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 0x561663bb9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 -- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 msgr2=0x561663bb9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 0x561663bb9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 -- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 msgr2=0x561663bd9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663bd9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 -- 172.21.15.5:0/1163546779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561663934b60 con 0x561663be1dd0 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a37fe700 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663bd9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f10a8d98700 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663be1dd0 0x561663bd9680 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f109c088f00 tx=0x7f109c059460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e63ce953bc8f2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.141+0000 7f109a7fc700 1 -- 172.21.15.5:0/1163546779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c059ab0 con 0x561663be1dd0 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f109a7fc700 1 -- 172.21.15.5:0/1163546779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c05d8e0 con 0x561663be1dd0 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f109a7fc700 1 -- 172.21.15.5:0/1163546779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c061ea0 con 0x561663be1dd0 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 -- 172.21.15.5:0/1163546779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663be1dd0 msgr2=0x561663bd9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663be1dd0 0x561663bd9680 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f109c088f00 tx=0x7f109c059460 comp rx=0 tx=0).stop 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 -- 172.21.15.5:0/1163546779 shutdown_connections 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663be1dd0 0x561663bd9680 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663bb5370 0x561663bb9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/1163546779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663bd9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 -- 172.21.15.5:0/1163546779 >> 172.21.15.5:0/1163546779 conn(0x561663b15210 msgr2=0x561663bb2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 -- 172.21.15.5:0/1163546779 shutdown_connections 2024-05-08T23:28:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.142+0000 7f10b69b5b80 1 -- 172.21.15.5:0/1163546779 wait complete. 2024-05-08T23:28:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.143+0000 7f10b69b5b80 1 -- start start 2024-05-08T23:28:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.143+0000 7f10b69b5b80 1 Processor -- start 2024-05-08T23:28:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663bf31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663c0bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10b69b5b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663be1dd0 0x561663be0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561663be1420 con 0x561663bb5370 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561663c0c250 con 0x561663bda7a0 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10b69b5b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561663bf2760 con 0x561663be1dd0 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10b69b5b80 10 client.0 _ll_register_callbacks cb 0x7f109c055df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663bf31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663bf31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48020/0 (socket says 172.21.15.5:48020) 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a3fff700 1 -- 172.21.15.5:0/122968158 learned_addr learned my addr 172.21.15.5:0/122968158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a37fe700 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663c0bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a8d98700 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663be1dd0 0x561663be0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a3fff700 1 -- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663be1dd0 msgr2=0x561663be0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a3fff700 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663be1dd0 0x561663be0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a3fff700 1 -- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 msgr2=0x561663c0bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a3fff700 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663c0bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.144+0000 7f10a3fff700 1 -- 172.21.15.5:0/122968158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a415cd70 con 0x561663bb5370 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10a3fff700 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663bf31b0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f108c02c950 tx=0x7f108c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1692321cd5e7ac30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f1082ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f1082ffd700 1 -- 172.21.15.5:0/122968158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0157d0 con 0x561663bb5370 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f1082ffd700 1 -- 172.21.15.5:0/122968158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f108c02cbd0 con 0x561663bb5370 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 1 -- 172.21.15.5:0/122968158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561663bda310 con 0x561663bb5370 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f1082ffd700 1 -- 172.21.15.5:0/122968158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f108c02cd90 con 0x561663bb5370 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 10 client.9099 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 1 -- 172.21.15.5:0/122968158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x561663acd560 con 0x561663bb5370 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f1082ffd700 1 -- 172.21.15.5:0/122968158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561663bda310 con 0x561663bb5370 2024-05-08T23:28:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f1082ffd700 1 -- 172.21.15.5:0/122968158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f108c021820 con 0x561663bb5370 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 10 client.9099 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 10 client.9099 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 20 client.9099 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x561663bde960 0x561663bb2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 4 client.9099 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10b69b5b80 1 -- 172.21.15.5:0/122968158 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f108c015930 con 0x561663bde960 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.145+0000 7f10a37fe700 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x561663bde960 0x561663bb2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10a37fe700 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x561663bde960 0x561663bb2d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=406ac87dc3777573 server_cookie=500e14b758ad3c42 in_seq=0 out_seq=0 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f1082ffd700 10 client.9099 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f1082ffd700 1 -- 172.21.15.5:0/122968158 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x7f108c015930 con 0x561663bde960 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f1082ffd700 10 client.9099 handle_command_reply: tid=0 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 client.9099 shutdown 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 -- 172.21.15.5:0/122968158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 msgr2=0x561663bf31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663bf31b0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f108c02c950 tx=0x7f108c00c000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 -- 172.21.15.5:0/122968158 shutdown_connections 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x561663bde960 0x561663bb2d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561663be1dd0 0x561663be0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561663bb5370 0x561663bf31b0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 --2- 172.21.15.5:0/122968158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561663bda7a0 0x561663c0bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 -- 172.21.15.5:0/122968158 >> 172.21.15.5:0/122968158 conn(0x561663b15210 msgr2=0x561663bb0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f1082ffd700 0 client.9099 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 -- 172.21.15.5:0/122968158 shutdown_connections 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.146+0000 7f10b69b5b80 1 -- 172.21.15.5:0/122968158 wait complete. 2024-05-08T23:28:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10b69b5b80 20 client.9099 trim_cache size 0 max 16384 2024-05-08T23:28:33.155 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.156 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:33.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 -- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f108406ac90 msgr2=0x7f108406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f108406ac90 0x7f108406d150 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f108c007fd0 tx=0x7f108c009040 comp rx=0 tx=0).stop 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 -- 172.21.15.5:0/147467072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10a4159c80 msgr2=0x7f10a415aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10a4159c80 0x7f10a415aab0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f109c002230 tx=0x7f109c009750 comp rx=0 tx=0).stop 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 -- 172.21.15.5:0/147467072 shutdown_connections 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f108406ac90 0x7f108406d150 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10a4159c80 0x7f10a415aab0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10a414c1c0 0x7f10a40cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 --2- 172.21.15.5:0/147467072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10a414c5a0 0x7f10a40cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 -- 172.21.15.5:0/147467072 >> 172.21.15.5:0/147467072 conn(0x7f10a40b5a30 msgr2=0x7f10a41566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 -- 172.21.15.5:0/147467072 shutdown_connections 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.147+0000 7f10aa7fb700 1 -- 172.21.15.5:0/147467072 wait complete. 2024-05-08T23:28:33.161 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:33.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T23:28:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.298+0000 7f925f487700 1 Processor -- start 2024-05-08T23:28:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.298+0000 7f925f487700 1 -- start start 2024-05-08T23:28:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.299+0000 7f925f487700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f925806b070 0x7f9258072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.299+0000 7f925f487700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f925806dcd0 0x7f925806e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.299+0000 7f925f487700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9258099df0 con 0x7f9258161d70 2024-05-08T23:28:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.299+0000 7f925f487700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580b2700 con 0x7f925806dcd0 2024-05-08T23:28:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.299+0000 7f925f487700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92580b29e0 con 0x7f925806b070 2024-05-08T23:28:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.299+0000 7f925da24700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f925806dcd0 0x7f925806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.299+0000 7f925d223700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9258161d70 0x7f925806a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47034/0 (socket says 172.21.15.5:47034) 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.299+0000 7f925d223700 1 -- 172.21.15.5:0/2964358215 learned_addr learned my addr 172.21.15.5:0/2964358215 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f925ca22700 1 --2- 172.21.15.5:0/2964358215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f925806b070 0x7f9258072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f924bfff700 1 -- 172.21.15.5:0/2964358215 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744593769 0 0) 0x7f9258099df0 con 0x7f9258161d70 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f924bfff700 1 -- 172.21.15.5:0/2964358215 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92440035b0 con 0x7f9258161d70 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f925da24700 1 -- 172.21.15.5:0/2964358215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f925806b070 msgr2=0x7f9258072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f925da24700 1 --2- 172.21.15.5:0/2964358215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f925806b070 0x7f9258072170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f925da24700 1 -- 172.21.15.5:0/2964358215 >> v1:172.21.15.5:6789/0 conn(0x7f9258161d70 legacy=0x7f925806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f925da24700 1 -- 172.21.15.5:0/2964358215 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580b53b0 con 0x7f925806dcd0 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f925da24700 1 --2- 172.21.15.5:0/2964358215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f925806e160 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9250009d90 tx=0x7f925000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3052838c34f747 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f924bfff700 1 -- 172.21.15.5:0/2964358215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925000e070 con 0x7f925806dcd0 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f924bfff700 1 -- 172.21.15.5:0/2964358215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925000bb20 con 0x7f925806dcd0 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.300+0000 7f924bfff700 1 -- 172.21.15.5:0/2964358215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250012430 con 0x7f925806dcd0 2024-05-08T23:28:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.301+0000 7f925f487700 1 -- 172.21.15.5:0/2964358215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 msgr2=0x7f925806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.301+0000 7f925f487700 1 --2- 172.21.15.5:0/2964358215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f925806e160 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9250009d90 tx=0x7f925000b330 comp rx=0 tx=0).stop 2024-05-08T23:28:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.301+0000 7f925f487700 1 -- 172.21.15.5:0/2964358215 shutdown_connections 2024-05-08T23:28:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.301+0000 7f925f487700 1 --2- 172.21.15.5:0/2964358215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f925806e160 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.301+0000 7f925f487700 1 --2- 172.21.15.5:0/2964358215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f925806b070 0x7f9258072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.301+0000 7f925f487700 1 -- 172.21.15.5:0/2964358215 >> 172.21.15.5:0/2964358215 conn(0x7f92580b5a30 msgr2=0x7f92580b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.301+0000 7f925f487700 1 -- 172.21.15.5:0/2964358215 shutdown_connections 2024-05-08T23:28:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.301+0000 7f925f487700 1 -- 172.21.15.5:0/2964358215 wait complete. 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.302+0000 7f925f487700 1 Processor -- start 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.302+0000 7f925f487700 1 -- start start 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925f487700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f925806b070 0x7f9258163210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925f487700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f92580ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925f487700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9258161d70 0x7f92580b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925f487700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580ad6a0 con 0x7f925806dcd0 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925f487700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580ad8f0 con 0x7f925806b070 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925f487700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92580adbd0 con 0x7f9258161d70 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925ca22700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f92580ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925ca22700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f92580ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48040/0 (socket says 172.21.15.5:48040) 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925ca22700 1 -- 172.21.15.5:0/3892289947 learned_addr learned my addr 172.21.15.5:0/3892289947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925d223700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f925806b070 0x7f9258163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.303+0000 7f925da24700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9258161d70 0x7f92580b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925ca22700 1 -- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9258161d70 msgr2=0x7f92580b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925ca22700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9258161d70 0x7f92580b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925ca22700 1 -- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f925806b070 msgr2=0x7f9258163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925ca22700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f925806b070 0x7f9258163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925ca22700 1 -- 172.21.15.5:0/3892289947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9250009a70 con 0x7f925806dcd0 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925da24700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9258161d70 0x7f92580b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925ca22700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f92580ae990 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e810 tx=0x7f924c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f9249ffb700 1 -- 172.21.15.5:0/3892289947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c013610 con 0x7f925806dcd0 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f9249ffb700 1 -- 172.21.15.5:0/3892289947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c013770 con 0x7f925806dcd0 2024-05-08T23:28:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925f487700 1 -- 172.21.15.5:0/3892289947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92580adeb0 con 0x7f925806dcd0 2024-05-08T23:28:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f9249ffb700 1 -- 172.21.15.5:0/3892289947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c013b20 con 0x7f925806dcd0 2024-05-08T23:28:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.304+0000 7f925f487700 1 -- 172.21.15.5:0/3892289947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92580ae470 con 0x7f925806dcd0 2024-05-08T23:28:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.306+0000 7f9249ffb700 1 -- 172.21.15.5:0/3892289947 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f924c013c80 con 0x7f925806dcd0 2024-05-08T23:28:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.307+0000 7f9249ffb700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f923c06ac90 0x7f923c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.307+0000 7f9249ffb700 1 -- 172.21.15.5:0/3892289947 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f924c088ee0 con 0x7f925806dcd0 2024-05-08T23:28:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.307+0000 7f925d223700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f923c06ac90 0x7f923c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.307+0000 7f925d223700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f923c06ac90 0x7f923c06d150 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9254007fd0 tx=0x7f9254007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.308+0000 7f926b641b80 1 Processor -- start 2024-05-08T23:28:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.309+0000 7f926b641b80 1 -- start start 2024-05-08T23:28:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f926b641b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff117110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f926b641b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff11c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f926b641b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff145dd0 0x558eff146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f926b641b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eff072150 con 0x558eff13e7a0 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f926b641b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eff06fb10 con 0x558eff119370 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f926b641b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558eff06b400 con 0x558eff145dd0 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff117110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff117110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48048/0 (socket says 172.21.15.5:48048) 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 -- 172.21.15.5:0/3073567818 learned_addr learned my addr 172.21.15.5:0/3073567818 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925da24700 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff145dd0 0x558eff146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925ca22700 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff11c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 -- 172.21.15.5:0/3073567818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff145dd0 msgr2=0x558eff146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff145dd0 0x558eff146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 -- 172.21.15.5:0/3073567818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 msgr2=0x558eff11c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff11c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 -- 172.21.15.5:0/3073567818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558efeeec3b0 con 0x558eff13e7a0 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f925d223700 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff117110 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f9254014610 tx=0x7f9254012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544d540170586a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.310+0000 7f9242ffd700 1 -- 172.21.15.5:0/3073567818 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254014ce0 con 0x558eff13e7a0 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f9242ffd700 1 -- 172.21.15.5:0/3073567818 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9254014e40 con 0x558eff13e7a0 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f9242ffd700 1 -- 172.21.15.5:0/3073567818 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254005ae0 con 0x558eff13e7a0 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 -- 172.21.15.5:0/3073567818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 msgr2=0x558eff117110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff117110 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f9254014610 tx=0x7f9254012000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 -- 172.21.15.5:0/3073567818 shutdown_connections 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff145dd0 0x558eff146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff117110 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 --2- 172.21.15.5:0/3073567818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff11c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 -- 172.21.15.5:0/3073567818 >> 172.21.15.5:0/3073567818 conn(0x558eff079210 msgr2=0x558eff160dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 -- 172.21.15.5:0/3073567818 shutdown_connections 2024-05-08T23:28:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.311+0000 7f926b641b80 1 -- 172.21.15.5:0/3073567818 wait complete. 2024-05-08T23:28:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.312+0000 7f926b641b80 1 -- start start 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.312+0000 7f926b641b80 1 Processor -- start 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f926b641b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 0x558eff13b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f926b641b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f926b641b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff145dd0 0x558eff1648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f926b641b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eff007560 con 0x558eff13e7a0 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f926b641b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eff007200 con 0x558eff145dd0 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f926b641b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558efeeee6a0 con 0x558eff119370 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f926b641b80 10 client.0 _ll_register_callbacks cb 0x558eff010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f925ca22700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f925ca22700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48050/0 (socket says 172.21.15.5:48050) 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f925ca22700 1 -- 172.21.15.5:0/207268046 learned_addr learned my addr 172.21.15.5:0/207268046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f925d223700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 0x558eff13b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.313+0000 7f925da24700 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff145dd0 0x558eff1648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f925ca22700 1 -- 172.21.15.5:0/207268046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 msgr2=0x558eff13b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f925ca22700 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 0x558eff13b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f925ca22700 1 -- 172.21.15.5:0/207268046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff145dd0 msgr2=0x558eff1648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f925ca22700 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff145dd0 0x558eff1648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f925ca22700 1 -- 172.21.15.5:0/207268046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92440035b0 con 0x558eff13e7a0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f925ca22700 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff149280 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f924c00ef20 tx=0x7f924c0582d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e96745675207f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f9237fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f9237fff700 1 -- 172.21.15.5:0/207268046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c06bdd0 con 0x558eff13e7a0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f926b641b80 1 -- 172.21.15.5:0/207268046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558efefc4270 con 0x558eff13e7a0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f9237fff700 1 -- 172.21.15.5:0/207268046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f924c060180 con 0x558eff13e7a0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f9237fff700 1 -- 172.21.15.5:0/207268046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f924c05fcf0 con 0x558eff13e7a0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f926b641b80 10 client.9114 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f926b641b80 1 -- 172.21.15.5:0/207268046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x558efee98b60 con 0x558eff13e7a0 2024-05-08T23:28:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f9237fff700 1 -- 172.21.15.5:0/207268046 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558efefc4270 con 0x558eff13e7a0 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f9237fff700 1 -- 172.21.15.5:0/207268046 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f924c05cae0 con 0x558eff13e7a0 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.314+0000 7f926b641b80 10 client.9114 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f926b641b80 10 client.9114 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f926b641b80 20 client.9114 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f926b641b80 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x558eff162250 0x558eff162660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f926b641b80 4 client.9114 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f926b641b80 1 -- 172.21.15.5:0/207268046 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558eff0ea7e0 con 0x558eff162250 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f925d223700 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x558eff162250 0x558eff162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f925d223700 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x558eff162250 0x558eff162660 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=227f0bd2fbc70aad server_cookie=d51d426193707fc6 in_seq=0 out_seq=0 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f9237fff700 10 client.9114 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.315+0000 7f9237fff700 1 -- 172.21.15.5:0/207268046 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558eff0ea7e0 con 0x558eff162250 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f9237fff700 10 client.9114 handle_command_reply: tid=0 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 client.9114 shutdown 2024-05-08T23:28:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 -- 172.21.15.5:0/207268046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 msgr2=0x558eff149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff149280 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f924c00ef20 tx=0x7f924c0582d0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 -- 172.21.15.5:0/207268046 shutdown_connections 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x558eff162250 0x558eff162660 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff145dd0 0x558eff1648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f9237fff700 0 client.9114 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 0x558eff13b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 --2- 172.21.15.5:0/207268046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff13e7a0 0x558eff149280 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 -- 172.21.15.5:0/207268046 >> 172.21.15.5:0/207268046 conn(0x558eff079210 msgr2=0x558eff143060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 -- 172.21.15.5:0/207268046 shutdown_connections 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 1 -- 172.21.15.5:0/207268046 wait complete. 2024-05-08T23:28:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.316+0000 7f926b641b80 20 client.9114 trim_cache size 0 max 16384 2024-05-08T23:28:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.335+0000 7f926b641b80 1 Processor -- start 2024-05-08T23:28:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.335+0000 7f926b641b80 1 -- start start 2024-05-08T23:28:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f926b641b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff1462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f926b641b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff13e7a0 0x558eff164810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f926b641b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff11be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f926b641b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eff13e130 con 0x558eff145dd0 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f926b641b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eff13ba90 con 0x558eff119370 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f926b641b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558eff1628f0 con 0x558eff13e7a0 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff11be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff11be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48052/0 (socket says 172.21.15.5:48052) 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 -- 172.21.15.5:0/311241332 learned_addr learned my addr 172.21.15.5:0/311241332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925d223700 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff1462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925ca22700 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff13e7a0 0x558eff164810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 -- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff13e7a0 msgr2=0x558eff164810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff13e7a0 0x558eff164810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 -- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 msgr2=0x558eff1462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff1462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 -- 172.21.15.5:0/311241332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eff031560 con 0x558eff145dd0 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925d223700 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff1462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f925da24700 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff11be60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f9250013000 tx=0x7f9250011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f462e52c3fdfd12e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.336+0000 7f92437fe700 1 -- 172.21.15.5:0/311241332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925002b070 con 0x558eff145dd0 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f92437fe700 1 -- 172.21.15.5:0/311241332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9250003d30 con 0x558eff145dd0 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f92437fe700 1 -- 172.21.15.5:0/311241332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250020040 con 0x558eff145dd0 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 -- 172.21.15.5:0/311241332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 msgr2=0x558eff11be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff11be60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f9250013000 tx=0x7f9250011000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 -- 172.21.15.5:0/311241332 shutdown_connections 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff11be60 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff119370 0x558eff1462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 --2- 172.21.15.5:0/311241332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff13e7a0 0x558eff164810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 -- 172.21.15.5:0/311241332 >> 172.21.15.5:0/311241332 conn(0x558eff079210 msgr2=0x558eff116410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 -- 172.21.15.5:0/311241332 shutdown_connections 2024-05-08T23:28:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.337+0000 7f926b641b80 1 -- 172.21.15.5:0/311241332 wait complete. 2024-05-08T23:28:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.338+0000 7f926b641b80 1 -- start start 2024-05-08T23:28:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.338+0000 7f926b641b80 1 Processor -- start 2024-05-08T23:28:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f926b641b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 0x558eff1b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f926b641b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff13e7a0 0x558eff1c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f926b641b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff179d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f926b641b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eff13d470 con 0x558eff145dd0 2024-05-08T23:28:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f926b641b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eff1ca670 con 0x558eff13e7a0 2024-05-08T23:28:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f926b641b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558eff1b33a0 con 0x558eff119370 2024-05-08T23:28:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f926b641b80 10 client.0 _ll_register_callbacks cb 0x558eff010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925da24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff179d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925da24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff179d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48058/0 (socket says 172.21.15.5:48058) 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925da24700 1 -- 172.21.15.5:0/2985470610 learned_addr learned my addr 172.21.15.5:0/2985470610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925ca22700 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff13e7a0 0x558eff1c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925d223700 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 0x558eff1b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925da24700 1 -- 172.21.15.5:0/2985470610 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 msgr2=0x558eff1b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925da24700 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 0x558eff1b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925da24700 1 -- 172.21.15.5:0/2985470610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff13e7a0 msgr2=0x558eff1c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925da24700 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff13e7a0 0x558eff1c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.339+0000 7f925da24700 1 -- 172.21.15.5:0/2985470610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92580b53b0 con 0x558eff145dd0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f925da24700 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff179d70 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9250029040 tx=0x7f9250004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcfe79b8d9e35b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f9237fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f9237fff700 1 -- 172.21.15.5:0/2985470610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250020040 con 0x558eff145dd0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f9237fff700 1 -- 172.21.15.5:0/2985470610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9250027890 con 0x558eff145dd0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 1 -- 172.21.15.5:0/2985470610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9254014e40 con 0x558eff145dd0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f9237fff700 1 -- 172.21.15.5:0/2985470610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925002b040 con 0x558eff145dd0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 10 client.9120 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 1 -- 172.21.15.5:0/2985470610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x558efeef0cd0 con 0x558eff145dd0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f9237fff700 1 -- 172.21.15.5:0/2985470610 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9254014e40 con 0x558eff145dd0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f9237fff700 1 -- 172.21.15.5:0/2985470610 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f9250023030 con 0x558eff145dd0 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 10 client.9120 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 10 client.9120 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 20 client.9120 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x558eff17a2b0 0x558eff148f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 4 client.9120 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f926b641b80 1 -- 172.21.15.5:0/2985470610 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558eff063ec0 con 0x558eff17a2b0 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.340+0000 7f925d223700 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x558eff17a2b0 0x558eff148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f9237fff700 10 client.9120 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f925d223700 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x558eff17a2b0 0x558eff148f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8403fc403377b06 server_cookie=16b7684d6b6427db in_seq=0 out_seq=0 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f9237fff700 1 -- 172.21.15.5:0/2985470610 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x558eff063ec0 con 0x558eff17a2b0 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f9237fff700 10 client.9120 handle_command_reply: tid=0 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 client.9120 shutdown 2024-05-08T23:28:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 -- 172.21.15.5:0/2985470610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 msgr2=0x558eff179d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff179d70 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9250029040 tx=0x7f9250004b40 comp rx=0 tx=0).stop 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 -- 172.21.15.5:0/2985470610 shutdown_connections 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x558eff17a2b0 0x558eff148f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eff145dd0 0x558eff179d70 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558eff119370 0x558eff1b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f9237fff700 0 client.9120 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 --2- 172.21.15.5:0/2985470610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558eff13e7a0 0x558eff1c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 -- 172.21.15.5:0/2985470610 >> 172.21.15.5:0/2985470610 conn(0x558eff079210 msgr2=0x558eff1461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 -- 172.21.15.5:0/2985470610 shutdown_connections 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 1 -- 172.21.15.5:0/2985470610 wait complete. 2024-05-08T23:28:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.341+0000 7f926b641b80 20 client.9120 trim_cache size 0 max 16384 2024-05-08T23:28:33.350 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:33.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.351 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 -- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f923c06ac90 msgr2=0x7f923c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f923c06ac90 0x7f923c06d150 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9254007fd0 tx=0x7f9254007dc0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 -- 172.21.15.5:0/3892289947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 msgr2=0x7f92580ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f92580ae990 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e810 tx=0x7f924c00eb20 comp rx=0 tx=0).stop 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 -- 172.21.15.5:0/3892289947 shutdown_connections 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f923c06ac90 0x7f923c06d150 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9258161d70 0x7f92580b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f925806b070 0x7f9258163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 --2- 172.21.15.5:0/3892289947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f925806dcd0 0x7f92580ae990 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 -- 172.21.15.5:0/3892289947 >> 172.21.15.5:0/3892289947 conn(0x7f92580b5a30 msgr2=0x7f925815fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 -- 172.21.15.5:0/3892289947 shutdown_connections 2024-05-08T23:28:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.342+0000 7f925f487700 1 -- 172.21.15.5:0/3892289947 wait complete. 2024-05-08T23:28:33.355 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:33.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 scrub status 2024-05-08T23:28:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.494+0000 7f4f353ef700 1 Processor -- start 2024-05-08T23:28:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.494+0000 7f4f353ef700 1 -- start start 2024-05-08T23:28:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.495+0000 7f4f353ef700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f300d1910 0x7f4f300cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.495+0000 7f4f353ef700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f300cf560 con 0x7f4f3015b8c0 2024-05-08T23:28:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.495+0000 7f4f353ef700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f300ccff0 con 0x7f4f3015b4e0 2024-05-08T23:28:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.495+0000 7f4f353ef700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f300cefe0 con 0x7f4f300d1910 2024-05-08T23:28:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.495+0000 7f4f2f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f300d1910 0x7f4f300cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.495+0000 7f4f2e7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4f3015b8c0 0x7f4f300d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47038/0 (socket says 172.21.15.5:47038) 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.495+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/3585325150 learned_addr learned my addr 172.21.15.5:0/3585325150 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.496+0000 7f4f2f7fe700 1 -- 172.21.15.5:0/3585325150 >> v1:172.21.15.49:6789/0 conn(0x7f4f3015b4e0 legacy=0x7f4f300b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.496+0000 7f4f2f7fe700 1 -- 172.21.15.5:0/3585325150 >> v1:172.21.15.5:6789/0 conn(0x7f4f3015b8c0 legacy=0x7f4f300d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.496+0000 7f4f2f7fe700 1 -- 172.21.15.5:0/3585325150 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300cd2d0 con 0x7f4f300d1910 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.496+0000 7f4f2dffb700 1 -- 172.21.15.5:0/3585325150 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240141867 0 0) 0x7f4f300cf560 con 0x7f4f3015b8c0 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.496+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/3585325150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300d1910 0x7f4f300cc2c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009d90 tx=0x7f4f2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad1ccff5c911c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.496+0000 7f4f2dffb700 1 -- 172.21.15.5:0/3585325150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800e070 con 0x7f4f300d1910 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.496+0000 7f4f2dffb700 1 -- 172.21.15.5:0/3585325150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2800bbd0 con 0x7f4f300d1910 2024-05-08T23:28:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.496+0000 7f4f2dffb700 1 -- 172.21.15.5:0/3585325150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28012430 con 0x7f4f300d1910 2024-05-08T23:28:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.497+0000 7f4f353ef700 1 -- 172.21.15.5:0/3585325150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300d1910 msgr2=0x7f4f300cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.497+0000 7f4f353ef700 1 --2- 172.21.15.5:0/3585325150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300d1910 0x7f4f300cc2c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009d90 tx=0x7f4f2800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.497+0000 7f4f353ef700 1 -- 172.21.15.5:0/3585325150 shutdown_connections 2024-05-08T23:28:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.497+0000 7f4f353ef700 1 --2- 172.21.15.5:0/3585325150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f300d1910 0x7f4f300cc2c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.497+0000 7f4f353ef700 1 -- 172.21.15.5:0/3585325150 >> 172.21.15.5:0/3585325150 conn(0x7f4f3008afe0 msgr2=0x7f4f3008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.497+0000 7f4f353ef700 1 -- 172.21.15.5:0/3585325150 shutdown_connections 2024-05-08T23:28:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.497+0000 7f4f353ef700 1 -- 172.21.15.5:0/3585325150 wait complete. 2024-05-08T23:28:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.498+0000 7f4f353ef700 1 Processor -- start 2024-05-08T23:28:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.498+0000 7f4f353ef700 1 -- start start 2024-05-08T23:28:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f353ef700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f300d1910 0x7f4f301f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f353ef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f3015b4e0 0x7f4f301f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f353ef700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f3015b8c0 0x7f4f30144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f353ef700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3015ea60 con 0x7f4f3015b4e0 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f353ef700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3015de70 con 0x7f4f3015b8c0 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f353ef700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f301481a0 con 0x7f4f300d1910 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f2e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f3015b4e0 0x7f4f301f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f2e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f3015b4e0 0x7f4f301f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48074/0 (socket says 172.21.15.5:48074) 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/3973851025 learned_addr learned my addr 172.21.15.5:0/3973851025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f3015b8c0 0x7f4f30144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.499+0000 7f4f2effd700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f300d1910 0x7f4f301f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f300d1910 msgr2=0x7f4f301f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f300d1910 0x7f4f301f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f3015b8c0 msgr2=0x7f4f30144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f3015b8c0 0x7f4f30144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/3973851025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f28009a70 con 0x7f4f3015b4e0 2024-05-08T23:28:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f3015b4e0 0x7f4f301f41e0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c550 tx=0x7f4f2400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f177fe700 1 -- 172.21.15.5:0/3973851025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24004c80 con 0x7f4f3015b4e0 2024-05-08T23:28:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f177fe700 1 -- 172.21.15.5:0/3973851025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f24004de0 con 0x7f4f3015b4e0 2024-05-08T23:28:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f353ef700 1 -- 172.21.15.5:0/3973851025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3015e0c0 con 0x7f4f3015b4e0 2024-05-08T23:28:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f177fe700 1 -- 172.21.15.5:0/3973851025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24007310 con 0x7f4f3015b4e0 2024-05-08T23:28:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.500+0000 7f4f353ef700 1 -- 172.21.15.5:0/3973851025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3015e680 con 0x7f4f3015b4e0 2024-05-08T23:28:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.502+0000 7f4f177fe700 1 -- 172.21.15.5:0/3973851025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4f24007470 con 0x7f4f3015b4e0 2024-05-08T23:28:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.502+0000 7f4f177fe700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.502+0000 7f4f177fe700 1 -- 172.21.15.5:0/3973851025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f24088e30 con 0x7f4f3015b4e0 2024-05-08T23:28:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.502+0000 7f4f2effd700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.503+0000 7f4f2effd700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4f20007fd0 tx=0x7f4f20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.504+0000 7f4f415a9b80 1 Processor -- start 2024-05-08T23:28:33.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f415a9b80 1 -- start start 2024-05-08T23:28:33.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 0x561cf737e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 0x561cf739a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf73531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf72aa150 con 0x561cf737b3d0 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf72a7b10 con 0x561cf737dfc0 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf72a3400 con 0x561cf7351370 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f2f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf73531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f2f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf73531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48078/0 (socket says 172.21.15.5:48078) 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f2f7fe700 1 -- 172.21.15.5:0/1376696775 learned_addr learned my addr 172.21.15.5:0/1376696775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f2effd700 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 0x561cf737e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.505+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 0x561cf739a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f2f7fe700 1 -- 172.21.15.5:0/1376696775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 msgr2=0x561cf739a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 0x561cf739a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f2f7fe700 1 -- 172.21.15.5:0/1376696775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 msgr2=0x561cf737e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 0x561cf737e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f2f7fe700 1 -- 172.21.15.5:0/1376696775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cf71243b0 con 0x561cf737b3d0 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf73531a0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f4f280124e0 tx=0x7f4f28010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370d061b93e65dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f14ff9700 1 -- 172.21.15.5:0/1376696775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28007000 con 0x561cf737b3d0 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f14ff9700 1 -- 172.21.15.5:0/1376696775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f28003c60 con 0x561cf737b3d0 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.506+0000 7f4f14ff9700 1 -- 172.21.15.5:0/1376696775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f280125e0 con 0x561cf737b3d0 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1376696775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 msgr2=0x561cf73531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf73531a0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f4f280124e0 tx=0x7f4f28010000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1376696775 shutdown_connections 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf73531a0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 0x561cf737e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1376696775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 0x561cf739a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1376696775 >> 172.21.15.5:0/1376696775 conn(0x561cf72b1210 msgr2=0x561cf7398dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1376696775 shutdown_connections 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.507+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1376696775 wait complete. 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.508+0000 7f4f415a9b80 1 -- start start 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.508+0000 7f4f415a9b80 1 Processor -- start 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 0x561cf738fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7372f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 0x561cf73919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf723f560 con 0x561cf737b3d0 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf723f200 con 0x561cf737dfc0 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf71266a0 con 0x561cf7351370 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f415a9b80 10 client.0 _ll_register_callbacks cb 0x561cf7248b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7372f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7372f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48080/0 (socket says 172.21.15.5:48080) 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/1990888332 learned_addr learned my addr 172.21.15.5:0/1990888332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 0x561cf73919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2effd700 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 0x561cf738fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/1990888332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 msgr2=0x561cf738fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 0x561cf738fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/1990888332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 msgr2=0x561cf73919a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 0x561cf73919a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/1990888332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f300cd2d0 con 0x561cf737b3d0 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7372f00 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f4f24001310 tx=0x7f4f240576c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9fe248b5048f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1990888332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f240581c0 con 0x561cf737b3d0 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1990888332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f24057d60 con 0x561cf737b3d0 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.509+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1990888332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cf71fc270 con 0x561cf737b3d0 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1990888332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f2405bb90 con 0x561cf737b3d0 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f415a9b80 10 client.9135 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1990888332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x561cf70d0b60 con 0x561cf737b3d0 2024-05-08T23:28:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1990888332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f24073020 con 0x561cf737b3d0 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1990888332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f4f24057250 con 0x561cf737b3d0 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f415a9b80 10 client.9135 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f415a9b80 10 client.9135 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f415a9b80 20 client.9135 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x561cf73728e0 0x561cf737d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f415a9b80 4 client.9135 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1990888332 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cf73227e0 con 0x561cf73728e0 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f2effd700 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x561cf73728e0 0x561cf737d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f2effd700 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x561cf73728e0 0x561cf737d750 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8d0eef856b20eb1 server_cookie=33445789ca8da3c7 in_seq=0 out_seq=0 2024-05-08T23:28:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.510+0000 7f4f0dffb700 10 client.9135 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1990888332 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cf73227e0 con 0x561cf73728e0 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f0dffb700 10 client.9135 handle_command_reply: tid=0 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 client.9135 shutdown 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1990888332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 msgr2=0x561cf7372f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7372f00 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f4f24001310 tx=0x7f4f240576c0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1990888332 shutdown_connections 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x561cf73728e0 0x561cf737d750 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf737dfc0 0x561cf73919a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf7351370 0x561cf738fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1990888332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7372f00 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1990888332 >> 172.21.15.5:0/1990888332 conn(0x561cf72b1210 msgr2=0x561cf7352390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f0dffb700 0 client.9135 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1990888332 shutdown_connections 2024-05-08T23:28:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.511+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1990888332 wait complete. 2024-05-08T23:28:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.512+0000 7f4f415a9b80 20 client.9135 trim_cache size 0 max 16384 2024-05-08T23:28:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f415a9b80 1 Processor -- start 2024-05-08T23:28:33.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f415a9b80 1 -- start start 2024-05-08T23:28:33.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf734d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf739c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7372d60 con 0x561cf737b3d0 2024-05-08T23:28:33.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf73995b0 con 0x561cf7351370 2024-05-08T23:28:33.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf7399d40 con 0x561cf737dfc0 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f2e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf739c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f2e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf739c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48088/0 (socket says 172.21.15.5:48088) 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.530+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/2718617716 learned_addr learned my addr 172.21.15.5:0/2718617716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2effd700 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf734d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 msgr2=0x561cf739c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 msgr2=0x561cf734d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf734d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/2718617716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cf7269560 con 0x561cf737b3d0 2024-05-08T23:28:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf739c260 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f4f24057cf0 tx=0x7f4f2406c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c47872826f141a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f157fa700 1 -- 172.21.15.5:0/2718617716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f240576c0 con 0x561cf737b3d0 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f157fa700 1 -- 172.21.15.5:0/2718617716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f24073070 con 0x561cf737b3d0 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.531+0000 7f4f157fa700 1 -- 172.21.15.5:0/2718617716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24058220 con 0x561cf737b3d0 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 -- 172.21.15.5:0/2718617716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 msgr2=0x561cf739c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf739c260 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f4f24057cf0 tx=0x7f4f2406c000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 -- 172.21.15.5:0/2718617716 shutdown_connections 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf734d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/2718617716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf739c260 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 -- 172.21.15.5:0/2718617716 >> 172.21.15.5:0/2718617716 conn(0x561cf72b1210 msgr2=0x561cf734f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 -- 172.21.15.5:0/2718617716 shutdown_connections 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 -- 172.21.15.5:0/2718617716 wait complete. 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 -- start start 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.532+0000 7f4f415a9b80 1 Processor -- start 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf7405c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7408870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f415a9b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf737eaa0 con 0x561cf737b3d0 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7409440 con 0x561cf7351370 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f415a9b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf737e850 con 0x561cf737dfc0 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f415a9b80 10 client.0 _ll_register_callbacks cb 0x561cf7248b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f2e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7408870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f2e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7408870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48096/0 (socket says 172.21.15.5:48096) 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.533+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/1529211725 learned_addr learned my addr 172.21.15.5:0/1529211725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2effd700 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf7405c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2f7fe700 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 msgr2=0x561cf739c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 msgr2=0x561cf7405c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf7405c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2e7fc700 1 -- 172.21.15.5:0/1529211725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cf70d0b60 con 0x561cf737b3d0 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2effd700 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf7405c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f2e7fc700 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7408870 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c520 tx=0x7f4f240872c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbffe6568ada4aa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1529211725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2405fec0 con 0x561cf737b3d0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1529211725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f24074080 con 0x561cf737b3d0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1529211725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f28003c60 con 0x561cf737b3d0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1529211725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f24057d60 con 0x561cf737b3d0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f415a9b80 10 client.9141 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1529211725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x561cf7128cd0 con 0x561cf737b3d0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1529211725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f28003c60 con 0x561cf737b3d0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.534+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1529211725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f4f2406bba0 con 0x561cf737b3d0 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 10 client.9141 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 10 client.9141 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 20 client.9141 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x561cf74029f0 0x561cf739c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 4 client.9141 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1529211725 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561cf729bec0 con 0x561cf74029f0 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f2effd700 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x561cf74029f0 0x561cf739c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f0dffb700 10 client.9141 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f2effd700 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x561cf74029f0 0x561cf739c7a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d40fb3d355b8520 server_cookie=bad96b65c4d10a56 in_seq=0 out_seq=0 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f0dffb700 1 -- 172.21.15.5:0/1529211725 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x561cf729bec0 con 0x561cf74029f0 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f0dffb700 10 client.9141 handle_command_reply: tid=0 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 client.9141 shutdown 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1529211725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 msgr2=0x561cf7408870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7408870 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c520 tx=0x7f4f240872c0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1529211725 shutdown_connections 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x561cf74029f0 0x561cf739c7a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x561cf737dfc0 0x561cf739c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561cf7351370 0x561cf7405c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 --2- 172.21.15.5:0/1529211725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cf737b3d0 0x561cf7408870 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f0dffb700 0 client.9141 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.535+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1529211725 >> 172.21.15.5:0/1529211725 conn(0x561cf72b1210 msgr2=0x561cf73beff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1529211725 shutdown_connections 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f415a9b80 1 -- 172.21.15.5:0/1529211725 wait complete. 2024-05-08T23:28:33.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f415a9b80 20 client.9141 trim_cache size 0 max 16384 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.546 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:33.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 -- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 msgr2=0x7f4f1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4f20007fd0 tx=0x7f4f20009040 comp rx=0 tx=0).stop 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 -- 172.21.15.5:0/3973851025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f3015b4e0 msgr2=0x7f4f301f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f3015b4e0 0x7f4f301f41e0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c550 tx=0x7f4f2400c860 comp rx=0 tx=0).stop 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 -- 172.21.15.5:0/3973851025 shutdown_connections 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f4f1c06ac90 0x7f4f1c06d150 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f3015b8c0 0x7f4f30144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4f300d1910 0x7f4f301f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 --2- 172.21.15.5:0/3973851025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f3015b4e0 0x7f4f301f41e0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 -- 172.21.15.5:0/3973851025 >> 172.21.15.5:0/3973851025 conn(0x7f4f3008afe0 msgr2=0x7f4f300af260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 -- 172.21.15.5:0/3973851025 shutdown_connections 2024-05-08T23:28:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.536+0000 7f4f353ef700 1 -- 172.21.15.5:0/3973851025 wait complete. 2024-05-08T23:28:33.550 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:33.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 scrub status 2024-05-08T23:28:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.688+0000 7f9c20a7e700 1 Processor -- start 2024-05-08T23:28:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.688+0000 7f9c20a7e700 1 -- start start 2024-05-08T23:28:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.689+0000 7f9c20a7e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c1c15d900 0x7f9c1c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.689+0000 7f9c20a7e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c06c080 0x7f9c1c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.689+0000 7f9c20a7e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1c15dce0 con 0x7f9c1c0cda40 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.689+0000 7f9c20a7e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c0ce5b0 con 0x7f9c1c15d900 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.689+0000 7f9c20a7e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c161510 con 0x7f9c1c06c080 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1a59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c1c15d900 0x7f9c1c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1a59c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c1c15d900 0x7f9c1c161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48102/0 (socket says 172.21.15.5:48102) 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1a59c700 1 -- 172.21.15.5:0/1292577601 learned_addr learned my addr 172.21.15.5:0/1292577601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1ad9d700 1 --2- 172.21.15.5:0/1292577601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c06c080 0x7f9c1c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1a59c700 1 -- 172.21.15.5:0/1292577601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c06c080 msgr2=0x7f9c1c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/1292577601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c06c080 0x7f9c1c0cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1a59c700 1 -- 172.21.15.5:0/1292577601 >> v1:172.21.15.49:6789/0 conn(0x7f9c1c0cda40 legacy=0x7f9c1c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1a59c700 1 -- 172.21.15.5:0/1292577601 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c1617f0 con 0x7f9c1c15d900 2024-05-08T23:28:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/1292577601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c15d900 0x7f9c1c161ff0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9c100098f0 tx=0x7f9c10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2661f0c53aabd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1959a700 1 -- 172.21.15.5:0/1292577601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10011830 con 0x7f9c1c15d900 2024-05-08T23:28:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1959a700 1 -- 172.21.15.5:0/1292577601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10011990 con 0x7f9c1c15d900 2024-05-08T23:28:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.690+0000 7f9c1959a700 1 -- 172.21.15.5:0/1292577601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10011d20 con 0x7f9c1c15d900 2024-05-08T23:28:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.691+0000 7f9c20a7e700 1 -- 172.21.15.5:0/1292577601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c15d900 msgr2=0x7f9c1c161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.691+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/1292577601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c15d900 0x7f9c1c161ff0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9c100098f0 tx=0x7f9c10009c00 comp rx=0 tx=0).stop 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.691+0000 7f9c20a7e700 1 -- 172.21.15.5:0/1292577601 shutdown_connections 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.691+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/1292577601 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c06c080 0x7f9c1c0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.691+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/1292577601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c15d900 0x7f9c1c161ff0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.691+0000 7f9c20a7e700 1 -- 172.21.15.5:0/1292577601 >> 172.21.15.5:0/1292577601 conn(0x7f9c1c0713c0 msgr2=0x7f9c1c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.691+0000 7f9c20a7e700 1 -- 172.21.15.5:0/1292577601 shutdown_connections 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.691+0000 7f9c20a7e700 1 -- 172.21.15.5:0/1292577601 wait complete. 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.693+0000 7f9c20a7e700 1 Processor -- start 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.693+0000 7f9c20a7e700 1 -- start start 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.693+0000 7f9c20a7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c06c080 0x7f9c1c0b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.693+0000 7f9c20a7e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c0cda40 0x7f9c1c0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.693+0000 7f9c20a7e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c1c15d900 0x7f9c1c15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.693+0000 7f9c20a7e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c150de0 con 0x7f9c1c06c080 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.693+0000 7f9c20a7e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c0ade10 con 0x7f9c1c15d900 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.693+0000 7f9c20a7e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c1f4290 con 0x7f9c1c0cda40 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c06c080 0x7f9c1c0b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c06c080 0x7f9c1c0b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48112/0 (socket says 172.21.15.5:48112) 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 -- 172.21.15.5:0/2955905689 learned_addr learned my addr 172.21.15.5:0/2955905689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c0cda40 0x7f9c1c0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1ad9d700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c1c15d900 0x7f9c1c15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 -- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c0cda40 msgr2=0x7f9c1c0b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c0cda40 0x7f9c1c0b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 -- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c1c15d900 msgr2=0x7f9c1c15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c1c15d900 0x7f9c1c15bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 -- 172.21.15.5:0/2955905689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c10009550 con 0x7f9c1c06c080 2024-05-08T23:28:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c06c080 0x7f9c1c0b09b0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9c10014040 tx=0x7f9c10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c0affd700 1 -- 172.21.15.5:0/2955905689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c100036a0 con 0x7f9c1c06c080 2024-05-08T23:28:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c0affd700 1 -- 172.21.15.5:0/2955905689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1001e070 con 0x7f9c1c06c080 2024-05-08T23:28:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c20a7e700 1 -- 172.21.15.5:0/2955905689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1c1f4570 con 0x7f9c1c06c080 2024-05-08T23:28:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c0affd700 1 -- 172.21.15.5:0/2955905689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10003a60 con 0x7f9c1c06c080 2024-05-08T23:28:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.694+0000 7f9c20a7e700 1 -- 172.21.15.5:0/2955905689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c1f4b30 con 0x7f9c1c06c080 2024-05-08T23:28:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.696+0000 7f9c0affd700 1 -- 172.21.15.5:0/2955905689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9c10003bc0 con 0x7f9c1c06c080 2024-05-08T23:28:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.696+0000 7f9c0affd700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bfc06ac90 0x7f9bfc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.696+0000 7f9c0affd700 1 -- 172.21.15.5:0/2955905689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c1008a2a0 con 0x7f9c1c06c080 2024-05-08T23:28:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.697+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bfc06ac90 0x7f9bfc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.697+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bfc06ac90 0x7f9bfc06d150 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c000f60 tx=0x7f9c0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.698+0000 7f9c2cc38b80 1 Processor -- start 2024-05-08T23:28:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c2cc38b80 1 -- start start 2024-05-08T23:28:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c92110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81c94370 0x563c81c970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81cc1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c81bed150 con 0x563c81cb97a0 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c81beab10 con 0x563c81c94370 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c81be6400 con 0x563c81cc0dd0 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c1a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c92110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c1a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c92110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48122/0 (socket says 172.21.15.5:48122) 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c1a59c700 1 -- 172.21.15.5:0/839609317 learned_addr learned my addr 172.21.15.5:0/839609317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81c94370 0x563c81c970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.699+0000 7f9c1ad9d700 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81cc1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9c1a59c700 1 -- 172.21.15.5:0/839609317 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 msgr2=0x563c81cc1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81cc1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9c1a59c700 1 -- 172.21.15.5:0/839609317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81c94370 msgr2=0x563c81c970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81c94370 0x563c81c970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9c1a59c700 1 -- 172.21.15.5:0/839609317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c81a673b0 con 0x563c81cb97a0 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c92110 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f9c10088880 tx=0x7f9c10059070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e3117fad98f1cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9bfbfff700 1 -- 172.21.15.5:0/839609317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c100591d0 con 0x563c81cb97a0 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9bfbfff700 1 -- 172.21.15.5:0/839609317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1005d000 con 0x563c81cb97a0 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.700+0000 7f9bfbfff700 1 -- 172.21.15.5:0/839609317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c100615c0 con 0x563c81cb97a0 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/839609317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 msgr2=0x563c81c92110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c92110 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f9c10088880 tx=0x7f9c10059070 comp rx=0 tx=0).stop 2024-05-08T23:28:33.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/839609317 shutdown_connections 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81cc1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c92110 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/839609317 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81c94370 0x563c81c970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/839609317 >> 172.21.15.5:0/839609317 conn(0x563c81bf4210 msgr2=0x563c81cdbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/839609317 shutdown_connections 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.701+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/839609317 wait complete. 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.702+0000 7f9c2cc38b80 1 -- start start 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.702+0000 7f9c2cc38b80 1 Processor -- start 2024-05-08T23:28:33.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81cc1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81cc24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cde0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c81b82560 con 0x563c81cb97a0 2024-05-08T23:28:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c81b82200 con 0x563c81cc0dd0 2024-05-08T23:28:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c81a696a0 con 0x563c81c94370 2024-05-08T23:28:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c2cc38b80 10 client.0 _ll_register_callbacks cb 0x563c81b8bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81cc24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81cc24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48128/0 (socket says 172.21.15.5:48128) 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 -- 172.21.15.5:0/430569482 learned_addr learned my addr 172.21.15.5:0/430569482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c1ad9d700 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cde0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81cc1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 -- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 msgr2=0x563c81cc1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81cc1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 -- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 msgr2=0x563c81cde0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cde0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 -- 172.21.15.5:0/430569482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c1f4b30 con 0x563c81cb97a0 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81cc24b0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c003a30 tx=0x7f9c0c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a622fd126da05674 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.703+0000 7f9bf97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9bf97fa700 1 -- 172.21.15.5:0/430569482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c003ea0 con 0x563c81cb97a0 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9bf97fa700 1 -- 172.21.15.5:0/430569482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c0c014ad0 con 0x563c81cb97a0 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/430569482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c81b3f270 con 0x563c81cb97a0 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9bf97fa700 1 -- 172.21.15.5:0/430569482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c0c014c90 con 0x563c81cb97a0 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 10 client.9153 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/430569482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x563c81a13b60 con 0x563c81cb97a0 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9bf97fa700 1 -- 172.21.15.5:0/430569482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563c81b3f270 con 0x563c81cb97a0 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9bf97fa700 1 -- 172.21.15.5:0/430569482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f9c0c020070 con 0x563c81cb97a0 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 10 client.9153 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 10 client.9153 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 20 client.9153 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x563c81cee120 0x563c81cee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 4 client.9153 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/430569482 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c81c657e0 con 0x563c81cee120 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.704+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x563c81cee120 0x563c81cee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.705+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x563c81cee120 0x563c81cee620 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8916785420141c71 server_cookie=2ef42304007dde9e in_seq=0 out_seq=0 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.705+0000 7f9bf97fa700 10 client.9153 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.705+0000 7f9bf97fa700 1 -- 172.21.15.5:0/430569482 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c81c657e0 con 0x563c81cee120 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.705+0000 7f9bf97fa700 10 client.9153 handle_command_reply: tid=0 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 client.9153 shutdown 2024-05-08T23:28:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/430569482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 msgr2=0x563c81cc24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81cc24b0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c003a30 tx=0x7f9c0c001120 comp rx=0 tx=0).stop 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/430569482 shutdown_connections 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x563c81cee120 0x563c81cee620 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cde0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81cc1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/430569482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81cc24b0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9bf97fa700 0 client.9153 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/430569482 >> 172.21.15.5:0/430569482 conn(0x563c81bf4210 msgr2=0x563c81c902c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/430569482 shutdown_connections 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/430569482 wait complete. 2024-05-08T23:28:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.706+0000 7f9c2cc38b80 20 client.9153 trim_cache size 0 max 16384 2024-05-08T23:28:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c2cc38b80 1 Processor -- start 2024-05-08T23:28:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c2cc38b80 1 -- start start 2024-05-08T23:28:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81c91f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cdeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c81cbdad0 con 0x563c81cb97a0 2024-05-08T23:28:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c81cbeba0 con 0x563c81cc0dd0 2024-05-08T23:28:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c81c8d1e0 con 0x563c81c94370 2024-05-08T23:28:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c19d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c19d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c98260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48140/0 (socket says 172.21.15.5:48140) 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.725+0000 7f9c19d9b700 1 -- 172.21.15.5:0/2901442341 learned_addr learned my addr 172.21.15.5:0/2901442341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c1ad9d700 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cdeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81c91f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c19d9b700 1 -- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 msgr2=0x563c81c91f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81c91f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c19d9b700 1 -- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 msgr2=0x563c81cdeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cdeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c19d9b700 1 -- 172.21.15.5:0/2901442341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c81bac560 con 0x563c81cb97a0 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c1ad9d700 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cdeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81c91f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c98260 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c012ae0 tx=0x7f9c0c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc26f872d490316 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c08ff9700 1 -- 172.21.15.5:0/2901442341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c014710 con 0x563c81cb97a0 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c08ff9700 1 -- 172.21.15.5:0/2901442341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c014e90 con 0x563c81cb97a0 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.726+0000 7f9c08ff9700 1 -- 172.21.15.5:0/2901442341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c012b60 con 0x563c81cb97a0 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/2901442341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 msgr2=0x563c81c98260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c98260 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c012ae0 tx=0x7f9c0c01d000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/2901442341 shutdown_connections 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cc0dd0 0x563c81cdeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81c94370 0x563c81c91f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/2901442341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81cb97a0 0x563c81c98260 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/2901442341 >> 172.21.15.5:0/2901442341 conn(0x563c81bf4210 msgr2=0x563c81cdde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/2901442341 shutdown_connections 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/2901442341 wait complete. 2024-05-08T23:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.727+0000 7f9c2cc38b80 1 -- start start 2024-05-08T23:28:33.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.728+0000 7f9c2cc38b80 1 Processor -- start 2024-05-08T23:28:33.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81c94370 0x563c81cc2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cb97a0 0x563c81c95f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c2cc38b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81c96470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c81d48d70 con 0x563c81c94370 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c81cea7b0 con 0x563c81cb97a0 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c2cc38b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c81cc2bf0 con 0x563c81cc0dd0 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c2cc38b80 10 client.0 _ll_register_callbacks cb 0x563c81b8bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c1a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81c94370 0x563c81cc2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c1a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81c94370 0x563c81cc2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48144/0 (socket says 172.21.15.5:48144) 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c1a59c700 1 -- 172.21.15.5:0/3414408646 learned_addr learned my addr 172.21.15.5:0/3414408646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cb97a0 0x563c81c95f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.729+0000 7f9c1ad9d700 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81c96470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c1a59c700 1 -- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 msgr2=0x563c81c96470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81c96470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c1a59c700 1 -- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cb97a0 msgr2=0x563c81c95f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cb97a0 0x563c81c95f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c1a59c700 1 -- 172.21.15.5:0/3414408646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c81a13b60 con 0x563c81c94370 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c1ad9d700 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81c96470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c1a59c700 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81c94370 0x563c81cc2f50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009130 tx=0x7f9c10020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dc601020ddb6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9bf97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9bf97fa700 1 -- 172.21.15.5:0/3414408646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1008a0b0 con 0x563c81c94370 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9bf97fa700 1 -- 172.21.15.5:0/3414408646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c100615c0 con 0x563c81c94370 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/3414408646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c1005d000 con 0x563c81c94370 2024-05-08T23:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9bf97fa700 1 -- 172.21.15.5:0/3414408646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c100591d0 con 0x563c81c94370 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 10 client.9159 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/3414408646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x563c81a6bcd0 con 0x563c81c94370 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9bf97fa700 1 -- 172.21.15.5:0/3414408646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1005d000 con 0x563c81c94370 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9bf97fa700 1 -- 172.21.15.5:0/3414408646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f9c1005f0c0 con 0x563c81c94370 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 10 client.9159 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 10 client.9159 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 20 client.9159 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x563c81d2de50 0x563c81cf4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 4 client.9159 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.730+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/3414408646 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563c81bdeec0 con 0x563c81d2de50 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x563c81d2de50 0x563c81cf4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9c19d9b700 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x563c81d2de50 0x563c81cf4790 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e9932fefd61fe38 server_cookie=a08685edbd6df100 in_seq=0 out_seq=0 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9bf97fa700 10 client.9159 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9bf97fa700 1 -- 172.21.15.5:0/3414408646 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x563c81bdeec0 con 0x563c81d2de50 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9bf97fa700 10 client.9159 handle_command_reply: tid=0 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9c2cc38b80 1 client.9159 shutdown 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/3414408646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81c94370 msgr2=0x563c81cc2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81c94370 0x563c81cc2f50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009130 tx=0x7f9c10020000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.731+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/3414408646 shutdown_connections 2024-05-08T23:28:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x563c81d2de50 0x563c81cf4790 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x563c81cc0dd0 0x563c81c96470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c81c94370 0x563c81cc2f50 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9bf97fa700 0 client.9159 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c2cc38b80 1 --2- 172.21.15.5:0/3414408646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c81cb97a0 0x563c81c95f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/3414408646 >> 172.21.15.5:0/3414408646 conn(0x563c81bf4210 msgr2=0x563c81cdc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/3414408646 shutdown_connections 2024-05-08T23:28:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c2cc38b80 1 -- 172.21.15.5:0/3414408646 wait complete. 2024-05-08T23:28:33.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c2cc38b80 20 client.9159 trim_cache size 0 max 16384 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.740 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 -- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bfc06ac90 msgr2=0x7f9bfc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bfc06ac90 0x7f9bfc06d150 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c000f60 tx=0x7f9c0c009040 comp rx=0 tx=0).stop 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 -- 172.21.15.5:0/2955905689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c06c080 msgr2=0x7f9c1c0b09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c06c080 0x7f9c1c0b09b0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9c10014040 tx=0x7f9c10004750 comp rx=0 tx=0).stop 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 -- 172.21.15.5:0/2955905689 shutdown_connections 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f9bfc06ac90 0x7f9bfc06d150 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c1c15d900 0x7f9c1c15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c1c06c080 0x7f9c1c0b09b0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 --2- 172.21.15.5:0/2955905689 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9c1c0cda40 0x7f9c1c0b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 -- 172.21.15.5:0/2955905689 >> 172.21.15.5:0/2955905689 conn(0x7f9c1c0713c0 msgr2=0x7f9c1c06dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 -- 172.21.15.5:0/2955905689 shutdown_connections 2024-05-08T23:28:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.732+0000 7f9c20a7e700 1 -- 172.21.15.5:0/2955905689 wait complete. 2024-05-08T23:28:33.746 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:33.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T23:28:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.885+0000 7f57ee5ac700 1 Processor -- start 2024-05-08T23:28:33.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.885+0000 7f57ee5ac700 1 -- start start 2024-05-08T23:28:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57ee5ac700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57e815b710 0x7f57e8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57ee5ac700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f57e80b26e0 0x7f57e8099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57ee5ac700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e80d04b0 con 0x7f57e80cff30 2024-05-08T23:28:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57ee5ac700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e80b0910 con 0x7f57e815b710 2024-05-08T23:28:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57ee5ac700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e80b0bf0 con 0x7f57e80b26e0 2024-05-08T23:28:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57e7fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57e815b710 0x7f57e8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57e7fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57e815b710 0x7f57e8099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48150/0 (socket says 172.21.15.5:48150) 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57e7fff700 1 -- 172.21.15.5:0/176987355 learned_addr learned my addr 172.21.15.5:0/176987355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57ecb49700 1 --2- 172.21.15.5:0/176987355 >> v2:172.21.15.49:3300/0 conn(0x7f57e80b26e0 0x7f57e8099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57e7fff700 1 -- 172.21.15.5:0/176987355 >> v2:172.21.15.49:3300/0 conn(0x7f57e80b26e0 msgr2=0x7f57e8099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57e7fff700 1 --2- 172.21.15.5:0/176987355 >> v2:172.21.15.49:3300/0 conn(0x7f57e80b26e0 0x7f57e8099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57e7fff700 1 -- 172.21.15.5:0/176987355 >> v1:172.21.15.49:6789/0 conn(0x7f57e80cff30 legacy=0x7f57e80cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57e7fff700 1 -- 172.21.15.5:0/176987355 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e80b0ed0 con 0x7f57e815b710 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.886+0000 7f57e7fff700 1 --2- 172.21.15.5:0/176987355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e815b710 0x7f57e8099a50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f57d80098f0 tx=0x7f57d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee21be1e1a38651 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57e6ffd700 1 -- 172.21.15.5:0/176987355 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8011830 con 0x7f57e815b710 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57e6ffd700 1 -- 172.21.15.5:0/176987355 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d8011990 con 0x7f57e815b710 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57e6ffd700 1 -- 172.21.15.5:0/176987355 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8011d20 con 0x7f57e815b710 2024-05-08T23:28:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57ee5ac700 1 -- 172.21.15.5:0/176987355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e815b710 msgr2=0x7f57e8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/176987355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e815b710 0x7f57e8099a50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f57d80098f0 tx=0x7f57d8009c00 comp rx=0 tx=0).stop 2024-05-08T23:28:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57ee5ac700 1 -- 172.21.15.5:0/176987355 shutdown_connections 2024-05-08T23:28:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/176987355 >> v2:172.21.15.49:3300/0 conn(0x7f57e80b26e0 0x7f57e8099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/176987355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e815b710 0x7f57e8099a50 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57ee5ac700 1 -- 172.21.15.5:0/176987355 >> 172.21.15.5:0/176987355 conn(0x7f57e80b2f60 msgr2=0x7f57e80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57ee5ac700 1 -- 172.21.15.5:0/176987355 shutdown_connections 2024-05-08T23:28:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.887+0000 7f57ee5ac700 1 -- 172.21.15.5:0/176987355 wait complete. 2024-05-08T23:28:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.889+0000 7f57ee5ac700 1 Processor -- start 2024-05-08T23:28:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.889+0000 7f57ee5ac700 1 -- start start 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57ee5ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57e80b26e0 0x7f57e8159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57ee5ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e80cff30 0x7f57e809d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57ee5ac700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f57e815b710 0x7f57e809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57ee5ac700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80b4c20 con 0x7f57e80cff30 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57ee5ac700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80b6e50 con 0x7f57e80b26e0 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57ee5ac700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80b4ee0 con 0x7f57e815b710 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e80cff30 0x7f57e809d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e80cff30 0x7f57e809d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48152/0 (socket says 172.21.15.5:48152) 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 -- 172.21.15.5:0/3910024879 learned_addr learned my addr 172.21.15.5:0/3910024879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e7fff700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57e80b26e0 0x7f57e8159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57ecb49700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f57e815b710 0x7f57e809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 -- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f57e815b710 msgr2=0x7f57e809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f57e815b710 0x7f57e809d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 -- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57e80b26e0 msgr2=0x7f57e8159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57e80b26e0 0x7f57e8159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 -- 172.21.15.5:0/3910024879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d8009550 con 0x7f57e80cff30 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e77fe700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e80cff30 0x7f57e809d2a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00c4b0 tx=0x7f57dc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.890+0000 7f57e4ff9700 1 -- 172.21.15.5:0/3910024879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0052d0 con 0x7f57e80cff30 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.891+0000 7f57e4ff9700 1 -- 172.21.15.5:0/3910024879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc005430 con 0x7f57e80cff30 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.891+0000 7f57ee5ac700 1 -- 172.21.15.5:0/3910024879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e80b51c0 con 0x7f57e80cff30 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.891+0000 7f57e4ff9700 1 -- 172.21.15.5:0/3910024879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0038c0 con 0x7f57e80cff30 2024-05-08T23:28:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.891+0000 7f57ee5ac700 1 -- 172.21.15.5:0/3910024879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e80b6ae0 con 0x7f57e80cff30 2024-05-08T23:28:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.892+0000 7f57e4ff9700 1 -- 172.21.15.5:0/3910024879 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f57dc003a20 con 0x7f57e80cff30 2024-05-08T23:28:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.893+0000 7f57e4ff9700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f57d406ac90 0x7f57d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.893+0000 7f57e4ff9700 1 -- 172.21.15.5:0/3910024879 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57dc088730 con 0x7f57e80cff30 2024-05-08T23:28:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.893+0000 7f57e7fff700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f57d406ac90 0x7f57d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.894+0000 7f57e7fff700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f57d406ac90 0x7f57d406d150 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009c40 tx=0x7f57d8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.895+0000 7f57fa766b80 1 Processor -- start 2024-05-08T23:28:33.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.895+0000 7f57fa766b80 1 -- start start 2024-05-08T23:28:33.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e538254fc0 0x55e5382553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e538271250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5382523d0 0x55e53822a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57fa766b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e538181150 con 0x55e538228370 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57fa766b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53817eb10 con 0x55e5382523d0 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57fa766b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e53817a400 con 0x55e538254fc0 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e538271250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e538271250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48166/0 (socket says 172.21.15.5:48166) 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e77fe700 1 -- 172.21.15.5:0/1621620849 learned_addr learned my addr 172.21.15.5:0/1621620849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e7fff700 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e538254fc0 0x55e5382553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57ecb49700 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5382523d0 0x55e53822a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e77fe700 1 -- 172.21.15.5:0/1621620849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e538254fc0 msgr2=0x55e5382553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e538254fc0 0x55e5382553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e77fe700 1 -- 172.21.15.5:0/1621620849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5382523d0 msgr2=0x55e53822a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5382523d0 0x55e53822a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.896+0000 7f57e77fe700 1 -- 172.21.15.5:0/1621620849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e537ffb3b0 con 0x55e538228370 2024-05-08T23:28:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e538271250 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f57dc086b80 tx=0x7f57dc0573f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0be17e6fd4b18d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57cdffb700 1 -- 172.21.15.5:0/1621620849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc05b170 con 0x55e538228370 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57cdffb700 1 -- 172.21.15.5:0/1621620849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc056e80 con 0x55e538228370 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57cdffb700 1 -- 172.21.15.5:0/1621620849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc057c20 con 0x55e538228370 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57fa766b80 1 -- 172.21.15.5:0/1621620849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 msgr2=0x55e538271250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e538271250 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f57dc086b80 tx=0x7f57dc0573f0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57fa766b80 1 -- 172.21.15.5:0/1621620849 shutdown_connections 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e5382523d0 0x55e53822a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e538254fc0 0x55e5382553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1621620849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e538271250 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57fa766b80 1 -- 172.21.15.5:0/1621620849 >> 172.21.15.5:0/1621620849 conn(0x55e538188210 msgr2=0x55e53826fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.897+0000 7f57fa766b80 1 -- 172.21.15.5:0/1621620849 shutdown_connections 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.898+0000 7f57fa766b80 1 -- 172.21.15.5:0/1621620849 wait complete. 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.898+0000 7f57fa766b80 1 -- start start 2024-05-08T23:28:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.899+0000 7f57fa766b80 1 Processor -- start 2024-05-08T23:28:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e538286e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 0x55e53824a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57fa766b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e538116560 con 0x55e538228370 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57fa766b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e538116200 con 0x55e538254fc0 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57fa766b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e537ffd6a0 con 0x55e5382523d0 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57fa766b80 10 client.0 _ll_register_callbacks cb 0x55e53811fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382868f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48168/0 (socket says 172.21.15.5:48168) 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 -- 172.21.15.5:0/1091681515 learned_addr learned my addr 172.21.15.5:0/1091681515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e538286e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57ecb49700 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 0x55e53824a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 -- 172.21.15.5:0/1091681515 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 msgr2=0x55e538286e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e538286e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 -- 172.21.15.5:0/1091681515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 msgr2=0x55e53824a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 0x55e53824a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 -- 172.21.15.5:0/1091681515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e80b6ae0 con 0x55e538228370 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57e7fff700 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382868f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f57e8072950 tx=0x7f57d800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151bca99d2b5b5d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.900+0000 7f57c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1091681515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d802e070 con 0x55e538228370 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1091681515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57d8003c90 con 0x55e538228370 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 1 -- 172.21.15.5:0/1091681515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5380d3270 con 0x55e538228370 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1091681515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d8020070 con 0x55e538228370 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 10 client.9171 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 1 -- 172.21.15.5:0/1091681515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55e537fa7b60 con 0x55e538228370 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1091681515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e5380d3270 con 0x55e538228370 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1091681515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f57d801a420 con 0x55e538228370 2024-05-08T23:28:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 10 client.9171 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 10 client.9171 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 20 client.9171 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55e538270540 0x55e538257300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 4 client.9171 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57fa766b80 1 -- 172.21.15.5:0/1091681515 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5381f97e0 con 0x55e538270540 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.901+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55e538270540 0x55e538257300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57c6ffd700 10 client.9171 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55e538270540 0x55e538257300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9286bbf44d22d405 server_cookie=1c77f93b9f381e33 in_seq=0 out_seq=0 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1091681515 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5381f97e0 con 0x55e538270540 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57c6ffd700 10 client.9171 handle_command_reply: tid=0 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 client.9171 shutdown 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 -- 172.21.15.5:0/1091681515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 msgr2=0x55e5382868f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382868f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f57e8072950 tx=0x7f57d800e000 comp rx=0 tx=0).stop 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 -- 172.21.15.5:0/1091681515 shutdown_connections 2024-05-08T23:28:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55e538270540 0x55e538257300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 0x55e53824a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382868f0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57c6ffd700 0 client.9171 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1091681515 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e538286e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.902+0000 7f57fa766b80 1 -- 172.21.15.5:0/1091681515 >> 172.21.15.5:0/1091681515 conn(0x55e538188210 msgr2=0x55e538224350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.903+0000 7f57fa766b80 1 -- 172.21.15.5:0/1091681515 shutdown_connections 2024-05-08T23:28:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.903+0000 7f57fa766b80 1 -- 172.21.15.5:0/1091681515 wait complete. 2024-05-08T23:28:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.903+0000 7f57fa766b80 20 client.9171 trim_cache size 0 max 16384 2024-05-08T23:28:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.921+0000 7f57fa766b80 1 Processor -- start 2024-05-08T23:28:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.921+0000 7f57fa766b80 1 -- start start 2024-05-08T23:28:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538228370 0x55e53824cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e53824a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538254fc0 0x55e538227300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57fa766b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e538225cf0 con 0x55e538254fc0 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57fa766b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e538250900 con 0x55e538228370 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57fa766b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e538287530 con 0x55e5382523d0 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57ecb49700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538254fc0 0x55e538227300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57ecb49700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538254fc0 0x55e538227300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48180/0 (socket says 172.21.15.5:48180) 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57ecb49700 1 -- 172.21.15.5:0/3359262412 learned_addr learned my addr 172.21.15.5:0/3359262412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57e7fff700 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538228370 0x55e53824cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57e77fe700 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e53824a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57ecb49700 1 -- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 msgr2=0x55e53824a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57ecb49700 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e53824a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57ecb49700 1 -- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538228370 msgr2=0x55e53824cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57ecb49700 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538228370 0x55e53824cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57ecb49700 1 -- 172.21.15.5:0/3359262412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e538140560 con 0x55e538254fc0 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.922+0000 7f57e77fe700 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e53824a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57ecb49700 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538254fc0 0x55e538227300 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f57e00154f0 tx=0x7f57e0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d572a6ff62668c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57ce7fc700 1 -- 172.21.15.5:0/3359262412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000e040 con 0x55e538254fc0 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57ce7fc700 1 -- 172.21.15.5:0/3359262412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000f040 con 0x55e538254fc0 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57ce7fc700 1 -- 172.21.15.5:0/3359262412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0021630 con 0x55e538254fc0 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57fa766b80 1 -- 172.21.15.5:0/3359262412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538254fc0 msgr2=0x55e538227300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57fa766b80 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538254fc0 0x55e538227300 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f57e00154f0 tx=0x7f57e0009750 comp rx=0 tx=0).stop 2024-05-08T23:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57fa766b80 1 -- 172.21.15.5:0/3359262412 shutdown_connections 2024-05-08T23:28:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57fa766b80 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538254fc0 0x55e538227300 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57fa766b80 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538228370 0x55e53824cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57fa766b80 1 --2- 172.21.15.5:0/3359262412 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e53824a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.923+0000 7f57fa766b80 1 -- 172.21.15.5:0/3359262412 >> 172.21.15.5:0/3359262412 conn(0x55e538188210 msgr2=0x55e5382558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.924+0000 7f57fa766b80 1 -- 172.21.15.5:0/3359262412 shutdown_connections 2024-05-08T23:28:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.924+0000 7f57fa766b80 1 -- 172.21.15.5:0/3359262412 wait complete. 2024-05-08T23:28:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.924+0000 7f57fa766b80 1 -- start start 2024-05-08T23:28:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.924+0000 7f57fa766b80 1 Processor -- start 2024-05-08T23:28:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e5382538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57fa766b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 0x55e538253df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57fa766b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e538229cf0 con 0x55e538228370 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57fa766b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53822b480 con 0x55e538254fc0 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57fa766b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5382497d0 con 0x55e5382523d0 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57fa766b80 10 client.0 _ll_register_callbacks cb 0x55e53811fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382dc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48186/0 (socket says 172.21.15.5:48186) 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.925+0000 7f57e7fff700 1 -- 172.21.15.5:0/1815194718 learned_addr learned my addr 172.21.15.5:0/1815194718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e5382538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57ecb49700 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 0x55e538253df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57e7fff700 1 -- 172.21.15.5:0/1815194718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 msgr2=0x55e5382538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57e7fff700 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e5382538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57e7fff700 1 -- 172.21.15.5:0/1815194718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 msgr2=0x55e538253df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57e7fff700 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 0x55e538253df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57e7fff700 1 -- 172.21.15.5:0/1815194718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e0009e70 con 0x55e538228370 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57e7fff700 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382dc6d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f57d801a650 tx=0x7f57d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27c0f60d3be7d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1815194718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d801ac10 con 0x55e538228370 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1815194718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57d801ad70 con 0x55e538228370 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57fa766b80 1 -- 172.21.15.5:0/1815194718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57dc056e80 con 0x55e538228370 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1815194718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d8026410 con 0x55e538228370 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57fa766b80 10 client.9177 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.926+0000 7f57fa766b80 1 -- 172.21.15.5:0/1815194718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55e537fffcd0 con 0x55e538228370 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1815194718 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc056e80 con 0x55e538228370 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1815194718 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f57d8026570 con 0x55e538228370 2024-05-08T23:28:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57fa766b80 10 client.9177 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57fa766b80 10 client.9177 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57fa766b80 20 client.9177 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55e538255c00 0x55e53824f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57fa766b80 4 client.9177 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57fa766b80 1 -- 172.21.15.5:0/1815194718 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e538172ec0 con 0x55e538255c00 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55e538255c00 0x55e53824f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57c6ffd700 10 client.9177 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57e77fe700 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55e538255c00 0x55e53824f630 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d395b9ccc783f33 server_cookie=ace93f2856c0b9c3 in_seq=0 out_seq=0 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57c6ffd700 1 -- 172.21.15.5:0/1815194718 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55e538172ec0 con 0x55e538255c00 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57c6ffd700 10 client.9177 handle_command_reply: tid=0 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.927+0000 7f57fa766b80 1 client.9177 shutdown 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 -- 172.21.15.5:0/1815194718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 msgr2=0x55e5382dc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382dc6d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f57d801a650 tx=0x7f57d80047c0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 -- 172.21.15.5:0/1815194718 shutdown_connections 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55e538255c00 0x55e53824f630 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e538254fc0 0x55e538253df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e538228370 0x55e5382dc6d0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57c6ffd700 0 client.9177 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 --2- 172.21.15.5:0/1815194718 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55e5382523d0 0x55e5382538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 -- 172.21.15.5:0/1815194718 >> 172.21.15.5:0/1815194718 conn(0x55e538188210 msgr2=0x55e538257a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 -- 172.21.15.5:0/1815194718 shutdown_connections 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 1 -- 172.21.15.5:0/1815194718 wait complete. 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57fa766b80 20 client.9177 trim_cache size 0 max 16384 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:33.935 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-05-08T23:28:33.936 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:33.936 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:33.936 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:33.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:33.936 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:33.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:33.936 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57ee5ac700 1 -- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f57d406ac90 msgr2=0x7f57d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f57d406ac90 0x7f57d406d150 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009c40 tx=0x7f57d8008030 comp rx=0 tx=0).stop 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57ee5ac700 1 -- 172.21.15.5:0/3910024879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e80cff30 msgr2=0x7f57e809d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e80cff30 0x7f57e809d2a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00c4b0 tx=0x7f57dc00c7c0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57ee5ac700 1 -- 172.21.15.5:0/3910024879 shutdown_connections 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.928+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f57d406ac90 0x7f57d406d150 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.929+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f57e815b710 0x7f57e809d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.929+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57e80b26e0 0x7f57e8159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.929+0000 7f57ee5ac700 1 --2- 172.21.15.5:0/3910024879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e80cff30 0x7f57e809d2a0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.929+0000 7f57ee5ac700 1 -- 172.21.15.5:0/3910024879 >> 172.21.15.5:0/3910024879 conn(0x7f57e80b2f60 msgr2=0x7f57e80cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.929+0000 7f57ee5ac700 1 -- 172.21.15.5:0/3910024879 shutdown_connections 2024-05-08T23:28:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:33.929+0000 7f57ee5ac700 1 -- 172.21.15.5:0/3910024879 wait complete. 2024-05-08T23:28:33.942 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:33.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.080+0000 7f3bbc9eb700 1 Processor -- start 2024-05-08T23:28:34.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.080+0000 7f3bbc9eb700 1 -- start start 2024-05-08T23:28:34.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bbc9eb700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bb80d0f50 0x7f3bb80cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bbc9eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80c6740 0x7f3bb80c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bbc9eb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb815ba40 con 0x7f3bb80b29e0 2024-05-08T23:28:34.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bbc9eb700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb80c7130 con 0x7f3bb80d0f50 2024-05-08T23:28:34.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bbc9eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80ca960 con 0x7f3bb80c6740 2024-05-08T23:28:34.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bb659c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3bb80b29e0 0x7f3bb815fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47050/0 (socket says 172.21.15.5:47050) 2024-05-08T23:28:34.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bb659c700 1 -- 172.21.15.5:0/487609059 learned_addr learned my addr 172.21.15.5:0/487609059 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:34.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/487609059 >> v2:172.21.15.49:3300/0 conn(0x7f3bb80d0f50 0x7f3bb80cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bb6d9d700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80c6740 0x7f3bb80c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.081+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845080042 0 0) 0x7f3bb815ba40 con 0x7f3bb80b29e0 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba00035b0 con 0x7f3bb80b29e0 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1103840409 0 0) 0x7f3ba00035b0 con 0x7f3bb80b29e0 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80c6740 msgr2=0x7f3bb80c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80c6740 0x7f3bb80c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 >> v2:172.21.15.49:3300/0 conn(0x7f3bb80d0f50 msgr2=0x7f3bb80cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 --2- 172.21.15.5:0/487609059 >> v2:172.21.15.49:3300/0 conn(0x7f3bb80d0f50 0x7f3bb80cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb80cabb0 con 0x7f3bb80b29e0 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f3ba8001670 con 0x7f3bb80b29e0 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/487609059 >> v2:172.21.15.49:3300/0 conn(0x7f3bb80d0f50 0x7f3bb80cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 >> v1:172.21.15.5:6789/0 conn(0x7f3bb80b29e0 legacy=0x7f3bb815fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80c6740 0x7f3ba0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3ba0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ba0004d90 0x7f3ba0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c7130 con 0x7f3ba0003f80 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80ca960 con 0x7f3ba0004d90 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba000ba40 con 0x7f3bb80c6740 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3ba0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb659c700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80c6740 0x7f3ba0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb6d9d700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ba0004d90 0x7f3ba0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb5d9b700 1 -- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80c6740 msgr2=0x7f3ba0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80c6740 0x7f3ba0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb5d9b700 1 -- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ba0004d90 msgr2=0x7f3ba0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ba0004d90 0x7f3ba0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.082+0000 7f3bb5d9b700 1 -- 172.21.15.5:0/487609059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba00035b0 con 0x7f3ba0003f80 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3ba0004850 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009d30 tx=0x7f3ba0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f9a67a5a5b8993 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac009e80 con 0x7f3ba0003f80 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac00e9a0 con 0x7f3ba0003f80 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bb559a700 1 -- 172.21.15.5:0/487609059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00ed30 con 0x7f3ba0003f80 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/487609059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 msgr2=0x7f3ba0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bbc9eb700 1 --2- 172.21.15.5:0/487609059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3ba0004850 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009d30 tx=0x7f3ba0005880 comp rx=0 tx=0).stop 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bb6d9d700 1 -- 172.21.15.5:0/487609059 reap_dead start 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/487609059 shutdown_connections 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/487609059 >> 172.21.15.5:0/487609059 conn(0x7f3bb80d03a0 msgr2=0x7f3bb80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/487609059 shutdown_connections 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.083+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/487609059 wait complete. 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.085+0000 7f3bbc9eb700 1 Processor -- start 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.085+0000 7f3bbc9eb700 1 -- start start 2024-05-08T23:28:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bbc9eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3bb8144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bbc9eb700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80ca2c0 0x7f3bb80aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bbc9eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb80af1b0 0x7f3bb8146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bbc9eb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80ca740 con 0x7f3ba0003f80 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bbc9eb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb815bbd0 con 0x7f3bb80af1b0 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bbc9eb700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb815d5c0 con 0x7f3bb80ca2c0 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3bb8144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3bb8144d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48202/0 (socket says 172.21.15.5:48202) 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 -- 172.21.15.5:0/4003115683 learned_addr learned my addr 172.21.15.5:0/4003115683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb6d9d700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb80af1b0 0x7f3bb8146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80ca2c0 0x7f3bb80aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 -- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80ca2c0 msgr2=0x7f3bb80aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80ca2c0 0x7f3bb80aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 -- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb80af1b0 msgr2=0x7f3bb8146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb80af1b0 0x7f3bb8146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 -- 172.21.15.5:0/4003115683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb815d8a0 con 0x7f3ba0003f80 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80ca2c0 0x7f3bb80aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb6d9d700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb80af1b0 0x7f3bb8146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.086+0000 7f3bb659c700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3bb8144d20 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8007290 tx=0x7f3ba8007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.087+0000 7f3ba6ffd700 1 -- 172.21.15.5:0/4003115683 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800f6c0 con 0x7f3ba0003f80 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.087+0000 7f3ba6ffd700 1 -- 172.21.15.5:0/4003115683 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba800f820 con 0x7f3ba0003f80 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.087+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb815db80 con 0x7f3ba0003f80 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.087+0000 7f3ba6ffd700 1 -- 172.21.15.5:0/4003115683 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800fbd0 con 0x7f3ba0003f80 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.087+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb815d1e0 con 0x7f3ba0003f80 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.088+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb815d400 con 0x7f3ba0003f80 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.088+0000 7f3ba6ffd700 1 -- 172.21.15.5:0/4003115683 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3ba800fd30 con 0x7f3ba0003f80 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.088+0000 7f3ba6ffd700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b9806ac90 0x7f3b9806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.089+0000 7f3ba6ffd700 1 -- 172.21.15.5:0/4003115683 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ba8089320 con 0x7f3ba0003f80 2024-05-08T23:28:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.089+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b9806ac90 0x7f3b9806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.089+0000 7f3bb5d9b700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b9806ac90 0x7f3b9806d150 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8157dd0 tx=0x7f3bac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.093+0000 7f3ba6ffd700 1 -- 172.21.15.5:0/4003115683 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3ba8054500 con 0x7f3ba0003f80 2024-05-08T23:28:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.358+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bb8068000 con 0x7f3ba0003f80 2024-05-08T23:28:34.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.359+0000 7f3ba6ffd700 1 -- 172.21.15.5:0/4003115683 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7f3ba8057b20 con 0x7f3ba0003f80 2024-05-08T23:28:34.359 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b9806ac90 msgr2=0x7f3b9806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b9806ac90 0x7f3b9806d150 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8157dd0 tx=0x7f3bac008030 comp rx=0 tx=0).stop 2024-05-08T23:28:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 msgr2=0x7f3bb8144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3bb8144d20 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8007290 tx=0x7f3ba8007650 comp rx=0 tx=0).stop 2024-05-08T23:28:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 shutdown_connections 2024-05-08T23:28:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3b9806ac90 0x7f3b9806d150 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bb80af1b0 0x7f3bb8146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba0003f80 0x7f3bb8144d20 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 --2- 172.21.15.5:0/4003115683 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bb80ca2c0 0x7f3bb80aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.361+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 >> 172.21.15.5:0/4003115683 conn(0x7f3bb80d03a0 msgr2=0x7f3bb806dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:34.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.362+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 shutdown_connections 2024-05-08T23:28:34.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.362+0000 7f3bbc9eb700 1 -- 172.21.15.5:0/4003115683 wait complete. 2024-05-08T23:28:34.363 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:34.376 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'b'] 2024-05-08T23:28:34.377 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:28:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.511+0000 7f5c0b59e700 1 Processor -- start 2024-05-08T23:28:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.511+0000 7f5c0b59e700 1 -- start start 2024-05-08T23:28:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.512+0000 7f5c0b59e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5c0c0b0a10 0x7f5c0c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.512+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c160250 0x7f5c0c06ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.512+0000 7f5c0b59e700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c0b54d0 con 0x7f5c0c06a960 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.512+0000 7f5c0b59e700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b1c40 con 0x7f5c0c0b0a10 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.512+0000 7f5c0b59e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c06b320 con 0x7f5c0c160250 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.512+0000 7f5c09d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5c0c06a960 0x7f5c0c161390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47066/0 (socket says 172.21.15.5:47066) 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.512+0000 7f5c09d9b700 1 -- 172.21.15.5:0/833347201 learned_addr learned my addr 172.21.15.5:0/833347201 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0ad9d700 1 --2- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c160250 0x7f5c0c06ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0a59c700 1 --2- 172.21.15.5:0/833347201 >> v2:172.21.15.49:3300/0 conn(0x7f5c0c0b0a10 0x7f5c0c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0959a700 1 -- 172.21.15.5:0/833347201 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438088642 0 0) 0x7f5c0c0b54d0 con 0x7f5c0c06a960 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0959a700 1 -- 172.21.15.5:0/833347201 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf40035b0 con 0x7f5c0c06a960 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0959a700 1 -- 172.21.15.5:0/833347201 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3634957604 0 0) 0x7f5bf40035b0 con 0x7f5c0c06a960 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0a59c700 1 -- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c160250 msgr2=0x7f5c0c06ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0a59c700 1 --2- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c160250 0x7f5c0c06ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0a59c700 1 -- 172.21.15.5:0/833347201 >> v1:172.21.15.5:6789/0 conn(0x7f5c0c06a960 legacy=0x7f5c0c161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0a59c700 1 -- 172.21.15.5:0/833347201 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0b4be0 con 0x7f5c0c0b0a10 2024-05-08T23:28:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0ad9d700 1 --2- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c160250 0x7f5c0c06ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0a59c700 1 --2- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c0b0a10 0x7f5c0c15e7f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009840 tx=0x7f5bfc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd9671a0c9ce964 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.513+0000 7f5c0959a700 1 -- 172.21.15.5:0/833347201 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00001670 con 0x7f5c0c0b0a10 2024-05-08T23:28:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0959a700 1 -- 172.21.15.5:0/833347201 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc004030 con 0x7f5c0c0b0a10 2024-05-08T23:28:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0959a700 1 -- 172.21.15.5:0/833347201 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0043a0 con 0x7f5c0c0b0a10 2024-05-08T23:28:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0b59e700 1 -- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c0b0a10 msgr2=0x7f5c0c15e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c0b0a10 0x7f5c0c15e7f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009840 tx=0x7f5bfc009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0b59e700 1 -- 172.21.15.5:0/833347201 shutdown_connections 2024-05-08T23:28:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c160250 0x7f5c0c06ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/833347201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c0b0a10 0x7f5c0c15e7f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0b59e700 1 -- 172.21.15.5:0/833347201 >> 172.21.15.5:0/833347201 conn(0x7f5c0c099a50 msgr2=0x7f5c0c0b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0b59e700 1 -- 172.21.15.5:0/833347201 shutdown_connections 2024-05-08T23:28:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.514+0000 7f5c0b59e700 1 -- 172.21.15.5:0/833347201 wait complete. 2024-05-08T23:28:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.516+0000 7f5c0b59e700 1 Processor -- start 2024-05-08T23:28:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.516+0000 7f5c0b59e700 1 -- start start 2024-05-08T23:28:34.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c06a960 0x7f5c0c09dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c0b0a10 0x7f5c0c1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c160250 0x7f5c0c151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0b59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c14fa80 con 0x7f5c0c06a960 2024-05-08T23:28:34.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0b59e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c09e320 con 0x7f5c0c160250 2024-05-08T23:28:34.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0b59e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c13ef00 con 0x7f5c0c0b0a10 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c06a960 0x7f5c0c09dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c06a960 0x7f5c0c09dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48216/0 (socket says 172.21.15.5:48216) 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 -- 172.21.15.5:0/2099926088 learned_addr learned my addr 172.21.15.5:0/2099926088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0ad9d700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c160250 0x7f5c0c151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c09d9b700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c0b0a10 0x7f5c0c1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 -- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c0b0a10 msgr2=0x7f5c0c1510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c0b0a10 0x7f5c0c1510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 -- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c160250 msgr2=0x7f5c0c151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c160250 0x7f5c0c151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 -- 172.21.15.5:0/2099926088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009a70 con 0x7f5c0c06a960 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.517+0000 7f5c0a59c700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c06a960 0x7f5c0c09dde0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009b90 tx=0x7f5bfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.518+0000 7f5bfaffd700 1 -- 172.21.15.5:0/2099926088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc003c30 con 0x7f5c0c06a960 2024-05-08T23:28:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.518+0000 7f5bfaffd700 1 -- 172.21.15.5:0/2099926088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc01f070 con 0x7f5c0c06a960 2024-05-08T23:28:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.518+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc0094a0 con 0x7f5c0c06a960 2024-05-08T23:28:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.518+0000 7f5bfaffd700 1 -- 172.21.15.5:0/2099926088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc011630 con 0x7f5c0c06a960 2024-05-08T23:28:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.518+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c14f3c0 con 0x7f5c0c06a960 2024-05-08T23:28:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.519+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c074b50 con 0x7f5c0c06a960 2024-05-08T23:28:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.519+0000 7f5bfaffd700 1 -- 172.21.15.5:0/2099926088 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5bfc0117b0 con 0x7f5c0c06a960 2024-05-08T23:28:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.520+0000 7f5bfaffd700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bec06ac90 0x7f5bec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.520+0000 7f5bfaffd700 1 -- 172.21.15.5:0/2099926088 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bfc021070 con 0x7f5c0c06a960 2024-05-08T23:28:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.520+0000 7f5c09d9b700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bec06ac90 0x7f5bec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.521+0000 7f5c09d9b700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bec06ac90 0x7f5bec06d150 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c06e7a0 tx=0x7f5c00001e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.525+0000 7f5bfaffd700 1 -- 172.21.15.5:0/2099926088 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5bfc055cb0 con 0x7f5c0c06a960 2024-05-08T23:28:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.841+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5c0c15eae0 con 0x7f5c0c06a960 2024-05-08T23:28:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.841+0000 7f5bfaffd700 1 -- 172.21.15.5:0/2099926088 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1925 (secure 0 0 0) 0x7f5bfc02b080 con 0x7f5c0c06a960 2024-05-08T23:28:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:34.841 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":469,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700348,"bytes_used":265920512,"bytes_avail":772828192768,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9895,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":109,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7052},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":7058},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8592}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-05-08T23:28:31.403877+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:28:34.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bec06ac90 msgr2=0x7f5bec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bec06ac90 0x7f5bec06d150 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c06e7a0 tx=0x7f5c00001e40 comp rx=0 tx=0).stop 2024-05-08T23:28:34.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c06a960 msgr2=0x7f5c0c09dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:34.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c06a960 0x7f5c0c09dde0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009b90 tx=0x7f5bfc004750 comp rx=0 tx=0).stop 2024-05-08T23:28:34.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 shutdown_connections 2024-05-08T23:28:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5bec06ac90 0x7f5bec06d150 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c0c160250 0x7f5c0c151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c06a960 0x7f5c0c09dde0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 --2- 172.21.15.5:0/2099926088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5c0c0b0a10 0x7f5c0c1510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 >> 172.21.15.5:0/2099926088 conn(0x7f5c0c099a50 msgr2=0x7f5c0c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 shutdown_connections 2024-05-08T23:28:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.844+0000 7f5c0b59e700 1 -- 172.21.15.5:0/2099926088 wait complete. 2024-05-08T23:28:34.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:28:34.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub resume 2024-05-08T23:28:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.997+0000 7fa1e3a76700 1 Processor -- start 2024-05-08T23:28:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.997+0000 7fa1e3a76700 1 -- start start 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e3a76700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1dc15ac00 0x7fa1dc15d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e3a76700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15d9b0 0x7fa1dc0b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e3a76700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0cd5c0 con 0x7fa1dc0b2cb0 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e3a76700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0b3120 con 0x7fa1dc15ac00 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e3a76700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc15df10 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e1812700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1dc15ac00 0x7fa1dc15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e2013700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa1dc0b2cb0 0x7fa1dc070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47082/0 (socket says 172.21.15.5:47082) 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e2013700 1 -- 172.21.15.5:0/2950883211 learned_addr learned my addr 172.21.15.5:0/2950883211 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e1011700 1 --2- 172.21.15.5:0/2950883211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15d9b0 0x7fa1dc0b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e1812700 1 -- 172.21.15.5:0/2950883211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15d9b0 msgr2=0x7fa1dc0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e1812700 1 --2- 172.21.15.5:0/2950883211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15d9b0 0x7fa1dc0b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e1812700 1 -- 172.21.15.5:0/2950883211 >> v1:172.21.15.5:6789/0 conn(0x7fa1dc0b2cb0 legacy=0x7fa1dc070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e1812700 1 -- 172.21.15.5:0/2950883211 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc15e1f0 con 0x7fa1dc15ac00 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e0810700 1 -- 172.21.15.5:0/2950883211 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532969431 0 0) 0x7fa1dc0cd5c0 con 0x7fa1dc0b2cb0 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.998+0000 7fa1e1011700 1 --2- 172.21.15.5:0/2950883211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15d9b0 0x7fa1dc0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e1812700 1 --2- 172.21.15.5:0/2950883211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15ac00 0x7fa1dc15d470 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009840 tx=0x7fa1d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ad8ef0ca52021f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e0810700 1 -- 172.21.15.5:0/2950883211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8011830 con 0x7fa1dc15ac00 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e0810700 1 -- 172.21.15.5:0/2950883211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8011990 con 0x7fa1dc15ac00 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e0810700 1 -- 172.21.15.5:0/2950883211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8011d20 con 0x7fa1dc15ac00 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e3a76700 1 -- 172.21.15.5:0/2950883211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15ac00 msgr2=0x7fa1dc15d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/2950883211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15ac00 0x7fa1dc15d470 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009840 tx=0x7fa1d8009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e3a76700 1 -- 172.21.15.5:0/2950883211 shutdown_connections 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/2950883211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15ac00 0x7fa1dc15d470 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/2950883211 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15d9b0 0x7fa1dc0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e3a76700 1 -- 172.21.15.5:0/2950883211 >> 172.21.15.5:0/2950883211 conn(0x7fa1dc0cf6c0 msgr2=0x7fa1dc0cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e3a76700 1 -- 172.21.15.5:0/2950883211 shutdown_connections 2024-05-08T23:28:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:34.999+0000 7fa1e3a76700 1 -- 172.21.15.5:0/2950883211 wait complete. 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.001+0000 7fa1e3a76700 1 Processor -- start 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.001+0000 7fa1e3a76700 1 -- start start 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.001+0000 7fa1e3a76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1dc0b2cb0 0x7fa1dc0acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e3a76700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15ac00 0x7fa1dc1408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e3a76700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15d9b0 0x7fa1dc1f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e3a76700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc140df0 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e3a76700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0ae2d0 con 0x7fa1dc0b2cb0 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e3a76700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0ad230 con 0x7fa1dc15ac00 2024-05-08T23:28:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15d9b0 0x7fa1dc1f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15d9b0 0x7fa1dc1f4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48246/0 (socket says 172.21.15.5:48246) 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 -- 172.21.15.5:0/1780467331 learned_addr learned my addr 172.21.15.5:0/1780467331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e1812700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1dc0b2cb0 0x7fa1dc0acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e1011700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15ac00 0x7fa1dc1408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 -- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15ac00 msgr2=0x7fa1dc1408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15ac00 0x7fa1dc1408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 -- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1dc0b2cb0 msgr2=0x7fa1dc0acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1dc0b2cb0 0x7fa1dc0acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 -- 172.21.15.5:0/1780467331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d80094a0 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e2013700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15d9b0 0x7fa1dc1f4df0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4002230 tx=0x7fa1d400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1d27fc700 1 -- 172.21.15.5:0/1780467331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400aac0 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1d27fc700 1 -- 172.21.15.5:0/1780467331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d400ac40 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1d27fc700 1 -- 172.21.15.5:0/1780467331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40070d0 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.002+0000 7fa1e3a76700 1 -- 172.21.15.5:0/1780467331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc0ad510 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.003+0000 7fa1e3a76700 1 -- 172.21.15.5:0/1780467331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc0adad0 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.004+0000 7fa1d27fc700 1 -- 172.21.15.5:0/1780467331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa1d4007250 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.004+0000 7fa1d27fc700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1c006ac90 0x7fa1c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.005+0000 7fa1d27fc700 1 -- 172.21.15.5:0/1780467331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1d4088d80 con 0x7fa1dc15d9b0 2024-05-08T23:28:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.005+0000 7fa1e1812700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1c006ac90 0x7fa1c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.005+0000 7fa1e1812700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1c006ac90 0x7fa1c006d150 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80090b0 tx=0x7fa1d8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.007+0000 7fa1efc30b80 1 Processor -- start 2024-05-08T23:28:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.007+0000 7fa1efc30b80 1 -- start start 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc58809250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ea3d0 0x55cc587c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc58719150 con 0x55cc587ecfc0 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc58716b10 con 0x55cc587ea3d0 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc58712400 con 0x55cc587c0370 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587ed3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48248/0 (socket says 172.21.15.5:48248) 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 -- 172.21.15.5:0/3595479453 learned_addr learned my addr 172.21.15.5:0/3595479453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e2013700 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ea3d0 0x55cc587c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1011700 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc58809250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 -- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 msgr2=0x55cc58809250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc58809250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 -- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ea3d0 msgr2=0x55cc587c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ea3d0 0x55cc587c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 -- 172.21.15.5:0/3595479453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc585933b0 con 0x55cc587ecfc0 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1011700 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc58809250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1e1812700 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587ed3a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8004040 tx=0x7fa1d8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d98363411d40d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1bf7fe700 1 -- 172.21.15.5:0/3595479453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d802c070 con 0x55cc587ecfc0 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1bf7fe700 1 -- 172.21.15.5:0/3595479453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d80037a0 con 0x55cc587ecfc0 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.008+0000 7fa1bf7fe700 1 -- 172.21.15.5:0/3595479453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8020070 con 0x55cc587ecfc0 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3595479453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 msgr2=0x55cc587ed3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587ed3a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8004040 tx=0x7fa1d8010000 comp rx=0 tx=0).stop 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3595479453 shutdown_connections 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ea3d0 0x55cc587c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587ed3a0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3595479453 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc58809250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3595479453 >> 172.21.15.5:0/3595479453 conn(0x55cc58720210 msgr2=0x55cc58807dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3595479453 shutdown_connections 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.009+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3595479453 wait complete. 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.010+0000 7fa1efc30b80 1 -- start start 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.010+0000 7fa1efc30b80 1 Processor -- start 2024-05-08T23:28:35.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587c0370 0x55cc587fee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587ea3d0 0x55cc587e1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc586ae560 con 0x55cc587ecfc0 2024-05-08T23:28:35.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc586ae200 con 0x55cc587c0370 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc585956a0 con 0x55cc587ea3d0 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1efc30b80 10 client.0 _ll_register_callbacks cb 0x55cc586b7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1e2013700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1e2013700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587df8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48254/0 (socket says 172.21.15.5:48254) 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1e2013700 1 -- 172.21.15.5:0/2567546483 learned_addr learned my addr 172.21.15.5:0/2567546483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1e1011700 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587ea3d0 0x55cc587e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.011+0000 7fa1e1812700 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587c0370 0x55cc587fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1e2013700 1 -- 172.21.15.5:0/2567546483 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587ea3d0 msgr2=0x55cc587e1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1e2013700 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587ea3d0 0x55cc587e1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1e2013700 1 -- 172.21.15.5:0/2567546483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587c0370 msgr2=0x55cc587fee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1e2013700 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587c0370 0x55cc587fee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1e2013700 1 -- 172.21.15.5:0/2567546483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc15e1f0 con 0x55cc587ecfc0 2024-05-08T23:28:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1e2013700 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587df8e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4057920 tx=0x7fa1d4058160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958143ac5d013a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1bcff9700 1 -- 172.21.15.5:0/2567546483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d405fd80 con 0x55cc587ecfc0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1bcff9700 1 -- 172.21.15.5:0/2567546483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1d405ba50 con 0x55cc587ecfc0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2567546483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc5866b270 con 0x55cc587ecfc0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1bcff9700 1 -- 172.21.15.5:0/2567546483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1d4060010 con 0x55cc587ecfc0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1efc30b80 10 client.9204 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2567546483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55cc5853fb60 con 0x55cc587ecfc0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1bcff9700 1 -- 172.21.15.5:0/2567546483 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cc5866b270 con 0x55cc587ecfc0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1bcff9700 1 -- 172.21.15.5:0/2567546483 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fa1d4084020 con 0x55cc587ecfc0 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1efc30b80 10 client.9204 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.012+0000 7fa1efc30b80 10 client.9204 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.013+0000 7fa1efc30b80 20 client.9204 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:35.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.013+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55cc587eb9a0 0x55cc587ebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.013+0000 7fa1efc30b80 4 client.9204 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.013+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2567546483 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc587917e0 con 0x55cc587eb9a0 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.013+0000 7fa1e1812700 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55cc587eb9a0 0x55cc587ebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.013+0000 7fa1bcff9700 10 client.9204 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.013+0000 7fa1e1812700 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55cc587eb9a0 0x55cc587ebdb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74581ad4b7348028 server_cookie=639f010ea982681f in_seq=0 out_seq=0 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1bcff9700 1 -- 172.21.15.5:0/2567546483 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc587917e0 con 0x55cc587eb9a0 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1bcff9700 10 client.9204 handle_command_reply: tid=0 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 client.9204 shutdown 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2567546483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 msgr2=0x55cc587df8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587df8e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4057920 tx=0x7fa1d4058160 comp rx=0 tx=0).stop 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2567546483 shutdown_connections 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55cc587eb9a0 0x55cc587ebdb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ecfc0 0x55cc587df8e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587c0370 0x55cc587fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1bcff9700 0 client.9204 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2567546483 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587ea3d0 0x55cc587e1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2567546483 >> 172.21.15.5:0/2567546483 conn(0x55cc58720210 msgr2=0x55cc587c2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2567546483 shutdown_connections 2024-05-08T23:28:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2567546483 wait complete. 2024-05-08T23:28:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.014+0000 7fa1efc30b80 20 client.9204 trim_cache size 0 max 16384 2024-05-08T23:28:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.032+0000 7fa1efc30b80 1 Processor -- start 2024-05-08T23:28:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.032+0000 7fa1efc30b80 1 -- start start 2024-05-08T23:28:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc587c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc587c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5880c0b0 con 0x55cc587ea3d0 2024-05-08T23:28:35.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc587be850 con 0x55cc587ecfc0 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc5880bed0 con 0x55cc587c0370 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc587c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc587c0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48256/0 (socket says 172.21.15.5:48256) 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 -- 172.21.15.5:0/2585268746 learned_addr learned my addr 172.21.15.5:0/2585268746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1812700 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc587c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e2013700 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 -- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 msgr2=0x55cc587c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc587c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 -- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 msgr2=0x55cc587e0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587e0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 -- 172.21.15.5:0/2585268746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc586d8560 con 0x55cc587ea3d0 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e2013700 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:35.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.033+0000 7fa1e1011700 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc587c0c90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00aad0 tx=0x7fa1cc00ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765c9dce1b87c933 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1bffff700 1 -- 172.21.15.5:0/2585268746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc014a80 con 0x55cc587ea3d0 2024-05-08T23:28:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1bffff700 1 -- 172.21.15.5:0/2585268746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc00c040 con 0x55cc587ea3d0 2024-05-08T23:28:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1bffff700 1 -- 172.21.15.5:0/2585268746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc014e00 con 0x55cc587ea3d0 2024-05-08T23:28:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2585268746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 msgr2=0x55cc587c0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc587c0c90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00aad0 tx=0x7fa1cc00ade0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2585268746 shutdown_connections 2024-05-08T23:28:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc587c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/2585268746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc587c0c90 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2585268746 >> 172.21.15.5:0/2585268746 conn(0x55cc58720210 msgr2=0x55cc58800970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2585268746 shutdown_connections 2024-05-08T23:28:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.034+0000 7fa1efc30b80 1 -- 172.21.15.5:0/2585268746 wait complete. 2024-05-08T23:28:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.035+0000 7fa1efc30b80 1 -- start start 2024-05-08T23:28:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.035+0000 7fa1efc30b80 1 Processor -- start 2024-05-08T23:28:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc587e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc5881d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1efc30b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc587e9cb0 con 0x55cc587ea3d0 2024-05-08T23:28:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc587c1130 con 0x55cc587ecfc0 2024-05-08T23:28:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1efc30b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc5880b260 con 0x55cc587c0370 2024-05-08T23:28:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1efc30b80 10 client.0 _ll_register_callbacks cb 0x7fa1d4057760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1011700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc5881d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1011700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc5881d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48260/0 (socket says 172.21.15.5:48260) 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1011700 1 -- 172.21.15.5:0/3516640854 learned_addr learned my addr 172.21.15.5:0/3516640854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e2013700 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1812700 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc587e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1011700 1 -- 172.21.15.5:0/3516640854 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 msgr2=0x55cc587e03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1011700 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc587e03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1011700 1 -- 172.21.15.5:0/3516640854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 msgr2=0x55cc587eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1011700 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587eff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.036+0000 7fa1e1011700 1 -- 172.21.15.5:0/3516640854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc00a7b0 con 0x55cc587ea3d0 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1e1011700 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc5881d500 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc008010 tx=0x7fa1cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18c710ab073cfd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1bcff9700 1 -- 172.21.15.5:0/3516640854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc01f890 con 0x55cc587ea3d0 2024-05-08T23:28:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3516640854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc587ff9a0 con 0x55cc587ea3d0 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1bcff9700 1 -- 172.21.15.5:0/3516640854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1cc01a070 con 0x55cc587ea3d0 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1bcff9700 1 -- 172.21.15.5:0/3516640854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1cc01f9f0 con 0x55cc587ea3d0 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 10 client.9210 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3516640854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55cc58597cd0 con 0x55cc587ea3d0 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1bcff9700 1 -- 172.21.15.5:0/3516640854 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cc587ff9a0 con 0x55cc587ea3d0 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1bcff9700 1 -- 172.21.15.5:0/3516640854 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fa1cc01fb50 con 0x55cc587ea3d0 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 10 client.9210 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 10 client.9210 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 20 client.9210 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55cc587c0750 0x55cc587f0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 4 client.9210 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3516640854 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x7fa1d80037a0 con 0x55cc587c0750 2024-05-08T23:28:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.037+0000 7fa1e1812700 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55cc587c0750 0x55cc587f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1bcff9700 10 client.9210 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1e1812700 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55cc587c0750 0x55cc587f0440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee3280b2b2aeaf1 server_cookie=74ec880cd5ffbaf0 in_seq=0 out_seq=0 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1bcff9700 1 -- 172.21.15.5:0/3516640854 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x7fa1d80037a0 con 0x55cc587c0750 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1bcff9700 10 client.9210 handle_command_reply: tid=0 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 client.9210 shutdown 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3516640854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 msgr2=0x55cc5881d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc5881d500 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc008010 tx=0x7fa1cc0047c0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3516640854 shutdown_connections 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x55cc587c0750 0x55cc587f0440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc587ecfc0 0x55cc587eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x55cc587c0370 0x55cc587e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1bcff9700 0 client.9210 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 --2- 172.21.15.5:0/3516640854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc587ea3d0 0x55cc5881d500 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3516640854 >> 172.21.15.5:0/3516640854 conn(0x55cc58720210 msgr2=0x55cc587ea810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3516640854 shutdown_connections 2024-05-08T23:28:35.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 1 -- 172.21.15.5:0/3516640854 wait complete. 2024-05-08T23:28:35.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.038+0000 7fa1efc30b80 20 client.9210 trim_cache size 0 max 16384 2024-05-08T23:28:35.048 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:35.048 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-08T23:28:35.048 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:35.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 -- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1c006ac90 msgr2=0x7fa1c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1c006ac90 0x7fa1c006d150 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80090b0 tx=0x7fa1d8008030 comp rx=0 tx=0).stop 2024-05-08T23:28:35.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 -- 172.21.15.5:0/1780467331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15d9b0 msgr2=0x7fa1dc1f4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15d9b0 0x7fa1dc1f4df0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4002230 tx=0x7fa1d400cf50 comp rx=0 tx=0).stop 2024-05-08T23:28:35.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 -- 172.21.15.5:0/1780467331 shutdown_connections 2024-05-08T23:28:35.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa1c006ac90 0x7fa1c006d150 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc15d9b0 0x7fa1dc1f4df0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1dc0b2cb0 0x7fa1dc0acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 --2- 172.21.15.5:0/1780467331 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa1dc15ac00 0x7fa1dc1408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 -- 172.21.15.5:0/1780467331 >> 172.21.15.5:0/1780467331 conn(0x7fa1dc0cf6c0 msgr2=0x7fa1dc0b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 -- 172.21.15.5:0/1780467331 shutdown_connections 2024-05-08T23:28:35.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.039+0000 7fa1e3a76700 1 -- 172.21.15.5:0/1780467331 wait complete. 2024-05-08T23:28:35.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 scrub status 2024-05-08T23:28:35.133 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:28:35.133+0000 7f8a4c50b700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T23:28:35.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.190+0000 7f98e2b8f700 1 Processor -- start 2024-05-08T23:28:35.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.190+0000 7f98e2b8f700 1 -- start start 2024-05-08T23:28:35.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.191+0000 7f98e2b8f700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98dc0d0a60 0x7f98dc0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.191+0000 7f98e2b8f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f98dc15fbb0 0x7f98dc0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.191+0000 7f98e2b8f700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0af6e0 con 0x7f98dc145040 2024-05-08T23:28:35.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.191+0000 7f98e2b8f700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0b6460 con 0x7f98dc15fbb0 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.191+0000 7f98e2b8f700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0aeb50 con 0x7f98dc0d0a60 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.191+0000 7f98e112c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f98dc15fbb0 0x7f98dc0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.191+0000 7f98e092b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f98dc145040 0x7f98dc0d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47098/0 (socket says 172.21.15.5:47098) 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.191+0000 7f98e092b700 1 -- 172.21.15.5:0/3001631746 learned_addr learned my addr 172.21.15.5:0/3001631746 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98dbfff700 1 --2- 172.21.15.5:0/3001631746 >> v2:172.21.15.49:3300/0 conn(0x7f98dc0d0a60 0x7f98dc0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98e112c700 1 -- 172.21.15.5:0/3001631746 >> v2:172.21.15.49:3300/0 conn(0x7f98dc0d0a60 msgr2=0x7f98dc0b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98e112c700 1 --2- 172.21.15.5:0/3001631746 >> v2:172.21.15.49:3300/0 conn(0x7f98dc0d0a60 0x7f98dc0b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98e112c700 1 -- 172.21.15.5:0/3001631746 >> v1:172.21.15.5:6789/0 conn(0x7f98dc145040 legacy=0x7f98dc0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98db7fe700 1 -- 172.21.15.5:0/3001631746 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269622381 0 0) 0x7f98dc0af6e0 con 0x7f98dc145040 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98e112c700 1 -- 172.21.15.5:0/3001631746 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc0aedd0 con 0x7f98dc15fbb0 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98e112c700 1 --2- 172.21.15.5:0/3001631746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc0afb70 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0b3000 tx=0x7f98d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b950f736ffeb662d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98db7fe700 1 -- 172.21.15.5:0/3001631746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400e070 con 0x7f98dc15fbb0 2024-05-08T23:28:35.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98db7fe700 1 -- 172.21.15.5:0/3001631746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d400bbd0 con 0x7f98dc15fbb0 2024-05-08T23:28:35.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.192+0000 7f98db7fe700 1 -- 172.21.15.5:0/3001631746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4012430 con 0x7f98dc15fbb0 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.193+0000 7f98e2b8f700 1 -- 172.21.15.5:0/3001631746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 msgr2=0x7f98dc0afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.193+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/3001631746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc0afb70 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0b3000 tx=0x7f98d400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.193+0000 7f98e2b8f700 1 -- 172.21.15.5:0/3001631746 shutdown_connections 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.193+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/3001631746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc0afb70 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.193+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/3001631746 >> v2:172.21.15.49:3300/0 conn(0x7f98dc0d0a60 0x7f98dc0b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.193+0000 7f98e2b8f700 1 -- 172.21.15.5:0/3001631746 >> 172.21.15.5:0/3001631746 conn(0x7f98dc099a50 msgr2=0x7f98dc0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.193+0000 7f98e2b8f700 1 -- 172.21.15.5:0/3001631746 shutdown_connections 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.193+0000 7f98e2b8f700 1 -- 172.21.15.5:0/3001631746 wait complete. 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.194+0000 7f98e2b8f700 1 Processor -- start 2024-05-08T23:28:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.194+0000 7f98e2b8f700 1 -- start start 2024-05-08T23:28:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e2b8f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98dc0d0a60 0x7f98dc15c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e2b8f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f98dc145040 0x7f98dc150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e2b8f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e2b8f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc150690 con 0x7f98dc15fbb0 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e2b8f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0ce1a0 con 0x7f98dc0d0a60 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e2b8f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc13cf50 con 0x7f98dc145040 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e112c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e112c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc14a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48278/0 (socket says 172.21.15.5:48278) 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.195+0000 7f98e112c700 1 -- 172.21.15.5:0/1126889582 learned_addr learned my addr 172.21.15.5:0/1126889582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e092b700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98dc0d0a60 0x7f98dc15c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98dbfff700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f98dc145040 0x7f98dc150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e112c700 1 -- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f98dc145040 msgr2=0x7f98dc150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e112c700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f98dc145040 0x7f98dc150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e112c700 1 -- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98dc0d0a60 msgr2=0x7f98dc15c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e112c700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98dc0d0a60 0x7f98dc15c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e112c700 1 -- 172.21.15.5:0/1126889582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4009a70 con 0x7f98dc15fbb0 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98dbfff700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f98dc145040 0x7f98dc150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e112c700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc14a7a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0b5790 tx=0x7f98d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98d97fa700 1 -- 172.21.15.5:0/1126889582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4012430 con 0x7f98dc15fbb0 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98d97fa700 1 -- 172.21.15.5:0/1126889582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d4020070 con 0x7f98dc15fbb0 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e2b8f700 1 -- 172.21.15.5:0/1126889582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc0cd870 con 0x7f98dc15fbb0 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98d97fa700 1 -- 172.21.15.5:0/1126889582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4004280 con 0x7f98dc15fbb0 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.196+0000 7f98e2b8f700 1 -- 172.21.15.5:0/1126889582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc0cde30 con 0x7f98dc15fbb0 2024-05-08T23:28:35.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.198+0000 7f98d97fa700 1 -- 172.21.15.5:0/1126889582 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f98d40043e0 con 0x7f98dc15fbb0 2024-05-08T23:28:35.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.198+0000 7f98d97fa700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f98c806ac90 0x7f98c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.198+0000 7f98d97fa700 1 -- 172.21.15.5:0/1126889582 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98d4022070 con 0x7f98dc15fbb0 2024-05-08T23:28:35.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.198+0000 7f98e092b700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f98c806ac90 0x7f98c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.199+0000 7f98e092b700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f98c806ac90 0x7f98c806d150 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f98cc000f60 tx=0x7f98cc007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.200+0000 7f98eed49b80 1 Processor -- start 2024-05-08T23:28:35.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.200+0000 7f98eed49b80 1 -- start start 2024-05-08T23:28:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09aa9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09aae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09ad8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98eed49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b09a04150 con 0x557b09ad07a0 2024-05-08T23:28:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98eed49b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b09a01b10 con 0x557b09aab370 2024-05-08T23:28:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98eed49b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b099fd400 con 0x557b09ad7dd0 2024-05-08T23:28:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e092b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09aa9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e092b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09aa9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48288/0 (socket says 172.21.15.5:48288) 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e092b700 1 -- 172.21.15.5:0/4019099783 learned_addr learned my addr 172.21.15.5:0/4019099783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98dbfff700 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09aae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e112c700 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09ad8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e092b700 1 -- 172.21.15.5:0/4019099783 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 msgr2=0x557b09ad8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e092b700 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09ad8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e092b700 1 -- 172.21.15.5:0/4019099783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 msgr2=0x557b09aae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e092b700 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09aae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.201+0000 7f98e092b700 1 -- 172.21.15.5:0/4019099783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b0987e3b0 con 0x557b09ad07a0 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.202+0000 7f98e092b700 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09aa9110 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f98cc015620 tx=0x7f98cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187fc3aaf60f6b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.202+0000 7f98c27fc700 1 -- 172.21.15.5:0/4019099783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc015e80 con 0x557b09ad07a0 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.202+0000 7f98c27fc700 1 -- 172.21.15.5:0/4019099783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc015320 con 0x557b09ad07a0 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.202+0000 7f98c27fc700 1 -- 172.21.15.5:0/4019099783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc021650 con 0x557b09ad07a0 2024-05-08T23:28:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.202+0000 7f98eed49b80 1 -- 172.21.15.5:0/4019099783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 msgr2=0x557b09aa9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.202+0000 7f98eed49b80 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09aa9110 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f98cc015620 tx=0x7f98cc012000 comp rx=0 tx=0).stop 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.203+0000 7f98eed49b80 1 -- 172.21.15.5:0/4019099783 shutdown_connections 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.203+0000 7f98eed49b80 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09ad8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.203+0000 7f98eed49b80 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09aa9110 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.203+0000 7f98eed49b80 1 --2- 172.21.15.5:0/4019099783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09aae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.203+0000 7f98eed49b80 1 -- 172.21.15.5:0/4019099783 >> 172.21.15.5:0/4019099783 conn(0x557b09a0b210 msgr2=0x557b09af2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.203+0000 7f98eed49b80 1 -- 172.21.15.5:0/4019099783 shutdown_connections 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.203+0000 7f98eed49b80 1 -- 172.21.15.5:0/4019099783 wait complete. 2024-05-08T23:28:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.203+0000 7f98eed49b80 1 -- start start 2024-05-08T23:28:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.204+0000 7f98eed49b80 1 Processor -- start 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09aab370 0x557b09af6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09ad07a0 0x557b09acf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09acf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98eed49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b09999560 con 0x557b09aab370 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98eed49b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b09999200 con 0x557b09ad07a0 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98eed49b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b098806a0 con 0x557b09ad7dd0 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98eed49b80 10 client.0 _ll_register_callbacks cb 0x557b099a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e092b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09aab370 0x557b09af6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e092b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09aab370 0x557b09af6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48302/0 (socket says 172.21.15.5:48302) 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e092b700 1 -- 172.21.15.5:0/2434437498 learned_addr learned my addr 172.21.15.5:0/2434437498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98dbfff700 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09ad07a0 0x557b09acf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e112c700 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09acf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e092b700 1 -- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 msgr2=0x557b09acf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e092b700 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09acf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e092b700 1 -- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09ad07a0 msgr2=0x557b09acf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e092b700 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09ad07a0 0x557b09acf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98e092b700 1 -- 172.21.15.5:0/2434437498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc00f040 con 0x557b09aab370 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.205+0000 7f98dbfff700 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09ad07a0 0x557b09acf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98e092b700 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09aab370 0x557b09af6a60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f98cc015c10 tx=0x7f98cc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd32ba40e60dfd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2434437498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc005be0 con 0x557b09aab370 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2434437498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98cc021be0 con 0x557b09aab370 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 1 -- 172.21.15.5:0/2434437498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b09956270 con 0x557b09aab370 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2434437498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98cc021da0 con 0x557b09aab370 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 10 client.9225 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 1 -- 172.21.15.5:0/2434437498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x557b0982ab60 con 0x557b09aab370 2024-05-08T23:28:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2434437498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b09956270 con 0x557b09aab370 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2434437498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f98cc02c470 con 0x557b09aab370 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 10 client.9225 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 10 client.9225 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 20 client.9225 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x557b09af5ad0 0x557b09af8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 4 client.9225 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.206+0000 7f98eed49b80 1 -- 172.21.15.5:0/2434437498 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b09a7c7e0 con 0x557b09af5ad0 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.207+0000 7f98dbfff700 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x557b09af5ad0 0x557b09af8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.207+0000 7f98bb7fe700 10 client.9225 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.207+0000 7f98dbfff700 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x557b09af5ad0 0x557b09af8510 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14bc669966f68bcb server_cookie=cc6c748c9700f84a in_seq=0 out_seq=0 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.207+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2434437498 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b09a7c7e0 con 0x557b09af5ad0 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.207+0000 7f98bb7fe700 10 client.9225 handle_command_reply: tid=0 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.207+0000 7f98eed49b80 1 client.9225 shutdown 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 -- 172.21.15.5:0/2434437498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09aab370 msgr2=0x557b09af6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09aab370 0x557b09af6a60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f98cc015c10 tx=0x7f98cc000ed0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 -- 172.21.15.5:0/2434437498 shutdown_connections 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x557b09af5ad0 0x557b09af8510 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09acf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09aab370 0x557b09af6a60 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2434437498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09ad07a0 0x557b09acf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98bb7fe700 0 client.9225 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 -- 172.21.15.5:0/2434437498 >> 172.21.15.5:0/2434437498 conn(0x557b09a0b210 msgr2=0x557b09af3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 -- 172.21.15.5:0/2434437498 shutdown_connections 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 1 -- 172.21.15.5:0/2434437498 wait complete. 2024-05-08T23:28:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.208+0000 7f98eed49b80 20 client.9225 trim_cache size 0 max 16384 2024-05-08T23:28:35.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.226+0000 7f98eed49b80 1 Processor -- start 2024-05-08T23:28:35.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.226+0000 7f98eed49b80 1 -- start start 2024-05-08T23:28:35.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09af28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09b525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09adadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98eed49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b09af76c0 con 0x557b09ad07a0 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98eed49b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b09af7e80 con 0x557b09aab370 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98eed49b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b09af33f0 con 0x557b09ad7dd0 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09b525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09b525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48310/0 (socket says 172.21.15.5:48310) 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98dbfff700 1 -- 172.21.15.5:0/2088850240 learned_addr learned my addr 172.21.15.5:0/2088850240 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98dbfff700 1 -- 172.21.15.5:0/2088850240 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 msgr2=0x557b09adadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98dbfff700 1 --2- 172.21.15.5:0/2088850240 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09adadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98dbfff700 1 -- 172.21.15.5:0/2088850240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 msgr2=0x557b09af28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98dbfff700 1 --2- 172.21.15.5:0/2088850240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09af28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.227+0000 7f98dbfff700 1 -- 172.21.15.5:0/2088850240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b099c3560 con 0x557b09ad07a0 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98dbfff700 1 --2- 172.21.15.5:0/2088850240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09b525c0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f98dc15c190 tx=0x7f98d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b59f06a0e0f662 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98c2ffd700 1 -- 172.21.15.5:0/2088850240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d001c070 con 0x557b09ad07a0 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98c2ffd700 1 -- 172.21.15.5:0/2088850240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0020070 con 0x557b09ad07a0 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98c2ffd700 1 -- 172.21.15.5:0/2088850240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0004240 con 0x557b09ad07a0 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 -- 172.21.15.5:0/2088850240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 msgr2=0x557b09b525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2088850240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09b525c0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f98dc15c190 tx=0x7f98d0008040 comp rx=0 tx=0).stop 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 -- 172.21.15.5:0/2088850240 shutdown_connections 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2088850240 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad7dd0 0x557b09adadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2088850240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09af28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2088850240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad07a0 0x557b09b525c0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 -- 172.21.15.5:0/2088850240 >> 172.21.15.5:0/2088850240 conn(0x557b09a0b210 msgr2=0x557b09aa90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 -- 172.21.15.5:0/2088850240 shutdown_connections 2024-05-08T23:28:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.228+0000 7f98eed49b80 1 -- 172.21.15.5:0/2088850240 wait complete. 2024-05-08T23:28:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.229+0000 7f98eed49b80 1 -- start start 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.229+0000 7f98eed49b80 1 Processor -- start 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09b519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad07a0 0x557b09b5c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98eed49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad7dd0 0x557b09b55440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98eed49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b09b55020 con 0x557b09ad7dd0 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98eed49b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b09b5cf40 con 0x557b09aab370 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98eed49b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b09aa6ce0 con 0x557b09ad07a0 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98eed49b80 10 client.0 _ll_register_callbacks cb 0x557b099a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad7dd0 0x557b09b55440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad7dd0 0x557b09b55440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48324/0 (socket says 172.21.15.5:48324) 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 -- 172.21.15.5:0/2364727057 learned_addr learned my addr 172.21.15.5:0/2364727057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e092b700 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09b519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98dbfff700 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad07a0 0x557b09b5c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 -- 172.21.15.5:0/2364727057 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad07a0 msgr2=0x557b09b5c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad07a0 0x557b09b5c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 -- 172.21.15.5:0/2364727057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 msgr2=0x557b09b519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09b519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 -- 172.21.15.5:0/2364727057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d000e040 con 0x557b09ad7dd0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98e112c700 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad7dd0 0x557b09b55440 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f98d400b370 tx=0x7f98d408b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a987d3e1862e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.231+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2364727057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4059ae0 con 0x557b09ad7dd0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2364727057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98d405aaf0 con 0x557b09ad7dd0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 1 -- 172.21.15.5:0/2364727057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98cc015320 con 0x557b09ad7dd0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2364727057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98d405a3f0 con 0x557b09ad7dd0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 10 client.9231 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 1 -- 172.21.15.5:0/2364727057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x557b09882cd0 con 0x557b09ad7dd0 2024-05-08T23:28:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2364727057 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d407a020 con 0x557b09ad7dd0 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2364727057 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f98d4062350 con 0x557b09ad7dd0 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 10 client.9231 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 10 client.9231 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 20 client.9231 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x557b09b55980 0x557b09aab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 4 client.9231 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98eed49b80 1 -- 172.21.15.5:0/2364727057 --> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b099f5ec0 con 0x557b09b55980 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.232+0000 7f98e092b700 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x557b09b55980 0x557b09aab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98e092b700 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x557b09b55980 0x557b09aab750 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f13d224a04515d2 server_cookie=e4b9bfd7aae0778b in_seq=0 out_seq=0 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98bb7fe700 10 client.9231 ms_handle_connect on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98bb7fe700 1 -- 172.21.15.5:0/2364727057 <== mds.1 v2:172.21.15.5:6833/1828661194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x557b099f5ec0 con 0x557b09b55980 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98bb7fe700 10 client.9231 handle_command_reply: tid=0 2024-05-08T23:28:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 client.9231 shutdown 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 -- 172.21.15.5:0/2364727057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad7dd0 msgr2=0x557b09b55440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad7dd0 0x557b09b55440 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f98d400b370 tx=0x7f98d408b390 comp rx=0 tx=0).stop 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 -- 172.21.15.5:0/2364727057 shutdown_connections 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.5:6833/1828661194,v1:172.21.15.5:6835/1828661194] conn(0x557b09b55980 0x557b09aab750 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b09ad7dd0 0x557b09b55440 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b09aab370 0x557b09b519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98bb7fe700 0 client.9231 ms_handle_reset on v2:172.21.15.5:6833/1828661194 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 --2- 172.21.15.5:0/2364727057 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x557b09ad07a0 0x557b09b5c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 -- 172.21.15.5:0/2364727057 >> 172.21.15.5:0/2364727057 conn(0x557b09a0b210 msgr2=0x557b09adadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 -- 172.21.15.5:0/2364727057 shutdown_connections 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 1 -- 172.21.15.5:0/2364727057 wait complete. 2024-05-08T23:28:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.233+0000 7f98eed49b80 20 client.9231 trim_cache size 0 max 16384 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:35.239 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 -- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f98c806ac90 msgr2=0x7f98c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f98c806ac90 0x7f98c806d150 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f98cc000f60 tx=0x7f98cc007be0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 -- 172.21.15.5:0/1126889582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 msgr2=0x7f98dc14a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc14a7a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0b5790 tx=0x7f98d4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 -- 172.21.15.5:0/1126889582 shutdown_connections 2024-05-08T23:28:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f98c806ac90 0x7f98c806d150 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98dc15fbb0 0x7f98dc14a7a0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98dc0d0a60 0x7f98dc15c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 --2- 172.21.15.5:0/1126889582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f98dc145040 0x7f98dc150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 -- 172.21.15.5:0/1126889582 >> 172.21.15.5:0/1126889582 conn(0x7f98dc099a50 msgr2=0x7f98dc06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 -- 172.21.15.5:0/1126889582 shutdown_connections 2024-05-08T23:28:35.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.234+0000 7f98e2b8f700 1 -- 172.21.15.5:0/1126889582 wait complete. 2024-05-08T23:28:35.247 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:35.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 scrub status 2024-05-08T23:28:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.386+0000 7f28eada0700 1 Processor -- start 2024-05-08T23:28:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.386+0000 7f28eada0700 1 -- start start 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28eada0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e40cef00 0x7f28e40b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28eada0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28e40cdde0 0x7f28e40ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28eada0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e40cb830 con 0x7f28e415a520 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28eada0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e40cba80 con 0x7f28e40cdde0 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28eada0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40cbd60 con 0x7f28e40cef00 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28ea59f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28e40cdde0 0x7f28e40ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28ea59f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28e40cdde0 0x7f28e40ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48332/0 (socket says 172.21.15.5:48332) 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28ea59f700 1 -- 172.21.15.5:0/3408022449 learned_addr learned my addr 172.21.15.5:0/3408022449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28e959d700 1 --2- 172.21.15.5:0/3408022449 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e40cef00 0x7f28e40b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28ea59f700 1 -- 172.21.15.5:0/3408022449 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e40cef00 msgr2=0x7f28e40b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28ea59f700 1 --2- 172.21.15.5:0/3408022449 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e40cef00 0x7f28e40b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28ea59f700 1 -- 172.21.15.5:0/3408022449 >> v1:172.21.15.49:6789/0 conn(0x7f28e415a520 legacy=0x7f28e40ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28ea59f700 1 -- 172.21.15.5:0/3408022449 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e40cc040 con 0x7f28e40cdde0 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.387+0000 7f28ea59f700 1 --2- 172.21.15.5:0/3408022449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e40ad850 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d150 tx=0x7f28dc00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffae858e3708d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28e8d9c700 1 -- 172.21.15.5:0/3408022449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc010070 con 0x7f28e40cdde0 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28e8d9c700 1 -- 172.21.15.5:0/3408022449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc00dd10 con 0x7f28e40cdde0 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28e8d9c700 1 -- 172.21.15.5:0/3408022449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00b310 con 0x7f28e40cdde0 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28eada0700 1 -- 172.21.15.5:0/3408022449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 msgr2=0x7f28e40ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28eada0700 1 --2- 172.21.15.5:0/3408022449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e40ad850 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d150 tx=0x7f28dc00d460 comp rx=0 tx=0).stop 2024-05-08T23:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28eada0700 1 -- 172.21.15.5:0/3408022449 shutdown_connections 2024-05-08T23:28:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28eada0700 1 --2- 172.21.15.5:0/3408022449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e40ad850 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28eada0700 1 --2- 172.21.15.5:0/3408022449 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e40cef00 0x7f28e40b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28eada0700 1 -- 172.21.15.5:0/3408022449 >> 172.21.15.5:0/3408022449 conn(0x7f28e40b1de0 msgr2=0x7f28e40b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28eada0700 1 -- 172.21.15.5:0/3408022449 shutdown_connections 2024-05-08T23:28:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.388+0000 7f28eada0700 1 -- 172.21.15.5:0/3408022449 wait complete. 2024-05-08T23:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.390+0000 7f28eada0700 1 Processor -- start 2024-05-08T23:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.390+0000 7f28eada0700 1 -- start start 2024-05-08T23:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28eada0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e4155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28eada0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e40cef00 0x7f28e40c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28eada0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e415a520 0x7f28e409ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28eada0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40b4380 con 0x7f28e40cdde0 2024-05-08T23:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28eada0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40c9a40 con 0x7f28e40cef00 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28eada0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40c9d20 con 0x7f28e415a520 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e4155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e4155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48338/0 (socket says 172.21.15.5:48338) 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 -- 172.21.15.5:0/3440547789 learned_addr learned my addr 172.21.15.5:0/3440547789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e959d700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e40cef00 0x7f28e40c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28ea59f700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e415a520 0x7f28e409ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 -- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e415a520 msgr2=0x7f28e409ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e415a520 0x7f28e409ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 -- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e40cef00 msgr2=0x7f28e40c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e40cef00 0x7f28e40c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 -- 172.21.15.5:0/3440547789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc00bd40 con 0x7f28e40cdde0 2024-05-08T23:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e4155750 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c450 tx=0x7f28e000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28da7fc700 1 -- 172.21.15.5:0/3440547789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e00052d0 con 0x7f28e40cdde0 2024-05-08T23:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28da7fc700 1 -- 172.21.15.5:0/3440547789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e0005430 con 0x7f28e40cdde0 2024-05-08T23:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.391+0000 7f28eada0700 1 -- 172.21.15.5:0/3440547789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e40ca000 con 0x7f28e40cdde0 2024-05-08T23:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.392+0000 7f28da7fc700 1 -- 172.21.15.5:0/3440547789 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e00038c0 con 0x7f28e40cdde0 2024-05-08T23:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.392+0000 7f28eada0700 1 -- 172.21.15.5:0/3440547789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e4155c90 con 0x7f28e40cdde0 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.396+0000 7f28f81bcb80 1 Processor -- start 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.396+0000 7f28f81bcb80 1 -- start start 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28da7fc700 1 -- 172.21.15.5:0/3440547789 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f28e0003a20 con 0x7f28e40cdde0 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b809150 con 0x558d3b8b0560 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b806b10 con 0x558d3b8c9040 2024-05-08T23:28:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d3b802400 con 0x558d3b8dd1d0 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e959d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e959d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48342/0 (socket says 172.21.15.5:48342) 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e959d700 1 -- 172.21.15.5:0/3240484779 learned_addr learned my addr 172.21.15.5:0/3240484779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28da7fc700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f28c806acc0 0x7f28c806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28ea59f700 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e959d700 1 -- 172.21.15.5:0/3240484779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 msgr2=0x558d3b8dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e959d700 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e959d700 1 -- 172.21.15.5:0/3240484779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 msgr2=0x558d3b8f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e959d700 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.397+0000 7f28e959d700 1 -- 172.21.15.5:0/3240484779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d3b6833b0 con 0x558d3b8b0560 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.398+0000 7f28ea59f700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f28c806acc0 0x7f28c806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.398+0000 7f28da7fc700 1 -- 172.21.15.5:0/3440547789 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28e00885b0 con 0x7f28e40cdde0 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.398+0000 7f28e959d700 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8b2d30 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009c30 tx=0x7f28d4009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8d14d4287120d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.398+0000 7f28c77fe700 1 -- 172.21.15.5:0/3240484779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4011c50 con 0x558d3b8b0560 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.398+0000 7f28c77fe700 1 -- 172.21.15.5:0/3240484779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d4011db0 con 0x558d3b8b0560 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.398+0000 7f28c77fe700 1 -- 172.21.15.5:0/3240484779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4004250 con 0x558d3b8b0560 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3240484779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 msgr2=0x558d3b8b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28ea59f700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f28c806acc0 0x7f28c806d180 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d4a0 tx=0x7f28dc013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8b2d30 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009c30 tx=0x7f28d4009f40 comp rx=0 tx=0).stop 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3240484779 shutdown_connections 2024-05-08T23:28:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8b2d30 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3240484779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3240484779 >> 172.21.15.5:0/3240484779 conn(0x558d3b810210 msgr2=0x558d3b8f7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3240484779 shutdown_connections 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3240484779 wait complete. 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 -- start start 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.399+0000 7f28f81bcb80 1 Processor -- start 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.400+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.400+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.400+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8b9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.400+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b79e560 con 0x558d3b8b0560 2024-05-08T23:28:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b79e200 con 0x558d3b8c9040 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d3b6856a0 con 0x558d3b8dd1d0 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28f81bcb80 10 client.0 _ll_register_callbacks cb 0x558d3b7a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8ef240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48344/0 (socket says 172.21.15.5:48344) 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 -- 172.21.15.5:0/2561384283 learned_addr learned my addr 172.21.15.5:0/2561384283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e959d700 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28ea59f700 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 -- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 msgr2=0x558d3b8b9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8b9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 -- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 msgr2=0x558d3b8ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 -- 172.21.15.5:0/2561384283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d4009860 con 0x558d3b8b0560 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8ef240 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f28e000a040 tx=0x7f28e0057120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9af02e83b1f961 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28c4ff9700 1 -- 172.21.15.5:0/2561384283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e005b120 con 0x558d3b8b0560 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28c4ff9700 1 -- 172.21.15.5:0/2561384283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28e00574e0 con 0x558d3b8b0560 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.401+0000 7f28f81bcb80 1 -- 172.21.15.5:0/2561384283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d3b75b270 con 0x558d3b8b0560 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28c4ff9700 1 -- 172.21.15.5:0/2561384283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e00584f0 con 0x558d3b8b0560 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28f81bcb80 10 client.9243 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28f81bcb80 1 -- 172.21.15.5:0/2561384283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x558d3b62fb60 con 0x558d3b8b0560 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28c4ff9700 1 -- 172.21.15.5:0/2561384283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e0084070 con 0x558d3b8b0560 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28c4ff9700 1 -- 172.21.15.5:0/2561384283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f28e005f440 con 0x558d3b8b0560 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28f81bcb80 10 client.9243 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:35.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28f81bcb80 10 client.9243 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28f81bcb80 20 client.9243 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x558d3b8b17d0 0x558d3b8b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28f81bcb80 4 client.9243 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28f81bcb80 1 -- 172.21.15.5:0/2561384283 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d3b8817e0 con 0x558d3b8b17d0 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.402+0000 7f28e959d700 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x558d3b8b17d0 0x558d3b8b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28c4ff9700 10 client.9243 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28e959d700 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x558d3b8b17d0 0x558d3b8b9c40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cef33edc4b7e6602 server_cookie=4ccef7ec1cb2c39a in_seq=0 out_seq=0 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28c4ff9700 1 -- 172.21.15.5:0/2561384283 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d3b8817e0 con 0x558d3b8b17d0 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28c4ff9700 10 client.9243 handle_command_reply: tid=0 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 client.9243 shutdown 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 -- 172.21.15.5:0/2561384283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 msgr2=0x558d3b8ef240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8ef240 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f28e000a040 tx=0x7f28e0057120 comp rx=0 tx=0).stop 2024-05-08T23:28:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 -- 172.21.15.5:0/2561384283 shutdown_connections 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x558d3b8b17d0 0x558d3b8b9c40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28c4ff9700 0 client.9243 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8b0560 0x558d3b8ef240 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/2561384283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.403+0000 7f28f81bcb80 1 -- 172.21.15.5:0/2561384283 >> 172.21.15.5:0/2561384283 conn(0x558d3b810210 msgr2=0x558d3b8af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.404+0000 7f28f81bcb80 1 -- 172.21.15.5:0/2561384283 shutdown_connections 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.404+0000 7f28f81bcb80 1 -- 172.21.15.5:0/2561384283 wait complete. 2024-05-08T23:28:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.404+0000 7f28f81bcb80 20 client.9243 trim_cache size 0 max 16384 2024-05-08T23:28:35.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.422+0000 7f28f81bcb80 1 Processor -- start 2024-05-08T23:28:35.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.422+0000 7f28f81bcb80 1 -- start start 2024-05-08T23:28:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8b0560 0x558d3b8dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8dc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8dd1d0 0x558d3b8f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b8b94a0 con 0x558d3b8dd1d0 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b8f0f70 con 0x558d3b8c9040 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d3b91a370 con 0x558d3b8b0560 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8dd1d0 0x558d3b8f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8dd1d0 0x558d3b8f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48356/0 (socket says 172.21.15.5:48356) 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 -- 172.21.15.5:0/235902430 learned_addr learned my addr 172.21.15.5:0/235902430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8b0560 0x558d3b8dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28e959d700 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8dc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 -- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8b0560 msgr2=0x558d3b8dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8b0560 0x558d3b8dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 -- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 msgr2=0x558d3b8dc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8dc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 -- 172.21.15.5:0/235902430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d3b7c8560 con 0x558d3b8dd1d0 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28e959d700 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8dc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.423+0000 7f28ea59f700 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8dd1d0 0x558d3b8f79a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d7b0 tx=0x7f28dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741e6d03a2960cdd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28c7fff700 1 -- 172.21.15.5:0/235902430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0046d0 con 0x558d3b8dd1d0 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28c7fff700 1 -- 172.21.15.5:0/235902430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc004830 con 0x558d3b8dd1d0 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28c7fff700 1 -- 172.21.15.5:0/235902430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc02b670 con 0x558d3b8dd1d0 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 -- 172.21.15.5:0/235902430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8dd1d0 msgr2=0x558d3b8f79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8dd1d0 0x558d3b8f79a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d7b0 tx=0x7f28dc012000 comp rx=0 tx=0).stop 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 -- 172.21.15.5:0/235902430 shutdown_connections 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8dd1d0 0x558d3b8f79a0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8b0560 0x558d3b8dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/235902430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8c9040 0x558d3b8dc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 -- 172.21.15.5:0/235902430 >> 172.21.15.5:0/235902430 conn(0x558d3b810210 msgr2=0x558d3b8a91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 -- 172.21.15.5:0/235902430 shutdown_connections 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.424+0000 7f28f81bcb80 1 -- 172.21.15.5:0/235902430 wait complete. 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.425+0000 7f28f81bcb80 1 -- start start 2024-05-08T23:28:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.425+0000 7f28f81bcb80 1 Processor -- start 2024-05-08T23:28:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8b0560 0x558d3b8dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8c9040 0x558d3b965a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28f81bcb80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b962760 con 0x558d3b8c9040 2024-05-08T23:28:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3b8dcdb0 con 0x558d3b8b0560 2024-05-08T23:28:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28f81bcb80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d3b9655e0 con 0x558d3b8dd1d0 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28f81bcb80 10 client.0 _ll_register_callbacks cb 0x558d3b7a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28e959d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8c9040 0x558d3b965a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28e959d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8c9040 0x558d3b965a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48366/0 (socket says 172.21.15.5:48366) 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28e959d700 1 -- 172.21.15.5:0/3440795502 learned_addr learned my addr 172.21.15.5:0/3440795502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28e9d9e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8b0560 0x558d3b8dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28ea59f700 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28e959d700 1 -- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 msgr2=0x558d3b8ef5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28e959d700 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8ef5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28e959d700 1 -- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8b0560 msgr2=0x558d3b8dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.426+0000 7f28e959d700 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8b0560 0x558d3b8dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28e959d700 1 -- 172.21.15.5:0/3440795502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e40cc040 con 0x558d3b8c9040 2024-05-08T23:28:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28e959d700 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8c9040 0x558d3b965a70 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009250 tx=0x7f28d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622e720b25ca4522 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28c4ff9700 1 -- 172.21.15.5:0/3440795502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4004450 con 0x558d3b8c9040 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28c4ff9700 1 -- 172.21.15.5:0/3440795502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28d402a350 con 0x558d3b8c9040 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3440795502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28d4011db0 con 0x558d3b8c9040 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28c4ff9700 1 -- 172.21.15.5:0/3440795502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28d4006c60 con 0x558d3b8c9040 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 10 client.9249 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3440795502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x558d3b687cd0 con 0x558d3b8c9040 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28c4ff9700 1 -- 172.21.15.5:0/3440795502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d4011db0 con 0x558d3b8c9040 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28c4ff9700 1 -- 172.21.15.5:0/3440795502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7f28d402a510 con 0x558d3b8c9040 2024-05-08T23:28:35.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 10 client.9249 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 10 client.9249 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 20 client.9249 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x558d3b961ae0 0x558d3b961ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 4 client.9249 mds_command: new command op to 7052 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.427+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3440795502 --> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558d3b7faec0 con 0x558d3b961ae0 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x558d3b961ae0 0x558d3b961ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28e9d9e700 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x558d3b961ae0 0x558d3b961ec0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83ff68165a19ab4c server_cookie=9f2f824ad36a7982 in_seq=0 out_seq=0 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28c4ff9700 10 client.9249 ms_handle_connect on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28c4ff9700 1 -- 172.21.15.5:0/3440795502 <== mds.2 v2:172.21.15.49:6835/873094942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x558d3b7faec0 con 0x558d3b961ae0 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28c4ff9700 10 client.9249 handle_command_reply: tid=0 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 client.9249 shutdown 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3440795502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8c9040 msgr2=0x558d3b965a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8c9040 0x558d3b965a70 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009250 tx=0x7f28d4014000 comp rx=0 tx=0).stop 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3440795502 shutdown_connections 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:6835/873094942,v1:172.21.15.49:6837/873094942] conn(0x558d3b961ae0 0x558d3b961ec0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x558d3b8dd1d0 0x558d3b8ef5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558d3b8b0560 0x558d3b8dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 --2- 172.21.15.5:0/3440795502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d3b8c9040 0x558d3b965a70 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28c4ff9700 0 client.9249 ms_handle_reset on v2:172.21.15.49:6835/873094942 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.428+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3440795502 >> 172.21.15.5:0/3440795502 conn(0x558d3b810210 msgr2=0x558d3b8b8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3440795502 shutdown_connections 2024-05-08T23:28:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28f81bcb80 1 -- 172.21.15.5:0/3440795502 wait complete. 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28f81bcb80 20 client.9249 trim_cache size 0 max 16384 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:35.437 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:35.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:35.438 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 -- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f28c806acc0 msgr2=0x7f28c806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f28c806acc0 0x7f28c806d180 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d4a0 tx=0x7f28dc013000 comp rx=0 tx=0).stop 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 -- 172.21.15.5:0/3440547789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 msgr2=0x7f28e4155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e4155750 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c450 tx=0x7f28e000c760 comp rx=0 tx=0).stop 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 -- 172.21.15.5:0/3440547789 shutdown_connections 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f28c806acc0 0x7f28c806d180 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f28e415a520 0x7f28e409ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e40cdde0 0x7f28e4155750 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 --2- 172.21.15.5:0/3440547789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28e40cef00 0x7f28e40c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 -- 172.21.15.5:0/3440547789 >> 172.21.15.5:0/3440547789 conn(0x7f28e40b1de0 msgr2=0x7f28e40b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 -- 172.21.15.5:0/3440547789 shutdown_connections 2024-05-08T23:28:35.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.429+0000 7f28eada0700 1 -- 172.21.15.5:0/3440547789 wait complete. 2024-05-08T23:28:35.443 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:35.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T23:28:35.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.581+0000 7fd36c4d9700 1 Processor -- start 2024-05-08T23:28:35.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.581+0000 7fd36c4d9700 1 -- start start 2024-05-08T23:28:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.582+0000 7fd36c4d9700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd364161da0 0x7fd36406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.582+0000 7fd36c4d9700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36415ed00 con 0x7fd3640b1b90 2024-05-08T23:28:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.582+0000 7fd36c4d9700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640d10f0 con 0x7fd36415d910 2024-05-08T23:28:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.582+0000 7fd36c4d9700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640cdee0 con 0x7fd364161da0 2024-05-08T23:28:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.582+0000 7fd369a74700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd364161da0 0x7fd36406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.582+0000 7fd36aa76700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd3640b1b90 0x7fd3640b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47102/0 (socket says 172.21.15.5:47102) 2024-05-08T23:28:35.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.582+0000 7fd36aa76700 1 -- 172.21.15.5:0/2875640083 learned_addr learned my addr 172.21.15.5:0/2875640083 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.582+0000 7fd36a275700 1 --1- 172.21.15.5:0/2875640083 >> v1:172.21.15.49:6789/0 conn(0x7fd36415d910 0x7fd36406a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:43016/0 (socket says 172.21.15.5:43016) 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.583+0000 7fd369a74700 1 -- 172.21.15.5:0/2875640083 >> v1:172.21.15.49:6789/0 conn(0x7fd36415d910 legacy=0x7fd36406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.583+0000 7fd369273700 1 -- 172.21.15.5:0/2875640083 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833329561 0 0) 0x7fd3640d10f0 con 0x7fd36415d910 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.583+0000 7fd369a74700 1 -- 172.21.15.5:0/2875640083 >> v1:172.21.15.5:6789/0 conn(0x7fd3640b1b90 legacy=0x7fd3640b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.583+0000 7fd369a74700 1 -- 172.21.15.5:0/2875640083 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364162230 con 0x7fd364161da0 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.583+0000 7fd369a74700 1 --2- 172.21.15.5:0/2875640083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364161da0 0x7fd36406b070 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd358009840 tx=0x7fd358009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80681adbe3d86534 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.583+0000 7fd369273700 1 -- 172.21.15.5:0/2875640083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358004230 con 0x7fd364161da0 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.583+0000 7fd369273700 1 -- 172.21.15.5:0/2875640083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd358004390 con 0x7fd364161da0 2024-05-08T23:28:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.583+0000 7fd369273700 1 -- 172.21.15.5:0/2875640083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3580036a0 con 0x7fd364161da0 2024-05-08T23:28:35.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.584+0000 7fd36c4d9700 1 -- 172.21.15.5:0/2875640083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364161da0 msgr2=0x7fd36406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.584+0000 7fd36c4d9700 1 --2- 172.21.15.5:0/2875640083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364161da0 0x7fd36406b070 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd358009840 tx=0x7fd358009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:35.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.584+0000 7fd36c4d9700 1 -- 172.21.15.5:0/2875640083 shutdown_connections 2024-05-08T23:28:35.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.584+0000 7fd36c4d9700 1 --2- 172.21.15.5:0/2875640083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364161da0 0x7fd36406b070 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.584+0000 7fd36c4d9700 1 -- 172.21.15.5:0/2875640083 >> 172.21.15.5:0/2875640083 conn(0x7fd3640d03a0 msgr2=0x7fd3640d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.584+0000 7fd36c4d9700 1 -- 172.21.15.5:0/2875640083 shutdown_connections 2024-05-08T23:28:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.584+0000 7fd36c4d9700 1 -- 172.21.15.5:0/2875640083 wait complete. 2024-05-08T23:28:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.585+0000 7fd36c4d9700 1 Processor -- start 2024-05-08T23:28:35.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.585+0000 7fd36c4d9700 1 -- start start 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36c4d9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3640b1b90 0x7fd3641f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36c4d9700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd36415d910 0x7fd36406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36c4d9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd364161da0 0x7fd36406f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36c4d9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641f4130 con 0x7fd3640b1b90 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36c4d9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641f35a0 con 0x7fd364161da0 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36c4d9700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641f45c0 con 0x7fd36415d910 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3640b1b90 0x7fd3641f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3640b1b90 0x7fd3641f74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48388/0 (socket says 172.21.15.5:48388) 2024-05-08T23:28:35.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 -- 172.21.15.5:0/210640790 learned_addr learned my addr 172.21.15.5:0/210640790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36aa76700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd364161da0 0x7fd36406f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd369a74700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd36415d910 0x7fd36406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 -- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd36415d910 msgr2=0x7fd36406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd36415d910 0x7fd36406eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 -- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd364161da0 msgr2=0x7fd36406f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd364161da0 0x7fd36406f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 -- 172.21.15.5:0/210640790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580094a0 con 0x7fd3640b1b90 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36aa76700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd364161da0 0x7fd36406f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd369a74700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd36415d910 0x7fd36406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.586+0000 7fd36a275700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3640b1b90 0x7fd3641f74d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd36000c520 tx=0x7fd36000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.587+0000 7fd356ffd700 1 -- 172.21.15.5:0/210640790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360004c80 con 0x7fd3640b1b90 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.587+0000 7fd356ffd700 1 -- 172.21.15.5:0/210640790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360004de0 con 0x7fd3640b1b90 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.587+0000 7fd36c4d9700 1 -- 172.21.15.5:0/210640790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640cc090 con 0x7fd3640b1b90 2024-05-08T23:28:35.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.587+0000 7fd356ffd700 1 -- 172.21.15.5:0/210640790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360007310 con 0x7fd3640b1b90 2024-05-08T23:28:35.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.587+0000 7fd36c4d9700 1 -- 172.21.15.5:0/210640790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3641f3ae0 con 0x7fd3640b1b90 2024-05-08T23:28:35.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.588+0000 7fd356ffd700 1 -- 172.21.15.5:0/210640790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd360007470 con 0x7fd3640b1b90 2024-05-08T23:28:35.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.589+0000 7fd356ffd700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd34806ac90 0x7fd34806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.589+0000 7fd356ffd700 1 -- 172.21.15.5:0/210640790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd360089080 con 0x7fd3640b1b90 2024-05-08T23:28:35.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.589+0000 7fd369a74700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd34806ac90 0x7fd34806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.590+0000 7fd369a74700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd34806ac90 0x7fd34806d150 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3641f11e0 tx=0x7fd358008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.591+0000 7fd378693b80 1 Processor -- start 2024-05-08T23:28:35.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.591+0000 7fd378693b80 1 -- start start 2024-05-08T23:28:35.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd378693b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be0f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd378693b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 0x56104be140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd378693b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 0x56104be3e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd378693b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104bd6a150 con 0x56104be367a0 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd378693b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104bd67b10 con 0x56104be3ddd0 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd378693b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56104bd63400 con 0x56104be11370 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be0f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be0f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48394/0 (socket says 172.21.15.5:48394) 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 -- 172.21.15.5:0/2554362899 learned_addr learned my addr 172.21.15.5:0/2554362899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd369a74700 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 0x56104be140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36aa76700 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 0x56104be3e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 -- 172.21.15.5:0/2554362899 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 msgr2=0x56104be140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 0x56104be140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 -- 172.21.15.5:0/2554362899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 msgr2=0x56104be3e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 0x56104be3e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 -- 172.21.15.5:0/2554362899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56104bbe43b0 con 0x56104be367a0 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd36a275700 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be0f110 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd360089050 tx=0x7fd360054260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c7444f778f1995 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.592+0000 7fd347fff700 1 -- 172.21.15.5:0/2554362899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36005bd50 con 0x56104be367a0 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd347fff700 1 -- 172.21.15.5:0/2554362899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360060310 con 0x56104be367a0 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd347fff700 1 -- 172.21.15.5:0/2554362899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360058180 con 0x56104be367a0 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 -- 172.21.15.5:0/2554362899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 msgr2=0x56104be0f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be0f110 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd360089050 tx=0x7fd360054260 comp rx=0 tx=0).stop 2024-05-08T23:28:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 -- 172.21.15.5:0/2554362899 shutdown_connections 2024-05-08T23:28:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 0x56104be3e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be0f110 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 --2- 172.21.15.5:0/2554362899 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 0x56104be140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 -- 172.21.15.5:0/2554362899 >> 172.21.15.5:0/2554362899 conn(0x56104bd71210 msgr2=0x56104be58dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 -- 172.21.15.5:0/2554362899 shutdown_connections 2024-05-08T23:28:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.593+0000 7fd378693b80 1 -- 172.21.15.5:0/2554362899 wait complete. 2024-05-08T23:28:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.594+0000 7fd378693b80 1 -- start start 2024-05-08T23:28:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.594+0000 7fd378693b80 1 Processor -- start 2024-05-08T23:28:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd378693b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be11370 0x56104be359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd378693b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be40220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd378693b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 0x56104be41630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd378693b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104bcff560 con 0x56104be367a0 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd378693b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104bcff200 con 0x56104be11370 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd378693b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56104bbe66a0 con 0x56104be3ddd0 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd378693b80 10 client.0 _ll_register_callbacks cb 0x56104bd08b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd369a74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be40220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd369a74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be40220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48396/0 (socket says 172.21.15.5:48396) 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.595+0000 7fd369a74700 1 -- 172.21.15.5:0/3701101329 learned_addr learned my addr 172.21.15.5:0/3701101329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd36aa76700 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 0x56104be41630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd36a275700 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be11370 0x56104be359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd369a74700 1 -- 172.21.15.5:0/3701101329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 msgr2=0x56104be41630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd369a74700 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 0x56104be41630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd369a74700 1 -- 172.21.15.5:0/3701101329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be11370 msgr2=0x56104be359a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd369a74700 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be11370 0x56104be359a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd369a74700 1 -- 172.21.15.5:0/3701101329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3641f3ae0 con 0x56104be367a0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd369a74700 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be40220 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd3580090b0 tx=0x7fd3580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb14e07242e645b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd3457fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd3457fa700 1 -- 172.21.15.5:0/3701101329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35802b070 con 0x56104be367a0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd378693b80 1 -- 172.21.15.5:0/3701101329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56104bcbc270 con 0x56104be367a0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd3457fa700 1 -- 172.21.15.5:0/3701101329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd358011840 con 0x56104be367a0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd3457fa700 1 -- 172.21.15.5:0/3701101329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd358003e70 con 0x56104be367a0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd378693b80 10 client.9264 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.596+0000 7fd378693b80 1 -- 172.21.15.5:0/3701101329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x56104bb90b60 con 0x56104be367a0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd3457fa700 1 -- 172.21.15.5:0/3701101329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56104bcbc270 con 0x56104be367a0 2024-05-08T23:28:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd3457fa700 1 -- 172.21.15.5:0/3701101329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fd358011a00 con 0x56104be367a0 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd378693b80 10 client.9264 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd378693b80 10 client.9264 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd378693b80 20 client.9264 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd378693b80 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56104be5a800 0x56104be33c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd378693b80 4 client.9264 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd378693b80 1 -- 172.21.15.5:0/3701101329 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56104bde27e0 con 0x56104be5a800 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd36a275700 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56104be5a800 0x56104be33c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd36a275700 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56104be5a800 0x56104be33c60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f472a422d32527ff server_cookie=f3576d37142c82a4 in_seq=0 out_seq=0 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.597+0000 7fd3457fa700 10 client.9264 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd3457fa700 1 -- 172.21.15.5:0/3701101329 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56104bde27e0 con 0x56104be5a800 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd3457fa700 10 client.9264 handle_command_reply: tid=0 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 client.9264 shutdown 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 -- 172.21.15.5:0/3701101329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 msgr2=0x56104be40220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be40220 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd3580090b0 tx=0x7fd3580047c0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 -- 172.21.15.5:0/3701101329 shutdown_connections 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56104be5a800 0x56104be33c60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 0x56104be41630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be11370 0x56104be359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd3457fa700 0 client.9264 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 --2- 172.21.15.5:0/3701101329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104be40220 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 -- 172.21.15.5:0/3701101329 >> 172.21.15.5:0/3701101329 conn(0x56104bd71210 msgr2=0x56104be3e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 -- 172.21.15.5:0/3701101329 shutdown_connections 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 1 -- 172.21.15.5:0/3701101329 wait complete. 2024-05-08T23:28:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.598+0000 7fd378693b80 20 client.9264 trim_cache size 0 max 16384 2024-05-08T23:28:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.617+0000 7fd378693b80 1 Processor -- start 2024-05-08T23:28:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.617+0000 7fd378693b80 1 -- start start 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd378693b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be11370 0x56104be33e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd378693b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be367a0 0x56104be3e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd378693b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 0x56104be3e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd378693b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104be14760 con 0x56104be11370 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd378693b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104be420e0 con 0x56104be367a0 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd378693b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56104be736b0 con 0x56104be3ddd0 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be11370 0x56104be33e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be11370 0x56104be33e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48412/0 (socket says 172.21.15.5:48412) 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 -- 172.21.15.5:0/1512832141 learned_addr learned my addr 172.21.15.5:0/1512832141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 -- 172.21.15.5:0/1512832141 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 msgr2=0x56104be3e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36aa76700 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 0x56104be3e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd369a74700 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be367a0 0x56104be3e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 0x56104be3e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 -- 172.21.15.5:0/1512832141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be367a0 msgr2=0x56104be3e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be367a0 0x56104be3e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 -- 172.21.15.5:0/1512832141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56104be59d90 con 0x56104be11370 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd36a275700 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be11370 0x56104be33e20 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fd360087600 tx=0x7fd360054390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab13b290881c3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd354ff9700 1 -- 172.21.15.5:0/1512832141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36005bd50 con 0x56104be11370 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd354ff9700 1 -- 172.21.15.5:0/1512832141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360056060 con 0x56104be11370 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.618+0000 7fd354ff9700 1 -- 172.21.15.5:0/1512832141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600870c0 con 0x56104be11370 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 -- 172.21.15.5:0/1512832141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be11370 msgr2=0x56104be33e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be11370 0x56104be33e20 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fd360087600 tx=0x7fd360054390 comp rx=0 tx=0).stop 2024-05-08T23:28:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 -- 172.21.15.5:0/1512832141 shutdown_connections 2024-05-08T23:28:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be3ddd0 0x56104be3e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be11370 0x56104be33e20 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 --2- 172.21.15.5:0/1512832141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be367a0 0x56104be3e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 -- 172.21.15.5:0/1512832141 >> 172.21.15.5:0/1512832141 conn(0x56104bd71210 msgr2=0x56104be0a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 -- 172.21.15.5:0/1512832141 shutdown_connections 2024-05-08T23:28:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.619+0000 7fd378693b80 1 -- 172.21.15.5:0/1512832141 wait complete. 2024-05-08T23:28:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.620+0000 7fd378693b80 1 -- start start 2024-05-08T23:28:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.620+0000 7fd378693b80 1 Processor -- start 2024-05-08T23:28:35.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd378693b80 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 0x56104be3aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd378693b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104bea83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd378693b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 0x56104be0d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd378693b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104bea8900 con 0x56104be367a0 2024-05-08T23:28:35.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd378693b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104bea8d30 con 0x56104be3ddd0 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd378693b80 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x56104be404d0 con 0x56104be11370 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd378693b80 10 client.0 _ll_register_callbacks cb 0x7fd358004290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd369a74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104bea83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd369a74700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104bea83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48422/0 (socket says 172.21.15.5:48422) 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd369a74700 1 -- 172.21.15.5:0/129903370 learned_addr learned my addr 172.21.15.5:0/129903370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd36aa76700 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 0x56104be0d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd36a275700 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 0x56104be3aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd369a74700 1 -- 172.21.15.5:0/129903370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 msgr2=0x56104be3aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd369a74700 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 0x56104be3aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd369a74700 1 -- 172.21.15.5:0/129903370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 msgr2=0x56104be0d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd369a74700 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 0x56104be0d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.621+0000 7fd369a74700 1 -- 172.21.15.5:0/129903370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56104bbe43b0 con 0x56104be367a0 2024-05-08T23:28:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd369a74700 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104bea83c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fd358000c00 tx=0x7fd358009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29c28d28a0a48bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd3457fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd3457fa700 1 -- 172.21.15.5:0/129903370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358026660 con 0x56104be367a0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd3457fa700 1 -- 172.21.15.5:0/129903370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3580267c0 con 0x56104be367a0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 1 -- 172.21.15.5:0/129903370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd360060310 con 0x56104be367a0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd3457fa700 1 -- 172.21.15.5:0/129903370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd358026980 con 0x56104be367a0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 10 client.9270 fetch_fsmap learned FSMap version 109 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 1 -- 172.21.15.5:0/129903370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x56104bd29560 con 0x56104be367a0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd3457fa700 1 -- 172.21.15.5:0/129903370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360060310 con 0x56104be367a0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd3457fa700 1 -- 172.21.15.5:0/129903370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 109) v1 ==== 3154+0+0 (secure 0 0 0) 0x7fd358026ae0 con 0x56104be367a0 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 10 client.9270 fetch_fsmap finished waiting for FSMap version 109 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 10 client.9270 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 20 client.9270 populate_metadata read hostname 'smithi005' 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56104be0da20 0x56104be34c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 4 client.9270 mds_command: new command op to 8592 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd378693b80 1 -- 172.21.15.5:0/129903370 --> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56104bd5bec0 con 0x56104be0da20 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.622+0000 7fd36a275700 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56104be0da20 0x56104be34c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd3457fa700 10 client.9270 ms_handle_connect on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd36a275700 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56104be0da20 0x56104be34c10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef438dfbbb3284c2 server_cookie=809e7fcb49382127 in_seq=0 out_seq=0 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd3457fa700 1 -- 172.21.15.5:0/129903370 <== mds.0 v2:172.21.15.5:6832/1510899872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x56104bd5bec0 con 0x56104be0da20 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd3457fa700 10 client.9270 handle_command_reply: tid=0 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 client.9270 shutdown 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 -- 172.21.15.5:0/129903370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 msgr2=0x56104bea83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104bea83c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fd358000c00 tx=0x7fd358009210 comp rx=0 tx=0).stop 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 -- 172.21.15.5:0/129903370 shutdown_connections 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.5:6832/1510899872,v1:172.21.15.5:6834/1510899872] conn(0x56104be0da20 0x56104be34c10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56104be3ddd0 0x56104be0d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x56104be11370 0x56104be3aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 --2- 172.21.15.5:0/129903370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104be367a0 0x56104bea83c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd3457fa700 0 client.9270 ms_handle_reset on v2:172.21.15.5:6832/1510899872 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 -- 172.21.15.5:0/129903370 >> 172.21.15.5:0/129903370 conn(0x56104bd71210 msgr2=0x56104be7be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 -- 172.21.15.5:0/129903370 shutdown_connections 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 1 -- 172.21.15.5:0/129903370 wait complete. 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.623+0000 7fd378693b80 20 client.9270 trim_cache size 0 max 16384 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-08T23:28:35.630 INFO:teuthology.orchestra.run.smithi005.stdout: "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0": { 2024-05-08T23:28:35.631 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/d1/d2/d3", 2024-05-08T23:28:35.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bdad5635-3e51-4a7f-8eaa-da82bd9ce8c0", 2024-05-08T23:28:35.631 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-08T23:28:35.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:35.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-08T23:28:35.631 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 -- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd34806ac90 msgr2=0x7fd34806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd34806ac90 0x7fd34806d150 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3641f11e0 tx=0x7fd358008030 comp rx=0 tx=0).stop 2024-05-08T23:28:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 -- 172.21.15.5:0/210640790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3640b1b90 msgr2=0x7fd3641f74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3640b1b90 0x7fd3641f74d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd36000c520 tx=0x7fd36000c8e0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 -- 172.21.15.5:0/210640790 shutdown_connections 2024-05-08T23:28:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd34806ac90 0x7fd34806d150 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd364161da0 0x7fd36406f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3640b1b90 0x7fd3641f74d0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 --2- 172.21.15.5:0/210640790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd36415d910 0x7fd36406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 -- 172.21.15.5:0/210640790 >> 172.21.15.5:0/210640790 conn(0x7fd3640d03a0 msgr2=0x7fd3640ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 -- 172.21.15.5:0/210640790 shutdown_connections 2024-05-08T23:28:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.624+0000 7fd36c4d9700 1 -- 172.21.15.5:0/210640790 wait complete. 2024-05-08T23:28:35.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:35.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:35.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.773+0000 7f8a88085700 1 Processor -- start 2024-05-08T23:28:35.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.773+0000 7f8a88085700 1 -- start start 2024-05-08T23:28:35.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.774+0000 7f8a88085700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006d470 0x7f8a8006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.774+0000 7f8a88085700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a800af650 con 0x7f8a8006c080 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.774+0000 7f8a88085700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8015c6d0 con 0x7f8a8015b710 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.774+0000 7f8a88085700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8006ed10 con 0x7f8a8006d470 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a85620700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8a8006c080 0x7f8a800cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47114/0 (socket says 172.21.15.5:47114) 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a85620700 1 -- 172.21.15.5:0/2804047770 learned_addr learned my addr 172.21.15.5:0/2804047770 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a86622700 1 --2- 172.21.15.5:0/2804047770 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006d470 0x7f8a8006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a84e1f700 1 -- 172.21.15.5:0/2804047770 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967727877 0 0) 0x7f8a800af650 con 0x7f8a8006c080 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a84e1f700 1 -- 172.21.15.5:0/2804047770 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a6c0035b0 con 0x7f8a8006c080 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a84e1f700 1 -- 172.21.15.5:0/2804047770 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497052161 0 0) 0x7f8a8015c6d0 con 0x7f8a8015b710 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a84e1f700 1 -- 172.21.15.5:0/2804047770 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a800af650 con 0x7f8a8015b710 2024-05-08T23:28:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a86622700 1 -- 172.21.15.5:0/2804047770 >> v1:172.21.15.49:6789/0 conn(0x7f8a8015b710 legacy=0x7f8a8006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a86622700 1 -- 172.21.15.5:0/2804047770 >> v1:172.21.15.5:6789/0 conn(0x7f8a8006c080 legacy=0x7f8a800cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a86622700 1 -- 172.21.15.5:0/2804047770 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8015baf0 con 0x7f8a8006d470 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.775+0000 7f8a86622700 1 --2- 172.21.15.5:0/2804047770 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006d470 0x7f8a8006e7d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009dc0 tx=0x7f8a7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d42980149097712 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a84e1f700 1 -- 172.21.15.5:0/2804047770 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800e070 con 0x7f8a8006d470 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a84e1f700 1 -- 172.21.15.5:0/2804047770 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800bb20 con 0x7f8a8006d470 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a84e1f700 1 -- 172.21.15.5:0/2804047770 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78012430 con 0x7f8a8006d470 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a88085700 1 -- 172.21.15.5:0/2804047770 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006d470 msgr2=0x7f8a8006e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a88085700 1 --2- 172.21.15.5:0/2804047770 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006d470 0x7f8a8006e7d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009dc0 tx=0x7f8a7800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a88085700 1 -- 172.21.15.5:0/2804047770 shutdown_connections 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a88085700 1 --2- 172.21.15.5:0/2804047770 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006d470 0x7f8a8006e7d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a88085700 1 -- 172.21.15.5:0/2804047770 >> 172.21.15.5:0/2804047770 conn(0x7f8a800713e0 msgr2=0x7f8a800717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a88085700 1 -- 172.21.15.5:0/2804047770 shutdown_connections 2024-05-08T23:28:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.776+0000 7f8a88085700 1 -- 172.21.15.5:0/2804047770 wait complete. 2024-05-08T23:28:35.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.778+0000 7f8a88085700 1 Processor -- start 2024-05-08T23:28:35.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.778+0000 7f8a88085700 1 -- start start 2024-05-08T23:28:35.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.778+0000 7f8a88085700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006c080 0x7f8a801453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.778+0000 7f8a88085700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a8006d470 0x7f8a801f1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.778+0000 7f8a88085700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a8015b710 0x7f8a800ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.778+0000 7f8a88085700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80145930 con 0x7f8a8015b710 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.778+0000 7f8a88085700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80151f70 con 0x7f8a8006d470 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.778+0000 7f8a88085700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80152250 con 0x7f8a8006c080 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a8015b710 0x7f8a800ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a8015b710 0x7f8a800ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48424/0 (socket says 172.21.15.5:48424) 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 -- 172.21.15.5:0/788200788 learned_addr learned my addr 172.21.15.5:0/788200788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a85620700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a8006d470 0x7f8a801f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a85e21700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006c080 0x7f8a801453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 -- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006c080 msgr2=0x7f8a801453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006c080 0x7f8a801453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 -- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a8006d470 msgr2=0x7f8a801f1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a8006d470 0x7f8a801f1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 -- 172.21.15.5:0/788200788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009a70 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a85620700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a8006d470 0x7f8a801f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a86622700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a8015b710 0x7f8a800ceb30 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800b370 tx=0x7f8a78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a727fc700 1 -- 172.21.15.5:0/788200788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7801b810 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a727fc700 1 -- 172.21.15.5:0/788200788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a78020070 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a80152530 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.779+0000 7f8a727fc700 1 -- 172.21.15.5:0/788200788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7801bbf0 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.780+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a80156c70 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.780+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a800cf9e0 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.781+0000 7f8a727fc700 1 -- 172.21.15.5:0/788200788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8a7801bd50 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.781+0000 7f8a727fc700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a6406ac90 0x7f8a6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.781+0000 7f8a727fc700 1 -- 172.21.15.5:0/788200788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a78022070 con 0x7f8a8015b710 2024-05-08T23:28:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.782+0000 7f8a85e21700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a6406ac90 0x7f8a6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.782+0000 7f8a85e21700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a6406ac90 0x7f8a6406d150 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8a74007fd0 tx=0x7f8a74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:35.786+0000 7f8a727fc700 1 -- 172.21.15.5:0/788200788 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8a78056d90 con 0x7f8a8015b710 2024-05-08T23:28:36.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.048+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a80152a50 con 0x7f8a8015b710 2024-05-08T23:28:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.049+0000 7f8a727fc700 1 -- 172.21.15.5:0/788200788 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7f8a80152a50 con 0x7f8a8015b710 2024-05-08T23:28:36.049 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:36.049 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a6406ac90 msgr2=0x7f8a6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a6406ac90 0x7f8a6406d150 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8a74007fd0 tx=0x7f8a74009040 comp rx=0 tx=0).stop 2024-05-08T23:28:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a8015b710 msgr2=0x7f8a800ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a8015b710 0x7f8a800ceb30 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800b370 tx=0x7f8a78004ad0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 shutdown_connections 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f8a6406ac90 0x7f8a6406d150 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a8015b710 0x7f8a800ceb30 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8a8006c080 0x7f8a801453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 --2- 172.21.15.5:0/788200788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a8006d470 0x7f8a801f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 >> 172.21.15.5:0/788200788 conn(0x7f8a800713e0 msgr2=0x7f8a800afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 shutdown_connections 2024-05-08T23:28:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.051+0000 7f8a88085700 1 -- 172.21.15.5:0/788200788 wait complete. 2024-05-08T23:28:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:36.066 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'b'] 2024-05-08T23:28:36.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:28:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.200+0000 7f36e37d4700 1 Processor -- start 2024-05-08T23:28:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.201+0000 7f36e37d4700 1 -- start start 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.201+0000 7f36e37d4700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f36dc0b2480 0x7f36dc09a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.201+0000 7f36e37d4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36dc0cbbc0 0x7f36dc0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.201+0000 7f36e37d4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36dc0d0840 con 0x7f36dc161030 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.201+0000 7f36e37d4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36dc06ccc0 con 0x7f36dc0cbbc0 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.201+0000 7f36e37d4700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36dc0d0660 con 0x7f36dc0b2480 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e1d71700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36dc0cbbc0 0x7f36dc0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e1570700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f36dc161030 0x7f36dc099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47120/0 (socket says 172.21.15.5:47120) 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e1570700 1 -- 172.21.15.5:0/1918099575 learned_addr learned my addr 172.21.15.5:0/1918099575 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e0d6f700 1 --2- 172.21.15.5:0/1918099575 >> v2:172.21.15.49:3300/0 conn(0x7f36dc0b2480 0x7f36dc09a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e1d71700 1 -- 172.21.15.5:0/1918099575 >> v2:172.21.15.49:3300/0 conn(0x7f36dc0b2480 msgr2=0x7f36dc09a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e1d71700 1 --2- 172.21.15.5:0/1918099575 >> v2:172.21.15.49:3300/0 conn(0x7f36dc0b2480 0x7f36dc09a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e1d71700 1 -- 172.21.15.5:0/1918099575 >> v1:172.21.15.5:6789/0 conn(0x7f36dc161030 legacy=0x7f36dc099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e1d71700 1 -- 172.21.15.5:0/1918099575 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc06c600 con 0x7f36dc0cbbc0 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e0d6f700 1 --2- 172.21.15.5:0/1918099575 >> v2:172.21.15.49:3300/0 conn(0x7f36dc0b2480 0x7f36dc09a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36d3fff700 1 -- 172.21.15.5:0/1918099575 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47112018 0 0) 0x7f36dc0d0840 con 0x7f36dc161030 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36e1d71700 1 --2- 172.21.15.5:0/1918099575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0cbbc0 0x7f36dc0cc010 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0b2430 tx=0x7f36d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73080be05e690ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36d3fff700 1 -- 172.21.15.5:0/1918099575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d400e070 con 0x7f36dc0cbbc0 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36d3fff700 1 -- 172.21.15.5:0/1918099575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d400bbd0 con 0x7f36dc0cbbc0 2024-05-08T23:28:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.202+0000 7f36d3fff700 1 -- 172.21.15.5:0/1918099575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4012430 con 0x7f36dc0cbbc0 2024-05-08T23:28:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.203+0000 7f36e37d4700 1 -- 172.21.15.5:0/1918099575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0cbbc0 msgr2=0x7f36dc0cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.203+0000 7f36e37d4700 1 --2- 172.21.15.5:0/1918099575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0cbbc0 0x7f36dc0cc010 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0b2430 tx=0x7f36d400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.203+0000 7f36e37d4700 1 -- 172.21.15.5:0/1918099575 shutdown_connections 2024-05-08T23:28:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.203+0000 7f36e37d4700 1 --2- 172.21.15.5:0/1918099575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0cbbc0 0x7f36dc0cc010 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.203+0000 7f36e37d4700 1 --2- 172.21.15.5:0/1918099575 >> v2:172.21.15.49:3300/0 conn(0x7f36dc0b2480 0x7f36dc09a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.203+0000 7f36e37d4700 1 -- 172.21.15.5:0/1918099575 >> 172.21.15.5:0/1918099575 conn(0x7f36dc0b2f20 msgr2=0x7f36dc0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.203+0000 7f36e37d4700 1 -- 172.21.15.5:0/1918099575 shutdown_connections 2024-05-08T23:28:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.203+0000 7f36e37d4700 1 -- 172.21.15.5:0/1918099575 wait complete. 2024-05-08T23:28:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.204+0000 7f36e37d4700 1 Processor -- start 2024-05-08T23:28:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e37d4700 1 -- start start 2024-05-08T23:28:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e37d4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0b2480 0x7f36dc1f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e37d4700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36dc0cbbc0 0x7f36dc0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e37d4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36dc161030 0x7f36dc159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e37d4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc0c9540 con 0x7f36dc0b2480 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e37d4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc0c5a40 con 0x7f36dc161030 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e37d4700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc0c8a40 con 0x7f36dc0cbbc0 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e1570700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0b2480 0x7f36dc1f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e1570700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0b2480 0x7f36dc1f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48440/0 (socket says 172.21.15.5:48440) 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.205+0000 7f36e1570700 1 -- 172.21.15.5:0/2329539261 learned_addr learned my addr 172.21.15.5:0/2329539261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e0d6f700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36dc0cbbc0 0x7f36dc0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e1d71700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36dc161030 0x7f36dc159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e1570700 1 -- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36dc0cbbc0 msgr2=0x7f36dc0c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e1570700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36dc0cbbc0 0x7f36dc0c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e1570700 1 -- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36dc161030 msgr2=0x7f36dc159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e1570700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36dc161030 0x7f36dc159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e1570700 1 -- 172.21.15.5:0/2329539261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4009a70 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e1570700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0b2480 0x7f36dc1f3aa0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f36d80011e0 tx=0x7f36d800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36d1ffb700 1 -- 172.21.15.5:0/2329539261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004c80 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36d1ffb700 1 -- 172.21.15.5:0/2329539261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d8004e00 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc0c8ca0 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36d1ffb700 1 -- 172.21.15.5:0/2329539261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8007310 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.206+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc0c9260 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.207+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc0c5810 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.208+0000 7f36d1ffb700 1 -- 172.21.15.5:0/2329539261 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f36d8007470 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.208+0000 7f36d1ffb700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36c006ac90 0x7f36c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.208+0000 7f36d1ffb700 1 -- 172.21.15.5:0/2329539261 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36d8088e90 con 0x7f36dc0b2480 2024-05-08T23:28:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.208+0000 7f36e0d6f700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36c006ac90 0x7f36c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.209+0000 7f36e0d6f700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36c006ac90 0x7f36c006d150 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0cb620 tx=0x7f36cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.213+0000 7f36d1ffb700 1 -- 172.21.15.5:0/2329539261 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f36dc0c5810 con 0x7f36dc0b2480 2024-05-08T23:28:36.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.525+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f36dc0c5810 con 0x7f36dc0b2480 2024-05-08T23:28:36.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.526+0000 7f36d1ffb700 1 -- 172.21.15.5:0/2329539261 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1966 (secure 0 0 0) 0x7f36dc0c5810 con 0x7f36dc0b2480 2024-05-08T23:28:36.526 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:36.526 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":471,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700348,"bytes_used":265920512,"bytes_avail":772828192768,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6653,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":109,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7052},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":7058},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8592}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-05-08T23:28:35.405031+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:28:36.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36c006ac90 msgr2=0x7f36c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:36.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36c006ac90 0x7f36c006d150 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0cb620 tx=0x7f36cc008040 comp rx=0 tx=0).stop 2024-05-08T23:28:36.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0b2480 msgr2=0x7f36dc1f3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:36.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0b2480 0x7f36dc1f3aa0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f36d80011e0 tx=0x7f36d800ca30 comp rx=0 tx=0).stop 2024-05-08T23:28:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 shutdown_connections 2024-05-08T23:28:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f36c006ac90 0x7f36c006d150 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36dc161030 0x7f36dc159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36dc0b2480 0x7f36dc1f3aa0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 --2- 172.21.15.5:0/2329539261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f36dc0cbbc0 0x7f36dc0c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 >> 172.21.15.5:0/2329539261 conn(0x7f36dc0b2f20 msgr2=0x7f36dc1600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 shutdown_connections 2024-05-08T23:28:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:36.528+0000 7f36e37d4700 1 -- 172.21.15.5:0/2329539261 wait complete. 2024-05-08T23:28:36.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:28:37.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.687+0000 7fcd15642700 1 Processor -- start 2024-05-08T23:28:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.687+0000 7fcd15642700 1 -- start start 2024-05-08T23:28:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd15642700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcd1009a0a0 0x7fcd1015e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd15642700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd10165400 0x7fcd101677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd15642700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1006c820 con 0x7fcd10099cc0 2024-05-08T23:28:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd15642700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1006bc10 con 0x7fcd1009a0a0 2024-05-08T23:28:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd15642700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1006bef0 con 0x7fcd10165400 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd0effd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcd10099cc0 0x7fcd100d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57922/0 (socket says 172.21.15.5:57922) 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd0effd700 1 -- 172.21.15.5:0/1533151553 learned_addr learned my addr 172.21.15.5:0/1533151553 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd0f7fe700 1 --2- 172.21.15.5:0/1533151553 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd10165400 0x7fcd101677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd0e7fc700 1 --2- 172.21.15.5:0/1533151553 >> v2:172.21.15.49:3300/0 conn(0x7fcd1009a0a0 0x7fcd1015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd0dffb700 1 -- 172.21.15.5:0/1533151553 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029983623 0 0) 0x7fcd1006c820 con 0x7fcd10099cc0 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.688+0000 7fcd0dffb700 1 -- 172.21.15.5:0/1533151553 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccf80035b0 con 0x7fcd10099cc0 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0dffb700 1 -- 172.21.15.5:0/1533151553 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 455211518 0 0) 0x7fccf80035b0 con 0x7fcd10099cc0 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0f7fe700 1 -- 172.21.15.5:0/1533151553 >> v2:172.21.15.49:3300/0 conn(0x7fcd1009a0a0 msgr2=0x7fcd1015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0f7fe700 1 --2- 172.21.15.5:0/1533151553 >> v2:172.21.15.49:3300/0 conn(0x7fcd1009a0a0 0x7fcd1015e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0f7fe700 1 -- 172.21.15.5:0/1533151553 >> v1:172.21.15.5:6789/0 conn(0x7fcd10099cc0 legacy=0x7fcd100d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0f7fe700 1 -- 172.21.15.5:0/1533151553 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1006c1d0 con 0x7fcd10165400 2024-05-08T23:28:37.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0f7fe700 1 --2- 172.21.15.5:0/1533151553 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd10165400 0x7fcd101677f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009d90 tx=0x7fcd0800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b274d89f60dbae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:37.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0dffb700 1 -- 172.21.15.5:0/1533151553 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800e070 con 0x7fcd10165400 2024-05-08T23:28:37.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0dffb700 1 -- 172.21.15.5:0/1533151553 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08003a20 con 0x7fcd10165400 2024-05-08T23:28:37.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.689+0000 7fcd0dffb700 1 -- 172.21.15.5:0/1533151553 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00001670 con 0x7fcd10165400 2024-05-08T23:28:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.690+0000 7fcd15642700 1 -- 172.21.15.5:0/1533151553 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd10165400 msgr2=0x7fcd101677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.690+0000 7fcd15642700 1 --2- 172.21.15.5:0/1533151553 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd10165400 0x7fcd101677f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009d90 tx=0x7fcd0800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.690+0000 7fcd15642700 1 -- 172.21.15.5:0/1533151553 shutdown_connections 2024-05-08T23:28:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.690+0000 7fcd15642700 1 --2- 172.21.15.5:0/1533151553 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd10165400 0x7fcd101677f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.690+0000 7fcd15642700 1 --2- 172.21.15.5:0/1533151553 >> v2:172.21.15.49:3300/0 conn(0x7fcd1009a0a0 0x7fcd1015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.690+0000 7fcd15642700 1 -- 172.21.15.5:0/1533151553 >> 172.21.15.5:0/1533151553 conn(0x7fcd100b2f80 msgr2=0x7fcd100b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.690+0000 7fcd15642700 1 -- 172.21.15.5:0/1533151553 shutdown_connections 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.690+0000 7fcd15642700 1 -- 172.21.15.5:0/1533151553 wait complete. 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.691+0000 7fcd15642700 1 Processor -- start 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.691+0000 7fcd15642700 1 -- start start 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd15642700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd10099cc0 0x7fcd101548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd15642700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd1009a0a0 0x7fcd100b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd15642700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd10165400 0x7fcd1015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd15642700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100c6000 con 0x7fcd10099cc0 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd15642700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100c53f0 con 0x7fcd10165400 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd15642700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100c56d0 con 0x7fcd1009a0a0 2024-05-08T23:28:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd10099cc0 0x7fcd101548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd10099cc0 0x7fcd101548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39980/0 (socket says 172.21.15.5:39980) 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd0effd700 1 -- 172.21.15.5:0/1384587150 learned_addr learned my addr 172.21.15.5:0/1384587150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd0e7fc700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd1009a0a0 0x7fcd100b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd0f7fe700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd10165400 0x7fcd1015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd0effd700 1 -- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd1009a0a0 msgr2=0x7fcd100b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.692+0000 7fcd0effd700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd1009a0a0 0x7fcd100b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fcd0effd700 1 -- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd10165400 msgr2=0x7fcd1015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fcd0effd700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd10165400 0x7fcd1015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fcd0effd700 1 -- 172.21.15.5:0/1384587150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd08009a70 con 0x7fcd10099cc0 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fcd0f7fe700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd10165400 0x7fcd1015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fcd0effd700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd10099cc0 0x7fcd101548d0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fcd000011e0 tx=0x7fcd0000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fccf77fe700 1 -- 172.21.15.5:0/1384587150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00004c80 con 0x7fcd10099cc0 2024-05-08T23:28:37.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fccf77fe700 1 -- 172.21.15.5:0/1384587150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd00004de0 con 0x7fcd10099cc0 2024-05-08T23:28:37.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd100c59b0 con 0x7fcd10099cc0 2024-05-08T23:28:37.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fccf77fe700 1 -- 172.21.15.5:0/1384587150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000072f0 con 0x7fcd10099cc0 2024-05-08T23:28:37.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.693+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10154e10 con 0x7fcd10099cc0 2024-05-08T23:28:37.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.694+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd100678f0 con 0x7fcd10099cc0 2024-05-08T23:28:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.694+0000 7fccf77fe700 1 -- 172.21.15.5:0/1384587150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcd00007450 con 0x7fcd10099cc0 2024-05-08T23:28:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.695+0000 7fccf77fe700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fccfc06ac90 0x7fccfc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.695+0000 7fccf77fe700 1 -- 172.21.15.5:0/1384587150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd00089160 con 0x7fcd10099cc0 2024-05-08T23:28:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.695+0000 7fcd0e7fc700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fccfc06ac90 0x7fccfc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.696+0000 7fcd0e7fc700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fccfc06ac90 0x7fccfc06d150 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcd1006ea80 tx=0x7fcd04009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.700+0000 7fccf77fe700 1 -- 172.21.15.5:0/1384587150 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcd000542c0 con 0x7fcd10099cc0 2024-05-08T23:28:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.969+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd100d0fd0 con 0x7fcd10099cc0 2024-05-08T23:28:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.970+0000 7fccf77fe700 1 -- 172.21.15.5:0/1384587150 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7fcd00007700 con 0x7fcd10099cc0 2024-05-08T23:28:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:37.970 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.972+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fccfc06ac90 msgr2=0x7fccfc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fccfc06ac90 0x7fccfc06d150 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcd1006ea80 tx=0x7fcd04009210 comp rx=0 tx=0).stop 2024-05-08T23:28:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd10099cc0 msgr2=0x7fcd101548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd10099cc0 0x7fcd101548d0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fcd000011e0 tx=0x7fcd0000ca50 comp rx=0 tx=0).stop 2024-05-08T23:28:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 shutdown_connections 2024-05-08T23:28:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fccfc06ac90 0x7fccfc06d150 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd10165400 0x7fcd1015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd10099cc0 0x7fcd101548d0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 --2- 172.21.15.5:0/1384587150 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd1009a0a0 0x7fcd100b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 >> 172.21.15.5:0/1384587150 conn(0x7fcd100b2f80 msgr2=0x7fcd1015c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 shutdown_connections 2024-05-08T23:28:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:37.973+0000 7fcd15642700 1 -- 172.21.15.5:0/1384587150 wait complete. 2024-05-08T23:28:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:37.986 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'b'] 2024-05-08T23:28:37.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:28:38.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.122+0000 7fc5d2396700 1 Processor -- start 2024-05-08T23:28:38.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.122+0000 7fc5d2396700 1 -- start start 2024-05-08T23:28:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.122+0000 7fc5d2396700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc0b2480 0x7fc5cc09a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.122+0000 7fc5d2396700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc5cc0cbbc0 0x7fc5cc0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.122+0000 7fc5d2396700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc0d0840 con 0x7fc5cc161030 2024-05-08T23:28:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.122+0000 7fc5d2396700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc06ccc0 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.122+0000 7fc5d2396700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0d0660 con 0x7fc5cc0b2480 2024-05-08T23:28:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5cbfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc5cc161030 0x7fc5cc099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57930/0 (socket says 172.21.15.5:57930) 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5cbfff700 1 -- 172.21.15.5:0/4281147177 learned_addr learned my addr 172.21.15.5:0/4281147177 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5d0933700 1 --2- 172.21.15.5:0/4281147177 >> v2:172.21.15.49:3300/0 conn(0x7fc5cc0cbbc0 0x7fc5cc0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5cb7fe700 1 --2- 172.21.15.5:0/4281147177 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc0b2480 0x7fc5cc09a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5caffd700 1 -- 172.21.15.5:0/4281147177 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563979257 0 0) 0x7fc5cc0d0840 con 0x7fc5cc161030 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5caffd700 1 -- 172.21.15.5:0/4281147177 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b40035b0 con 0x7fc5cc161030 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5caffd700 1 -- 172.21.15.5:0/4281147177 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434327588 0 0) 0x7fc5b40035b0 con 0x7fc5cc161030 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5d0933700 1 -- 172.21.15.5:0/4281147177 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc0b2480 msgr2=0x7fc5cc09a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5d0933700 1 --2- 172.21.15.5:0/4281147177 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc0b2480 0x7fc5cc09a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5d0933700 1 -- 172.21.15.5:0/4281147177 >> v1:172.21.15.5:6789/0 conn(0x7fc5cc161030 legacy=0x7fc5cc099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.123+0000 7fc5d0933700 1 -- 172.21.15.5:0/4281147177 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc06c600 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d0933700 1 --2- 172.21.15.5:0/4281147177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc0cc050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009d90 tx=0x7fc5c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4824c8822d3dcc2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5caffd700 1 -- 172.21.15.5:0/4281147177 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400e070 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5caffd700 1 -- 172.21.15.5:0/4281147177 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c4003a20 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5caffd700 1 -- 172.21.15.5:0/4281147177 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc001670 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d2396700 1 -- 172.21.15.5:0/4281147177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0cbbc0 msgr2=0x7fc5cc0cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d2396700 1 --2- 172.21.15.5:0/4281147177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc0cc050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009d90 tx=0x7fc5c400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d2396700 1 -- 172.21.15.5:0/4281147177 shutdown_connections 2024-05-08T23:28:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d2396700 1 --2- 172.21.15.5:0/4281147177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc0cc050 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d2396700 1 --2- 172.21.15.5:0/4281147177 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc0b2480 0x7fc5cc09a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d2396700 1 -- 172.21.15.5:0/4281147177 >> 172.21.15.5:0/4281147177 conn(0x7fc5cc0b2f20 msgr2=0x7fc5cc0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d2396700 1 -- 172.21.15.5:0/4281147177 shutdown_connections 2024-05-08T23:28:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.124+0000 7fc5d2396700 1 -- 172.21.15.5:0/4281147177 wait complete. 2024-05-08T23:28:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.126+0000 7fc5d2396700 1 Processor -- start 2024-05-08T23:28:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.126+0000 7fc5d2396700 1 -- start start 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5d2396700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0b2480 0x7fc5cc0c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5d2396700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5d2396700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc161030 0x7fc5cc0c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5d2396700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc155f50 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5d2396700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c53d0 con 0x7fc5cc0b2480 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5d2396700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c5ea0 con 0x7fc5cc161030 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39984/0 (socket says 172.21.15.5:39984) 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 -- 172.21.15.5:0/1519048581 learned_addr learned my addr 172.21.15.5:0/1519048581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cbfff700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0b2480 0x7fc5cc0c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5d0933700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc161030 0x7fc5cc0c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 -- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc161030 msgr2=0x7fc5cc0c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc161030 0x7fc5cc0c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 -- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0b2480 msgr2=0x7fc5cc0c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0b2480 0x7fc5cc0c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 -- 172.21.15.5:0/1519048581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c4009a70 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cbfff700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0b2480 0x7fc5cc0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.127+0000 7fc5cb7fe700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc155a10 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e870 tx=0x7fc5c000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.128+0000 7fc5c8ff9700 1 -- 172.21.15.5:0/1519048581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0009590 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.128+0000 7fc5c8ff9700 1 -- 172.21.15.5:0/1519048581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c000ee70 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.128+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc0c5600 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.128+0000 7fc5c8ff9700 1 -- 172.21.15.5:0/1519048581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c000ca50 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.128+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc0c5bc0 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.129+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc068000 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.129+0000 7fc5c8ff9700 1 -- 172.21.15.5:0/1519048581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc5c000cbb0 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.130+0000 7fc5c8ff9700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc5b806ac90 0x7fc5b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.130+0000 7fc5c8ff9700 1 -- 172.21.15.5:0/1519048581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5c0089370 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.130+0000 7fc5cbfff700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc5b806ac90 0x7fc5b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.131+0000 7fc5cbfff700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc5b806ac90 0x7fc5b806d150 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc001860 tx=0x7fc5bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:38.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.135+0000 7fc5c8ff9700 1 -- 172.21.15.5:0/1519048581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc5c00544b0 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:28:38.440 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:28:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.452+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc5cc0d0f50 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.452+0000 7fc5c8ff9700 1 -- 172.21.15.5:0/1519048581 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1966 (secure 0 0 0) 0x7fc5c000ce60 con 0x7fc5cc0cbbc0 2024-05-08T23:28:38.454 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:38.455 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":473,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":705854,"bytes_used":265936896,"bytes_avail":772828176384,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7336,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":109,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7052},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":7058},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8592}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-05-08T23:28:35.405031+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc5b806ac90 msgr2=0x7fc5b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc5b806ac90 0x7fc5b806d150 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc001860 tx=0x7fc5bc009040 comp rx=0 tx=0).stop 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5cc0cbbc0 msgr2=0x7fc5cc155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc155a10 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e870 tx=0x7fc5c000ec30 comp rx=0 tx=0).stop 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 shutdown_connections 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc5b806ac90 0x7fc5b806d150 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc5cc161030 0x7fc5cc0c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5cc0b2480 0x7fc5cc0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 --2- 172.21.15.5:0/1519048581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5cc0cbbc0 0x7fc5cc155a10 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 >> 172.21.15.5:0/1519048581 conn(0x7fc5cc0b2f20 msgr2=0x7fc5cc099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 shutdown_connections 2024-05-08T23:28:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:38.455+0000 7fc5d2396700 1 -- 172.21.15.5:0/1519048581 wait complete. 2024-05-08T23:28:38.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:28:39.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.619+0000 7fc2b4efe700 1 Processor -- start 2024-05-08T23:28:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.619+0000 7fc2b4efe700 1 -- start start 2024-05-08T23:28:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2b4efe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b015da10 0x7fc2b00d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2b4efe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2b006c3d0 0x7fc2b006c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2b4efe700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b00cf990 con 0x7fc2b006bd80 2024-05-08T23:28:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2b4efe700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b006cdc0 con 0x7fc2b006c3d0 2024-05-08T23:28:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2b4efe700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b009ab40 con 0x7fc2b015da10 2024-05-08T23:28:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2ae59c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b015da10 0x7fc2b00d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2aed9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2b006c3d0 0x7fc2b006c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2ae59c700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b015da10 0x7fc2b00d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:47278/0 (socket says 172.21.15.5:47278) 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2aed9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2b006c3d0 0x7fc2b006c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46772/0 (socket says 172.21.15.5:46772) 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2add9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc2b006bd80 0x7fc2b009a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:43530/0 (socket says 172.21.15.5:43530) 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.620+0000 7fc2ae59c700 1 -- 172.21.15.5:0/4212054888 learned_addr learned my addr 172.21.15.5:0/4212054888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ad59a700 1 -- 172.21.15.5:0/4212054888 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015820730 0 0) 0x7fc2b00cf990 con 0x7fc2b006bd80 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ad59a700 1 -- 172.21.15.5:0/4212054888 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2980035b0 con 0x7fc2b006bd80 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ae59c700 1 -- 172.21.15.5:0/4212054888 >> v2:172.21.15.49:3300/0 conn(0x7fc2b006c3d0 msgr2=0x7fc2b006c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/4212054888 >> v2:172.21.15.49:3300/0 conn(0x7fc2b006c3d0 0x7fc2b006c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ae59c700 1 -- 172.21.15.5:0/4212054888 >> v1:172.21.15.49:6789/0 conn(0x7fc2b006bd80 legacy=0x7fc2b009a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ae59c700 1 -- 172.21.15.5:0/4212054888 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b00d04b0 con 0x7fc2b015da10 2024-05-08T23:28:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2aed9d700 1 --2- 172.21.15.5:0/4212054888 >> v2:172.21.15.49:3300/0 conn(0x7fc2b006c3d0 0x7fc2b006c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/4212054888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b015da10 0x7fc2b00d0940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40098f0 tx=0x7fc2a4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f0a6cc9eeb31b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ad59a700 1 -- 172.21.15.5:0/4212054888 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4011830 con 0x7fc2b015da10 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ad59a700 1 -- 172.21.15.5:0/4212054888 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a4011990 con 0x7fc2b015da10 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.621+0000 7fc2ad59a700 1 -- 172.21.15.5:0/4212054888 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4011d20 con 0x7fc2b015da10 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.622+0000 7fc2b4efe700 1 -- 172.21.15.5:0/4212054888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b015da10 msgr2=0x7fc2b00d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.622+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/4212054888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b015da10 0x7fc2b00d0940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40098f0 tx=0x7fc2a4009c00 comp rx=0 tx=0).stop 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.622+0000 7fc2b4efe700 1 -- 172.21.15.5:0/4212054888 shutdown_connections 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.622+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/4212054888 >> v2:172.21.15.49:3300/0 conn(0x7fc2b006c3d0 0x7fc2b006c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.622+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/4212054888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b015da10 0x7fc2b00d0940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.622+0000 7fc2b4efe700 1 -- 172.21.15.5:0/4212054888 >> 172.21.15.5:0/4212054888 conn(0x7fc2b00b3010 msgr2=0x7fc2b00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.622+0000 7fc2b4efe700 1 -- 172.21.15.5:0/4212054888 shutdown_connections 2024-05-08T23:28:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.622+0000 7fc2b4efe700 1 -- 172.21.15.5:0/4212054888 wait complete. 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.623+0000 7fc2b4efe700 1 Processor -- start 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.624+0000 7fc2b4efe700 1 -- start start 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.624+0000 7fc2b4efe700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b006bd80 0x7fc2b014b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.624+0000 7fc2b4efe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b006c3d0 0x7fc2b01f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.624+0000 7fc2b4efe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015da10 0x7fc2b009eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.624+0000 7fc2b4efe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b014c460 con 0x7fc2b006c3d0 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.624+0000 7fc2b4efe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b01f7850 con 0x7fc2b015da10 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.624+0000 7fc2b4efe700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b014bbc0 con 0x7fc2b006bd80 2024-05-08T23:28:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b006c3d0 0x7fc2b01f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b006c3d0 0x7fc2b01f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39992/0 (socket says 172.21.15.5:39992) 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 -- 172.21.15.5:0/3185982318 learned_addr learned my addr 172.21.15.5:0/3185982318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2aed9d700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015da10 0x7fc2b009eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b006bd80 0x7fc2b014b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 -- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b006bd80 msgr2=0x7fc2b014b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b006bd80 0x7fc2b014b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 -- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015da10 msgr2=0x7fc2b009eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015da10 0x7fc2b009eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 -- 172.21.15.5:0/3185982318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8009a70 con 0x7fc2b006c3d0 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2aed9d700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015da10 0x7fc2b009eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2add9b700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b006c3d0 0x7fc2b01f7310 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c500 tx=0x7fc2a000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc29effd700 1 -- 172.21.15.5:0/3185982318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000cea0 con 0x7fc2b006c3d0 2024-05-08T23:28:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc29effd700 1 -- 172.21.15.5:0/3185982318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0005cb0 con 0x7fc2b006c3d0 2024-05-08T23:28:39.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a4009550 con 0x7fc2b006c3d0 2024-05-08T23:28:39.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc29effd700 1 -- 172.21.15.5:0/3185982318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0007310 con 0x7fc2b006c3d0 2024-05-08T23:28:39.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.625+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b014c180 con 0x7fc2b006c3d0 2024-05-08T23:28:39.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.626+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b00b76c0 con 0x7fc2b006c3d0 2024-05-08T23:28:39.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.627+0000 7fc29effd700 1 -- 172.21.15.5:0/3185982318 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc2a0007470 con 0x7fc2b006c3d0 2024-05-08T23:28:39.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.627+0000 7fc29effd700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.628+0000 7fc29effd700 1 -- 172.21.15.5:0/3185982318 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2a0088730 con 0x7fc2b006c3d0 2024-05-08T23:28:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.628+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:39.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.628+0000 7fc2ae59c700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009130 tx=0x7fc2a401d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:39.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.632+0000 7fc29effd700 1 -- 172.21.15.5:0/3185982318 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc2a0053900 con 0x7fc2b006c3d0 2024-05-08T23:28:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.898+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2b0068000 con 0x7fc2b006c3d0 2024-05-08T23:28:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.899+0000 7fc29effd700 1 -- 172.21.15.5:0/3185982318 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7fc2a0007720 con 0x7fc2b006c3d0 2024-05-08T23:28:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:39.900 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 msgr2=0x7fc29006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009130 tx=0x7fc2a401d040 comp rx=0 tx=0).stop 2024-05-08T23:28:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b006c3d0 msgr2=0x7fc2b01f7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b006c3d0 0x7fc2b01f7310 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c500 tx=0x7fc2a000c810 comp rx=0 tx=0).stop 2024-05-08T23:28:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 shutdown_connections 2024-05-08T23:28:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc29006ac90 0x7fc29006d150 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2b015da10 0x7fc2b009eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2b006bd80 0x7fc2b014b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 --2- 172.21.15.5:0/3185982318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b006c3d0 0x7fc2b01f7310 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.901+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 >> 172.21.15.5:0/3185982318 conn(0x7fc2b00b3010 msgr2=0x7fc2b015ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.902+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 shutdown_connections 2024-05-08T23:28:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:39.902+0000 7fc2b4efe700 1 -- 172.21.15.5:0/3185982318 wait complete. 2024-05-08T23:28:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:39.915 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'b'] 2024-05-08T23:28:39.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:28:40.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.050+0000 7fa61bcde700 1 Processor -- start 2024-05-08T23:28:40.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.051+0000 7fa61bcde700 1 -- start start 2024-05-08T23:28:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.051+0000 7fa61bcde700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa61406e610 0x7fa6140b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61bcde700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa6140af360 0x7fa61415e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61bcde700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6140b3e60 con 0x7fa6140aef80 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61bcde700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61415ec50 con 0x7fa6140af360 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61bcde700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61406b970 con 0x7fa61406e610 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61a27b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa6140af360 0x7fa61415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa619279700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa61406e610 0x7fa6140b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61a27b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa6140af360 0x7fa61415e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46786/0 (socket says 172.21.15.5:46786) 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61a27b700 1 -- 172.21.15.5:0/2035205775 learned_addr learned my addr 172.21.15.5:0/2035205775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa619a7a700 1 --1- 172.21.15.5:0/2035205775 >> v1:172.21.15.49:6789/0 conn(0x7fa6140aef80 0x7fa61406df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:43534/0 (socket says 172.21.15.5:43534) 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61a27b700 1 -- 172.21.15.5:0/2035205775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa61406e610 msgr2=0x7fa6140b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61a27b700 1 --2- 172.21.15.5:0/2035205775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa61406e610 0x7fa6140b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61a27b700 1 -- 172.21.15.5:0/2035205775 >> v1:172.21.15.49:6789/0 conn(0x7fa6140aef80 legacy=0x7fa61406df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61a27b700 1 -- 172.21.15.5:0/2035205775 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6140b4140 con 0x7fa6140af360 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa618a78700 1 -- 172.21.15.5:0/2035205775 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069012055 0 0) 0x7fa6140b3e60 con 0x7fa6140aef80 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.052+0000 7fa61a27b700 1 --2- 172.21.15.5:0/2035205775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140af360 0x7fa61415e710 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009e50 tx=0x7fa60c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9ca4c64d9b90a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa618a78700 1 -- 172.21.15.5:0/2035205775 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00e070 con 0x7fa6140af360 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa618a78700 1 -- 172.21.15.5:0/2035205775 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c00b910 con 0x7fa6140af360 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa618a78700 1 -- 172.21.15.5:0/2035205775 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00bcc0 con 0x7fa6140af360 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa61bcde700 1 -- 172.21.15.5:0/2035205775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140af360 msgr2=0x7fa61415e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa61bcde700 1 --2- 172.21.15.5:0/2035205775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140af360 0x7fa61415e710 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009e50 tx=0x7fa60c00b330 comp rx=0 tx=0).stop 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa61bcde700 1 -- 172.21.15.5:0/2035205775 shutdown_connections 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa61bcde700 1 --2- 172.21.15.5:0/2035205775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140af360 0x7fa61415e710 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa61bcde700 1 --2- 172.21.15.5:0/2035205775 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa61406e610 0x7fa6140b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa61bcde700 1 -- 172.21.15.5:0/2035205775 >> 172.21.15.5:0/2035205775 conn(0x7fa6140cfce0 msgr2=0x7fa6140d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa61bcde700 1 -- 172.21.15.5:0/2035205775 shutdown_connections 2024-05-08T23:28:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.053+0000 7fa61bcde700 1 -- 172.21.15.5:0/2035205775 wait complete. 2024-05-08T23:28:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.055+0000 7fa61bcde700 1 Processor -- start 2024-05-08T23:28:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.055+0000 7fa61bcde700 1 -- start start 2024-05-08T23:28:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.055+0000 7fa61bcde700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61406e610 0x7fa614142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.055+0000 7fa61bcde700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140aef80 0x7fa6141e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa61bcde700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa6140af360 0x7fa6141e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa61bcde700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614143730 con 0x7fa61406e610 2024-05-08T23:28:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa61bcde700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614143cf0 con 0x7fa6140aef80 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa61bcde700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa614156b30 con 0x7fa6140af360 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619a7a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61406e610 0x7fa614142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619279700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140aef80 0x7fa6141e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619a7a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61406e610 0x7fa614142ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39994/0 (socket says 172.21.15.5:39994) 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619a7a700 1 -- 172.21.15.5:0/3856910875 learned_addr learned my addr 172.21.15.5:0/3856910875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa61a27b700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa6140af360 0x7fa6141e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619279700 1 -- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa6140af360 msgr2=0x7fa6141e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619279700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa6140af360 0x7fa6141e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619279700 1 -- 172.21.15.5:0/3856910875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61406e610 msgr2=0x7fa614142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619279700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61406e610 0x7fa614142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619279700 1 -- 172.21.15.5:0/3856910875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c009b30 con 0x7fa6140aef80 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619a7a700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61406e610 0x7fa614142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa619279700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140aef80 0x7fa6141e8a40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa60400e8d0 tx=0x7fa60400ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.056+0000 7fa60a7fc700 1 -- 172.21.15.5:0/3856910875 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604013680 con 0x7fa6140aef80 2024-05-08T23:28:40.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.057+0000 7fa60a7fc700 1 -- 172.21.15.5:0/3856910875 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6040137e0 con 0x7fa6140aef80 2024-05-08T23:28:40.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.057+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa614143420 con 0x7fa6140aef80 2024-05-08T23:28:40.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.057+0000 7fa60a7fc700 1 -- 172.21.15.5:0/3856910875 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604013b50 con 0x7fa6140aef80 2024-05-08T23:28:40.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.057+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6141e9890 con 0x7fa6140aef80 2024-05-08T23:28:40.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.058+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6141e9a80 con 0x7fa6140aef80 2024-05-08T23:28:40.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.058+0000 7fa60a7fc700 1 -- 172.21.15.5:0/3856910875 <== mon.1 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa604013cb0 con 0x7fa6140aef80 2024-05-08T23:28:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.059+0000 7fa60a7fc700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa5f806ac90 0x7fa5f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.059+0000 7fa60a7fc700 1 -- 172.21.15.5:0/3856910875 <== mon.1 v2:172.21.15.49:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa604089340 con 0x7fa6140aef80 2024-05-08T23:28:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.059+0000 7fa619a7a700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa5f806ac90 0x7fa5f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.059+0000 7fa619a7a700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa5f806ac90 0x7fa5f806d150 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa610000f60 tx=0x7fa610007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.064+0000 7fa60a7fc700 1 -- 172.21.15.5:0/3856910875 <== mon.1 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa604054470 con 0x7fa6140aef80 2024-05-08T23:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.391+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa6140b2f30 con 0x7fa6140aef80 2024-05-08T23:28:40.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.394+0000 7fa60a7fc700 1 -- 172.21.15.5:0/3856910875 <== mon.1 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1946 (secure 0 0 0) 0x7fa6040177b0 con 0x7fa6140aef80 2024-05-08T23:28:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:40.394 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":475,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":705854,"bytes_used":265936896,"bytes_avail":772828176384,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":109,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7052},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":7058},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8592}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-05-08T23:28:35.405031+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa5f806ac90 msgr2=0x7fa5f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa5f806ac90 0x7fa5f806d150 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa610000f60 tx=0x7fa610007ea0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140aef80 msgr2=0x7fa6141e8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140aef80 0x7fa6141e8a40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa60400e8d0 tx=0x7fa60400ebe0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 shutdown_connections 2024-05-08T23:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa5f806ac90 0x7fa5f806d150 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa6140af360 0x7fa6141e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa61406e610 0x7fa614142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 --2- 172.21.15.5:0/3856910875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6140aef80 0x7fa6141e8a40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 >> 172.21.15.5:0/3856910875 conn(0x7fa6140cfce0 msgr2=0x7fa614071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 shutdown_connections 2024-05-08T23:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:40.397+0000 7fa61bcde700 1 -- 172.21.15.5:0/3856910875 wait complete. 2024-05-08T23:28:40.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:28:41.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:41.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.557+0000 7f08e814a700 1 Processor -- start 2024-05-08T23:28:41.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.557+0000 7f08e814a700 1 -- start start 2024-05-08T23:28:41.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e814a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08e0155cd0 0x7f08e015fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e814a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e006f970 0x7f08e015c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e814a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f08e00cc530 0x7f08e00cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e814a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e00ccf20 con 0x7f08e0155cd0 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e814a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e015f810 con 0x7f08e00cc530 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e814a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e01560b0 con 0x7f08e006f970 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e5ee6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08e0155cd0 0x7f08e015fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e5ee6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08e0155cd0 0x7f08e015fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40002/0 (socket says 172.21.15.5:40002) 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.558+0000 7f08e5ee6700 1 -- 172.21.15.5:0/1941250733 learned_addr learned my addr 172.21.15.5:0/1941250733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e56e5700 1 --2- 172.21.15.5:0/1941250733 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e006f970 0x7f08e015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e66e7700 1 --2- 172.21.15.5:0/1941250733 >> v2:172.21.15.49:3300/0 conn(0x7f08e00cc530 0x7f08e00cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e5ee6700 1 -- 172.21.15.5:0/1941250733 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e006f970 msgr2=0x7f08e015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e5ee6700 1 --2- 172.21.15.5:0/1941250733 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e006f970 0x7f08e015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e5ee6700 1 -- 172.21.15.5:0/1941250733 >> v2:172.21.15.49:3300/0 conn(0x7f08e00cc530 msgr2=0x7f08e00cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e5ee6700 1 --2- 172.21.15.5:0/1941250733 >> v2:172.21.15.49:3300/0 conn(0x7f08e00cc530 0x7f08e00cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e5ee6700 1 -- 172.21.15.5:0/1941250733 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e015cb90 con 0x7f08e0155cd0 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e66e7700 1 --2- 172.21.15.5:0/1941250733 >> v2:172.21.15.49:3300/0 conn(0x7f08e00cc530 0x7f08e00cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e5ee6700 1 --2- 172.21.15.5:0/1941250733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e0155cd0 0x7f08e015fca0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f08e01597a0 tx=0x7f08dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04b2aa7ec92152e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e4ee4700 1 -- 172.21.15.5:0/1941250733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00b870 con 0x7f08e0155cd0 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e4ee4700 1 -- 172.21.15.5:0/1941250733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc00b9d0 con 0x7f08e0155cd0 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.559+0000 7f08e4ee4700 1 -- 172.21.15.5:0/1941250733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00bd60 con 0x7f08e0155cd0 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 -- 172.21.15.5:0/1941250733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e0155cd0 msgr2=0x7f08e015fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 --2- 172.21.15.5:0/1941250733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e0155cd0 0x7f08e015fca0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f08e01597a0 tx=0x7f08dc009bd0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 -- 172.21.15.5:0/1941250733 shutdown_connections 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 --2- 172.21.15.5:0/1941250733 >> v2:172.21.15.49:3300/0 conn(0x7f08e00cc530 0x7f08e00cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 --2- 172.21.15.5:0/1941250733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e0155cd0 0x7f08e015fca0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 --2- 172.21.15.5:0/1941250733 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e006f970 0x7f08e015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 -- 172.21.15.5:0/1941250733 >> 172.21.15.5:0/1941250733 conn(0x7f08e00cfce0 msgr2=0x7f08e00d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 -- 172.21.15.5:0/1941250733 shutdown_connections 2024-05-08T23:28:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.560+0000 7f08e814a700 1 -- 172.21.15.5:0/1941250733 wait complete. 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.561+0000 7f08e814a700 1 Processor -- start 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.561+0000 7f08e814a700 1 -- start start 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e814a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e006f970 0x7f08e0146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e814a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e00cc530 0x7f08e01561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e814a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08e0155cd0 0x7f08e00b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e814a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0146bd0 con 0x7f08e006f970 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e814a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0156730 con 0x7f08e0155cd0 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e814a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e00ca860 con 0x7f08e00cc530 2024-05-08T23:28:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e5ee6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e006f970 0x7f08e0146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e5ee6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e006f970 0x7f08e0146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40016/0 (socket says 172.21.15.5:40016) 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e5ee6700 1 -- 172.21.15.5:0/77690577 learned_addr learned my addr 172.21.15.5:0/77690577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e66e7700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08e0155cd0 0x7f08e00b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e56e5700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e00cc530 0x7f08e01561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e5ee6700 1 -- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e00cc530 msgr2=0x7f08e01561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e5ee6700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e00cc530 0x7f08e01561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e5ee6700 1 -- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08e0155cd0 msgr2=0x7f08e00b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e5ee6700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08e0155cd0 0x7f08e00b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.562+0000 7f08e5ee6700 1 -- 172.21.15.5:0/77690577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc009550 con 0x7f08e006f970 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.563+0000 7f08e5ee6700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e006f970 0x7f08e0146690 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f08dc014040 tx=0x7f08dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.563+0000 7f08d67fc700 1 -- 172.21.15.5:0/77690577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc004520 con 0x7f08e006f970 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.563+0000 7f08d67fc700 1 -- 172.21.15.5:0/77690577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc01e070 con 0x7f08e006f970 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.563+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e00b30d0 con 0x7f08e006f970 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.563+0000 7f08d67fc700 1 -- 172.21.15.5:0/77690577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0038c0 con 0x7f08e006f970 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.563+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e00b3690 con 0x7f08e006f970 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.564+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e00cdb60 con 0x7f08e006f970 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.564+0000 7f08d67fc700 1 -- 172.21.15.5:0/77690577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f08dc003a20 con 0x7f08e006f970 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.565+0000 7f08d67fc700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f08c406acc0 0x7f08c406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.565+0000 7f08d67fc700 1 -- 172.21.15.5:0/77690577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08dc020070 con 0x7f08e006f970 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.565+0000 7f08e56e5700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f08c406acc0 0x7f08c406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.566+0000 7f08e56e5700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f08c406acc0 0x7f08c406d180 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f08e01f6c80 tx=0x7f08d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:41.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.570+0000 7f08d67fc700 1 -- 172.21.15.5:0/77690577 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f08dc0553d0 con 0x7f08e006f970 2024-05-08T23:28:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.835+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08e0068000 con 0x7f08e006f970 2024-05-08T23:28:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.836+0000 7f08d67fc700 1 -- 172.21.15.5:0/77690577 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7f08dc0589f0 con 0x7f08e006f970 2024-05-08T23:28:41.836 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:41.836 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f08c406acc0 msgr2=0x7f08c406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f08c406acc0 0x7f08c406d180 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f08e01f6c80 tx=0x7f08d0008040 comp rx=0 tx=0).stop 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e006f970 msgr2=0x7f08e0146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e006f970 0x7f08e0146690 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f08dc014040 tx=0x7f08dc004750 comp rx=0 tx=0).stop 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 shutdown_connections 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f08c406acc0 0x7f08c406d180 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08e0155cd0 0x7f08e00b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08e006f970 0x7f08e0146690 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 --2- 172.21.15.5:0/77690577 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f08e00cc530 0x7f08e01561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 >> 172.21.15.5:0/77690577 conn(0x7f08e00cfce0 msgr2=0x7f08e015ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 shutdown_connections 2024-05-08T23:28:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.838+0000 7f08e814a700 1 -- 172.21.15.5:0/77690577 wait complete. 2024-05-08T23:28:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:41.852 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'b'] 2024-05-08T23:28:41.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:28:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.988+0000 7f62fc376700 1 Processor -- start 2024-05-08T23:28:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.988+0000 7f62fc376700 1 -- start start 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fc376700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62f406a960 0x7f62f40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fc376700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62f40cf760 0x7f62f40cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fc376700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f40cdbb0 con 0x7f62f406b140 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fc376700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40cd310 con 0x7f62f40cf760 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fc376700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40cd5f0 con 0x7f62f406a960 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fa913700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62f40cf760 0x7f62f40cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fa913700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62f40cf760 0x7f62f40cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40028/0 (socket says 172.21.15.5:40028) 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fa913700 1 -- 172.21.15.5:0/688584261 learned_addr learned my addr 172.21.15.5:0/688584261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62f9911700 1 --2- 172.21.15.5:0/688584261 >> v2:172.21.15.49:3300/0 conn(0x7f62f406a960 0x7f62f40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fa913700 1 -- 172.21.15.5:0/688584261 >> v2:172.21.15.49:3300/0 conn(0x7f62f406a960 msgr2=0x7f62f40ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fa913700 1 --2- 172.21.15.5:0/688584261 >> v2:172.21.15.49:3300/0 conn(0x7f62f406a960 0x7f62f40ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fa913700 1 -- 172.21.15.5:0/688584261 >> v1:172.21.15.49:6789/0 conn(0x7f62f406b140 legacy=0x7f62f415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62fa913700 1 -- 172.21.15.5:0/688584261 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40cd8d0 con 0x7f62f40cf760 2024-05-08T23:28:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.989+0000 7f62f9911700 1 --2- 172.21.15.5:0/688584261 >> v2:172.21.15.49:3300/0 conn(0x7f62f406a960 0x7f62f40ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fa913700 1 --2- 172.21.15.5:0/688584261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f40cf760 0x7f62f40cfbd0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f62f406d750 tx=0x7f62ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbeb1edd62c5b699 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62f9110700 1 -- 172.21.15.5:0/688584261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00e070 con 0x7f62f40cf760 2024-05-08T23:28:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62f9110700 1 -- 172.21.15.5:0/688584261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec00b9c0 con 0x7f62f40cf760 2024-05-08T23:28:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62f9110700 1 -- 172.21.15.5:0/688584261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00bd70 con 0x7f62f40cf760 2024-05-08T23:28:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fc376700 1 -- 172.21.15.5:0/688584261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f40cf760 msgr2=0x7f62f40cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fc376700 1 --2- 172.21.15.5:0/688584261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f40cf760 0x7f62f40cfbd0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f62f406d750 tx=0x7f62ec00b330 comp rx=0 tx=0).stop 2024-05-08T23:28:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fc376700 1 -- 172.21.15.5:0/688584261 shutdown_connections 2024-05-08T23:28:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fc376700 1 --2- 172.21.15.5:0/688584261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f40cf760 0x7f62f40cfbd0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fc376700 1 --2- 172.21.15.5:0/688584261 >> v2:172.21.15.49:3300/0 conn(0x7f62f406a960 0x7f62f40ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fc376700 1 -- 172.21.15.5:0/688584261 >> 172.21.15.5:0/688584261 conn(0x7f62f40d0320 msgr2=0x7f62f40d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fc376700 1 -- 172.21.15.5:0/688584261 shutdown_connections 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.990+0000 7f62fc376700 1 -- 172.21.15.5:0/688584261 wait complete. 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.992+0000 7f62fc376700 1 Processor -- start 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.992+0000 7f62fc376700 1 -- start start 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.992+0000 7f62fc376700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f406a960 0x7f62f40c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.992+0000 7f62fc376700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f406b140 0x7f62f41470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fc376700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62f40cf760 0x7f62f40c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fc376700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f41475f0 con 0x7f62f406b140 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fc376700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4141450 con 0x7f62f406a960 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fc376700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40c4400 con 0x7f62f40cf760 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f406b140 0x7f62f41470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f406b140 0x7f62f41470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40040/0 (socket says 172.21.15.5:40040) 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 -- 172.21.15.5:0/3604552459 learned_addr learned my addr 172.21.15.5:0/3604552459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fa112700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f406a960 0x7f62f40c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fa913700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62f40cf760 0x7f62f40c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 -- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62f40cf760 msgr2=0x7f62f40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62f40cf760 0x7f62f40c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 -- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f406a960 msgr2=0x7f62f40c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f406a960 0x7f62f40c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 -- 172.21.15.5:0/3604552459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec009b30 con 0x7f62f406b140 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fa913700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62f40cf760 0x7f62f40c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62f9911700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f406b140 0x7f62f41470b0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e810 tx=0x7f62e400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62eaffd700 1 -- 172.21.15.5:0/3604552459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400c030 con 0x7f62f406b140 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62eaffd700 1 -- 172.21.15.5:0/3604552459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e4013610 con 0x7f62f406b140 2024-05-08T23:28:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f40c3cc0 con 0x7f62f406b140 2024-05-08T23:28:41.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62eaffd700 1 -- 172.21.15.5:0/3604552459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40139a0 con 0x7f62f406b140 2024-05-08T23:28:41.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.993+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f40c7000 con 0x7f62f406b140 2024-05-08T23:28:41.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.994+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f40cb740 con 0x7f62f406b140 2024-05-08T23:28:41.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.995+0000 7f62eaffd700 1 -- 172.21.15.5:0/3604552459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f62e4013b00 con 0x7f62f406b140 2024-05-08T23:28:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.995+0000 7f62eaffd700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62d806ac90 0x7f62d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.996+0000 7f62eaffd700 1 -- 172.21.15.5:0/3604552459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62e4088d20 con 0x7f62f406b140 2024-05-08T23:28:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.996+0000 7f62fa112700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62d806ac90 0x7f62d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:41.996+0000 7f62fa112700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62d806ac90 0x7f62d806d150 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f62f0000f60 tx=0x7f62f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.000+0000 7f62eaffd700 1 -- 172.21.15.5:0/3604552459 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f62e4053f00 con 0x7f62f406b140 2024-05-08T23:28:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.311+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f62f4068000 con 0x7f62f406b140 2024-05-08T23:28:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.311+0000 7f62eaffd700 1 -- 172.21.15.5:0/3604552459 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1946 (secure 0 0 0) 0x7f62e4057520 con 0x7f62f406b140 2024-05-08T23:28:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:42.312 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":477,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":705854,"bytes_used":265936896,"bytes_avail":772828176384,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":109,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7052},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":7058},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8592}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-05-08T23:28:35.405031+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62d806ac90 msgr2=0x7f62d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62d806ac90 0x7f62d806d150 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f62f0000f60 tx=0x7f62f0009040 comp rx=0 tx=0).stop 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f406b140 msgr2=0x7f62f41470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f406b140 0x7f62f41470b0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e810 tx=0x7f62e400ebd0 comp rx=0 tx=0).stop 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 shutdown_connections 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f62d806ac90 0x7f62d806d150 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f62f40cf760 0x7f62f40c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62f406a960 0x7f62f40c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 --2- 172.21.15.5:0/3604552459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f406b140 0x7f62f41470b0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 >> 172.21.15.5:0/3604552459 conn(0x7f62f40d0320 msgr2=0x7f62f40ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 shutdown_connections 2024-05-08T23:28:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:42.314+0000 7f62fc376700 1 -- 172.21.15.5:0/3604552459 wait complete. 2024-05-08T23:28:42.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-05-08T23:28:43.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.475+0000 7f4122674700 1 Processor -- start 2024-05-08T23:28:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.475+0000 7f4122674700 1 -- start start 2024-05-08T23:28:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f4122674700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f411c15d7a0 0x7f411c161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f4122674700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c0d3f40 0x7f411c0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f4122674700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c162450 con 0x7f411c15db80 2024-05-08T23:28:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f4122674700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c0cee00 con 0x7f411c15d7a0 2024-05-08T23:28:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f4122674700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0b27e0 con 0x7f411c0d3f40 2024-05-08T23:28:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f411bfff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f411c15d7a0 0x7f411c161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f4120c11700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f411c15db80 0x7f411c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57934/0 (socket says 172.21.15.5:57934) 2024-05-08T23:28:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f4120c11700 1 -- 172.21.15.5:0/1871927401 learned_addr learned my addr 172.21.15.5:0/1871927401 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.476+0000 7f411b7fe700 1 --2- 172.21.15.5:0/1871927401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c0d3f40 0x7f411c0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411bfff700 1 -- 172.21.15.5:0/1871927401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c0d3f40 msgr2=0x7f411c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411bfff700 1 --2- 172.21.15.5:0/1871927401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c0d3f40 0x7f411c0b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411bfff700 1 -- 172.21.15.5:0/1871927401 >> v1:172.21.15.5:6789/0 conn(0x7f411c15db80 legacy=0x7f411c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411bfff700 1 -- 172.21.15.5:0/1871927401 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0d4400 con 0x7f411c15d7a0 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411affd700 1 -- 172.21.15.5:0/1871927401 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347353015 0 0) 0x7f4114001490 con 0x7f411c15db80 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411bfff700 1 --2- 172.21.15.5:0/1871927401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c15d7a0 0x7f411c161f10 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f410c009840 tx=0x7f410c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7aa60afc6e615e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411affd700 1 -- 172.21.15.5:0/1871927401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c011830 con 0x7f411c15d7a0 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411affd700 1 -- 172.21.15.5:0/1871927401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c011990 con 0x7f411c15d7a0 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.477+0000 7f411affd700 1 -- 172.21.15.5:0/1871927401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c011d20 con 0x7f411c15d7a0 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.478+0000 7f4122674700 1 -- 172.21.15.5:0/1871927401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c15d7a0 msgr2=0x7f411c161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.478+0000 7f4122674700 1 --2- 172.21.15.5:0/1871927401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c15d7a0 0x7f411c161f10 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f410c009840 tx=0x7f410c009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.478+0000 7f4122674700 1 -- 172.21.15.5:0/1871927401 shutdown_connections 2024-05-08T23:28:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.478+0000 7f4122674700 1 --2- 172.21.15.5:0/1871927401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c15d7a0 0x7f411c161f10 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.478+0000 7f4122674700 1 --2- 172.21.15.5:0/1871927401 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c0d3f40 0x7f411c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.478+0000 7f4122674700 1 -- 172.21.15.5:0/1871927401 >> 172.21.15.5:0/1871927401 conn(0x7f411c08afe0 msgr2=0x7f411c0d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.478+0000 7f4122674700 1 -- 172.21.15.5:0/1871927401 shutdown_connections 2024-05-08T23:28:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.478+0000 7f4122674700 1 -- 172.21.15.5:0/1871927401 wait complete. 2024-05-08T23:28:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.479+0000 7f4122674700 1 Processor -- start 2024-05-08T23:28:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.479+0000 7f4122674700 1 -- start start 2024-05-08T23:28:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f4122674700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c0d3f40 0x7f411c15f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f4122674700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f411c15d7a0 0x7f411c0cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f4122674700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c15db80 0x7f411c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f4122674700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0cc090 con 0x7f411c0d3f40 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f4122674700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c15fa90 con 0x7f411c15d7a0 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f4122674700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c15fd70 con 0x7f411c15db80 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f411bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c0d3f40 0x7f411c15f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f411bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c0d3f40 0x7f411c15f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40054/0 (socket says 172.21.15.5:40054) 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.480+0000 7f411bfff700 1 -- 172.21.15.5:0/2785124324 learned_addr learned my addr 172.21.15.5:0/2785124324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f4120c11700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c15db80 0x7f411c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f411b7fe700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f411c15d7a0 0x7f411c0cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f411bfff700 1 -- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c15db80 msgr2=0x7f411c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f411bfff700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c15db80 0x7f411c0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f411bfff700 1 -- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f411c15d7a0 msgr2=0x7f411c0cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f411bfff700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f411c15d7a0 0x7f411c0cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f411bfff700 1 -- 172.21.15.5:0/2785124324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0094a0 con 0x7f411c0d3f40 2024-05-08T23:28:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f4120c11700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c15db80 0x7f411c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f411bfff700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c0d3f40 0x7f411c15f390 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f410c0090b0 tx=0x7f410c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f4118ff9700 1 -- 172.21.15.5:0/2785124324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0038a0 con 0x7f411c0d3f40 2024-05-08T23:28:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f4118ff9700 1 -- 172.21.15.5:0/2785124324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c01f070 con 0x7f411c0d3f40 2024-05-08T23:28:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f4118ff9700 1 -- 172.21.15.5:0/2785124324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c003c20 con 0x7f411c0d3f40 2024-05-08T23:28:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c160050 con 0x7f411c0d3f40 2024-05-08T23:28:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.481+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c160610 con 0x7f411c0d3f40 2024-05-08T23:28:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.482+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c068000 con 0x7f411c0d3f40 2024-05-08T23:28:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.483+0000 7f4118ff9700 1 -- 172.21.15.5:0/2785124324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f410c01a430 con 0x7f411c0d3f40 2024-05-08T23:28:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.483+0000 7f4118ff9700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f410806ac90 0x7f410806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.483+0000 7f4118ff9700 1 -- 172.21.15.5:0/2785124324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f410c021070 con 0x7f411c0d3f40 2024-05-08T23:28:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.483+0000 7f411b7fe700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f410806ac90 0x7f410806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.484+0000 7f411b7fe700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f410806ac90 0x7f410806d150 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f411c1f7070 tx=0x7f4110008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:43.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.488+0000 7f4118ff9700 1 -- 172.21.15.5:0/2785124324 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f410c055f40 con 0x7f411c0d3f40 2024-05-08T23:28:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.755+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f411c0b4e90 con 0x7f411c0d3f40 2024-05-08T23:28:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.756+0000 7f4118ff9700 1 -- 172.21.15.5:0/2785124324 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7f410c02b090 con 0x7f411c0d3f40 2024-05-08T23:28:43.756 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:43.756 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f410806ac90 msgr2=0x7f410806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f410806ac90 0x7f410806d150 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f411c1f7070 tx=0x7f4110008040 comp rx=0 tx=0).stop 2024-05-08T23:28:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c0d3f40 msgr2=0x7f411c15f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c0d3f40 0x7f411c15f390 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f410c0090b0 tx=0x7f410c004750 comp rx=0 tx=0).stop 2024-05-08T23:28:43.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 shutdown_connections 2024-05-08T23:28:43.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f410806ac90 0x7f410806d150 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f411c15db80 0x7f411c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f411c0d3f40 0x7f411c15f390 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 --2- 172.21.15.5:0/2785124324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f411c15d7a0 0x7f411c0cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 >> 172.21.15.5:0/2785124324 conn(0x7f411c08afe0 msgr2=0x7f411c0d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:43.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 shutdown_connections 2024-05-08T23:28:43.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.758+0000 7f4122674700 1 -- 172.21.15.5:0/2785124324 wait complete. 2024-05-08T23:28:43.760 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:43.772 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'b'] 2024-05-08T23:28:43.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:28:43.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.908+0000 7f10efbfd700 1 Processor -- start 2024-05-08T23:28:43.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.908+0000 7f10efbfd700 1 -- start start 2024-05-08T23:28:43.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10efbfd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e815b810 0x7f10e80cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10efbfd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10e809a160 0x7f10e80ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10efbfd700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e809aca0 con 0x7f10e80d0520 2024-05-08T23:28:43.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10efbfd700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e80cd9e0 con 0x7f10e809a160 2024-05-08T23:28:43.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10efbfd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80cf220 con 0x7f10e815b810 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10ee19a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10e809a160 0x7f10e80ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10ee19a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10e809a160 0x7f10e80ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40066/0 (socket says 172.21.15.5:40066) 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10ee19a700 1 -- 172.21.15.5:0/3158845129 learned_addr learned my addr 172.21.15.5:0/3158845129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10ed999700 1 --2- 172.21.15.5:0/3158845129 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e815b810 0x7f10e80cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10ee19a700 1 -- 172.21.15.5:0/3158845129 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e815b810 msgr2=0x7f10e80cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10ee19a700 1 --2- 172.21.15.5:0/3158845129 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e815b810 0x7f10e80cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10ee19a700 1 -- 172.21.15.5:0/3158845129 >> v1:172.21.15.49:6789/0 conn(0x7f10e80d0520 legacy=0x7f10e8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.909+0000 7f10ee19a700 1 -- 172.21.15.5:0/3158845129 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80cf500 con 0x7f10e809a160 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10ee19a700 1 --2- 172.21.15.5:0/3158845129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80ccdd0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009e80 tx=0x7f10e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45dcee183feee11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10ec997700 1 -- 172.21.15.5:0/3158845129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e000e070 con 0x7f10e809a160 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10ec997700 1 -- 172.21.15.5:0/3158845129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e000b980 con 0x7f10e809a160 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10ec997700 1 -- 172.21.15.5:0/3158845129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e000bd30 con 0x7f10e809a160 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10efbfd700 1 -- 172.21.15.5:0/3158845129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 msgr2=0x7f10e80ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10efbfd700 1 --2- 172.21.15.5:0/3158845129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80ccdd0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009e80 tx=0x7f10e000b330 comp rx=0 tx=0).stop 2024-05-08T23:28:43.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10efbfd700 1 -- 172.21.15.5:0/3158845129 shutdown_connections 2024-05-08T23:28:43.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10efbfd700 1 --2- 172.21.15.5:0/3158845129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80ccdd0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10efbfd700 1 --2- 172.21.15.5:0/3158845129 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e815b810 0x7f10e80cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.910+0000 7f10efbfd700 1 -- 172.21.15.5:0/3158845129 >> 172.21.15.5:0/3158845129 conn(0x7f10e80b3010 msgr2=0x7f10e80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:43.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.911+0000 7f10efbfd700 1 -- 172.21.15.5:0/3158845129 shutdown_connections 2024-05-08T23:28:43.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.911+0000 7f10efbfd700 1 -- 172.21.15.5:0/3158845129 wait complete. 2024-05-08T23:28:43.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.912+0000 7f10efbfd700 1 Processor -- start 2024-05-08T23:28:43.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.912+0000 7f10efbfd700 1 -- start start 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10efbfd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10efbfd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e80d0520 0x7f10e80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10efbfd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10e815b810 0x7f10e80b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10efbfd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80b5e20 con 0x7f10e809a160 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10efbfd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80b4840 con 0x7f10e815b810 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10efbfd700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80b4b20 con 0x7f10e80d0520 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40068/0 (socket says 172.21.15.5:40068) 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 -- 172.21.15.5:0/2567188569 learned_addr learned my addr 172.21.15.5:0/2567188569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ee19a700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10e815b810 0x7f10e80b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed198700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e80d0520 0x7f10e80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 -- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e80d0520 msgr2=0x7f10e80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e80d0520 0x7f10e80b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 -- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10e815b810 msgr2=0x7f10e80b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10e815b810 0x7f10e80b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 -- 172.21.15.5:0/2567188569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0009b30 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed198700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e80d0520 0x7f10e80b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10ed999700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80c9720 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f10e400d800 tx=0x7f10e400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10de7fc700 1 -- 172.21.15.5:0/2567188569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e40125f0 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10de7fc700 1 -- 172.21.15.5:0/2567188569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e4012750 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e80b4e00 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.913+0000 7f10de7fc700 1 -- 172.21.15.5:0/2567188569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4012b00 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.914+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e80b53c0 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.914+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e80b5580 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.915+0000 7f10de7fc700 1 -- 172.21.15.5:0/2567188569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f10e4012c60 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.916+0000 7f10de7fc700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10cc06ac90 0x7f10cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.916+0000 7f10de7fc700 1 -- 172.21.15.5:0/2567188569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e4089640 con 0x7f10e809a160 2024-05-08T23:28:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.916+0000 7f10ed198700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10cc06ac90 0x7f10cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.916+0000 7f10ed198700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10cc06ac90 0x7f10cc06d150 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f10e80b2710 tx=0x7f10d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:43.920+0000 7f10de7fc700 1 -- 172.21.15.5:0/2567188569 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f10e40548b0 con 0x7f10e809a160 2024-05-08T23:28:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.229+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f10e8068000 con 0x7f10e809a160 2024-05-08T23:28:44.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.230+0000 7f10de7fc700 1 -- 172.21.15.5:0/2567188569 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1870 (secure 0 0 0) 0x7f10e4057ed0 con 0x7f10e809a160 2024-05-08T23:28:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:44.231 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":478,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1715210456,"num_in_osds":8,"osd_in_since":1715210448,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":705854,"bytes_used":265936896,"bytes_avail":772828176384,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":109,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7052},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":7058},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8592}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-05-08T23:28:41.406572+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:28:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10cc06ac90 msgr2=0x7f10cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10cc06ac90 0x7f10cc06d150 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f10e80b2710 tx=0x7f10d8009040 comp rx=0 tx=0).stop 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 msgr2=0x7f10e80c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80c9720 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f10e400d800 tx=0x7f10e400dbc0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 shutdown_connections 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f10cc06ac90 0x7f10cc06d150 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10e815b810 0x7f10e80b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10e809a160 0x7f10e80c9720 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 --2- 172.21.15.5:0/2567188569 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f10e80d0520 0x7f10e80b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 >> 172.21.15.5:0/2567188569 conn(0x7f10e80b3010 msgr2=0x7f10e80ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.232+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 shutdown_connections 2024-05-08T23:28:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.233+0000 7f10efbfd700 1 -- 172.21.15.5:0/2567188569 wait complete. 2024-05-08T23:28:44.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]'} 2024-05-08T23:28:44.246 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:28:44.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:28:44.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:28:44.341 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:28:44.342 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:28:44.342 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:44.364 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:28:44.365 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:44.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:28:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.621+0000 7f6209cd0700 1 Processor -- start 2024-05-08T23:28:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.621+0000 7f6209cd0700 1 -- start start 2024-05-08T23:28:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f6209cd0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f62040ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f6209cd0700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62040d0a90 con 0x7f620406b0e0 2024-05-08T23:28:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f6209cd0700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620406b720 con 0x7f62040d0f20 2024-05-08T23:28:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f6209cd0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62040d0440 con 0x7f620415e830 2024-05-08T23:28:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f6203fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f620406b0e0 0x7f6204157600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57946/0 (socket says 172.21.15.5:57946) 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f6203fff700 1 -- 172.21.15.5:0/4122426003 learned_addr learned my addr 172.21.15.5:0/4122426003 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f62037fe700 1 --2- 172.21.15.5:0/4122426003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f62040ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f62027fc700 1 -- 172.21.15.5:0/4122426003 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163979740 0 0) 0x7f62040d0a90 con 0x7f620406b0e0 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.622+0000 7f62027fc700 1 -- 172.21.15.5:0/4122426003 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f00035b0 con 0x7f620406b0e0 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62027fc700 1 -- 172.21.15.5:0/4122426003 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995178625 0 0) 0x7f620406b720 con 0x7f62040d0f20 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62027fc700 1 -- 172.21.15.5:0/4122426003 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62040d0a90 con 0x7f62040d0f20 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62037fe700 1 -- 172.21.15.5:0/4122426003 >> v1:172.21.15.49:6789/0 conn(0x7f62040d0f20 legacy=0x7f620406a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62037fe700 1 -- 172.21.15.5:0/4122426003 >> v1:172.21.15.5:6789/0 conn(0x7f620406b0e0 legacy=0x7f6204157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62037fe700 1 -- 172.21.15.5:0/4122426003 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62040d0720 con 0x7f620415e830 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62037fe700 1 --2- 172.21.15.5:0/4122426003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f62040ceb50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009870 tx=0x7f61f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88503cf9f667e5a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62027fc700 1 -- 172.21.15.5:0/4122426003 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4011830 con 0x7f620415e830 2024-05-08T23:28:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62027fc700 1 -- 172.21.15.5:0/4122426003 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f4011990 con 0x7f620415e830 2024-05-08T23:28:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.623+0000 7f62027fc700 1 -- 172.21.15.5:0/4122426003 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4011d20 con 0x7f620415e830 2024-05-08T23:28:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.624+0000 7f6209cd0700 1 -- 172.21.15.5:0/4122426003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 msgr2=0x7f62040ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.624+0000 7f6209cd0700 1 --2- 172.21.15.5:0/4122426003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f62040ceb50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009870 tx=0x7f61f4009b80 comp rx=0 tx=0).stop 2024-05-08T23:28:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.624+0000 7f6209cd0700 1 -- 172.21.15.5:0/4122426003 shutdown_connections 2024-05-08T23:28:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.624+0000 7f6209cd0700 1 --2- 172.21.15.5:0/4122426003 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f62040ceb50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.624+0000 7f6209cd0700 1 -- 172.21.15.5:0/4122426003 >> 172.21.15.5:0/4122426003 conn(0x7f62040b5810 msgr2=0x7f62040b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.624+0000 7f6209cd0700 1 -- 172.21.15.5:0/4122426003 shutdown_connections 2024-05-08T23:28:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.624+0000 7f6209cd0700 1 -- 172.21.15.5:0/4122426003 wait complete. 2024-05-08T23:28:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.625+0000 7f6209cd0700 1 Processor -- start 2024-05-08T23:28:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.625+0000 7f6209cd0700 1 -- start start 2024-05-08T23:28:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6209cd0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f620406b0e0 0x7f62040ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6209cd0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62040d0f20 0x7f62040ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6209cd0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f6204142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6209cd0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62040ae440 con 0x7f62040d0f20 2024-05-08T23:28:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6209cd0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62040c99e0 con 0x7f620406b0e0 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6209cd0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62040add80 con 0x7f620415e830 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6202ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62040d0f20 0x7f62040ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6202ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62040d0f20 0x7f62040ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40072/0 (socket says 172.21.15.5:40072) 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6202ffd700 1 -- 172.21.15.5:0/2003088484 learned_addr learned my addr 172.21.15.5:0/2003088484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f62037fe700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f620406b0e0 0x7f62040ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.626+0000 7f6203fff700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f6204142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6202ffd700 1 -- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 msgr2=0x7f6204142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6202ffd700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f6204142b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6202ffd700 1 -- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f620406b0e0 msgr2=0x7f62040ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6202ffd700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f620406b0e0 0x7f62040ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6202ffd700 1 -- 172.21.15.5:0/2003088484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f40094a0 con 0x7f62040d0f20 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6203fff700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f6204142b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6202ffd700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62040d0f20 0x7f62040ae860 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00c520 tx=0x7f61ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f61fbfff700 1 -- 172.21.15.5:0/2003088484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0052d0 con 0x7f62040d0f20 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f61fbfff700 1 -- 172.21.15.5:0/2003088484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec005430 con 0x7f62040d0f20 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f61fbfff700 1 -- 172.21.15.5:0/2003088484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0038a0 con 0x7f62040d0f20 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62040ae060 con 0x7f62040d0f20 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.627+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62040c9670 con 0x7f62040d0f20 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.628+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204068000 con 0x7f62040d0f20 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.628+0000 7f61fbfff700 1 -- 172.21.15.5:0/2003088484 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f61ec003a00 con 0x7f62040d0f20 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.629+0000 7f61fbfff700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f61e406ac90 0x7f61e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.629+0000 7f61fbfff700 1 -- 172.21.15.5:0/2003088484 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61ec0887a0 con 0x7f62040d0f20 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.629+0000 7f62037fe700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f61e406ac90 0x7f61e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.630+0000 7f62037fe700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f61e406ac90 0x7f61e406d150 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f61f40090b0 tx=0x7f61f401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.634+0000 7f61fbfff700 1 -- 172.21.15.5:0/2003088484 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f61ec053a10 con 0x7f62040d0f20 2024-05-08T23:28:44.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.853+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62040cdf40 con 0x7f62040d0f20 2024-05-08T23:28:44.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.854+0000 7f61fbfff700 1 -- 172.21.15.5:0/2003088484 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+15944 (secure 0 0 0) 0x7f61ec003cb0 con 0x7f62040d0f20 2024-05-08T23:28:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:44.854 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:28:00.273637+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:27:57.260067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T23:27:58.243420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:28:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.856+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f61e406ac90 msgr2=0x7f61e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.856+0000 7f6209cd0700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f61e406ac90 0x7f61e406d150 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f61f40090b0 tx=0x7f61f401c040 comp rx=0 tx=0).stop 2024-05-08T23:28:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.856+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62040d0f20 msgr2=0x7f62040ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.856+0000 7f6209cd0700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62040d0f20 0x7f62040ae860 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00c520 tx=0x7f61ec00c830 comp rx=0 tx=0).stop 2024-05-08T23:28:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.857+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 shutdown_connections 2024-05-08T23:28:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.857+0000 7f6209cd0700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f61e406ac90 0x7f61e406d150 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.857+0000 7f6209cd0700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f620415e830 0x7f6204142b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.857+0000 7f6209cd0700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f620406b0e0 0x7f62040ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.857+0000 7f6209cd0700 1 --2- 172.21.15.5:0/2003088484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62040d0f20 0x7f62040ae860 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.857+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 >> 172.21.15.5:0/2003088484 conn(0x7f62040b5810 msgr2=0x7f620415ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.857+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 shutdown_connections 2024-05-08T23:28:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:44.857+0000 7f6209cd0700 1 -- 172.21.15.5:0/2003088484 wait complete. 2024-05-08T23:28:44.871 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:27:57.260067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:27:58.243420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:28:44.871 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:28:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.011+0000 7f5a47b1d700 1 Processor -- start 2024-05-08T23:28:45.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.011+0000 7f5a47b1d700 1 -- start start 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a47b1d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4009a160 0x7f5a400b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a47b1d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5a400cff30 0x7f5a400b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a47b1d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a400d04b0 con 0x7f5a4015b710 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a47b1d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a400b0910 con 0x7f5a400cff30 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a47b1d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400b0bf0 con 0x7f5a4009a160 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a460ba700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5a400cff30 0x7f5a400b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a458b9700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5a4015b710 0x7f5a40099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57948/0 (socket says 172.21.15.5:57948) 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a460ba700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5a400cff30 0x7f5a400b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40084/0 (socket says 172.21.15.5:40084) 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a458b9700 1 -- 172.21.15.5:0/212803379 learned_addr learned my addr 172.21.15.5:0/212803379 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a450b8700 1 --2- 172.21.15.5:0/212803379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4009a160 0x7f5a400b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a460ba700 1 -- 172.21.15.5:0/212803379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4009a160 msgr2=0x7f5a400b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a460ba700 1 --2- 172.21.15.5:0/212803379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4009a160 0x7f5a400b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a460ba700 1 -- 172.21.15.5:0/212803379 >> v1:172.21.15.5:6789/0 conn(0x7f5a4015b710 legacy=0x7f5a40099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a460ba700 1 -- 172.21.15.5:0/212803379 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a400b0ed0 con 0x7f5a400cff30 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.012+0000 7f5a448b7700 1 -- 172.21.15.5:0/212803379 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383815226 0 0) 0x7f5a400d04b0 con 0x7f5a4015b710 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a460ba700 1 --2- 172.21.15.5:0/212803379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a400b1c10 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009d90 tx=0x7f5a3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b426095d8ca6ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a448b7700 1 -- 172.21.15.5:0/212803379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800e070 con 0x7f5a400cff30 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a448b7700 1 -- 172.21.15.5:0/212803379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800bb20 con 0x7f5a400cff30 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a448b7700 1 -- 172.21.15.5:0/212803379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38012430 con 0x7f5a400cff30 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a47b1d700 1 -- 172.21.15.5:0/212803379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 msgr2=0x7f5a400b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/212803379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a400b1c10 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009d90 tx=0x7f5a3800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a47b1d700 1 -- 172.21.15.5:0/212803379 shutdown_connections 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/212803379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a400b1c10 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/212803379 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4009a160 0x7f5a400b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a47b1d700 1 -- 172.21.15.5:0/212803379 >> 172.21.15.5:0/212803379 conn(0x7f5a400b2f60 msgr2=0x7f5a400b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a47b1d700 1 -- 172.21.15.5:0/212803379 shutdown_connections 2024-05-08T23:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.013+0000 7f5a47b1d700 1 -- 172.21.15.5:0/212803379 wait complete. 2024-05-08T23:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.015+0000 7f5a47b1d700 1 Processor -- start 2024-05-08T23:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.015+0000 7f5a47b1d700 1 -- start start 2024-05-08T23:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a47b1d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a4009a160 0x7f5a40151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a47b1d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a4014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a47b1d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4015b710 0x7f5a401f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a47b1d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400b7070 con 0x7f5a400cff30 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a47b1d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a40151a20 con 0x7f5a4009a160 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a47b1d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400c8d90 con 0x7f5a4015b710 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a4014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a4014b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40086/0 (socket says 172.21.15.5:40086) 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 -- 172.21.15.5:0/375835961 learned_addr learned my addr 172.21.15.5:0/375835961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a460ba700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4015b710 0x7f5a401f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a458b9700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a4009a160 0x7f5a40151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 -- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4015b710 msgr2=0x7f5a401f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4015b710 0x7f5a401f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 -- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a4009a160 msgr2=0x7f5a40151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a4009a160 0x7f5a40151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 -- 172.21.15.5:0/375835961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009a70 con 0x7f5a400cff30 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a450b8700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a4014b230 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9d0 tx=0x7f5a3000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a367fc700 1 -- 172.21.15.5:0/375835961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a30009730 con 0x7f5a400cff30 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a367fc700 1 -- 172.21.15.5:0/375835961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3000ee70 con 0x7f5a400cff30 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a400cb490 con 0x7f5a400cff30 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.016+0000 7f5a367fc700 1 -- 172.21.15.5:0/375835961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000ca10 con 0x7f5a400cff30 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.017+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a400cae40 con 0x7f5a400cff30 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.017+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a40074bc0 con 0x7f5a400cff30 2024-05-08T23:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.018+0000 7f5a367fc700 1 -- 172.21.15.5:0/375835961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5a3000cb70 con 0x7f5a400cff30 2024-05-08T23:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.019+0000 7f5a367fc700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a2406ac90 0x7f5a2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.019+0000 7f5a367fc700 1 -- 172.21.15.5:0/375835961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a300892d0 con 0x7f5a400cff30 2024-05-08T23:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.019+0000 7f5a458b9700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a2406ac90 0x7f5a2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.020+0000 7f5a458b9700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a2406ac90 0x7f5a2406d150 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000f60 tx=0x7f5a3c007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.024+0000 7f5a367fc700 1 -- 172.21.15.5:0/375835961 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5a300544a0 con 0x7f5a400cff30 2024-05-08T23:28:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.242+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a400cda30 con 0x7f5a400cff30 2024-05-08T23:28:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.242+0000 7f5a367fc700 1 -- 172.21.15.5:0/375835961 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a30057ac0 con 0x7f5a400cff30 2024-05-08T23:28:45.242 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a2406ac90 msgr2=0x7f5a2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a2406ac90 0x7f5a2406d150 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000f60 tx=0x7f5a3c007e70 comp rx=0 tx=0).stop 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 msgr2=0x7f5a4014b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a4014b230 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9d0 tx=0x7f5a3000ece0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 shutdown_connections 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f5a2406ac90 0x7f5a2406d150 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5a4015b710 0x7f5a401f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a4009a160 0x7f5a40151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 --2- 172.21.15.5:0/375835961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a400cff30 0x7f5a4014b230 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 >> 172.21.15.5:0/375835961 conn(0x7f5a400b2f60 msgr2=0x7f5a40070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 shutdown_connections 2024-05-08T23:28:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.244+0000 7f5a47b1d700 1 -- 172.21.15.5:0/375835961 wait complete. 2024-05-08T23:28:45.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:28:45.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.397+0000 7fdaa2fba700 1 Processor -- start 2024-05-08T23:28:45.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.397+0000 7fdaa2fba700 1 -- start start 2024-05-08T23:28:45.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.398+0000 7fdaa2fba700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.398+0000 7fdaa2fba700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fda9c0d3d20 0x7fda9c0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.398+0000 7fdaa2fba700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9c0d0850 con 0x7fda9c0ce7a0 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.398+0000 7fdaa2fba700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c0cf340 con 0x7fda9c0d3d20 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.398+0000 7fdaa2fba700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0cf610 con 0x7fda9c0cff30 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.398+0000 7fda9bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fda9c0d3d20 0x7fda9c0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.398+0000 7fdaa0d56700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fdaa1557700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fda9c0ce7a0 0x7fda9c0cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:43580/0 (socket says 172.21.15.5:43580) 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fda9c0d3d20 0x7fda9c0ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46852/0 (socket says 172.21.15.5:46852) 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fdaa1557700 1 -- 172.21.15.5:0/126641693 learned_addr learned my addr 172.21.15.5:0/126641693 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9bfff700 1 -- 172.21.15.5:0/126641693 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 msgr2=0x7fda9c0d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9bfff700 1 --2- 172.21.15.5:0/126641693 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9bfff700 1 -- 172.21.15.5:0/126641693 >> v1:172.21.15.49:6789/0 conn(0x7fda9c0ce7a0 legacy=0x7fda9c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9bfff700 1 -- 172.21.15.5:0/126641693 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c0d1fc0 con 0x7fda9c0d3d20 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fdaa0d56700 1 --2- 172.21.15.5:0/126641693 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9bfff700 1 --2- 172.21.15.5:0/126641693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0ce260 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fda90009840 tx=0x7fda90009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63e0355e9a207782 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9b7fe700 1 -- 172.21.15.5:0/126641693 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011830 con 0x7fda9c0d3d20 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9b7fe700 1 -- 172.21.15.5:0/126641693 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90011990 con 0x7fda9c0d3d20 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.399+0000 7fda9b7fe700 1 -- 172.21.15.5:0/126641693 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011d20 con 0x7fda9c0d3d20 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.400+0000 7fdaa2fba700 1 -- 172.21.15.5:0/126641693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0d3d20 msgr2=0x7fda9c0ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.400+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/126641693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0ce260 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fda90009840 tx=0x7fda90009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.400+0000 7fdaa2fba700 1 -- 172.21.15.5:0/126641693 shutdown_connections 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.400+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/126641693 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.400+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/126641693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0ce260 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.400+0000 7fdaa2fba700 1 -- 172.21.15.5:0/126641693 >> 172.21.15.5:0/126641693 conn(0x7fda9c08afe0 msgr2=0x7fda9c0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.400+0000 7fdaa2fba700 1 -- 172.21.15.5:0/126641693 shutdown_connections 2024-05-08T23:28:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.400+0000 7fdaa2fba700 1 -- 172.21.15.5:0/126641693 wait complete. 2024-05-08T23:28:45.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.401+0000 7fdaa2fba700 1 Processor -- start 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.401+0000 7fdaa2fba700 1 -- start start 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.402+0000 7fdaa2fba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0ce7a0 0x7fda9c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.402+0000 7fdaa2fba700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.402+0000 7fdaa2fba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.402+0000 7fdaa2fba700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0ca080 con 0x7fda9c0d3d20 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.402+0000 7fdaa2fba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c15ee50 con 0x7fda9c0ce7a0 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.402+0000 7fdaa2fba700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c15e240 con 0x7fda9c0cff30 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40100/0 (socket says 172.21.15.5:40100) 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 -- 172.21.15.5:0/3048826122 learned_addr learned my addr 172.21.15.5:0/3048826122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa0d56700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0ce7a0 0x7fda9c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fda9bfff700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 -- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 msgr2=0x7fda9c0ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 -- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0ce7a0 msgr2=0x7fda9c15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0ce7a0 0x7fda9c15a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 -- 172.21.15.5:0/3048826122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c0094a0 con 0x7fda9c0d3d20 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa0d56700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0ce7a0 0x7fda9c15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa1557700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0caaa0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fda9400cab0 tx=0x7fda9400cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fda997fa700 1 -- 172.21.15.5:0/3048826122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9400a9a0 con 0x7fda9c0d3d20 2024-05-08T23:28:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fda997fa700 1 -- 172.21.15.5:0/3048826122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9400ab00 con 0x7fda9c0d3d20 2024-05-08T23:28:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda900094a0 con 0x7fda9c0d3d20 2024-05-08T23:28:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fda997fa700 1 -- 172.21.15.5:0/3048826122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9400ae70 con 0x7fda9c0d3d20 2024-05-08T23:28:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.403+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c15e800 con 0x7fda9c0d3d20 2024-05-08T23:28:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.404+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c074bc0 con 0x7fda9c0d3d20 2024-05-08T23:28:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.405+0000 7fda997fa700 1 -- 172.21.15.5:0/3048826122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fda9401b3b0 con 0x7fda9c0d3d20 2024-05-08T23:28:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.405+0000 7fda997fa700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda8806ac90 0x7fda8806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.406+0000 7fda997fa700 1 -- 172.21.15.5:0/3048826122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda94088de0 con 0x7fda9c0d3d20 2024-05-08T23:28:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.406+0000 7fdaa0d56700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda8806ac90 0x7fda8806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.406+0000 7fdaa0d56700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda8806ac90 0x7fda8806d150 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009dc0 tx=0x7fda8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.411+0000 7fda997fa700 1 -- 172.21.15.5:0/3048826122 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fda94054070 con 0x7fda9c0d3d20 2024-05-08T23:28:45.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.630+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda9c0d4520 con 0x7fda9c0d3d20 2024-05-08T23:28:45.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.630+0000 7fda997fa700 1 -- 172.21.15.5:0/3048826122 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda94017020 con 0x7fda9c0d3d20 2024-05-08T23:28:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:28:45.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda8806ac90 msgr2=0x7fda8806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda8806ac90 0x7fda8806d150 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009dc0 tx=0x7fda8c008040 comp rx=0 tx=0).stop 2024-05-08T23:28:45.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c0d3d20 msgr2=0x7fda9c0caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0caaa0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fda9400cab0 tx=0x7fda9400cdc0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 shutdown_connections 2024-05-08T23:28:45.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fda8806ac90 0x7fda8806d150 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c0d3d20 0x7fda9c0caaa0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda9c0ce7a0 0x7fda9c15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 --2- 172.21.15.5:0/3048826122 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fda9c0cff30 0x7fda9c0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 >> 172.21.15.5:0/3048826122 conn(0x7fda9c08afe0 msgr2=0x7fda9c0cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 shutdown_connections 2024-05-08T23:28:45.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.633+0000 7fdaa2fba700 1 -- 172.21.15.5:0/3048826122 wait complete. 2024-05-08T23:28:45.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:28:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.786+0000 7fe8063d1700 1 Processor -- start 2024-05-08T23:28:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.786+0000 7fe8063d1700 1 -- start start 2024-05-08T23:28:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.787+0000 7fe8063d1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe80015ba10 0x7fe8000cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.787+0000 7fe8063d1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80015f5f0 con 0x7fe80006b070 2024-05-08T23:28:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.787+0000 7fe8063d1700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80015f8d0 con 0x7fe8000cfde0 2024-05-08T23:28:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.787+0000 7fe8063d1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80015f3e0 con 0x7fe80015ba10 2024-05-08T23:28:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.787+0000 7fe80496e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe80006b070 0x7fe8000ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57952/0 (socket says 172.21.15.5:57952) 2024-05-08T23:28:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe80496e700 1 -- 172.21.15.5:0/3315564560 learned_addr learned my addr 172.21.15.5:0/3315564560 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7fffff700 1 --2- 172.21.15.5:0/3315564560 >> v2:172.21.15.49:3300/0 conn(0x7fe80015ba10 0x7fe8000cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547856950 0 0) 0x7fe80015f5f0 con 0x7fe80006b070 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e80035b0 con 0x7fe80006b070 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596626461 0 0) 0x7fe80015f8d0 con 0x7fe8000cfde0 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe80015f5f0 con 0x7fe8000cfde0 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041261842 0 0) 0x7fe7e80035b0 con 0x7fe80006b070 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 >> v2:172.21.15.49:3300/0 conn(0x7fe80015ba10 msgr2=0x7fe8000cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 --2- 172.21.15.5:0/3315564560 >> v2:172.21.15.49:3300/0 conn(0x7fe80015ba10 0x7fe8000cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 >> v1:172.21.15.49:6789/0 conn(0x7fe8000cfde0 legacy=0x7fe80006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80015fbb0 con 0x7fe80006b070 2024-05-08T23:28:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fe7f80019f0 con 0x7fe80006b070 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 >> v1:172.21.15.5:6789/0 conn(0x7fe80006b070 legacy=0x7fe8000ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8003b90 0x7fe7e8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7e8004850 0x7fe7e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8005210 0x7fe7e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015f3e0 con 0x7fe7e8005210 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e800c090 con 0x7fe7e8003b90 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e800c250 con 0x7fe7e8004850 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7fffff700 1 --2- 172.21.15.5:0/3315564560 >> v2:172.21.15.49:3300/0 conn(0x7fe80015ba10 0x7fe8000cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe80496e700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8005210 0x7fe7e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7ff7fe700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7e8004850 0x7fe7e8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.788+0000 7fe7fffff700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8003b90 0x7fe7e8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe80496e700 1 -- 172.21.15.5:0/3315564560 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7e8004850 msgr2=0x7fe7e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe80496e700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe7e8004850 0x7fe7e8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe80496e700 1 -- 172.21.15.5:0/3315564560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8003b90 msgr2=0x7fe7e8003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe80496e700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8003b90 0x7fe7e8003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe80496e700 1 -- 172.21.15.5:0/3315564560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e80035b0 con 0x7fe7e8005210 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe80496e700 1 -- 172.21.15.5:0/3315564560 reap_dead start 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe80496e700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8005210 0x7fe7e800ba10 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800a7a0 tx=0x7fe7e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfd77b9bf89deb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8012600 con 0x7fe7e8005210 2024-05-08T23:28:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f8012780 con 0x7fe7e8005210 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.789+0000 7fe7feffd700 1 -- 172.21.15.5:0/3315564560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8012b30 con 0x7fe7e8005210 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.790+0000 7fe8063d1700 1 -- 172.21.15.5:0/3315564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8005210 msgr2=0x7fe7e800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.790+0000 7fe8063d1700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8005210 0x7fe7e800ba10 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800a7a0 tx=0x7fe7e8005880 comp rx=0 tx=0).stop 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.790+0000 7fe8063d1700 1 -- 172.21.15.5:0/3315564560 shutdown_connections 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.790+0000 7fe8063d1700 1 --2- 172.21.15.5:0/3315564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7e8005210 0x7fe7e800ba10 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.790+0000 7fe8063d1700 1 -- 172.21.15.5:0/3315564560 >> 172.21.15.5:0/3315564560 conn(0x7fe800099a50 msgr2=0x7fe8000d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.790+0000 7fe8063d1700 1 -- 172.21.15.5:0/3315564560 shutdown_connections 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.790+0000 7fe8063d1700 1 -- 172.21.15.5:0/3315564560 wait complete. 2024-05-08T23:28:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.791+0000 7fe8063d1700 1 Processor -- start 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.791+0000 7fe8063d1700 1 -- start start 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe8063d1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8005210 0x7fe8001547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe8063d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8000b14c0 0x7fe80014eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe8063d1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8001560b0 0x7fe8001564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe8063d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80014fea0 con 0x7fe8000b14c0 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe8063d1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80014f8e0 con 0x7fe7e8005210 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe8063d1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80014fbc0 con 0x7fe8001560b0 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe7ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8000b14c0 0x7fe80014eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe7ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8000b14c0 0x7fe80014eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40132/0 (socket says 172.21.15.5:40132) 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe7ff7fe700 1 -- 172.21.15.5:0/1584418571 learned_addr learned my addr 172.21.15.5:0/1584418571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe80496e700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8001560b0 0x7fe8001564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe7fffff700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8005210 0x7fe8001547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe7ff7fe700 1 -- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8001560b0 msgr2=0x7fe8001564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe7ff7fe700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8001560b0 0x7fe8001564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe7ff7fe700 1 -- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8005210 msgr2=0x7fe8001547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.792+0000 7fe7ff7fe700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8005210 0x7fe8001547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe7ff7fe700 1 -- 172.21.15.5:0/1584418571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800151270 con 0x7fe8000b14c0 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe80496e700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8001560b0 0x7fe8001564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe7fffff700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8005210 0x7fe8001547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe7ff7fe700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8000b14c0 0x7fe80014eed0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400f770 tx=0x7fe7f400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe7fcff9700 1 -- 172.21.15.5:0/1584418571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4005320 con 0x7fe8000b14c0 2024-05-08T23:28:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe7fcff9700 1 -- 172.21.15.5:0/1584418571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f40054a0 con 0x7fe8000b14c0 2024-05-08T23:28:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8000b1940 con 0x7fe8000b14c0 2024-05-08T23:28:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe7fcff9700 1 -- 172.21.15.5:0/1584418571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4006aa0 con 0x7fe8000b14c0 2024-05-08T23:28:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.793+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80014f410 con 0x7fe8000b14c0 2024-05-08T23:28:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.794+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80014fa20 con 0x7fe8000b14c0 2024-05-08T23:28:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.794+0000 7fe7fcff9700 1 -- 172.21.15.5:0/1584418571 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe7f4006c00 con 0x7fe8000b14c0 2024-05-08T23:28:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.795+0000 7fe7fcff9700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7ec06ac90 0x7fe7ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.795+0000 7fe7fcff9700 1 -- 172.21.15.5:0/1584418571 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f4088bb0 con 0x7fe8000b14c0 2024-05-08T23:28:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.795+0000 7fe7fffff700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7ec06ac90 0x7fe7ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.796+0000 7fe7fffff700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7ec06ac90 0x7fe7ec06d150 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009d70 tx=0x7fe7f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:45.800+0000 7fe7fcff9700 1 -- 172.21.15.5:0/1584418571 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe80014fa20 con 0x7fe8000b14c0 2024-05-08T23:28:46.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.015+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe80014fa20 con 0x7fe8000b14c0 2024-05-08T23:28:46.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.015+0000 7fe7fcff9700 1 -- 172.21.15.5:0/1584418571 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe80014fa20 con 0x7fe8000b14c0 2024-05-08T23:28:46.016 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:28:46.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7ec06ac90 msgr2=0x7fe7ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7ec06ac90 0x7fe7ec06d150 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009d70 tx=0x7fe7f0008040 comp rx=0 tx=0).stop 2024-05-08T23:28:46.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8000b14c0 msgr2=0x7fe80014eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8000b14c0 0x7fe80014eed0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400f770 tx=0x7fe7f400fb30 comp rx=0 tx=0).stop 2024-05-08T23:28:46.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 shutdown_connections 2024-05-08T23:28:46.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe7ec06ac90 0x7fe7ec06d150 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe8001560b0 0x7fe8001564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7e8005210 0x7fe8001547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 --2- 172.21.15.5:0/1584418571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8000b14c0 0x7fe80014eed0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.017+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 >> 172.21.15.5:0/1584418571 conn(0x7fe800099a50 msgr2=0x7fe8000b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.018+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 shutdown_connections 2024-05-08T23:28:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.018+0000 7fe8063d1700 1 -- 172.21.15.5:0/1584418571 wait complete. 2024-05-08T23:28:46.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.170+0000 7f2cd63a3700 1 Processor -- start 2024-05-08T23:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.171+0000 7f2cd63a3700 1 -- start start 2024-05-08T23:28:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.171+0000 7f2cd63a3700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cd0160f00 0x7f2cd00b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.171+0000 7f2cd63a3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2cd00d22d0 0x7f2cd00ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.171+0000 7f2cd63a3700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd00ce3f0 con 0x7f2cd00b2f60 2024-05-08T23:28:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.171+0000 7f2cd63a3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd00cdc50 con 0x7f2cd00d22d0 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.171+0000 7f2cd63a3700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd00b42c0 con 0x7f2cd0160f00 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cd4940700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2cd00d22d0 0x7f2cd00ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cd4940700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2cd00d22d0 0x7f2cd00ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40134/0 (socket says 172.21.15.5:40134) 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cd4940700 1 -- 172.21.15.5:0/1883268884 learned_addr learned my addr 172.21.15.5:0/1883268884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2ccffff700 1 --2- 172.21.15.5:0/1883268884 >> v2:172.21.15.49:3300/0 conn(0x7f2cd0160f00 0x7f2cd00b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cd4940700 1 -- 172.21.15.5:0/1883268884 >> v2:172.21.15.49:3300/0 conn(0x7f2cd0160f00 msgr2=0x7f2cd00b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cd4940700 1 --2- 172.21.15.5:0/1883268884 >> v2:172.21.15.49:3300/0 conn(0x7f2cd0160f00 0x7f2cd00b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cd4940700 1 -- 172.21.15.5:0/1883268884 >> v1:172.21.15.49:6789/0 conn(0x7f2cd00b2f60 legacy=0x7f2cd00d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cd4940700 1 -- 172.21.15.5:0/1883268884 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd00b3740 con 0x7f2cd00d22d0 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cd4940700 1 --2- 172.21.15.5:0/1883268884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd00d22d0 0x7f2cd00ce860 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00d1180 tx=0x7f2cc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7571dde337774efa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cceffd700 1 -- 172.21.15.5:0/1883268884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800e070 con 0x7f2cd00d22d0 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cceffd700 1 -- 172.21.15.5:0/1883268884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8003a20 con 0x7f2cd00d22d0 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.172+0000 7f2cceffd700 1 -- 172.21.15.5:0/1883268884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8003dd0 con 0x7f2cd00d22d0 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.173+0000 7f2cd63a3700 1 -- 172.21.15.5:0/1883268884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd00d22d0 msgr2=0x7f2cd00ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.173+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/1883268884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd00d22d0 0x7f2cd00ce860 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00d1180 tx=0x7f2cc800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.173+0000 7f2cd63a3700 1 -- 172.21.15.5:0/1883268884 shutdown_connections 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.173+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/1883268884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd00d22d0 0x7f2cd00ce860 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.173+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/1883268884 >> v2:172.21.15.49:3300/0 conn(0x7f2cd0160f00 0x7f2cd00b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.173+0000 7f2cd63a3700 1 -- 172.21.15.5:0/1883268884 >> 172.21.15.5:0/1883268884 conn(0x7f2cd008afe0 msgr2=0x7f2cd00d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.173+0000 7f2cd63a3700 1 -- 172.21.15.5:0/1883268884 shutdown_connections 2024-05-08T23:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.173+0000 7f2cd63a3700 1 -- 172.21.15.5:0/1883268884 wait complete. 2024-05-08T23:28:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.174+0000 7f2cd63a3700 1 Processor -- start 2024-05-08T23:28:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.175+0000 7f2cd63a3700 1 -- start start 2024-05-08T23:28:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.175+0000 7f2cd63a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cd00b2f60 0x7f2cd00b8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.175+0000 7f2cd63a3700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2cd00d22d0 0x7f2cd00b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.175+0000 7f2cd63a3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd0160f00 0x7f2cd014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.175+0000 7f2cd63a3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00b0a70 con 0x7f2cd0160f00 2024-05-08T23:28:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.175+0000 7f2cd63a3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00b8aa0 con 0x7f2cd00b2f60 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.175+0000 7f2cd63a3700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00c7850 con 0x7f2cd00d22d0 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.175+0000 7f2cd4940700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd0160f00 0x7f2cd014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd4940700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd0160f00 0x7f2cd014a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40140/0 (socket says 172.21.15.5:40140) 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd4940700 1 -- 172.21.15.5:0/3356607382 learned_addr learned my addr 172.21.15.5:0/3356607382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2ccffff700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cd00b2f60 0x7f2cd00b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2ccf7fe700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2cd00d22d0 0x7f2cd00b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd4940700 1 -- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2cd00d22d0 msgr2=0x7f2cd00b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd4940700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2cd00d22d0 0x7f2cd00b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd4940700 1 -- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cd00b2f60 msgr2=0x7f2cd00b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd4940700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cd00b2f60 0x7f2cd00b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd4940700 1 -- 172.21.15.5:0/3356607382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc8009b30 con 0x7f2cd0160f00 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd4940700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd0160f00 0x7f2cd014a120 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800b370 tx=0x7f2cc8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cccff9700 1 -- 172.21.15.5:0/3356607382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8004440 con 0x7f2cd0160f00 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cccff9700 1 -- 172.21.15.5:0/3356607382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc80092e0 con 0x7f2cd0160f00 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd00b03d0 con 0x7f2cd0160f00 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cccff9700 1 -- 172.21.15.5:0/3356607382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8004800 con 0x7f2cd0160f00 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.176+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd00b8fe0 con 0x7f2cd0160f00 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.177+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd00b0bb0 con 0x7f2cd0160f00 2024-05-08T23:28:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.178+0000 7f2cccff9700 1 -- 172.21.15.5:0/3356607382 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2cc8012430 con 0x7f2cd0160f00 2024-05-08T23:28:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.178+0000 7f2cccff9700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2cb006ac90 0x7f2cb006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.179+0000 7f2cccff9700 1 -- 172.21.15.5:0/3356607382 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cc808a670 con 0x7f2cd0160f00 2024-05-08T23:28:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.179+0000 7f2ccffff700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2cb006ac90 0x7f2cb006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.179+0000 7f2ccffff700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2cb006ac90 0x7f2cb006d150 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8006fd0 tx=0x7f2cb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:46.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.183+0000 7f2cccff9700 1 -- 172.21.15.5:0/3356607382 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2cd00b0bb0 con 0x7f2cd0160f00 2024-05-08T23:28:46.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.417+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2cd00b0bb0 con 0x7f2cd0160f00 2024-05-08T23:28:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.418+0000 7f2cccff9700 1 -- 172.21.15.5:0/3356607382 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2cd00b0bb0 con 0x7f2cd0160f00 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2cb006ac90 msgr2=0x7f2cb006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2cb006ac90 0x7f2cb006d150 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8006fd0 tx=0x7f2cb8008040 comp rx=0 tx=0).stop 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd0160f00 msgr2=0x7f2cd014a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd0160f00 0x7f2cd014a120 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800b370 tx=0x7f2cc8004ad0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 shutdown_connections 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f2cb006ac90 0x7f2cb006d150 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd0160f00 0x7f2cd014a120 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cd00b2f60 0x7f2cd00b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 --2- 172.21.15.5:0/3356607382 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f2cd00d22d0 0x7f2cd00b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 >> 172.21.15.5:0/3356607382 conn(0x7f2cd008afe0 msgr2=0x7f2cd015ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 shutdown_connections 2024-05-08T23:28:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.420+0000 7f2cd63a3700 1 -- 172.21.15.5:0/3356607382 wait complete. 2024-05-08T23:28:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-08T23:28:46.434 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:28:46.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:46.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T23:28:46.484 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:28:46.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:46.485 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:28:46.504 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:28:46.504 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:46.526 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:28:46.526 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:28:46.527 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:46.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:28:46.566 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:28:46.566 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:46.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:28:46.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:46.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.751+0000 7fc335c67700 1 Processor -- start 2024-05-08T23:28:46.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.751+0000 7fc335c67700 1 -- start start 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.751+0000 7fc335c67700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc330161030 0x7fc330099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.751+0000 7fc335c67700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc3300b2480 0x7fc330099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.751+0000 7fc335c67700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3300d0840 con 0x7fc33009a4d0 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc335c67700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33006ccc0 con 0x7fc330161030 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc335c67700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33009a920 con 0x7fc3300b2480 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32f7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc330161030 0x7fc330099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32ffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc33009a4d0 0x7fc3300cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57968/0 (socket says 172.21.15.5:57968) 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32ffff700 1 -- 172.21.15.5:0/4246376107 learned_addr learned my addr 172.21.15.5:0/4246376107 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32effd700 1 --2- 172.21.15.5:0/4246376107 >> v2:172.21.15.49:3300/0 conn(0x7fc3300b2480 0x7fc330099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32f7fe700 1 -- 172.21.15.5:0/4246376107 >> v2:172.21.15.49:3300/0 conn(0x7fc3300b2480 msgr2=0x7fc330099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32f7fe700 1 --2- 172.21.15.5:0/4246376107 >> v2:172.21.15.49:3300/0 conn(0x7fc3300b2480 0x7fc330099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32f7fe700 1 -- 172.21.15.5:0/4246376107 >> v1:172.21.15.5:6789/0 conn(0x7fc33009a4d0 legacy=0x7fc3300cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32e7fc700 1 -- 172.21.15.5:0/4246376107 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351455407 0 0) 0x7fc3300d0840 con 0x7fc33009a4d0 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32f7fe700 1 -- 172.21.15.5:0/4246376107 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33006c600 con 0x7fc330161030 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32f7fe700 1 --2- 172.21.15.5:0/4246376107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc330099a50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc3300b2430 tx=0x7fc320009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1cccb3dd13a685 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.752+0000 7fc32e7fc700 1 -- 172.21.15.5:0/4246376107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320011830 con 0x7fc330161030 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc32e7fc700 1 -- 172.21.15.5:0/4246376107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc320011990 con 0x7fc330161030 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc32e7fc700 1 -- 172.21.15.5:0/4246376107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320011d20 con 0x7fc330161030 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc335c67700 1 -- 172.21.15.5:0/4246376107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 msgr2=0x7fc330099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc335c67700 1 --2- 172.21.15.5:0/4246376107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc330099a50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc3300b2430 tx=0x7fc320009b20 comp rx=0 tx=0).stop 2024-05-08T23:28:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc335c67700 1 -- 172.21.15.5:0/4246376107 shutdown_connections 2024-05-08T23:28:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc335c67700 1 --2- 172.21.15.5:0/4246376107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc330099a50 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc335c67700 1 --2- 172.21.15.5:0/4246376107 >> v2:172.21.15.49:3300/0 conn(0x7fc3300b2480 0x7fc330099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc335c67700 1 -- 172.21.15.5:0/4246376107 >> 172.21.15.5:0/4246376107 conn(0x7fc3300b2f20 msgr2=0x7fc3300b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc335c67700 1 -- 172.21.15.5:0/4246376107 shutdown_connections 2024-05-08T23:28:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.753+0000 7fc335c67700 1 -- 172.21.15.5:0/4246376107 wait complete. 2024-05-08T23:28:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.754+0000 7fc335c67700 1 Processor -- start 2024-05-08T23:28:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.755+0000 7fc335c67700 1 -- start start 2024-05-08T23:28:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.755+0000 7fc335c67700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc33009a4d0 0x7fc3300c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.755+0000 7fc335c67700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3300b2480 0x7fc3300b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.755+0000 7fc335c67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc3300b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.755+0000 7fc335c67700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300ca300 con 0x7fc330161030 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc335c67700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300c9e70 con 0x7fc33009a4d0 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc335c67700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300c9260 con 0x7fc3300b2480 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc3300b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc3300b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40172/0 (socket says 172.21.15.5:40172) 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 -- 172.21.15.5:0/910022025 learned_addr learned my addr 172.21.15.5:0/910022025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32f7fe700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc33009a4d0 0x7fc3300c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32effd700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3300b2480 0x7fc3300b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 -- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3300b2480 msgr2=0x7fc3300b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3300b2480 0x7fc3300b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 -- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc33009a4d0 msgr2=0x7fc3300c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc33009a4d0 0x7fc3300c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 -- 172.21.15.5:0/910022025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200094a0 con 0x7fc330161030 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32effd700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3300b2480 0x7fc3300b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc32ffff700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc3300b6da0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc32800cba0 tx=0x7fc32800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc317fff700 1 -- 172.21.15.5:0/910022025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32800aad0 con 0x7fc330161030 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc317fff700 1 -- 172.21.15.5:0/910022025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32800ac50 con 0x7fc330161030 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3300c9540 con 0x7fc330161030 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.756+0000 7fc317fff700 1 -- 172.21.15.5:0/910022025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280070d0 con 0x7fc330161030 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.757+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3300c9b00 con 0x7fc330161030 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.758+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3300ca000 con 0x7fc330161030 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.758+0000 7fc317fff700 1 -- 172.21.15.5:0/910022025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc328007230 con 0x7fc330161030 2024-05-08T23:28:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.759+0000 7fc317fff700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc31c06ac90 0x7fc31c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.759+0000 7fc317fff700 1 -- 172.21.15.5:0/910022025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc328089430 con 0x7fc330161030 2024-05-08T23:28:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.760+0000 7fc32f7fe700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc31c06ac90 0x7fc31c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.760+0000 7fc32f7fe700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc31c06ac90 0x7fc31c06d150 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc3200090b0 tx=0x7fc320008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:46.763+0000 7fc317fff700 1 -- 172.21.15.5:0/910022025 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc3280545f0 con 0x7fc330161030 2024-05-08T23:28:47.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.030+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc330068000 con 0x7fc330161030 2024-05-08T23:28:47.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.031+0000 7fc317fff700 1 -- 172.21.15.5:0/910022025 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7fc328057c10 con 0x7fc330161030 2024-05-08T23:28:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:47.031 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc31c06ac90 msgr2=0x7fc31c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc31c06ac90 0x7fc31c06d150 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc3200090b0 tx=0x7fc320008030 comp rx=0 tx=0).stop 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 msgr2=0x7fc3300b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc3300b6da0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc32800cba0 tx=0x7fc32800ceb0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 shutdown_connections 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc31c06ac90 0x7fc31c06d150 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc330161030 0x7fc3300b6da0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc33009a4d0 0x7fc3300c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 --2- 172.21.15.5:0/910022025 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc3300b2480 0x7fc3300b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 >> 172.21.15.5:0/910022025 conn(0x7fc3300b2f20 msgr2=0x7fc3300cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 shutdown_connections 2024-05-08T23:28:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.033+0000 7fc335c67700 1 -- 172.21.15.5:0/910022025 wait complete. 2024-05-08T23:28:47.035 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:47.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:28:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.184+0000 7f3bd1a79700 1 Processor -- start 2024-05-08T23:28:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.184+0000 7f3bd1a79700 1 -- start start 2024-05-08T23:28:47.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.185+0000 7f3bd1a79700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bcc15ef60 0x7f3bcc0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.185+0000 7f3bd1a79700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3bcc0cf000 0x7f3bcc0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.185+0000 7f3bd1a79700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc0cfe10 con 0x7f3bcc0cf9a0 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.185+0000 7f3bd1a79700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0cd400 con 0x7f3bcc0cf000 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.185+0000 7f3bd1a79700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc09ad40 con 0x7f3bcc15ef60 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.185+0000 7f3bcaffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3bcc0cf000 0x7f3bcc0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bcaffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3bcc0cf000 0x7f3bcc0cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40188/0 (socket says 172.21.15.5:40188) 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bcaffd700 1 -- 172.21.15.5:0/520115484 learned_addr learned my addr 172.21.15.5:0/520115484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bcb7fe700 1 --2- 172.21.15.5:0/520115484 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc15ef60 0x7f3bcc0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bcaffd700 1 -- 172.21.15.5:0/520115484 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc15ef60 msgr2=0x7f3bcc0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bcaffd700 1 --2- 172.21.15.5:0/520115484 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc15ef60 0x7f3bcc0b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bcaffd700 1 -- 172.21.15.5:0/520115484 >> v1:172.21.15.49:6789/0 conn(0x7f3bcc0cf9a0 legacy=0x7f3bcc099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bcaffd700 1 -- 172.21.15.5:0/520115484 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0ccb60 con 0x7f3bcc0cf000 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bca7fc700 1 -- 172.21.15.5:0/520115484 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999835419 0 0) 0x7f3bcc0cfe10 con 0x7f3bcc0cf9a0 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bcaffd700 1 --2- 172.21.15.5:0/520115484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cf000 0x7f3bcc0cf460 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc06e820 tx=0x7f3bc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f9774e4e2bb976 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bca7fc700 1 -- 172.21.15.5:0/520115484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011830 con 0x7f3bcc0cf000 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bca7fc700 1 -- 172.21.15.5:0/520115484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0011990 con 0x7f3bcc0cf000 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.186+0000 7f3bca7fc700 1 -- 172.21.15.5:0/520115484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011d20 con 0x7f3bcc0cf000 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.187+0000 7f3bd1a79700 1 -- 172.21.15.5:0/520115484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cf000 msgr2=0x7f3bcc0cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.187+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/520115484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cf000 0x7f3bcc0cf460 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc06e820 tx=0x7f3bc0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.187+0000 7f3bd1a79700 1 -- 172.21.15.5:0/520115484 shutdown_connections 2024-05-08T23:28:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.187+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/520115484 >> v2:172.21.15.49:3300/0 conn(0x7f3bcc15ef60 0x7f3bcc0b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.187+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/520115484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc0cf000 0x7f3bcc0cf460 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.187+0000 7f3bd1a79700 1 -- 172.21.15.5:0/520115484 >> 172.21.15.5:0/520115484 conn(0x7f3bcc0b2f20 msgr2=0x7f3bcc0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.187+0000 7f3bd1a79700 1 -- 172.21.15.5:0/520115484 shutdown_connections 2024-05-08T23:28:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.187+0000 7f3bd1a79700 1 -- 172.21.15.5:0/520115484 wait complete. 2024-05-08T23:28:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.188+0000 7f3bd1a79700 1 Processor -- start 2024-05-08T23:28:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bd1a79700 1 -- start start 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bd1a79700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc0cf000 0x7f3bcc154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bd1a79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc0cf9a0 0x7f3bcc06cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bd1a79700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15ef60 0x7f3bcc06d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bd1a79700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0ca9a0 con 0x7f3bcc15ef60 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bd1a79700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc155060 con 0x7f3bcc0cf9a0 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bd1a79700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0c8500 con 0x7f3bcc0cf000 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bcbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15ef60 0x7f3bcc06d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bcbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15ef60 0x7f3bcc06d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40196/0 (socket says 172.21.15.5:40196) 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.189+0000 7f3bcbfff700 1 -- 172.21.15.5:0/762861377 learned_addr learned my addr 172.21.15.5:0/762861377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bcaffd700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc0cf9a0 0x7f3bcc06cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bcb7fe700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc0cf000 0x7f3bcc154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bcbfff700 1 -- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc0cf000 msgr2=0x7f3bcc154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bcbfff700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc0cf000 0x7f3bcc154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bcbfff700 1 -- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc0cf9a0 msgr2=0x7f3bcc06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bcbfff700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc0cf9a0 0x7f3bcc06cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bcbfff700 1 -- 172.21.15.5:0/762861377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc0009550 con 0x7f3bcc15ef60 2024-05-08T23:28:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bcbfff700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15ef60 0x7f3bcc06d000 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c7a0 tx=0x7f3bc400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bb3fff700 1 -- 172.21.15.5:0/762861377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4003e30 con 0x7f3bcc15ef60 2024-05-08T23:28:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bb3fff700 1 -- 172.21.15.5:0/762861377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc4005070 con 0x7f3bcc15ef60 2024-05-08T23:28:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc0c9510 con 0x7f3bcc15ef60 2024-05-08T23:28:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bb3fff700 1 -- 172.21.15.5:0/762861377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4005420 con 0x7f3bcc15ef60 2024-05-08T23:28:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.190+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc0c8a30 con 0x7f3bcc15ef60 2024-05-08T23:28:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.191+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc068000 con 0x7f3bcc15ef60 2024-05-08T23:28:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.192+0000 7f3bb3fff700 1 -- 172.21.15.5:0/762861377 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3bc4005580 con 0x7f3bcc15ef60 2024-05-08T23:28:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.192+0000 7f3bb3fff700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb806ac90 0x7f3bb806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.192+0000 7f3bb3fff700 1 -- 172.21.15.5:0/762861377 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bc4088a20 con 0x7f3bcc15ef60 2024-05-08T23:28:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.193+0000 7f3bcb7fe700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb806ac90 0x7f3bb806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.193+0000 7f3bcb7fe700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb806ac90 0x7f3bb806d150 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009be0 tx=0x7f3bbc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.197+0000 7f3bb3fff700 1 -- 172.21.15.5:0/762861377 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3bc4057570 con 0x7f3bcc15ef60 2024-05-08T23:28:47.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.421+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bcc0b6bb0 con 0x7f3bcc15ef60 2024-05-08T23:28:47.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.422+0000 7f3bb3fff700 1 -- 172.21.15.5:0/762861377 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+15944 (secure 0 0 0) 0x7f3bc4017090 con 0x7f3bcc15ef60 2024-05-08T23:28:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:47.422 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:28:00.273637+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:27:57.260067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T23:27:58.243420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb806ac90 msgr2=0x7f3bb806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb806ac90 0x7f3bb806d150 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009be0 tx=0x7f3bbc009210 comp rx=0 tx=0).stop 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15ef60 msgr2=0x7f3bcc06d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15ef60 0x7f3bcc06d000 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c7a0 tx=0x7f3bc400cab0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 shutdown_connections 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3bb806ac90 0x7f3bb806d150 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bcc15ef60 0x7f3bcc06d000 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3bcc0cf000 0x7f3bcc154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 --2- 172.21.15.5:0/762861377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bcc0cf9a0 0x7f3bcc06cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.424+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 >> 172.21.15.5:0/762861377 conn(0x7f3bcc0b2f20 msgr2=0x7f3bcc0b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.425+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 shutdown_connections 2024-05-08T23:28:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.425+0000 7f3bd1a79700 1 -- 172.21.15.5:0/762861377 wait complete. 2024-05-08T23:28:47.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:20:57.351375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:27:57.260067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:27:58.243420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:28:47.439 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:28:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.579+0000 7f3666a68700 1 Processor -- start 2024-05-08T23:28:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.579+0000 7f3666a68700 1 -- start start 2024-05-08T23:28:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3666a68700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f36600af8e0 0x7f366015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3666a68700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36600b3d00 0x7f36600b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3666a68700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36600b46d0 con 0x7f366015a7c0 2024-05-08T23:28:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3666a68700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36600aecf0 con 0x7f36600b3d00 2024-05-08T23:28:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3666a68700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36600b4950 con 0x7f36600af8e0 2024-05-08T23:28:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3665005700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36600b3d00 0x7f36600b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3665005700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36600b3d00 0x7f36600b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43996/0 (socket says 172.21.15.5:43996) 2024-05-08T23:28:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3665005700 1 -- 172.21.15.5:0/687927334 learned_addr learned my addr 172.21.15.5:0/687927334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f365ffff700 1 --2- 172.21.15.5:0/687927334 >> v2:172.21.15.49:3300/0 conn(0x7f36600af8e0 0x7f366015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3665005700 1 -- 172.21.15.5:0/687927334 >> v2:172.21.15.49:3300/0 conn(0x7f36600af8e0 msgr2=0x7f366015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3665005700 1 --2- 172.21.15.5:0/687927334 >> v2:172.21.15.49:3300/0 conn(0x7f36600af8e0 0x7f366015aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3665005700 1 -- 172.21.15.5:0/687927334 >> v1:172.21.15.5:6789/0 conn(0x7f366015a7c0 legacy=0x7f36600af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.580+0000 7f3665005700 1 -- 172.21.15.5:0/687927334 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366006de70 con 0x7f36600b3d00 2024-05-08T23:28:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f3665005700 1 --2- 172.21.15.5:0/687927334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600b3d00 0x7f36600b4190 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f36600718f0 tx=0x7f365800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6712cf566b8ae47a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f365f7fe700 1 -- 172.21.15.5:0/687927334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800e070 con 0x7f36600b3d00 2024-05-08T23:28:47.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f365f7fe700 1 -- 172.21.15.5:0/687927334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800b9c0 con 0x7f36600b3d00 2024-05-08T23:28:47.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f365f7fe700 1 -- 172.21.15.5:0/687927334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800bd70 con 0x7f36600b3d00 2024-05-08T23:28:47.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f3666a68700 1 -- 172.21.15.5:0/687927334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600b3d00 msgr2=0x7f36600b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f3666a68700 1 --2- 172.21.15.5:0/687927334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600b3d00 0x7f36600b4190 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f36600718f0 tx=0x7f365800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:47.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f3666a68700 1 -- 172.21.15.5:0/687927334 shutdown_connections 2024-05-08T23:28:47.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f3666a68700 1 --2- 172.21.15.5:0/687927334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600b3d00 0x7f36600b4190 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f3666a68700 1 --2- 172.21.15.5:0/687927334 >> v2:172.21.15.49:3300/0 conn(0x7f36600af8e0 0x7f366015aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.581+0000 7f3666a68700 1 -- 172.21.15.5:0/687927334 >> 172.21.15.5:0/687927334 conn(0x7f36600b5810 msgr2=0x7f36600b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.582+0000 7f3666a68700 1 -- 172.21.15.5:0/687927334 shutdown_connections 2024-05-08T23:28:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.582+0000 7f3666a68700 1 -- 172.21.15.5:0/687927334 wait complete. 2024-05-08T23:28:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.583+0000 7f3666a68700 1 Processor -- start 2024-05-08T23:28:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.583+0000 7f3666a68700 1 -- start start 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3666a68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600af8e0 0x7f3660141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3666a68700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36600b3d00 0x7f366009b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3666a68700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f366015a7c0 0x7f366009d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3666a68700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36600cd480 con 0x7f36600af8e0 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3666a68700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601f1eb0 con 0x7f36600b3d00 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3666a68700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36600ccae0 con 0x7f366015a7c0 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3664804700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600af8e0 0x7f3660141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3664804700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600af8e0 0x7f3660141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44006/0 (socket says 172.21.15.5:44006) 2024-05-08T23:28:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3664804700 1 -- 172.21.15.5:0/2262142819 learned_addr learned my addr 172.21.15.5:0/2262142819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f365ffff700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36600b3d00 0x7f366009b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3665005700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f366015a7c0 0x7f366009d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3664804700 1 -- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f366015a7c0 msgr2=0x7f366009d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3664804700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f366015a7c0 0x7f366009d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3664804700 1 -- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36600b3d00 msgr2=0x7f366009b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3664804700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36600b3d00 0x7f366009b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f3664804700 1 -- 172.21.15.5:0/2262142819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009b30 con 0x7f36600af8e0 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.584+0000 7f365ffff700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36600b3d00 0x7f366009b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.585+0000 7f3664804700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600af8e0 0x7f3660141d80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f365400c450 tx=0x7f365400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.585+0000 7f365d7fa700 1 -- 172.21.15.5:0/2262142819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540052d0 con 0x7f36600af8e0 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.585+0000 7f365d7fa700 1 -- 172.21.15.5:0/2262142819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654005430 con 0x7f36600af8e0 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.585+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36600ccdc0 con 0x7f36600af8e0 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.585+0000 7f365d7fa700 1 -- 172.21.15.5:0/2262142819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540038c0 con 0x7f36600af8e0 2024-05-08T23:28:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.585+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36601422c0 con 0x7f36600af8e0 2024-05-08T23:28:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.586+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3660074bc0 con 0x7f36600af8e0 2024-05-08T23:28:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.586+0000 7f365d7fa700 1 -- 172.21.15.5:0/2262142819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3654003a20 con 0x7f36600af8e0 2024-05-08T23:28:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.587+0000 7f365d7fa700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f364406ac90 0x7f364406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.587+0000 7f365d7fa700 1 -- 172.21.15.5:0/2262142819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3654088640 con 0x7f36600af8e0 2024-05-08T23:28:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.587+0000 7f365ffff700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f364406ac90 0x7f364406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.587+0000 7f365ffff700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f364406ac90 0x7f364406d150 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f364c009a00 tx=0x7f364c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.591+0000 7f365d7fa700 1 -- 172.21.15.5:0/2262142819 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3654053830 con 0x7f36600af8e0 2024-05-08T23:28:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.806+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36600ae470 con 0x7f36600af8e0 2024-05-08T23:28:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.806+0000 7f365d7fa700 1 -- 172.21.15.5:0/2262142819 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3654056e50 con 0x7f36600af8e0 2024-05-08T23:28:47.806 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.808+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f364406ac90 msgr2=0x7f364406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.808+0000 7f3666a68700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f364406ac90 0x7f364406d150 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f364c009a00 tx=0x7f364c008040 comp rx=0 tx=0).stop 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.808+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600af8e0 msgr2=0x7f3660141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.808+0000 7f3666a68700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600af8e0 0x7f3660141d80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f365400c450 tx=0x7f365400c810 comp rx=0 tx=0).stop 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.809+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 shutdown_connections 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.809+0000 7f3666a68700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f364406ac90 0x7f364406d150 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.809+0000 7f3666a68700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f366015a7c0 0x7f366009d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.809+0000 7f3666a68700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36600af8e0 0x7f3660141d80 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.809+0000 7f3666a68700 1 --2- 172.21.15.5:0/2262142819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36600b3d00 0x7f366009b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.809+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 >> 172.21.15.5:0/2262142819 conn(0x7f36600b5810 msgr2=0x7f36600b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:47.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.809+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 shutdown_connections 2024-05-08T23:28:47.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.809+0000 7f3666a68700 1 -- 172.21.15.5:0/2262142819 wait complete. 2024-05-08T23:28:47.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:28:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.962+0000 7fb42ed79700 1 Processor -- start 2024-05-08T23:28:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.962+0000 7fb42ed79700 1 -- start start 2024-05-08T23:28:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.963+0000 7fb42ed79700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb42815b4e0 0x7fb4280b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.963+0000 7fb42ed79700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb42815b8c0 0x7fb4280d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.963+0000 7fb42ed79700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4280cf560 con 0x7fb4280d1740 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.963+0000 7fb42ed79700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4280cefe0 con 0x7fb42815b4e0 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.963+0000 7fb42ed79700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280d1bb0 con 0x7fb42815b8c0 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.963+0000 7fb42d316700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb4280d1740 0x7fb4280cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34616/0 (socket says 172.21.15.5:34616) 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.963+0000 7fb42d316700 1 -- 172.21.15.5:0/39842058 learned_addr learned my addr 172.21.15.5:0/39842058 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb42cb15700 1 --2- 172.21.15.5:0/39842058 >> v2:172.21.15.49:3300/0 conn(0x7fb42815b4e0 0x7fb4280b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb427fff700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb42815b8c0 0x7fb4280d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266845232 0 0) 0x7fb4280cf560 con 0x7fb4280d1740 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4140035b0 con 0x7fb4280d1740 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2410712724 0 0) 0x7fb4140035b0 con 0x7fb4280d1740 2024-05-08T23:28:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb42815b8c0 msgr2=0x7fb4280d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb42815b8c0 0x7fb4280d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 >> v2:172.21.15.49:3300/0 conn(0x7fb42815b4e0 msgr2=0x7fb4280b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 --2- 172.21.15.5:0/39842058 >> v2:172.21.15.49:3300/0 conn(0x7fb42815b4e0 0x7fb4280b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280ccff0 con 0x7fb4280d1740 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fb4200019f0 con 0x7fb4280d1740 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb42cb15700 1 --2- 172.21.15.5:0/39842058 >> v2:172.21.15.49:3300/0 conn(0x7fb42815b4e0 0x7fb4280b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 >> v1:172.21.15.5:6789/0 conn(0x7fb4280d1740 legacy=0x7fb4280cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb414003a00 0x7fb414003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb42815b8c0 0x7fb414004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414004d90 0x7fb414005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280cefe0 con 0x7fb414003a00 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280d1bb0 con 0x7fb414004d90 2024-05-08T23:28:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41400ba10 con 0x7fb42815b8c0 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb42cb15700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb414003a00 0x7fb414003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb42d316700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414004d90 0x7fb414005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.964+0000 7fb427fff700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb42815b8c0 0x7fb414004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb42cb15700 1 -- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb42815b8c0 msgr2=0x7fb414004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb42cb15700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb42815b8c0 0x7fb414004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb42cb15700 1 -- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414004d90 msgr2=0x7fb414005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb42cb15700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414004d90 0x7fb414005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb42cb15700 1 -- 172.21.15.5:0/39842058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4140035b0 con 0x7fb414003a00 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb42cb15700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb414003a00 0x7fb414003e20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb418009d30 tx=0x7fb414005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f160078ae9a218e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418009e80 con 0x7fb414003a00 2024-05-08T23:28:47.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41800e9a0 con 0x7fb414003a00 2024-05-08T23:28:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.965+0000 7fb4277fe700 1 -- 172.21.15.5:0/39842058 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41800ed30 con 0x7fb414003a00 2024-05-08T23:28:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.966+0000 7fb42ed79700 1 -- 172.21.15.5:0/39842058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb414003a00 msgr2=0x7fb414003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.966+0000 7fb42ed79700 1 --2- 172.21.15.5:0/39842058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb414003a00 0x7fb414003e20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb418009d30 tx=0x7fb414005880 comp rx=0 tx=0).stop 2024-05-08T23:28:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.966+0000 7fb42d316700 1 -- 172.21.15.5:0/39842058 reap_dead start 2024-05-08T23:28:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.966+0000 7fb42ed79700 1 -- 172.21.15.5:0/39842058 shutdown_connections 2024-05-08T23:28:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.966+0000 7fb42ed79700 1 -- 172.21.15.5:0/39842058 >> 172.21.15.5:0/39842058 conn(0x7fb42808afe0 msgr2=0x7fb42808b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:47.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.966+0000 7fb42ed79700 1 -- 172.21.15.5:0/39842058 shutdown_connections 2024-05-08T23:28:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.966+0000 7fb42ed79700 1 -- 172.21.15.5:0/39842058 wait complete. 2024-05-08T23:28:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.967+0000 7fb42ed79700 1 Processor -- start 2024-05-08T23:28:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.967+0000 7fb42ed79700 1 -- start start 2024-05-08T23:28:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42ed79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414003a00 0x7fb42815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42ed79700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb4281f4e90 0x7fb4281f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42ed79700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4280af630 0x7fb4280afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42ed79700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42814c330 con 0x7fb4280af630 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42ed79700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280afff0 con 0x7fb414003a00 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42ed79700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42815d210 con 0x7fb4281f4e90 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42d316700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4280af630 0x7fb4280afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42d316700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4280af630 0x7fb4280afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44034/0 (socket says 172.21.15.5:44034) 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42d316700 1 -- 172.21.15.5:0/1107994144 learned_addr learned my addr 172.21.15.5:0/1107994144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb427fff700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb4281f4e90 0x7fb4281f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42cb15700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414003a00 0x7fb42815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42d316700 1 -- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb4281f4e90 msgr2=0x7fb4281f06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42d316700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb4281f4e90 0x7fb4281f06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42d316700 1 -- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414003a00 msgr2=0x7fb42815c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42d316700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414003a00 0x7fb42815c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.968+0000 7fb42d316700 1 -- 172.21.15.5:0/1107994144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42815c690 con 0x7fb4280af630 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.969+0000 7fb42d316700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4280af630 0x7fb4280afab0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb4280cef60 tx=0x7fb42000a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.969+0000 7fb4257fa700 1 -- 172.21.15.5:0/1107994144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4200133f0 con 0x7fb4280af630 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.969+0000 7fb4257fa700 1 -- 172.21.15.5:0/1107994144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb420013550 con 0x7fb4280af630 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.969+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42815c970 con 0x7fb4280af630 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.969+0000 7fb4257fa700 1 -- 172.21.15.5:0/1107994144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420013900 con 0x7fb4280af630 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.969+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42815cf30 con 0x7fb4280af630 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.970+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4280c5ff0 con 0x7fb4280af630 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.970+0000 7fb4257fa700 1 -- 172.21.15.5:0/1107994144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb420013a60 con 0x7fb4280af630 2024-05-08T23:28:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.971+0000 7fb4257fa700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb40806ac90 0x7fb40806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.971+0000 7fb4257fa700 1 -- 172.21.15.5:0/1107994144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb420089cd0 con 0x7fb4280af630 2024-05-08T23:28:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.971+0000 7fb42cb15700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb40806ac90 0x7fb40806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.972+0000 7fb42cb15700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb40806ac90 0x7fb40806d150 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb418000c00 tx=0x7fb418008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:47.976+0000 7fb4257fa700 1 -- 172.21.15.5:0/1107994144 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb4280c5ff0 con 0x7fb4280af630 2024-05-08T23:28:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.191+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4280c5ff0 con 0x7fb4280af630 2024-05-08T23:28:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.192+0000 7fb4257fa700 1 -- 172.21.15.5:0/1107994144 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4280c5ff0 con 0x7fb4280af630 2024-05-08T23:28:48.192 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-08T23:28:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb40806ac90 msgr2=0x7fb40806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb40806ac90 0x7fb40806d150 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb418000c00 tx=0x7fb418008030 comp rx=0 tx=0).stop 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4280af630 msgr2=0x7fb4280afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4280af630 0x7fb4280afab0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb4280cef60 tx=0x7fb42000a640 comp rx=0 tx=0).stop 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 shutdown_connections 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fb40806ac90 0x7fb40806d150 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4280af630 0x7fb4280afab0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb414003a00 0x7fb42815c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 --2- 172.21.15.5:0/1107994144 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb4281f4e90 0x7fb4281f06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 >> 172.21.15.5:0/1107994144 conn(0x7fb42808afe0 msgr2=0x7fb428155b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 shutdown_connections 2024-05-08T23:28:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.194+0000 7fb42ed79700 1 -- 172.21.15.5:0/1107994144 wait complete. 2024-05-08T23:28:48.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:28:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.348+0000 7f89b9865700 1 Processor -- start 2024-05-08T23:28:48.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.348+0000 7f89b9865700 1 -- start start 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.348+0000 7f89b9865700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f89b4159da0 0x7f89b415d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b9865700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b406ede0 0x7f89b406afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b9865700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b415e6b0 con 0x7f89b415ac70 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b9865700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b415b0f0 con 0x7f89b4159da0 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b9865700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b415de10 con 0x7f89b406ede0 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89abfff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f89b415ac70 0x7f89b406e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34632/0 (socket says 172.21.15.5:34632) 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89abfff700 1 -- 172.21.15.5:0/3577485802 learned_addr learned my addr 172.21.15.5:0/3577485802 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b2ffd700 1 --2- 172.21.15.5:0/3577485802 >> v2:172.21.15.49:3300/0 conn(0x7f89b4159da0 0x7f89b415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b37fe700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b406ede0 0x7f89b406afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912049839 0 0) 0x7f89b415e6b0 con 0x7f89b415ac70 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a00035b0 con 0x7f89b415ac70 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1043021754 0 0) 0x7f89a00035b0 con 0x7f89b415ac70 2024-05-08T23:28:48.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b406ede0 msgr2=0x7f89b406afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b406ede0 0x7f89b406afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 >> v2:172.21.15.49:3300/0 conn(0x7f89b4159da0 msgr2=0x7f89b415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 --2- 172.21.15.5:0/3577485802 >> v2:172.21.15.49:3300/0 conn(0x7f89b4159da0 0x7f89b415d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b415e0f0 con 0x7f89b415ac70 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f899c001670 con 0x7f89b415ac70 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 >> v1:172.21.15.5:6789/0 conn(0x7f89b415ac70 legacy=0x7f89b406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a0003a00 0x7f89a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b406ede0 0x7f89a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0004d90 0x7f89a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b415b0f0 con 0x7f89a0003a00 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b415de10 con 0x7f89a0004d90 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.349+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a000bb70 con 0x7f89b406ede0 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b2ffd700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a0003a00 0x7f89a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89abfff700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b406ede0 0x7f89a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b37fe700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0004d90 0x7f89a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b2ffd700 1 -- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b406ede0 msgr2=0x7f89a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b2ffd700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b406ede0 0x7f89a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b2ffd700 1 -- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0004d90 msgr2=0x7f89a0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b2ffd700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0004d90 0x7f89a0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b2ffd700 1 -- 172.21.15.5:0/3577485802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00035b0 con 0x7f89a0003a00 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b37fe700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0004d90 0x7f89a0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b2ffd700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a0003a00 0x7f89a0003e20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f89a400be60 tx=0x7f89a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e7a0ab5a2c9faf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4012040 con 0x7f89a0003a00 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a400ddc0 con 0x7f89a0003a00 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.350+0000 7f89b27fc700 1 -- 172.21.15.5:0/3577485802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a40185f0 con 0x7f89a0003a00 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.351+0000 7f89b9865700 1 -- 172.21.15.5:0/3577485802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a0003a00 msgr2=0x7f89a0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.351+0000 7f89b9865700 1 --2- 172.21.15.5:0/3577485802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a0003a00 0x7f89a0003e20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f89a400be60 tx=0x7f89a0005880 comp rx=0 tx=0).stop 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.351+0000 7f89b37fe700 1 -- 172.21.15.5:0/3577485802 reap_dead start 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.351+0000 7f89b9865700 1 -- 172.21.15.5:0/3577485802 shutdown_connections 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.351+0000 7f89b9865700 1 -- 172.21.15.5:0/3577485802 >> 172.21.15.5:0/3577485802 conn(0x7f89b40cfce0 msgr2=0x7f89b40d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.351+0000 7f89b9865700 1 -- 172.21.15.5:0/3577485802 shutdown_connections 2024-05-08T23:28:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.351+0000 7f89b9865700 1 -- 172.21.15.5:0/3577485802 wait complete. 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.352+0000 7f89b9865700 1 Processor -- start 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.352+0000 7f89b9865700 1 -- start start 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b9865700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0003a00 0x7f89b41f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b9865700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b41e4240 0x7f89b413c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b9865700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b415b230 0x7f89b415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b9865700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4150a10 con 0x7f89b415b230 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b9865700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b415bbf0 con 0x7f89a0003a00 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b9865700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41e52a0 con 0x7f89b41e4240 2024-05-08T23:28:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b415b230 0x7f89b415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b415b230 0x7f89b415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44046/0 (socket says 172.21.15.5:44046) 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.353+0000 7f89b37fe700 1 -- 172.21.15.5:0/1721317406 learned_addr learned my addr 172.21.15.5:0/1721317406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89abfff700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b41e4240 0x7f89b413c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b2ffd700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0003a00 0x7f89b41f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b37fe700 1 -- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b41e4240 msgr2=0x7f89b413c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b37fe700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b41e4240 0x7f89b413c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b37fe700 1 -- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0003a00 msgr2=0x7f89b41f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b37fe700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0003a00 0x7f89b41f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b37fe700 1 -- 172.21.15.5:0/1721317406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b41e4a00 con 0x7f89b415b230 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b37fe700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b415b230 0x7f89b415b6b0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f89b40b2bc0 tx=0x7f89ac0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b0ff9700 1 -- 172.21.15.5:0/1721317406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac00e050 con 0x7f89b415b230 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b0ff9700 1 -- 172.21.15.5:0/1721317406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac00f040 con 0x7f89b415b230 2024-05-08T23:28:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b41e4ce0 con 0x7f89b415b230 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b0ff9700 1 -- 172.21.15.5:0/1721317406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac010db0 con 0x7f89b415b230 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.354+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b41e46c0 con 0x7f89b415b230 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.355+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b4150b50 con 0x7f89b415b230 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.356+0000 7f89b0ff9700 1 -- 172.21.15.5:0/1721317406 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f89ac015440 con 0x7f89b415b230 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.356+0000 7f89b0ff9700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f899406ac90 0x7f899406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.356+0000 7f89b0ff9700 1 -- 172.21.15.5:0/1721317406 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89ac089460 con 0x7f89b415b230 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.356+0000 7f89b2ffd700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f899406ac90 0x7f899406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.357+0000 7f89b2ffd700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f899406ac90 0x7f899406d150 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f89a4000c00 tx=0x7f89a4013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.361+0000 7f89b0ff9700 1 -- 172.21.15.5:0/1721317406 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f89b4150b50 con 0x7f89b415b230 2024-05-08T23:28:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.578+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89b4150b50 con 0x7f89b415b230 2024-05-08T23:28:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.579+0000 7f89b0ff9700 1 -- 172.21.15.5:0/1721317406 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89b4150b50 con 0x7f89b415b230 2024-05-08T23:28:48.579 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-08T23:28:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f899406ac90 msgr2=0x7f899406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f899406ac90 0x7f899406d150 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f89a4000c00 tx=0x7f89a4013000 comp rx=0 tx=0).stop 2024-05-08T23:28:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b415b230 msgr2=0x7f89b415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b415b230 0x7f89b415b6b0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f89b40b2bc0 tx=0x7f89ac0104a0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 shutdown_connections 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f899406ac90 0x7f899406d150 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b415b230 0x7f89b415b6b0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89a0003a00 0x7f89b41f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 --2- 172.21.15.5:0/1721317406 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f89b41e4240 0x7f89b413c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 >> 172.21.15.5:0/1721317406 conn(0x7f89b40cfce0 msgr2=0x7f89b415a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 shutdown_connections 2024-05-08T23:28:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.581+0000 7f89b9865700 1 -- 172.21.15.5:0/1721317406 wait complete. 2024-05-08T23:28:48.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.731+0000 7f450c0be700 1 Processor -- start 2024-05-08T23:28:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.731+0000 7f450c0be700 1 -- start start 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.732+0000 7f450c0be700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f45040ced90 0x7f45040d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.732+0000 7f450c0be700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45040d40b0 con 0x7f45040c6990 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.732+0000 7f450c0be700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45040d0f50 con 0x7f45040d0850 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.732+0000 7f450c0be700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45040d25e0 con 0x7f45040ced90 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.732+0000 7f4509e5a700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f45040c6990 0x7f45040d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34642/0 (socket says 172.21.15.5:34642) 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.732+0000 7f4509e5a700 1 -- 172.21.15.5:0/1411126140 learned_addr learned my addr 172.21.15.5:0/1411126140 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.732+0000 7f450a65b700 1 --2- 172.21.15.5:0/1411126140 >> v2:172.21.15.49:3300/0 conn(0x7f45040ced90 0x7f45040d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.732+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663270887 0 0) 0x7f45040d40b0 con 0x7f45040c6990 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44f00035b0 con 0x7f45040c6990 2024-05-08T23:28:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102514257 0 0) 0x7f45040d0f50 con 0x7f45040d0850 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45040d40b0 con 0x7f45040d0850 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3271306201 0 0) 0x7f44f00035b0 con 0x7f45040c6990 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 >> v2:172.21.15.49:3300/0 conn(0x7f45040ced90 msgr2=0x7f45040d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 --2- 172.21.15.5:0/1411126140 >> v2:172.21.15.49:3300/0 conn(0x7f45040ced90 0x7f45040d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 >> v1:172.21.15.49:6789/0 conn(0x7f45040d0850 legacy=0x7f45040ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040d0ce0 con 0x7f45040c6990 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f4500001670 con 0x7f45040c6990 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 >> v1:172.21.15.5:6789/0 conn(0x7f45040c6990 legacy=0x7f45040d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f44f0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44f0004850 0x7f44f0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44f0005210 0x7f44f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d25e0 con 0x7f44f0003b90 2024-05-08T23:28:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f000c090 con 0x7f44f0004850 2024-05-08T23:28:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f000c250 con 0x7f44f0005210 2024-05-08T23:28:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4509e5a700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f44f0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4509659700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44f0004850 0x7f44f0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f450a65b700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44f0005210 0x7f44f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4509e5a700 1 -- 172.21.15.5:0/1411126140 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44f0005210 msgr2=0x7f44f000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4509e5a700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f44f0005210 0x7f44f000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4509e5a700 1 -- 172.21.15.5:0/1411126140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44f0004850 msgr2=0x7f44f0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4509e5a700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44f0004850 0x7f44f0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f4509e5a700 1 -- 172.21.15.5:0/1411126140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f00035b0 con 0x7f44f0003b90 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.733+0000 7f450a65b700 1 -- 172.21.15.5:0/1411126140 reap_dead start 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f4509e5a700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f44f0003f90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f450000c240 tx=0x7f44f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae9e08155808a90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000cd80 con 0x7f44f0003b90 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500005960 con 0x7f44f0003b90 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f4508e58700 1 -- 172.21.15.5:0/1411126140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500005d10 con 0x7f44f0003b90 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f450c0be700 1 -- 172.21.15.5:0/1411126140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 msgr2=0x7f44f0003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f450c0be700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f44f0003f90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f450000c240 tx=0x7f44f0005880 comp rx=0 tx=0).stop 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f450c0be700 1 -- 172.21.15.5:0/1411126140 shutdown_connections 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f450c0be700 1 --2- 172.21.15.5:0/1411126140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f44f0003f90 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.734+0000 7f450c0be700 1 -- 172.21.15.5:0/1411126140 >> 172.21.15.5:0/1411126140 conn(0x7f450409c950 msgr2=0x7f450408afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.735+0000 7f450c0be700 1 -- 172.21.15.5:0/1411126140 shutdown_connections 2024-05-08T23:28:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.735+0000 7f450c0be700 1 -- 172.21.15.5:0/1411126140 wait complete. 2024-05-08T23:28:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.736+0000 7f450c0be700 1 Processor -- start 2024-05-08T23:28:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.736+0000 7f450c0be700 1 -- start start 2024-05-08T23:28:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f450c0be700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f45040c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f450c0be700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450415b7c0 0x7f450415bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f450c0be700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45041fad00 0x7f45041fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f450c0be700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45041fba70 con 0x7f44f0003b90 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f450c0be700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450415c180 con 0x7f45041fad00 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f450c0be700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040b5270 con 0x7f450415b7c0 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f45040c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f45040c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44072/0 (socket says 172.21.15.5:44072) 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 -- 172.21.15.5:0/550592015 learned_addr learned my addr 172.21.15.5:0/550592015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509659700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450415b7c0 0x7f450415bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f450a65b700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45041fad00 0x7f45041fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 -- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450415b7c0 msgr2=0x7f450415bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450415b7c0 0x7f450415bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 -- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45041fad00 msgr2=0x7f45041fb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45041fad00 0x7f45041fb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 -- 172.21.15.5:0/550592015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040b4940 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.737+0000 7f4509e5a700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f45040c96f0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f450000c830 tx=0x7f45040d24e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.738+0000 7f44fa7fc700 1 -- 172.21.15.5:0/550592015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500004c80 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.738+0000 7f44fa7fc700 1 -- 172.21.15.5:0/550592015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500004de0 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.738+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45040b4c20 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.738+0000 7f44fa7fc700 1 -- 172.21.15.5:0/550592015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000072f0 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.738+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45040b5700 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.739+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45040b5550 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.739+0000 7f44fa7fc700 1 -- 172.21.15.5:0/550592015 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4500001a80 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.739+0000 7f44fa7fc700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44e806ac90 0x7f44e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.740+0000 7f44fa7fc700 1 -- 172.21.15.5:0/550592015 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4500089d00 con 0x7f44f0003b90 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.740+0000 7f4509659700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44e806ac90 0x7f44e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.740+0000 7f4509659700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44e806ac90 0x7f44e806d150 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f45040b6040 tx=0x7f44f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:48.744+0000 7f44fa7fc700 1 -- 172.21.15.5:0/550592015 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4500054f70 con 0x7f44f0003b90 2024-05-08T23:28:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.010+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4504068000 con 0x7f44f0003b90 2024-05-08T23:28:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.011+0000 7f44fa7fc700 1 -- 172.21.15.5:0/550592015 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7f4500058590 con 0x7f44f0003b90 2024-05-08T23:28:49.011 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:49.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44e806ac90 msgr2=0x7f44e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44e806ac90 0x7f44e806d150 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f45040b6040 tx=0x7f44f4009040 comp rx=0 tx=0).stop 2024-05-08T23:28:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 msgr2=0x7f45040c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f45040c96f0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f450000c830 tx=0x7f45040d24e0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 shutdown_connections 2024-05-08T23:28:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44e806ac90 0x7f44e806d150 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45041fad00 0x7f45041fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f0003b90 0x7f45040c96f0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 --2- 172.21.15.5:0/550592015 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450415b7c0 0x7f450415bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 >> 172.21.15.5:0/550592015 conn(0x7f450409c950 msgr2=0x7f4504163560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 shutdown_connections 2024-05-08T23:28:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.013+0000 7f450c0be700 1 -- 172.21.15.5:0/550592015 wait complete. 2024-05-08T23:28:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:49.027 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:28:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.163+0000 7fdb6f860700 1 Processor -- start 2024-05-08T23:28:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.163+0000 7fdb6f860700 1 -- start start 2024-05-08T23:28:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.164+0000 7fdb6f860700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdb680d3f40 0x7fdb68161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.164+0000 7fdb6f860700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815db80 0x7fdb680cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.164+0000 7fdb6f860700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb68162450 con 0x7fdb6815d7a0 2024-05-08T23:28:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.164+0000 7fdb6f860700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb680cee00 con 0x7fdb680d3f40 2024-05-08T23:28:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.164+0000 7fdb6f860700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680b27e0 con 0x7fdb6815db80 2024-05-08T23:28:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.164+0000 7fdb6cdfb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdb680d3f40 0x7fdb68161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.164+0000 7fdb6cdfb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdb680d3f40 0x7fdb68161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44082/0 (socket says 172.21.15.5:44082) 2024-05-08T23:28:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.164+0000 7fdb6cdfb700 1 -- 172.21.15.5:0/3988608215 learned_addr learned my addr 172.21.15.5:0/3988608215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb6ddfd700 1 --2- 172.21.15.5:0/3988608215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815db80 0x7fdb680cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb6cdfb700 1 -- 172.21.15.5:0/3988608215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815db80 msgr2=0x7fdb680cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb6cdfb700 1 --2- 172.21.15.5:0/3988608215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815db80 0x7fdb680cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb6cdfb700 1 -- 172.21.15.5:0/3988608215 >> v1:172.21.15.49:6789/0 conn(0x7fdb6815d7a0 legacy=0x7fdb680b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb6cdfb700 1 -- 172.21.15.5:0/3988608215 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb680d4400 con 0x7fdb680d3f40 2024-05-08T23:28:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb6ddfd700 1 --2- 172.21.15.5:0/3988608215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815db80 0x7fdb680cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb6cdfb700 1 --2- 172.21.15.5:0/3988608215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb68161f10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fdb58006fd0 tx=0x7fdb58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3048f24f4950ab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb5ffff700 1 -- 172.21.15.5:0/3988608215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58005e30 con 0x7fdb680d3f40 2024-05-08T23:28:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb5ffff700 1 -- 172.21.15.5:0/3988608215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb58006030 con 0x7fdb680d3f40 2024-05-08T23:28:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.165+0000 7fdb5ffff700 1 -- 172.21.15.5:0/3988608215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580063c0 con 0x7fdb680d3f40 2024-05-08T23:28:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.166+0000 7fdb6f860700 1 -- 172.21.15.5:0/3988608215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 msgr2=0x7fdb68161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.166+0000 7fdb6f860700 1 --2- 172.21.15.5:0/3988608215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb68161f10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fdb58006fd0 tx=0x7fdb58009bd0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.166+0000 7fdb6f860700 1 -- 172.21.15.5:0/3988608215 shutdown_connections 2024-05-08T23:28:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.166+0000 7fdb6f860700 1 --2- 172.21.15.5:0/3988608215 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815db80 0x7fdb680cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.166+0000 7fdb6f860700 1 --2- 172.21.15.5:0/3988608215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb68161f10 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.166+0000 7fdb6f860700 1 -- 172.21.15.5:0/3988608215 >> 172.21.15.5:0/3988608215 conn(0x7fdb6808afe0 msgr2=0x7fdb680d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.166+0000 7fdb6f860700 1 -- 172.21.15.5:0/3988608215 shutdown_connections 2024-05-08T23:28:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.166+0000 7fdb6f860700 1 -- 172.21.15.5:0/3988608215 wait complete. 2024-05-08T23:28:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.168+0000 7fdb6f860700 1 Processor -- start 2024-05-08T23:28:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.168+0000 7fdb6f860700 1 -- start start 2024-05-08T23:28:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6f860700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb680ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6f860700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815d7a0 0x7fdb681609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6f860700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6815db80 0x7fdb6814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6f860700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68160ee0 con 0x7fdb680d3f40 2024-05-08T23:28:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6f860700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6814e000 con 0x7fdb6815db80 2024-05-08T23:28:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6f860700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680b2a50 con 0x7fdb6815d7a0 2024-05-08T23:28:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb680ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb680ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44084/0 (socket says 172.21.15.5:44084) 2024-05-08T23:28:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 -- 172.21.15.5:0/151937758 learned_addr learned my addr 172.21.15.5:0/151937758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6cdfb700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815d7a0 0x7fdb681609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6ddfd700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6815db80 0x7fdb6814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 -- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815d7a0 msgr2=0x7fdb681609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815d7a0 0x7fdb681609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 -- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6815db80 msgr2=0x7fdb6814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6815db80 0x7fdb6814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 -- 172.21.15.5:0/151937758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58009500 con 0x7fdb680d3f40 2024-05-08T23:28:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.169+0000 7fdb6d5fc700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb680ca810 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400b4a0 tx=0x7fdb6400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.170+0000 7fdb5dffb700 1 -- 172.21.15.5:0/151937758 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64011830 con 0x7fdb680d3f40 2024-05-08T23:28:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.170+0000 7fdb5dffb700 1 -- 172.21.15.5:0/151937758 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb64011990 con 0x7fdb680d3f40 2024-05-08T23:28:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.170+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb680ccd00 con 0x7fdb680d3f40 2024-05-08T23:28:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.170+0000 7fdb5dffb700 1 -- 172.21.15.5:0/151937758 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64011d40 con 0x7fdb680d3f40 2024-05-08T23:28:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.170+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb680cd2c0 con 0x7fdb680d3f40 2024-05-08T23:28:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.171+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb68068000 con 0x7fdb680d3f40 2024-05-08T23:28:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.171+0000 7fdb5dffb700 1 -- 172.21.15.5:0/151937758 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdb6401a400 con 0x7fdb680d3f40 2024-05-08T23:28:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.172+0000 7fdb5dffb700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb4c06ac90 0x7fdb4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.172+0000 7fdb5dffb700 1 -- 172.21.15.5:0/151937758 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb64089210 con 0x7fdb680d3f40 2024-05-08T23:28:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.172+0000 7fdb6cdfb700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb4c06ac90 0x7fdb4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.173+0000 7fdb6cdfb700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb4c06ac90 0x7fdb4c06d150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdb58013040 tx=0x7fdb58008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.176+0000 7fdb5dffb700 1 -- 172.21.15.5:0/151937758 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdb640542d0 con 0x7fdb680d3f40 2024-05-08T23:28:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.395+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb680d0e30 con 0x7fdb680d3f40 2024-05-08T23:28:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.396+0000 7fdb5dffb700 1 -- 172.21.15.5:0/151937758 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+15944 (secure 0 0 0) 0x7fdb640578f0 con 0x7fdb680d3f40 2024-05-08T23:28:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:49.396 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:28:00.273637+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:27:57.260067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T23:27:58.243420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:28:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb4c06ac90 msgr2=0x7fdb4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb4c06ac90 0x7fdb4c06d150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdb58013040 tx=0x7fdb58008070 comp rx=0 tx=0).stop 2024-05-08T23:28:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 msgr2=0x7fdb680ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb680ca810 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400b4a0 tx=0x7fdb6400b860 comp rx=0 tx=0).stop 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 shutdown_connections 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fdb4c06ac90 0x7fdb4c06d150 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb6815db80 0x7fdb6814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb680d3f40 0x7fdb680ca810 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 --2- 172.21.15.5:0/151937758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fdb6815d7a0 0x7fdb681609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 >> 172.21.15.5:0/151937758 conn(0x7fdb6808afe0 msgr2=0x7fdb680cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 shutdown_connections 2024-05-08T23:28:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.398+0000 7fdb6f860700 1 -- 172.21.15.5:0/151937758 wait complete. 2024-05-08T23:28:49.412 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:28:49.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.548+0000 7f3f75837700 1 Processor -- start 2024-05-08T23:28:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.548+0000 7f3f75837700 1 -- start start 2024-05-08T23:28:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f75837700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3f7015d2d0 0x7f3f700cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f75837700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f700d3420 0x7f3f700d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f75837700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f700d0f80 con 0x7f3f70161850 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f75837700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f700ce290 con 0x7f3f7015d2d0 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f75837700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f700cedd0 con 0x7f3f700d3420 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6effd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3f7015d2d0 0x7f3f700cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6f7fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3f70161850 0x7f3f70161c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34652/0 (socket says 172.21.15.5:34652) 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6f7fe700 1 -- 172.21.15.5:0/2395789749 learned_addr learned my addr 172.21.15.5:0/2395789749 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6e7fc700 1 --2- 172.21.15.5:0/2395789749 >> v2:172.21.15.49:3300/0 conn(0x7f3f700d3420 0x7f3f700d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6effd700 1 -- 172.21.15.5:0/2395789749 >> v2:172.21.15.49:3300/0 conn(0x7f3f700d3420 msgr2=0x7f3f700d38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6effd700 1 --2- 172.21.15.5:0/2395789749 >> v2:172.21.15.49:3300/0 conn(0x7f3f700d3420 0x7f3f700d38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6effd700 1 -- 172.21.15.5:0/2395789749 >> v1:172.21.15.5:6789/0 conn(0x7f3f70161850 legacy=0x7f3f70161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6effd700 1 -- 172.21.15.5:0/2395789749 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700d3df0 con 0x7f3f7015d2d0 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6e7fc700 1 --2- 172.21.15.5:0/2395789749 >> v2:172.21.15.49:3300/0 conn(0x7f3f700d3420 0x7f3f700d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6effd700 1 --2- 172.21.15.5:0/2395789749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f7015d2d0 0x7f3f700cf260 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f3f7009d0e0 tx=0x7f3f60009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e837db6d3571f4d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.549+0000 7f3f6dffb700 1 -- 172.21.15.5:0/2395789749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60011830 con 0x7f3f7015d2d0 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f6dffb700 1 -- 172.21.15.5:0/2395789749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f60011990 con 0x7f3f7015d2d0 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f6dffb700 1 -- 172.21.15.5:0/2395789749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60011d20 con 0x7f3f7015d2d0 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f75837700 1 -- 172.21.15.5:0/2395789749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f7015d2d0 msgr2=0x7f3f700cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f75837700 1 --2- 172.21.15.5:0/2395789749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f7015d2d0 0x7f3f700cf260 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f3f7009d0e0 tx=0x7f3f60009b20 comp rx=0 tx=0).stop 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f75837700 1 -- 172.21.15.5:0/2395789749 shutdown_connections 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f75837700 1 --2- 172.21.15.5:0/2395789749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f7015d2d0 0x7f3f700cf260 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f75837700 1 --2- 172.21.15.5:0/2395789749 >> v2:172.21.15.49:3300/0 conn(0x7f3f700d3420 0x7f3f700d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f75837700 1 -- 172.21.15.5:0/2395789749 >> 172.21.15.5:0/2395789749 conn(0x7f3f7009c9b0 msgr2=0x7f3f7008afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f75837700 1 -- 172.21.15.5:0/2395789749 shutdown_connections 2024-05-08T23:28:49.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.550+0000 7f3f75837700 1 -- 172.21.15.5:0/2395789749 wait complete. 2024-05-08T23:28:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.551+0000 7f3f75837700 1 Processor -- start 2024-05-08T23:28:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.552+0000 7f3f75837700 1 -- start start 2024-05-08T23:28:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.552+0000 7f3f75837700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f700d3420 0x7f3f7009d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.552+0000 7f3f75837700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f7015d2d0 0x7f3f7015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.552+0000 7f3f75837700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f70161850 0x7f3f700c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.552+0000 7f3f75837700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700c8120 con 0x7f3f700d3420 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.552+0000 7f3f75837700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700c8ca0 con 0x7f3f70161850 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.552+0000 7f3f75837700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700c8370 con 0x7f3f7015d2d0 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.552+0000 7f3f6effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f700d3420 0x7f3f7009d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f700d3420 0x7f3f7009d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44110/0 (socket says 172.21.15.5:44110) 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6effd700 1 -- 172.21.15.5:0/4065281559 learned_addr learned my addr 172.21.15.5:0/4065281559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6f7fe700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f70161850 0x7f3f700c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6e7fc700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f7015d2d0 0x7f3f7015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6effd700 1 -- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f7015d2d0 msgr2=0x7f3f7015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6effd700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f7015d2d0 0x7f3f7015c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6effd700 1 -- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f70161850 msgr2=0x7f3f700c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6effd700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f70161850 0x7f3f700c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6effd700 1 -- 172.21.15.5:0/4065281559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f600094a0 con 0x7f3f700d3420 2024-05-08T23:28:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f6effd700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f700d3420 0x7f3f7009d8a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3f60000c00 tx=0x7f3f60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f57fff700 1 -- 172.21.15.5:0/4065281559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f600036a0 con 0x7f3f700d3420 2024-05-08T23:28:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f57fff700 1 -- 172.21.15.5:0/4065281559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6001f070 con 0x7f3f700d3420 2024-05-08T23:28:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f700c8650 con 0x7f3f700d3420 2024-05-08T23:28:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f57fff700 1 -- 172.21.15.5:0/4065281559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60003a60 con 0x7f3f700d3420 2024-05-08T23:28:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.553+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7009dde0 con 0x7f3f700d3420 2024-05-08T23:28:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.554+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f700c8f80 con 0x7f3f700d3420 2024-05-08T23:28:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.555+0000 7f3f57fff700 1 -- 172.21.15.5:0/4065281559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3f60003bc0 con 0x7f3f700d3420 2024-05-08T23:28:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.555+0000 7f3f57fff700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3f5c06ac90 0x7f3f5c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.555+0000 7f3f57fff700 1 -- 172.21.15.5:0/4065281559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f60021070 con 0x7f3f700d3420 2024-05-08T23:28:49.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.555+0000 7f3f6e7fc700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3f5c06ac90 0x7f3f5c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.556+0000 7f3f6e7fc700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3f5c06ac90 0x7f3f5c06d150 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3f700c9df0 tx=0x7f3f64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.560+0000 7f3f57fff700 1 -- 172.21.15.5:0/4065281559 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3f60055ed0 con 0x7f3f700d3420 2024-05-08T23:28:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.823+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f700678f0 con 0x7f3f700d3420 2024-05-08T23:28:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.824+0000 7f3f57fff700 1 -- 172.21.15.5:0/4065281559 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7f3f6002b020 con 0x7f3f700d3420 2024-05-08T23:28:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:49.825 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3f5c06ac90 msgr2=0x7f3f5c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3f5c06ac90 0x7f3f5c06d150 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3f700c9df0 tx=0x7f3f64008040 comp rx=0 tx=0).stop 2024-05-08T23:28:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f700d3420 msgr2=0x7f3f7009d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f700d3420 0x7f3f7009d8a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3f60000c00 tx=0x7f3f60004750 comp rx=0 tx=0).stop 2024-05-08T23:28:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 shutdown_connections 2024-05-08T23:28:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3f5c06ac90 0x7f3f5c06d150 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f70161850 0x7f3f700c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f700d3420 0x7f3f7009d8a0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 --2- 172.21.15.5:0/4065281559 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f7015d2d0 0x7f3f7015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 >> 172.21.15.5:0/4065281559 conn(0x7f3f7009c9b0 msgr2=0x7f3f700b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.826+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 shutdown_connections 2024-05-08T23:28:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.827+0000 7f3f75837700 1 -- 172.21.15.5:0/4065281559 wait complete. 2024-05-08T23:28:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:49.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:49.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.978+0000 7f05b067b700 1 Processor -- start 2024-05-08T23:28:49.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.978+0000 7f05b067b700 1 -- start start 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05b067b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05a8161030 0x7f05a8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05b067b700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a80d0840 con 0x7f05a80b2480 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05b067b700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a806c600 con 0x7f05a80cbbc0 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05b067b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a806ccc0 con 0x7f05a8161030 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05ae417700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05a8161030 0x7f05a8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05adc16700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f05a80b2480 0x7f05a8099f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34662/0 (socket says 172.21.15.5:34662) 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05adc16700 1 -- 172.21.15.5:0/13795553 learned_addr learned my addr 172.21.15.5:0/13795553 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05ae417700 1 -- 172.21.15.5:0/13795553 >> v1:172.21.15.49:6789/0 conn(0x7f05a80cbbc0 legacy=0x7f05a80cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05ae417700 1 -- 172.21.15.5:0/13795553 >> v1:172.21.15.5:6789/0 conn(0x7f05a80b2480 legacy=0x7f05a8099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.979+0000 7f05ae417700 1 -- 172.21.15.5:0/13795553 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a806c8e0 con 0x7f05a8161030 2024-05-08T23:28:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05ae417700 1 --2- 172.21.15.5:0/13795553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a8161030 0x7f05a8099a50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009840 tx=0x7f05a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63fab8503300fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05ad415700 1 -- 172.21.15.5:0/13795553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4011830 con 0x7f05a8161030 2024-05-08T23:28:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05ad415700 1 -- 172.21.15.5:0/13795553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a4011990 con 0x7f05a8161030 2024-05-08T23:28:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05ad415700 1 -- 172.21.15.5:0/13795553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4011d20 con 0x7f05a8161030 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05b067b700 1 -- 172.21.15.5:0/13795553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a8161030 msgr2=0x7f05a8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05b067b700 1 --2- 172.21.15.5:0/13795553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a8161030 0x7f05a8099a50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009840 tx=0x7f05a4009b50 comp rx=0 tx=0).stop 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05b067b700 1 -- 172.21.15.5:0/13795553 shutdown_connections 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05b067b700 1 --2- 172.21.15.5:0/13795553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a8161030 0x7f05a8099a50 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.980+0000 7f05b067b700 1 -- 172.21.15.5:0/13795553 >> 172.21.15.5:0/13795553 conn(0x7f05a80b2f20 msgr2=0x7f05a80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.981+0000 7f05b067b700 1 -- 172.21.15.5:0/13795553 shutdown_connections 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.981+0000 7f05b067b700 1 -- 172.21.15.5:0/13795553 wait complete. 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.982+0000 7f05b067b700 1 Processor -- start 2024-05-08T23:28:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.982+0000 7f05b067b700 1 -- start start 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05b067b700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05a80b2480 0x7f05a81525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05b067b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a80cbbc0 0x7f05a80c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05b067b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05a8161030 0x7f05a815f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05b067b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80d0660 con 0x7f05a80cbbc0 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05b067b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8152af0 con 0x7f05a8161030 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05b067b700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c5f80 con 0x7f05a80b2480 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a80cbbc0 0x7f05a80c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a80cbbc0 0x7f05a80c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44138/0 (socket says 172.21.15.5:44138) 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 -- 172.21.15.5:0/714490989 learned_addr learned my addr 172.21.15.5:0/714490989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05aec18700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05a8161030 0x7f05a815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 -- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05a80b2480 msgr2=0x7f05a81525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05ae417700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05a80b2480 0x7f05a81525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05a80b2480 0x7f05a81525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 -- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05a8161030 msgr2=0x7f05a815f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05a8161030 0x7f05a815f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 -- 172.21.15.5:0/714490989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40094a0 con 0x7f05a80cbbc0 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05aec18700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05a8161030 0x7f05a815f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05adc16700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a80cbbc0 0x7f05a80c9ca0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f059800c520 tx=0x7f059800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f059effd700 1 -- 172.21.15.5:0/714490989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598005d70 con 0x7f05a80cbbc0 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f059effd700 1 -- 172.21.15.5:0/714490989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05980070d0 con 0x7f05a80cbbc0 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a80c5370 con 0x7f05a80cbbc0 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.983+0000 7f059effd700 1 -- 172.21.15.5:0/714490989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598007440 con 0x7f05a80cbbc0 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.984+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a80c5930 con 0x7f05a80cbbc0 2024-05-08T23:28:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.984+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a80c84e0 con 0x7f05a80cbbc0 2024-05-08T23:28:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.985+0000 7f059effd700 1 -- 172.21.15.5:0/714490989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f05980075a0 con 0x7f05a80cbbc0 2024-05-08T23:28:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.986+0000 7f059effd700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f058c06ac90 0x7f058c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.986+0000 7f059effd700 1 -- 172.21.15.5:0/714490989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0598088690 con 0x7f05a80cbbc0 2024-05-08T23:28:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.986+0000 7f05ae417700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f058c06ac90 0x7f058c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.986+0000 7f05ae417700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f058c06ac90 0x7f058c06d150 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f05a80b4260 tx=0x7f05a4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:49.991+0000 7f059effd700 1 -- 172.21.15.5:0/714490989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f05980539b0 con 0x7f05a80cbbc0 2024-05-08T23:28:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.256+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05a8068000 con 0x7f05a80cbbc0 2024-05-08T23:28:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.256+0000 7f059effd700 1 -- 172.21.15.5:0/714490989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4768 (secure 0 0 0) 0x7f0598056fd0 con 0x7f05a80cbbc0 2024-05-08T23:28:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:50.257 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:27:59.271749+0000","modified":"2024-05-08T23:28:32.366916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8592,"mds_1":7058,"mds_2":7052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7052":{"gid":7052,"name":"b","rank":2,"incarnation":103,"state":"up:active","state_seq":10,"addr":"172.21.15.49:6837/873094942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":873094942},{"type":"v1","addr":"172.21.15.49:6837","nonce":873094942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7058":{"gid":7058,"name":"a","rank":1,"incarnation":101,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6835/1828661194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":1828661194},{"type":"v1","addr":"172.21.15.5:6835","nonce":1828661194}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8592":{"gid":8592,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":11,"addr":"172.21.15.5:6834/1510899872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1510899872},{"type":"v1","addr":"172.21.15.5:6834","nonce":1510899872}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T23:28:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f058c06ac90 msgr2=0x7f058c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f058c06ac90 0x7f058c06d150 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f05a80b4260 tx=0x7f05a4008030 comp rx=0 tx=0).stop 2024-05-08T23:28:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a80cbbc0 msgr2=0x7f05a80c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a80cbbc0 0x7f05a80c9ca0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f059800c520 tx=0x7f059800c8e0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 shutdown_connections 2024-05-08T23:28:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f058c06ac90 0x7f058c06d150 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05a8161030 0x7f05a815f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f05a80b2480 0x7f05a81525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 --2- 172.21.15.5:0/714490989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a80cbbc0 0x7f05a80c9ca0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 >> 172.21.15.5:0/714490989 conn(0x7f05a80b2f20 msgr2=0x7f05a80cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 shutdown_connections 2024-05-08T23:28:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.259+0000 7f05b067b700 1 -- 172.21.15.5:0/714490989 wait complete. 2024-05-08T23:28:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-05-08T23:28:50.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:28:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.409+0000 7fc20c8a6700 1 Processor -- start 2024-05-08T23:28:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.409+0000 7fc20c8a6700 1 -- start start 2024-05-08T23:28:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc20c8a6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc20815a750 0x7fc2080d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc20c8a6700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2080c5720 0x7fc2080c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc20c8a6700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2080cfbd0 con 0x7fc2080af860 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc20c8a6700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2080c6110 con 0x7fc2080c5720 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc20c8a6700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2080cf440 con 0x7fc20815a750 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc206d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2080c5720 0x7fc2080c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc205d9b700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc2080af860 0x7fc2080ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34666/0 (socket says 172.21.15.5:34666) 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc206d9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2080c5720 0x7fc2080c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44148/0 (socket says 172.21.15.5:44148) 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc205d9b700 1 -- 172.21.15.5:0/748722739 learned_addr learned my addr 172.21.15.5:0/748722739 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc20659c700 1 --2- 172.21.15.5:0/748722739 >> v2:172.21.15.49:3300/0 conn(0x7fc20815a750 0x7fc2080d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc206d9d700 1 -- 172.21.15.5:0/748722739 >> v2:172.21.15.49:3300/0 conn(0x7fc20815a750 msgr2=0x7fc2080d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc206d9d700 1 --2- 172.21.15.5:0/748722739 >> v2:172.21.15.49:3300/0 conn(0x7fc20815a750 0x7fc2080d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc206d9d700 1 -- 172.21.15.5:0/748722739 >> v1:172.21.15.5:6789/0 conn(0x7fc2080af860 legacy=0x7fc2080ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc20559a700 1 -- 172.21.15.5:0/748722739 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890829003 0 0) 0x7fc2080cfbd0 con 0x7fc2080af860 2024-05-08T23:28:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc206d9d700 1 -- 172.21.15.5:0/748722739 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2080cb780 con 0x7fc2080c5720 2024-05-08T23:28:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.410+0000 7fc206d9d700 1 --2- 172.21.15.5:0/748722739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2080c5720 0x7fc2080c5bd0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fc208071570 tx=0x7fc200009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ff4a4bfed0b62f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20559a700 1 -- 172.21.15.5:0/748722739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200011830 con 0x7fc2080c5720 2024-05-08T23:28:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20559a700 1 -- 172.21.15.5:0/748722739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc200011990 con 0x7fc2080c5720 2024-05-08T23:28:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20559a700 1 -- 172.21.15.5:0/748722739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200011d20 con 0x7fc2080c5720 2024-05-08T23:28:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20c8a6700 1 -- 172.21.15.5:0/748722739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2080c5720 msgr2=0x7fc2080c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/748722739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2080c5720 0x7fc2080c5bd0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fc208071570 tx=0x7fc200009b20 comp rx=0 tx=0).stop 2024-05-08T23:28:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20c8a6700 1 -- 172.21.15.5:0/748722739 shutdown_connections 2024-05-08T23:28:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/748722739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2080c5720 0x7fc2080c5bd0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/748722739 >> v2:172.21.15.49:3300/0 conn(0x7fc20815a750 0x7fc2080d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20c8a6700 1 -- 172.21.15.5:0/748722739 >> 172.21.15.5:0/748722739 conn(0x7fc2080d03a0 msgr2=0x7fc2080d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20c8a6700 1 -- 172.21.15.5:0/748722739 shutdown_connections 2024-05-08T23:28:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.411+0000 7fc20c8a6700 1 -- 172.21.15.5:0/748722739 wait complete. 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.413+0000 7fc20c8a6700 1 Processor -- start 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.413+0000 7fc20c8a6700 1 -- start start 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc20c8a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080af860 0x7fc2080b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc20c8a6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2080c5720 0x7fc208143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc20c8a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20815a750 0x7fc2081e5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc20c8a6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2081e42f0 con 0x7fc20815a750 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc20c8a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2081e45d0 con 0x7fc2080af860 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc20c8a6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2081e48b0 con 0x7fc2080c5720 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20815a750 0x7fc2081e5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20815a750 0x7fc2081e5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44164/0 (socket says 172.21.15.5:44164) 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 -- 172.21.15.5:0/3676353223 learned_addr learned my addr 172.21.15.5:0/3676353223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc205d9b700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2080c5720 0x7fc208143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc20659c700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080af860 0x7fc2080b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 -- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2080c5720 msgr2=0x7fc208143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2080c5720 0x7fc208143d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 -- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080af860 msgr2=0x7fc2080b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080af860 0x7fc2080b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 -- 172.21.15.5:0/3676353223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2000094a0 con 0x7fc20815a750 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.414+0000 7fc206d9d700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20815a750 0x7fc2081e5390 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fc2000090b0 tx=0x7fc200004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.415+0000 7fc1f6ffd700 1 -- 172.21.15.5:0/3676353223 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200004590 con 0x7fc20815a750 2024-05-08T23:28:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.415+0000 7fc1f6ffd700 1 -- 172.21.15.5:0/3676353223 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20001f070 con 0x7fc20815a750 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.415+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2081e4b90 con 0x7fc20815a750 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.415+0000 7fc1f6ffd700 1 -- 172.21.15.5:0/3676353223 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000038a0 con 0x7fc20815a750 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.415+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20814d1c0 con 0x7fc20815a750 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.416+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2081e50b0 con 0x7fc20815a750 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.416+0000 7fc1f6ffd700 1 -- 172.21.15.5:0/3676353223 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc200003a00 con 0x7fc20815a750 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.417+0000 7fc1f6ffd700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc1e806ac90 0x7fc1e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.417+0000 7fc1f6ffd700 1 -- 172.21.15.5:0/3676353223 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc200021070 con 0x7fc20815a750 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.417+0000 7fc20659c700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc1e806ac90 0x7fc1e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.417+0000 7fc20659c700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc1e806ac90 0x7fc1e806d150 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80113e0 tx=0x7fc1f800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.422+0000 7fc1f6ffd700 1 -- 172.21.15.5:0/3676353223 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc2081e50b0 con 0x7fc20815a750 2024-05-08T23:28:50.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.639+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2081e50b0 con 0x7fc20815a750 2024-05-08T23:28:50.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.641+0000 7fc1f6ffd700 1 -- 172.21.15.5:0/3676353223 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+15944 (secure 0 0 0) 0x7fc2081e50b0 con 0x7fc20815a750 2024-05-08T23:28:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:50.641 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:28:00.273637+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:27:57.260067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T23:27:58.243420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:28:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc1e806ac90 msgr2=0x7fc1e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc1e806ac90 0x7fc1e806d150 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80113e0 tx=0x7fc1f800b040 comp rx=0 tx=0).stop 2024-05-08T23:28:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20815a750 msgr2=0x7fc2081e5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20815a750 0x7fc2081e5390 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fc2000090b0 tx=0x7fc200004750 comp rx=0 tx=0).stop 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 shutdown_connections 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc1e806ac90 0x7fc1e806d150 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20815a750 0x7fc2081e5390 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080af860 0x7fc2080b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 --2- 172.21.15.5:0/3676353223 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc2080c5720 0x7fc208143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 >> 172.21.15.5:0/3676353223 conn(0x7fc2080d03a0 msgr2=0x7fc2080cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 shutdown_connections 2024-05-08T23:28:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.643+0000 7fc20c8a6700 1 -- 172.21.15.5:0/3676353223 wait complete. 2024-05-08T23:28:50.657 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:28:50.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.795+0000 7fd75c612700 1 Processor -- start 2024-05-08T23:28:50.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.795+0000 7fd75c612700 1 -- start start 2024-05-08T23:28:50.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.796+0000 7fd75c612700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd75409a160 0x7fd7540b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.796+0000 7fd75c612700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd7540b2510 0x7fd7540b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.796+0000 7fd75c612700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7540b4280 con 0x7fd75415ba00 2024-05-08T23:28:50.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.796+0000 7fd75c612700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7540b4560 con 0x7fd75409a160 2024-05-08T23:28:50.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.796+0000 7fd75c612700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7540cccf0 con 0x7fd7540b2510 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.796+0000 7fd759bad700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd75409a160 0x7fd7540b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.796+0000 7fd759bad700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd75409a160 0x7fd7540b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44168/0 (socket says 172.21.15.5:44168) 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.796+0000 7fd759bad700 1 -- 172.21.15.5:0/1232148087 learned_addr learned my addr 172.21.15.5:0/1232148087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd75abaf700 1 --2- 172.21.15.5:0/1232148087 >> v2:172.21.15.49:3300/0 conn(0x7fd7540b2510 0x7fd7540b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd759bad700 1 -- 172.21.15.5:0/1232148087 >> v2:172.21.15.49:3300/0 conn(0x7fd7540b2510 msgr2=0x7fd7540b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd759bad700 1 --2- 172.21.15.5:0/1232148087 >> v2:172.21.15.49:3300/0 conn(0x7fd7540b2510 0x7fd7540b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd759bad700 1 -- 172.21.15.5:0/1232148087 >> v1:172.21.15.49:6789/0 conn(0x7fd75415ba00 legacy=0x7fd754099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd759bad700 1 -- 172.21.15.5:0/1232148087 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7540ccfd0 con 0x7fd75409a160 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd75abaf700 1 --2- 172.21.15.5:0/1232148087 >> v2:172.21.15.49:3300/0 conn(0x7fd7540b2510 0x7fd7540b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd759bad700 1 --2- 172.21.15.5:0/1232148087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd75409a160 0x7fd7540b3a40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fd7540724a0 tx=0x7fd744009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa61cd854caa143 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd7593ac700 1 -- 172.21.15.5:0/1232148087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74400b870 con 0x7fd75409a160 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd7593ac700 1 -- 172.21.15.5:0/1232148087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74400b9d0 con 0x7fd75409a160 2024-05-08T23:28:50.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.797+0000 7fd7593ac700 1 -- 172.21.15.5:0/1232148087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74400bd60 con 0x7fd75409a160 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.798+0000 7fd75c612700 1 -- 172.21.15.5:0/1232148087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd75409a160 msgr2=0x7fd7540b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.798+0000 7fd75c612700 1 --2- 172.21.15.5:0/1232148087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd75409a160 0x7fd7540b3a40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fd7540724a0 tx=0x7fd744009bd0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.798+0000 7fd75c612700 1 -- 172.21.15.5:0/1232148087 shutdown_connections 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.798+0000 7fd75c612700 1 --2- 172.21.15.5:0/1232148087 >> v2:172.21.15.49:3300/0 conn(0x7fd7540b2510 0x7fd7540b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.798+0000 7fd75c612700 1 --2- 172.21.15.5:0/1232148087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd75409a160 0x7fd7540b3a40 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.798+0000 7fd75c612700 1 -- 172.21.15.5:0/1232148087 >> 172.21.15.5:0/1232148087 conn(0x7fd7540b2f20 msgr2=0x7fd7540b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.798+0000 7fd75c612700 1 -- 172.21.15.5:0/1232148087 shutdown_connections 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.798+0000 7fd75c612700 1 -- 172.21.15.5:0/1232148087 wait complete. 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.800+0000 7fd75c612700 1 Processor -- start 2024-05-08T23:28:50.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.800+0000 7fd75c612700 1 -- start start 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75c612700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd75409a160 0x7fd75409b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75c612700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7540b2510 0x7fd7541f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75c612700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd75415ba00 0x7fd7541f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75c612700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7541f1340 con 0x7fd7540b2510 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75c612700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7541f2780 con 0x7fd75409a160 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75c612700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7541f2a60 con 0x7fd75415ba00 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7540b2510 0x7fd7541f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7540b2510 0x7fd7541f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44180/0 (socket says 172.21.15.5:44180) 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 -- 172.21.15.5:0/2570700516 learned_addr learned my addr 172.21.15.5:0/2570700516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75a3ae700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd75409a160 0x7fd75409b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75abaf700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd75415ba00 0x7fd7541f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 -- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd75415ba00 msgr2=0x7fd7541f0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd75415ba00 0x7fd7541f0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 -- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd75409a160 msgr2=0x7fd75409b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd75409a160 0x7fd75409b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 -- 172.21.15.5:0/2570700516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd744009550 con 0x7fd7540b2510 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd759bad700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7540b2510 0x7fd7541f08c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fd744000c00 tx=0x7fd744004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd74affd700 1 -- 172.21.15.5:0/2570700516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744004540 con 0x7fd7540b2510 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd74affd700 1 -- 172.21.15.5:0/2570700516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7440036a0 con 0x7fd7540b2510 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7541f2d40 con 0x7fd7540b2510 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.801+0000 7fd74affd700 1 -- 172.21.15.5:0/2570700516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744003a10 con 0x7fd7540b2510 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.802+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7541f3300 con 0x7fd7540b2510 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.802+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd754068000 con 0x7fd7540b2510 2024-05-08T23:28:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.803+0000 7fd74affd700 1 -- 172.21.15.5:0/2570700516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd744021070 con 0x7fd7540b2510 2024-05-08T23:28:50.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.803+0000 7fd74affd700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd73806ac90 0x7fd73806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:50.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.804+0000 7fd74affd700 1 -- 172.21.15.5:0/2570700516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd74408aaa0 con 0x7fd7540b2510 2024-05-08T23:28:50.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.804+0000 7fd75a3ae700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd73806ac90 0x7fd73806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:50.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.804+0000 7fd75a3ae700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd73806ac90 0x7fd73806d150 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd750000f60 tx=0x7fd750009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:50.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:50.808+0000 7fd74affd700 1 -- 172.21.15.5:0/2570700516 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd744055e30 con 0x7fd7540b2510 2024-05-08T23:28:51.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.074+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd7541f34c0 con 0x7fd7540b2510 2024-05-08T23:28:51.075 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-08T23:28:51.075+0000 7f8a49d06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:28:51.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.082+0000 7fd74affd700 1 -- 172.21.15.5:0/2570700516 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v110) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd744019e60 con 0x7fd7540b2510 2024-05-08T23:28:51.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd73806ac90 msgr2=0x7fd73806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:51.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd73806ac90 0x7fd73806d150 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd750000f60 tx=0x7fd750009040 comp rx=0 tx=0).stop 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7540b2510 msgr2=0x7fd7541f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7540b2510 0x7fd7541f08c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fd744000c00 tx=0x7fd744004750 comp rx=0 tx=0).stop 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 shutdown_connections 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fd73806ac90 0x7fd73806d150 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd75415ba00 0x7fd7541f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd75409a160 0x7fd75409b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 --2- 172.21.15.5:0/2570700516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7540b2510 0x7fd7541f08c0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 >> 172.21.15.5:0/2570700516 conn(0x7fd7540b2f20 msgr2=0x7fd75415e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 shutdown_connections 2024-05-08T23:28:51.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.086+0000 7fd75c612700 1 -- 172.21.15.5:0/2570700516 wait complete. 2024-05-08T23:28:51.088 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:28:51.088 INFO:tasks.ceph.mds.b.smithi049.stderr: -1099> 2024-05-08T23:28:28.952+0000 7f7187517700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1) n(v0 rc2024-05-08T23:28:12.068944+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={8161=pAsLsXsFsx/pAsLsXsFsx@4},l=8161 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x564e1f5e7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.068944+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-05-08T23:28:12.068944+0000 2=0+2)","error_str":""},"return_code":-61} 2024-05-08T23:28:51.088 INFO:tasks.ceph.mds.b.smithi049.stderr: -1085> 2024-05-08T23:28:28.952+0000 7f7187517700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={8161=pAsLsXsFsx/AsLsXsFsx@2},l=8161 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x564e1f5e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-05-08T23:28:51.089 INFO:tasks.ceph.mds.a.smithi005.stderr: -1201> 2024-05-08T23:28:28.951+0000 7f59c8b28700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-05-08T23:28:12.067944+0000 1=0+1) n(v2 rc2024-05-08T23:28:12.325938+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={8161=pAsLsXsFsx/pAsLsXsFsx@4},l=8161 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x557432a06b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.067944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.325938+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.067944+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-05-08T23:28:12.325938+0000 4=0+4)","error_str":""},"return_code":-61} 2024-05-08T23:28:51.089 INFO:tasks.ceph.mds.a.smithi005.stderr: -1081> 2024-05-08T23:28:28.953+0000 7f59ceb34700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1) n(v1 rc2024-05-08T23:28:12.325938+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={8161=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x557432a07080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.068944+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.068944+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-05-08T23:28:12.325938+0000 3=0+3)","error_str":""},"return_code":-61} 2024-05-08T23:28:51.093 INFO:tasks.ceph.mds.c.smithi005.stderr: -6819> 2024-05-08T23:28:14.802+0000 7fa0ab37b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T23:28:51.102 INFO:tasks.ceph.mds.c.smithi005.stderr: -2519> 2024-05-08T23:28:28.951+0000 7fa0a7b74700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1) n(v3 rc2024-05-08T23:28:12.325938+0000 6=0+6) (inest mix dirty) (ifile excl) (iversion lock) caps={8161=pAsLsXsFsx/pAsLsXsFsx@5},l=8161 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ab1daee580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.325938+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-05-08T23:28:12.325938+0000 6=0+6)","error_str":""},"return_code":-61} 2024-05-08T23:28:51.102 INFO:tasks.ceph.mds.c.smithi005.stderr: -2417> 2024-05-08T23:28:28.953+0000 7fa0adb80700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1) n(v4 rc2024-05-08T23:28:12.325938+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={8161=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55ab1daeeb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:28:12.325938+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-05-08T23:28:12.066944+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-05-08T23:28:12.325938+0000 5=0+5)","error_str":""},"return_code":-61} 2024-05-08T23:28:51.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:28:51.117 INFO:tasks.ceph.mds.b.smithi049.stdout:starting mds.b at 2024-05-08T23:28:51.119 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-08T23:28:51.134 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-08T23:28:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.248+0000 7f0f93b44700 1 Processor -- start 2024-05-08T23:28:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.248+0000 7f0f93b44700 1 -- start start 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f93b44700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0f8c15b8c0 0x7f0f8c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f93b44700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f8c0d1740 0x7f0f8c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f93b44700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c0cf560 con 0x7f0f8c15b4e0 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f93b44700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c0cefe0 con 0x7f0f8c0d1740 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f93b44700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c0ccff0 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f920e1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f8c0d1740 0x7f0f8c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f920e1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f8c0d1740 0x7f0f8c0cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44208/0 (socket says 172.21.15.5:44208) 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f920e1700 1 -- 172.21.15.5:0/1352157774 learned_addr learned my addr 172.21.15.5:0/1352157774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f910df700 1 --2- 172.21.15.5:0/1352157774 >> v2:172.21.15.49:3300/0 conn(0x7f0f8c15b8c0 0x7f0f8c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f920e1700 1 -- 172.21.15.5:0/1352157774 >> v2:172.21.15.49:3300/0 conn(0x7f0f8c15b8c0 msgr2=0x7f0f8c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f920e1700 1 --2- 172.21.15.5:0/1352157774 >> v2:172.21.15.49:3300/0 conn(0x7f0f8c15b8c0 0x7f0f8c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f920e1700 1 -- 172.21.15.5:0/1352157774 >> v1:172.21.15.49:6789/0 conn(0x7f0f8c15b4e0 legacy=0x7f0f8c0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f920e1700 1 -- 172.21.15.5:0/1352157774 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0cd2d0 con 0x7f0f8c0d1740 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.249+0000 7f0f920e1700 1 --2- 172.21.15.5:0/1352157774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0d1740 0x7f0f8c0cc2c0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c15f1e0 tx=0x7f0f8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5546b5e7d80090 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f908de700 1 -- 172.21.15.5:0/1352157774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400e070 con 0x7f0f8c0d1740 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f908de700 1 -- 172.21.15.5:0/1352157774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8400b9c0 con 0x7f0f8c0d1740 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f908de700 1 -- 172.21.15.5:0/1352157774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400bd70 con 0x7f0f8c0d1740 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f93b44700 1 -- 172.21.15.5:0/1352157774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0d1740 msgr2=0x7f0f8c0cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f93b44700 1 --2- 172.21.15.5:0/1352157774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0d1740 0x7f0f8c0cc2c0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c15f1e0 tx=0x7f0f8400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f93b44700 1 -- 172.21.15.5:0/1352157774 shutdown_connections 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f93b44700 1 --2- 172.21.15.5:0/1352157774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0d1740 0x7f0f8c0cc2c0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f93b44700 1 --2- 172.21.15.5:0/1352157774 >> v2:172.21.15.49:3300/0 conn(0x7f0f8c15b8c0 0x7f0f8c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f93b44700 1 -- 172.21.15.5:0/1352157774 >> 172.21.15.5:0/1352157774 conn(0x7f0f8c08afe0 msgr2=0x7f0f8c08b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f93b44700 1 -- 172.21.15.5:0/1352157774 shutdown_connections 2024-05-08T23:28:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.250+0000 7f0f93b44700 1 -- 172.21.15.5:0/1352157774 wait complete. 2024-05-08T23:28:51.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.252+0000 7f0f93b44700 1 Processor -- start 2024-05-08T23:28:51.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.252+0000 7f0f93b44700 1 -- start start 2024-05-08T23:28:51.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f93b44700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f8c0d1740 0x7f0f8c14f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:51.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f93b44700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15b4e0 0x7f0f8c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:51.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f93b44700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c15b8c0 0x7f0f8c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:51.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f93b44700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c06f8c0 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f93b44700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c8ef0 con 0x7f0f8c15b4e0 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f93b44700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0caf00 con 0x7f0f8c0d1740 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f920e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c15b8c0 0x7f0f8c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f920e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c15b8c0 0x7f0f8c0b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44220/0 (socket says 172.21.15.5:44220) 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.253+0000 7f0f920e1700 1 -- 172.21.15.5:0/2512835867 learned_addr learned my addr 172.21.15.5:0/2512835867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f910df700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15b4e0 0x7f0f8c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f918e0700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f8c0d1740 0x7f0f8c14f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f920e1700 1 -- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f8c0d1740 msgr2=0x7f0f8c14f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f920e1700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f8c0d1740 0x7f0f8c14f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f920e1700 1 -- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15b4e0 msgr2=0x7f0f8c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f920e1700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15b4e0 0x7f0f8c0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f920e1700 1 -- 172.21.15.5:0/2512835867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f84009b30 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f918e0700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f8c0d1740 0x7f0f8c14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f920e1700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c15b8c0 0x7f0f8c0b06a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0cef90 tx=0x7f0f84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f827fc700 1 -- 172.21.15.5:0/2512835867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840099b0 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f827fc700 1 -- 172.21.15.5:0/2512835867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f840092e0 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c0cb1e0 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f827fc700 1 -- 172.21.15.5:0/2512835867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8401c640 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.254+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c14fdb0 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.255+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c0c9320 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.256+0000 7f0f827fc700 1 -- 172.21.15.5:0/2512835867 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0f8401c7a0 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.256+0000 7f0f827fc700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f7006ac90 0x7f0f7006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.256+0000 7f0f827fc700 1 -- 172.21.15.5:0/2512835867 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0f8408b0f0 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.256+0000 7f0f918e0700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f7006ac90 0x7f0f7006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.257+0000 7f0f918e0700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f7006ac90 0x7f0f7006d150 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0f88000f60 tx=0x7f0f88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.261+0000 7f0f827fc700 1 -- 172.21.15.5:0/2512835867 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0f84056260 con 0x7f0f8c15b8c0 2024-05-08T23:28:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:51.527+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0f8c074bc0 con 0x7f0f8c15b8c0 2024-05-08T23:28:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.086+0000 7f0f827fc700 1 -- 172.21.15.5:0/2512835867 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v111) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0f84059880 con 0x7f0f8c15b8c0 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f7006ac90 msgr2=0x7f0f7006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f7006ac90 0x7f0f7006d150 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0f88000f60 tx=0x7f0f88009040 comp rx=0 tx=0).stop 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c15b8c0 msgr2=0x7f0f8c0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c15b8c0 0x7f0f8c0b06a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0cef90 tx=0x7f0f84004ad0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 shutdown_connections 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0f7006ac90 0x7f0f7006d150 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c15b8c0 0x7f0f8c0b06a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0f8c0d1740 0x7f0f8c14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 --2- 172.21.15.5:0/2512835867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f8c15b4e0 0x7f0f8c0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 >> 172.21.15.5:0/2512835867 conn(0x7f0f8c08afe0 msgr2=0x7f0f8c06fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:52.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.090+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 shutdown_connections 2024-05-08T23:28:52.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.091+0000 7f0f93b44700 1 -- 172.21.15.5:0/2512835867 wait complete. 2024-05-08T23:28:52.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:28:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.296+0000 7f09e1316700 1 Processor -- start 2024-05-08T23:28:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.296+0000 7f09e1316700 1 -- start start 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09e1316700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0cff90 0x7f09dc0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09e1316700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09dc0d41b0 0x7f09dc0ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09e1316700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09dc0d08b0 con 0x7f09dc15dc50 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09e1316700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09dc15eae0 con 0x7f09dc0d41b0 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09e1316700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc0d18f0 con 0x7f09dc0cff90 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09db7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09dc0d41b0 0x7f09dc0ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09da7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f09dc15dc50 0x7f09dc0d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34676/0 (socket says 172.21.15.5:34676) 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09da7fc700 1 -- 172.21.15.5:0/2635882682 learned_addr learned my addr 172.21.15.5:0/2635882682 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09daffd700 1 --2- 172.21.15.5:0/2635882682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0cff90 0x7f09dc0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09db7fe700 1 -- 172.21.15.5:0/2635882682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0cff90 msgr2=0x7f09dc0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09db7fe700 1 --2- 172.21.15.5:0/2635882682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0cff90 0x7f09dc0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09db7fe700 1 -- 172.21.15.5:0/2635882682 >> v1:172.21.15.5:6789/0 conn(0x7f09dc15dc50 legacy=0x7f09dc0d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09db7fe700 1 -- 172.21.15.5:0/2635882682 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc15e140 con 0x7f09dc0d41b0 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09d9ffb700 1 -- 172.21.15.5:0/2635882682 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366010336 0 0) 0x7f09dc0d08b0 con 0x7f09dc15dc50 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.297+0000 7f09db7fe700 1 --2- 172.21.15.5:0/2635882682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0d41b0 0x7f09dc0ce400 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009dc0 tx=0x7f09d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0f7ef4fe746f15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09d9ffb700 1 -- 172.21.15.5:0/2635882682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400e070 con 0x7f09dc0d41b0 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09d9ffb700 1 -- 172.21.15.5:0/2635882682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d400bae0 con 0x7f09dc0d41b0 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09d9ffb700 1 -- 172.21.15.5:0/2635882682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400be90 con 0x7f09dc0d41b0 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09e1316700 1 -- 172.21.15.5:0/2635882682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0d41b0 msgr2=0x7f09dc0ce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09e1316700 1 --2- 172.21.15.5:0/2635882682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0d41b0 0x7f09dc0ce400 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009dc0 tx=0x7f09d400b330 comp rx=0 tx=0).stop 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09e1316700 1 -- 172.21.15.5:0/2635882682 shutdown_connections 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09e1316700 1 --2- 172.21.15.5:0/2635882682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0d41b0 0x7f09dc0ce400 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09e1316700 1 --2- 172.21.15.5:0/2635882682 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0cff90 0x7f09dc0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09e1316700 1 -- 172.21.15.5:0/2635882682 >> 172.21.15.5:0/2635882682 conn(0x7f09dc08afe0 msgr2=0x7f09dc0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09e1316700 1 -- 172.21.15.5:0/2635882682 shutdown_connections 2024-05-08T23:28:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.298+0000 7f09e1316700 1 -- 172.21.15.5:0/2635882682 wait complete. 2024-05-08T23:28:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.300+0000 7f09e1316700 1 Processor -- start 2024-05-08T23:28:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.300+0000 7f09e1316700 1 -- start start 2024-05-08T23:28:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09e1316700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0cff90 0x7f09dc15cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09e1316700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0d41b0 0x7f09dc0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09e1316700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09dc15dc50 0x7f09dc1e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09e1316700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc15fc20 con 0x7f09dc0cff90 2024-05-08T23:28:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09e1316700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc15efa0 con 0x7f09dc15dc50 2024-05-08T23:28:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09e1316700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc15f280 con 0x7f09dc0d41b0 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0cff90 0x7f09dc15cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0cff90 0x7f09dc15cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44246/0 (socket says 172.21.15.5:44246) 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 -- 172.21.15.5:0/2434504394 learned_addr learned my addr 172.21.15.5:0/2434504394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09da7fc700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0d41b0 0x7f09dc0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09db7fe700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09dc15dc50 0x7f09dc1e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 -- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0d41b0 msgr2=0x7f09dc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0d41b0 0x7f09dc0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 -- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09dc15dc50 msgr2=0x7f09dc1e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09dc15dc50 0x7f09dc1e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 -- 172.21.15.5:0/2434504394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4009a70 con 0x7f09dc0cff90 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09db7fe700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09dc15dc50 0x7f09dc1e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09da7fc700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0d41b0 0x7f09dc0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09daffd700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0cff90 0x7f09dc15cab0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00e9b0 tx=0x7f09cc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09c37fe700 1 -- 172.21.15.5:0/2434504394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc009710 con 0x7f09dc0cff90 2024-05-08T23:28:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09c37fe700 1 -- 172.21.15.5:0/2434504394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc00c850 con 0x7f09dc0cff90 2024-05-08T23:28:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09c37fe700 1 -- 172.21.15.5:0/2434504394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00cbc0 con 0x7f09dc0cff90 2024-05-08T23:28:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.301+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc15f560 con 0x7f09dc0cff90 2024-05-08T23:28:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.302+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc15cff0 con 0x7f09dc0cff90 2024-05-08T23:28:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.303+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc08cde0 con 0x7f09dc0cff90 2024-05-08T23:28:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.303+0000 7f09c37fe700 1 -- 172.21.15.5:0/2434504394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f09cc00cd20 con 0x7f09dc0cff90 2024-05-08T23:28:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.304+0000 7f09c37fe700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09c806ac90 0x7f09c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.304+0000 7f09c37fe700 1 -- 172.21.15.5:0/2434504394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f09cc0894a0 con 0x7f09dc0cff90 2024-05-08T23:28:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.304+0000 7f09da7fc700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09c806ac90 0x7f09c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.304+0000 7f09da7fc700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09c806ac90 0x7f09c806d150 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0a1300 tx=0x7f09d0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.308+0000 7f09c37fe700 1 -- 172.21.15.5:0/2434504394 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f09cc054610 con 0x7f09dc0cff90 2024-05-08T23:28:52.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:52.530+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f09dc15fe90 con 0x7f09dc0cff90 2024-05-08T23:28:53.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.487+0000 7f09c37fe700 1 -- 172.21.15.5:0/2434504394 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7f09cc057c30 con 0x7f09dc0cff90 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09c806ac90 msgr2=0x7f09c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09c806ac90 0x7f09c806d150 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0a1300 tx=0x7f09d0007d30 comp rx=0 tx=0).stop 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0cff90 msgr2=0x7f09dc15cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0cff90 0x7f09dc15cab0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00e9b0 tx=0x7f09cc00ed70 comp rx=0 tx=0).stop 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 shutdown_connections 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f09c806ac90 0x7f09c806d150 secure :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0a1300 tx=0x7f09d0007d30 comp rx=0 tx=0).stop 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09dc15dc50 0x7f09dc1e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09dc0cff90 0x7f09dc15cab0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 --2- 172.21.15.5:0/2434504394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f09dc0d41b0 0x7f09dc0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.491+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 >> 172.21.15.5:0/2434504394 conn(0x7f09dc08afe0 msgr2=0x7f09dc0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.492+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 shutdown_connections 2024-05-08T23:28:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.492+0000 7f09e1316700 1 -- 172.21.15.5:0/2434504394 wait complete. 2024-05-08T23:28:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:28:53.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:28:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.660+0000 7f6bdd71a700 1 Processor -- start 2024-05-08T23:28:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.660+0000 7f6bdd71a700 1 -- start start 2024-05-08T23:28:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bdd71a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bd815a7c0 0x7f6bd80cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bdd71a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd80727b0 0x7f6bd80c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bdd71a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd80cd7c0 con 0x7f6bd8072260 2024-05-08T23:28:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bdd71a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd80cd1b0 con 0x7f6bd815a7c0 2024-05-08T23:28:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bdd71a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd815aba0 con 0x7f6bd80727b0 2024-05-08T23:28:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd6ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bd815a7c0 0x7f6bd80cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd67fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd80727b0 0x7f6bd80c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd67fc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd80727b0 0x7f6bd80c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:41134/0 (socket says 172.21.15.5:41134) 2024-05-08T23:28:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd77fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6bd8072260 0x7f6bd80b24f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:46972/0 (socket says 172.21.15.5:46972) 2024-05-08T23:28:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd67fc700 1 -- 172.21.15.5:0/362533761 learned_addr learned my addr 172.21.15.5:0/362533761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd6ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bd815a7c0 0x7f6bd80cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36356/0 (socket says 172.21.15.5:36356) 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd67fc700 1 -- 172.21.15.5:0/362533761 >> v2:172.21.15.49:3300/0 conn(0x7f6bd815a7c0 msgr2=0x7f6bd80cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd67fc700 1 --2- 172.21.15.5:0/362533761 >> v2:172.21.15.49:3300/0 conn(0x7f6bd815a7c0 0x7f6bd80cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd67fc700 1 -- 172.21.15.5:0/362533761 >> v1:172.21.15.49:6789/0 conn(0x7f6bd8072260 legacy=0x7f6bd80b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd67fc700 1 -- 172.21.15.5:0/362533761 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd80b3e70 con 0x7f6bd80727b0 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd6ffd700 1 --2- 172.21.15.5:0/362533761 >> v2:172.21.15.49:3300/0 conn(0x7f6bd815a7c0 0x7f6bd80cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd67fc700 1 --2- 172.21.15.5:0/362533761 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd80727b0 0x7f6bd80c9a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0098f0 tx=0x7f6bcc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affc85e017831475 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd5ffb700 1 -- 172.21.15.5:0/362533761 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc011830 con 0x7f6bd80727b0 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.661+0000 7f6bd5ffb700 1 -- 172.21.15.5:0/362533761 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc011990 con 0x7f6bd80727b0 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bd5ffb700 1 -- 172.21.15.5:0/362533761 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc011d20 con 0x7f6bd80727b0 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bdd71a700 1 -- 172.21.15.5:0/362533761 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd80727b0 msgr2=0x7f6bd80c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/362533761 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd80727b0 0x7f6bd80c9a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0098f0 tx=0x7f6bcc009c00 comp rx=0 tx=0).stop 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bdd71a700 1 -- 172.21.15.5:0/362533761 shutdown_connections 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/362533761 >> v2:172.21.15.49:3300/0 conn(0x7f6bd815a7c0 0x7f6bd80cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/362533761 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd80727b0 0x7f6bd80c9a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bdd71a700 1 -- 172.21.15.5:0/362533761 >> 172.21.15.5:0/362533761 conn(0x7f6bd806a960 msgr2=0x7f6bd806ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bdd71a700 1 -- 172.21.15.5:0/362533761 shutdown_connections 2024-05-08T23:28:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.663+0000 7f6bdd71a700 1 -- 172.21.15.5:0/362533761 wait complete. 2024-05-08T23:28:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 Processor -- start 2024-05-08T23:28:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 -- start start 2024-05-08T23:28:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd8072260 0x7f6bd81484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bd80727b0 0x7f6bd815c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bd815a7c0 0x7f6bd815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd815b860 con 0x7f6bd815a7c0 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd815bb40 con 0x7f6bd80727b0 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd815be20 con 0x7f6bd8072260 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bd815a7c0 0x7f6bd815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd6ffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd8072260 0x7f6bd81484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bd80727b0 0x7f6bd815c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bd815a7c0 0x7f6bd815ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44250/0 (socket says 172.21.15.5:44250) 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd6ffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd8072260 0x7f6bd81484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.5:41148/0 (socket says 172.21.15.5:41148) 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 -- 172.21.15.5:0/1617807324 learned_addr learned my addr 172.21.15.5:0/1617807324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 -- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd8072260 msgr2=0x7f6bd81484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd8072260 0x7f6bd81484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 -- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bd80727b0 msgr2=0x7f6bd815c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bd80727b0 0x7f6bd815c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 -- 172.21.15.5:0/1617807324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc80094a0 con 0x7f6bd815a7c0 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd67fc700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bd80727b0 0x7f6bd815c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bd77fe700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bd815a7c0 0x7f6bd815ad10 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000cba0 tx=0x7f6bd000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bbf7fe700 1 -- 172.21.15.5:0/1617807324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd000aaa0 con 0x7f6bd815a7c0 2024-05-08T23:28:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bbf7fe700 1 -- 172.21.15.5:0/1617807324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd000ac00 con 0x7f6bd815a7c0 2024-05-08T23:28:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.665+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc009550 con 0x7f6bd815a7c0 2024-05-08T23:28:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.666+0000 7f6bbf7fe700 1 -- 172.21.15.5:0/1617807324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0007070 con 0x7f6bd815a7c0 2024-05-08T23:28:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.666+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd815c3e0 con 0x7f6bd815a7c0 2024-05-08T23:28:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.667+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd8074bc0 con 0x7f6bd815a7c0 2024-05-08T23:28:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.667+0000 7f6bbf7fe700 1 -- 172.21.15.5:0/1617807324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6bd00071d0 con 0x7f6bd815a7c0 2024-05-08T23:28:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.668+0000 7f6bbf7fe700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bc406ac90 0x7f6bc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.668+0000 7f6bbf7fe700 1 -- 172.21.15.5:0/1617807324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5529+0+0 (secure 0 0 0) 0x7f6bd0088eb0 con 0x7f6bd815a7c0 2024-05-08T23:28:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.668+0000 7f6bd6ffd700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bc406ac90 0x7f6bc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.668+0000 7f6bd6ffd700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bc406ac90 0x7f6bc406d150 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009cd0 tx=0x7f6bc8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:53.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.673+0000 7f6bbf7fe700 1 -- 172.21.15.5:0/1617807324 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6bd0054160 con 0x7f6bd815a7c0 2024-05-08T23:28:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:53.888+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6bd80ced00 con 0x7f6bd815a7c0 2024-05-08T23:28:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.498+0000 7f6bbf7fe700 1 -- 172.21.15.5:0/1617807324 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6bd0017030 con 0x7f6bd815a7c0 2024-05-08T23:28:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.503+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bc406ac90 msgr2=0x7f6bc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.503+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bc406ac90 0x7f6bc406d150 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009cd0 tx=0x7f6bc8009210 comp rx=0 tx=0).stop 2024-05-08T23:28:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.503+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bd815a7c0 msgr2=0x7f6bd815ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.503+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bd815a7c0 0x7f6bd815ad10 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000cba0 tx=0x7f6bd000ceb0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.504+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 shutdown_connections 2024-05-08T23:28:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.504+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f6bc406ac90 0x7f6bc406d150 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.504+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bd815a7c0 0x7f6bd815ad10 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.504+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6bd8072260 0x7f6bd81484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.504+0000 7f6bdd71a700 1 --2- 172.21.15.5:0/1617807324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bd80727b0 0x7f6bd815c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.504+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 >> 172.21.15.5:0/1617807324 conn(0x7f6bd806a960 msgr2=0x7f6bd815e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.504+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 shutdown_connections 2024-05-08T23:28:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.504+0000 7f6bdd71a700 1 -- 172.21.15.5:0/1617807324 wait complete. 2024-05-08T23:28:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-08T23:28:54.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T23:28:54.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.663+0000 7fc6461e7700 1 Processor -- start 2024-05-08T23:28:54.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.663+0000 7fc6461e7700 1 -- start start 2024-05-08T23:28:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc6461e7700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc64015b4e0 0x7fc6400b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc6461e7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc6400cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc6461e7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6400cf560 con 0x7fc64015b8c0 2024-05-08T23:28:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc6461e7700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6400cefe0 con 0x7fc64015b4e0 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc6461e7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6400ccff0 con 0x7fc6400d1910 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc63ffff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc6400cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc63f7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc64015b4e0 0x7fc6400b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc63effd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc64015b8c0 0x7fc6400d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:46974/0 (socket says 172.21.15.5:46974) 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.664+0000 7fc63effd700 1 -- 172.21.15.5:0/2739111117 learned_addr learned my addr 172.21.15.5:0/2739111117 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63ffff700 1 -- 172.21.15.5:0/2739111117 >> v2:172.21.15.49:3300/0 conn(0x7fc64015b4e0 msgr2=0x7fc6400b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63ffff700 1 --2- 172.21.15.5:0/2739111117 >> v2:172.21.15.49:3300/0 conn(0x7fc64015b4e0 0x7fc6400b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63ffff700 1 -- 172.21.15.5:0/2739111117 >> v1:172.21.15.49:6789/0 conn(0x7fc64015b8c0 legacy=0x7fc6400d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63ffff700 1 -- 172.21.15.5:0/2739111117 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6400cd2d0 con 0x7fc6400d1910 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63f7fe700 1 --2- 172.21.15.5:0/2739111117 >> v2:172.21.15.49:3300/0 conn(0x7fc64015b4e0 0x7fc6400b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63ffff700 1 --2- 172.21.15.5:0/2739111117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc6400cc2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc638009d90 tx=0x7fc63800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2f302fcb4ff038 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63e7fc700 1 -- 172.21.15.5:0/2739111117 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63800e070 con 0x7fc6400d1910 2024-05-08T23:28:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63e7fc700 1 -- 172.21.15.5:0/2739111117 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63800bb20 con 0x7fc6400d1910 2024-05-08T23:28:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.665+0000 7fc63e7fc700 1 -- 172.21.15.5:0/2739111117 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638012430 con 0x7fc6400d1910 2024-05-08T23:28:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.666+0000 7fc6461e7700 1 -- 172.21.15.5:0/2739111117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 msgr2=0x7fc6400cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.666+0000 7fc6461e7700 1 --2- 172.21.15.5:0/2739111117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc6400cc2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc638009d90 tx=0x7fc63800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.666+0000 7fc6461e7700 1 -- 172.21.15.5:0/2739111117 shutdown_connections 2024-05-08T23:28:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.666+0000 7fc6461e7700 1 --2- 172.21.15.5:0/2739111117 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc6400cc2c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.666+0000 7fc6461e7700 1 --2- 172.21.15.5:0/2739111117 >> v2:172.21.15.49:3300/0 conn(0x7fc64015b4e0 0x7fc6400b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.666+0000 7fc6461e7700 1 -- 172.21.15.5:0/2739111117 >> 172.21.15.5:0/2739111117 conn(0x7fc64008afe0 msgr2=0x7fc64008b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.666+0000 7fc6461e7700 1 -- 172.21.15.5:0/2739111117 shutdown_connections 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.666+0000 7fc6461e7700 1 -- 172.21.15.5:0/2739111117 wait complete. 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.668+0000 7fc6461e7700 1 Processor -- start 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.668+0000 7fc6461e7700 1 -- start start 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc6461e7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc64006ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc6461e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc64015b4e0 0x7fc64006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc6461e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64015b8c0 0x7fc6400c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc6461e7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64014bcc0 con 0x7fc64015b8c0 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc6461e7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6400c7010 con 0x7fc64015b4e0 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc6461e7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6400c6400 con 0x7fc6400d1910 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64015b8c0 0x7fc6400c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64015b8c0 0x7fc6400c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44262/0 (socket says 172.21.15.5:44262) 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63ffff700 1 -- 172.21.15.5:0/1717010040 learned_addr learned my addr 172.21.15.5:0/1717010040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63f7fe700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc64006ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63effd700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc64015b4e0 0x7fc64006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63ffff700 1 -- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 msgr2=0x7fc64006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63ffff700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc64006ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63ffff700 1 -- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc64015b4e0 msgr2=0x7fc64006d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63ffff700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc64015b4e0 0x7fc64006d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.669+0000 7fc63ffff700 1 -- 172.21.15.5:0/1717010040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6300094a0 con 0x7fc64015b8c0 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.670+0000 7fc63ffff700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64015b8c0 0x7fc6400c8570 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc638014040 tx=0x7fc638004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.670+0000 7fc627fff700 1 -- 172.21.15.5:0/1717010040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638012430 con 0x7fc64015b8c0 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.670+0000 7fc627fff700 1 -- 172.21.15.5:0/1717010040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc638020070 con 0x7fc64015b8c0 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.670+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc638009a70 con 0x7fc64015b8c0 2024-05-08T23:28:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.670+0000 7fc627fff700 1 -- 172.21.15.5:0/1717010040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638003ea0 con 0x7fc64015b8c0 2024-05-08T23:28:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.670+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6400c69c0 con 0x7fc64015b8c0 2024-05-08T23:28:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.671+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6400679e0 con 0x7fc64015b8c0 2024-05-08T23:28:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.671+0000 7fc627fff700 1 -- 172.21.15.5:0/1717010040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc638004280 con 0x7fc64015b8c0 2024-05-08T23:28:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.672+0000 7fc627fff700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc62c06ac90 0x7fc62c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.672+0000 7fc627fff700 1 -- 172.21.15.5:0/1717010040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc638022070 con 0x7fc64015b8c0 2024-05-08T23:28:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.672+0000 7fc63f7fe700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc62c06ac90 0x7fc62c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.673+0000 7fc63f7fe700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc62c06ac90 0x7fc62c06d150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc630009d00 tx=0x7fc630008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.677+0000 7fc627fff700 1 -- 172.21.15.5:0/1717010040 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc638056230 con 0x7fc64015b8c0 2024-05-08T23:28:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.894+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc640074bc0 con 0x7fc64015b8c0 2024-05-08T23:28:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.898+0000 7fc627fff700 1 -- 172.21.15.5:0/1717010040 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc638059850 con 0x7fc64015b8c0 2024-05-08T23:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc62c06ac90 msgr2=0x7fc62c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc62c06ac90 0x7fc62c06d150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc630009d00 tx=0x7fc630008040 comp rx=0 tx=0).stop 2024-05-08T23:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64015b8c0 msgr2=0x7fc6400c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64015b8c0 0x7fc6400c8570 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc638014040 tx=0x7fc638004ad0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 shutdown_connections 2024-05-08T23:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc62c06ac90 0x7fc62c06d150 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64015b8c0 0x7fc6400c8570 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc6400d1910 0x7fc64006ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 --2- 172.21.15.5:0/1717010040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc64015b4e0 0x7fc64006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 >> 172.21.15.5:0/1717010040 conn(0x7fc64008afe0 msgr2=0x7fc64008b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 shutdown_connections 2024-05-08T23:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:54.901+0000 7fc6461e7700 1 -- 172.21.15.5:0/1717010040 wait complete. 2024-05-08T23:28:54.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-05-08T23:28:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.056+0000 7f56470b2700 1 Processor -- start 2024-05-08T23:28:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.056+0000 7f56470b2700 1 -- start start 2024-05-08T23:28:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.057+0000 7f56470b2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f564015ef60 0x7f56400b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.057+0000 7f56470b2700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56400cfb90 0x7f56400b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.057+0000 7f56470b2700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564009ad40 con 0x7f56400cf000 2024-05-08T23:28:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.057+0000 7f56470b2700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56400cd400 con 0x7f56400cfb90 2024-05-08T23:28:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.057+0000 7f56470b2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56400ccb60 con 0x7f564015ef60 2024-05-08T23:28:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.057+0000 7f564564f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56400cfb90 0x7f56400b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.057+0000 7f563ffff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f56400cf000 0x7f56400cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34678/0 (socket says 172.21.15.5:34678) 2024-05-08T23:28:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.057+0000 7f563ffff700 1 -- 172.21.15.5:0/1115113350 learned_addr learned my addr 172.21.15.5:0/1115113350 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f5644e4e700 1 --2- 172.21.15.5:0/1115113350 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f564015ef60 0x7f56400b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f564564f700 1 -- 172.21.15.5:0/1115113350 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f564015ef60 msgr2=0x7f56400b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f564564f700 1 --2- 172.21.15.5:0/1115113350 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f564015ef60 0x7f56400b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f564564f700 1 -- 172.21.15.5:0/1115113350 >> v1:172.21.15.5:6789/0 conn(0x7f56400cf000 legacy=0x7f56400cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f564564f700 1 -- 172.21.15.5:0/1115113350 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400cce40 con 0x7f56400cfb90 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f563f7fe700 1 -- 172.21.15.5:0/1115113350 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188277254 0 0) 0x7f564009ad40 con 0x7f56400cf000 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f564564f700 1 --2- 172.21.15.5:0/1115113350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cfb90 0x7f56400b4620 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5638009d90 tx=0x7f563800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f82fcba4332380a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f563f7fe700 1 -- 172.21.15.5:0/1115113350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800e070 con 0x7f56400cfb90 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f563f7fe700 1 -- 172.21.15.5:0/1115113350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563800bbd0 con 0x7f56400cfb90 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.058+0000 7f563f7fe700 1 -- 172.21.15.5:0/1115113350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638012430 con 0x7f56400cfb90 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.059+0000 7f56470b2700 1 -- 172.21.15.5:0/1115113350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cfb90 msgr2=0x7f56400b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.059+0000 7f56470b2700 1 --2- 172.21.15.5:0/1115113350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cfb90 0x7f56400b4620 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5638009d90 tx=0x7f563800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.059+0000 7f56470b2700 1 -- 172.21.15.5:0/1115113350 shutdown_connections 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.059+0000 7f56470b2700 1 --2- 172.21.15.5:0/1115113350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cfb90 0x7f56400b4620 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.059+0000 7f56470b2700 1 --2- 172.21.15.5:0/1115113350 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f564015ef60 0x7f56400b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.059+0000 7f56470b2700 1 -- 172.21.15.5:0/1115113350 >> 172.21.15.5:0/1115113350 conn(0x7f56400b2f20 msgr2=0x7f56400b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:55.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.059+0000 7f56470b2700 1 -- 172.21.15.5:0/1115113350 shutdown_connections 2024-05-08T23:28:55.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.059+0000 7f56470b2700 1 -- 172.21.15.5:0/1115113350 wait complete. 2024-05-08T23:28:55.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.060+0000 7f56470b2700 1 Processor -- start 2024-05-08T23:28:55.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.060+0000 7f56470b2700 1 -- start start 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f56470b2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cf000 0x7f564014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f56470b2700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f56400cfb90 0x7f564009d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f56470b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f564015ef60 0x7f564009d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f56470b2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564009e080 con 0x7f56400cf000 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f56470b2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564009e360 con 0x7f564015ef60 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f56470b2700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564009e640 con 0x7f56400cfb90 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f5644e4e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cf000 0x7f564014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f5644e4e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cf000 0x7f564014fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44278/0 (socket says 172.21.15.5:44278) 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.061+0000 7f5644e4e700 1 -- 172.21.15.5:0/1715999027 learned_addr learned my addr 172.21.15.5:0/1715999027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:55.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f563ffff700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f56400cfb90 0x7f564009d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f564564f700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f564015ef60 0x7f564009d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f5644e4e700 1 -- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f56400cfb90 msgr2=0x7f564009d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f5644e4e700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f56400cfb90 0x7f564009d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f5644e4e700 1 -- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f564015ef60 msgr2=0x7f564009d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f5644e4e700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f564015ef60 0x7f564009d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f5644e4e700 1 -- 172.21.15.5:0/1715999027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5638009a70 con 0x7f56400cf000 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f564564f700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f564015ef60 0x7f564009d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f563ffff700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f56400cfb90 0x7f564009d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f5644e4e700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cf000 0x7f564014fbc0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f563000d800 tx=0x7f563000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f563d7fa700 1 -- 172.21.15.5:0/1715999027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56300125f0 con 0x7f56400cf000 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f563d7fa700 1 -- 172.21.15.5:0/1715999027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630012750 con 0x7f56400cf000 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564009e920 con 0x7f56400cf000 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f563d7fa700 1 -- 172.21.15.5:0/1715999027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630012b00 con 0x7f56400cf000 2024-05-08T23:28:55.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.062+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5640150100 con 0x7f56400cf000 2024-05-08T23:28:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.063+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564009ee40 con 0x7f56400cf000 2024-05-08T23:28:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.064+0000 7f563d7fa700 1 -- 172.21.15.5:0/1715999027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5630012c60 con 0x7f56400cf000 2024-05-08T23:28:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.064+0000 7f563d7fa700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f562006ac90 0x7f562006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.064+0000 7f563d7fa700 1 -- 172.21.15.5:0/1715999027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f56300894e0 con 0x7f56400cf000 2024-05-08T23:28:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.064+0000 7f563ffff700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f562006ac90 0x7f562006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.065+0000 7f563ffff700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f562006ac90 0x7f562006d150 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f56400cae70 tx=0x7f5628009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:55.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.069+0000 7f563d7fa700 1 -- 172.21.15.5:0/1715999027 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5630054910 con 0x7f56400cf000 2024-05-08T23:28:55.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.378+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) v1 -- 0x7f56400cc180 con 0x7f56400cf000 2024-05-08T23:28:55.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.495+0000 7f563d7fa700 1 -- 172.21.15.5:0/1715999027 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v406) v1 ==== 165+0+0 (secure 0 0 0) 0x7f5630057f30 con 0x7f56400cf000 2024-05-08T23:28:55.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.497+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f562006ac90 msgr2=0x7f562006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:55.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.497+0000 7f56470b2700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f562006ac90 0x7f562006d150 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f56400cae70 tx=0x7f5628009040 comp rx=0 tx=0).stop 2024-05-08T23:28:55.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.497+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cf000 msgr2=0x7f564014fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:55.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.497+0000 7f56470b2700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cf000 0x7f564014fbc0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f563000d800 tx=0x7f563000dbc0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.498+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 shutdown_connections 2024-05-08T23:28:55.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.498+0000 7f56470b2700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f562006ac90 0x7f562006d150 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.498+0000 7f56470b2700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f564015ef60 0x7f564009d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.498+0000 7f56470b2700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56400cf000 0x7f564014fbc0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.498+0000 7f56470b2700 1 --2- 172.21.15.5:0/1715999027 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f56400cfb90 0x7f564009d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:55.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.498+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 >> 172.21.15.5:0/1715999027 conn(0x7f56400b2f20 msgr2=0x7f564015a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:55.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.498+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 shutdown_connections 2024-05-08T23:28:55.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:55.498+0000 7f56470b2700 1 -- 172.21.15.5:0/1715999027 wait complete. 2024-05-08T23:28:55.512 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-05-08T23:28:55.512 INFO:tasks.cephfs_test_runner: 2024-05-08T23:28:55.512 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T23:28:55.512 INFO:tasks.cephfs_test_runner:Ran 6 tests in 435.850s 2024-05-08T23:28:55.513 INFO:tasks.cephfs_test_runner: 2024-05-08T23:28:55.513 INFO:tasks.cephfs_test_runner:OK 2024-05-08T23:28:55.513 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-08T23:28:55.536 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T23:28:55.548 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T23:28:55.548 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:28:55.548 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:55.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:28:55.573 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-08T23:28:55.573 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:55.643 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2962 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-05-08T23:28:55.644 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:28:55.644 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:55.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:28:55.670 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-08T23:28:55.670 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:28:55.734 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244112k,nr_inodes=4061028,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18176 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252508k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-05-08T23:28:55.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:55.734 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:28:55.757 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:28:55.757 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:28:55.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:55.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:28:55.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:55.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:28:55.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:55.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-08T23:28:55.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:55.989 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-08T23:28:56.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:56.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-08T23:28:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-05-08T23:28:56.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:28:56.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:56.097 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-08T23:28:56.123 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T23:28:56.123 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T23:28:56.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:56.123 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T23:28:56.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:56.215 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T23:28:56.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:56.284 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-05-08T23:28:56.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:56.368 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-08T23:28:56.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:28:56.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-05-08T23:28:56.461 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-05-08T23:28:56.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:28:56.463 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T23:28:56.475 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T23:28:56.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:28:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.622+0000 7ff2e5438700 1 Processor -- start 2024-05-08T23:28:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.622+0000 7ff2e5438700 1 -- start start 2024-05-08T23:28:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.622+0000 7ff2e5438700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e00ac1c0 0x7ff2e00ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2e5438700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e015f190 con 0x7ff2e015b7b0 2024-05-08T23:28:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2e5438700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e015f470 con 0x7ff2e00b2420 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2e5438700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e01600f0 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2deffd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff2e015b7b0 0x7ff2e00cb3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34688/0 (socket says 172.21.15.5:34688) 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2deffd700 1 -- 172.21.15.5:0/2645261754 learned_addr learned my addr 172.21.15.5:0/2645261754 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2de7fc700 1 --2- 172.21.15.5:0/2645261754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e00ac1c0 0x7ff2e00ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2ddffb700 1 -- 172.21.15.5:0/2645261754 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402367073 0 0) 0x7ff2e015f190 con 0x7ff2e015b7b0 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2ddffb700 1 -- 172.21.15.5:0/2645261754 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c80035b0 con 0x7ff2e015b7b0 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2ddffb700 1 -- 172.21.15.5:0/2645261754 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966791722 0 0) 0x7ff2e015f470 con 0x7ff2e00b2420 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2ddffb700 1 -- 172.21.15.5:0/2645261754 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e015f190 con 0x7ff2e00b2420 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2de7fc700 1 -- 172.21.15.5:0/2645261754 >> v1:172.21.15.49:6789/0 conn(0x7ff2e00b2420 legacy=0x7ff2e00b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2de7fc700 1 -- 172.21.15.5:0/2645261754 >> v1:172.21.15.5:6789/0 conn(0x7ff2e015b7b0 legacy=0x7ff2e00cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2de7fc700 1 -- 172.21.15.5:0/2645261754 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e015f750 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2de7fc700 1 --2- 172.21.15.5:0/2645261754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e00ac1c0 0x7ff2e00ac640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009870 tx=0x7ff2d4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e93ab4b677aeae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.623+0000 7ff2ddffb700 1 -- 172.21.15.5:0/2645261754 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4011830 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2ddffb700 1 -- 172.21.15.5:0/2645261754 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4011990 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2ddffb700 1 -- 172.21.15.5:0/2645261754 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4011d20 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2e5438700 1 -- 172.21.15.5:0/2645261754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e00ac1c0 msgr2=0x7ff2e00ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2e5438700 1 --2- 172.21.15.5:0/2645261754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e00ac1c0 0x7ff2e00ac640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009870 tx=0x7ff2d4009b80 comp rx=0 tx=0).stop 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2e5438700 1 -- 172.21.15.5:0/2645261754 shutdown_connections 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2e5438700 1 --2- 172.21.15.5:0/2645261754 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e00ac1c0 0x7ff2e00ac640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2e5438700 1 -- 172.21.15.5:0/2645261754 >> 172.21.15.5:0/2645261754 conn(0x7ff2e00cf270 msgr2=0x7ff2e00cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2e5438700 1 -- 172.21.15.5:0/2645261754 shutdown_connections 2024-05-08T23:28:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.624+0000 7ff2e5438700 1 -- 172.21.15.5:0/2645261754 wait complete. 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.626+0000 7ff2e5438700 1 Processor -- start 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.626+0000 7ff2e5438700 1 -- start start 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.626+0000 7ff2e5438700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e00ac1c0 0x7ff2e015c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.626+0000 7ff2e5438700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e00b2420 0x7ff2e014f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.626+0000 7ff2e5438700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e015e490 0x7ff2e006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.626+0000 7ff2e5438700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e015ce40 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.626+0000 7ff2e5438700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e015d120 con 0x7ff2e00b2420 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.626+0000 7ff2e5438700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e015d400 con 0x7ff2e015e490 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e00ac1c0 0x7ff2e015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e00ac1c0 0x7ff2e015c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44294/0 (socket says 172.21.15.5:44294) 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 -- 172.21.15.5:0/1392667357 learned_addr learned my addr 172.21.15.5:0/1392667357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2de7fc700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e00b2420 0x7ff2e014f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2df7fe700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e015e490 0x7ff2e006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 -- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e015e490 msgr2=0x7ff2e006d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e015e490 0x7ff2e006d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 -- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e00b2420 msgr2=0x7ff2e014f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e00b2420 0x7ff2e014f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 -- 172.21.15.5:0/1392667357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40094a0 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2deffd700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e00ac1c0 0x7ff2e015c760 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c700 tx=0x7ff2d000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2c77fe700 1 -- 172.21.15.5:0/1392667357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0005d70 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2c77fe700 1 -- 172.21.15.5:0/1392667357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d00070d0 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.627+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e015d6e0 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.628+0000 7ff2c77fe700 1 -- 172.21.15.5:0/1392667357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0007460 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.628+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e014fd50 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.629+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e00c8560 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.629+0000 7ff2c77fe700 1 -- 172.21.15.5:0/1392667357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff2d00075c0 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.630+0000 7ff2c77fe700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff2cc06ac90 0x7ff2cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.630+0000 7ff2c77fe700 1 -- 172.21.15.5:0/1392667357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff2d0088510 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.630+0000 7ff2de7fc700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff2cc06ac90 0x7ff2cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.630+0000 7ff2de7fc700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff2cc06ac90 0x7ff2cc06d150 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097c0 tx=0x7ff2d401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:56.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.635+0000 7ff2c77fe700 1 -- 172.21.15.5:0/1392667357 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff2d0053940 con 0x7ff2e00ac1c0 2024-05-08T23:28:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.848+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2e015fd10 con 0x7ff2cc06ac90 2024-05-08T23:28:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.851+0000 7ff2c77fe700 1 -- 172.21.15.5:0/1392667357 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51311 (secure 0 0 0) 0x7ff2d4011b00 con 0x7ff2cc06ac90 2024-05-08T23:28:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff2cc06ac90 msgr2=0x7ff2cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff2cc06ac90 0x7ff2cc06d150 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097c0 tx=0x7ff2d401c040 comp rx=0 tx=0).stop 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e00ac1c0 msgr2=0x7ff2e015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e00ac1c0 0x7ff2e015c760 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c700 tx=0x7ff2d000ca10 comp rx=0 tx=0).stop 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 shutdown_connections 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7ff2cc06ac90 0x7ff2cc06d150 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff2e015e490 0x7ff2e006d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e00ac1c0 0x7ff2e015c760 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 --2- 172.21.15.5:0/1392667357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e00b2420 0x7ff2e014f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 >> 172.21.15.5:0/1392667357 conn(0x7ff2e00cf270 msgr2=0x7ff2e00ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 shutdown_connections 2024-05-08T23:28:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:56.853+0000 7ff2e5438700 1 -- 172.21.15.5:0/1392667357 wait complete. 2024-05-08T23:28:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:28:56.867 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-05-08T23:28:55.410100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259840,"kb_used_data":10912,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714880,"statfs":{"total":773094113280,"available":772828037120,"internally_reserved":0,"allocated":11173888,"data_stored":5590384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":254791221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-121676,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-245,"num_write_kb":-203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001492"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":229,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970501+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:28:01.970501+0000","last_peered":"2024-05-08T23:28:01.970501+0000","last_clean":"2024-05-08T23:28:01.970501+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:28:01.970501+0000","last_undegraded":"2024-05-08T23:28:01.970501+0000","last_fullsized":"2024-05-08T23:28:01.970501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1814528,"data_stored":1111961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.82499999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.628},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.873,"5min":1.038,"15min":1.038},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.81200000000000006,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46600000000000003}]},{"osd":2,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.54900000000000004},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76700000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.67900000000000005}]},{"osd":3,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76800000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.36499999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.85499999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69199999999999995,"5min":0.87,"15min":0.87},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93500000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.89700000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574947,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1253376,"data_stored":559180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.54000000000000004},"min":{"1min":0.44500000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.39700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83299999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.88200000000000001,"15min":0.94999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.86899999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.48899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73999999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.496},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83099999999999996,"5min":0.85699999999999998,"15min":1.115},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72199999999999998,"5min":0.878,"15min":1.0229999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.98899999999999999,"15min":1.0009999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.752,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.41099999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78900000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":1.052},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81799999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.877,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.46899999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1245184,"data_stored":543799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52000000000000002,"5min":0.61099999999999999,"15min":0.91000000000000003},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.93100000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.41099999999999998},"min":{"1min":0.28699999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.55400000000000005,"5min":0.63700000000000001,"15min":0.88400000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.42699999999999999,"15min":0.44700000000000001},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60999999999999999,"5min":0.61399999999999999,"15min":0.96999999999999997},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64300000000000002,"5min":0.65300000000000002,"15min":1.01},"last":0.44},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.47399999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.627,"5min":0.68200000000000005,"15min":1.028},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":0.91100000000000003,"15min":1.1479999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.67500000000000004},"min":{"1min":0.45500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88500000000000001,"5min":1.0169999999999999,"15min":1.1579999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86499999999999999,"5min":0.89000000000000001,"15min":1.0489999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.67800000000000005},"min":{"1min":0.503,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.115},"last":0.55800000000000005}]},{"osd":6,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95199999999999996,"5min":1.0089999999999999,"15min":1.2050000000000001},"last":0.627},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65700000000000003,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.105},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.47999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1,"5min":1,"15min":1.1359999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.68799999999999994},"min":{"1min":0.55600000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":1.0609999999999999},"last":0.60199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1810432,"data_stored":1112375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.63400000000000001,"15min":7.7610000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40500000000000003,"15min":0.65700000000000003},"min":{"1min":0.27200000000000002,"5min":0.249,"15min":0.24399999999999999},"max":{"1min":0.58699999999999997,"5min":0.65800000000000003,"15min":7.9889999999999999},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41899999999999998,"15min":0.71499999999999997},"min":{"1min":0.32700000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":7.7869999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.436,"15min":0.69099999999999995},"min":{"1min":0.32100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56599999999999995,"5min":0.69899999999999995,"15min":8.0519999999999996},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41299999999999998,"15min":0.44500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":2.9420000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.46400000000000002,"15min":0.73799999999999999},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67800000000000005,"5min":0.79800000000000004,"15min":7.6440000000000001},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59899999999999998,"15min":0.874},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.93100000000000005,"15min":7.9139999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59599999999999997,"15min":0.84999999999999998},"min":{"1min":0.47899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.96199999999999997,"15min":7.1879999999999997},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.86099999999999999},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":7.1120000000000001},"last":0.502},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.94799999999999995},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90600000000000003,"5min":0.94199999999999995,"15min":7.6970000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63900000000000001,"15min":0.91400000000000003},"min":{"1min":0.48199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95099999999999996,"15min":7.734},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.89600000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83099999999999996,"5min":0.88,"15min":8.0220000000000002},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65100000000000002,"15min":0.90800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":7.9539999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.627,"15min":0.87},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.91300000000000003,"15min":7.1600000000000001},"last":0.77200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1204224,"data_stored":520821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.39900000000000002,"15min":0.39200000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.40300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.68400000000000005},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45000000000000001,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.629,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.42899999999999999},"min":{"1min":0.35699999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53000000000000003,"5min":0.64800000000000002,"15min":0.67300000000000004},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.46800000000000003},"min":{"1min":0.34599999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.627,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.377}]},{"osd":4,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.54400000000000004,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.879,"5min":0.93799999999999994,"15min":1.024},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.79800000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.502,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0129999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.0549999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65000000000000002,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83099999999999996,"5min":0.89200000000000002,"15min":1.159},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.53500000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.55400000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91200000000000003,"5min":0.92500000000000004,"15min":1.194},"last":0.71999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":602558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":1.46},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.41499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.2749999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.27700000000000002},"max":{"1min":0.54800000000000004,"5min":1.327,"15min":1.327},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":0.68400000000000005,"5min":1.109,"15min":1.5649999999999999},"last":0.31}]},{"osd":2,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44700000000000001,"15min":0.77200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63,"5min":1.1479999999999999,"15min":11.964},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48199999999999998,"15min":0.51100000000000001},"min":{"1min":0.35699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.73899999999999999,"15min":1.593},"last":0.438}]},{"osd":4,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63900000000000001,"15min":0.65800000000000003},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94299999999999995,"5min":1.214,"15min":2.4590000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.536},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.016,"5min":1.2969999999999999,"15min":1.3100000000000001},"last":0.57499999999999996}]},{"osd":6,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":1.0389999999999999,"15min":1.5029999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.4850000000000001,"15min":2.5230000000000001},"last":0.76700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1327104,"data_stored":618653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.625},"min":{"1min":0.27200000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71799999999999997,"5min":0.75700000000000001,"15min":2.0209999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.68999999999999995},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.78400000000000003,"5min":0.91200000000000003,"15min":4.0960000000000001},"last":0.34599999999999997}]},{"osd":1,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56599999999999995,"15min":0.65000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68799999999999994,"5min":1.0349999999999999,"15min":2.145},"last":0.435},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.66500000000000004},"min":{"1min":0.32800000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76500000000000001,"5min":0.88500000000000001,"15min":2.2530000000000001},"last":0.374}]},{"osd":2,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":2.3769999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.749},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":4.0389999999999997},"last":0.42399999999999999}]},{"osd":3,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56599999999999995,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.747,"5min":0.98299999999999998,"15min":1.0800000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":1.0289999999999999},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70299999999999996,"5min":0.93000000000000005,"15min":12.127000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.56799999999999995},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.79700000000000004,"15min":0.877},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.56299999999999994},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.79000000000000004,"15min":0.84299999999999997},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53500000000000003,"15min":0.60499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.81499999999999995,"15min":0.93700000000000006},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.67300000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68300000000000005,"5min":0.877,"15min":1.24},"last":0.68300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":521037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.32700000000000001,"15min":0.27800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88700000000000001,"15min":4.6710000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.83699999999999997},"min":{"1min":0.41499999999999998,"5min":0.35199999999999998,"15min":0.30499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":8.798},"last":0.57399999999999995}]},{"osd":1,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.73499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74099999999999999,"5min":0.82899999999999996,"15min":4.71},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.89900000000000002},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78900000000000003,"5min":1.0109999999999999,"15min":8.6880000000000006},"last":0.65000000000000002}]},{"osd":2,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.73199999999999998},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":1.0009999999999999,"15min":4.8940000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.626,"15min":0.76000000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.443},"max":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":4.8559999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60799999999999998,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69099999999999995,"5min":0.91800000000000004,"15min":8.8770000000000007},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.621,"15min":0.93899999999999995},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85899999999999999,"5min":1.0489999999999999,"15min":10.427},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48999999999999999,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.874},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54600000000000004,"15min":0.56499999999999995},"min":{"1min":0.35099999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.86799999999999999,"15min":0.95899999999999996},"last":0.67800000000000005}]},{"osd":6,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93999999999999995,"5min":1.141,"15min":1.141},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60899999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":1.01,"15min":1.01},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65400000000000003,"15min":0.66900000000000004},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86299999999999999,"5min":1.097,"15min":1.135},"last":0.68799999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:28:56.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:28:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.006+0000 7fc7149a0700 1 Processor -- start 2024-05-08T23:28:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.006+0000 7fc7149a0700 1 -- start start 2024-05-08T23:28:57.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc7149a0700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc710159380 0x7fc710071e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc7149a0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc710066b80 0x7fc71006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc7149a0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc71006beb0 0x7fc7100b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc7149a0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71006c320 con 0x7fc71006beb0 2024-05-08T23:28:57.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc7149a0700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71015fea0 con 0x7fc710159380 2024-05-08T23:28:57.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc7149a0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc710159760 con 0x7fc710066b80 2024-05-08T23:28:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70ed9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc71006beb0 0x7fc7100b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70ed9d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc71006beb0 0x7fc7100b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44296/0 (socket says 172.21.15.5:44296) 2024-05-08T23:28:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70ed9d700 1 -- 172.21.15.5:0/2840029675 learned_addr learned my addr 172.21.15.5:0/2840029675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70e59c700 1 --2- 172.21.15.5:0/2840029675 >> v2:172.21.15.49:3300/0 conn(0x7fc710159380 0x7fc710071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/2840029675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc710066b80 0x7fc71006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70ed9d700 1 -- 172.21.15.5:0/2840029675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc710066b80 msgr2=0x7fc71006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/2840029675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc710066b80 0x7fc71006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70ed9d700 1 -- 172.21.15.5:0/2840029675 >> v2:172.21.15.49:3300/0 conn(0x7fc710159380 msgr2=0x7fc710071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.007+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/2840029675 >> v2:172.21.15.49:3300/0 conn(0x7fc710159380 0x7fc710071e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.008+0000 7fc70ed9d700 1 -- 172.21.15.5:0/2840029675 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71015f2d0 con 0x7fc71006beb0 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.008+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/2840029675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc71006beb0 0x7fc7100b2af0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fc71006d240 tx=0x7fc70800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a310c5eab93383 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.008+0000 7fc70d59a700 1 -- 172.21.15.5:0/2840029675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70800e070 con 0x7fc71006beb0 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.008+0000 7fc70d59a700 1 -- 172.21.15.5:0/2840029675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70800b9c0 con 0x7fc71006beb0 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.008+0000 7fc70d59a700 1 -- 172.21.15.5:0/2840029675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70800bd70 con 0x7fc71006beb0 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.008+0000 7fc7149a0700 1 -- 172.21.15.5:0/2840029675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc71006beb0 msgr2=0x7fc7100b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.008+0000 7fc7149a0700 1 --2- 172.21.15.5:0/2840029675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc71006beb0 0x7fc7100b2af0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fc71006d240 tx=0x7fc70800b330 comp rx=0 tx=0).stop 2024-05-08T23:28:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.009+0000 7fc7149a0700 1 -- 172.21.15.5:0/2840029675 shutdown_connections 2024-05-08T23:28:57.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.009+0000 7fc7149a0700 1 --2- 172.21.15.5:0/2840029675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc71006beb0 0x7fc7100b2af0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.009+0000 7fc7149a0700 1 --2- 172.21.15.5:0/2840029675 >> v2:172.21.15.49:3300/0 conn(0x7fc710159380 0x7fc710071e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.009+0000 7fc7149a0700 1 --2- 172.21.15.5:0/2840029675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc710066b80 0x7fc71006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.009+0000 7fc7149a0700 1 -- 172.21.15.5:0/2840029675 >> 172.21.15.5:0/2840029675 conn(0x7fc710071350 msgr2=0x7fc710071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:57.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.009+0000 7fc7149a0700 1 -- 172.21.15.5:0/2840029675 shutdown_connections 2024-05-08T23:28:57.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.009+0000 7fc7149a0700 1 -- 172.21.15.5:0/2840029675 wait complete. 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.010+0000 7fc7149a0700 1 Processor -- start 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.011+0000 7fc7149a0700 1 -- start start 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.011+0000 7fc7149a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710066b80 0x7fc71014d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc7149a0700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc71006beb0 0x7fc71014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc7149a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc710159380 0x7fc7100af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc7149a0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71014dba0 con 0x7fc710066b80 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc7149a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71014f490 con 0x7fc710159380 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc7149a0700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71014e880 con 0x7fc71006beb0 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710066b80 0x7fc71014d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710066b80 0x7fc71014d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44304/0 (socket says 172.21.15.5:44304) 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 -- 172.21.15.5:0/237554339 learned_addr learned my addr 172.21.15.5:0/237554339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc71006beb0 0x7fc71014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc710159380 0x7fc7100af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 -- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc71006beb0 msgr2=0x7fc71014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc71006beb0 0x7fc71014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 -- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc710159380 msgr2=0x7fc7100af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc710159380 0x7fc7100af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 -- 172.21.15.5:0/237554339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708009b30 con 0x7fc710066b80 2024-05-08T23:28:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc70e59c700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710066b80 0x7fc71014d660 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d830 tx=0x7fc70400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc6feffd700 1 -- 172.21.15.5:0/237554339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040125f0 con 0x7fc710066b80 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc6feffd700 1 -- 172.21.15.5:0/237554339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc704012750 con 0x7fc710066b80 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71014eb60 con 0x7fc710066b80 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.012+0000 7fc6feffd700 1 -- 172.21.15.5:0/237554339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704012b00 con 0x7fc710066b80 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.013+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71014f120 con 0x7fc710066b80 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.014+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7100b2700 con 0x7fc710066b80 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.014+0000 7fc6feffd700 1 -- 172.21.15.5:0/237554339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc704012c60 con 0x7fc710066b80 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.015+0000 7fc6feffd700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6f006ac90 0x7fc6f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.015+0000 7fc6feffd700 1 -- 172.21.15.5:0/237554339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc7040890a0 con 0x7fc710066b80 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.015+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6f006ac90 0x7fc6f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.015+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6f006ac90 0x7fc6f006d150 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b0a70 tx=0x7fc700008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.019+0000 7fc6feffd700 1 -- 172.21.15.5:0/237554339 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc704054610 con 0x7fc710066b80 2024-05-08T23:28:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.232+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc71015fb30 con 0x7fc6f006ac90 2024-05-08T23:28:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.234+0000 7fc6feffd700 1 -- 172.21.15.5:0/237554339 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51311 (secure 0 0 0) 0x7fc700009e40 con 0x7fc6f006ac90 2024-05-08T23:28:57.235 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:57.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6f006ac90 msgr2=0x7fc6f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6f006ac90 0x7fc6f006d150 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b0a70 tx=0x7fc700008040 comp rx=0 tx=0).stop 2024-05-08T23:28:57.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710066b80 msgr2=0x7fc71014d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710066b80 0x7fc71014d660 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d830 tx=0x7fc70400db40 comp rx=0 tx=0).stop 2024-05-08T23:28:57.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 shutdown_connections 2024-05-08T23:28:57.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc6f006ac90 0x7fc6f006d150 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc710159380 0x7fc7100af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710066b80 0x7fc71014d660 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 --2- 172.21.15.5:0/237554339 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc71006beb0 0x7fc71014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 >> 172.21.15.5:0/237554339 conn(0x7fc710071350 msgr2=0x7fc7100b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 shutdown_connections 2024-05-08T23:28:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.237+0000 7fc7149a0700 1 -- 172.21.15.5:0/237554339 wait complete. 2024-05-08T23:28:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:28:57.251 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-05-08T23:28:55.410100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259840,"kb_used_data":10912,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714880,"statfs":{"total":773094113280,"available":772828037120,"internally_reserved":0,"allocated":11173888,"data_stored":5590384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":254791221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-121676,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-245,"num_write_kb":-203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001492"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":229,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T23:28:01.970501+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:28:01.970501+0000","last_peered":"2024-05-08T23:28:01.970501+0000","last_clean":"2024-05-08T23:28:01.970501+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:28:01.970501+0000","last_undegraded":"2024-05-08T23:28:01.970501+0000","last_fullsized":"2024-05-08T23:28:01.970501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1814528,"data_stored":1111961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.82499999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.628},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.873,"5min":1.038,"15min":1.038},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.81200000000000006,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46600000000000003}]},{"osd":2,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.54900000000000004},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76700000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.67900000000000005}]},{"osd":3,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76800000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.36499999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.85499999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69199999999999995,"5min":0.87,"15min":0.87},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93500000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.89700000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574947,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1253376,"data_stored":559180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.54000000000000004},"min":{"1min":0.44500000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.39700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83299999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.88200000000000001,"15min":0.94999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.86899999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.48899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73999999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.496},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83099999999999996,"5min":0.85699999999999998,"15min":1.115},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72199999999999998,"5min":0.878,"15min":1.0229999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.98899999999999999,"15min":1.0009999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.752,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.41099999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78900000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":1.052},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81799999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.877,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.46899999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1245184,"data_stored":543799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52000000000000002,"5min":0.61099999999999999,"15min":0.91000000000000003},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.93100000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.41099999999999998},"min":{"1min":0.28699999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.55400000000000005,"5min":0.63700000000000001,"15min":0.88400000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.42699999999999999,"15min":0.44700000000000001},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60999999999999999,"5min":0.61399999999999999,"15min":0.96999999999999997},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64300000000000002,"5min":0.65300000000000002,"15min":1.01},"last":0.44},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.47399999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.627,"5min":0.68200000000000005,"15min":1.028},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":0.91100000000000003,"15min":1.1479999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.67500000000000004},"min":{"1min":0.45500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88500000000000001,"5min":1.0169999999999999,"15min":1.1579999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86499999999999999,"5min":0.89000000000000001,"15min":1.0489999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.67800000000000005},"min":{"1min":0.503,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.115},"last":0.55800000000000005}]},{"osd":6,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95199999999999996,"5min":1.0089999999999999,"15min":1.2050000000000001},"last":0.627},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65700000000000003,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.105},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.47999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1,"5min":1,"15min":1.1359999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.68799999999999994},"min":{"1min":0.55600000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":1.0609999999999999},"last":0.60199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1810432,"data_stored":1112375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.63400000000000001,"15min":7.7610000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40500000000000003,"15min":0.65700000000000003},"min":{"1min":0.27200000000000002,"5min":0.249,"15min":0.24399999999999999},"max":{"1min":0.58699999999999997,"5min":0.65800000000000003,"15min":7.9889999999999999},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41899999999999998,"15min":0.71499999999999997},"min":{"1min":0.32700000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":7.7869999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.436,"15min":0.69099999999999995},"min":{"1min":0.32100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56599999999999995,"5min":0.69899999999999995,"15min":8.0519999999999996},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41299999999999998,"15min":0.44500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":2.9420000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.46400000000000002,"15min":0.73799999999999999},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67800000000000005,"5min":0.79800000000000004,"15min":7.6440000000000001},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59899999999999998,"15min":0.874},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.93100000000000005,"15min":7.9139999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59599999999999997,"15min":0.84999999999999998},"min":{"1min":0.47899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.96199999999999997,"15min":7.1879999999999997},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.86099999999999999},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":7.1120000000000001},"last":0.502},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.94799999999999995},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90600000000000003,"5min":0.94199999999999995,"15min":7.6970000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63900000000000001,"15min":0.91400000000000003},"min":{"1min":0.48199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95099999999999996,"15min":7.734},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.89600000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83099999999999996,"5min":0.88,"15min":8.0220000000000002},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65100000000000002,"15min":0.90800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":7.9539999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.627,"15min":0.87},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.91300000000000003,"15min":7.1600000000000001},"last":0.77200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1204224,"data_stored":520821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.39900000000000002,"15min":0.39200000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.40300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.68400000000000005},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45000000000000001,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.629,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.42899999999999999},"min":{"1min":0.35699999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53000000000000003,"5min":0.64800000000000002,"15min":0.67300000000000004},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.46800000000000003},"min":{"1min":0.34599999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.627,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.377}]},{"osd":4,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.54400000000000004,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.879,"5min":0.93799999999999994,"15min":1.024},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.79800000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.502,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0129999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.0549999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65000000000000002,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83099999999999996,"5min":0.89200000000000002,"15min":1.159},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.53500000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.55400000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91200000000000003,"5min":0.92500000000000004,"15min":1.194},"last":0.71999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":602558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":1.46},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.41499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.2749999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.27700000000000002},"max":{"1min":0.54800000000000004,"5min":1.327,"15min":1.327},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":0.68400000000000005,"5min":1.109,"15min":1.5649999999999999},"last":0.31}]},{"osd":2,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44700000000000001,"15min":0.77200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63,"5min":1.1479999999999999,"15min":11.964},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48199999999999998,"15min":0.51100000000000001},"min":{"1min":0.35699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.73899999999999999,"15min":1.593},"last":0.438}]},{"osd":4,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63900000000000001,"15min":0.65800000000000003},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94299999999999995,"5min":1.214,"15min":2.4590000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.536},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.016,"5min":1.2969999999999999,"15min":1.3100000000000001},"last":0.57499999999999996}]},{"osd":6,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":1.0389999999999999,"15min":1.5029999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.4850000000000001,"15min":2.5230000000000001},"last":0.76700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1327104,"data_stored":618653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.625},"min":{"1min":0.27200000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71799999999999997,"5min":0.75700000000000001,"15min":2.0209999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.68999999999999995},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.78400000000000003,"5min":0.91200000000000003,"15min":4.0960000000000001},"last":0.34599999999999997}]},{"osd":1,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56599999999999995,"15min":0.65000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68799999999999994,"5min":1.0349999999999999,"15min":2.145},"last":0.435},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.66500000000000004},"min":{"1min":0.32800000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76500000000000001,"5min":0.88500000000000001,"15min":2.2530000000000001},"last":0.374}]},{"osd":2,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":2.3769999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.749},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":4.0389999999999997},"last":0.42399999999999999}]},{"osd":3,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56599999999999995,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.747,"5min":0.98299999999999998,"15min":1.0800000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":1.0289999999999999},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70299999999999996,"5min":0.93000000000000005,"15min":12.127000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.56799999999999995},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.79700000000000004,"15min":0.877},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.56299999999999994},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.79000000000000004,"15min":0.84299999999999997},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53500000000000003,"15min":0.60499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.81499999999999995,"15min":0.93700000000000006},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.67300000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68300000000000005,"5min":0.877,"15min":1.24},"last":0.68300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":521037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.32700000000000001,"15min":0.27800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88700000000000001,"15min":4.6710000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.83699999999999997},"min":{"1min":0.41499999999999998,"5min":0.35199999999999998,"15min":0.30499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":8.798},"last":0.57399999999999995}]},{"osd":1,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.73499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74099999999999999,"5min":0.82899999999999996,"15min":4.71},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.89900000000000002},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78900000000000003,"5min":1.0109999999999999,"15min":8.6880000000000006},"last":0.65000000000000002}]},{"osd":2,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.73199999999999998},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":1.0009999999999999,"15min":4.8940000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.626,"15min":0.76000000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.443},"max":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":4.8559999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60799999999999998,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69099999999999995,"5min":0.91800000000000004,"15min":8.8770000000000007},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.621,"15min":0.93899999999999995},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85899999999999999,"5min":1.0489999999999999,"15min":10.427},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48999999999999999,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.874},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54600000000000004,"15min":0.56499999999999995},"min":{"1min":0.35099999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.86799999999999999,"15min":0.95899999999999996},"last":0.67800000000000005}]},{"osd":6,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93999999999999995,"5min":1.141,"15min":1.141},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60899999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":1.01,"15min":1.01},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65400000000000003,"15min":0.66900000000000004},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86299999999999999,"5min":1.097,"15min":1.135},"last":0.68799999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:28:57.252 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T23:28:57.253 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:28:57.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.389+0000 7fbee284e700 1 Processor -- start 2024-05-08T23:28:57.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.389+0000 7fbee284e700 1 -- start start 2024-05-08T23:28:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.390+0000 7fbee284e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0b1260 0x7fbedc15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.390+0000 7fbee284e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbedc0d1070 0x7fbedc0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.390+0000 7fbee284e700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc0b1b80 con 0x7fbedc0cbfc0 2024-05-08T23:28:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.390+0000 7fbee284e700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc15c740 con 0x7fbedc0d1070 2024-05-08T23:28:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.390+0000 7fbee284e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0cca10 con 0x7fbedc0b1260 2024-05-08T23:28:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.390+0000 7fbedb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbedc0d1070 0x7fbedc0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedb7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbedc0d1070 0x7fbedc0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34994/0 (socket says 172.21.15.5:34994) 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedb7fe700 1 -- 172.21.15.5:0/787540856 learned_addr learned my addr 172.21.15.5:0/787540856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedbfff700 1 --2- 172.21.15.5:0/787540856 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0b1260 0x7fbedc15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedb7fe700 1 -- 172.21.15.5:0/787540856 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0b1260 msgr2=0x7fbedc15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedb7fe700 1 --2- 172.21.15.5:0/787540856 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0b1260 0x7fbedc15ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedb7fe700 1 -- 172.21.15.5:0/787540856 >> v1:172.21.15.49:6789/0 conn(0x7fbedc0cbfc0 legacy=0x7fbedc0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedb7fe700 1 -- 172.21.15.5:0/787540856 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc15c080 con 0x7fbedc0d1070 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedb7fe700 1 --2- 172.21.15.5:0/787540856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0d1070 0x7fbedc0b1640 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fbed00098f0 tx=0x7fbed0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910462b029c6f72b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedaffd700 1 -- 172.21.15.5:0/787540856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000b8a0 con 0x7fbedc0d1070 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedaffd700 1 -- 172.21.15.5:0/787540856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed000ba00 con 0x7fbedc0d1070 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.391+0000 7fbedaffd700 1 -- 172.21.15.5:0/787540856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000bd90 con 0x7fbedc0d1070 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.392+0000 7fbee284e700 1 -- 172.21.15.5:0/787540856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0d1070 msgr2=0x7fbedc0b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.392+0000 7fbee284e700 1 --2- 172.21.15.5:0/787540856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0d1070 0x7fbedc0b1640 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fbed00098f0 tx=0x7fbed0009c00 comp rx=0 tx=0).stop 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.392+0000 7fbee284e700 1 -- 172.21.15.5:0/787540856 shutdown_connections 2024-05-08T23:28:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.392+0000 7fbee284e700 1 --2- 172.21.15.5:0/787540856 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0b1260 0x7fbedc15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.392+0000 7fbee284e700 1 --2- 172.21.15.5:0/787540856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0d1070 0x7fbedc0b1640 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.392+0000 7fbee284e700 1 -- 172.21.15.5:0/787540856 >> 172.21.15.5:0/787540856 conn(0x7fbedc099a50 msgr2=0x7fbedc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:57.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.392+0000 7fbee284e700 1 -- 172.21.15.5:0/787540856 shutdown_connections 2024-05-08T23:28:57.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.392+0000 7fbee284e700 1 -- 172.21.15.5:0/787540856 wait complete. 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.393+0000 7fbee284e700 1 Processor -- start 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbee284e700 1 -- start start 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbee284e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0b1260 0x7fbedc153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbee284e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbedc0cbfc0 0x7fbedc1f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbee284e700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0d1070 0x7fbedc0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbee284e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0c49c0 con 0x7fbedc0b1260 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbee284e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0c5820 con 0x7fbedc0cbfc0 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbee284e700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc153ab0 con 0x7fbedc0d1070 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbedbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0b1260 0x7fbedc153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbedbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0b1260 0x7fbedc153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35006/0 (socket says 172.21.15.5:35006) 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.394+0000 7fbedbfff700 1 -- 172.21.15.5:0/1011388158 learned_addr learned my addr 172.21.15.5:0/1011388158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbedb7fe700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbedc0cbfc0 0x7fbedc1f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbee0deb700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0d1070 0x7fbedc0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbedbfff700 1 -- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0d1070 msgr2=0x7fbedc0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbedbfff700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0d1070 0x7fbedc0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbedbfff700 1 -- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbedc0cbfc0 msgr2=0x7fbedc1f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbedbfff700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbedc0cbfc0 0x7fbedc1f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbedbfff700 1 -- 172.21.15.5:0/1011388158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed0009550 con 0x7fbedc0b1260 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbedbfff700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0b1260 0x7fbedc153570 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00e870 tx=0x7fbecc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbed8ff9700 1 -- 172.21.15.5:0/1011388158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00c070 con 0x7fbedc0b1260 2024-05-08T23:28:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbed8ff9700 1 -- 172.21.15.5:0/1011388158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc013610 con 0x7fbedc0b1260 2024-05-08T23:28:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc1f3990 con 0x7fbedc0b1260 2024-05-08T23:28:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbed8ff9700 1 -- 172.21.15.5:0/1011388158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0139c0 con 0x7fbedc0b1260 2024-05-08T23:28:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.395+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc0c4ef0 con 0x7fbedc0b1260 2024-05-08T23:28:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.396+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc068000 con 0x7fbedc0b1260 2024-05-08T23:28:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.397+0000 7fbed8ff9700 1 -- 172.21.15.5:0/1011388158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbecc013b20 con 0x7fbedc0b1260 2024-05-08T23:28:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.397+0000 7fbed8ff9700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbec806ac90 0x7fbec806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.397+0000 7fbed8ff9700 1 -- 172.21.15.5:0/1011388158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbecc088b50 con 0x7fbedc0b1260 2024-05-08T23:28:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.397+0000 7fbedb7fe700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbec806ac90 0x7fbec806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.398+0000 7fbedb7fe700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbec806ac90 0x7fbec806d150 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009c40 tx=0x7fbed0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.402+0000 7fbed8ff9700 1 -- 172.21.15.5:0/1011388158 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbecc053f80 con 0x7fbedc0b1260 2024-05-08T23:28:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.614+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbedc0cc430 con 0x7fbec806ac90 2024-05-08T23:28:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.617+0000 7fbed8ff9700 1 -- 172.21.15.5:0/1011388158 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51276 (secure 0 0 0) 0x7fbed000bb70 con 0x7fbec806ac90 2024-05-08T23:28:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbec806ac90 msgr2=0x7fbec806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbec806ac90 0x7fbec806d150 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009c40 tx=0x7fbed0008030 comp rx=0 tx=0).stop 2024-05-08T23:28:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0b1260 msgr2=0x7fbedc153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0b1260 0x7fbedc153570 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00e870 tx=0x7fbecc00eb80 comp rx=0 tx=0).stop 2024-05-08T23:28:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 shutdown_connections 2024-05-08T23:28:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbec806ac90 0x7fbec806d150 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbedc0d1070 0x7fbedc0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc0b1260 0x7fbedc153570 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 --2- 172.21.15.5:0/1011388158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbedc0cbfc0 0x7fbedc1f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 >> 172.21.15.5:0/1011388158 conn(0x7fbedc099a50 msgr2=0x7fbedc15a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 shutdown_connections 2024-05-08T23:28:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.619+0000 7fbee284e700 1 -- 172.21.15.5:0/1011388158 wait complete. 2024-05-08T23:28:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:28:57.633 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-05-08T23:28:57.410478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259752,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714968,"statfs":{"total":773094113280,"available":772828127232,"internally_reserved":0,"allocated":11051008,"data_stored":5442328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":254791221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-121676,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-245,"num_write_kb":-203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.332005"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T23:28:53.636914+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:28:53.636914+0000","last_peered":"2024-05-08T23:28:53.636914+0000","last_clean":"2024-05-08T23:28:53.636914+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:28:53.636914+0000","last_undegraded":"2024-05-08T23:28:53.636914+0000","last_fullsized":"2024-05-08T23:28:53.636914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574947,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1826816,"data_stored":1123067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.82499999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.434},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.873,"5min":1.038,"15min":1.038},"last":0.504},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.81200000000000006,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53900000000000003}]},{"osd":2,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.54900000000000004},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76700000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68200000000000005}]},{"osd":3,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76800000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.36499999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.85499999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69199999999999995,"5min":0.87,"15min":0.87},"last":0.52000000000000002}]},{"osd":5,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93500000000000005},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.89700000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.41099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.54000000000000004},"min":{"1min":0.44500000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.39700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83299999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47699999999999998}]},{"osd":1,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.88200000000000001,"15min":0.94999999999999996},"last":0.496},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.86899999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.48899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73999999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83099999999999996,"5min":0.85699999999999998,"15min":1.115},"last":0.54800000000000004}]},{"osd":3,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72199999999999998,"5min":0.878,"15min":1.0229999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.98899999999999999,"15min":1.0009999999999999},"last":0.37}]},{"osd":4,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.752,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.41099999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78900000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.50800000000000001}]},{"osd":5,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":1.052},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81799999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55600000000000005}]},{"osd":7,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.877,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.46899999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52000000000000002,"5min":0.61099999999999999,"15min":0.91000000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.93100000000000005},"last":0.28399999999999997}]},{"osd":2,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.41099999999999998},"min":{"1min":0.28699999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.55400000000000005,"5min":0.63700000000000001,"15min":0.88400000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.42699999999999999,"15min":0.44700000000000001},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60999999999999999,"5min":0.61399999999999999,"15min":0.96999999999999997},"last":0.377}]},{"osd":3,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64300000000000002,"5min":0.65300000000000002,"15min":1.01},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.47399999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.627,"5min":0.68200000000000005,"15min":1.028},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":0.91100000000000003,"15min":1.1479999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.67500000000000004},"min":{"1min":0.45500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88500000000000001,"5min":1.0169999999999999,"15min":1.1579999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86499999999999999,"5min":0.89000000000000001,"15min":1.0489999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.67800000000000005},"min":{"1min":0.503,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.115},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95199999999999996,"5min":1.0089999999999999,"15min":1.2050000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65700000000000003,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.105},"last":0.441}]},{"osd":7,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.47999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1,"5min":1,"15min":1.1359999999999999},"last":0.502},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.68799999999999994},"min":{"1min":0.55600000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":1.0609999999999999},"last":0.42799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1826816,"data_stored":1123067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.63400000000000001,"15min":7.7610000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40500000000000003,"15min":0.65700000000000003},"min":{"1min":0.27200000000000002,"5min":0.249,"15min":0.24399999999999999},"max":{"1min":0.58699999999999997,"5min":0.65800000000000003,"15min":7.9889999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41899999999999998,"15min":0.71499999999999997},"min":{"1min":0.32700000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":7.7869999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.436,"15min":0.69099999999999995},"min":{"1min":0.32100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56599999999999995,"5min":0.69899999999999995,"15min":8.0519999999999996},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41299999999999998,"15min":0.44500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":2.9420000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.497,"5min":0.46400000000000002,"15min":0.73799999999999999},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67800000000000005,"5min":0.79800000000000004,"15min":7.6440000000000001},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59899999999999998,"15min":0.874},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.93100000000000005,"15min":7.9139999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59599999999999997,"15min":0.84999999999999998},"min":{"1min":0.47899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.96199999999999997,"15min":7.1879999999999997},"last":0.70799999999999996}]},{"osd":5,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.86099999999999999},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":7.1120000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.94799999999999995},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90600000000000003,"5min":0.94199999999999995,"15min":7.6970000000000001},"last":0.79400000000000004}]},{"osd":6,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63900000000000001,"15min":0.91400000000000003},"min":{"1min":0.48199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95099999999999996,"15min":7.734},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.89600000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83099999999999996,"5min":0.88,"15min":8.0220000000000002},"last":0.81000000000000005}]},{"osd":7,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65100000000000002,"15min":0.90800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":7.9539999999999997},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.627,"15min":0.87},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.91300000000000003,"15min":7.1600000000000001},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574947,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.39900000000000002,"15min":0.39200000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.40300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.68400000000000005},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45000000000000001,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.629,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.42899999999999999},"min":{"1min":0.35699999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53000000000000003,"5min":0.64800000000000002,"15min":0.67300000000000004},"last":0.622}]},{"osd":3,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.46800000000000003},"min":{"1min":0.34599999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.627,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.54400000000000004,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.879,"5min":0.93799999999999994,"15min":1.024},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.79800000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.502,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0129999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.0549999999999999},"last":0.82399999999999995}]},{"osd":6,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65000000000000002,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83099999999999996,"5min":0.89200000000000002,"15min":1.159},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.53500000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.83199999999999996}]},{"osd":7,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.55400000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91200000000000003,"5min":0.92500000000000004,"15min":1.194},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":1.46},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.41499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.2749999999999999},"last":0.89100000000000001}]},{"osd":1,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.27700000000000002},"max":{"1min":0.54800000000000004,"5min":1.327,"15min":1.327},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":0.68400000000000005,"5min":1.109,"15min":1.5649999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44700000000000001,"15min":0.77200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63,"5min":1.1479999999999999,"15min":11.964},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48199999999999998,"15min":0.51100000000000001},"min":{"1min":0.35699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.73899999999999999,"15min":1.593},"last":1.0249999999999999}]},{"osd":4,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63900000000000001,"15min":0.65800000000000003},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94299999999999995,"5min":1.214,"15min":2.4590000000000001},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.536},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.016,"5min":1.2969999999999999,"15min":1.3100000000000001},"last":1.113}]},{"osd":6,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.097},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":1.0389999999999999,"15min":1.5029999999999999},"last":1.073}]},{"osd":7,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.4850000000000001,"15min":2.5230000000000001},"last":1.2010000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.625},"min":{"1min":0.27200000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71799999999999997,"5min":0.75700000000000001,"15min":2.0209999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.68999999999999995},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.78400000000000003,"5min":0.91200000000000003,"15min":4.0960000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56599999999999995,"15min":0.65000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68799999999999994,"5min":1.0349999999999999,"15min":2.145},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.66500000000000004},"min":{"1min":0.32800000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76500000000000001,"5min":0.88500000000000001,"15min":2.2530000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":2.3769999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.749},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":4.0389999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56599999999999995,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.747,"5min":0.98299999999999998,"15min":1.0800000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":1.0289999999999999},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70299999999999996,"5min":0.93000000000000005,"15min":12.127000000000001},"last":0.629}]},{"osd":5,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.56799999999999995},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.79700000000000004,"15min":0.877},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.56299999999999994},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.79000000000000004,"15min":0.84299999999999997},"last":0.437}]},{"osd":6,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53500000000000003,"15min":0.60499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.81499999999999995,"15min":0.93700000000000006},"last":0.495},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.67300000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68300000000000005,"5min":0.877,"15min":1.24},"last":0.73599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.32700000000000001,"15min":0.27800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88700000000000001,"15min":4.6710000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.83699999999999997},"min":{"1min":0.41499999999999998,"5min":0.35199999999999998,"15min":0.30499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":8.798},"last":0.495}]},{"osd":1,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.73499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74099999999999999,"5min":0.82899999999999996,"15min":4.71},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.89900000000000002},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78900000000000003,"5min":1.0109999999999999,"15min":8.6880000000000006},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.73199999999999998},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":1.0009999999999999,"15min":4.8940000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.626,"15min":0.76000000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.443},"max":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":4.8559999999999999},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60799999999999998,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69099999999999995,"5min":0.91800000000000004,"15min":8.8770000000000007},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.621,"15min":0.93899999999999995},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85899999999999999,"5min":1.0489999999999999,"15min":10.427},"last":0.67300000000000004}]},{"osd":4,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48999999999999999,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.874},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54600000000000004,"15min":0.56499999999999995},"min":{"1min":0.35099999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.86799999999999999,"15min":0.95899999999999996},"last":0.75700000000000001}]},{"osd":6,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93999999999999995,"5min":1.141,"15min":1.141},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60899999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":1.01,"15min":1.01},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65400000000000003,"15min":0.66900000000000004},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86299999999999999,"5min":1.097,"15min":1.135},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:28:57.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:28:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.770+0000 7f0fcb282700 1 Processor -- start 2024-05-08T23:28:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.771+0000 7f0fcb282700 1 -- start start 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.771+0000 7f0fcb282700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40ccbd0 0x7f0fc4168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.771+0000 7f0fcb282700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc409a870 con 0x7f0fc4161030 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fcb282700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc40d0840 con 0x7f0fc40b2480 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fcb282700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc406ccc0 con 0x7f0fc40ccbd0 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fc901e700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0fc4161030 0x7f0fc4099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43742/0 (socket says 172.21.15.5:43742) 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fc901e700 1 -- 172.21.15.5:0/1297595738 learned_addr learned my addr 172.21.15.5:0/1297595738 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fc981f700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40ccbd0 0x7f0fc4168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158809911 0 0) 0x7f0fc409a870 con 0x7f0fc4161030 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb00035b0 con 0x7f0fc4161030 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859826723 0 0) 0x7f0fc40d0840 con 0x7f0fc40b2480 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fc409a870 con 0x7f0fc40b2480 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099330154 0 0) 0x7f0fb00035b0 con 0x7f0fc4161030 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40ccbd0 msgr2=0x7f0fc4168580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40ccbd0 0x7f0fc4168580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 >> v1:172.21.15.49:6789/0 conn(0x7f0fc40b2480 legacy=0x7f0fc409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc406c600 con 0x7f0fc4161030 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f0fc0001670 con 0x7f0fc4161030 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 >> v1:172.21.15.5:6789/0 conn(0x7f0fc4161030 legacy=0x7f0fc4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40ccbd0 0x7f0fb0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fb0004120 0x7f0fb0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb0004d90 0x7f0fb0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc406ccc0 con 0x7f0fb0004d90 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb000baf0 con 0x7f0fb0004120 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.772+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb000bcb0 con 0x7f0fc40ccbd0 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc981f700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb0004d90 0x7f0fb0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc901e700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40ccbd0 0x7f0fb0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc881d700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fb0004120 0x7f0fb0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc981f700 1 -- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40ccbd0 msgr2=0x7f0fb0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc981f700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40ccbd0 0x7f0fb0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc981f700 1 -- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fb0004120 msgr2=0x7f0fb0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc981f700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fb0004120 0x7f0fb0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc981f700 1 -- 172.21.15.5:0/1297595738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00035b0 con 0x7f0fb0004d90 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fc981f700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb0004d90 0x7f0fb0005210 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009f90 tx=0x7f0fb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6ed343042e24c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00ebd0 con 0x7f0fb0004d90 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc00f040 con 0x7f0fb0004d90 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.773+0000 7f0fbbfff700 1 -- 172.21.15.5:0/1297595738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc014480 con 0x7f0fb0004d90 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.774+0000 7f0fcb282700 1 -- 172.21.15.5:0/1297595738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb0004d90 msgr2=0x7f0fb0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.774+0000 7f0fcb282700 1 --2- 172.21.15.5:0/1297595738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb0004d90 0x7f0fb0005210 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009f90 tx=0x7f0fb0005880 comp rx=0 tx=0).stop 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.774+0000 7f0fc981f700 1 -- 172.21.15.5:0/1297595738 reap_dead start 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.774+0000 7f0fcb282700 1 -- 172.21.15.5:0/1297595738 shutdown_connections 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.774+0000 7f0fcb282700 1 -- 172.21.15.5:0/1297595738 >> 172.21.15.5:0/1297595738 conn(0x7f0fc40b2f20 msgr2=0x7f0fc40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.774+0000 7f0fcb282700 1 -- 172.21.15.5:0/1297595738 shutdown_connections 2024-05-08T23:28:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.774+0000 7f0fcb282700 1 -- 172.21.15.5:0/1297595738 wait complete. 2024-05-08T23:28:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.776+0000 7f0fcb282700 1 Processor -- start 2024-05-08T23:28:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.776+0000 7f0fcb282700 1 -- start start 2024-05-08T23:28:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fcb282700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fc40b5460 0x7f0fc40b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fcb282700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40c67e0 0x7f0fc40c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fcb282700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fc40ca6d0 0x7f0fc40cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fcb282700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40d0660 con 0x7f0fc40b5460 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fcb282700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40cb0c0 con 0x7f0fc40ca6d0 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fcb282700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40c5e00 con 0x7f0fc40c67e0 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fc40b5460 0x7f0fc40b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fc40b5460 0x7f0fc40b5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35014/0 (socket says 172.21.15.5:35014) 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 -- 172.21.15.5:0/288490164 learned_addr learned my addr 172.21.15.5:0/288490164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc881d700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40c67e0 0x7f0fc40c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc981f700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fc40ca6d0 0x7f0fc40cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 -- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40c67e0 msgr2=0x7f0fc40c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40c67e0 0x7f0fc40c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 -- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fc40ca6d0 msgr2=0x7f0fc40cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fc40ca6d0 0x7f0fc40cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 -- 172.21.15.5:0/288490164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40c6da0 con 0x7f0fc40b5460 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.777+0000 7f0fc901e700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fc40b5460 0x7f0fc40b5840 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0007370 tx=0x7f0fc0007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.778+0000 7f0fb9ffb700 1 -- 172.21.15.5:0/288490164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000f870 con 0x7f0fc40b5460 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.778+0000 7f0fb9ffb700 1 -- 172.21.15.5:0/288490164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc000fa10 con 0x7f0fc40b5460 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.778+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc40c7080 con 0x7f0fc40b5460 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.778+0000 7f0fb9ffb700 1 -- 172.21.15.5:0/288490164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000fdc0 con 0x7f0fc40b5460 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.778+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc40c7640 con 0x7f0fc40b5460 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.779+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc4068000 con 0x7f0fc40b5460 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.779+0000 7f0fb9ffb700 1 -- 172.21.15.5:0/288490164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0fc001d460 con 0x7f0fc40b5460 2024-05-08T23:28:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.780+0000 7f0fb9ffb700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0fa806ac90 0x7f0fa806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.780+0000 7f0fb9ffb700 1 -- 172.21.15.5:0/288490164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f0fc0089160 con 0x7f0fc40b5460 2024-05-08T23:28:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.780+0000 7f0fc881d700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0fa806ac90 0x7f0fa806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.781+0000 7f0fc881d700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0fa806ac90 0x7f0fa806d150 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4141390 tx=0x7f0fb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:57.785+0000 7f0fb9ffb700 1 -- 172.21.15.5:0/288490164 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0fc0054590 con 0x7f0fc40b5460 2024-05-08T23:28:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.000+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fc406bb90 con 0x7f0fc40b5460 2024-05-08T23:28:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.001+0000 7f0fb9ffb700 1 -- 172.21.15.5:0/288490164 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+12677 (secure 0 0 0) 0x7f0fc0057bb0 con 0x7f0fc40b5460 2024-05-08T23:28:58.001 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:28:58.001 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":99,"fsid":"4fd8ca56-5ac1-431d-b985-b50a13a8130a","created":"2024-05-08T23:20:45.273592+0000","modified":"2024-05-08T23:28:54.486487+0000","last_up_change":"2024-05-08T23:20:56.597803+0000","last_in_change":"2024-05-08T23:20:48.674248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:20:57.351375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d7f7048-8067-4dbf-aee2-245243a044ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":47134},{"type":"v1","addr":"172.21.15.5:6825","nonce":47134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":47134},{"type":"v1","addr":"172.21.15.5:6827","nonce":47134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":47134},{"type":"v1","addr":"172.21.15.5:6831","nonce":47134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":47134},{"type":"v1","addr":"172.21.15.5:6829","nonce":47134}]},"public_addr":"172.21.15.5:6825/47134","cluster_addr":"172.21.15.5:6827/47134","heartbeat_back_addr":"172.21.15.5:6831/47134","heartbeat_front_addr":"172.21.15.5:6829/47134","state":["exists","up"]},{"osd":1,"uuid":"1128f99e-4bac-4e93-a928-21f11858a313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":47133},{"type":"v1","addr":"172.21.15.5:6816","nonce":47133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":47133},{"type":"v1","addr":"172.21.15.5:6819","nonce":47133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":47133},{"type":"v1","addr":"172.21.15.5:6823","nonce":47133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":47133},{"type":"v1","addr":"172.21.15.5:6821","nonce":47133}]},"public_addr":"172.21.15.5:6816/47133","cluster_addr":"172.21.15.5:6819/47133","heartbeat_back_addr":"172.21.15.5:6823/47133","heartbeat_front_addr":"172.21.15.5:6821/47133","state":["exists","up"]},{"osd":2,"uuid":"99a97325-4923-4c97-aa94-52a2eaa66907","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":47131},{"type":"v1","addr":"172.21.15.5:6801","nonce":47131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":47131},{"type":"v1","addr":"172.21.15.5:6803","nonce":47131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":47131},{"type":"v1","addr":"172.21.15.5:6807","nonce":47131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":47131},{"type":"v1","addr":"172.21.15.5:6805","nonce":47131}]},"public_addr":"172.21.15.5:6801/47131","cluster_addr":"172.21.15.5:6803/47131","heartbeat_back_addr":"172.21.15.5:6807/47131","heartbeat_front_addr":"172.21.15.5:6805/47131","state":["exists","up"]},{"osd":3,"uuid":"e2783735-4661-4f39-b55c-00c3fcdc8124","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":47142},{"type":"v1","addr":"172.21.15.5:6809","nonce":47142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":47142},{"type":"v1","addr":"172.21.15.5:6811","nonce":47142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":47142},{"type":"v1","addr":"172.21.15.5:6817","nonce":47142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":47142},{"type":"v1","addr":"172.21.15.5:6813","nonce":47142}]},"public_addr":"172.21.15.5:6809/47142","cluster_addr":"172.21.15.5:6811/47142","heartbeat_back_addr":"172.21.15.5:6817/47142","heartbeat_front_addr":"172.21.15.5:6813/47142","state":["exists","up"]},{"osd":4,"uuid":"90b7fb65-a3d7-4e06-ab90-c04c884689c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":47400},{"type":"v1","addr":"172.21.15.49:6817","nonce":47400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":47400},{"type":"v1","addr":"172.21.15.49:6819","nonce":47400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":47400},{"type":"v1","addr":"172.21.15.49:6825","nonce":47400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":47400},{"type":"v1","addr":"172.21.15.49:6821","nonce":47400}]},"public_addr":"172.21.15.49:6817/47400","cluster_addr":"172.21.15.49:6819/47400","heartbeat_back_addr":"172.21.15.49:6825/47400","heartbeat_front_addr":"172.21.15.49:6821/47400","state":["exists","up"]},{"osd":5,"uuid":"f2df4605-e5ec-41b9-8ec6-0bfc843354c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":47399},{"type":"v1","addr":"172.21.15.49:6801","nonce":47399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":47399},{"type":"v1","addr":"172.21.15.49:6803","nonce":47399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":47399},{"type":"v1","addr":"172.21.15.49:6807","nonce":47399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":47399},{"type":"v1","addr":"172.21.15.49:6805","nonce":47399}]},"public_addr":"172.21.15.49:6801/47399","cluster_addr":"172.21.15.49:6803/47399","heartbeat_back_addr":"172.21.15.49:6807/47399","heartbeat_front_addr":"172.21.15.49:6805/47399","state":["exists","up"]},{"osd":6,"uuid":"ddcb50c8-ad38-42a2-a994-16033633bee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":47398},{"type":"v1","addr":"172.21.15.49:6809","nonce":47398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":47398},{"type":"v1","addr":"172.21.15.49:6811","nonce":47398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":47398},{"type":"v1","addr":"172.21.15.49:6815","nonce":47398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":47398},{"type":"v1","addr":"172.21.15.49:6813","nonce":47398}]},"public_addr":"172.21.15.49:6809/47398","cluster_addr":"172.21.15.49:6811/47398","heartbeat_back_addr":"172.21.15.49:6815/47398","heartbeat_front_addr":"172.21.15.49:6813/47398","state":["exists","up"]},{"osd":7,"uuid":"e8e302a4-2b11-42b9-bc89-265425c636f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":47408},{"type":"v1","addr":"172.21.15.49:6824","nonce":47408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":47408},{"type":"v1","addr":"172.21.15.49:6827","nonce":47408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":47408},{"type":"v1","addr":"172.21.15.49:6831","nonce":47408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":47408},{"type":"v1","addr":"172.21.15.49:6829","nonce":47408}]},"public_addr":"172.21.15.49:6824/47408","cluster_addr":"172.21.15.49:6827/47408","heartbeat_back_addr":"172.21.15.49:6831/47408","heartbeat_front_addr":"172.21.15.49:6829/47408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.863762+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.871339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.891389+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.894641+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.833419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.853895+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:20:53.828586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/1828661194":"2024-05-09T23:28:51.076209+0000","172.21.15.5:6833/1828661194":"2024-05-09T23:28:51.076209+0000","172.21.15.5:6834/1510899872":"2024-05-09T23:28:51.076214+0000","172.21.15.5:6832/1510899872":"2024-05-09T23:28:51.076214+0000","172.21.15.49:6837/873094942":"2024-05-09T23:28:51.076191+0000","172.21.15.49:6835/873094942":"2024-05-09T23:28:51.076191+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.003+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0fa806ac90 msgr2=0x7f0fa806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.003+0000 7f0fcb282700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0fa806ac90 0x7f0fa806d150 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4141390 tx=0x7f0fb4009040 comp rx=0 tx=0).stop 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.003+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fc40b5460 msgr2=0x7f0fc40b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.003+0000 7f0fcb282700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fc40b5460 0x7f0fc40b5840 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0007370 tx=0x7f0fc0007730 comp rx=0 tx=0).stop 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.003+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 shutdown_connections 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.003+0000 7f0fcb282700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f0fa806ac90 0x7f0fa806d150 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.003+0000 7f0fcb282700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fc40ca6d0 0x7f0fc40cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.003+0000 7f0fcb282700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fc40b5460 0x7f0fc40b5840 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.004+0000 7f0fcb282700 1 --2- 172.21.15.5:0/288490164 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0fc40c67e0 0x7f0fc40c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.004+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 >> 172.21.15.5:0/288490164 conn(0x7f0fc40b2f20 msgr2=0x7f0fc40cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.004+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 shutdown_connections 2024-05-08T23:28:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:58.004+0000 7f0fcb282700 1 -- 172.21.15.5:0/288490164 wait complete. 2024-05-08T23:28:59.018 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T23:28:59.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.170+0000 7f9489af5700 1 Processor -- start 2024-05-08T23:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.170+0000 7f9489af5700 1 -- start start 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9489af5700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f948415fbb0 0x7f94840b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9489af5700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94840af6e0 con 0x7f94840d0a60 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9489af5700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94840aeb50 con 0x7f9484145040 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9489af5700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94840b6460 con 0x7f948415fbb0 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f947affd700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f94840d0a60 0x7f94840afb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43758/0 (socket says 172.21.15.5:43758) 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f947affd700 1 -- 172.21.15.5:0/1226089274 learned_addr learned my addr 172.21.15.5:0/1226089274 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9483fff700 1 --2- 172.21.15.5:0/1226089274 >> v2:172.21.15.49:3300/0 conn(0x7f948415fbb0 0x7f94840b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378339694 0 0) 0x7f94840af6e0 con 0x7f94840d0a60 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94700035b0 con 0x7f94840d0a60 2024-05-08T23:28:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055958595 0 0) 0x7f94840aeb50 con 0x7f9484145040 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94840af6e0 con 0x7f9484145040 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 131776704 0 0) 0x7f94700035b0 con 0x7f94840d0a60 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 >> v2:172.21.15.49:3300/0 conn(0x7f948415fbb0 msgr2=0x7f94840b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 --2- 172.21.15.5:0/1226089274 >> v2:172.21.15.49:3300/0 conn(0x7f948415fbb0 0x7f94840b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 >> v1:172.21.15.49:6789/0 conn(0x7f9484145040 legacy=0x7f94840d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840aedd0 con 0x7f94840d0a60 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f946c001670 con 0x7f94840d0a60 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 >> v1:172.21.15.5:6789/0 conn(0x7f94840d0a60 legacy=0x7f94840afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9470003be0 0x7f9470003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.171+0000 7f9482ffd700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9470004850 0x7f9470004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9482ffd700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9470005260 0x7f947000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840b6460 con 0x7f9470004850 2024-05-08T23:28:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947000c190 con 0x7f9470005260 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947000c350 con 0x7f9470003be0 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f947affd700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9470004850 0x7f9470004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f94837fe700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9470003be0 0x7f9470003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9483fff700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9470005260 0x7f947000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f947affd700 1 -- 172.21.15.5:0/1226089274 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9470003be0 msgr2=0x7f9470003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f947affd700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9470003be0 0x7f9470003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f947affd700 1 -- 172.21.15.5:0/1226089274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9470005260 msgr2=0x7f947000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f947affd700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9470005260 0x7f947000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f947affd700 1 -- 172.21.15.5:0/1226089274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700035b0 con 0x7f9470004850 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9483fff700 1 -- 172.21.15.5:0/1226089274 reap_dead start 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f947affd700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9470004850 0x7f9470004cd0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c110 tx=0x7f9470005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b35f90a5c3eb86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c00cc10 con 0x7f9470004850 2024-05-08T23:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c00cd70 con 0x7f9470004850 2024-05-08T23:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.172+0000 7f9482ffd700 1 -- 172.21.15.5:0/1226089274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c005b80 con 0x7f9470004850 2024-05-08T23:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.173+0000 7f9489af5700 1 -- 172.21.15.5:0/1226089274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9470004850 msgr2=0x7f9470004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.173+0000 7f9489af5700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9470004850 0x7f9470004cd0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c110 tx=0x7f9470005880 comp rx=0 tx=0).stop 2024-05-08T23:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.173+0000 7f9489af5700 1 -- 172.21.15.5:0/1226089274 shutdown_connections 2024-05-08T23:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.173+0000 7f9489af5700 1 --2- 172.21.15.5:0/1226089274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9470004850 0x7f9470004cd0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.173+0000 7f9489af5700 1 -- 172.21.15.5:0/1226089274 >> 172.21.15.5:0/1226089274 conn(0x7f9484099a50 msgr2=0x7f94840b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.173+0000 7f9489af5700 1 -- 172.21.15.5:0/1226089274 shutdown_connections 2024-05-08T23:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.173+0000 7f9489af5700 1 -- 172.21.15.5:0/1226089274 wait complete. 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.174+0000 7f9489af5700 1 Processor -- start 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f9489af5700 1 -- start start 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f9489af5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9470004850 0x7f94841f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f9489af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94841524f0 0x7f9484152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f9489af5700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9484152eb0 0x7f94841f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f9489af5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840cb6c0 con 0x7f94841524f0 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f9489af5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948412af80 con 0x7f9470004850 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f9489af5700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948412b260 con 0x7f9484152eb0 2024-05-08T23:28:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f947affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94841524f0 0x7f9484152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.175+0000 7f947affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94841524f0 0x7f9484152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35020/0 (socket says 172.21.15.5:35020) 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f947affd700 1 -- 172.21.15.5:0/3455270773 learned_addr learned my addr 172.21.15.5:0/3455270773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f94837fe700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9470004850 0x7f94841f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f9483fff700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9484152eb0 0x7f94841f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f947affd700 1 -- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9484152eb0 msgr2=0x7f94841f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f947affd700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9484152eb0 0x7f94841f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f947affd700 1 -- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9470004850 msgr2=0x7f94841f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f947affd700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9470004850 0x7f94841f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f947affd700 1 -- 172.21.15.5:0/3455270773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948412b540 con 0x7f94841524f0 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f947affd700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94841524f0 0x7f9484152970 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f946c000c00 tx=0x7f946c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c00cee0 con 0x7f94841524f0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c0072d0 con 0x7f94841524f0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948412b820 con 0x7f94841524f0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c007660 con 0x7f94841524f0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.176+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94841f64d0 con 0x7f94841524f0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.177+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94840af370 con 0x7f94841524f0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.178+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f946c01e430 con 0x7f94841524f0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.178+0000 7f9480ff9700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f946406ac90 0x7f946406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.178+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f946c08a930 con 0x7f94841524f0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.178+0000 7f94837fe700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f946406ac90 0x7f946406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.179+0000 7f9480ff9700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f9464070c30 0x7f9464073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.179+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9464073700 con 0x7f9464070c30 2024-05-08T23:28:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.179+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f946c08aca0 con 0x7f94841524f0 2024-05-08T23:28:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.179+0000 7f9483fff700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f9464070c30 0x7f9464073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.179+0000 7f94837fe700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f946406ac90 0x7f946406d150 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9484069190 tx=0x7f9474009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.179+0000 7f9483fff700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f9464070c30 0x7f9464073050 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.180+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 <== osd.0 v2:172.21.15.5:6824/47134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9464073700 con 0x7f9464070c30 2024-05-08T23:28:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.195+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 --> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f94841f6690 con 0x7f9464070c30 2024-05-08T23:28:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9480ff9700 1 -- 172.21.15.5:0/3455270773 <== osd.0 v2:172.21.15.5:6824/47134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f94841f6690 con 0x7f9464070c30 2024-05-08T23:28:59.196 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:28:59.196 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f9464070c30 msgr2=0x7f9464073050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.5:6824/47134,v1:172.21.15.5:6825/47134] conn(0x7f9464070c30 0x7f9464073050 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f946406ac90 msgr2=0x7f946406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f946406ac90 0x7f946406d150 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9484069190 tx=0x7f9474009040 comp rx=0 tx=0).stop 2024-05-08T23:28:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94841524f0 msgr2=0x7f9484152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 --2- 172.21.15.5:0/3455270773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94841524f0 0x7f9484152970 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f946c000c00 tx=0x7f946c001930 comp rx=0 tx=0).stop 2024-05-08T23:28:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9483fff700 1 -- 172.21.15.5:0/3455270773 reap_dead start 2024-05-08T23:28:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 shutdown_connections 2024-05-08T23:28:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 >> 172.21.15.5:0/3455270773 conn(0x7f9484099a50 msgr2=0x7f94840b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 shutdown_connections 2024-05-08T23:28:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.196+0000 7f9489af5700 1 -- 172.21.15.5:0/3455270773 wait complete. 2024-05-08T23:28:59.210 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T23:28:59.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.347+0000 7f996f052700 1 Processor -- start 2024-05-08T23:28:59.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.347+0000 7f996f052700 1 -- start start 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f996f052700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f996815b900 0x7f99680cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f996f052700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680b1410 con 0x7f996809a5e0 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f996f052700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680b0e30 con 0x7f9968099a50 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f996f052700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99680cffd0 con 0x7f996815b900 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f996d5ef700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f996809a5e0 0x7f99680b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43760/0 (socket says 172.21.15.5:43760) 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f996d5ef700 1 -- 172.21.15.5:0/668889349 learned_addr learned my addr 172.21.15.5:0/668889349 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f996cdee700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f996815b900 0x7f99680cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880472507 0 0) 0x7f99680b1410 con 0x7f996809a5e0 2024-05-08T23:28:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.348+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99580035b0 con 0x7f996809a5e0 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284940115 0 0) 0x7f99680b0e30 con 0x7f9968099a50 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99680b1410 con 0x7f9968099a50 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3021236172 0 0) 0x7f99580035b0 con 0x7f996809a5e0 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f996815b900 msgr2=0x7f99680cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f996815b900 0x7f99680cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 >> v1:172.21.15.49:6789/0 conn(0x7f9968099a50 legacy=0x7f9968099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680b10a0 con 0x7f996809a5e0 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7f99600019f0 con 0x7f996809a5e0 2024-05-08T23:28:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 >> v1:172.21.15.5:6789/0 conn(0x7f996809a5e0 legacy=0x7f99680b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9958003bc0 0x7f9958003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f9958004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f996815b900 0x7f9958005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99680cffd0 con 0x7f9958004850 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995800bac0 con 0x7f9958003bc0 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995800bc80 con 0x7f996815b900 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f996cdee700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9958003bc0 0x7f9958003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f9967fff700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f9958004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f996d5ef700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f996815b900 0x7f9958005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f9967fff700 1 -- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f996815b900 msgr2=0x7f9958005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f9967fff700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f996815b900 0x7f9958005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f9967fff700 1 -- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9958003bc0 msgr2=0x7f9958003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f9967fff700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9958003bc0 0x7f9958003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f9967fff700 1 -- 172.21.15.5:0/668889349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99580035b0 con 0x7f9958004850 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.349+0000 7f996cdee700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9958003bc0 0x7f9958003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f9967fff700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f9958004cd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f9954001940 tx=0x7f9958005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b70fac53efdf894 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954001430 con 0x7f9958004850 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9954001d70 con 0x7f9958004850 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f99677fe700 1 -- 172.21.15.5:0/668889349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954005510 con 0x7f9958004850 2024-05-08T23:28:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f996f052700 1 -- 172.21.15.5:0/668889349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 msgr2=0x7f9958004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f996f052700 1 --2- 172.21.15.5:0/668889349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f9958004cd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f9954001940 tx=0x7f9958005880 comp rx=0 tx=0).stop 2024-05-08T23:28:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f996d5ef700 1 -- 172.21.15.5:0/668889349 reap_dead start 2024-05-08T23:28:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f996f052700 1 -- 172.21.15.5:0/668889349 shutdown_connections 2024-05-08T23:28:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f996f052700 1 -- 172.21.15.5:0/668889349 >> 172.21.15.5:0/668889349 conn(0x7f99680b2f20 msgr2=0x7f99680b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f996f052700 1 -- 172.21.15.5:0/668889349 shutdown_connections 2024-05-08T23:28:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.350+0000 7f996f052700 1 -- 172.21.15.5:0/668889349 wait complete. 2024-05-08T23:28:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.352+0000 7f996f052700 1 Processor -- start 2024-05-08T23:28:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.352+0000 7f996f052700 1 -- start start 2024-05-08T23:28:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996f052700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f99680cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996f052700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f996809ead0 0x7f99681f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996f052700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99680c8500 0x7f99680c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996f052700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99681f4400 con 0x7f9958004850 2024-05-08T23:28:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996f052700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99680c8ec0 con 0x7f996809ead0 2024-05-08T23:28:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996f052700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996809ef50 con 0x7f99680c8500 2024-05-08T23:28:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f99680cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f99680cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35032/0 (socket says 172.21.15.5:35032) 2024-05-08T23:28:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 -- 172.21.15.5:0/3238228829 learned_addr learned my addr 172.21.15.5:0/3238228829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996d5ef700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99680c8500 0x7f99680c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f9967fff700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f996809ead0 0x7f99681f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 -- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99680c8500 msgr2=0x7f99680c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99680c8500 0x7f99680c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 -- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f996809ead0 msgr2=0x7f99681f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f996809ead0 0x7f99681f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 -- 172.21.15.5:0/3238228829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996815d2d0 con 0x7f9958004850 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.353+0000 7f996cdee700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f99680cb250 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f995c011cd0 tx=0x7f995c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.354+0000 7f99657fa700 1 -- 172.21.15.5:0/3238228829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00fb30 con 0x7f9958004850 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.354+0000 7f99657fa700 1 -- 172.21.15.5:0/3238228829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c00fc90 con 0x7f9958004850 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.354+0000 7f99657fa700 1 -- 172.21.15.5:0/3238228829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c004020 con 0x7f9958004850 2024-05-08T23:28:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.354+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99681f6e20 con 0x7f9958004850 2024-05-08T23:28:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.354+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99681f73a0 con 0x7f9958004850 2024-05-08T23:28:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.355+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9968068000 con 0x7f9958004850 2024-05-08T23:28:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.355+0000 7f99657fa700 1 -- 172.21.15.5:0/3238228829 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f995c004180 con 0x7f9958004850 2024-05-08T23:28:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.356+0000 7f99657fa700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f994c06acc0 0x7f994c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.356+0000 7f99657fa700 1 -- 172.21.15.5:0/3238228829 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f995c089610 con 0x7f9958004850 2024-05-08T23:28:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.356+0000 7f9967fff700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f994c06acc0 0x7f994c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.356+0000 7f9967fff700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f994c06acc0 0x7f994c06d180 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f99680b4b50 tx=0x7f995400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.361+0000 7f99657fa700 1 -- 172.21.15.5:0/3238228829 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f995c054a40 con 0x7f9958004850 2024-05-08T23:28:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.577+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9968067a50 con 0x7f994c06acc0 2024-05-08T23:28:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.577+0000 7f99657fa700 1 -- 172.21.15.5:0/3238228829 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9968067a50 con 0x7f994c06acc0 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.579+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f994c06acc0 msgr2=0x7f994c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.579+0000 7f996f052700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f994c06acc0 0x7f994c06d180 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f99680b4b50 tx=0x7f995400a000 comp rx=0 tx=0).stop 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.579+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 msgr2=0x7f99680cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.579+0000 7f996f052700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f99680cb250 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f995c011cd0 tx=0x7f995c00f520 comp rx=0 tx=0).stop 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.580+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 shutdown_connections 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.580+0000 7f996f052700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f994c06acc0 0x7f994c06d180 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.580+0000 7f996f052700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f99680c8500 0x7f99680c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.580+0000 7f996f052700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9958004850 0x7f99680cb250 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.580+0000 7f996f052700 1 --2- 172.21.15.5:0/3238228829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f996809ead0 0x7f99681f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.580+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 >> 172.21.15.5:0/3238228829 conn(0x7f99680b2f20 msgr2=0x7f9968070580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.580+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 shutdown_connections 2024-05-08T23:28:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.580+0000 7f996f052700 1 -- 172.21.15.5:0/3238228829 wait complete. 2024-05-08T23:28:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T23:28:59.594 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T23:28:59.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:28:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.735+0000 7fe2829dc700 1 Processor -- start 2024-05-08T23:28:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.735+0000 7fe2829dc700 1 -- start start 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe2829dc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c15d880 0x7fe27c0af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe2829dc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe27c06f8c0 0x7fe27c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe2829dc700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c0b3870 con 0x7fe27c0ad7a0 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe2829dc700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c0b32f0 con 0x7fe27c06f8c0 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe2829dc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c06fd70 con 0x7fe27c15d880 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe280f79700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe27c0ad7a0 0x7fe27c06e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43776/0 (socket says 172.21.15.5:43776) 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe280f79700 1 -- 172.21.15.5:0/663606426 learned_addr learned my addr 172.21.15.5:0/663606426 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe27bfff700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c15d880 0x7fe27c0af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe27b7fe700 1 --2- 172.21.15.5:0/663606426 >> v2:172.21.15.49:3300/0 conn(0x7fe27c06f8c0 0x7fe27c0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136471160 0 0) 0x7fe27c0b3870 con 0x7fe27c0ad7a0 2024-05-08T23:28:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2640035b0 con 0x7fe27c0ad7a0 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.736+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522652028 0 0) 0x7fe2640035b0 con 0x7fe27c0ad7a0 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c15d880 msgr2=0x7fe27c0af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c15d880 0x7fe27c0af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 >> v2:172.21.15.49:3300/0 conn(0x7fe27c06f8c0 msgr2=0x7fe27c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 --2- 172.21.15.5:0/663606426 >> v2:172.21.15.49:3300/0 conn(0x7fe27c06f8c0 0x7fe27c0ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c0b4ee0 con 0x7fe27c0ad7a0 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27b7fe700 1 --2- 172.21.15.5:0/663606426 >> v2:172.21.15.49:3300/0 conn(0x7fe27c06f8c0 0x7fe27c0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4235341250 0 0) 0x7fe2740019f0 con 0x7fe27c0ad7a0 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 >> v1:172.21.15.5:6789/0 conn(0x7fe27c0ad7a0 legacy=0x7fe27c06e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264003a00 0x7fe264003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c15d880 0x7fe264004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264004d90 0x7fe264005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0b32f0 con 0x7fe264004d90 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c06fd70 con 0x7fe264003a00 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26400ba10 con 0x7fe27c15d880 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe280f79700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264004d90 0x7fe264005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27bfff700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264003a00 0x7fe264003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27b7fe700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c15d880 0x7fe264004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe280f79700 1 -- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c15d880 msgr2=0x7fe264004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe280f79700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c15d880 0x7fe264004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe280f79700 1 -- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264003a00 msgr2=0x7fe264003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe280f79700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe264003a00 0x7fe264003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe280f79700 1 -- 172.21.15.5:0/663606426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640035b0 con 0x7fe264004d90 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe280f79700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264004d90 0x7fe264005210 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe27400a830 tx=0x7fe264005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e0dc5ce3c3a1fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.737+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27400a320 con 0x7fe264004d90 2024-05-08T23:28:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe274012600 con 0x7fe264004d90 2024-05-08T23:28:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe27affd700 1 -- 172.21.15.5:0/663606426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2740129b0 con 0x7fe264004d90 2024-05-08T23:28:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe2829dc700 1 -- 172.21.15.5:0/663606426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264004d90 msgr2=0x7fe264005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe2829dc700 1 --2- 172.21.15.5:0/663606426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264004d90 0x7fe264005210 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe27400a830 tx=0x7fe264005880 comp rx=0 tx=0).stop 2024-05-08T23:28:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe280f79700 1 -- 172.21.15.5:0/663606426 reap_dead start 2024-05-08T23:28:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe2829dc700 1 -- 172.21.15.5:0/663606426 shutdown_connections 2024-05-08T23:28:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe2829dc700 1 -- 172.21.15.5:0/663606426 >> 172.21.15.5:0/663606426 conn(0x7fe27c0b5810 msgr2=0x7fe27c0b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe2829dc700 1 -- 172.21.15.5:0/663606426 shutdown_connections 2024-05-08T23:28:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.738+0000 7fe2829dc700 1 -- 172.21.15.5:0/663606426 wait complete. 2024-05-08T23:28:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.739+0000 7fe2829dc700 1 Processor -- start 2024-05-08T23:28:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.740+0000 7fe2829dc700 1 -- start start 2024-05-08T23:28:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe2829dc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe27c0d0620 0x7fe27c14a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe2829dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe27c0d0a00 0x7fe27c14a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe2829dc700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c148020 0x7fe27c1484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe2829dc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c1499f0 con 0x7fe27c0d0620 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe2829dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c148de0 con 0x7fe27c0d0a00 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe2829dc700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c1490c0 con 0x7fe27c148020 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe27c0d0620 0x7fe27c14a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe27c0d0620 0x7fe27c14a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35058/0 (socket says 172.21.15.5:35058) 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 -- 172.21.15.5:0/1133821071 learned_addr learned my addr 172.21.15.5:0/1133821071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe280f79700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c148020 0x7fe27c1484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27b7fe700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe27c0d0a00 0x7fe27c14a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 -- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c148020 msgr2=0x7fe27c1484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe27c148020 0x7fe27c1484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 -- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe27c0d0a00 msgr2=0x7fe27c14a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe27c0d0a00 0x7fe27c14a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 -- 172.21.15.5:0/1133821071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c1493a0 con 0x7fe27c0d0620 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27b7fe700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe27c0d0a00 0x7fe27c14a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe27bfff700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe27c0d0620 0x7fe27c14a020 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0119a0 tx=0x7fe26c011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00f940 con 0x7fe27c0d0620 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c00faa0 con 0x7fe27c0d0620 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c149680 con 0x7fe27c0d0620 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00fe30 con 0x7fe27c0d0620 2024-05-08T23:28:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.741+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c14d830 con 0x7fe27c0d0620 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.742+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe27c0b5c20 con 0x7fe27c0d0620 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.743+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe26c01d400 con 0x7fe27c0d0620 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.744+0000 7fe278ff9700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe26806acc0 0x7fe26806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.744+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe26c089800 con 0x7fe27c0d0620 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.744+0000 7fe278ff9700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fe268070c60 0x7fe268073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.744+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 --> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe268073730 con 0x7fe268070c60 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.744+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe26c08e2c0 con 0x7fe27c0d0620 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.744+0000 7fe27b7fe700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe26806acc0 0x7fe26806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.744+0000 7fe280f79700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fe268070c60 0x7fe268073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.744+0000 7fe280f79700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fe268070c60 0x7fe268073080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.745+0000 7fe27b7fe700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe26806acc0 0x7fe26806d180 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe27c15ba70 tx=0x7fe270008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.745+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 <== osd.1 v2:172.21.15.5:6814/47133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe268073730 con 0x7fe268070c60 2024-05-08T23:28:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 --> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe27c0b3500 con 0x7fe268070c60 2024-05-08T23:28:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe278ff9700 1 -- 172.21.15.5:0/1133821071 <== osd.1 v2:172.21.15.5:6814/47133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe274012400 con 0x7fe268070c60 2024-05-08T23:28:59.761 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:28:59.762 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:28:59.762 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:28:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fe268070c60 msgr2=0x7fe268073080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.5:6814/47133,v1:172.21.15.5:6816/47133] conn(0x7fe268070c60 0x7fe268073080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe26806acc0 msgr2=0x7fe26806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe26806acc0 0x7fe26806d180 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe27c15ba70 tx=0x7fe270008040 comp rx=0 tx=0).stop 2024-05-08T23:28:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe27c0d0620 msgr2=0x7fe27c14a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 --2- 172.21.15.5:0/1133821071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe27c0d0620 0x7fe27c14a020 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0119a0 tx=0x7fe26c011cb0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe280f79700 1 -- 172.21.15.5:0/1133821071 reap_dead start 2024-05-08T23:28:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 shutdown_connections 2024-05-08T23:28:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.761+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 >> 172.21.15.5:0/1133821071 conn(0x7fe27c0b5810 msgr2=0x7fe27c06e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.762+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 shutdown_connections 2024-05-08T23:28:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.762+0000 7fe2829dc700 1 -- 172.21.15.5:0/1133821071 wait complete. 2024-05-08T23:28:59.775 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T23:28:59.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.912+0000 7fc951d2f700 1 Processor -- start 2024-05-08T23:28:59.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.912+0000 7fc951d2f700 1 -- start start 2024-05-08T23:28:59.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.913+0000 7fc951d2f700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc94c0cd4e0 0x7fc94c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.913+0000 7fc951d2f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cecf0 0x7fc94c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.913+0000 7fc951d2f700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94c15c280 con 0x7fc94c06f130 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.913+0000 7fc951d2f700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94c06e780 con 0x7fc94c0cd4e0 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.913+0000 7fc951d2f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c06ea60 con 0x7fc94c0cecf0 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.913+0000 7fc94b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc94c0cd4e0 0x7fc94c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.913+0000 7fc94b7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc94c0cd4e0 0x7fc94c06f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35068/0 (socket says 172.21.15.5:35068) 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.913+0000 7fc94b7fe700 1 -- 172.21.15.5:0/3097740088 learned_addr learned my addr 172.21.15.5:0/3097740088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94bfff700 1 --2- 172.21.15.5:0/3097740088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cecf0 0x7fc94c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94b7fe700 1 -- 172.21.15.5:0/3097740088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cecf0 msgr2=0x7fc94c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94b7fe700 1 --2- 172.21.15.5:0/3097740088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cecf0 0x7fc94c15c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94b7fe700 1 -- 172.21.15.5:0/3097740088 >> v1:172.21.15.49:6789/0 conn(0x7fc94c06f130 legacy=0x7fc94c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94b7fe700 1 -- 172.21.15.5:0/3097740088 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c06ed40 con 0x7fc94c0cd4e0 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94bfff700 1 --2- 172.21.15.5:0/3097740088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cecf0 0x7fc94c15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94b7fe700 1 --2- 172.21.15.5:0/3097740088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cd4e0 0x7fc94c06f910 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009e50 tx=0x7fc93c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e530993e2201b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94a7fc700 1 -- 172.21.15.5:0/3097740088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00e070 con 0x7fc94c0cd4e0 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94a7fc700 1 -- 172.21.15.5:0/3097740088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c00b9c0 con 0x7fc94c0cd4e0 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.914+0000 7fc94a7fc700 1 -- 172.21.15.5:0/3097740088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00bd70 con 0x7fc94c0cd4e0 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.915+0000 7fc951d2f700 1 -- 172.21.15.5:0/3097740088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cd4e0 msgr2=0x7fc94c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.915+0000 7fc951d2f700 1 --2- 172.21.15.5:0/3097740088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cd4e0 0x7fc94c06f910 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009e50 tx=0x7fc93c00b330 comp rx=0 tx=0).stop 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.915+0000 7fc951d2f700 1 -- 172.21.15.5:0/3097740088 shutdown_connections 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.915+0000 7fc951d2f700 1 --2- 172.21.15.5:0/3097740088 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cecf0 0x7fc94c15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.915+0000 7fc951d2f700 1 --2- 172.21.15.5:0/3097740088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cd4e0 0x7fc94c06f910 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.915+0000 7fc951d2f700 1 -- 172.21.15.5:0/3097740088 >> 172.21.15.5:0/3097740088 conn(0x7fc94c0d0320 msgr2=0x7fc94c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.915+0000 7fc951d2f700 1 -- 172.21.15.5:0/3097740088 shutdown_connections 2024-05-08T23:28:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.915+0000 7fc951d2f700 1 -- 172.21.15.5:0/3097740088 wait complete. 2024-05-08T23:28:59.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.916+0000 7fc951d2f700 1 Processor -- start 2024-05-08T23:28:59.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.916+0000 7fc951d2f700 1 -- start start 2024-05-08T23:28:59.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc951d2f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc94c06f130 0x7fc94c0c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc951d2f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cd4e0 0x7fc94c1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc951d2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cecf0 0x7fc94c15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc951d2f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c0b3f70 con 0x7fc94c0cecf0 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc951d2f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c1f3c10 con 0x7fc94c06f130 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc951d2f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c1f3ee0 con 0x7fc94c0cd4e0 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc94bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cecf0 0x7fc94c15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc94bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cecf0 0x7fc94c15d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35084/0 (socket says 172.21.15.5:35084) 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.917+0000 7fc94bfff700 1 -- 172.21.15.5:0/6216146 learned_addr learned my addr 172.21.15.5:0/6216146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94b7fe700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc94c06f130 0x7fc94c0c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94affd700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cd4e0 0x7fc94c1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94bfff700 1 -- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cd4e0 msgr2=0x7fc94c1f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94bfff700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cd4e0 0x7fc94c1f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94bfff700 1 -- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc94c06f130 msgr2=0x7fc94c0c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94bfff700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc94c06f130 0x7fc94c0c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94bfff700 1 -- 172.21.15.5:0/6216146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c009b30 con 0x7fc94c0cecf0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94affd700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cd4e0 0x7fc94c1f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc94bfff700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cecf0 0x7fc94c15d1d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c7f0 tx=0x7fc94400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc943fff700 1 -- 172.21.15.5:0/6216146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944011800 con 0x7fc94c0cecf0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc943fff700 1 -- 172.21.15.5:0/6216146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc944011980 con 0x7fc94c0cecf0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc943fff700 1 -- 172.21.15.5:0/6216146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944011d30 con 0x7fc94c0cecf0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c1f41c0 con 0x7fc94c0cecf0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.918+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c0b41c0 con 0x7fc94c0cecf0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.919+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94c068000 con 0x7fc94c0cecf0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.920+0000 7fc943fff700 1 -- 172.21.15.5:0/6216146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc944022350 con 0x7fc94c0cecf0 2024-05-08T23:28:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.920+0000 7fc943fff700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc92c06ac90 0x7fc92c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:59.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.920+0000 7fc943fff700 1 -- 172.21.15.5:0/6216146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc944089070 con 0x7fc94c0cecf0 2024-05-08T23:28:59.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.920+0000 7fc94b7fe700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc92c06ac90 0x7fc92c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:59.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.921+0000 7fc94b7fe700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc92c06ac90 0x7fc92c06d150 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00b370 tx=0x7fc93c009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:59.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:28:59.925+0000 7fc943fff700 1 -- 172.21.15.5:0/6216146 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc9440544b0 con 0x7fc94c0cecf0 2024-05-08T23:29:00.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.142+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc94c0d0e40 con 0x7fc92c06ac90 2024-05-08T23:29:00.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.143+0000 7fc943fff700 1 -- 172.21.15.5:0/6216146 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc93c021070 con 0x7fc92c06ac90 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc92c06ac90 msgr2=0x7fc92c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc92c06ac90 0x7fc92c06d150 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00b370 tx=0x7fc93c009000 comp rx=0 tx=0).stop 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cecf0 msgr2=0x7fc94c15d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cecf0 0x7fc94c15d1d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c7f0 tx=0x7fc94400cb00 comp rx=0 tx=0).stop 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 shutdown_connections 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fc92c06ac90 0x7fc92c06d150 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc94c0cecf0 0x7fc94c15d1d0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc94c06f130 0x7fc94c0c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 --2- 172.21.15.5:0/6216146 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc94c0cd4e0 0x7fc94c1f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 >> 172.21.15.5:0/6216146 conn(0x7fc94c0d0320 msgr2=0x7fc94c0cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:00.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 shutdown_connections 2024-05-08T23:29:00.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.145+0000 7fc951d2f700 1 -- 172.21.15.5:0/6216146 wait complete. 2024-05-08T23:29:00.147 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T23:29:00.159 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T23:29:00.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:29:00.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.300+0000 7feb575ff700 1 Processor -- start 2024-05-08T23:29:00.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.301+0000 7feb575ff700 1 -- start start 2024-05-08T23:29:00.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb575ff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b0d80 0x7feb500b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb575ff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb500b4d50 0x7feb5006a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb575ff700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb50145040 con 0x7feb5006aea0 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb575ff700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5006b330 con 0x7feb500b4d50 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb575ff700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb500b38c0 con 0x7feb500b0d80 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb54b9a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb500b4d50 0x7feb5006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb55b9c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7feb5006aea0 0x7feb500d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43788/0 (socket says 172.21.15.5:43788) 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb55b9c700 1 -- 172.21.15.5:0/2795121966 learned_addr learned my addr 172.21.15.5:0/2795121966 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb5539b700 1 --2- 172.21.15.5:0/2795121966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b0d80 0x7feb500b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb54b9a700 1 -- 172.21.15.5:0/2795121966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b0d80 msgr2=0x7feb500b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb54b9a700 1 --2- 172.21.15.5:0/2795121966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b0d80 0x7feb500b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb54b9a700 1 -- 172.21.15.5:0/2795121966 >> v1:172.21.15.5:6789/0 conn(0x7feb5006aea0 legacy=0x7feb500d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb54b9a700 1 -- 172.21.15.5:0/2795121966 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500b16e0 con 0x7feb500b4d50 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb47fff700 1 -- 172.21.15.5:0/2795121966 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092235164 0 0) 0x7feb50145040 con 0x7feb5006aea0 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.302+0000 7feb54b9a700 1 --2- 172.21.15.5:0/2795121966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b4d50 0x7feb5006a960 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7feb40009840 tx=0x7feb40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7824d0c191b2913 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb47fff700 1 -- 172.21.15.5:0/2795121966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40011830 con 0x7feb500b4d50 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb47fff700 1 -- 172.21.15.5:0/2795121966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb40011990 con 0x7feb500b4d50 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb47fff700 1 -- 172.21.15.5:0/2795121966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40011d20 con 0x7feb500b4d50 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb575ff700 1 -- 172.21.15.5:0/2795121966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b4d50 msgr2=0x7feb5006a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb575ff700 1 --2- 172.21.15.5:0/2795121966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b4d50 0x7feb5006a960 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7feb40009840 tx=0x7feb40009b50 comp rx=0 tx=0).stop 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb575ff700 1 -- 172.21.15.5:0/2795121966 shutdown_connections 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb575ff700 1 --2- 172.21.15.5:0/2795121966 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b0d80 0x7feb500b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb575ff700 1 --2- 172.21.15.5:0/2795121966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b4d50 0x7feb5006a960 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb575ff700 1 -- 172.21.15.5:0/2795121966 >> 172.21.15.5:0/2795121966 conn(0x7feb50099a50 msgr2=0x7feb500b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb575ff700 1 -- 172.21.15.5:0/2795121966 shutdown_connections 2024-05-08T23:29:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.303+0000 7feb575ff700 1 -- 172.21.15.5:0/2795121966 wait complete. 2024-05-08T23:29:00.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb575ff700 1 Processor -- start 2024-05-08T23:29:00.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb575ff700 1 -- start start 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb575ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5006aea0 0x7feb500cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb575ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b0d80 0x7feb501f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb575ff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b4d50 0x7feb501f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb575ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb501f67d0 con 0x7feb500b0d80 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb575ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500cf5b0 con 0x7feb5006aea0 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb575ff700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5009b060 con 0x7feb500b4d50 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb54b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b0d80 0x7feb501f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb54b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b0d80 0x7feb501f2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35100/0 (socket says 172.21.15.5:35100) 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.305+0000 7feb54b9a700 1 -- 172.21.15.5:0/917037248 learned_addr learned my addr 172.21.15.5:0/917037248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb55b9c700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b4d50 0x7feb501f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb5539b700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5006aea0 0x7feb500cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb54b9a700 1 -- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b4d50 msgr2=0x7feb501f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb54b9a700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7feb500b4d50 0x7feb501f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb54b9a700 1 -- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5006aea0 msgr2=0x7feb500cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb54b9a700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb5006aea0 0x7feb500cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb54b9a700 1 -- 172.21.15.5:0/917037248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400094a0 con 0x7feb500b0d80 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb54b9a700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b0d80 0x7feb501f2e40 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7feb40014040 tx=0x7feb40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400036a0 con 0x7feb500b0d80 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4001e070 con 0x7feb500b0d80 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5009b340 con 0x7feb500b0d80 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.306+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40003a40 con 0x7feb500b0d80 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.307+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5009b900 con 0x7feb500b0d80 2024-05-08T23:29:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.307+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb500b55a0 con 0x7feb500b0d80 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.308+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feb40003ba0 con 0x7feb500b0d80 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.308+0000 7feb45ffb700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb3406ac90 0x7feb3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.308+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7feb4008a040 con 0x7feb500b0d80 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.309+0000 7feb45ffb700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7feb34070c30 0x7feb34073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.309+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 --> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb34073700 con 0x7feb34070c30 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.309+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb4008c700 con 0x7feb500b0d80 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.309+0000 7feb55b9c700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7feb34070c30 0x7feb34073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.309+0000 7feb5539b700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb3406ac90 0x7feb3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.309+0000 7feb55b9c700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7feb34070c30 0x7feb34073050 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.309+0000 7feb5539b700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb3406ac90 0x7feb3406d150 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009be0 tx=0x7feb4c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.310+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 <== osd.2 v2:172.21.15.5:6800/47131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7feb34073700 con 0x7feb34070c30 2024-05-08T23:29:00.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 --> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feb500b1ca0 con 0x7feb34070c30 2024-05-08T23:29:00.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb45ffb700 1 -- 172.21.15.5:0/917037248 <== osd.2 v2:172.21.15.5:6800/47131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7feb48007260 con 0x7feb34070c30 2024-05-08T23:29:00.326 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:29:00.326 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:29:00.326 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:29:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7feb34070c30 msgr2=0x7feb34073050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.5:6800/47131,v1:172.21.15.5:6801/47131] conn(0x7feb34070c30 0x7feb34073050 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb3406ac90 msgr2=0x7feb3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7feb3406ac90 0x7feb3406d150 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009be0 tx=0x7feb4c009210 comp rx=0 tx=0).stop 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b0d80 msgr2=0x7feb501f2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 --2- 172.21.15.5:0/917037248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb500b0d80 0x7feb501f2e40 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7feb40014040 tx=0x7feb40004750 comp rx=0 tx=0).stop 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb55b9c700 1 -- 172.21.15.5:0/917037248 reap_dead start 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 shutdown_connections 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 >> 172.21.15.5:0/917037248 conn(0x7feb50099a50 msgr2=0x7feb50144960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.326+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 shutdown_connections 2024-05-08T23:29:00.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.327+0000 7feb575ff700 1 -- 172.21.15.5:0/917037248 wait complete. 2024-05-08T23:29:00.340 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T23:29:00.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.477+0000 7fe0293e7700 1 Processor -- start 2024-05-08T23:29:00.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.478+0000 7fe0293e7700 1 -- start start 2024-05-08T23:29:00.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.478+0000 7fe0293e7700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0240c5d30 0x7fe0240b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.478+0000 7fe0293e7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0240b1450 0x7fe0240b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.478+0000 7fe0293e7700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0240ce5c0 con 0x7fe02415a8e0 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.478+0000 7fe0293e7700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0240ce430 con 0x7fe0240c5d30 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.478+0000 7fe0293e7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0240b08f0 con 0x7fe0240b1450 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0227fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0240c5d30 0x7fe0240b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0227fc700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0240c5d30 0x7fe0240b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35112/0 (socket says 172.21.15.5:35112) 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0227fc700 1 -- 172.21.15.5:0/2845234722 learned_addr learned my addr 172.21.15.5:0/2845234722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe022ffd700 1 --1- 172.21.15.5:0/2845234722 >> v1:172.21.15.49:6789/0 conn(0x7fe02415a8e0 0x7fe0240c5620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:44836/0 (socket says 172.21.15.5:44836) 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0237fe700 1 --2- 172.21.15.5:0/2845234722 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0240b1450 0x7fe0240b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0227fc700 1 -- 172.21.15.5:0/2845234722 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0240b1450 msgr2=0x7fe0240b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0227fc700 1 --2- 172.21.15.5:0/2845234722 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0240b1450 0x7fe0240b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0227fc700 1 -- 172.21.15.5:0/2845234722 >> v1:172.21.15.49:6789/0 conn(0x7fe02415a8e0 legacy=0x7fe0240c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0227fc700 1 -- 172.21.15.5:0/2845234722 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0240b0b60 con 0x7fe0240c5d30 2024-05-08T23:29:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe021ffb700 1 -- 172.21.15.5:0/2845234722 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634405886 0 0) 0x7fe0240ce5c0 con 0x7fe02415a8e0 2024-05-08T23:29:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe0227fc700 1 --2- 172.21.15.5:0/2845234722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240b1b90 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe0180098f0 tx=0x7fe018009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e299639dfe77822 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe021ffb700 1 -- 172.21.15.5:0/2845234722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018011830 con 0x7fe0240c5d30 2024-05-08T23:29:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe021ffb700 1 -- 172.21.15.5:0/2845234722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe018011990 con 0x7fe0240c5d30 2024-05-08T23:29:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.479+0000 7fe021ffb700 1 -- 172.21.15.5:0/2845234722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018011d20 con 0x7fe0240c5d30 2024-05-08T23:29:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.480+0000 7fe0293e7700 1 -- 172.21.15.5:0/2845234722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 msgr2=0x7fe0240b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.480+0000 7fe0293e7700 1 --2- 172.21.15.5:0/2845234722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240b1b90 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe0180098f0 tx=0x7fe018009c00 comp rx=0 tx=0).stop 2024-05-08T23:29:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.480+0000 7fe0293e7700 1 -- 172.21.15.5:0/2845234722 shutdown_connections 2024-05-08T23:29:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.480+0000 7fe0293e7700 1 --2- 172.21.15.5:0/2845234722 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe0240b1450 0x7fe0240b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.480+0000 7fe0293e7700 1 --2- 172.21.15.5:0/2845234722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240b1b90 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.480+0000 7fe0293e7700 1 -- 172.21.15.5:0/2845234722 >> 172.21.15.5:0/2845234722 conn(0x7fe0240d03a0 msgr2=0x7fe0240d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.480+0000 7fe0293e7700 1 -- 172.21.15.5:0/2845234722 shutdown_connections 2024-05-08T23:29:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.480+0000 7fe0293e7700 1 -- 172.21.15.5:0/2845234722 wait complete. 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.481+0000 7fe0293e7700 1 Processor -- start 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0293e7700 1 -- start start 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0293e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0240b1450 0x7fe0240aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0293e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0293e7700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe02415a8e0 0x7fe02414b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0293e7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe024150100 con 0x7fe0240c5d30 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0293e7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02414b6c0 con 0x7fe0240b1450 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0293e7700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02414f7d0 con 0x7fe02415a8e0 2024-05-08T23:29:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0227fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0227fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35114/0 (socket says 172.21.15.5:35114) 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.482+0000 7fe0227fc700 1 -- 172.21.15.5:0/1210293479 learned_addr learned my addr 172.21.15.5:0/1210293479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0237fe700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe02415a8e0 0x7fe02414b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe022ffd700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0240b1450 0x7fe0240aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0227fc700 1 -- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe02415a8e0 msgr2=0x7fe02414b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0227fc700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe02415a8e0 0x7fe02414b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0227fc700 1 -- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0240b1450 msgr2=0x7fe0240aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0227fc700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0240b1450 0x7fe0240aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0227fc700 1 -- 172.21.15.5:0/1210293479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018009550 con 0x7fe0240c5d30 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0227fc700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240c9e50 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe018009c40 tx=0x7fe018004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe00b7fe700 1 -- 172.21.15.5:0/1210293479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018004300 con 0x7fe0240c5d30 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe00b7fe700 1 -- 172.21.15.5:0/1210293479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01801f070 con 0x7fe0240c5d30 2024-05-08T23:29:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02414fab0 con 0x7fe0240c5d30 2024-05-08T23:29:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe00b7fe700 1 -- 172.21.15.5:0/1210293479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180036a0 con 0x7fe0240c5d30 2024-05-08T23:29:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.483+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02414f0e0 con 0x7fe0240c5d30 2024-05-08T23:29:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.484+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe024068000 con 0x7fe0240c5d30 2024-05-08T23:29:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.485+0000 7fe00b7fe700 1 -- 172.21.15.5:0/1210293479 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe018021030 con 0x7fe0240c5d30 2024-05-08T23:29:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.485+0000 7fe00b7fe700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe01006ac90 0x7fe01006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.485+0000 7fe00b7fe700 1 -- 172.21.15.5:0/1210293479 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe01808a500 con 0x7fe0240c5d30 2024-05-08T23:29:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.486+0000 7fe022ffd700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe01006ac90 0x7fe01006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.486+0000 7fe022ffd700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe01006ac90 0x7fe01006d150 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe014007fd0 tx=0x7fe014007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.490+0000 7fe00b7fe700 1 -- 172.21.15.5:0/1210293479 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe018055940 con 0x7fe0240c5d30 2024-05-08T23:29:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.710+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0240cfa60 con 0x7fe01006ac90 2024-05-08T23:29:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.711+0000 7fe00b7fe700 1 -- 172.21.15.5:0/1210293479 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe014001370 con 0x7fe01006ac90 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe01006ac90 msgr2=0x7fe01006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe01006ac90 0x7fe01006d150 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe014007fd0 tx=0x7fe014007de0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 msgr2=0x7fe0240c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240c9e50 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe018009c40 tx=0x7fe018004750 comp rx=0 tx=0).stop 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 shutdown_connections 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fe01006ac90 0x7fe01006d150 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe02415a8e0 0x7fe02414b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0240b1450 0x7fe0240aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 --2- 172.21.15.5:0/1210293479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0240c5d30 0x7fe0240c9e50 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 >> 172.21.15.5:0/1210293479 conn(0x7fe0240d03a0 msgr2=0x7fe0240caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.713+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 shutdown_connections 2024-05-08T23:29:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.714+0000 7fe0293e7700 1 -- 172.21.15.5:0/1210293479 wait complete. 2024-05-08T23:29:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T23:29:00.727 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T23:29:00.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:29:00.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.868+0000 7f86e16ad700 1 Processor -- start 2024-05-08T23:29:00.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.868+0000 7f86e16ad700 1 -- start start 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86e16ad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc15d700 0x7f86dc0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86e16ad700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f86dc161c00 0x7f86dc0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86e16ad700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc0b49b0 con 0x7f86dc0b38b0 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86e16ad700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86dc0b3da0 con 0x7f86dc161c00 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86e16ad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0b4080 con 0x7f86dc15d700 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86da7fc700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f86dc0b38b0 0x7f86dc1614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:44844/0 (socket says 172.21.15.5:44844) 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86db7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f86dc161c00 0x7f86dc0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86da7fc700 1 -- 172.21.15.5:0/368091091 learned_addr learned my addr 172.21.15.5:0/368091091 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86daffd700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc15d700 0x7f86dc0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86d9ffb700 1 -- 172.21.15.5:0/368091091 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005284202 0 0) 0x7f86dc0b49b0 con 0x7f86dc0b38b0 2024-05-08T23:29:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86d9ffb700 1 -- 172.21.15.5:0/368091091 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c40035b0 con 0x7f86dc0b38b0 2024-05-08T23:29:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86db7fe700 1 -- 172.21.15.5:0/368091091 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc15d700 msgr2=0x7f86dc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86db7fe700 1 --2- 172.21.15.5:0/368091091 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc15d700 0x7f86dc0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86db7fe700 1 -- 172.21.15.5:0/368091091 >> v1:172.21.15.49:6789/0 conn(0x7f86dc0b38b0 legacy=0x7f86dc1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.869+0000 7f86db7fe700 1 -- 172.21.15.5:0/368091091 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0b4360 con 0x7f86dc161c00 2024-05-08T23:29:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86db7fe700 1 --2- 172.21.15.5:0/368091091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86dc161c00 0x7f86dc0b0140 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009d90 tx=0x7f86d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b973f4f2f1dff2a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86d9ffb700 1 -- 172.21.15.5:0/368091091 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400e070 con 0x7f86dc161c00 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86d9ffb700 1 -- 172.21.15.5:0/368091091 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400bb20 con 0x7f86dc161c00 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86d9ffb700 1 -- 172.21.15.5:0/368091091 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400be90 con 0x7f86dc161c00 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86e16ad700 1 -- 172.21.15.5:0/368091091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86dc161c00 msgr2=0x7f86dc0b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86e16ad700 1 --2- 172.21.15.5:0/368091091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86dc161c00 0x7f86dc0b0140 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009d90 tx=0x7f86d400b330 comp rx=0 tx=0).stop 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86e16ad700 1 -- 172.21.15.5:0/368091091 shutdown_connections 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86e16ad700 1 --2- 172.21.15.5:0/368091091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86dc161c00 0x7f86dc0b0140 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86e16ad700 1 --2- 172.21.15.5:0/368091091 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc15d700 0x7f86dc0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.870+0000 7f86e16ad700 1 -- 172.21.15.5:0/368091091 >> 172.21.15.5:0/368091091 conn(0x7f86dc08afe0 msgr2=0x7f86dc0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.871+0000 7f86e16ad700 1 -- 172.21.15.5:0/368091091 shutdown_connections 2024-05-08T23:29:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.871+0000 7f86e16ad700 1 -- 172.21.15.5:0/368091091 wait complete. 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.872+0000 7f86e16ad700 1 Processor -- start 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.872+0000 7f86e16ad700 1 -- start start 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86e16ad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc0b38b0 0x7f86dc1f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86e16ad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86dc15d700 0x7f86dc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86e16ad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86dc161c00 0x7f86dc0b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86e16ad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1f41d0 con 0x7f86dc161c00 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86e16ad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1f3550 con 0x7f86dc15d700 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86e16ad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1f3830 con 0x7f86dc0b38b0 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86dc161c00 0x7f86dc0b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86dc161c00 0x7f86dc0b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35128/0 (socket says 172.21.15.5:35128) 2024-05-08T23:29:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86db7fe700 1 -- 172.21.15.5:0/3413011030 learned_addr learned my addr 172.21.15.5:0/3413011030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86daffd700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc0b38b0 0x7f86dc1f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86da7fc700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86dc15d700 0x7f86dc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86db7fe700 1 -- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc0b38b0 msgr2=0x7f86dc1f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86db7fe700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f86dc0b38b0 0x7f86dc1f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86db7fe700 1 -- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86dc15d700 msgr2=0x7f86dc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86db7fe700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86dc15d700 0x7f86dc0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.873+0000 7f86db7fe700 1 -- 172.21.15.5:0/3413011030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4009a70 con 0x7f86dc161c00 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.874+0000 7f86db7fe700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86dc161c00 0x7f86dc0b1b80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f86d4014040 tx=0x7f86d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.874+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4004960 con 0x7f86dc161c00 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.874+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d4020070 con 0x7f86dc161c00 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.874+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc1f3b10 con 0x7f86dc161c00 2024-05-08T23:29:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.874+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40276c0 con 0x7f86dc161c00 2024-05-08T23:29:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.874+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc0cd590 con 0x7f86dc161c00 2024-05-08T23:29:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.875+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86dc0cf850 con 0x7f86dc161c00 2024-05-08T23:29:00.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.875+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f86d4012430 con 0x7f86dc161c00 2024-05-08T23:29:00.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.876+0000 7f86c37fe700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c806ac90 0x7f86c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.876+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f86d4022070 con 0x7f86dc161c00 2024-05-08T23:29:00.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.876+0000 7f86daffd700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c806ac90 0x7f86c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.876+0000 7f86c37fe700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f86c8070ac0 0x7f86c8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:00.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.876+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 --> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86c8073590 con 0x7f86c8070ac0 2024-05-08T23:29:00.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.876+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d408b7b0 con 0x7f86dc161c00 2024-05-08T23:29:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.876+0000 7f86da7fc700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f86c8070ac0 0x7f86c8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.877+0000 7f86daffd700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c806ac90 0x7f86c806d150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0d0db0 tx=0x7f86cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.877+0000 7f86da7fc700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f86c8070ac0 0x7f86c8072ee0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.877+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 <== osd.3 v2:172.21.15.5:6808/47142 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f86c8073590 con 0x7f86c8070ac0 2024-05-08T23:29:00.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.893+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 --> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f86dc074bc0 con 0x7f86c8070ac0 2024-05-08T23:29:00.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.893+0000 7f86c37fe700 1 -- 172.21.15.5:0/3413011030 <== osd.3 v2:172.21.15.5:6808/47142 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f86dc074bc0 con 0x7f86c8070ac0 2024-05-08T23:29:00.893 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:29:00.893 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:29:00.893 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:29:00.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.893+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f86c8070ac0 msgr2=0x7f86c8072ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.893+0000 7f86e16ad700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.5:6808/47142,v1:172.21.15.5:6809/47142] conn(0x7f86c8070ac0 0x7f86c8072ee0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c806ac90 msgr2=0x7f86c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86e16ad700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f86c806ac90 0x7f86c806d150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0d0db0 tx=0x7f86cc008040 comp rx=0 tx=0).stop 2024-05-08T23:29:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86dc161c00 msgr2=0x7f86dc0b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86e16ad700 1 --2- 172.21.15.5:0/3413011030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86dc161c00 0x7f86dc0b1b80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f86d4014040 tx=0x7f86d4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:29:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86db7fe700 1 -- 172.21.15.5:0/3413011030 reap_dead start 2024-05-08T23:29:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 shutdown_connections 2024-05-08T23:29:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 >> 172.21.15.5:0/3413011030 conn(0x7f86dc08afe0 msgr2=0x7f86dc15bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 shutdown_connections 2024-05-08T23:29:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:00.894+0000 7f86e16ad700 1 -- 172.21.15.5:0/3413011030 wait complete. 2024-05-08T23:29:00.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T23:29:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.044+0000 7fbe84f3d700 1 Processor -- start 2024-05-08T23:29:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.044+0000 7fbe84f3d700 1 -- start start 2024-05-08T23:29:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.045+0000 7fbe84f3d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe800cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.045+0000 7fbe84f3d700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe800b1410 con 0x7fbe8009a5e0 2024-05-08T23:29:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.045+0000 7fbe84f3d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe800b0e30 con 0x7fbe80099a50 2024-05-08T23:29:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.045+0000 7fbe84f3d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800cffd0 con 0x7fbe8015b900 2024-05-08T23:29:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.045+0000 7fbe7ed9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbe8009a5e0 0x7fbe800b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43802/0 (socket says 172.21.15.5:43802) 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.045+0000 7fbe7ed9d700 1 -- 172.21.15.5:0/3768347773 learned_addr learned my addr 172.21.15.5:0/3768347773 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.045+0000 7fbe7e59c700 1 --2- 172.21.15.5:0/3768347773 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe800cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.045+0000 7fbe7d59a700 1 -- 172.21.15.5:0/3768347773 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113348787 0 0) 0x7fbe800b1410 con 0x7fbe8009a5e0 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7d59a700 1 -- 172.21.15.5:0/3768347773 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe680035b0 con 0x7fbe8009a5e0 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7d59a700 1 -- 172.21.15.5:0/3768347773 <== mon.1 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159424289 0 0) 0x7fbe800b0e30 con 0x7fbe80099a50 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7d59a700 1 -- 172.21.15.5:0/3768347773 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe800b1410 con 0x7fbe80099a50 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7e59c700 1 -- 172.21.15.5:0/3768347773 >> v1:172.21.15.49:6789/0 conn(0x7fbe80099a50 legacy=0x7fbe80099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7e59c700 1 -- 172.21.15.5:0/3768347773 >> v1:172.21.15.5:6789/0 conn(0x7fbe8009a5e0 legacy=0x7fbe800b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7e59c700 1 -- 172.21.15.5:0/3768347773 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800b10a0 con 0x7fbe8015b900 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7e59c700 1 --2- 172.21.15.5:0/3768347773 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe800cf9e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009870 tx=0x7fbe70009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529a679de3a92900 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7d59a700 1 -- 172.21.15.5:0/3768347773 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70011830 con 0x7fbe8015b900 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7d59a700 1 -- 172.21.15.5:0/3768347773 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70011990 con 0x7fbe8015b900 2024-05-08T23:29:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.046+0000 7fbe7d59a700 1 -- 172.21.15.5:0/3768347773 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70011d20 con 0x7fbe8015b900 2024-05-08T23:29:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.047+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3768347773 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 msgr2=0x7fbe800cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.047+0000 7fbe84f3d700 1 --2- 172.21.15.5:0/3768347773 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe800cf9e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009870 tx=0x7fbe70009b80 comp rx=0 tx=0).stop 2024-05-08T23:29:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.047+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3768347773 shutdown_connections 2024-05-08T23:29:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.047+0000 7fbe84f3d700 1 --2- 172.21.15.5:0/3768347773 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe800cf9e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.047+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3768347773 >> 172.21.15.5:0/3768347773 conn(0x7fbe800b2f20 msgr2=0x7fbe800b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.047+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3768347773 shutdown_connections 2024-05-08T23:29:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.047+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3768347773 wait complete. 2024-05-08T23:29:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.048+0000 7fbe84f3d700 1 Processor -- start 2024-05-08T23:29:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.048+0000 7fbe84f3d700 1 -- start start 2024-05-08T23:29:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe84f3d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe80099a50 0x7fbe8015e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe84f3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe8009a5e0 0x7fbe801f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe84f3d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe8015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe84f3d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8015ed40 con 0x7fbe8009a5e0 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe84f3d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801f1df0 con 0x7fbe80099a50 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe84f3d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800b5590 con 0x7fbe8015b900 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe7dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe8009a5e0 0x7fbe801f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe7dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe8009a5e0 0x7fbe801f18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35130/0 (socket says 172.21.15.5:35130) 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe7dd9b700 1 -- 172.21.15.5:0/3517581674 learned_addr learned my addr 172.21.15.5:0/3517581674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe7ed9d700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe8015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.049+0000 7fbe7e59c700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe80099a50 0x7fbe8015e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe7dd9b700 1 -- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 msgr2=0x7fbe8015a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe7dd9b700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe8015a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe7dd9b700 1 -- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe80099a50 msgr2=0x7fbe8015e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe7dd9b700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe80099a50 0x7fbe8015e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe7dd9b700 1 -- 172.21.15.5:0/3517581674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe700094a0 con 0x7fbe8009a5e0 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe7dd9b700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe8009a5e0 0x7fbe801f18b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400c520 tx=0x7fbe7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe6effd700 1 -- 172.21.15.5:0/3517581674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740052d0 con 0x7fbe8009a5e0 2024-05-08T23:29:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe6effd700 1 -- 172.21.15.5:0/3517581674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe74005430 con 0x7fbe8009a5e0 2024-05-08T23:29:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe800b44f0 con 0x7fbe8009a5e0 2024-05-08T23:29:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe6effd700 1 -- 172.21.15.5:0/3517581674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740038c0 con 0x7fbe8009a5e0 2024-05-08T23:29:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.050+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe800b4ab0 con 0x7fbe8009a5e0 2024-05-08T23:29:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.051+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe80068000 con 0x7fbe8009a5e0 2024-05-08T23:29:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.052+0000 7fbe6effd700 1 -- 172.21.15.5:0/3517581674 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbe74003a20 con 0x7fbe8009a5e0 2024-05-08T23:29:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.052+0000 7fbe6effd700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe6006ac90 0x7fbe6006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.052+0000 7fbe6effd700 1 -- 172.21.15.5:0/3517581674 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbe74088540 con 0x7fbe8009a5e0 2024-05-08T23:29:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.052+0000 7fbe7e59c700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe6006ac90 0x7fbe6006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.053+0000 7fbe7e59c700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe6006ac90 0x7fbe6006d150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbe700090b0 tx=0x7fbe7001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.057+0000 7fbe6effd700 1 -- 172.21.15.5:0/3517581674 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbe74053960 con 0x7fbe8009a5e0 2024-05-08T23:29:01.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.275+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe800b1f80 con 0x7fbe6006ac90 2024-05-08T23:29:01.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.276+0000 7fbe6effd700 1 -- 172.21.15.5:0/3517581674 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbe800b1f80 con 0x7fbe6006ac90 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe6006ac90 msgr2=0x7fbe6006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe6006ac90 0x7fbe6006d150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbe700090b0 tx=0x7fbe7001c040 comp rx=0 tx=0).stop 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe8009a5e0 msgr2=0x7fbe801f18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe8009a5e0 0x7fbe801f18b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400c520 tx=0x7fbe7400c830 comp rx=0 tx=0).stop 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 shutdown_connections 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbe6006ac90 0x7fbe6006d150 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe8015b900 0x7fbe8015a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe80099a50 0x7fbe8015e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 --2- 172.21.15.5:0/3517581674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe8009a5e0 0x7fbe801f18b0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 >> 172.21.15.5:0/3517581674 conn(0x7fbe800b2f20 msgr2=0x7fbe8006ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 shutdown_connections 2024-05-08T23:29:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.278+0000 7fbe84f3d700 1 -- 172.21.15.5:0/3517581674 wait complete. 2024-05-08T23:29:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T23:29:01.292 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T23:29:01.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:29:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.432+0000 7f48bb374700 1 Processor -- start 2024-05-08T23:29:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.433+0000 7f48bb374700 1 -- start start 2024-05-08T23:29:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48bb374700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48b406a960 0x7f48b4099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48bb374700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f48b40aea70 0x7f48b40aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48bb374700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b415c920 con 0x7f48b40b5500 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48bb374700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b40ad6b0 con 0x7f48b406a960 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48bb374700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b40b2700 con 0x7f48b40aea70 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b9110700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48b406a960 0x7f48b4099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b890f700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f48b40b5500 0x7f48b406ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43810/0 (socket says 172.21.15.5:43810) 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b890f700 1 -- 172.21.15.5:0/127726586 learned_addr learned my addr 172.21.15.5:0/127726586 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b9911700 1 --2- 172.21.15.5:0/127726586 >> v2:172.21.15.49:3300/0 conn(0x7f48b40aea70 0x7f48b40aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b9110700 1 -- 172.21.15.5:0/127726586 >> v2:172.21.15.49:3300/0 conn(0x7f48b40aea70 msgr2=0x7f48b40aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b9110700 1 --2- 172.21.15.5:0/127726586 >> v2:172.21.15.49:3300/0 conn(0x7f48b40aea70 0x7f48b40aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b9110700 1 -- 172.21.15.5:0/127726586 >> v1:172.21.15.5:6789/0 conn(0x7f48b40b5500 legacy=0x7f48b406ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b9110700 1 -- 172.21.15.5:0/127726586 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40b2910 con 0x7f48b406a960 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48abfff700 1 -- 172.21.15.5:0/127726586 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293195931 0 0) 0x7f48b415c920 con 0x7f48b40b5500 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.434+0000 7f48b9110700 1 --2- 172.21.15.5:0/127726586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b406a960 0x7f48b4099a50 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009840 tx=0x7f48b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e59de7f630c5e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48abfff700 1 -- 172.21.15.5:0/127726586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0011830 con 0x7f48b406a960 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48abfff700 1 -- 172.21.15.5:0/127726586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0011990 con 0x7f48b406a960 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48abfff700 1 -- 172.21.15.5:0/127726586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0011d20 con 0x7f48b406a960 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48bb374700 1 -- 172.21.15.5:0/127726586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b406a960 msgr2=0x7f48b4099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48bb374700 1 --2- 172.21.15.5:0/127726586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b406a960 0x7f48b4099a50 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009840 tx=0x7f48b0009b50 comp rx=0 tx=0).stop 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48bb374700 1 -- 172.21.15.5:0/127726586 shutdown_connections 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48bb374700 1 --2- 172.21.15.5:0/127726586 >> v2:172.21.15.49:3300/0 conn(0x7f48b40aea70 0x7f48b40aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48bb374700 1 --2- 172.21.15.5:0/127726586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b406a960 0x7f48b4099a50 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48bb374700 1 -- 172.21.15.5:0/127726586 >> 172.21.15.5:0/127726586 conn(0x7f48b40b5a30 msgr2=0x7f48b40b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48bb374700 1 -- 172.21.15.5:0/127726586 shutdown_connections 2024-05-08T23:29:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.435+0000 7f48bb374700 1 -- 172.21.15.5:0/127726586 wait complete. 2024-05-08T23:29:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.437+0000 7f48bb374700 1 Processor -- start 2024-05-08T23:29:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.437+0000 7f48bb374700 1 -- start start 2024-05-08T23:29:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48bb374700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b406a960 0x7f48b409dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48bb374700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48b40aea70 0x7f48b4143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48bb374700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b40b5500 0x7f48b4144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48bb374700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4143cb0 con 0x7f48b40b5500 2024-05-08T23:29:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48bb374700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4144ec0 con 0x7f48b406a960 2024-05-08T23:29:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48bb374700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b409b900 con 0x7f48b40aea70 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48b9911700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b40b5500 0x7f48b4144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48b9911700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b40b5500 0x7f48b4144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35150/0 (socket says 172.21.15.5:35150) 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48b9911700 1 -- 172.21.15.5:0/3412194988 learned_addr learned my addr 172.21.15.5:0/3412194988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.438+0000 7f48b9110700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b406a960 0x7f48b409dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48b890f700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48b40aea70 0x7f48b4143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48b9911700 1 -- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48b40aea70 msgr2=0x7f48b4143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48b9911700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f48b40aea70 0x7f48b4143770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48b9911700 1 -- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b406a960 msgr2=0x7f48b409dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48b9911700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b406a960 0x7f48b409dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48b9911700 1 -- 172.21.15.5:0/3412194988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b00094a0 con 0x7f48b40b5500 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48b9911700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b40b5500 0x7f48b4144980 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00efc0 tx=0x7f48ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac00cd40 con 0x7f48b40b5500 2024-05-08T23:29:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac00f040 con 0x7f48b40b5500 2024-05-08T23:29:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b4145160 con 0x7f48b40b5500 2024-05-08T23:29:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac0146c0 con 0x7f48b40b5500 2024-05-08T23:29:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.439+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b4145720 con 0x7f48b40b5500 2024-05-08T23:29:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.440+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48b415c5b0 con 0x7f48b40b5500 2024-05-08T23:29:01.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.441+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f48ac014820 con 0x7f48b40b5500 2024-05-08T23:29:01.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.441+0000 7f48a9ffb700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f489806ac90 0x7f489806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.441+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f48ac08a740 con 0x7f48b40b5500 2024-05-08T23:29:01.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.441+0000 7f48a9ffb700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f4898070c30 0x7f4898073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.441+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 --> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4898073700 con 0x7f4898070c30 2024-05-08T23:29:01.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.441+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ac08aab0 con 0x7f48b40b5500 2024-05-08T23:29:01.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.441+0000 7f48b9110700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f489806ac90 0x7f489806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.442+0000 7f48b890f700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f4898070c30 0x7f4898073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.442+0000 7f48b9110700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f489806ac90 0x7f489806d150 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f48b00090b0 tx=0x7f48b001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.442+0000 7f48b890f700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f4898070c30 0x7f4898073050 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.443+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 <== osd.4 v2:172.21.15.49:6816/47400 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4898073700 con 0x7f4898070c30 2024-05-08T23:29:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.458+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 --> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f48b4074bc0 con 0x7f4898070c30 2024-05-08T23:29:01.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.459+0000 7f48a9ffb700 1 -- 172.21.15.5:0/3412194988 <== osd.4 v2:172.21.15.49:6816/47400 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f48b4074bc0 con 0x7f4898070c30 2024-05-08T23:29:01.459 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:29:01.459 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:29:01.459 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:29:01.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.459+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f4898070c30 msgr2=0x7f4898073050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.459+0000 7f48bb374700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6816/47400,v1:172.21.15.49:6817/47400] conn(0x7f4898070c30 0x7f4898073050 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.459+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f489806ac90 msgr2=0x7f489806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.459+0000 7f48bb374700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f489806ac90 0x7f489806d150 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f48b00090b0 tx=0x7f48b001c040 comp rx=0 tx=0).stop 2024-05-08T23:29:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.459+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b40b5500 msgr2=0x7f48b4144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.459+0000 7f48bb374700 1 --2- 172.21.15.5:0/3412194988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b40b5500 0x7f48b4144980 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00efc0 tx=0x7f48ac00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.459+0000 7f48b9911700 1 -- 172.21.15.5:0/3412194988 reap_dead start 2024-05-08T23:29:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.460+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 shutdown_connections 2024-05-08T23:29:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.460+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 >> 172.21.15.5:0/3412194988 conn(0x7f48b40b5a30 msgr2=0x7f48b415ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.460+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 shutdown_connections 2024-05-08T23:29:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.460+0000 7f48bb374700 1 -- 172.21.15.5:0/3412194988 wait complete. 2024-05-08T23:29:01.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T23:29:01.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.609+0000 7f03dfc23700 1 Processor -- start 2024-05-08T23:29:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.610+0000 7f03dfc23700 1 -- start start 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dfc23700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d81614f0 0x7f03d8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dfc23700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03d80d0650 0x7f03d81618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dfc23700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d8161e10 con 0x7f03d815d0a0 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dfc23700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d809a030 con 0x7f03d80d0650 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dfc23700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d809a310 con 0x7f03d81614f0 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dd1be700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03d80d0650 0x7f03d81618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03de1c0700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f03d815d0a0 0x7f03d815d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43822/0 (socket says 172.21.15.5:43822) 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03de1c0700 1 -- 172.21.15.5:0/3925780987 learned_addr learned my addr 172.21.15.5:0/3925780987 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dd9bf700 1 --2- 172.21.15.5:0/3925780987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d81614f0 0x7f03d8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dd1be700 1 -- 172.21.15.5:0/3925780987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d81614f0 msgr2=0x7f03d8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dd1be700 1 --2- 172.21.15.5:0/3925780987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d81614f0 0x7f03d8099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dd1be700 1 -- 172.21.15.5:0/3925780987 >> v1:172.21.15.5:6789/0 conn(0x7f03d815d0a0 legacy=0x7f03d815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dc9bd700 1 -- 172.21.15.5:0/3925780987 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713137672 0 0) 0x7f03d8161e10 con 0x7f03d815d0a0 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dd1be700 1 -- 172.21.15.5:0/3925780987 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d809a5f0 con 0x7f03d80d0650 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dd9bf700 1 --2- 172.21.15.5:0/3925780987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d81614f0 0x7f03d8099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dd1be700 1 --2- 172.21.15.5:0/3925780987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80d0650 0x7f03d81618d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009840 tx=0x7f03c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c5f635ae39e64b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.611+0000 7f03dc9bd700 1 -- 172.21.15.5:0/3925780987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8011830 con 0x7f03d80d0650 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dc9bd700 1 -- 172.21.15.5:0/3925780987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c8011990 con 0x7f03d80d0650 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dc9bd700 1 -- 172.21.15.5:0/3925780987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8011d20 con 0x7f03d80d0650 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dfc23700 1 -- 172.21.15.5:0/3925780987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80d0650 msgr2=0x7f03d81618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dfc23700 1 --2- 172.21.15.5:0/3925780987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80d0650 0x7f03d81618d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009840 tx=0x7f03c8009b50 comp rx=0 tx=0).stop 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dfc23700 1 -- 172.21.15.5:0/3925780987 shutdown_connections 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dfc23700 1 --2- 172.21.15.5:0/3925780987 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d81614f0 0x7f03d8099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dfc23700 1 --2- 172.21.15.5:0/3925780987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80d0650 0x7f03d81618d0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dfc23700 1 -- 172.21.15.5:0/3925780987 >> 172.21.15.5:0/3925780987 conn(0x7f03d80b2f20 msgr2=0x7f03d80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dfc23700 1 -- 172.21.15.5:0/3925780987 shutdown_connections 2024-05-08T23:29:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.612+0000 7f03dfc23700 1 -- 172.21.15.5:0/3925780987 wait complete. 2024-05-08T23:29:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.613+0000 7f03dfc23700 1 Processor -- start 2024-05-08T23:29:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.614+0000 7f03dfc23700 1 -- start start 2024-05-08T23:29:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.614+0000 7f03dfc23700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d80d0650 0x7f03d81f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.614+0000 7f03dfc23700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d0a0 0x7f03d80b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.614+0000 7f03dfc23700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d81614f0 0x7f03d80b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.614+0000 7f03dfc23700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d815f990 con 0x7f03d81614f0 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.614+0000 7f03dfc23700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d81f3f40 con 0x7f03d80d0650 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.614+0000 7f03dfc23700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d815f060 con 0x7f03d815d0a0 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.614+0000 7f03de1c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d81614f0 0x7f03d80b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03de1c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d81614f0 0x7f03d80b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35172/0 (socket says 172.21.15.5:35172) 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03de1c0700 1 -- 172.21.15.5:0/1476100982 learned_addr learned my addr 172.21.15.5:0/1476100982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03dd1be700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d0a0 0x7f03d80b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03dd9bf700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d80d0650 0x7f03d81f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03de1c0700 1 -- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d0a0 msgr2=0x7f03d80b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03de1c0700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d0a0 0x7f03d80b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03de1c0700 1 -- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d80d0650 msgr2=0x7f03d81f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03de1c0700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d80d0650 0x7f03d81f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03de1c0700 1 -- 172.21.15.5:0/1476100982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c80094a0 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03dd1be700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d0a0 0x7f03d80b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03de1c0700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d81614f0 0x7f03d80b5170 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f03d0002230 tx=0x7f03d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03ce7fc700 1 -- 172.21.15.5:0/1476100982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d000ac20 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03ce7fc700 1 -- 172.21.15.5:0/1476100982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d000ada0 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d815f340 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03ce7fc700 1 -- 172.21.15.5:0/1476100982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0007310 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.615+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d815fe20 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.616+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d806d3e0 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.617+0000 7f03ce7fc700 1 -- 172.21.15.5:0/1476100982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f03d0007470 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.617+0000 7f03ce7fc700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.617+0000 7f03ce7fc700 1 -- 172.21.15.5:0/1476100982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f03d0088bc0 con 0x7f03d81614f0 2024-05-08T23:29:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.617+0000 7f03dd9bf700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.618+0000 7f03dd9bf700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006fd0 tx=0x7f03d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.622+0000 7f03ce7fc700 1 -- 172.21.15.5:0/1476100982 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f03d0054080 con 0x7f03d81614f0 2024-05-08T23:29:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.843+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f03d80cfa00 con 0x7f03bc06ac90 2024-05-08T23:29:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.844+0000 7f03ce7fc700 1 -- 172.21.15.5:0/1476100982 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f03d400ae30 con 0x7f03bc06ac90 2024-05-08T23:29:01.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 msgr2=0x7f03bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006fd0 tx=0x7f03d4008040 comp rx=0 tx=0).stop 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d81614f0 msgr2=0x7f03d80b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d81614f0 0x7f03d80b5170 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f03d0002230 tx=0x7f03d000a060 comp rx=0 tx=0).stop 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 shutdown_connections 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f03bc06ac90 0x7f03bc06d150 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d81614f0 0x7f03d80b5170 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d80d0650 0x7f03d81f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 --2- 172.21.15.5:0/1476100982 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f03d815d0a0 0x7f03d80b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 >> 172.21.15.5:0/1476100982 conn(0x7f03d80b2f20 msgr2=0x7f03d80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 shutdown_connections 2024-05-08T23:29:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:01.846+0000 7f03dfc23700 1 -- 172.21.15.5:0/1476100982 wait complete. 2024-05-08T23:29:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T23:29:01.860 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T23:29:01.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:29:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.000+0000 7f3409dae700 1 Processor -- start 2024-05-08T23:29:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.000+0000 7f3409dae700 1 -- start start 2024-05-08T23:29:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.001+0000 7f3409dae700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3404161ff0 0x7f340406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.001+0000 7f3409dae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 0x7f34040ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.001+0000 7f3409dae700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340406c2e0 con 0x7f340415d900 2024-05-08T23:29:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.001+0000 7f3409dae700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340415dce0 con 0x7f3404161ff0 2024-05-08T23:29:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.001+0000 7f3409dae700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040ce5b0 con 0x7f340406beb0 2024-05-08T23:29:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.001+0000 7f3402ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3404161ff0 0x7f340406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.001+0000 7f3402ffd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3404161ff0 0x7f340406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35176/0 (socket says 172.21.15.5:35176) 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.001+0000 7f3402ffd700 1 -- 172.21.15.5:0/3862002186 learned_addr learned my addr 172.21.15.5:0/3862002186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f3403fff700 1 --2- 172.21.15.5:0/3862002186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 0x7f34040ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f3402ffd700 1 -- 172.21.15.5:0/3862002186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 msgr2=0x7f34040ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f3402ffd700 1 --2- 172.21.15.5:0/3862002186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 0x7f34040ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f3402ffd700 1 -- 172.21.15.5:0/3862002186 >> v1:172.21.15.49:6789/0 conn(0x7f340415d900 legacy=0x7f34040cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f3402ffd700 1 -- 172.21.15.5:0/3862002186 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404161510 con 0x7f3404161ff0 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f3402ffd700 1 --2- 172.21.15.5:0/3862002186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 0x7f340406b970 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0098f0 tx=0x7f33ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9312e8c130bd335 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f34027fc700 1 -- 172.21.15.5:0/3862002186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec00b8a0 con 0x7f3404161ff0 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f34027fc700 1 -- 172.21.15.5:0/3862002186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec00ba00 con 0x7f3404161ff0 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.002+0000 7f34027fc700 1 -- 172.21.15.5:0/3862002186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec00bd90 con 0x7f3404161ff0 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.003+0000 7f3409dae700 1 -- 172.21.15.5:0/3862002186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 msgr2=0x7f340406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.003+0000 7f3409dae700 1 --2- 172.21.15.5:0/3862002186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 0x7f340406b970 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0098f0 tx=0x7f33ec009c00 comp rx=0 tx=0).stop 2024-05-08T23:29:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.003+0000 7f3409dae700 1 -- 172.21.15.5:0/3862002186 shutdown_connections 2024-05-08T23:29:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.003+0000 7f3409dae700 1 --2- 172.21.15.5:0/3862002186 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 0x7f34040ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.003+0000 7f3409dae700 1 --2- 172.21.15.5:0/3862002186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 0x7f340406b970 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.003+0000 7f3409dae700 1 -- 172.21.15.5:0/3862002186 >> 172.21.15.5:0/3862002186 conn(0x7f34040713c0 msgr2=0x7f34040717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.003+0000 7f3409dae700 1 -- 172.21.15.5:0/3862002186 shutdown_connections 2024-05-08T23:29:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.003+0000 7f3409dae700 1 -- 172.21.15.5:0/3862002186 wait complete. 2024-05-08T23:29:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.004+0000 7f3409dae700 1 Processor -- start 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.004+0000 7f3409dae700 1 -- start start 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3409dae700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 0x7f340415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3409dae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415d900 0x7f34041562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3409dae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 0x7f3404156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3409dae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404150090 con 0x7f3404161ff0 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3409dae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34041f42f0 con 0x7f340415d900 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3409dae700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340415f110 con 0x7f340406beb0 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3403fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 0x7f3404156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3403fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 0x7f3404156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35182/0 (socket says 172.21.15.5:35182) 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f3403fff700 1 -- 172.21.15.5:0/1104664348 learned_addr learned my addr 172.21.15.5:0/1104664348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.005+0000 7f34037fe700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 0x7f340415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3402ffd700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415d900 0x7f34041562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3403fff700 1 -- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 msgr2=0x7f340415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3403fff700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 0x7f340415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3403fff700 1 -- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415d900 msgr2=0x7f34041562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3403fff700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415d900 0x7f34041562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3403fff700 1 -- 172.21.15.5:0/1104664348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec009550 con 0x7f3404161ff0 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f34037fe700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f340406beb0 0x7f340415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3402ffd700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415d900 0x7f34041562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3403fff700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 0x7f3404156800 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007d90 tx=0x7f33fc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00ede0 con 0x7f3404161ff0 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc014420 con 0x7f3404161ff0 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34040b2af0 con 0x7f3404161ff0 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0147d0 con 0x7f3404161ff0 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.006+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34040b30b0 con 0x7f3404161ff0 2024-05-08T23:29:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.007+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34040cf7c0 con 0x7f3404161ff0 2024-05-08T23:29:02.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.008+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f33fc014930 con 0x7f3404161ff0 2024-05-08T23:29:02.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.008+0000 7f33fbfff700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e406ac90 0x7f33e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.008+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f33fc0897b0 con 0x7f3404161ff0 2024-05-08T23:29:02.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.008+0000 7f33fbfff700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7f33e4070c30 0x7f33e4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.008+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 --> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33e4073700 con 0x7f33e4070c30 2024-05-08T23:29:02.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.008+0000 7f34037fe700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e406ac90 0x7f33e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.008+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33fc08be70 con 0x7f3404161ff0 2024-05-08T23:29:02.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.008+0000 7f3402ffd700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7f33e4070c30 0x7f33e4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.009+0000 7f34037fe700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e406ac90 0x7f33e406d150 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f33f4000f60 tx=0x7f33f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.009+0000 7f3402ffd700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7f33e4070c30 0x7f33e4073050 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.010+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 <== osd.5 v2:172.21.15.49:6800/47399 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f33e4073700 con 0x7f33e4070c30 2024-05-08T23:29:02.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.025+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 --> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3404161ad0 con 0x7f33e4070c30 2024-05-08T23:29:02.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.025+0000 7f33fbfff700 1 -- 172.21.15.5:0/1104664348 <== osd.5 v2:172.21.15.49:6800/47399 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f33ec00bb70 con 0x7f33e4070c30 2024-05-08T23:29:02.025 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:29:02.026 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:29:02.026 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7f33e4070c30 msgr2=0x7f33e4073050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6800/47399,v1:172.21.15.49:6801/47399] conn(0x7f33e4070c30 0x7f33e4073050 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e406ac90 msgr2=0x7f33e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f33e406ac90 0x7f33e406d150 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f33f4000f60 tx=0x7f33f4009040 comp rx=0 tx=0).stop 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 msgr2=0x7f3404156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 --2- 172.21.15.5:0/1104664348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3404161ff0 0x7f3404156800 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007d90 tx=0x7f33fc00e3c0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3403fff700 1 -- 172.21.15.5:0/1104664348 reap_dead start 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 shutdown_connections 2024-05-08T23:29:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 >> 172.21.15.5:0/1104664348 conn(0x7f34040713c0 msgr2=0x7f340406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 shutdown_connections 2024-05-08T23:29:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.026+0000 7f3409dae700 1 -- 172.21.15.5:0/1104664348 wait complete. 2024-05-08T23:29:02.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T23:29:02.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.177+0000 7f2581e8f700 1 Processor -- start 2024-05-08T23:29:02.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.177+0000 7f2581e8f700 1 -- start start 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f2581e8f700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f257c160f00 0x7f257c0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f2581e8f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c0d22d0 0x7f257c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f2581e8f700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257c0ce3f0 con 0x7f257c0b2f60 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f2581e8f700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c0cdc50 con 0x7f257c160f00 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f2581e8f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0b42c0 con 0x7f257c0d22d0 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257bfff700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c0d22d0 0x7f257c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257affd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f257c0b2f60 0x7f257c0d1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.5:44872/0 (socket says 172.21.15.5:44872) 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257b7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f257c160f00 0x7f257c0b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257affd700 1 -- 172.21.15.5:0/172800622 learned_addr learned my addr 172.21.15.5:0/172800622 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257b7fe700 1 -- 172.21.15.5:0/172800622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c0d22d0 msgr2=0x7f257c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257b7fe700 1 --2- 172.21.15.5:0/172800622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c0d22d0 0x7f257c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257b7fe700 1 -- 172.21.15.5:0/172800622 >> v1:172.21.15.49:6789/0 conn(0x7f257c0b2f60 legacy=0x7f257c0d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257b7fe700 1 -- 172.21.15.5:0/172800622 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0b3740 con 0x7f257c160f00 2024-05-08T23:29:02.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.178+0000 7f257bfff700 1 --2- 172.21.15.5:0/172800622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c0d22d0 0x7f257c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:29:02.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f257b7fe700 1 --2- 172.21.15.5:0/172800622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c160f00 0x7f257c0b47f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f256c009820 tx=0x7f256c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf0c706137447e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f257a7fc700 1 -- 172.21.15.5:0/172800622 <== mon.1 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c005e30 con 0x7f257c160f00 2024-05-08T23:29:02.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f257a7fc700 1 -- 172.21.15.5:0/172800622 <== mon.1 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c006030 con 0x7f257c160f00 2024-05-08T23:29:02.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f257a7fc700 1 -- 172.21.15.5:0/172800622 <== mon.1 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c0063c0 con 0x7f257c160f00 2024-05-08T23:29:02.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f2581e8f700 1 -- 172.21.15.5:0/172800622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c160f00 msgr2=0x7f257c0b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f2581e8f700 1 --2- 172.21.15.5:0/172800622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c160f00 0x7f257c0b47f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f256c009820 tx=0x7f256c009b30 comp rx=0 tx=0).stop 2024-05-08T23:29:02.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f2581e8f700 1 -- 172.21.15.5:0/172800622 shutdown_connections 2024-05-08T23:29:02.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f2581e8f700 1 --2- 172.21.15.5:0/172800622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c0d22d0 0x7f257c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f2581e8f700 1 --2- 172.21.15.5:0/172800622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c160f00 0x7f257c0b47f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f2581e8f700 1 -- 172.21.15.5:0/172800622 >> 172.21.15.5:0/172800622 conn(0x7f257c08afe0 msgr2=0x7f257c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f2581e8f700 1 -- 172.21.15.5:0/172800622 shutdown_connections 2024-05-08T23:29:02.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.179+0000 7f2581e8f700 1 -- 172.21.15.5:0/172800622 wait complete. 2024-05-08T23:29:02.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.181+0000 7f2581e8f700 1 Processor -- start 2024-05-08T23:29:02.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.181+0000 7f2581e8f700 1 -- start start 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f2581e8f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c0b2f60 0x7f257c1f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f2581e8f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f257c0d22d0 0x7f257c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f2581e8f700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c160f00 0x7f257c06cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f2581e8f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c1f6c70 con 0x7f257c0d22d0 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f2581e8f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0c9d90 con 0x7f257c0b2f60 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f2581e8f700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0caed0 con 0x7f257c160f00 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f257c0d22d0 0x7f257c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f257c0d22d0 0x7f257c0c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35192/0 (socket says 172.21.15.5:35192) 2024-05-08T23:29:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257affd700 1 -- 172.21.15.5:0/2882370443 learned_addr learned my addr 172.21.15.5:0/2882370443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257b7fe700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c0b2f60 0x7f257c1f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257bfff700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c160f00 0x7f257c06cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257affd700 1 -- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c160f00 msgr2=0x7f257c06cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257affd700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c160f00 0x7f257c06cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257affd700 1 -- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c0b2f60 msgr2=0x7f257c1f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257affd700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c0b2f60 0x7f257c1f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.182+0000 7f257affd700 1 -- 172.21.15.5:0/2882370443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574009a70 con 0x7f257c0d22d0 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.183+0000 7f257affd700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f257c0d22d0 0x7f257c0c92b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f257000c480 tx=0x7f257000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.183+0000 7f2563fff700 1 -- 172.21.15.5:0/2882370443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570005d70 con 0x7f257c0d22d0 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.183+0000 7f2563fff700 1 -- 172.21.15.5:0/2882370443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257000ce80 con 0x7f257c0d22d0 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.183+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c009430 con 0x7f257c0d22d0 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.183+0000 7f2563fff700 1 -- 172.21.15.5:0/2882370443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700072d0 con 0x7f257c0d22d0 2024-05-08T23:29:02.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.183+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c0cb490 con 0x7f257c0d22d0 2024-05-08T23:29:02.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.184+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c0cad30 con 0x7f257c0d22d0 2024-05-08T23:29:02.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.185+0000 7f2563fff700 1 -- 172.21.15.5:0/2882370443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2570007430 con 0x7f257c0d22d0 2024-05-08T23:29:02.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.185+0000 7f2563fff700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f256806ac90 0x7f256806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.185+0000 7f2563fff700 1 -- 172.21.15.5:0/2882370443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2570088390 con 0x7f257c0d22d0 2024-05-08T23:29:02.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.185+0000 7f257b7fe700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f256806ac90 0x7f256806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.186+0000 7f257b7fe700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f256806ac90 0x7f256806d150 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f256c000c00 tx=0x7f256c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.190+0000 7f2563fff700 1 -- 172.21.15.5:0/2882370443 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f257c0cad30 con 0x7f257c0d22d0 2024-05-08T23:29:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.409+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f257c0b3fe0 con 0x7f256806ac90 2024-05-08T23:29:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.409+0000 7f2563fff700 1 -- 172.21.15.5:0/2882370443 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f256c004140 con 0x7f256806ac90 2024-05-08T23:29:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.411+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f256806ac90 msgr2=0x7f256806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.411+0000 7f2581e8f700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f256806ac90 0x7f256806d150 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f256c000c00 tx=0x7f256c008070 comp rx=0 tx=0).stop 2024-05-08T23:29:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f257c0d22d0 msgr2=0x7f257c0c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f257c0d22d0 0x7f257c0c92b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f257000c480 tx=0x7f257000c790 comp rx=0 tx=0).stop 2024-05-08T23:29:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 shutdown_connections 2024-05-08T23:29:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f256806ac90 0x7f256806d150 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f257c160f00 0x7f257c06cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f257c0b2f60 0x7f257c1f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 --2- 172.21.15.5:0/2882370443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f257c0d22d0 0x7f257c0c92b0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 >> 172.21.15.5:0/2882370443 conn(0x7f257c08afe0 msgr2=0x7f257c0b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 shutdown_connections 2024-05-08T23:29:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.412+0000 7f2581e8f700 1 -- 172.21.15.5:0/2882370443 wait complete. 2024-05-08T23:29:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T23:29:02.426 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T23:29:02.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:29:02.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.567+0000 7fa2413a8700 1 Processor -- start 2024-05-08T23:29:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.567+0000 7fa2413a8700 1 -- start start 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.568+0000 7fa2413a8700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa23c15dde0 0x7fa23c0cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.568+0000 7fa2413a8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c0d2500 0x7fa23c0d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.568+0000 7fa2413a8700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23c0ceef0 con 0x7fa23c0b4c80 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.568+0000 7fa2413a8700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23c15e1c0 con 0x7fa23c15dde0 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.568+0000 7fa2413a8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0cf140 con 0x7fa23c0d2500 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.568+0000 7fa23affd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa23c15dde0 0x7fa23c0cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.568+0000 7fa23affd700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa23c15dde0 0x7fa23c0cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35204/0 (socket says 172.21.15.5:35204) 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.568+0000 7fa23affd700 1 -- 172.21.15.5:0/3229186835 learned_addr learned my addr 172.21.15.5:0/3229186835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa23b7fe700 1 --2- 172.21.15.5:0/3229186835 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c0d2500 0x7fa23c0d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa23affd700 1 -- 172.21.15.5:0/3229186835 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c0d2500 msgr2=0x7fa23c0d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa23affd700 1 --2- 172.21.15.5:0/3229186835 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c0d2500 0x7fa23c0d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa23affd700 1 -- 172.21.15.5:0/3229186835 >> v1:172.21.15.49:6789/0 conn(0x7fa23c0b4c80 legacy=0x7fa23c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa23affd700 1 -- 172.21.15.5:0/3229186835 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0cf420 con 0x7fa23c15dde0 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa23b7fe700 1 --2- 172.21.15.5:0/3229186835 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c0d2500 0x7fa23c0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa23affd700 1 --2- 172.21.15.5:0/3229186835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c15dde0 0x7fa23c0cffa0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0098f0 tx=0x7fa22c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78d5e58c6274823 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa239ffb700 1 -- 172.21.15.5:0/3229186835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c004230 con 0x7fa23c15dde0 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa239ffb700 1 -- 172.21.15.5:0/3229186835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c004390 con 0x7fa23c15dde0 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.569+0000 7fa239ffb700 1 -- 172.21.15.5:0/3229186835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0036a0 con 0x7fa23c15dde0 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.570+0000 7fa2413a8700 1 -- 172.21.15.5:0/3229186835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c15dde0 msgr2=0x7fa23c0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.570+0000 7fa2413a8700 1 --2- 172.21.15.5:0/3229186835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c15dde0 0x7fa23c0cffa0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0098f0 tx=0x7fa22c009c00 comp rx=0 tx=0).stop 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.570+0000 7fa2413a8700 1 -- 172.21.15.5:0/3229186835 shutdown_connections 2024-05-08T23:29:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.570+0000 7fa2413a8700 1 --2- 172.21.15.5:0/3229186835 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c0d2500 0x7fa23c0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.570+0000 7fa2413a8700 1 --2- 172.21.15.5:0/3229186835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c15dde0 0x7fa23c0cffa0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.570+0000 7fa2413a8700 1 -- 172.21.15.5:0/3229186835 >> 172.21.15.5:0/3229186835 conn(0x7fa23c08afe0 msgr2=0x7fa23c0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.570+0000 7fa2413a8700 1 -- 172.21.15.5:0/3229186835 shutdown_connections 2024-05-08T23:29:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.570+0000 7fa2413a8700 1 -- 172.21.15.5:0/3229186835 wait complete. 2024-05-08T23:29:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.571+0000 7fa2413a8700 1 Processor -- start 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.571+0000 7fa2413a8700 1 -- start start 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa2413a8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c0b4c80 0x7fa23c1f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa2413a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23c0d2500 0x7fa23c0cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa2413a8700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c15dde0 0x7fa23c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa2413a8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c1f3330 con 0x7fa23c0b4c80 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa2413a8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0c8490 con 0x7fa23c0d2500 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa2413a8700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0cc920 con 0x7fa23c15dde0 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa23affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c0b4c80 0x7fa23c1f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa23affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c0b4c80 0x7fa23c1f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35210/0 (socket says 172.21.15.5:35210) 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa23affd700 1 -- 172.21.15.5:0/3676395366 learned_addr learned my addr 172.21.15.5:0/3676395366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa23b7fe700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c15dde0 0x7fa23c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.572+0000 7fa23a7fc700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23c0d2500 0x7fa23c0cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa23affd700 1 -- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c15dde0 msgr2=0x7fa23c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa23affd700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c15dde0 0x7fa23c06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa23affd700 1 -- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23c0d2500 msgr2=0x7fa23c0cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa23affd700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23c0d2500 0x7fa23c0cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa23affd700 1 -- 172.21.15.5:0/3676395366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c009550 con 0x7fa23c0b4c80 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa23b7fe700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa23c15dde0 0x7fa23c06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa23a7fc700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23c0d2500 0x7fa23c0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa23affd700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c0b4c80 0x7fa23c1f2df0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009130 tx=0x7fa22c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00b860 con 0x7fa23c0b4c80 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c00b9c0 con 0x7fa23c0b4c80 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00bd70 con 0x7fa23c0b4c80 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23c0cc080 con 0x7fa23c0b4c80 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.573+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c0cc640 con 0x7fa23c0b4c80 2024-05-08T23:29:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.574+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa23c0cf9e0 con 0x7fa23c0b4c80 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.574+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa22c01b440 con 0x7fa23c0b4c80 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.575+0000 7fa2237fe700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa22806ac90 0x7fa22806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.575+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa22c08ab70 con 0x7fa23c0b4c80 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.575+0000 7fa2237fe700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fa228070c30 0x7fa228073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.575+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa228073700 con 0x7fa228070c30 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.575+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa22c08f2a0 con 0x7fa23c0b4c80 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.575+0000 7fa23a7fc700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa22806ac90 0x7fa22806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.575+0000 7fa23b7fe700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fa228070c30 0x7fa228073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.576+0000 7fa23a7fc700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa22806ac90 0x7fa22806d150 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa23c1f68f0 tx=0x7fa230009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.576+0000 7fa23b7fe700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fa228070c30 0x7fa228073050 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.577+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 <== osd.6 v2:172.21.15.49:6808/47398 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa228073700 con 0x7fa228070c30 2024-05-08T23:29:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.593+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 --> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa23c0c85d0 con 0x7fa228070c30 2024-05-08T23:29:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.593+0000 7fa2237fe700 1 -- 172.21.15.5:0/3676395366 <== osd.6 v2:172.21.15.49:6808/47398 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa23400e5a0 con 0x7fa228070c30 2024-05-08T23:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:29:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fa228070c30 msgr2=0x7fa228073050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6808/47398,v1:172.21.15.49:6809/47398] conn(0x7fa228070c30 0x7fa228073050 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa22806ac90 msgr2=0x7fa22806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa22806ac90 0x7fa22806d150 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa23c1f68f0 tx=0x7fa230009040 comp rx=0 tx=0).stop 2024-05-08T23:29:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c0b4c80 msgr2=0x7fa23c1f2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 --2- 172.21.15.5:0/3676395366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23c0b4c80 0x7fa23c1f2df0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009130 tx=0x7fa22c004750 comp rx=0 tx=0).stop 2024-05-08T23:29:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa23b7fe700 1 -- 172.21.15.5:0/3676395366 reap_dead start 2024-05-08T23:29:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 shutdown_connections 2024-05-08T23:29:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 >> 172.21.15.5:0/3676395366 conn(0x7fa23c08afe0 msgr2=0x7fa23c15c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 shutdown_connections 2024-05-08T23:29:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.594+0000 7fa2413a8700 1 -- 172.21.15.5:0/3676395366 wait complete. 2024-05-08T23:29:02.607 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T23:29:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.744+0000 7f450f1ea700 1 Processor -- start 2024-05-08T23:29:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.744+0000 7f450f1ea700 1 -- start start 2024-05-08T23:29:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f450f1ea700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450815a770 0x7f45080ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f450f1ea700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45080cea20 0x7f45080cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f450f1ea700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45080cf790 con 0x7f45080ccbd0 2024-05-08T23:29:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f450f1ea700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45080caa30 con 0x7f45080cea20 2024-05-08T23:29:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f450f1ea700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45080cfa70 con 0x7f450815a770 2024-05-08T23:29:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f4507fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45080cea20 0x7f45080cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f4507fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45080cea20 0x7f45080cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35216/0 (socket says 172.21.15.5:35216) 2024-05-08T23:29:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f4507fff700 1 -- 172.21.15.5:0/422587799 learned_addr learned my addr 172.21.15.5:0/422587799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f450cf86700 1 --2- 172.21.15.5:0/422587799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450815a770 0x7f45080ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f4507fff700 1 -- 172.21.15.5:0/422587799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450815a770 msgr2=0x7f45080ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f4507fff700 1 --2- 172.21.15.5:0/422587799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450815a770 0x7f45080ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f4507fff700 1 -- 172.21.15.5:0/422587799 >> v1:172.21.15.49:6789/0 conn(0x7f45080ccbd0 legacy=0x7f45080cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.745+0000 7f4507fff700 1 -- 172.21.15.5:0/422587799 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080cfd50 con 0x7f45080cea20 2024-05-08T23:29:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f4507fff700 1 --2- 172.21.15.5:0/422587799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080cea20 0x7f45080cee80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f44f400bb90 tx=0x7f44f400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4c6080ad796c30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f45077fe700 1 -- 172.21.15.5:0/422587799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400dd50 con 0x7f45080cea20 2024-05-08T23:29:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f45077fe700 1 -- 172.21.15.5:0/422587799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f4010470 con 0x7f45080cea20 2024-05-08T23:29:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f45077fe700 1 -- 172.21.15.5:0/422587799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4010800 con 0x7f45080cea20 2024-05-08T23:29:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f450f1ea700 1 -- 172.21.15.5:0/422587799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080cea20 msgr2=0x7f45080cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f450f1ea700 1 --2- 172.21.15.5:0/422587799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080cea20 0x7f45080cee80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f44f400bb90 tx=0x7f44f400bea0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f450f1ea700 1 -- 172.21.15.5:0/422587799 shutdown_connections 2024-05-08T23:29:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f450f1ea700 1 --2- 172.21.15.5:0/422587799 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f450815a770 0x7f45080ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f450f1ea700 1 --2- 172.21.15.5:0/422587799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080cea20 0x7f45080cee80 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f450f1ea700 1 -- 172.21.15.5:0/422587799 >> 172.21.15.5:0/422587799 conn(0x7f45080b1f80 msgr2=0x7f45080b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f450f1ea700 1 -- 172.21.15.5:0/422587799 shutdown_connections 2024-05-08T23:29:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.746+0000 7f450f1ea700 1 -- 172.21.15.5:0/422587799 wait complete. 2024-05-08T23:29:02.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.748+0000 7f450f1ea700 1 Processor -- start 2024-05-08T23:29:02.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.748+0000 7f450f1ea700 1 -- start start 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450f1ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080ccbd0 0x7f4508152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450f1ea700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45080cea20 0x7f45080c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450f1ea700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f450815a770 0x7f4508158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450f1ea700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4508154840 con 0x7f45080ccbd0 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450f1ea700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45081534e0 con 0x7f450815a770 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450f1ea700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45081f33a0 con 0x7f45080cea20 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080ccbd0 0x7f4508152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080ccbd0 0x7f4508152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35230/0 (socket says 172.21.15.5:35230) 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 -- 172.21.15.5:0/3238135402 learned_addr learned my addr 172.21.15.5:0/3238135402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 -- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45080cea20 msgr2=0x7f45080c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f4507fff700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45080cea20 0x7f45080c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450d787700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f450815a770 0x7f4508158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45080cea20 0x7f45080c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 -- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f450815a770 msgr2=0x7f4508158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f450815a770 0x7f4508158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 -- 172.21.15.5:0/3238135402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f400b7f0 con 0x7f45080ccbd0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f4507fff700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45080cea20 0x7f45080c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450d787700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f450815a770 0x7f4508158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.749+0000 7f450cf86700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080ccbd0 0x7f4508152fa0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00d800 tx=0x7f44fc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.750+0000 7f45057fa700 1 -- 172.21.15.5:0/3238135402 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0125f0 con 0x7f45080ccbd0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.750+0000 7f45057fa700 1 -- 172.21.15.5:0/3238135402 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc012750 con 0x7f45080ccbd0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.750+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4508153cc0 con 0x7f45080ccbd0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.750+0000 7f45057fa700 1 -- 172.21.15.5:0/3238135402 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc012b00 con 0x7f45080ccbd0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.750+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4508154050 con 0x7f45080ccbd0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.751+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45081f34f0 con 0x7f45080ccbd0 2024-05-08T23:29:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.751+0000 7f45057fa700 1 -- 172.21.15.5:0/3238135402 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f44fc012c60 con 0x7f45080ccbd0 2024-05-08T23:29:02.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.752+0000 7f45057fa700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44ec06ac90 0x7f44ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.752+0000 7f45057fa700 1 -- 172.21.15.5:0/3238135402 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f44fc089490 con 0x7f45080ccbd0 2024-05-08T23:29:02.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.752+0000 7f4507fff700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44ec06ac90 0x7f44ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.752+0000 7f4507fff700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44ec06ac90 0x7f44ec06d150 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f44f400b340 tx=0x7f44f401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.756+0000 7f45057fa700 1 -- 172.21.15.5:0/3238135402 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f45081f34f0 con 0x7f45080ccbd0 2024-05-08T23:29:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.973+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f450815ab50 con 0x7f44ec06ac90 2024-05-08T23:29:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.974+0000 7f45057fa700 1 -- 172.21.15.5:0/3238135402 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f44f400ded0 con 0x7f44ec06ac90 2024-05-08T23:29:02.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44ec06ac90 msgr2=0x7f44ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44ec06ac90 0x7f44ec06d150 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f44f400b340 tx=0x7f44f401b040 comp rx=0 tx=0).stop 2024-05-08T23:29:02.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080ccbd0 msgr2=0x7f4508152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080ccbd0 0x7f4508152fa0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00d800 tx=0x7f44fc00dbc0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 shutdown_connections 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f44ec06ac90 0x7f44ec06d150 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f450815a770 0x7f4508158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45080ccbd0 0x7f4508152fa0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 --2- 172.21.15.5:0/3238135402 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45080cea20 0x7f45080c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 >> 172.21.15.5:0/3238135402 conn(0x7f45080b1f80 msgr2=0x7f450806dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 shutdown_connections 2024-05-08T23:29:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:02.976+0000 7f450f1ea700 1 -- 172.21.15.5:0/3238135402 wait complete. 2024-05-08T23:29:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T23:29:02.990 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T23:29:02.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:29:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.131+0000 7f7a42c17700 1 Processor -- start 2024-05-08T23:29:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.131+0000 7f7a42c17700 1 -- start start 2024-05-08T23:29:03.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.131+0000 7f7a42c17700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a3c15d8c0 0x7f7a3c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a42c17700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a3c0b5550 0x7f7a3c1618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a42c17700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c099ce0 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a42c17700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c06fde0 con 0x7f7a3c15d8c0 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a42c17700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c06f260 con 0x7f7a3c0b5550 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a409b3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a3c15d8c0 0x7f7a3c0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a409b3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a3c15d8c0 0x7f7a3c0ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35242/0 (socket says 172.21.15.5:35242) 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a409b3700 1 -- 172.21.15.5:0/3209728633 learned_addr learned my addr 172.21.15.5:0/3209728633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a411b4700 1 --2- 172.21.15.5:0/3209728633 >> v2:172.21.15.49:3300/0 conn(0x7f7a3c0b5550 0x7f7a3c1618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a409b3700 1 -- 172.21.15.5:0/3209728633 >> v2:172.21.15.49:3300/0 conn(0x7f7a3c0b5550 msgr2=0x7f7a3c1618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a409b3700 1 --2- 172.21.15.5:0/3209728633 >> v2:172.21.15.49:3300/0 conn(0x7f7a3c0b5550 0x7f7a3c1618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a409b3700 1 -- 172.21.15.5:0/3209728633 >> v1:172.21.15.49:6789/0 conn(0x7f7a3c0b50d0 legacy=0x7f7a3c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a409b3700 1 -- 172.21.15.5:0/3209728633 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c06f540 con 0x7f7a3c15d8c0 2024-05-08T23:29:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a409b3700 1 --2- 172.21.15.5:0/3209728633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c15d8c0 0x7f7a3c0ad260 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0b4650 tx=0x7f7a2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4c8f4bd17716b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a3b7fe700 1 -- 172.21.15.5:0/3209728633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c004230 con 0x7f7a3c15d8c0 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a3b7fe700 1 -- 172.21.15.5:0/3209728633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c004390 con 0x7f7a3c15d8c0 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.132+0000 7f7a3b7fe700 1 -- 172.21.15.5:0/3209728633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0036a0 con 0x7f7a3c15d8c0 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.133+0000 7f7a42c17700 1 -- 172.21.15.5:0/3209728633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c15d8c0 msgr2=0x7f7a3c0ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.133+0000 7f7a42c17700 1 --2- 172.21.15.5:0/3209728633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c15d8c0 0x7f7a3c0ad260 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0b4650 tx=0x7f7a2c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.133+0000 7f7a42c17700 1 -- 172.21.15.5:0/3209728633 shutdown_connections 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.133+0000 7f7a42c17700 1 --2- 172.21.15.5:0/3209728633 >> v2:172.21.15.49:3300/0 conn(0x7f7a3c0b5550 0x7f7a3c1618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.133+0000 7f7a42c17700 1 --2- 172.21.15.5:0/3209728633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c15d8c0 0x7f7a3c0ad260 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.133+0000 7f7a42c17700 1 -- 172.21.15.5:0/3209728633 >> 172.21.15.5:0/3209728633 conn(0x7f7a3c0b5a30 msgr2=0x7f7a3c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.133+0000 7f7a42c17700 1 -- 172.21.15.5:0/3209728633 shutdown_connections 2024-05-08T23:29:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.133+0000 7f7a42c17700 1 -- 172.21.15.5:0/3209728633 wait complete. 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.134+0000 7f7a42c17700 1 Processor -- start 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.134+0000 7f7a42c17700 1 -- start start 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.135+0000 7f7a42c17700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c0b50d0 0x7f7a3c13d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.135+0000 7f7a42c17700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7a3c0b5550 0x7f7a3c153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.135+0000 7f7a42c17700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a3c15d8c0 0x7f7a3c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.135+0000 7f7a42c17700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c13deb0 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.135+0000 7f7a42c17700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c155fa0 con 0x7f7a3c15d8c0 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.135+0000 7f7a42c17700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c156280 con 0x7f7a3c0b5550 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c0b50d0 0x7f7a3c13d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c0b50d0 0x7f7a3c13d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35244/0 (socket says 172.21.15.5:35244) 2024-05-08T23:29:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 -- 172.21.15.5:0/4024743582 learned_addr learned my addr 172.21.15.5:0/4024743582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a3bfff700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7a3c0b5550 0x7f7a3c153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a411b4700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a3c15d8c0 0x7f7a3c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 -- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7a3c0b5550 msgr2=0x7f7a3c153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7a3c0b5550 0x7f7a3c153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 -- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a3c15d8c0 msgr2=0x7f7a3c15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a3c15d8c0 0x7f7a3c15b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 -- 172.21.15.5:0/4024743582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009550 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a3bfff700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7a3c0b5550 0x7f7a3c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a409b3700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c0b50d0 0x7f7a3c13d750 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0705b0 tx=0x7f7a2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c011630 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c011790 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c011b40 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c13e150 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.136+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c153960 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.137+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a3c153b50 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.138+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7a2c011ca0 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.138+0000 7f7a397fa700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a2806ac90 0x7f7a2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.138+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7a2c08a790 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.138+0000 7f7a397fa700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7f7a28070c30 0x7f7a28073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.138+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 --> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a28073700 con 0x7f7a28070c30 2024-05-08T23:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.138+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a2c08ce50 con 0x7f7a3c0b50d0 2024-05-08T23:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.138+0000 7f7a3bfff700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a2806ac90 0x7f7a2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.139+0000 7f7a411b4700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7f7a28070c30 0x7f7a28073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.139+0000 7f7a3bfff700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a2806ac90 0x7f7a2806d150 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c1616f0 tx=0x7f7a30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.139+0000 7f7a411b4700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7f7a28070c30 0x7f7a28073050 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.140+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 <== osd.7 v2:172.21.15.49:6822/47408 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7a28073700 con 0x7f7a28070c30 2024-05-08T23:29:03.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.155+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 --> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7a3c0ad7a0 con 0x7f7a28070c30 2024-05-08T23:29:03.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a397fa700 1 -- 172.21.15.5:0/4024743582 <== osd.7 v2:172.21.15.49:6822/47408 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7a3400f400 con 0x7f7a28070c30 2024-05-08T23:29:03.156 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-08T23:29:03.156 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:29:03.156 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-08T23:29:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7f7a28070c30 msgr2=0x7f7a28073050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a42c17700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6822/47408,v1:172.21.15.49:6824/47408] conn(0x7f7a28070c30 0x7f7a28073050 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a2806ac90 msgr2=0x7f7a2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a42c17700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f7a2806ac90 0x7f7a2806d150 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c1616f0 tx=0x7f7a30009040 comp rx=0 tx=0).stop 2024-05-08T23:29:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c0b50d0 msgr2=0x7f7a3c13d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a42c17700 1 --2- 172.21.15.5:0/4024743582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a3c0b50d0 0x7f7a3c13d750 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0705b0 tx=0x7f7a2c004750 comp rx=0 tx=0).stop 2024-05-08T23:29:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a411b4700 1 -- 172.21.15.5:0/4024743582 reap_dead start 2024-05-08T23:29:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 shutdown_connections 2024-05-08T23:29:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.156+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 >> 172.21.15.5:0/4024743582 conn(0x7f7a3c0b5a30 msgr2=0x7f7a3c158290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.157+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 shutdown_connections 2024-05-08T23:29:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.157+0000 7f7a42c17700 1 -- 172.21.15.5:0/4024743582 wait complete. 2024-05-08T23:29:03.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T23:29:03.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.307+0000 7f595a71a700 1 Processor -- start 2024-05-08T23:29:03.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.308+0000 7f595a71a700 1 -- start start 2024-05-08T23:29:03.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.308+0000 7f595a71a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59540b2aa0 0x7f59540c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.308+0000 7f595a71a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f59540c60d0 0x7f59540ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.308+0000 7f595a71a700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540c71f0 con 0x7f595415db00 2024-05-08T23:29:03.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.308+0000 7f595a71a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540ce630 con 0x7f59540b2aa0 2024-05-08T23:29:03.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.308+0000 7f595a71a700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540c6950 con 0x7f59540c60d0 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f59537fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59540b2aa0 0x7f59540c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f59537fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59540b2aa0 0x7f59540c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35246/0 (socket says 172.21.15.5:35246) 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f59537fe700 1 -- 172.21.15.5:0/611819163 learned_addr learned my addr 172.21.15.5:0/611819163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f5958cb7700 1 --2- 172.21.15.5:0/611819163 >> v2:172.21.15.49:3300/0 conn(0x7f59540c60d0 0x7f59540ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f59537fe700 1 -- 172.21.15.5:0/611819163 >> v2:172.21.15.49:3300/0 conn(0x7f59540c60d0 msgr2=0x7f59540ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f59537fe700 1 --2- 172.21.15.5:0/611819163 >> v2:172.21.15.49:3300/0 conn(0x7f59540c60d0 0x7f59540ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f59537fe700 1 -- 172.21.15.5:0/611819163 >> v1:172.21.15.49:6789/0 conn(0x7f595415db00 legacy=0x7f59540cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f59537fe700 1 -- 172.21.15.5:0/611819163 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540c6c30 con 0x7f59540b2aa0 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f59537fe700 1 --2- 172.21.15.5:0/611819163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f59540c5b90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f595406d560 tx=0x7f593c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab46e2d2c8b9c4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f5952ffd700 1 -- 172.21.15.5:0/611819163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c011830 con 0x7f59540b2aa0 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f5952ffd700 1 -- 172.21.15.5:0/611819163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c011990 con 0x7f59540b2aa0 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.309+0000 7f5952ffd700 1 -- 172.21.15.5:0/611819163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c011d20 con 0x7f59540b2aa0 2024-05-08T23:29:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.310+0000 7f595a71a700 1 -- 172.21.15.5:0/611819163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 msgr2=0x7f59540c5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.310+0000 7f595a71a700 1 --2- 172.21.15.5:0/611819163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f59540c5b90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f595406d560 tx=0x7f593c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.310+0000 7f595a71a700 1 -- 172.21.15.5:0/611819163 shutdown_connections 2024-05-08T23:29:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.310+0000 7f595a71a700 1 --2- 172.21.15.5:0/611819163 >> v2:172.21.15.49:3300/0 conn(0x7f59540c60d0 0x7f59540ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.310+0000 7f595a71a700 1 --2- 172.21.15.5:0/611819163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f59540c5b90 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.310+0000 7f595a71a700 1 -- 172.21.15.5:0/611819163 >> 172.21.15.5:0/611819163 conn(0x7f59540b1f80 msgr2=0x7f59540b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.310+0000 7f595a71a700 1 -- 172.21.15.5:0/611819163 shutdown_connections 2024-05-08T23:29:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.310+0000 7f595a71a700 1 -- 172.21.15.5:0/611819163 wait complete. 2024-05-08T23:29:03.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.311+0000 7f595a71a700 1 Processor -- start 2024-05-08T23:29:03.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.312+0000 7f595a71a700 1 -- start start 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.312+0000 7f595a71a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f5954157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.312+0000 7f595a71a700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f59540c60d0 0x7f59540c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.312+0000 7f595a71a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f595415db00 0x7f59540ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.312+0000 7f595a71a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541f76f0 con 0x7f59540b2aa0 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.312+0000 7f595a71a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540c7420 con 0x7f595415db00 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.312+0000 7f595a71a700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5954157da0 con 0x7f59540c60d0 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f5954157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f5954157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35248/0 (socket says 172.21.15.5:35248) 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 -- 172.21.15.5:0/440741047 learned_addr learned my addr 172.21.15.5:0/440741047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5958cb7700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f595415db00 0x7f59540ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f59537fe700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f59540c60d0 0x7f59540c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 -- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f59540c60d0 msgr2=0x7f59540c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f59540c60d0 0x7f59540c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 -- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f595415db00 msgr2=0x7f59540ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f595415db00 0x7f59540ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 -- 172.21.15.5:0/440741047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c009550 con 0x7f59540b2aa0 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5958cb7700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f595415db00 0x7f59540ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5953fff700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f5954157860 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f594400c450 tx=0x7f594400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5950ff9700 1 -- 172.21.15.5:0/440741047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440052d0 con 0x7f59540b2aa0 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5950ff9700 1 -- 172.21.15.5:0/440741047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5944005430 con 0x7f59540b2aa0 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59541f7b10 con 0x7f59540b2aa0 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f5950ff9700 1 -- 172.21.15.5:0/440741047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440038c0 con 0x7f59540b2aa0 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.313+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59541f7030 con 0x7f59540b2aa0 2024-05-08T23:29:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.314+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5954068000 con 0x7f59540b2aa0 2024-05-08T23:29:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.315+0000 7f5950ff9700 1 -- 172.21.15.5:0/440741047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5944003a20 con 0x7f59540b2aa0 2024-05-08T23:29:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.315+0000 7f5950ff9700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f593406ac90 0x7f593406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.316+0000 7f5950ff9700 1 -- 172.21.15.5:0/440741047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f59440884a0 con 0x7f59540b2aa0 2024-05-08T23:29:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.316+0000 7f59537fe700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f593406ac90 0x7f593406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.316+0000 7f59537fe700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f593406ac90 0x7f593406d150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f595406d560 tx=0x7f593c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.320+0000 7f5950ff9700 1 -- 172.21.15.5:0/440741047 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5944053960 con 0x7f59540b2aa0 2024-05-08T23:29:03.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.540+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f59540c5670 con 0x7f593406ac90 2024-05-08T23:29:03.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.541+0000 7f5950ff9700 1 -- 172.21.15.5:0/440741047 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f593c011b80 con 0x7f593406ac90 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f593406ac90 msgr2=0x7f593406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f593406ac90 0x7f593406d150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f595406d560 tx=0x7f593c008030 comp rx=0 tx=0).stop 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 msgr2=0x7f5954157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f5954157860 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f594400c450 tx=0x7f594400c810 comp rx=0 tx=0).stop 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 shutdown_connections 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f593406ac90 0x7f593406d150 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f595415db00 0x7f59540ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59540b2aa0 0x7f5954157860 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 --2- 172.21.15.5:0/440741047 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f59540c60d0 0x7f59540c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.543+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 >> 172.21.15.5:0/440741047 conn(0x7f59540b1f80 msgr2=0x7f59540afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.544+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 shutdown_connections 2024-05-08T23:29:03.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.544+0000 7f595a71a700 1 -- 172.21.15.5:0/440741047 wait complete. 2024-05-08T23:29:03.545 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T23:29:03.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:29:03.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.694+0000 7fbcaa6c1700 1 Processor -- start 2024-05-08T23:29:03.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.694+0000 7fbcaa6c1700 1 -- start start 2024-05-08T23:29:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbcaa6c1700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbca415ef60 0x7fbca40b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbcaa6c1700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbca40cfb70 0x7fbca40b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbcaa6c1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca409ad40 con 0x7fbca40cf000 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbcaa6c1700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca40cd400 con 0x7fbca40cfb70 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbcaa6c1700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca40ccb60 con 0x7fbca415ef60 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca8c5e700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbca40cfb70 0x7fbca40b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca37fe700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbca40cf000 0x7fbca40cf460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43828/0 (socket says 172.21.15.5:43828) 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca37fe700 1 -- 172.21.15.5:0/4087470893 learned_addr learned my addr 172.21.15.5:0/4087470893 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca3fff700 1 --2- 172.21.15.5:0/4087470893 >> v2:172.21.15.49:3300/0 conn(0x7fbca415ef60 0x7fbca40b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca8c5e700 1 -- 172.21.15.5:0/4087470893 >> v2:172.21.15.49:3300/0 conn(0x7fbca415ef60 msgr2=0x7fbca40b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca8c5e700 1 --2- 172.21.15.5:0/4087470893 >> v2:172.21.15.49:3300/0 conn(0x7fbca415ef60 0x7fbca40b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca8c5e700 1 -- 172.21.15.5:0/4087470893 >> v1:172.21.15.5:6789/0 conn(0x7fbca40cf000 legacy=0x7fbca40cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca8c5e700 1 -- 172.21.15.5:0/4087470893 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40cce40 con 0x7fbca40cfb70 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.695+0000 7fbca8c5e700 1 --2- 172.21.15.5:0/4087470893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca40cfb70 0x7fbca40b4620 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fbca406e820 tx=0x7fbc9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636bf82efd3778d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbca2ffd700 1 -- 172.21.15.5:0/4087470893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c00e070 con 0x7fbca40cfb70 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbca2ffd700 1 -- 172.21.15.5:0/4087470893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c00bad0 con 0x7fbca40cfb70 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbca2ffd700 1 -- 172.21.15.5:0/4087470893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c00be80 con 0x7fbca40cfb70 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/4087470893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca40cfb70 msgr2=0x7fbca40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/4087470893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca40cfb70 0x7fbca40b4620 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fbca406e820 tx=0x7fbc9c00b330 comp rx=0 tx=0).stop 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/4087470893 shutdown_connections 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/4087470893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca40cfb70 0x7fbca40b4620 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/4087470893 >> v2:172.21.15.49:3300/0 conn(0x7fbca415ef60 0x7fbca40b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/4087470893 >> 172.21.15.5:0/4087470893 conn(0x7fbca40b2f20 msgr2=0x7fbca40b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/4087470893 shutdown_connections 2024-05-08T23:29:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.696+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/4087470893 wait complete. 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.698+0000 7fbcaa6c1700 1 Processor -- start 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.698+0000 7fbcaa6c1700 1 -- start start 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbcaa6c1700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbca40cf000 0x7fbca406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbcaa6c1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbca40cfb70 0x7fbca40caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbcaa6c1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca415ef60 0x7fbca41f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbcaa6c1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca406d950 con 0x7fbca415ef60 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbcaa6c1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca40c8500 con 0x7fbca40cfb70 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbcaa6c1700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca40c87e0 con 0x7fbca40cf000 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbca8c5e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca415ef60 0x7fbca41f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbca8c5e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca415ef60 0x7fbca41f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35268/0 (socket says 172.21.15.5:35268) 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbca8c5e700 1 -- 172.21.15.5:0/3868196206 learned_addr learned my addr 172.21.15.5:0/3868196206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbca3fff700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbca40cf000 0x7fbca406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.699+0000 7fbca37fe700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbca40cfb70 0x7fbca40caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca8c5e700 1 -- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbca40cf000 msgr2=0x7fbca406d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca8c5e700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbca40cf000 0x7fbca406d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca8c5e700 1 -- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbca40cfb70 msgr2=0x7fbca40caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca8c5e700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbca40cfb70 0x7fbca40caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca8c5e700 1 -- 172.21.15.5:0/3868196206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c009a70 con 0x7fbca415ef60 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca8c5e700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca415ef60 0x7fbca41f63a0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b370 tx=0x7fbc9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca0ff9700 1 -- 172.21.15.5:0/3868196206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c012430 con 0x7fbca415ef60 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca0ff9700 1 -- 172.21.15.5:0/3868196206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c0125b0 con 0x7fbca415ef60 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca40c8ac0 con 0x7fbca415ef60 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbca0ff9700 1 -- 172.21.15.5:0/3868196206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c012960 con 0x7fbca415ef60 2024-05-08T23:29:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.700+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca40c9080 con 0x7fbca415ef60 2024-05-08T23:29:03.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.702+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca4068000 con 0x7fbca415ef60 2024-05-08T23:29:03.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.702+0000 7fbca0ff9700 1 -- 172.21.15.5:0/3868196206 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbc9c012ac0 con 0x7fbca415ef60 2024-05-08T23:29:03.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.702+0000 7fbca0ff9700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc9006ac90 0x7fbc9006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.702+0000 7fbca0ff9700 1 -- 172.21.15.5:0/3868196206 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbc9c08b560 con 0x7fbca415ef60 2024-05-08T23:29:03.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.703+0000 7fbca3fff700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc9006ac90 0x7fbc9006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.703+0000 7fbca3fff700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc9006ac90 0x7fbc9006d150 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbca4073ac0 tx=0x7fbc94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.707+0000 7fbca0ff9700 1 -- 172.21.15.5:0/3868196206 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbc9c056a20 con 0x7fbca415ef60 2024-05-08T23:29:03.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.920+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbca409aa60 con 0x7fbc9006ac90 2024-05-08T23:29:03.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.923+0000 7fbca0ff9700 1 -- 172.21.15.5:0/3868196206 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51169 (secure 0 0 0) 0x7fbc940076d0 con 0x7fbc9006ac90 2024-05-08T23:29:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:29:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc9006ac90 msgr2=0x7fbc9006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc9006ac90 0x7fbc9006d150 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbca4073ac0 tx=0x7fbc94008040 comp rx=0 tx=0).stop 2024-05-08T23:29:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca415ef60 msgr2=0x7fbca41f63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca415ef60 0x7fbca41f63a0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b370 tx=0x7fbc9c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 shutdown_connections 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fbc9006ac90 0x7fbc9006d150 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbca415ef60 0x7fbca41f63a0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbca40cf000 0x7fbca406d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 --2- 172.21.15.5:0/3868196206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbca40cfb70 0x7fbca40caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 >> 172.21.15.5:0/3868196206 conn(0x7fbca40b2f20 msgr2=0x7fbca415a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 shutdown_connections 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:03.926+0000 7fbcaa6c1700 1 -- 172.21.15.5:0/3868196206 wait complete. 2024-05-08T23:29:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:29:03.940 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-05-08T23:29:03.411803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259752,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714968,"statfs":{"total":773094113280,"available":772828127232,"internally_reserved":0,"allocated":11051008,"data_stored":5442328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":254791221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.924994"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T23:28:55.218902+0000","last_change":"2024-05-08T23:20:58.968489+0000","last_active":"2024-05-08T23:28:55.218902+0000","last_peered":"2024-05-08T23:28:55.218902+0000","last_clean":"2024-05-08T23:28:55.218902+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:28:55.218902+0000","last_undegraded":"2024-05-08T23:28:55.218902+0000","last_fullsized":"2024-05-08T23:28:55.218902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:20:57.599705+0000","last_clean_scrub_stamp":"2024-05-08T23:20:57.599705+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:51.308641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1826816,"data_stored":1123067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.82499999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.873,"5min":1.038,"15min":1.038},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.81200000000000006,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.54900000000000004},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76700000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46000000000000002}]},{"osd":3,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76800000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.626}]},{"osd":4,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.36499999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.85499999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69199999999999995,"5min":0.87,"15min":0.87},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93500000000000005},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed May 8 23:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.89700000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.54000000000000004},"min":{"1min":0.44500000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.39700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83299999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.5}]},{"osd":1,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71999999999999997,"5min":0.88200000000000001,"15min":0.94999999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.86899999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.48899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73999999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83099999999999996,"5min":0.85699999999999998,"15min":1.115},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72199999999999998,"5min":0.878,"15min":1.0229999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.98899999999999999,"15min":1.0009999999999999},"last":0.67600000000000005}]},{"osd":4,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.752,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.41099999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78900000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.36299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":1.052},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81799999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.877,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.46899999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52000000000000002,"5min":0.61099999999999999,"15min":0.91000000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.42099999999999999},"min":{"1min":0.28799999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.93100000000000005},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.41099999999999998},"min":{"1min":0.28699999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.55400000000000005,"5min":0.63700000000000001,"15min":0.88400000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.42699999999999999,"15min":0.44700000000000001},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60999999999999999,"5min":0.61399999999999999,"15min":0.96999999999999997},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64300000000000002,"5min":0.65300000000000002,"15min":1.01},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.47399999999999998},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.627,"5min":0.68200000000000005,"15min":1.028},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84599999999999997,"5min":0.91100000000000003,"15min":1.1479999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.67500000000000004},"min":{"1min":0.45500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88500000000000001,"5min":1.0169999999999999,"15min":1.1579999999999999},"last":0.75}]},{"osd":5,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86499999999999999,"5min":0.89000000000000001,"15min":1.0489999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.67800000000000005},"min":{"1min":0.503,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.115},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95199999999999996,"5min":1.0089999999999999,"15min":1.2050000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65700000000000003,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.105},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed May 8 23:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.47999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1,"5min":1,"15min":1.1359999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.68799999999999994},"min":{"1min":0.55600000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":1.0609999999999999},"last":0.72999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1826816,"data_stored":1123067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.63400000000000001,"15min":7.7610000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40500000000000003,"15min":0.65700000000000003},"min":{"1min":0.27200000000000002,"5min":0.249,"15min":0.24399999999999999},"max":{"1min":0.58699999999999997,"5min":0.65800000000000003,"15min":7.9889999999999999},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41899999999999998,"15min":0.71499999999999997},"min":{"1min":0.32700000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":7.7869999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.436,"15min":0.69099999999999995},"min":{"1min":0.32100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56599999999999995,"5min":0.69899999999999995,"15min":8.0519999999999996},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41299999999999998,"15min":0.44500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":2.9420000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.46400000000000002,"15min":0.73799999999999999},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67800000000000005,"5min":0.79800000000000004,"15min":7.6440000000000001},"last":0.442}]},{"osd":4,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59899999999999998,"15min":0.874},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.93100000000000005,"15min":7.9139999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59599999999999997,"15min":0.84999999999999998},"min":{"1min":0.47899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.96199999999999997,"15min":7.1879999999999997},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.86099999999999999},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":7.1120000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.94799999999999995},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90600000000000003,"5min":0.94199999999999995,"15min":7.6970000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63900000000000001,"15min":0.91400000000000003},"min":{"1min":0.48199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95099999999999996,"15min":7.734},"last":0.75},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.89600000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83099999999999996,"5min":0.88,"15min":8.0220000000000002},"last":0.64100000000000001}]},{"osd":7,"last update":"Wed May 8 23:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65100000000000002,"15min":0.90800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":7.9539999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.627,"15min":0.87},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.91300000000000003,"15min":7.1600000000000001},"last":0.65300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.39900000000000002,"15min":0.39200000000000002},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.40300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.68400000000000005},"last":0.32300000000000001}]},{"osd":1,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45000000000000001,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.629,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.378},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.42899999999999999},"min":{"1min":0.35699999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53000000000000003,"5min":0.64800000000000002,"15min":0.67300000000000004},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.46800000000000003},"min":{"1min":0.34599999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.627,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.54400000000000004,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.879,"5min":0.93799999999999994,"15min":1.024},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.79800000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.502,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0129999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.0549999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65000000000000002,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83099999999999996,"5min":0.89200000000000002,"15min":1.159},"last":0.755},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.53500000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64100000000000001}]},{"osd":7,"last update":"Wed May 8 23:28:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.55400000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91200000000000003,"5min":0.92500000000000004,"15min":1.194},"last":0.626}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":1.46},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.41499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.2749999999999999},"last":0.308}]},{"osd":1,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.27700000000000002},"max":{"1min":0.54800000000000004,"5min":1.327,"15min":1.327},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":0.68400000000000005,"5min":1.109,"15min":1.5649999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44700000000000001,"15min":0.77200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63,"5min":1.1479999999999999,"15min":11.964},"last":0.376},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48199999999999998,"15min":0.51100000000000001},"min":{"1min":0.35699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.73899999999999999,"15min":1.593},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63900000000000001,"15min":0.65800000000000003},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.503},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94299999999999995,"5min":1.214,"15min":2.4590000000000001},"last":0.43099999999999999}]},{"osd":5,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.536},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.016,"5min":1.2969999999999999,"15min":1.3100000000000001},"last":0.441}]},{"osd":6,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":1.0389999999999999,"15min":1.5029999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed May 8 23:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.4850000000000001,"15min":2.5230000000000001},"last":0.51800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.625},"min":{"1min":0.27200000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71799999999999997,"5min":0.75700000000000001,"15min":2.0209999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55200000000000005,"15min":0.68999999999999995},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.78400000000000003,"5min":0.91200000000000003,"15min":4.0960000000000001},"last":0.66900000000000004}]},{"osd":1,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56599999999999995,"15min":0.65000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68799999999999994,"5min":1.0349999999999999,"15min":2.145},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.66500000000000004},"min":{"1min":0.32800000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76500000000000001,"5min":0.88500000000000001,"15min":2.2530000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":2.3769999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59399999999999997,"15min":0.749},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":4.0389999999999997},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56599999999999995,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.747,"5min":0.98299999999999998,"15min":1.0800000000000001},"last":0.62},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":1.0289999999999999},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70299999999999996,"5min":0.93000000000000005,"15min":12.127000000000001},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.56799999999999995},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.79700000000000004,"15min":0.877},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51600000000000001,"15min":0.56299999999999994},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.79000000000000004,"15min":0.84299999999999997},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53500000000000003,"15min":0.60499999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.81499999999999995,"15min":0.93700000000000006},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.628,"15min":0.67300000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed May 8 23:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68300000000000005,"5min":0.877,"15min":1.24},"last":0.71499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.32700000000000001,"15min":0.27800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88700000000000001,"15min":4.6710000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.83699999999999997},"min":{"1min":0.41499999999999998,"5min":0.35199999999999998,"15min":0.30499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":8.798},"last":0.498}]},{"osd":1,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.73499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74099999999999999,"5min":0.82899999999999996,"15min":4.71},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.89900000000000002},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78900000000000003,"5min":1.0109999999999999,"15min":8.6880000000000006},"last":0.46100000000000002}]},{"osd":2,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.73199999999999998},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":1.0009999999999999,"15min":4.8940000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.626,"15min":0.76000000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.443},"max":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":4.8559999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60799999999999998,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69099999999999995,"5min":0.91800000000000004,"15min":8.8770000000000007},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.621,"15min":0.93899999999999995},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85899999999999999,"5min":1.0489999999999999,"15min":10.427},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48999999999999999,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.874},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54600000000000004,"15min":0.56499999999999995},"min":{"1min":0.35099999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.86799999999999999,"15min":0.95899999999999996},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93999999999999995,"5min":1.141,"15min":1.141},"last":0.63},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 8 23:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60899999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":1.01,"15min":1.01},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65400000000000003,"15min":0.66900000000000004},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86299999999999999,"5min":1.097,"15min":1.135},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:29:03.943 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T23:20:57.599705+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=20, tm_sec=57, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=28, tm_sec=58, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:29:03.944 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T23:29:08.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:29:08.530 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:29:23.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:29:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.092+0000 7fa59771d700 1 Processor -- start 2024-05-08T23:29:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.092+0000 7fa59771d700 1 -- start start 2024-05-08T23:29:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.093+0000 7fa59771d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa59015db00 0x7fa5900cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.093+0000 7fa59771d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa59006f990 0x7fa59006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.093+0000 7fa59771d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59006ad20 con 0x7fa5900caa50 2024-05-08T23:29:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.093+0000 7fa59771d700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59015dee0 con 0x7fa59006f990 2024-05-08T23:29:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.093+0000 7fa59771d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5900ccad0 con 0x7fa59015db00 2024-05-08T23:29:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.093+0000 7fa595cba700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa59006f990 0x7fa59006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.093+0000 7fa595cba700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa59006f990 0x7fa59006e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34110/0 (socket says 172.21.15.5:34110) 2024-05-08T23:29:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.093+0000 7fa595cba700 1 -- 172.21.15.5:0/2449351817 learned_addr learned my addr 172.21.15.5:0/2449351817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa5954b9700 1 --2- 172.21.15.5:0/2449351817 >> v2:172.21.15.49:3300/0 conn(0x7fa59015db00 0x7fa5900cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa595cba700 1 -- 172.21.15.5:0/2449351817 >> v2:172.21.15.49:3300/0 conn(0x7fa59015db00 msgr2=0x7fa5900cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa595cba700 1 --2- 172.21.15.5:0/2449351817 >> v2:172.21.15.49:3300/0 conn(0x7fa59015db00 0x7fa5900cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa595cba700 1 -- 172.21.15.5:0/2449351817 >> v1:172.21.15.49:6789/0 conn(0x7fa5900caa50 legacy=0x7fa59006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa595cba700 1 -- 172.21.15.5:0/2449351817 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59006b000 con 0x7fa59006f990 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa5954b9700 1 --2- 172.21.15.5:0/2449351817 >> v2:172.21.15.49:3300/0 conn(0x7fa59015db00 0x7fa5900cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa595cba700 1 --2- 172.21.15.5:0/2449351817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa59006e320 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa5900cdf50 tx=0x7fa58800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76507e2363d2625e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa583fff700 1 -- 172.21.15.5:0/2449351817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800e070 con 0x7fa59006f990 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa583fff700 1 -- 172.21.15.5:0/2449351817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa588003a20 con 0x7fa59006f990 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.094+0000 7fa583fff700 1 -- 172.21.15.5:0/2449351817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588003dd0 con 0x7fa59006f990 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.095+0000 7fa59771d700 1 -- 172.21.15.5:0/2449351817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 msgr2=0x7fa59006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.095+0000 7fa59771d700 1 --2- 172.21.15.5:0/2449351817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa59006e320 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa5900cdf50 tx=0x7fa58800b330 comp rx=0 tx=0).stop 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.095+0000 7fa59771d700 1 -- 172.21.15.5:0/2449351817 shutdown_connections 2024-05-08T23:29:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.095+0000 7fa59771d700 1 --2- 172.21.15.5:0/2449351817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa59006e320 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.095+0000 7fa59771d700 1 --2- 172.21.15.5:0/2449351817 >> v2:172.21.15.49:3300/0 conn(0x7fa59015db00 0x7fa5900cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.095+0000 7fa59771d700 1 -- 172.21.15.5:0/2449351817 >> 172.21.15.5:0/2449351817 conn(0x7fa590099a50 msgr2=0x7fa5900d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:24.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.095+0000 7fa59771d700 1 -- 172.21.15.5:0/2449351817 shutdown_connections 2024-05-08T23:29:24.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.095+0000 7fa59771d700 1 -- 172.21.15.5:0/2449351817 wait complete. 2024-05-08T23:29:24.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.096+0000 7fa59771d700 1 Processor -- start 2024-05-08T23:29:24.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.097+0000 7fa59771d700 1 -- start start 2024-05-08T23:29:24.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa59771d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa5900c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa59771d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5900caa50 0x7fa5901535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa59771d700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59015db00 0x7fa590153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa59771d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900b5760 con 0x7fa59006f990 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa59771d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900b4ae0 con 0x7fa5900caa50 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa59771d700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900b4dc0 con 0x7fa59015db00 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa5900c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa5900c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34124/0 (socket says 172.21.15.5:34124) 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 -- 172.21.15.5:0/1211261972 learned_addr learned my addr 172.21.15.5:0/1211261972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa595cba700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59015db00 0x7fa590153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa594cb8700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5900caa50 0x7fa5901535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 -- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59015db00 msgr2=0x7fa590153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59015db00 0x7fa590153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 -- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5900caa50 msgr2=0x7fa5901535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5900caa50 0x7fa5901535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 -- 172.21.15.5:0/1211261972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588009b30 con 0x7fa59006f990 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa5954b9700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa5900c9640 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c7f0 tx=0x7fa58400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa581ffb700 1 -- 172.21.15.5:0/1211261972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584011800 con 0x7fa59006f990 2024-05-08T23:29:24.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa581ffb700 1 -- 172.21.15.5:0/1211261972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584011960 con 0x7fa59006f990 2024-05-08T23:29:24.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.098+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5900b50a0 con 0x7fa59006f990 2024-05-08T23:29:24.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.099+0000 7fa581ffb700 1 -- 172.21.15.5:0/1211261972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584011d10 con 0x7fa59006f990 2024-05-08T23:29:24.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.099+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5900c9b80 con 0x7fa59006f990 2024-05-08T23:29:24.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.100+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5900b58a0 con 0x7fa59006f990 2024-05-08T23:29:24.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.100+0000 7fa581ffb700 1 -- 172.21.15.5:0/1211261972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa584018360 con 0x7fa59006f990 2024-05-08T23:29:24.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.101+0000 7fa581ffb700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57406ac90 0x7fa57406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.101+0000 7fa581ffb700 1 -- 172.21.15.5:0/1211261972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa5840890f0 con 0x7fa59006f990 2024-05-08T23:29:24.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.101+0000 7fa594cb8700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57406ac90 0x7fa57406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.102+0000 7fa594cb8700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57406ac90 0x7fa57406d150 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa5900b00f0 tx=0x7fa58c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.106+0000 7fa581ffb700 1 -- 172.21.15.5:0/1211261972 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa5900b58a0 con 0x7fa59006f990 2024-05-08T23:29:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.321+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 --> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5900d1170 con 0x7fa57406ac90 2024-05-08T23:29:24.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.323+0000 7fa581ffb700 1 -- 172.21.15.5:0/1211261972 <== mgr.4100 v2:172.21.15.49:6832/46734 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50851 (secure 0 0 0) 0x7fa58c015430 con 0x7fa57406ac90 2024-05-08T23:29:24.324 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:29:24.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57406ac90 msgr2=0x7fa57406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57406ac90 0x7fa57406d150 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa5900b00f0 tx=0x7fa58c009040 comp rx=0 tx=0).stop 2024-05-08T23:29:24.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 msgr2=0x7fa5900c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa5900c9640 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c7f0 tx=0x7fa58400cbb0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 shutdown_connections 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7fa57406ac90 0x7fa57406d150 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa59015db00 0x7fa590153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59006f990 0x7fa5900c9640 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 --2- 172.21.15.5:0/1211261972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5900caa50 0x7fa5901535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 >> 172.21.15.5:0/1211261972 conn(0x7fa590099a50 msgr2=0x7fa5900cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 shutdown_connections 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.326+0000 7fa59771d700 1 -- 172.21.15.5:0/1211261972 wait complete. 2024-05-08T23:29:24.327 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-08T23:29:24.340 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-08T23:29:23.416050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259736,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714984,"statfs":{"total":773094113280,"available":772828143616,"internally_reserved":0,"allocated":11051008,"data_stored":5442328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12747,"internal_metadata":254791221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002502"},"pg_stats":[{"pgid":"1.0","version":"99'119","reported_seq":246,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T23:29:03.680389+0000","last_change":"2024-05-08T23:29:03.680518+0000","last_active":"2024-05-08T23:29:03.680389+0000","last_peered":"2024-05-08T23:29:03.680389+0000","last_clean":"2024-05-08T23:29:03.680389+0000","last_became_active":"2024-05-08T23:20:58.967485+0000","last_became_peered":"2024-05-08T23:20:58.967485+0000","last_unstale":"2024-05-08T23:29:03.680389+0000","last_undegraded":"2024-05-08T23:29:03.680389+0000","last_fullsized":"2024-05-08T23:29:03.680389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'119","last_scrub_stamp":"2024-05-08T23:29:03.680361+0000","last_deep_scrub":"99'119","last_deep_scrub_stamp":"2024-05-08T23:29:03.680361+0000","last_clean_scrub_stamp":"2024-05-08T23:29:03.680361+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:53.592132+0000","scrub_duration":0.0045022209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1826816,"data_stored":1123067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:29:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.73699999999999999,"5min":0.85999999999999999,"15min":0.99299999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.34300000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72199999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71899999999999997}]},{"osd":1,"last update":"Wed May 8 23:29:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70699999999999996,"5min":0.873,"15min":1.038},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.56299999999999994},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.35199999999999998},"max":{"1min":0.81399999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed May 8 23:29:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67800000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.89900000000000002},"last":0.76000000000000001}]},{"osd":3,"last update":"Wed May 8 23:29:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.45300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.75800000000000001,"5min":0.92100000000000004,"15min":0.95399999999999996},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed May 8 23:29:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.497,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.71699999999999997,"5min":0.74199999999999999,"15min":0.85499999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67200000000000004,"5min":0.82499999999999996,"15min":0.87},"last":0.47399999999999998}]},{"osd":5,"last update":"Wed May 8 23:29:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.36099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69199999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.5},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.71699999999999997,"5min":0.78700000000000003,"15min":0.93500000000000005},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed May 8 23:29:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.63},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38800000000000001},"max":{"1min":0.80700000000000005,"5min":0.85399999999999998,"15min":0.89700000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.314},"max":{"1min":0.76500000000000001,"5min":0.95799999999999996,"15min":0.98799999999999999},"last":0.63400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:29:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67800000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.35599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79000000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.50600000000000001}]},{"osd":1,"last update":"Wed May 8 23:29:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73199999999999998,"5min":0.82999999999999996,"15min":0.94999999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.63900000000000001,"5min":0.84799999999999998,"15min":0.86899999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Wed May 8 23:29:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":1.0329999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.34499999999999997},"max":{"1min":0.71399999999999997,"5min":0.83099999999999996,"15min":1.115},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed May 8 23:29:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.39900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":0.878,"15min":1.0229999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.505,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70399999999999996,"5min":0.81499999999999995,"15min":1.0009999999999999},"last":0.63}]},{"osd":4,"last update":"Wed May 8 23:29:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.54500000000000004},"min":{"1min":0.30499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73299999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.373,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.749,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.42399999999999999}]},{"osd":5,"last update":"Wed May 8 23:29:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53900000000000003,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":1.052},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.753,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:29:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40400000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.755,"5min":0.877,"15min":0.92100000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.33800000000000002,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.84599999999999997,"15min":1.0980000000000001},"last":0.67400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:29:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.376,"15min":0.39100000000000001},"min":{"1min":0.311,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.91000000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40899999999999997,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.93100000000000005},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed May 8 23:29:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.40999999999999998},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55400000000000005,"5min":0.63700000000000001,"15min":0.88400000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.433,"15min":0.44600000000000001},"min":{"1min":0.37,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.96999999999999997},"last":0.371}]},{"osd":3,"last update":"Wed May 8 23:29:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47199999999999998,"15min":0.48999999999999999},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.32700000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":1.01},"last":0.437},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45100000000000001,"15min":0.47199999999999998},"min":{"1min":0.38300000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.65600000000000003,"5min":0.67600000000000005,"15min":1.028},"last":0.46999999999999997}]},{"osd":4,"last update":"Wed May 8 23:29:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.52400000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.1479999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65500000000000003,"15min":0.67400000000000004},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94699999999999995,"5min":1.0169999999999999,"15min":1.1579999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Wed May 8 23:29:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":0.89000000000000001,"15min":1.0489999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.67500000000000004},"min":{"1min":0.51500000000000001,"5min":0.38100000000000001,"15min":0.32500000000000001},"max":{"1min":0.92700000000000005,"5min":0.97699999999999998,"15min":1.115},"last":0.60599999999999998}]},{"osd":6,"last update":"Wed May 8 23:29:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.43099999999999999},"max":{"1min":0.85399999999999998,"5min":1.0089999999999999,"15min":1.2050000000000001},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.67800000000000005},"min":{"1min":0.441,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82099999999999995,"5min":0.95899999999999996,"15min":1.105},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed May 8 23:29:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66400000000000003,"15min":0.68999999999999995},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.34999999999999998},"max":{"1min":0.91200000000000003,"5min":1,"15min":1.1359999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":1.0609999999999999},"last":0.67000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1826816,"data_stored":1123067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:29:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39100000000000001,"15min":0.63500000000000001},"min":{"1min":0.26700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":7.7610000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.41299999999999998,"15min":0.63200000000000001},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.24399999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":7.9889999999999999},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed May 8 23:29:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42799999999999999,"15min":0.68500000000000005},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.628,"5min":0.67400000000000004,"15min":7.7869999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.45000000000000001,"15min":0.66700000000000004},"min":{"1min":0.35399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":8.0519999999999996},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed May 8 23:29:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.44500000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59699999999999998,"5min":0.67100000000000004,"15min":2.9420000000000002},"last":0.443},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47199999999999998,"15min":0.71099999999999997},"min":{"1min":0.35099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":7.6440000000000001},"last":0.499}]},{"osd":4,"last update":"Wed May 8 23:29:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61199999999999999,"15min":0.84599999999999997},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.40100000000000002},"max":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":7.9139999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60899999999999999,"15min":0.82799999999999996},"min":{"1min":0.41899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88600000000000001,"5min":0.96199999999999997,"15min":7.1879999999999997},"last":0.73299999999999998}]},{"osd":5,"last update":"Wed May 8 23:29:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.83899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83099999999999996,"5min":0.91700000000000004,"15min":7.1120000000000001},"last":0.876},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.92000000000000004},"min":{"1min":0.52200000000000002,"5min":0.44800000000000001,"15min":0.44500000000000001},"max":{"1min":0.93999999999999995,"5min":0.94199999999999995,"15min":7.6970000000000001},"last":0.83599999999999997}]},{"osd":6,"last update":"Wed May 8 23:29:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65100000000000002,"15min":0.88700000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.877,"5min":0.89200000000000002,"15min":7.734},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.63700000000000001,"15min":0.87},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":8.0220000000000002},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed May 8 23:29:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66500000000000004,"15min":0.88400000000000001},"min":{"1min":0.55100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":7.9539999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64600000000000002,"15min":0.84999999999999998},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90000000000000002,"5min":0.91300000000000003,"15min":7.1600000000000001},"last":0.84699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39900000000000002,"15min":0.39200000000000002},"min":{"1min":0.29399999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.53700000000000003,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.42399999999999999,"15min":0.40600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63100000000000001,"5min":0.65800000000000003,"15min":0.68400000000000005},"last":0.308}]},{"osd":1,"last update":"Wed May 8 23:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.42199999999999999},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.29299999999999998},"max":{"1min":0.60099999999999998,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.45200000000000001,"15min":0.432},"min":{"1min":0.34699999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.44700000000000001}]},{"osd":3,"last update":"Wed May 8 23:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.436,"15min":0.42699999999999999},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.49199999999999999,"15min":0.46999999999999997},"min":{"1min":0.377,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66300000000000003,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed May 8 23:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.024},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.628},"min":{"1min":0.498,"5min":0.379,"15min":0.379},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed May 8 23:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.54200000000000004,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97199999999999998,"5min":0.98599999999999999,"15min":1.0129999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90500000000000003,"5min":0.92100000000000004,"15min":1.0549999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed May 8 23:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.66000000000000003,"15min":0.67400000000000004},"min":{"1min":0.50600000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85999999999999999,"5min":0.89200000000000002,"15min":1.159},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70599999999999996,"15min":0.69399999999999995},"min":{"1min":0.59199999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.68000000000000005},"min":{"1min":0.56100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93700000000000006,"5min":1.071,"15min":1.071},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.625,"5min":0.49099999999999999,"15min":0.47499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.194},"last":0.69899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.40799999999999997},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":1.46},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.41499999999999998},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.2749999999999999},"last":0.32700000000000001}]},{"osd":1,"last update":"Wed May 8 23:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.0609999999999999,"5min":1.327,"15min":1.327},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45000000000000001,"15min":0.46300000000000002},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":0.97499999999999998,"5min":1.109,"15min":1.5649999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed May 8 23:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.73699999999999999},"min":{"1min":0.36399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.95399999999999996,"5min":1.1479999999999999,"15min":11.964},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.593},"last":0.622}]},{"osd":4,"last update":"Wed May 8 23:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93700000000000006,"5min":1.383,"15min":1.383},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.63100000000000001,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.40799999999999997},"max":{"1min":1.0069999999999999,"5min":1.214,"15min":2.4590000000000001},"last":0.59799999999999998}]},{"osd":5,"last update":"Wed May 8 23:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.432},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.536},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.113,"5min":1.2969999999999999,"15min":1.3100000000000001},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed May 8 23:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.67300000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.097,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.498},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.496,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.5029999999999999},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed May 8 23:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.69199999999999995},"min":{"1min":0.497,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0349999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68600000000000005,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2010000000000001,"5min":1.4850000000000001,"15min":2.5230000000000001},"last":0.65400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55500000000000005,"15min":0.61899999999999999},"min":{"1min":0.41699999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":2.0209999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.67500000000000004},"min":{"1min":0.34599999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.82599999999999996,"5min":0.91200000000000003,"15min":4.0960000000000001},"last":0.65500000000000003}]},{"osd":1,"last update":"Wed May 8 23:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.64000000000000001},"min":{"1min":0.39900000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81399999999999995,"5min":1.0349999999999999,"15min":2.145},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57299999999999995,"15min":0.65700000000000003},"min":{"1min":0.374,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80700000000000005,"5min":0.88500000000000001,"15min":2.2530000000000001},"last":0.629}]},{"osd":2,"last update":"Wed May 8 23:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.65300000000000002},"min":{"1min":0.46899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.81799999999999995,"15min":2.3769999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.73299999999999998},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73799999999999999,"5min":1.0209999999999999,"15min":4.0389999999999997},"last":0.66600000000000004}]},{"osd":3,"last update":"Wed May 8 23:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.60799999999999998},"min":{"1min":0.48099999999999998,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.78100000000000003,"5min":0.98299999999999998,"15min":1.0800000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.98299999999999998},"min":{"1min":0.501,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83899999999999997,"5min":0.93000000000000005,"15min":12.127000000000001},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed May 8 23:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.501,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.747,"5min":0.79700000000000004,"15min":0.877},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.30499999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72499999999999998,"5min":0.79000000000000004,"15min":0.84299999999999997},"last":0.45900000000000002}]},{"osd":6,"last update":"Wed May 8 23:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53900000000000003,"15min":0.59999999999999998},"min":{"1min":0.36899999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.93700000000000006},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.66900000000000004},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83599999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57599999999999996}]},{"osd":7,"last update":"Wed May 8 23:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.61599999999999999},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77500000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77300000000000002,"5min":0.877,"15min":1.24},"last":0.53100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1232896,"data_stored":532699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:29:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.67400000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":4.6710000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.80600000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.30499999999999999},"max":{"1min":0.745,"5min":0.96099999999999997,"15min":8.798},"last":0.69899999999999995}]},{"osd":1,"last update":"Wed May 8 23:29:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56399999999999995,"15min":0.71599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70999999999999996,"5min":0.82899999999999996,"15min":4.71},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59199999999999997,"15min":0.86499999999999999},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.35399999999999998},"max":{"1min":0.72799999999999998,"5min":0.876,"15min":8.6880000000000006},"last":0.71299999999999997}]},{"osd":2,"last update":"Wed May 8 23:29:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.624,"15min":0.71899999999999997},"min":{"1min":0.36399999999999999,"5min":0.34599999999999997,"15min":0.34300000000000003},"max":{"1min":0.78900000000000003,"5min":1.0009999999999999,"15min":4.8940000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.625,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.443},"max":{"1min":0.82899999999999996,"5min":0.83499999999999996,"15min":4.8559999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed May 8 23:29:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.83899999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":8.8770000000000007},"last":0.751},{"interface":"front","average":{"1min":0.625,"5min":0.623,"15min":0.90400000000000003},"min":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.78200000000000003,"5min":1.0489999999999999,"15min":10.427},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed May 8 23:29:07 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48999999999999999,"15min":0.53000000000000003},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77400000000000002,"5min":0.81200000000000006,"15min":0.874},"last":0.497},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.56399999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.95899999999999996},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed May 8 23:29:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53600000000000003,"15min":0.56699999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.32600000000000001},"max":{"1min":0.79000000000000004,"5min":1.141,"15min":1.141},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":1.038,"15min":1.038},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed May 8 23:29:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.63500000000000001},"min":{"1min":0.42899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.01},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.66700000000000004},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41299999999999998},"max":{"1min":0.82399999999999995,"5min":1.097,"15min":1.135},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:29:24.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T23:29:24.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.478+0000 7f14bb7e6700 1 Processor -- start 2024-05-08T23:29:24.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.478+0000 7f14bb7e6700 1 -- start start 2024-05-08T23:29:24.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14bb7e6700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f14b40c4af0 0x7f14b40ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14bb7e6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b416bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14bb7e6700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40b00b0 con 0x7f14b414b790 2024-05-08T23:29:24.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14bb7e6700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b40af010 con 0x7f14b40c4af0 2024-05-08T23:29:24.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14bb7e6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40af2f0 con 0x7f14b40ae800 2024-05-08T23:29:24.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14b9582700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f14b414b790 0x7f14b40c43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33072/0 (socket says 172.21.15.5:33072) 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14b9582700 1 -- 172.21.15.5:0/3670931029 learned_addr learned my addr 172.21.15.5:0/3670931029 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14b9d83700 1 --2- 172.21.15.5:0/3670931029 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b416bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14b8d81700 1 --2- 172.21.15.5:0/3670931029 >> v2:172.21.15.49:3300/0 conn(0x7f14b40c4af0 0x7f14b40ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14abfff700 1 -- 172.21.15.5:0/3670931029 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626961266 0 0) 0x7f14b40b00b0 con 0x7f14b414b790 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14abfff700 1 -- 172.21.15.5:0/3670931029 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a00035b0 con 0x7f14b414b790 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.479+0000 7f14b9d83700 1 -- 172.21.15.5:0/3670931029 >> v2:172.21.15.49:3300/0 conn(0x7f14b40c4af0 msgr2=0x7f14b40ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.480+0000 7f14b9d83700 1 --2- 172.21.15.5:0/3670931029 >> v2:172.21.15.49:3300/0 conn(0x7f14b40c4af0 0x7f14b40ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.480+0000 7f14b9d83700 1 -- 172.21.15.5:0/3670931029 >> v1:172.21.15.5:6789/0 conn(0x7f14b414b790 legacy=0x7f14b40c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.480+0000 7f14b9d83700 1 -- 172.21.15.5:0/3670931029 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40af5d0 con 0x7f14b40ae800 2024-05-08T23:29:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.480+0000 7f14b8d81700 1 --2- 172.21.15.5:0/3670931029 >> v2:172.21.15.49:3300/0 conn(0x7f14b40c4af0 0x7f14b40ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:29:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.480+0000 7f14b9d83700 1 --2- 172.21.15.5:0/3670931029 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b416bc60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d90 tx=0x7f14ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b20b1e49299764 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.480+0000 7f14abfff700 1 -- 172.21.15.5:0/3670931029 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00e070 con 0x7f14b40ae800 2024-05-08T23:29:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.480+0000 7f14abfff700 1 -- 172.21.15.5:0/3670931029 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac00bb20 con 0x7f14b40ae800 2024-05-08T23:29:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.480+0000 7f14abfff700 1 -- 172.21.15.5:0/3670931029 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00be90 con 0x7f14b40ae800 2024-05-08T23:29:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.481+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3670931029 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 msgr2=0x7f14b416bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.481+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3670931029 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b416bc60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d90 tx=0x7f14ac00b330 comp rx=0 tx=0).stop 2024-05-08T23:29:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.481+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3670931029 shutdown_connections 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.481+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3670931029 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b416bc60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.481+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3670931029 >> v2:172.21.15.49:3300/0 conn(0x7f14b40c4af0 0x7f14b40ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.481+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3670931029 >> 172.21.15.5:0/3670931029 conn(0x7f14b408afe0 msgr2=0x7f14b408b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.481+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3670931029 shutdown_connections 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.481+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3670931029 wait complete. 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.482+0000 7f14bb7e6700 1 Processor -- start 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.482+0000 7f14bb7e6700 1 -- start start 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14bb7e6700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b4161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14bb7e6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b40c4af0 0x7f14b40c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14bb7e6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b414b790 0x7f14b40b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14bb7e6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b4161540 con 0x7f14b414b790 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14bb7e6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41ec540 con 0x7f14b40c4af0 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14bb7e6700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41ec820 con 0x7f14b40ae800 2024-05-08T23:29:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9d83700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b414b790 0x7f14b40b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9d83700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b414b790 0x7f14b40b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34132/0 (socket says 172.21.15.5:34132) 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9d83700 1 -- 172.21.15.5:0/3509101261 learned_addr learned my addr 172.21.15.5:0/3509101261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9582700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b4161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b8d81700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b40c4af0 0x7f14b40c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9d83700 1 -- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 msgr2=0x7f14b4161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9d83700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b4161000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9d83700 1 -- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b40c4af0 msgr2=0x7f14b40c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9d83700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b40c4af0 0x7f14b40c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.483+0000 7f14b9d83700 1 -- 172.21.15.5:0/3509101261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a40094a0 con 0x7f14b414b790 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.484+0000 7f14b9d83700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b414b790 0x7f14b40b61d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f14ac014040 tx=0x7f14ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.484+0000 7f14a9ffb700 1 -- 172.21.15.5:0/3509101261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac004960 con 0x7f14b414b790 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.484+0000 7f14a9ffb700 1 -- 172.21.15.5:0/3509101261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac020070 con 0x7f14b414b790 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.484+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac009a70 con 0x7f14b414b790 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.484+0000 7f14a9ffb700 1 -- 172.21.15.5:0/3509101261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0276c0 con 0x7f14b414b790 2024-05-08T23:29:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.484+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b41ecde0 con 0x7f14b414b790 2024-05-08T23:29:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.485+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b40679e0 con 0x7f14b414b790 2024-05-08T23:29:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.486+0000 7f14a9ffb700 1 -- 172.21.15.5:0/3509101261 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f14ac012430 con 0x7f14b414b790 2024-05-08T23:29:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.486+0000 7f14a9ffb700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f149806ac90 0x7f149806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.486+0000 7f14a9ffb700 1 -- 172.21.15.5:0/3509101261 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f14ac022070 con 0x7f14b414b790 2024-05-08T23:29:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.486+0000 7f14b9582700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f149806ac90 0x7f149806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.487+0000 7f14b9582700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f149806ac90 0x7f149806d150 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f14b0005910 tx=0x7f14b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.491+0000 7f14a9ffb700 1 -- 172.21.15.5:0/3509101261 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f14ac0561e0 con 0x7f14b414b790 2024-05-08T23:29:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.706+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f14b406d1e0 con 0x7f14b414b790 2024-05-08T23:29:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.710+0000 7f14a9ffb700 1 -- 172.21.15.5:0/3509101261 <== mon.0 v2:172.21.15.5:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f14ac0561e0 con 0x7f14b414b790 2024-05-08T23:29:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.711+0000 7f14a9ffb700 1 -- 172.21.15.5:0/3509101261 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f14ac059800 con 0x7f14b414b790 2024-05-08T23:29:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f149806ac90 msgr2=0x7f149806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f149806ac90 0x7f149806d150 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f14b0005910 tx=0x7f14b0009040 comp rx=0 tx=0).stop 2024-05-08T23:29:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b414b790 msgr2=0x7f14b40b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b414b790 0x7f14b40b61d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f14ac014040 tx=0x7f14ac004ad0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 shutdown_connections 2024-05-08T23:29:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f149806ac90 0x7f149806d150 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b414b790 0x7f14b40b61d0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f14b40ae800 0x7f14b4161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 --2- 172.21.15.5:0/3509101261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14b40c4af0 0x7f14b40c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 >> 172.21.15.5:0/3509101261 conn(0x7f14b408afe0 msgr2=0x7f14b40b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 shutdown_connections 2024-05-08T23:29:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.714+0000 7f14bb7e6700 1 -- 172.21.15.5:0/3509101261 wait complete. 2024-05-08T23:29:24.728 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:29:24.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:29:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.864+0000 7f3ddeaad700 1 Processor -- start 2024-05-08T23:29:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.864+0000 7f3ddeaad700 1 -- start start 2024-05-08T23:29:24.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3ddeaad700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3dd815dcc0 0x7f3dd806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3ddeaad700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3dd806fa10 0x7f3dd80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3ddeaad700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd80cef60 con 0x7f3dd815e310 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3ddeaad700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd815eee0 con 0x7f3dd806fa10 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3ddeaad700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd80ce980 con 0x7f3dd815dcc0 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3dd7fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3dd806fa10 0x7f3dd80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3ddd04a700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3dd815e310 0x7f3dd815e6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33088/0 (socket says 172.21.15.5:33088) 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3ddd04a700 1 -- 172.21.15.5:0/707001728 learned_addr learned my addr 172.21.15.5:0/707001728 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3ddc849700 1 --2- 172.21.15.5:0/707001728 >> v2:172.21.15.49:3300/0 conn(0x7f3dd815dcc0 0x7f3dd806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3dd7fff700 1 -- 172.21.15.5:0/707001728 >> v2:172.21.15.49:3300/0 conn(0x7f3dd815dcc0 msgr2=0x7f3dd806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3dd7fff700 1 --2- 172.21.15.5:0/707001728 >> v2:172.21.15.49:3300/0 conn(0x7f3dd815dcc0 0x7f3dd806ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.865+0000 7f3dd7fff700 1 -- 172.21.15.5:0/707001728 >> v1:172.21.15.5:6789/0 conn(0x7f3dd815e310 legacy=0x7f3dd815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3dd77fe700 1 -- 172.21.15.5:0/707001728 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811211246 0 0) 0x7f3dd80cef60 con 0x7f3dd815e310 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3dd7fff700 1 -- 172.21.15.5:0/707001728 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd815e0a0 con 0x7f3dd806fa10 2024-05-08T23:29:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3ddc849700 1 --2- 172.21.15.5:0/707001728 >> v2:172.21.15.49:3300/0 conn(0x7f3dd815dcc0 0x7f3dd806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3dd7fff700 1 --2- 172.21.15.5:0/707001728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80b26d0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f3dd809e440 tx=0x7f3dcc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ea64dde9601dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3dd77fe700 1 -- 172.21.15.5:0/707001728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc011830 con 0x7f3dd806fa10 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3dd77fe700 1 -- 172.21.15.5:0/707001728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3dcc011990 con 0x7f3dd806fa10 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3dd77fe700 1 -- 172.21.15.5:0/707001728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc011d40 con 0x7f3dd806fa10 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3ddeaad700 1 -- 172.21.15.5:0/707001728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 msgr2=0x7f3dd80b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/707001728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80b26d0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f3dd809e440 tx=0x7f3dcc009b20 comp rx=0 tx=0).stop 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3ddeaad700 1 -- 172.21.15.5:0/707001728 shutdown_connections 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/707001728 >> v2:172.21.15.49:3300/0 conn(0x7f3dd815dcc0 0x7f3dd806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.866+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/707001728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80b26d0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.867+0000 7f3ddeaad700 1 -- 172.21.15.5:0/707001728 >> 172.21.15.5:0/707001728 conn(0x7f3dd8099a50 msgr2=0x7f3dd80b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.867+0000 7f3ddeaad700 1 -- 172.21.15.5:0/707001728 shutdown_connections 2024-05-08T23:29:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.867+0000 7f3ddeaad700 1 -- 172.21.15.5:0/707001728 wait complete. 2024-05-08T23:29:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.868+0000 7f3ddeaad700 1 Processor -- start 2024-05-08T23:29:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.868+0000 7f3ddeaad700 1 -- start start 2024-05-08T23:29:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddeaad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddeaad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dd815dcc0 0x7f3dd80c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddeaad700 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3dd815e310 0x7f3dd8153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddeaad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80c8320 con 0x7f3dd806fa10 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddeaad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80c8550 con 0x7f3dd815dcc0 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddeaad700 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80c8830 con 0x7f3dd815e310 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddc849700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddc849700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34152/0 (socket says 172.21.15.5:34152) 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddc849700 1 -- 172.21.15.5:0/1406636283 learned_addr learned my addr 172.21.15.5:0/1406636283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddd04a700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3dd815e310 0x7f3dd8153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3dd7fff700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dd815dcc0 0x7f3dd80c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddc849700 1 -- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3dd815e310 msgr2=0x7f3dd8153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddc849700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3dd815e310 0x7f3dd8153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddc849700 1 -- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dd815dcc0 msgr2=0x7f3dd80c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddc849700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dd815dcc0 0x7f3dd80c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3ddc849700 1 -- 172.21.15.5:0/1406636283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0094a0 con 0x7f3dd806fa10 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.869+0000 7f3dd7fff700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dd815dcc0 0x7f3dd80c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.870+0000 7f3ddc849700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80ccc20 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800e810 tx=0x7f3dc800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.870+0000 7f3dd57fa700 1 -- 172.21.15.5:0/1406636283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8004230 con 0x7f3dd806fa10 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.870+0000 7f3dd57fa700 1 -- 172.21.15.5:0/1406636283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3dc80043b0 con 0x7f3dd806fa10 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.870+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd80b2d90 con 0x7f3dd806fa10 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.870+0000 7f3dd57fa700 1 -- 172.21.15.5:0/1406636283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8003900 con 0x7f3dd806fa10 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.870+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd80b48e0 con 0x7f3dd806fa10 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.871+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd8068000 con 0x7f3dd806fa10 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.871+0000 7f3dd57fa700 1 -- 172.21.15.5:0/1406636283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3dc8003a60 con 0x7f3dd806fa10 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.872+0000 7f3dd57fa700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3dc406ac90 0x7f3dc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:24.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.872+0000 7f3dd57fa700 1 -- 172.21.15.5:0/1406636283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f3dc8088b70 con 0x7f3dd806fa10 2024-05-08T23:29:24.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.872+0000 7f3dd7fff700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3dc406ac90 0x7f3dc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:24.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.873+0000 7f3dd7fff700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3dc406ac90 0x7f3dc406d150 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0090b0 tx=0x7f3dcc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:24.877+0000 7f3dd57fa700 1 -- 172.21.15.5:0/1406636283 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3dc8053fa0 con 0x7f3dd806fa10 2024-05-08T23:29:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.147+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dd80cd980 con 0x7f3dd806fa10 2024-05-08T23:29:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.148+0000 7f3dd57fa700 1 -- 172.21.15.5:0/1406636283 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+3521 (secure 0 0 0) 0x7f3dc8020de0 con 0x7f3dd806fa10 2024-05-08T23:29:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-08T23:29:25.150 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6836/3350715214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3350715214},{"type":"v1","addr":"172.21.15.49:6836","nonce":3350715214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/940556691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6835","nonce":940556691},{"type":"v1","addr":"172.21.15.49:6837","nonce":940556691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111},{"gid":9453,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/4136914472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4136914472},{"type":"v1","addr":"172.21.15.5:6833","nonce":4136914472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111},{"gid":9459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/771882130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":771882130},{"type":"v1","addr":"172.21.15.5:6835","nonce":771882130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[]} 2024-05-08T23:29:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.150+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3dc406ac90 msgr2=0x7f3dc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.150+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3dc406ac90 0x7f3dc406d150 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0090b0 tx=0x7f3dcc01c040 comp rx=0 tx=0).stop 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 msgr2=0x7f3dd80ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80ccc20 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800e810 tx=0x7f3dc800eb20 comp rx=0 tx=0).stop 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 shutdown_connections 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:6832/46734,v1:172.21.15.49:6833/46734] conn(0x7f3dc406ac90 0x7f3dc406d150 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3dd815e310 0x7f3dd8153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd806fa10 0x7f3dd80ccc20 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 --2- 172.21.15.5:0/1406636283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dd815dcc0 0x7f3dd80c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 >> 172.21.15.5:0/1406636283 conn(0x7f3dd8099a50 msgr2=0x7f3dd80ad180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 shutdown_connections 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-08T23:29:25.151+0000 7f3ddeaad700 1 -- 172.21.15.5:0/1406636283 wait complete. 2024-05-08T23:29:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 111 2024-05-08T23:29:25.165 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-08T23:29:25.165 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T23:29:25.165 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:29:25.165 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.174 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:29:25.174 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:29:25.174 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.225 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:29:25.225 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:29:25.225 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.236 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:29:25.236 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:29:25.236 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.288 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:29:25.288 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T23:29:25.288 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T23:29:25.288 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.325 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T23:29:25.325 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T23:29:25.326 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.406 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T23:29:25.407 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T23:29:25.407 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.478 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T23:29:25.478 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T23:29:25.478 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.549 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T23:29:25.549 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T23:29:25.550 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.579 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T23:29:25.580 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T23:29:25.580 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.656 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T23:29:25.657 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T23:29:25.657 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.729 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T23:29:25.729 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T23:29:25.729 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.810 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T23:29:25.811 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T23:29:25.811 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T23:29:25.811 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.839 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T23:29:25.839 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T23:29:25.839 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.879 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T23:29:25.880 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T23:29:25.880 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T23:29:25.880 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.894 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T23:29:25.894 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T23:29:25.894 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.934 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T23:29:25.934 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T23:29:25.934 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:29:25.988 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T23:29:25.988 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T23:29:25.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Scrub error on inode' | head -n 1 2024-05-08T23:29:26.032 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:29:26.032 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T23:29:26.325 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:29:26.325 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T23:29:26.415 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:29:26.415 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T23:29:26.487 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:29:26.487 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T23:29:26.559 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:29:26.559 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T23:29:26.750 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:29:26.750 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T23:29:26.841 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:29:26.841 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T23:29:26.917 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:29:26.917 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T23:29:26.997 INFO:tasks.ceph:Archiving mon data... 2024-05-08T23:29:26.999 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/data/mon.a.tgz 2024-05-08T23:29:26.999 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-05-08T23:29:27.021 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.lgZqMlrZeE 2024-05-08T23:29:27.021 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.lgZqMlrZeE 2024-05-08T23:29:27.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.lgZqMlrZeE 2024-05-08T23:29:27.293 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.lgZqMlrZeE is 438KB 2024-05-08T23:29:27.369 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.lgZqMlrZeE 2024-05-08T23:29:27.390 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/data/mon.b.tgz 2024-05-08T23:29:27.390 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-05-08T23:29:27.411 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.OKuRDGslxf 2024-05-08T23:29:27.411 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.OKuRDGslxf 2024-05-08T23:29:27.587 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.OKuRDGslxf 2024-05-08T23:29:27.677 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.OKuRDGslxf is 453KB 2024-05-08T23:29:27.773 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.OKuRDGslxf 2024-05-08T23:29:27.791 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/data/mon.c.tgz 2024-05-08T23:29:27.791 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-05-08T23:29:27.847 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.Ex70zGniWe 2024-05-08T23:29:27.847 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Ex70zGniWe 2024-05-08T23:29:28.004 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.Ex70zGniWe 2024-05-08T23:29:28.093 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.Ex70zGniWe is 453KB 2024-05-08T23:29:28.192 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.Ex70zGniWe 2024-05-08T23:29:28.209 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T23:29:28.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T23:29:28.211 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T23:29:28.277 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-05-08T23:29:28.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-08T23:29:28.286 DEBUG:teuthology.orchestra.run.smithi049:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-08T23:29:28.320 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-08T23:29:28.345 INFO:teuthology.orchestra.run.smithi049.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-08T23:29:28.351 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T23:29:28.353 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/remote/smithi005/crash 2024-05-08T23:29:28.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T23:29:28.402 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/remote/smithi049/crash 2024-05-08T23:29:28.403 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T23:29:28.431 INFO:tasks.ceph:Shutting down logrotate 2024-05-08T23:29:28.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:29:28.447 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:29:28.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T23:29:28.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T23:29:28.584 INFO:tasks.ceph:Compressing logs... 2024-05-08T23:29:28.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:29:28.600 DEBUG:teuthology.orchestra.run.smithi049:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:29:31.733 INFO:tasks.ceph:Archiving logs... 2024-05-08T23:29:31.733 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/remote/smithi005/log 2024-05-08T23:29:31.734 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:29:32.640 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/remote/smithi049/log 2024-05-08T23:29:32.641 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:29:33.004 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T23:29:33.013 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T23:29:33.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:29:33.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:29:33.108 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T23:29:33.118 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T23:29:33.118 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:29:33.121 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:29:33.138 INFO:teuthology.orchestra.run.smithi005.stderr:bash: ntpq: command not found 2024-05-08T23:29:33.139 INFO:teuthology.orchestra.run.smithi049.stderr:bash: ntpq: command not found 2024-05-08T23:29:33.144 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:29:33.144 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-08T23:29:33.144 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +1174us[+1136us] +/- 42ms 2024-05-08T23:29:33.144 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -1232us[-1270us] +/- 50ms 2024-05-08T23:29:33.145 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -347us[ -385us] +/- 66ms 2024-05-08T23:29:33.145 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:29:33.146 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:29:33.146 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-05-08T23:29:33.146 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +1192us[+1154us] +/- 42ms 2024-05-08T23:29:33.146 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -1236us[-1274us] +/- 50ms 2024-05-08T23:29:33.146 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -355us[ -355us] +/- 66ms 2024-05-08T23:29:33.147 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:29:33.147 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T23:29:33.187 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T23:29:33.188 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T23:29:33.244 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:29:33.342 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:29:33.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:29:33.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:33.456 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:29:33.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:33.491 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T23:29:33.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T23:29:33.544 INFO:teuthology.task.internal:Duration was 1403.372068 seconds 2024-05-08T23:29:33.544 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T23:29:33.592 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T23:29:33.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:29:33.595 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:29:33.647 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:29:34.090 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T23:29:34.091 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-05-08T23:29:34.091 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:29:34.119 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-05-08T23:29:34.119 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:29:34.146 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T23:29:34.146 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:29:34.162 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:29:34.235 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T23:29:34.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:29:34.238 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:29:34.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T23:29:34.355 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T23:29:34.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:29:34.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:29:34.415 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T23:29:34.429 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:29:34.431 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:29:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-05-08T23:29:34.483 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-05-08T23:29:34.498 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:29:34.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:34.539 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:29:34.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:34.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T23:29:34.566 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T23:29:34.567 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/remote/smithi005 2024-05-08T23:29:34.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:29:34.634 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698161/remote/smithi049 2024-05-08T23:29:34.634 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:29:34.674 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T23:29:34.674 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:29:34.677 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:29:34.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T23:29:34.736 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T23:29:34.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T23:29:34.769 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T23:29:34.770 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:29:34.772 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:29:34.789 INFO:teuthology.orchestra.run.smithi049.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:29 /home/ubuntu/cephtest 2024-05-08T23:29:34.791 INFO:teuthology.orchestra.run.smithi005.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:29 /home/ubuntu/cephtest 2024-05-08T23:29:34.793 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T23:29:34.830 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T23:29:34.907 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/multimds_misc} duration: 1403.3720679283142 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T23:29:34.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:29:34.991 INFO:teuthology.run:pass